2024-09-15T22:24:44.553 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-15T22:24:44.578 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T22:24:44.666 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906262 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7906262' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 8921c8ef947807b717db041d85f0761b7914d5fc subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 8921c8ef947807b717db041d85f0761b7914d5fc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: ad35db06e8924912f01e32b35576297c0f1352b4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3393 sha1: 8921c8ef947807b717db041d85f0761b7914d5fc sleep_before_teardown: 0 subset: 294/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ad35db06e8924912f01e32b35576297c0f1352b4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi111.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGOlbuYrRPc64mhUQwqwUVAYp05Qlii27vwr1FJMvtlwQIX5Q69skHQYO32T6zud6tU96eLJrfsPWiFZbo6WIp4= smithi121.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOnkSn/bPUZ4mKwijt2AUQNIrOPgIuclja+0YHamkXJ6CVd0aK1BAZun8PlrSjYD3fcfeFZhnhLiT8lQjH4WPUs= smithi192.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOEe9zPjF8lH82GpUjwJ8HBXryOTq7oCS5FQV5RmlByL9HxXZ5sn6c7JYRnVbvmtg1nF5cJT6ChWWCOfjOH9WxU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_21:24:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-15T22:24:44.666 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa; will attempt to use it 2024-09-15T22:24:44.667 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks 2024-09-15T22:24:44.667 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-15T22:24:44.668 INFO:teuthology.task.internal:Checking packages... 2024-09-15T22:24:44.690 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8921c8ef947807b717db041d85f0761b7914d5fc' 2024-09-15T22:24:44.690 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-15T22:24:44.690 INFO:teuthology.packaging:ref: None 2024-09-15T22:24:44.690 INFO:teuthology.packaging:tag: None 2024-09-15T22:24:44.690 INFO:teuthology.packaging:branch: squid 2024-09-15T22:24:44.690 INFO:teuthology.packaging:sha1: 8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:24:44.690 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-15T22:24:44.926 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-328.gad35db06 2024-09-15T22:24:44.927 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-15T22:24:44.934 INFO:teuthology.task.internal:no buildpackages task found 2024-09-15T22:24:44.934 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-15T22:24:44.940 INFO:teuthology.task.internal:Saving configuration 2024-09-15T22:24:44.961 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-15T22:24:44.968 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-15T22:24:44.990 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi111.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906262', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 22:19:16.076528', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGOlbuYrRPc64mhUQwqwUVAYp05Qlii27vwr1FJMvtlwQIX5Q69skHQYO32T6zud6tU96eLJrfsPWiFZbo6WIp4='} 2024-09-15T22:24:45.011 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi121.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906262', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 22:19:16.074955', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOnkSn/bPUZ4mKwijt2AUQNIrOPgIuclja+0YHamkXJ6CVd0aK1BAZun8PlrSjYD3fcfeFZhnhLiT8lQjH4WPUs='} 2024-09-15T22:24:45.033 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906262', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 22:19:16.073226', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOEe9zPjF8lH82GpUjwJ8HBXryOTq7oCS5FQV5RmlByL9HxXZ5sn6c7JYRnVbvmtg1nF5cJT6ChWWCOfjOH9WxU='} 2024-09-15T22:24:45.033 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-15T22:24:45.039 INFO:teuthology.task.internal:roles: ubuntu@smithi111.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-15T22:24:45.039 INFO:teuthology.task.internal:roles: ubuntu@smithi121.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-15T22:24:45.039 INFO:teuthology.task.internal:roles: ubuntu@smithi192.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-15T22:24:45.039 INFO:teuthology.run_tasks:Running task console_log... 2024-09-15T22:24:45.123 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2973adb520>, signals=[15]) 2024-09-15T22:24:45.123 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-15T22:24:45.130 INFO:teuthology.task.internal:Opening connections... 2024-09-15T22:24:45.130 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi111.front.sepia.ceph.com 2024-09-15T22:24:45.131 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T22:24:45.205 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi121.front.sepia.ceph.com 2024-09-15T22:24:45.206 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T22:24:45.286 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi192.front.sepia.ceph.com 2024-09-15T22:24:45.287 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T22:24:45.364 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-15T22:24:45.372 DEBUG:teuthology.orchestra.run.smithi111:> uname -m 2024-09-15T22:24:45.390 INFO:teuthology.orchestra.run.smithi111.stdout:x86_64 2024-09-15T22:24:45.391 DEBUG:teuthology.orchestra.run.smithi111:> cat /etc/os-release 2024-09-15T22:24:45.449 INFO:teuthology.orchestra.run.smithi111.stdout:NAME="CentOS Stream" 2024-09-15T22:24:45.449 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION="9" 2024-09-15T22:24:45.449 INFO:teuthology.orchestra.run.smithi111.stdout:ID="centos" 2024-09-15T22:24:45.449 INFO:teuthology.orchestra.run.smithi111.stdout:ID_LIKE="rhel fedora" 2024-09-15T22:24:45.449 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION_ID="9" 2024-09-15T22:24:45.449 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM_ID="platform:el9" 2024-09-15T22:24:45.449 INFO:teuthology.orchestra.run.smithi111.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T22:24:45.449 INFO:teuthology.orchestra.run.smithi111.stdout:ANSI_COLOR="0;31" 2024-09-15T22:24:45.450 INFO:teuthology.orchestra.run.smithi111.stdout:LOGO="fedora-logo-icon" 2024-09-15T22:24:45.450 INFO:teuthology.orchestra.run.smithi111.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T22:24:45.450 INFO:teuthology.orchestra.run.smithi111.stdout:HOME_URL="https://centos.org/" 2024-09-15T22:24:45.450 INFO:teuthology.orchestra.run.smithi111.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T22:24:45.450 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T22:24:45.450 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T22:24:45.451 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server 2024-09-15T22:24:45.477 DEBUG:teuthology.orchestra.run.smithi121:> uname -m 2024-09-15T22:24:45.495 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64 2024-09-15T22:24:45.496 DEBUG:teuthology.orchestra.run.smithi121:> cat /etc/os-release 2024-09-15T22:24:45.553 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="CentOS Stream" 2024-09-15T22:24:45.554 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="9" 2024-09-15T22:24:45.554 INFO:teuthology.orchestra.run.smithi121.stdout:ID="centos" 2024-09-15T22:24:45.554 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="rhel fedora" 2024-09-15T22:24:45.554 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="9" 2024-09-15T22:24:45.554 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el9" 2024-09-15T22:24:45.554 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T22:24:45.554 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31" 2024-09-15T22:24:45.554 INFO:teuthology.orchestra.run.smithi121.stdout:LOGO="fedora-logo-icon" 2024-09-15T22:24:45.554 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T22:24:45.554 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://centos.org/" 2024-09-15T22:24:45.554 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T22:24:45.555 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T22:24:45.555 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T22:24:45.555 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2024-09-15T22:24:45.577 DEBUG:teuthology.orchestra.run.smithi192:> uname -m 2024-09-15T22:24:45.596 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2024-09-15T22:24:45.596 DEBUG:teuthology.orchestra.run.smithi192:> cat /etc/os-release 2024-09-15T22:24:45.654 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="CentOS Stream" 2024-09-15T22:24:45.654 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="9" 2024-09-15T22:24:45.654 INFO:teuthology.orchestra.run.smithi192.stdout:ID="centos" 2024-09-15T22:24:45.654 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE="rhel fedora" 2024-09-15T22:24:45.655 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="9" 2024-09-15T22:24:45.655 INFO:teuthology.orchestra.run.smithi192.stdout:PLATFORM_ID="platform:el9" 2024-09-15T22:24:45.655 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T22:24:45.655 INFO:teuthology.orchestra.run.smithi192.stdout:ANSI_COLOR="0;31" 2024-09-15T22:24:45.655 INFO:teuthology.orchestra.run.smithi192.stdout:LOGO="fedora-logo-icon" 2024-09-15T22:24:45.655 INFO:teuthology.orchestra.run.smithi192.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T22:24:45.655 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://centos.org/" 2024-09-15T22:24:45.655 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T22:24:45.655 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T22:24:45.655 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T22:24:45.655 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2024-09-15T22:24:45.677 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-15T22:24:45.690 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-15T22:24:45.702 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-15T22:24:45.703 DEBUG:teuthology.orchestra.run.smithi111:> test '!' -e /home/ubuntu/cephtest 2024-09-15T22:24:45.706 DEBUG:teuthology.orchestra.run.smithi121:> test '!' -e /home/ubuntu/cephtest 2024-09-15T22:24:45.708 DEBUG:teuthology.orchestra.run.smithi192:> test '!' -e /home/ubuntu/cephtest 2024-09-15T22:24:45.725 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-15T22:24:45.736 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-15T22:24:45.736 DEBUG:teuthology.orchestra.run.smithi111:> test -z $(ls -A /var/lib/ceph) 2024-09-15T22:24:45.764 DEBUG:teuthology.orchestra.run.smithi121:> test -z $(ls -A /var/lib/ceph) 2024-09-15T22:24:45.767 DEBUG:teuthology.orchestra.run.smithi192:> test -z $(ls -A /var/lib/ceph) 2024-09-15T22:24:45.871 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-15T22:24:45.951 INFO:teuthology.run_tasks:Running task kernel... 2024-09-15T22:24:45.967 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-15T22:24:45.967 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-09-15T22:24:45.967 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-09-15T22:24:45.968 DEBUG:teuthology.orchestra.run.smithi111:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T22:24:45.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T22:24:45.986 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-15T22:24:46.164 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-15T22:24:46.165 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-15T22:24:46.165 DEBUG:teuthology.orchestra.run.smithi111:> uname -r 2024-09-15T22:24:46.183 INFO:teuthology.orchestra.run.smithi111.stdout:5.14.0-503.el9.x86_64 2024-09-15T22:24:46.183 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-15T22:24:46.183 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-15T22:24:46.184 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum upgrade -y linux-firmware 2024-09-15T22:24:48.468 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.2 MB 00:00 2024-09-15T22:24:52.610 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - AppStream 6.7 MB/s | 20 MB 00:03 2024-09-15T22:24:56.886 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-15T22:24:59.112 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-15T22:25:00.232 INFO:teuthology.orchestra.run.smithi111.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-15T22:25:05.713 INFO:teuthology.orchestra.run.smithi111.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-15T22:25:07.829 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-15T22:25:07.830 INFO:teuthology.orchestra.run.smithi111.stdout:Nothing to do. 2024-09-15T22:25:07.830 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-15T22:25:07.940 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-15T22:25:07.941 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-15T22:25:08.097 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-15T22:25:08.097 DEBUG:teuthology.orchestra.run.smithi111:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-15T22:25:08.100 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi111.front.sepia.ceph.com to complete... 2024-09-15T22:25:14.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15 22:25:14 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-15T22:25:14.443 INFO:teuthology.orchestra.run.smithi111.stderr:FINISHED --2024-09-15 22:25:14-- 2024-09-15T22:25:14.443 INFO:teuthology.orchestra.run.smithi111.stderr:Total wall clock time: 6.3s 2024-09-15T22:25:14.443 INFO:teuthology.orchestra.run.smithi111.stderr:Downloaded: 1 files, 343M in 6.0s (57.0 MB/s) 2024-09-15T22:25:14.446 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-15T22:25:14.446 DEBUG:teuthology.orchestra.run.smithi111:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-15T22:25:16.806 INFO:teuthology.orchestra.run.smithi111.stdout:Verifying... ######################################## 2024-09-15T22:25:17.002 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing... ######################################## 2024-09-15T22:25:17.105 INFO:teuthology.orchestra.run.smithi111.stdout:Updating / installing... 2024-09-15T22:25:25.075 INFO:teuthology.orchestra.run.smithi111.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-15T22:26:05.511 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi111.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-15T22:26:05.512 DEBUG:teuthology.orchestra.run.smithi111:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-15T22:26:05.596 INFO:teuthology.orchestra.run.smithi111.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-15T22:26:05.596 INFO:teuthology.orchestra.run.smithi111.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-15T22:26:05.596 INFO:teuthology.orchestra.run.smithi111.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-15T22:26:05.596 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-15T22:26:05.596 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-15T22:26:05.596 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-15T22:26:05.596 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-15T22:26:05.596 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-15T22:26:05.596 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-15T22:26:05.597 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-15T22:26:05.597 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-15T22:26:05.597 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-15T22:26:05.597 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-15T22:26:05.597 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-15T22:26:05.597 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-15T22:26:05.597 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-15T22:26:05.597 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-15T22:26:05.597 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-15T22:26:05.597 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-15T22:26:05.598 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-15T22:26:05.598 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-15T22:26:05.598 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-15T22:26:05.598 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-15T22:26:05.598 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-15T22:26:05.598 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-15T22:26:05.598 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-15T22:26:05.598 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-15T22:26:05.598 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-15T22:26:05.598 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-15T22:26:05.598 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-15T22:26:05.598 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-15T22:26:05.599 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-15T22:26:05.599 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-15T22:26:05.599 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-15T22:26:05.599 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-15T22:26:05.599 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-15T22:26:05.599 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-15T22:26:05.599 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-15T22:26:05.599 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-15T22:26:05.599 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-15T22:26:05.599 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-15T22:26:05.599 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-15T22:26:05.600 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-15T22:26:05.600 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-15T22:26:05.600 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-15T22:26:05.600 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-15T22:26:05.600 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-15T22:26:05.600 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-15T22:26:05.600 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-15T22:26:05.600 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-15T22:26:05.600 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-15T22:26:05.600 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-15T22:26:05.600 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-15T22:26:05.600 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-15T22:26:05.601 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-15T22:26:05.601 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-15T22:26:05.601 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-15T22:26:05.601 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-15T22:26:05.601 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-15T22:26:05.601 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-15T22:26:05.601 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-15T22:26:05.601 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-15T22:26:05.601 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-15T22:26:05.601 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-15T22:26:05.601 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-15T22:26:05.601 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-15T22:26:05.602 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-15T22:26:05.602 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-15T22:26:05.602 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-15T22:26:05.602 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-15T22:26:05.602 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-15T22:26:05.602 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-15T22:26:05.602 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-15T22:26:05.602 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-15T22:26:05.602 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-15T22:26:05.602 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-15T22:26:05.602 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-15T22:26:05.602 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-15T22:26:05.603 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-15T22:26:05.603 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-15T22:26:05.603 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-15T22:26:05.603 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-15T22:26:05.603 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-15T22:26:05.603 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-15T22:26:05.603 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-15T22:26:05.603 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-15T22:26:05.603 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-15T22:26:05.603 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-15T22:26:05.603 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-15T22:26:05.603 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-15T22:26:05.603 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-15T22:26:05.604 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-15T22:26:05.604 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-15T22:26:05.604 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-15T22:26:05.604 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-15T22:26:05.604 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-15T22:26:05.604 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-15T22:26:05.604 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-15T22:26:05.604 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-15T22:26:05.604 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-15T22:26:05.604 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-15T22:26:05.604 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-15T22:26:05.605 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-15T22:26:05.605 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-15T22:26:05.605 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-15T22:26:05.605 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-15T22:26:05.605 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-15T22:26:05.605 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-15T22:26:05.605 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-15T22:26:05.605 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-15T22:26:05.605 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-15T22:26:05.605 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-15T22:26:05.605 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-15T22:26:05.606 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-15T22:26:05.606 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-15T22:26:05.606 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-15T22:26:05.606 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-15T22:26:05.606 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-15T22:26:05.606 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-15T22:26:05.606 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-15T22:26:05.606 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-15T22:26:05.606 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-15T22:26:05.606 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-15T22:26:05.607 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-15T22:26:05.607 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-15T22:26:05.607 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-15T22:26:05.607 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-15T22:26:05.607 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-15T22:26:05.607 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-15T22:26:05.607 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-15T22:26:05.607 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-15T22:26:05.607 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-15T22:26:05.607 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-15T22:26:05.607 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-15T22:26:05.608 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-15T22:26:05.608 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-15T22:26:05.608 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-15T22:26:05.608 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-15T22:26:05.608 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-15T22:26:05.608 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-15T22:26:05.608 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-15T22:26:05.608 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-15T22:26:05.608 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-15T22:26:05.608 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-15T22:26:05.609 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-15T22:26:05.609 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-15T22:26:05.609 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-15T22:26:05.609 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-15T22:26:05.609 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-15T22:26:05.609 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-15T22:26:05.609 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-15T22:26:05.609 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-15T22:26:05.609 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-15T22:26:05.609 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-15T22:26:05.610 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-15T22:26:05.610 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-15T22:26:05.610 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-15T22:26:05.610 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-15T22:26:05.610 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-15T22:26:05.610 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-15T22:26:05.610 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-15T22:26:05.610 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-15T22:26:05.610 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-15T22:26:05.610 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-15T22:26:05.611 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-15T22:26:05.611 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-15T22:26:05.611 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-15T22:26:05.611 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-15T22:26:05.611 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-15T22:26:05.611 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-15T22:26:05.611 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-15T22:26:05.611 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-15T22:26:05.611 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-15T22:26:05.612 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-15T22:26:05.612 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-15T22:26:05.612 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-15T22:26:05.612 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-15T22:26:05.612 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-15T22:26:05.612 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-15T22:26:05.612 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-15T22:26:05.612 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-15T22:26:05.612 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-15T22:26:05.612 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-15T22:26:05.613 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-15T22:26:05.613 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-15T22:26:05.613 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-15T22:26:05.613 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-15T22:26:05.613 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-15T22:26:05.613 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-15T22:26:05.613 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-15T22:26:05.613 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-15T22:26:05.613 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-15T22:26:05.613 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-15T22:26:05.614 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-15T22:26:05.614 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-15T22:26:05.614 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-15T22:26:05.614 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-15T22:26:05.614 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-15T22:26:05.614 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-15T22:26:05.614 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-15T22:26:05.614 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-15T22:26:05.614 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-15T22:26:05.614 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-15T22:26:05.615 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-15T22:26:05.615 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-15T22:26:05.615 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-15T22:26:05.615 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-15T22:26:05.615 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-15T22:26:05.615 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-15T22:26:05.615 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-15T22:26:05.615 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-15T22:26:05.615 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-15T22:26:05.615 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-15T22:26:05.616 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-15T22:26:05.616 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-15T22:26:05.616 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-15T22:26:05.616 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-15T22:26:05.616 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-15T22:26:05.616 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-15T22:26:05.616 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-15T22:26:05.616 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-15T22:26:05.616 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-15T22:26:05.616 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-15T22:26:05.617 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-15T22:26:05.617 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-15T22:26:05.617 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-15T22:26:05.617 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-15T22:26:05.617 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-15T22:26:05.617 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-15T22:26:05.617 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-15T22:26:05.617 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-15T22:26:05.617 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-15T22:26:05.617 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-15T22:26:05.617 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-15T22:26:05.617 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-15T22:26:05.618 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-15T22:26:05.618 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-15T22:26:05.618 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-15T22:26:05.618 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-15T22:26:05.618 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-15T22:26:05.618 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-15T22:26:05.618 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-15T22:26:05.618 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-15T22:26:05.618 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-15T22:26:05.618 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-15T22:26:05.618 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-15T22:26:05.619 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-15T22:26:05.619 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-15T22:26:05.619 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-15T22:26:05.619 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-15T22:26:05.619 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-15T22:26:05.619 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-15T22:26:05.619 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-15T22:26:05.619 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-15T22:26:05.619 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-15T22:26:05.619 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-15T22:26:05.619 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-15T22:26:05.620 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-15T22:26:05.620 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-15T22:26:05.620 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-15T22:26:05.620 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-15T22:26:05.620 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-15T22:26:05.620 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-15T22:26:05.620 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-15T22:26:05.620 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-15T22:26:05.620 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-15T22:26:05.620 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-15T22:26:05.620 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-15T22:26:05.621 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-15T22:26:05.621 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-15T22:26:05.621 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-15T22:26:05.621 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-15T22:26:05.621 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-15T22:26:05.621 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-15T22:26:05.621 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-15T22:26:05.621 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-15T22:26:05.621 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-15T22:26:05.621 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-15T22:26:05.621 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-15T22:26:05.622 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-15T22:26:05.622 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-15T22:26:05.622 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-15T22:26:05.622 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-15T22:26:05.622 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-15T22:26:05.622 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-15T22:26:05.622 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-15T22:26:05.623 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-15T22:26:05.623 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-15T22:26:05.623 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-15T22:26:05.623 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-15T22:26:05.623 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-15T22:26:05.623 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-15T22:26:05.623 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-15T22:26:05.623 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-15T22:26:05.623 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-15T22:26:05.623 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-15T22:26:05.623 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-15T22:26:05.624 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-15T22:26:05.624 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-15T22:26:05.624 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-15T22:26:05.624 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-15T22:26:05.624 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-15T22:26:05.624 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-15T22:26:05.624 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-15T22:26:05.625 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-15T22:26:05.625 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-15T22:26:05.625 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-15T22:26:05.625 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-15T22:26:05.625 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-15T22:26:05.625 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-15T22:26:05.625 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-15T22:26:05.625 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-15T22:26:05.625 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-15T22:26:05.625 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-15T22:26:05.625 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-15T22:26:05.626 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-15T22:26:05.626 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-15T22:26:05.626 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-15T22:26:05.626 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-15T22:26:05.626 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-15T22:26:05.626 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-15T22:26:05.626 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-15T22:26:05.626 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-15T22:26:05.626 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-15T22:26:05.626 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-15T22:26:05.626 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-15T22:26:05.627 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-15T22:26:05.627 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-15T22:26:05.627 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-15T22:26:05.627 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-15T22:26:05.627 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-15T22:26:05.627 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-15T22:26:05.627 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-15T22:26:05.627 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-15T22:26:05.627 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-15T22:26:05.627 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-15T22:26:05.627 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-15T22:26:05.628 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-15T22:26:05.628 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-15T22:26:05.628 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-15T22:26:05.628 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-15T22:26:05.628 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-15T22:26:05.628 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-15T22:26:05.628 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-15T22:26:05.629 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-15T22:26:05.629 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-15T22:26:05.629 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-15T22:26:05.629 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-15T22:26:05.629 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-15T22:26:05.629 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-15T22:26:05.629 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-15T22:26:05.629 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-15T22:26:05.629 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-15T22:26:05.629 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-15T22:26:05.629 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-15T22:26:05.630 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-15T22:26:05.630 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-15T22:26:05.630 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-15T22:26:05.630 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-15T22:26:05.630 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-15T22:26:05.630 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-15T22:26:05.630 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-15T22:26:05.630 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-15T22:26:05.630 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-15T22:26:05.630 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-15T22:26:05.630 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-15T22:26:05.631 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-15T22:26:05.631 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-15T22:26:05.631 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-15T22:26:05.631 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-15T22:26:05.631 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-15T22:26:05.631 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-15T22:26:05.631 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-15T22:26:05.631 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-15T22:26:05.631 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-15T22:26:05.631 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-15T22:26:05.631 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-15T22:26:05.631 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-15T22:26:05.632 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-15T22:26:05.632 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-15T22:26:05.632 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-15T22:26:05.632 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-15T22:26:05.632 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-15T22:26:05.632 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-15T22:26:05.632 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-15T22:26:05.632 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-15T22:26:05.632 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-15T22:26:05.632 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-15T22:26:05.632 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-15T22:26:05.633 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-15T22:26:05.633 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-15T22:26:05.633 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-15T22:26:05.633 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-15T22:26:05.633 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-15T22:26:05.633 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-15T22:26:05.633 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-15T22:26:05.633 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-15T22:26:05.633 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-15T22:26:05.633 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-15T22:26:05.633 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-15T22:26:05.634 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-15T22:26:05.634 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-15T22:26:05.634 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-15T22:26:05.634 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-15T22:26:05.634 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-15T22:26:05.634 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-15T22:26:05.634 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-15T22:26:05.634 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-15T22:26:05.634 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-15T22:26:05.634 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-15T22:26:05.634 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-15T22:26:05.635 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-15T22:26:05.635 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-15T22:26:05.635 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-15T22:26:05.635 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-15T22:26:05.635 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-15T22:26:05.635 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-15T22:26:05.635 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-15T22:26:05.635 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-15T22:26:05.635 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-15T22:26:05.635 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-15T22:26:05.636 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-15T22:26:05.636 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-15T22:26:05.636 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-15T22:26:05.636 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-15T22:26:05.636 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-15T22:26:05.636 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-15T22:26:05.636 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-15T22:26:05.636 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-15T22:26:05.636 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-15T22:26:05.636 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-15T22:26:05.636 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-15T22:26:05.636 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-15T22:26:05.637 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-15T22:26:05.637 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-15T22:26:05.637 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-15T22:26:05.637 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-15T22:26:05.637 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-15T22:26:05.637 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-15T22:26:05.637 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-15T22:26:05.637 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-15T22:26:05.637 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-15T22:26:05.637 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-15T22:26:05.638 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-15T22:26:05.638 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-15T22:26:05.638 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-15T22:26:05.638 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-15T22:26:05.638 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-15T22:26:05.638 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-15T22:26:05.638 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-15T22:26:05.638 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-15T22:26:05.638 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-15T22:26:05.638 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-15T22:26:05.638 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-15T22:26:05.639 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-15T22:26:05.639 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-15T22:26:05.639 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-15T22:26:05.639 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-15T22:26:05.639 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-15T22:26:05.639 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-15T22:26:05.639 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-15T22:26:05.639 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-15T22:26:05.639 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-15T22:26:05.639 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-15T22:26:05.639 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-15T22:26:05.639 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-15T22:26:05.640 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-15T22:26:05.640 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-15T22:26:05.640 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-15T22:26:05.640 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-15T22:26:05.640 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-15T22:26:05.640 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-15T22:26:05.640 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-15T22:26:05.640 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-15T22:26:05.640 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-15T22:26:05.640 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-15T22:26:05.640 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-15T22:26:05.641 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-15T22:26:05.641 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-15T22:26:05.641 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-15T22:26:05.641 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-15T22:26:05.641 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-15T22:26:05.641 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-15T22:26:05.641 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-15T22:26:05.641 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-15T22:26:05.641 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-15T22:26:05.641 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-15T22:26:05.641 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-15T22:26:05.642 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-15T22:26:05.642 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-15T22:26:05.642 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-15T22:26:05.642 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-15T22:26:05.642 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-15T22:26:05.642 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-15T22:26:05.642 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-15T22:26:05.642 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-15T22:26:05.642 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-15T22:26:05.642 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-15T22:26:05.642 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-15T22:26:05.642 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-15T22:26:05.643 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-15T22:26:05.643 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-15T22:26:05.643 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-15T22:26:05.643 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-15T22:26:05.643 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-15T22:26:05.643 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-15T22:26:05.643 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-15T22:26:05.643 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-15T22:26:05.643 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-15T22:26:05.643 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-15T22:26:05.643 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-15T22:26:05.644 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-15T22:26:05.644 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-15T22:26:05.644 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-15T22:26:05.644 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-15T22:26:05.644 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-15T22:26:05.644 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-15T22:26:05.644 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-15T22:26:05.644 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-15T22:26:05.644 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-15T22:26:05.644 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-15T22:26:05.644 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-15T22:26:05.645 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-15T22:26:05.645 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-15T22:26:05.645 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-15T22:26:05.645 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-15T22:26:05.645 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-15T22:26:05.645 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-15T22:26:05.645 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-15T22:26:05.645 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-15T22:26:05.645 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-15T22:26:05.645 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-15T22:26:05.645 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-15T22:26:05.645 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-15T22:26:05.646 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-15T22:26:05.646 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-15T22:26:05.646 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-15T22:26:05.646 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-15T22:26:05.646 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-15T22:26:05.646 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-15T22:26:05.646 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-15T22:26:05.646 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-15T22:26:05.646 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-15T22:26:05.646 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-15T22:26:05.646 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-15T22:26:05.647 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-15T22:26:05.647 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-15T22:26:05.647 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-15T22:26:05.647 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-15T22:26:05.647 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-15T22:26:05.647 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-15T22:26:05.647 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-15T22:26:05.647 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-15T22:26:05.647 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-15T22:26:05.647 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-15T22:26:05.647 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-15T22:26:05.647 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-15T22:26:05.648 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-15T22:26:05.648 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-15T22:26:05.648 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-15T22:26:05.648 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-15T22:26:05.648 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-15T22:26:05.648 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-15T22:26:05.648 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-15T22:26:05.648 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-15T22:26:05.648 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-15T22:26:05.648 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-15T22:26:05.649 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-15T22:26:05.649 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-15T22:26:05.649 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-15T22:26:05.649 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-15T22:26:05.649 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-15T22:26:05.649 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-15T22:26:05.649 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-15T22:26:05.649 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-15T22:26:05.649 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-15T22:26:05.649 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-15T22:26:05.649 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-15T22:26:05.650 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-15T22:26:05.650 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-15T22:26:05.650 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-15T22:26:05.650 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-15T22:26:05.650 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-15T22:26:05.650 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-15T22:26:05.650 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-15T22:26:05.650 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-15T22:26:05.650 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-15T22:26:05.650 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-15T22:26:05.650 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-15T22:26:05.650 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-15T22:26:05.650 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-15T22:26:05.651 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-15T22:26:05.651 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-15T22:26:05.651 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-15T22:26:05.651 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-15T22:26:05.651 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-15T22:26:05.651 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-15T22:26:05.651 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-15T22:26:05.651 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-15T22:26:05.651 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-15T22:26:05.651 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-15T22:26:05.651 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-15T22:26:05.651 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-15T22:26:05.651 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-15T22:26:05.652 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-15T22:26:05.652 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-15T22:26:05.652 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-15T22:26:05.652 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-15T22:26:05.652 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-15T22:26:05.652 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-15T22:26:05.652 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-15T22:26:05.652 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-15T22:26:05.652 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-15T22:26:05.652 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-15T22:26:05.652 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-15T22:26:05.652 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-15T22:26:05.653 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-15T22:26:05.653 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-15T22:26:05.653 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-15T22:26:05.653 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-15T22:26:05.653 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-15T22:26:05.653 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-15T22:26:05.653 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-15T22:26:05.653 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-15T22:26:05.653 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-15T22:26:05.653 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-15T22:26:05.653 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-15T22:26:05.654 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-15T22:26:05.654 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-15T22:26:05.654 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-15T22:26:05.654 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-15T22:26:05.654 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-15T22:26:05.654 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-15T22:26:05.654 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-15T22:26:05.654 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-15T22:26:05.654 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-15T22:26:05.654 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-15T22:26:05.654 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-15T22:26:05.654 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-15T22:26:05.655 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-15T22:26:05.655 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-15T22:26:05.655 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-15T22:26:05.655 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-15T22:26:05.655 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-15T22:26:05.655 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-15T22:26:05.655 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-15T22:26:05.655 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-15T22:26:05.655 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-15T22:26:05.655 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-15T22:26:05.655 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-15T22:26:05.655 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-15T22:26:05.655 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-15T22:26:05.655 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-15T22:26:05.656 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-15T22:26:05.656 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-15T22:26:05.656 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-15T22:26:05.656 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-15T22:26:05.656 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-15T22:26:05.656 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-15T22:26:05.656 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-15T22:26:05.656 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-15T22:26:05.656 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-15T22:26:05.656 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-15T22:26:05.656 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-15T22:26:05.656 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-15T22:26:05.657 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-15T22:26:05.657 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-15T22:26:05.657 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-15T22:26:05.657 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-15T22:26:05.657 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-15T22:26:05.657 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-15T22:26:05.657 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-15T22:26:05.657 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-15T22:26:05.657 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-15T22:26:05.657 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-15T22:26:05.657 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-15T22:26:05.657 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-15T22:26:05.658 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-15T22:26:05.658 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-15T22:26:05.658 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-15T22:26:05.658 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-15T22:26:05.658 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-15T22:26:05.658 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-15T22:26:05.658 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-15T22:26:05.658 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-15T22:26:05.658 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-15T22:26:05.658 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-15T22:26:05.658 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-15T22:26:05.658 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-15T22:26:05.658 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-15T22:26:05.659 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-15T22:26:05.659 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-15T22:26:05.659 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-15T22:26:05.659 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-15T22:26:05.659 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-15T22:26:05.659 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-15T22:26:05.659 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-15T22:26:05.659 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-15T22:26:05.659 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-15T22:26:05.659 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-15T22:26:05.659 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-15T22:26:05.660 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-15T22:26:05.660 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-15T22:26:05.660 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-15T22:26:05.660 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-15T22:26:05.660 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-15T22:26:05.660 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-15T22:26:05.660 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-15T22:26:05.660 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-15T22:26:05.660 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-15T22:26:05.660 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-15T22:26:05.660 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-15T22:26:05.660 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-15T22:26:05.660 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-15T22:26:05.661 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-15T22:26:05.661 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-15T22:26:05.661 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-15T22:26:05.661 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-15T22:26:05.661 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-15T22:26:05.661 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-15T22:26:05.661 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-15T22:26:05.661 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-15T22:26:05.661 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-15T22:26:05.661 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-15T22:26:05.661 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-15T22:26:05.662 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-15T22:26:05.662 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-15T22:26:05.662 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-15T22:26:05.662 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-15T22:26:05.662 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-15T22:26:05.662 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-15T22:26:05.662 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-15T22:26:05.662 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-15T22:26:05.662 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-15T22:26:05.662 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-15T22:26:05.662 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-15T22:26:05.662 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-15T22:26:05.662 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-15T22:26:05.662 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-15T22:26:05.663 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-15T22:26:05.663 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-15T22:26:05.663 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-15T22:26:05.663 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-15T22:26:05.663 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-15T22:26:05.663 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-15T22:26:05.663 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-15T22:26:05.663 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-15T22:26:05.663 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-15T22:26:05.663 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-15T22:26:05.663 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-15T22:26:05.664 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-15T22:26:05.664 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-15T22:26:05.664 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-15T22:26:05.664 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-15T22:26:05.664 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-15T22:26:05.664 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-15T22:26:05.664 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-15T22:26:05.664 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-15T22:26:05.664 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-15T22:26:05.664 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-15T22:26:05.664 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-15T22:26:05.665 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-15T22:26:05.665 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-15T22:26:05.665 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-15T22:26:05.665 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-15T22:26:05.665 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-15T22:26:05.665 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-15T22:26:05.665 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-15T22:26:05.665 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-15T22:26:05.665 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-15T22:26:05.665 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-15T22:26:05.666 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-15T22:26:05.666 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-15T22:26:05.666 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-15T22:26:05.666 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-15T22:26:05.666 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-15T22:26:05.666 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-15T22:26:05.666 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-15T22:26:05.666 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-15T22:26:05.666 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-15T22:26:05.666 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-15T22:26:05.666 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-15T22:26:05.667 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-15T22:26:05.667 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-15T22:26:05.667 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-15T22:26:05.667 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-15T22:26:05.667 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-15T22:26:05.667 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-15T22:26:05.667 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-15T22:26:05.667 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-15T22:26:05.667 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-15T22:26:05.667 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-15T22:26:05.668 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-15T22:26:05.668 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-15T22:26:05.668 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-15T22:26:05.668 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-15T22:26:05.668 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-15T22:26:05.668 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-15T22:26:05.668 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-15T22:26:05.668 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-15T22:26:05.668 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-15T22:26:05.668 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-15T22:26:05.668 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-15T22:26:05.669 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-15T22:26:05.669 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-15T22:26:05.669 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-15T22:26:05.669 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-15T22:26:05.669 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-15T22:26:05.669 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-15T22:26:05.669 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-15T22:26:05.669 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-15T22:26:05.669 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-15T22:26:05.669 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-15T22:26:05.669 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-15T22:26:05.670 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-15T22:26:05.670 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-15T22:26:05.670 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-15T22:26:05.670 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-15T22:26:05.670 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-15T22:26:05.670 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-15T22:26:05.670 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-15T22:26:05.670 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-15T22:26:05.670 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-15T22:26:05.670 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-15T22:26:05.670 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-15T22:26:05.671 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-15T22:26:05.671 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-15T22:26:05.671 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-15T22:26:05.671 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-15T22:26:05.671 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-15T22:26:05.671 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-15T22:26:05.671 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-15T22:26:05.671 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-15T22:26:05.671 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-15T22:26:05.671 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-15T22:26:05.671 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-15T22:26:05.672 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-15T22:26:05.672 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-15T22:26:05.672 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-15T22:26:05.672 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-15T22:26:05.672 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-15T22:26:05.672 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-15T22:26:05.672 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-15T22:26:05.672 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-15T22:26:05.672 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-15T22:26:05.672 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-15T22:26:05.672 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-15T22:26:05.673 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-15T22:26:05.673 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-15T22:26:05.673 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-15T22:26:05.673 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-15T22:26:05.673 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-15T22:26:05.673 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-15T22:26:05.673 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-15T22:26:05.673 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-15T22:26:05.673 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-15T22:26:05.673 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-15T22:26:05.673 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-15T22:26:05.674 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-15T22:26:05.674 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-15T22:26:05.674 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-15T22:26:05.674 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-15T22:26:05.674 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-15T22:26:05.674 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-15T22:26:05.674 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-15T22:26:05.674 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-15T22:26:05.674 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-15T22:26:05.674 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-15T22:26:05.675 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-15T22:26:05.675 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-15T22:26:05.675 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-15T22:26:05.675 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-15T22:26:05.675 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-15T22:26:05.675 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-15T22:26:05.675 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-15T22:26:05.675 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-15T22:26:05.675 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-15T22:26:05.675 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-15T22:26:05.675 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-15T22:26:05.675 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-15T22:26:05.676 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-15T22:26:05.676 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-15T22:26:05.676 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-15T22:26:05.676 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-15T22:26:05.676 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-15T22:26:05.676 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-15T22:26:05.676 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-15T22:26:05.676 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-15T22:26:05.676 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-15T22:26:05.676 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-15T22:26:05.676 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-15T22:26:05.677 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-15T22:26:05.677 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-15T22:26:05.677 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-15T22:26:05.677 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-15T22:26:05.677 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-15T22:26:05.677 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-15T22:26:05.677 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-15T22:26:05.677 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-15T22:26:05.677 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-15T22:26:05.677 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-15T22:26:05.678 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-15T22:26:05.678 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-15T22:26:05.678 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-15T22:26:05.678 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-15T22:26:05.678 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-15T22:26:05.678 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-15T22:26:05.678 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-15T22:26:05.678 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-15T22:26:05.678 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-15T22:26:05.678 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-15T22:26:05.678 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-15T22:26:05.678 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-15T22:26:05.679 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-15T22:26:05.679 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-15T22:26:05.679 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-15T22:26:05.679 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-15T22:26:05.679 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-15T22:26:05.679 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-15T22:26:05.679 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-15T22:26:05.679 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-15T22:26:05.679 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-15T22:26:05.679 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-15T22:26:05.679 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-15T22:26:05.679 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-15T22:26:05.680 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-15T22:26:05.680 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-15T22:26:05.680 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-15T22:26:05.680 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-15T22:26:05.680 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-15T22:26:05.680 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-15T22:26:05.680 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-15T22:26:05.680 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-15T22:26:05.680 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-15T22:26:05.680 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-15T22:26:05.680 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-15T22:26:05.680 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-15T22:26:05.681 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-15T22:26:05.681 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-15T22:26:05.681 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-15T22:26:05.681 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-15T22:26:05.681 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-15T22:26:05.681 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-15T22:26:05.681 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-15T22:26:05.681 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-15T22:26:05.681 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-15T22:26:05.681 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-15T22:26:05.682 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-15T22:26:05.682 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-15T22:26:05.682 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-15T22:26:05.682 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-15T22:26:05.682 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-15T22:26:05.682 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-15T22:26:05.682 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-15T22:26:05.682 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-15T22:26:05.682 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-15T22:26:05.682 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-15T22:26:05.682 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-15T22:26:05.683 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-15T22:26:05.683 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-15T22:26:05.683 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-15T22:26:05.683 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-15T22:26:05.683 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-15T22:26:05.683 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-15T22:26:05.683 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-15T22:26:05.683 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-15T22:26:05.683 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-15T22:26:05.684 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-15T22:26:05.684 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-15T22:26:05.684 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-15T22:26:05.684 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-15T22:26:05.684 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-15T22:26:05.684 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-15T22:26:05.684 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-15T22:26:05.684 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-15T22:26:05.684 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-15T22:26:05.684 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-15T22:26:05.684 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-15T22:26:05.684 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-15T22:26:05.685 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-15T22:26:05.685 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-15T22:26:05.685 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-15T22:26:05.685 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-15T22:26:05.685 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-15T22:26:05.685 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-15T22:26:05.685 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-15T22:26:05.685 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-15T22:26:05.685 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-15T22:26:05.685 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-15T22:26:05.685 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-15T22:26:05.685 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-15T22:26:05.686 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-15T22:26:05.686 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-15T22:26:05.688 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-15T22:26:05.688 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-15T22:26:05.688 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-15T22:26:05.688 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-15T22:26:05.688 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-15T22:26:05.688 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-15T22:26:05.688 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-15T22:26:05.689 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-15T22:26:05.689 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-15T22:26:05.689 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-15T22:26:05.689 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-15T22:26:05.689 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-15T22:26:05.689 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-15T22:26:05.689 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-15T22:26:05.689 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-15T22:26:05.689 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-15T22:26:05.689 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-15T22:26:05.689 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-15T22:26:05.689 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-15T22:26:05.690 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-15T22:26:05.690 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-15T22:26:05.690 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-15T22:26:05.690 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-15T22:26:05.690 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-15T22:26:05.690 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-15T22:26:05.690 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-15T22:26:05.690 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-15T22:26:05.690 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-15T22:26:05.690 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-15T22:26:05.690 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-15T22:26:05.690 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-15T22:26:05.690 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-15T22:26:05.690 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-15T22:26:05.690 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-15T22:26:05.691 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-15T22:26:05.691 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-15T22:26:05.691 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-15T22:26:05.691 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-15T22:26:05.691 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-15T22:26:05.691 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-15T22:26:05.691 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-15T22:26:05.691 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-15T22:26:05.691 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-15T22:26:05.691 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-15T22:26:05.691 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-15T22:26:05.691 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-15T22:26:05.691 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-15T22:26:05.691 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-15T22:26:05.692 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-15T22:26:05.692 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-15T22:26:05.692 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-15T22:26:05.692 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-15T22:26:05.692 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-15T22:26:05.692 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-15T22:26:05.692 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-15T22:26:05.692 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-15T22:26:05.692 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-15T22:26:05.692 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-15T22:26:05.692 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-15T22:26:05.692 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-15T22:26:05.692 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-15T22:26:05.692 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-15T22:26:05.693 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-15T22:26:05.693 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-15T22:26:05.693 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-15T22:26:05.693 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-15T22:26:05.693 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-15T22:26:05.693 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-15T22:26:05.693 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-15T22:26:05.693 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-15T22:26:05.693 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-15T22:26:05.693 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-15T22:26:05.693 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-15T22:26:05.693 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-15T22:26:05.693 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-15T22:26:05.694 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-15T22:26:05.694 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-15T22:26:05.694 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-15T22:26:05.694 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-15T22:26:05.694 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-15T22:26:05.694 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-15T22:26:05.694 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-15T22:26:05.694 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-15T22:26:05.694 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-15T22:26:05.694 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-15T22:26:05.694 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-15T22:26:05.694 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-15T22:26:05.694 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-15T22:26:05.694 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-15T22:26:05.694 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-15T22:26:05.694 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-15T22:26:05.695 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-15T22:26:05.695 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-15T22:26:05.695 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-15T22:26:05.695 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-15T22:26:05.695 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-15T22:26:05.695 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-15T22:26:05.695 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-15T22:26:05.695 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-15T22:26:05.695 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-15T22:26:05.695 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-15T22:26:05.695 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-15T22:26:05.695 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-15T22:26:05.695 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-15T22:26:05.696 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-15T22:26:05.696 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-15T22:26:05.696 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-15T22:26:05.696 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-15T22:26:05.696 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-15T22:26:05.696 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-15T22:26:05.696 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-15T22:26:05.696 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-15T22:26:05.696 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-15T22:26:05.696 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-15T22:26:05.696 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-15T22:26:05.696 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-15T22:26:05.696 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-15T22:26:05.696 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-15T22:26:05.696 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-15T22:26:05.697 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-15T22:26:05.697 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-15T22:26:05.697 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-15T22:26:05.697 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-15T22:26:05.697 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-15T22:26:05.697 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-15T22:26:05.697 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-15T22:26:05.697 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-15T22:26:05.697 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-15T22:26:05.697 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-15T22:26:05.697 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-15T22:26:05.697 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-15T22:26:05.698 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-15T22:26:05.698 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-15T22:26:05.698 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-15T22:26:05.698 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-15T22:26:05.698 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-15T22:26:05.698 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-15T22:26:05.698 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-15T22:26:05.698 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-15T22:26:05.698 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-15T22:26:05.698 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-15T22:26:05.698 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-15T22:26:05.698 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-15T22:26:05.698 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-15T22:26:05.698 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-15T22:26:05.698 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-15T22:26:05.699 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-15T22:26:05.699 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-15T22:26:05.699 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-15T22:26:05.699 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-15T22:26:05.699 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-15T22:26:05.699 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-15T22:26:05.699 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-15T22:26:05.699 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-15T22:26:05.699 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-15T22:26:05.699 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-15T22:26:05.699 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-15T22:26:05.699 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-15T22:26:05.699 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-15T22:26:05.699 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-15T22:26:05.699 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-15T22:26:05.700 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-15T22:26:05.700 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-15T22:26:05.700 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-15T22:26:05.700 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-15T22:26:05.700 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-15T22:26:05.700 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-15T22:26:05.700 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-15T22:26:05.700 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-15T22:26:05.700 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-15T22:26:05.700 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-15T22:26:05.700 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-15T22:26:05.700 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-15T22:26:05.700 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-15T22:26:05.700 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-15T22:26:05.700 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-15T22:26:05.701 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-15T22:26:05.701 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-15T22:26:05.701 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-15T22:26:05.701 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-15T22:26:05.701 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-15T22:26:05.701 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-15T22:26:05.701 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-15T22:26:05.701 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-15T22:26:05.701 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-15T22:26:05.701 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-15T22:26:05.701 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-15T22:26:05.701 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-15T22:26:05.701 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-15T22:26:05.701 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-15T22:26:05.701 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-15T22:26:05.702 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-15T22:26:05.702 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-15T22:26:05.702 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-15T22:26:05.702 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-15T22:26:05.702 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-15T22:26:05.702 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-15T22:26:05.702 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-15T22:26:05.702 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-15T22:26:05.702 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-15T22:26:05.702 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-15T22:26:05.702 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-15T22:26:05.702 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-15T22:26:05.702 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-15T22:26:05.702 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-15T22:26:05.702 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-15T22:26:05.703 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-15T22:26:05.703 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-15T22:26:05.703 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-15T22:26:05.703 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-15T22:26:05.703 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-15T22:26:05.703 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-15T22:26:05.703 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-15T22:26:05.703 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-15T22:26:05.703 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-15T22:26:05.703 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-15T22:26:05.703 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-15T22:26:05.703 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-15T22:26:05.703 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-15T22:26:05.703 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-15T22:26:05.703 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-15T22:26:05.704 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-15T22:26:05.704 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-15T22:26:05.704 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-15T22:26:05.704 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-15T22:26:05.704 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-15T22:26:05.704 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-15T22:26:05.704 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-15T22:26:05.704 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-15T22:26:05.704 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-15T22:26:05.704 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-15T22:26:05.704 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-15T22:26:05.704 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-15T22:26:05.704 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-15T22:26:05.704 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-15T22:26:05.705 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-15T22:26:05.705 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-15T22:26:05.705 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-15T22:26:05.705 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-15T22:26:05.705 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-15T22:26:05.705 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-15T22:26:05.705 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-15T22:26:05.705 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-15T22:26:05.705 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-15T22:26:05.705 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-15T22:26:05.705 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-15T22:26:05.705 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-15T22:26:05.705 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-15T22:26:05.705 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-15T22:26:05.706 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-15T22:26:05.706 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-15T22:26:05.706 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-15T22:26:05.706 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-15T22:26:05.706 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-15T22:26:05.706 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-15T22:26:05.706 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-15T22:26:05.706 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-15T22:26:05.706 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-15T22:26:05.706 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-15T22:26:05.706 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-15T22:26:05.706 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-15T22:26:05.706 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-15T22:26:05.706 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-15T22:26:05.706 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-15T22:26:05.707 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-15T22:26:05.707 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-15T22:26:05.707 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-15T22:26:05.707 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-15T22:26:05.707 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-15T22:26:05.707 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-15T22:26:05.707 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-15T22:26:05.707 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-15T22:26:05.707 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-15T22:26:05.707 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-15T22:26:05.707 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-15T22:26:05.707 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-15T22:26:05.707 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-15T22:26:05.707 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-15T22:26:05.707 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-15T22:26:05.708 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-15T22:26:05.708 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-15T22:26:05.708 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-15T22:26:05.708 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-15T22:26:05.708 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-15T22:26:05.708 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-15T22:26:05.708 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-15T22:26:05.708 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-15T22:26:05.708 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-15T22:26:05.708 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-15T22:26:05.708 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-15T22:26:05.708 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-15T22:26:05.708 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-15T22:26:05.708 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-15T22:26:05.708 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-15T22:26:05.709 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-15T22:26:05.709 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-15T22:26:05.709 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-15T22:26:05.709 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-15T22:26:05.709 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-15T22:26:05.709 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-15T22:26:05.709 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-15T22:26:05.709 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-15T22:26:05.709 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-15T22:26:05.709 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-15T22:26:05.709 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-15T22:26:05.709 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-15T22:26:05.709 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-15T22:26:05.709 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-15T22:26:05.710 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-15T22:26:05.710 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-15T22:26:05.710 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-15T22:26:05.710 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-15T22:26:05.710 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-15T22:26:05.710 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-15T22:26:05.710 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-15T22:26:05.710 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-15T22:26:05.710 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-15T22:26:05.710 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-15T22:26:05.710 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-15T22:26:05.710 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-15T22:26:05.710 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-15T22:26:05.710 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-15T22:26:05.710 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-15T22:26:05.711 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-15T22:26:05.711 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-15T22:26:05.711 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-15T22:26:05.711 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-15T22:26:05.711 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-15T22:26:05.711 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-15T22:26:05.711 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-15T22:26:05.711 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-15T22:26:05.711 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-15T22:26:05.711 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-15T22:26:05.711 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-15T22:26:05.711 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-15T22:26:05.711 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-15T22:26:05.711 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-15T22:26:05.711 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-15T22:26:05.711 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-15T22:26:05.712 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-15T22:26:05.712 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-15T22:26:05.712 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-15T22:26:05.712 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-15T22:26:05.712 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-15T22:26:05.712 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-15T22:26:05.712 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-15T22:26:05.712 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-15T22:26:05.712 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-15T22:26:05.712 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-15T22:26:05.712 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-15T22:26:05.712 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-15T22:26:05.712 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-15T22:26:05.712 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-15T22:26:05.713 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-15T22:26:05.713 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-15T22:26:05.713 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-15T22:26:05.713 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-15T22:26:05.713 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-15T22:26:05.713 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-15T22:26:05.713 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-15T22:26:05.713 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-15T22:26:05.713 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-15T22:26:05.713 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-15T22:26:05.713 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-15T22:26:05.713 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-15T22:26:05.713 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-15T22:26:05.714 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-15T22:26:05.714 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-15T22:26:05.714 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-15T22:26:05.714 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-15T22:26:05.714 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-15T22:26:05.714 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-15T22:26:05.714 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-15T22:26:05.714 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-15T22:26:05.714 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-15T22:26:05.714 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-15T22:26:05.714 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-15T22:26:05.714 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-15T22:26:05.714 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-15T22:26:05.714 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-15T22:26:05.715 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-15T22:26:05.715 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-15T22:26:05.715 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-15T22:26:05.715 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-15T22:26:05.715 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-15T22:26:05.715 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-15T22:26:05.715 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-15T22:26:05.715 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-15T22:26:05.715 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-15T22:26:05.715 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-15T22:26:05.715 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-15T22:26:05.715 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-15T22:26:05.715 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-15T22:26:05.716 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-15T22:26:05.716 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-15T22:26:05.716 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-15T22:26:05.716 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-15T22:26:05.716 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-15T22:26:05.716 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-15T22:26:05.716 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-15T22:26:05.716 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-15T22:26:05.716 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-15T22:26:05.716 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-15T22:26:05.716 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-15T22:26:05.716 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-15T22:26:05.716 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-15T22:26:05.717 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-15T22:26:05.717 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-15T22:26:05.717 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-15T22:26:05.717 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-15T22:26:05.717 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-15T22:26:05.717 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-15T22:26:05.717 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-15T22:26:05.717 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-15T22:26:05.717 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-15T22:26:05.717 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-15T22:26:05.717 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-15T22:26:05.717 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-15T22:26:05.717 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-15T22:26:05.717 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-15T22:26:05.718 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-15T22:26:05.718 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-15T22:26:05.718 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-15T22:26:05.718 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-15T22:26:05.718 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-15T22:26:05.718 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-15T22:26:05.718 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-15T22:26:05.718 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-15T22:26:05.718 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-15T22:26:05.718 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-15T22:26:05.718 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-15T22:26:05.718 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-15T22:26:05.718 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-15T22:26:05.718 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-15T22:26:05.719 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-15T22:26:05.719 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-15T22:26:05.719 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-15T22:26:05.719 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-15T22:26:05.719 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-15T22:26:05.719 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-15T22:26:05.719 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-15T22:26:05.719 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-15T22:26:05.719 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-15T22:26:05.719 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-15T22:26:05.719 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-15T22:26:05.719 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-15T22:26:05.719 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-15T22:26:05.719 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-15T22:26:05.720 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-15T22:26:05.720 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-15T22:26:05.720 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-15T22:26:05.720 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-15T22:26:05.720 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-15T22:26:05.720 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-15T22:26:05.720 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-15T22:26:05.720 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-15T22:26:05.720 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-15T22:26:05.720 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-15T22:26:05.720 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-15T22:26:05.720 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-15T22:26:05.720 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-15T22:26:05.720 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-15T22:26:05.721 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-15T22:26:05.721 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-15T22:26:05.721 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-15T22:26:05.721 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-15T22:26:05.721 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-15T22:26:05.721 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-15T22:26:05.721 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-15T22:26:05.721 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-15T22:26:05.721 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-15T22:26:05.721 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-15T22:26:05.721 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-15T22:26:05.721 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-15T22:26:05.721 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-15T22:26:05.721 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-15T22:26:05.722 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-15T22:26:05.722 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-15T22:26:05.722 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-15T22:26:05.722 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-15T22:26:05.722 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-15T22:26:05.722 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-15T22:26:05.722 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-15T22:26:05.722 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-15T22:26:05.722 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-15T22:26:05.722 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-15T22:26:05.722 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-15T22:26:05.722 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-15T22:26:05.722 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-15T22:26:05.722 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-15T22:26:05.723 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-15T22:26:05.723 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-15T22:26:05.723 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-15T22:26:05.723 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-15T22:26:05.723 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-15T22:26:05.723 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-15T22:26:05.723 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-15T22:26:05.723 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-15T22:26:05.723 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-15T22:26:05.723 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-15T22:26:05.723 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-15T22:26:05.723 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-15T22:26:05.723 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-15T22:26:05.723 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-15T22:26:05.724 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-15T22:26:05.724 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-15T22:26:05.724 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-15T22:26:05.724 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-15T22:26:05.724 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-15T22:26:05.724 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-15T22:26:05.724 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-15T22:26:05.724 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-15T22:26:05.724 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-15T22:26:05.724 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-15T22:26:05.724 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-15T22:26:05.724 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-15T22:26:05.724 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-15T22:26:05.725 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-15T22:26:05.725 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-15T22:26:05.725 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-15T22:26:05.725 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-15T22:26:05.725 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-15T22:26:05.725 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-15T22:26:05.725 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-15T22:26:05.725 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-15T22:26:05.725 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-15T22:26:05.725 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-15T22:26:05.725 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-15T22:26:05.725 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-15T22:26:05.725 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-15T22:26:05.725 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-15T22:26:05.726 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-15T22:26:05.726 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-15T22:26:05.726 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-15T22:26:05.726 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-15T22:26:05.726 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-15T22:26:05.726 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-15T22:26:05.726 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-15T22:26:05.726 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-15T22:26:05.726 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-15T22:26:05.726 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-15T22:26:05.726 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-15T22:26:05.726 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-15T22:26:05.726 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-15T22:26:05.726 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-15T22:26:05.726 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-15T22:26:05.726 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-15T22:26:05.727 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-15T22:26:05.727 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-15T22:26:05.727 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-15T22:26:05.727 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-15T22:26:05.727 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-15T22:26:05.727 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-15T22:26:05.727 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-15T22:26:05.727 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-15T22:26:05.727 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-15T22:26:05.727 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-15T22:26:05.727 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-15T22:26:05.727 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-15T22:26:05.727 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-15T22:26:05.727 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-15T22:26:05.727 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-15T22:26:05.727 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-15T22:26:05.728 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-15T22:26:05.728 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-15T22:26:05.728 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-15T22:26:05.728 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-15T22:26:05.728 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-15T22:26:05.728 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-15T22:26:05.728 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-15T22:26:05.728 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-15T22:26:05.728 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-15T22:26:05.728 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-15T22:26:05.728 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-15T22:26:05.728 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-15T22:26:05.728 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-15T22:26:05.729 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-15T22:26:05.729 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-15T22:26:05.729 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-15T22:26:05.729 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-15T22:26:05.729 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-15T22:26:05.729 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-15T22:26:05.729 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-15T22:26:05.729 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-15T22:26:05.729 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-15T22:26:05.729 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-15T22:26:05.729 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-15T22:26:05.729 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-15T22:26:05.729 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-15T22:26:05.729 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-15T22:26:05.729 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-15T22:26:05.730 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-15T22:26:05.730 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-15T22:26:05.730 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-15T22:26:05.730 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-15T22:26:05.730 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-15T22:26:05.730 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-15T22:26:05.730 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-15T22:26:05.730 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-15T22:26:05.730 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-15T22:26:05.730 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-15T22:26:05.730 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-15T22:26:05.730 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-15T22:26:05.730 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-15T22:26:05.730 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-15T22:26:05.731 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-15T22:26:05.731 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-15T22:26:05.731 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-15T22:26:05.731 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-15T22:26:05.731 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-15T22:26:05.731 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-15T22:26:05.731 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-15T22:26:05.731 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-15T22:26:05.731 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-15T22:26:05.731 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-15T22:26:05.731 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-15T22:26:05.731 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-15T22:26:05.731 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-15T22:26:05.731 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-15T22:26:05.731 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-15T22:26:05.732 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-15T22:26:05.732 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-15T22:26:05.732 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-15T22:26:05.732 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-15T22:26:05.732 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-15T22:26:05.732 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-15T22:26:05.732 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-15T22:26:05.732 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-15T22:26:05.732 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-15T22:26:05.732 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-15T22:26:05.732 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-15T22:26:05.732 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-15T22:26:05.732 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-15T22:26:05.733 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-15T22:26:05.733 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-15T22:26:05.733 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-15T22:26:05.733 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-15T22:26:05.733 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-15T22:26:05.733 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-15T22:26:05.733 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-15T22:26:05.733 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-15T22:26:05.733 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-15T22:26:05.733 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-15T22:26:05.733 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-15T22:26:05.733 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-15T22:26:05.733 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-15T22:26:05.733 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-15T22:26:05.734 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-15T22:26:05.734 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-15T22:26:05.734 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-15T22:26:05.734 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-15T22:26:05.734 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-15T22:26:05.734 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-15T22:26:05.734 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-15T22:26:05.734 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-15T22:26:05.734 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-15T22:26:05.734 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-15T22:26:05.734 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-15T22:26:05.734 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-15T22:26:05.734 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-15T22:26:05.735 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-15T22:26:05.735 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-15T22:26:05.735 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-15T22:26:05.735 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-15T22:26:05.735 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-15T22:26:05.735 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-15T22:26:05.735 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-15T22:26:05.735 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-15T22:26:05.735 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-15T22:26:05.735 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-15T22:26:05.735 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-15T22:26:05.735 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-15T22:26:05.735 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-15T22:26:05.735 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-15T22:26:05.736 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-15T22:26:05.736 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-15T22:26:05.736 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-15T22:26:05.736 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-15T22:26:05.736 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-15T22:26:05.736 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-15T22:26:05.736 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-15T22:26:05.736 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-15T22:26:05.736 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-15T22:26:05.736 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-15T22:26:05.736 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-15T22:26:05.736 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-15T22:26:05.736 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-15T22:26:05.736 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-15T22:26:05.737 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-15T22:26:05.737 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-15T22:26:05.737 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-15T22:26:05.737 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-15T22:26:05.737 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-15T22:26:05.737 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-15T22:26:05.737 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-15T22:26:05.737 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-15T22:26:05.737 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-15T22:26:05.737 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-15T22:26:05.737 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-15T22:26:05.737 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-15T22:26:05.737 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-15T22:26:05.738 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-15T22:26:05.738 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-15T22:26:05.738 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-15T22:26:05.738 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-15T22:26:05.738 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-15T22:26:05.738 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-15T22:26:05.738 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-15T22:26:05.738 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-15T22:26:05.738 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-15T22:26:05.738 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-15T22:26:05.738 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-15T22:26:05.738 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-15T22:26:05.738 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-15T22:26:05.738 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-15T22:26:05.739 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-15T22:26:05.739 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-15T22:26:05.739 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-15T22:26:05.739 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-15T22:26:05.739 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-15T22:26:05.739 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-15T22:26:05.739 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-15T22:26:05.739 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-15T22:26:05.739 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-15T22:26:05.739 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-15T22:26:05.739 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-15T22:26:05.739 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-15T22:26:05.739 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-15T22:26:05.739 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-15T22:26:05.739 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-15T22:26:05.740 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-15T22:26:05.740 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-15T22:26:05.740 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-15T22:26:05.740 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-15T22:26:05.740 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-15T22:26:05.740 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-15T22:26:05.740 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-15T22:26:05.740 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-15T22:26:05.740 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-15T22:26:05.740 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-15T22:26:05.740 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-15T22:26:05.740 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-15T22:26:05.740 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-15T22:26:05.741 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-15T22:26:05.741 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-15T22:26:05.741 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-15T22:26:05.741 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-15T22:26:05.741 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-15T22:26:05.741 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-15T22:26:05.741 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-15T22:26:05.741 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-15T22:26:05.741 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-15T22:26:05.741 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-15T22:26:05.741 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-15T22:26:05.741 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-15T22:26:05.741 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-15T22:26:05.742 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-15T22:26:05.742 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-15T22:26:05.742 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-15T22:26:05.742 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-15T22:26:05.742 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-15T22:26:05.742 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-15T22:26:05.742 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-15T22:26:05.742 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-15T22:26:05.742 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-15T22:26:05.742 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-15T22:26:05.742 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-15T22:26:05.742 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-15T22:26:05.742 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-15T22:26:05.742 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-15T22:26:05.743 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-15T22:26:05.743 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-15T22:26:05.743 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-15T22:26:05.743 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-15T22:26:05.743 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-15T22:26:05.743 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-15T22:26:05.743 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-15T22:26:05.743 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-15T22:26:05.743 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-15T22:26:05.743 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-15T22:26:05.743 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-15T22:26:05.743 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-15T22:26:05.743 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-15T22:26:05.743 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-15T22:26:05.744 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-15T22:26:05.744 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-15T22:26:05.744 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-15T22:26:05.744 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-15T22:26:05.744 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-15T22:26:05.744 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-15T22:26:05.744 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-15T22:26:05.744 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-15T22:26:05.744 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-15T22:26:05.744 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-15T22:26:05.744 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-15T22:26:05.744 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-15T22:26:05.744 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-15T22:26:05.745 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-15T22:26:05.745 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-15T22:26:05.745 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-15T22:26:05.745 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-15T22:26:05.745 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-15T22:26:05.745 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-15T22:26:05.745 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-15T22:26:05.745 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-15T22:26:05.745 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-15T22:26:05.745 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-15T22:26:05.745 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-15T22:26:05.745 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-15T22:26:05.745 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-15T22:26:05.746 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-15T22:26:05.746 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-15T22:26:05.746 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-15T22:26:05.746 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-15T22:26:05.746 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-15T22:26:05.746 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-15T22:26:05.746 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-15T22:26:05.746 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-15T22:26:05.746 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-15T22:26:05.746 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-15T22:26:05.746 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-15T22:26:05.746 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-15T22:26:05.746 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-15T22:26:05.746 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-15T22:26:05.747 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-15T22:26:05.747 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-15T22:26:05.747 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-15T22:26:05.747 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-15T22:26:05.747 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-15T22:26:05.747 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-15T22:26:05.747 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-15T22:26:05.747 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-15T22:26:05.747 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-15T22:26:05.747 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-15T22:26:05.747 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-15T22:26:05.747 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-15T22:26:05.747 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-15T22:26:05.748 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-15T22:26:05.748 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-15T22:26:05.748 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-15T22:26:05.748 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-15T22:26:05.748 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-15T22:26:05.748 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-15T22:26:05.748 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-15T22:26:05.748 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-15T22:26:05.748 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-15T22:26:05.748 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-15T22:26:05.748 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-15T22:26:05.748 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-15T22:26:05.749 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-15T22:26:05.749 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-15T22:26:05.749 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-15T22:26:05.749 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-15T22:26:05.749 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-15T22:26:05.749 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-15T22:26:05.749 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-15T22:26:05.749 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-15T22:26:05.749 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-15T22:26:05.749 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-15T22:26:05.749 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-15T22:26:05.749 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-15T22:26:05.750 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-15T22:26:05.750 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-15T22:26:05.750 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-15T22:26:05.750 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-15T22:26:05.750 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-15T22:26:05.750 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-15T22:26:05.750 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-15T22:26:05.750 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-15T22:26:05.750 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-15T22:26:05.750 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-15T22:26:05.750 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-15T22:26:05.750 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-15T22:26:05.750 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-15T22:26:05.751 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-15T22:26:05.751 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-15T22:26:05.751 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-15T22:26:05.751 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-15T22:26:05.751 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-15T22:26:05.751 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-15T22:26:05.751 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-15T22:26:05.751 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-15T22:26:05.751 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-15T22:26:05.751 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-15T22:26:05.751 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-15T22:26:05.751 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-15T22:26:05.752 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-15T22:26:05.752 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-15T22:26:05.752 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-15T22:26:05.752 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-15T22:26:05.752 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-15T22:26:05.752 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-15T22:26:05.752 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-15T22:26:05.752 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-15T22:26:05.752 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-15T22:26:05.752 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-15T22:26:05.752 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-15T22:26:05.752 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-15T22:26:05.753 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-15T22:26:05.753 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-15T22:26:05.753 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-15T22:26:05.753 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-15T22:26:05.753 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-15T22:26:05.753 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-15T22:26:05.753 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-15T22:26:05.753 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-15T22:26:05.753 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-15T22:26:05.753 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-15T22:26:05.753 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-15T22:26:05.753 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-15T22:26:05.753 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-15T22:26:05.753 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-15T22:26:05.754 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-15T22:26:05.754 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-15T22:26:05.754 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-15T22:26:05.754 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-15T22:26:05.754 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-15T22:26:05.754 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-15T22:26:05.754 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-15T22:26:05.754 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-15T22:26:05.754 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-15T22:26:05.754 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-15T22:26:05.754 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-15T22:26:05.754 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-15T22:26:05.754 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-15T22:26:05.754 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-15T22:26:05.754 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-15T22:26:05.755 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-15T22:26:05.755 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-15T22:26:05.755 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-15T22:26:05.755 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-15T22:26:05.755 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-15T22:26:05.755 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-15T22:26:05.755 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-15T22:26:05.755 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-15T22:26:05.755 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-15T22:26:05.755 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-15T22:26:05.755 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-15T22:26:05.755 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-15T22:26:05.755 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-15T22:26:05.756 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-15T22:26:05.756 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-15T22:26:05.756 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-15T22:26:05.756 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-15T22:26:05.756 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-15T22:26:05.756 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-15T22:26:05.756 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-15T22:26:05.756 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-15T22:26:05.756 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-15T22:26:05.756 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-15T22:26:05.756 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-15T22:26:05.756 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-15T22:26:05.756 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-15T22:26:05.757 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-15T22:26:05.757 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-15T22:26:05.757 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-15T22:26:05.757 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-15T22:26:05.757 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-15T22:26:05.757 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-15T22:26:05.757 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-15T22:26:05.757 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-15T22:26:05.757 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-15T22:26:05.761 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-15T22:26:05.761 DEBUG:teuthology.orchestra.run.smithi111:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-15T22:26:05.785 INFO:teuthology.orchestra.run.smithi111.stdout:postinstall scriptlet (using /bin/sh): 2024-09-15T22:26:05.785 INFO:teuthology.orchestra.run.smithi111.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-15T22:26:05.785 INFO:teuthology.orchestra.run.smithi111.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-15T22:26:05.785 INFO:teuthology.orchestra.run.smithi111.stdout:fi 2024-09-15T22:26:05.785 INFO:teuthology.orchestra.run.smithi111.stdout:for file in vmlinuz System.map config; do 2024-09-15T22:26:05.785 INFO:teuthology.orchestra.run.smithi111.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-15T22:26:05.785 INFO:teuthology.orchestra.run.smithi111.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-15T22:26:05.785 INFO:teuthology.orchestra.run.smithi111.stdout: fi 2024-09-15T22:26:05.785 INFO:teuthology.orchestra.run.smithi111.stdout:done 2024-09-15T22:26:05.785 INFO:teuthology.orchestra.run.smithi111.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-15T22:26:05.785 INFO:teuthology.orchestra.run.smithi111.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-15T22:26:05.786 INFO:teuthology.orchestra.run.smithi111.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-15T22:26:05.786 INFO:teuthology.orchestra.run.smithi111.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-15T22:26:05.786 INFO:teuthology.orchestra.run.smithi111.stdout:fi 2024-09-15T22:26:05.786 INFO:teuthology.orchestra.run.smithi111.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-15T22:26:05.786 INFO:teuthology.orchestra.run.smithi111.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-15T22:26:05.786 INFO:teuthology.orchestra.run.smithi111.stdout:fi 2024-09-15T22:26:05.786 INFO:teuthology.orchestra.run.smithi111.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-15T22:26:05.786 INFO:teuthology.orchestra.run.smithi111.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-15T22:26:05.786 INFO:teuthology.orchestra.run.smithi111.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-15T22:26:05.786 INFO:teuthology.orchestra.run.smithi111.stdout:fi 2024-09-15T22:26:05.787 DEBUG:teuthology.orchestra.run.smithi111:> sudo rpm -qi grub2-tools 2024-09-15T22:26:05.859 INFO:teuthology.orchestra.run.smithi111.stdout:Name : grub2-tools 2024-09-15T22:26:05.859 INFO:teuthology.orchestra.run.smithi111.stdout:Epoch : 1 2024-09-15T22:26:05.860 INFO:teuthology.orchestra.run.smithi111.stdout:Version : 2.06 2024-09-15T22:26:05.860 INFO:teuthology.orchestra.run.smithi111.stdout:Release : 82.el9 2024-09-15T22:26:05.860 INFO:teuthology.orchestra.run.smithi111.stdout:Architecture: x86_64 2024-09-15T22:26:05.860 INFO:teuthology.orchestra.run.smithi111.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-15T22:26:05.860 INFO:teuthology.orchestra.run.smithi111.stdout:Group : Unspecified 2024-09-15T22:26:05.860 INFO:teuthology.orchestra.run.smithi111.stdout:Size : 8274670 2024-09-15T22:26:05.860 INFO:teuthology.orchestra.run.smithi111.stdout:License : GPLv3+ 2024-09-15T22:26:05.860 INFO:teuthology.orchestra.run.smithi111.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-15T22:26:05.860 INFO:teuthology.orchestra.run.smithi111.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-15T22:26:05.860 INFO:teuthology.orchestra.run.smithi111.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-15T22:26:05.860 INFO:teuthology.orchestra.run.smithi111.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-15T22:26:05.860 INFO:teuthology.orchestra.run.smithi111.stdout:Packager : builder@centos.org 2024-09-15T22:26:05.860 INFO:teuthology.orchestra.run.smithi111.stdout:Vendor : CentOS 2024-09-15T22:26:05.860 INFO:teuthology.orchestra.run.smithi111.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-15T22:26:05.861 INFO:teuthology.orchestra.run.smithi111.stdout:Summary : Support tools for GRUB. 2024-09-15T22:26:05.861 INFO:teuthology.orchestra.run.smithi111.stdout:Description : 2024-09-15T22:26:05.861 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:26:05.861 INFO:teuthology.orchestra.run.smithi111.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-15T22:26:05.861 INFO:teuthology.orchestra.run.smithi111.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-15T22:26:05.861 INFO:teuthology.orchestra.run.smithi111.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-15T22:26:05.861 INFO:teuthology.orchestra.run.smithi111.stdout:hardware devices. 2024-09-15T22:26:05.861 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:26:05.861 INFO:teuthology.orchestra.run.smithi111.stdout:This subpackage provides tools for support of all platforms. 2024-09-15T22:26:05.862 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-15T22:26:05.862 INFO:teuthology.task.kernel:Updating grub on smithi111 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-15T22:26:05.862 DEBUG:teuthology.orchestra.run.smithi111:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-15T22:26:06.496 INFO:teuthology.orchestra.run.smithi111.stderr:Generating grub configuration file ... 2024-09-15T22:26:07.852 INFO:teuthology.orchestra.run.smithi111.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-15T22:26:07.882 INFO:teuthology.orchestra.run.smithi111.stderr:done 2024-09-15T22:26:07.884 DEBUG:teuthology.orchestra.run.smithi111:> mktemp 2024-09-15T22:26:07.900 INFO:teuthology.orchestra.run.smithi111.stdout:/tmp/tmp.nVTTsvrXrS 2024-09-15T22:26:07.900 DEBUG:teuthology.orchestra.run.smithi111:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.nVTTsvrXrS 2024-09-15T22:26:07.966 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0666 /tmp/tmp.nVTTsvrXrS 2024-09-15T22:26:08.105 DEBUG:teuthology.orchestra.remote:smithi111:/tmp/tmp.nVTTsvrXrS is 6KB 2024-09-15T22:26:08.154 DEBUG:teuthology.orchestra.run.smithi111:> rm -fr /tmp/tmp.nVTTsvrXrS 2024-09-15T22:26:08.169 DEBUG:teuthology.orchestra.run.smithi111:> sudo /bin/ls /boot/loader/entries || true 2024-09-15T22:26:08.239 INFO:teuthology.orchestra.run.smithi111.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-15T22:26:08.239 INFO:teuthology.orchestra.run.smithi111.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-15T22:26:08.240 INFO:teuthology.orchestra.run.smithi111.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-15T22:26:08.240 INFO:teuthology.orchestra.run.smithi111.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-15T22:26:08.241 DEBUG:teuthology.orchestra.run.smithi111:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-15T22:26:08.393 DEBUG:teuthology.orchestra.run.smithi111:> sudo shutdown -r now 2024-09-15T22:26:38.396 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-15T22:26:38.397 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-15T22:26:38.398 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T22:26:56.961 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-09-15T22:27:05.962 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-15T22:27:05.963 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T22:27:09.025 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-09-15T22:27:21.025 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-15T22:27:21.026 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T22:27:24.093 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-09-15T22:27:39.094 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-15T22:27:39.096 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T22:27:57.629 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2024-09-15T22:28:15.630 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-15T22:28:15.631 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T22:28:16.478 DEBUG:teuthology.orchestra.run.smithi111:> true 2024-09-15T22:28:17.109 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-15T22:28:17.109 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-15T22:28:17.110 DEBUG:teuthology.orchestra.run.smithi111:> uname -r 2024-09-15T22:28:17.145 INFO:teuthology.orchestra.run.smithi111.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-15T22:28:17.145 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-15T22:28:17.146 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T22:28:17.146 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T22:28:18.147 DEBUG:teuthology.parallel:result is None 2024-09-15T22:28:18.147 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-15T22:28:18.160 INFO:teuthology.task.internal:Creating test directory... 2024-09-15T22:28:18.160 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T22:28:18.164 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T22:28:18.167 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T22:28:18.195 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-15T22:28:18.202 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-15T22:28:18.211 INFO:teuthology.task.internal:Creating archive directory... 2024-09-15T22:28:18.212 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T22:28:18.239 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T22:28:18.242 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T22:28:18.389 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-15T22:28:18.397 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-15T22:28:18.397 DEBUG:teuthology.orchestra.run.smithi121:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T22:28:18.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T22:28:18.416 DEBUG:teuthology.orchestra.run.smithi192:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T22:28:18.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T22:28:18.434 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T22:28:18.438 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T22:28:18.459 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T22:28:18.502 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T22:28:18.517 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T22:28:18.526 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T22:28:18.547 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T22:28:18.602 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T22:28:18.631 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T22:28:18.634 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-15T22:28:18.642 INFO:teuthology.task.internal:Configuring sudo... 2024-09-15T22:28:18.642 DEBUG:teuthology.orchestra.run.smithi111:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T22:28:18.646 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T22:28:18.648 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T22:28:18.701 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-15T22:28:18.712 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-15T22:28:18.712 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T22:28:18.716 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T22:28:18.719 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T22:28:18.748 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T22:28:18.855 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T22:28:18.941 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T22:28:19.005 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T22:28:19.081 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-15T22:28:19.082 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T22:28:19.136 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T22:28:19.186 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T22:28:19.234 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T22:28:19.319 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T22:28:19.360 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-15T22:28:19.361 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T22:28:19.427 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T22:28:19.474 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T22:28:19.533 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T22:28:19.573 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T22:28:19.620 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T22:28:19.620 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T22:28:19.687 DEBUG:teuthology.orchestra.run.smithi111:> sudo service rsyslog restart 2024-09-15T22:28:19.690 DEBUG:teuthology.orchestra.run.smithi121:> sudo service rsyslog restart 2024-09-15T22:28:19.693 DEBUG:teuthology.orchestra.run.smithi192:> sudo service rsyslog restart 2024-09-15T22:28:19.730 INFO:teuthology.orchestra.run.smithi121.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T22:28:19.764 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T22:28:19.773 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T22:28:20.195 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-15T22:28:20.205 INFO:teuthology.task.internal:Starting timer... 2024-09-15T22:28:20.205 INFO:teuthology.run_tasks:Running task pcp... 2024-09-15T22:28:20.280 INFO:teuthology.run_tasks:Running task selinux... 2024-09-15T22:28:20.289 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-15T22:28:20.290 DEBUG:teuthology.orchestra.run.smithi111:> sudo service auditd rotate 2024-09-15T22:28:20.432 INFO:teuthology.orchestra.run.smithi111.stdout:Rotating logs: 2024-09-15T22:28:20.437 DEBUG:teuthology.orchestra.run.smithi121:> sudo service auditd rotate 2024-09-15T22:28:20.493 INFO:teuthology.orchestra.run.smithi121.stdout:Rotating logs: 2024-09-15T22:28:20.495 DEBUG:teuthology.orchestra.run.smithi192:> sudo service auditd rotate 2024-09-15T22:28:20.550 INFO:teuthology.orchestra.run.smithi192.stdout:Rotating logs: 2024-09-15T22:28:20.552 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-15T22:28:20.552 DEBUG:teuthology.orchestra.run.smithi111:> /usr/sbin/getenforce 2024-09-15T22:28:20.595 INFO:teuthology.orchestra.run.smithi111.stdout:Permissive 2024-09-15T22:28:20.595 DEBUG:teuthology.orchestra.run.smithi121:> /usr/sbin/getenforce 2024-09-15T22:28:20.629 INFO:teuthology.orchestra.run.smithi121.stdout:Permissive 2024-09-15T22:28:20.629 DEBUG:teuthology.orchestra.run.smithi192:> /usr/sbin/getenforce 2024-09-15T22:28:20.660 INFO:teuthology.orchestra.run.smithi192.stdout:Permissive 2024-09-15T22:28:20.661 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi111.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi121.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi192.front.sepia.ceph.com': 'permissive'} 2024-09-15T22:28:20.661 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T22:28:20.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T22:28:20.719 DEBUG:teuthology.orchestra.run.smithi121:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T22:28:20.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T22:28:20.748 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T22:28:20.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T22:28:20.774 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-15T22:28:20.775 DEBUG:teuthology.orchestra.run.smithi111:> sudo /usr/sbin/setenforce permissive 2024-09-15T22:28:20.843 DEBUG:teuthology.orchestra.run.smithi121:> sudo /usr/sbin/setenforce permissive 2024-09-15T22:28:20.879 DEBUG:teuthology.orchestra.run.smithi192:> sudo /usr/sbin/setenforce permissive 2024-09-15T22:28:20.907 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-15T22:28:20.917 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-15T22:28:20.934 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-15T22:28:20.935 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi111.front.sepia.ceph.com,smithi121.front.sepia.ceph.com,smithi192.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-15T22:33:02.414 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi111.front.sepia.ceph.com'), Remote(name='ubuntu@smithi121.front.sepia.ceph.com'), Remote(name='ubuntu@smithi192.front.sepia.ceph.com')] 2024-09-15T22:33:02.415 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-15T22:33:02.416 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T22:33:02.500 DEBUG:teuthology.orchestra.run.smithi111:> true 2024-09-15T22:33:02.622 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com' 2024-09-15T22:33:02.623 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-09-15T22:33:02.623 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T22:33:02.699 DEBUG:teuthology.orchestra.run.smithi121:> true 2024-09-15T22:33:02.773 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-09-15T22:33:02.773 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-09-15T22:33:02.774 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T22:33:02.857 DEBUG:teuthology.orchestra.run.smithi192:> true 2024-09-15T22:33:02.938 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-09-15T22:33:02.938 INFO:teuthology.run_tasks:Running task clock... 2024-09-15T22:33:03.129 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-15T22:33:03.129 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T22:33:03.130 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T22:33:03.134 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T22:33:03.134 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T22:33:03.136 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T22:33:03.136 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T22:33:03.167 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T22:33:03.168 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T22:33:03.182 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T22:33:03.184 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T22:33:03.191 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T22:33:03.207 INFO:teuthology.orchestra.run.smithi192.stderr:sudo: ntpd: command not found 2024-09-15T22:33:03.211 INFO:teuthology.orchestra.run.smithi121.stderr:sudo: ntpd: command not found 2024-09-15T22:33:03.218 INFO:teuthology.orchestra.run.smithi192.stdout:506 Cannot talk to daemon 2024-09-15T22:33:03.220 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T22:33:03.222 INFO:teuthology.orchestra.run.smithi121.stdout:506 Cannot talk to daemon 2024-09-15T22:33:03.232 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T22:33:03.237 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T22:33:03.246 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T22:33:03.252 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T22:33:03.271 INFO:teuthology.orchestra.run.smithi111.stderr:sudo: ntpd: command not found 2024-09-15T22:33:03.295 INFO:teuthology.orchestra.run.smithi111.stdout:506 Cannot talk to daemon 2024-09-15T22:33:03.304 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-09-15T22:33:03.308 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T22:33:03.308 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-09-15T22:33:03.308 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:33:03.308 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:33:03.308 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:33:03.309 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:33:03.312 INFO:teuthology.orchestra.run.smithi121.stderr:bash: line 1: ntpq: command not found 2024-09-15T22:33:03.316 INFO:teuthology.orchestra.run.smithi121.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T22:33:03.316 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================== 2024-09-15T22:33:03.316 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:33:03.316 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:33:03.316 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:33:03.316 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:33:03.323 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T22:33:03.353 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T22:33:03.444 INFO:teuthology.orchestra.run.smithi111.stderr:bash: line 1: ntpq: command not found 2024-09-15T22:33:03.451 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T22:33:03.451 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2024-09-15T22:33:03.451 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:33:03.451 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:33:03.452 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:33:03.452 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:33:03.453 INFO:teuthology.run_tasks:Running task install... 2024-09-15T22:33:03.461 DEBUG:teuthology.task.install:project ceph 2024-09-15T22:33:03.461 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}} 2024-09-15T22:33:03.462 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'} 2024-09-15T22:33:03.462 INFO:teuthology.task.install:Using flavor: default 2024-09-15T22:33:03.467 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-15T22:33:03.467 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-15T22:33:03.468 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-15T22:33:03.468 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:33:03.470 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-15T22:33:03.470 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:33:03.471 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-15T22:33:03.471 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:33:03.619 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-15T22:33:03.620 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-15T22:33:03.623 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-15T22:33:03.623 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-15T22:33:03.635 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-15T22:33:03.635 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-15T22:33:03.728 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T22:33:03.728 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T22:33:03.728 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T22:33:03.738 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T22:33:03.739 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-15T22:33:03.739 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T22:33:03.741 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T22:33:03.741 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-15T22:33:03.741 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T22:33:03.754 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-15T22:33:03.754 DEBUG:teuthology.orchestra.run.smithi192:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T22:33:03.768 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-15T22:33:03.768 DEBUG:teuthology.orchestra.run.smithi121:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T22:33:03.797 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-15T22:33:03.797 DEBUG:teuthology.orchestra.run.smithi111:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T22:33:03.820 DEBUG:teuthology.orchestra.run.smithi192:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T22:33:03.834 DEBUG:teuthology.orchestra.run.smithi121:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T22:33:03.895 DEBUG:teuthology.orchestra.run.smithi111:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T22:33:03.896 DEBUG:teuthology.orchestra.run.smithi192:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T22:33:03.910 DEBUG:teuthology.orchestra.run.smithi121:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T22:33:03.960 INFO:teuthology.orchestra.run.smithi192.stdout:check_obsoletes = 1 2024-09-15T22:33:03.962 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum clean all 2024-09-15T22:33:03.973 DEBUG:teuthology.orchestra.run.smithi111:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T22:33:03.974 INFO:teuthology.orchestra.run.smithi121.stdout:check_obsoletes = 1 2024-09-15T22:33:03.976 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum clean all 2024-09-15T22:33:04.028 INFO:teuthology.orchestra.run.smithi111.stdout:check_obsoletes = 1 2024-09-15T22:33:04.031 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum clean all 2024-09-15T22:33:04.187 INFO:teuthology.orchestra.run.smithi192.stdout:45 files removed 2024-09-15T22:33:04.201 INFO:teuthology.orchestra.run.smithi121.stdout:45 files removed 2024-09-15T22:33:04.214 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-15T22:33:04.227 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-15T22:33:04.299 INFO:teuthology.orchestra.run.smithi111.stdout:45 files removed 2024-09-15T22:33:04.332 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-15T22:33:04.750 INFO:teuthology.orchestra.run.smithi192.stdout:ceph packages for x86_64 325 kB/s | 83 kB 00:00 2024-09-15T22:33:04.780 INFO:teuthology.orchestra.run.smithi121.stdout:ceph packages for x86_64 302 kB/s | 83 kB 00:00 2024-09-15T22:33:05.018 INFO:teuthology.orchestra.run.smithi111.stdout:ceph packages for x86_64 286 kB/s | 83 kB 00:00 2024-09-15T22:33:05.026 INFO:teuthology.orchestra.run.smithi192.stdout:ceph noarch packages 50 kB/s | 12 kB 00:00 2024-09-15T22:33:05.064 INFO:teuthology.orchestra.run.smithi121.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-15T22:33:05.293 INFO:teuthology.orchestra.run.smithi192.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-15T22:33:05.327 INFO:teuthology.orchestra.run.smithi111.stdout:ceph noarch packages 44 kB/s | 12 kB 00:00 2024-09-15T22:33:05.340 INFO:teuthology.orchestra.run.smithi121.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-15T22:33:05.603 INFO:teuthology.orchestra.run.smithi111.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-15T22:33:05.827 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-15T22:33:06.089 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-15T22:33:06.128 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-15T22:33:07.745 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-15T22:33:08.013 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-15T22:33:08.266 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - AppStream 19 MB/s | 20 MB 00:01 2024-09-15T22:33:12.691 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-15T22:33:13.353 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-15T22:33:13.806 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-15T22:33:14.150 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - Extras packages 72 kB/s | 19 kB 00:00 2024-09-15T22:33:14.812 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 76 kB/s | 19 kB 00:00 2024-09-15T22:33:15.531 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 19 kB 00:00 2024-09-15T22:33:15.679 INFO:teuthology.orchestra.run.smithi121.stdout:Extra Packages for Enterprise Linux 16 MB/s | 23 MB 00:01 2024-09-15T22:33:15.959 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-15T22:33:16.654 INFO:teuthology.orchestra.run.smithi111.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-15T22:33:21.126 INFO:teuthology.orchestra.run.smithi121.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-09-15T22:33:21.422 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-15T22:33:23.051 INFO:teuthology.orchestra.run.smithi111.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-15T22:33:23.380 INFO:teuthology.orchestra.run.smithi121.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T22:33:23.381 INFO:teuthology.orchestra.run.smithi121.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T22:33:23.389 INFO:teuthology.orchestra.run.smithi121.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-15T22:33:23.390 INFO:teuthology.orchestra.run.smithi121.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-15T22:33:23.391 INFO:teuthology.orchestra.run.smithi121.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-15T22:33:23.392 INFO:teuthology.orchestra.run.smithi121.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-15T22:33:23.393 INFO:teuthology.orchestra.run.smithi121.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-15T22:33:23.395 INFO:teuthology.orchestra.run.smithi121.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-15T22:33:23.396 INFO:teuthology.orchestra.run.smithi121.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-15T22:33:23.458 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-15T22:33:23.481 INFO:teuthology.orchestra.run.smithi121.stdout:====================================================================================== 2024-09-15T22:33:23.481 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-09-15T22:33:23.481 INFO:teuthology.orchestra.run.smithi121.stdout:====================================================================================== 2024-09-15T22:33:23.481 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-09-15T22:33:23.481 INFO:teuthology.orchestra.run.smithi121.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-15T22:33:23.481 INFO:teuthology.orchestra.run.smithi121.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-15T22:33:23.481 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-15T22:33:23.481 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-15T22:33:23.482 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-15T22:33:23.482 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-15T22:33:23.482 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-15T22:33:23.482 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-15T22:33:23.482 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-15T22:33:23.482 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-15T22:33:23.482 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-15T22:33:23.482 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-15T22:33:23.482 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-15T22:33:23.482 INFO:teuthology.orchestra.run.smithi121.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-15T22:33:23.483 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-15T22:33:23.483 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-15T22:33:23.483 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-15T22:33:23.483 INFO:teuthology.orchestra.run.smithi121.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-15T22:33:23.483 INFO:teuthology.orchestra.run.smithi121.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-15T22:33:23.483 INFO:teuthology.orchestra.run.smithi121.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-15T22:33:23.483 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-15T22:33:23.483 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-15T22:33:23.483 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-15T22:33:23.483 INFO:teuthology.orchestra.run.smithi121.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-15T22:33:23.483 INFO:teuthology.orchestra.run.smithi121.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-15T22:33:23.484 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-15T22:33:23.484 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-15T22:33:23.484 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-15T22:33:23.484 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-15T22:33:23.484 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-15T22:33:23.484 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-15T22:33:23.484 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-15T22:33:23.484 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-15T22:33:23.484 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-15T22:33:23.484 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-15T22:33:23.484 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-15T22:33:23.485 INFO:teuthology.orchestra.run.smithi121.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-15T22:33:23.485 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrading: 2024-09-15T22:33:23.485 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-15T22:33:23.485 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-15T22:33:23.485 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-15T22:33:23.485 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-15T22:33:23.485 INFO:teuthology.orchestra.run.smithi121.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-15T22:33:23.485 INFO:teuthology.orchestra.run.smithi121.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-15T22:33:23.485 INFO:teuthology.orchestra.run.smithi121.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-15T22:33:23.485 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-15T22:33:23.486 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-09-15T22:33:23.486 INFO:teuthology.orchestra.run.smithi121.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T22:33:23.486 INFO:teuthology.orchestra.run.smithi121.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-15T22:33:23.486 INFO:teuthology.orchestra.run.smithi121.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T22:33:23.486 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-15T22:33:23.486 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-15T22:33:23.486 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-15T22:33:23.486 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-15T22:33:23.486 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-15T22:33:23.486 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-15T22:33:23.486 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-15T22:33:23.486 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-15T22:33:23.487 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T22:33:23.487 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T22:33:23.487 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T22:33:23.487 INFO:teuthology.orchestra.run.smithi121.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T22:33:23.487 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T22:33:23.487 INFO:teuthology.orchestra.run.smithi121.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T22:33:23.487 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T22:33:23.487 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T22:33:23.488 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-15T22:33:23.488 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T22:33:23.488 INFO:teuthology.orchestra.run.smithi121.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T22:33:23.488 INFO:teuthology.orchestra.run.smithi121.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T22:33:23.488 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T22:33:23.488 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T22:33:23.489 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-15T22:33:23.489 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T22:33:23.489 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-15T22:33:23.489 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T22:33:23.489 INFO:teuthology.orchestra.run.smithi121.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-15T22:33:23.489 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T22:33:23.489 INFO:teuthology.orchestra.run.smithi121.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T22:33:23.489 INFO:teuthology.orchestra.run.smithi121.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T22:33:23.490 INFO:teuthology.orchestra.run.smithi121.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T22:33:23.490 INFO:teuthology.orchestra.run.smithi121.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T22:33:23.490 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T22:33:23.490 INFO:teuthology.orchestra.run.smithi121.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T22:33:23.490 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-15T22:33:23.490 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T22:33:23.490 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T22:33:23.490 INFO:teuthology.orchestra.run.smithi121.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T22:33:23.490 INFO:teuthology.orchestra.run.smithi121.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-15T22:33:23.490 INFO:teuthology.orchestra.run.smithi121.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T22:33:23.490 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T22:33:23.490 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T22:33:23.490 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-15T22:33:23.491 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-15T22:33:23.491 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T22:33:23.491 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T22:33:23.491 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T22:33:23.491 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T22:33:23.491 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-15T22:33:23.491 INFO:teuthology.orchestra.run.smithi121.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-15T22:33:23.491 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T22:33:23.491 INFO:teuthology.orchestra.run.smithi121.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T22:33:23.491 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T22:33:23.491 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T22:33:23.491 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T22:33:23.492 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T22:33:23.492 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T22:33:23.492 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-15T22:33:23.492 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T22:33:23.492 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T22:33:23.492 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T22:33:23.492 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T22:33:23.492 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T22:33:23.492 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T22:33:23.492 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T22:33:23.493 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T22:33:23.493 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T22:33:23.493 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T22:33:23.493 INFO:teuthology.orchestra.run.smithi121.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T22:33:23.493 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T22:33:23.493 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T22:33:23.493 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T22:33:23.493 INFO:teuthology.orchestra.run.smithi121.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T22:33:23.493 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T22:33:23.493 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T22:33:23.493 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T22:33:23.493 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T22:33:23.493 INFO:teuthology.orchestra.run.smithi121.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T22:33:23.494 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T22:33:23.494 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T22:33:23.494 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T22:33:23.494 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T22:33:23.494 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-15T22:33:23.494 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T22:33:23.494 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T22:33:23.494 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T22:33:23.494 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T22:33:23.494 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T22:33:23.494 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T22:33:23.494 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T22:33:23.494 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T22:33:23.495 INFO:teuthology.orchestra.run.smithi121.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T22:33:23.495 INFO:teuthology.orchestra.run.smithi121.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T22:33:23.495 INFO:teuthology.orchestra.run.smithi121.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T22:33:23.495 INFO:teuthology.orchestra.run.smithi121.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-15T22:33:23.495 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T22:33:23.495 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T22:33:23.495 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T22:33:23.495 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T22:33:23.495 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T22:33:23.495 INFO:teuthology.orchestra.run.smithi121.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T22:33:23.495 INFO:teuthology.orchestra.run.smithi121.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T22:33:23.495 INFO:teuthology.orchestra.run.smithi121.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T22:33:23.495 INFO:teuthology.orchestra.run.smithi121.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T22:33:23.495 INFO:teuthology.orchestra.run.smithi121.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-15T22:33:23.496 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T22:33:23.496 INFO:teuthology.orchestra.run.smithi121.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-15T22:33:23.496 INFO:teuthology.orchestra.run.smithi121.stdout:Installing weak dependencies: 2024-09-15T22:33:23.496 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T22:33:23.496 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-15T22:33:23.496 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-09-15T22:33:23.496 INFO:teuthology.orchestra.run.smithi121.stdout:====================================================================================== 2024-09-15T22:33:23.496 INFO:teuthology.orchestra.run.smithi121.stdout:Install 148 Packages 2024-09-15T22:33:23.496 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrade 8 Packages 2024-09-15T22:33:23.496 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-15T22:33:23.497 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 226 M 2024-09-15T22:33:23.497 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-09-15T22:33:23.702 INFO:teuthology.orchestra.run.smithi192.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T22:33:23.703 INFO:teuthology.orchestra.run.smithi192.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T22:33:23.716 INFO:teuthology.orchestra.run.smithi192.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-15T22:33:23.717 INFO:teuthology.orchestra.run.smithi192.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-15T22:33:23.718 INFO:teuthology.orchestra.run.smithi192.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-15T22:33:23.719 INFO:teuthology.orchestra.run.smithi192.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-15T22:33:23.721 INFO:teuthology.orchestra.run.smithi192.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-15T22:33:23.724 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-15T22:33:23.725 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-15T22:33:23.789 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-15T22:33:23.810 INFO:teuthology.orchestra.run.smithi192.stdout:====================================================================================== 2024-09-15T22:33:23.810 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-15T22:33:23.810 INFO:teuthology.orchestra.run.smithi192.stdout:====================================================================================== 2024-09-15T22:33:23.810 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-15T22:33:23.810 INFO:teuthology.orchestra.run.smithi192.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-15T22:33:23.811 INFO:teuthology.orchestra.run.smithi192.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-15T22:33:23.811 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-15T22:33:23.811 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-15T22:33:23.811 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-15T22:33:23.811 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-15T22:33:23.811 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-15T22:33:23.811 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-15T22:33:23.811 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-15T22:33:23.811 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-15T22:33:23.811 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-15T22:33:23.811 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-15T22:33:23.811 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-15T22:33:23.812 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-15T22:33:23.812 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-15T22:33:23.812 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-15T22:33:23.812 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-15T22:33:23.812 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-15T22:33:23.812 INFO:teuthology.orchestra.run.smithi192.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-15T22:33:23.812 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-15T22:33:23.812 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-15T22:33:23.812 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-15T22:33:23.812 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-15T22:33:23.812 INFO:teuthology.orchestra.run.smithi192.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-15T22:33:23.812 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-15T22:33:23.813 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-15T22:33:23.813 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-15T22:33:23.813 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-15T22:33:23.813 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-15T22:33:23.813 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-15T22:33:23.813 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-15T22:33:23.813 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-15T22:33:23.813 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-15T22:33:23.813 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-15T22:33:23.813 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-15T22:33:23.813 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-15T22:33:23.813 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-15T22:33:23.814 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrading: 2024-09-15T22:33:23.814 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-15T22:33:23.814 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-15T22:33:23.814 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-15T22:33:23.814 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-15T22:33:23.814 INFO:teuthology.orchestra.run.smithi192.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-15T22:33:23.814 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-15T22:33:23.814 INFO:teuthology.orchestra.run.smithi192.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-15T22:33:23.814 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-15T22:33:23.814 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-15T22:33:23.814 INFO:teuthology.orchestra.run.smithi192.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T22:33:23.814 INFO:teuthology.orchestra.run.smithi192.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-15T22:33:23.814 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T22:33:23.815 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-15T22:33:23.815 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-15T22:33:23.815 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-15T22:33:23.815 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-15T22:33:23.815 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-15T22:33:23.815 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-15T22:33:23.815 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-15T22:33:23.815 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-15T22:33:23.815 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T22:33:23.815 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T22:33:23.815 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T22:33:23.815 INFO:teuthology.orchestra.run.smithi192.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T22:33:23.815 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T22:33:23.816 INFO:teuthology.orchestra.run.smithi192.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T22:33:23.816 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T22:33:23.816 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T22:33:23.816 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-15T22:33:23.816 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T22:33:23.816 INFO:teuthology.orchestra.run.smithi192.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T22:33:23.816 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T22:33:23.816 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T22:33:23.816 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T22:33:23.816 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-15T22:33:23.816 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T22:33:23.816 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-15T22:33:23.816 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T22:33:23.817 INFO:teuthology.orchestra.run.smithi192.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-15T22:33:23.817 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T22:33:23.817 INFO:teuthology.orchestra.run.smithi192.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T22:33:23.817 INFO:teuthology.orchestra.run.smithi192.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T22:33:23.817 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T22:33:23.817 INFO:teuthology.orchestra.run.smithi192.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T22:33:23.817 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T22:33:23.817 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T22:33:23.817 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-15T22:33:23.817 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T22:33:23.817 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T22:33:23.817 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T22:33:23.817 INFO:teuthology.orchestra.run.smithi192.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-15T22:33:23.818 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T22:33:23.818 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T22:33:23.818 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T22:33:23.818 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-15T22:33:23.818 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-15T22:33:23.818 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T22:33:23.818 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T22:33:23.818 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T22:33:23.818 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T22:33:23.818 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-15T22:33:23.819 INFO:teuthology.orchestra.run.smithi192.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-15T22:33:23.819 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T22:33:23.819 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T22:33:23.819 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T22:33:23.819 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T22:33:23.819 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T22:33:23.819 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T22:33:23.819 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T22:33:23.819 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-15T22:33:23.819 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T22:33:23.819 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T22:33:23.819 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T22:33:23.819 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T22:33:23.820 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T22:33:23.820 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T22:33:23.820 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T22:33:23.820 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T22:33:23.820 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T22:33:23.820 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T22:33:23.820 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T22:33:23.820 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T22:33:23.820 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T22:33:23.820 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T22:33:23.820 INFO:teuthology.orchestra.run.smithi192.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T22:33:23.821 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T22:33:23.821 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T22:33:23.821 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T22:33:23.821 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T22:33:23.821 INFO:teuthology.orchestra.run.smithi192.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T22:33:23.821 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T22:33:23.821 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T22:33:23.821 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T22:33:23.821 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T22:33:23.821 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-15T22:33:23.821 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T22:33:23.821 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T22:33:23.822 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T22:33:23.822 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T22:33:23.822 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T22:33:23.822 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T22:33:23.822 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T22:33:23.822 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T22:33:23.822 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T22:33:23.822 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T22:33:23.822 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T22:33:23.822 INFO:teuthology.orchestra.run.smithi192.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-15T22:33:23.822 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T22:33:23.822 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T22:33:23.823 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T22:33:23.823 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T22:33:23.823 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T22:33:23.823 INFO:teuthology.orchestra.run.smithi192.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T22:33:23.823 INFO:teuthology.orchestra.run.smithi192.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T22:33:23.823 INFO:teuthology.orchestra.run.smithi192.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T22:33:23.823 INFO:teuthology.orchestra.run.smithi192.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T22:33:23.823 INFO:teuthology.orchestra.run.smithi192.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-15T22:33:23.823 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T22:33:23.823 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-15T22:33:23.823 INFO:teuthology.orchestra.run.smithi192.stdout:Installing weak dependencies: 2024-09-15T22:33:23.824 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T22:33:23.824 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T22:33:23.824 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-15T22:33:23.824 INFO:teuthology.orchestra.run.smithi192.stdout:====================================================================================== 2024-09-15T22:33:23.824 INFO:teuthology.orchestra.run.smithi192.stdout:Install 148 Packages 2024-09-15T22:33:23.824 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrade 8 Packages 2024-09-15T22:33:23.824 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T22:33:23.824 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 226 M 2024-09-15T22:33:23.825 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-15T22:33:23.919 INFO:teuthology.orchestra.run.smithi121.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 56 kB/s | 6.5 kB 00:00 2024-09-15T22:33:24.094 INFO:teuthology.orchestra.run.smithi121.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 5.0 MB/s | 896 kB 00:00 2024-09-15T22:33:24.153 INFO:teuthology.orchestra.run.smithi121.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 2.4 MB/s | 145 kB 00:00 2024-09-15T22:33:24.295 INFO:teuthology.orchestra.run.smithi121.stdout:(4/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-15T22:33:24.379 INFO:teuthology.orchestra.run.smithi121.stdout:(5/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 11 MB/s | 2.4 MB 00:00 2024-09-15T22:33:24.429 INFO:teuthology.orchestra.run.smithi121.stdout:(6/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 11 MB/s | 1.5 MB 00:00 2024-09-15T22:33:24.465 INFO:teuthology.orchestra.run.smithi192.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 40 kB/s | 6.5 kB 00:00 2024-09-15T22:33:24.754 INFO:teuthology.orchestra.run.smithi121.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 13 MB/s | 4.7 MB 00:00 2024-09-15T22:33:25.132 INFO:teuthology.orchestra.run.smithi192.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 1.3 MB/s | 896 kB 00:00 2024-09-15T22:33:25.171 INFO:teuthology.orchestra.run.smithi121.stdout:(8/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 15 MB/s | 21 MB 00:01 2024-09-15T22:33:25.199 INFO:teuthology.orchestra.run.smithi192.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 2.1 MB/s | 145 kB 00:00 2024-09-15T22:33:25.213 INFO:teuthology.orchestra.run.smithi121.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 610 kB/s | 25 kB 00:00 2024-09-15T22:33:25.472 INFO:teuthology.orchestra.run.smithi121.stdout:(10/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 16 MB/s | 17 MB 00:01 2024-09-15T22:33:25.588 INFO:teuthology.orchestra.run.smithi121.stdout:(11/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 13 MB/s | 11 MB 00:00 2024-09-15T22:33:25.606 INFO:teuthology.orchestra.run.smithi111.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T22:33:25.607 INFO:teuthology.orchestra.run.smithi111.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T22:33:25.613 INFO:teuthology.orchestra.run.smithi121.stdout:(12/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 1.6 MB/s | 228 kB 00:00 2024-09-15T22:33:25.618 INFO:teuthology.orchestra.run.smithi111.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-15T22:33:25.619 INFO:teuthology.orchestra.run.smithi111.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-15T22:33:25.620 INFO:teuthology.orchestra.run.smithi111.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-15T22:33:25.621 INFO:teuthology.orchestra.run.smithi111.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-15T22:33:25.623 INFO:teuthology.orchestra.run.smithi111.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-15T22:33:25.625 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-15T22:33:25.626 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-15T22:33:25.639 INFO:teuthology.orchestra.run.smithi121.stdout:(13/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 541 kB/s | 27 kB 00:00 2024-09-15T22:33:25.664 INFO:teuthology.orchestra.run.smithi121.stdout:(14/156): libcephfs-devel-19.1.1-326.g8921c8ef. 630 kB/s | 32 kB 00:00 2024-09-15T22:33:25.690 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2024-09-15T22:33:25.691 INFO:teuthology.orchestra.run.smithi192.stdout:(4/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 4.9 MB/s | 2.4 MB 00:00 2024-09-15T22:33:25.706 INFO:teuthology.orchestra.run.smithi121.stdout:(15/156): libcephsqlite-19.1.1-326.g8921c8ef.el 3.9 MB/s | 163 kB 00:00 2024-09-15T22:33:25.708 INFO:teuthology.orchestra.run.smithi111.stdout:====================================================================================== 2024-09-15T22:33:25.708 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2024-09-15T22:33:25.708 INFO:teuthology.orchestra.run.smithi111.stdout:====================================================================================== 2024-09-15T22:33:25.708 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2024-09-15T22:33:25.708 INFO:teuthology.orchestra.run.smithi111.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-15T22:33:25.709 INFO:teuthology.orchestra.run.smithi111.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-15T22:33:25.709 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-15T22:33:25.709 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-15T22:33:25.709 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-15T22:33:25.709 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-15T22:33:25.709 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-15T22:33:25.709 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-15T22:33:25.709 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-15T22:33:25.709 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-15T22:33:25.709 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-15T22:33:25.709 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-15T22:33:25.709 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-15T22:33:25.710 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-15T22:33:25.710 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-15T22:33:25.710 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-15T22:33:25.710 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-15T22:33:25.710 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-15T22:33:25.710 INFO:teuthology.orchestra.run.smithi111.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-15T22:33:25.710 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-15T22:33:25.710 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-15T22:33:25.710 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-15T22:33:25.710 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-15T22:33:25.710 INFO:teuthology.orchestra.run.smithi111.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-15T22:33:25.710 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-15T22:33:25.710 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-15T22:33:25.710 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-15T22:33:25.710 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-15T22:33:25.711 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-15T22:33:25.711 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-15T22:33:25.711 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-15T22:33:25.711 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-15T22:33:25.711 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-15T22:33:25.711 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-15T22:33:25.711 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-15T22:33:25.711 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-15T22:33:25.711 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-15T22:33:25.711 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrading: 2024-09-15T22:33:25.711 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-15T22:33:25.711 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-15T22:33:25.711 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-15T22:33:25.711 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-15T22:33:25.712 INFO:teuthology.orchestra.run.smithi111.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-15T22:33:25.712 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-15T22:33:25.712 INFO:teuthology.orchestra.run.smithi111.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-15T22:33:25.712 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-15T22:33:25.712 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2024-09-15T22:33:25.712 INFO:teuthology.orchestra.run.smithi111.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T22:33:25.712 INFO:teuthology.orchestra.run.smithi111.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-15T22:33:25.712 INFO:teuthology.orchestra.run.smithi111.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T22:33:25.712 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-15T22:33:25.712 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-15T22:33:25.712 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-15T22:33:25.712 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-15T22:33:25.712 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-15T22:33:25.712 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-15T22:33:25.712 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-15T22:33:25.713 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-15T22:33:25.713 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T22:33:25.713 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T22:33:25.713 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T22:33:25.713 INFO:teuthology.orchestra.run.smithi111.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T22:33:25.713 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T22:33:25.713 INFO:teuthology.orchestra.run.smithi111.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T22:33:25.713 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T22:33:25.713 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T22:33:25.713 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-15T22:33:25.713 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T22:33:25.713 INFO:teuthology.orchestra.run.smithi111.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T22:33:25.713 INFO:teuthology.orchestra.run.smithi111.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T22:33:25.714 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T22:33:25.714 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T22:33:25.714 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-15T22:33:25.714 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T22:33:25.714 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-15T22:33:25.714 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T22:33:25.714 INFO:teuthology.orchestra.run.smithi111.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-15T22:33:25.714 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T22:33:25.714 INFO:teuthology.orchestra.run.smithi111.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T22:33:25.714 INFO:teuthology.orchestra.run.smithi111.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T22:33:25.714 INFO:teuthology.orchestra.run.smithi111.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T22:33:25.714 INFO:teuthology.orchestra.run.smithi111.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T22:33:25.715 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T22:33:25.715 INFO:teuthology.orchestra.run.smithi111.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T22:33:25.715 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-15T22:33:25.715 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T22:33:25.715 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T22:33:25.715 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T22:33:25.715 INFO:teuthology.orchestra.run.smithi111.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-15T22:33:25.715 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T22:33:25.715 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T22:33:25.715 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T22:33:25.715 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-15T22:33:25.715 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-15T22:33:25.716 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T22:33:25.716 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T22:33:25.716 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T22:33:25.716 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T22:33:25.716 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-15T22:33:25.716 INFO:teuthology.orchestra.run.smithi111.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-15T22:33:25.716 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T22:33:25.716 INFO:teuthology.orchestra.run.smithi111.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T22:33:25.716 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T22:33:25.716 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T22:33:25.716 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T22:33:25.716 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T22:33:25.716 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T22:33:25.716 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-15T22:33:25.717 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T22:33:25.717 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T22:33:25.717 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T22:33:25.717 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T22:33:25.717 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T22:33:25.717 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T22:33:25.717 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T22:33:25.717 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T22:33:25.717 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T22:33:25.717 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T22:33:25.717 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T22:33:25.717 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T22:33:25.717 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T22:33:25.717 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T22:33:25.718 INFO:teuthology.orchestra.run.smithi111.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T22:33:25.718 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T22:33:25.718 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T22:33:25.718 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T22:33:25.718 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T22:33:25.718 INFO:teuthology.orchestra.run.smithi111.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T22:33:25.718 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T22:33:25.718 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T22:33:25.718 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T22:33:25.718 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T22:33:25.718 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-15T22:33:25.718 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T22:33:25.718 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T22:33:25.718 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T22:33:25.719 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T22:33:25.719 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T22:33:25.719 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T22:33:25.719 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T22:33:25.719 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T22:33:25.719 INFO:teuthology.orchestra.run.smithi111.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T22:33:25.719 INFO:teuthology.orchestra.run.smithi111.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T22:33:25.719 INFO:teuthology.orchestra.run.smithi111.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T22:33:25.719 INFO:teuthology.orchestra.run.smithi111.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-15T22:33:25.719 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T22:33:25.719 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T22:33:25.719 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T22:33:25.719 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T22:33:25.719 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T22:33:25.719 INFO:teuthology.orchestra.run.smithi111.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T22:33:25.720 INFO:teuthology.orchestra.run.smithi111.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T22:33:25.720 INFO:teuthology.orchestra.run.smithi111.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T22:33:25.720 INFO:teuthology.orchestra.run.smithi111.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T22:33:25.720 INFO:teuthology.orchestra.run.smithi111.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-15T22:33:25.720 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T22:33:25.720 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-15T22:33:25.720 INFO:teuthology.orchestra.run.smithi111.stdout:Installing weak dependencies: 2024-09-15T22:33:25.720 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T22:33:25.720 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:33:25.720 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2024-09-15T22:33:25.720 INFO:teuthology.orchestra.run.smithi111.stdout:====================================================================================== 2024-09-15T22:33:25.720 INFO:teuthology.orchestra.run.smithi111.stdout:Install 148 Packages 2024-09-15T22:33:25.720 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrade 8 Packages 2024-09-15T22:33:25.720 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:33:25.721 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 226 M 2024-09-15T22:33:25.721 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2024-09-15T22:33:25.748 INFO:teuthology.orchestra.run.smithi121.stdout:(16/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 6.7 MB/s | 751 kB 00:00 2024-09-15T22:33:25.773 INFO:teuthology.orchestra.run.smithi121.stdout:(17/156): librados-devel-19.1.1-326.g8921c8ef.e 1.9 MB/s | 127 kB 00:00 2024-09-15T22:33:25.882 INFO:teuthology.orchestra.run.smithi121.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 3.7 MB/s | 506 kB 00:00 2024-09-15T22:33:25.950 INFO:teuthology.orchestra.run.smithi192.stdout:(5/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 3.3 MB/s | 5.4 MB 00:01 2024-09-15T22:33:25.966 INFO:teuthology.orchestra.run.smithi121.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 541 kB/s | 45 kB 00:00 2024-09-15T22:33:26.041 INFO:teuthology.orchestra.run.smithi192.stdout:(6/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 4.3 MB/s | 1.5 MB 00:00 2024-09-15T22:33:26.058 INFO:teuthology.orchestra.run.smithi121.stdout:(20/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 19 MB/s | 5.3 MB 00:00 2024-09-15T22:33:26.083 INFO:teuthology.orchestra.run.smithi121.stdout:(21/156): python3-ceph-common-19.1.1-326.g8921c 1.1 MB/s | 136 kB 00:00 2024-09-15T22:33:26.116 INFO:teuthology.orchestra.run.smithi121.stdout:(22/156): python3-cephfs-19.1.1-326.g8921c8ef.e 2.7 MB/s | 161 kB 00:00 2024-09-15T22:33:26.158 INFO:teuthology.orchestra.run.smithi121.stdout:(23/156): python3-rados-19.1.1-326.g8921c8ef.el 4.2 MB/s | 320 kB 00:00 2024-09-15T22:33:26.192 INFO:teuthology.orchestra.run.smithi121.stdout:(24/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 3.9 MB/s | 300 kB 00:00 2024-09-15T22:33:26.234 INFO:teuthology.orchestra.run.smithi121.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.3 MB/s | 99 kB 00:00 2024-09-15T22:33:26.253 INFO:teuthology.orchestra.run.smithi111.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 31 kB/s | 6.5 kB 00:00 2024-09-15T22:33:26.259 INFO:teuthology.orchestra.run.smithi121.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-15T22:33:26.343 INFO:teuthology.orchestra.run.smithi121.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 2.0 MB/s | 170 kB 00:00 2024-09-15T22:33:26.418 INFO:teuthology.orchestra.run.smithi121.stdout:(28/156): ceph-grafana-dashboards-19.1.1-326.g8 425 kB/s | 32 kB 00:00 2024-09-15T22:33:26.510 INFO:teuthology.orchestra.run.smithi121.stdout:(29/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 1.6 MB/s | 146 kB 00:00 2024-09-15T22:33:26.625 INFO:teuthology.orchestra.run.smithi192.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 7.0 MB/s | 4.7 MB 00:00 2024-09-15T22:33:26.802 INFO:teuthology.orchestra.run.smithi121.stdout:(30/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 12 MB/s | 3.6 MB 00:00 2024-09-15T22:33:26.861 INFO:teuthology.orchestra.run.smithi121.stdout:(31/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 5.0 MB/s | 3.1 MB 00:00 2024-09-15T22:33:26.944 INFO:teuthology.orchestra.run.smithi121.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 2.9 MB/s | 249 kB 00:00 2024-09-15T22:33:27.045 INFO:teuthology.orchestra.run.smithi121.stdout:(33/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 493 kB/s | 49 kB 00:00 2024-09-15T22:33:27.120 INFO:teuthology.orchestra.run.smithi111.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 1.0 MB/s | 896 kB 00:00 2024-09-15T22:33:27.128 INFO:teuthology.orchestra.run.smithi121.stdout:(34/156): ceph-prometheus-alerts-19.1.1-326.g89 198 kB/s | 16 kB 00:00 2024-09-15T22:33:27.221 INFO:teuthology.orchestra.run.smithi121.stdout:(35/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 3.0 MB/s | 285 kB 00:00 2024-09-15T22:33:27.295 INFO:teuthology.orchestra.run.smithi111.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 832 kB/s | 145 kB 00:00 2024-09-15T22:33:27.362 INFO:teuthology.orchestra.run.smithi121.stdout:(36/156): ceph-mgr-diskprediction-local-19.1.1- 13 MB/s | 7.4 MB 00:00 2024-09-15T22:33:27.413 INFO:teuthology.orchestra.run.smithi121.stdout:(37/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 3.9 MB/s | 765 kB 00:00 2024-09-15T22:33:27.438 INFO:teuthology.orchestra.run.smithi121.stdout:(38/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 243 kB/s | 18 kB 00:00 2024-09-15T22:33:27.522 INFO:teuthology.orchestra.run.smithi121.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 736 kB/s | 80 kB 00:00 2024-09-15T22:33:27.580 INFO:teuthology.orchestra.run.smithi121.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-15T22:33:27.614 INFO:teuthology.orchestra.run.smithi121.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 4.5 MB/s | 801 kB 00:00 2024-09-15T22:33:27.639 INFO:teuthology.orchestra.run.smithi121.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-09-15T22:33:27.681 INFO:teuthology.orchestra.run.smithi121.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-15T22:33:27.870 INFO:teuthology.orchestra.run.smithi111.stdout:(4/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 3.0 MB/s | 5.4 MB 00:01 2024-09-15T22:33:28.017 INFO:teuthology.orchestra.run.smithi111.stdout:(5/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 3.3 MB/s | 2.4 MB 00:00 2024-09-15T22:33:28.075 INFO:teuthology.orchestra.run.smithi192.stdout:(8/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9. 7.3 MB/s | 11 MB 00:01 2024-09-15T22:33:28.131 INFO:teuthology.orchestra.run.smithi121.stdout:(44/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 17 MB/s | 49 MB 00:02 2024-09-15T22:33:28.159 INFO:teuthology.orchestra.run.smithi121.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 283 kB/s | 135 kB 00:00 2024-09-15T22:33:28.162 INFO:teuthology.orchestra.run.smithi111.stdout:(6/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 5.2 MB/s | 1.5 MB 00:00 2024-09-15T22:33:28.193 INFO:teuthology.orchestra.run.smithi121.stdout:(46/156): python3-cryptography-36.0.1-4.el9.x86 2.1 MB/s | 1.2 MB 00:00 2024-09-15T22:33:28.235 INFO:teuthology.orchestra.run.smithi121.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-09-15T22:33:28.260 INFO:teuthology.orchestra.run.smithi121.stdout:(48/156): python3-requests-2.25.1-8.el9.noarch. 974 kB/s | 125 kB 00:00 2024-09-15T22:33:28.267 INFO:teuthology.orchestra.run.smithi192.stdout:(9/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86_ 7.6 MB/s | 17 MB 00:02 2024-09-15T22:33:28.293 INFO:teuthology.orchestra.run.smithi192.stdout:(10/156): ceph-selinux-19.1.1-326.g8921c8ef.el9 116 kB/s | 25 kB 00:00 2024-09-15T22:33:28.351 INFO:teuthology.orchestra.run.smithi192.stdout:(11/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 3.8 MB/s | 228 kB 00:00 2024-09-15T22:33:28.401 INFO:teuthology.orchestra.run.smithi192.stdout:(12/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 541 kB/s | 27 kB 00:00 2024-09-15T22:33:28.463 INFO:teuthology.orchestra.run.smithi111.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 11 MB/s | 4.7 MB 00:00 2024-09-15T22:33:28.602 INFO:teuthology.orchestra.run.smithi192.stdout:(13/156): ceph-common-19.1.1-326.g8921c8ef.el9. 4.9 MB/s | 21 MB 00:04 2024-09-15T22:33:28.627 INFO:teuthology.orchestra.run.smithi192.stdout:(14/156): libcephfs-devel-19.1.1-326.g8921c8ef. 141 kB/s | 32 kB 00:00 2024-09-15T22:33:28.685 INFO:teuthology.orchestra.run.smithi192.stdout:(15/156): libcephsqlite-19.1.1-326.g8921c8ef.el 2.7 MB/s | 163 kB 00:00 2024-09-15T22:33:28.736 INFO:teuthology.orchestra.run.smithi192.stdout:(16/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 5.5 MB/s | 751 kB 00:00 2024-09-15T22:33:28.761 INFO:teuthology.orchestra.run.smithi192.stdout:(17/156): librados-devel-19.1.1-326.g8921c8ef.e 1.6 MB/s | 127 kB 00:00 2024-09-15T22:33:28.845 INFO:teuthology.orchestra.run.smithi192.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 4.5 MB/s | 506 kB 00:00 2024-09-15T22:33:28.904 INFO:teuthology.orchestra.run.smithi192.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 775 kB/s | 45 kB 00:00 2024-09-15T22:33:28.954 INFO:teuthology.orchestra.run.smithi192.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 2.7 MB/s | 136 kB 00:00 2024-09-15T22:33:28.977 INFO:teuthology.orchestra.run.smithi121.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 149 kB/s | 107 kB 00:00 2024-09-15T22:33:29.021 INFO:teuthology.orchestra.run.smithi192.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 2.4 MB/s | 161 kB 00:00 2024-09-15T22:33:29.038 INFO:teuthology.orchestra.run.smithi111.stdout:(8/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9. 18 MB/s | 11 MB 00:00 2024-09-15T22:33:29.069 INFO:teuthology.orchestra.run.smithi121.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 787 kB/s | 689 kB 00:00 2024-09-15T22:33:29.105 INFO:teuthology.orchestra.run.smithi192.stdout:(22/156): python3-rados-19.1.1-326.g8921c8ef.el 3.8 MB/s | 320 kB 00:00 2024-09-15T22:33:29.105 INFO:teuthology.orchestra.run.smithi111.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 380 kB/s | 25 kB 00:00 2024-09-15T22:33:29.127 INFO:teuthology.orchestra.run.smithi121.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-09-15T22:33:29.247 INFO:teuthology.orchestra.run.smithi192.stdout:(23/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 11 MB/s | 5.3 MB 00:00 2024-09-15T22:33:29.272 INFO:teuthology.orchestra.run.smithi192.stdout:(24/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 1.8 MB/s | 300 kB 00:00 2024-09-15T22:33:29.305 INFO:teuthology.orchestra.run.smithi192.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.7 MB/s | 99 kB 00:00 2024-09-15T22:33:29.331 INFO:teuthology.orchestra.run.smithi192.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.4 MB/s | 85 kB 00:00 2024-09-15T22:33:29.406 INFO:teuthology.orchestra.run.smithi192.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 2.2 MB/s | 170 kB 00:00 2024-09-15T22:33:29.481 INFO:teuthology.orchestra.run.smithi192.stdout:(28/156): ceph-grafana-dashboards-19.1.1-326.g8 425 kB/s | 32 kB 00:00 2024-09-15T22:33:29.557 INFO:teuthology.orchestra.run.smithi192.stdout:(29/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 1.9 MB/s | 146 kB 00:00 2024-09-15T22:33:29.569 INFO:teuthology.orchestra.run.smithi121.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 37 kB/s | 22 kB 00:00 2024-09-15T22:33:29.670 INFO:teuthology.orchestra.run.smithi121.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 60 kB/s | 33 kB 00:00 2024-09-15T22:33:29.715 INFO:teuthology.orchestra.run.smithi192.stdout:(30/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 7.6 MB/s | 3.1 MB 00:00 2024-09-15T22:33:29.861 INFO:teuthology.orchestra.run.smithi121.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 396 kB/s | 313 kB 00:00 2024-09-15T22:33:30.203 INFO:teuthology.orchestra.run.smithi121.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 34 kB/s | 18 kB 00:00 2024-09-15T22:33:30.345 INFO:teuthology.orchestra.run.smithi121.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 23 kB/s | 11 kB 00:00 2024-09-15T22:33:30.441 INFO:teuthology.orchestra.run.smithi192.stdout:(31/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 4.0 MB/s | 3.6 MB 00:00 2024-09-15T22:33:30.541 INFO:teuthology.orchestra.run.smithi192.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 2.4 MB/s | 249 kB 00:00 2024-09-15T22:33:30.591 INFO:teuthology.orchestra.run.smithi192.stdout:(33/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 990 kB/s | 49 kB 00:00 2024-09-15T22:33:30.604 INFO:teuthology.orchestra.run.smithi121.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.9 MB/s | 3.0 MB 00:01 2024-09-15T22:33:30.642 INFO:teuthology.orchestra.run.smithi192.stdout:(34/156): ceph-prometheus-alerts-19.1.1-326.g89 329 kB/s | 16 kB 00:00 2024-09-15T22:33:30.734 INFO:teuthology.orchestra.run.smithi192.stdout:(35/156): ceph-mgr-diskprediction-local-19.1.1- 7.3 MB/s | 7.4 MB 00:01 2024-09-15T22:33:30.767 INFO:teuthology.orchestra.run.smithi192.stdout:(36/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 2.2 MB/s | 285 kB 00:00 2024-09-15T22:33:30.817 INFO:teuthology.orchestra.run.smithi192.stdout:(37/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 9.0 MB/s | 765 kB 00:00 2024-09-15T22:33:30.843 INFO:teuthology.orchestra.run.smithi192.stdout:(38/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 242 kB/s | 18 kB 00:00 2024-09-15T22:33:30.945 INFO:teuthology.orchestra.run.smithi121.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 216 kB/s | 160 kB 00:00 2024-09-15T22:33:31.012 INFO:teuthology.orchestra.run.smithi121.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 68 kB/s | 45 kB 00:00 2024-09-15T22:33:31.047 INFO:teuthology.orchestra.run.smithi111.stdout:(10/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 25 MB/s | 49 MB 00:01 2024-09-15T22:33:31.097 INFO:teuthology.orchestra.run.smithi111.stdout:(11/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 4.5 MB/s | 228 kB 00:00 2024-09-15T22:33:31.147 INFO:teuthology.orchestra.run.smithi111.stdout:(12/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 545 kB/s | 27 kB 00:00 2024-09-15T22:33:31.189 INFO:teuthology.orchestra.run.smithi111.stdout:(13/156): libcephfs-devel-19.1.1-326.g8921c8ef. 762 kB/s | 32 kB 00:00 2024-09-15T22:33:31.218 INFO:teuthology.orchestra.run.smithi192.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 199 kB/s | 80 kB 00:00 2024-09-15T22:33:31.285 INFO:teuthology.orchestra.run.smithi111.stdout:(14/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 7.7 MB/s | 751 kB 00:00 2024-09-15T22:33:31.339 INFO:teuthology.orchestra.run.smithi111.stdout:(15/156): libcephsqlite-19.1.1-326.g8921c8ef.el 3.0 MB/s | 163 kB 00:00 2024-09-15T22:33:31.418 INFO:teuthology.orchestra.run.smithi192.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 957 kB/s | 191 kB 00:00 2024-09-15T22:33:31.463 INFO:teuthology.orchestra.run.smithi121.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 779 kB/s | 662 kB 00:00 2024-09-15T22:33:31.532 INFO:teuthology.orchestra.run.smithi111.stdout:(16/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 5.0 MB/s | 17 MB 00:03 2024-09-15T22:33:31.535 INFO:teuthology.orchestra.run.smithi192.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-09-15T22:33:31.557 INFO:teuthology.orchestra.run.smithi111.stdout:(17/156): librados-devel-19.1.1-326.g8921c8ef.e 584 kB/s | 127 kB 00:00 2024-09-15T22:33:31.563 INFO:teuthology.orchestra.run.smithi121.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 44 kB/s | 24 kB 00:00 2024-09-15T22:33:31.577 INFO:teuthology.orchestra.run.smithi192.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 801 kB 00:00 2024-09-15T22:33:31.663 INFO:teuthology.orchestra.run.smithi111.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 3.8 MB/s | 506 kB 00:00 2024-09-15T22:33:31.686 INFO:teuthology.orchestra.run.smithi192.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 986 kB/s | 106 kB 00:00 2024-09-15T22:33:31.725 INFO:teuthology.orchestra.run.smithi111.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 740 kB/s | 45 kB 00:00 2024-09-15T22:33:31.796 INFO:teuthology.orchestra.run.smithi121.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 683 kB/s | 581 kB 00:00 2024-09-15T22:33:31.803 INFO:teuthology.orchestra.run.smithi192.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-09-15T22:33:31.808 INFO:teuthology.orchestra.run.smithi111.stdout:(20/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 21 MB/s | 5.3 MB 00:00 2024-09-15T22:33:31.833 INFO:teuthology.orchestra.run.smithi111.stdout:(21/156): python3-ceph-common-19.1.1-326.g8921c 1.2 MB/s | 136 kB 00:00 2024-09-15T22:33:31.845 INFO:teuthology.orchestra.run.smithi192.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 4.0 MB/s | 1.2 MB 00:00 2024-09-15T22:33:31.858 INFO:teuthology.orchestra.run.smithi111.stdout:(22/156): python3-cephfs-19.1.1-326.g8921c8ef.e 3.2 MB/s | 161 kB 00:00 2024-09-15T22:33:31.909 INFO:teuthology.orchestra.run.smithi111.stdout:(23/156): python3-rados-19.1.1-326.g8921c8ef.el 4.2 MB/s | 320 kB 00:00 2024-09-15T22:33:31.920 INFO:teuthology.orchestra.run.smithi192.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-09-15T22:33:31.942 INFO:teuthology.orchestra.run.smithi111.stdout:(24/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 3.5 MB/s | 300 kB 00:00 2024-09-15T22:33:31.968 INFO:teuthology.orchestra.run.smithi111.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.7 MB/s | 99 kB 00:00 2024-09-15T22:33:32.018 INFO:teuthology.orchestra.run.smithi111.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.1 MB/s | 85 kB 00:00 2024-09-15T22:33:32.038 INFO:teuthology.orchestra.run.smithi121.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 82 kB/s | 47 kB 00:00 2024-09-15T22:33:32.068 INFO:teuthology.orchestra.run.smithi111.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 3.3 MB/s | 170 kB 00:00 2024-09-15T22:33:32.118 INFO:teuthology.orchestra.run.smithi111.stdout:(28/156): ceph-grafana-dashboards-19.1.1-326.g8 641 kB/s | 32 kB 00:00 2024-09-15T22:33:32.169 INFO:teuthology.orchestra.run.smithi111.stdout:(29/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 2.9 MB/s | 146 kB 00:00 2024-09-15T22:33:32.311 INFO:teuthology.orchestra.run.smithi111.stdout:(30/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 25 MB/s | 3.6 MB 00:00 2024-09-15T22:33:32.354 INFO:teuthology.orchestra.run.smithi192.stdout:(47/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 12 MB/s | 49 MB 00:04 2024-09-15T22:33:32.372 INFO:teuthology.orchestra.run.smithi121.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 362 kB/s | 292 kB 00:00 2024-09-15T22:33:32.379 INFO:teuthology.orchestra.run.smithi192.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 402 kB/s | 215 kB 00:00 2024-09-15T22:33:32.478 INFO:teuthology.orchestra.run.smithi111.stdout:(31/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 6.1 MB/s | 3.1 MB 00:00 2024-09-15T22:33:32.504 INFO:teuthology.orchestra.run.smithi192.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 856 kB/s | 107 kB 00:00 2024-09-15T22:33:32.530 INFO:teuthology.orchestra.run.smithi192.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 1.1 MB/s | 689 kB 00:00 2024-09-15T22:33:32.555 INFO:teuthology.orchestra.run.smithi192.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-09-15T22:33:32.555 INFO:teuthology.orchestra.run.smithi121.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 251 kB/s | 188 kB 00:00 2024-09-15T22:33:32.578 INFO:teuthology.orchestra.run.smithi111.stdout:(32/156): ceph-mgr-diskprediction-local-19.1.1- 28 MB/s | 7.4 MB 00:00 2024-09-15T22:33:32.580 INFO:teuthology.orchestra.run.smithi192.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-15T22:33:32.595 INFO:teuthology.orchestra.run.smithi111.stdout:(33/156): ceph-mgr-modules-core-19.1.1-326.g892 2.1 MB/s | 249 kB 00:00 2024-09-15T22:33:32.605 INFO:teuthology.orchestra.run.smithi192.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 652 kB/s | 33 kB 00:00 2024-09-15T22:33:32.614 INFO:teuthology.orchestra.run.smithi121.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 68 kB/s | 38 kB 00:00 2024-09-15T22:33:32.628 INFO:teuthology.orchestra.run.smithi111.stdout:(34/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 991 kB/s | 49 kB 00:00 2024-09-15T22:33:32.647 INFO:teuthology.orchestra.run.smithi192.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 943 kB 00:00 2024-09-15T22:33:32.673 INFO:teuthology.orchestra.run.smithi192.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 272 kB/s | 18 kB 00:00 2024-09-15T22:33:32.698 INFO:teuthology.orchestra.run.smithi192.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-15T22:33:32.723 INFO:teuthology.orchestra.run.smithi192.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-15T22:33:32.737 INFO:teuthology.orchestra.run.smithi111.stdout:(35/156): ceph-prometheus-alerts-19.1.1-326.g89 116 kB/s | 16 kB 00:00 2024-09-15T22:33:32.749 INFO:teuthology.orchestra.run.smithi192.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-09-15T22:33:32.762 INFO:teuthology.orchestra.run.smithi111.stdout:(36/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 2.1 MB/s | 285 kB 00:00 2024-09-15T22:33:32.782 INFO:teuthology.orchestra.run.smithi192.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-15T22:33:32.804 INFO:teuthology.orchestra.run.smithi111.stdout:(37/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 440 kB/s | 18 kB 00:00 2024-09-15T22:33:32.841 INFO:teuthology.orchestra.run.smithi192.stdout:(60/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-09-15T22:33:32.854 INFO:teuthology.orchestra.run.smithi111.stdout:(38/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 6.4 MB/s | 765 kB 00:00 2024-09-15T22:33:32.883 INFO:teuthology.orchestra.run.smithi192.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 244 kB/s | 24 kB 00:00 2024-09-15T22:33:32.897 INFO:teuthology.orchestra.run.smithi111.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 868 kB/s | 80 kB 00:00 2024-09-15T22:33:32.908 INFO:teuthology.orchestra.run.smithi192.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 712 kB/s | 47 kB 00:00 2024-09-15T22:33:32.933 INFO:teuthology.orchestra.run.smithi192.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-15T22:33:32.955 INFO:teuthology.orchestra.run.smithi111.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-15T22:33:32.958 INFO:teuthology.orchestra.run.smithi192.stdout:(64/156): libtool-2.4.6-46.el9.x86_64.rpm 2.7 MB/s | 581 kB 00:00 2024-09-15T22:33:32.984 INFO:teuthology.orchestra.run.smithi192.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 769 kB/s | 38 kB 00:00 2024-09-15T22:33:33.009 INFO:teuthology.orchestra.run.smithi192.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-09-15T22:33:33.126 INFO:teuthology.orchestra.run.smithi192.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 30 MB/s | 4.9 MB 00:00 2024-09-15T22:33:33.197 INFO:teuthology.orchestra.run.smithi111.stdout:(41/156): ceph-common-19.1.1-326.g8921c8ef.el9. 3.0 MB/s | 21 MB 00:07 2024-09-15T22:33:33.201 INFO:teuthology.orchestra.run.smithi192.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-09-15T22:33:33.222 INFO:teuthology.orchestra.run.smithi111.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 949 kB/s | 253 kB 00:00 2024-09-15T22:33:33.243 INFO:teuthology.orchestra.run.smithi192.stdout:(69/156): postgresql-13.16-1.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-09-15T22:33:33.256 INFO:teuthology.orchestra.run.smithi111.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 3.1 MB/s | 106 kB 00:00 2024-09-15T22:33:33.268 INFO:teuthology.orchestra.run.smithi192.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-15T22:33:33.289 INFO:teuthology.orchestra.run.smithi111.stdout:(44/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.8 MB/s | 801 kB 00:00 2024-09-15T22:33:33.310 INFO:teuthology.orchestra.run.smithi192.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 4.8 MB/s | 910 kB 00:00 2024-09-15T22:33:33.315 INFO:teuthology.orchestra.run.smithi111.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-15T22:33:33.340 INFO:teuthology.orchestra.run.smithi111.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-15T22:33:33.352 INFO:teuthology.orchestra.run.smithi192.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-09-15T22:33:33.366 INFO:teuthology.orchestra.run.smithi111.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-15T22:33:33.378 INFO:teuthology.orchestra.run.smithi192.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-15T22:33:33.408 INFO:teuthology.orchestra.run.smithi111.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 5.8 MB/s | 1.2 MB 00:00 2024-09-15T22:33:33.420 INFO:teuthology.orchestra.run.smithi192.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-15T22:33:33.495 INFO:teuthology.orchestra.run.smithi192.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-09-15T22:33:33.514 INFO:teuthology.orchestra.run.smithi121.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.3 MB/s | 4.9 MB 00:01 2024-09-15T22:33:33.522 INFO:teuthology.orchestra.run.smithi192.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-15T22:33:33.545 INFO:teuthology.orchestra.run.smithi192.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-09-15T22:33:33.563 INFO:teuthology.orchestra.run.smithi192.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 609 kB/s | 26 kB 00:00 2024-09-15T22:33:33.573 INFO:teuthology.orchestra.run.smithi121.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 1.7 MB/s | 1.6 MB 00:00 2024-09-15T22:33:33.575 INFO:teuthology.orchestra.run.smithi111.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 642 kB/s | 107 kB 00:00 2024-09-15T22:33:33.579 INFO:teuthology.orchestra.run.smithi192.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 568 kB/s | 19 kB 00:00 2024-09-15T22:33:33.605 INFO:teuthology.orchestra.run.smithi192.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-09-15T22:33:33.617 INFO:teuthology.orchestra.run.smithi111.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-09-15T22:33:33.630 INFO:teuthology.orchestra.run.smithi192.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 692 kB/s | 35 kB 00:00 2024-09-15T22:33:33.633 INFO:teuthology.orchestra.run.smithi111.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 372 kB/s | 22 kB 00:00 2024-09-15T22:33:33.667 INFO:teuthology.orchestra.run.smithi111.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 943 kB 00:00 2024-09-15T22:33:33.684 INFO:teuthology.orchestra.run.smithi111.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 652 kB/s | 33 kB 00:00 2024-09-15T22:33:33.690 INFO:teuthology.orchestra.run.smithi121.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2024-09-15T22:33:33.709 INFO:teuthology.orchestra.run.smithi111.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-09-15T22:33:33.722 INFO:teuthology.orchestra.run.smithi192.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-15T22:33:33.726 INFO:teuthology.orchestra.run.smithi111.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-09-15T22:33:33.747 INFO:teuthology.orchestra.run.smithi192.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-15T22:33:33.751 INFO:teuthology.orchestra.run.smithi111.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-09-15T22:33:33.772 INFO:teuthology.orchestra.run.smithi192.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-09-15T22:33:33.798 INFO:teuthology.orchestra.run.smithi192.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-15T22:33:33.822 INFO:teuthology.orchestra.run.smithi111.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-09-15T22:33:33.823 INFO:teuthology.orchestra.run.smithi192.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-09-15T22:33:33.848 INFO:teuthology.orchestra.run.smithi192.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-09-15T22:33:33.852 INFO:teuthology.orchestra.run.smithi111.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 452 kB/s | 45 kB 00:00 2024-09-15T22:33:33.874 INFO:teuthology.orchestra.run.smithi192.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-15T22:33:33.877 INFO:teuthology.orchestra.run.smithi111.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-09-15T22:33:33.899 INFO:teuthology.orchestra.run.smithi192.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-15T22:33:33.919 INFO:teuthology.orchestra.run.smithi111.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 6.7 MB/s | 662 kB 00:00 2024-09-15T22:33:33.924 INFO:teuthology.orchestra.run.smithi192.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-15T22:33:33.944 INFO:teuthology.orchestra.run.smithi111.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 366 kB/s | 24 kB 00:00 2024-09-15T22:33:33.949 INFO:teuthology.orchestra.run.smithi192.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-15T22:33:33.978 INFO:teuthology.orchestra.run.smithi111.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 4.5 MB/s | 581 kB 00:00 2024-09-15T22:33:34.003 INFO:teuthology.orchestra.run.smithi111.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 566 kB/s | 47 kB 00:00 2024-09-15T22:33:34.025 INFO:teuthology.orchestra.run.smithi192.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-15T22:33:34.037 INFO:teuthology.orchestra.run.smithi111.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-15T22:33:34.050 INFO:teuthology.orchestra.run.smithi192.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-09-15T22:33:34.062 INFO:teuthology.orchestra.run.smithi111.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 659 kB/s | 38 kB 00:00 2024-09-15T22:33:34.075 INFO:teuthology.orchestra.run.smithi192.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 885 kB/s | 44 kB 00:00 2024-09-15T22:33:34.087 INFO:teuthology.orchestra.run.smithi111.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-15T22:33:34.100 INFO:teuthology.orchestra.run.smithi192.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-15T22:33:34.134 INFO:teuthology.orchestra.run.smithi192.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 614 kB/s | 36 kB 00:00 2024-09-15T22:33:34.159 INFO:teuthology.orchestra.run.smithi192.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 813 kB/s | 47 kB 00:00 2024-09-15T22:33:34.185 INFO:teuthology.orchestra.run.smithi192.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-15T22:33:34.210 INFO:teuthology.orchestra.run.smithi192.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-15T22:33:34.221 INFO:teuthology.orchestra.run.smithi111.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-15T22:33:34.235 INFO:teuthology.orchestra.run.smithi192.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-15T22:33:34.261 INFO:teuthology.orchestra.run.smithi192.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-15T22:33:34.286 INFO:teuthology.orchestra.run.smithi192.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-09-15T22:33:34.288 INFO:teuthology.orchestra.run.smithi111.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-09-15T22:33:34.323 INFO:teuthology.orchestra.run.smithi121.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 183 kB/s | 137 kB 00:00 2024-09-15T22:33:34.330 INFO:teuthology.orchestra.run.smithi111.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 8.2 MB/s | 910 kB 00:00 2024-09-15T22:33:34.355 INFO:teuthology.orchestra.run.smithi111.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-15T22:33:34.403 INFO:teuthology.orchestra.run.smithi192.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 157 kB/s | 22 kB 00:00 2024-09-15T22:33:34.430 INFO:teuthology.orchestra.run.smithi111.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 13 MB/s | 4.9 MB 00:00 2024-09-15T22:33:34.440 INFO:teuthology.orchestra.run.smithi121.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 984 kB/s | 910 kB 00:00 2024-09-15T22:33:34.445 INFO:teuthology.orchestra.run.smithi192.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 5.3 MB/s | 863 kB 00:00 2024-09-15T22:33:34.464 INFO:teuthology.orchestra.run.smithi111.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-09-15T22:33:34.489 INFO:teuthology.orchestra.run.smithi111.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-15T22:33:34.565 INFO:teuthology.orchestra.run.smithi111.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 815 kB/s | 61 kB 00:00 2024-09-15T22:33:34.620 INFO:teuthology.orchestra.run.smithi192.stdout:(105/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-09-15T22:33:34.656 INFO:teuthology.orchestra.run.smithi111.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 18 MB/s | 5.8 MB 00:00 2024-09-15T22:33:34.670 INFO:teuthology.orchestra.run.smithi192.stdout:(106/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.0 MB/s | 548 kB 00:00 2024-09-15T22:33:34.681 INFO:teuthology.orchestra.run.smithi111.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.1 MB/s | 245 kB 00:00 2024-09-15T22:33:34.695 INFO:teuthology.orchestra.run.smithi192.stdout:(107/156): gperftools-libs-2.9.1-3.el9.x86_64.r 1.2 MB/s | 308 kB 00:00 2024-09-15T22:33:34.715 INFO:teuthology.orchestra.run.smithi111.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-15T22:33:34.721 INFO:teuthology.orchestra.run.smithi192.stdout:(108/156): grpc-data-1.46.7-10.el9.noarch.rpm 198 kB/s | 19 kB 00:00 2024-09-15T22:33:34.741 INFO:teuthology.orchestra.run.smithi111.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 436 kB/s | 26 kB 00:00 2024-09-15T22:33:34.746 INFO:teuthology.orchestra.run.smithi192.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 516 kB/s | 26 kB 00:00 2024-09-15T22:33:34.766 INFO:teuthology.orchestra.run.smithi111.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-09-15T22:33:34.771 INFO:teuthology.orchestra.run.smithi192.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 967 kB/s | 49 kB 00:00 2024-09-15T22:33:34.797 INFO:teuthology.orchestra.run.smithi192.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-15T22:33:34.800 INFO:teuthology.orchestra.run.smithi111.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-15T22:33:34.825 INFO:teuthology.orchestra.run.smithi111.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 595 kB/s | 35 kB 00:00 2024-09-15T22:33:34.830 INFO:teuthology.orchestra.run.smithi192.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-09-15T22:33:34.897 INFO:teuthology.orchestra.run.smithi192.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 839 kB 00:00 2024-09-15T22:33:34.915 INFO:teuthology.orchestra.run.smithi121.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 4.7 MB/s | 5.8 MB 00:01 2024-09-15T22:33:34.922 INFO:teuthology.orchestra.run.smithi192.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-15T22:33:34.942 INFO:teuthology.orchestra.run.smithi111.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-09-15T22:33:34.956 INFO:teuthology.orchestra.run.smithi192.stdout:(115/156): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-09-15T22:33:34.967 INFO:teuthology.orchestra.run.smithi111.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.7 MB/s | 250 kB 00:00 2024-09-15T22:33:34.981 INFO:teuthology.orchestra.run.smithi192.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 550 kB/s | 32 kB 00:00 2024-09-15T22:33:35.001 INFO:teuthology.orchestra.run.smithi111.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-15T22:33:35.007 INFO:teuthology.orchestra.run.smithi192.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 570 kB/s | 14 kB 00:00 2024-09-15T22:33:35.026 INFO:teuthology.orchestra.run.smithi111.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-15T22:33:35.040 INFO:teuthology.orchestra.run.smithi192.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-15T22:33:35.051 INFO:teuthology.orchestra.run.smithi111.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-09-15T22:33:35.085 INFO:teuthology.orchestra.run.smithi111.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-15T22:33:35.107 INFO:teuthology.orchestra.run.smithi192.stdout:(119/156): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-09-15T22:33:35.110 INFO:teuthology.orchestra.run.smithi111.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-15T22:33:35.133 INFO:teuthology.orchestra.run.smithi192.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-09-15T22:33:35.158 INFO:teuthology.orchestra.run.smithi192.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-09-15T22:33:35.183 INFO:teuthology.orchestra.run.smithi192.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 2.9 MB/s | 220 kB 00:00 2024-09-15T22:33:35.194 INFO:teuthology.orchestra.run.smithi111.stdout:(89/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-09-15T22:33:35.200 INFO:teuthology.orchestra.run.smithi192.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 648 kB/s | 11 kB 00:00 2024-09-15T22:33:35.219 INFO:teuthology.orchestra.run.smithi111.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 470 kB/s | 51 kB 00:00 2024-09-15T22:33:35.224 INFO:teuthology.orchestra.run.smithi121.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.1 MB/s | 1.0 MB 00:00 2024-09-15T22:33:35.225 INFO:teuthology.orchestra.run.smithi192.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-09-15T22:33:35.242 INFO:teuthology.orchestra.run.smithi192.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 424 kB/s | 18 kB 00:00 2024-09-15T22:33:35.249 INFO:teuthology.orchestra.run.smithi121.stdout:(74/156): python3-attrs-20.3.0-7.el9.noarch.rpm 108 kB/s | 87 kB 00:00 2024-09-15T22:33:35.259 INFO:teuthology.orchestra.run.smithi192.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 699 kB/s | 23 kB 00:00 2024-09-15T22:33:35.275 INFO:teuthology.orchestra.run.smithi111.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.4 MB/s | 279 kB 00:00 2024-09-15T22:33:35.285 INFO:teuthology.orchestra.run.smithi192.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 464 kB/s | 19 kB 00:00 2024-09-15T22:33:35.303 INFO:teuthology.orchestra.run.smithi111.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 494 kB/s | 54 kB 00:00 2024-09-15T22:33:35.310 INFO:teuthology.orchestra.run.smithi192.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-09-15T22:33:35.337 INFO:teuthology.orchestra.run.smithi111.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 688 kB/s | 42 kB 00:00 2024-09-15T22:33:35.343 INFO:teuthology.orchestra.run.smithi192.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-15T22:33:35.370 INFO:teuthology.orchestra.run.smithi111.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 661 kB/s | 44 kB 00:00 2024-09-15T22:33:35.377 INFO:teuthology.orchestra.run.smithi192.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-15T22:33:35.395 INFO:teuthology.orchestra.run.smithi111.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-15T22:33:35.410 INFO:teuthology.orchestra.run.smithi192.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 8.1 MB/s | 1.0 MB 00:00 2024-09-15T22:33:35.429 INFO:teuthology.orchestra.run.smithi111.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 612 kB/s | 36 kB 00:00 2024-09-15T22:33:35.436 INFO:teuthology.orchestra.run.smithi192.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 986 kB/s | 58 kB 00:00 2024-09-15T22:33:35.454 INFO:teuthology.orchestra.run.smithi111.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 809 kB/s | 47 kB 00:00 2024-09-15T22:33:35.461 INFO:teuthology.orchestra.run.smithi192.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-15T22:33:35.486 INFO:teuthology.orchestra.run.smithi192.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-15T22:33:35.488 INFO:teuthology.orchestra.run.smithi111.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-15T22:33:35.511 INFO:teuthology.orchestra.run.smithi192.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-15T22:33:35.537 INFO:teuthology.orchestra.run.smithi192.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-15T22:33:35.546 INFO:teuthology.orchestra.run.smithi111.stdout:(99/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 1.6 MB/s | 96 kB 00:00 2024-09-15T22:33:35.562 INFO:teuthology.orchestra.run.smithi192.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-15T22:33:35.572 INFO:teuthology.orchestra.run.smithi111.stdout:(100/156): uuid-1.6.2-55.el9.x86_64.rpm 493 kB/s | 58 kB 00:00 2024-09-15T22:33:35.588 INFO:teuthology.orchestra.run.smithi192.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-15T22:33:35.605 INFO:teuthology.orchestra.run.smithi111.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-15T22:33:35.613 INFO:teuthology.orchestra.run.smithi192.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-15T22:33:35.638 INFO:teuthology.orchestra.run.smithi192.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-15T22:33:35.656 INFO:teuthology.orchestra.run.smithi111.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 531 kB/s | 45 kB 00:00 2024-09-15T22:33:35.672 INFO:teuthology.orchestra.run.smithi192.stdout:(141/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.8 MB/s | 2.0 MB 00:00 2024-09-15T22:33:35.706 INFO:teuthology.orchestra.run.smithi192.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 808 kB/s | 74 kB 00:00 2024-09-15T22:33:35.731 INFO:teuthology.orchestra.run.smithi192.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-15T22:33:35.756 INFO:teuthology.orchestra.run.smithi192.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-15T22:33:35.781 INFO:teuthology.orchestra.run.smithi192.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 400 kB/s | 20 kB 00:00 2024-09-15T22:33:35.815 INFO:teuthology.orchestra.run.smithi192.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-09-15T22:33:35.840 INFO:teuthology.orchestra.run.smithi192.stdout:(147/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-15T22:33:35.841 INFO:teuthology.orchestra.run.smithi121.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 100 kB/s | 61 kB 00:00 2024-09-15T22:33:35.898 INFO:teuthology.orchestra.run.smithi111.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-09-15T22:33:35.899 INFO:teuthology.orchestra.run.smithi192.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-15T22:33:35.923 INFO:teuthology.orchestra.run.smithi111.stdout:(104/156): lua-devel-5.4.4-4.el9.x86_64.rpm 70 kB/s | 22 kB 00:00 2024-09-15T22:33:36.015 INFO:teuthology.orchestra.run.smithi111.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 4.6 MB/s | 548 kB 00:00 2024-09-15T22:33:36.041 INFO:teuthology.orchestra.run.smithi111.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 780 kB/s | 19 kB 00:00 2024-09-15T22:33:36.066 INFO:teuthology.orchestra.run.smithi111.stdout:(107/156): gperftools-libs-2.9.1-3.el9.x86_64.r 2.1 MB/s | 308 kB 00:00 2024-09-15T22:33:36.100 INFO:teuthology.orchestra.run.smithi111.stdout:(108/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-09-15T22:33:36.100 INFO:teuthology.orchestra.run.smithi121.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.0 MB/s | 6.0 MB 00:01 2024-09-15T22:33:36.116 INFO:teuthology.orchestra.run.smithi192.stdout:(149/156): librados2-19.1.1-326.g8921c8ef.el9.x 11 MB/s | 3.4 MB 00:00 2024-09-15T22:33:36.125 INFO:teuthology.orchestra.run.smithi111.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 443 kB/s | 26 kB 00:00 2024-09-15T22:33:36.133 INFO:teuthology.orchestra.run.smithi121.stdout:(77/156): python3-devel-3.9.19-8.el9.x86_64.rpm 277 kB/s | 245 kB 00:00 2024-09-15T22:33:36.167 INFO:teuthology.orchestra.run.smithi111.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.1 MB/s | 67 kB 00:00 2024-09-15T22:33:36.192 INFO:teuthology.orchestra.run.smithi111.stdout:(111/156): liboath-2.6.7-2.el9.x86_64.rpm 527 kB/s | 49 kB 00:00 2024-09-15T22:33:36.216 INFO:teuthology.orchestra.run.smithi192.stdout:(150/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 8.4 MB/s | 3.2 MB 00:00 2024-09-15T22:33:36.217 INFO:teuthology.orchestra.run.smithi111.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 3.1 MB/s | 151 kB 00:00 2024-09-15T22:33:36.276 INFO:teuthology.orchestra.run.smithi111.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-09-15T22:33:36.309 INFO:teuthology.orchestra.run.smithi111.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-09-15T22:33:36.376 INFO:teuthology.orchestra.run.smithi111.stdout:(115/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.5 MB/s | 839 kB 00:00 2024-09-15T22:33:36.402 INFO:teuthology.orchestra.run.smithi111.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 350 kB/s | 32 kB 00:00 2024-09-15T22:33:36.427 INFO:teuthology.orchestra.run.smithi111.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 277 kB/s | 14 kB 00:00 2024-09-15T22:33:36.461 INFO:teuthology.orchestra.run.smithi111.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-15T22:33:36.508 INFO:teuthology.orchestra.run.smithi192.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 140 kB/s | 41 kB 00:00 2024-09-15T22:33:36.536 INFO:teuthology.orchestra.run.smithi111.stdout:(119/156): libarrow-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 4.4 MB 00:00 2024-09-15T22:33:36.561 INFO:teuthology.orchestra.run.smithi111.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-09-15T22:33:36.586 INFO:teuthology.orchestra.run.smithi111.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-09-15T22:33:36.609 INFO:teuthology.orchestra.run.smithi121.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 40 kB/s | 19 kB 00:00 2024-09-15T22:33:36.612 INFO:teuthology.orchestra.run.smithi111.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 2.9 MB/s | 220 kB 00:00 2024-09-15T22:33:36.637 INFO:teuthology.orchestra.run.smithi111.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-15T22:33:36.642 INFO:teuthology.orchestra.run.smithi121.stdout:(79/156): python3-jinja2-2.11.3-6.el9.noarch.rp 310 kB/s | 248 kB 00:00 2024-09-15T22:33:36.663 INFO:teuthology.orchestra.run.smithi111.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 212 kB/s | 11 kB 00:00 2024-09-15T22:33:36.667 INFO:teuthology.orchestra.run.smithi121.stdout:(80/156): python3-jsonpatch-1.21-16.el9.noarch. 46 kB/s | 26 kB 00:00 2024-09-15T22:33:36.688 INFO:teuthology.orchestra.run.smithi111.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-15T22:33:36.725 INFO:teuthology.orchestra.run.smithi192.stdout:(152/156): NetworkManager-1.49.90-1.el9.x86_64. 2.8 MB/s | 2.3 MB 00:00 2024-09-15T22:33:36.738 INFO:teuthology.orchestra.run.smithi111.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 309 kB/s | 23 kB 00:00 2024-09-15T22:33:36.763 INFO:teuthology.orchestra.run.smithi111.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-09-15T22:33:36.788 INFO:teuthology.orchestra.run.smithi111.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-09-15T22:33:36.822 INFO:teuthology.orchestra.run.smithi111.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-15T22:33:36.856 INFO:teuthology.orchestra.run.smithi111.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-15T22:33:36.867 INFO:teuthology.orchestra.run.smithi192.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 9.6 MB/s | 1.4 MB 00:00 2024-09-15T22:33:36.889 INFO:teuthology.orchestra.run.smithi111.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 1.7 MB/s | 58 kB 00:00 2024-09-15T22:33:36.892 INFO:teuthology.orchestra.run.smithi192.stdout:(154/156): NetworkManager-tui-1.49.90-1.el9.x86 648 kB/s | 249 kB 00:00 2024-09-15T22:33:36.934 INFO:teuthology.orchestra.run.smithi192.stdout:(155/156): NetworkManager-libnm-1.49.90-1.el9.x 2.2 MB/s | 1.8 MB 00:00 2024-09-15T22:33:36.940 INFO:teuthology.orchestra.run.smithi111.stdout:(132/156): python3-grpcio-1.46.7-10.el9.x86_64. 5.4 MB/s | 2.0 MB 00:00 2024-09-15T22:33:36.965 INFO:teuthology.orchestra.run.smithi111.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 661 kB/s | 16 kB 00:00 2024-09-15T22:33:36.990 INFO:teuthology.orchestra.run.smithi111.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-15T22:33:37.032 INFO:teuthology.orchestra.run.smithi111.stdout:(135/156): python3-kubernetes-26.1.0-2.el9.noar 3.8 MB/s | 1.0 MB 00:00 2024-09-15T22:33:37.057 INFO:teuthology.orchestra.run.smithi111.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 976 kB/s | 90 kB 00:00 2024-09-15T22:33:37.107 INFO:teuthology.orchestra.run.smithi111.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 243 kB/s | 28 kB 00:00 2024-09-15T22:33:37.133 INFO:teuthology.orchestra.run.smithi111.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-09-15T22:33:37.158 INFO:teuthology.orchestra.run.smithi111.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-15T22:33:37.184 INFO:teuthology.orchestra.run.smithi111.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 707 kB/s | 36 kB 00:00 2024-09-15T22:33:37.184 INFO:teuthology.orchestra.run.smithi192.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 7.6 MB/s | 2.4 MB 00:00 2024-09-15T22:33:37.190 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-15T22:33:37.190 INFO:teuthology.orchestra.run.smithi192.stdout:Total 17 MB/s | 226 MB 00:13 2024-09-15T22:33:37.192 INFO:teuthology.orchestra.run.smithi121.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 63 kB/s | 35 kB 00:00 2024-09-15T22:33:37.209 INFO:teuthology.orchestra.run.smithi111.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-15T22:33:37.234 INFO:teuthology.orchestra.run.smithi111.stdout:(142/156): python3-routes-2.5.1-5.el9.noarch.rp 1.0 MB/s | 188 kB 00:00 2024-09-15T22:33:37.259 INFO:teuthology.orchestra.run.smithi111.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-15T22:33:37.285 INFO:teuthology.orchestra.run.smithi111.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-15T22:33:37.310 INFO:teuthology.orchestra.run.smithi111.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-15T22:33:37.326 INFO:teuthology.orchestra.run.smithi121.stdout:(82/156): python3-mako-1.1.4-6.el9.noarch.rpm 240 kB/s | 172 kB 00:00 2024-09-15T22:33:37.335 INFO:teuthology.orchestra.run.smithi111.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-15T22:33:37.361 INFO:teuthology.orchestra.run.smithi111.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-15T22:33:37.411 INFO:teuthology.orchestra.run.smithi111.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-15T22:33:37.594 INFO:teuthology.orchestra.run.smithi111.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-09-15T22:33:37.678 INFO:teuthology.orchestra.run.smithi111.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 22 MB/s | 1.8 MB 00:00 2024-09-15T22:33:37.712 INFO:teuthology.orchestra.run.smithi111.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 1.2 MB/s | 41 kB 00:00 2024-09-15T22:33:37.793 INFO:teuthology.orchestra.run.smithi121.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.5 MB/s | 5.1 MB 00:01 2024-09-15T22:33:37.837 INFO:teuthology.orchestra.run.smithi111.stdout:(152/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 6.6 MB/s | 3.2 MB 00:00 2024-09-15T22:33:37.962 INFO:teuthology.orchestra.run.smithi111.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 994 kB/s | 249 kB 00:00 2024-09-15T22:33:37.976 INFO:teuthology.orchestra.run.smithi121.stdout:(84/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 320 kB/s | 250 kB 00:00 2024-09-15T22:33:38.054 INFO:teuthology.orchestra.run.smithi111.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 26 MB/s | 2.4 MB 00:00 2024-09-15T22:33:38.093 INFO:teuthology.orchestra.run.smithi121.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 290 kB/s | 222 kB 00:00 2024-09-15T22:33:38.112 INFO:teuthology.orchestra.run.smithi111.stdout:(155/156): librados2-19.1.1-326.g8921c8ef.el9.x 4.4 MB/s | 3.4 MB 00:00 2024-09-15T22:33:38.146 INFO:teuthology.orchestra.run.smithi111.stdout:(156/156): openssl-3.2.2-6.el9.x86_64.rpm 4.4 MB/s | 1.4 MB 00:00 2024-09-15T22:33:38.151 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-09-15T22:33:38.151 INFO:teuthology.orchestra.run.smithi111.stdout:Total 18 MB/s | 226 MB 00:12 2024-09-15T22:33:38.427 INFO:teuthology.orchestra.run.smithi121.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 122 kB/s | 77 kB 00:00 2024-09-15T22:33:38.431 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-15T22:33:38.579 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-15T22:33:38.580 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-15T22:33:38.594 INFO:teuthology.orchestra.run.smithi121.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 68 kB/s | 42 kB 00:00 2024-09-15T22:33:38.853 INFO:teuthology.orchestra.run.smithi121.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 355 kB/s | 269 kB 00:00 2024-09-15T22:33:39.153 INFO:teuthology.orchestra.run.smithi121.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 219 kB/s | 159 kB 00:00 2024-09-15T22:33:39.445 INFO:teuthology.orchestra.run.smithi121.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 328 kB/s | 279 kB 00:00 2024-09-15T22:33:39.470 INFO:teuthology.orchestra.run.smithi121.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 83 kB/s | 51 kB 00:00 2024-09-15T22:33:39.812 INFO:teuthology.orchestra.run.smithi121.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 81 kB/s | 54 kB 00:00 2024-09-15T22:33:40.070 INFO:teuthology.orchestra.run.smithi121.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 69 kB/s | 42 kB 00:00 2024-09-15T22:33:40.254 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2024-09-15T22:33:40.370 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-15T22:33:40.370 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-15T22:33:40.429 INFO:teuthology.orchestra.run.smithi121.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 72 kB/s | 44 kB 00:00 2024-09-15T22:33:40.445 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2024-09-15T22:33:40.445 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2024-09-15T22:33:40.813 INFO:teuthology.orchestra.run.smithi121.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 298 kB/s | 221 kB 00:00 2024-09-15T22:33:40.980 INFO:teuthology.orchestra.run.smithi121.stdout:(96/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-15T22:33:41.005 INFO:teuthology.orchestra.run.smithi121.stdout:(97/156): qatlib-service-24.02.0-1.el9.x86_64.r 62 kB/s | 36 kB 00:00 2024-09-15T22:33:41.488 INFO:teuthology.orchestra.run.smithi121.stdout:(98/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 70 kB/s | 47 kB 00:00 2024-09-15T22:33:41.605 INFO:teuthology.orchestra.run.smithi121.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 96 kB/s | 58 kB 00:00 2024-09-15T22:33:41.822 INFO:teuthology.orchestra.run.smithi121.stdout:(100/156): socat-1.7.4.1-6.el9.x86_64.rpm 361 kB/s | 304 kB 00:00 2024-09-15T22:33:42.139 INFO:teuthology.orchestra.run.smithi121.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 147 kB/s | 96 kB 00:00 2024-09-15T22:33:42.264 INFO:teuthology.orchestra.run.smithi121.stdout:(102/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 97 kB/s | 64 kB 00:00 2024-09-15T22:33:42.289 INFO:teuthology.orchestra.run.smithi121.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 149 kB/s | 22 kB 00:00 2024-09-15T22:33:42.382 INFO:teuthology.orchestra.run.smithi121.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 5.8 MB/s | 548 kB 00:00 2024-09-15T22:33:42.423 INFO:teuthology.orchestra.run.smithi121.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 7.2 MB/s | 308 kB 00:00 2024-09-15T22:33:42.449 INFO:teuthology.orchestra.run.smithi121.stdout:(106/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 71 kB/s | 45 kB 00:00 2024-09-15T22:33:42.474 INFO:teuthology.orchestra.run.smithi121.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 387 kB/s | 19 kB 00:00 2024-09-15T22:33:42.516 INFO:teuthology.orchestra.run.smithi121.stdout:(108/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.4 MB/s | 863 kB 00:00 2024-09-15T22:33:42.549 INFO:teuthology.orchestra.run.smithi121.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 385 kB/s | 26 kB 00:00 2024-09-15T22:33:42.575 INFO:teuthology.orchestra.run.smithi121.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 3.8 MB/s | 67 kB 00:00 2024-09-15T22:33:42.609 INFO:teuthology.orchestra.run.smithi121.stdout:(111/156): liboath-2.6.7-2.el9.x86_64.rpm 538 kB/s | 49 kB 00:00 2024-09-15T22:33:42.634 INFO:teuthology.orchestra.run.smithi121.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.6 MB/s | 151 kB 00:00 2024-09-15T22:33:42.701 INFO:teuthology.orchestra.run.smithi121.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-09-15T22:33:42.726 INFO:teuthology.orchestra.run.smithi121.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-15T22:33:42.760 INFO:teuthology.orchestra.run.smithi121.stdout:(115/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.4 MB/s | 839 kB 00:00 2024-09-15T22:33:42.785 INFO:teuthology.orchestra.run.smithi121.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 548 kB/s | 32 kB 00:00 2024-09-15T22:33:42.791 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2024-09-15T22:33:42.792 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2024-09-15T22:33:42.810 INFO:teuthology.orchestra.run.smithi121.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 282 kB/s | 14 kB 00:00 2024-09-15T22:33:42.835 INFO:teuthology.orchestra.run.smithi121.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-15T22:33:42.877 INFO:teuthology.orchestra.run.smithi121.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-15T22:33:42.903 INFO:teuthology.orchestra.run.smithi121.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-15T22:33:42.936 INFO:teuthology.orchestra.run.smithi121.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-15T22:33:43.004 INFO:teuthology.orchestra.run.smithi121.stdout:(122/156): libarrow-9.0.0-11.el9.x86_64.rpm 8.0 MB/s | 4.4 MB 00:00 2024-09-15T22:33:43.029 INFO:teuthology.orchestra.run.smithi121.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-09-15T22:33:43.054 INFO:teuthology.orchestra.run.smithi121.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-09-15T22:33:43.079 INFO:teuthology.orchestra.run.smithi121.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-15T22:33:43.104 INFO:teuthology.orchestra.run.smithi121.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 460 kB/s | 23 kB 00:00 2024-09-15T22:33:43.130 INFO:teuthology.orchestra.run.smithi121.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-15T22:33:43.155 INFO:teuthology.orchestra.run.smithi121.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-09-15T22:33:43.180 INFO:teuthology.orchestra.run.smithi121.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-15T22:33:43.214 INFO:teuthology.orchestra.run.smithi121.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-15T22:33:43.247 INFO:teuthology.orchestra.run.smithi121.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 1.7 MB/s | 58 kB 00:00 2024-09-15T22:33:43.298 INFO:teuthology.orchestra.run.smithi121.stdout:(132/156): python3-grpcio-1.46.7-10.el9.x86_64. 5.2 MB/s | 2.0 MB 00:00 2024-09-15T22:33:43.323 INFO:teuthology.orchestra.run.smithi121.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-15T22:33:43.365 INFO:teuthology.orchestra.run.smithi121.stdout:(134/156): python3-kubernetes-26.1.0-2.el9.noar 4.3 MB/s | 1.0 MB 00:00 2024-09-15T22:33:43.390 INFO:teuthology.orchestra.run.smithi121.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-09-15T22:33:43.416 INFO:teuthology.orchestra.run.smithi121.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-09-15T22:33:43.441 INFO:teuthology.orchestra.run.smithi121.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-09-15T22:33:43.466 INFO:teuthology.orchestra.run.smithi121.stdout:(138/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.2 MB/s | 272 kB 00:00 2024-09-15T22:33:43.492 INFO:teuthology.orchestra.run.smithi121.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-15T22:33:43.542 INFO:teuthology.orchestra.run.smithi121.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 476 kB/s | 36 kB 00:00 2024-09-15T22:33:43.567 INFO:teuthology.orchestra.run.smithi121.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-15T22:33:43.601 INFO:teuthology.orchestra.run.smithi121.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 685 kB/s | 74 kB 00:00 2024-09-15T22:33:43.634 INFO:teuthology.orchestra.run.smithi121.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-09-15T22:33:43.659 INFO:teuthology.orchestra.run.smithi121.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-15T22:33:43.685 INFO:teuthology.orchestra.run.smithi121.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-15T22:33:43.710 INFO:teuthology.orchestra.run.smithi121.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-15T22:33:43.735 INFO:teuthology.orchestra.run.smithi121.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-15T22:33:43.802 INFO:teuthology.orchestra.run.smithi121.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-15T22:33:44.003 INFO:teuthology.orchestra.run.smithi121.stdout:(149/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 12 MB/s | 3.2 MB 00:00 2024-09-15T22:33:44.120 INFO:teuthology.orchestra.run.smithi121.stdout:(150/156): librados2-19.1.1-326.g8921c8ef.el9.x 8.4 MB/s | 3.4 MB 00:00 2024-09-15T22:33:44.195 INFO:teuthology.orchestra.run.smithi121.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 545 kB/s | 41 kB 00:00 2024-09-15T22:33:44.278 INFO:teuthology.orchestra.run.smithi121.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 2.9 MB/s | 249 kB 00:00 2024-09-15T22:33:44.420 INFO:teuthology.orchestra.run.smithi121.stdout:(153/156): NetworkManager-1.49.90-1.el9.x86_64. 3.8 MB/s | 2.3 MB 00:00 2024-09-15T22:33:44.462 INFO:teuthology.orchestra.run.smithi121.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 3.9 MB/s | 1.8 MB 00:00 2024-09-15T22:33:44.521 INFO:teuthology.orchestra.run.smithi121.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 5.6 MB/s | 1.4 MB 00:00 2024-09-15T22:33:45.104 INFO:teuthology.orchestra.run.smithi121.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.5 MB/s | 2.4 MB 00:00 2024-09-15T22:33:45.111 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-09-15T22:33:45.111 INFO:teuthology.orchestra.run.smithi121.stdout:Total 10 MB/s | 226 MB 00:21 2024-09-15T22:33:46.344 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-09-15T22:33:46.497 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-09-15T22:33:46.497 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-09-15T22:33:46.635 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-15T22:33:46.704 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-15T22:33:46.838 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-15T22:33:47.024 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-15T22:33:47.073 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-15T22:33:47.098 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-15T22:33:47.150 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-15T22:33:47.486 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-15T22:33:47.590 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T22:33:47.695 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T22:33:47.724 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T22:33:47.799 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T22:33:47.828 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T22:33:47.977 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-15T22:33:48.025 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-15T22:33:48.053 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T22:33:48.157 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T22:33:48.179 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T22:33:48.242 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T22:33:48.264 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T22:33:48.287 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T22:33:48.304 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-09-15T22:33:48.304 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-09-15T22:33:48.432 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T22:33:48.565 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T22:33:48.728 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-15T22:33:48.839 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T22:33:48.925 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T22:33:49.167 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T22:33:49.282 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-15T22:33:49.377 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-15T22:33:49.405 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-15T22:33:49.440 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-15T22:33:49.522 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-15T22:33:49.574 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-15T22:33:49.606 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-15T22:33:49.650 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-15T22:33:49.682 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-15T22:33:49.710 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-15T22:33:49.779 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-15T22:33:49.797 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2024-09-15T22:33:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-15T22:33:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-15T22:33:49.907 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-15T22:33:49.968 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-15T22:33:49.984 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-15T22:33:50.025 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-15T22:33:50.068 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-15T22:33:50.084 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-15T22:33:50.097 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-15T22:33:50.166 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-15T22:33:50.212 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-15T22:33:50.221 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-15T22:33:50.270 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-15T22:33:50.294 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-15T22:33:50.302 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-15T22:33:50.357 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-15T22:33:50.435 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-15T22:33:50.489 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-15T22:33:50.737 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-15T22:33:50.844 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T22:33:51.006 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T22:33:51.016 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-15T22:33:51.037 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T22:33:51.101 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-15T22:33:51.124 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T22:33:51.141 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-15T22:33:51.159 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T22:33:51.184 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-15T22:33:51.222 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-15T22:33:51.264 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-15T22:33:51.376 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-15T22:33:51.419 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-15T22:33:51.453 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-15T22:33:51.475 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-15T22:33:51.482 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T22:33:51.530 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-15T22:33:51.604 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-15T22:33:51.607 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T22:33:51.634 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-15T22:33:51.641 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T22:33:51.679 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-15T22:33:51.715 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T22:33:51.733 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-15T22:33:51.742 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T22:33:51.769 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-15T22:33:51.787 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T22:33:51.813 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-15T22:33:51.863 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-15T22:33:51.998 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-15T22:33:52.043 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T22:33:52.048 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-15T22:33:52.089 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-15T22:33:52.191 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-15T22:33:52.231 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-15T22:33:52.236 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T22:33:52.349 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-15T22:33:52.438 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-15T22:33:52.497 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-15T22:33:52.522 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-15T22:33:52.576 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T22:33:52.602 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-15T22:33:52.692 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T22:33:52.732 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-15T22:33:52.779 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-15T22:33:52.807 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-15T22:33:52.841 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-15T22:33:52.880 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-15T22:33:52.911 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-15T22:33:52.942 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-15T22:33:52.976 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-15T22:33:52.992 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T22:33:53.010 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-15T22:33:53.051 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-15T22:33:53.094 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-15T22:33:53.123 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-15T22:33:53.189 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-15T22:33:53.241 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-15T22:33:53.251 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-15T22:33:53.290 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-15T22:33:53.297 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-15T22:33:53.334 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-15T22:33:53.401 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-15T22:33:53.415 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-15T22:33:53.448 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-15T22:33:53.471 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-15T22:33:53.534 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-15T22:33:53.588 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-15T22:33:53.636 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-15T22:33:53.669 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-15T22:33:53.766 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-15T22:33:53.830 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-15T22:33:53.882 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-15T22:33:53.927 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-15T22:33:54.008 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-15T22:33:54.043 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-15T22:33:54.087 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-15T22:33:54.115 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-15T22:33:54.169 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-15T22:33:54.198 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-15T22:33:54.220 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-15T22:33:54.254 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-15T22:33:54.259 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-15T22:33:54.351 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-15T22:33:54.351 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-15T22:33:54.401 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-15T22:33:54.446 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-15T22:33:54.508 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-15T22:33:54.532 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-15T22:33:54.590 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-15T22:33:54.601 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-15T22:33:54.781 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-15T22:33:54.793 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-09-15T22:33:54.805 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-15T22:33:54.882 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-15T22:33:54.924 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-15T22:33:54.962 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-15T22:33:55.024 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-15T22:33:55.058 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-15T22:33:55.083 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-15T22:33:55.134 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-15T22:33:55.239 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-15T22:33:55.472 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-15T22:33:55.504 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-15T22:33:55.513 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-15T22:33:55.540 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-15T22:33:55.584 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-15T22:33:55.584 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T22:33:55.622 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-15T22:33:55.628 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-15T22:33:55.692 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-15T22:33:55.715 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T22:33:55.744 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T22:33:55.835 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T22:33:55.865 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T22:33:55.890 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-15T22:33:55.958 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-15T22:33:56.013 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-15T22:33:56.029 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-15T22:33:56.070 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-15T22:33:56.098 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T22:33:56.115 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-15T22:33:56.154 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-15T22:33:56.191 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-15T22:33:56.202 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T22:33:56.233 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T22:33:56.249 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-15T22:33:56.290 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-15T22:33:56.294 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T22:33:56.317 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T22:33:56.327 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-15T22:33:56.342 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T22:33:56.377 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-15T22:33:56.526 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T22:33:56.558 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-15T22:33:56.602 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-15T22:33:56.644 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-15T22:33:56.658 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T22:33:56.761 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-15T22:33:56.765 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-15T22:33:56.802 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-15T22:33:56.829 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-15T22:33:56.845 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-15T22:33:56.847 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T22:33:56.874 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-15T22:33:56.899 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-15T22:33:56.908 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-15T22:33:56.936 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-15T22:33:56.942 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T22:33:56.969 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-15T22:33:57.016 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-15T22:33:57.162 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T22:33:57.176 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-15T22:33:57.224 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-15T22:33:57.261 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-15T22:33:57.277 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-15T22:33:57.282 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-15T22:33:57.304 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-15T22:33:57.323 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T22:33:57.352 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-15T22:33:57.369 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-15T22:33:57.391 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-15T22:33:57.397 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-15T22:33:57.440 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-15T22:33:57.465 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-15T22:33:57.487 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T22:33:57.505 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-15T22:33:57.514 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-15T22:33:57.538 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-15T22:33:57.558 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-15T22:33:57.590 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-15T22:33:57.590 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-15T22:33:57.623 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-15T22:33:57.642 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-15T22:33:57.674 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-15T22:33:57.690 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-15T22:33:57.702 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-15T22:33:57.722 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-15T22:33:57.771 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-15T22:33:57.773 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-15T22:33:57.824 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-15T22:33:57.827 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-15T22:33:57.848 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-15T22:33:57.858 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-15T22:33:57.894 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-15T22:33:57.957 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-15T22:33:57.971 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-15T22:33:58.005 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-15T22:33:58.014 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-15T22:33:58.048 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-15T22:33:58.076 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-15T22:33:58.145 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-15T22:33:58.191 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-15T22:33:58.264 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-15T22:33:58.358 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-15T22:33:58.393 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T22:33:58.396 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-15T22:33:58.451 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-15T22:33:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T22:33:58.495 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-15T22:33:58.891 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-15T22:33:58.949 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-15T22:33:58.963 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-15T22:33:58.983 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-15T22:33:59.047 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-15T22:33:59.077 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-15T22:33:59.099 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-15T22:33:59.120 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-15T22:33:59.157 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-15T22:33:59.158 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-15T22:33:59.198 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-15T22:33:59.211 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-15T22:33:59.341 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-15T22:33:59.351 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-15T22:33:59.385 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-15T22:33:59.419 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-15T22:33:59.474 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-15T22:33:59.546 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-15T22:33:59.587 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-15T22:33:59.620 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-15T22:33:59.622 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-15T22:33:59.657 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-15T22:33:59.669 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-15T22:33:59.705 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-15T22:33:59.751 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-15T22:33:59.768 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-15T22:33:59.793 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-15T22:33:59.929 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-15T22:33:59.970 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-15T22:34:00.011 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-15T22:34:00.112 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-15T22:34:00.145 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-15T22:34:00.189 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-15T22:34:00.219 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-15T22:34:00.244 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-15T22:34:00.317 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-15T22:34:00.436 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-15T22:34:00.468 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-15T22:34:00.495 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-15T22:34:00.530 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-15T22:34:00.565 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-15T22:34:00.569 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-15T22:34:00.601 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-15T22:34:00.631 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-15T22:34:00.665 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-15T22:34:00.698 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-15T22:34:00.711 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-15T22:34:00.740 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-15T22:34:00.774 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-15T22:34:00.833 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T22:34:00.861 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-15T22:34:00.868 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T22:34:00.905 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-15T22:34:00.936 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-15T22:34:01.039 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-15T22:34:01.096 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-15T22:34:01.415 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T22:34:01.454 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T22:34:01.554 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T22:34:01.555 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T22:34:01.555 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T22:34:01.555 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T22:34:01.589 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T22:34:01.816 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-15T22:34:01.859 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-15T22:34:01.893 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-15T22:34:01.991 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-15T22:34:02.048 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-15T22:34:02.094 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-15T22:34:02.180 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-15T22:34:02.235 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-15T22:34:02.262 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-15T22:34:02.353 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-15T22:34:02.390 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-15T22:34:02.416 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-15T22:34:02.422 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-15T22:34:02.445 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-15T22:34:02.523 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-15T22:34:02.758 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-15T22:34:02.800 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T22:34:02.964 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T22:34:02.998 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-15T22:34:03.054 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-15T22:34:03.163 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-15T22:34:03.268 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-15T22:34:03.570 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-15T22:34:03.604 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T22:34:03.678 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T22:34:03.705 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-15T22:34:04.489 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-15T22:34:04.575 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-15T22:34:04.603 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-15T22:34:04.635 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-15T22:34:04.938 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-15T22:34:04.982 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T22:34:05.073 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T22:34:05.103 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-15T22:34:05.140 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-15T22:34:05.622 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-15T22:34:05.660 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T22:34:05.731 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T22:34:05.753 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-15T22:34:06.222 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T22:34:06.260 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T22:34:06.905 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T22:34:06.955 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T22:34:07.131 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T22:34:07.131 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T22:34:07.132 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T22:34:07.132 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:34:07.182 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T22:34:08.012 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T22:34:08.058 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T22:34:08.618 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T22:34:08.762 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T22:34:08.862 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T22:34:08.862 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T22:34:08.862 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T22:34:08.862 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-15T22:34:08.896 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T22:34:13.340 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T22:34:13.340 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /sys 2024-09-15T22:34:13.340 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /proc 2024-09-15T22:34:13.340 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /mnt 2024-09-15T22:34:13.340 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /var/tmp 2024-09-15T22:34:13.340 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /home 2024-09-15T22:34:13.340 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /root 2024-09-15T22:34:13.341 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /tmp 2024-09-15T22:34:13.341 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T22:34:13.525 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T22:34:13.542 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T22:34:13.542 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:34:13.542 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T22:34:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T22:34:15.011 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T22:34:15.038 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T22:34:15.038 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:34:15.038 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T22:34:15.039 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T22:34:15.039 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T22:34:15.039 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T22:34:15.261 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T22:34:15.286 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T22:34:15.286 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:34:15.286 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T22:34:15.286 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T22:34:15.286 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T22:34:15.286 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T22:34:15.721 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T22:34:15.744 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T22:34:15.744 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:34:15.744 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T22:34:15.744 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T22:34:15.744 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T22:34:15.745 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T22:34:15.850 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-15T22:34:15.907 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-15T22:34:15.963 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-15T22:34:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-15T22:34:16.196 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-15T22:34:16.274 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-15T22:34:16.362 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-15T22:34:16.502 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-15T22:34:16.538 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-15T22:34:16.569 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-15T22:34:16.597 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-15T22:34:17.059 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T22:34:17.088 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T22:34:17.119 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-15T22:34:17.181 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-15T22:34:17.211 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T22:34:18.262 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T22:34:18.296 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T22:34:19.259 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T22:34:19.288 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T22:34:19.390 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T22:34:19.559 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-15T22:34:19.590 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T22:34:19.614 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T22:34:19.614 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:34:19.614 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T22:34:19.614 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T22:34:19.614 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T22:34:19.614 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T22:34:19.666 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T22:34:19.678 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T22:34:19.775 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-15T22:34:19.831 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-15T22:34:19.842 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T22:34:19.842 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /sys 2024-09-15T22:34:19.843 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /proc 2024-09-15T22:34:19.843 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /mnt 2024-09-15T22:34:19.843 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /var/tmp 2024-09-15T22:34:19.843 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /home 2024-09-15T22:34:19.843 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /root 2024-09-15T22:34:19.843 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /tmp 2024-09-15T22:34:19.843 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:34:20.053 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T22:34:20.088 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T22:34:20.088 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:34:20.089 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T22:34:20.089 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:34:20.699 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T22:34:20.699 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /sys 2024-09-15T22:34:20.699 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /proc 2024-09-15T22:34:20.699 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /mnt 2024-09-15T22:34:20.699 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /var/tmp 2024-09-15T22:34:20.699 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /home 2024-09-15T22:34:20.699 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /root 2024-09-15T22:34:20.700 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /tmp 2024-09-15T22:34:20.700 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-15T22:34:20.757 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-15T22:34:20.785 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T22:34:20.808 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T22:34:20.808 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:34:20.809 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T22:34:20.809 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T22:34:20.809 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T22:34:20.809 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T22:34:20.852 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T22:34:20.866 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T22:34:20.875 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T22:34:20.875 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:34:20.875 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T22:34:20.876 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T22:34:20.883 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T22:34:20.883 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:34:20.883 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T22:34:20.884 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-15T22:34:20.947 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T22:34:20.970 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T22:34:20.971 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:34:20.971 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-15T22:34:20.971 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T22:34:20.971 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T22:34:20.971 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T22:34:21.287 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T22:34:21.311 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T22:34:21.311 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:34:21.311 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T22:34:21.311 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T22:34:21.311 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T22:34:21.311 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T22:34:21.707 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T22:34:21.765 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T22:34:21.765 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:34:21.765 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T22:34:21.765 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T22:34:21.765 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T22:34:21.765 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:34:22.024 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T22:34:22.075 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T22:34:22.076 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:34:22.076 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T22:34:22.076 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T22:34:22.076 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T22:34:22.076 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:34:22.353 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T22:34:22.380 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T22:34:22.380 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:34:22.380 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T22:34:22.380 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T22:34:22.380 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T22:34:22.380 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-15T22:34:22.550 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T22:34:22.600 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T22:34:22.600 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:34:22.600 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T22:34:22.601 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T22:34:22.601 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T22:34:22.601 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:34:22.612 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T22:34:22.636 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T22:34:22.636 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:34:22.636 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T22:34:22.636 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T22:34:22.636 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T22:34:22.636 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-15T22:34:22.733 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-15T22:34:22.792 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-15T22:34:22.857 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-15T22:34:23.071 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T22:34:23.081 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-15T22:34:23.094 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T22:34:23.094 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:34:23.094 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T22:34:23.094 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T22:34:23.095 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T22:34:23.095 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-15T22:34:23.137 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-15T22:34:23.201 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-15T22:34:23.221 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-15T22:34:23.258 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-15T22:34:23.313 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-15T22:34:23.321 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-15T22:34:23.499 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-15T22:34:23.520 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-15T22:34:23.547 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-15T22:34:23.575 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-15T22:34:23.609 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-15T22:34:23.624 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-15T22:34:23.635 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-15T22:34:23.712 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-15T22:34:23.860 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-15T22:34:23.913 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-15T22:34:23.945 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-15T22:34:23.973 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-15T22:34:24.234 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T22:34:24.268 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T22:34:24.309 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-15T22:34:24.376 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-15T22:34:24.415 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T22:34:24.426 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T22:34:24.455 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T22:34:24.487 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-15T22:34:24.549 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-15T22:34:24.578 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T22:34:25.618 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T22:34:25.631 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T22:34:25.655 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T22:34:25.675 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T22:34:25.874 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-15T22:34:25.915 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-15T22:34:25.955 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-15T22:34:25.988 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-15T22:34:26.006 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-15T22:34:26.189 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-15T22:34:26.262 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-15T22:34:26.292 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-15T22:34:26.321 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-15T22:34:26.368 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-15T22:34:26.620 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T22:34:26.647 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T22:34:26.749 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T22:34:26.761 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T22:34:26.792 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T22:34:26.910 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-15T22:34:26.929 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T22:34:26.941 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T22:34:26.965 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T22:34:26.965 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:34:26.965 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T22:34:26.966 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T22:34:26.966 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T22:34:26.966 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-15T22:34:26.982 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T22:34:27.008 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T22:34:27.020 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T22:34:27.023 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T22:34:27.045 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T22:34:27.119 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-15T22:34:27.122 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-15T22:34:27.153 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T22:34:27.172 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-15T22:34:27.204 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T22:34:27.205 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:34:27.205 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T22:34:27.205 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T22:34:27.205 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T22:34:27.205 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:34:27.262 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T22:34:27.289 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T22:34:27.396 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-15T22:34:27.465 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-15T22:34:27.873 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-15T22:34:27.953 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-15T22:34:27.994 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-15T22:34:28.075 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-15T22:34:28.099 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-15T22:34:28.122 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-15T22:34:28.127 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T22:34:28.148 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-15T22:34:28.151 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T22:34:28.151 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:34:28.151 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T22:34:28.151 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T22:34:28.152 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T22:34:28.152 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-15T22:34:28.174 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-15T22:34:28.174 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T22:34:28.194 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T22:34:28.217 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T22:34:28.218 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:34:28.218 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T22:34:28.218 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-15T22:34:28.258 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T22:34:28.258 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T22:34:28.262 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T22:34:28.285 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T22:34:28.285 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:34:28.285 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-15T22:34:28.285 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T22:34:28.285 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T22:34:28.285 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-15T22:34:28.342 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T22:34:28.367 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-15T22:34:28.373 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T22:34:28.373 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T22:34:28.463 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-15T22:34:28.506 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T22:34:28.552 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T22:34:28.552 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T22:34:28.558 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T22:34:28.558 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:34:28.558 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T22:34:28.558 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T22:34:28.558 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T22:34:28.558 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:34:28.596 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T22:34:28.620 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T22:34:28.620 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:34:28.620 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T22:34:28.620 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T22:34:28.620 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T22:34:28.620 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-15T22:34:28.623 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T22:34:28.636 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T22:34:28.674 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T22:34:28.674 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:34:28.674 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T22:34:28.674 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:34:28.699 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T22:34:28.724 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T22:34:28.775 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T22:34:28.776 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:34:28.776 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-15T22:34:28.776 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T22:34:28.776 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T22:34:28.776 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:34:29.108 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T22:34:29.160 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T22:34:29.160 INFO:teuthology.orchestra.run.smithi111.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T22:34:29.161 INFO:teuthology.orchestra.run.smithi111.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T22:34:29.161 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T22:34:29.161 INFO:teuthology.orchestra.run.smithi111.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T22:34:29.161 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:34:33.183 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-15T22:34:33.224 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-15T22:34:33.265 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-15T22:34:33.297 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-15T22:34:33.324 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-15T22:34:33.501 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-15T22:34:33.546 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-15T22:34:33.577 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-15T22:34:33.606 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-15T22:34:33.644 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-15T22:34:34.062 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-15T22:34:34.104 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-15T22:34:34.148 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-15T22:34:34.202 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-15T22:34:34.206 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T22:34:34.237 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-15T22:34:34.250 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T22:34:34.270 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T22:34:34.451 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T22:34:34.451 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-15T22:34:34.451 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-15T22:34:34.451 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-15T22:34:34.451 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-15T22:34:34.451 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-15T22:34:34.451 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-15T22:34:34.451 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-15T22:34:34.451 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T22:34:34.451 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T22:34:34.452 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T22:34:34.452 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-15T22:34:34.452 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-15T22:34:34.452 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T22:34:34.452 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-15T22:34:34.452 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-15T22:34:34.452 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-15T22:34:34.452 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-15T22:34:34.452 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-15T22:34:34.452 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-15T22:34:34.452 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-15T22:34:34.452 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-15T22:34:34.453 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-15T22:34:34.453 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-15T22:34:34.453 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-15T22:34:34.453 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-15T22:34:34.453 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-15T22:34:34.453 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-15T22:34:34.453 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-15T22:34:34.453 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-15T22:34:34.453 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-15T22:34:34.453 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-15T22:34:34.465 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-15T22:34:34.465 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-15T22:34:34.465 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-15T22:34:34.465 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-15T22:34:34.465 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-15T22:34:34.465 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-15T22:34:34.465 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-15T22:34:34.465 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-15T22:34:34.465 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-15T22:34:34.466 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-15T22:34:34.466 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-15T22:34:34.466 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-15T22:34:34.466 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-15T22:34:34.466 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-15T22:34:34.466 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-15T22:34:34.466 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-15T22:34:34.466 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-15T22:34:34.466 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-15T22:34:34.466 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-15T22:34:34.466 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-15T22:34:34.466 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-15T22:34:34.467 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-15T22:34:34.467 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-15T22:34:34.467 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-15T22:34:34.467 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-15T22:34:34.467 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-15T22:34:34.467 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-15T22:34:34.467 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-15T22:34:34.467 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-15T22:34:34.467 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-15T22:34:34.467 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-15T22:34:34.467 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-15T22:34:34.467 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-15T22:34:34.467 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-15T22:34:34.467 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-15T22:34:34.468 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-15T22:34:34.468 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-15T22:34:34.468 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-15T22:34:34.468 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-15T22:34:34.468 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-15T22:34:34.468 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-15T22:34:34.468 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-15T22:34:34.468 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-15T22:34:34.468 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-15T22:34:34.468 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-15T22:34:34.468 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-15T22:34:34.468 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-15T22:34:34.469 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-15T22:34:34.469 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-15T22:34:34.469 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-15T22:34:34.469 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-15T22:34:34.469 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-15T22:34:34.469 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-15T22:34:34.469 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-15T22:34:34.469 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-15T22:34:34.469 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-15T22:34:34.469 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-15T22:34:34.469 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-15T22:34:34.469 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-15T22:34:34.469 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-15T22:34:34.470 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-15T22:34:34.470 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-15T22:34:34.470 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-15T22:34:34.470 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-15T22:34:34.470 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-15T22:34:34.470 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-15T22:34:34.470 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-15T22:34:34.470 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-15T22:34:34.470 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-15T22:34:34.470 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-15T22:34:34.470 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-15T22:34:34.470 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-15T22:34:34.470 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-15T22:34:34.470 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-15T22:34:34.471 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-15T22:34:34.471 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-15T22:34:34.471 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-15T22:34:34.471 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-15T22:34:34.471 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-15T22:34:34.471 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-15T22:34:34.471 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-15T22:34:34.471 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-15T22:34:34.471 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-15T22:34:34.471 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-15T22:34:34.471 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-15T22:34:34.471 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-15T22:34:34.471 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-15T22:34:34.471 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-15T22:34:34.471 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-15T22:34:34.472 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-15T22:34:34.472 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-15T22:34:34.472 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-15T22:34:34.472 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-15T22:34:34.472 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-15T22:34:34.472 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-15T22:34:34.472 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-15T22:34:34.472 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-15T22:34:34.472 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-15T22:34:34.472 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-15T22:34:34.472 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-15T22:34:34.472 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-15T22:34:34.473 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-15T22:34:34.473 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-15T22:34:34.473 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-15T22:34:34.473 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-15T22:34:34.473 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-15T22:34:34.473 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-15T22:34:34.473 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-15T22:34:34.473 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-15T22:34:34.473 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-15T22:34:34.473 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-15T22:34:34.473 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-15T22:34:34.473 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-15T22:34:34.473 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-15T22:34:34.473 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-15T22:34:34.474 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-15T22:34:34.474 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-15T22:34:34.474 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-15T22:34:34.474 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-15T22:34:34.474 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-15T22:34:34.474 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-15T22:34:34.474 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-15T22:34:34.474 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-15T22:34:34.474 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-15T22:34:34.474 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-15T22:34:34.474 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-15T22:34:34.474 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-15T22:34:34.474 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-15T22:34:34.474 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-15T22:34:34.474 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-15T22:34:34.475 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-15T22:34:34.475 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-15T22:34:34.478 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-15T22:34:34.543 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-15T22:34:34.573 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-15T22:34:34.612 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-15T22:34:34.664 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-15T22:34:35.080 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-15T22:34:35.155 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-15T22:34:35.195 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-15T22:34:35.275 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-15T22:34:35.308 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-15T22:34:35.342 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-15T22:34:35.368 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-15T22:34:35.368 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T22:34:35.444 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T22:34:35.444 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T22:34:35.528 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T22:34:35.553 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-15T22:34:35.558 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T22:34:35.558 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T22:34:35.696 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T22:34:35.738 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T22:34:35.738 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T22:34:35.738 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T22:34:35.779 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T22:34:35.822 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T22:34:35.885 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T22:34:37.040 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-15T22:34:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T22:34:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T22:34:37.119 INFO:teuthology.orchestra.run.smithi192.stdout:Upgraded: 2024-09-15T22:34:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-15T22:34:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-15T22:34:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-15T22:34:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-15T22:34:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-15T22:34:37.119 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-15T22:34:37.120 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-15T22:34:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T22:34:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: automake-1.16.2-8.el9.noarch 2024-09-15T22:34:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: bison-3.7.4-5.el9.x86_64 2024-09-15T22:34:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T22:34:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.120 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:37.121 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-15T22:34:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: flex-2.6.4-9.el9.x86_64 2024-09-15T22:34:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T22:34:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T22:34:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T22:34:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-15T22:34:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T22:34:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T22:34:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-15T22:34:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T22:34:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T22:34:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T22:34:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T22:34:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T22:34:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T22:34:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T22:34:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T22:34:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-15T22:34:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T22:34:37.123 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-15T22:34:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-15T22:34:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T22:34:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T22:34:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T22:34:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T22:34:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T22:34:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T22:34:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-15T22:34:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T22:34:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-13.16-1.el9.x86_64 2024-09-15T22:34:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-15T22:34:37.124 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-15T22:34:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-15T22:34:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T22:34:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T22:34:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T22:34:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-15T22:34:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T22:34:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T22:34:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T22:34:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T22:34:37.125 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T22:34:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T22:34:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T22:34:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-15T22:34:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-15T22:34:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T22:34:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T22:34:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T22:34:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T22:34:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T22:34:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T22:34:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T22:34:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T22:34:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T22:34:37.126 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T22:34:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T22:34:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T22:34:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T22:34:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T22:34:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T22:34:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T22:34:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T22:34:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T22:34:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T22:34:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T22:34:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T22:34:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T22:34:37.127 INFO:teuthology.orchestra.run.smithi192.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T22:34:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T22:34:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T22:34:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T22:34:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T22:34:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T22:34:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T22:34:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T22:34:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T22:34:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T22:34:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-15T22:34:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T22:34:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T22:34:37.128 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T22:34:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T22:34:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T22:34:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T22:34:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T22:34:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T22:34:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T22:34:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T22:34:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T22:34:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-15T22:34:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T22:34:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T22:34:37.129 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T22:34:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T22:34:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T22:34:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T22:34:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T22:34:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T22:34:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T22:34:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T22:34:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-15T22:34:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-15T22:34:37.130 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T22:34:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-15T22:34:37.131 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T22:34:37.131 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-15T22:34:37.151 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-15T22:34:37.217 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-15T22:34:37.323 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-15T22:34:37.354 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-15T22:34:37.379 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-15T22:34:37.449 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-15T22:34:37.449 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T22:34:37.532 DEBUG:teuthology.parallel:result is None 2024-09-15T22:34:37.539 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T22:34:37.540 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T22:34:37.648 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T22:34:37.673 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-15T22:34:37.688 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T22:34:37.689 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T22:34:37.934 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T22:34:37.934 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T22:34:38.044 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T22:34:38.124 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T22:34:38.369 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T22:34:38.369 INFO:teuthology.orchestra.run.smithi111.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-15T22:34:38.369 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:34:41.380 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T22:34:41.380 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-15T22:34:41.380 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-15T22:34:41.380 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-15T22:34:41.380 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-15T22:34:41.380 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-15T22:34:41.380 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-15T22:34:41.380 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-15T22:34:41.380 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T22:34:41.381 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T22:34:41.381 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T22:34:41.381 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-15T22:34:41.381 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-15T22:34:41.381 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T22:34:41.381 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-15T22:34:41.381 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-15T22:34:41.381 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-15T22:34:41.381 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-15T22:34:41.381 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-15T22:34:41.381 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-15T22:34:41.381 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-15T22:34:41.381 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-15T22:34:41.382 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-15T22:34:41.382 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-15T22:34:41.382 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-15T22:34:41.382 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-15T22:34:41.382 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-15T22:34:41.382 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-15T22:34:41.382 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-15T22:34:41.382 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-15T22:34:41.382 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-15T22:34:41.382 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-15T22:34:41.382 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-15T22:34:41.383 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-15T22:34:41.383 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-15T22:34:41.383 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-15T22:34:41.383 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-15T22:34:41.383 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-15T22:34:41.383 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-15T22:34:41.383 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-15T22:34:41.383 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-15T22:34:41.383 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-15T22:34:41.383 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-15T22:34:41.383 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-15T22:34:41.383 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-15T22:34:41.384 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-15T22:34:41.384 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-15T22:34:41.384 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-15T22:34:41.384 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-15T22:34:41.384 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-15T22:34:41.384 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-15T22:34:41.384 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-15T22:34:41.384 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-15T22:34:41.384 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-15T22:34:41.384 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-15T22:34:41.384 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-15T22:34:41.384 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-15T22:34:41.384 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-15T22:34:41.385 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-15T22:34:41.385 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-15T22:34:41.385 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-15T22:34:41.385 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-15T22:34:41.385 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-15T22:34:41.385 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-15T22:34:41.385 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-15T22:34:41.385 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-15T22:34:41.385 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-15T22:34:41.385 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-15T22:34:41.385 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-15T22:34:41.385 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-15T22:34:41.385 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-15T22:34:41.385 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-15T22:34:41.386 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-15T22:34:41.386 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-15T22:34:41.386 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-15T22:34:41.386 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-15T22:34:41.386 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-15T22:34:41.386 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-15T22:34:41.386 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-15T22:34:41.386 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-15T22:34:41.386 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-15T22:34:41.386 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-15T22:34:41.386 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-15T22:34:41.386 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-15T22:34:41.387 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-15T22:34:41.387 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-15T22:34:41.387 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-15T22:34:41.387 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-15T22:34:41.387 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-15T22:34:41.387 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-15T22:34:41.387 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-15T22:34:41.387 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-15T22:34:41.387 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-15T22:34:41.388 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-15T22:34:41.388 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-15T22:34:41.388 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-15T22:34:41.388 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-15T22:34:41.388 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-15T22:34:41.388 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-15T22:34:41.388 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-15T22:34:41.388 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-15T22:34:41.388 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-15T22:34:41.388 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-15T22:34:41.388 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-15T22:34:41.388 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-15T22:34:41.388 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-15T22:34:41.389 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-15T22:34:41.389 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-15T22:34:41.389 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-15T22:34:41.389 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-15T22:34:41.389 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-15T22:34:41.389 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-15T22:34:41.389 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-15T22:34:41.389 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-15T22:34:41.389 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-15T22:34:41.389 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-15T22:34:41.389 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-15T22:34:41.389 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-15T22:34:41.389 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-15T22:34:41.390 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-15T22:34:41.390 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-15T22:34:41.390 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-15T22:34:41.390 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-15T22:34:41.390 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-15T22:34:41.390 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-15T22:34:41.390 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-15T22:34:41.390 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-15T22:34:41.390 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-15T22:34:41.390 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-15T22:34:41.390 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-15T22:34:41.390 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-15T22:34:41.390 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-15T22:34:41.390 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-15T22:34:41.390 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-15T22:34:41.391 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-15T22:34:41.391 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-15T22:34:41.391 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-15T22:34:41.391 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-15T22:34:41.391 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-15T22:34:41.391 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-15T22:34:41.391 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-15T22:34:41.391 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-15T22:34:41.391 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-15T22:34:41.391 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-15T22:34:41.391 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-15T22:34:41.391 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-15T22:34:41.391 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-15T22:34:41.392 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-15T22:34:41.392 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-15T22:34:41.392 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-15T22:34:41.392 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-15T22:34:41.392 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-15T22:34:41.392 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-15T22:34:41.392 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-15T22:34:41.392 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-15T22:34:41.392 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-15T22:34:41.393 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-15T22:34:41.393 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-15T22:34:41.393 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-15T22:34:41.393 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-15T22:34:41.393 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-15T22:34:41.394 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-15T22:34:41.394 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-15T22:34:41.394 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-15T22:34:43.976 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T22:34:43.976 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-15T22:34:43.976 INFO:teuthology.orchestra.run.smithi121.stdout:Upgraded: 2024-09-15T22:34:43.976 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-15T22:34:43.976 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-15T22:34:43.976 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-15T22:34:43.976 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-15T22:34:43.976 INFO:teuthology.orchestra.run.smithi121.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.977 INFO:teuthology.orchestra.run.smithi121.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.977 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-15T22:34:43.977 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-15T22:34:43.977 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-09-15T22:34:43.977 INFO:teuthology.orchestra.run.smithi121.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T22:34:43.977 INFO:teuthology.orchestra.run.smithi121.stdout: automake-1.16.2-8.el9.noarch 2024-09-15T22:34:43.977 INFO:teuthology.orchestra.run.smithi121.stdout: bison-3.7.4-5.el9.x86_64 2024-09-15T22:34:43.977 INFO:teuthology.orchestra.run.smithi121.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T22:34:43.977 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.977 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.977 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.978 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.978 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:43.978 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.978 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.978 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.978 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:43.978 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:43.978 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:43.978 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:43.978 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:43.978 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.978 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.979 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:43.979 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.979 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.979 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.979 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:43.979 INFO:teuthology.orchestra.run.smithi121.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:43.979 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.979 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.979 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:43.979 INFO:teuthology.orchestra.run.smithi121.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-15T22:34:43.979 INFO:teuthology.orchestra.run.smithi121.stdout: flex-2.6.4-9.el9.x86_64 2024-09-15T22:34:43.979 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T22:34:43.979 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T22:34:43.980 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T22:34:43.980 INFO:teuthology.orchestra.run.smithi121.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-15T22:34:43.980 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T22:34:43.980 INFO:teuthology.orchestra.run.smithi121.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T22:34:43.980 INFO:teuthology.orchestra.run.smithi121.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-15T22:34:43.980 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T22:34:43.980 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T22:34:43.980 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.980 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.980 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.980 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T22:34:43.980 INFO:teuthology.orchestra.run.smithi121.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T22:34:43.981 INFO:teuthology.orchestra.run.smithi121.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T22:34:43.981 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T22:34:43.981 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T22:34:43.981 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.981 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.981 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T22:34:43.981 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.981 INFO:teuthology.orchestra.run.smithi121.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-15T22:34:43.981 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T22:34:43.981 INFO:teuthology.orchestra.run.smithi121.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-15T22:34:43.981 INFO:teuthology.orchestra.run.smithi121.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-15T22:34:43.982 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T22:34:43.982 INFO:teuthology.orchestra.run.smithi121.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T22:34:43.982 INFO:teuthology.orchestra.run.smithi121.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T22:34:43.982 INFO:teuthology.orchestra.run.smithi121.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T22:34:43.982 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T22:34:43.982 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T22:34:43.982 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-15T22:34:43.982 INFO:teuthology.orchestra.run.smithi121.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T22:34:43.982 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-13.16-1.el9.x86_64 2024-09-15T22:34:43.982 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-15T22:34:43.982 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-15T22:34:43.983 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-15T22:34:43.983 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T22:34:43.983 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T22:34:43.983 INFO:teuthology.orchestra.run.smithi121.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T22:34:43.983 INFO:teuthology.orchestra.run.smithi121.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-15T22:34:43.983 INFO:teuthology.orchestra.run.smithi121.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T22:34:43.983 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T22:34:43.983 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T22:34:43.983 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.983 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.983 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.983 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T22:34:43.983 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T22:34:43.984 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T22:34:43.984 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T22:34:43.984 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-15T22:34:43.984 INFO:teuthology.orchestra.run.smithi121.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-15T22:34:43.984 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T22:34:43.984 INFO:teuthology.orchestra.run.smithi121.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T22:34:43.984 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T22:34:43.984 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T22:34:43.984 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T22:34:43.984 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T22:34:43.984 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T22:34:43.984 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T22:34:43.985 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T22:34:43.985 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T22:34:43.985 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T22:34:43.985 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T22:34:43.985 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T22:34:43.985 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T22:34:43.985 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T22:34:43.985 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T22:34:43.985 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T22:34:43.985 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T22:34:43.985 INFO:teuthology.orchestra.run.smithi121.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T22:34:43.985 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T22:34:43.985 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T22:34:43.986 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T22:34:43.986 INFO:teuthology.orchestra.run.smithi121.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T22:34:43.986 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T22:34:43.986 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T22:34:43.986 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T22:34:43.986 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T22:34:43.986 INFO:teuthology.orchestra.run.smithi121.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T22:34:43.986 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T22:34:43.986 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T22:34:43.986 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T22:34:43.986 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T22:34:43.986 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-15T22:34:43.986 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T22:34:43.987 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.987 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.987 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T22:34:43.987 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T22:34:43.987 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T22:34:43.987 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.987 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T22:34:43.987 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T22:34:43.987 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T22:34:43.987 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T22:34:43.987 INFO:teuthology.orchestra.run.smithi121.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T22:34:43.987 INFO:teuthology.orchestra.run.smithi121.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T22:34:43.987 INFO:teuthology.orchestra.run.smithi121.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T22:34:43.987 INFO:teuthology.orchestra.run.smithi121.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-15T22:34:43.988 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T22:34:43.988 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T22:34:43.988 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T22:34:43.988 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T22:34:43.988 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T22:34:43.988 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T22:34:43.988 INFO:teuthology.orchestra.run.smithi121.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T22:34:43.988 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.988 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.988 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:43.988 INFO:teuthology.orchestra.run.smithi121.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T22:34:43.988 INFO:teuthology.orchestra.run.smithi121.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T22:34:43.988 INFO:teuthology.orchestra.run.smithi121.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T22:34:43.989 INFO:teuthology.orchestra.run.smithi121.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-15T22:34:43.989 INFO:teuthology.orchestra.run.smithi121.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-15T22:34:43.989 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T22:34:43.989 INFO:teuthology.orchestra.run.smithi121.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-15T22:34:43.989 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-15T22:34:43.989 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-15T22:34:44.157 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-15T22:34:44.157 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-15T22:34:44.157 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-15T22:34:44.157 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-15T22:34:44.157 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-15T22:34:44.157 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-15T22:34:44.157 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-15T22:34:44.157 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T22:34:44.157 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T22:34:44.157 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T22:34:44.157 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-15T22:34:44.158 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-15T22:34:44.158 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T22:34:44.158 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-15T22:34:44.158 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-15T22:34:44.158 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-15T22:34:44.158 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-15T22:34:44.158 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-15T22:34:44.158 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-15T22:34:44.158 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-15T22:34:44.158 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-15T22:34:44.158 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-15T22:34:44.158 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-15T22:34:44.158 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-15T22:34:44.159 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-15T22:34:44.159 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-15T22:34:44.159 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-15T22:34:44.171 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-15T22:34:44.171 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-15T22:34:44.171 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-15T22:34:44.171 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-15T22:34:44.171 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-15T22:34:44.172 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-15T22:34:44.172 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-15T22:34:44.172 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-15T22:34:44.172 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-15T22:34:44.172 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-15T22:34:44.172 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-15T22:34:44.172 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-15T22:34:44.172 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-15T22:34:44.172 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-15T22:34:44.172 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-15T22:34:44.172 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-15T22:34:44.172 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-15T22:34:44.172 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-15T22:34:44.173 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-15T22:34:44.173 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-15T22:34:44.173 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-15T22:34:44.173 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-15T22:34:44.173 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-15T22:34:44.173 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-15T22:34:44.173 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-15T22:34:44.173 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-15T22:34:44.173 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-15T22:34:44.173 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-15T22:34:44.174 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-15T22:34:44.174 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-15T22:34:44.174 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-15T22:34:44.174 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-15T22:34:44.174 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-15T22:34:44.174 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-15T22:34:44.174 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-15T22:34:44.174 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-15T22:34:44.174 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-15T22:34:44.174 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-15T22:34:44.174 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-15T22:34:44.174 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-15T22:34:44.175 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-15T22:34:44.175 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-15T22:34:44.175 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-15T22:34:44.175 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-15T22:34:44.175 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-15T22:34:44.175 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-15T22:34:44.175 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-15T22:34:44.175 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-15T22:34:44.175 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-15T22:34:44.175 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-15T22:34:44.175 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-15T22:34:44.175 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-15T22:34:44.175 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-15T22:34:44.176 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-15T22:34:44.176 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-15T22:34:44.176 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-15T22:34:44.176 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-15T22:34:44.176 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-15T22:34:44.176 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-15T22:34:44.176 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-15T22:34:44.176 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-15T22:34:44.176 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-15T22:34:44.176 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-15T22:34:44.176 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-15T22:34:44.176 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-15T22:34:44.177 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-15T22:34:44.177 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-15T22:34:44.177 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-15T22:34:44.177 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-15T22:34:44.177 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-15T22:34:44.177 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-15T22:34:44.177 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-15T22:34:44.177 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-15T22:34:44.177 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-15T22:34:44.177 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-15T22:34:44.177 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-15T22:34:44.177 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-15T22:34:44.178 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-15T22:34:44.178 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-15T22:34:44.178 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-15T22:34:44.178 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-15T22:34:44.178 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-15T22:34:44.178 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-15T22:34:44.178 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-15T22:34:44.178 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-15T22:34:44.178 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-15T22:34:44.178 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-15T22:34:44.178 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-15T22:34:44.178 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-15T22:34:44.179 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-15T22:34:44.179 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-15T22:34:44.179 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-15T22:34:44.179 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-15T22:34:44.179 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-15T22:34:44.179 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-15T22:34:44.179 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-15T22:34:44.179 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-15T22:34:44.179 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-15T22:34:44.179 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-15T22:34:44.179 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-15T22:34:44.179 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-15T22:34:44.179 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-15T22:34:44.180 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-15T22:34:44.180 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-15T22:34:44.180 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-15T22:34:44.180 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-15T22:34:44.180 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-15T22:34:44.180 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-15T22:34:44.180 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-15T22:34:44.180 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-15T22:34:44.180 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-15T22:34:44.180 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-15T22:34:44.180 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-15T22:34:44.180 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-15T22:34:44.181 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-15T22:34:44.181 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-15T22:34:44.181 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-15T22:34:44.181 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-15T22:34:44.181 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-15T22:34:44.181 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-15T22:34:44.181 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-15T22:34:44.181 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-15T22:34:44.181 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-15T22:34:44.181 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-15T22:34:44.181 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-15T22:34:44.181 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-15T22:34:44.181 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-15T22:34:44.182 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-15T22:34:44.182 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-15T22:34:44.182 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-15T22:34:44.182 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-15T22:34:44.182 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-15T22:34:44.182 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-15T22:34:44.182 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-15T22:34:44.182 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-15T22:34:44.182 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-15T22:34:44.435 DEBUG:teuthology.parallel:result is None 2024-09-15T22:34:46.574 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T22:34:46.574 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:34:46.575 INFO:teuthology.orchestra.run.smithi111.stdout:Upgraded: 2024-09-15T22:34:46.575 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-15T22:34:46.575 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-15T22:34:46.575 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-15T22:34:46.575 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-15T22:34:46.575 INFO:teuthology.orchestra.run.smithi111.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.575 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.575 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-15T22:34:46.575 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-15T22:34:46.575 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2024-09-15T22:34:46.576 INFO:teuthology.orchestra.run.smithi111.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T22:34:46.576 INFO:teuthology.orchestra.run.smithi111.stdout: automake-1.16.2-8.el9.noarch 2024-09-15T22:34:46.576 INFO:teuthology.orchestra.run.smithi111.stdout: bison-3.7.4-5.el9.x86_64 2024-09-15T22:34:46.576 INFO:teuthology.orchestra.run.smithi111.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T22:34:46.576 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.576 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.576 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.576 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.576 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:46.576 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.576 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.576 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.576 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:46.576 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:46.577 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:46.577 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:46.577 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:46.577 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.577 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.577 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:46.577 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.577 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.577 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.577 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:46.577 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:46.577 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.577 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.577 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T22:34:46.578 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-15T22:34:46.578 INFO:teuthology.orchestra.run.smithi111.stdout: flex-2.6.4-9.el9.x86_64 2024-09-15T22:34:46.578 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T22:34:46.578 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T22:34:46.578 INFO:teuthology.orchestra.run.smithi111.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T22:34:46.578 INFO:teuthology.orchestra.run.smithi111.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-15T22:34:46.578 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T22:34:46.578 INFO:teuthology.orchestra.run.smithi111.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T22:34:46.578 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-15T22:34:46.578 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T22:34:46.578 INFO:teuthology.orchestra.run.smithi111.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T22:34:46.578 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.578 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.579 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.579 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T22:34:46.579 INFO:teuthology.orchestra.run.smithi111.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T22:34:46.579 INFO:teuthology.orchestra.run.smithi111.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T22:34:46.579 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T22:34:46.579 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T22:34:46.579 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.579 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.579 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T22:34:46.579 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.579 INFO:teuthology.orchestra.run.smithi111.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-15T22:34:46.579 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T22:34:46.579 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-15T22:34:46.579 INFO:teuthology.orchestra.run.smithi111.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-15T22:34:46.579 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T22:34:46.580 INFO:teuthology.orchestra.run.smithi111.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T22:34:46.580 INFO:teuthology.orchestra.run.smithi111.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T22:34:46.580 INFO:teuthology.orchestra.run.smithi111.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T22:34:46.580 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T22:34:46.580 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T22:34:46.580 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-15T22:34:46.580 INFO:teuthology.orchestra.run.smithi111.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T22:34:46.580 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-13.16-1.el9.x86_64 2024-09-15T22:34:46.580 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-15T22:34:46.580 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-15T22:34:46.581 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-15T22:34:46.581 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T22:34:46.581 INFO:teuthology.orchestra.run.smithi111.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T22:34:46.581 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T22:34:46.581 INFO:teuthology.orchestra.run.smithi111.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-15T22:34:46.581 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T22:34:46.581 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T22:34:46.581 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T22:34:46.581 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.581 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.581 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.581 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T22:34:46.582 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T22:34:46.582 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T22:34:46.582 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T22:34:46.582 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-15T22:34:46.582 INFO:teuthology.orchestra.run.smithi111.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-15T22:34:46.582 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T22:34:46.582 INFO:teuthology.orchestra.run.smithi111.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T22:34:46.582 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T22:34:46.582 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T22:34:46.582 INFO:teuthology.orchestra.run.smithi111.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T22:34:46.583 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T22:34:46.583 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T22:34:46.583 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T22:34:46.583 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T22:34:46.583 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T22:34:46.583 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T22:34:46.583 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T22:34:46.583 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T22:34:46.583 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T22:34:46.583 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T22:34:46.584 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T22:34:46.584 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T22:34:46.584 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T22:34:46.584 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T22:34:46.584 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T22:34:46.584 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T22:34:46.584 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T22:34:46.584 INFO:teuthology.orchestra.run.smithi111.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T22:34:46.584 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T22:34:46.584 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T22:34:46.584 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T22:34:46.585 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T22:34:46.585 INFO:teuthology.orchestra.run.smithi111.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T22:34:46.585 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T22:34:46.585 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T22:34:46.585 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T22:34:46.585 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T22:34:46.585 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-15T22:34:46.585 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T22:34:46.585 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.585 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.585 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T22:34:46.585 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T22:34:46.586 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T22:34:46.586 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.586 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T22:34:46.586 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T22:34:46.586 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T22:34:46.586 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T22:34:46.586 INFO:teuthology.orchestra.run.smithi111.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T22:34:46.586 INFO:teuthology.orchestra.run.smithi111.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T22:34:46.586 INFO:teuthology.orchestra.run.smithi111.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T22:34:46.586 INFO:teuthology.orchestra.run.smithi111.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-15T22:34:46.586 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T22:34:46.586 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T22:34:46.586 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T22:34:46.586 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T22:34:46.587 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T22:34:46.587 INFO:teuthology.orchestra.run.smithi111.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T22:34:46.587 INFO:teuthology.orchestra.run.smithi111.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T22:34:46.587 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.587 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.587 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T22:34:46.587 INFO:teuthology.orchestra.run.smithi111.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T22:34:46.587 INFO:teuthology.orchestra.run.smithi111.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T22:34:46.587 INFO:teuthology.orchestra.run.smithi111.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T22:34:46.587 INFO:teuthology.orchestra.run.smithi111.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-15T22:34:46.587 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-15T22:34:46.587 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T22:34:46.588 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-15T22:34:46.588 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:34:46.588 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2024-09-15T22:34:47.172 DEBUG:teuthology.parallel:result is None 2024-09-15T22:34:47.172 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:34:47.318 DEBUG:teuthology.orchestra.run.smithi111:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T22:34:47.365 INFO:teuthology.orchestra.run.smithi111.stdout:19.1.1-326.g8921c8ef.el9 2024-09-15T22:34:47.366 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-15T22:34:47.366 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-15T22:34:47.368 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:34:47.507 DEBUG:teuthology.orchestra.run.smithi121:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T22:34:47.533 INFO:teuthology.orchestra.run.smithi121.stdout:19.1.1-326.g8921c8ef.el9 2024-09-15T22:34:47.533 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-15T22:34:47.534 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-15T22:34:47.535 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:34:47.669 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T22:34:47.696 INFO:teuthology.orchestra.run.smithi192.stdout:19.1.1-326.g8921c8ef.el9 2024-09-15T22:34:47.696 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-15T22:34:47.696 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-15T22:34:47.698 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-15T22:34:47.698 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-15T22:34:47.698 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T22:34:47.760 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-15T22:34:47.761 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T22:34:47.793 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T22:34:47.793 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T22:34:47.825 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-15T22:34:47.825 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-15T22:34:47.825 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T22:34:47.886 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T22:34:47.984 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-15T22:34:47.984 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T22:34:48.014 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T22:34:48.083 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T22:34:48.083 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T22:34:48.113 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T22:34:48.183 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-15T22:34:48.183 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-15T22:34:48.183 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T22:34:48.239 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T22:34:48.335 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-15T22:34:48.335 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T22:34:48.364 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T22:34:48.434 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T22:34:48.434 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T22:34:48.464 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T22:34:48.533 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-15T22:34:48.533 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-15T22:34:48.533 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T22:34:48.590 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T22:34:48.687 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-15T22:34:48.687 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T22:34:48.717 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T22:34:48.786 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T22:34:48.786 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T22:34:48.816 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T22:34:48.885 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-15T22:34:48.974 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'subvols': {'create': 2}} 2024-09-15T22:34:48.975 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:34:48.975 INFO:tasks.cephadm:Cluster fsid is b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T22:34:48.975 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-15T22:34:48.975 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.111', 'mon.b': '172.21.15.121', 'mon.c': '172.21.15.192'} 2024-09-15T22:34:48.976 INFO:tasks.cephadm:First mon is mon.a on smithi111 2024-09-15T22:34:48.976 INFO:tasks.cephadm:First mgr is x 2024-09-15T22:34:48.976 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-15T22:34:48.976 DEBUG:teuthology.orchestra.run.smithi111:> sudo hostname $(hostname -s) 2024-09-15T22:34:49.034 DEBUG:teuthology.orchestra.run.smithi121:> sudo hostname $(hostname -s) 2024-09-15T22:34:49.064 DEBUG:teuthology.orchestra.run.smithi192:> sudo hostname $(hostname -s) 2024-09-15T22:34:49.094 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-15T22:34:49.094 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:34:49.232 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56081/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-326-g8921c8ef', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-326.g8921c8ef'}, 'url': 'https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-15 16:06:42.385367', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-15T22:34:49.374 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref squid, sha1 8921c8ef947807b717db041d85f0761b7914d5fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:34:49.375 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm 2024-09-15T22:34:49.375 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm 2024-09-15T22:34:49.375 DEBUG:teuthology.orchestra.run.smithi111:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-15T22:34:49.756 INFO:teuthology.orchestra.run.smithi111.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 15 22:34 /home/ubuntu/cephtest/cephadm 2024-09-15T22:34:49.757 DEBUG:teuthology.orchestra.run.smithi121:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-15T22:34:50.124 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 15 22:34 /home/ubuntu/cephtest/cephadm 2024-09-15T22:34:50.125 DEBUG:teuthology.orchestra.run.smithi192:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-15T22:34:50.482 INFO:teuthology.orchestra.run.smithi192.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 15 22:34 /home/ubuntu/cephtest/cephadm 2024-09-15T22:34:50.483 DEBUG:teuthology.orchestra.run.smithi111:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-15T22:34:50.520 DEBUG:teuthology.orchestra.run.smithi121:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-15T22:34:50.540 DEBUG:teuthology.orchestra.run.smithi192:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-15T22:34:50.570 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc on all hosts... 2024-09-15T22:34:50.570 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-15T22:34:50.575 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-15T22:34:50.582 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-15T22:34:52.091 INFO:teuthology.orchestra.run.smithi192.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-15T22:34:52.286 INFO:teuthology.orchestra.run.smithi121.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-15T22:34:53.532 INFO:teuthology.orchestra.run.smithi111.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-15T22:35:18.418 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T22:35:18.418 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-15T22:35:18.418 INFO:teuthology.orchestra.run.smithi192.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-15T22:35:18.418 INFO:teuthology.orchestra.run.smithi192.stdout: "repo_digests": [ 2024-09-15T22:35:18.418 INFO:teuthology.orchestra.run.smithi192.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-15T22:35:18.418 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T22:35:18.418 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T22:35:36.611 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-15T22:35:36.611 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-15T22:35:36.611 INFO:teuthology.orchestra.run.smithi121.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-15T22:35:36.611 INFO:teuthology.orchestra.run.smithi121.stdout: "repo_digests": [ 2024-09-15T22:35:36.611 INFO:teuthology.orchestra.run.smithi121.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-15T22:35:36.611 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-09-15T22:35:36.611 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-15T22:35:40.940 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-15T22:35:40.941 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-15T22:35:40.941 INFO:teuthology.orchestra.run.smithi111.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-15T22:35:40.941 INFO:teuthology.orchestra.run.smithi111.stdout: "repo_digests": [ 2024-09-15T22:35:40.941 INFO:teuthology.orchestra.run.smithi111.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-15T22:35:40.941 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-15T22:35:40.941 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-15T22:35:40.971 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /etc/ceph 2024-09-15T22:35:41.030 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /etc/ceph 2024-09-15T22:35:41.065 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /etc/ceph 2024-09-15T22:35:41.098 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 777 /etc/ceph 2024-09-15T22:35:41.153 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 777 /etc/ceph 2024-09-15T22:35:41.182 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 777 /etc/ceph 2024-09-15T22:35:41.211 INFO:tasks.cephadm:Writing seed config... 2024-09-15T22:35:41.212 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-15T22:35:41.212 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-15T22:35:41.213 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-15T22:35:41.213 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-15T22:35:41.213 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-15T22:35:41.213 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-15T22:35:41.213 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-15T22:35:41.213 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-15T22:35:41.213 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-15T22:35:41.213 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-15T22:35:41.213 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-15T22:35:41.213 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-15T22:35:41.213 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-15T22:35:41.213 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-15T22:35:41.213 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-15T22:35:41.214 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-15T22:35:41.214 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-15T22:35:41.214 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-15T22:35:41.214 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-15T22:35:41.214 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-15T22:35:41.214 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-15T22:35:41.214 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-15T22:35:41.214 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-15T22:35:41.214 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-15T22:35:41.214 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-15T22:35:41.214 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-15T22:35:41.214 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-15T22:35:41.214 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-15T22:35:41.214 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-15T22:35:41.215 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-15T22:35:41.215 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-15T22:35:41.215 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-15T22:35:41.215 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-15T22:35:41.215 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-15T22:35:41.215 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-15T22:35:41.215 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-15T22:35:41.215 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-15T22:35:41.215 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-15T22:35:41.215 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-15T22:35:41.215 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-15T22:35:41.215 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-15T22:35:41.215 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-15T22:35:41.215 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-15T22:35:41.215 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-15T22:35:41.216 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-15T22:35:41.216 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-15T22:35:41.216 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-15T22:35:41.216 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-15T22:35:41.216 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-15T22:35:41.217 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-15T22:35:41.217 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-15T22:35:41.251 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = b7614d3c-73b2-11ef-bceb-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-15T22:35:41.252 DEBUG:teuthology.orchestra.run.smithi111:mon.a> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mon.a.service 2024-09-15T22:35:41.295 DEBUG:teuthology.orchestra.run.smithi111:mgr.x> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mgr.x.service 2024-09-15T22:35:41.340 INFO:tasks.cephadm:Bootstrapping... 2024-09-15T22:35:41.340 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc -v bootstrap --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.111 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-15T22:35:41.593 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2024-09-15T22:35:41.594 INFO:teuthology.orchestra.run.smithi111.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc', '-v', 'bootstrap', '--fsid', 'b7614d3c-73b2-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.111', '--skip-admin-label'] 2024-09-15T22:35:41.594 INFO:teuthology.orchestra.run.smithi111.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-15T22:35:41.594 INFO:teuthology.orchestra.run.smithi111.stdout:Verifying podman|docker is present... 2024-09-15T22:35:41.638 INFO:teuthology.orchestra.run.smithi111.stdout:/bin/podman: stdout 5.2.2 2024-09-15T22:35:41.638 INFO:teuthology.orchestra.run.smithi111.stdout:Verifying lvm2 is present... 2024-09-15T22:35:41.639 INFO:teuthology.orchestra.run.smithi111.stdout:Verifying time synchronization is in place... 2024-09-15T22:35:41.653 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-15T22:35:41.653 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-15T22:35:41.665 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-15T22:35:41.665 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout inactive 2024-09-15T22:35:41.679 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout enabled 2024-09-15T22:35:41.691 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout active 2024-09-15T22:35:41.691 INFO:teuthology.orchestra.run.smithi111.stdout:Unit chronyd.service is enabled and running 2024-09-15T22:35:41.691 INFO:teuthology.orchestra.run.smithi111.stdout:Repeating the final host check... 2024-09-15T22:35:41.733 INFO:teuthology.orchestra.run.smithi111.stdout:/bin/podman: stdout 5.2.2 2024-09-15T22:35:41.733 INFO:teuthology.orchestra.run.smithi111.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-15T22:35:41.733 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl is present 2024-09-15T22:35:41.733 INFO:teuthology.orchestra.run.smithi111.stdout:lvcreate is present 2024-09-15T22:35:41.746 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-15T22:35:41.746 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-15T22:35:41.758 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-15T22:35:41.758 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout inactive 2024-09-15T22:35:41.772 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout enabled 2024-09-15T22:35:41.784 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout active 2024-09-15T22:35:41.784 INFO:teuthology.orchestra.run.smithi111.stdout:Unit chronyd.service is enabled and running 2024-09-15T22:35:41.784 INFO:teuthology.orchestra.run.smithi111.stdout:Host looks OK 2024-09-15T22:35:41.784 INFO:teuthology.orchestra.run.smithi111.stdout:Cluster fsid: b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T22:35:41.784 INFO:teuthology.orchestra.run.smithi111.stdout:Acquiring lock 140518446285824 on /run/cephadm/b7614d3c-73b2-11ef-bceb-c7b262605968.lock 2024-09-15T22:35:41.784 INFO:teuthology.orchestra.run.smithi111.stdout:Lock 140518446285824 acquired on /run/cephadm/b7614d3c-73b2-11ef-bceb-c7b262605968.lock 2024-09-15T22:35:41.785 INFO:teuthology.orchestra.run.smithi111.stdout:Verifying IP 172.21.15.111 port 3300 ... 2024-09-15T22:35:41.785 INFO:teuthology.orchestra.run.smithi111.stdout:Verifying IP 172.21.15.111 port 6789 ... 2024-09-15T22:35:41.785 INFO:teuthology.orchestra.run.smithi111.stdout:Base mon IP(s) is [172.21.15.111:3300, 172.21.15.111:6789], mon addrv is [v2:172.21.15.111:3300,v1:172.21.15.111:6789] 2024-09-15T22:35:41.792 INFO:teuthology.orchestra.run.smithi111.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.111 metric 100 2024-09-15T22:35:41.792 INFO:teuthology.orchestra.run.smithi111.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.111 metric 100 2024-09-15T22:35:41.798 INFO:teuthology.orchestra.run.smithi111.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-15T22:35:41.798 INFO:teuthology.orchestra.run.smithi111.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-15T22:35:41.799 INFO:teuthology.orchestra.run.smithi111.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-15T22:35:41.804 INFO:teuthology.orchestra.run.smithi111.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-15T22:35:41.804 INFO:teuthology.orchestra.run.smithi111.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-15T22:35:41.805 INFO:teuthology.orchestra.run.smithi111.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-15T22:35:41.805 INFO:teuthology.orchestra.run.smithi111.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-15T22:35:41.805 INFO:teuthology.orchestra.run.smithi111.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link noprefixroute 2024-09-15T22:35:41.805 INFO:teuthology.orchestra.run.smithi111.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-15T22:35:41.805 INFO:teuthology.orchestra.run.smithi111.stdout:Mon IP `172.21.15.111` is in CIDR network `172.21.0.0/20` 2024-09-15T22:35:41.805 INFO:teuthology.orchestra.run.smithi111.stdout:Mon IP `172.21.15.111` is in CIDR network `172.21.0.0/20` 2024-09-15T22:35:41.806 INFO:teuthology.orchestra.run.smithi111.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-15T22:35:41.806 INFO:teuthology.orchestra.run.smithi111.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-15T22:35:41.807 INFO:teuthology.orchestra.run.smithi111.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-15T22:35:42.335 INFO:teuthology.orchestra.run.smithi111.stdout:/bin/podman: stdout f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee 2024-09-15T22:35:42.335 INFO:teuthology.orchestra.run.smithi111.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-15T22:35:42.335 INFO:teuthology.orchestra.run.smithi111.stdout:/bin/podman: stderr Getting image source signatures 2024-09-15T22:35:42.335 INFO:teuthology.orchestra.run.smithi111.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-15T22:35:42.335 INFO:teuthology.orchestra.run.smithi111.stdout:/bin/podman: stderr Copying blob sha256:c0574da0c3174f3dc49daa274908f97d06456bd6a3cfab87e94022c1369a83a6 2024-09-15T22:35:42.335 INFO:teuthology.orchestra.run.smithi111.stdout:/bin/podman: stderr Copying config sha256:f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee 2024-09-15T22:35:42.335 INFO:teuthology.orchestra.run.smithi111.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-15T22:35:43.723 INFO:teuthology.orchestra.run.smithi111.stdout:ceph: stdout ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable) 2024-09-15T22:35:43.724 INFO:teuthology.orchestra.run.smithi111.stdout:Ceph version: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable) 2024-09-15T22:35:43.724 INFO:teuthology.orchestra.run.smithi111.stdout:Extracting ceph user uid/gid from container image... 2024-09-15T22:35:45.019 INFO:teuthology.orchestra.run.smithi111.stdout:stat: stdout 167 167 2024-09-15T22:35:45.019 INFO:teuthology.orchestra.run.smithi111.stdout:Creating initial keys... 2024-09-15T22:35:46.251 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph-authtool: stdout AQBBYedmhjfbGxAAspFenIuBo9HMfXzLe06ZbA== 2024-09-15T22:35:47.373 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph-authtool: stdout AQBCYedmAWcmJBAA8XfJ42g5PHzpY92B/kB6Nw== 2024-09-15T22:35:48.520 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph-authtool: stdout AQBDYedmcb7oMBAAkN66pJDtzFCINAZrnBXxlQ== 2024-09-15T22:35:48.521 INFO:teuthology.orchestra.run.smithi111.stdout:Creating initial monmap... 2024-09-15T22:35:49.586 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-15T22:35:49.587 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-15T22:35:49.587 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T22:35:49.587 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-15T22:35:49.587 INFO:teuthology.orchestra.run.smithi111.stdout:monmaptool for a [v2:172.21.15.111:3300,v1:172.21.15.111:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-15T22:35:49.587 INFO:teuthology.orchestra.run.smithi111.stdout:setting min_mon_release = quincy 2024-09-15T22:35:49.587 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/monmaptool: set fsid to b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T22:35:49.587 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-15T22:35:49.587 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:35:49.587 INFO:teuthology.orchestra.run.smithi111.stdout:Creating mon... 2024-09-15T22:35:50.633 INFO:teuthology.orchestra.run.smithi111.stdout:create mon.a on 2024-09-15T22:35:50.939 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-15T22:35:51.169 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-15T22:35:51.425 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-b7614d3c-73b2-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-b7614d3c-73b2-11ef-bceb-c7b262605968.target. 2024-09-15T22:35:51.425 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-b7614d3c-73b2-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-b7614d3c-73b2-11ef-bceb-c7b262605968.target. 2024-09-15T22:35:51.742 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mon.a 2024-09-15T22:35:51.749 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-15T22:35:52.002 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b7614d3c-73b2-11ef-bceb-c7b262605968.target.wants/ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@.service. 2024-09-15T22:35:52.601 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-15T22:35:52.601 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout disabled 2024-09-15T22:35:52.614 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-15T22:35:52.614 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout inactive 2024-09-15T22:35:52.614 INFO:teuthology.orchestra.run.smithi111.stdout:firewalld.service is not enabled 2024-09-15T22:35:52.614 INFO:teuthology.orchestra.run.smithi111.stdout:Not possible to enable service . firewalld.service is not available 2024-09-15T22:35:52.615 INFO:teuthology.orchestra.run.smithi111.stdout:Waiting for mon to start... 2024-09-15T22:35:52.615 INFO:teuthology.orchestra.run.smithi111.stdout:Waiting for mon... 2024-09-15T22:35:52.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:52 smithi111 systemd[1]: Started Ceph mon.a for b7614d3c-73b2-11ef-bceb-c7b262605968. 2024-09-15T22:35:53.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:53 smithi111 ceph-mon[30270]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-15T22:35:53.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:53 smithi111 ceph-mon[30270]: monmap epoch 1 2024-09-15T22:35:53.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:53 smithi111 ceph-mon[30270]: fsid b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T22:35:53.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:53 smithi111 ceph-mon[30270]: last_changed 2024-09-15T22:35:48.940270+0000 2024-09-15T22:35:53.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:53 smithi111 ceph-mon[30270]: created 2024-09-15T22:35:48.940270+0000 2024-09-15T22:35:53.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:53 smithi111 ceph-mon[30270]: min_mon_release 19 (squid) 2024-09-15T22:35:53.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:53 smithi111 ceph-mon[30270]: election_strategy: 1 2024-09-15T22:35:53.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:53 smithi111 ceph-mon[30270]: 0: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.a 2024-09-15T22:35:53.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:53 smithi111 ceph-mon[30270]: fsmap 2024-09-15T22:35:53.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:53 smithi111 ceph-mon[30270]: osdmap e1: 0 total, 0 up, 0 in 2024-09-15T22:35:53.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:53 smithi111 ceph-mon[30270]: mgrmap e1: no daemons active 2024-09-15T22:35:53.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:53 smithi111 ceph-mon[30270]: from='client.? 172.21.15.111:0/3541306284' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-15T22:35:54.093 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout cluster: 2024-09-15T22:35:54.093 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout id: b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T22:35:54.093 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-15T22:35:54.093 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 2024-09-15T22:35:54.094 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout services: 2024-09-15T22:35:54.094 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.668008s) 2024-09-15T22:35:54.094 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-15T22:35:54.094 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-15T22:35:54.094 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 2024-09-15T22:35:54.094 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout data: 2024-09-15T22:35:54.094 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-15T22:35:54.094 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-15T22:35:54.094 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-15T22:35:54.094 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout pgs: 2024-09-15T22:35:54.094 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 2024-09-15T22:35:54.094 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.162+0000 7fd408a00640 1 Processor -- start 2024-09-15T22:35:54.095 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.163+0000 7fd408a00640 1 -- start start 2024-09-15T22:35:54.095 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.163+0000 7fd408a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd40407ddb0 0x7fd40407c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:54.095 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.163+0000 7fd408a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd40407c7a0 con 0x7fd40407ddb0 2024-09-15T22:35:54.095 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.164+0000 7fd403400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd40407ddb0 0x7fd40407c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:54.095 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.164+0000 7fd403400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd40407ddb0 0x7fd40407c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54594/0 (socket says 172.21.15.111:54594) 2024-09-15T22:35:54.095 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.164+0000 7fd403400640 1 -- 172.21.15.111:0/3370916432 learned_addr learned my addr 172.21.15.111:0/3370916432 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:35:54.095 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.164+0000 7fd403400640 1 -- 172.21.15.111:0/3370916432 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd40407c920 con 0x7fd40407ddb0 2024-09-15T22:35:54.095 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.167+0000 7fd403400640 1 --2- 172.21.15.111:0/3370916432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd40407ddb0 0x7fd40407c260 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4009820 tx=0x7fd3f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb7ba68478371c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:54.095 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.167+0000 7fd402000640 1 -- 172.21.15.111:0/3370916432 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3f4037070 con 0x7fd40407ddb0 2024-09-15T22:35:54.095 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.167+0000 7fd402000640 1 -- 172.21.15.111:0/3370916432 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3f4032440 con 0x7fd40407ddb0 2024-09-15T22:35:54.095 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.167+0000 7fd402000640 1 -- 172.21.15.111:0/3370916432 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3f4032740 con 0x7fd40407ddb0 2024-09-15T22:35:54.095 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.168+0000 7fd408a00640 1 -- 172.21.15.111:0/3370916432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd40407ddb0 msgr2=0x7fd40407c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:54.096 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.168+0000 7fd408a00640 1 --2- 172.21.15.111:0/3370916432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd40407ddb0 0x7fd40407c260 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4009820 tx=0x7fd3f402a270 comp rx=0 tx=0).stop 2024-09-15T22:35:54.096 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.168+0000 7fd408a00640 1 -- 172.21.15.111:0/3370916432 shutdown_connections 2024-09-15T22:35:54.096 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.168+0000 7fd408a00640 1 --2- 172.21.15.111:0/3370916432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd40407ddb0 0x7fd40407c260 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:54.096 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.168+0000 7fd408a00640 1 -- 172.21.15.111:0/3370916432 >> 172.21.15.111:0/3370916432 conn(0x7fd404102d70 msgr2=0x7fd404105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:54.096 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.168+0000 7fd408a00640 1 -- 172.21.15.111:0/3370916432 shutdown_connections 2024-09-15T22:35:54.096 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.168+0000 7fd408a00640 1 -- 172.21.15.111:0/3370916432 wait complete. 2024-09-15T22:35:54.096 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.169+0000 7fd408a00640 1 Processor -- start 2024-09-15T22:35:54.096 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.169+0000 7fd408a00640 1 -- start start 2024-09-15T22:35:54.096 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.169+0000 7fd408a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd40407ddb0 0x7fd40419cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:54.096 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.169+0000 7fd408a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd40407cec0 con 0x7fd40407ddb0 2024-09-15T22:35:54.096 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.170+0000 7fd403400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd40407ddb0 0x7fd40419cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:54.097 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.170+0000 7fd403400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd40407ddb0 0x7fd40419cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54606/0 (socket says 172.21.15.111:54606) 2024-09-15T22:35:54.097 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.170+0000 7fd403400640 1 -- 172.21.15.111:0/3541306284 learned_addr learned my addr 172.21.15.111:0/3541306284 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:35:54.097 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.170+0000 7fd403400640 1 -- 172.21.15.111:0/3541306284 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd40419d460 con 0x7fd40407ddb0 2024-09-15T22:35:54.097 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.171+0000 7fd403400640 1 --2- 172.21.15.111:0/3541306284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd40407ddb0 0x7fd40419cf20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd3f40090b0 tx=0x7fd3f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:54.097 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.171+0000 7fd3fbe00640 1 -- 172.21.15.111:0/3541306284 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3f4041070 con 0x7fd40407ddb0 2024-09-15T22:35:54.097 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.171+0000 7fd3fbe00640 1 -- 172.21.15.111:0/3541306284 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3f403c7a0 con 0x7fd40407ddb0 2024-09-15T22:35:54.097 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.171+0000 7fd3fbe00640 1 -- 172.21.15.111:0/3541306284 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3f4037040 con 0x7fd40407ddb0 2024-09-15T22:35:54.097 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.171+0000 7fd408a00640 1 -- 172.21.15.111:0/3541306284 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd40419d6f0 con 0x7fd40407ddb0 2024-09-15T22:35:54.097 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.171+0000 7fd408a00640 1 -- 172.21.15.111:0/3541306284 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4041a03e0 con 0x7fd40407ddb0 2024-09-15T22:35:54.097 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.171+0000 7fd3fbe00640 1 -- 172.21.15.111:0/3541306284 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd3f4044070 con 0x7fd40407ddb0 2024-09-15T22:35:54.097 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.171+0000 7fd3fbe00640 1 -- 172.21.15.111:0/3541306284 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd3f403d3f0 con 0x7fd40407ddb0 2024-09-15T22:35:54.097 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.172+0000 7fd408a00640 1 -- 172.21.15.111:0/3541306284 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd40407c2e0 con 0x7fd40407ddb0 2024-09-15T22:35:54.097 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.175+0000 7fd3fbe00640 1 -- 172.21.15.111:0/3541306284 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fd3f4046450 con 0x7fd40407ddb0 2024-09-15T22:35:54.098 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.236+0000 7fd408a00640 1 -- 172.21.15.111:0/3541306284 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fd40402d910 con 0x7fd40407ddb0 2024-09-15T22:35:54.098 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.237+0000 7fd3fbe00640 1 -- 172.21.15.111:0/3541306284 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fd3f4046630 con 0x7fd40407ddb0 2024-09-15T22:35:54.098 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.238+0000 7fd408a00640 1 -- 172.21.15.111:0/3541306284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd40407ddb0 msgr2=0x7fd40419cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:54.098 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.238+0000 7fd408a00640 1 --2- 172.21.15.111:0/3541306284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd40407ddb0 0x7fd40419cf20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd3f40090b0 tx=0x7fd3f40046c0 comp rx=0 tx=0).stop 2024-09-15T22:35:54.098 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.238+0000 7fd408a00640 1 -- 172.21.15.111:0/3541306284 shutdown_connections 2024-09-15T22:35:54.098 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.238+0000 7fd408a00640 1 --2- 172.21.15.111:0/3541306284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd40407ddb0 0x7fd40419cf20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:54.098 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.238+0000 7fd408a00640 1 -- 172.21.15.111:0/3541306284 >> 172.21.15.111:0/3541306284 conn(0x7fd404102d70 msgr2=0x7fd404103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:54.098 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.238+0000 7fd408a00640 1 -- 172.21.15.111:0/3541306284 shutdown_connections 2024-09-15T22:35:54.098 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:53.238+0000 7fd408a00640 1 -- 172.21.15.111:0/3541306284 wait complete. 2024-09-15T22:35:54.098 INFO:teuthology.orchestra.run.smithi111.stdout:mon is available 2024-09-15T22:35:54.098 INFO:teuthology.orchestra.run.smithi111.stdout:Assimilating anything we can from ceph.conf... 2024-09-15T22:35:54.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:54 smithi111 ceph-mon[30270]: from='client.? 172.21.15.111:0/2532207504' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-15T22:35:54.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:54 smithi111 ceph-mon[30270]: from='client.? 172.21.15.111:0/2532207504' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-15T22:35:55.356 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 2024-09-15T22:35:55.356 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout [global] 2024-09-15T22:35:55.356 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout fsid = b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T22:35:55.357 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.111:3300,v1:172.21.15.111:6789] 2024-09-15T22:35:55.357 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-15T22:35:55.357 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-15T22:35:55.357 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-15T22:35:55.357 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-15T22:35:55.357 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 2024-09-15T22:35:55.357 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-15T22:35:55.357 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-15T22:35:55.357 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 2024-09-15T22:35:55.357 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout [osd] 2024-09-15T22:35:55.357 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-15T22:35:55.357 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-15T22:35:55.357 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-15T22:35:55.357 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-15T22:35:55.358 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-15T22:35:55.358 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.593+0000 7f98c7600640 1 Processor -- start 2024-09-15T22:35:55.358 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.593+0000 7f98c7600640 1 -- start start 2024-09-15T22:35:55.358 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.593+0000 7f98c7600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98c0109930 0x7f98c0109d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:55.358 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.593+0000 7f98c7600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f98c010a300 con 0x7f98c0109930 2024-09-15T22:35:55.358 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.594+0000 7f98c4c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98c0109930 0x7f98c0109d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:55.358 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.594+0000 7f98c4c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98c0109930 0x7f98c0109d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54620/0 (socket says 172.21.15.111:54620) 2024-09-15T22:35:55.358 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.594+0000 7f98c4c00640 1 -- 172.21.15.111:0/930272026 learned_addr learned my addr 172.21.15.111:0/930272026 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:35:55.358 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.594+0000 7f98c4c00640 1 -- 172.21.15.111:0/930272026 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c010ab20 con 0x7f98c0109930 2024-09-15T22:35:55.358 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.595+0000 7f98c4c00640 1 --2- 172.21.15.111:0/930272026 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98c0109930 0x7f98c0109d30 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009820 tx=0x7f98b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee1e0b66349afc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:55.358 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.595+0000 7f98bf400640 1 -- 172.21.15.111:0/930272026 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98b4037070 con 0x7f98c0109930 2024-09-15T22:35:55.359 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.595+0000 7f98bf400640 1 -- 172.21.15.111:0/930272026 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98b4032440 con 0x7f98c0109930 2024-09-15T22:35:55.360 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.595+0000 7f98bf400640 1 -- 172.21.15.111:0/930272026 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98b4032740 con 0x7f98c0109930 2024-09-15T22:35:55.360 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.596+0000 7f98c7600640 1 -- 172.21.15.111:0/930272026 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98c0109930 msgr2=0x7f98c0109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:55.360 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.596+0000 7f98c7600640 1 --2- 172.21.15.111:0/930272026 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98c0109930 0x7f98c0109d30 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009820 tx=0x7f98b402a270 comp rx=0 tx=0).stop 2024-09-15T22:35:55.360 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.596+0000 7f98c7600640 1 -- 172.21.15.111:0/930272026 shutdown_connections 2024-09-15T22:35:55.360 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.596+0000 7f98c7600640 1 --2- 172.21.15.111:0/930272026 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98c0109930 0x7f98c0109d30 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:55.360 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.596+0000 7f98c7600640 1 -- 172.21.15.111:0/930272026 >> 172.21.15.111:0/930272026 conn(0x7f98c007cfa0 msgr2=0x7f98c007d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:55.360 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.596+0000 7f98c7600640 1 -- 172.21.15.111:0/930272026 shutdown_connections 2024-09-15T22:35:55.360 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.596+0000 7f98c7600640 1 -- 172.21.15.111:0/930272026 wait complete. 2024-09-15T22:35:55.360 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.597+0000 7f98c7600640 1 Processor -- start 2024-09-15T22:35:55.360 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.597+0000 7f98c7600640 1 -- start start 2024-09-15T22:35:55.361 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.598+0000 7f98c7600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98c0109930 0x7f98c019d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:55.361 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.598+0000 7f98c7600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f98c010b0c0 con 0x7f98c0109930 2024-09-15T22:35:55.361 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.598+0000 7f98c4c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98c0109930 0x7f98c019d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:55.361 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.598+0000 7f98c4c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98c0109930 0x7f98c019d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54628/0 (socket says 172.21.15.111:54628) 2024-09-15T22:35:55.361 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.598+0000 7f98c4c00640 1 -- 172.21.15.111:0/2532207504 learned_addr learned my addr 172.21.15.111:0/2532207504 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:35:55.361 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.598+0000 7f98c4c00640 1 -- 172.21.15.111:0/2532207504 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c019d630 con 0x7f98c0109930 2024-09-15T22:35:55.361 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.598+0000 7f98c4c00640 1 --2- 172.21.15.111:0/2532207504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98c0109930 0x7f98c019d0f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f98b40090b0 tx=0x7f98b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:55.361 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.599+0000 7f98bd600640 1 -- 172.21.15.111:0/2532207504 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98b4041070 con 0x7f98c0109930 2024-09-15T22:35:55.361 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.599+0000 7f98bd600640 1 -- 172.21.15.111:0/2532207504 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98b403c7a0 con 0x7f98c0109930 2024-09-15T22:35:55.361 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.599+0000 7f98bd600640 1 -- 172.21.15.111:0/2532207504 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98b4037040 con 0x7f98c0109930 2024-09-15T22:35:55.362 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.599+0000 7f98c7600640 1 -- 172.21.15.111:0/2532207504 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c019d8c0 con 0x7f98c0109930 2024-09-15T22:35:55.362 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.599+0000 7f98c7600640 1 -- 172.21.15.111:0/2532207504 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c01a05b0 con 0x7f98c0109930 2024-09-15T22:35:55.362 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.599+0000 7f98bd600640 1 -- 172.21.15.111:0/2532207504 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f98b4044070 con 0x7f98c0109930 2024-09-15T22:35:55.362 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.599+0000 7f98bd600640 1 -- 172.21.15.111:0/2532207504 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f98b403d3f0 con 0x7f98c0109930 2024-09-15T22:35:55.362 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.600+0000 7f98c7600640 1 -- 172.21.15.111:0/2532207504 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98c010b710 con 0x7f98c0109930 2024-09-15T22:35:55.362 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.603+0000 7f98bd600640 1 -- 172.21.15.111:0/2532207504 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f98b4046450 con 0x7f98c0109930 2024-09-15T22:35:55.362 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.662+0000 7f98c7600640 1 -- 172.21.15.111:0/2532207504 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f98c01a1c80 con 0x7f98c0109930 2024-09-15T22:35:55.362 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.667+0000 7f98bd600640 1 -- 172.21.15.111:0/2532207504 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7f98b4046630 con 0x7f98c0109930 2024-09-15T22:35:55.362 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.668+0000 7f98bd600640 1 -- 172.21.15.111:0/2532207504 <== mon.0 v2:172.21.15.111:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f98b40584b0 con 0x7f98c0109930 2024-09-15T22:35:55.362 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.669+0000 7f98c7600640 1 -- 172.21.15.111:0/2532207504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98c0109930 msgr2=0x7f98c019d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:55.362 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.669+0000 7f98c7600640 1 --2- 172.21.15.111:0/2532207504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98c0109930 0x7f98c019d0f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f98b40090b0 tx=0x7f98b40046c0 comp rx=0 tx=0).stop 2024-09-15T22:35:55.363 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.669+0000 7f98c7600640 1 -- 172.21.15.111:0/2532207504 shutdown_connections 2024-09-15T22:35:55.363 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.669+0000 7f98c7600640 1 --2- 172.21.15.111:0/2532207504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98c0109930 0x7f98c019d0f0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:55.363 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.669+0000 7f98c7600640 1 -- 172.21.15.111:0/2532207504 >> 172.21.15.111:0/2532207504 conn(0x7f98c007cfa0 msgr2=0x7f98c01912d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:55.363 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.669+0000 7f98c7600640 1 -- 172.21.15.111:0/2532207504 shutdown_connections 2024-09-15T22:35:55.363 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:54.669+0000 7f98c7600640 1 -- 172.21.15.111:0/2532207504 wait complete. 2024-09-15T22:35:55.363 INFO:teuthology.orchestra.run.smithi111.stdout:Generating new minimal ceph.conf... 2024-09-15T22:35:56.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:55 smithi111 ceph-mon[30270]: from='client.? 172.21.15.111:0/4114362210' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:35:56.610 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.843+0000 7f6b83200640 1 Processor -- start 2024-09-15T22:35:56.610 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.844+0000 7f6b83200640 1 -- start start 2024-09-15T22:35:56.610 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.844+0000 7f6b83200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b7c107700 0x7f6b7c107b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:56.610 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.844+0000 7f6b83200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c1080d0 con 0x7f6b7c107700 2024-09-15T22:35:56.610 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.844+0000 7f6b7be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b7c107700 0x7f6b7c107b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:56.610 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.844+0000 7f6b7be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b7c107700 0x7f6b7c107b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54636/0 (socket says 172.21.15.111:54636) 2024-09-15T22:35:56.610 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.844+0000 7f6b7be00640 1 -- 172.21.15.111:0/634367490 learned_addr learned my addr 172.21.15.111:0/634367490 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:35:56.610 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.845+0000 7f6b7be00640 1 -- 172.21.15.111:0/634367490 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b7c108900 con 0x7f6b7c107700 2024-09-15T22:35:56.611 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.846+0000 7f6b7be00640 1 --2- 172.21.15.111:0/634367490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b7c107700 0x7f6b7c107b00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c009820 tx=0x7f6b6c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9186504dcf798f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:56.611 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.846+0000 7f6b7b400640 1 -- 172.21.15.111:0/634367490 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b6c037070 con 0x7f6b7c107700 2024-09-15T22:35:56.611 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.846+0000 7f6b7b400640 1 -- 172.21.15.111:0/634367490 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6b6c032900 con 0x7f6b7c107700 2024-09-15T22:35:56.611 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.846+0000 7f6b7b400640 1 -- 172.21.15.111:0/634367490 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b6c030850 con 0x7f6b7c107700 2024-09-15T22:35:56.611 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.846+0000 7f6b83200640 1 -- 172.21.15.111:0/634367490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b7c107700 msgr2=0x7f6b7c107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:56.611 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.846+0000 7f6b83200640 1 --2- 172.21.15.111:0/634367490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b7c107700 0x7f6b7c107b00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c009820 tx=0x7f6b6c02a270 comp rx=0 tx=0).stop 2024-09-15T22:35:56.611 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.847+0000 7f6b83200640 1 -- 172.21.15.111:0/634367490 shutdown_connections 2024-09-15T22:35:56.611 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.847+0000 7f6b83200640 1 --2- 172.21.15.111:0/634367490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b7c107700 0x7f6b7c107b00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:56.611 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.847+0000 7f6b83200640 1 -- 172.21.15.111:0/634367490 >> 172.21.15.111:0/634367490 conn(0x7f6b7c102f30 msgr2=0x7f6b7c105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:56.611 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.847+0000 7f6b83200640 1 -- 172.21.15.111:0/634367490 shutdown_connections 2024-09-15T22:35:56.611 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.847+0000 7f6b83200640 1 -- 172.21.15.111:0/634367490 wait complete. 2024-09-15T22:35:56.611 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.847+0000 7f6b83200640 1 Processor -- start 2024-09-15T22:35:56.611 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.848+0000 7f6b83200640 1 -- start start 2024-09-15T22:35:56.612 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.848+0000 7f6b83200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b7c107700 0x7f6b7c07daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:35:56.612 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.848+0000 7f6b83200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c10bcb0 con 0x7f6b7c107700 2024-09-15T22:35:56.612 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.848+0000 7f6b7be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b7c107700 0x7f6b7c07daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:35:56.612 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.848+0000 7f6b7be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b7c107700 0x7f6b7c07daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54640/0 (socket says 172.21.15.111:54640) 2024-09-15T22:35:56.612 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.848+0000 7f6b7be00640 1 -- 172.21.15.111:0/4114362210 learned_addr learned my addr 172.21.15.111:0/4114362210 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:35:56.612 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.848+0000 7f6b7be00640 1 -- 172.21.15.111:0/4114362210 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b7c07e030 con 0x7f6b7c107700 2024-09-15T22:35:56.612 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.849+0000 7f6b7be00640 1 --2- 172.21.15.111:0/4114362210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b7c107700 0x7f6b7c07daf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0097f0 tx=0x7f6b6c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:35:56.612 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.849+0000 7f6b79600640 1 -- 172.21.15.111:0/4114362210 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b6c037040 con 0x7f6b7c107700 2024-09-15T22:35:56.612 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.849+0000 7f6b79600640 1 -- 172.21.15.111:0/4114362210 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6b6c040070 con 0x7f6b7c107700 2024-09-15T22:35:56.612 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.849+0000 7f6b79600640 1 -- 172.21.15.111:0/4114362210 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b6c03a390 con 0x7f6b7c107700 2024-09-15T22:35:56.612 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.849+0000 7f6b83200640 1 -- 172.21.15.111:0/4114362210 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b7c07c220 con 0x7f6b7c107700 2024-09-15T22:35:56.612 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.849+0000 7f6b83200640 1 -- 172.21.15.111:0/4114362210 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b7c07c700 con 0x7f6b7c107700 2024-09-15T22:35:56.612 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.849+0000 7f6b79600640 1 -- 172.21.15.111:0/4114362210 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6b6c03a5e0 con 0x7f6b7c107700 2024-09-15T22:35:56.613 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.850+0000 7f6b79600640 1 -- 172.21.15.111:0/4114362210 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6b6c045950 con 0x7f6b7c107700 2024-09-15T22:35:56.613 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.850+0000 7f6b83200640 1 -- 172.21.15.111:0/4114362210 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b7c107b80 con 0x7f6b7c107700 2024-09-15T22:35:56.613 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.853+0000 7f6b79600640 1 -- 172.21.15.111:0/4114362210 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f6b6c045460 con 0x7f6b7c107700 2024-09-15T22:35:56.613 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.912+0000 7f6b83200640 1 -- 172.21.15.111:0/4114362210 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f6b7c07cc30 con 0x7f6b7c107700 2024-09-15T22:35:56.613 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.913+0000 7f6b79600640 1 -- 172.21.15.111:0/4114362210 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f6b6c03b430 con 0x7f6b7c107700 2024-09-15T22:35:56.613 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.914+0000 7f6b83200640 1 -- 172.21.15.111:0/4114362210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b7c107700 msgr2=0x7f6b7c07daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:35:56.613 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.914+0000 7f6b83200640 1 --2- 172.21.15.111:0/4114362210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b7c107700 0x7f6b7c07daf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0097f0 tx=0x7f6b6c030d80 comp rx=0 tx=0).stop 2024-09-15T22:35:56.613 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.914+0000 7f6b83200640 1 -- 172.21.15.111:0/4114362210 shutdown_connections 2024-09-15T22:35:56.613 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.914+0000 7f6b83200640 1 --2- 172.21.15.111:0/4114362210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b7c107700 0x7f6b7c07daf0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:35:56.613 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.914+0000 7f6b83200640 1 -- 172.21.15.111:0/4114362210 >> 172.21.15.111:0/4114362210 conn(0x7f6b7c102f30 msgr2=0x7f6b7c103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:35:56.613 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.914+0000 7f6b83200640 1 -- 172.21.15.111:0/4114362210 shutdown_connections 2024-09-15T22:35:56.613 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:55.914+0000 7f6b83200640 1 -- 172.21.15.111:0/4114362210 wait complete. 2024-09-15T22:35:56.614 INFO:teuthology.orchestra.run.smithi111.stdout:Restarting the monitor... 2024-09-15T22:35:56.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:56 smithi111 systemd[1]: Stopping Ceph mon.a for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T22:35:57.215 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:56 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-a[30266]: 2024-09-15T22:35:56.947+0000 7f7d20600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:35:57.215 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:56 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-a[30266]: 2024-09-15T22:35:56.947+0000 7f7d20600640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-15T22:35:57.471 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:57 smithi111 podman[30508]: 2024-09-15 22:35:57.214630792 +0000 UTC m=+0.376972476 container died 4da2ff5e8daf493df014aeb5867b7fcde289731ae0a2189456b4e4844968ddef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-15T22:35:57.804 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:57 smithi111 podman[30508]: 2024-09-15 22:35:57.635951752 +0000 UTC m=+0.798293436 container cleanup 4da2ff5e8daf493df014aeb5867b7fcde289731ae0a2189456b4e4844968ddef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e) 2024-09-15T22:35:57.804 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:57 smithi111 bash[30508]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-a 2024-09-15T22:35:58.057 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:57 smithi111 podman[30521]: 2024-09-15 22:35:57.803443705 +0000 UTC m=+0.579504309 container remove 4da2ff5e8daf493df014aeb5867b7fcde289731ae0a2189456b4e4844968ddef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731) 2024-09-15T22:35:58.369 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 systemd[1]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-15T22:35:58.369 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 systemd[1]: Stopped Ceph mon.a for b7614d3c-73b2-11ef-bceb-c7b262605968. 2024-09-15T22:35:58.369 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 systemd[1]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mon.a.service: Consumed 1.182s CPU time. 2024-09-15T22:35:58.369 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 systemd[1]: Starting Ceph mon.a for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T22:35:58.639 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 podman[30610]: 2024-09-15 22:35:58.272265917 +0000 UTC m=+0.027277683 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:35:58.639 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 podman[30610]: 2024-09-15 22:35:58.383199179 +0000 UTC m=+0.138210921 container create 200de39e07abb43550a97f99fafefa8bc7eb2f15408fdb45d95fab09f14d6c02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-a, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:35:58.639 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 podman[30610]: 2024-09-15 22:35:58.598674698 +0000 UTC m=+0.353686441 container init 200de39e07abb43550a97f99fafefa8bc7eb2f15408fdb45d95fab09f14d6c02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-09-15T22:35:58.639 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 podman[30610]: 2024-09-15 22:35:58.605133626 +0000 UTC m=+0.360145369 container start 200de39e07abb43550a97f99fafefa8bc7eb2f15408fdb45d95fab09f14d6c02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-a, ceph=True, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-15T22:35:58.732 INFO:teuthology.orchestra.run.smithi111.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-15T22:35:58.918 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: set uid:gid to 167:167 (ceph:ceph) 2024-09-15T22:35:58.918 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable), process ceph-mon, pid 2 2024-09-15T22:35:58.918 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: pidfile_write: ignore empty --pid-file 2024-09-15T22:35:58.918 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: load: jerasure load: lrc 2024-09-15T22:35:58.918 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: RocksDB version: 7.9.2 2024-09-15T22:35:58.918 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Git sha 0 2024-09-15T22:35:58.918 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Compile date 2024-09-15 15:04:35 2024-09-15T22:35:58.918 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: DB SUMMARY 2024-09-15T22:35:58.919 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: DB Session ID: 79SI5ZPIMX4OH556APTQ 2024-09-15T22:35:58.919 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: CURRENT file: CURRENT 2024-09-15T22:35:58.919 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: IDENTITY file: IDENTITY 2024-09-15T22:35:58.919 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-15T22:35:58.919 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-15T22:35:58.919 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106254 ; 2024-09-15T22:35:58.919 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.error_if_exists: 0 2024-09-15T22:35:58.919 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.create_if_missing: 0 2024-09-15T22:35:58.919 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.paranoid_checks: 1 2024-09-15T22:35:58.919 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-15T22:35:58.919 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-15T22:35:58.919 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-15T22:35:58.919 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.env: 0x5651243d5940 2024-09-15T22:35:58.920 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.fs: PosixFileSystem 2024-09-15T22:35:58.920 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.info_log: 0x565142d7d6a0 2024-09-15T22:35:58.920 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.max_file_opening_threads: 16 2024-09-15T22:35:58.920 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.statistics: (nil) 2024-09-15T22:35:58.920 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.use_fsync: 0 2024-09-15T22:35:58.920 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.max_log_file_size: 0 2024-09-15T22:35:58.920 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-15T22:35:58.920 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-15T22:35:58.920 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.keep_log_file_num: 1000 2024-09-15T22:35:58.920 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.recycle_log_file_num: 0 2024-09-15T22:35:58.920 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.allow_fallocate: 1 2024-09-15T22:35:58.920 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.allow_mmap_reads: 0 2024-09-15T22:35:58.920 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.allow_mmap_writes: 0 2024-09-15T22:35:58.921 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.use_direct_reads: 0 2024-09-15T22:35:58.921 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-15T22:35:58.921 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.create_missing_column_families: 0 2024-09-15T22:35:58.921 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.db_log_dir: 2024-09-15T22:35:58.921 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.wal_dir: 2024-09-15T22:35:58.921 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-15T22:35:58.921 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-15T22:35:58.921 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-15T22:35:58.921 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-15T22:35:58.921 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-15T22:35:58.921 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-15T22:35:58.921 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.advise_random_on_open: 1 2024-09-15T22:35:58.921 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.db_write_buffer_size: 0 2024-09-15T22:35:58.922 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.write_buffer_manager: 0x565142d8cdc0 2024-09-15T22:35:58.922 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-15T22:35:58.922 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-15T22:35:58.922 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-15T22:35:58.922 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.rate_limiter: (nil) 2024-09-15T22:35:58.922 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-15T22:35:58.922 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.wal_recovery_mode: 2 2024-09-15T22:35:58.922 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.enable_thread_tracking: 0 2024-09-15T22:35:58.922 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.enable_pipelined_write: 0 2024-09-15T22:35:58.922 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.unordered_write: 0 2024-09-15T22:35:58.922 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-15T22:35:58.922 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-15T22:35:58.923 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-15T22:35:58.923 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-15T22:35:58.923 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.row_cache: None 2024-09-15T22:35:58.923 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.wal_filter: None 2024-09-15T22:35:58.923 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-15T22:35:58.923 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.allow_ingest_behind: 0 2024-09-15T22:35:58.923 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.two_write_queues: 0 2024-09-15T22:35:58.923 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.manual_wal_flush: 0 2024-09-15T22:35:58.923 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.wal_compression: 0 2024-09-15T22:35:58.923 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.atomic_flush: 0 2024-09-15T22:35:58.923 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-15T22:35:58.923 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-15T22:35:58.923 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-15T22:35:58.924 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.log_readahead_size: 0 2024-09-15T22:35:58.924 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-15T22:35:58.924 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.best_efforts_recovery: 0 2024-09-15T22:35:58.924 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-15T22:35:58.924 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-15T22:35:58.924 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.allow_data_in_errors: 0 2024-09-15T22:35:58.924 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.db_host_id: __hostname__ 2024-09-15T22:35:58.924 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-15T22:35:58.924 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.max_background_jobs: 2 2024-09-15T22:35:58.924 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.max_background_compactions: -1 2024-09-15T22:35:58.924 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.max_subcompactions: 1 2024-09-15T22:35:58.924 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-15T22:35:58.924 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-15T22:35:58.925 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-15T22:35:58.925 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.max_total_wal_size: 0 2024-09-15T22:35:58.925 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-15T22:35:58.925 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-15T22:35:58.925 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-15T22:35:58.925 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-15T22:35:58.925 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.max_open_files: -1 2024-09-15T22:35:58.925 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.bytes_per_sync: 0 2024-09-15T22:35:58.925 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-15T22:35:58.925 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-15T22:35:58.925 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.compaction_readahead_size: 0 2024-09-15T22:35:58.925 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.max_background_flushes: -1 2024-09-15T22:35:58.926 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Compression algorithms supported: 2024-09-15T22:35:58.926 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: kZSTD supported: 0 2024-09-15T22:35:58.926 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: kXpressCompression supported: 0 2024-09-15T22:35:58.926 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: kBZip2Compression supported: 0 2024-09-15T22:35:58.926 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-15T22:35:58.926 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: kLZ4Compression supported: 1 2024-09-15T22:35:58.926 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: kZlibCompression supported: 1 2024-09-15T22:35:58.926 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: kLZ4HCCompression supported: 1 2024-09-15T22:35:58.926 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: kSnappyCompression supported: 1 2024-09-15T22:35:58.926 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-15T22:35:58.926 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-15T22:35:58.926 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-15T22:35:58.927 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-15T22:35:58.927 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-15T22:35:58.927 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.merge_operator: 2024-09-15T22:35:58.927 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.compaction_filter: None 2024-09-15T22:35:58.927 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.compaction_filter_factory: None 2024-09-15T22:35:58.927 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.sst_partitioner_factory: None 2024-09-15T22:35:58.927 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-15T22:35:58.927 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-15T22:35:58.927 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x565142d7d300) 2024-09-15T22:35:58.927 INFO:journalctl@ceph.mon.a.smithi111.stdout: cache_index_and_filter_blocks: 1 2024-09-15T22:35:58.927 INFO:journalctl@ceph.mon.a.smithi111.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-15T22:35:58.927 INFO:journalctl@ceph.mon.a.smithi111.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-15T22:35:58.927 INFO:journalctl@ceph.mon.a.smithi111.stdout: pin_top_level_index_and_filter: 1 2024-09-15T22:35:58.927 INFO:journalctl@ceph.mon.a.smithi111.stdout: index_type: 0 2024-09-15T22:35:58.928 INFO:journalctl@ceph.mon.a.smithi111.stdout: data_block_index_type: 0 2024-09-15T22:35:58.928 INFO:journalctl@ceph.mon.a.smithi111.stdout: index_shortening: 1 2024-09-15T22:35:58.928 INFO:journalctl@ceph.mon.a.smithi111.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-15T22:35:58.928 INFO:journalctl@ceph.mon.a.smithi111.stdout: checksum: 4 2024-09-15T22:35:58.928 INFO:journalctl@ceph.mon.a.smithi111.stdout: no_block_cache: 0 2024-09-15T22:35:58.928 INFO:journalctl@ceph.mon.a.smithi111.stdout: block_cache: 0x565142d711f0 2024-09-15T22:35:58.928 INFO:journalctl@ceph.mon.a.smithi111.stdout: block_cache_name: BinnedLRUCache 2024-09-15T22:35:58.928 INFO:journalctl@ceph.mon.a.smithi111.stdout: block_cache_options: 2024-09-15T22:35:58.928 INFO:journalctl@ceph.mon.a.smithi111.stdout: capacity : 536870912 2024-09-15T22:35:58.928 INFO:journalctl@ceph.mon.a.smithi111.stdout: num_shard_bits : 4 2024-09-15T22:35:58.928 INFO:journalctl@ceph.mon.a.smithi111.stdout: strict_capacity_limit : 0 2024-09-15T22:35:58.928 INFO:journalctl@ceph.mon.a.smithi111.stdout: high_pri_pool_ratio: 0.000 2024-09-15T22:35:58.929 INFO:journalctl@ceph.mon.a.smithi111.stdout: block_cache_compressed: (nil) 2024-09-15T22:35:58.929 INFO:journalctl@ceph.mon.a.smithi111.stdout: persistent_cache: (nil) 2024-09-15T22:35:58.929 INFO:journalctl@ceph.mon.a.smithi111.stdout: block_size: 4096 2024-09-15T22:35:58.929 INFO:journalctl@ceph.mon.a.smithi111.stdout: block_size_deviation: 10 2024-09-15T22:35:58.929 INFO:journalctl@ceph.mon.a.smithi111.stdout: block_restart_interval: 16 2024-09-15T22:35:58.929 INFO:journalctl@ceph.mon.a.smithi111.stdout: index_block_restart_interval: 1 2024-09-15T22:35:58.929 INFO:journalctl@ceph.mon.a.smithi111.stdout: metadata_block_size: 4096 2024-09-15T22:35:58.929 INFO:journalctl@ceph.mon.a.smithi111.stdout: partition_filters: 0 2024-09-15T22:35:58.929 INFO:journalctl@ceph.mon.a.smithi111.stdout: use_delta_encoding: 1 2024-09-15T22:35:58.929 INFO:journalctl@ceph.mon.a.smithi111.stdout: filter_policy: bloomfilter 2024-09-15T22:35:58.929 INFO:journalctl@ceph.mon.a.smithi111.stdout: whole_key_filtering: 1 2024-09-15T22:35:58.930 INFO:journalctl@ceph.mon.a.smithi111.stdout: verify_compression: 0 2024-09-15T22:35:58.930 INFO:journalctl@ceph.mon.a.smithi111.stdout: read_amp_bytes_per_bit: 0 2024-09-15T22:35:58.930 INFO:journalctl@ceph.mon.a.smithi111.stdout: format_version: 5 2024-09-15T22:35:58.930 INFO:journalctl@ceph.mon.a.smithi111.stdout: enable_index_compression: 1 2024-09-15T22:35:58.930 INFO:journalctl@ceph.mon.a.smithi111.stdout: block_align: 0 2024-09-15T22:35:58.930 INFO:journalctl@ceph.mon.a.smithi111.stdout: max_auto_readahead_size: 262144 2024-09-15T22:35:58.930 INFO:journalctl@ceph.mon.a.smithi111.stdout: prepopulate_block_cache: 0 2024-09-15T22:35:58.930 INFO:journalctl@ceph.mon.a.smithi111.stdout: initial_auto_readahead_size: 8192 2024-09-15T22:35:58.930 INFO:journalctl@ceph.mon.a.smithi111.stdout: num_file_reads_for_auto_readahead: 2 2024-09-15T22:35:58.930 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.write_buffer_size: 33554432 2024-09-15T22:35:58.930 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.max_write_buffer_number: 2 2024-09-15T22:35:58.930 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.compression: NoCompression 2024-09-15T22:35:58.930 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.bottommost_compression: Disabled 2024-09-15T22:35:58.930 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.prefix_extractor: nullptr 2024-09-15T22:35:58.930 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-15T22:35:58.930 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.num_levels: 7 2024-09-15T22:35:58.931 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-15T22:35:58.931 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-15T22:35:58.931 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-15T22:35:58.931 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-15T22:35:58.931 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-15T22:35:58.931 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-15T22:35:58.931 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-15T22:35:58.931 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-15T22:35:58.931 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-15T22:35:58.931 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-15T22:35:58.931 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-15T22:35:58.931 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-15T22:35:58.931 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-15T22:35:58.932 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.compression_opts.level: 32767 2024-09-15T22:35:58.932 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.compression_opts.strategy: 0 2024-09-15T22:35:58.932 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-15T22:35:58.932 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-15T22:35:58.932 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-15T22:35:58.932 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-15T22:35:58.932 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.compression_opts.enabled: false 2024-09-15T22:35:58.932 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-15T22:35:58.932 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-15T22:35:58.932 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-15T22:35:58.932 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-15T22:35:58.932 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.target_file_size_base: 67108864 2024-09-15T22:35:58.932 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-15T22:35:58.933 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-15T22:35:58.933 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-15T22:35:58.933 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-15T22:35:58.933 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-15T22:35:58.933 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-15T22:35:58.933 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-15T22:35:58.933 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-15T22:35:58.933 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-15T22:35:58.933 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-15T22:35:58.933 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-15T22:35:58.933 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-15T22:35:58.933 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-15T22:35:58.934 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-15T22:35:58.934 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.arena_block_size: 1048576 2024-09-15T22:35:58.934 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-15T22:35:58.934 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-15T22:35:58.934 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.disable_auto_compactions: 0 2024-09-15T22:35:58.934 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-15T22:35:58.934 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-15T22:35:58.934 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-15T22:35:58.935 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-15T22:35:58.935 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-15T22:35:58.935 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-15T22:35:58.935 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-15T22:35:58.936 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-15T22:35:58.936 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-15T22:35:58.936 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-15T22:35:58.936 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-15T22:35:58.936 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.inplace_update_support: 0 2024-09-15T22:35:58.936 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-15T22:35:58.936 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-15T22:35:58.936 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-15T22:35:58.936 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-15T22:35:58.936 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.bloom_locality: 0 2024-09-15T22:35:58.936 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.max_successive_merges: 0 2024-09-15T22:35:58.936 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-15T22:35:58.936 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.paranoid_file_checks: 0 2024-09-15T22:35:58.937 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.force_consistency_checks: 1 2024-09-15T22:35:58.937 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.report_bg_io_stats: 0 2024-09-15T22:35:58.937 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.ttl: 2592000 2024-09-15T22:35:58.937 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-15T22:35:58.937 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-15T22:35:58.937 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-15T22:35:58.937 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.enable_blob_files: false 2024-09-15T22:35:58.937 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.min_blob_size: 0 2024-09-15T22:35:58.937 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.blob_file_size: 268435456 2024-09-15T22:35:58.937 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-15T22:35:58.937 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-15T22:35:58.937 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-15T22:35:58.937 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-15T22:35:58.937 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-15T22:35:58.938 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.blob_file_starting_level: 0 2024-09-15T22:35:58.938 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-15T22:35:58.938 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-15T22:35:58.938 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-15T22:35:58.938 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5a62ecac-907a-4529-b4b6-db9eab152df6 2024-09-15T22:35:58.938 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726439758646485, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-15T22:35:58.938 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-15T22:35:58.938 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726439758648140, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101210, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 332, "table_properties": {"data_size": 99095, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14538, "raw_average_key_size": 49, "raw_value_size": 91251, "raw_average_value_size": 313, "num_data_blocks": 12, "num_entries": 291, "num_filter_entries": 291, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726439758, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5a62ecac-907a-4529-b4b6-db9eab152df6", "db_session_id": "79SI5ZPIMX4OH556APTQ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-15T22:35:58.938 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726439758648316, "job": 1, "event": "recovery_finished"} 2024-09-15T22:35:58.938 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-15T22:35:58.938 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-15T22:35:58.938 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x565142da0e00 2024-09-15T22:35:58.938 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: DB pointer 0x565142e9c000 2024-09-15T22:35:58.938 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-15T22:35:58.938 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-15T22:35:58.939 INFO:journalctl@ceph.mon.a.smithi111.stdout: ** DB Stats ** 2024-09-15T22:35:58.939 INFO:journalctl@ceph.mon.a.smithi111.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-15T22:35:58.939 INFO:journalctl@ceph.mon.a.smithi111.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-15T22:35:58.939 INFO:journalctl@ceph.mon.a.smithi111.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-15T22:35:58.939 INFO:journalctl@ceph.mon.a.smithi111.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-15T22:35:58.939 INFO:journalctl@ceph.mon.a.smithi111.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-15T22:35:58.939 INFO:journalctl@ceph.mon.a.smithi111.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-15T22:35:58.939 INFO:journalctl@ceph.mon.a.smithi111.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-15T22:35:58.939 INFO:journalctl@ceph.mon.a.smithi111.stdout: 2024-09-15T22:35:58.939 INFO:journalctl@ceph.mon.a.smithi111.stdout: ** Compaction Stats [default] ** 2024-09-15T22:35:58.939 INFO:journalctl@ceph.mon.a.smithi111.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-15T22:35:58.939 INFO:journalctl@ceph.mon.a.smithi111.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-15T22:35:58.939 INFO:journalctl@ceph.mon.a.smithi111.stdout: L0 2/0 100.70 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 95.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-15T22:35:58.939 INFO:journalctl@ceph.mon.a.smithi111.stdout: Sum 2/0 100.70 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 95.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-15T22:35:58.939 INFO:journalctl@ceph.mon.a.smithi111.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 95.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-15T22:35:58.940 INFO:journalctl@ceph.mon.a.smithi111.stdout: 2024-09-15T22:35:58.940 INFO:journalctl@ceph.mon.a.smithi111.stdout: ** Compaction Stats [default] ** 2024-09-15T22:35:58.940 INFO:journalctl@ceph.mon.a.smithi111.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-15T22:35:58.940 INFO:journalctl@ceph.mon.a.smithi111.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-15T22:35:58.940 INFO:journalctl@ceph.mon.a.smithi111.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 95.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-15T22:35:58.940 INFO:journalctl@ceph.mon.a.smithi111.stdout: 2024-09-15T22:35:58.940 INFO:journalctl@ceph.mon.a.smithi111.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-15T22:35:58.940 INFO:journalctl@ceph.mon.a.smithi111.stdout: 2024-09-15T22:35:58.940 INFO:journalctl@ceph.mon.a.smithi111.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-15T22:35:58.940 INFO:journalctl@ceph.mon.a.smithi111.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-15T22:35:58.940 INFO:journalctl@ceph.mon.a.smithi111.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-15T22:35:58.940 INFO:journalctl@ceph.mon.a.smithi111.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-15T22:35:58.940 INFO:journalctl@ceph.mon.a.smithi111.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-15T22:35:58.940 INFO:journalctl@ceph.mon.a.smithi111.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-15T22:35:58.941 INFO:journalctl@ceph.mon.a.smithi111.stdout: Cumulative compaction: 0.00 GB write, 17.24 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-15T22:35:58.941 INFO:journalctl@ceph.mon.a.smithi111.stdout: Interval compaction: 0.00 GB write, 17.24 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-15T22:35:58.941 INFO:journalctl@ceph.mon.a.smithi111.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-15T22:35:58.941 INFO:journalctl@ceph.mon.a.smithi111.stdout: Block cache BinnedLRUCache@0x565142d711f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-15T22:35:58.941 INFO:journalctl@ceph.mon.a.smithi111.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-15T22:35:58.941 INFO:journalctl@ceph.mon.a.smithi111.stdout: 2024-09-15T22:35:58.941 INFO:journalctl@ceph.mon.a.smithi111.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-15T22:35:58.941 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: starting mon.a rank 0 at public addrs [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] at bind addrs [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T22:35:58.941 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: mon.a@-1(???) e1 preinit fsid b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T22:35:58.941 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: mon.a@-1(???).mds e1 new map 2024-09-15T22:35:58.941 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: mon.a@-1(???).mds e1 print_map 2024-09-15T22:35:58.941 INFO:journalctl@ceph.mon.a.smithi111.stdout: e1 2024-09-15T22:35:58.941 INFO:journalctl@ceph.mon.a.smithi111.stdout: btime 2024-09-15T22:35:52:571222+0000 2024-09-15T22:35:58.941 INFO:journalctl@ceph.mon.a.smithi111.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-15T22:35:58.941 INFO:journalctl@ceph.mon.a.smithi111.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-15T22:35:58.942 INFO:journalctl@ceph.mon.a.smithi111.stdout: legacy client fscid: -1 2024-09-15T22:35:58.942 INFO:journalctl@ceph.mon.a.smithi111.stdout: 2024-09-15T22:35:58.942 INFO:journalctl@ceph.mon.a.smithi111.stdout: No filesystems configured 2024-09-15T22:35:58.942 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-15T22:35:58.942 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T22:35:58.942 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T22:35:58.942 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T22:35:58.942 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-15T22:35:58.942 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-15T22:35:58.942 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: monmap epoch 1 2024-09-15T22:35:58.942 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: fsid b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T22:35:58.942 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: last_changed 2024-09-15T22:35:48.940270+0000 2024-09-15T22:35:58.942 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: created 2024-09-15T22:35:48.940270+0000 2024-09-15T22:35:58.942 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: min_mon_release 19 (squid) 2024-09-15T22:35:58.942 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: election_strategy: 1 2024-09-15T22:35:58.943 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: 0: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.a 2024-09-15T22:35:58.943 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: fsmap 2024-09-15T22:35:58.943 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: osdmap e1: 0 total, 0 up, 0 in 2024-09-15T22:35:58.943 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 ceph-mon[30629]: mgrmap e1: no daemons active 2024-09-15T22:35:58.943 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 bash[30610]: 200de39e07abb43550a97f99fafefa8bc7eb2f15408fdb45d95fab09f14d6c02 2024-09-15T22:35:58.943 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:35:58 smithi111 systemd[1]: Started Ceph mon.a for b7614d3c-73b2-11ef-bceb-c7b262605968. 2024-09-15T22:36:00.357 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.615+0000 7fe86b200640 1 Processor -- start 2024-09-15T22:36:00.358 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.615+0000 7fe86b200640 1 -- start start 2024-09-15T22:36:00.358 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.615+0000 7fe86b200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe864109930 0x7fe864109d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:00.358 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.615+0000 7fe86b200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe86410a300 con 0x7fe864109930 2024-09-15T22:36:00.358 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.615+0000 7fe863e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe864109930 0x7fe864109d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:00.358 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.616+0000 7fe863e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe864109930 0x7fe864109d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54648/0 (socket says 172.21.15.111:54648) 2024-09-15T22:36:00.358 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.616+0000 7fe863e00640 1 -- 172.21.15.111:0/28777377 learned_addr learned my addr 172.21.15.111:0/28777377 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:00.358 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.616+0000 7fe863e00640 1 -- 172.21.15.111:0/28777377 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86410ab20 con 0x7fe864109930 2024-09-15T22:36:00.358 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.619+0000 7fe863e00640 1 --2- 172.21.15.111:0/28777377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe864109930 0x7fe864109d30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe854009820 tx=0x7fe85402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50013f4243bca963 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:00.358 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.619+0000 7fe863400640 1 -- 172.21.15.111:0/28777377 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe854037070 con 0x7fe864109930 2024-09-15T22:36:00.358 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.619+0000 7fe863400640 1 -- 172.21.15.111:0/28777377 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe854032900 con 0x7fe864109930 2024-09-15T22:36:00.359 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.619+0000 7fe863400640 1 -- 172.21.15.111:0/28777377 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe854030850 con 0x7fe864109930 2024-09-15T22:36:00.359 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.619+0000 7fe86b200640 1 -- 172.21.15.111:0/28777377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe864109930 msgr2=0x7fe864109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:00.359 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.619+0000 7fe86b200640 1 --2- 172.21.15.111:0/28777377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe864109930 0x7fe864109d30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe854009820 tx=0x7fe85402a270 comp rx=0 tx=0).stop 2024-09-15T22:36:00.359 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.620+0000 7fe86b200640 1 -- 172.21.15.111:0/28777377 shutdown_connections 2024-09-15T22:36:00.359 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.620+0000 7fe86b200640 1 --2- 172.21.15.111:0/28777377 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe864109930 0x7fe864109d30 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:00.359 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.620+0000 7fe86b200640 1 -- 172.21.15.111:0/28777377 >> 172.21.15.111:0/28777377 conn(0x7fe86407cfa0 msgr2=0x7fe86407d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:00.359 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.620+0000 7fe86b200640 1 -- 172.21.15.111:0/28777377 shutdown_connections 2024-09-15T22:36:00.359 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.620+0000 7fe86b200640 1 -- 172.21.15.111:0/28777377 wait complete. 2024-09-15T22:36:00.359 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.621+0000 7fe86b200640 1 Processor -- start 2024-09-15T22:36:00.360 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.621+0000 7fe86b200640 1 -- start start 2024-09-15T22:36:00.360 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.621+0000 7fe86b200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe864109930 0x7fe86419d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:00.360 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.621+0000 7fe86b200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe86410ded0 con 0x7fe864109930 2024-09-15T22:36:00.360 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.621+0000 7fe863e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe864109930 0x7fe86419d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:00.360 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.621+0000 7fe863e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe864109930 0x7fe86419d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54660/0 (socket says 172.21.15.111:54660) 2024-09-15T22:36:00.360 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.621+0000 7fe863e00640 1 -- 172.21.15.111:0/3573326296 learned_addr learned my addr 172.21.15.111:0/3573326296 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:00.360 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.622+0000 7fe863e00640 1 -- 172.21.15.111:0/3573326296 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86419de80 con 0x7fe864109930 2024-09-15T22:36:00.360 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.622+0000 7fe863e00640 1 --2- 172.21.15.111:0/3573326296 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe864109930 0x7fe86419d940 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe8540097f0 tx=0x7fe854030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:00.360 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.622+0000 7fe861600640 1 -- 172.21.15.111:0/3573326296 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe854037040 con 0x7fe864109930 2024-09-15T22:36:00.360 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.622+0000 7fe861600640 1 -- 172.21.15.111:0/3573326296 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe854040070 con 0x7fe864109930 2024-09-15T22:36:00.360 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.622+0000 7fe861600640 1 -- 172.21.15.111:0/3573326296 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe85403a390 con 0x7fe864109930 2024-09-15T22:36:00.360 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.622+0000 7fe86b200640 1 -- 172.21.15.111:0/3573326296 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe86419e110 con 0x7fe864109930 2024-09-15T22:36:00.361 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.623+0000 7fe86b200640 1 -- 172.21.15.111:0/3573326296 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8641a0e00 con 0x7fe864109930 2024-09-15T22:36:00.361 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.623+0000 7fe861600640 1 -- 172.21.15.111:0/3573326296 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe85403a5e0 con 0x7fe864109930 2024-09-15T22:36:00.361 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.623+0000 7fe861600640 1 -- 172.21.15.111:0/3573326296 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe854045950 con 0x7fe864109930 2024-09-15T22:36:00.361 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.624+0000 7fe86b200640 1 -- 172.21.15.111:0/3573326296 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe864109db0 con 0x7fe864109930 2024-09-15T22:36:00.361 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.627+0000 7fe861600640 1 -- 172.21.15.111:0/3573326296 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fe854045460 con 0x7fe864109930 2024-09-15T22:36:00.361 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.686+0000 7fe86b200640 1 -- 172.21.15.111:0/3573326296 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fe86419e590 con 0x7fe864109930 2024-09-15T22:36:00.361 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.688+0000 7fe861600640 1 -- 172.21.15.111:0/3573326296 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fe85403b430 con 0x7fe864109930 2024-09-15T22:36:00.361 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.689+0000 7fe86b200640 1 -- 172.21.15.111:0/3573326296 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe864109930 msgr2=0x7fe86419d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:00.361 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.689+0000 7fe86b200640 1 --2- 172.21.15.111:0/3573326296 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe864109930 0x7fe86419d940 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe8540097f0 tx=0x7fe854030d80 comp rx=0 tx=0).stop 2024-09-15T22:36:00.361 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.689+0000 7fe86b200640 1 -- 172.21.15.111:0/3573326296 shutdown_connections 2024-09-15T22:36:00.361 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.689+0000 7fe86b200640 1 --2- 172.21.15.111:0/3573326296 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe864109930 0x7fe86419d940 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:00.361 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.689+0000 7fe86b200640 1 -- 172.21.15.111:0/3573326296 >> 172.21.15.111:0/3573326296 conn(0x7fe86407cfa0 msgr2=0x7fe864107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:00.361 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.689+0000 7fe86b200640 1 -- 172.21.15.111:0/3573326296 shutdown_connections 2024-09-15T22:36:00.362 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:35:59.690+0000 7fe86b200640 1 -- 172.21.15.111:0/3573326296 wait complete. 2024-09-15T22:36:00.362 INFO:teuthology.orchestra.run.smithi111.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-15T22:36:00.381 INFO:teuthology.orchestra.run.smithi111.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-15T22:36:00.381 INFO:teuthology.orchestra.run.smithi111.stdout:Creating mgr... 2024-09-15T22:36:00.382 INFO:teuthology.orchestra.run.smithi111.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-15T22:36:00.382 INFO:teuthology.orchestra.run.smithi111.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-15T22:36:00.722 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mgr.x 2024-09-15T22:36:00.723 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-15T22:36:00.963 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b7614d3c-73b2-11ef-bceb-c7b262605968.target.wants/ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@.service. 2024-09-15T22:36:00.996 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:00 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/3573326296' entity='client.admin' 2024-09-15T22:36:01.309 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:00 smithi111 systemd[1]: Starting Ceph mgr.x for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T22:36:01.594 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:01 smithi111 podman[30838]: 2024-09-15 22:36:01.212678054 +0000 UTC m=+0.027138289 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T22:36:01.595 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:01 smithi111 podman[30838]: 2024-09-15 22:36:01.341545061 +0000 UTC m=+0.156005295 container create 4fb6ded66cc6e4cb5312f6d51dacc12b9f4467b257c66856930945b1c5601e37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-09-15T22:36:01.595 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:01 smithi111 podman[30838]: 2024-09-15 22:36:01.59402027 +0000 UTC m=+0.408480504 container init 4fb6ded66cc6e4cb5312f6d51dacc12b9f4467b257c66856930945b1c5601e37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-15T22:36:01.712 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-15T22:36:01.712 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout disabled 2024-09-15T22:36:01.726 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-15T22:36:01.726 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout inactive 2024-09-15T22:36:01.726 INFO:teuthology.orchestra.run.smithi111.stdout:firewalld.service is not enabled 2024-09-15T22:36:01.727 INFO:teuthology.orchestra.run.smithi111.stdout:Not possible to enable service . firewalld.service is not available 2024-09-15T22:36:01.743 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-15T22:36:01.743 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout disabled 2024-09-15T22:36:01.757 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-15T22:36:01.757 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout inactive 2024-09-15T22:36:01.757 INFO:teuthology.orchestra.run.smithi111.stdout:firewalld.service is not enabled 2024-09-15T22:36:01.757 INFO:teuthology.orchestra.run.smithi111.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-15T22:36:01.757 INFO:teuthology.orchestra.run.smithi111.stdout:Waiting for mgr to start... 2024-09-15T22:36:01.758 INFO:teuthology.orchestra.run.smithi111.stdout:Waiting for mgr... 2024-09-15T22:36:01.899 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:01 smithi111 podman[30838]: 2024-09-15 22:36:01.600215219 +0000 UTC m=+0.414675455 container start 4fb6ded66cc6e4cb5312f6d51dacc12b9f4467b257c66856930945b1c5601e37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-15T22:36:01.900 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:01 smithi111 bash[30838]: 4fb6ded66cc6e4cb5312f6d51dacc12b9f4467b257c66856930945b1c5601e37 2024-09-15T22:36:01.900 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:01 smithi111 systemd[1]: Started Ceph mgr.x for b7614d3c-73b2-11ef-bceb-c7b262605968. 2024-09-15T22:36:01.900 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:01 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:01.769+0000 7f2c2da34040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T22:36:01.900 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:01 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:01.841+0000 7f2c2da34040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T22:36:02.164 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:02 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:02.030+0000 7f2c2da34040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T22:36:02.164 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:02 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:02.112+0000 7f2c2da34040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T22:36:02.568 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:02 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:02.294+0000 7f2c2da34040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:36:02.569 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:02 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:02.465+0000 7f2c2da34040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T22:36:02.851 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:02 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/3018757583' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-15T22:36:03.137 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:02 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:02.849+0000 7f2c2da34040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T22:36:03.137 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:02 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:02.924+0000 7f2c2da34040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T22:36:03.138 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:03 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:03.135+0000 7f2c2da34040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T22:36:03.575 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 2024-09-15T22:36:03.578 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout { 2024-09-15T22:36:03.578 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "fsid": "b7614d3c-73b2-11ef-bceb-c7b262605968", 2024-09-15T22:36:03.578 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "health": { 2024-09-15T22:36:03.578 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-15T22:36:03.578 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-15T22:36:03.578 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-15T22:36:03.578 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:36:03.579 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-15T22:36:03.579 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-15T22:36:03.579 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 0 2024-09-15T22:36:03.579 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout ], 2024-09-15T22:36:03.579 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-15T22:36:03.579 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "a" 2024-09-15T22:36:03.579 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout ], 2024-09-15T22:36:03.579 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-15T22:36:03.579 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-15T22:36:03.579 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:36:03.579 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-15T22:36:03.579 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-15T22:36:03.579 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:36:03.579 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-15T22:36:03.580 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:36:03.580 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-15T22:36:03.580 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-15T22:36:03.580 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-15T22:36:03.587 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-15T22:36:03.587 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-15T22:36:03.588 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-15T22:36:03.588 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:36:03.588 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-15T22:36:03.588 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-15T22:36:03.588 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-15T22:36:03.588 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-15T22:36:03.588 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-15T22:36:03.588 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-15T22:36:03.588 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-15T22:36:03.588 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-15T22:36:03.588 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-15T22:36:03.588 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:36:03.588 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-15T22:36:03.588 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:36:03.589 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "btime": "2024-09-15T22:35:52:571222+0000", 2024-09-15T22:36:03.589 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-15T22:36:03.589 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-15T22:36:03.589 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:36:03.589 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-15T22:36:03.589 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-15T22:36:03.589 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-15T22:36:03.589 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-15T22:36:03.589 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-15T22:36:03.589 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-15T22:36:03.589 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "restful" 2024-09-15T22:36:03.589 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout ], 2024-09-15T22:36:03.589 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T22:36:03.589 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:36:03.590 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-15T22:36:03.590 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:36:03.590 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "modified": "2024-09-15T22:35:52.571986+0000", 2024-09-15T22:36:03.590 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T22:36:03.590 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:36:03.590 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-15T22:36:03.590 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout } 2024-09-15T22:36:03.590 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.439+0000 7fbd07000640 1 Processor -- start 2024-09-15T22:36:03.590 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.440+0000 7fbd07000640 1 -- start start 2024-09-15T22:36:03.590 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.440+0000 7fbd07000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd0007ddb0 0x7fbd0007c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:03.590 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.440+0000 7fbd07000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0007c7a0 con 0x7fbd0007ddb0 2024-09-15T22:36:03.590 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.440+0000 7fbd05c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd0007ddb0 0x7fbd0007c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:03.590 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.441+0000 7fbd05c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd0007ddb0 0x7fbd0007c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38560/0 (socket says 172.21.15.111:38560) 2024-09-15T22:36:03.590 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.441+0000 7fbd05c00640 1 -- 172.21.15.111:0/963082589 learned_addr learned my addr 172.21.15.111:0/963082589 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:03.590 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.441+0000 7fbd05c00640 1 -- 172.21.15.111:0/963082589 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0007c920 con 0x7fbd0007ddb0 2024-09-15T22:36:03.591 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.442+0000 7fbd05c00640 1 --2- 172.21.15.111:0/963082589 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd0007ddb0 0x7fbd0007c260 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4009820 tx=0x7fbcf402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3b30dc471ebef7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:03.591 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.442+0000 7fbcffe00640 1 -- 172.21.15.111:0/963082589 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcf4037070 con 0x7fbd0007ddb0 2024-09-15T22:36:03.591 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.442+0000 7fbcffe00640 1 -- 172.21.15.111:0/963082589 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbcf4032900 con 0x7fbd0007ddb0 2024-09-15T22:36:03.591 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.442+0000 7fbcffe00640 1 -- 172.21.15.111:0/963082589 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcf4030850 con 0x7fbd0007ddb0 2024-09-15T22:36:03.591 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.443+0000 7fbd07000640 1 -- 172.21.15.111:0/963082589 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd0007ddb0 msgr2=0x7fbd0007c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:03.591 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.443+0000 7fbd07000640 1 --2- 172.21.15.111:0/963082589 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd0007ddb0 0x7fbd0007c260 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4009820 tx=0x7fbcf402a270 comp rx=0 tx=0).stop 2024-09-15T22:36:03.591 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.443+0000 7fbd07000640 1 -- 172.21.15.111:0/963082589 shutdown_connections 2024-09-15T22:36:03.591 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.443+0000 7fbd07000640 1 --2- 172.21.15.111:0/963082589 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd0007ddb0 0x7fbd0007c260 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:03.591 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.443+0000 7fbd07000640 1 -- 172.21.15.111:0/963082589 >> 172.21.15.111:0/963082589 conn(0x7fbd00102d30 msgr2=0x7fbd001051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:03.591 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.443+0000 7fbd07000640 1 -- 172.21.15.111:0/963082589 shutdown_connections 2024-09-15T22:36:03.591 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.443+0000 7fbd07000640 1 -- 172.21.15.111:0/963082589 wait complete. 2024-09-15T22:36:03.591 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.444+0000 7fbd07000640 1 Processor -- start 2024-09-15T22:36:03.591 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.445+0000 7fbd07000640 1 -- start start 2024-09-15T22:36:03.591 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.445+0000 7fbd07000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd0007ddb0 0x7fbd0019d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:03.592 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.445+0000 7fbd07000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbd0010dbb0 con 0x7fbd0007ddb0 2024-09-15T22:36:03.592 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.445+0000 7fbd05c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd0007ddb0 0x7fbd0019d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:03.592 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.445+0000 7fbd05c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd0007ddb0 0x7fbd0019d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38570/0 (socket says 172.21.15.111:38570) 2024-09-15T22:36:03.592 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.445+0000 7fbd05c00640 1 -- 172.21.15.111:0/3018757583 learned_addr learned my addr 172.21.15.111:0/3018757583 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:03.592 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.446+0000 7fbd05c00640 1 -- 172.21.15.111:0/3018757583 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd0019dce0 con 0x7fbd0007ddb0 2024-09-15T22:36:03.592 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.446+0000 7fbd05c00640 1 --2- 172.21.15.111:0/3018757583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd0007ddb0 0x7fbd0019d7a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40097f0 tx=0x7fbcf4030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:03.592 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.446+0000 7fbcfe000640 1 -- 172.21.15.111:0/3018757583 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcf4042070 con 0x7fbd0007ddb0 2024-09-15T22:36:03.592 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.446+0000 7fbcfe000640 1 -- 172.21.15.111:0/3018757583 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbcf4032ca0 con 0x7fbd0007ddb0 2024-09-15T22:36:03.592 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.446+0000 7fbd07000640 1 -- 172.21.15.111:0/3018757583 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd0019df70 con 0x7fbd0007ddb0 2024-09-15T22:36:03.592 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.446+0000 7fbcfe000640 1 -- 172.21.15.111:0/3018757583 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcf4037040 con 0x7fbd0007ddb0 2024-09-15T22:36:03.592 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.446+0000 7fbd07000640 1 -- 172.21.15.111:0/3018757583 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd001a0c60 con 0x7fbd0007ddb0 2024-09-15T22:36:03.592 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.447+0000 7fbcfe000640 1 -- 172.21.15.111:0/3018757583 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbcf4045070 con 0x7fbd0007ddb0 2024-09-15T22:36:03.592 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.447+0000 7fbcfe000640 1 -- 172.21.15.111:0/3018757583 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbcf4047a90 con 0x7fbd0007ddb0 2024-09-15T22:36:03.592 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.447+0000 7fbd07000640 1 -- 172.21.15.111:0/3018757583 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd0007c2e0 con 0x7fbd0007ddb0 2024-09-15T22:36:03.592 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.450+0000 7fbcfe000640 1 -- 172.21.15.111:0/3018757583 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fbcf403db10 con 0x7fbd0007ddb0 2024-09-15T22:36:03.593 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.515+0000 7fbd07000640 1 -- 172.21.15.111:0/3018757583 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbd001a11d0 con 0x7fbd0007ddb0 2024-09-15T22:36:03.593 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.516+0000 7fbcfe000640 1 -- 172.21.15.111:0/3018757583 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fbcf403d430 con 0x7fbd0007ddb0 2024-09-15T22:36:03.593 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.517+0000 7fbd07000640 1 -- 172.21.15.111:0/3018757583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd0007ddb0 msgr2=0x7fbd0019d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:03.593 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.517+0000 7fbd07000640 1 --2- 172.21.15.111:0/3018757583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd0007ddb0 0x7fbd0019d7a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40097f0 tx=0x7fbcf4030ec0 comp rx=0 tx=0).stop 2024-09-15T22:36:03.593 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.517+0000 7fbd07000640 1 -- 172.21.15.111:0/3018757583 shutdown_connections 2024-09-15T22:36:03.593 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.517+0000 7fbd07000640 1 --2- 172.21.15.111:0/3018757583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd0007ddb0 0x7fbd0019d7a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:03.593 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.517+0000 7fbd07000640 1 -- 172.21.15.111:0/3018757583 >> 172.21.15.111:0/3018757583 conn(0x7fbd00102d30 msgr2=0x7fbd001039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:03.593 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.517+0000 7fbd07000640 1 -- 172.21.15.111:0/3018757583 shutdown_connections 2024-09-15T22:36:03.593 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:02.517+0000 7fbd07000640 1 -- 172.21.15.111:0/3018757583 wait complete. 2024-09-15T22:36:03.593 INFO:teuthology.orchestra.run.smithi111.stdout:mgr not available, waiting (1/15)... 2024-09-15T22:36:04.666 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:04 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:04.233+0000 7f2c2da34040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T22:36:04.666 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:04 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:04.313+0000 7f2c2da34040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T22:36:05.573 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:05 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:05.255+0000 7f2c2da34040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T22:36:05.574 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:05 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:05.394+0000 7f2c2da34040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T22:36:05.574 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:05 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:05.572+0000 7f2c2da34040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:36:05.859 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:05 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:05.656+0000 7f2c2da34040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T22:36:05.859 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:05 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:05.843+0000 7f2c2da34040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T22:36:06.165 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:05 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:05.939+0000 7f2c2da34040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T22:36:06.165 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:06 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:06.020+0000 7f2c2da34040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T22:36:06.518 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:06 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:06.200+0000 7f2c2da34040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T22:36:06.518 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:06 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:06.275+0000 7f2c2da34040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T22:36:06.518 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:06 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:06.352+0000 7f2c2da34040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T22:36:06.518 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:06 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:06.447+0000 7f2c2da34040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T22:36:06.518 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:06 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:06.516+0000 7f2c2da34040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T22:36:06.915 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:06 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:06.585+0000 7f2c2da34040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T22:36:07.916 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:07 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:07.504+0000 7f2c2da34040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T22:36:07.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:07 smithi111 ceph-mon[30629]: Activating manager daemon x 2024-09-15T22:36:07.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:07 smithi111 ceph-mon[30629]: mgrmap e2: x(active, starting, since 0.00432879s) 2024-09-15T22:36:07.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:07 smithi111 ceph-mon[30629]: from='mgr.14100 172.21.15.111:0/709251733' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-15T22:36:07.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:07 smithi111 ceph-mon[30629]: from='mgr.14100 172.21.15.111:0/709251733' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-15T22:36:07.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:07 smithi111 ceph-mon[30629]: from='mgr.14100 172.21.15.111:0/709251733' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-15T22:36:07.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:07 smithi111 ceph-mon[30629]: from='mgr.14100 172.21.15.111:0/709251733' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:36:07.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:07 smithi111 ceph-mon[30629]: from='mgr.14100 172.21.15.111:0/709251733' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-15T22:36:07.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:07 smithi111 ceph-mon[30629]: Manager daemon x is now available 2024-09-15T22:36:07.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:07 smithi111 ceph-mon[30629]: from='mgr.14100 172.21.15.111:0/709251733' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-15T22:36:07.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:07 smithi111 ceph-mon[30629]: from='mgr.14100 172.21.15.111:0/709251733' entity='mgr.x' 2024-09-15T22:36:07.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:07 smithi111 ceph-mon[30629]: from='mgr.14100 172.21.15.111:0/709251733' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-15T22:36:07.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:07 smithi111 ceph-mon[30629]: from='mgr.14100 172.21.15.111:0/709251733' entity='mgr.x' 2024-09-15T22:36:07.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:07 smithi111 ceph-mon[30629]: from='mgr.14100 172.21.15.111:0/709251733' entity='mgr.x' 2024-09-15T22:36:09.854 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:09 smithi111 ceph-mon[30629]: mgrmap e3: x(active, since 1.01032s) 2024-09-15T22:36:10.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:10 smithi111 ceph-mon[30629]: mgrmap e4: x(active, since 2s) 2024-09-15T22:36:14.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:13 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/3246185212' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-15T22:36:14.586 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 2024-09-15T22:36:14.586 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout { 2024-09-15T22:36:14.586 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "fsid": "b7614d3c-73b2-11ef-bceb-c7b262605968", 2024-09-15T22:36:14.586 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "health": { 2024-09-15T22:36:14.586 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-15T22:36:14.586 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-15T22:36:14.586 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-15T22:36:14.586 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:36:14.586 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-15T22:36:14.586 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-15T22:36:14.587 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 0 2024-09-15T22:36:14.587 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout ], 2024-09-15T22:36:14.587 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-15T22:36:14.587 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "a" 2024-09-15T22:36:14.587 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout ], 2024-09-15T22:36:14.587 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-09-15T22:36:14.587 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-15T22:36:14.587 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:36:14.587 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-15T22:36:14.587 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-15T22:36:14.587 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:36:14.588 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-15T22:36:14.588 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:36:14.588 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-15T22:36:14.588 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-15T22:36:14.588 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-15T22:36:14.596 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-15T22:36:14.596 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-15T22:36:14.596 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-15T22:36:14.596 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:36:14.596 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-15T22:36:14.597 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-15T22:36:14.597 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-15T22:36:14.597 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-15T22:36:14.597 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-15T22:36:14.597 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-15T22:36:14.597 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-15T22:36:14.597 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-15T22:36:14.597 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-15T22:36:14.597 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:36:14.597 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-15T22:36:14.597 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:36:14.597 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "btime": "2024-09-15T22:35:52:571222+0000", 2024-09-15T22:36:14.597 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-15T22:36:14.597 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-15T22:36:14.598 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:36:14.598 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-15T22:36:14.598 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-15T22:36:14.598 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-15T22:36:14.598 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-15T22:36:14.598 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-15T22:36:14.598 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-15T22:36:14.598 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "restful" 2024-09-15T22:36:14.598 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout ], 2024-09-15T22:36:14.598 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T22:36:14.598 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:36:14.598 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-15T22:36:14.598 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T22:36:14.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "modified": "2024-09-15T22:35:52.571986+0000", 2024-09-15T22:36:14.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T22:36:14.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout }, 2024-09-15T22:36:14.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-15T22:36:14.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout } 2024-09-15T22:36:14.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.586+0000 7f595c400640 1 Processor -- start 2024-09-15T22:36:14.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.586+0000 7f595c400640 1 -- start start 2024-09-15T22:36:14.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.587+0000 7f595c400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5954107700 0x7f5954107b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:14.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.587+0000 7f595c400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f59541080d0 con 0x7f5954107700 2024-09-15T22:36:14.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.587+0000 7f5959a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5954107700 0x7f5954107b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:14.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.587+0000 7f5959a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5954107700 0x7f5954107b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48580/0 (socket says 172.21.15.111:48580) 2024-09-15T22:36:14.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.587+0000 7f5959a00640 1 -- 172.21.15.111:0/928474289 learned_addr learned my addr 172.21.15.111:0/928474289 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:14.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.587+0000 7f5959a00640 1 -- 172.21.15.111:0/928474289 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5954108900 con 0x7f5954107700 2024-09-15T22:36:14.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.588+0000 7f5959a00640 1 --2- 172.21.15.111:0/928474289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5954107700 0x7f5954107b00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5948009820 tx=0x7f594802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d106afc6db158d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:14.599 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.589+0000 7f5953e00640 1 -- 172.21.15.111:0/928474289 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5948037070 con 0x7f5954107700 2024-09-15T22:36:14.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.589+0000 7f5953e00640 1 -- 172.21.15.111:0/928474289 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5948032900 con 0x7f5954107700 2024-09-15T22:36:14.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.589+0000 7f5953e00640 1 -- 172.21.15.111:0/928474289 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5948030850 con 0x7f5954107700 2024-09-15T22:36:14.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.589+0000 7f595c400640 1 -- 172.21.15.111:0/928474289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5954107700 msgr2=0x7f5954107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:14.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.589+0000 7f595c400640 1 --2- 172.21.15.111:0/928474289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5954107700 0x7f5954107b00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5948009820 tx=0x7f594802a270 comp rx=0 tx=0).stop 2024-09-15T22:36:14.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.589+0000 7f595c400640 1 -- 172.21.15.111:0/928474289 shutdown_connections 2024-09-15T22:36:14.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.589+0000 7f595c400640 1 --2- 172.21.15.111:0/928474289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5954107700 0x7f5954107b00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.589+0000 7f595c400640 1 -- 172.21.15.111:0/928474289 >> 172.21.15.111:0/928474289 conn(0x7f5954102f30 msgr2=0x7f5954105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:14.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.589+0000 7f595c400640 1 -- 172.21.15.111:0/928474289 shutdown_connections 2024-09-15T22:36:14.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.590+0000 7f595c400640 1 -- 172.21.15.111:0/928474289 wait complete. 2024-09-15T22:36:14.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.590+0000 7f595c400640 1 Processor -- start 2024-09-15T22:36:14.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.591+0000 7f595c400640 1 -- start start 2024-09-15T22:36:14.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.591+0000 7f595c400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5954107700 0x7f595419d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:14.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.591+0000 7f595c400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f595410bcb0 con 0x7f5954107700 2024-09-15T22:36:14.600 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.591+0000 7f5959a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5954107700 0x7f595419d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:14.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.591+0000 7f5959a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5954107700 0x7f595419d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48590/0 (socket says 172.21.15.111:48590) 2024-09-15T22:36:14.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.591+0000 7f5959a00640 1 -- 172.21.15.111:0/3246185212 learned_addr learned my addr 172.21.15.111:0/3246185212 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:14.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.591+0000 7f5959a00640 1 -- 172.21.15.111:0/3246185212 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595419de70 con 0x7f5954107700 2024-09-15T22:36:14.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.592+0000 7f5959a00640 1 --2- 172.21.15.111:0/3246185212 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5954107700 0x7f595419d930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f59480097f0 tx=0x7f5948030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:14.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.592+0000 7f5952000640 1 -- 172.21.15.111:0/3246185212 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5948037040 con 0x7f5954107700 2024-09-15T22:36:14.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.592+0000 7f5952000640 1 -- 172.21.15.111:0/3246185212 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5948040070 con 0x7f5954107700 2024-09-15T22:36:14.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.592+0000 7f595c400640 1 -- 172.21.15.111:0/3246185212 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f595419e100 con 0x7f5954107700 2024-09-15T22:36:14.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.592+0000 7f5952000640 1 -- 172.21.15.111:0/3246185212 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f594803a390 con 0x7f5954107700 2024-09-15T22:36:14.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.592+0000 7f595c400640 1 -- 172.21.15.111:0/3246185212 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59541a0df0 con 0x7f5954107700 2024-09-15T22:36:14.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.593+0000 7f5952000640 1 -- 172.21.15.111:0/3246185212 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f5948032900 con 0x7f5954107700 2024-09-15T22:36:14.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.593+0000 7f595c400640 1 -- 172.21.15.111:0/3246185212 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5954107b80 con 0x7f5954107700 2024-09-15T22:36:14.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.594+0000 7f5952000640 1 --2- 172.21.15.111:0/3246185212 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7f593c0391b0 0x7f593c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:14.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.594+0000 7f5952000640 1 -- 172.21.15.111:0/3246185212 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f594806c360 con 0x7f5954107700 2024-09-15T22:36:14.601 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.594+0000 7f5959000640 1 --2- 172.21.15.111:0/3246185212 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7f593c0391b0 0x7f593c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:14.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.595+0000 7f5959000640 1 --2- 172.21.15.111:0/3246185212 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7f593c0391b0 0x7f593c03b670 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f59400098c0 tx=0x7f5940006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:14.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.600+0000 7f5952000640 1 -- 172.21.15.111:0/3246185212 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f594806c0c0 con 0x7f5954107700 2024-09-15T22:36:14.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.836+0000 7f595c400640 1 -- 172.21.15.111:0/3246185212 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f595419e580 con 0x7f5954107700 2024-09-15T22:36:14.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.837+0000 7f5952000640 1 -- 172.21.15.111:0/3246185212 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f5948032bf0 con 0x7f5954107700 2024-09-15T22:36:14.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.840+0000 7f595c400640 1 -- 172.21.15.111:0/3246185212 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7f593c0391b0 msgr2=0x7f593c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:14.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.840+0000 7f595c400640 1 --2- 172.21.15.111:0/3246185212 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7f593c0391b0 0x7f593c03b670 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f59400098c0 tx=0x7f5940006db0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.840+0000 7f595c400640 1 -- 172.21.15.111:0/3246185212 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5954107700 msgr2=0x7f595419d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:14.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.840+0000 7f595c400640 1 --2- 172.21.15.111:0/3246185212 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5954107700 0x7f595419d930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f59480097f0 tx=0x7f5948030d80 comp rx=0 tx=0).stop 2024-09-15T22:36:14.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.840+0000 7f595c400640 1 -- 172.21.15.111:0/3246185212 shutdown_connections 2024-09-15T22:36:14.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.840+0000 7f595c400640 1 --2- 172.21.15.111:0/3246185212 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7f593c0391b0 0x7f593c03b670 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.840+0000 7f595c400640 1 --2- 172.21.15.111:0/3246185212 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5954107700 0x7f595419d930 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:14.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.840+0000 7f595c400640 1 -- 172.21.15.111:0/3246185212 >> 172.21.15.111:0/3246185212 conn(0x7f5954102f30 msgr2=0x7f5954103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:14.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.840+0000 7f595c400640 1 -- 172.21.15.111:0/3246185212 shutdown_connections 2024-09-15T22:36:14.602 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:13.840+0000 7f595c400640 1 -- 172.21.15.111:0/3246185212 wait complete. 2024-09-15T22:36:14.602 INFO:teuthology.orchestra.run.smithi111.stdout:mgr is available 2024-09-15T22:36:15.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:15 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/2734023315' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-15T22:36:16.031 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 2024-09-15T22:36:16.031 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout [global] 2024-09-15T22:36:16.031 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout fsid = b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T22:36:16.031 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.111:3300,v1:172.21.15.111:6789] 2024-09-15T22:36:16.031 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-15T22:36:16.031 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-15T22:36:16.032 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-15T22:36:16.032 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-15T22:36:16.032 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 2024-09-15T22:36:16.032 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-15T22:36:16.032 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-15T22:36:16.032 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 2024-09-15T22:36:16.032 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout [osd] 2024-09-15T22:36:16.032 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-15T22:36:16.032 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-15T22:36:16.032 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-15T22:36:16.032 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-15T22:36:16.032 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-15T22:36:16.032 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.141+0000 7fc4f0e00640 1 Processor -- start 2024-09-15T22:36:16.033 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.141+0000 7fc4f0e00640 1 -- start start 2024-09-15T22:36:16.033 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.142+0000 7fc4f0e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4ec109910 0x7fc4ec109d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:16.033 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.142+0000 7fc4f0e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec10a2e0 con 0x7fc4ec109910 2024-09-15T22:36:16.033 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.142+0000 7fc4e9e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4ec109910 0x7fc4ec109d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:16.033 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.142+0000 7fc4e9e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4ec109910 0x7fc4ec109d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48606/0 (socket says 172.21.15.111:48606) 2024-09-15T22:36:16.033 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.142+0000 7fc4e9e00640 1 -- 172.21.15.111:0/1968048563 learned_addr learned my addr 172.21.15.111:0/1968048563 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:16.033 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.143+0000 7fc4e9e00640 1 -- 172.21.15.111:0/1968048563 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ec10ab10 con 0x7fc4ec109910 2024-09-15T22:36:16.033 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.144+0000 7fc4e9e00640 1 --2- 172.21.15.111:0/1968048563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4ec109910 0x7fc4ec109d10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc009820 tx=0x7fc4dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b474bae66564c06c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:16.033 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.144+0000 7fc4e8a00640 1 -- 172.21.15.111:0/1968048563 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4dc037070 con 0x7fc4ec109910 2024-09-15T22:36:16.033 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.144+0000 7fc4e8a00640 1 -- 172.21.15.111:0/1968048563 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc4dc032900 con 0x7fc4ec109910 2024-09-15T22:36:16.033 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.144+0000 7fc4e8a00640 1 -- 172.21.15.111:0/1968048563 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4dc030850 con 0x7fc4ec109910 2024-09-15T22:36:16.033 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.144+0000 7fc4f0e00640 1 -- 172.21.15.111:0/1968048563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4ec109910 msgr2=0x7fc4ec109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:16.034 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.144+0000 7fc4f0e00640 1 --2- 172.21.15.111:0/1968048563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4ec109910 0x7fc4ec109d10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc009820 tx=0x7fc4dc02a270 comp rx=0 tx=0).stop 2024-09-15T22:36:16.034 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.145+0000 7fc4f0e00640 1 -- 172.21.15.111:0/1968048563 shutdown_connections 2024-09-15T22:36:16.034 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.145+0000 7fc4f0e00640 1 --2- 172.21.15.111:0/1968048563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4ec109910 0x7fc4ec109d10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.034 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.145+0000 7fc4f0e00640 1 -- 172.21.15.111:0/1968048563 >> 172.21.15.111:0/1968048563 conn(0x7fc4ec07cef0 msgr2=0x7fc4ec07d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:16.034 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.145+0000 7fc4f0e00640 1 -- 172.21.15.111:0/1968048563 shutdown_connections 2024-09-15T22:36:16.034 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.145+0000 7fc4f0e00640 1 -- 172.21.15.111:0/1968048563 wait complete. 2024-09-15T22:36:16.034 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.145+0000 7fc4f0e00640 1 Processor -- start 2024-09-15T22:36:16.034 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.146+0000 7fc4f0e00640 1 -- start start 2024-09-15T22:36:16.034 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.146+0000 7fc4f0e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4ec109910 0x7fc4ec19d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:16.034 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.146+0000 7fc4f0e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec10dec0 con 0x7fc4ec109910 2024-09-15T22:36:16.034 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.146+0000 7fc4e9e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4ec109910 0x7fc4ec19d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:16.034 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.146+0000 7fc4e9e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4ec109910 0x7fc4ec19d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48622/0 (socket says 172.21.15.111:48622) 2024-09-15T22:36:16.034 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.146+0000 7fc4e9e00640 1 -- 172.21.15.111:0/2734023315 learned_addr learned my addr 172.21.15.111:0/2734023315 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:16.035 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.146+0000 7fc4e9e00640 1 -- 172.21.15.111:0/2734023315 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ec19de50 con 0x7fc4ec109910 2024-09-15T22:36:16.035 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.147+0000 7fc4e9e00640 1 --2- 172.21.15.111:0/2734023315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4ec109910 0x7fc4ec19d910 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc009940 tx=0x7fc4dc032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:16.035 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.147+0000 7fc4e2a00640 1 -- 172.21.15.111:0/2734023315 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4dc042070 con 0x7fc4ec109910 2024-09-15T22:36:16.035 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.147+0000 7fc4e2a00640 1 -- 172.21.15.111:0/2734023315 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc4dc0308c0 con 0x7fc4ec109910 2024-09-15T22:36:16.035 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.147+0000 7fc4e2a00640 1 -- 172.21.15.111:0/2734023315 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4dc037040 con 0x7fc4ec109910 2024-09-15T22:36:16.035 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.147+0000 7fc4f0e00640 1 -- 172.21.15.111:0/2734023315 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4ec19e0e0 con 0x7fc4ec109910 2024-09-15T22:36:16.035 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.147+0000 7fc4f0e00640 1 -- 172.21.15.111:0/2734023315 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4ec1a0dd0 con 0x7fc4ec109910 2024-09-15T22:36:16.035 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.148+0000 7fc4e2a00640 1 -- 172.21.15.111:0/2734023315 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7fc4dc045070 con 0x7fc4ec109910 2024-09-15T22:36:16.035 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.148+0000 7fc4f0e00640 1 -- 172.21.15.111:0/2734023315 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4ec10e040 con 0x7fc4ec109910 2024-09-15T22:36:16.035 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.148+0000 7fc4e2a00640 1 --2- 172.21.15.111:0/2734023315 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7fc4d00391b0 0x7fc4d003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:16.035 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.148+0000 7fc4e2a00640 1 -- 172.21.15.111:0/2734023315 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc4dc06c2a0 con 0x7fc4ec109910 2024-09-15T22:36:16.035 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.149+0000 7fc4e9400640 1 --2- 172.21.15.111:0/2734023315 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7fc4d00391b0 0x7fc4d003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:16.035 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.149+0000 7fc4e9400640 1 --2- 172.21.15.111:0/2734023315 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7fc4d00391b0 0x7fc4d003b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80098c0 tx=0x7fc4d8006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:16.036 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.153+0000 7fc4e2a00640 1 -- 172.21.15.111:0/2734023315 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc4dc04b170 con 0x7fc4ec109910 2024-09-15T22:36:16.036 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.326+0000 7fc4f0e00640 1 -- 172.21.15.111:0/2734023315 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fc4ec109d90 con 0x7fc4ec109910 2024-09-15T22:36:16.036 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.327+0000 7fc4e2a00640 1 -- 172.21.15.111:0/2734023315 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+431 (secure 0 0 0) 0x7fc4dc06bcc0 con 0x7fc4ec109910 2024-09-15T22:36:16.036 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.330+0000 7fc4f0e00640 1 -- 172.21.15.111:0/2734023315 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7fc4d00391b0 msgr2=0x7fc4d003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:16.036 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.330+0000 7fc4f0e00640 1 --2- 172.21.15.111:0/2734023315 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7fc4d00391b0 0x7fc4d003b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80098c0 tx=0x7fc4d8006db0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.036 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.330+0000 7fc4f0e00640 1 -- 172.21.15.111:0/2734023315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4ec109910 msgr2=0x7fc4ec19d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:16.036 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.330+0000 7fc4f0e00640 1 --2- 172.21.15.111:0/2734023315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4ec109910 0x7fc4ec19d910 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc009940 tx=0x7fc4dc032ce0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.036 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.330+0000 7fc4f0e00640 1 -- 172.21.15.111:0/2734023315 shutdown_connections 2024-09-15T22:36:16.036 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.330+0000 7fc4f0e00640 1 --2- 172.21.15.111:0/2734023315 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7fc4d00391b0 0x7fc4d003b670 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.036 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.330+0000 7fc4f0e00640 1 --2- 172.21.15.111:0/2734023315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4ec109910 0x7fc4ec19d910 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:16.036 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.330+0000 7fc4f0e00640 1 -- 172.21.15.111:0/2734023315 >> 172.21.15.111:0/2734023315 conn(0x7fc4ec07cef0 msgr2=0x7fc4ec107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:16.036 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.330+0000 7fc4f0e00640 1 -- 172.21.15.111:0/2734023315 shutdown_connections 2024-09-15T22:36:16.036 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:15.330+0000 7fc4f0e00640 1 -- 172.21.15.111:0/2734023315 wait complete. 2024-09-15T22:36:16.037 INFO:teuthology.orchestra.run.smithi111.stdout:Enabling cephadm module... 2024-09-15T22:36:17.098 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:16 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/1634022469' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-15T22:36:17.098 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:16 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: ignoring --setuser ceph since I am not root 2024-09-15T22:36:17.098 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:16 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: ignoring --setgroup ceph since I am not root 2024-09-15T22:36:17.098 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:17 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:17.019+0000 7f0ade836040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T22:36:17.098 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:17 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:17.096+0000 7f0ade836040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T22:36:17.364 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:17 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:17.281+0000 7f0ade836040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T22:36:17.364 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:17 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:17.362+0000 7f0ade836040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T22:36:17.526 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.517+0000 7fd54c000640 1 Processor -- start 2024-09-15T22:36:17.527 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.518+0000 7fd54c000640 1 -- start start 2024-09-15T22:36:17.527 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.518+0000 7fd54c000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd544081860 0x7fd544081c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:17.527 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.518+0000 7fd54c000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd544082230 con 0x7fd544081860 2024-09-15T22:36:17.527 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.518+0000 7fd549600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd544081860 0x7fd544081c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:17.527 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.518+0000 7fd549600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd544081860 0x7fd544081c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48630/0 (socket says 172.21.15.111:48630) 2024-09-15T22:36:17.527 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.518+0000 7fd549600640 1 -- 172.21.15.111:0/1314764394 learned_addr learned my addr 172.21.15.111:0/1314764394 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:17.527 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.518+0000 7fd549600640 1 -- 172.21.15.111:0/1314764394 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54410efa0 con 0x7fd544081860 2024-09-15T22:36:17.527 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.519+0000 7fd549600640 1 --2- 172.21.15.111:0/1314764394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd544081860 0x7fd544081c60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd538009820 tx=0x7fd53802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb890c4936eaabd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:17.528 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.520+0000 7fd543e00640 1 -- 172.21.15.111:0/1314764394 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd538037070 con 0x7fd544081860 2024-09-15T22:36:17.528 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.520+0000 7fd543e00640 1 -- 172.21.15.111:0/1314764394 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd538032900 con 0x7fd544081860 2024-09-15T22:36:17.528 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.520+0000 7fd543e00640 1 -- 172.21.15.111:0/1314764394 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd538030850 con 0x7fd544081860 2024-09-15T22:36:17.528 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.520+0000 7fd54c000640 1 -- 172.21.15.111:0/1314764394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd544081860 msgr2=0x7fd544081c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:17.528 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.520+0000 7fd54c000640 1 --2- 172.21.15.111:0/1314764394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd544081860 0x7fd544081c60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd538009820 tx=0x7fd53802a270 comp rx=0 tx=0).stop 2024-09-15T22:36:17.528 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.520+0000 7fd54c000640 1 -- 172.21.15.111:0/1314764394 shutdown_connections 2024-09-15T22:36:17.528 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.520+0000 7fd54c000640 1 --2- 172.21.15.111:0/1314764394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd544081860 0x7fd544081c60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:17.528 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.520+0000 7fd54c000640 1 -- 172.21.15.111:0/1314764394 >> 172.21.15.111:0/1314764394 conn(0x7fd54407cfa0 msgr2=0x7fd54407f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:17.528 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.520+0000 7fd54c000640 1 -- 172.21.15.111:0/1314764394 shutdown_connections 2024-09-15T22:36:17.528 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.521+0000 7fd54c000640 1 -- 172.21.15.111:0/1314764394 wait complete. 2024-09-15T22:36:17.528 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.521+0000 7fd54c000640 1 Processor -- start 2024-09-15T22:36:17.529 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.521+0000 7fd54c000640 1 -- start start 2024-09-15T22:36:17.529 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.522+0000 7fd54c000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd544081860 0x7fd5441a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:17.529 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.522+0000 7fd54c000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd544112350 con 0x7fd544081860 2024-09-15T22:36:17.529 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.522+0000 7fd549600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd544081860 0x7fd5441a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:17.529 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.522+0000 7fd549600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd544081860 0x7fd5441a1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48642/0 (socket says 172.21.15.111:48642) 2024-09-15T22:36:17.529 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.522+0000 7fd549600640 1 -- 172.21.15.111:0/1634022469 learned_addr learned my addr 172.21.15.111:0/1634022469 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:17.529 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.522+0000 7fd549600640 1 -- 172.21.15.111:0/1634022469 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5441a2270 con 0x7fd544081860 2024-09-15T22:36:17.529 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.522+0000 7fd549600640 1 --2- 172.21.15.111:0/1634022469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd544081860 0x7fd5441a1d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd5380097f0 tx=0x7fd538030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:17.529 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.523+0000 7fd542000640 1 -- 172.21.15.111:0/1634022469 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd538037040 con 0x7fd544081860 2024-09-15T22:36:17.529 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.523+0000 7fd542000640 1 -- 172.21.15.111:0/1634022469 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd538040070 con 0x7fd544081860 2024-09-15T22:36:17.529 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.523+0000 7fd542000640 1 -- 172.21.15.111:0/1634022469 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd53803a390 con 0x7fd544081860 2024-09-15T22:36:17.529 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.523+0000 7fd54c000640 1 -- 172.21.15.111:0/1634022469 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5441a2500 con 0x7fd544081860 2024-09-15T22:36:17.530 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.523+0000 7fd54c000640 1 -- 172.21.15.111:0/1634022469 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5441a51f0 con 0x7fd544081860 2024-09-15T22:36:17.530 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.524+0000 7fd542000640 1 -- 172.21.15.111:0/1634022469 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7fd53803b430 con 0x7fd544081860 2024-09-15T22:36:17.530 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.524+0000 7fd54c000640 1 -- 172.21.15.111:0/1634022469 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd544081ce0 con 0x7fd544081860 2024-09-15T22:36:17.530 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.525+0000 7fd542000640 1 --2- 172.21.15.111:0/1634022469 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7fd52c0391b0 0x7fd52c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:17.530 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.525+0000 7fd542000640 1 -- 172.21.15.111:0/1634022469 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd53806c3e0 con 0x7fd544081860 2024-09-15T22:36:17.530 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.525+0000 7fd548c00640 1 --2- 172.21.15.111:0/1634022469 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7fd52c0391b0 0x7fd52c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:17.530 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.525+0000 7fd548c00640 1 --2- 172.21.15.111:0/1634022469 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7fd52c0391b0 0x7fd52c03b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd5300098c0 tx=0x7fd530006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:17.530 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.529+0000 7fd542000640 1 -- 172.21.15.111:0/1634022469 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd5380371e0 con 0x7fd544081860 2024-09-15T22:36:17.530 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.751+0000 7fd54c000640 1 -- 172.21.15.111:0/1634022469 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fd5441a2980 con 0x7fd544081860 2024-09-15T22:36:17.530 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.803+0000 7fd542000640 1 -- 172.21.15.111:0/1634022469 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fd538032dd0 con 0x7fd544081860 2024-09-15T22:36:17.530 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.804+0000 7fd542000640 1 -- 172.21.15.111:0/1634022469 <== mon.0 v2:172.21.15.111:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7fd538032900 con 0x7fd544081860 2024-09-15T22:36:17.531 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.806+0000 7fd54c000640 1 -- 172.21.15.111:0/1634022469 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7fd52c0391b0 msgr2=0x7fd52c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:17.531 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.806+0000 7fd54c000640 1 --2- 172.21.15.111:0/1634022469 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7fd52c0391b0 0x7fd52c03b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd5300098c0 tx=0x7fd530006db0 comp rx=0 tx=0).stop 2024-09-15T22:36:17.531 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.806+0000 7fd54c000640 1 -- 172.21.15.111:0/1634022469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd544081860 msgr2=0x7fd5441a1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:17.531 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.806+0000 7fd54c000640 1 --2- 172.21.15.111:0/1634022469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd544081860 0x7fd5441a1d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd5380097f0 tx=0x7fd538030d80 comp rx=0 tx=0).stop 2024-09-15T22:36:17.531 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.806+0000 7fd54c000640 1 -- 172.21.15.111:0/1634022469 shutdown_connections 2024-09-15T22:36:17.531 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.806+0000 7fd54c000640 1 --2- 172.21.15.111:0/1634022469 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7fd52c0391b0 0x7fd52c03b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:17.531 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.806+0000 7fd54c000640 1 --2- 172.21.15.111:0/1634022469 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd544081860 0x7fd5441a1d30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:17.531 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.806+0000 7fd54c000640 1 -- 172.21.15.111:0/1634022469 >> 172.21.15.111:0/1634022469 conn(0x7fd54407cfa0 msgr2=0x7fd54407dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:17.531 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.807+0000 7fd54c000640 1 -- 172.21.15.111:0/1634022469 shutdown_connections 2024-09-15T22:36:17.531 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:16.807+0000 7fd54c000640 1 -- 172.21.15.111:0/1634022469 wait complete. 2024-09-15T22:36:17.663 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:17 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:17.539+0000 7f0ade836040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:36:17.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:17 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/1634022469' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-15T22:36:17.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:17 smithi111 ceph-mon[30629]: mgrmap e5: x(active, since 9s) 2024-09-15T22:36:17.916 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:17 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:17.705+0000 7f0ade836040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T22:36:18.313 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:18 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:18.081+0000 7f0ade836040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T22:36:18.313 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:18 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:18.170+0000 7f0ade836040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T22:36:18.665 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:18 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:18.386+0000 7f0ade836040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T22:36:19.082 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:18 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/4081412294' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-15T22:36:19.417 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout { 2024-09-15T22:36:19.417 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-15T22:36:19.417 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-15T22:36:19.418 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-15T22:36:19.418 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-15T22:36:19.418 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout } 2024-09-15T22:36:19.418 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.102+0000 7f81e3c00640 1 Processor -- start 2024-09-15T22:36:19.418 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.102+0000 7f81e3c00640 1 -- start start 2024-09-15T22:36:19.418 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.102+0000 7f81e3c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81dc109910 0x7f81dc109d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:19.418 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.102+0000 7f81e3c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc10a2e0 con 0x7f81dc109910 2024-09-15T22:36:19.418 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.103+0000 7f81e1200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81dc109910 0x7f81dc109d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:19.418 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.103+0000 7f81e1200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81dc109910 0x7f81dc109d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48668/0 (socket says 172.21.15.111:48668) 2024-09-15T22:36:19.418 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.103+0000 7f81e1200640 1 -- 172.21.15.111:0/4192884193 learned_addr learned my addr 172.21.15.111:0/4192884193 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:19.418 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.103+0000 7f81e1200640 1 -- 172.21.15.111:0/4192884193 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81dc10ab10 con 0x7f81dc109910 2024-09-15T22:36:19.418 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.104+0000 7f81e1200640 1 --2- 172.21.15.111:0/4192884193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81dc109910 0x7f81dc109d10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f81d0009820 tx=0x7f81d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb38affe20892a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:19.418 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.104+0000 7f81db400640 1 -- 172.21.15.111:0/4192884193 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81d0037070 con 0x7f81dc109910 2024-09-15T22:36:19.419 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.104+0000 7f81db400640 1 -- 172.21.15.111:0/4192884193 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f81d0032900 con 0x7f81dc109910 2024-09-15T22:36:19.419 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.104+0000 7f81db400640 1 -- 172.21.15.111:0/4192884193 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81d0030850 con 0x7f81dc109910 2024-09-15T22:36:19.419 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.105+0000 7f81e3c00640 1 -- 172.21.15.111:0/4192884193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81dc109910 msgr2=0x7f81dc109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:19.419 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.105+0000 7f81e3c00640 1 --2- 172.21.15.111:0/4192884193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81dc109910 0x7f81dc109d10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f81d0009820 tx=0x7f81d002a270 comp rx=0 tx=0).stop 2024-09-15T22:36:19.419 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.105+0000 7f81e3c00640 1 -- 172.21.15.111:0/4192884193 shutdown_connections 2024-09-15T22:36:19.419 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.105+0000 7f81e3c00640 1 --2- 172.21.15.111:0/4192884193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81dc109910 0x7f81dc109d10 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:19.419 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.105+0000 7f81e3c00640 1 -- 172.21.15.111:0/4192884193 >> 172.21.15.111:0/4192884193 conn(0x7f81dc07cf50 msgr2=0x7f81dc07d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:19.419 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.105+0000 7f81e3c00640 1 -- 172.21.15.111:0/4192884193 shutdown_connections 2024-09-15T22:36:19.419 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.105+0000 7f81e3c00640 1 -- 172.21.15.111:0/4192884193 wait complete. 2024-09-15T22:36:19.419 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.106+0000 7f81e3c00640 1 Processor -- start 2024-09-15T22:36:19.419 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.106+0000 7f81e3c00640 1 -- start start 2024-09-15T22:36:19.419 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.107+0000 7f81e3c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81dc109910 0x7f81dc19d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:19.419 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.107+0000 7f81e3c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc10dec0 con 0x7f81dc109910 2024-09-15T22:36:19.420 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.107+0000 7f81e1200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81dc109910 0x7f81dc19d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:19.420 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.107+0000 7f81e1200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81dc109910 0x7f81dc19d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48680/0 (socket says 172.21.15.111:48680) 2024-09-15T22:36:19.420 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.107+0000 7f81e1200640 1 -- 172.21.15.111:0/4081412294 learned_addr learned my addr 172.21.15.111:0/4081412294 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:19.420 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.107+0000 7f81e1200640 1 -- 172.21.15.111:0/4081412294 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81dc19de50 con 0x7f81dc109910 2024-09-15T22:36:19.420 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.108+0000 7f81e1200640 1 --2- 172.21.15.111:0/4081412294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81dc109910 0x7f81dc19d910 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f81d0009940 tx=0x7f81d0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:19.420 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.108+0000 7f81d9600640 1 -- 172.21.15.111:0/4081412294 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81d0042070 con 0x7f81dc109910 2024-09-15T22:36:19.420 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.108+0000 7f81d9600640 1 -- 172.21.15.111:0/4081412294 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f81d00308c0 con 0x7f81dc109910 2024-09-15T22:36:19.420 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.108+0000 7f81d9600640 1 -- 172.21.15.111:0/4081412294 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81d0037040 con 0x7f81dc109910 2024-09-15T22:36:19.420 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.108+0000 7f81e3c00640 1 -- 172.21.15.111:0/4081412294 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81dc19e0e0 con 0x7f81dc109910 2024-09-15T22:36:19.420 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.108+0000 7f81e3c00640 1 -- 172.21.15.111:0/4081412294 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81dc1a0dd0 con 0x7f81dc109910 2024-09-15T22:36:19.420 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.109+0000 7f81d9600640 1 -- 172.21.15.111:0/4081412294 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f81d0045070 con 0x7f81dc109910 2024-09-15T22:36:19.420 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.109+0000 7f81d9600640 1 --2- 172.21.15.111:0/4081412294 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7f81c4039200 0x7f81c403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:19.420 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.109+0000 7f81d9600640 1 -- 172.21.15.111:0/4081412294 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f81d006c460 con 0x7f81dc109910 2024-09-15T22:36:19.421 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.109+0000 7f81e3c00640 1 -- 172.21.15.111:0/4081412294 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81dc10e040 con 0x7f81dc109910 2024-09-15T22:36:19.421 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.110+0000 7f81dbe00640 1 -- 172.21.15.111:0/4081412294 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7f81c4039200 msgr2=0x7f81c403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/3350425104 2024-09-15T22:36:19.421 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.110+0000 7f81dbe00640 1 --2- 172.21.15.111:0/4081412294 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7f81c4039200 0x7f81c403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:36:19.421 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.115+0000 7f81d9600640 1 -- 172.21.15.111:0/4081412294 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f81d003add0 con 0x7f81dc109910 2024-09-15T22:36:19.421 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.311+0000 7f81dbe00640 1 -- 172.21.15.111:0/4081412294 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7f81c4039200 msgr2=0x7f81c403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/3350425104 2024-09-15T22:36:19.421 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.311+0000 7f81dbe00640 1 --2- 172.21.15.111:0/4081412294 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7f81c4039200 0x7f81c403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T22:36:19.421 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.357+0000 7f81e3c00640 1 -- 172.21.15.111:0/4081412294 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f81dc1a0fa0 con 0x7f81dc109910 2024-09-15T22:36:19.421 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.357+0000 7f81d9600640 1 -- 172.21.15.111:0/4081412294 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f81d006be80 con 0x7f81dc109910 2024-09-15T22:36:19.421 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.360+0000 7f81e3c00640 1 -- 172.21.15.111:0/4081412294 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7f81c4039200 msgr2=0x7f81c403b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:36:19.421 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.360+0000 7f81e3c00640 1 --2- 172.21.15.111:0/4081412294 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7f81c4039200 0x7f81c403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:19.421 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.360+0000 7f81e3c00640 1 -- 172.21.15.111:0/4081412294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81dc109910 msgr2=0x7f81dc19d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:19.421 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.360+0000 7f81e3c00640 1 --2- 172.21.15.111:0/4081412294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81dc109910 0x7f81dc19d910 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f81d0009940 tx=0x7f81d0032ce0 comp rx=0 tx=0).stop 2024-09-15T22:36:19.421 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.360+0000 7f81e3c00640 1 -- 172.21.15.111:0/4081412294 shutdown_connections 2024-09-15T22:36:19.422 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.360+0000 7f81e3c00640 1 --2- 172.21.15.111:0/4081412294 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7f81c4039200 0x7f81c403b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:19.422 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.360+0000 7f81e3c00640 1 --2- 172.21.15.111:0/4081412294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81dc109910 0x7f81dc19d910 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:19.422 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.360+0000 7f81e3c00640 1 -- 172.21.15.111:0/4081412294 >> 172.21.15.111:0/4081412294 conn(0x7f81dc07cf50 msgr2=0x7f81dc107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:19.422 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.360+0000 7f81e3c00640 1 -- 172.21.15.111:0/4081412294 shutdown_connections 2024-09-15T22:36:19.422 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:18.360+0000 7f81e3c00640 1 -- 172.21.15.111:0/4081412294 wait complete. 2024-09-15T22:36:19.422 INFO:teuthology.orchestra.run.smithi111.stdout:Waiting for the mgr to restart... 2024-09-15T22:36:19.422 INFO:teuthology.orchestra.run.smithi111.stdout:Waiting for mgr epoch 5... 2024-09-15T22:36:19.665 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:19 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:19.502+0000 7f0ade836040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T22:36:19.666 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:19 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:19.581+0000 7f0ade836040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T22:36:20.804 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:20 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:20.507+0000 7f0ade836040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T22:36:20.805 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:20 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:20.643+0000 7f0ade836040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T22:36:20.805 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:20 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:20.802+0000 7f0ade836040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:36:21.123 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:20 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:20.874+0000 7f0ade836040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T22:36:21.123 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:21 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:21.041+0000 7f0ade836040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T22:36:21.123 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:21 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:21.121+0000 7f0ade836040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T22:36:21.413 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:21 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:21.191+0000 7f0ade836040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T22:36:21.413 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:21 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:21.368+0000 7f0ade836040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T22:36:21.666 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:21 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:21.444+0000 7f0ade836040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T22:36:21.666 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:21 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:21.522+0000 7f0ade836040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T22:36:21.666 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:21 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:21.617+0000 7f0ade836040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T22:36:22.165 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:21 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:21.688+0000 7f0ade836040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T22:36:22.166 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:21 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:21.756+0000 7f0ade836040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T22:36:22.916 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:22 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:22.650+0000 7f0ade836040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T22:36:22.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:22 smithi111 ceph-mon[30629]: Active manager daemon x restarted 2024-09-15T22:36:22.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:22 smithi111 ceph-mon[30629]: Activating manager daemon x 2024-09-15T22:36:22.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:22 smithi111 ceph-mon[30629]: osdmap e2: 0 total, 0 up, 0 in 2024-09-15T22:36:22.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:22 smithi111 ceph-mon[30629]: mgrmap e6: x(active, starting, since 0.00418079s) 2024-09-15T22:36:22.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:22 smithi111 ceph-mon[30629]: from='mgr.14116 172.21.15.111:0/472659009' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:36:22.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:22 smithi111 ceph-mon[30629]: from='mgr.14116 172.21.15.111:0/472659009' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-15T22:36:22.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:22 smithi111 ceph-mon[30629]: from='mgr.14116 172.21.15.111:0/472659009' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-15T22:36:22.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:22 smithi111 ceph-mon[30629]: from='mgr.14116 172.21.15.111:0/472659009' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-15T22:36:22.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:22 smithi111 ceph-mon[30629]: from='mgr.14116 172.21.15.111:0/472659009' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-15T22:36:22.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:22 smithi111 ceph-mon[30629]: Manager daemon x is now available 2024-09-15T22:36:22.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:22 smithi111 ceph-mon[30629]: from='mgr.14116 172.21.15.111:0/472659009' entity='mgr.x' 2024-09-15T22:36:22.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:22 smithi111 ceph-mon[30629]: from='mgr.14116 172.21.15.111:0/472659009' entity='mgr.x' 2024-09-15T22:36:22.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:22 smithi111 ceph-mon[30629]: from='mgr.14116 172.21.15.111:0/472659009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:36:22.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:22 smithi111 ceph-mon[30629]: from='mgr.14116 172.21.15.111:0/472659009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:36:23.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:23 smithi111 ceph-mon[30629]: Found migration_current of "None". Setting to last migration. 2024-09-15T22:36:23.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:23 smithi111 ceph-mon[30629]: from='mgr.14116 172.21.15.111:0/472659009' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-15T22:36:23.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:23 smithi111 ceph-mon[30629]: from='mgr.14116 172.21.15.111:0/472659009' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-15T22:36:23.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:23 smithi111 ceph-mon[30629]: from='mgr.14116 172.21.15.111:0/472659009' entity='mgr.x' 2024-09-15T22:36:23.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:23 smithi111 ceph-mon[30629]: from='mgr.14116 172.21.15.111:0/472659009' entity='mgr.x' 2024-09-15T22:36:23.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:23 smithi111 ceph-mon[30629]: mgrmap e7: x(active, since 1.00675s) 2024-09-15T22:36:24.417 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout { 2024-09-15T22:36:24.417 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-15T22:36:24.417 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-15T22:36:24.417 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout } 2024-09-15T22:36:24.417 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.000+0000 7f5b65600640 1 Processor -- start 2024-09-15T22:36:24.417 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.000+0000 7f5b65600640 1 -- start start 2024-09-15T22:36:24.417 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.001+0000 7f5b65600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b6007ddd0 0x7f5b6007c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:24.418 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.001+0000 7f5b65600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6007c7c0 con 0x7f5b6007ddd0 2024-09-15T22:36:24.418 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.001+0000 7f5b5fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b6007ddd0 0x7f5b6007c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:24.418 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.001+0000 7f5b5fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b6007ddd0 0x7f5b6007c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48696/0 (socket says 172.21.15.111:48696) 2024-09-15T22:36:24.418 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.001+0000 7f5b5fe00640 1 -- 172.21.15.111:0/1110445468 learned_addr learned my addr 172.21.15.111:0/1110445468 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:24.418 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.001+0000 7f5b5fe00640 1 -- 172.21.15.111:0/1110445468 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6007c940 con 0x7f5b6007ddd0 2024-09-15T22:36:24.418 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.002+0000 7f5b5fe00640 1 --2- 172.21.15.111:0/1110445468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b6007ddd0 0x7f5b6007c280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009820 tx=0x7f5b5002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d9c7f1c445ab7a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:24.418 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.003+0000 7f5b5f400640 1 -- 172.21.15.111:0/1110445468 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b50037070 con 0x7f5b6007ddd0 2024-09-15T22:36:24.418 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.003+0000 7f5b5f400640 1 -- 172.21.15.111:0/1110445468 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5b50032900 con 0x7f5b6007ddd0 2024-09-15T22:36:24.418 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.003+0000 7f5b5f400640 1 -- 172.21.15.111:0/1110445468 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b500308e0 con 0x7f5b6007ddd0 2024-09-15T22:36:24.418 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.003+0000 7f5b65600640 1 -- 172.21.15.111:0/1110445468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b6007ddd0 msgr2=0x7f5b6007c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:24.418 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.003+0000 7f5b65600640 1 --2- 172.21.15.111:0/1110445468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b6007ddd0 0x7f5b6007c280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009820 tx=0x7f5b5002a300 comp rx=0 tx=0).stop 2024-09-15T22:36:24.419 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.004+0000 7f5b65600640 1 -- 172.21.15.111:0/1110445468 shutdown_connections 2024-09-15T22:36:24.419 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.004+0000 7f5b65600640 1 --2- 172.21.15.111:0/1110445468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b6007ddd0 0x7f5b6007c280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.419 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.004+0000 7f5b65600640 1 -- 172.21.15.111:0/1110445468 >> 172.21.15.111:0/1110445468 conn(0x7f5b60102d30 msgr2=0x7f5b601051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:24.419 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.004+0000 7f5b65600640 1 -- 172.21.15.111:0/1110445468 shutdown_connections 2024-09-15T22:36:24.419 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.004+0000 7f5b65600640 1 -- 172.21.15.111:0/1110445468 wait complete. 2024-09-15T22:36:24.419 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.005+0000 7f5b65600640 1 Processor -- start 2024-09-15T22:36:24.419 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.005+0000 7f5b65600640 1 -- start start 2024-09-15T22:36:24.419 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.005+0000 7f5b65600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b6007ddd0 0x7f5b6019d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:24.419 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.005+0000 7f5b65600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6010db90 con 0x7f5b6007ddd0 2024-09-15T22:36:24.419 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.005+0000 7f5b5fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b6007ddd0 0x7f5b6019d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:24.419 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.005+0000 7f5b5fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b6007ddd0 0x7f5b6019d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48710/0 (socket says 172.21.15.111:48710) 2024-09-15T22:36:24.419 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.005+0000 7f5b5fe00640 1 -- 172.21.15.111:0/3924969955 learned_addr learned my addr 172.21.15.111:0/3924969955 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:24.419 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.006+0000 7f5b5fe00640 1 -- 172.21.15.111:0/3924969955 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6019dcf0 con 0x7f5b6007ddd0 2024-09-15T22:36:24.420 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.006+0000 7f5b5fe00640 1 --2- 172.21.15.111:0/3924969955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b6007ddd0 0x7f5b6019d7b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5b5002a820 tx=0x7f5b50030900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:24.420 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.006+0000 7f5b5d600640 1 -- 172.21.15.111:0/3924969955 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b50037040 con 0x7f5b6007ddd0 2024-09-15T22:36:24.420 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.006+0000 7f5b5d600640 1 -- 172.21.15.111:0/3924969955 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5b50040070 con 0x7f5b6007ddd0 2024-09-15T22:36:24.420 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.006+0000 7f5b5d600640 1 -- 172.21.15.111:0/3924969955 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b5003bdd0 con 0x7f5b6007ddd0 2024-09-15T22:36:24.420 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.006+0000 7f5b65600640 1 -- 172.21.15.111:0/3924969955 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6019df80 con 0x7f5b6007ddd0 2024-09-15T22:36:24.420 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.006+0000 7f5b65600640 1 -- 172.21.15.111:0/3924969955 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b601a0c70 con 0x7f5b6007ddd0 2024-09-15T22:36:24.420 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.008+0000 7f5b5d600640 1 -- 172.21.15.111:0/3924969955 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f5b50030a90 con 0x7f5b6007ddd0 2024-09-15T22:36:24.420 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.008+0000 7f5b5d600640 1 --2- 172.21.15.111:0/3924969955 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7f5b440391b0 0x7f5b4403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:24.420 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.008+0000 7f5b5d600640 1 -- 172.21.15.111:0/3924969955 --> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5b4403bd80 con 0x7f5b440391b0 2024-09-15T22:36:24.420 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.008+0000 7f5b5d600640 1 -- 172.21.15.111:0/3924969955 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5b5006c0a0 con 0x7f5b6007ddd0 2024-09-15T22:36:24.420 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.008+0000 7f5b57e00640 1 -- 172.21.15.111:0/3924969955 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7f5b440391b0 msgr2=0x7f5b4403b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/3350425104 2024-09-15T22:36:24.420 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.008+0000 7f5b57e00640 1 --2- 172.21.15.111:0/3924969955 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7f5b440391b0 0x7f5b4403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:36:24.421 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.209+0000 7f5b57e00640 1 -- 172.21.15.111:0/3924969955 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7f5b440391b0 msgr2=0x7f5b4403b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/3350425104 2024-09-15T22:36:24.421 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.209+0000 7f5b57e00640 1 --2- 172.21.15.111:0/3924969955 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7f5b440391b0 0x7f5b4403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T22:36:24.421 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.610+0000 7f5b57e00640 1 -- 172.21.15.111:0/3924969955 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7f5b440391b0 msgr2=0x7f5b4403b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/3350425104 2024-09-15T22:36:24.421 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:20.610+0000 7f5b57e00640 1 --2- 172.21.15.111:0/3924969955 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7f5b440391b0 0x7f5b4403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T22:36:24.421 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:21.411+0000 7f5b57e00640 1 -- 172.21.15.111:0/3924969955 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7f5b440391b0 msgr2=0x7f5b4403b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/3350425104 2024-09-15T22:36:24.421 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:21.411+0000 7f5b57e00640 1 --2- 172.21.15.111:0/3924969955 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7f5b440391b0 0x7f5b4403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T22:36:24.421 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:22.654+0000 7f5b5d600640 1 -- 172.21.15.111:0/3924969955 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7f5b5006aea0 con 0x7f5b6007ddd0 2024-09-15T22:36:24.421 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:22.654+0000 7f5b5d600640 1 -- 172.21.15.111:0/3924969955 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7f5b440391b0 msgr2=0x7f5b4403b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:36:24.421 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:22.654+0000 7f5b5d600640 1 --2- 172.21.15.111:0/3924969955 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7f5b440391b0 0x7f5b4403b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.421 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:23.656+0000 7f5b5d600640 1 -- 172.21.15.111:0/3924969955 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f5b5004bd10 con 0x7f5b6007ddd0 2024-09-15T22:36:24.421 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:23.656+0000 7f5b5d600640 1 --2- 172.21.15.111:0/3924969955 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f5b4403cae0 0x7f5b4403eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:24.421 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:23.656+0000 7f5b5d600640 1 -- 172.21.15.111:0/3924969955 --> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5b5003de70 con 0x7f5b4403cae0 2024-09-15T22:36:24.421 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:23.657+0000 7f5b57e00640 1 --2- 172.21.15.111:0/3924969955 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f5b4403cae0 0x7f5b4403eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:24.422 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:23.659+0000 7f5b57e00640 1 --2- 172.21.15.111:0/3924969955 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f5b4403cae0 0x7f5b4403eed0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5b48003980 tx=0x7f5b480092b0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:24.422 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:23.660+0000 7f5b5d600640 1 -- 172.21.15.111:0/3924969955 <== mgr.14116 v2:172.21.15.111:6800/2107775445 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f5b5003de70 con 0x7f5b4403cae0 2024-09-15T22:36:24.422 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:23.667+0000 7f5b65600640 1 -- 172.21.15.111:0/3924969955 --> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f5b6007c300 con 0x7f5b4403cae0 2024-09-15T22:36:24.422 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:23.667+0000 7f5b5d600640 1 -- 172.21.15.111:0/3924969955 <== mgr.14116 v2:172.21.15.111:6800/2107775445 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f5b6007c300 con 0x7f5b4403cae0 2024-09-15T22:36:24.422 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:23.667+0000 7f5b65600640 1 -- 172.21.15.111:0/3924969955 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f5b4403cae0 msgr2=0x7f5b4403eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:24.422 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:23.667+0000 7f5b65600640 1 --2- 172.21.15.111:0/3924969955 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f5b4403cae0 0x7f5b4403eed0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5b48003980 tx=0x7f5b480092b0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.422 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:23.667+0000 7f5b65600640 1 -- 172.21.15.111:0/3924969955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b6007ddd0 msgr2=0x7f5b6019d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:24.422 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:23.667+0000 7f5b65600640 1 --2- 172.21.15.111:0/3924969955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b6007ddd0 0x7f5b6019d7b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5b5002a820 tx=0x7f5b50030900 comp rx=0 tx=0).stop 2024-09-15T22:36:24.423 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:23.668+0000 7f5b65600640 1 -- 172.21.15.111:0/3924969955 shutdown_connections 2024-09-15T22:36:24.423 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:23.668+0000 7f5b65600640 1 --2- 172.21.15.111:0/3924969955 >> [v2:172.21.15.111:6800/3350425104,v1:172.21.15.111:6801/3350425104] conn(0x7f5b440391b0 0x7f5b4403b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.423 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:23.668+0000 7f5b65600640 1 --2- 172.21.15.111:0/3924969955 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f5b4403cae0 0x7f5b4403eed0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.423 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:23.668+0000 7f5b65600640 1 --2- 172.21.15.111:0/3924969955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5b6007ddd0 0x7f5b6019d7b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:24.423 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:23.668+0000 7f5b65600640 1 -- 172.21.15.111:0/3924969955 >> 172.21.15.111:0/3924969955 conn(0x7f5b60102d30 msgr2=0x7f5b601039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:24.423 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:23.668+0000 7f5b65600640 1 -- 172.21.15.111:0/3924969955 shutdown_connections 2024-09-15T22:36:24.423 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:23.668+0000 7f5b65600640 1 -- 172.21.15.111:0/3924969955 wait complete. 2024-09-15T22:36:24.423 INFO:teuthology.orchestra.run.smithi111.stdout:mgr epoch 5 is available 2024-09-15T22:36:24.423 INFO:teuthology.orchestra.run.smithi111.stdout:Setting orchestrator backend to cephadm... 2024-09-15T22:36:24.666 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:24 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:24.435+0000 7f0a3da00640 -1 log_channel(cephadm) log [ERR] : [15/Sep/2024:22:36:24] ENGINE Error in HTTPServer.serve 2024-09-15T22:36:24.666 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:24 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: Traceback (most recent call last): 2024-09-15T22:36:24.666 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:24 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T22:36:24.666 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:24 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: self._connections.run(self.expiration_interval) 2024-09-15T22:36:24.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:24 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T22:36:24.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:24 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: self._run(expiration_interval) 2024-09-15T22:36:24.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:24 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T22:36:24.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:24 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T22:36:24.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:24 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T22:36:24.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:24 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T22:36:24.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:24 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T22:36:24.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:24 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: s = self.context.wrap_socket( 2024-09-15T22:36:24.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:24 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T22:36:24.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:24 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: return self.sslsocket_class._create( 2024-09-15T22:36:24.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:24 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T22:36:24.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:24 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: self.do_handshake() 2024-09-15T22:36:24.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:24 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T22:36:24.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:24 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: self._sslobj.do_handshake() 2024-09-15T22:36:24.668 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:24 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T22:36:24.668 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:24 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:25.100 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:24 smithi111 ceph-mon[30629]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-15T22:36:25.100 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:24 smithi111 ceph-mon[30629]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-15T22:36:25.100 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:24 smithi111 ceph-mon[30629]: from='mgr.14116 172.21.15.111:0/472659009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:36:25.811 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.896+0000 7f463ce00640 1 Processor -- start 2024-09-15T22:36:25.811 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.896+0000 7f463ce00640 1 -- start start 2024-09-15T22:36:25.811 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.897+0000 7f463ce00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4638109930 0x7f4638109d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:25.811 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.897+0000 7f463ce00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f463810a300 con 0x7f4638109930 2024-09-15T22:36:25.812 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.897+0000 7f4635e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4638109930 0x7f4638109d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:25.812 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.897+0000 7f4635e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4638109930 0x7f4638109d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57140/0 (socket says 172.21.15.111:57140) 2024-09-15T22:36:25.812 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.897+0000 7f4635e00640 1 -- 172.21.15.111:0/4228312366 learned_addr learned my addr 172.21.15.111:0/4228312366 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:25.812 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.897+0000 7f4635e00640 1 -- 172.21.15.111:0/4228312366 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463810ab20 con 0x7f4638109930 2024-09-15T22:36:25.812 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.898+0000 7f4635e00640 1 --2- 172.21.15.111:0/4228312366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4638109930 0x7f4638109d30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4628009820 tx=0x7f462802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abece47c6b687f08 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:25.813 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.899+0000 7f4635400640 1 -- 172.21.15.111:0/4228312366 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4628037070 con 0x7f4638109930 2024-09-15T22:36:25.813 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.899+0000 7f4635400640 1 -- 172.21.15.111:0/4228312366 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4628032900 con 0x7f4638109930 2024-09-15T22:36:25.813 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.899+0000 7f4635400640 1 -- 172.21.15.111:0/4228312366 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4628030850 con 0x7f4638109930 2024-09-15T22:36:25.813 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.899+0000 7f463ce00640 1 -- 172.21.15.111:0/4228312366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4638109930 msgr2=0x7f4638109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:25.813 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.899+0000 7f463ce00640 1 --2- 172.21.15.111:0/4228312366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4638109930 0x7f4638109d30 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4628009820 tx=0x7f462802a270 comp rx=0 tx=0).stop 2024-09-15T22:36:25.814 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.899+0000 7f463ce00640 1 -- 172.21.15.111:0/4228312366 shutdown_connections 2024-09-15T22:36:25.814 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.899+0000 7f463ce00640 1 --2- 172.21.15.111:0/4228312366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4638109930 0x7f4638109d30 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:25.814 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.899+0000 7f463ce00640 1 -- 172.21.15.111:0/4228312366 >> 172.21.15.111:0/4228312366 conn(0x7f463807cfa0 msgr2=0x7f463807d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:25.814 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.899+0000 7f463ce00640 1 -- 172.21.15.111:0/4228312366 shutdown_connections 2024-09-15T22:36:25.814 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.900+0000 7f463ce00640 1 -- 172.21.15.111:0/4228312366 wait complete. 2024-09-15T22:36:25.815 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.900+0000 7f463ce00640 1 Processor -- start 2024-09-15T22:36:25.815 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.901+0000 7f463ce00640 1 -- start start 2024-09-15T22:36:25.815 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.901+0000 7f463ce00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4638109930 0x7f463819d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:25.815 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.901+0000 7f463ce00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f463810ded0 con 0x7f4638109930 2024-09-15T22:36:25.815 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.901+0000 7f4635e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4638109930 0x7f463819d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:25.816 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.901+0000 7f4635e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4638109930 0x7f463819d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57154/0 (socket says 172.21.15.111:57154) 2024-09-15T22:36:25.816 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.901+0000 7f4635e00640 1 -- 172.21.15.111:0/1353455808 learned_addr learned my addr 172.21.15.111:0/1353455808 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:25.816 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.901+0000 7f4635e00640 1 -- 172.21.15.111:0/1353455808 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463819de80 con 0x7f4638109930 2024-09-15T22:36:25.816 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.902+0000 7f4635e00640 1 --2- 172.21.15.111:0/1353455808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4638109930 0x7f463819d940 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f46280097f0 tx=0x7f4628030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:25.816 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.902+0000 7f462f400640 1 -- 172.21.15.111:0/1353455808 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4628037040 con 0x7f4638109930 2024-09-15T22:36:25.817 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.902+0000 7f462f400640 1 -- 172.21.15.111:0/1353455808 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4628040070 con 0x7f4638109930 2024-09-15T22:36:25.817 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.902+0000 7f462f400640 1 -- 172.21.15.111:0/1353455808 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f462803a390 con 0x7f4638109930 2024-09-15T22:36:25.817 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.902+0000 7f463ce00640 1 -- 172.21.15.111:0/1353455808 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f463819e110 con 0x7f4638109930 2024-09-15T22:36:25.817 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.902+0000 7f463ce00640 1 -- 172.21.15.111:0/1353455808 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46381a0e00 con 0x7f4638109930 2024-09-15T22:36:25.817 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.903+0000 7f462f400640 1 -- 172.21.15.111:0/1353455808 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f4628032900 con 0x7f4638109930 2024-09-15T22:36:25.817 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.903+0000 7f463ce00640 1 -- 172.21.15.111:0/1353455808 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4638109db0 con 0x7f4638109930 2024-09-15T22:36:25.818 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.904+0000 7f462f400640 1 --2- 172.21.15.111:0/1353455808 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f461c039200 0x7f461c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:25.818 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.904+0000 7f462f400640 1 -- 172.21.15.111:0/1353455808 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f462806c7f0 con 0x7f4638109930 2024-09-15T22:36:25.818 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.904+0000 7f462d400640 1 --2- 172.21.15.111:0/1353455808 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f461c039200 0x7f461c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:25.818 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.905+0000 7f462d400640 1 --2- 172.21.15.111:0/1353455808 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f461c039200 0x7f461c03b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f46200098c0 tx=0x7f4620006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:25.818 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:24.909+0000 7f462f400640 1 -- 172.21.15.111:0/1353455808 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f462806c140 con 0x7f4638109930 2024-09-15T22:36:25.819 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:25.098+0000 7f463ce00640 1 -- 172.21.15.111:0/1353455808 --> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f4638107c50 con 0x7f461c039200 2024-09-15T22:36:25.819 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:25.104+0000 7f462f400640 1 -- 172.21.15.111:0/1353455808 <== mgr.14116 v2:172.21.15.111:6800/2107775445 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4638107c50 con 0x7f461c039200 2024-09-15T22:36:25.819 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:25.106+0000 7f463ce00640 1 -- 172.21.15.111:0/1353455808 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f461c039200 msgr2=0x7f461c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:25.820 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:25.106+0000 7f463ce00640 1 --2- 172.21.15.111:0/1353455808 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f461c039200 0x7f461c03b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f46200098c0 tx=0x7f4620006db0 comp rx=0 tx=0).stop 2024-09-15T22:36:25.820 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:25.106+0000 7f463ce00640 1 -- 172.21.15.111:0/1353455808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4638109930 msgr2=0x7f463819d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:25.820 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:25.106+0000 7f463ce00640 1 --2- 172.21.15.111:0/1353455808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4638109930 0x7f463819d940 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f46280097f0 tx=0x7f4628030d80 comp rx=0 tx=0).stop 2024-09-15T22:36:25.820 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:25.106+0000 7f463ce00640 1 -- 172.21.15.111:0/1353455808 shutdown_connections 2024-09-15T22:36:25.820 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:25.106+0000 7f463ce00640 1 --2- 172.21.15.111:0/1353455808 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f461c039200 0x7f461c03b6c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:25.821 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:25.106+0000 7f463ce00640 1 --2- 172.21.15.111:0/1353455808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4638109930 0x7f463819d940 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:25.821 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:25.106+0000 7f463ce00640 1 -- 172.21.15.111:0/1353455808 >> 172.21.15.111:0/1353455808 conn(0x7f463807cfa0 msgr2=0x7f4638107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:25.821 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:25.107+0000 7f463ce00640 1 -- 172.21.15.111:0/1353455808 shutdown_connections 2024-09-15T22:36:25.821 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:25.107+0000 7f463ce00640 1 -- 172.21.15.111:0/1353455808 wait complete. 2024-09-15T22:36:26.102 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:25 smithi111 ceph-mon[30629]: [15/Sep/2024:22:36:24] ENGINE Bus STARTING 2024-09-15T22:36:26.102 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:25 smithi111 ceph-mon[30629]: [15/Sep/2024:22:36:24] ENGINE Serving on https://172.21.15.111:7150 2024-09-15T22:36:26.102 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:25 smithi111 ceph-mon[30629]: [15/Sep/2024:22:36:24] ENGINE Error in HTTPServer.serve 2024-09-15T22:36:26.102 INFO:journalctl@ceph.mon.a.smithi111.stdout: Traceback (most recent call last): 2024-09-15T22:36:26.102 INFO:journalctl@ceph.mon.a.smithi111.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T22:36:26.102 INFO:journalctl@ceph.mon.a.smithi111.stdout: self._connections.run(self.expiration_interval) 2024-09-15T22:36:26.102 INFO:journalctl@ceph.mon.a.smithi111.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T22:36:26.102 INFO:journalctl@ceph.mon.a.smithi111.stdout: self._run(expiration_interval) 2024-09-15T22:36:26.102 INFO:journalctl@ceph.mon.a.smithi111.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T22:36:26.102 INFO:journalctl@ceph.mon.a.smithi111.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T22:36:26.102 INFO:journalctl@ceph.mon.a.smithi111.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T22:36:26.103 INFO:journalctl@ceph.mon.a.smithi111.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T22:36:26.103 INFO:journalctl@ceph.mon.a.smithi111.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T22:36:26.103 INFO:journalctl@ceph.mon.a.smithi111.stdout: s = self.context.wrap_socket( 2024-09-15T22:36:26.103 INFO:journalctl@ceph.mon.a.smithi111.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T22:36:26.103 INFO:journalctl@ceph.mon.a.smithi111.stdout: return self.sslsocket_class._create( 2024-09-15T22:36:26.103 INFO:journalctl@ceph.mon.a.smithi111.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T22:36:26.103 INFO:journalctl@ceph.mon.a.smithi111.stdout: self.do_handshake() 2024-09-15T22:36:26.103 INFO:journalctl@ceph.mon.a.smithi111.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T22:36:26.103 INFO:journalctl@ceph.mon.a.smithi111.stdout: self._sslobj.do_handshake() 2024-09-15T22:36:26.103 INFO:journalctl@ceph.mon.a.smithi111.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T22:36:26.103 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:25 smithi111 ceph-mon[30629]: [15/Sep/2024:22:36:24] ENGINE Serving on http://172.21.15.111:8765 2024-09-15T22:36:26.103 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:25 smithi111 ceph-mon[30629]: [15/Sep/2024:22:36:24] ENGINE Bus STARTED 2024-09-15T22:36:26.103 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:25 smithi111 ceph-mon[30629]: mgrmap e8: x(active, since 2s) 2024-09-15T22:36:26.104 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:25 smithi111 ceph-mon[30629]: from='mgr.14116 172.21.15.111:0/472659009' entity='mgr.x' 2024-09-15T22:36:26.104 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:25 smithi111 ceph-mon[30629]: from='mgr.14116 172.21.15.111:0/472659009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:36:26.898 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:26 smithi111 ceph-mon[30629]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:36:27.237 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-15T22:36:27.238 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.317+0000 7f6049400640 1 Processor -- start 2024-09-15T22:36:27.238 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.317+0000 7f6049400640 1 -- start start 2024-09-15T22:36:27.238 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.318+0000 7f6049400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6044109990 0x7f6044109d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:27.238 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.318+0000 7f6049400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f604410a360 con 0x7f6044109990 2024-09-15T22:36:27.238 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.318+0000 7f6042800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6044109990 0x7f6044109d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:27.238 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.318+0000 7f6042800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6044109990 0x7f6044109d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57162/0 (socket says 172.21.15.111:57162) 2024-09-15T22:36:27.238 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.318+0000 7f6042800640 1 -- 172.21.15.111:0/3373443364 learned_addr learned my addr 172.21.15.111:0/3373443364 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:27.238 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.318+0000 7f6042800640 1 -- 172.21.15.111:0/3373443364 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604410ab40 con 0x7f6044109990 2024-09-15T22:36:27.238 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.319+0000 7f6042800640 1 --2- 172.21.15.111:0/3373443364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6044109990 0x7f6044109d90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6034009820 tx=0x7f603402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f23b004e3642419 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:27.238 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.320+0000 7f6041400640 1 -- 172.21.15.111:0/3373443364 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6034037070 con 0x7f6044109990 2024-09-15T22:36:27.238 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.320+0000 7f6041400640 1 -- 172.21.15.111:0/3373443364 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6034032900 con 0x7f6044109990 2024-09-15T22:36:27.238 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.320+0000 7f6041400640 1 -- 172.21.15.111:0/3373443364 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6034030850 con 0x7f6044109990 2024-09-15T22:36:27.238 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.320+0000 7f6049400640 1 -- 172.21.15.111:0/3373443364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6044109990 msgr2=0x7f6044109d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:27.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.320+0000 7f6049400640 1 --2- 172.21.15.111:0/3373443364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6044109990 0x7f6044109d90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6034009820 tx=0x7f603402a270 comp rx=0 tx=0).stop 2024-09-15T22:36:27.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.320+0000 7f6049400640 1 -- 172.21.15.111:0/3373443364 shutdown_connections 2024-09-15T22:36:27.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.320+0000 7f6049400640 1 --2- 172.21.15.111:0/3373443364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6044109990 0x7f6044109d90 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:27.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.320+0000 7f6049400640 1 -- 172.21.15.111:0/3373443364 >> 172.21.15.111:0/3373443364 conn(0x7f604407cfd0 msgr2=0x7f604407d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:27.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.320+0000 7f6049400640 1 -- 172.21.15.111:0/3373443364 shutdown_connections 2024-09-15T22:36:27.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.321+0000 7f6049400640 1 -- 172.21.15.111:0/3373443364 wait complete. 2024-09-15T22:36:27.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.321+0000 7f6049400640 1 Processor -- start 2024-09-15T22:36:27.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.321+0000 7f6049400640 1 -- start start 2024-09-15T22:36:27.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.322+0000 7f6049400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6044109990 0x7f604419d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:27.239 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.322+0000 7f6049400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f604410def0 con 0x7f6044109990 2024-09-15T22:36:27.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.322+0000 7f6042800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6044109990 0x7f604419d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:27.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.322+0000 7f6042800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6044109990 0x7f604419d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57176/0 (socket says 172.21.15.111:57176) 2024-09-15T22:36:27.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.322+0000 7f6042800640 1 -- 172.21.15.111:0/967884005 learned_addr learned my addr 172.21.15.111:0/967884005 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:27.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.322+0000 7f6042800640 1 -- 172.21.15.111:0/967884005 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604419def0 con 0x7f6044109990 2024-09-15T22:36:27.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.323+0000 7f6042800640 1 --2- 172.21.15.111:0/967884005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6044109990 0x7f604419d9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f60340097f0 tx=0x7f6034030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:27.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.323+0000 7f603b400640 1 -- 172.21.15.111:0/967884005 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6034042070 con 0x7f6044109990 2024-09-15T22:36:27.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.323+0000 7f603b400640 1 -- 172.21.15.111:0/967884005 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6034032ca0 con 0x7f6044109990 2024-09-15T22:36:27.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.323+0000 7f6049400640 1 -- 172.21.15.111:0/967884005 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f604419e180 con 0x7f6044109990 2024-09-15T22:36:27.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.323+0000 7f603b400640 1 -- 172.21.15.111:0/967884005 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6034037040 con 0x7f6044109990 2024-09-15T22:36:27.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.323+0000 7f6049400640 1 -- 172.21.15.111:0/967884005 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60441a0e70 con 0x7f6044109990 2024-09-15T22:36:27.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.324+0000 7f603b400640 1 -- 172.21.15.111:0/967884005 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f6034045070 con 0x7f6044109990 2024-09-15T22:36:27.240 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.324+0000 7f6049400640 1 -- 172.21.15.111:0/967884005 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f604410e070 con 0x7f6044109990 2024-09-15T22:36:27.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.324+0000 7f603b400640 1 --2- 172.21.15.111:0/967884005 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f6028039200 0x7f602803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:27.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.325+0000 7f603b400640 1 -- 172.21.15.111:0/967884005 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f603406c230 con 0x7f6044109990 2024-09-15T22:36:27.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.325+0000 7f6041e00640 1 --2- 172.21.15.111:0/967884005 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f6028039200 0x7f602803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:27.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.325+0000 7f6041e00640 1 --2- 172.21.15.111:0/967884005 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f6028039200 0x7f602803b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f60300098c0 tx=0x7f6030006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:27.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.329+0000 7f603b400640 1 -- 172.21.15.111:0/967884005 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f603404b170 con 0x7f6044109990 2024-09-15T22:36:27.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.507+0000 7f6049400640 1 -- 172.21.15.111:0/967884005 --> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f6044107c60 con 0x7f6028039200 2024-09-15T22:36:27.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.508+0000 7f603b400640 1 -- 172.21.15.111:0/967884005 <== mgr.14116 v2:172.21.15.111:6800/2107775445 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f6044107c60 con 0x7f6028039200 2024-09-15T22:36:27.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.510+0000 7f6049400640 1 -- 172.21.15.111:0/967884005 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f6028039200 msgr2=0x7f602803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:27.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.510+0000 7f6049400640 1 --2- 172.21.15.111:0/967884005 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f6028039200 0x7f602803b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f60300098c0 tx=0x7f6030006db0 comp rx=0 tx=0).stop 2024-09-15T22:36:27.241 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.510+0000 7f6049400640 1 -- 172.21.15.111:0/967884005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6044109990 msgr2=0x7f604419d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:27.242 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.510+0000 7f6049400640 1 --2- 172.21.15.111:0/967884005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6044109990 0x7f604419d9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f60340097f0 tx=0x7f6034030ec0 comp rx=0 tx=0).stop 2024-09-15T22:36:27.242 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.510+0000 7f6049400640 1 -- 172.21.15.111:0/967884005 shutdown_connections 2024-09-15T22:36:27.242 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.510+0000 7f6049400640 1 --2- 172.21.15.111:0/967884005 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f6028039200 0x7f602803b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:27.242 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.510+0000 7f6049400640 1 --2- 172.21.15.111:0/967884005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6044109990 0x7f604419d9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:27.242 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.510+0000 7f6049400640 1 -- 172.21.15.111:0/967884005 >> 172.21.15.111:0/967884005 conn(0x7f604407cfd0 msgr2=0x7f6044107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:27.242 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.511+0000 7f6049400640 1 -- 172.21.15.111:0/967884005 shutdown_connections 2024-09-15T22:36:27.242 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:26.511+0000 7f6049400640 1 -- 172.21.15.111:0/967884005 wait complete. 2024-09-15T22:36:27.242 INFO:teuthology.orchestra.run.smithi111.stdout:Generating ssh key... 2024-09-15T22:36:27.758 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:27 smithi111 ceph-mon[30629]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:36:28.666 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:28 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: Generating public/private rsa key pair. 2024-09-15T22:36:28.666 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:28 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: Your identification has been saved in /tmp/tmpyp2yeap7/key 2024-09-15T22:36:28.666 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:28 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: Your public key has been saved in /tmp/tmpyp2yeap7/key.pub 2024-09-15T22:36:28.666 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:28 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: The key fingerprint is: 2024-09-15T22:36:28.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:28 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: SHA256:d+kVKV88rYBF6KWSI7edOOWHKOEUW0ntyYjaxDL+LSA ceph-b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T22:36:28.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:28 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: The key's randomart image is: 2024-09-15T22:36:28.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:28 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: +---[RSA 3072]----+ 2024-09-15T22:36:28.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:28 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: | ..o oo | 2024-09-15T22:36:28.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:28 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: | . o oo. o.| 2024-09-15T22:36:28.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:28 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: | . = *.+o o.+| 2024-09-15T22:36:28.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:28 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: | o O * B = +.| 2024-09-15T22:36:28.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:28 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: | . O +SX.oo + | 2024-09-15T22:36:28.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:28 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: | E + + =.=o.. | 2024-09-15T22:36:28.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:28 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: | . o o . .. | 2024-09-15T22:36:28.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:28 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: | o . | 2024-09-15T22:36:28.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:28 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: | . | 2024-09-15T22:36:28.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:28 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: +----[SHA256]-----+ 2024-09-15T22:36:29.117 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.749+0000 7f0f8f000640 1 Processor -- start 2024-09-15T22:36:29.117 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.749+0000 7f0f8f000640 1 -- start start 2024-09-15T22:36:29.117 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.750+0000 7f0f8f000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f88109930 0x7f0f88109d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:29.117 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.750+0000 7f0f8f000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8810a300 con 0x7f0f88109930 2024-09-15T22:36:29.117 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.750+0000 7f0f87e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f88109930 0x7f0f88109d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:29.117 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.750+0000 7f0f87e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f88109930 0x7f0f88109d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57184/0 (socket says 172.21.15.111:57184) 2024-09-15T22:36:29.117 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.750+0000 7f0f87e00640 1 -- 172.21.15.111:0/2432227182 learned_addr learned my addr 172.21.15.111:0/2432227182 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:29.117 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.750+0000 7f0f87e00640 1 -- 172.21.15.111:0/2432227182 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f8810ab20 con 0x7f0f88109930 2024-09-15T22:36:29.117 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.751+0000 7f0f87e00640 1 --2- 172.21.15.111:0/2432227182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f88109930 0x7f0f88109d30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0f78009820 tx=0x7f0f7802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d70233cc474819c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:29.118 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.752+0000 7f0f86a00640 1 -- 172.21.15.111:0/2432227182 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f78037070 con 0x7f0f88109930 2024-09-15T22:36:29.118 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.752+0000 7f0f86a00640 1 -- 172.21.15.111:0/2432227182 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0f78032900 con 0x7f0f88109930 2024-09-15T22:36:29.118 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.752+0000 7f0f86a00640 1 -- 172.21.15.111:0/2432227182 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f78030850 con 0x7f0f88109930 2024-09-15T22:36:29.118 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.752+0000 7f0f8f000640 1 -- 172.21.15.111:0/2432227182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f88109930 msgr2=0x7f0f88109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:29.118 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.752+0000 7f0f8f000640 1 --2- 172.21.15.111:0/2432227182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f88109930 0x7f0f88109d30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0f78009820 tx=0x7f0f7802a270 comp rx=0 tx=0).stop 2024-09-15T22:36:29.118 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.752+0000 7f0f8f000640 1 -- 172.21.15.111:0/2432227182 shutdown_connections 2024-09-15T22:36:29.118 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.752+0000 7f0f8f000640 1 --2- 172.21.15.111:0/2432227182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f88109930 0x7f0f88109d30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.118 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.752+0000 7f0f8f000640 1 -- 172.21.15.111:0/2432227182 >> 172.21.15.111:0/2432227182 conn(0x7f0f8807cfa0 msgr2=0x7f0f8807d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:29.118 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.752+0000 7f0f8f000640 1 -- 172.21.15.111:0/2432227182 shutdown_connections 2024-09-15T22:36:29.118 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.752+0000 7f0f8f000640 1 -- 172.21.15.111:0/2432227182 wait complete. 2024-09-15T22:36:29.118 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.753+0000 7f0f8f000640 1 Processor -- start 2024-09-15T22:36:29.118 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.753+0000 7f0f8f000640 1 -- start start 2024-09-15T22:36:29.119 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.754+0000 7f0f8f000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f88109930 0x7f0f8819d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:29.119 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.754+0000 7f0f8f000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8810ded0 con 0x7f0f88109930 2024-09-15T22:36:29.119 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.754+0000 7f0f87e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f88109930 0x7f0f8819d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:29.119 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.754+0000 7f0f87e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f88109930 0x7f0f8819d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57194/0 (socket says 172.21.15.111:57194) 2024-09-15T22:36:29.119 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.754+0000 7f0f87e00640 1 -- 172.21.15.111:0/4188826849 learned_addr learned my addr 172.21.15.111:0/4188826849 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:29.119 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.754+0000 7f0f87e00640 1 -- 172.21.15.111:0/4188826849 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f8819de80 con 0x7f0f88109930 2024-09-15T22:36:29.119 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.755+0000 7f0f87e00640 1 --2- 172.21.15.111:0/4188826849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f88109930 0x7f0f8819d940 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0f780097f0 tx=0x7f0f78030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:29.119 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.755+0000 7f0f84c00640 1 -- 172.21.15.111:0/4188826849 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f78037040 con 0x7f0f88109930 2024-09-15T22:36:29.119 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.755+0000 7f0f84c00640 1 -- 172.21.15.111:0/4188826849 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0f78040070 con 0x7f0f88109930 2024-09-15T22:36:29.119 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.755+0000 7f0f84c00640 1 -- 172.21.15.111:0/4188826849 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f7803a390 con 0x7f0f88109930 2024-09-15T22:36:29.119 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.755+0000 7f0f8f000640 1 -- 172.21.15.111:0/4188826849 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f8819e110 con 0x7f0f88109930 2024-09-15T22:36:29.119 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.755+0000 7f0f8f000640 1 -- 172.21.15.111:0/4188826849 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f881a0e00 con 0x7f0f88109930 2024-09-15T22:36:29.119 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.756+0000 7f0f8f000640 1 -- 172.21.15.111:0/4188826849 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f88109db0 con 0x7f0f88109930 2024-09-15T22:36:29.119 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.756+0000 7f0f84c00640 1 -- 172.21.15.111:0/4188826849 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f0f7803b430 con 0x7f0f88109930 2024-09-15T22:36:29.120 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.757+0000 7f0f84c00640 1 --2- 172.21.15.111:0/4188826849 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f0f6c039200 0x7f0f6c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:29.120 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.757+0000 7f0f84c00640 1 -- 172.21.15.111:0/4188826849 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0f7806c7d0 con 0x7f0f88109930 2024-09-15T22:36:29.120 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.757+0000 7f0f87400640 1 --2- 172.21.15.111:0/4188826849 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f0f6c039200 0x7f0f6c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:29.120 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.758+0000 7f0f87400640 1 --2- 172.21.15.111:0/4188826849 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f0f6c039200 0x7f0f6c03b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0f740098c0 tx=0x7f0f74006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:29.120 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.764+0000 7f0f84c00640 1 -- 172.21.15.111:0/4188826849 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0f7806c120 con 0x7f0f88109930 2024-09-15T22:36:29.120 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:27.941+0000 7f0f8f000640 1 -- 172.21.15.111:0/4188826849 --> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f0f88107c50 con 0x7f0f6c039200 2024-09-15T22:36:29.120 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:28.398+0000 7f0f84c00640 1 -- 172.21.15.111:0/4188826849 <== mgr.14116 v2:172.21.15.111:6800/2107775445 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0f88107c50 con 0x7f0f6c039200 2024-09-15T22:36:29.120 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:28.400+0000 7f0f8f000640 1 -- 172.21.15.111:0/4188826849 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f0f6c039200 msgr2=0x7f0f6c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:29.120 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:28.400+0000 7f0f8f000640 1 --2- 172.21.15.111:0/4188826849 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f0f6c039200 0x7f0f6c03b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0f740098c0 tx=0x7f0f74006db0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.120 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:28.400+0000 7f0f8f000640 1 -- 172.21.15.111:0/4188826849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f88109930 msgr2=0x7f0f8819d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:29.120 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:28.400+0000 7f0f8f000640 1 --2- 172.21.15.111:0/4188826849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f88109930 0x7f0f8819d940 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0f780097f0 tx=0x7f0f78030d80 comp rx=0 tx=0).stop 2024-09-15T22:36:29.120 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:28.400+0000 7f0f8f000640 1 -- 172.21.15.111:0/4188826849 shutdown_connections 2024-09-15T22:36:29.121 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:28.400+0000 7f0f8f000640 1 --2- 172.21.15.111:0/4188826849 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f0f6c039200 0x7f0f6c03b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.121 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:28.400+0000 7f0f8f000640 1 --2- 172.21.15.111:0/4188826849 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f88109930 0x7f0f8819d940 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:29.121 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:28.400+0000 7f0f8f000640 1 -- 172.21.15.111:0/4188826849 >> 172.21.15.111:0/4188826849 conn(0x7f0f8807cfa0 msgr2=0x7f0f88107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:29.121 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:28.401+0000 7f0f8f000640 1 -- 172.21.15.111:0/4188826849 shutdown_connections 2024-09-15T22:36:29.121 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:28.401+0000 7f0f8f000640 1 -- 172.21.15.111:0/4188826849 wait complete. 2024-09-15T22:36:29.648 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:29 smithi111 ceph-mon[30629]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:36:29.648 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:29 smithi111 ceph-mon[30629]: Generating ssh key... 2024-09-15T22:36:29.648 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:29 smithi111 ceph-mon[30629]: from='mgr.14116 172.21.15.111:0/472659009' entity='mgr.x' 2024-09-15T22:36:29.649 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:29 smithi111 ceph-mon[30629]: from='mgr.14116 172.21.15.111:0/472659009' entity='mgr.x' 2024-09-15T22:36:30.538 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:30 smithi111 ceph-mon[30629]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:36:30.549 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCWKCBLmWKt6EZ4bc1UdpbSUq0inAxSTYz/nq36B8tVHl18FhGU/hrKS2h/bxY4YCdrMoxj/xBoKGf46ue/aLao1CiaFeY3dsFltHqHcsyrRZIbXZ/L05f/fDiF5VtNGDAlMAUul+sbPCIyIX0yk2iTSWgRv8ITAADIZMD07NDpsntHSJzgSiQGoD7CYZ9/moFPVHniuzvlbwlE9//UD8LJogTckjVkjTZ0Ukjx2wbtNEihmCmJkHX0YFUH59q9w1MIcsLL6OtwjevxaVUxsFNXF0IwQzjyWzJlknaz+GvuUfpnZlSHwWMdPJOaYeUsdtWPYfLo/GSnA75BMAIWTha7L43B8NoFFS5AKb0oysNRKoxfMVKAe8+0Yn7OVrSEj/y/WkpFhxy0ERyBJVm6XsgdhLK4hY37cPjwUU/VPMR9G7pgW3a1NPqZ0lcexwODi8wuhRla5NIw+BWEkXY1YpytWo6cv8ub00KShJOCkxCaAb+lWloCbp6cjam/bJMyLJ0= ceph-b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T22:36:30.549 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.631+0000 7f146fe00640 1 Processor -- start 2024-09-15T22:36:30.549 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.631+0000 7f146fe00640 1 -- start start 2024-09-15T22:36:30.549 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.631+0000 7f146fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1468107700 0x7f1468107b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:30.549 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.631+0000 7f146fe00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f14681080d0 con 0x7f1468107700 2024-09-15T22:36:30.549 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.632+0000 7f146d400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1468107700 0x7f1468107b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:30.549 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.632+0000 7f146d400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1468107700 0x7f1468107b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57204/0 (socket says 172.21.15.111:57204) 2024-09-15T22:36:30.550 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.632+0000 7f146d400640 1 -- 172.21.15.111:0/2997855836 learned_addr learned my addr 172.21.15.111:0/2997855836 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:30.550 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.632+0000 7f146d400640 1 -- 172.21.15.111:0/2997855836 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1468108900 con 0x7f1468107700 2024-09-15T22:36:30.550 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.633+0000 7f146d400640 1 --2- 172.21.15.111:0/2997855836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1468107700 0x7f1468107b00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f145c009820 tx=0x7f145c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ef36f5f6493004 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:30.550 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.633+0000 7f1467e00640 1 -- 172.21.15.111:0/2997855836 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f145c037070 con 0x7f1468107700 2024-09-15T22:36:30.550 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.633+0000 7f1467e00640 1 -- 172.21.15.111:0/2997855836 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f145c032900 con 0x7f1468107700 2024-09-15T22:36:30.550 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.633+0000 7f1467e00640 1 -- 172.21.15.111:0/2997855836 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f145c030850 con 0x7f1468107700 2024-09-15T22:36:30.550 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.634+0000 7f146fe00640 1 -- 172.21.15.111:0/2997855836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1468107700 msgr2=0x7f1468107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:30.550 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.634+0000 7f146fe00640 1 --2- 172.21.15.111:0/2997855836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1468107700 0x7f1468107b00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f145c009820 tx=0x7f145c02a270 comp rx=0 tx=0).stop 2024-09-15T22:36:30.550 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.634+0000 7f146fe00640 1 -- 172.21.15.111:0/2997855836 shutdown_connections 2024-09-15T22:36:30.550 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.634+0000 7f146fe00640 1 --2- 172.21.15.111:0/2997855836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1468107700 0x7f1468107b00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:30.550 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.634+0000 7f146fe00640 1 -- 172.21.15.111:0/2997855836 >> 172.21.15.111:0/2997855836 conn(0x7f1468102f30 msgr2=0x7f1468105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:30.550 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.634+0000 7f146fe00640 1 -- 172.21.15.111:0/2997855836 shutdown_connections 2024-09-15T22:36:30.550 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.634+0000 7f146fe00640 1 -- 172.21.15.111:0/2997855836 wait complete. 2024-09-15T22:36:30.550 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.635+0000 7f146fe00640 1 Processor -- start 2024-09-15T22:36:30.551 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.635+0000 7f146fe00640 1 -- start start 2024-09-15T22:36:30.551 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.636+0000 7f146fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1468107700 0x7f146807daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:30.551 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.636+0000 7f146fe00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f146810bcb0 con 0x7f1468107700 2024-09-15T22:36:30.551 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.636+0000 7f146d400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1468107700 0x7f146807daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:30.551 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.636+0000 7f146d400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1468107700 0x7f146807daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57208/0 (socket says 172.21.15.111:57208) 2024-09-15T22:36:30.551 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.636+0000 7f146d400640 1 -- 172.21.15.111:0/690220051 learned_addr learned my addr 172.21.15.111:0/690220051 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:30.551 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.636+0000 7f146d400640 1 -- 172.21.15.111:0/690220051 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f146807e030 con 0x7f1468107700 2024-09-15T22:36:30.551 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.637+0000 7f146d400640 1 --2- 172.21.15.111:0/690220051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1468107700 0x7f146807daf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f145c0097f0 tx=0x7f145c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:30.551 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.637+0000 7f1466000640 1 -- 172.21.15.111:0/690220051 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f145c037040 con 0x7f1468107700 2024-09-15T22:36:30.551 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.637+0000 7f1466000640 1 -- 172.21.15.111:0/690220051 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f145c040070 con 0x7f1468107700 2024-09-15T22:36:30.551 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.637+0000 7f146fe00640 1 -- 172.21.15.111:0/690220051 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f146807c220 con 0x7f1468107700 2024-09-15T22:36:30.551 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.637+0000 7f146fe00640 1 -- 172.21.15.111:0/690220051 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f146807c700 con 0x7f1468107700 2024-09-15T22:36:30.551 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.637+0000 7f1466000640 1 -- 172.21.15.111:0/690220051 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f145c03a390 con 0x7f1468107700 2024-09-15T22:36:30.551 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.638+0000 7f1466000640 1 -- 172.21.15.111:0/690220051 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f145c032900 con 0x7f1468107700 2024-09-15T22:36:30.551 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.638+0000 7f146fe00640 1 -- 172.21.15.111:0/690220051 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1468107b80 con 0x7f1468107700 2024-09-15T22:36:30.552 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.638+0000 7f1466000640 1 --2- 172.21.15.111:0/690220051 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f14500391b0 0x7f145003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:30.552 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.638+0000 7f1466000640 1 -- 172.21.15.111:0/690220051 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f145c06c810 con 0x7f1468107700 2024-09-15T22:36:30.552 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.639+0000 7f146ca00640 1 --2- 172.21.15.111:0/690220051 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f14500391b0 0x7f145003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:30.552 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.639+0000 7f146ca00640 1 --2- 172.21.15.111:0/690220051 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f14500391b0 0x7f145003b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f14540098c0 tx=0x7f1454006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:30.552 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.643+0000 7f1466000640 1 -- 172.21.15.111:0/690220051 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f145c06c160 con 0x7f1468107700 2024-09-15T22:36:30.552 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.821+0000 7f146fe00640 1 -- 172.21.15.111:0/690220051 --> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f1468104160 con 0x7f14500391b0 2024-09-15T22:36:30.552 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.822+0000 7f1466000640 1 -- 172.21.15.111:0/690220051 <== mgr.14116 v2:172.21.15.111:6800/2107775445 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f1468104160 con 0x7f14500391b0 2024-09-15T22:36:30.552 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.824+0000 7f146fe00640 1 -- 172.21.15.111:0/690220051 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f14500391b0 msgr2=0x7f145003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:30.552 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.824+0000 7f146fe00640 1 --2- 172.21.15.111:0/690220051 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f14500391b0 0x7f145003b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f14540098c0 tx=0x7f1454006db0 comp rx=0 tx=0).stop 2024-09-15T22:36:30.552 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.824+0000 7f146fe00640 1 -- 172.21.15.111:0/690220051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1468107700 msgr2=0x7f146807daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:30.552 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.824+0000 7f146fe00640 1 --2- 172.21.15.111:0/690220051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1468107700 0x7f146807daf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f145c0097f0 tx=0x7f145c030d80 comp rx=0 tx=0).stop 2024-09-15T22:36:30.552 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.825+0000 7f146fe00640 1 -- 172.21.15.111:0/690220051 shutdown_connections 2024-09-15T22:36:30.552 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.825+0000 7f146fe00640 1 --2- 172.21.15.111:0/690220051 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f14500391b0 0x7f145003b670 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:30.552 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.825+0000 7f146fe00640 1 --2- 172.21.15.111:0/690220051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1468107700 0x7f146807daf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:30.553 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.825+0000 7f146fe00640 1 -- 172.21.15.111:0/690220051 >> 172.21.15.111:0/690220051 conn(0x7f1468102f30 msgr2=0x7f1468103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:30.553 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.825+0000 7f146fe00640 1 -- 172.21.15.111:0/690220051 shutdown_connections 2024-09-15T22:36:30.553 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:29.825+0000 7f146fe00640 1 -- 172.21.15.111:0/690220051 wait complete. 2024-09-15T22:36:30.553 INFO:teuthology.orchestra.run.smithi111.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-15T22:36:30.553 INFO:teuthology.orchestra.run.smithi111.stdout:Adding key to root@localhost authorized_keys... 2024-09-15T22:36:30.558 INFO:teuthology.orchestra.run.smithi111.stdout:Adding host smithi111... 2024-09-15T22:36:32.247 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:31 smithi111 ceph-mon[30629]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi111", "addr": "172.21.15.111", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:36:33.130 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:32 smithi111 ceph-mon[30629]: Deploying cephadm binary to smithi111 2024-09-15T22:36:35.872 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout Added host 'smithi111' with addr '172.21.15.111' 2024-09-15T22:36:35.872 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.104+0000 7f6192000640 1 Processor -- start 2024-09-15T22:36:35.872 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.104+0000 7f6192000640 1 -- start start 2024-09-15T22:36:35.873 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.105+0000 7f6192000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f618c109840 0x7f618c109c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:35.873 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.105+0000 7f6192000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f618c10a210 con 0x7f618c109840 2024-09-15T22:36:35.873 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.105+0000 7f6190c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f618c109840 0x7f618c109c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:35.873 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.105+0000 7f6190c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f618c109840 0x7f618c109c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41528/0 (socket says 172.21.15.111:41528) 2024-09-15T22:36:35.873 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.105+0000 7f6190c00640 1 -- 172.21.15.111:0/1316698583 learned_addr learned my addr 172.21.15.111:0/1316698583 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:35.873 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.105+0000 7f6190c00640 1 -- 172.21.15.111:0/1316698583 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618c10a950 con 0x7f618c109840 2024-09-15T22:36:35.873 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.106+0000 7f6190c00640 1 --2- 172.21.15.111:0/1316698583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f618c109840 0x7f618c109c40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6180009820 tx=0x7f618002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84037d73f26870b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:35.873 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.106+0000 7f618b400640 1 -- 172.21.15.111:0/1316698583 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6180037070 con 0x7f618c109840 2024-09-15T22:36:35.873 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.106+0000 7f618b400640 1 -- 172.21.15.111:0/1316698583 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6180032900 con 0x7f618c109840 2024-09-15T22:36:35.873 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.106+0000 7f618b400640 1 -- 172.21.15.111:0/1316698583 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6180030850 con 0x7f618c109840 2024-09-15T22:36:35.873 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.107+0000 7f6192000640 1 -- 172.21.15.111:0/1316698583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f618c109840 msgr2=0x7f618c109c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:35.873 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.107+0000 7f6192000640 1 --2- 172.21.15.111:0/1316698583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f618c109840 0x7f618c109c40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6180009820 tx=0x7f618002a270 comp rx=0 tx=0).stop 2024-09-15T22:36:35.873 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.107+0000 7f6192000640 1 -- 172.21.15.111:0/1316698583 shutdown_connections 2024-09-15T22:36:35.874 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.107+0000 7f6192000640 1 --2- 172.21.15.111:0/1316698583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f618c109840 0x7f618c109c40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:35.874 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.107+0000 7f6192000640 1 -- 172.21.15.111:0/1316698583 >> 172.21.15.111:0/1316698583 conn(0x7f618c07cd90 msgr2=0x7f618c07d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:35.874 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.107+0000 7f6192000640 1 -- 172.21.15.111:0/1316698583 shutdown_connections 2024-09-15T22:36:35.874 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.107+0000 7f6192000640 1 -- 172.21.15.111:0/1316698583 wait complete. 2024-09-15T22:36:35.874 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.108+0000 7f6192000640 1 Processor -- start 2024-09-15T22:36:35.874 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.108+0000 7f6192000640 1 -- start start 2024-09-15T22:36:35.874 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.108+0000 7f6192000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f618c109840 0x7f618c19d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:35.874 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.108+0000 7f6192000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f618c10dd00 con 0x7f618c109840 2024-09-15T22:36:35.874 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.109+0000 7f6190c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f618c109840 0x7f618c19d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:35.874 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.109+0000 7f6190c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f618c109840 0x7f618c19d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41536/0 (socket says 172.21.15.111:41536) 2024-09-15T22:36:35.874 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.109+0000 7f6190c00640 1 -- 172.21.15.111:0/994250977 learned_addr learned my addr 172.21.15.111:0/994250977 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:35.874 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.109+0000 7f6190c00640 1 -- 172.21.15.111:0/994250977 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618c19dd20 con 0x7f618c109840 2024-09-15T22:36:35.874 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.109+0000 7f6190c00640 1 --2- 172.21.15.111:0/994250977 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f618c109840 0x7f618c19d7e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f61800097f0 tx=0x7f6180030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:35.875 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.109+0000 7f6189600640 1 -- 172.21.15.111:0/994250977 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6180042070 con 0x7f618c109840 2024-09-15T22:36:35.875 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.109+0000 7f6189600640 1 -- 172.21.15.111:0/994250977 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6180032ca0 con 0x7f618c109840 2024-09-15T22:36:35.875 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.109+0000 7f6189600640 1 -- 172.21.15.111:0/994250977 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6180037040 con 0x7f618c109840 2024-09-15T22:36:35.875 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.109+0000 7f6192000640 1 -- 172.21.15.111:0/994250977 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f618c19dfb0 con 0x7f618c109840 2024-09-15T22:36:35.875 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.110+0000 7f6192000640 1 -- 172.21.15.111:0/994250977 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f618c1a0ca0 con 0x7f618c109840 2024-09-15T22:36:35.875 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.111+0000 7f6189600640 1 -- 172.21.15.111:0/994250977 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f6180045070 con 0x7f618c109840 2024-09-15T22:36:35.875 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.111+0000 7f6192000640 1 -- 172.21.15.111:0/994250977 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f618c10de80 con 0x7f618c109840 2024-09-15T22:36:35.875 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.111+0000 7f6189600640 1 --2- 172.21.15.111:0/994250977 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f6174039200 0x7f617403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:35.875 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.111+0000 7f6189600640 1 -- 172.21.15.111:0/994250977 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f618006c170 con 0x7f618c109840 2024-09-15T22:36:35.875 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.111+0000 7f618be00640 1 --2- 172.21.15.111:0/994250977 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f6174039200 0x7f617403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:35.875 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.112+0000 7f618be00640 1 --2- 172.21.15.111:0/994250977 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f6174039200 0x7f617403b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f61780098c0 tx=0x7f6178006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:35.875 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.116+0000 7f6189600640 1 -- 172.21.15.111:0/994250977 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f618004b170 con 0x7f618c109840 2024-09-15T22:36:35.875 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:31.295+0000 7f6192000640 1 -- 172.21.15.111:0/994250977 --> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi111", "addr": "172.21.15.111", "target": ["mon-mgr", ""]}) -- 0x7f618c107cd0 con 0x7f6174039200 2024-09-15T22:36:35.876 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:35.116+0000 7f6189600640 1 -- 172.21.15.111:0/994250977 <== mgr.14116 v2:172.21.15.111:6800/2107775445 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f618c107cd0 con 0x7f6174039200 2024-09-15T22:36:35.876 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:35.119+0000 7f6192000640 1 -- 172.21.15.111:0/994250977 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f6174039200 msgr2=0x7f617403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:35.876 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:35.119+0000 7f6192000640 1 --2- 172.21.15.111:0/994250977 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f6174039200 0x7f617403b6c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f61780098c0 tx=0x7f6178006db0 comp rx=0 tx=0).stop 2024-09-15T22:36:35.876 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:35.119+0000 7f6192000640 1 -- 172.21.15.111:0/994250977 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f618c109840 msgr2=0x7f618c19d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:35.876 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:35.119+0000 7f6192000640 1 --2- 172.21.15.111:0/994250977 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f618c109840 0x7f618c19d7e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f61800097f0 tx=0x7f6180030ec0 comp rx=0 tx=0).stop 2024-09-15T22:36:35.876 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:35.119+0000 7f6192000640 1 -- 172.21.15.111:0/994250977 shutdown_connections 2024-09-15T22:36:35.876 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:35.119+0000 7f6192000640 1 --2- 172.21.15.111:0/994250977 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f6174039200 0x7f617403b6c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:35.876 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:35.119+0000 7f6192000640 1 --2- 172.21.15.111:0/994250977 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f618c109840 0x7f618c19d7e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:35.876 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:35.119+0000 7f6192000640 1 -- 172.21.15.111:0/994250977 >> 172.21.15.111:0/994250977 conn(0x7f618c07cd90 msgr2=0x7f618c1075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:35.876 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:35.119+0000 7f6192000640 1 -- 172.21.15.111:0/994250977 shutdown_connections 2024-09-15T22:36:35.876 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:35.119+0000 7f6192000640 1 -- 172.21.15.111:0/994250977 wait complete. 2024-09-15T22:36:35.876 INFO:teuthology.orchestra.run.smithi111.stdout:Deploying unmanaged mon service... 2024-09-15T22:36:36.376 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:36 smithi111 ceph-mon[30629]: from='mgr.14116 172.21.15.111:0/472659009' entity='mgr.x' 2024-09-15T22:36:36.376 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:36 smithi111 ceph-mon[30629]: Added host smithi111 2024-09-15T22:36:36.376 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:36 smithi111 ceph-mon[30629]: from='mgr.14116 172.21.15.111:0/472659009' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:36:37.443 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-15T22:36:37.443 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.378+0000 7ff4fe800640 1 Processor -- start 2024-09-15T22:36:37.443 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.378+0000 7ff4fe800640 1 -- start start 2024-09-15T22:36:37.443 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.379+0000 7ff4fe800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4f807ddd0 0x7ff4f807c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:37.443 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.379+0000 7ff4fe800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f807c7c0 con 0x7ff4f807ddd0 2024-09-15T22:36:37.443 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.379+0000 7ff4fd400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4f807ddd0 0x7ff4f807c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:37.443 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.379+0000 7ff4fd400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4f807ddd0 0x7ff4f807c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41542/0 (socket says 172.21.15.111:41542) 2024-09-15T22:36:37.443 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.379+0000 7ff4fd400640 1 -- 172.21.15.111:0/2891128780 learned_addr learned my addr 172.21.15.111:0/2891128780 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:37.444 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.380+0000 7ff4fd400640 1 -- 172.21.15.111:0/2891128780 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f807c940 con 0x7ff4f807ddd0 2024-09-15T22:36:37.444 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.381+0000 7ff4fd400640 1 --2- 172.21.15.111:0/2891128780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4f807ddd0 0x7ff4f807c280 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec009820 tx=0x7ff4ec02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466ac73c7ed2dbd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:37.444 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.381+0000 7ff4f7e00640 1 -- 172.21.15.111:0/2891128780 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4ec037070 con 0x7ff4f807ddd0 2024-09-15T22:36:37.444 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.381+0000 7ff4f7e00640 1 -- 172.21.15.111:0/2891128780 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff4ec032900 con 0x7ff4f807ddd0 2024-09-15T22:36:37.444 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.381+0000 7ff4f7e00640 1 -- 172.21.15.111:0/2891128780 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4ec0308e0 con 0x7ff4f807ddd0 2024-09-15T22:36:37.444 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.381+0000 7ff4fe800640 1 -- 172.21.15.111:0/2891128780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4f807ddd0 msgr2=0x7ff4f807c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:37.444 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.381+0000 7ff4fe800640 1 --2- 172.21.15.111:0/2891128780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4f807ddd0 0x7ff4f807c280 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec009820 tx=0x7ff4ec02a300 comp rx=0 tx=0).stop 2024-09-15T22:36:37.444 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.381+0000 7ff4fe800640 1 -- 172.21.15.111:0/2891128780 shutdown_connections 2024-09-15T22:36:37.444 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.381+0000 7ff4fe800640 1 --2- 172.21.15.111:0/2891128780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4f807ddd0 0x7ff4f807c280 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:37.444 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.381+0000 7ff4fe800640 1 -- 172.21.15.111:0/2891128780 >> 172.21.15.111:0/2891128780 conn(0x7ff4f8102d30 msgr2=0x7ff4f81051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:37.444 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.382+0000 7ff4fe800640 1 -- 172.21.15.111:0/2891128780 shutdown_connections 2024-09-15T22:36:37.444 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.382+0000 7ff4fe800640 1 -- 172.21.15.111:0/2891128780 wait complete. 2024-09-15T22:36:37.445 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.382+0000 7ff4fe800640 1 Processor -- start 2024-09-15T22:36:37.445 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.383+0000 7ff4fe800640 1 -- start start 2024-09-15T22:36:37.445 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.383+0000 7ff4fe800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4f807ddd0 0x7ff4f819d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:37.445 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.383+0000 7ff4fe800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f810db90 con 0x7ff4f807ddd0 2024-09-15T22:36:37.445 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.383+0000 7ff4fd400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4f807ddd0 0x7ff4f819d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:37.445 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.383+0000 7ff4fd400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4f807ddd0 0x7ff4f819d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41552/0 (socket says 172.21.15.111:41552) 2024-09-15T22:36:37.445 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.383+0000 7ff4fd400640 1 -- 172.21.15.111:0/3834549821 learned_addr learned my addr 172.21.15.111:0/3834549821 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:37.445 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.383+0000 7ff4fd400640 1 -- 172.21.15.111:0/3834549821 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f819dcf0 con 0x7ff4f807ddd0 2024-09-15T22:36:37.445 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.384+0000 7ff4fd400640 1 --2- 172.21.15.111:0/3834549821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4f807ddd0 0x7ff4f819d7b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec02a820 tx=0x7ff4ec030900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:37.445 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.384+0000 7ff4f6000640 1 -- 172.21.15.111:0/3834549821 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4ec037040 con 0x7ff4f807ddd0 2024-09-15T22:36:37.445 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.384+0000 7ff4f6000640 1 -- 172.21.15.111:0/3834549821 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff4ec040070 con 0x7ff4f807ddd0 2024-09-15T22:36:37.445 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.384+0000 7ff4fe800640 1 -- 172.21.15.111:0/3834549821 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f819df80 con 0x7ff4f807ddd0 2024-09-15T22:36:37.445 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.384+0000 7ff4f6000640 1 -- 172.21.15.111:0/3834549821 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4ec03bdd0 con 0x7ff4f807ddd0 2024-09-15T22:36:37.446 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.384+0000 7ff4fe800640 1 -- 172.21.15.111:0/3834549821 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f81a0c70 con 0x7ff4f807ddd0 2024-09-15T22:36:37.446 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.385+0000 7ff4f6000640 1 -- 172.21.15.111:0/3834549821 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7ff4ec03b5a0 con 0x7ff4f807ddd0 2024-09-15T22:36:37.446 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.385+0000 7ff4fe800640 1 -- 172.21.15.111:0/3834549821 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f807c300 con 0x7ff4f807ddd0 2024-09-15T22:36:37.446 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.386+0000 7ff4f6000640 1 --2- 172.21.15.111:0/3834549821 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7ff4e00391b0 0x7ff4e003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:37.446 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.386+0000 7ff4f6000640 1 -- 172.21.15.111:0/3834549821 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff4ec06c3d0 con 0x7ff4f807ddd0 2024-09-15T22:36:37.446 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.386+0000 7ff4fca00640 1 --2- 172.21.15.111:0/3834549821 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7ff4e00391b0 0x7ff4e003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:37.446 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.387+0000 7ff4fca00640 1 --2- 172.21.15.111:0/3834549821 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7ff4e00391b0 0x7ff4e003b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff4e40098c0 tx=0x7ff4e4006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:37.446 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.391+0000 7ff4f6000640 1 -- 172.21.15.111:0/3834549821 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff4ec0371e0 con 0x7ff4f807ddd0 2024-09-15T22:36:37.446 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.579+0000 7ff4fe800640 1 -- 172.21.15.111:0/3834549821 --> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7ff4f8104080 con 0x7ff4e00391b0 2024-09-15T22:36:37.446 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.584+0000 7ff4f6000640 1 -- 172.21.15.111:0/3834549821 <== mgr.14116 v2:172.21.15.111:6800/2107775445 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff4f8104080 con 0x7ff4e00391b0 2024-09-15T22:36:37.446 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.586+0000 7ff4fe800640 1 -- 172.21.15.111:0/3834549821 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7ff4e00391b0 msgr2=0x7ff4e003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:37.446 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.586+0000 7ff4fe800640 1 --2- 172.21.15.111:0/3834549821 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7ff4e00391b0 0x7ff4e003b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff4e40098c0 tx=0x7ff4e4006db0 comp rx=0 tx=0).stop 2024-09-15T22:36:37.446 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.586+0000 7ff4fe800640 1 -- 172.21.15.111:0/3834549821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4f807ddd0 msgr2=0x7ff4f819d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:37.447 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.586+0000 7ff4fe800640 1 --2- 172.21.15.111:0/3834549821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4f807ddd0 0x7ff4f819d7b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec02a820 tx=0x7ff4ec030900 comp rx=0 tx=0).stop 2024-09-15T22:36:37.447 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.586+0000 7ff4fe800640 1 -- 172.21.15.111:0/3834549821 shutdown_connections 2024-09-15T22:36:37.447 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.586+0000 7ff4fe800640 1 --2- 172.21.15.111:0/3834549821 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7ff4e00391b0 0x7ff4e003b670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:37.447 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.586+0000 7ff4fe800640 1 --2- 172.21.15.111:0/3834549821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4f807ddd0 0x7ff4f819d7b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:37.447 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.586+0000 7ff4fe800640 1 -- 172.21.15.111:0/3834549821 >> 172.21.15.111:0/3834549821 conn(0x7ff4f8102d30 msgr2=0x7ff4f8103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:37.447 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.586+0000 7ff4fe800640 1 -- 172.21.15.111:0/3834549821 shutdown_connections 2024-09-15T22:36:37.447 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:36.587+0000 7ff4fe800640 1 -- 172.21.15.111:0/3834549821 wait complete. 2024-09-15T22:36:37.447 INFO:teuthology.orchestra.run.smithi111.stdout:Deploying unmanaged mgr service... 2024-09-15T22:36:37.809 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:37 smithi111 ceph-mon[30629]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:36:37.809 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:37 smithi111 ceph-mon[30629]: Saving service mon spec with placement count:5 2024-09-15T22:36:37.809 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:37 smithi111 ceph-mon[30629]: from='mgr.14116 172.21.15.111:0/472659009' entity='mgr.x' 2024-09-15T22:36:39.917 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-15T22:36:39.917 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.830+0000 7f9c89600640 1 Processor -- start 2024-09-15T22:36:39.917 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.831+0000 7f9c89600640 1 -- start start 2024-09-15T22:36:39.917 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.831+0000 7f9c89600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c8407ddb0 0x7f9c8407c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:39.917 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.831+0000 7f9c89600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8407c7a0 con 0x7f9c8407ddb0 2024-09-15T22:36:39.917 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.831+0000 7f9c83e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c8407ddb0 0x7f9c8407c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:39.917 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.831+0000 7f9c83e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c8407ddb0 0x7f9c8407c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41556/0 (socket says 172.21.15.111:41556) 2024-09-15T22:36:39.917 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.831+0000 7f9c83e00640 1 -- 172.21.15.111:0/143764339 learned_addr learned my addr 172.21.15.111:0/143764339 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:39.918 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.832+0000 7f9c83e00640 1 -- 172.21.15.111:0/143764339 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8407c920 con 0x7f9c8407ddb0 2024-09-15T22:36:39.918 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.832+0000 7f9c83e00640 1 --2- 172.21.15.111:0/143764339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c8407ddb0 0x7f9c8407c260 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9c74009820 tx=0x7f9c7402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33faf5c07250994 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:39.918 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.833+0000 7f9c83400640 1 -- 172.21.15.111:0/143764339 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c74037070 con 0x7f9c8407ddb0 2024-09-15T22:36:39.918 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.833+0000 7f9c83400640 1 -- 172.21.15.111:0/143764339 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9c74032900 con 0x7f9c8407ddb0 2024-09-15T22:36:39.918 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.833+0000 7f9c83400640 1 -- 172.21.15.111:0/143764339 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c74030850 con 0x7f9c8407ddb0 2024-09-15T22:36:39.918 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.833+0000 7f9c89600640 1 -- 172.21.15.111:0/143764339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c8407ddb0 msgr2=0x7f9c8407c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:39.918 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.833+0000 7f9c89600640 1 --2- 172.21.15.111:0/143764339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c8407ddb0 0x7f9c8407c260 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9c74009820 tx=0x7f9c7402a270 comp rx=0 tx=0).stop 2024-09-15T22:36:39.918 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.833+0000 7f9c89600640 1 -- 172.21.15.111:0/143764339 shutdown_connections 2024-09-15T22:36:39.918 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.834+0000 7f9c89600640 1 --2- 172.21.15.111:0/143764339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c8407ddb0 0x7f9c8407c260 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:39.918 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.834+0000 7f9c89600640 1 -- 172.21.15.111:0/143764339 >> 172.21.15.111:0/143764339 conn(0x7f9c84102d70 msgr2=0x7f9c84105190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:39.918 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.834+0000 7f9c89600640 1 -- 172.21.15.111:0/143764339 shutdown_connections 2024-09-15T22:36:39.918 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.834+0000 7f9c89600640 1 -- 172.21.15.111:0/143764339 wait complete. 2024-09-15T22:36:39.918 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.834+0000 7f9c89600640 1 Processor -- start 2024-09-15T22:36:39.919 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.835+0000 7f9c89600640 1 -- start start 2024-09-15T22:36:39.919 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.835+0000 7f9c89600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c8407ddb0 0x7f9c8419d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:39.919 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.835+0000 7f9c89600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8410dbd0 con 0x7f9c8407ddb0 2024-09-15T22:36:39.919 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.835+0000 7f9c83e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c8407ddb0 0x7f9c8419d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:39.919 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.835+0000 7f9c83e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c8407ddb0 0x7f9c8419d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41572/0 (socket says 172.21.15.111:41572) 2024-09-15T22:36:39.919 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.835+0000 7f9c83e00640 1 -- 172.21.15.111:0/576975718 learned_addr learned my addr 172.21.15.111:0/576975718 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:39.919 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.835+0000 7f9c83e00640 1 -- 172.21.15.111:0/576975718 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8419dcf0 con 0x7f9c8407ddb0 2024-09-15T22:36:39.919 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.836+0000 7f9c83e00640 1 --2- 172.21.15.111:0/576975718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c8407ddb0 0x7f9c8419d7b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9c740097f0 tx=0x7f9c74030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:39.919 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.836+0000 7f9c81600640 1 -- 172.21.15.111:0/576975718 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c74042070 con 0x7f9c8407ddb0 2024-09-15T22:36:39.919 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.836+0000 7f9c81600640 1 -- 172.21.15.111:0/576975718 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9c74032ca0 con 0x7f9c8407ddb0 2024-09-15T22:36:39.919 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.836+0000 7f9c81600640 1 -- 172.21.15.111:0/576975718 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c74037040 con 0x7f9c8407ddb0 2024-09-15T22:36:39.919 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.836+0000 7f9c89600640 1 -- 172.21.15.111:0/576975718 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c8419df80 con 0x7f9c8407ddb0 2024-09-15T22:36:39.919 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.836+0000 7f9c89600640 1 -- 172.21.15.111:0/576975718 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c841a0c70 con 0x7f9c8407ddb0 2024-09-15T22:36:39.920 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.837+0000 7f9c81600640 1 -- 172.21.15.111:0/576975718 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f9c74045070 con 0x7f9c8407ddb0 2024-09-15T22:36:39.920 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.837+0000 7f9c89600640 1 -- 172.21.15.111:0/576975718 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c8407c2e0 con 0x7f9c8407ddb0 2024-09-15T22:36:39.920 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.838+0000 7f9c81600640 1 --2- 172.21.15.111:0/576975718 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f9c680391b0 0x7f9c6803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:39.920 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.838+0000 7f9c81600640 1 -- 172.21.15.111:0/576975718 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9c7406c330 con 0x7f9c8407ddb0 2024-09-15T22:36:39.920 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.838+0000 7f9c7b400640 1 --2- 172.21.15.111:0/576975718 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f9c680391b0 0x7f9c6803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:39.920 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.839+0000 7f9c7b400640 1 --2- 172.21.15.111:0/576975718 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f9c680391b0 0x7f9c6803b670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c0098c0 tx=0x7f9c6c006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:39.920 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:38.843+0000 7f9c81600640 1 -- 172.21.15.111:0/576975718 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9c74045360 con 0x7f9c8407ddb0 2024-09-15T22:36:39.920 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:39.033+0000 7f9c89600640 1 -- 172.21.15.111:0/576975718 --> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f9c84104080 con 0x7f9c680391b0 2024-09-15T22:36:39.920 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:39.037+0000 7f9c81600640 1 -- 172.21.15.111:0/576975718 <== mgr.14116 v2:172.21.15.111:6800/2107775445 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9c84104080 con 0x7f9c680391b0 2024-09-15T22:36:39.920 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:39.040+0000 7f9c89600640 1 -- 172.21.15.111:0/576975718 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f9c680391b0 msgr2=0x7f9c6803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:39.920 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:39.040+0000 7f9c89600640 1 --2- 172.21.15.111:0/576975718 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f9c680391b0 0x7f9c6803b670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c0098c0 tx=0x7f9c6c006db0 comp rx=0 tx=0).stop 2024-09-15T22:36:39.920 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:39.040+0000 7f9c89600640 1 -- 172.21.15.111:0/576975718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c8407ddb0 msgr2=0x7f9c8419d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:39.920 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:39.040+0000 7f9c89600640 1 --2- 172.21.15.111:0/576975718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c8407ddb0 0x7f9c8419d7b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9c740097f0 tx=0x7f9c74030ec0 comp rx=0 tx=0).stop 2024-09-15T22:36:39.921 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:39.040+0000 7f9c89600640 1 -- 172.21.15.111:0/576975718 shutdown_connections 2024-09-15T22:36:39.921 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:39.040+0000 7f9c89600640 1 --2- 172.21.15.111:0/576975718 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f9c680391b0 0x7f9c6803b670 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:39.921 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:39.040+0000 7f9c89600640 1 --2- 172.21.15.111:0/576975718 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9c8407ddb0 0x7f9c8419d7b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:39.921 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:39.040+0000 7f9c89600640 1 -- 172.21.15.111:0/576975718 >> 172.21.15.111:0/576975718 conn(0x7f9c84102d70 msgr2=0x7f9c84103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:39.921 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:39.040+0000 7f9c89600640 1 -- 172.21.15.111:0/576975718 shutdown_connections 2024-09-15T22:36:39.921 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:39.040+0000 7f9c89600640 1 -- 172.21.15.111:0/576975718 wait complete. 2024-09-15T22:36:40.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:40 smithi111 ceph-mon[30629]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:36:40.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:40 smithi111 ceph-mon[30629]: Saving service mgr spec with placement count:2 2024-09-15T22:36:40.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:40 smithi111 ceph-mon[30629]: from='mgr.14116 172.21.15.111:0/472659009' entity='mgr.x' 2024-09-15T22:36:40.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:40 smithi111 ceph-mon[30629]: from='mgr.14116 172.21.15.111:0/472659009' entity='mgr.x' 2024-09-15T22:36:40.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:40 smithi111 ceph-mon[30629]: from='mgr.14116 172.21.15.111:0/472659009' entity='mgr.x' 2024-09-15T22:36:41.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:41 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/2783356798' entity='client.admin' 2024-09-15T22:36:41.927 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.414+0000 7f03a9200640 1 Processor -- start 2024-09-15T22:36:41.933 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.414+0000 7f03a9200640 1 -- start start 2024-09-15T22:36:41.933 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.414+0000 7f03a9200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03a41075e0 0x7f03a41079e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:41.933 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.414+0000 7f03a9200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f03a4107fb0 con 0x7f03a41075e0 2024-09-15T22:36:41.933 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.415+0000 7f03a3400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03a41075e0 0x7f03a41079e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:41.933 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.415+0000 7f03a3400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03a41075e0 0x7f03a41079e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41576/0 (socket says 172.21.15.111:41576) 2024-09-15T22:36:41.933 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.415+0000 7f03a3400640 1 -- 172.21.15.111:0/193108101 learned_addr learned my addr 172.21.15.111:0/193108101 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:41.933 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.415+0000 7f03a3400640 1 -- 172.21.15.111:0/193108101 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a4108790 con 0x7f03a41075e0 2024-09-15T22:36:41.933 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.416+0000 7f03a3400640 1 --2- 172.21.15.111:0/193108101 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03a41075e0 0x7f03a41079e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0394009820 tx=0x7f039402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201314ff094332e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:41.933 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.416+0000 7f03a2000640 1 -- 172.21.15.111:0/193108101 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0394037070 con 0x7f03a41075e0 2024-09-15T22:36:41.933 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.416+0000 7f03a2000640 1 -- 172.21.15.111:0/193108101 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0394032920 con 0x7f03a41075e0 2024-09-15T22:36:41.933 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.416+0000 7f03a2000640 1 -- 172.21.15.111:0/193108101 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0394030810 con 0x7f03a41075e0 2024-09-15T22:36:41.934 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.417+0000 7f03a9200640 1 -- 172.21.15.111:0/193108101 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03a41075e0 msgr2=0x7f03a41079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:41.934 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.417+0000 7f03a9200640 1 --2- 172.21.15.111:0/193108101 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03a41075e0 0x7f03a41079e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0394009820 tx=0x7f039402a270 comp rx=0 tx=0).stop 2024-09-15T22:36:41.934 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.417+0000 7f03a9200640 1 -- 172.21.15.111:0/193108101 shutdown_connections 2024-09-15T22:36:41.934 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.417+0000 7f03a9200640 1 --2- 172.21.15.111:0/193108101 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03a41075e0 0x7f03a41079e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:41.934 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.417+0000 7f03a9200640 1 -- 172.21.15.111:0/193108101 >> 172.21.15.111:0/193108101 conn(0x7f03a4102d30 msgr2=0x7f03a41051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:41.934 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.417+0000 7f03a9200640 1 -- 172.21.15.111:0/193108101 shutdown_connections 2024-09-15T22:36:41.934 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.417+0000 7f03a9200640 1 -- 172.21.15.111:0/193108101 wait complete. 2024-09-15T22:36:41.934 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.418+0000 7f03a9200640 1 Processor -- start 2024-09-15T22:36:41.934 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.418+0000 7f03a9200640 1 -- start start 2024-09-15T22:36:41.934 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.418+0000 7f03a9200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03a41075e0 0x7f03a419d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:41.934 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.418+0000 7f03a9200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f03a410bb40 con 0x7f03a41075e0 2024-09-15T22:36:41.934 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.419+0000 7f03a3400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03a41075e0 0x7f03a419d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:41.935 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.419+0000 7f03a3400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03a41075e0 0x7f03a419d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41582/0 (socket says 172.21.15.111:41582) 2024-09-15T22:36:41.935 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.419+0000 7f03a3400640 1 -- 172.21.15.111:0/2783356798 learned_addr learned my addr 172.21.15.111:0/2783356798 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:41.935 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.419+0000 7f03a3400640 1 -- 172.21.15.111:0/2783356798 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a419dd50 con 0x7f03a41075e0 2024-09-15T22:36:41.935 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.419+0000 7f03a3400640 1 --2- 172.21.15.111:0/2783356798 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03a41075e0 0x7f03a419d810 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0394030560 tx=0x7f0394032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:41.935 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.420+0000 7f039be00640 1 -- 172.21.15.111:0/2783356798 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0394037040 con 0x7f03a41075e0 2024-09-15T22:36:41.935 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.420+0000 7f039be00640 1 -- 172.21.15.111:0/2783356798 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0394040070 con 0x7f03a41075e0 2024-09-15T22:36:41.935 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.420+0000 7f03a9200640 1 -- 172.21.15.111:0/2783356798 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03a419dfe0 con 0x7f03a41075e0 2024-09-15T22:36:41.935 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.420+0000 7f039be00640 1 -- 172.21.15.111:0/2783356798 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f039403a6d0 con 0x7f03a41075e0 2024-09-15T22:36:41.935 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.420+0000 7f03a9200640 1 -- 172.21.15.111:0/2783356798 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03a41a0cd0 con 0x7f03a41075e0 2024-09-15T22:36:41.935 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.421+0000 7f039be00640 1 -- 172.21.15.111:0/2783356798 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f039403a940 con 0x7f03a41075e0 2024-09-15T22:36:41.935 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.421+0000 7f03a9200640 1 -- 172.21.15.111:0/2783356798 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03a4107a60 con 0x7f03a41075e0 2024-09-15T22:36:41.935 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.421+0000 7f039be00640 1 --2- 172.21.15.111:0/2783356798 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f03880391b0 0x7f038803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:41.935 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.421+0000 7f039be00640 1 -- 172.21.15.111:0/2783356798 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f039406ca40 con 0x7f03a41075e0 2024-09-15T22:36:41.935 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.422+0000 7f03a2a00640 1 --2- 172.21.15.111:0/2783356798 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f03880391b0 0x7f038803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:41.936 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.422+0000 7f03a2a00640 1 --2- 172.21.15.111:0/2783356798 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f03880391b0 0x7f038803b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f03900098c0 tx=0x7f0390006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:41.936 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.425+0000 7f039be00640 1 -- 172.21.15.111:0/2783356798 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0394032920 con 0x7f03a41075e0 2024-09-15T22:36:41.936 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.619+0000 7f03a9200640 1 -- 172.21.15.111:0/2783356798 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f03a41a12c0 con 0x7f03a41075e0 2024-09-15T22:36:41.936 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.622+0000 7f039be00640 1 -- 172.21.15.111:0/2783356798 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f039406c390 con 0x7f03a41075e0 2024-09-15T22:36:41.936 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.625+0000 7f03a9200640 1 -- 172.21.15.111:0/2783356798 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f03880391b0 msgr2=0x7f038803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:41.936 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.625+0000 7f03a9200640 1 --2- 172.21.15.111:0/2783356798 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f03880391b0 0x7f038803b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f03900098c0 tx=0x7f0390006db0 comp rx=0 tx=0).stop 2024-09-15T22:36:41.936 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.625+0000 7f03a9200640 1 -- 172.21.15.111:0/2783356798 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03a41075e0 msgr2=0x7f03a419d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:41.936 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.625+0000 7f03a9200640 1 --2- 172.21.15.111:0/2783356798 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03a41075e0 0x7f03a419d810 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0394030560 tx=0x7f0394032670 comp rx=0 tx=0).stop 2024-09-15T22:36:41.936 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.625+0000 7f03a9200640 1 -- 172.21.15.111:0/2783356798 shutdown_connections 2024-09-15T22:36:41.936 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.625+0000 7f03a9200640 1 --2- 172.21.15.111:0/2783356798 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f03880391b0 0x7f038803b670 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:41.936 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.625+0000 7f03a9200640 1 --2- 172.21.15.111:0/2783356798 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03a41075e0 0x7f03a419d810 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:41.936 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.625+0000 7f03a9200640 1 -- 172.21.15.111:0/2783356798 >> 172.21.15.111:0/2783356798 conn(0x7f03a4102d30 msgr2=0x7f03a4103930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:41.936 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.625+0000 7f03a9200640 1 -- 172.21.15.111:0/2783356798 shutdown_connections 2024-09-15T22:36:41.936 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:40.626+0000 7f03a9200640 1 -- 172.21.15.111:0/2783356798 wait complete. 2024-09-15T22:36:43.386 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:43 smithi111 ceph-mon[30629]: from='mgr.14116 172.21.15.111:0/472659009' entity='mgr.x' 2024-09-15T22:36:43.386 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:43 smithi111 ceph-mon[30629]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:36:43.386 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:43 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/3789967720' entity='client.admin' 2024-09-15T22:36:43.397 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.478+0000 7fdf4ae00640 1 Processor -- start 2024-09-15T22:36:43.397 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.478+0000 7fdf4ae00640 1 -- start start 2024-09-15T22:36:43.398 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.479+0000 7fdf4ae00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf4407e6e0 0x7fdf4407eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:43.398 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.479+0000 7fdf4ae00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4407f0b0 con 0x7fdf4407e6e0 2024-09-15T22:36:43.398 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.479+0000 7fdf43e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf4407e6e0 0x7fdf4407eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:43.398 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.479+0000 7fdf43e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf4407e6e0 0x7fdf4407eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52366/0 (socket says 172.21.15.111:52366) 2024-09-15T22:36:43.398 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.479+0000 7fdf43e00640 1 -- 172.21.15.111:0/1931137667 learned_addr learned my addr 172.21.15.111:0/1931137667 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:43.398 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.480+0000 7fdf43e00640 1 -- 172.21.15.111:0/1931137667 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4407f930 con 0x7fdf4407e6e0 2024-09-15T22:36:43.398 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.481+0000 7fdf43e00640 1 --2- 172.21.15.111:0/1931137667 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf4407e6e0 0x7fdf4407eae0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009820 tx=0x7fdf3402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e008fa835ae89b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:43.398 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.481+0000 7fdf42a00640 1 -- 172.21.15.111:0/1931137667 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf34037070 con 0x7fdf4407e6e0 2024-09-15T22:36:43.398 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.481+0000 7fdf42a00640 1 -- 172.21.15.111:0/1931137667 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf34032920 con 0x7fdf4407e6e0 2024-09-15T22:36:43.398 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.481+0000 7fdf42a00640 1 -- 172.21.15.111:0/1931137667 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf34030810 con 0x7fdf4407e6e0 2024-09-15T22:36:43.398 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.482+0000 7fdf4ae00640 1 -- 172.21.15.111:0/1931137667 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf4407e6e0 msgr2=0x7fdf4407eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:43.398 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.482+0000 7fdf4ae00640 1 --2- 172.21.15.111:0/1931137667 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf4407e6e0 0x7fdf4407eae0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdf34009820 tx=0x7fdf3402a270 comp rx=0 tx=0).stop 2024-09-15T22:36:43.398 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.482+0000 7fdf4ae00640 1 -- 172.21.15.111:0/1931137667 shutdown_connections 2024-09-15T22:36:43.399 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.482+0000 7fdf4ae00640 1 --2- 172.21.15.111:0/1931137667 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf4407e6e0 0x7fdf4407eae0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:43.399 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.482+0000 7fdf4ae00640 1 -- 172.21.15.111:0/1931137667 >> 172.21.15.111:0/1931137667 conn(0x7fdf4407cf00 msgr2=0x7fdf4407d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:43.399 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.482+0000 7fdf4ae00640 1 -- 172.21.15.111:0/1931137667 shutdown_connections 2024-09-15T22:36:43.399 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.482+0000 7fdf4ae00640 1 -- 172.21.15.111:0/1931137667 wait complete. 2024-09-15T22:36:43.399 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.483+0000 7fdf4ae00640 1 Processor -- start 2024-09-15T22:36:43.399 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.483+0000 7fdf4ae00640 1 -- start start 2024-09-15T22:36:43.399 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.484+0000 7fdf4ae00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf4407e6e0 0x7fdf441a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:43.399 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.484+0000 7fdf4ae00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdf44080cf0 con 0x7fdf4407e6e0 2024-09-15T22:36:43.399 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.484+0000 7fdf43e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf4407e6e0 0x7fdf441a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:43.399 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.484+0000 7fdf43e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf4407e6e0 0x7fdf441a1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52380/0 (socket says 172.21.15.111:52380) 2024-09-15T22:36:43.399 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.484+0000 7fdf43e00640 1 -- 172.21.15.111:0/3789967720 learned_addr learned my addr 172.21.15.111:0/3789967720 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:43.399 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.484+0000 7fdf43e00640 1 -- 172.21.15.111:0/3789967720 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf441a2250 con 0x7fdf4407e6e0 2024-09-15T22:36:43.399 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.485+0000 7fdf43e00640 1 --2- 172.21.15.111:0/3789967720 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf4407e6e0 0x7fdf441a1d10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdf340097f0 tx=0x7fdf340327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:43.399 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.485+0000 7fdf40c00640 1 -- 172.21.15.111:0/3789967720 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf34037040 con 0x7fdf4407e6e0 2024-09-15T22:36:43.400 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.485+0000 7fdf40c00640 1 -- 172.21.15.111:0/3789967720 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf34030cc0 con 0x7fdf4407e6e0 2024-09-15T22:36:43.400 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.485+0000 7fdf40c00640 1 -- 172.21.15.111:0/3789967720 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdf3403a6b0 con 0x7fdf4407e6e0 2024-09-15T22:36:43.400 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.485+0000 7fdf4ae00640 1 -- 172.21.15.111:0/3789967720 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf441a24e0 con 0x7fdf4407e6e0 2024-09-15T22:36:43.400 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.485+0000 7fdf4ae00640 1 -- 172.21.15.111:0/3789967720 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf441a51c0 con 0x7fdf4407e6e0 2024-09-15T22:36:43.400 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.487+0000 7fdf40c00640 1 -- 172.21.15.111:0/3789967720 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fdf3403baf0 con 0x7fdf4407e6e0 2024-09-15T22:36:43.400 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.487+0000 7fdf4ae00640 1 -- 172.21.15.111:0/3789967720 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf4407eae0 con 0x7fdf4407e6e0 2024-09-15T22:36:43.400 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.487+0000 7fdf40c00640 1 --2- 172.21.15.111:0/3789967720 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7fdf28039200 0x7fdf2803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:43.400 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.487+0000 7fdf40c00640 1 -- 172.21.15.111:0/3789967720 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdf3406c870 con 0x7fdf4407e6e0 2024-09-15T22:36:43.400 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.487+0000 7fdf43400640 1 --2- 172.21.15.111:0/3789967720 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7fdf28039200 0x7fdf2803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:43.400 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.488+0000 7fdf43400640 1 --2- 172.21.15.111:0/3789967720 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7fdf28039200 0x7fdf2803b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdf300098c0 tx=0x7fdf30006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:43.400 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.492+0000 7fdf40c00640 1 -- 172.21.15.111:0/3789967720 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdf340371e0 con 0x7fdf4407e6e0 2024-09-15T22:36:43.400 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.686+0000 7fdf4ae00640 1 -- 172.21.15.111:0/3789967720 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fdf441a54e0 con 0x7fdf4407e6e0 2024-09-15T22:36:43.401 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.689+0000 7fdf40c00640 1 -- 172.21.15.111:0/3789967720 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fdf3406c1c0 con 0x7fdf4407e6e0 2024-09-15T22:36:43.401 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.692+0000 7fdf4ae00640 1 -- 172.21.15.111:0/3789967720 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7fdf28039200 msgr2=0x7fdf2803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:43.401 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.692+0000 7fdf4ae00640 1 --2- 172.21.15.111:0/3789967720 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7fdf28039200 0x7fdf2803b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdf300098c0 tx=0x7fdf30006db0 comp rx=0 tx=0).stop 2024-09-15T22:36:43.401 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.692+0000 7fdf4ae00640 1 -- 172.21.15.111:0/3789967720 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf4407e6e0 msgr2=0x7fdf441a1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:43.401 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.692+0000 7fdf4ae00640 1 --2- 172.21.15.111:0/3789967720 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf4407e6e0 0x7fdf441a1d10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdf340097f0 tx=0x7fdf340327b0 comp rx=0 tx=0).stop 2024-09-15T22:36:43.401 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.692+0000 7fdf4ae00640 1 -- 172.21.15.111:0/3789967720 shutdown_connections 2024-09-15T22:36:43.401 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.692+0000 7fdf4ae00640 1 --2- 172.21.15.111:0/3789967720 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7fdf28039200 0x7fdf2803b6c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:43.401 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.692+0000 7fdf4ae00640 1 --2- 172.21.15.111:0/3789967720 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdf4407e6e0 0x7fdf441a1d10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:43.401 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.692+0000 7fdf4ae00640 1 -- 172.21.15.111:0/3789967720 >> 172.21.15.111:0/3789967720 conn(0x7fdf4407cf00 msgr2=0x7fdf441088a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:43.401 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.692+0000 7fdf4ae00640 1 -- 172.21.15.111:0/3789967720 shutdown_connections 2024-09-15T22:36:43.402 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:42.693+0000 7fdf4ae00640 1 -- 172.21.15.111:0/3789967720 wait complete. 2024-09-15T22:36:43.402 INFO:teuthology.orchestra.run.smithi111.stdout:Enabling the dashboard module... 2024-09-15T22:36:44.555 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:44 smithi111 ceph-mon[30629]: from='mgr.14116 172.21.15.111:0/472659009' entity='mgr.x' 2024-09-15T22:36:44.556 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:44 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/140643030' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-15T22:36:44.916 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:44 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: ignoring --setuser ceph since I am not root 2024-09-15T22:36:44.916 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:44 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: ignoring --setgroup ceph since I am not root 2024-09-15T22:36:44.916 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:44 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:44.712+0000 7f2a0f410040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T22:36:44.916 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:44 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:44.781+0000 7f2a0f410040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T22:36:45.225 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:44 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:44.967+0000 7f2a0f410040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T22:36:45.225 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:45 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:45.049+0000 7f2a0f410040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T22:36:45.225 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:45 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:45.223+0000 7f2a0f410040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:36:45.475 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:45 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/140643030' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-15T22:36:45.475 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:45 smithi111 ceph-mon[30629]: mgrmap e9: x(active, since 21s) 2024-09-15T22:36:45.476 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:45 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:45.389+0000 7f2a0f410040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T22:36:46.041 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:45 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:45.759+0000 7f2a0f410040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T22:36:46.041 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:45 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:45.831+0000 7f2a0f410040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T22:36:46.041 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:46 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:46.039+0000 7f2a0f410040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T22:36:46.056 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.001+0000 7fa991e00640 1 Processor -- start 2024-09-15T22:36:46.061 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.001+0000 7fa991e00640 1 -- start start 2024-09-15T22:36:46.061 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.001+0000 7fa991e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa98c07dd90 0x7fa98c07c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:46.061 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.001+0000 7fa991e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c07c780 con 0x7fa98c07dd90 2024-09-15T22:36:46.061 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.002+0000 7fa990a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa98c07dd90 0x7fa98c07c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:46.061 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.002+0000 7fa990a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa98c07dd90 0x7fa98c07c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52388/0 (socket says 172.21.15.111:52388) 2024-09-15T22:36:46.061 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.002+0000 7fa990a00640 1 -- 172.21.15.111:0/1190946937 learned_addr learned my addr 172.21.15.111:0/1190946937 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:46.061 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.002+0000 7fa990a00640 1 -- 172.21.15.111:0/1190946937 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa98c07c900 con 0x7fa98c07dd90 2024-09-15T22:36:46.061 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.003+0000 7fa990a00640 1 --2- 172.21.15.111:0/1190946937 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa98c07dd90 0x7fa98c07c240 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa980009820 tx=0x7fa98002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc1fd8ef4d5d443 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:46.062 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.003+0000 7fa98b400640 1 -- 172.21.15.111:0/1190946937 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa980037070 con 0x7fa98c07dd90 2024-09-15T22:36:46.062 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.004+0000 7fa98b400640 1 -- 172.21.15.111:0/1190946937 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa980032920 con 0x7fa98c07dd90 2024-09-15T22:36:46.062 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.004+0000 7fa98b400640 1 -- 172.21.15.111:0/1190946937 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa980030810 con 0x7fa98c07dd90 2024-09-15T22:36:46.062 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.004+0000 7fa991e00640 1 -- 172.21.15.111:0/1190946937 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa98c07dd90 msgr2=0x7fa98c07c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:46.062 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.004+0000 7fa991e00640 1 --2- 172.21.15.111:0/1190946937 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa98c07dd90 0x7fa98c07c240 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa980009820 tx=0x7fa98002a270 comp rx=0 tx=0).stop 2024-09-15T22:36:46.062 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.004+0000 7fa991e00640 1 -- 172.21.15.111:0/1190946937 shutdown_connections 2024-09-15T22:36:46.062 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.004+0000 7fa991e00640 1 --2- 172.21.15.111:0/1190946937 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa98c07dd90 0x7fa98c07c240 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:46.062 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.004+0000 7fa991e00640 1 -- 172.21.15.111:0/1190946937 >> 172.21.15.111:0/1190946937 conn(0x7fa98c102d30 msgr2=0x7fa98c1051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:46.062 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.004+0000 7fa991e00640 1 -- 172.21.15.111:0/1190946937 shutdown_connections 2024-09-15T22:36:46.062 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.005+0000 7fa991e00640 1 -- 172.21.15.111:0/1190946937 wait complete. 2024-09-15T22:36:46.062 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.005+0000 7fa991e00640 1 Processor -- start 2024-09-15T22:36:46.062 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.006+0000 7fa991e00640 1 -- start start 2024-09-15T22:36:46.062 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.006+0000 7fa991e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa98c07dd90 0x7fa98c19d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:46.062 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.006+0000 7fa991e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c10dbc0 con 0x7fa98c07dd90 2024-09-15T22:36:46.063 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.006+0000 7fa990a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa98c07dd90 0x7fa98c19d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:46.063 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.006+0000 7fa990a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa98c07dd90 0x7fa98c19d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52392/0 (socket says 172.21.15.111:52392) 2024-09-15T22:36:46.063 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.006+0000 7fa990a00640 1 -- 172.21.15.111:0/140643030 learned_addr learned my addr 172.21.15.111:0/140643030 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:46.063 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.006+0000 7fa990a00640 1 -- 172.21.15.111:0/140643030 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa98c19dcf0 con 0x7fa98c07dd90 2024-09-15T22:36:46.063 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.007+0000 7fa990a00640 1 --2- 172.21.15.111:0/140643030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa98c07dd90 0x7fa98c19d7b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa9800097f0 tx=0x7fa9800327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:46.063 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.007+0000 7fa989600640 1 -- 172.21.15.111:0/140643030 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa980037040 con 0x7fa98c07dd90 2024-09-15T22:36:46.063 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.007+0000 7fa989600640 1 -- 172.21.15.111:0/140643030 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa980030c40 con 0x7fa98c07dd90 2024-09-15T22:36:46.063 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.007+0000 7fa989600640 1 -- 172.21.15.111:0/140643030 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa98003a5d0 con 0x7fa98c07dd90 2024-09-15T22:36:46.063 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.007+0000 7fa991e00640 1 -- 172.21.15.111:0/140643030 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa98c19df80 con 0x7fa98c07dd90 2024-09-15T22:36:46.063 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.007+0000 7fa991e00640 1 -- 172.21.15.111:0/140643030 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa98c1a0c70 con 0x7fa98c07dd90 2024-09-15T22:36:46.063 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.008+0000 7fa991e00640 1 -- 172.21.15.111:0/140643030 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa98c07c2c0 con 0x7fa98c07dd90 2024-09-15T22:36:46.063 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.008+0000 7fa989600640 1 -- 172.21.15.111:0/140643030 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fa98003b3f0 con 0x7fa98c07dd90 2024-09-15T22:36:46.063 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.009+0000 7fa989600640 1 --2- 172.21.15.111:0/140643030 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7fa974039200 0x7fa97403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:46.064 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.009+0000 7fa989600640 1 -- 172.21.15.111:0/140643030 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa98006c850 con 0x7fa98c07dd90 2024-09-15T22:36:46.064 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.010+0000 7fa98be00640 1 --2- 172.21.15.111:0/140643030 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7fa974039200 0x7fa97403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:46.064 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.011+0000 7fa98be00640 1 --2- 172.21.15.111:0/140643030 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7fa974039200 0x7fa97403b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa9780098c0 tx=0x7fa978006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:46.064 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.015+0000 7fa989600640 1 -- 172.21.15.111:0/140643030 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa98003b6e0 con 0x7fa98c07dd90 2024-09-15T22:36:46.064 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.244+0000 7fa991e00640 1 -- 172.21.15.111:0/140643030 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fa98c1a0e40 con 0x7fa98c07dd90 2024-09-15T22:36:46.064 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.468+0000 7fa989600640 1 -- 172.21.15.111:0/140643030 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fa98006c1a0 con 0x7fa98c07dd90 2024-09-15T22:36:46.064 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.470+0000 7fa989600640 1 -- 172.21.15.111:0/140643030 <== mon.0 v2:172.21.15.111:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7fa98003a7c0 con 0x7fa98c07dd90 2024-09-15T22:36:46.064 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.472+0000 7fa991e00640 1 -- 172.21.15.111:0/140643030 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7fa974039200 msgr2=0x7fa97403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:46.064 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.472+0000 7fa991e00640 1 --2- 172.21.15.111:0/140643030 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7fa974039200 0x7fa97403b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa9780098c0 tx=0x7fa978006db0 comp rx=0 tx=0).stop 2024-09-15T22:36:46.064 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.472+0000 7fa991e00640 1 -- 172.21.15.111:0/140643030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa98c07dd90 msgr2=0x7fa98c19d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:46.064 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.472+0000 7fa991e00640 1 --2- 172.21.15.111:0/140643030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa98c07dd90 0x7fa98c19d7b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa9800097f0 tx=0x7fa9800327b0 comp rx=0 tx=0).stop 2024-09-15T22:36:46.064 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.472+0000 7fa991e00640 1 -- 172.21.15.111:0/140643030 shutdown_connections 2024-09-15T22:36:46.065 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.472+0000 7fa991e00640 1 --2- 172.21.15.111:0/140643030 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7fa974039200 0x7fa97403b6c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:46.065 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.472+0000 7fa991e00640 1 --2- 172.21.15.111:0/140643030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa98c07dd90 0x7fa98c19d7b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:46.065 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.472+0000 7fa991e00640 1 -- 172.21.15.111:0/140643030 >> 172.21.15.111:0/140643030 conn(0x7fa98c102d30 msgr2=0x7fa98c103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:46.065 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.472+0000 7fa991e00640 1 -- 172.21.15.111:0/140643030 shutdown_connections 2024-09-15T22:36:46.065 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:44.472+0000 7fa991e00640 1 -- 172.21.15.111:0/140643030 wait complete. 2024-09-15T22:36:47.156 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:46 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/552233365' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-15T22:36:47.415 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:47 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:47.179+0000 7f2a0f410040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T22:36:47.416 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:47 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:47.260+0000 7f2a0f410040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T22:36:47.530 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout { 2024-09-15T22:36:47.536 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-15T22:36:47.537 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-15T22:36:47.537 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-15T22:36:47.537 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-15T22:36:47.537 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout } 2024-09-15T22:36:47.537 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.584+0000 7f222be00640 1 Processor -- start 2024-09-15T22:36:47.537 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.584+0000 7f222be00640 1 -- start start 2024-09-15T22:36:47.537 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.585+0000 7f222be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f222c107fe0 0x7f222c10a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:47.537 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.585+0000 7f222be00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f222c07bdd0 con 0x7f222c107fe0 2024-09-15T22:36:47.537 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.585+0000 7f222aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f222c107fe0 0x7f222c10a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:47.537 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.585+0000 7f222aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f222c107fe0 0x7f222c10a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52422/0 (socket says 172.21.15.111:52422) 2024-09-15T22:36:47.537 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.585+0000 7f222aa00640 1 -- 172.21.15.111:0/4038677182 learned_addr learned my addr 172.21.15.111:0/4038677182 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:47.537 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.585+0000 7f222aa00640 1 -- 172.21.15.111:0/4038677182 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222c10a910 con 0x7f222c107fe0 2024-09-15T22:36:47.537 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.586+0000 7f222aa00640 1 --2- 172.21.15.111:0/4038677182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f222c107fe0 0x7f222c10a3d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f221c009820 tx=0x7f221c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac403a6ea8ae3c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:47.538 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.587+0000 7f222a000640 1 -- 172.21.15.111:0/4038677182 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f221c037070 con 0x7f222c107fe0 2024-09-15T22:36:47.538 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.587+0000 7f222a000640 1 -- 172.21.15.111:0/4038677182 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f221c032920 con 0x7f222c107fe0 2024-09-15T22:36:47.538 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.587+0000 7f222a000640 1 -- 172.21.15.111:0/4038677182 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f221c030810 con 0x7f222c107fe0 2024-09-15T22:36:47.538 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.587+0000 7f222be00640 1 -- 172.21.15.111:0/4038677182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f222c107fe0 msgr2=0x7f222c10a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:47.538 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.587+0000 7f222be00640 1 --2- 172.21.15.111:0/4038677182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f222c107fe0 0x7f222c10a3d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f221c009820 tx=0x7f221c02a270 comp rx=0 tx=0).stop 2024-09-15T22:36:47.538 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.587+0000 7f222be00640 1 -- 172.21.15.111:0/4038677182 shutdown_connections 2024-09-15T22:36:47.538 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.587+0000 7f222be00640 1 --2- 172.21.15.111:0/4038677182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f222c107fe0 0x7f222c10a3d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:47.538 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.587+0000 7f222be00640 1 -- 172.21.15.111:0/4038677182 >> 172.21.15.111:0/4038677182 conn(0x7f222c101bc0 msgr2=0x7f222c103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:47.538 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.588+0000 7f222be00640 1 -- 172.21.15.111:0/4038677182 shutdown_connections 2024-09-15T22:36:47.538 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.588+0000 7f222be00640 1 -- 172.21.15.111:0/4038677182 wait complete. 2024-09-15T22:36:47.538 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.588+0000 7f222be00640 1 Processor -- start 2024-09-15T22:36:47.538 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.589+0000 7f222be00640 1 -- start start 2024-09-15T22:36:47.538 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.589+0000 7f222be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f222c107fe0 0x7f222c107860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:47.539 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.589+0000 7f222be00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f222c10b990 con 0x7f222c107fe0 2024-09-15T22:36:47.539 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.589+0000 7f222aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f222c107fe0 0x7f222c107860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:47.539 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.589+0000 7f222aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f222c107fe0 0x7f222c107860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52428/0 (socket says 172.21.15.111:52428) 2024-09-15T22:36:47.539 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.589+0000 7f222aa00640 1 -- 172.21.15.111:0/552233365 learned_addr learned my addr 172.21.15.111:0/552233365 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:47.539 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.589+0000 7f222aa00640 1 -- 172.21.15.111:0/552233365 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222c107da0 con 0x7f222c107fe0 2024-09-15T22:36:47.539 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.590+0000 7f222aa00640 1 --2- 172.21.15.111:0/552233365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f222c107fe0 0x7f222c107860 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f221c0097f0 tx=0x7f221c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:47.539 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.590+0000 7f2223400640 1 -- 172.21.15.111:0/552233365 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f221c037040 con 0x7f222c107fe0 2024-09-15T22:36:47.539 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.590+0000 7f2223400640 1 -- 172.21.15.111:0/552233365 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f221c030c40 con 0x7f222c107fe0 2024-09-15T22:36:47.539 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.590+0000 7f222be00640 1 -- 172.21.15.111:0/552233365 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f222c105f90 con 0x7f222c107fe0 2024-09-15T22:36:47.539 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.590+0000 7f2223400640 1 -- 172.21.15.111:0/552233365 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f221c03a5d0 con 0x7f222c107fe0 2024-09-15T22:36:47.539 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.590+0000 7f222be00640 1 -- 172.21.15.111:0/552233365 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f222c1064d0 con 0x7f222c107fe0 2024-09-15T22:36:47.539 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.591+0000 7f2223400640 1 -- 172.21.15.111:0/552233365 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f221c03b3f0 con 0x7f222c107fe0 2024-09-15T22:36:47.539 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.591+0000 7f222be00640 1 -- 172.21.15.111:0/552233365 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f222c108440 con 0x7f222c107fe0 2024-09-15T22:36:47.540 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.592+0000 7f2223400640 1 --2- 172.21.15.111:0/552233365 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f2210039260 0x7f221003b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:47.540 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.592+0000 7f2223400640 1 -- 172.21.15.111:0/552233365 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f221c06c860 con 0x7f222c107fe0 2024-09-15T22:36:47.540 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.592+0000 7f2223e00640 1 -- 172.21.15.111:0/552233365 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f2210039260 msgr2=0x7f221003b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/2107775445 2024-09-15T22:36:47.540 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.592+0000 7f2223e00640 1 --2- 172.21.15.111:0/552233365 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f2210039260 0x7f221003b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:36:47.540 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.596+0000 7f2223400640 1 -- 172.21.15.111:0/552233365 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f221c0371e0 con 0x7f222c107fe0 2024-09-15T22:36:47.540 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.793+0000 7f2223e00640 1 -- 172.21.15.111:0/552233365 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f2210039260 msgr2=0x7f221003b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/2107775445 2024-09-15T22:36:47.540 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.793+0000 7f2223e00640 1 --2- 172.21.15.111:0/552233365 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f2210039260 0x7f221003b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T22:36:47.540 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.824+0000 7f222be00640 1 -- 172.21.15.111:0/552233365 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f222c106750 con 0x7f222c107fe0 2024-09-15T22:36:47.540 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.825+0000 7f2223400640 1 -- 172.21.15.111:0/552233365 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f221c03a770 con 0x7f222c107fe0 2024-09-15T22:36:47.540 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.827+0000 7f222be00640 1 -- 172.21.15.111:0/552233365 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f2210039260 msgr2=0x7f221003b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:36:47.540 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.827+0000 7f222be00640 1 --2- 172.21.15.111:0/552233365 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f2210039260 0x7f221003b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:47.540 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.827+0000 7f222be00640 1 -- 172.21.15.111:0/552233365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f222c107fe0 msgr2=0x7f222c107860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:47.540 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.827+0000 7f222be00640 1 --2- 172.21.15.111:0/552233365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f222c107fe0 0x7f222c107860 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f221c0097f0 tx=0x7f221c0327b0 comp rx=0 tx=0).stop 2024-09-15T22:36:47.541 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.827+0000 7f222be00640 1 -- 172.21.15.111:0/552233365 shutdown_connections 2024-09-15T22:36:47.541 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.827+0000 7f222be00640 1 --2- 172.21.15.111:0/552233365 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f2210039260 0x7f221003b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:47.541 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.827+0000 7f222be00640 1 --2- 172.21.15.111:0/552233365 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f222c107fe0 0x7f222c107860 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:47.541 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.827+0000 7f222be00640 1 -- 172.21.15.111:0/552233365 >> 172.21.15.111:0/552233365 conn(0x7f222c101bc0 msgr2=0x7f222c1027c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:47.541 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.828+0000 7f222be00640 1 -- 172.21.15.111:0/552233365 shutdown_connections 2024-09-15T22:36:47.541 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:46.828+0000 7f222be00640 1 -- 172.21.15.111:0/552233365 wait complete. 2024-09-15T22:36:47.541 INFO:teuthology.orchestra.run.smithi111.stdout:Waiting for the mgr to restart... 2024-09-15T22:36:47.541 INFO:teuthology.orchestra.run.smithi111.stdout:Waiting for mgr epoch 9... 2024-09-15T22:36:48.336 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:48 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:48.197+0000 7f2a0f410040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T22:36:48.336 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:48 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:48.334+0000 7f2a0f410040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T22:36:48.665 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:48 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:48.498+0000 7f2a0f410040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:36:48.666 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:48 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:48.569+0000 7f2a0f410040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T22:36:49.071 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:48 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:48.740+0000 7f2a0f410040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T22:36:49.072 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:48 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:48.822+0000 7f2a0f410040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T22:36:49.072 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:48 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:48.892+0000 7f2a0f410040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T22:36:49.072 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:49 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:49.070+0000 7f2a0f410040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T22:36:49.389 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:49 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:49.145+0000 7f2a0f410040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T22:36:49.389 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:49 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:49.223+0000 7f2a0f410040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T22:36:49.390 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:49 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:49.318+0000 7f2a0f410040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T22:36:49.390 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:49 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:49.387+0000 7f2a0f410040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T22:36:49.665 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:49 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:49.457+0000 7f2a0f410040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T22:36:50.666 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:50 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:50.354+0000 7f2a0f410040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T22:36:50.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:50 smithi111 ceph-mon[30629]: Active manager daemon x restarted 2024-09-15T22:36:50.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:50 smithi111 ceph-mon[30629]: Activating manager daemon x 2024-09-15T22:36:50.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:50 smithi111 ceph-mon[30629]: osdmap e3: 0 total, 0 up, 0 in 2024-09-15T22:36:50.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:50 smithi111 ceph-mon[30629]: mgrmap e10: x(active, starting, since 0.00399703s) 2024-09-15T22:36:50.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:36:50.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-15T22:36:50.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-15T22:36:50.667 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-15T22:36:50.667 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-15T22:36:50.667 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:50 smithi111 ceph-mon[30629]: Manager daemon x is now available 2024-09-15T22:36:50.667 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:36:51.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:51 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-15T22:36:51.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:51 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-15T22:36:51.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:51 smithi111 ceph-mon[30629]: mgrmap e11: x(active, since 1.00705s) 2024-09-15T22:36:51.666 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:51 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:51.327+0000 7f296cc00640 -1 log_channel(cephadm) log [ERR] : [15/Sep/2024:22:36:51] ENGINE Error in HTTPServer.serve 2024-09-15T22:36:51.666 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:51 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: Traceback (most recent call last): 2024-09-15T22:36:51.666 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:51 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T22:36:51.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:51 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: self._connections.run(self.expiration_interval) 2024-09-15T22:36:51.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:51 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T22:36:51.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:51 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: self._run(expiration_interval) 2024-09-15T22:36:51.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:51 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T22:36:51.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:51 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T22:36:51.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:51 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T22:36:51.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:51 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T22:36:51.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:51 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T22:36:51.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:51 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: s = self.context.wrap_socket( 2024-09-15T22:36:51.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:51 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T22:36:51.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:51 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: return self.sslsocket_class._create( 2024-09-15T22:36:51.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:51 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T22:36:51.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:51 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: self.do_handshake() 2024-09-15T22:36:51.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:51 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T22:36:51.667 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:51 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: self._sslobj.do_handshake() 2024-09-15T22:36:51.668 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:51 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T22:36:51.668 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:36:51 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:36:52.591 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:52 smithi111 ceph-mon[30629]: [15/Sep/2024:22:36:51] ENGINE Bus STARTING 2024-09-15T22:36:52.591 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:52 smithi111 ceph-mon[30629]: [15/Sep/2024:22:36:51] ENGINE Serving on https://172.21.15.111:7150 2024-09-15T22:36:52.591 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:52 smithi111 ceph-mon[30629]: [15/Sep/2024:22:36:51] ENGINE Error in HTTPServer.serve 2024-09-15T22:36:52.591 INFO:journalctl@ceph.mon.a.smithi111.stdout: Traceback (most recent call last): 2024-09-15T22:36:52.591 INFO:journalctl@ceph.mon.a.smithi111.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T22:36:52.591 INFO:journalctl@ceph.mon.a.smithi111.stdout: self._connections.run(self.expiration_interval) 2024-09-15T22:36:52.591 INFO:journalctl@ceph.mon.a.smithi111.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T22:36:52.591 INFO:journalctl@ceph.mon.a.smithi111.stdout: self._run(expiration_interval) 2024-09-15T22:36:52.591 INFO:journalctl@ceph.mon.a.smithi111.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T22:36:52.591 INFO:journalctl@ceph.mon.a.smithi111.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T22:36:52.591 INFO:journalctl@ceph.mon.a.smithi111.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T22:36:52.591 INFO:journalctl@ceph.mon.a.smithi111.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T22:36:52.592 INFO:journalctl@ceph.mon.a.smithi111.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T22:36:52.592 INFO:journalctl@ceph.mon.a.smithi111.stdout: s = self.context.wrap_socket( 2024-09-15T22:36:52.592 INFO:journalctl@ceph.mon.a.smithi111.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T22:36:52.592 INFO:journalctl@ceph.mon.a.smithi111.stdout: return self.sslsocket_class._create( 2024-09-15T22:36:52.592 INFO:journalctl@ceph.mon.a.smithi111.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T22:36:52.592 INFO:journalctl@ceph.mon.a.smithi111.stdout: self.do_handshake() 2024-09-15T22:36:52.592 INFO:journalctl@ceph.mon.a.smithi111.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T22:36:52.592 INFO:journalctl@ceph.mon.a.smithi111.stdout: self._sslobj.do_handshake() 2024-09-15T22:36:52.592 INFO:journalctl@ceph.mon.a.smithi111.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T22:36:52.592 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:52 smithi111 ceph-mon[30629]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-15T22:36:52.592 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:52 smithi111 ceph-mon[30629]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-15T22:36:52.592 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:52 smithi111 ceph-mon[30629]: [15/Sep/2024:22:36:51] ENGINE Serving on http://172.21.15.111:8765 2024-09-15T22:36:52.592 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:52 smithi111 ceph-mon[30629]: [15/Sep/2024:22:36:51] ENGINE Bus STARTED 2024-09-15T22:36:52.592 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:52 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:36:52.604 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout { 2024-09-15T22:36:52.605 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-15T22:36:52.605 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-15T22:36:52.605 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout } 2024-09-15T22:36:52.605 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.075+0000 7f3236200640 1 Processor -- start 2024-09-15T22:36:52.605 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.075+0000 7f3236200640 1 -- start start 2024-09-15T22:36:52.605 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.076+0000 7f3236200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32301099c0 0x7f3230109dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:52.605 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.076+0000 7f3236200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f323010a300 con 0x7f32301099c0 2024-09-15T22:36:52.605 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.076+0000 7f322f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32301099c0 0x7f3230109dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:52.605 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.076+0000 7f322f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32301099c0 0x7f3230109dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52430/0 (socket says 172.21.15.111:52430) 2024-09-15T22:36:52.605 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.076+0000 7f322f400640 1 -- 172.21.15.111:0/1638067543 learned_addr learned my addr 172.21.15.111:0/1638067543 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:52.605 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.076+0000 7f322f400640 1 -- 172.21.15.111:0/1638067543 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f323010a480 con 0x7f32301099c0 2024-09-15T22:36:52.606 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.077+0000 7f322f400640 1 --2- 172.21.15.111:0/1638067543 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32301099c0 0x7f3230109dc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3220009820 tx=0x7f322002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3247c0bb405e5f93 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:52.606 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.078+0000 7f322e000640 1 -- 172.21.15.111:0/1638067543 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3220037070 con 0x7f32301099c0 2024-09-15T22:36:52.606 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.078+0000 7f322e000640 1 -- 172.21.15.111:0/1638067543 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3220032920 con 0x7f32301099c0 2024-09-15T22:36:52.606 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.078+0000 7f322e000640 1 -- 172.21.15.111:0/1638067543 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3220030810 con 0x7f32301099c0 2024-09-15T22:36:52.606 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.078+0000 7f3236200640 1 -- 172.21.15.111:0/1638067543 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32301099c0 msgr2=0x7f3230109dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:52.606 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.078+0000 7f3236200640 1 --2- 172.21.15.111:0/1638067543 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32301099c0 0x7f3230109dc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3220009820 tx=0x7f322002a270 comp rx=0 tx=0).stop 2024-09-15T22:36:52.606 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.078+0000 7f3236200640 1 -- 172.21.15.111:0/1638067543 shutdown_connections 2024-09-15T22:36:52.606 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.078+0000 7f3236200640 1 --2- 172.21.15.111:0/1638067543 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32301099c0 0x7f3230109dc0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:52.606 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.079+0000 7f3236200640 1 -- 172.21.15.111:0/1638067543 >> 172.21.15.111:0/1638067543 conn(0x7f323007d0a0 msgr2=0x7f323007d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:52.606 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.079+0000 7f3236200640 1 -- 172.21.15.111:0/1638067543 shutdown_connections 2024-09-15T22:36:52.606 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.079+0000 7f3236200640 1 -- 172.21.15.111:0/1638067543 wait complete. 2024-09-15T22:36:52.607 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.080+0000 7f3236200640 1 Processor -- start 2024-09-15T22:36:52.607 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.080+0000 7f3236200640 1 -- start start 2024-09-15T22:36:52.607 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.080+0000 7f3236200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32301099c0 0x7f323019d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:52.607 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.080+0000 7f3236200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f323010dde0 con 0x7f32301099c0 2024-09-15T22:36:52.607 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.080+0000 7f322f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32301099c0 0x7f323019d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:52.607 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.080+0000 7f322f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32301099c0 0x7f323019d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52446/0 (socket says 172.21.15.111:52446) 2024-09-15T22:36:52.607 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.080+0000 7f322f400640 1 -- 172.21.15.111:0/2350486998 learned_addr learned my addr 172.21.15.111:0/2350486998 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:52.607 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.081+0000 7f322f400640 1 -- 172.21.15.111:0/2350486998 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f323019dec0 con 0x7f32301099c0 2024-09-15T22:36:52.607 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.081+0000 7f322f400640 1 --2- 172.21.15.111:0/2350486998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32301099c0 0x7f323019d980 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f32200097f0 tx=0x7f32200327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:52.607 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.081+0000 7f3227e00640 1 -- 172.21.15.111:0/2350486998 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3220037040 con 0x7f32301099c0 2024-09-15T22:36:52.607 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.081+0000 7f3236200640 1 -- 172.21.15.111:0/2350486998 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f323019e150 con 0x7f32301099c0 2024-09-15T22:36:52.607 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.081+0000 7f3227e00640 1 -- 172.21.15.111:0/2350486998 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3220030c40 con 0x7f32301099c0 2024-09-15T22:36:52.607 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.081+0000 7f3227e00640 1 -- 172.21.15.111:0/2350486998 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f322003a5d0 con 0x7f32301099c0 2024-09-15T22:36:52.607 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.081+0000 7f3236200640 1 -- 172.21.15.111:0/2350486998 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32301a0e40 con 0x7f32301099c0 2024-09-15T22:36:52.607 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.083+0000 7f3227e00640 1 -- 172.21.15.111:0/2350486998 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f322003b3f0 con 0x7f32301099c0 2024-09-15T22:36:52.608 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.083+0000 7f3227e00640 1 --2- 172.21.15.111:0/2350486998 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f3214039200 0x7f321403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:52.608 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.083+0000 7f3227e00640 1 -- 172.21.15.111:0/2350486998 --> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f321403bdd0 con 0x7f3214039200 2024-09-15T22:36:52.608 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.083+0000 7f3227e00640 1 -- 172.21.15.111:0/2350486998 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f322006c880 con 0x7f32301099c0 2024-09-15T22:36:52.608 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.083+0000 7f322ea00640 1 -- 172.21.15.111:0/2350486998 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f3214039200 msgr2=0x7f321403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/2107775445 2024-09-15T22:36:52.608 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.083+0000 7f322ea00640 1 --2- 172.21.15.111:0/2350486998 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f3214039200 0x7f321403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:36:52.608 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.284+0000 7f322ea00640 1 -- 172.21.15.111:0/2350486998 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f3214039200 msgr2=0x7f321403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/2107775445 2024-09-15T22:36:52.608 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.284+0000 7f322ea00640 1 --2- 172.21.15.111:0/2350486998 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f3214039200 0x7f321403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T22:36:52.608 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.685+0000 7f322ea00640 1 -- 172.21.15.111:0/2350486998 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f3214039200 msgr2=0x7f321403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/2107775445 2024-09-15T22:36:52.608 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:48.685+0000 7f322ea00640 1 --2- 172.21.15.111:0/2350486998 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f3214039200 0x7f321403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T22:36:52.608 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:49.486+0000 7f322ea00640 1 -- 172.21.15.111:0/2350486998 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f3214039200 msgr2=0x7f321403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.111:6800/2107775445 2024-09-15T22:36:52.608 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:49.486+0000 7f322ea00640 1 --2- 172.21.15.111:0/2350486998 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f3214039200 0x7f321403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T22:36:52.608 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:50.359+0000 7f3227e00640 1 -- 172.21.15.111:0/2350486998 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7f3220041d10 con 0x7f32301099c0 2024-09-15T22:36:52.608 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:50.359+0000 7f3227e00640 1 -- 172.21.15.111:0/2350486998 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f3214039200 msgr2=0x7f321403b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:36:52.609 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:50.359+0000 7f3227e00640 1 --2- 172.21.15.111:0/2350486998 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f3214039200 0x7f321403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:52.609 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:51.360+0000 7f3227e00640 1 -- 172.21.15.111:0/2350486998 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7f3220042300 con 0x7f32301099c0 2024-09-15T22:36:52.609 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:51.361+0000 7f3227e00640 1 --2- 172.21.15.111:0/2350486998 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f321403ccc0 0x7f321403f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:52.609 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:51.361+0000 7f3227e00640 1 -- 172.21.15.111:0/2350486998 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3220047e50 con 0x7f321403ccc0 2024-09-15T22:36:52.609 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:51.361+0000 7f322ea00640 1 --2- 172.21.15.111:0/2350486998 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f321403ccc0 0x7f321403f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:52.609 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:51.362+0000 7f322ea00640 1 --2- 172.21.15.111:0/2350486998 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f321403ccc0 0x7f321403f0b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3218003980 tx=0x7f3218006e10 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:52.609 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:51.363+0000 7f3227e00640 1 -- 172.21.15.111:0/2350486998 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f3220047e50 con 0x7f321403ccc0 2024-09-15T22:36:52.609 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:51.370+0000 7f3236200640 1 -- 172.21.15.111:0/2350486998 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f3230109e40 con 0x7f321403ccc0 2024-09-15T22:36:52.609 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:51.371+0000 7f3227e00640 1 -- 172.21.15.111:0/2350486998 <== mgr.14148 v2:172.21.15.111:6800/3125866964 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f3230109e40 con 0x7f321403ccc0 2024-09-15T22:36:52.609 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:51.371+0000 7f3236200640 1 -- 172.21.15.111:0/2350486998 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f321403ccc0 msgr2=0x7f321403f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:52.609 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:51.371+0000 7f3236200640 1 --2- 172.21.15.111:0/2350486998 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f321403ccc0 0x7f321403f0b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3218003980 tx=0x7f3218006e10 comp rx=0 tx=0).stop 2024-09-15T22:36:52.609 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:51.371+0000 7f3236200640 1 -- 172.21.15.111:0/2350486998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32301099c0 msgr2=0x7f323019d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:52.609 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:51.371+0000 7f3236200640 1 --2- 172.21.15.111:0/2350486998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32301099c0 0x7f323019d980 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f32200097f0 tx=0x7f32200327b0 comp rx=0 tx=0).stop 2024-09-15T22:36:52.609 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:51.371+0000 7f3236200640 1 -- 172.21.15.111:0/2350486998 shutdown_connections 2024-09-15T22:36:52.610 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:51.371+0000 7f3236200640 1 --2- 172.21.15.111:0/2350486998 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f321403ccc0 0x7f321403f0b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:52.610 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:51.371+0000 7f3236200640 1 --2- 172.21.15.111:0/2350486998 >> [v2:172.21.15.111:6800/2107775445,v1:172.21.15.111:6801/2107775445] conn(0x7f3214039200 0x7f321403b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:52.610 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:51.371+0000 7f3236200640 1 --2- 172.21.15.111:0/2350486998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f32301099c0 0x7f323019d980 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:52.610 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:51.371+0000 7f3236200640 1 -- 172.21.15.111:0/2350486998 >> 172.21.15.111:0/2350486998 conn(0x7f323007d0a0 msgr2=0x7f3230107520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:52.610 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:51.371+0000 7f3236200640 1 -- 172.21.15.111:0/2350486998 shutdown_connections 2024-09-15T22:36:52.610 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:51.371+0000 7f3236200640 1 -- 172.21.15.111:0/2350486998 wait complete. 2024-09-15T22:36:52.610 INFO:teuthology.orchestra.run.smithi111.stdout:mgr epoch 9 is available 2024-09-15T22:36:52.610 INFO:teuthology.orchestra.run.smithi111.stdout:Generating a dashboard self-signed certificate... 2024-09-15T22:36:54.040 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-15T22:36:54.040 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.058+0000 7f88b3a00640 1 Processor -- start 2024-09-15T22:36:54.040 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.059+0000 7f88b3a00640 1 -- start start 2024-09-15T22:36:54.040 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.059+0000 7f88b3a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88ac109930 0x7f88ac109d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:54.040 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.059+0000 7f88b3a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac10a300 con 0x7f88ac109930 2024-09-15T22:36:54.040 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.059+0000 7f88b1000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88ac109930 0x7f88ac109d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:54.040 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.059+0000 7f88b1000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88ac109930 0x7f88ac109d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53782/0 (socket says 172.21.15.111:53782) 2024-09-15T22:36:54.040 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.059+0000 7f88b1000640 1 -- 172.21.15.111:0/1745942703 learned_addr learned my addr 172.21.15.111:0/1745942703 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:54.040 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.060+0000 7f88b1000640 1 -- 172.21.15.111:0/1745942703 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ac10ab20 con 0x7f88ac109930 2024-09-15T22:36:54.041 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.061+0000 7f88b1000640 1 --2- 172.21.15.111:0/1745942703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88ac109930 0x7f88ac109d30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009820 tx=0x7f88a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bfeb5b0ce9b2b11 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:54.041 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.061+0000 7f88ab400640 1 -- 172.21.15.111:0/1745942703 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88a0037070 con 0x7f88ac109930 2024-09-15T22:36:54.041 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.061+0000 7f88ab400640 1 -- 172.21.15.111:0/1745942703 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88a0032920 con 0x7f88ac109930 2024-09-15T22:36:54.041 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.061+0000 7f88ab400640 1 -- 172.21.15.111:0/1745942703 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88a0030810 con 0x7f88ac109930 2024-09-15T22:36:54.041 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.062+0000 7f88b3a00640 1 -- 172.21.15.111:0/1745942703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88ac109930 msgr2=0x7f88ac109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:54.041 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.062+0000 7f88b3a00640 1 --2- 172.21.15.111:0/1745942703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88ac109930 0x7f88ac109d30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009820 tx=0x7f88a002a270 comp rx=0 tx=0).stop 2024-09-15T22:36:54.041 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.062+0000 7f88b3a00640 1 -- 172.21.15.111:0/1745942703 shutdown_connections 2024-09-15T22:36:54.041 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.062+0000 7f88b3a00640 1 --2- 172.21.15.111:0/1745942703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88ac109930 0x7f88ac109d30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:54.041 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.062+0000 7f88b3a00640 1 -- 172.21.15.111:0/1745942703 >> 172.21.15.111:0/1745942703 conn(0x7f88ac07cfa0 msgr2=0x7f88ac07d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:54.041 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.062+0000 7f88b3a00640 1 -- 172.21.15.111:0/1745942703 shutdown_connections 2024-09-15T22:36:54.041 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.062+0000 7f88b3a00640 1 -- 172.21.15.111:0/1745942703 wait complete. 2024-09-15T22:36:54.041 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.063+0000 7f88b3a00640 1 Processor -- start 2024-09-15T22:36:54.042 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.063+0000 7f88b3a00640 1 -- start start 2024-09-15T22:36:54.042 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.063+0000 7f88b3a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88ac109930 0x7f88ac081b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:54.042 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.063+0000 7f88b3a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac10ded0 con 0x7f88ac109930 2024-09-15T22:36:54.042 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.064+0000 7f88b1000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88ac109930 0x7f88ac081b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:54.042 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.064+0000 7f88b1000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88ac109930 0x7f88ac081b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53794/0 (socket says 172.21.15.111:53794) 2024-09-15T22:36:54.042 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.064+0000 7f88b1000640 1 -- 172.21.15.111:0/1214910735 learned_addr learned my addr 172.21.15.111:0/1214910735 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:54.042 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.064+0000 7f88b1000640 1 -- 172.21.15.111:0/1214910735 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ac082040 con 0x7f88ac109930 2024-09-15T22:36:54.042 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.064+0000 7f88b1000640 1 --2- 172.21.15.111:0/1214910735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88ac109930 0x7f88ac081b00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f88a00097f0 tx=0x7f88a00327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:54.042 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.065+0000 7f88a9600640 1 -- 172.21.15.111:0/1214910735 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88a0037040 con 0x7f88ac109930 2024-09-15T22:36:54.042 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.065+0000 7f88a9600640 1 -- 172.21.15.111:0/1214910735 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88a0030cc0 con 0x7f88ac109930 2024-09-15T22:36:54.042 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.065+0000 7f88a9600640 1 -- 172.21.15.111:0/1214910735 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88a003a640 con 0x7f88ac109930 2024-09-15T22:36:54.042 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.065+0000 7f88b3a00640 1 -- 172.21.15.111:0/1214910735 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88ac0822d0 con 0x7f88ac109930 2024-09-15T22:36:54.042 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.065+0000 7f88b3a00640 1 -- 172.21.15.111:0/1214910735 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88ac07e890 con 0x7f88ac109930 2024-09-15T22:36:54.042 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.066+0000 7f88a9600640 1 -- 172.21.15.111:0/1214910735 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7f88a003b3f0 con 0x7f88ac109930 2024-09-15T22:36:54.043 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.066+0000 7f88b3a00640 1 -- 172.21.15.111:0/1214910735 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88ac0824a0 con 0x7f88ac109930 2024-09-15T22:36:54.043 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.066+0000 7f88a9600640 1 --2- 172.21.15.111:0/1214910735 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8894039140 0x7f889403b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:54.043 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.066+0000 7f88a9600640 1 -- 172.21.15.111:0/1214910735 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f88a006c770 con 0x7f88ac109930 2024-09-15T22:36:54.043 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.067+0000 7f88abe00640 1 --2- 172.21.15.111:0/1214910735 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8894039140 0x7f889403b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:54.043 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.067+0000 7f88abe00640 1 --2- 172.21.15.111:0/1214910735 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8894039140 0x7f889403b600 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f889c0098c0 tx=0x7f889c006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:54.043 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.072+0000 7f88a9600640 1 -- 172.21.15.111:0/1214910735 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f88ac0824a0 con 0x7f88ac109930 2024-09-15T22:36:54.043 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.259+0000 7f88b3a00640 1 -- 172.21.15.111:0/1214910735 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f88ac07efe0 con 0x7f8894039140 2024-09-15T22:36:54.043 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.327+0000 7f88a9600640 1 -- 172.21.15.111:0/1214910735 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f88ac07efe0 con 0x7f8894039140 2024-09-15T22:36:54.043 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.330+0000 7f88b3a00640 1 -- 172.21.15.111:0/1214910735 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8894039140 msgr2=0x7f889403b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:54.043 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.330+0000 7f88b3a00640 1 --2- 172.21.15.111:0/1214910735 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8894039140 0x7f889403b600 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f889c0098c0 tx=0x7f889c006db0 comp rx=0 tx=0).stop 2024-09-15T22:36:54.043 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.330+0000 7f88b3a00640 1 -- 172.21.15.111:0/1214910735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88ac109930 msgr2=0x7f88ac081b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:54.043 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.330+0000 7f88b3a00640 1 --2- 172.21.15.111:0/1214910735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88ac109930 0x7f88ac081b00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f88a00097f0 tx=0x7f88a00327b0 comp rx=0 tx=0).stop 2024-09-15T22:36:54.044 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.330+0000 7f88b3a00640 1 -- 172.21.15.111:0/1214910735 shutdown_connections 2024-09-15T22:36:54.044 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.330+0000 7f88b3a00640 1 --2- 172.21.15.111:0/1214910735 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8894039140 0x7f889403b600 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:54.044 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.330+0000 7f88b3a00640 1 --2- 172.21.15.111:0/1214910735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88ac109930 0x7f88ac081b00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:54.044 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.330+0000 7f88b3a00640 1 -- 172.21.15.111:0/1214910735 >> 172.21.15.111:0/1214910735 conn(0x7f88ac07cfa0 msgr2=0x7f88ac107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:54.044 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.330+0000 7f88b3a00640 1 -- 172.21.15.111:0/1214910735 shutdown_connections 2024-09-15T22:36:54.044 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:53.330+0000 7f88b3a00640 1 -- 172.21.15.111:0/1214910735 wait complete. 2024-09-15T22:36:54.044 INFO:teuthology.orchestra.run.smithi111.stdout:Creating initial admin user... 2024-09-15T22:36:54.564 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:54 smithi111 ceph-mon[30629]: from='client.14160 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:36:54.565 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:54 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:36:54.565 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:54 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:36:54.565 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:54 smithi111 ceph-mon[30629]: mgrmap e12: x(active, since 2s) 2024-09-15T22:36:54.565 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:54 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:36:56.274 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:55 smithi111 ceph-mon[30629]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:36:56.274 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:55 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:36:56.623 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$2DIFM2i.iSd.fOJfh9rV6upTL6a9XGrskZK6xc9AB3SzL1xH28x5W", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726439814, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-15T22:36:56.624 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.562+0000 7fedbf200640 1 Processor -- start 2024-09-15T22:36:56.624 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.562+0000 7fedbf200640 1 -- start start 2024-09-15T22:36:56.624 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.562+0000 7fedbf200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedb8106410 0x7fedb8106810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:56.624 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.562+0000 7fedbf200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fedb8106de0 con 0x7fedb8106410 2024-09-15T22:36:56.624 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.562+0000 7fedbde00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedb8106410 0x7fedb8106810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:56.624 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.562+0000 7fedbde00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedb8106410 0x7fedb8106810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53804/0 (socket says 172.21.15.111:53804) 2024-09-15T22:36:56.624 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.562+0000 7fedbde00640 1 -- 172.21.15.111:0/49314972 learned_addr learned my addr 172.21.15.111:0/49314972 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:56.624 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.563+0000 7fedbde00640 1 -- 172.21.15.111:0/49314972 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb8107610 con 0x7fedb8106410 2024-09-15T22:36:56.624 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.564+0000 7fedbde00640 1 --2- 172.21.15.111:0/49314972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedb8106410 0x7fedb8106810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fedac009820 tx=0x7fedac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8812a9fee264f7d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:56.624 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.564+0000 7fedbca00640 1 -- 172.21.15.111:0/49314972 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedac037070 con 0x7fedb8106410 2024-09-15T22:36:56.624 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.564+0000 7fedbca00640 1 -- 172.21.15.111:0/49314972 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedac032920 con 0x7fedb8106410 2024-09-15T22:36:56.624 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.564+0000 7fedbca00640 1 -- 172.21.15.111:0/49314972 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedac030810 con 0x7fedb8106410 2024-09-15T22:36:56.624 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.565+0000 7fedbf200640 1 -- 172.21.15.111:0/49314972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedb8106410 msgr2=0x7fedb8106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:56.625 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.565+0000 7fedbf200640 1 --2- 172.21.15.111:0/49314972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedb8106410 0x7fedb8106810 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fedac009820 tx=0x7fedac02a270 comp rx=0 tx=0).stop 2024-09-15T22:36:56.625 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.565+0000 7fedbf200640 1 -- 172.21.15.111:0/49314972 shutdown_connections 2024-09-15T22:36:56.625 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.565+0000 7fedbf200640 1 --2- 172.21.15.111:0/49314972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedb8106410 0x7fedb8106810 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:56.625 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.565+0000 7fedbf200640 1 -- 172.21.15.111:0/49314972 >> 172.21.15.111:0/49314972 conn(0x7fedb8101bc0 msgr2=0x7fedb8103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:56.625 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.565+0000 7fedbf200640 1 -- 172.21.15.111:0/49314972 shutdown_connections 2024-09-15T22:36:56.625 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.565+0000 7fedbf200640 1 -- 172.21.15.111:0/49314972 wait complete. 2024-09-15T22:36:56.625 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.566+0000 7fedbf200640 1 Processor -- start 2024-09-15T22:36:56.625 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.566+0000 7fedbf200640 1 -- start start 2024-09-15T22:36:56.625 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.567+0000 7fedbf200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedb8106410 0x7fedb819d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:56.625 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.567+0000 7fedbf200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fedb810a9c0 con 0x7fedb8106410 2024-09-15T22:36:56.625 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.567+0000 7fedbde00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedb8106410 0x7fedb819d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:56.625 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.567+0000 7fedbde00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedb8106410 0x7fedb819d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53812/0 (socket says 172.21.15.111:53812) 2024-09-15T22:36:56.625 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.567+0000 7fedbde00640 1 -- 172.21.15.111:0/1661007971 learned_addr learned my addr 172.21.15.111:0/1661007971 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:56.625 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.567+0000 7fedbde00640 1 -- 172.21.15.111:0/1661007971 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb819dd50 con 0x7fedb8106410 2024-09-15T22:36:56.626 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.568+0000 7fedbde00640 1 --2- 172.21.15.111:0/1661007971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedb8106410 0x7fedb819d810 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fedac0097f0 tx=0x7fedac0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:56.626 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.568+0000 7fedb2a00640 1 -- 172.21.15.111:0/1661007971 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedac037040 con 0x7fedb8106410 2024-09-15T22:36:56.626 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.568+0000 7fedb2a00640 1 -- 172.21.15.111:0/1661007971 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedac030c40 con 0x7fedb8106410 2024-09-15T22:36:56.626 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.568+0000 7fedb2a00640 1 -- 172.21.15.111:0/1661007971 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedac03a5b0 con 0x7fedb8106410 2024-09-15T22:36:56.626 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.568+0000 7fedbf200640 1 -- 172.21.15.111:0/1661007971 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedb819dfe0 con 0x7fedb8106410 2024-09-15T22:36:56.626 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.568+0000 7fedbf200640 1 -- 172.21.15.111:0/1661007971 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedb81a0cd0 con 0x7fedb8106410 2024-09-15T22:36:56.626 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.569+0000 7fedb2a00640 1 -- 172.21.15.111:0/1661007971 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fedac03a7a0 con 0x7fedb8106410 2024-09-15T22:36:56.626 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.569+0000 7fedbf200640 1 -- 172.21.15.111:0/1661007971 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedb810ab40 con 0x7fedb8106410 2024-09-15T22:36:56.626 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.570+0000 7fedb2a00640 1 --2- 172.21.15.111:0/1661007971 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7feda0039260 0x7feda003b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:56.626 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.570+0000 7fedb2a00640 1 -- 172.21.15.111:0/1661007971 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fedac06c980 con 0x7fedb8106410 2024-09-15T22:36:56.626 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.570+0000 7fedbd400640 1 --2- 172.21.15.111:0/1661007971 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7feda0039260 0x7feda003b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:56.626 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.571+0000 7fedbd400640 1 --2- 172.21.15.111:0/1661007971 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7feda0039260 0x7feda003b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7feda80098c0 tx=0x7feda8006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:56.626 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.575+0000 7fedb2a00640 1 -- 172.21.15.111:0/1661007971 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fedac03abe0 con 0x7fedb8106410 2024-09-15T22:36:56.626 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.771+0000 7fedbf200640 1 -- 172.21.15.111:0/1661007971 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fedb81a0ea0 con 0x7feda0039260 2024-09-15T22:36:56.627 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.997+0000 7fedb2a00640 1 -- 172.21.15.111:0/1661007971 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fedb81a0ea0 con 0x7feda0039260 2024-09-15T22:36:56.627 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.999+0000 7fedbf200640 1 -- 172.21.15.111:0/1661007971 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7feda0039260 msgr2=0x7feda003b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:56.627 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.999+0000 7fedbf200640 1 --2- 172.21.15.111:0/1661007971 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7feda0039260 0x7feda003b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7feda80098c0 tx=0x7feda8006db0 comp rx=0 tx=0).stop 2024-09-15T22:36:56.627 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.999+0000 7fedbf200640 1 -- 172.21.15.111:0/1661007971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedb8106410 msgr2=0x7fedb819d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:56.627 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.999+0000 7fedbf200640 1 --2- 172.21.15.111:0/1661007971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedb8106410 0x7fedb819d810 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fedac0097f0 tx=0x7fedac0327b0 comp rx=0 tx=0).stop 2024-09-15T22:36:56.627 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.999+0000 7fedbf200640 1 -- 172.21.15.111:0/1661007971 shutdown_connections 2024-09-15T22:36:56.627 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.999+0000 7fedbf200640 1 --2- 172.21.15.111:0/1661007971 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7feda0039260 0x7feda003b720 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:56.627 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.999+0000 7fedbf200640 1 --2- 172.21.15.111:0/1661007971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedb8106410 0x7fedb819d810 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:56.627 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.999+0000 7fedbf200640 1 -- 172.21.15.111:0/1661007971 >> 172.21.15.111:0/1661007971 conn(0x7fedb8101bc0 msgr2=0x7fedb8102810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:56.627 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.999+0000 7fedbf200640 1 -- 172.21.15.111:0/1661007971 shutdown_connections 2024-09-15T22:36:56.627 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:54.999+0000 7fedbf200640 1 -- 172.21.15.111:0/1661007971 wait complete. 2024-09-15T22:36:56.627 INFO:teuthology.orchestra.run.smithi111.stdout:Fetching dashboard port number... 2024-09-15T22:36:58.051 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stdout 8443 2024-09-15T22:36:58.051 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.133+0000 7fad19600640 1 Processor -- start 2024-09-15T22:36:58.051 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.134+0000 7fad19600640 1 -- start start 2024-09-15T22:36:58.051 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.134+0000 7fad19600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad1407de00 0x7fad1407c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:58.051 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.134+0000 7fad19600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fad1407c830 con 0x7fad1407de00 2024-09-15T22:36:58.051 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.134+0000 7fad12a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad1407de00 0x7fad1407c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:58.051 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.134+0000 7fad12a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad1407de00 0x7fad1407c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53816/0 (socket says 172.21.15.111:53816) 2024-09-15T22:36:58.051 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.134+0000 7fad12a00640 1 -- 172.21.15.111:0/2408656438 learned_addr learned my addr 172.21.15.111:0/2408656438 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:58.051 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.135+0000 7fad12a00640 1 -- 172.21.15.111:0/2408656438 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1407c9b0 con 0x7fad1407de00 2024-09-15T22:36:58.051 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.136+0000 7fad12a00640 1 --2- 172.21.15.111:0/2408656438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad1407de00 0x7fad1407c260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fad04009820 tx=0x7fad0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7d8f7226b529a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:58.052 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.136+0000 7fad12000640 1 -- 172.21.15.111:0/2408656438 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad04037070 con 0x7fad1407de00 2024-09-15T22:36:58.052 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.136+0000 7fad12000640 1 -- 172.21.15.111:0/2408656438 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad04032920 con 0x7fad1407de00 2024-09-15T22:36:58.052 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.136+0000 7fad12000640 1 -- 172.21.15.111:0/2408656438 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad04030810 con 0x7fad1407de00 2024-09-15T22:36:58.052 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.136+0000 7fad19600640 1 -- 172.21.15.111:0/2408656438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad1407de00 msgr2=0x7fad1407c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:58.052 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.136+0000 7fad19600640 1 --2- 172.21.15.111:0/2408656438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad1407de00 0x7fad1407c260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fad04009820 tx=0x7fad0402a270 comp rx=0 tx=0).stop 2024-09-15T22:36:58.052 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.136+0000 7fad19600640 1 -- 172.21.15.111:0/2408656438 shutdown_connections 2024-09-15T22:36:58.052 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.136+0000 7fad19600640 1 --2- 172.21.15.111:0/2408656438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad1407de00 0x7fad1407c260 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:58.052 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.136+0000 7fad19600640 1 -- 172.21.15.111:0/2408656438 >> 172.21.15.111:0/2408656438 conn(0x7fad14102f70 msgr2=0x7fad14105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:58.052 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.137+0000 7fad19600640 1 -- 172.21.15.111:0/2408656438 shutdown_connections 2024-09-15T22:36:58.052 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.137+0000 7fad19600640 1 -- 172.21.15.111:0/2408656438 wait complete. 2024-09-15T22:36:58.052 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.137+0000 7fad19600640 1 Processor -- start 2024-09-15T22:36:58.052 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.138+0000 7fad19600640 1 -- start start 2024-09-15T22:36:58.052 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.138+0000 7fad19600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad1407de00 0x7fad1419d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:58.053 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.138+0000 7fad19600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fad1410ddc0 con 0x7fad1407de00 2024-09-15T22:36:58.053 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.138+0000 7fad12a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad1407de00 0x7fad1419d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:58.053 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.138+0000 7fad12a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad1407de00 0x7fad1419d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53826/0 (socket says 172.21.15.111:53826) 2024-09-15T22:36:58.053 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.138+0000 7fad12a00640 1 -- 172.21.15.111:0/1867687544 learned_addr learned my addr 172.21.15.111:0/1867687544 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:58.053 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.138+0000 7fad12a00640 1 -- 172.21.15.111:0/1867687544 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1419de60 con 0x7fad1407de00 2024-09-15T22:36:58.053 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.139+0000 7fad12a00640 1 --2- 172.21.15.111:0/1867687544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad1407de00 0x7fad1419d920 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fad04030380 tx=0x7fad04032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:58.053 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.139+0000 7fad0be00640 1 -- 172.21.15.111:0/1867687544 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad04037040 con 0x7fad1407de00 2024-09-15T22:36:58.053 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.139+0000 7fad0be00640 1 -- 172.21.15.111:0/1867687544 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad04040070 con 0x7fad1407de00 2024-09-15T22:36:58.053 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.139+0000 7fad19600640 1 -- 172.21.15.111:0/1867687544 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad1419e0f0 con 0x7fad1407de00 2024-09-15T22:36:58.053 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.139+0000 7fad0be00640 1 -- 172.21.15.111:0/1867687544 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad0403a7b0 con 0x7fad1407de00 2024-09-15T22:36:58.053 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.139+0000 7fad19600640 1 -- 172.21.15.111:0/1867687544 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad141a0de0 con 0x7fad1407de00 2024-09-15T22:36:58.053 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.140+0000 7fad0be00640 1 -- 172.21.15.111:0/1867687544 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fad0403b420 con 0x7fad1407de00 2024-09-15T22:36:58.053 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.140+0000 7fad19600640 1 -- 172.21.15.111:0/1867687544 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad1407c2e0 con 0x7fad1407de00 2024-09-15T22:36:58.053 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.141+0000 7fad0be00640 1 --2- 172.21.15.111:0/1867687544 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7facf80392a0 0x7facf803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:58.054 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.141+0000 7fad0be00640 1 -- 172.21.15.111:0/1867687544 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fad0406cad0 con 0x7fad1407de00 2024-09-15T22:36:58.054 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.141+0000 7fad0a000640 1 --2- 172.21.15.111:0/1867687544 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7facf80392a0 0x7facf803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:58.054 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.141+0000 7fad0a000640 1 --2- 172.21.15.111:0/1867687544 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7facf80392a0 0x7facf803b760 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7facfc0098c0 tx=0x7facfc006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:58.054 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.145+0000 7fad0be00640 1 -- 172.21.15.111:0/1867687544 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fad04042b50 con 0x7fad1407de00 2024-09-15T22:36:58.054 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.322+0000 7fad19600640 1 -- 172.21.15.111:0/1867687544 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fad141a13d0 con 0x7fad1407de00 2024-09-15T22:36:58.054 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.323+0000 7fad0be00640 1 -- 172.21.15.111:0/1867687544 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fad04043360 con 0x7fad1407de00 2024-09-15T22:36:58.054 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.325+0000 7fad19600640 1 -- 172.21.15.111:0/1867687544 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7facf80392a0 msgr2=0x7facf803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:58.054 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.325+0000 7fad19600640 1 --2- 172.21.15.111:0/1867687544 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7facf80392a0 0x7facf803b760 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7facfc0098c0 tx=0x7facfc006db0 comp rx=0 tx=0).stop 2024-09-15T22:36:58.054 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.325+0000 7fad19600640 1 -- 172.21.15.111:0/1867687544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad1407de00 msgr2=0x7fad1419d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:58.054 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.325+0000 7fad19600640 1 --2- 172.21.15.111:0/1867687544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad1407de00 0x7fad1419d920 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fad04030380 tx=0x7fad04032670 comp rx=0 tx=0).stop 2024-09-15T22:36:58.054 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.325+0000 7fad19600640 1 -- 172.21.15.111:0/1867687544 shutdown_connections 2024-09-15T22:36:58.054 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.325+0000 7fad19600640 1 --2- 172.21.15.111:0/1867687544 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7facf80392a0 0x7facf803b760 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:58.054 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.325+0000 7fad19600640 1 --2- 172.21.15.111:0/1867687544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad1407de00 0x7fad1419d920 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:58.054 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.325+0000 7fad19600640 1 -- 172.21.15.111:0/1867687544 >> 172.21.15.111:0/1867687544 conn(0x7fad14102f70 msgr2=0x7fad14103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:58.054 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.325+0000 7fad19600640 1 -- 172.21.15.111:0/1867687544 shutdown_connections 2024-09-15T22:36:58.055 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:57.325+0000 7fad19600640 1 -- 172.21.15.111:0/1867687544 wait complete. 2024-09-15T22:36:58.067 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-15T22:36:58.068 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout disabled 2024-09-15T22:36:58.081 INFO:teuthology.orchestra.run.smithi111.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-15T22:36:58.081 INFO:teuthology.orchestra.run.smithi111.stdout:systemctl: stdout inactive 2024-09-15T22:36:58.081 INFO:teuthology.orchestra.run.smithi111.stdout:firewalld.service is not enabled 2024-09-15T22:36:58.081 INFO:teuthology.orchestra.run.smithi111.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-15T22:36:58.084 INFO:teuthology.orchestra.run.smithi111.stdout:Ceph Dashboard is now available at: 2024-09-15T22:36:58.085 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:36:58.085 INFO:teuthology.orchestra.run.smithi111.stdout: URL: https://smithi111.front.sepia.ceph.com:8443/ 2024-09-15T22:36:58.085 INFO:teuthology.orchestra.run.smithi111.stdout: User: admin 2024-09-15T22:36:58.085 INFO:teuthology.orchestra.run.smithi111.stdout: Password: feaiuo1816 2024-09-15T22:36:58.085 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:36:58.085 INFO:teuthology.orchestra.run.smithi111.stdout:Saving cluster configuration to /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/config directory 2024-09-15T22:36:58.385 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:57 smithi111 ceph-mon[30629]: mgrmap e13: x(active, since 6s) 2024-09-15T22:36:58.385 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:57 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/1867687544' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-15T22:36:59.625 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.623+0000 7f7319400640 1 Processor -- start 2024-09-15T22:36:59.625 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.624+0000 7f7319400640 1 -- start start 2024-09-15T22:36:59.626 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.624+0000 7f7319400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7314105fe0 0x7f73141063e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:59.626 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.624+0000 7f7319400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f731407bdd0 con 0x7f7314105fe0 2024-09-15T22:36:59.626 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.624+0000 7f7312800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7314105fe0 0x7f73141063e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:59.626 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.624+0000 7f7312800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7314105fe0 0x7f73141063e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53834/0 (socket says 172.21.15.111:53834) 2024-09-15T22:36:59.626 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.624+0000 7f7312800640 1 -- 172.21.15.111:0/3094772772 learned_addr learned my addr 172.21.15.111:0/3094772772 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:59.626 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.625+0000 7f7312800640 1 -- 172.21.15.111:0/3094772772 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7314106920 con 0x7f7314105fe0 2024-09-15T22:36:59.626 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.626+0000 7f7312800640 1 --2- 172.21.15.111:0/3094772772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7314105fe0 0x7f73141063e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7304009820 tx=0x7f730402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d542e09cde52f19 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:59.626 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.626+0000 7f7311400640 1 -- 172.21.15.111:0/3094772772 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7304037070 con 0x7f7314105fe0 2024-09-15T22:36:59.626 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.626+0000 7f7311400640 1 -- 172.21.15.111:0/3094772772 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7304032920 con 0x7f7314105fe0 2024-09-15T22:36:59.626 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.626+0000 7f7311400640 1 -- 172.21.15.111:0/3094772772 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7304030810 con 0x7f7314105fe0 2024-09-15T22:36:59.626 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.627+0000 7f7319400640 1 -- 172.21.15.111:0/3094772772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7314105fe0 msgr2=0x7f73141063e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:59.626 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.627+0000 7f7319400640 1 --2- 172.21.15.111:0/3094772772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7314105fe0 0x7f73141063e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7304009820 tx=0x7f730402a270 comp rx=0 tx=0).stop 2024-09-15T22:36:59.627 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.627+0000 7f7319400640 1 -- 172.21.15.111:0/3094772772 shutdown_connections 2024-09-15T22:36:59.627 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.627+0000 7f7319400640 1 --2- 172.21.15.111:0/3094772772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7314105fe0 0x7f73141063e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.627 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.627+0000 7f7319400640 1 -- 172.21.15.111:0/3094772772 >> 172.21.15.111:0/3094772772 conn(0x7f7314101e70 msgr2=0x7f7314104290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:59.627 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.627+0000 7f7319400640 1 -- 172.21.15.111:0/3094772772 shutdown_connections 2024-09-15T22:36:59.627 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.627+0000 7f7319400640 1 -- 172.21.15.111:0/3094772772 wait complete. 2024-09-15T22:36:59.627 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.628+0000 7f7319400640 1 Processor -- start 2024-09-15T22:36:59.627 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.628+0000 7f7319400640 1 -- start start 2024-09-15T22:36:59.627 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.628+0000 7f7319400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7314105fe0 0x7f731419d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:59.627 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.628+0000 7f7319400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7314107a60 con 0x7f7314105fe0 2024-09-15T22:36:59.627 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.628+0000 7f7312800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7314105fe0 0x7f731419d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:59.627 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.628+0000 7f7312800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7314105fe0 0x7f731419d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53836/0 (socket says 172.21.15.111:53836) 2024-09-15T22:36:59.627 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.628+0000 7f7312800640 1 -- 172.21.15.111:0/2616148014 learned_addr learned my addr 172.21.15.111:0/2616148014 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:36:59.627 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.629+0000 7f7312800640 1 -- 172.21.15.111:0/2616148014 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f731419de20 con 0x7f7314105fe0 2024-09-15T22:36:59.628 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.629+0000 7f7312800640 1 --2- 172.21.15.111:0/2616148014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7314105fe0 0x7f731419d8e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f73040097f0 tx=0x7f73040327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:59.628 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.629+0000 7f730b400640 1 -- 172.21.15.111:0/2616148014 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7304037040 con 0x7f7314105fe0 2024-09-15T22:36:59.628 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.629+0000 7f730b400640 1 -- 172.21.15.111:0/2616148014 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7304030cc0 con 0x7f7314105fe0 2024-09-15T22:36:59.628 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.629+0000 7f730b400640 1 -- 172.21.15.111:0/2616148014 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f730403a6b0 con 0x7f7314105fe0 2024-09-15T22:36:59.628 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.629+0000 7f7319400640 1 -- 172.21.15.111:0/2616148014 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f731419e0b0 con 0x7f7314105fe0 2024-09-15T22:36:59.628 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.629+0000 7f7319400640 1 -- 172.21.15.111:0/2616148014 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73141a0da0 con 0x7f7314105fe0 2024-09-15T22:36:59.628 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.630+0000 7f730b400640 1 -- 172.21.15.111:0/2616148014 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f730403b3f0 con 0x7f7314105fe0 2024-09-15T22:36:59.628 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.631+0000 7f7319400640 1 -- 172.21.15.111:0/2616148014 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73141063e0 con 0x7f7314105fe0 2024-09-15T22:36:59.628 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.631+0000 7f730b400640 1 --2- 172.21.15.111:0/2616148014 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f72f8039300 0x7f72f803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:36:59.628 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.631+0000 7f730b400640 1 -- 172.21.15.111:0/2616148014 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f730406ca90 con 0x7f7314105fe0 2024-09-15T22:36:59.628 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.631+0000 7f7311e00640 1 --2- 172.21.15.111:0/2616148014 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f72f8039300 0x7f72f803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:36:59.628 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.632+0000 7f7311e00640 1 --2- 172.21.15.111:0/2616148014 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f72f8039300 0x7f72f803b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f73000098c0 tx=0x7f7300006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:36:59.628 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.636+0000 7f730b400640 1 -- 172.21.15.111:0/2616148014 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7304041890 con 0x7f7314105fe0 2024-09-15T22:36:59.628 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.904+0000 7f7319400640 1 -- 172.21.15.111:0/2616148014 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f731402d9b0 con 0x7f7314105fe0 2024-09-15T22:36:59.629 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.907+0000 7f730b400640 1 -- 172.21.15.111:0/2616148014 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f73040371e0 con 0x7f7314105fe0 2024-09-15T22:36:59.629 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-15T22:36:59.629 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.909+0000 7f7319400640 1 -- 172.21.15.111:0/2616148014 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f72f8039300 msgr2=0x7f72f803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:59.629 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.909+0000 7f7319400640 1 --2- 172.21.15.111:0/2616148014 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f72f8039300 0x7f72f803b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f73000098c0 tx=0x7f7300006db0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.629 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.909+0000 7f7319400640 1 -- 172.21.15.111:0/2616148014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7314105fe0 msgr2=0x7f731419d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:36:59.629 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.909+0000 7f7319400640 1 --2- 172.21.15.111:0/2616148014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7314105fe0 0x7f731419d8e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f73040097f0 tx=0x7f73040327b0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.629 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.909+0000 7f7319400640 1 -- 172.21.15.111:0/2616148014 shutdown_connections 2024-09-15T22:36:59.629 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.909+0000 7f7319400640 1 --2- 172.21.15.111:0/2616148014 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f72f8039300 0x7f72f803b7c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.629 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.909+0000 7f7319400640 1 --2- 172.21.15.111:0/2616148014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7314105fe0 0x7f731419d8e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:36:59.629 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.909+0000 7f7319400640 1 -- 172.21.15.111:0/2616148014 >> 172.21.15.111:0/2616148014 conn(0x7f7314101e70 msgr2=0x7f7314191bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:36:59.629 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.909+0000 7f7319400640 1 -- 172.21.15.111:0/2616148014 shutdown_connections 2024-09-15T22:36:59.629 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/ceph: stderr 2024-09-15T22:36:58.910+0000 7f7319400640 1 -- 172.21.15.111:0/2616148014 wait complete. 2024-09-15T22:36:59.629 INFO:teuthology.orchestra.run.smithi111.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-15T22:36:59.629 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:36:59.630 INFO:teuthology.orchestra.run.smithi111.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-15T22:36:59.630 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:36:59.630 INFO:teuthology.orchestra.run.smithi111.stdout:Or, if you are only running a single cluster on this host: 2024-09-15T22:36:59.630 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:36:59.630 INFO:teuthology.orchestra.run.smithi111.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-15T22:36:59.630 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:36:59.630 INFO:teuthology.orchestra.run.smithi111.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-15T22:36:59.630 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:36:59.630 INFO:teuthology.orchestra.run.smithi111.stdout: ceph telemetry on 2024-09-15T22:36:59.630 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:36:59.630 INFO:teuthology.orchestra.run.smithi111.stdout:For more information see: 2024-09-15T22:36:59.630 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:36:59.630 INFO:teuthology.orchestra.run.smithi111.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-15T22:36:59.630 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:36:59.630 INFO:teuthology.orchestra.run.smithi111.stdout:Bootstrap complete. 2024-09-15T22:36:59.681 INFO:tasks.cephadm:Fetching config... 2024-09-15T22:36:59.682 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-15T22:36:59.682 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-15T22:36:59.718 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-15T22:36:59.718 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-15T22:36:59.718 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-15T22:36:59.795 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-15T22:36:59.796 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-15T22:36:59.796 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-15T22:36:59.892 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-15T22:36:59.892 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-15T22:36:59.892 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-15T22:36:59.968 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-15T22:36:59.968 DEBUG:teuthology.orchestra.run.smithi111:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCWKCBLmWKt6EZ4bc1UdpbSUq0inAxSTYz/nq36B8tVHl18FhGU/hrKS2h/bxY4YCdrMoxj/xBoKGf46ue/aLao1CiaFeY3dsFltHqHcsyrRZIbXZ/L05f/fDiF5VtNGDAlMAUul+sbPCIyIX0yk2iTSWgRv8ITAADIZMD07NDpsntHSJzgSiQGoD7CYZ9/moFPVHniuzvlbwlE9//UD8LJogTckjVkjTZ0Ukjx2wbtNEihmCmJkHX0YFUH59q9w1MIcsLL6OtwjevxaVUxsFNXF0IwQzjyWzJlknaz+GvuUfpnZlSHwWMdPJOaYeUsdtWPYfLo/GSnA75BMAIWTha7L43B8NoFFS5AKb0oysNRKoxfMVKAe8+0Yn7OVrSEj/y/WkpFhxy0ERyBJVm6XsgdhLK4hY37cPjwUU/VPMR9G7pgW3a1NPqZ0lcexwODi8wuhRla5NIw+BWEkXY1YpytWo6cv8ub00KShJOCkxCaAb+lWloCbp6cjam/bJMyLJ0= ceph-b7614d3c-73b2-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-15T22:37:00.092 INFO:teuthology.orchestra.run.smithi111.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCWKCBLmWKt6EZ4bc1UdpbSUq0inAxSTYz/nq36B8tVHl18FhGU/hrKS2h/bxY4YCdrMoxj/xBoKGf46ue/aLao1CiaFeY3dsFltHqHcsyrRZIbXZ/L05f/fDiF5VtNGDAlMAUul+sbPCIyIX0yk2iTSWgRv8ITAADIZMD07NDpsntHSJzgSiQGoD7CYZ9/moFPVHniuzvlbwlE9//UD8LJogTckjVkjTZ0Ukjx2wbtNEihmCmJkHX0YFUH59q9w1MIcsLL6OtwjevxaVUxsFNXF0IwQzjyWzJlknaz+GvuUfpnZlSHwWMdPJOaYeUsdtWPYfLo/GSnA75BMAIWTha7L43B8NoFFS5AKb0oysNRKoxfMVKAe8+0Yn7OVrSEj/y/WkpFhxy0ERyBJVm6XsgdhLK4hY37cPjwUU/VPMR9G7pgW3a1NPqZ0lcexwODi8wuhRla5NIw+BWEkXY1YpytWo6cv8ub00KShJOCkxCaAb+lWloCbp6cjam/bJMyLJ0= ceph-b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T22:37:00.115 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:36:59 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/2616148014' entity='client.admin' 2024-09-15T22:37:00.117 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCWKCBLmWKt6EZ4bc1UdpbSUq0inAxSTYz/nq36B8tVHl18FhGU/hrKS2h/bxY4YCdrMoxj/xBoKGf46ue/aLao1CiaFeY3dsFltHqHcsyrRZIbXZ/L05f/fDiF5VtNGDAlMAUul+sbPCIyIX0yk2iTSWgRv8ITAADIZMD07NDpsntHSJzgSiQGoD7CYZ9/moFPVHniuzvlbwlE9//UD8LJogTckjVkjTZ0Ukjx2wbtNEihmCmJkHX0YFUH59q9w1MIcsLL6OtwjevxaVUxsFNXF0IwQzjyWzJlknaz+GvuUfpnZlSHwWMdPJOaYeUsdtWPYfLo/GSnA75BMAIWTha7L43B8NoFFS5AKb0oysNRKoxfMVKAe8+0Yn7OVrSEj/y/WkpFhxy0ERyBJVm6XsgdhLK4hY37cPjwUU/VPMR9G7pgW3a1NPqZ0lcexwODi8wuhRla5NIw+BWEkXY1YpytWo6cv8ub00KShJOCkxCaAb+lWloCbp6cjam/bJMyLJ0= ceph-b7614d3c-73b2-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-15T22:37:00.159 INFO:teuthology.orchestra.run.smithi121.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCWKCBLmWKt6EZ4bc1UdpbSUq0inAxSTYz/nq36B8tVHl18FhGU/hrKS2h/bxY4YCdrMoxj/xBoKGf46ue/aLao1CiaFeY3dsFltHqHcsyrRZIbXZ/L05f/fDiF5VtNGDAlMAUul+sbPCIyIX0yk2iTSWgRv8ITAADIZMD07NDpsntHSJzgSiQGoD7CYZ9/moFPVHniuzvlbwlE9//UD8LJogTckjVkjTZ0Ukjx2wbtNEihmCmJkHX0YFUH59q9w1MIcsLL6OtwjevxaVUxsFNXF0IwQzjyWzJlknaz+GvuUfpnZlSHwWMdPJOaYeUsdtWPYfLo/GSnA75BMAIWTha7L43B8NoFFS5AKb0oysNRKoxfMVKAe8+0Yn7OVrSEj/y/WkpFhxy0ERyBJVm6XsgdhLK4hY37cPjwUU/VPMR9G7pgW3a1NPqZ0lcexwODi8wuhRla5NIw+BWEkXY1YpytWo6cv8ub00KShJOCkxCaAb+lWloCbp6cjam/bJMyLJ0= ceph-b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T22:37:00.172 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCWKCBLmWKt6EZ4bc1UdpbSUq0inAxSTYz/nq36B8tVHl18FhGU/hrKS2h/bxY4YCdrMoxj/xBoKGf46ue/aLao1CiaFeY3dsFltHqHcsyrRZIbXZ/L05f/fDiF5VtNGDAlMAUul+sbPCIyIX0yk2iTSWgRv8ITAADIZMD07NDpsntHSJzgSiQGoD7CYZ9/moFPVHniuzvlbwlE9//UD8LJogTckjVkjTZ0Ukjx2wbtNEihmCmJkHX0YFUH59q9w1MIcsLL6OtwjevxaVUxsFNXF0IwQzjyWzJlknaz+GvuUfpnZlSHwWMdPJOaYeUsdtWPYfLo/GSnA75BMAIWTha7L43B8NoFFS5AKb0oysNRKoxfMVKAe8+0Yn7OVrSEj/y/WkpFhxy0ERyBJVm6XsgdhLK4hY37cPjwUU/VPMR9G7pgW3a1NPqZ0lcexwODi8wuhRla5NIw+BWEkXY1YpytWo6cv8ub00KShJOCkxCaAb+lWloCbp6cjam/bJMyLJ0= ceph-b7614d3c-73b2-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-15T22:37:00.214 INFO:teuthology.orchestra.run.smithi192.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCWKCBLmWKt6EZ4bc1UdpbSUq0inAxSTYz/nq36B8tVHl18FhGU/hrKS2h/bxY4YCdrMoxj/xBoKGf46ue/aLao1CiaFeY3dsFltHqHcsyrRZIbXZ/L05f/fDiF5VtNGDAlMAUul+sbPCIyIX0yk2iTSWgRv8ITAADIZMD07NDpsntHSJzgSiQGoD7CYZ9/moFPVHniuzvlbwlE9//UD8LJogTckjVkjTZ0Ukjx2wbtNEihmCmJkHX0YFUH59q9w1MIcsLL6OtwjevxaVUxsFNXF0IwQzjyWzJlknaz+GvuUfpnZlSHwWMdPJOaYeUsdtWPYfLo/GSnA75BMAIWTha7L43B8NoFFS5AKb0oysNRKoxfMVKAe8+0Yn7OVrSEj/y/WkpFhxy0ERyBJVm6XsgdhLK4hY37cPjwUU/VPMR9G7pgW3a1NPqZ0lcexwODi8wuhRla5NIw+BWEkXY1YpytWo6cv8ub00KShJOCkxCaAb+lWloCbp6cjam/bJMyLJ0= ceph-b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T22:37:00.227 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-15T22:37:00.591 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:01.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.194+0000 7fdb7fe00640 1 -- 172.21.15.111:0/2701931626 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb80101400 msgr2=0x7fdb80101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:01.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.194+0000 7fdb7fe00640 1 --2- 172.21.15.111:0/2701931626 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb80101400 0x7fdb80101800 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c009880 tx=0x7fdb6c02a5d0 comp rx=0 tx=0).stop 2024-09-15T22:37:01.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.194+0000 7fdb7fe00640 1 -- 172.21.15.111:0/2701931626 shutdown_connections 2024-09-15T22:37:01.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.194+0000 7fdb7fe00640 1 --2- 172.21.15.111:0/2701931626 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb80101400 0x7fdb80101800 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:01.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.194+0000 7fdb7fe00640 1 -- 172.21.15.111:0/2701931626 >> 172.21.15.111:0/2701931626 conn(0x7fdb800fcb90 msgr2=0x7fdb800fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:01.198 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.194+0000 7fdb7fe00640 1 -- 172.21.15.111:0/2701931626 shutdown_connections 2024-09-15T22:37:01.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.194+0000 7fdb7fe00640 1 -- 172.21.15.111:0/2701931626 wait complete. 2024-09-15T22:37:01.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.195+0000 7fdb7fe00640 1 Processor -- start 2024-09-15T22:37:01.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.195+0000 7fdb7fe00640 1 -- start start 2024-09-15T22:37:01.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.195+0000 7fdb7fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb80101400 0x7fdb80075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:01.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.195+0000 7fdb7fe00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdb80107510 con 0x7fdb80101400 2024-09-15T22:37:01.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.196+0000 7fdb7ea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb80101400 0x7fdb80075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:01.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.196+0000 7fdb7ea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb80101400 0x7fdb80075eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:32780/0 (socket says 172.21.15.111:32780) 2024-09-15T22:37:01.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.196+0000 7fdb7ea00640 1 -- 172.21.15.111:0/1648918359 learned_addr learned my addr 172.21.15.111:0/1648918359 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:37:01.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.196+0000 7fdb7ea00640 1 -- 172.21.15.111:0/1648918359 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb800763f0 con 0x7fdb80101400 2024-09-15T22:37:01.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.196+0000 7fdb7ea00640 1 --2- 172.21.15.111:0/1648918359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb80101400 0x7fdb80075eb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c004f60 tx=0x7fdb6c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:01.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.197+0000 7fdb77400640 1 -- 172.21.15.111:0/1648918359 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb6c03f070 con 0x7fdb80101400 2024-09-15T22:37:01.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.197+0000 7fdb77400640 1 -- 172.21.15.111:0/1648918359 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb6c004cc0 con 0x7fdb80101400 2024-09-15T22:37:01.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.197+0000 7fdb7fe00640 1 -- 172.21.15.111:0/1648918359 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb800745e0 con 0x7fdb80101400 2024-09-15T22:37:01.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.197+0000 7fdb77400640 1 -- 172.21.15.111:0/1648918359 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb6c035030 con 0x7fdb80101400 2024-09-15T22:37:01.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.197+0000 7fdb7fe00640 1 -- 172.21.15.111:0/1648918359 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb80074b20 con 0x7fdb80101400 2024-09-15T22:37:01.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.198+0000 7fdb7fe00640 1 -- 172.21.15.111:0/1648918359 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb80107c90 con 0x7fdb80101400 2024-09-15T22:37:01.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.198+0000 7fdb77400640 1 -- 172.21.15.111:0/1648918359 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fdb6c042070 con 0x7fdb80101400 2024-09-15T22:37:01.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.199+0000 7fdb77400640 1 --2- 172.21.15.111:0/1648918359 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fdb64039300 0x7fdb6403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:01.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.199+0000 7fdb77400640 1 -- 172.21.15.111:0/1648918359 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdb6c06d2e0 con 0x7fdb80101400 2024-09-15T22:37:01.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.199+0000 7fdb7e000640 1 --2- 172.21.15.111:0/1648918359 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fdb64039300 0x7fdb6403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:01.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.200+0000 7fdb7e000640 1 --2- 172.21.15.111:0/1648918359 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fdb64039300 0x7fdb6403b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdb700098c0 tx=0x7fdb70006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:01.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.204+0000 7fdb77400640 1 -- 172.21.15.111:0/1648918359 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdb6c0433b0 con 0x7fdb80101400 2024-09-15T22:37:01.391 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.389+0000 7fdb7fe00640 1 -- 172.21.15.111:0/1648918359 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fdb8018ffd0 con 0x7fdb80101400 2024-09-15T22:37:01.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.392+0000 7fdb77400640 1 -- 172.21.15.111:0/1648918359 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fdb6c02f300 con 0x7fdb80101400 2024-09-15T22:37:01.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.394+0000 7fdb7fe00640 1 -- 172.21.15.111:0/1648918359 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fdb64039300 msgr2=0x7fdb6403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:01.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.394+0000 7fdb7fe00640 1 --2- 172.21.15.111:0/1648918359 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fdb64039300 0x7fdb6403b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdb700098c0 tx=0x7fdb70006db0 comp rx=0 tx=0).stop 2024-09-15T22:37:01.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.395+0000 7fdb7fe00640 1 -- 172.21.15.111:0/1648918359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb80101400 msgr2=0x7fdb80075eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:01.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.395+0000 7fdb7fe00640 1 --2- 172.21.15.111:0/1648918359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb80101400 0x7fdb80075eb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c004f60 tx=0x7fdb6c0048f0 comp rx=0 tx=0).stop 2024-09-15T22:37:01.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.395+0000 7fdb7fe00640 1 -- 172.21.15.111:0/1648918359 shutdown_connections 2024-09-15T22:37:01.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.395+0000 7fdb7fe00640 1 --2- 172.21.15.111:0/1648918359 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fdb64039300 0x7fdb6403b7c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:01.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.395+0000 7fdb7fe00640 1 --2- 172.21.15.111:0/1648918359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb80101400 0x7fdb80075eb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:01.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.395+0000 7fdb7fe00640 1 -- 172.21.15.111:0/1648918359 >> 172.21.15.111:0/1648918359 conn(0x7fdb800fcb90 msgr2=0x7fdb800fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:01.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.395+0000 7fdb7fe00640 1 -- 172.21.15.111:0/1648918359 shutdown_connections 2024-09-15T22:37:01.397 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:01.395+0000 7fdb7fe00640 1 -- 172.21.15.111:0/1648918359 wait complete. 2024-09-15T22:37:02.229 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-15T22:37:02.230 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-15T22:37:02.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:02 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/1648918359' entity='client.admin' 2024-09-15T22:37:02.937 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:03.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:03 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:03.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:03 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:03.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:03 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi111", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:37:03.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:03 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:03.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:03 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:37:03.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:03 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:03.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.797+0000 7f43c1600640 1 -- 172.21.15.111:0/1019134249 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43b4031760 con 0x7f43c4103620 2024-09-15T22:37:03.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.798+0000 7f43c3e00640 1 -- 172.21.15.111:0/1019134249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43c4103620 msgr2=0x7f43c4103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:03.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.798+0000 7f43c3e00640 1 --2- 172.21.15.111:0/1019134249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43c4103620 0x7f43c4103a20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009880 tx=0x7f43b402a5d0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.798+0000 7f43c3e00640 1 -- 172.21.15.111:0/1019134249 shutdown_connections 2024-09-15T22:37:03.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.798+0000 7f43c3e00640 1 --2- 172.21.15.111:0/1019134249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43c4103620 0x7f43c4103a20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:03.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.798+0000 7f43c3e00640 1 -- 172.21.15.111:0/1019134249 >> 172.21.15.111:0/1019134249 conn(0x7f43c40fedd0 msgr2=0x7f43c41011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:03.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.798+0000 7f43c3e00640 1 -- 172.21.15.111:0/1019134249 shutdown_connections 2024-09-15T22:37:03.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.798+0000 7f43c3e00640 1 -- 172.21.15.111:0/1019134249 wait complete. 2024-09-15T22:37:03.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.799+0000 7f43c3e00640 1 Processor -- start 2024-09-15T22:37:03.919 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.799+0000 7f43c3e00640 1 -- start start 2024-09-15T22:37:03.919 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.800+0000 7f43c3e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43c4103620 0x7f43c4198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:03.919 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.800+0000 7f43c3e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f43c4109730 con 0x7f43c4103620 2024-09-15T22:37:03.919 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.800+0000 7f43c2a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43c4103620 0x7f43c4198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:03.919 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.800+0000 7f43c2a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43c4103620 0x7f43c4198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:32802/0 (socket says 172.21.15.111:32802) 2024-09-15T22:37:03.919 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.800+0000 7f43c2a00640 1 -- 172.21.15.111:0/1704144390 learned_addr learned my addr 172.21.15.111:0/1704144390 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:37:03.919 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.800+0000 7f43c2a00640 1 -- 172.21.15.111:0/1704144390 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c4199320 con 0x7f43c4103620 2024-09-15T22:37:03.919 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.801+0000 7f43c2a00640 1 --2- 172.21.15.111:0/1704144390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43c4103620 0x7f43c4198de0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f43b4004f60 tx=0x7f43b40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:03.919 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.801+0000 7f43bb400640 1 -- 172.21.15.111:0/1704144390 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43b403f070 con 0x7f43c4103620 2024-09-15T22:37:03.919 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.801+0000 7f43bb400640 1 -- 172.21.15.111:0/1704144390 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43b4004d40 con 0x7f43c4103620 2024-09-15T22:37:03.919 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.801+0000 7f43bb400640 1 -- 172.21.15.111:0/1704144390 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43b4035030 con 0x7f43c4103620 2024-09-15T22:37:03.919 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.801+0000 7f43c3e00640 1 -- 172.21.15.111:0/1704144390 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c41995b0 con 0x7f43c4103620 2024-09-15T22:37:03.919 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.801+0000 7f43c3e00640 1 -- 172.21.15.111:0/1704144390 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c419c2a0 con 0x7f43c4103620 2024-09-15T22:37:03.920 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.802+0000 7f43bb400640 1 -- 172.21.15.111:0/1704144390 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f43b4042070 con 0x7f43c4103620 2024-09-15T22:37:03.920 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.802+0000 7f43c3e00640 1 -- 172.21.15.111:0/1704144390 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c4109eb0 con 0x7f43c4103620 2024-09-15T22:37:03.920 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.803+0000 7f43bb400640 1 --2- 172.21.15.111:0/1704144390 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f43a80392a0 0x7f43a803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:03.920 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.803+0000 7f43bb400640 1 -- 172.21.15.111:0/1704144390 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f43b406d000 con 0x7f43c4103620 2024-09-15T22:37:03.920 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.803+0000 7f43c2000640 1 --2- 172.21.15.111:0/1704144390 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f43a80392a0 0x7f43a803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:03.920 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.804+0000 7f43c2000640 1 --2- 172.21.15.111:0/1704144390 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f43a80392a0 0x7f43a803b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f43b00098c0 tx=0x7f43b0006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:03.920 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.808+0000 7f43bb400640 1 -- 172.21.15.111:0/1704144390 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f43b404a260 con 0x7f43c4103620 2024-09-15T22:37:03.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:03.995+0000 7f43c3e00640 1 -- 172.21.15.111:0/1704144390 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f43c402d830 con 0x7f43a80392a0 2024-09-15T22:37:04.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:04.000+0000 7f43bb400640 1 -- 172.21.15.111:0/1704144390 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f43c402d830 con 0x7f43a80392a0 2024-09-15T22:37:04.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:04.003+0000 7f43c3e00640 1 -- 172.21.15.111:0/1704144390 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f43a80392a0 msgr2=0x7f43a803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:04.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:04.003+0000 7f43c3e00640 1 --2- 172.21.15.111:0/1704144390 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f43a80392a0 0x7f43a803b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f43b00098c0 tx=0x7f43b0006db0 comp rx=0 tx=0).stop 2024-09-15T22:37:04.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:04.003+0000 7f43c3e00640 1 -- 172.21.15.111:0/1704144390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43c4103620 msgr2=0x7f43c4198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:04.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:04.003+0000 7f43c3e00640 1 --2- 172.21.15.111:0/1704144390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43c4103620 0x7f43c4198de0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f43b4004f60 tx=0x7f43b40048f0 comp rx=0 tx=0).stop 2024-09-15T22:37:04.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:04.003+0000 7f43c3e00640 1 -- 172.21.15.111:0/1704144390 shutdown_connections 2024-09-15T22:37:04.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:04.003+0000 7f43c3e00640 1 --2- 172.21.15.111:0/1704144390 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f43a80392a0 0x7f43a803b760 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:04.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:04.003+0000 7f43c3e00640 1 --2- 172.21.15.111:0/1704144390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43c4103620 0x7f43c4198de0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:04.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:04.003+0000 7f43c3e00640 1 -- 172.21.15.111:0/1704144390 >> 172.21.15.111:0/1704144390 conn(0x7f43c40fedd0 msgr2=0x7f43c40ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:04.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:04.003+0000 7f43c3e00640 1 -- 172.21.15.111:0/1704144390 shutdown_connections 2024-09-15T22:37:04.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:04.003+0000 7f43c3e00640 1 -- 172.21.15.111:0/1704144390 wait complete. 2024-09-15T22:37:04.880 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi121 2024-09-15T22:37:04.881 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-15T22:37:04.881 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/etc/ceph/ceph.conf 2024-09-15T22:37:04.900 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-15T22:37:04.900 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:37:04.958 INFO:tasks.cephadm:Adding host smithi121 to orchestrator... 2024-09-15T22:37:04.958 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph orch host add smithi121 2024-09-15T22:37:05.279 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:04 smithi111 ceph-mon[30629]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:37:05.279 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:05 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:05.279 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:05 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:37:05.279 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:05 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:05.279 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:05 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:37:05.279 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:05 smithi111 ceph-mon[30629]: Updating smithi111:/etc/ceph/ceph.conf 2024-09-15T22:37:05.279 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:05 smithi111 ceph-mon[30629]: Updating smithi111:/var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:37:05.758 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:06.295 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:06 smithi111 ceph-mon[30629]: Updating smithi111:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:37:06.296 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:06 smithi111 ceph-mon[30629]: Updating smithi111:/var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:37:06.296 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:06 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:06.296 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:06 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:06.296 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:06 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:06.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.406+0000 7fd6e6000640 1 -- 172.21.15.111:0/1832685007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6e0076720 msgr2=0x7fd6e0076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:06.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.406+0000 7fd6e6000640 1 --2- 172.21.15.111:0/1832685007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6e0076720 0x7fd6e0076b00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0009880 tx=0x7fd6d002a5d0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.406+0000 7fd6e6000640 1 -- 172.21.15.111:0/1832685007 shutdown_connections 2024-09-15T22:37:06.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.406+0000 7fd6e6000640 1 --2- 172.21.15.111:0/1832685007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6e0076720 0x7fd6e0076b00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:06.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.406+0000 7fd6e6000640 1 -- 172.21.15.111:0/1832685007 >> 172.21.15.111:0/1832685007 conn(0x7fd6e00ff4e0 msgr2=0x7fd6e0101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:06.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.406+0000 7fd6e6000640 1 -- 172.21.15.111:0/1832685007 shutdown_connections 2024-09-15T22:37:06.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.406+0000 7fd6e6000640 1 -- 172.21.15.111:0/1832685007 wait complete. 2024-09-15T22:37:06.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.407+0000 7fd6e6000640 1 Processor -- start 2024-09-15T22:37:06.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.408+0000 7fd6e6000640 1 -- start start 2024-09-15T22:37:06.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.408+0000 7fd6e6000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6e0076720 0x7fd6e01a33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:06.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.408+0000 7fd6e6000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e01122d0 con 0x7fd6e0076720 2024-09-15T22:37:06.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.408+0000 7fd6df400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6e0076720 0x7fd6e01a33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:06.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.408+0000 7fd6df400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6e0076720 0x7fd6e01a33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:32818/0 (socket says 172.21.15.111:32818) 2024-09-15T22:37:06.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.408+0000 7fd6df400640 1 -- 172.21.15.111:0/295779779 learned_addr learned my addr 172.21.15.111:0/295779779 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:37:06.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.409+0000 7fd6df400640 1 -- 172.21.15.111:0/295779779 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e01a3910 con 0x7fd6e0076720 2024-09-15T22:37:06.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.409+0000 7fd6df400640 1 --2- 172.21.15.111:0/295779779 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6e0076720 0x7fd6e01a33d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0009230 tx=0x7fd6d0003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:06.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.409+0000 7fd6dcc00640 1 -- 172.21.15.111:0/295779779 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6d003f070 con 0x7fd6e0076720 2024-09-15T22:37:06.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.409+0000 7fd6dcc00640 1 -- 172.21.15.111:0/295779779 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6d003bc80 con 0x7fd6e0076720 2024-09-15T22:37:06.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.409+0000 7fd6dcc00640 1 -- 172.21.15.111:0/295779779 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6d0035030 con 0x7fd6e0076720 2024-09-15T22:37:06.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.409+0000 7fd6e6000640 1 -- 172.21.15.111:0/295779779 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e01a3ba0 con 0x7fd6e0076720 2024-09-15T22:37:06.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.409+0000 7fd6e6000640 1 -- 172.21.15.111:0/295779779 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e019d590 con 0x7fd6e0076720 2024-09-15T22:37:06.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.410+0000 7fd6dcc00640 1 -- 172.21.15.111:0/295779779 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fd6d0042070 con 0x7fd6e0076720 2024-09-15T22:37:06.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.411+0000 7fd6e6000640 1 -- 172.21.15.111:0/295779779 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6e0077e60 con 0x7fd6e0076720 2024-09-15T22:37:06.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.411+0000 7fd6dcc00640 1 --2- 172.21.15.111:0/295779779 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd6c4039300 0x7fd6c403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:06.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.411+0000 7fd6dcc00640 1 -- 172.21.15.111:0/295779779 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd6d006ccf0 con 0x7fd6e0076720 2024-09-15T22:37:06.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.411+0000 7fd6d6a00640 1 --2- 172.21.15.111:0/295779779 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd6c4039300 0x7fd6c403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:06.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.412+0000 7fd6d6a00640 1 --2- 172.21.15.111:0/295779779 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd6c4039300 0x7fd6c403b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd6c80098c0 tx=0x7fd6c8006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:06.417 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.416+0000 7fd6dcc00640 1 -- 172.21.15.111:0/295779779 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd6d00372d0 con 0x7fd6e0076720 2024-09-15T22:37:06.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:06.594+0000 7fd6e6000640 1 -- 172.21.15.111:0/295779779 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi121", "target": ["mon-mgr", ""]}) -- 0x7fd6e019eac0 con 0x7fd6c4039300 2024-09-15T22:37:07.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:07 smithi111 ceph-mon[30629]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi121", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:37:08.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:08 smithi111 ceph-mon[30629]: Deploying cephadm binary to smithi121 2024-09-15T22:37:09.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:09.727+0000 7fd6dcc00640 1 -- 172.21.15.111:0/295779779 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fd6e019eac0 con 0x7fd6c4039300 2024-09-15T22:37:09.729 INFO:teuthology.orchestra.run.smithi111.stdout:Added host 'smithi121' with addr '172.21.15.121' 2024-09-15T22:37:09.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:09.729+0000 7fd6e6000640 1 -- 172.21.15.111:0/295779779 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd6c4039300 msgr2=0x7fd6c403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:09.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:09.729+0000 7fd6e6000640 1 --2- 172.21.15.111:0/295779779 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd6c4039300 0x7fd6c403b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd6c80098c0 tx=0x7fd6c8006db0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:09.729+0000 7fd6e6000640 1 -- 172.21.15.111:0/295779779 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6e0076720 msgr2=0x7fd6e01a33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:09.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:09.729+0000 7fd6e6000640 1 --2- 172.21.15.111:0/295779779 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6e0076720 0x7fd6e01a33d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0009230 tx=0x7fd6d0003d40 comp rx=0 tx=0).stop 2024-09-15T22:37:09.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:09.730+0000 7fd6e6000640 1 -- 172.21.15.111:0/295779779 shutdown_connections 2024-09-15T22:37:09.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:09.730+0000 7fd6e6000640 1 --2- 172.21.15.111:0/295779779 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd6c4039300 0x7fd6c403b7c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:09.730+0000 7fd6e6000640 1 --2- 172.21.15.111:0/295779779 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd6e0076720 0x7fd6e01a33d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:09.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:09.730+0000 7fd6e6000640 1 -- 172.21.15.111:0/295779779 >> 172.21.15.111:0/295779779 conn(0x7fd6e00ff4e0 msgr2=0x7fd6e00ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:09.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:09.730+0000 7fd6e6000640 1 -- 172.21.15.111:0/295779779 shutdown_connections 2024-09-15T22:37:09.732 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:09.730+0000 7fd6e6000640 1 -- 172.21.15.111:0/295779779 wait complete. 2024-09-15T22:37:10.531 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-15T22:37:10.891 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:10.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:10 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:10.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:10 smithi111 ceph-mon[30629]: Added host smithi121 2024-09-15T22:37:10.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:10 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:37:10.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:10 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:10.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:10 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:10.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:10 smithi111 ceph-mon[30629]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:11.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.497+0000 7f27c1800640 1 -- 172.21.15.111:0/528675186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27bc107660 msgr2=0x7f27bc107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:11.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.497+0000 7f27c1800640 1 --2- 172.21.15.111:0/528675186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27bc107660 0x7f27bc107a40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f27ac009880 tx=0x7f27ac02a5d0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.498+0000 7f27c1800640 1 -- 172.21.15.111:0/528675186 shutdown_connections 2024-09-15T22:37:11.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.498+0000 7f27c1800640 1 --2- 172.21.15.111:0/528675186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27bc107660 0x7f27bc107a40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.498+0000 7f27c1800640 1 -- 172.21.15.111:0/528675186 >> 172.21.15.111:0/528675186 conn(0x7f27bc0fd2b0 msgr2=0x7f27bc0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:11.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.498+0000 7f27c1800640 1 -- 172.21.15.111:0/528675186 shutdown_connections 2024-09-15T22:37:11.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.498+0000 7f27c1800640 1 -- 172.21.15.111:0/528675186 wait complete. 2024-09-15T22:37:11.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.499+0000 7f27c1800640 1 Processor -- start 2024-09-15T22:37:11.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.499+0000 7f27c1800640 1 -- start start 2024-09-15T22:37:11.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.499+0000 7f27c1800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27bc107660 0x7f27bc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:11.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.499+0000 7f27c1800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc10bc70 con 0x7f27bc107660 2024-09-15T22:37:11.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.500+0000 7f27baa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27bc107660 0x7f27bc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:11.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.500+0000 7f27baa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27bc107660 0x7f27bc0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47270/0 (socket says 172.21.15.111:47270) 2024-09-15T22:37:11.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.500+0000 7f27baa00640 1 -- 172.21.15.111:0/1498934783 learned_addr learned my addr 172.21.15.111:0/1498934783 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:37:11.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.500+0000 7f27baa00640 1 -- 172.21.15.111:0/1498934783 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27bc0763e0 con 0x7f27bc107660 2024-09-15T22:37:11.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.501+0000 7f27baa00640 1 --2- 172.21.15.111:0/1498934783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27bc107660 0x7f27bc0744f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f27ac009230 tx=0x7f27ac003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:11.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.501+0000 7f27b3400640 1 -- 172.21.15.111:0/1498934783 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27ac03f070 con 0x7f27bc107660 2024-09-15T22:37:11.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.501+0000 7f27b3400640 1 -- 172.21.15.111:0/1498934783 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27ac03bd40 con 0x7f27bc107660 2024-09-15T22:37:11.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.501+0000 7f27b3400640 1 -- 172.21.15.111:0/1498934783 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27ac035030 con 0x7f27bc107660 2024-09-15T22:37:11.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.502+0000 7f27c1800640 1 -- 172.21.15.111:0/1498934783 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27bc074a90 con 0x7f27bc107660 2024-09-15T22:37:11.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.502+0000 7f27c1800640 1 -- 172.21.15.111:0/1498934783 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27bc074f50 con 0x7f27bc107660 2024-09-15T22:37:11.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.503+0000 7f27c1800640 1 -- 172.21.15.111:0/1498934783 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27bc102d10 con 0x7f27bc107660 2024-09-15T22:37:11.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.503+0000 7f27b3400640 1 -- 172.21.15.111:0/1498934783 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f27ac042070 con 0x7f27bc107660 2024-09-15T22:37:11.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.503+0000 7f27b3400640 1 --2- 172.21.15.111:0/1498934783 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f27a00392a0 0x7f27a003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:11.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.503+0000 7f27b3400640 1 -- 172.21.15.111:0/1498934783 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f27ac06d000 con 0x7f27bc107660 2024-09-15T22:37:11.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.503+0000 7f27ba000640 1 --2- 172.21.15.111:0/1498934783 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f27a00392a0 0x7f27a003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:11.505 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.504+0000 7f27ba000640 1 --2- 172.21.15.111:0/1498934783 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f27a00392a0 0x7f27a003b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f27a80098c0 tx=0x7f27a8006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:11.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.508+0000 7f27b3400640 1 -- 172.21.15.111:0/1498934783 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f27ac031570 con 0x7f27bc107660 2024-09-15T22:37:11.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.697+0000 7f27c1800640 1 -- 172.21.15.111:0/1498934783 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f27bc075a20 con 0x7f27a00392a0 2024-09-15T22:37:11.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.699+0000 7f27b3400640 1 -- 172.21.15.111:0/1498934783 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f27bc075a20 con 0x7f27a00392a0 2024-09-15T22:37:11.700 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:37:11.700 INFO:teuthology.orchestra.run.smithi111.stdout:[{"addr": "172.21.15.111", "hostname": "smithi111", "labels": [], "status": ""}, {"addr": "172.21.15.121", "hostname": "smithi121", "labels": [], "status": ""}] 2024-09-15T22:37:11.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.701+0000 7f27c1800640 1 -- 172.21.15.111:0/1498934783 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f27a00392a0 msgr2=0x7f27a003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:11.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.701+0000 7f27c1800640 1 --2- 172.21.15.111:0/1498934783 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f27a00392a0 0x7f27a003b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f27a80098c0 tx=0x7f27a8006db0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.701+0000 7f27c1800640 1 -- 172.21.15.111:0/1498934783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27bc107660 msgr2=0x7f27bc0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:11.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.701+0000 7f27c1800640 1 --2- 172.21.15.111:0/1498934783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27bc107660 0x7f27bc0744f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f27ac009230 tx=0x7f27ac003d80 comp rx=0 tx=0).stop 2024-09-15T22:37:11.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.701+0000 7f27c1800640 1 -- 172.21.15.111:0/1498934783 shutdown_connections 2024-09-15T22:37:11.703 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.701+0000 7f27c1800640 1 --2- 172.21.15.111:0/1498934783 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f27a00392a0 0x7f27a003b760 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.701+0000 7f27c1800640 1 --2- 172.21.15.111:0/1498934783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27bc107660 0x7f27bc0744f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:11.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.701+0000 7f27c1800640 1 -- 172.21.15.111:0/1498934783 >> 172.21.15.111:0/1498934783 conn(0x7f27bc0fd2b0 msgr2=0x7f27bc0fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:11.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.701+0000 7f27c1800640 1 -- 172.21.15.111:0/1498934783 shutdown_connections 2024-09-15T22:37:11.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:11.701+0000 7f27c1800640 1 -- 172.21.15.111:0/1498934783 wait complete. 2024-09-15T22:37:12.384 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:12 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:12.424 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi192 2024-09-15T22:37:12.424 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T22:37:12.424 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/etc/ceph/ceph.conf 2024-09-15T22:37:12.443 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T22:37:12.443 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:37:12.501 INFO:tasks.cephadm:Adding host smithi192 to orchestrator... 2024-09-15T22:37:12.501 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph orch host add smithi192 2024-09-15T22:37:12.863 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:13.271 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:13 smithi111 ceph-mon[30629]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:37:13.271 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:13 smithi111 ceph-mon[30629]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:13.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.381+0000 7f6c79200640 1 -- 172.21.15.111:0/2739215923 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c74107460 msgr2=0x7f6c74107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:13.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.381+0000 7f6c79200640 1 --2- 172.21.15.111:0/2739215923 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c74107460 0x7f6c74107840 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009880 tx=0x7f6c6402a5d0 comp rx=0 tx=0).stop 2024-09-15T22:37:13.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.381+0000 7f6c79200640 1 -- 172.21.15.111:0/2739215923 shutdown_connections 2024-09-15T22:37:13.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.381+0000 7f6c79200640 1 --2- 172.21.15.111:0/2739215923 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c74107460 0x7f6c74107840 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:13.383 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.381+0000 7f6c79200640 1 -- 172.21.15.111:0/2739215923 >> 172.21.15.111:0/2739215923 conn(0x7f6c740fd0b0 msgr2=0x7f6c740ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:13.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.381+0000 7f6c79200640 1 -- 172.21.15.111:0/2739215923 shutdown_connections 2024-09-15T22:37:13.384 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.381+0000 7f6c79200640 1 -- 172.21.15.111:0/2739215923 wait complete. 2024-09-15T22:37:13.385 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.382+0000 7f6c79200640 1 Processor -- start 2024-09-15T22:37:13.385 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.382+0000 7f6c79200640 1 -- start start 2024-09-15T22:37:13.386 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.382+0000 7f6c79200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c74107460 0x7f6c74196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:13.386 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.382+0000 7f6c79200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7410ba90 con 0x7f6c74107460 2024-09-15T22:37:13.386 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.383+0000 7f6c73400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c74107460 0x7f6c74196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:13.386 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.383+0000 7f6c73400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c74107460 0x7f6c74196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47288/0 (socket says 172.21.15.111:47288) 2024-09-15T22:37:13.386 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.383+0000 7f6c73400640 1 -- 172.21.15.111:0/3803071291 learned_addr learned my addr 172.21.15.111:0/3803071291 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:37:13.386 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.383+0000 7f6c73400640 1 -- 172.21.15.111:0/3803071291 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c741974b0 con 0x7f6c74107460 2024-09-15T22:37:13.386 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.383+0000 7f6c73400640 1 --2- 172.21.15.111:0/3803071291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c74107460 0x7f6c74196f70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009230 tx=0x7f6c64003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:13.386 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.384+0000 7f6c70c00640 1 -- 172.21.15.111:0/3803071291 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c6403f070 con 0x7f6c74107460 2024-09-15T22:37:13.386 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.384+0000 7f6c70c00640 1 -- 172.21.15.111:0/3803071291 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c6403bd40 con 0x7f6c74107460 2024-09-15T22:37:13.386 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.384+0000 7f6c70c00640 1 -- 172.21.15.111:0/3803071291 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c64035030 con 0x7f6c74107460 2024-09-15T22:37:13.386 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.384+0000 7f6c79200640 1 -- 172.21.15.111:0/3803071291 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c741977a0 con 0x7f6c74107460 2024-09-15T22:37:13.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.384+0000 7f6c79200640 1 -- 172.21.15.111:0/3803071291 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c7419b840 con 0x7f6c74107460 2024-09-15T22:37:13.387 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.385+0000 7f6c79200640 1 -- 172.21.15.111:0/3803071291 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c74102b10 con 0x7f6c74107460 2024-09-15T22:37:13.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.385+0000 7f6c70c00640 1 -- 172.21.15.111:0/3803071291 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f6c64042070 con 0x7f6c74107460 2024-09-15T22:37:13.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.386+0000 7f6c70c00640 1 --2- 172.21.15.111:0/3803071291 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6c58039300 0x7f6c5803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:13.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.386+0000 7f6c70c00640 1 -- 172.21.15.111:0/3803071291 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6c6406cf60 con 0x7f6c74107460 2024-09-15T22:37:13.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.386+0000 7f6c6be00640 1 --2- 172.21.15.111:0/3803071291 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6c58039300 0x7f6c5803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:13.388 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.387+0000 7f6c6be00640 1 --2- 172.21.15.111:0/3803071291 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6c58039300 0x7f6c5803b7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c0098c0 tx=0x7f6c5c006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:13.392 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.390+0000 7f6c70c00640 1 -- 172.21.15.111:0/3803071291 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6c64030d60 con 0x7f6c74107460 2024-09-15T22:37:13.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:13.571+0000 7f6c79200640 1 -- 172.21.15.111:0/3803071291 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}) -- 0x7f6c741078c0 con 0x7f6c58039300 2024-09-15T22:37:14.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:14 smithi111 ceph-mon[30629]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi192", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:37:14.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:14 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:14.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:14 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:14.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:14 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:14.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:14 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:14.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:14 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi121", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:37:14.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:14 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:14.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:14 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:37:15.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:15 smithi111 ceph-mon[30629]: Updating smithi121:/etc/ceph/ceph.conf 2024-09-15T22:37:15.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:15 smithi111 ceph-mon[30629]: Updating smithi121:/var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:37:15.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:15 smithi111 ceph-mon[30629]: Updating smithi121:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:37:15.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:15 smithi111 ceph-mon[30629]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:15.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:15 smithi111 ceph-mon[30629]: Updating smithi121:/var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:37:15.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:15 smithi111 ceph-mon[30629]: Deploying cephadm binary to smithi192 2024-09-15T22:37:15.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:15 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:15.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:15 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:15.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:15 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:16.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:16.822+0000 7f6c70c00640 1 -- 172.21.15.111:0/3803071291 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f6c741078c0 con 0x7f6c58039300 2024-09-15T22:37:16.824 INFO:teuthology.orchestra.run.smithi111.stdout:Added host 'smithi192' with addr '172.21.15.192' 2024-09-15T22:37:16.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:16.825+0000 7f6c79200640 1 -- 172.21.15.111:0/3803071291 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6c58039300 msgr2=0x7f6c5803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:16.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:16.825+0000 7f6c79200640 1 --2- 172.21.15.111:0/3803071291 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6c58039300 0x7f6c5803b7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c0098c0 tx=0x7f6c5c006db0 comp rx=0 tx=0).stop 2024-09-15T22:37:16.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:16.825+0000 7f6c79200640 1 -- 172.21.15.111:0/3803071291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c74107460 msgr2=0x7f6c74196f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:16.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:16.825+0000 7f6c79200640 1 --2- 172.21.15.111:0/3803071291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c74107460 0x7f6c74196f70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009230 tx=0x7f6c64003d80 comp rx=0 tx=0).stop 2024-09-15T22:37:16.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:16.825+0000 7f6c79200640 1 -- 172.21.15.111:0/3803071291 shutdown_connections 2024-09-15T22:37:16.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:16.825+0000 7f6c79200640 1 --2- 172.21.15.111:0/3803071291 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6c58039300 0x7f6c5803b7c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:16.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:16.825+0000 7f6c79200640 1 --2- 172.21.15.111:0/3803071291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c74107460 0x7f6c74196f70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:16.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:16.825+0000 7f6c79200640 1 -- 172.21.15.111:0/3803071291 >> 172.21.15.111:0/3803071291 conn(0x7f6c740fd0b0 msgr2=0x7f6c740fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:16.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:16.825+0000 7f6c79200640 1 -- 172.21.15.111:0/3803071291 shutdown_connections 2024-09-15T22:37:16.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:16.825+0000 7f6c79200640 1 -- 172.21.15.111:0/3803071291 wait complete. 2024-09-15T22:37:17.536 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:17 smithi111 ceph-mon[30629]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:17.536 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:17 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:17.536 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:17 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:37:17.536 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:17 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:17.536 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:17 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:17.572 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-15T22:37:17.931 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:18.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.453+0000 7faadf200640 1 -- 172.21.15.111:0/1451736618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faae0076080 msgr2=0x7faae0074420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:18.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.453+0000 7faadf200640 1 --2- 172.21.15.111:0/1451736618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faae0076080 0x7faae0074420 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faad0009880 tx=0x7faad002a5d0 comp rx=0 tx=0).stop 2024-09-15T22:37:18.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.453+0000 7faadf200640 1 -- 172.21.15.111:0/1451736618 shutdown_connections 2024-09-15T22:37:18.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.453+0000 7faadf200640 1 --2- 172.21.15.111:0/1451736618 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faae0076080 0x7faae0074420 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:18.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.453+0000 7faadf200640 1 -- 172.21.15.111:0/1451736618 >> 172.21.15.111:0/1451736618 conn(0x7faae0078130 msgr2=0x7faae0078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:18.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.453+0000 7faadf200640 1 -- 172.21.15.111:0/1451736618 shutdown_connections 2024-09-15T22:37:18.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.453+0000 7faadf200640 1 -- 172.21.15.111:0/1451736618 wait complete. 2024-09-15T22:37:18.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.454+0000 7faadf200640 1 Processor -- start 2024-09-15T22:37:18.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.454+0000 7faadf200640 1 -- start start 2024-09-15T22:37:18.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.454+0000 7faadf200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faae0076080 0x7faae019d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:18.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.454+0000 7faadf200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faae01120b0 con 0x7faae0076080 2024-09-15T22:37:18.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.454+0000 7faadde00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faae0076080 0x7faae019d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:18.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.455+0000 7faadde00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faae0076080 0x7faae019d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47316/0 (socket says 172.21.15.111:47316) 2024-09-15T22:37:18.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.455+0000 7faadde00640 1 -- 172.21.15.111:0/875622213 learned_addr learned my addr 172.21.15.111:0/875622213 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:37:18.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.455+0000 7faadde00640 1 -- 172.21.15.111:0/875622213 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae019dae0 con 0x7faae0076080 2024-09-15T22:37:18.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.455+0000 7faadde00640 1 --2- 172.21.15.111:0/875622213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faae0076080 0x7faae019d5a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faad0009230 tx=0x7faad0003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:18.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.455+0000 7faad7400640 1 -- 172.21.15.111:0/875622213 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faad003f070 con 0x7faae0076080 2024-09-15T22:37:18.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.455+0000 7faad7400640 1 -- 172.21.15.111:0/875622213 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faad003bd40 con 0x7faae0076080 2024-09-15T22:37:18.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.455+0000 7faadf200640 1 -- 172.21.15.111:0/875622213 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae019dd70 con 0x7faae0076080 2024-09-15T22:37:18.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.456+0000 7faad7400640 1 -- 172.21.15.111:0/875622213 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faad0035030 con 0x7faae0076080 2024-09-15T22:37:18.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.456+0000 7faadf200640 1 -- 172.21.15.111:0/875622213 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae019e1d0 con 0x7faae0076080 2024-09-15T22:37:18.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.457+0000 7faad7400640 1 -- 172.21.15.111:0/875622213 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7faad0042070 con 0x7faae0076080 2024-09-15T22:37:18.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.457+0000 7faadf200640 1 -- 172.21.15.111:0/875622213 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faae0075930 con 0x7faae0076080 2024-09-15T22:37:18.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.457+0000 7faad7400640 1 --2- 172.21.15.111:0/875622213 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faac4039300 0x7faac403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:18.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.457+0000 7faad7400640 1 -- 172.21.15.111:0/875622213 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faad006cfc0 con 0x7faae0076080 2024-09-15T22:37:18.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.457+0000 7faad5400640 1 --2- 172.21.15.111:0/875622213 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faac4039300 0x7faac403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:18.460 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.458+0000 7faad5400640 1 --2- 172.21.15.111:0/875622213 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faac4039300 0x7faac403b7c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faac800ad30 tx=0x7faac80093f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:18.464 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.462+0000 7faad7400640 1 -- 172.21.15.111:0/875622213 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faad00412c0 con 0x7faae0076080 2024-09-15T22:37:18.642 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.640+0000 7faadf200640 1 -- 172.21.15.111:0/875622213 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faae019e990 con 0x7faac4039300 2024-09-15T22:37:18.642 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:18 smithi111 ceph-mon[30629]: Added host smithi192 2024-09-15T22:37:18.642 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:18 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:18.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.641+0000 7faad7400640 1 -- 172.21.15.111:0/875622213 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7faae019e990 con 0x7faac4039300 2024-09-15T22:37:18.643 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:37:18.643 INFO:teuthology.orchestra.run.smithi111.stdout:[{"addr": "172.21.15.111", "hostname": "smithi111", "labels": [], "status": ""}, {"addr": "172.21.15.121", "hostname": "smithi121", "labels": [], "status": ""}, {"addr": "172.21.15.192", "hostname": "smithi192", "labels": [], "status": ""}] 2024-09-15T22:37:18.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.643+0000 7faadf200640 1 -- 172.21.15.111:0/875622213 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faac4039300 msgr2=0x7faac403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:18.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.643+0000 7faadf200640 1 --2- 172.21.15.111:0/875622213 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faac4039300 0x7faac403b7c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faac800ad30 tx=0x7faac80093f0 comp rx=0 tx=0).stop 2024-09-15T22:37:18.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.643+0000 7faadf200640 1 -- 172.21.15.111:0/875622213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faae0076080 msgr2=0x7faae019d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:18.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.643+0000 7faadf200640 1 --2- 172.21.15.111:0/875622213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faae0076080 0x7faae019d5a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faad0009230 tx=0x7faad0003d80 comp rx=0 tx=0).stop 2024-09-15T22:37:18.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.644+0000 7faadf200640 1 -- 172.21.15.111:0/875622213 shutdown_connections 2024-09-15T22:37:18.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.644+0000 7faadf200640 1 --2- 172.21.15.111:0/875622213 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faac4039300 0x7faac403b7c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:18.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.644+0000 7faadf200640 1 --2- 172.21.15.111:0/875622213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faae0076080 0x7faae019d5a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:18.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.644+0000 7faadf200640 1 -- 172.21.15.111:0/875622213 >> 172.21.15.111:0/875622213 conn(0x7faae0078130 msgr2=0x7faae0104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:18.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.644+0000 7faadf200640 1 -- 172.21.15.111:0/875622213 shutdown_connections 2024-09-15T22:37:18.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:18.644+0000 7faadf200640 1 -- 172.21.15.111:0/875622213 wait complete. 2024-09-15T22:37:19.334 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-15T22:37:19.334 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-15T22:37:19.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:19 smithi111 ceph-mon[30629]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:19.690 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:20.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.251+0000 7f1123c00640 1 -- 172.21.15.111:0/623299343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f111c107660 msgr2=0x7f111c107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:20.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.251+0000 7f1123c00640 1 --2- 172.21.15.111:0/623299343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f111c107660 0x7f111c107a40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1110009880 tx=0x7f111002a5d0 comp rx=0 tx=0).stop 2024-09-15T22:37:20.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.251+0000 7f1123c00640 1 -- 172.21.15.111:0/623299343 shutdown_connections 2024-09-15T22:37:20.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.251+0000 7f1123c00640 1 --2- 172.21.15.111:0/623299343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f111c107660 0x7f111c107a40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:20.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.251+0000 7f1123c00640 1 -- 172.21.15.111:0/623299343 >> 172.21.15.111:0/623299343 conn(0x7f111c0fd2b0 msgr2=0x7f111c0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:20.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.251+0000 7f1123c00640 1 -- 172.21.15.111:0/623299343 shutdown_connections 2024-09-15T22:37:20.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.251+0000 7f1123c00640 1 -- 172.21.15.111:0/623299343 wait complete. 2024-09-15T22:37:20.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.252+0000 7f1123c00640 1 Processor -- start 2024-09-15T22:37:20.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.252+0000 7f1123c00640 1 -- start start 2024-09-15T22:37:20.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.252+0000 7f1123c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f111c107660 0x7f111c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:20.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.252+0000 7f1123c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f111c10bc70 con 0x7f111c107660 2024-09-15T22:37:20.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.253+0000 7f1121200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f111c107660 0x7f111c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:20.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.253+0000 7f1121200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f111c107660 0x7f111c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47340/0 (socket says 172.21.15.111:47340) 2024-09-15T22:37:20.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.253+0000 7f1121200640 1 -- 172.21.15.111:0/3440763325 learned_addr learned my addr 172.21.15.111:0/3440763325 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:37:20.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.253+0000 7f1121200640 1 -- 172.21.15.111:0/3440763325 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f111c0763e0 con 0x7f111c107660 2024-09-15T22:37:20.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.253+0000 7f1121200640 1 --2- 172.21.15.111:0/3440763325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f111c107660 0x7f111c0744f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1110009230 tx=0x7f1110003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:20.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.254+0000 7f1119600640 1 -- 172.21.15.111:0/3440763325 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f111003f070 con 0x7f111c107660 2024-09-15T22:37:20.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.254+0000 7f1119600640 1 -- 172.21.15.111:0/3440763325 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f111003bd40 con 0x7f111c107660 2024-09-15T22:37:20.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.254+0000 7f1119600640 1 -- 172.21.15.111:0/3440763325 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1110035030 con 0x7f111c107660 2024-09-15T22:37:20.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.254+0000 7f1123c00640 1 -- 172.21.15.111:0/3440763325 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f111c074a90 con 0x7f111c107660 2024-09-15T22:37:20.257 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.254+0000 7f1123c00640 1 -- 172.21.15.111:0/3440763325 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f111c074f50 con 0x7f111c107660 2024-09-15T22:37:20.257 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.255+0000 7f1119600640 1 -- 172.21.15.111:0/3440763325 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f1110042070 con 0x7f111c107660 2024-09-15T22:37:20.257 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.255+0000 7f1123c00640 1 -- 172.21.15.111:0/3440763325 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f111c102d10 con 0x7f111c107660 2024-09-15T22:37:20.257 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.255+0000 7f1119600640 1 --2- 172.21.15.111:0/3440763325 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1104039300 0x7f110403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:20.258 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.255+0000 7f1119600640 1 -- 172.21.15.111:0/3440763325 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f111006cf20 con 0x7f111c107660 2024-09-15T22:37:20.258 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.256+0000 7f111be00640 1 --2- 172.21.15.111:0/3440763325 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1104039300 0x7f110403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:20.258 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.256+0000 7f111be00640 1 --2- 172.21.15.111:0/3440763325 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1104039300 0x7f110403b7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f11080098c0 tx=0x7f1108006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:20.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.261+0000 7f1119600640 1 -- 172.21.15.111:0/3440763325 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1110030d90 con 0x7f111c107660 2024-09-15T22:37:20.445 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:20.443+0000 7f1123c00640 1 -- 172.21.15.111:0/3440763325 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f111c075a20 con 0x7f111c107660 2024-09-15T22:37:20.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:20 smithi111 ceph-mon[30629]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:37:21.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:21.431+0000 7f1119600640 1 -- 172.21.15.111:0/3440763325 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f1110036df0 con 0x7f111c107660 2024-09-15T22:37:21.433 INFO:teuthology.orchestra.run.smithi111.stderr:adjusted tunables profile to default 2024-09-15T22:37:21.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:21.434+0000 7f1123c00640 1 -- 172.21.15.111:0/3440763325 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1104039300 msgr2=0x7f110403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:21.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:21.434+0000 7f1123c00640 1 --2- 172.21.15.111:0/3440763325 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1104039300 0x7f110403b7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f11080098c0 tx=0x7f1108006db0 comp rx=0 tx=0).stop 2024-09-15T22:37:21.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:21.434+0000 7f1123c00640 1 -- 172.21.15.111:0/3440763325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f111c107660 msgr2=0x7f111c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:21.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:21.434+0000 7f1123c00640 1 --2- 172.21.15.111:0/3440763325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f111c107660 0x7f111c0744f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1110009230 tx=0x7f1110003d80 comp rx=0 tx=0).stop 2024-09-15T22:37:21.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:21.434+0000 7f1123c00640 1 -- 172.21.15.111:0/3440763325 shutdown_connections 2024-09-15T22:37:21.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:21.434+0000 7f1123c00640 1 --2- 172.21.15.111:0/3440763325 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1104039300 0x7f110403b7c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:21.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:21.434+0000 7f1123c00640 1 --2- 172.21.15.111:0/3440763325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f111c107660 0x7f111c0744f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:21.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:21.434+0000 7f1123c00640 1 -- 172.21.15.111:0/3440763325 >> 172.21.15.111:0/3440763325 conn(0x7f111c0fd2b0 msgr2=0x7f111c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:21.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:21.434+0000 7f1123c00640 1 -- 172.21.15.111:0/3440763325 shutdown_connections 2024-09-15T22:37:21.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:21.434+0000 7f1123c00640 1 -- 172.21.15.111:0/3440763325 wait complete. 2024-09-15T22:37:21.873 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:21 smithi111 ceph-mon[30629]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:21.874 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:21 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/3440763325' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-15T22:37:21.874 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:21 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:21.874 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:21 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:21.874 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:21 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:21.874 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:21 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:21.874 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:21 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:37:21.874 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:21 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:21.874 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:21 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:37:22.418 INFO:tasks.cephadm:Adding mon.a on smithi111 2024-09-15T22:37:22.418 INFO:tasks.cephadm:Adding mon.b on smithi121 2024-09-15T22:37:22.418 INFO:tasks.cephadm:Adding mon.c on smithi192 2024-09-15T22:37:22.418 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi111:172.21.15.111=a;smithi121:172.21.15.121=b;smithi192:172.21.15.192=c' 2024-09-15T22:37:22.649 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:37:22.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:22 smithi111 ceph-mon[30629]: Updating smithi192:/etc/ceph/ceph.conf 2024-09-15T22:37:22.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:22 smithi111 ceph-mon[30629]: Updating smithi192:/var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:37:22.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:22 smithi111 ceph-mon[30629]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:37:22.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:22 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/3440763325' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-15T22:37:22.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:22 smithi111 ceph-mon[30629]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:37:22.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:22 smithi111 ceph-mon[30629]: Updating smithi192:/var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:37:22.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:22 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:22.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:22 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:22.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:22 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:23.087 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.077+0000 7fa975d87640 1 -- 172.21.15.192:0/219312574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa970107690 msgr2=0x7fa970107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:23.087 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.077+0000 7fa975d87640 1 --2- 172.21.15.192:0/219312574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa970107690 0x7fa970107a70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa960009c20 tx=0x7fa96002ab20 comp rx=0 tx=0).stop 2024-09-15T22:37:23.087 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.077+0000 7fa975d87640 1 -- 172.21.15.192:0/219312574 shutdown_connections 2024-09-15T22:37:23.087 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.077+0000 7fa975d87640 1 --2- 172.21.15.192:0/219312574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa970107690 0x7fa970107a70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:23.087 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.077+0000 7fa975d87640 1 -- 172.21.15.192:0/219312574 >> 172.21.15.192:0/219312574 conn(0x7fa9700fb920 msgr2=0x7fa9700fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:23.088 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.077+0000 7fa975d87640 1 -- 172.21.15.192:0/219312574 shutdown_connections 2024-09-15T22:37:23.088 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.077+0000 7fa975d87640 1 -- 172.21.15.192:0/219312574 wait complete. 2024-09-15T22:37:23.088 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.077+0000 7fa975d87640 1 Processor -- start 2024-09-15T22:37:23.088 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.077+0000 7fa975d87640 1 -- start start 2024-09-15T22:37:23.088 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.078+0000 7fa975d87640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa970107690 0x7fa970197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:23.088 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.078+0000 7fa975d87640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa97010bc80 con 0x7fa970107690 2024-09-15T22:37:23.088 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.078+0000 7fa96f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa970107690 0x7fa970197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:23.088 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.078+0000 7fa96f7fe640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa970107690 0x7fa970197120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.192:58564/0 (socket says 172.21.15.192:58564) 2024-09-15T22:37:23.088 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.078+0000 7fa96f7fe640 1 -- 172.21.15.192:0/1251207395 learned_addr learned my addr 172.21.15.192:0/1251207395 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-09-15T22:37:23.088 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.078+0000 7fa96f7fe640 1 -- 172.21.15.192:0/1251207395 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa970197660 con 0x7fa970107690 2024-09-15T22:37:23.088 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.078+0000 7fa96f7fe640 1 --2- 172.21.15.192:0/1251207395 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa970107690 0x7fa970197120 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa960030040 tx=0x7fa96002eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:23.088 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.079+0000 7fa96cff9640 1 -- 172.21.15.192:0/1251207395 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa960047070 con 0x7fa970107690 2024-09-15T22:37:23.088 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.079+0000 7fa975d87640 1 -- 172.21.15.192:0/1251207395 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa970197950 con 0x7fa970107690 2024-09-15T22:37:23.089 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.079+0000 7fa96cff9640 1 -- 172.21.15.192:0/1251207395 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9600368e0 con 0x7fa970107690 2024-09-15T22:37:23.089 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.079+0000 7fa96cff9640 1 -- 172.21.15.192:0/1251207395 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa960042420 con 0x7fa970107690 2024-09-15T22:37:23.089 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.079+0000 7fa975d87640 1 -- 172.21.15.192:0/1251207395 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa97019b9f0 con 0x7fa970107690 2024-09-15T22:37:23.089 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.080+0000 7fa975d87640 1 -- 172.21.15.192:0/1251207395 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa970102d30 con 0x7fa970107690 2024-09-15T22:37:23.089 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.080+0000 7fa96cff9640 1 -- 172.21.15.192:0/1251207395 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7fa96004a070 con 0x7fa970107690 2024-09-15T22:37:23.089 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.080+0000 7fa96cff9640 1 --2- 172.21.15.192:0/1251207395 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa9580392a0 0x7fa95803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:23.089 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.080+0000 7fa96cff9640 1 -- 172.21.15.192:0/1251207395 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa96003d030 con 0x7fa970107690 2024-09-15T22:37:23.089 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.080+0000 7fa96effd640 1 --2- 172.21.15.192:0/1251207395 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa9580392a0 0x7fa95803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:23.089 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.081+0000 7fa96effd640 1 --2- 172.21.15.192:0/1251207395 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa9580392a0 0x7fa95803b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa9640098c0 tx=0x7fa964006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:23.089 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.086+0000 7fa96cff9640 1 -- 172.21.15.192:0/1251207395 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa960032030 con 0x7fa970107690 2024-09-15T22:37:23.271 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.269+0000 7fa975d87640 1 -- 172.21.15.192:0/1251207395 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi111:172.21.15.111=a;smithi121:172.21.15.121=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}) -- 0x7fa970107b50 con 0x7fa9580392a0 2024-09-15T22:37:23.278 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.276+0000 7fa96cff9640 1 -- 172.21.15.192:0/1251207395 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa970107b50 con 0x7fa9580392a0 2024-09-15T22:37:23.278 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mon update... 2024-09-15T22:37:23.280 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.279+0000 7fa975d87640 1 -- 172.21.15.192:0/1251207395 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa9580392a0 msgr2=0x7fa95803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:23.280 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.279+0000 7fa975d87640 1 --2- 172.21.15.192:0/1251207395 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa9580392a0 0x7fa95803b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa9640098c0 tx=0x7fa964006db0 comp rx=0 tx=0).stop 2024-09-15T22:37:23.280 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.279+0000 7fa975d87640 1 -- 172.21.15.192:0/1251207395 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa970107690 msgr2=0x7fa970197120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:23.280 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.279+0000 7fa975d87640 1 --2- 172.21.15.192:0/1251207395 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa970107690 0x7fa970197120 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa960030040 tx=0x7fa96002eb50 comp rx=0 tx=0).stop 2024-09-15T22:37:23.280 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.279+0000 7fa975d87640 1 -- 172.21.15.192:0/1251207395 shutdown_connections 2024-09-15T22:37:23.281 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.279+0000 7fa975d87640 1 --2- 172.21.15.192:0/1251207395 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa9580392a0 0x7fa95803b760 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:23.281 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.279+0000 7fa975d87640 1 --2- 172.21.15.192:0/1251207395 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa970107690 0x7fa970197120 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:23.281 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.279+0000 7fa975d87640 1 -- 172.21.15.192:0/1251207395 >> 172.21.15.192:0/1251207395 conn(0x7fa9700fb920 msgr2=0x7fa9700fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:23.281 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.279+0000 7fa975d87640 1 -- 172.21.15.192:0/1251207395 shutdown_connections 2024-09-15T22:37:23.281 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:23.279+0000 7fa975d87640 1 -- 172.21.15.192:0/1251207395 wait complete. 2024-09-15T22:37:23.873 DEBUG:teuthology.orchestra.run.smithi121:mon.b> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mon.b.service 2024-09-15T22:37:23.876 DEBUG:teuthology.orchestra.run.smithi192:mon.c> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mon.c.service 2024-09-15T22:37:23.878 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-15T22:37:23.878 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-15T22:37:23.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:23 smithi111 ceph-mon[30629]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:23.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:23 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:23.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:23 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:37:23.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:23 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:23.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:23 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:37:23.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:23 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:23.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:23 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:37:23.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:23 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:24.574 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:37:24.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:24 smithi111 ceph-mon[30629]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi111:172.21.15.111=a;smithi121:172.21.15.121=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:37:24.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:24 smithi111 ceph-mon[30629]: Saving service mon spec with placement smithi111:172.21.15.111=a;smithi121:172.21.15.121=b;smithi192:172.21.15.192=c;count:3 2024-09-15T22:37:24.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:24 smithi111 ceph-mon[30629]: Deploying daemon mon.c on smithi192 2024-09-15T22:37:25.426 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.009+0000 7f93c8dc3640 1 -- 172.21.15.192:0/1623257821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93c4100570 msgr2=0x7f93c4100950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:25.427 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.009+0000 7f93c8dc3640 1 --2- 172.21.15.192:0/1623257821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93c4100570 0x7f93c4100950 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009880 tx=0x7f93b402a670 comp rx=0 tx=0).stop 2024-09-15T22:37:25.427 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.009+0000 7f93c8dc3640 1 -- 172.21.15.192:0/1623257821 shutdown_connections 2024-09-15T22:37:25.427 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.009+0000 7f93c8dc3640 1 --2- 172.21.15.192:0/1623257821 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93c4100570 0x7f93c4100950 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:25.427 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.009+0000 7f93c8dc3640 1 -- 172.21.15.192:0/1623257821 >> 172.21.15.192:0/1623257821 conn(0x7f93c40fc1a0 msgr2=0x7f93c40fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:25.427 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.009+0000 7f93c8dc3640 1 -- 172.21.15.192:0/1623257821 shutdown_connections 2024-09-15T22:37:25.427 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.010+0000 7f93c8dc3640 1 -- 172.21.15.192:0/1623257821 wait complete. 2024-09-15T22:37:25.427 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.010+0000 7f93c8dc3640 1 Processor -- start 2024-09-15T22:37:25.427 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.011+0000 7f93c8dc3640 1 -- start start 2024-09-15T22:37:25.427 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.011+0000 7f93c8dc3640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93c4100570 0x7f93c41a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:25.427 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.011+0000 7f93c8dc3640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f93c4112300 con 0x7f93c4100570 2024-09-15T22:37:25.427 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.011+0000 7f93c259c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93c4100570 0x7f93c41a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:25.427 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.011+0000 7f93c259c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93c4100570 0x7f93c41a33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.192:36062/0 (socket says 172.21.15.192:36062) 2024-09-15T22:37:25.427 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.011+0000 7f93c259c640 1 -- 172.21.15.192:0/2189238001 learned_addr learned my addr 172.21.15.192:0/2189238001 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-09-15T22:37:25.428 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.011+0000 7f93c259c640 1 -- 172.21.15.192:0/2189238001 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c41a3920 con 0x7f93c4100570 2024-09-15T22:37:25.428 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.012+0000 7f93c259c640 1 --2- 172.21.15.192:0/2189238001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93c4100570 0x7f93c41a33e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009230 tx=0x7f93b4003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:25.428 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.012+0000 7f93bbfff640 1 -- 172.21.15.192:0/2189238001 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93b403f070 con 0x7f93c4100570 2024-09-15T22:37:25.428 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.012+0000 7f93bbfff640 1 -- 172.21.15.192:0/2189238001 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93b403bd40 con 0x7f93c4100570 2024-09-15T22:37:25.428 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.012+0000 7f93c8dc3640 1 -- 172.21.15.192:0/2189238001 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c41a3bb0 con 0x7f93c4100570 2024-09-15T22:37:25.428 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.012+0000 7f93bbfff640 1 -- 172.21.15.192:0/2189238001 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93b4035030 con 0x7f93c4100570 2024-09-15T22:37:25.428 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.012+0000 7f93c8dc3640 1 -- 172.21.15.192:0/2189238001 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c419d5a0 con 0x7f93c4100570 2024-09-15T22:37:25.428 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T22:37:25.428 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"b7614d3c-73b2-11ef-bceb-c7b262605968","modified":"2024-09-15T22:35:48.940270Z","created":"2024-09-15T22:35:48.940270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:3300","nonce":0},{"type":"v1","addr":"172.21.15.111:6789","nonce":0}]},"addr":"172.21.15.111:6789/0","public_addr":"172.21.15.111:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-15T22:37:25.429 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.013+0000 7f93bbfff640 1 -- 172.21.15.192:0/2189238001 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f93b4042070 con 0x7f93c4100570 2024-09-15T22:37:25.429 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.013+0000 7f93bbfff640 1 --2- 172.21.15.192:0/2189238001 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f93b0039300 0x7f93b003b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:25.429 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.013+0000 7f93bbfff640 1 -- 172.21.15.192:0/2189238001 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f93b406cf10 con 0x7f93c4100570 2024-09-15T22:37:25.429 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.013+0000 7f93c8dc3640 1 -- 172.21.15.192:0/2189238001 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93c406a7c0 con 0x7f93c4100570 2024-09-15T22:37:25.429 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.014+0000 7f93b9d9b640 1 --2- 172.21.15.192:0/2189238001 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f93b0039300 0x7f93b003b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:25.429 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.014+0000 7f93b9d9b640 1 --2- 172.21.15.192:0/2189238001 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f93b0039300 0x7f93b003b7c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0098c0 tx=0x7f93ac006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:25.429 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.019+0000 7f93bbfff640 1 -- 172.21.15.192:0/2189238001 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f93b4031de0 con 0x7f93c4100570 2024-09-15T22:37:25.429 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.251+0000 7f93c8dc3640 1 -- 172.21.15.192:0/2189238001 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f93c41009d0 con 0x7f93c4100570 2024-09-15T22:37:25.429 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.252+0000 7f93bbfff640 1 -- 172.21.15.192:0/2189238001 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f93b403a8e0 con 0x7f93c4100570 2024-09-15T22:37:25.429 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-09-15T22:37:25.429 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.255+0000 7f93c8dc3640 1 -- 172.21.15.192:0/2189238001 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f93b0039300 msgr2=0x7f93b003b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:25.429 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.255+0000 7f93c8dc3640 1 --2- 172.21.15.192:0/2189238001 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f93b0039300 0x7f93b003b7c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f93ac0098c0 tx=0x7f93ac006db0 comp rx=0 tx=0).stop 2024-09-15T22:37:25.430 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.255+0000 7f93c8dc3640 1 -- 172.21.15.192:0/2189238001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93c4100570 msgr2=0x7f93c41a33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:25.430 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.255+0000 7f93c8dc3640 1 --2- 172.21.15.192:0/2189238001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93c4100570 0x7f93c41a33e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009230 tx=0x7f93b4003d80 comp rx=0 tx=0).stop 2024-09-15T22:37:25.430 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.255+0000 7f93c8dc3640 1 -- 172.21.15.192:0/2189238001 shutdown_connections 2024-09-15T22:37:25.430 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.255+0000 7f93c8dc3640 1 --2- 172.21.15.192:0/2189238001 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f93b0039300 0x7f93b003b7c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:25.430 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.255+0000 7f93c8dc3640 1 --2- 172.21.15.192:0/2189238001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93c4100570 0x7f93c41a33e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:25.430 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.255+0000 7f93c8dc3640 1 -- 172.21.15.192:0/2189238001 >> 172.21.15.192:0/2189238001 conn(0x7f93c40fc1a0 msgr2=0x7f93c40fc580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:25.430 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.255+0000 7f93c8dc3640 1 -- 172.21.15.192:0/2189238001 shutdown_connections 2024-09-15T22:37:25.430 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:25.255+0000 7f93c8dc3640 1 -- 172.21.15.192:0/2189238001 wait complete. 2024-09-15T22:37:25.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:25 smithi111 ceph-mon[30629]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:25.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:25 smithi111 ceph-mon[30629]: from='client.? 172.21.15.192:0/2189238001' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:37:26.949 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-15T22:37:26.949 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-15T22:37:27.312 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:37:27.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:27 smithi111 ceph-mon[30629]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:27.958 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.955+0000 7ff940f91640 1 -- 172.21.15.192:0/1430640639 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff93c1028e0 msgr2=0x7ff93c102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:27.959 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.955+0000 7ff940f91640 1 --2- 172.21.15.192:0/1430640639 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff93c1028e0 0x7ff93c102cc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff930009880 tx=0x7ff93002a670 comp rx=0 tx=0).stop 2024-09-15T22:37:27.959 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.955+0000 7ff940f91640 1 -- 172.21.15.192:0/1430640639 shutdown_connections 2024-09-15T22:37:27.959 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.955+0000 7ff940f91640 1 --2- 172.21.15.192:0/1430640639 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff93c1028e0 0x7ff93c102cc0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:27.959 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.955+0000 7ff940f91640 1 -- 172.21.15.192:0/1430640639 >> 172.21.15.192:0/1430640639 conn(0x7ff93c0fc280 msgr2=0x7ff93c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:27.959 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.955+0000 7ff940f91640 1 -- 172.21.15.192:0/1430640639 shutdown_connections 2024-09-15T22:37:27.959 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.955+0000 7ff940f91640 1 -- 172.21.15.192:0/1430640639 wait complete. 2024-09-15T22:37:27.959 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.956+0000 7ff940f91640 1 Processor -- start 2024-09-15T22:37:27.959 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.956+0000 7ff940f91640 1 -- start start 2024-09-15T22:37:27.959 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.956+0000 7ff940f91640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff93c1028e0 0x7ff93c197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:27.959 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.956+0000 7ff940f91640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c10bdb0 con 0x7ff93c1028e0 2024-09-15T22:37:27.959 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.957+0000 7ff93a59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff93c1028e0 0x7ff93c197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:27.959 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.957+0000 7ff93a59c640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff93c1028e0 0x7ff93c197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.192:36070/0 (socket says 172.21.15.192:36070) 2024-09-15T22:37:27.960 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.957+0000 7ff93a59c640 1 -- 172.21.15.192:0/475233628 learned_addr learned my addr 172.21.15.192:0/475233628 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-09-15T22:37:27.960 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.957+0000 7ff93a59c640 1 -- 172.21.15.192:0/475233628 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff93c1978a0 con 0x7ff93c1028e0 2024-09-15T22:37:27.961 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.958+0000 7ff93a59c640 1 --2- 172.21.15.192:0/475233628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff93c1028e0 0x7ff93c197360 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff930009230 tx=0x7ff930003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:27.961 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.958+0000 7ff92b7fe640 1 -- 172.21.15.192:0/475233628 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff93003f070 con 0x7ff93c1028e0 2024-09-15T22:37:27.961 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.958+0000 7ff940f91640 1 -- 172.21.15.192:0/475233628 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff93c197b30 con 0x7ff93c1028e0 2024-09-15T22:37:27.961 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.958+0000 7ff92b7fe640 1 -- 172.21.15.192:0/475233628 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff93003bc80 con 0x7ff93c1028e0 2024-09-15T22:37:27.961 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.958+0000 7ff92b7fe640 1 -- 172.21.15.192:0/475233628 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff930035030 con 0x7ff93c1028e0 2024-09-15T22:37:27.961 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.958+0000 7ff940f91640 1 -- 172.21.15.192:0/475233628 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff93c197f90 con 0x7ff93c1028e0 2024-09-15T22:37:27.961 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.959+0000 7ff940f91640 1 -- 172.21.15.192:0/475233628 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff93c06a450 con 0x7ff93c1028e0 2024-09-15T22:37:27.961 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.959+0000 7ff92b7fe640 1 -- 172.21.15.192:0/475233628 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7ff930042070 con 0x7ff93c1028e0 2024-09-15T22:37:27.961 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.960+0000 7ff92b7fe640 1 --2- 172.21.15.192:0/475233628 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ff9240392a0 0x7ff92403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:27.961 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.960+0000 7ff92b7fe640 1 -- 172.21.15.192:0/475233628 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff93006d0c0 con 0x7ff93c1028e0 2024-09-15T22:37:27.962 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.960+0000 7ff939d9b640 1 --2- 172.21.15.192:0/475233628 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ff9240392a0 0x7ff92403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:27.962 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.960+0000 7ff939d9b640 1 --2- 172.21.15.192:0/475233628 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ff9240392a0 0x7ff92403b760 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0098c0 tx=0x7ff92c006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:27.965 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:27.964+0000 7ff92b7fe640 1 -- 172.21.15.192:0/475233628 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff9300372d0 con 0x7ff93c1028e0 2024-09-15T22:37:28.224 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:28.222+0000 7ff940f91640 1 -- 172.21.15.192:0/475233628 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff93c1986c0 con 0x7ff93c1028e0 2024-09-15T22:37:28.224 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:28.223+0000 7ff92b7fe640 1 -- 172.21.15.192:0/475233628 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7ff930031de0 con 0x7ff93c1028e0 2024-09-15T22:37:28.225 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T22:37:28.225 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":1,"fsid":"b7614d3c-73b2-11ef-bceb-c7b262605968","modified":"2024-09-15T22:35:48.940270Z","created":"2024-09-15T22:35:48.940270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:3300","nonce":0},{"type":"v1","addr":"172.21.15.111:6789","nonce":0}]},"addr":"172.21.15.111:6789/0","public_addr":"172.21.15.111:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-15T22:37:28.225 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 1 2024-09-15T22:37:28.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:28.225+0000 7ff940f91640 1 -- 172.21.15.192:0/475233628 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ff9240392a0 msgr2=0x7ff92403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:28.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:28.225+0000 7ff940f91640 1 --2- 172.21.15.192:0/475233628 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ff9240392a0 0x7ff92403b760 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0098c0 tx=0x7ff92c006db0 comp rx=0 tx=0).stop 2024-09-15T22:37:28.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:28.225+0000 7ff940f91640 1 -- 172.21.15.192:0/475233628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff93c1028e0 msgr2=0x7ff93c197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:28.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:28.225+0000 7ff940f91640 1 --2- 172.21.15.192:0/475233628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff93c1028e0 0x7ff93c197360 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff930009230 tx=0x7ff930003d40 comp rx=0 tx=0).stop 2024-09-15T22:37:28.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:28.225+0000 7ff940f91640 1 -- 172.21.15.192:0/475233628 shutdown_connections 2024-09-15T22:37:28.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:28.225+0000 7ff940f91640 1 --2- 172.21.15.192:0/475233628 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ff9240392a0 0x7ff92403b760 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:28.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:28.225+0000 7ff940f91640 1 --2- 172.21.15.192:0/475233628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff93c1028e0 0x7ff93c197360 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:28.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:28.225+0000 7ff940f91640 1 -- 172.21.15.192:0/475233628 >> 172.21.15.192:0/475233628 conn(0x7ff93c0fc280 msgr2=0x7ff93c1056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:28.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:28.226+0000 7ff940f91640 1 -- 172.21.15.192:0/475233628 shutdown_connections 2024-09-15T22:37:28.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:28.226+0000 7ff940f91640 1 -- 172.21.15.192:0/475233628 wait complete. 2024-09-15T22:37:28.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:28 smithi111 ceph-mon[30629]: from='client.? 172.21.15.192:0/475233628' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:37:29.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:29 smithi192 systemd[1]: Started Ceph mon.c for b7614d3c-73b2-11ef-bceb-c7b262605968. 2024-09-15T22:37:29.944 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-15T22:37:29.944 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-15T22:37:30.207 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:37:34.221 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.218+0000 7f1c1ce16640 1 -- 172.21.15.192:0/3105335931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c00003530 msgr2=0x7f1c000059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:34.221 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.218+0000 7f1c1ce16640 1 --2- 172.21.15.192:0/3105335931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c00003530 0x7f1c000059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1c181016c0 tx=0x7f1c0802c3f0 comp rx=0 tx=0).stop 2024-09-15T22:37:34.222 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.219+0000 7f1c1ce16640 1 -- 172.21.15.192:0/3105335931 shutdown_connections 2024-09-15T22:37:34.222 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.219+0000 7f1c1ce16640 1 --2- 172.21.15.192:0/3105335931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c00003530 0x7f1c000059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:34.222 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.219+0000 7f1c1ce16640 1 -- 172.21.15.192:0/3105335931 >> 172.21.15.192:0/3105335931 conn(0x7f1c180fc280 msgr2=0x7f1c180fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:34.222 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.219+0000 7f1c1ce16640 1 -- 172.21.15.192:0/3105335931 shutdown_connections 2024-09-15T22:37:34.222 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.219+0000 7f1c1ce16640 1 -- 172.21.15.192:0/3105335931 wait complete. 2024-09-15T22:37:34.222 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.220+0000 7f1c1ce16640 1 Processor -- start 2024-09-15T22:37:34.222 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.220+0000 7f1c1ce16640 1 -- start start 2024-09-15T22:37:34.223 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.220+0000 7f1c1ce16640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c00003530 0x7f1c18102210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:34.223 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.220+0000 7f1c1ce16640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c18102750 0x7f1c18102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:34.224 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.220+0000 7f1c1ce16640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1c18108690 con 0x7f1c00003530 2024-09-15T22:37:34.224 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.220+0000 7f1c1ce16640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1c18108510 con 0x7f1c18102750 2024-09-15T22:37:34.224 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.221+0000 7f1c15d9b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c18102750 0x7f1c18102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:34.224 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.221+0000 7f1c15d9b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c18102750 0x7f1c18102bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:45392/0 (socket says 172.21.15.192:45392) 2024-09-15T22:37:34.224 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.221+0000 7f1c15d9b640 1 -- 172.21.15.192:0/3884113458 learned_addr learned my addr 172.21.15.192:0/3884113458 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-09-15T22:37:34.224 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.221+0000 7f1c15d9b640 1 -- 172.21.15.192:0/3884113458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c18102750 msgr2=0x7f1c18102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-15T22:37:34.224 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.221+0000 7f1c15d9b640 1 -- 172.21.15.192:0/3884113458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c18102750 msgr2=0x7f1c18102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T22:37:34.224 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.221+0000 7f1c15d9b640 1 --2- 172.21.15.192:0/3884113458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c18102750 0x7f1c18102bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T22:37:34.224 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.221+0000 7f1c1659c640 1 --2- 172.21.15.192:0/3884113458 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c00003530 0x7f1c18102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:34.224 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.221+0000 7f1c15d9b640 1 --2- 172.21.15.192:0/3884113458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c18102750 0x7f1c18102bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:37:34.224 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.221+0000 7f1c1659c640 1 -- 172.21.15.192:0/3884113458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c18102750 msgr2=0x7f1c18102bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:37:34.225 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.221+0000 7f1c1659c640 1 --2- 172.21.15.192:0/3884113458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c18102750 0x7f1c18102bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:34.225 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.221+0000 7f1c1659c640 1 -- 172.21.15.192:0/3884113458 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c18104100 con 0x7f1c00003530 2024-09-15T22:37:34.225 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.221+0000 7f1c1659c640 1 --2- 172.21.15.192:0/3884113458 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c00003530 0x7f1c18102210 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1c08014cc0 tx=0x7f1c08034b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:34.225 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.222+0000 7f1c077fe640 1 -- 172.21.15.192:0/3884113458 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1c080125f0 con 0x7f1c00003530 2024-09-15T22:37:34.225 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.222+0000 7f1c1ce16640 1 -- 172.21.15.192:0/3884113458 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c181a3460 con 0x7f1c00003530 2024-09-15T22:37:34.225 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.222+0000 7f1c077fe640 1 -- 172.21.15.192:0/3884113458 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c08042070 con 0x7f1c00003530 2024-09-15T22:37:34.225 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.222+0000 7f1c1ce16640 1 -- 172.21.15.192:0/3884113458 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c181a39f0 con 0x7f1c00003530 2024-09-15T22:37:34.225 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.222+0000 7f1c077fe640 1 -- 172.21.15.192:0/3884113458 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1c0803db30 con 0x7f1c00003530 2024-09-15T22:37:34.226 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.223+0000 7f1c077fe640 1 -- 172.21.15.192:0/3884113458 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f1c0803dcd0 con 0x7f1c00003530 2024-09-15T22:37:34.226 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.223+0000 7f1c1ce16640 1 -- 172.21.15.192:0/3884113458 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c18101cc0 con 0x7f1c00003530 2024-09-15T22:37:34.226 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.223+0000 7f1c077fe640 1 --2- 172.21.15.192:0/3884113458 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1bf8039040 0x7f1bf803b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:34.226 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.223+0000 7f1c077fe640 1 -- 172.21.15.192:0/3884113458 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1c0806e9e0 con 0x7f1c00003530 2024-09-15T22:37:34.226 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.223+0000 7f1c15d9b640 1 --2- 172.21.15.192:0/3884113458 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1bf8039040 0x7f1bf803b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:34.226 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.224+0000 7f1c15d9b640 1 --2- 172.21.15.192:0/3884113458 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1bf8039040 0x7f1bf803b500 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c009400 tx=0x7f1c0c007040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:34.230 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.228+0000 7f1c077fe640 1 -- 172.21.15.192:0/3884113458 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1c08044310 con 0x7f1c00003530 2024-09-15T22:37:34.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:37:34.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:37:34.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: mon.a calling monitor election 2024-09-15T22:37:34.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:37:34.417 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:34.417 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:37:34.417 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: mon.c calling monitor election 2024-09-15T22:37:34.417 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:37:34.417 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:34.417 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:37:34.417 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:37:34.417 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-15T22:37:34.417 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: monmap epoch 2 2024-09-15T22:37:34.417 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: fsid b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T22:37:34.417 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: last_changed 2024-09-15T22:37:29.083163+0000 2024-09-15T22:37:34.417 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: created 2024-09-15T22:35:48.940270+0000 2024-09-15T22:37:34.418 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: min_mon_release 19 (squid) 2024-09-15T22:37:34.418 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: election_strategy: 1 2024-09-15T22:37:34.418 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: 0: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.a 2024-09-15T22:37:34.418 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-09-15T22:37:34.418 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: fsmap 2024-09-15T22:37:34.418 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:37:34.418 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: mgrmap e13: x(active, since 43s) 2024-09-15T22:37:34.418 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: overall HEALTH_OK 2024-09-15T22:37:34.418 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:34.418 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:34.418 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:34.419 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:37:34.419 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:34 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:34.465 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.464+0000 7f1c1ce16640 1 -- 172.21.15.192:0/3884113458 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1c181a3cd0 con 0x7f1c00003530 2024-09-15T22:37:34.466 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:37:34.466 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:37:34.466 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: mon.a calling monitor election 2024-09-15T22:37:34.466 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:37:34.466 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:34.466 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:37:34.467 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: mon.c calling monitor election 2024-09-15T22:37:34.467 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:37:34.467 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:34.467 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:37:34.467 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:37:34.467 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-15T22:37:34.467 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: monmap epoch 2 2024-09-15T22:37:34.467 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: fsid b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T22:37:34.467 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: last_changed 2024-09-15T22:37:29.083163+0000 2024-09-15T22:37:34.467 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: created 2024-09-15T22:35:48.940270+0000 2024-09-15T22:37:34.467 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: min_mon_release 19 (squid) 2024-09-15T22:37:34.467 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: election_strategy: 1 2024-09-15T22:37:34.467 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: 0: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.a 2024-09-15T22:37:34.467 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-09-15T22:37:34.467 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: fsmap 2024-09-15T22:37:34.468 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:37:34.468 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: mgrmap e13: x(active, since 43s) 2024-09-15T22:37:34.468 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: overall HEALTH_OK 2024-09-15T22:37:34.468 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:34.468 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:34.468 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:34.468 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:37:34.468 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:34 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:34.469 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.465+0000 7f1c077fe640 1 -- 172.21.15.192:0/3884113458 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f1c0803d480 con 0x7f1c00003530 2024-09-15T22:37:34.469 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T22:37:34.469 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":2,"fsid":"b7614d3c-73b2-11ef-bceb-c7b262605968","modified":"2024-09-15T22:37:29.083163Z","created":"2024-09-15T22:35:48.940270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:3300","nonce":0},{"type":"v1","addr":"172.21.15.111:6789","nonce":0}]},"addr":"172.21.15.111:6789/0","public_addr":"172.21.15.111:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-15T22:37:34.470 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 2 2024-09-15T22:37:34.470 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.467+0000 7f1c1ce16640 1 -- 172.21.15.192:0/3884113458 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1bf8039040 msgr2=0x7f1bf803b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:34.470 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.467+0000 7f1c1ce16640 1 --2- 172.21.15.192:0/3884113458 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1bf8039040 0x7f1bf803b500 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c009400 tx=0x7f1c0c007040 comp rx=0 tx=0).stop 2024-09-15T22:37:34.470 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.467+0000 7f1c1ce16640 1 -- 172.21.15.192:0/3884113458 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c00003530 msgr2=0x7f1c18102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:34.470 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.467+0000 7f1c1ce16640 1 --2- 172.21.15.192:0/3884113458 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c00003530 0x7f1c18102210 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1c08014cc0 tx=0x7f1c08034b60 comp rx=0 tx=0).stop 2024-09-15T22:37:34.470 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.467+0000 7f1c1ce16640 1 -- 172.21.15.192:0/3884113458 shutdown_connections 2024-09-15T22:37:34.470 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.467+0000 7f1c1ce16640 1 --2- 172.21.15.192:0/3884113458 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1bf8039040 0x7f1bf803b500 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:34.470 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.467+0000 7f1c1ce16640 1 --2- 172.21.15.192:0/3884113458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c18102750 0x7f1c18102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:34.470 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.467+0000 7f1c1ce16640 1 --2- 172.21.15.192:0/3884113458 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c00003530 0x7f1c18102210 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:34.470 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.467+0000 7f1c1ce16640 1 -- 172.21.15.192:0/3884113458 >> 172.21.15.192:0/3884113458 conn(0x7f1c180fc280 msgr2=0x7f1c1806a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:34.470 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.467+0000 7f1c1ce16640 1 -- 172.21.15.192:0/3884113458 shutdown_connections 2024-09-15T22:37:34.470 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:34.467+0000 7f1c1ce16640 1 -- 172.21.15.192:0/3884113458 wait complete. 2024-09-15T22:37:35.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:35 smithi192 ceph-mon[35427]: Deploying daemon mon.b on smithi121 2024-09-15T22:37:35.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:35 smithi192 ceph-mon[35427]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:35.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:35 smithi192 ceph-mon[35427]: from='client.? 172.21.15.192:0/3884113458' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:37:35.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:35 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:37:35.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:35 smithi111 ceph-mon[30629]: Deploying daemon mon.b on smithi121 2024-09-15T22:37:35.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:35 smithi111 ceph-mon[30629]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:35.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:35 smithi111 ceph-mon[30629]: from='client.? 172.21.15.192:0/3884113458' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:37:35.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:35 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:37:36.058 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-15T22:37:36.059 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-15T22:37:36.309 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:37:36.732 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.728+0000 7f657574d640 1 -- 172.21.15.192:0/3080945525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6570102820 msgr2=0x7f6570102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:36.732 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.728+0000 7f657574d640 1 --2- 172.21.15.192:0/3080945525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6570102820 0x7f6570102c00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6560009880 tx=0x7f656002a670 comp rx=0 tx=0).stop 2024-09-15T22:37:36.732 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.728+0000 7f657574d640 1 -- 172.21.15.192:0/3080945525 shutdown_connections 2024-09-15T22:37:36.732 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.728+0000 7f657574d640 1 --2- 172.21.15.192:0/3080945525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6570102820 0x7f6570102c00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:36.732 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.728+0000 7f657574d640 1 -- 172.21.15.192:0/3080945525 >> 172.21.15.192:0/3080945525 conn(0x7f65700fc220 msgr2=0x7f65700fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:36.732 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.728+0000 7f657574d640 1 -- 172.21.15.192:0/3080945525 shutdown_connections 2024-09-15T22:37:36.732 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.728+0000 7f657574d640 1 -- 172.21.15.192:0/3080945525 wait complete. 2024-09-15T22:37:36.732 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.729+0000 7f657574d640 1 Processor -- start 2024-09-15T22:37:36.732 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.729+0000 7f657574d640 1 -- start start 2024-09-15T22:37:36.733 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.729+0000 7f657574d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6570102820 0x7f6570100480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:36.733 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.729+0000 7f657574d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65701009c0 0x7f6570100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:36.733 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.729+0000 7f657574d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f657010bd70 con 0x7f65701009c0 2024-09-15T22:37:36.733 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.729+0000 7f657574d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f657010bbf0 con 0x7f6570102820 2024-09-15T22:37:36.733 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.729+0000 7f656effd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6570102820 0x7f6570100480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:36.733 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.729+0000 7f656effd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6570102820 0x7f6570100480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:45400/0 (socket says 172.21.15.192:45400) 2024-09-15T22:37:36.733 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.729+0000 7f656effd640 1 -- 172.21.15.192:0/3108180963 learned_addr learned my addr 172.21.15.192:0/3108180963 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-09-15T22:37:36.734 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.729+0000 7f656effd640 1 -- 172.21.15.192:0/3108180963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6570102820 msgr2=0x7f6570100480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-15T22:37:36.734 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.729+0000 7f656effd640 1 -- 172.21.15.192:0/3108180963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6570102820 msgr2=0x7f6570100480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T22:37:36.734 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.729+0000 7f656effd640 1 --2- 172.21.15.192:0/3108180963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6570102820 0x7f6570100480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T22:37:36.734 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.729+0000 7f656effd640 1 --2- 172.21.15.192:0/3108180963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6570102820 0x7f6570100480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:37:36.734 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.730+0000 7f6567fff640 1 --2- 172.21.15.192:0/3108180963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65701009c0 0x7f6570100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:36.734 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.730+0000 7f6567fff640 1 -- 172.21.15.192:0/3108180963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6570102820 msgr2=0x7f6570100480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:37:36.734 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.730+0000 7f6567fff640 1 --2- 172.21.15.192:0/3108180963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6570102820 0x7f6570100480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:36.734 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.730+0000 7f6567fff640 1 -- 172.21.15.192:0/3108180963 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6570102370 con 0x7f65701009c0 2024-09-15T22:37:36.734 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.730+0000 7f6567fff640 1 --2- 172.21.15.192:0/3108180963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65701009c0 0x7f6570100e20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f655800b400 tx=0x7f655800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:36.735 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.731+0000 7f656cff9640 1 -- 172.21.15.192:0/3108180963 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f6558011630 con 0x7f65701009c0 2024-09-15T22:37:36.735 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.731+0000 7f656cff9640 1 -- 172.21.15.192:0/3108180963 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6558011cb0 con 0x7f65701009c0 2024-09-15T22:37:36.736 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.731+0000 7f657574d640 1 -- 172.21.15.192:0/3108180963 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65701a36c0 con 0x7f65701009c0 2024-09-15T22:37:36.736 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.731+0000 7f656cff9640 1 -- 172.21.15.192:0/3108180963 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f6558010d80 con 0x7f65701009c0 2024-09-15T22:37:36.736 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.731+0000 7f657574d640 1 -- 172.21.15.192:0/3108180963 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65701a3c50 con 0x7f65701009c0 2024-09-15T22:37:36.736 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.732+0000 7f657574d640 1 -- 172.21.15.192:0/3108180963 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f657006ad10 con 0x7f65701009c0 2024-09-15T22:37:36.736 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.732+0000 7f656cff9640 1 -- 172.21.15.192:0/3108180963 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f6558018020 con 0x7f65701009c0 2024-09-15T22:37:36.736 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.732+0000 7f656cff9640 1 --2- 172.21.15.192:0/3108180963 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f655c039030 0x7f655c03b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:36.736 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.732+0000 7f656cff9640 1 -- 172.21.15.192:0/3108180963 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f655804bee0 con 0x7f65701009c0 2024-09-15T22:37:36.736 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.733+0000 7f656effd640 1 --2- 172.21.15.192:0/3108180963 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f655c039030 0x7f655c03b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:36.736 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.733+0000 7f656effd640 1 --2- 172.21.15.192:0/3108180963 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f655c039030 0x7f655c03b4f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f65600096c0 tx=0x7f6560005a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:36.739 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.738+0000 7f656cff9640 1 -- 172.21.15.192:0/3108180963 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f655801f020 con 0x7f65701009c0 2024-09-15T22:37:36.966 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.964+0000 7f657574d640 1 -- 172.21.15.192:0/3108180963 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f65701a3f60 con 0x7f65701009c0 2024-09-15T22:37:36.966 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.965+0000 7f656cff9640 1 -- 172.21.15.192:0/3108180963 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f65580172c0 con 0x7f65701009c0 2024-09-15T22:37:36.967 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T22:37:36.967 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":2,"fsid":"b7614d3c-73b2-11ef-bceb-c7b262605968","modified":"2024-09-15T22:37:29.083163Z","created":"2024-09-15T22:35:48.940270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:3300","nonce":0},{"type":"v1","addr":"172.21.15.111:6789","nonce":0}]},"addr":"172.21.15.111:6789/0","public_addr":"172.21.15.111:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-15T22:37:36.967 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 2 2024-09-15T22:37:36.969 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.967+0000 7f657574d640 1 -- 172.21.15.192:0/3108180963 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f655c039030 msgr2=0x7f655c03b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:36.969 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.967+0000 7f657574d640 1 --2- 172.21.15.192:0/3108180963 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f655c039030 0x7f655c03b4f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f65600096c0 tx=0x7f6560005a40 comp rx=0 tx=0).stop 2024-09-15T22:37:36.969 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.967+0000 7f657574d640 1 -- 172.21.15.192:0/3108180963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65701009c0 msgr2=0x7f6570100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:36.969 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.967+0000 7f657574d640 1 --2- 172.21.15.192:0/3108180963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65701009c0 0x7f6570100e20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f655800b400 tx=0x7f655800b8c0 comp rx=0 tx=0).stop 2024-09-15T22:37:36.969 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.967+0000 7f657574d640 1 -- 172.21.15.192:0/3108180963 shutdown_connections 2024-09-15T22:37:36.969 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.967+0000 7f657574d640 1 --2- 172.21.15.192:0/3108180963 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f655c039030 0x7f655c03b4f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:36.969 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.967+0000 7f657574d640 1 --2- 172.21.15.192:0/3108180963 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65701009c0 0x7f6570100e20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:36.969 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.967+0000 7f657574d640 1 --2- 172.21.15.192:0/3108180963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6570102820 0x7f6570100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:36.969 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.967+0000 7f657574d640 1 -- 172.21.15.192:0/3108180963 >> 172.21.15.192:0/3108180963 conn(0x7f65700fc220 msgr2=0x7f65701055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:36.969 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.967+0000 7f657574d640 1 -- 172.21.15.192:0/3108180963 shutdown_connections 2024-09-15T22:37:36.969 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:36.967+0000 7f657574d640 1 -- 172.21.15.192:0/3108180963 wait complete. 2024-09-15T22:37:37.724 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 podman[34796]: 2024-09-15 22:37:37.722267092 +0000 UTC m=+0.335214222 container init d9e180790a275da76614d717a65f703bcedb54720a6560540f4efd31123ed3bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-15T22:37:37.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:37 smithi192 ceph-mon[35427]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:37.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:37 smithi192 ceph-mon[35427]: from='client.? 172.21.15.192:0/3108180963' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:37:37.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:37 smithi111 ceph-mon[30629]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:37.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:37 smithi111 ceph-mon[30629]: from='client.? 172.21.15.192:0/3108180963' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:37:38.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 podman[34796]: 2024-09-15 22:37:37.725493024 +0000 UTC m=+0.338440155 container start d9e180790a275da76614d717a65f703bcedb54720a6560540f4efd31123ed3bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-b, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:37:38.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: set uid:gid to 167:167 (ceph:ceph) 2024-09-15T22:37:38.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable), process ceph-mon, pid 2 2024-09-15T22:37:38.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: pidfile_write: ignore empty --pid-file 2024-09-15T22:37:38.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: load: jerasure load: lrc 2024-09-15T22:37:38.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: RocksDB version: 7.9.2 2024-09-15T22:37:38.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Git sha 0 2024-09-15T22:37:38.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Compile date 2024-09-15 15:04:35 2024-09-15T22:37:38.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: DB SUMMARY 2024-09-15T22:37:38.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: DB Session ID: BW30JSFRH66HAVU8Q5D6 2024-09-15T22:37:38.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: CURRENT file: CURRENT 2024-09-15T22:37:38.142 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: IDENTITY file: IDENTITY 2024-09-15T22:37:38.142 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-15T22:37:38.142 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-15T22:37:38.142 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-15T22:37:38.142 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.error_if_exists: 0 2024-09-15T22:37:38.142 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.create_if_missing: 0 2024-09-15T22:37:38.142 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.paranoid_checks: 1 2024-09-15T22:37:38.142 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-15T22:37:38.142 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-15T22:37:38.142 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-15T22:37:38.143 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.env: 0x555fdd445940 2024-09-15T22:37:38.143 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.fs: PosixFileSystem 2024-09-15T22:37:38.143 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.info_log: 0x555fdf77d6e0 2024-09-15T22:37:38.143 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.max_file_opening_threads: 16 2024-09-15T22:37:38.143 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.statistics: (nil) 2024-09-15T22:37:38.143 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.use_fsync: 0 2024-09-15T22:37:38.143 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.max_log_file_size: 0 2024-09-15T22:37:38.143 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-15T22:37:38.143 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-15T22:37:38.143 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.keep_log_file_num: 1000 2024-09-15T22:37:38.143 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.recycle_log_file_num: 0 2024-09-15T22:37:38.143 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.allow_fallocate: 1 2024-09-15T22:37:38.145 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.allow_mmap_reads: 0 2024-09-15T22:37:38.145 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.allow_mmap_writes: 0 2024-09-15T22:37:38.146 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.use_direct_reads: 0 2024-09-15T22:37:38.146 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-15T22:37:38.146 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.create_missing_column_families: 0 2024-09-15T22:37:38.146 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.db_log_dir: 2024-09-15T22:37:38.146 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.wal_dir: 2024-09-15T22:37:38.146 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-15T22:37:38.146 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-15T22:37:38.146 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-15T22:37:38.146 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-15T22:37:38.146 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-15T22:37:38.146 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-15T22:37:38.146 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.advise_random_on_open: 1 2024-09-15T22:37:38.146 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.db_write_buffer_size: 0 2024-09-15T22:37:38.147 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.write_buffer_manager: 0x555fdf78cdc0 2024-09-15T22:37:38.147 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-15T22:37:38.147 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-15T22:37:38.147 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-15T22:37:38.147 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.rate_limiter: (nil) 2024-09-15T22:37:38.147 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-15T22:37:38.147 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.wal_recovery_mode: 2 2024-09-15T22:37:38.147 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.enable_thread_tracking: 0 2024-09-15T22:37:38.147 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.enable_pipelined_write: 0 2024-09-15T22:37:38.147 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.unordered_write: 0 2024-09-15T22:37:38.147 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-15T22:37:38.147 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-15T22:37:38.147 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-15T22:37:38.148 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-15T22:37:38.148 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.row_cache: None 2024-09-15T22:37:38.148 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.wal_filter: None 2024-09-15T22:37:38.148 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-15T22:37:38.148 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.allow_ingest_behind: 0 2024-09-15T22:37:38.148 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.two_write_queues: 0 2024-09-15T22:37:38.148 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.manual_wal_flush: 0 2024-09-15T22:37:38.148 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.wal_compression: 0 2024-09-15T22:37:38.148 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.atomic_flush: 0 2024-09-15T22:37:38.148 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-15T22:37:38.148 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-15T22:37:38.148 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-15T22:37:38.148 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.log_readahead_size: 0 2024-09-15T22:37:38.148 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-15T22:37:38.148 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.best_efforts_recovery: 0 2024-09-15T22:37:38.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-15T22:37:38.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-15T22:37:38.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.allow_data_in_errors: 0 2024-09-15T22:37:38.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.db_host_id: __hostname__ 2024-09-15T22:37:38.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-15T22:37:38.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.max_background_jobs: 2 2024-09-15T22:37:38.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.max_background_compactions: -1 2024-09-15T22:37:38.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.max_subcompactions: 1 2024-09-15T22:37:38.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-15T22:37:38.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-15T22:37:38.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-15T22:37:38.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.max_total_wal_size: 0 2024-09-15T22:37:38.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-15T22:37:38.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-15T22:37:38.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-15T22:37:38.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-15T22:37:38.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.max_open_files: -1 2024-09-15T22:37:38.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.bytes_per_sync: 0 2024-09-15T22:37:38.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-15T22:37:38.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-15T22:37:38.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.compaction_readahead_size: 0 2024-09-15T22:37:38.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.max_background_flushes: -1 2024-09-15T22:37:38.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Compression algorithms supported: 2024-09-15T22:37:38.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: kZSTD supported: 0 2024-09-15T22:37:38.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: kXpressCompression supported: 0 2024-09-15T22:37:38.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: kBZip2Compression supported: 0 2024-09-15T22:37:38.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-15T22:37:38.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: kLZ4Compression supported: 1 2024-09-15T22:37:38.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: kZlibCompression supported: 1 2024-09-15T22:37:38.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: kLZ4HCCompression supported: 1 2024-09-15T22:37:38.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: kSnappyCompression supported: 1 2024-09-15T22:37:38.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-15T22:37:38.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-15T22:37:38.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-15T22:37:38.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-15T22:37:38.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-15T22:37:38.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.merge_operator: 2024-09-15T22:37:38.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.compaction_filter: None 2024-09-15T22:37:38.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.compaction_filter_factory: None 2024-09-15T22:37:38.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.sst_partitioner_factory: None 2024-09-15T22:37:38.152 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-15T22:37:38.152 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-15T22:37:38.152 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555fdf77d320) 2024-09-15T22:37:38.152 INFO:journalctl@ceph.mon.b.smithi121.stdout: cache_index_and_filter_blocks: 1 2024-09-15T22:37:38.152 INFO:journalctl@ceph.mon.b.smithi121.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-15T22:37:38.152 INFO:journalctl@ceph.mon.b.smithi121.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-15T22:37:38.152 INFO:journalctl@ceph.mon.b.smithi121.stdout: pin_top_level_index_and_filter: 1 2024-09-15T22:37:38.152 INFO:journalctl@ceph.mon.b.smithi121.stdout: index_type: 0 2024-09-15T22:37:38.152 INFO:journalctl@ceph.mon.b.smithi121.stdout: data_block_index_type: 0 2024-09-15T22:37:38.152 INFO:journalctl@ceph.mon.b.smithi121.stdout: index_shortening: 1 2024-09-15T22:37:38.152 INFO:journalctl@ceph.mon.b.smithi121.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-15T22:37:38.153 INFO:journalctl@ceph.mon.b.smithi121.stdout: checksum: 4 2024-09-15T22:37:38.153 INFO:journalctl@ceph.mon.b.smithi121.stdout: no_block_cache: 0 2024-09-15T22:37:38.153 INFO:journalctl@ceph.mon.b.smithi121.stdout: block_cache: 0x555fdf7711f0 2024-09-15T22:37:38.153 INFO:journalctl@ceph.mon.b.smithi121.stdout: block_cache_name: BinnedLRUCache 2024-09-15T22:37:38.153 INFO:journalctl@ceph.mon.b.smithi121.stdout: block_cache_options: 2024-09-15T22:37:38.153 INFO:journalctl@ceph.mon.b.smithi121.stdout: capacity : 536870912 2024-09-15T22:37:38.153 INFO:journalctl@ceph.mon.b.smithi121.stdout: num_shard_bits : 4 2024-09-15T22:37:38.153 INFO:journalctl@ceph.mon.b.smithi121.stdout: strict_capacity_limit : 0 2024-09-15T22:37:38.153 INFO:journalctl@ceph.mon.b.smithi121.stdout: high_pri_pool_ratio: 0.000 2024-09-15T22:37:38.153 INFO:journalctl@ceph.mon.b.smithi121.stdout: block_cache_compressed: (nil) 2024-09-15T22:37:38.153 INFO:journalctl@ceph.mon.b.smithi121.stdout: persistent_cache: (nil) 2024-09-15T22:37:38.154 INFO:journalctl@ceph.mon.b.smithi121.stdout: block_size: 4096 2024-09-15T22:37:38.154 INFO:journalctl@ceph.mon.b.smithi121.stdout: block_size_deviation: 10 2024-09-15T22:37:38.154 INFO:journalctl@ceph.mon.b.smithi121.stdout: block_restart_interval: 16 2024-09-15T22:37:38.154 INFO:journalctl@ceph.mon.b.smithi121.stdout: index_block_restart_interval: 1 2024-09-15T22:37:38.154 INFO:journalctl@ceph.mon.b.smithi121.stdout: metadata_block_size: 4096 2024-09-15T22:37:38.154 INFO:journalctl@ceph.mon.b.smithi121.stdout: partition_filters: 0 2024-09-15T22:37:38.154 INFO:journalctl@ceph.mon.b.smithi121.stdout: use_delta_encoding: 1 2024-09-15T22:37:38.154 INFO:journalctl@ceph.mon.b.smithi121.stdout: filter_policy: bloomfilter 2024-09-15T22:37:38.154 INFO:journalctl@ceph.mon.b.smithi121.stdout: whole_key_filtering: 1 2024-09-15T22:37:38.154 INFO:journalctl@ceph.mon.b.smithi121.stdout: verify_compression: 0 2024-09-15T22:37:38.155 INFO:journalctl@ceph.mon.b.smithi121.stdout: read_amp_bytes_per_bit: 0 2024-09-15T22:37:38.155 INFO:journalctl@ceph.mon.b.smithi121.stdout: format_version: 5 2024-09-15T22:37:38.155 INFO:journalctl@ceph.mon.b.smithi121.stdout: enable_index_compression: 1 2024-09-15T22:37:38.155 INFO:journalctl@ceph.mon.b.smithi121.stdout: block_align: 0 2024-09-15T22:37:38.155 INFO:journalctl@ceph.mon.b.smithi121.stdout: max_auto_readahead_size: 262144 2024-09-15T22:37:38.155 INFO:journalctl@ceph.mon.b.smithi121.stdout: prepopulate_block_cache: 0 2024-09-15T22:37:38.155 INFO:journalctl@ceph.mon.b.smithi121.stdout: initial_auto_readahead_size: 8192 2024-09-15T22:37:38.155 INFO:journalctl@ceph.mon.b.smithi121.stdout: num_file_reads_for_auto_readahead: 2 2024-09-15T22:37:38.155 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.write_buffer_size: 33554432 2024-09-15T22:37:38.155 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.max_write_buffer_number: 2 2024-09-15T22:37:38.155 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.compression: NoCompression 2024-09-15T22:37:38.155 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.bottommost_compression: Disabled 2024-09-15T22:37:38.155 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.prefix_extractor: nullptr 2024-09-15T22:37:38.156 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-15T22:37:38.156 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.num_levels: 7 2024-09-15T22:37:38.156 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-15T22:37:38.156 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-15T22:37:38.156 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-15T22:37:38.156 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-15T22:37:38.156 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-15T22:37:38.156 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-15T22:37:38.156 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-15T22:37:38.156 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-15T22:37:38.156 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-15T22:37:38.156 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-15T22:37:38.156 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-15T22:37:38.157 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-15T22:37:38.157 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-15T22:37:38.157 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.compression_opts.level: 32767 2024-09-15T22:37:38.157 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.compression_opts.strategy: 0 2024-09-15T22:37:38.157 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-15T22:37:38.157 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-15T22:37:38.157 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-15T22:37:38.157 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-15T22:37:38.157 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.compression_opts.enabled: false 2024-09-15T22:37:38.157 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-15T22:37:38.157 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-15T22:37:38.157 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-15T22:37:38.157 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-15T22:37:38.158 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.target_file_size_base: 67108864 2024-09-15T22:37:38.158 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-15T22:37:38.158 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-15T22:37:38.158 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-15T22:37:38.158 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-15T22:37:38.158 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-15T22:37:38.158 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-15T22:37:38.158 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-15T22:37:38.158 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-15T22:37:38.158 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-15T22:37:38.158 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-15T22:37:38.158 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-15T22:37:38.158 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-15T22:37:38.158 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-15T22:37:38.159 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-15T22:37:38.159 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.arena_block_size: 1048576 2024-09-15T22:37:38.159 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-15T22:37:38.159 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-15T22:37:38.159 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.disable_auto_compactions: 0 2024-09-15T22:37:38.159 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-15T22:37:38.159 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-15T22:37:38.159 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-15T22:37:38.159 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-15T22:37:38.159 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-15T22:37:38.159 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-15T22:37:38.159 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-15T22:37:38.159 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-15T22:37:38.159 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-15T22:37:38.159 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-15T22:37:38.160 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-15T22:37:38.160 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.inplace_update_support: 0 2024-09-15T22:37:38.160 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-15T22:37:38.160 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-15T22:37:38.160 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-15T22:37:38.160 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-15T22:37:38.160 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.bloom_locality: 0 2024-09-15T22:37:38.160 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.max_successive_merges: 0 2024-09-15T22:37:38.160 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-15T22:37:38.160 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.paranoid_file_checks: 0 2024-09-15T22:37:38.160 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.force_consistency_checks: 1 2024-09-15T22:37:38.160 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.report_bg_io_stats: 0 2024-09-15T22:37:38.160 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.ttl: 2592000 2024-09-15T22:37:38.161 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-15T22:37:38.161 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-15T22:37:38.161 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-15T22:37:38.161 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.enable_blob_files: false 2024-09-15T22:37:38.161 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.min_blob_size: 0 2024-09-15T22:37:38.161 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.blob_file_size: 268435456 2024-09-15T22:37:38.161 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-15T22:37:38.161 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-15T22:37:38.161 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-15T22:37:38.161 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-15T22:37:38.161 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-15T22:37:38.161 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.blob_file_starting_level: 0 2024-09-15T22:37:38.161 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-15T22:37:38.161 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-15T22:37:38.161 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-15T22:37:38.162 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d37dcef9-fb1f-4651-af5b-87dd615fc121 2024-09-15T22:37:38.162 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726439857751914, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-15T22:37:38.162 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-15T22:37:38.162 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726439857752311, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726439857, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d37dcef9-fb1f-4651-af5b-87dd615fc121", "db_session_id": "BW30JSFRH66HAVU8Q5D6", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-15T22:37:38.162 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726439857752406, "job": 1, "event": "recovery_finished"} 2024-09-15T22:37:38.162 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-15T22:37:38.162 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-15T22:37:38.162 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555fdf79ee00 2024-09-15T22:37:38.162 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: DB pointer 0x555fdf89a000 2024-09-15T22:37:38.162 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-15T22:37:38.162 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-15T22:37:38.162 INFO:journalctl@ceph.mon.b.smithi121.stdout: ** DB Stats ** 2024-09-15T22:37:38.162 INFO:journalctl@ceph.mon.b.smithi121.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-15T22:37:38.162 INFO:journalctl@ceph.mon.b.smithi121.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-15T22:37:38.163 INFO:journalctl@ceph.mon.b.smithi121.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-15T22:37:38.163 INFO:journalctl@ceph.mon.b.smithi121.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-15T22:37:38.163 INFO:journalctl@ceph.mon.b.smithi121.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-15T22:37:38.163 INFO:journalctl@ceph.mon.b.smithi121.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-15T22:37:38.163 INFO:journalctl@ceph.mon.b.smithi121.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-15T22:37:38.163 INFO:journalctl@ceph.mon.b.smithi121.stdout: 2024-09-15T22:37:38.163 INFO:journalctl@ceph.mon.b.smithi121.stdout: ** Compaction Stats [default] ** 2024-09-15T22:37:38.163 INFO:journalctl@ceph.mon.b.smithi121.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-15T22:37:38.163 INFO:journalctl@ceph.mon.b.smithi121.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-15T22:37:38.163 INFO:journalctl@ceph.mon.b.smithi121.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-15T22:37:38.163 INFO:journalctl@ceph.mon.b.smithi121.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-15T22:37:38.163 INFO:journalctl@ceph.mon.b.smithi121.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-15T22:37:38.163 INFO:journalctl@ceph.mon.b.smithi121.stdout: 2024-09-15T22:37:38.163 INFO:journalctl@ceph.mon.b.smithi121.stdout: ** Compaction Stats [default] ** 2024-09-15T22:37:38.164 INFO:journalctl@ceph.mon.b.smithi121.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-15T22:37:38.164 INFO:journalctl@ceph.mon.b.smithi121.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-15T22:37:38.164 INFO:journalctl@ceph.mon.b.smithi121.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-15T22:37:38.164 INFO:journalctl@ceph.mon.b.smithi121.stdout: 2024-09-15T22:37:38.164 INFO:journalctl@ceph.mon.b.smithi121.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-15T22:37:38.164 INFO:journalctl@ceph.mon.b.smithi121.stdout: 2024-09-15T22:37:38.164 INFO:journalctl@ceph.mon.b.smithi121.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-15T22:37:38.164 INFO:journalctl@ceph.mon.b.smithi121.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-15T22:37:38.164 INFO:journalctl@ceph.mon.b.smithi121.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-15T22:37:38.164 INFO:journalctl@ceph.mon.b.smithi121.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-15T22:37:38.164 INFO:journalctl@ceph.mon.b.smithi121.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-15T22:37:38.164 INFO:journalctl@ceph.mon.b.smithi121.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-15T22:37:38.164 INFO:journalctl@ceph.mon.b.smithi121.stdout: Cumulative compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-15T22:37:38.165 INFO:journalctl@ceph.mon.b.smithi121.stdout: Interval compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-15T22:37:38.165 INFO:journalctl@ceph.mon.b.smithi121.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-15T22:37:38.165 INFO:journalctl@ceph.mon.b.smithi121.stdout: Block cache BinnedLRUCache@0x555fdf7711f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-15T22:37:38.165 INFO:journalctl@ceph.mon.b.smithi121.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-15T22:37:38.165 INFO:journalctl@ceph.mon.b.smithi121.stdout: 2024-09-15T22:37:38.165 INFO:journalctl@ceph.mon.b.smithi121.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-15T22:37:38.165 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-15T22:37:38.165 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: using public_addr v2:172.21.15.121:0/0 -> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] 2024-09-15T22:37:38.165 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: starting mon.b rank -1 at public addrs [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] at bind addrs [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T22:37:38.165 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: mon.b@-1(???) e0 preinit fsid b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T22:37:38.165 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: mon.b@-1(synchronizing).mds e1 new map 2024-09-15T22:37:38.165 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-15T22:37:38.165 INFO:journalctl@ceph.mon.b.smithi121.stdout: e1 2024-09-15T22:37:38.166 INFO:journalctl@ceph.mon.b.smithi121.stdout: btime 2024-09-15T22:35:52:571222+0000 2024-09-15T22:37:38.166 INFO:journalctl@ceph.mon.b.smithi121.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-15T22:37:38.166 INFO:journalctl@ceph.mon.b.smithi121.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-15T22:37:38.166 INFO:journalctl@ceph.mon.b.smithi121.stdout: legacy client fscid: -1 2024-09-15T22:37:38.166 INFO:journalctl@ceph.mon.b.smithi121.stdout: 2024-09-15T22:37:38.166 INFO:journalctl@ceph.mon.b.smithi121.stdout: No filesystems configured 2024-09-15T22:37:38.166 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-15T22:37:38.166 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-15T22:37:38.166 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-15T22:37:38.166 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-15T22:37:38.166 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-15T22:37:38.166 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-15T22:37:38.166 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-15T22:37:38.167 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T22:37:38.167 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T22:37:38.167 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T22:37:38.167 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: Updating smithi121:/etc/ceph/ceph.conf 2024-09-15T22:37:38.167 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: Updating smithi121:/var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:37:38.167 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: Updating smithi121:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:37:38.167 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:38.167 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: Updating smithi121:/var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:37:38.167 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: Deploying cephadm binary to smithi192 2024-09-15T22:37:38.167 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:38.167 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:38.167 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:38.167 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:38.168 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:38.168 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:37:38.168 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:38.168 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:38.168 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: Added host smithi192 2024-09-15T22:37:38.168 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:38.168 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:38.168 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:37:38.168 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:38.168 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/3440763325' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-15T22:37:38.168 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:38.168 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:38.169 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:38.169 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:38.169 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi192", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:37:38.169 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:38.169 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:37:38.169 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: Updating smithi192:/etc/ceph/ceph.conf 2024-09-15T22:37:38.169 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: Updating smithi192:/var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:37:38.169 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: Updating smithi192:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:37:38.169 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/3440763325' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-15T22:37:38.169 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:37:38.169 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: Updating smithi192:/var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:37:38.169 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:38.169 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:38.170 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:38.170 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:38.170 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:38.170 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:37:38.170 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:38.170 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:37:38.170 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:38.170 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:37:38.170 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:38.171 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi111:172.21.15.111=a;smithi121:172.21.15.121=b;smithi192:172.21.15.192=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:37:38.171 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: Saving service mon spec with placement smithi111:172.21.15.111=a;smithi121:172.21.15.121=b;smithi192:172.21.15.192=c;count:3 2024-09-15T22:37:38.171 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: Deploying daemon mon.c on smithi192 2024-09-15T22:37:38.171 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:38.171 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='client.? 172.21.15.192:0/2189238001' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:37:38.171 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:38.171 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='client.? 172.21.15.192:0/475233628' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:37:38.171 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:37:38.171 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:37:38.171 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: mon.a calling monitor election 2024-09-15T22:37:38.171 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:37:38.171 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:38.171 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:37:38.172 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: mon.c calling monitor election 2024-09-15T22:37:38.172 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:37:38.172 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:38.172 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:37:38.172 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:37:38.172 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-15T22:37:38.172 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: monmap epoch 2 2024-09-15T22:37:38.172 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: fsid b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T22:37:38.172 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: last_changed 2024-09-15T22:37:29.083163+0000 2024-09-15T22:37:38.172 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: created 2024-09-15T22:35:48.940270+0000 2024-09-15T22:37:38.172 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: min_mon_release 19 (squid) 2024-09-15T22:37:38.172 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: election_strategy: 1 2024-09-15T22:37:38.172 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: 0: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.a 2024-09-15T22:37:38.173 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-09-15T22:37:38.173 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: fsmap 2024-09-15T22:37:38.173 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:37:38.173 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: mgrmap e13: x(active, since 43s) 2024-09-15T22:37:38.173 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: overall HEALTH_OK 2024-09-15T22:37:38.173 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:38.173 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:38.173 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:38.173 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:37:38.173 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:38.173 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: Deploying daemon mon.b on smithi121 2024-09-15T22:37:38.173 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:38.174 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='client.? 172.21.15.192:0/3884113458' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:37:38.174 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:37:38.174 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:38.174 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: from='client.? 172.21.15.192:0/3108180963' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:37:38.174 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 ceph-mon[34815]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-15T22:37:38.174 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 bash[34796]: d9e180790a275da76614d717a65f703bcedb54720a6560540f4efd31123ed3bb 2024-09-15T22:37:38.174 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:37 smithi121 systemd[1]: Started Ceph mon.b for b7614d3c-73b2-11ef-bceb-c7b262605968. 2024-09-15T22:37:38.591 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-15T22:37:38.591 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-15T22:37:38.839 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:37:43.133 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.129+0000 7f1436193640 1 -- 172.21.15.192:0/1177058573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1418003530 msgr2=0x7f14180059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:43.133 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.129+0000 7f1436193640 1 --2- 172.21.15.192:0/1177058573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1418003530 0x7f14180059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f14301016c0 tx=0x7f142002c3f0 comp rx=0 tx=0).stop 2024-09-15T22:37:43.133 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.129+0000 7f1436193640 1 -- 172.21.15.192:0/1177058573 shutdown_connections 2024-09-15T22:37:43.133 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.129+0000 7f1436193640 1 --2- 172.21.15.192:0/1177058573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1418003530 0x7f14180059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:43.133 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.129+0000 7f1436193640 1 -- 172.21.15.192:0/1177058573 >> 172.21.15.192:0/1177058573 conn(0x7f14300fc280 msgr2=0x7f14300fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:43.133 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.130+0000 7f1436193640 1 -- 172.21.15.192:0/1177058573 shutdown_connections 2024-09-15T22:37:43.133 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.130+0000 7f1436193640 1 -- 172.21.15.192:0/1177058573 wait complete. 2024-09-15T22:37:43.134 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.130+0000 7f1436193640 1 Processor -- start 2024-09-15T22:37:43.134 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.130+0000 7f1436193640 1 -- start start 2024-09-15T22:37:43.134 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.131+0000 7f1436193640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1418003530 0x7f1430197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:43.134 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.131+0000 7f1436193640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1430197890 0x7f143019bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:43.134 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.131+0000 7f1436193640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f143019c160 0x7f143019c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:43.134 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.131+0000 7f1436193640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1430108690 con 0x7f143019c160 2024-09-15T22:37:43.134 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.131+0000 7f1436193640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1430108510 con 0x7f1418003530 2024-09-15T22:37:43.134 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.131+0000 7f1436193640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1430108810 con 0x7f1430197890 2024-09-15T22:37:43.134 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.131+0000 7f142effd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1430197890 0x7f143019bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:43.134 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.131+0000 7f142effd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1430197890 0x7f143019bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:45414/0 (socket says 172.21.15.192:45414) 2024-09-15T22:37:43.134 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.131+0000 7f142effd640 1 -- 172.21.15.192:0/3255723579 learned_addr learned my addr 172.21.15.192:0/3255723579 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-09-15T22:37:43.134 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.131+0000 7f142f7fe640 1 --2- 172.21.15.192:0/3255723579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1418003530 0x7f1430197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:43.134 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.131+0000 7f142effd640 1 -- 172.21.15.192:0/3255723579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1430197890 msgr2=0x7f143019bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-15T22:37:43.135 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.131+0000 7f142effd640 1 -- 172.21.15.192:0/3255723579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1430197890 msgr2=0x7f143019bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T22:37:43.135 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.131+0000 7f142effd640 1 --2- 172.21.15.192:0/3255723579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1430197890 0x7f143019bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T22:37:43.135 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.131+0000 7f142ffff640 1 --2- 172.21.15.192:0/3255723579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f143019c160 0x7f143019c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:43.135 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.131+0000 7f142effd640 1 --2- 172.21.15.192:0/3255723579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1430197890 0x7f143019bc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:37:43.135 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.131+0000 7f142f7fe640 1 -- 172.21.15.192:0/3255723579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1418003530 msgr2=0x7f1430197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-15T22:37:43.135 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.131+0000 7f142f7fe640 1 -- 172.21.15.192:0/3255723579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1418003530 msgr2=0x7f1430197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T22:37:43.135 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.131+0000 7f142f7fe640 1 --2- 172.21.15.192:0/3255723579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1418003530 0x7f1430197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T22:37:43.135 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.131+0000 7f142f7fe640 1 --2- 172.21.15.192:0/3255723579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1418003530 0x7f1430197350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:37:43.135 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.131+0000 7f142ffff640 1 -- 172.21.15.192:0/3255723579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1430197890 msgr2=0x7f143019bc20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:37:43.135 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.131+0000 7f142ffff640 1 --2- 172.21.15.192:0/3255723579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1430197890 0x7f143019bc20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:43.135 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.131+0000 7f142ffff640 1 -- 172.21.15.192:0/3255723579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1418003530 msgr2=0x7f1430197350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:37:43.135 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.131+0000 7f142ffff640 1 --2- 172.21.15.192:0/3255723579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1418003530 0x7f1430197350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:43.135 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.131+0000 7f142ffff640 1 -- 172.21.15.192:0/3255723579 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14301a3ed0 con 0x7f143019c160 2024-09-15T22:37:43.136 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.132+0000 7f142ffff640 1 --2- 172.21.15.192:0/3255723579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f143019c160 0x7f143019c610 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f142800bd50 tx=0x7f142800be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:43.137 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.132+0000 7f142cff9640 1 -- 172.21.15.192:0/3255723579 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1428019070 con 0x7f143019c160 2024-09-15T22:37:43.137 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.132+0000 7f142cff9640 1 -- 172.21.15.192:0/3255723579 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1428014920 con 0x7f143019c160 2024-09-15T22:37:43.137 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.132+0000 7f142cff9640 1 -- 172.21.15.192:0/3255723579 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1428012990 con 0x7f143019c160 2024-09-15T22:37:43.137 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.132+0000 7f1436193640 1 -- 172.21.15.192:0/3255723579 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14301a4100 con 0x7f143019c160 2024-09-15T22:37:43.137 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.132+0000 7f1436193640 1 -- 172.21.15.192:0/3255723579 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14301a4690 con 0x7f143019c160 2024-09-15T22:37:43.137 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.133+0000 7f142cff9640 1 -- 172.21.15.192:0/3255723579 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f142801c070 con 0x7f143019c160 2024-09-15T22:37:43.137 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.133+0000 7f142cff9640 1 --2- 172.21.15.192:0/3255723579 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f141c039730 0x7f141c03bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:43.138 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.133+0000 7f142cff9640 1 -- 172.21.15.192:0/3255723579 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1428014ad0 con 0x7f143019c160 2024-09-15T22:37:43.138 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.133+0000 7f1436193640 1 -- 172.21.15.192:0/3255723579 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1430101cc0 con 0x7f143019c160 2024-09-15T22:37:43.138 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.134+0000 7f142f7fe640 1 --2- 172.21.15.192:0/3255723579 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f141c039730 0x7f141c03bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:43.138 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.134+0000 7f142f7fe640 1 --2- 172.21.15.192:0/3255723579 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f141c039730 0x7f141c03bbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1420014cc0 tx=0x7f1420005f60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:43.140 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.138+0000 7f142cff9640 1 -- 172.21.15.192:0/3255723579 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f14280139f0 con 0x7f143019c160 2024-09-15T22:37:43.140 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:37:43.140 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: mon.a calling monitor election 2024-09-15T22:37:43.140 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: mon.c calling monitor election 2024-09-15T22:37:43.140 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:37:43.140 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:37:43.141 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:43.141 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:37:43.141 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:37:43.141 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: mon.b calling monitor election 2024-09-15T22:37:43.141 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:43.141 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:37:43.141 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:37:43.141 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:43.141 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:37:43.141 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-15T22:37:43.141 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: monmap epoch 3 2024-09-15T22:37:43.141 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: fsid b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T22:37:43.141 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: last_changed 2024-09-15T22:37:37.820290+0000 2024-09-15T22:37:43.141 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: created 2024-09-15T22:35:48.940270+0000 2024-09-15T22:37:43.142 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: min_mon_release 19 (squid) 2024-09-15T22:37:43.142 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: election_strategy: 1 2024-09-15T22:37:43.142 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: 0: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.a 2024-09-15T22:37:43.142 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-09-15T22:37:43.142 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: 2: [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] mon.b 2024-09-15T22:37:43.142 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: fsmap 2024-09-15T22:37:43.142 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:37:43.142 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: mgrmap e13: x(active, since 52s) 2024-09-15T22:37:43.142 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: overall HEALTH_OK 2024-09-15T22:37:43.142 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:43.142 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:43.142 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:43.142 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:42 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:43.143 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:37:43.143 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: mon.a calling monitor election 2024-09-15T22:37:43.143 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: mon.c calling monitor election 2024-09-15T22:37:43.143 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:37:43.143 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:37:43.143 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:43.143 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:37:43.143 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:37:43.143 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: mon.b calling monitor election 2024-09-15T22:37:43.143 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:43.143 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:37:43.143 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:37:43.144 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:43.144 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:37:43.144 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-15T22:37:43.144 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: monmap epoch 3 2024-09-15T22:37:43.144 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: fsid b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T22:37:43.144 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: last_changed 2024-09-15T22:37:37.820290+0000 2024-09-15T22:37:43.144 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: created 2024-09-15T22:35:48.940270+0000 2024-09-15T22:37:43.144 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: min_mon_release 19 (squid) 2024-09-15T22:37:43.144 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: election_strategy: 1 2024-09-15T22:37:43.144 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: 0: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.a 2024-09-15T22:37:43.144 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-09-15T22:37:43.144 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: 2: [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] mon.b 2024-09-15T22:37:43.144 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: fsmap 2024-09-15T22:37:43.145 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:37:43.145 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: mgrmap e13: x(active, since 52s) 2024-09-15T22:37:43.145 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: overall HEALTH_OK 2024-09-15T22:37:43.145 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:43.145 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:43.145 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:43.145 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:42 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:43.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:37:43.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: mon.a calling monitor election 2024-09-15T22:37:43.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: mon.c calling monitor election 2024-09-15T22:37:43.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:37:43.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:37:43.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:43.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:37:43.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:37:43.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: mon.b calling monitor election 2024-09-15T22:37:43.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:43.167 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:37:43.167 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:37:43.167 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:43.167 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:37:43.167 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-15T22:37:43.167 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: monmap epoch 3 2024-09-15T22:37:43.167 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: fsid b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T22:37:43.168 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: last_changed 2024-09-15T22:37:37.820290+0000 2024-09-15T22:37:43.168 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: created 2024-09-15T22:35:48.940270+0000 2024-09-15T22:37:43.168 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: min_mon_release 19 (squid) 2024-09-15T22:37:43.168 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: election_strategy: 1 2024-09-15T22:37:43.168 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: 0: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.a 2024-09-15T22:37:43.168 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: 1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.c 2024-09-15T22:37:43.168 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: 2: [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] mon.b 2024-09-15T22:37:43.168 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: fsmap 2024-09-15T22:37:43.168 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:37:43.168 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: mgrmap e13: x(active, since 52s) 2024-09-15T22:37:43.168 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: overall HEALTH_OK 2024-09-15T22:37:43.168 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:43.168 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:43.168 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:43.169 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:42 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:43.386 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.384+0000 7f1436193640 1 -- 172.21.15.192:0/3255723579 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f14301a4980 con 0x7f143019c160 2024-09-15T22:37:43.386 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.384+0000 7f142cff9640 1 -- 172.21.15.192:0/3255723579 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1289 (secure 0 0 0) 0x7f1428013c50 con 0x7f143019c160 2024-09-15T22:37:43.386 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T22:37:43.386 INFO:teuthology.orchestra.run.smithi192.stdout:{"epoch":3,"fsid":"b7614d3c-73b2-11ef-bceb-c7b262605968","modified":"2024-09-15T22:37:37.820290Z","created":"2024-09-15T22:35:48.940270Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:3300","nonce":0},{"type":"v1","addr":"172.21.15.111:6789","nonce":0}]},"addr":"172.21.15.111:6789/0","public_addr":"172.21.15.111:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:3300","nonce":0},{"type":"v1","addr":"172.21.15.121:6789","nonce":0}]},"addr":"172.21.15.121:6789/0","public_addr":"172.21.15.121:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-15T22:37:43.387 INFO:teuthology.orchestra.run.smithi192.stderr:dumped monmap epoch 3 2024-09-15T22:37:43.388 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.387+0000 7f1436193640 1 -- 172.21.15.192:0/3255723579 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f141c039730 msgr2=0x7f141c03bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:43.389 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.387+0000 7f1436193640 1 --2- 172.21.15.192:0/3255723579 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f141c039730 0x7f141c03bbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1420014cc0 tx=0x7f1420005f60 comp rx=0 tx=0).stop 2024-09-15T22:37:43.389 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.387+0000 7f1436193640 1 -- 172.21.15.192:0/3255723579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f143019c160 msgr2=0x7f143019c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:43.389 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.387+0000 7f1436193640 1 --2- 172.21.15.192:0/3255723579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f143019c160 0x7f143019c610 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f142800bd50 tx=0x7f142800be40 comp rx=0 tx=0).stop 2024-09-15T22:37:43.389 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.387+0000 7f1436193640 1 -- 172.21.15.192:0/3255723579 shutdown_connections 2024-09-15T22:37:43.389 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.387+0000 7f1436193640 1 --2- 172.21.15.192:0/3255723579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f143019c160 0x7f143019c610 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:43.389 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.387+0000 7f1436193640 1 --2- 172.21.15.192:0/3255723579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1430197890 0x7f143019bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:43.389 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.387+0000 7f1436193640 1 --2- 172.21.15.192:0/3255723579 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f141c039730 0x7f141c03bbf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:43.389 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.387+0000 7f1436193640 1 --2- 172.21.15.192:0/3255723579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1418003530 0x7f1430197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:43.389 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.387+0000 7f1436193640 1 -- 172.21.15.192:0/3255723579 >> 172.21.15.192:0/3255723579 conn(0x7f14300fc280 msgr2=0x7f1430105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:43.389 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.387+0000 7f1436193640 1 -- 172.21.15.192:0/3255723579 shutdown_connections 2024-09-15T22:37:43.389 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:43.387+0000 7f1436193640 1 -- 172.21.15.192:0/3255723579 wait complete. 2024-09-15T22:37:43.921 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:43 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:37:43.921 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:43 smithi192 ceph-mon[35427]: from='client.? 172.21.15.192:0/3255723579' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:37:43.921 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:43 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:37:43.922 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-15T22:37:43.922 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-15T22:37:44.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:43 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:37:44.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:43 smithi111 ceph-mon[30629]: from='client.? 172.21.15.192:0/3255723579' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:37:44.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:43 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:37:44.306 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:44.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:43 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:37:44.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:43 smithi121 ceph-mon[34815]: from='client.? 172.21.15.192:0/3255723579' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:37:44.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:43 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:37:44.890 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:37:44 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:37:44.820+0000 7f29ad200640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-15T22:37:44.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.898+0000 7f4317200640 1 -- 172.21.15.111:0/2127776759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4310106390 msgr2=0x7f4310106770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:44.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.898+0000 7f4317200640 1 --2- 172.21.15.111:0/2127776759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4310106390 0x7f4310106770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4304009880 tx=0x7f430402a5d0 comp rx=0 tx=0).stop 2024-09-15T22:37:44.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.898+0000 7f4317200640 1 -- 172.21.15.111:0/2127776759 shutdown_connections 2024-09-15T22:37:44.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.898+0000 7f4317200640 1 --2- 172.21.15.111:0/2127776759 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4310106390 0x7f4310106770 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:44.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.899+0000 7f4317200640 1 -- 172.21.15.111:0/2127776759 >> 172.21.15.111:0/2127776759 conn(0x7f4310076010 msgr2=0x7f4310076420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:44.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.899+0000 7f4317200640 1 -- 172.21.15.111:0/2127776759 shutdown_connections 2024-09-15T22:37:44.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.899+0000 7f4317200640 1 -- 172.21.15.111:0/2127776759 wait complete. 2024-09-15T22:37:44.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.900+0000 7f4317200640 1 Processor -- start 2024-09-15T22:37:44.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.900+0000 7f4317200640 1 -- start start 2024-09-15T22:37:44.902 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.900+0000 7f4317200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4310106390 0x7f4310100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:44.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.900+0000 7f4317200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4310100d40 0x7f43101011a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:44.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.900+0000 7f4317200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43101a3400 0x7f43101a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:44.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.900+0000 7f4317200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f431010bba0 con 0x7f4310100d40 2024-09-15T22:37:44.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.900+0000 7f4317200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f431010ba20 con 0x7f4310106390 2024-09-15T22:37:44.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.900+0000 7f4317200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f431010bd20 con 0x7f43101a3400 2024-09-15T22:37:44.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.901+0000 7f4315e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4310106390 0x7f4310100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:44.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.901+0000 7f4315e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4310106390 0x7f4310100800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:56542/0 (socket says 172.21.15.111:56542) 2024-09-15T22:37:44.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.901+0000 7f4315e00640 1 -- 172.21.15.111:0/2216391944 learned_addr learned my addr 172.21.15.111:0/2216391944 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:37:44.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.901+0000 7f4315400640 1 --2- 172.21.15.111:0/2216391944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4310100d40 0x7f43101011a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:44.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.901+0000 7f4316800640 1 --2- 172.21.15.111:0/2216391944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43101a3400 0x7f43101a57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:44.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.901+0000 7f4315e00640 1 -- 172.21.15.111:0/2216391944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43101a3400 msgr2=0x7f43101a57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:44.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.901+0000 7f4315e00640 1 --2- 172.21.15.111:0/2216391944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43101a3400 0x7f43101a57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:44.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.901+0000 7f4315e00640 1 -- 172.21.15.111:0/2216391944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4310100d40 msgr2=0x7f43101011a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:44.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.901+0000 7f4315e00640 1 --2- 172.21.15.111:0/2216391944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4310100d40 0x7f43101011a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:44.904 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.901+0000 7f4315e00640 1 -- 172.21.15.111:0/2216391944 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43101a5ec0 con 0x7f4310106390 2024-09-15T22:37:44.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.901+0000 7f4316800640 1 --2- 172.21.15.111:0/2216391944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43101a3400 0x7f43101a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:37:44.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.902+0000 7f4315e00640 1 --2- 172.21.15.111:0/2216391944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4310106390 0x7f4310100800 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4304006010 tx=0x7f4304003c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:44.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.902+0000 7f430aa00640 1 -- 172.21.15.111:0/2216391944 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430403f070 con 0x7f4310106390 2024-09-15T22:37:44.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.902+0000 7f430aa00640 1 -- 172.21.15.111:0/2216391944 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f430403b460 con 0x7f4310106390 2024-09-15T22:37:44.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.902+0000 7f4317200640 1 -- 172.21.15.111:0/2216391944 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43101a6150 con 0x7f4310106390 2024-09-15T22:37:44.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.902+0000 7f430aa00640 1 -- 172.21.15.111:0/2216391944 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4304035030 con 0x7f4310106390 2024-09-15T22:37:44.905 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.902+0000 7f4317200640 1 -- 172.21.15.111:0/2216391944 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43101a6680 con 0x7f4310106390 2024-09-15T22:37:44.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.903+0000 7f430aa00640 1 -- 172.21.15.111:0/2216391944 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f4304042070 con 0x7f4310106390 2024-09-15T22:37:44.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.904+0000 7f4317200640 1 -- 172.21.15.111:0/2216391944 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4310106770 con 0x7f4310106390 2024-09-15T22:37:44.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.904+0000 7f430aa00640 1 --2- 172.21.15.111:0/2216391944 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f42f8039730 0x7f42f803bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:44.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.904+0000 7f430aa00640 1 -- 172.21.15.111:0/2216391944 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f430406cfd0 con 0x7f4310106390 2024-09-15T22:37:44.906 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.904+0000 7f4315400640 1 --2- 172.21.15.111:0/2216391944 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f42f8039730 0x7f42f803bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:44.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.905+0000 7f4315400640 1 --2- 172.21.15.111:0/2216391944 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f42f8039730 0x7f42f803bbf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4300009950 tx=0x7f4300008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:44.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:44.910+0000 7f430aa00640 1 -- 172.21.15.111:0/2216391944 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4304049030 con 0x7f4310106390 2024-09-15T22:37:45.146 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:45.144+0000 7f4317200640 1 -- 172.21.15.111:0/2216391944 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f43101a69a0 con 0x7f4310106390 2024-09-15T22:37:45.146 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:45 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:45.147 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:45 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:45.147 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:45 smithi111 ceph-mon[30629]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:45.147 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:45 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:45.147 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:45 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:45.147 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:45 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:45.147 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:45 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:37:45.147 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:45 smithi111 ceph-mon[30629]: Updating smithi111:/etc/ceph/ceph.conf 2024-09-15T22:37:45.147 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:45 smithi111 ceph-mon[30629]: Updating smithi121:/etc/ceph/ceph.conf 2024-09-15T22:37:45.147 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:45 smithi111 ceph-mon[30629]: Updating smithi192:/etc/ceph/ceph.conf 2024-09-15T22:37:45.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:45.145+0000 7f430aa00640 1 -- 172.21.15.111:0/2216391944 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f43040433b0 con 0x7f4310106390 2024-09-15T22:37:45.148 INFO:teuthology.orchestra.run.smithi111.stdout:# minimal ceph.conf for b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T22:37:45.148 INFO:teuthology.orchestra.run.smithi111.stdout:[global] 2024-09-15T22:37:45.148 INFO:teuthology.orchestra.run.smithi111.stdout: fsid = b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T22:37:45.148 INFO:teuthology.orchestra.run.smithi111.stdout: mon_host = [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] 2024-09-15T22:37:45.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:45.147+0000 7f4317200640 1 -- 172.21.15.111:0/2216391944 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f42f8039730 msgr2=0x7f42f803bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:45.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:45.147+0000 7f4317200640 1 --2- 172.21.15.111:0/2216391944 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f42f8039730 0x7f42f803bbf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4300009950 tx=0x7f4300008040 comp rx=0 tx=0).stop 2024-09-15T22:37:45.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:45.147+0000 7f4317200640 1 -- 172.21.15.111:0/2216391944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4310106390 msgr2=0x7f4310100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:45.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:45.147+0000 7f4317200640 1 --2- 172.21.15.111:0/2216391944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4310106390 0x7f4310100800 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4304006010 tx=0x7f4304003c00 comp rx=0 tx=0).stop 2024-09-15T22:37:45.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:45.147+0000 7f4317200640 1 -- 172.21.15.111:0/2216391944 shutdown_connections 2024-09-15T22:37:45.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:45.148+0000 7f4317200640 1 --2- 172.21.15.111:0/2216391944 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43101a3400 0x7f43101a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:45.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:45.148+0000 7f4317200640 1 --2- 172.21.15.111:0/2216391944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4310100d40 0x7f43101011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:45.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:45.148+0000 7f4317200640 1 --2- 172.21.15.111:0/2216391944 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f42f8039730 0x7f42f803bbf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:45.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:45.148+0000 7f4317200640 1 --2- 172.21.15.111:0/2216391944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4310106390 0x7f4310100800 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:45.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:45.148+0000 7f4317200640 1 -- 172.21.15.111:0/2216391944 >> 172.21.15.111:0/2216391944 conn(0x7f4310076010 msgr2=0x7f4310102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:45.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:45.148+0000 7f4317200640 1 -- 172.21.15.111:0/2216391944 shutdown_connections 2024-09-15T22:37:45.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:45.148+0000 7f4317200640 1 -- 172.21.15.111:0/2216391944 wait complete. 2024-09-15T22:37:45.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:45 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:45.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:45 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:45.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:45 smithi192 ceph-mon[35427]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:45.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:45 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:45.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:45 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:45.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:45 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:45.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:45 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:37:45.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:45 smithi192 ceph-mon[35427]: Updating smithi111:/etc/ceph/ceph.conf 2024-09-15T22:37:45.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:45 smithi192 ceph-mon[35427]: Updating smithi121:/etc/ceph/ceph.conf 2024-09-15T22:37:45.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:45 smithi192 ceph-mon[35427]: Updating smithi192:/etc/ceph/ceph.conf 2024-09-15T22:37:45.333 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:45 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:45.333 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:45 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:45.333 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:45 smithi121 ceph-mon[34815]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:45.333 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:45 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:45.333 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:45 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:45.333 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:45 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:45.333 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:45 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:37:45.333 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:45 smithi121 ceph-mon[34815]: Updating smithi111:/etc/ceph/ceph.conf 2024-09-15T22:37:45.333 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:45 smithi121 ceph-mon[34815]: Updating smithi121:/etc/ceph/ceph.conf 2024-09-15T22:37:45.333 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:45 smithi121 ceph-mon[34815]: Updating smithi192:/etc/ceph/ceph.conf 2024-09-15T22:37:46.008 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-15T22:37:46.009 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-15T22:37:46.009 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/ceph/ceph.conf 2024-09-15T22:37:46.071 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-15T22:37:46.071 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:37:46.170 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-15T22:37:46.170 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/ceph/ceph.conf 2024-09-15T22:37:46.194 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:46 smithi121 ceph-mon[34815]: Updating smithi192:/var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:37:46.194 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:46 smithi121 ceph-mon[34815]: Updating smithi121:/var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:37:46.194 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.194 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.194 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.195 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.195 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:46 smithi121 ceph-mon[34815]: Updating smithi111:/var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:37:46.195 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:46 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/2216391944' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:46.195 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.195 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.195 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.195 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.195 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.195 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.195 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.195 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.195 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:37:46.196 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:37:46.196 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:46.199 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-15T22:37:46.199 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:37:46.264 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T22:37:46.264 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.conf 2024-09-15T22:37:46.277 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:46 smithi111 ceph-mon[30629]: Updating smithi192:/var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:37:46.277 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:46 smithi111 ceph-mon[30629]: Updating smithi121:/var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:37:46.277 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.277 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.278 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.278 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.278 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:46 smithi111 ceph-mon[30629]: Updating smithi111:/var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:37:46.278 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:46 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/2216391944' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:46.278 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.278 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.278 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.278 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.278 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.278 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.278 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.278 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.278 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:37:46.278 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:37:46.279 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:46.291 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:46 smithi192 ceph-mon[35427]: Updating smithi192:/var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:37:46.292 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:46 smithi192 ceph-mon[35427]: Updating smithi121:/var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:37:46.292 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.292 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.292 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.292 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.292 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:46 smithi192 ceph-mon[35427]: Updating smithi111:/var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:37:46.292 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:46 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/2216391944' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:46.292 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.292 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.292 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.292 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.292 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.292 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.292 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.293 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:46.293 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:37:46.293 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:37:46.293 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:46.297 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T22:37:46.297 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:37:46.366 INFO:tasks.cephadm:Adding mgr.x on smithi111 2024-09-15T22:37:46.366 INFO:tasks.cephadm:Adding mgr.y on smithi121 2024-09-15T22:37:46.366 INFO:tasks.cephadm:Adding mgr.z on smithi192 2024-09-15T22:37:46.366 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi111=x;smithi121=y;smithi192=z' 2024-09-15T22:37:46.645 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:37:47.057 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.041+0000 7f4acacb5640 1 -- 172.21.15.192:0/1096611411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac41005e0 msgr2=0x7f4ac41009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:47.058 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.041+0000 7f4acacb5640 1 --2- 172.21.15.192:0/1096611411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac41005e0 0x7f4ac41009e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009880 tx=0x7f4ac002a5d0 comp rx=0 tx=0).stop 2024-09-15T22:37:47.058 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.041+0000 7f4acacb5640 1 -- 172.21.15.192:0/1096611411 shutdown_connections 2024-09-15T22:37:47.058 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.041+0000 7f4acacb5640 1 --2- 172.21.15.192:0/1096611411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac41005e0 0x7f4ac41009e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:47.058 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.041+0000 7f4acacb5640 1 -- 172.21.15.192:0/1096611411 >> 172.21.15.192:0/1096611411 conn(0x7f4ac40fbdb0 msgr2=0x7f4ac40fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:47.058 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.041+0000 7f4acacb5640 1 -- 172.21.15.192:0/1096611411 shutdown_connections 2024-09-15T22:37:47.059 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.041+0000 7f4acacb5640 1 -- 172.21.15.192:0/1096611411 wait complete. 2024-09-15T22:37:47.059 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.042+0000 7f4acacb5640 1 Processor -- start 2024-09-15T22:37:47.059 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.042+0000 7f4acacb5640 1 -- start start 2024-09-15T22:37:47.059 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.042+0000 7f4acacb5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ac41005e0 0x7f4ac406a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:47.059 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.042+0000 7f4acacb5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac4199d20 0x7f4ac419c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:47.059 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.042+0000 7f4acacb5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ac419c650 0x7f4ac419eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:47.060 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.042+0000 7f4acacb5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac41066d0 con 0x7f4ac4199d20 2024-09-15T22:37:47.060 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.042+0000 7f4acacb5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac4106550 con 0x7f4ac41005e0 2024-09-15T22:37:47.060 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.042+0000 7f4acacb5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac4106850 con 0x7f4ac419c650 2024-09-15T22:37:47.060 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.042+0000 7f4ac9252640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ac419c650 0x7f4ac419eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:47.060 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.042+0000 7f4ac9252640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ac419c650 0x7f4ac419eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:50282/0 (socket says 172.21.15.192:50282) 2024-09-15T22:37:47.061 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.042+0000 7f4ac9252640 1 -- 172.21.15.192:0/4242035653 learned_addr learned my addr 172.21.15.192:0/4242035653 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-09-15T22:37:47.061 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.042+0000 7f4abbfff640 1 --2- 172.21.15.192:0/4242035653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac4199d20 0x7f4ac419c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:47.061 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.042+0000 7f4ac8a51640 1 --2- 172.21.15.192:0/4242035653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ac41005e0 0x7f4ac406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:47.061 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.042+0000 7f4ac9252640 1 -- 172.21.15.192:0/4242035653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ac41005e0 msgr2=0x7f4ac406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:47.061 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.042+0000 7f4ac9252640 1 --2- 172.21.15.192:0/4242035653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ac41005e0 0x7f4ac406a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:47.062 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.042+0000 7f4ac9252640 1 -- 172.21.15.192:0/4242035653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac4199d20 msgr2=0x7f4ac419c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:47.062 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.042+0000 7f4ac9252640 1 --2- 172.21.15.192:0/4242035653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac4199d20 0x7f4ac419c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:47.062 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.042+0000 7f4ac9252640 1 -- 172.21.15.192:0/4242035653 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac419f420 con 0x7f4ac419c650 2024-09-15T22:37:47.062 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.042+0000 7f4abbfff640 1 --2- 172.21.15.192:0/4242035653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac4199d20 0x7f4ac419c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:37:47.062 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.043+0000 7f4ac9252640 1 --2- 172.21.15.192:0/4242035653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ac419c650 0x7f4ac419eb10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00ba50 tx=0x7f4abc00bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:47.062 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.043+0000 7f4ab9ffb640 1 -- 172.21.15.192:0/4242035653 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc019070 con 0x7f4ac419c650 2024-09-15T22:37:47.062 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.043+0000 7f4ab9ffb640 1 -- 172.21.15.192:0/4242035653 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4abc00cbe0 con 0x7f4ac419c650 2024-09-15T22:37:47.062 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.043+0000 7f4ab9ffb640 1 -- 172.21.15.192:0/4242035653 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc0076a0 con 0x7f4ac419c650 2024-09-15T22:37:47.062 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.043+0000 7f4acacb5640 1 -- 172.21.15.192:0/4242035653 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac419f6b0 con 0x7f4ac419c650 2024-09-15T22:37:47.062 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.043+0000 7f4acacb5640 1 -- 172.21.15.192:0/4242035653 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac419fc20 con 0x7f4ac419c650 2024-09-15T22:37:47.063 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.044+0000 7f4ab9ffb640 1 -- 172.21.15.192:0/4242035653 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f4abc01c070 con 0x7f4ac419c650 2024-09-15T22:37:47.063 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.044+0000 7f4ab9ffb640 1 --2- 172.21.15.192:0/4242035653 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4aac039730 0x7f4aac03bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:47.063 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.044+0000 7f4ab9ffb640 1 -- 172.21.15.192:0/4242035653 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4abc04bdb0 con 0x7f4ac419c650 2024-09-15T22:37:47.063 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.044+0000 7f4ac8a51640 1 --2- 172.21.15.192:0/4242035653 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4aac039730 0x7f4aac03bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:47.063 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.044+0000 7f4acacb5640 1 -- 172.21.15.192:0/4242035653 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ac418f490 con 0x7f4ac419c650 2024-09-15T22:37:47.063 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.045+0000 7f4ac8a51640 1 --2- 172.21.15.192:0/4242035653 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4aac039730 0x7f4aac03bbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009230 tx=0x7f4ac00315d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:47.063 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.049+0000 7f4ab9ffb640 1 -- 172.21.15.192:0/4242035653 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4abc0112a0 con 0x7f4ac419c650 2024-09-15T22:37:47.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.227+0000 7f4acacb5640 1 -- 172.21.15.192:0/4242035653 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi111=x;smithi121=y;smithi192=z", "target": ["mon-mgr", ""]}) -- 0x7f4ac40626c0 con 0x7f4aac039730 2024-09-15T22:37:47.234 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.233+0000 7f4ab9ffb640 1 -- 172.21.15.192:0/4242035653 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4ac40626c0 con 0x7f4aac039730 2024-09-15T22:37:47.235 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mgr update... 2024-09-15T22:37:47.237 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.235+0000 7f4acacb5640 1 -- 172.21.15.192:0/4242035653 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4aac039730 msgr2=0x7f4aac03bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:47.237 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.235+0000 7f4acacb5640 1 --2- 172.21.15.192:0/4242035653 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4aac039730 0x7f4aac03bbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009230 tx=0x7f4ac00315d0 comp rx=0 tx=0).stop 2024-09-15T22:37:47.237 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.235+0000 7f4acacb5640 1 -- 172.21.15.192:0/4242035653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ac419c650 msgr2=0x7f4ac419eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:47.237 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.235+0000 7f4acacb5640 1 --2- 172.21.15.192:0/4242035653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ac419c650 0x7f4ac419eb10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00ba50 tx=0x7f4abc00bf10 comp rx=0 tx=0).stop 2024-09-15T22:37:47.238 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.235+0000 7f4acacb5640 1 -- 172.21.15.192:0/4242035653 shutdown_connections 2024-09-15T22:37:47.238 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.235+0000 7f4acacb5640 1 --2- 172.21.15.192:0/4242035653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ac419c650 0x7f4ac419eb10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:47.238 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.235+0000 7f4acacb5640 1 --2- 172.21.15.192:0/4242035653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac4199d20 0x7f4ac419c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:47.238 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.235+0000 7f4acacb5640 1 --2- 172.21.15.192:0/4242035653 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4aac039730 0x7f4aac03bbf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:47.238 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.235+0000 7f4acacb5640 1 --2- 172.21.15.192:0/4242035653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ac41005e0 0x7f4ac406a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:47.238 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.235+0000 7f4acacb5640 1 -- 172.21.15.192:0/4242035653 >> 172.21.15.192:0/4242035653 conn(0x7f4ac40fbdb0 msgr2=0x7f4ac40fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:47.238 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.235+0000 7f4acacb5640 1 -- 172.21.15.192:0/4242035653 shutdown_connections 2024-09-15T22:37:47.238 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:37:47.235+0000 7f4acacb5640 1 -- 172.21.15.192:0/4242035653 wait complete. 2024-09-15T22:37:47.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:47 smithi192 ceph-mon[35427]: Reconfiguring mon.a (unknown last config time)... 2024-09-15T22:37:47.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:47 smithi192 ceph-mon[35427]: Reconfiguring daemon mon.a on smithi111 2024-09-15T22:37:47.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:47 smithi192 ceph-mon[35427]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:47.377 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:47 smithi111 ceph-mon[30629]: Reconfiguring mon.a (unknown last config time)... 2024-09-15T22:37:47.377 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:47 smithi111 ceph-mon[30629]: Reconfiguring daemon mon.a on smithi111 2024-09-15T22:37:47.377 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:47 smithi111 ceph-mon[30629]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:47.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:47 smithi121 ceph-mon[34815]: Reconfiguring mon.a (unknown last config time)... 2024-09-15T22:37:47.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:47 smithi121 ceph-mon[34815]: Reconfiguring daemon mon.a on smithi111 2024-09-15T22:37:47.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:47 smithi121 ceph-mon[34815]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:47.833 DEBUG:teuthology.orchestra.run.smithi121:mgr.y> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mgr.y.service 2024-09-15T22:37:47.836 DEBUG:teuthology.orchestra.run.smithi192:mgr.z> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mgr.z.service 2024-09-15T22:37:47.838 INFO:tasks.cephadm:Deploying OSDs... 2024-09-15T22:37:47.838 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-15T22:37:47.839 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T22:37:47.875 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T22:37:47.875 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_1 2024-09-15T22:37:47.952 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T22:37:47.952 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:37:47.952 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-09-15T22:37:47.952 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:37:47.952 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:37:47.952 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-09-15 22:37:04.321642902 +0000 2024-09-15T22:37:47.952 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-09-15 22:32:37.172099479 +0000 2024-09-15T22:37:47.952 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-09-15 22:32:37.172099479 +0000 2024-09-15T22:37:47.953 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-09-15 22:32:37.172099479 +0000 2024-09-15T22:37:47.953 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T22:37:48.048 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-09-15T22:37:48.049 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-09-15T22:37:48.049 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000341937 s, 1.5 MB/s 2024-09-15T22:37:48.051 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T22:37:48.088 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_2 2024-09-15T22:37:48.165 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T22:37:48.165 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:37:48.165 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-09-15T22:37:48.168 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:37:48.168 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:37:48.168 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-09-15 22:37:04.374641986 +0000 2024-09-15T22:37:48.169 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-09-15 22:32:37.649091480 +0000 2024-09-15T22:37:48.169 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-09-15 22:32:37.649091480 +0000 2024-09-15T22:37:48.169 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-09-15 22:32:37.649091480 +0000 2024-09-15T22:37:48.169 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T22:37:48.258 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-09-15T22:37:48.258 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-09-15T22:37:48.259 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.00030053 s, 1.7 MB/s 2024-09-15T22:37:48.261 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T22:37:48.301 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_3 2024-09-15T22:37:48.376 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T22:37:48.377 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:37:48.377 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-09-15T22:37:48.377 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:37:48.377 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:37:48.377 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-09-15 22:37:04.427641070 +0000 2024-09-15T22:37:48.377 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-09-15 22:32:38.123083534 +0000 2024-09-15T22:37:48.377 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-09-15 22:32:38.123083534 +0000 2024-09-15T22:37:48.377 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-09-15 22:32:38.123083534 +0000 2024-09-15T22:37:48.377 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T22:37:48.471 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-09-15T22:37:48.471 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-09-15T22:37:48.471 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000302437 s, 1.7 MB/s 2024-09-15T22:37:48.473 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T22:37:48.510 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_4 2024-09-15T22:37:48.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:48 smithi192 ceph-mon[35427]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi111=x;smithi121=y;smithi192=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:37:48.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:48 smithi192 ceph-mon[35427]: Saving service mgr spec with placement smithi111=x;smithi121=y;smithi192=z;count:3 2024-09-15T22:37:48.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:48 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:48.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:48 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:48.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:48 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:48.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:48 smithi192 ceph-mon[35427]: Reconfiguring mgr.x (unknown last config time)... 2024-09-15T22:37:48.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:48 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:37:48.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:48 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:37:48.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:48 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:48.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:48 smithi192 ceph-mon[35427]: Reconfiguring daemon mgr.x on smithi111 2024-09-15T22:37:48.586 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T22:37:48.586 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:37:48.586 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-09-15T22:37:48.586 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:37:48.586 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:37:48.587 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-09-15 22:37:04.479640170 +0000 2024-09-15T22:37:48.587 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-09-15 22:32:38.620075202 +0000 2024-09-15T22:37:48.587 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-09-15 22:32:38.620075202 +0000 2024-09-15T22:37:48.587 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-09-15 22:32:38.620075202 +0000 2024-09-15T22:37:48.587 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T22:37:48.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:48 smithi121 ceph-mon[34815]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi111=x;smithi121=y;smithi192=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:37:48.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:48 smithi121 ceph-mon[34815]: Saving service mgr spec with placement smithi111=x;smithi121=y;smithi192=z;count:3 2024-09-15T22:37:48.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:48 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:48.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:48 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:48.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:48 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:48.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:48 smithi121 ceph-mon[34815]: Reconfiguring mgr.x (unknown last config time)... 2024-09-15T22:37:48.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:48 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:37:48.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:48 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:37:48.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:48 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:48.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:48 smithi121 ceph-mon[34815]: Reconfiguring daemon mgr.x on smithi111 2024-09-15T22:37:48.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:48 smithi111 ceph-mon[30629]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi111=x;smithi121=y;smithi192=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:37:48.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:48 smithi111 ceph-mon[30629]: Saving service mgr spec with placement smithi111=x;smithi121=y;smithi192=z;count:3 2024-09-15T22:37:48.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:48 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:48.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:48 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:48.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:48 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:48.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:48 smithi111 ceph-mon[30629]: Reconfiguring mgr.x (unknown last config time)... 2024-09-15T22:37:48.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:48 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:37:48.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:48 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:37:48.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:48 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:48.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:48 smithi111 ceph-mon[30629]: Reconfiguring daemon mgr.x on smithi111 2024-09-15T22:37:48.680 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2024-09-15T22:37:48.680 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2024-09-15T22:37:48.680 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000263699 s, 1.9 MB/s 2024-09-15T22:37:48.682 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T22:37:48.759 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-15T22:37:48.759 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T22:37:48.778 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T22:37:48.778 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_1 2024-09-15T22:37:48.838 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T22:37:48.838 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:37:48.838 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-15T22:37:48.838 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:37:48.838 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:37:48.838 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-09-15 22:37:37.199848028 +0000 2024-09-15T22:37:48.838 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-09-15 22:32:02.163681069 +0000 2024-09-15T22:37:48.838 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-09-15 22:32:02.163681069 +0000 2024-09-15T22:37:48.838 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-09-15 22:32:02.163681069 +0000 2024-09-15T22:37:48.839 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T22:37:48.906 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-09-15T22:37:48.906 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-09-15T22:37:48.907 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000305011 s, 1.7 MB/s 2024-09-15T22:37:48.908 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T22:37:48.967 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_2 2024-09-15T22:37:49.025 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T22:37:49.026 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:37:49.026 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-15T22:37:49.026 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:37:49.026 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:37:49.026 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-09-15 22:37:37.199848028 +0000 2024-09-15T22:37:49.026 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-09-15 22:32:02.494676929 +0000 2024-09-15T22:37:49.026 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-09-15 22:32:02.494676929 +0000 2024-09-15T22:37:49.026 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-09-15 22:32:02.494676929 +0000 2024-09-15T22:37:49.026 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T22:37:49.092 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-09-15T22:37:49.092 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-09-15T22:37:49.092 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000236027 s, 2.2 MB/s 2024-09-15T22:37:49.093 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T22:37:49.151 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_3 2024-09-15T22:37:49.208 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T22:37:49.208 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:37:49.208 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-15T22:37:49.208 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:37:49.209 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:37:49.209 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-09-15 22:37:37.200848015 +0000 2024-09-15T22:37:49.209 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-09-15 22:32:02.824672802 +0000 2024-09-15T22:37:49.209 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-09-15 22:32:02.824672802 +0000 2024-09-15T22:37:49.209 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-09-15 22:32:02.824672802 +0000 2024-09-15T22:37:49.209 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T22:37:49.273 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-09-15T22:37:49.273 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-09-15T22:37:49.273 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000184182 s, 2.8 MB/s 2024-09-15T22:37:49.274 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T22:37:49.331 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_4 2024-09-15T22:37:49.387 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T22:37:49.387 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:37:49.387 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-09-15T22:37:49.387 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:37:49.387 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:37:49.387 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-09-15 22:37:37.200848015 +0000 2024-09-15T22:37:49.387 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-09-15 22:32:03.155668662 +0000 2024-09-15T22:37:49.387 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-09-15 22:32:03.155668662 +0000 2024-09-15T22:37:49.387 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-09-15 22:32:03.155668662 +0000 2024-09-15T22:37:49.388 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T22:37:49.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:49 smithi111 ceph-mon[30629]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:49.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:49 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:49.450 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-09-15T22:37:49.450 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-09-15T22:37:49.450 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000197484 s, 2.6 MB/s 2024-09-15T22:37:49.452 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T22:37:49.508 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T22:37:49.508 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T22:37:49.527 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T22:37:49.528 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_1 2024-09-15T22:37:49.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:49 smithi192 ceph-mon[35427]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:49.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:49 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:49.582 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T22:37:49.583 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:37:49.583 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-15T22:37:49.583 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:37:49.583 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:37:49.583 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-09-15 22:37:27.904712093 +0000 2024-09-15T22:37:49.584 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-09-15 22:32:02.169835271 +0000 2024-09-15T22:37:49.584 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-09-15 22:32:02.169835271 +0000 2024-09-15T22:37:49.584 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-09-15 22:32:02.169835271 +0000 2024-09-15T22:37:49.584 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T22:37:49.601 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:49 smithi121 ceph-mon[34815]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:49.601 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:49 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:49.650 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-09-15T22:37:49.650 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-09-15T22:37:49.651 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000322816 s, 1.6 MB/s 2024-09-15T22:37:49.652 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T22:37:49.710 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_2 2024-09-15T22:37:49.767 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T22:37:49.767 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:37:49.767 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-15T22:37:49.767 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:37:49.767 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:37:49.767 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-09-15 22:37:27.904712093 +0000 2024-09-15T22:37:49.768 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-09-15 22:32:02.496828811 +0000 2024-09-15T22:37:49.768 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-09-15 22:32:02.496828811 +0000 2024-09-15T22:37:49.768 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-09-15 22:32:02.496828811 +0000 2024-09-15T22:37:49.768 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T22:37:49.833 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-09-15T22:37:49.833 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-09-15T22:37:49.833 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000228245 s, 2.2 MB/s 2024-09-15T22:37:49.834 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T22:37:49.892 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_3 2024-09-15T22:37:49.948 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T22:37:49.948 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:37:49.948 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-09-15T22:37:49.949 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:37:49.949 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:37:49.949 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-09-15 22:37:27.905712074 +0000 2024-09-15T22:37:49.949 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-09-15 22:32:02.848821857 +0000 2024-09-15T22:37:49.949 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-09-15 22:32:02.848821857 +0000 2024-09-15T22:37:49.949 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-09-15 22:32:02.848821857 +0000 2024-09-15T22:37:49.949 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T22:37:50.012 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-09-15T22:37:50.013 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-09-15T22:37:50.013 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000194716 s, 2.6 MB/s 2024-09-15T22:37:50.014 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T22:37:50.071 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_4 2024-09-15T22:37:50.126 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T22:37:50.127 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:37:50.127 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-15T22:37:50.127 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:37:50.127 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:37:50.127 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-09-15 22:37:27.905712074 +0000 2024-09-15T22:37:50.127 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-09-15 22:32:03.183815240 +0000 2024-09-15T22:37:50.127 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-09-15 22:32:03.183815240 +0000 2024-09-15T22:37:50.127 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-09-15 22:32:03.183815240 +0000 2024-09-15T22:37:50.128 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T22:37:50.191 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-09-15T22:37:50.191 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-09-15T22:37:50.191 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000226448 s, 2.3 MB/s 2024-09-15T22:37:50.192 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T22:37:50.249 INFO:tasks.cephadm:Deploying osd.0 on smithi111 with /dev/vg_nvme/lv_4... 2024-09-15T22:37:50.249 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-15T22:37:50.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:50.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:50 smithi192 ceph-mon[35427]: Reconfiguring mon.b (monmap changed)... 2024-09-15T22:37:50.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:37:50.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:37:50.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:50.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:50 smithi192 ceph-mon[35427]: Reconfiguring daemon mon.b on smithi121 2024-09-15T22:37:50.605 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:50.605 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:50 smithi121 ceph-mon[34815]: Reconfiguring mon.b (monmap changed)... 2024-09-15T22:37:50.605 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:37:50.605 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:37:50.605 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:50.605 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:50 smithi121 ceph-mon[34815]: Reconfiguring daemon mon.b on smithi121 2024-09-15T22:37:50.608 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:50.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:50.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:50 smithi111 ceph-mon[30629]: Reconfiguring mon.b (monmap changed)... 2024-09-15T22:37:50.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:37:50.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:37:50.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:50.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:50 smithi111 ceph-mon[30629]: Reconfiguring daemon mon.b on smithi121 2024-09-15T22:37:51.792 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:51 smithi192 ceph-mon[35427]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:51.792 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:51 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:51.792 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:51 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:51.792 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:51 smithi192 ceph-mon[35427]: Reconfiguring mon.c (monmap changed)... 2024-09-15T22:37:51.792 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:51 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:37:51.792 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:51 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:37:51.792 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:51 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:51.792 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:51 smithi192 ceph-mon[35427]: Reconfiguring daemon mon.c on smithi192 2024-09-15T22:37:51.796 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:51 smithi111 ceph-mon[30629]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:51.796 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:51 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:51.796 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:51 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:51.796 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:51 smithi111 ceph-mon[30629]: Reconfiguring mon.c (monmap changed)... 2024-09-15T22:37:51.796 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:51 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:37:51.796 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:51 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:37:51.796 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:51 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:51.796 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:51 smithi111 ceph-mon[30629]: Reconfiguring daemon mon.c on smithi192 2024-09-15T22:37:51.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:51 smithi121 ceph-mon[34815]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:51.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:51 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:51.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:51 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:51.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:51 smithi121 ceph-mon[34815]: Reconfiguring mon.c (monmap changed)... 2024-09-15T22:37:51.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:51 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:37:51.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:51 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:37:51.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:51 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:51.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:51 smithi121 ceph-mon[34815]: Reconfiguring daemon mon.c on smithi192 2024-09-15T22:37:53.005 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:52 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:53.005 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:52 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:53.005 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:52 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:37:53.006 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:52 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:53.006 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:52 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:37:53.006 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:52 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:53.006 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:52 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:37:53.006 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:52 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T22:37:53.006 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:52 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:37:53.006 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:52 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:53.006 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:52 smithi111 ceph-mon[30629]: Deploying daemon mgr.z on smithi192 2024-09-15T22:37:53.006 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:52 smithi111 ceph-mon[30629]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:53.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:52 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:53.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:52 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:53.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:52 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:37:53.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:52 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:53.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:52 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:37:53.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:52 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:53.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:52 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:37:53.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:52 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T22:37:53.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:52 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:37:53.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:52 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:53.066 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:52 smithi192 ceph-mon[35427]: Deploying daemon mgr.z on smithi192 2024-09-15T22:37:53.066 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:52 smithi192 ceph-mon[35427]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:53.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:52 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:53.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:52 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:53.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:52 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:37:53.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:52 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:53.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:52 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:37:53.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:52 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:53.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:52 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:37:53.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:52 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T22:37:53.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:52 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:37:53.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:52 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:53.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:52 smithi121 ceph-mon[34815]: Deploying daemon mgr.z on smithi192 2024-09-15T22:37:53.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:52 smithi121 ceph-mon[34815]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:53.601 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:37:53.632 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_4 2024-09-15T22:37:53.761 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:53 smithi192 systemd[1]: Starting Ceph mgr.z for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T22:37:53.987 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:37:54.046 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:53 smithi192 podman[37221]: 2024-09-15 22:37:53.76016949 +0000 UTC m=+0.110142676 container create 983cb48dfec383b62c06bd24f21dcfac2803e854a16459fddcd21879470e4eb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-15T22:37:54.046 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:53 smithi192 podman[37221]: 2024-09-15 22:37:53.666186455 +0000 UTC m=+0.016159655 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:37:54.046 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:53 smithi192 podman[37221]: 2024-09-15 22:37:53.962386802 +0000 UTC m=+0.312359988 container init 983cb48dfec383b62c06bd24f21dcfac2803e854a16459fddcd21879470e4eb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-15T22:37:54.047 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:53 smithi192 podman[37221]: 2024-09-15 22:37:53.96604825 +0000 UTC m=+0.316021444 container start 983cb48dfec383b62c06bd24f21dcfac2803e854a16459fddcd21879470e4eb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-15T22:37:54.047 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:53 smithi192 ceph-mgr[37240]: -- 172.21.15.192:0/3644392421 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5628060af0e0 con 0x56280608c800 2024-09-15T22:37:54.047 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:54 smithi192 bash[37221]: 983cb48dfec383b62c06bd24f21dcfac2803e854a16459fddcd21879470e4eb5 2024-09-15T22:37:54.305 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:54 smithi192 systemd[1]: Started Ceph mgr.z for b7614d3c-73b2-11ef-bceb-c7b262605968. 2024-09-15T22:37:54.305 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:54 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-15T22:37:54.093+0000 7f1e2fecd040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T22:37:54.305 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:54 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-15T22:37:54.155+0000 7f1e2fecd040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T22:37:54.305 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:54 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-15T22:37:54.303+0000 7f1e2fecd040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T22:37:54.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.559+0000 7f7943e00640 1 -- 172.21.15.111:0/4114349242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7944078790 msgr2=0x7f7944079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:54.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.559+0000 7f7943e00640 1 --2- 172.21.15.111:0/4114349242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7944078790 0x7f7944079030 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f793c009be0 tx=0x7f793c02a6c0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.559+0000 7f7943e00640 1 -- 172.21.15.111:0/4114349242 shutdown_connections 2024-09-15T22:37:54.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.559+0000 7f7943e00640 1 --2- 172.21.15.111:0/4114349242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7944078790 0x7f7944079030 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.559+0000 7f7943e00640 1 --2- 172.21.15.111:0/4114349242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7944077df0 0x7f7944078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.559+0000 7f7943e00640 1 --2- 172.21.15.111:0/4114349242 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7944076ba0 0x7f7944076fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.559+0000 7f7943e00640 1 -- 172.21.15.111:0/4114349242 >> 172.21.15.111:0/4114349242 conn(0x7f79440ff3e0 msgr2=0x7f7944101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:37:54.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.559+0000 7f7943e00640 1 -- 172.21.15.111:0/4114349242 shutdown_connections 2024-09-15T22:37:54.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.559+0000 7f7943e00640 1 -- 172.21.15.111:0/4114349242 wait complete. 2024-09-15T22:37:54.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.560+0000 7f7943e00640 1 Processor -- start 2024-09-15T22:37:54.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.561+0000 7f7943e00640 1 -- start start 2024-09-15T22:37:54.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.561+0000 7f7943e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7944076ba0 0x7f794419e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.561+0000 7f7943e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7944077df0 0x7f794419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.561+0000 7f7943e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7944078790 0x7f79441a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.561+0000 7f7943e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7944113cf0 con 0x7f7944078790 2024-09-15T22:37:54.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.561+0000 7f7943e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7944113b70 con 0x7f7944076ba0 2024-09-15T22:37:54.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.561+0000 7f7943e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7944113e70 con 0x7f7944077df0 2024-09-15T22:37:54.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.561+0000 7f7942000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7944077df0 0x7f794419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.561+0000 7f7942000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7944077df0 0x7f794419e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:39422/0 (socket says 172.21.15.111:39422) 2024-09-15T22:37:54.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.561+0000 7f7942000640 1 -- 172.21.15.111:0/3391860575 learned_addr learned my addr 172.21.15.111:0/3391860575 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:37:54.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.561+0000 7f7942a00640 1 --2- 172.21.15.111:0/3391860575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7944076ba0 0x7f794419e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.561+0000 7f7943400640 1 --2- 172.21.15.111:0/3391860575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7944078790 0x7f79441a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.562+0000 7f7942a00640 1 -- 172.21.15.111:0/3391860575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7944077df0 msgr2=0x7f794419e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:54.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.562+0000 7f7942a00640 1 --2- 172.21.15.111:0/3391860575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7944077df0 0x7f794419e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.562+0000 7f7942a00640 1 -- 172.21.15.111:0/3391860575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7944078790 msgr2=0x7f79441a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:37:54.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.562+0000 7f7942a00640 1 --2- 172.21.15.111:0/3391860575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7944078790 0x7f79441a5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:37:54.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.562+0000 7f7942a00640 1 -- 172.21.15.111:0/3391860575 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79441a5ec0 con 0x7f7944076ba0 2024-09-15T22:37:54.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.562+0000 7f7942000640 1 --2- 172.21.15.111:0/3391860575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7944077df0 0x7f794419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:37:54.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.562+0000 7f7942a00640 1 --2- 172.21.15.111:0/3391860575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7944076ba0 0x7f794419e290 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f793400e9f0 tx=0x7f793400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:54.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.562+0000 7f793b400640 1 -- 172.21.15.111:0/3391860575 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793400ccc0 con 0x7f7944076ba0 2024-09-15T22:37:54.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.562+0000 7f7943e00640 1 -- 172.21.15.111:0/3391860575 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79441a61b0 con 0x7f7944076ba0 2024-09-15T22:37:54.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.562+0000 7f793b400640 1 -- 172.21.15.111:0/3391860575 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7934005020 con 0x7f7944076ba0 2024-09-15T22:37:54.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.562+0000 7f793b400640 1 -- 172.21.15.111:0/3391860575 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79340049e0 con 0x7f7944076ba0 2024-09-15T22:37:54.567 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.562+0000 7f7943e00640 1 -- 172.21.15.111:0/3391860575 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79441a66f0 con 0x7f7944076ba0 2024-09-15T22:37:54.568 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:54 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-15T22:37:54.369+0000 7f1e2fecd040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T22:37:54.568 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:54 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-15T22:37:54.518+0000 7f1e2fecd040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:37:54.568 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.563+0000 7f7943e00640 1 -- 172.21.15.111:0/3391860575 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79441105d0 con 0x7f7944076ba0 2024-09-15T22:37:54.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.564+0000 7f793b400640 1 -- 172.21.15.111:0/3391860575 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 13) ==== 46518+0+0 (secure 0 0 0) 0x7f7934004bc0 con 0x7f7944076ba0 2024-09-15T22:37:54.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.564+0000 7f793b400640 1 --2- 172.21.15.111:0/3391860575 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7928039500 0x7f792803b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:37:54.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.564+0000 7f793b400640 1 -- 172.21.15.111:0/3391860575 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f793404c5b0 con 0x7f7944076ba0 2024-09-15T22:37:54.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.564+0000 7f7942000640 1 --2- 172.21.15.111:0/3391860575 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7928039500 0x7f792803b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:37:54.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.565+0000 7f7942000640 1 --2- 172.21.15.111:0/3391860575 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7928039500 0x7f792803b9c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f794419f7b0 tx=0x7f7930009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:37:54.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.569+0000 7f793b400640 1 -- 172.21.15.111:0/3391860575 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f793401ede0 con 0x7f7944076ba0 2024-09-15T22:37:54.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:54.755+0000 7f7943e00640 1 -- 172.21.15.111:0/3391860575 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7944062630 con 0x7f7928039500 2024-09-15T22:37:54.971 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:54 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-15T22:37:54.658+0000 7f1e2fecd040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T22:37:54.971 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:54 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-15T22:37:54.969+0000 7f1e2fecd040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T22:37:55.223 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:55 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-15T22:37:55.029+0000 7f1e2fecd040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T22:37:55.223 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:55 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:55.223 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:55 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:55.223 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:55 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:55.223 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:55 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:37:55.223 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:55 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T22:37:55.223 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:55 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:37:55.223 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:55 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:55.223 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:55 smithi192 ceph-mon[35427]: Deploying daemon mgr.y on smithi121 2024-09-15T22:37:55.223 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:55 smithi192 ceph-mon[35427]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:55.224 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:55 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:37:55.224 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:55 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:37:55.224 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:55 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:55.368 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:55 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:55.369 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:55 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:55.369 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:55 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:55.369 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:55 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:37:55.369 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:55 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T22:37:55.369 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:55 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:37:55.370 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:55 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:55.370 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:55 smithi121 ceph-mon[34815]: Deploying daemon mgr.y on smithi121 2024-09-15T22:37:55.370 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:55 smithi121 ceph-mon[34815]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:55.370 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:55 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:37:55.370 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:55 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:37:55.371 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:55 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:55.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:55 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:55.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:55 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:55.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:55 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:55.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:55 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:37:55.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:55 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T22:37:55.417 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:55 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:37:55.417 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:55 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:55.417 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:55 smithi111 ceph-mon[30629]: Deploying daemon mgr.y on smithi121 2024-09-15T22:37:55.417 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:55 smithi111 ceph-mon[30629]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:55.418 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:55 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:37:55.418 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:55 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:37:55.418 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:55 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:55.564 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:55 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-15T22:37:55.221+0000 7f1e2fecd040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T22:37:56.386 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:56 smithi192 ceph-mon[35427]: from='client.24110 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:37:56.387 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:56 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-15T22:37:56.150+0000 7f1e2fecd040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T22:37:56.387 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:56 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-15T22:37:56.219+0000 7f1e2fecd040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T22:37:56.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:56 smithi121 ceph-mon[34815]: from='client.24110 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:37:56.390 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 22:37:56 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-y[36178]: 2024-09-15T22:37:56.355+0000 7fe618c68040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T22:37:56.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:56 smithi111 ceph-mon[30629]: from='client.24110 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:37:56.890 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 22:37:56 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-y[36178]: 2024-09-15T22:37:56.509+0000 7fe618c68040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T22:37:56.890 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 22:37:56 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-y[36178]: 2024-09-15T22:37:56.579+0000 7fe618c68040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T22:37:56.890 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 22:37:56 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-y[36178]: 2024-09-15T22:37:56.736+0000 7fe618c68040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:37:57.184 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 22:37:56 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-y[36178]: 2024-09-15T22:37:56.892+0000 7fe618c68040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T22:37:57.267 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:56 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-15T22:37:56.995+0000 7f1e2fecd040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T22:37:57.267 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:57 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-15T22:37:57.117+0000 7f1e2fecd040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T22:37:57.267 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:57 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-15T22:37:57.265+0000 7f1e2fecd040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:37:57.483 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 22:37:57 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-y[36178]: 2024-09-15T22:37:57.217+0000 7fe618c68040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T22:37:57.483 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 22:37:57 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-y[36178]: 2024-09-15T22:37:57.281+0000 7fe618c68040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T22:37:57.483 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:57.483 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:57.483 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:57.484 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:57.484 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:57 smithi121 ceph-mon[34815]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:57.484 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:37:57.564 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:57 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-15T22:37:57.332+0000 7f1e2fecd040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T22:37:57.565 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:57 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-15T22:37:57.488+0000 7f1e2fecd040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T22:37:57.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:57.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:57.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:57.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:57.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:57 smithi192 ceph-mon[35427]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:57.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:37:57.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:57.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:57.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:57.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:57.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:57 smithi111 ceph-mon[30629]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:57.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:37:57.864 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:57 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-15T22:37:57.567+0000 7f1e2fecd040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T22:37:57.864 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:57 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-15T22:37:57.630+0000 7f1e2fecd040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T22:37:57.864 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:57 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-15T22:37:57.794+0000 7f1e2fecd040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T22:37:57.864 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:57 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-15T22:37:57.862+0000 7f1e2fecd040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T22:37:57.890 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 22:37:57 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-y[36178]: 2024-09-15T22:37:57.481+0000 7fe618c68040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T22:37:58.131 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:57 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-15T22:37:57.928+0000 7f1e2fecd040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T22:37:58.132 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:58 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-15T22:37:58.011+0000 7f1e2fecd040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T22:37:58.132 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:58 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-15T22:37:58.071+0000 7f1e2fecd040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T22:37:58.132 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:58 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-15T22:37:58.130+0000 7f1e2fecd040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T22:37:58.640 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 22:37:58 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-y[36178]: 2024-09-15T22:37:58.379+0000 7fe618c68040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T22:37:58.641 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 22:37:58 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-y[36178]: 2024-09-15T22:37:58.447+0000 7fe618c68040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T22:37:58.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:58 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:58.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:58 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:58.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:58 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:58.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:58 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:58.642 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:58 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:58.642 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:58 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:37:58.642 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:58 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:58.642 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:58 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:37:58.642 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:58 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:58.642 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:58 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:37:58.643 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:58 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:58.643 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:58 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/2363244923' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04f39604-be0e-4261-b316-67fa087c6666"}]: dispatch 2024-09-15T22:37:58.643 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:58 smithi121 ceph-mon[34815]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04f39604-be0e-4261-b316-67fa087c6666"}]: dispatch 2024-09-15T22:37:58.643 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:58 smithi121 ceph-mon[34815]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "04f39604-be0e-4261-b316-67fa087c6666"}]': finished 2024-09-15T22:37:58.643 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:58 smithi121 ceph-mon[34815]: osdmap e5: 1 total, 0 up, 1 in 2024-09-15T22:37:58.643 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:58 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:37:58.644 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:58 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/4164650520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:37:58.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:58 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:58.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:58 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:58.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:58 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:58.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:58 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:58.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:58 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:58.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:58 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:37:58.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:58 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:58.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:58 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:37:58.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:58 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:58.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:58 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:37:58.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:58 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:58.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:58 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/2363244923' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04f39604-be0e-4261-b316-67fa087c6666"}]: dispatch 2024-09-15T22:37:58.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:58 smithi192 ceph-mon[35427]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04f39604-be0e-4261-b316-67fa087c6666"}]: dispatch 2024-09-15T22:37:58.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:58 smithi192 ceph-mon[35427]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "04f39604-be0e-4261-b316-67fa087c6666"}]': finished 2024-09-15T22:37:58.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:58 smithi192 ceph-mon[35427]: osdmap e5: 1 total, 0 up, 1 in 2024-09-15T22:37:58.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:58 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:37:58.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:58 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/4164650520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:37:58.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:58 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:58.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:58 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:58.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:58 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:58.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:58 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:58.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:58 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:58.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:58 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:37:58.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:58 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:58.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:58 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:37:58.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:58 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:37:58.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:58 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:37:58.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:58 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:37:58.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:58 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/2363244923' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04f39604-be0e-4261-b316-67fa087c6666"}]: dispatch 2024-09-15T22:37:58.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:58 smithi111 ceph-mon[30629]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04f39604-be0e-4261-b316-67fa087c6666"}]: dispatch 2024-09-15T22:37:58.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:58 smithi111 ceph-mon[30629]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "04f39604-be0e-4261-b316-67fa087c6666"}]': finished 2024-09-15T22:37:58.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:58 smithi111 ceph-mon[30629]: osdmap e5: 1 total, 0 up, 1 in 2024-09-15T22:37:58.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:58 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:37:58.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:58 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/4164650520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:37:59.315 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 22:37:58 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z[37236]: 2024-09-15T22:37:58.920+0000 7f1e2fecd040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T22:37:59.423 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 22:37:59 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-y[36178]: 2024-09-15T22:37:59.166+0000 7fe618c68040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T22:37:59.423 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 22:37:59 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-y[36178]: 2024-09-15T22:37:59.280+0000 7fe618c68040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T22:37:59.424 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 22:37:59 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-y[36178]: 2024-09-15T22:37:59.422+0000 7fe618c68040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:37:59.699 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 22:37:59 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-y[36178]: 2024-09-15T22:37:59.482+0000 7fe618c68040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T22:37:59.699 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 22:37:59 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-y[36178]: 2024-09-15T22:37:59.630+0000 7fe618c68040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T22:37:59.699 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 22:37:59 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-y[36178]: 2024-09-15T22:37:59.697+0000 7fe618c68040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T22:37:59.699 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:59 smithi121 ceph-mon[34815]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:59.699 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:59 smithi121 ceph-mon[34815]: Standby manager daemon z started 2024-09-15T22:37:59.699 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:59 smithi121 ceph-mon[34815]: from='mgr.? 172.21.15.192:0/690582534' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-15T22:37:59.700 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:59 smithi121 ceph-mon[34815]: from='mgr.? 172.21.15.192:0/690582534' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:37:59.700 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:59 smithi121 ceph-mon[34815]: from='mgr.? 172.21.15.192:0/690582534' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-15T22:37:59.700 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:37:59 smithi121 ceph-mon[34815]: from='mgr.? 172.21.15.192:0/690582534' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:37:59.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:37:59.796+0000 7f793b400640 1 -- 172.21.15.111:0/3391860575 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mgrmap(e 14) ==== 92409+0+0 (secure 0 0 0) 0x7f793400ad10 con 0x7f7944076ba0 2024-09-15T22:37:59.799 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:59 smithi111 ceph-mon[30629]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:59.799 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:59 smithi111 ceph-mon[30629]: Standby manager daemon z started 2024-09-15T22:37:59.800 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:59 smithi111 ceph-mon[30629]: from='mgr.? 172.21.15.192:0/690582534' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-15T22:37:59.800 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:59 smithi111 ceph-mon[30629]: from='mgr.? 172.21.15.192:0/690582534' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:37:59.800 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:59 smithi111 ceph-mon[30629]: from='mgr.? 172.21.15.192:0/690582534' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-15T22:37:59.800 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:37:59 smithi111 ceph-mon[30629]: from='mgr.? 172.21.15.192:0/690582534' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:37:59.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:59 smithi192 ceph-mon[35427]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:37:59.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:59 smithi192 ceph-mon[35427]: Standby manager daemon z started 2024-09-15T22:37:59.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:59 smithi192 ceph-mon[35427]: from='mgr.? 172.21.15.192:0/690582534' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-15T22:37:59.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:59 smithi192 ceph-mon[35427]: from='mgr.? 172.21.15.192:0/690582534' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:37:59.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:59 smithi192 ceph-mon[35427]: from='mgr.? 172.21.15.192:0/690582534' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-15T22:37:59.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:37:59 smithi192 ceph-mon[35427]: from='mgr.? 172.21.15.192:0/690582534' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:37:59.982 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 22:37:59 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-y[36178]: 2024-09-15T22:37:59.757+0000 7fe618c68040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T22:37:59.982 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 22:37:59 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-y[36178]: 2024-09-15T22:37:59.916+0000 7fe618c68040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T22:37:59.982 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 22:37:59 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-y[36178]: 2024-09-15T22:37:59.981+0000 7fe618c68040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T22:38:00.246 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 22:38:00 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-y[36178]: 2024-09-15T22:38:00.046+0000 7fe618c68040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T22:38:00.247 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 22:38:00 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-y[36178]: 2024-09-15T22:38:00.128+0000 7fe618c68040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T22:38:00.247 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 22:38:00 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-y[36178]: 2024-09-15T22:38:00.187+0000 7fe618c68040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T22:38:00.247 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 22:38:00 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-y[36178]: 2024-09-15T22:38:00.245+0000 7fe618c68040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T22:38:01.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:00 smithi192 ceph-mon[35427]: mgrmap e14: x(active, since 69s), standbys: z 2024-09-15T22:38:01.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:00 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-15T22:38:01.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:00 smithi192 ceph-mon[35427]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:01.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:00 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:01.140 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 22:38:01 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-y[36178]: 2024-09-15T22:38:01.034+0000 7fe618c68040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T22:38:01.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:00 smithi121 ceph-mon[34815]: mgrmap e14: x(active, since 69s), standbys: z 2024-09-15T22:38:01.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:00 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-15T22:38:01.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:00 smithi121 ceph-mon[34815]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:01.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:00 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:01.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:00 smithi111 ceph-mon[30629]: mgrmap e14: x(active, since 69s), standbys: z 2024-09-15T22:38:01.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:00 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-15T22:38:01.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:00 smithi111 ceph-mon[30629]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:01.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:00 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:01.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:38:01.814+0000 7f793b400640 1 -- 172.21.15.111:0/3391860575 <== mon.2 v2:172.21.15.121:3300/0 8 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f79340203f0 con 0x7f7944076ba0 2024-09-15T22:38:02.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:01 smithi192 ceph-mon[35427]: Standby manager daemon y started 2024-09-15T22:38:02.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:01 smithi192 ceph-mon[35427]: from='mgr.? 172.21.15.121:0/1202382308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-15T22:38:02.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:01 smithi192 ceph-mon[35427]: from='mgr.? 172.21.15.121:0/1202382308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:38:02.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:01 smithi192 ceph-mon[35427]: from='mgr.? 172.21.15.121:0/1202382308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-15T22:38:02.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:01 smithi192 ceph-mon[35427]: from='mgr.? 172.21.15.121:0/1202382308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:38:02.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:01 smithi121 ceph-mon[34815]: Standby manager daemon y started 2024-09-15T22:38:02.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:01 smithi121 ceph-mon[34815]: from='mgr.? 172.21.15.121:0/1202382308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-15T22:38:02.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:01 smithi121 ceph-mon[34815]: from='mgr.? 172.21.15.121:0/1202382308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:38:02.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:01 smithi121 ceph-mon[34815]: from='mgr.? 172.21.15.121:0/1202382308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-15T22:38:02.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:01 smithi121 ceph-mon[34815]: from='mgr.? 172.21.15.121:0/1202382308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:38:02.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:01 smithi111 ceph-mon[30629]: Standby manager daemon y started 2024-09-15T22:38:02.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:01 smithi111 ceph-mon[30629]: from='mgr.? 172.21.15.121:0/1202382308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-15T22:38:02.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:01 smithi111 ceph-mon[30629]: from='mgr.? 172.21.15.121:0/1202382308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:38:02.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:01 smithi111 ceph-mon[30629]: from='mgr.? 172.21.15.121:0/1202382308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-15T22:38:02.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:01 smithi111 ceph-mon[30629]: from='mgr.? 172.21.15.121:0/1202382308' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:38:03.114 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:02 smithi111 ceph-mon[30629]: mgrmap e15: x(active, since 71s), standbys: z, y 2024-09-15T22:38:03.114 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:02 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-15T22:38:03.114 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:02 smithi111 ceph-mon[30629]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:03.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:02 smithi121 ceph-mon[34815]: mgrmap e15: x(active, since 71s), standbys: z, y 2024-09-15T22:38:03.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:02 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-15T22:38:03.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:02 smithi121 ceph-mon[34815]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:03.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:02 smithi192 ceph-mon[35427]: mgrmap e15: x(active, since 71s), standbys: z, y 2024-09-15T22:38:03.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:02 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-15T22:38:03.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:02 smithi192 ceph-mon[35427]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:05.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:05 smithi111 ceph-mon[30629]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:05.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:05 smithi192 ceph-mon[35427]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:05.843 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:05 smithi121 ceph-mon[34815]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:07.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:07 smithi111 ceph-mon[30629]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:07.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:07 smithi192 ceph-mon[35427]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:07.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:07 smithi121 ceph-mon[34815]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:08.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:08 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-15T22:38:08.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:08 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:08.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:08 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-15T22:38:08.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:08 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:08.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:08 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-15T22:38:08.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:08 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:09.599 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:09 smithi111 ceph-mon[30629]: Deploying daemon osd.0 on smithi111 2024-09-15T22:38:09.600 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:09 smithi111 ceph-mon[30629]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:09.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:09 smithi192 ceph-mon[35427]: Deploying daemon osd.0 on smithi111 2024-09-15T22:38:09.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:09 smithi192 ceph-mon[35427]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:09.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:09 smithi121 ceph-mon[34815]: Deploying daemon osd.0 on smithi111 2024-09-15T22:38:09.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:09 smithi121 ceph-mon[34815]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:11.705 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:11 smithi111 ceph-mon[30629]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:11.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:11 smithi192 ceph-mon[35427]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:11.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:11 smithi121 ceph-mon[34815]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:13.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:13 smithi192 ceph-mon[35427]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:13.818 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:13 smithi111 ceph-mon[30629]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:13.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:13 smithi121 ceph-mon[34815]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:15.522 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:15 smithi111 ceph-mon[30629]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:15.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:15 smithi192 ceph-mon[35427]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:15.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:15 smithi121 ceph-mon[34815]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:16.743 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:16 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:16.743 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:16 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:16.743 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:16 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:16.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:16 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:16.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:16 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:16.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:16 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:16.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:16 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:16.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:16 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:16.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:16 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:17.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:17 smithi111 ceph-mon[30629]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:17.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:17 smithi192 ceph-mon[35427]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:17.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:17 smithi121 ceph-mon[34815]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:19.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:19 smithi192 ceph-mon[35427]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:19.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:19 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:19.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:19 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:19.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:19 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:19.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:19 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:19.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:19 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:19.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:19 smithi121 ceph-mon[34815]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:19.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:19 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:19.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:19 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:19.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:19 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:19.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:19 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:19.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:19 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:19.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:19 smithi111 ceph-mon[30629]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:19.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:19 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:19.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:19 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:19.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:19 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:19.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:19 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:19.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:19 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:21.045 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 0 on host 'smithi111' 2024-09-15T22:38:21.046 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:38:21.043+0000 7f793b400640 1 -- 172.21.15.111:0/3391860575 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7944062630 con 0x7f7928039500 2024-09-15T22:38:21.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:38:21.047+0000 7f7943e00640 1 -- 172.21.15.111:0/3391860575 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7928039500 msgr2=0x7f792803b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:21.049 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:38:21.047+0000 7f7943e00640 1 --2- 172.21.15.111:0/3391860575 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7928039500 0x7f792803b9c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f794419f7b0 tx=0x7f7930009260 comp rx=0 tx=0).stop 2024-09-15T22:38:21.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:38:21.047+0000 7f7943e00640 1 -- 172.21.15.111:0/3391860575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7944076ba0 msgr2=0x7f794419e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:21.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:38:21.047+0000 7f7943e00640 1 --2- 172.21.15.111:0/3391860575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7944076ba0 0x7f794419e290 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f793400e9f0 tx=0x7f793400eeb0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:38:21.047+0000 7f7943e00640 1 -- 172.21.15.111:0/3391860575 shutdown_connections 2024-09-15T22:38:21.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:38:21.047+0000 7f7943e00640 1 --2- 172.21.15.111:0/3391860575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7944078790 0x7f79441a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:38:21.047+0000 7f7943e00640 1 --2- 172.21.15.111:0/3391860575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7944077df0 0x7f794419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:38:21.047+0000 7f7943e00640 1 --2- 172.21.15.111:0/3391860575 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7928039500 0x7f792803b9c0 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f794419f7b0 tx=0x7f7930009260 comp rx=0 tx=0).stop 2024-09-15T22:38:21.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:38:21.047+0000 7f7943e00640 1 --2- 172.21.15.111:0/3391860575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7944076ba0 0x7f794419e290 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:21.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:38:21.047+0000 7f7943e00640 1 -- 172.21.15.111:0/3391860575 >> 172.21.15.111:0/3391860575 conn(0x7f79440ff3e0 msgr2=0x7f7944100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:21.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:38:21.048+0000 7f7943e00640 1 -- 172.21.15.111:0/3391860575 shutdown_connections 2024-09-15T22:38:21.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:38:21.048+0000 7f7943e00640 1 -- 172.21.15.111:0/3391860575 wait complete. 2024-09-15T22:38:21.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:21 smithi111 ceph-mon[30629]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:21.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:21 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:21.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:21 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:21.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:21 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:21.752 DEBUG:teuthology.orchestra.run.smithi111:osd.0> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.0.service 2024-09-15T22:38:21.756 INFO:tasks.cephadm:Deploying osd.1 on smithi121 with /dev/vg_nvme/lv_4... 2024-09-15T22:38:21.756 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-15T22:38:21.784 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:21 smithi121 ceph-mon[34815]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:21.784 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:21 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:21.785 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:21 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:21.785 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:21 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:21.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:21 smithi192 ceph-mon[35427]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:21.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:21 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:21.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:21 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:21.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:21 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:22.010 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:38:22.523 INFO:journalctl@ceph.osd.0.smithi111.stdout:Sep 15 22:38:22 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-0[44883]: 2024-09-15T22:38:22.398+0000 7fdd7be1b640 -1 osd.0 0 log_to_monitors true 2024-09-15T22:38:22.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:22 smithi192 ceph-mon[35427]: from='osd.0 [v2:172.21.15.111:6802/1386823446,v1:172.21.15.111:6803/1386823446]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-15T22:38:22.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:22 smithi121 ceph-mon[34815]: from='osd.0 [v2:172.21.15.111:6802/1386823446,v1:172.21.15.111:6803/1386823446]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-15T22:38:22.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:22 smithi111 ceph-mon[30629]: from='osd.0 [v2:172.21.15.111:6802/1386823446,v1:172.21.15.111:6803/1386823446]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-15T22:38:23.575 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:23 smithi111 ceph-mon[30629]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:23.575 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:23 smithi111 ceph-mon[30629]: from='osd.0 [v2:172.21.15.111:6802/1386823446,v1:172.21.15.111:6803/1386823446]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-15T22:38:23.575 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:23 smithi111 ceph-mon[30629]: osdmap e6: 1 total, 0 up, 1 in 2024-09-15T22:38:23.575 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:23 smithi111 ceph-mon[30629]: from='osd.0 [v2:172.21.15.111:6802/1386823446,v1:172.21.15.111:6803/1386823446]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-15T22:38:23.575 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:23 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:38:23.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:23 smithi121 ceph-mon[34815]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:23.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:23 smithi121 ceph-mon[34815]: from='osd.0 [v2:172.21.15.111:6802/1386823446,v1:172.21.15.111:6803/1386823446]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-15T22:38:23.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:23 smithi121 ceph-mon[34815]: osdmap e6: 1 total, 0 up, 1 in 2024-09-15T22:38:23.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:23 smithi121 ceph-mon[34815]: from='osd.0 [v2:172.21.15.111:6802/1386823446,v1:172.21.15.111:6803/1386823446]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-15T22:38:23.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:23 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:38:23.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:23 smithi192 ceph-mon[35427]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:23.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:23 smithi192 ceph-mon[35427]: from='osd.0 [v2:172.21.15.111:6802/1386823446,v1:172.21.15.111:6803/1386823446]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-15T22:38:23.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:23 smithi192 ceph-mon[35427]: osdmap e6: 1 total, 0 up, 1 in 2024-09-15T22:38:23.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:23 smithi192 ceph-mon[35427]: from='osd.0 [v2:172.21.15.111:6802/1386823446,v1:172.21.15.111:6803/1386823446]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-15T22:38:23.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:23 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:38:24.331 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-15T22:38:24.359 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi121:vg_nvme/lv_4 2024-09-15T22:38:24.651 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:38:24.772 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:24 smithi121 ceph-mon[34815]: from='osd.0 [v2:172.21.15.111:6802/1386823446,v1:172.21.15.111:6803/1386823446]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-15T22:38:24.772 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:24 smithi121 ceph-mon[34815]: osdmap e7: 1 total, 0 up, 1 in 2024-09-15T22:38:24.772 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:24 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:38:24.772 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:24 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:38:24.772 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:24 smithi121 ceph-mon[34815]: from='osd.0 [v2:172.21.15.111:6802/1386823446,v1:172.21.15.111:6803/1386823446]' entity='osd.0' 2024-09-15T22:38:24.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:24 smithi192 ceph-mon[35427]: from='osd.0 [v2:172.21.15.111:6802/1386823446,v1:172.21.15.111:6803/1386823446]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-15T22:38:24.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:24 smithi192 ceph-mon[35427]: osdmap e7: 1 total, 0 up, 1 in 2024-09-15T22:38:24.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:24 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:38:24.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:24 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:38:24.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:24 smithi192 ceph-mon[35427]: from='osd.0 [v2:172.21.15.111:6802/1386823446,v1:172.21.15.111:6803/1386823446]' entity='osd.0' 2024-09-15T22:38:24.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:24 smithi111 ceph-mon[30629]: from='osd.0 [v2:172.21.15.111:6802/1386823446,v1:172.21.15.111:6803/1386823446]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-15T22:38:24.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:24 smithi111 ceph-mon[30629]: osdmap e7: 1 total, 0 up, 1 in 2024-09-15T22:38:24.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:24 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:38:24.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:24 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:38:24.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:24 smithi111 ceph-mon[30629]: from='osd.0 [v2:172.21.15.111:6802/1386823446,v1:172.21.15.111:6803/1386823446]' entity='osd.0' 2024-09-15T22:38:24.916 INFO:journalctl@ceph.osd.0.smithi111.stdout:Sep 15 22:38:24 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-0[44883]: 2024-09-15T22:38:24.525+0000 7fdd77400640 -1 osd.0 0 waiting for initial osdmap 2024-09-15T22:38:24.916 INFO:journalctl@ceph.osd.0.smithi111.stdout:Sep 15 22:38:24 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-0[44883]: 2024-09-15T22:38:24.542+0000 7fdd6ca00640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:38:25.125 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.124+0000 7f3b0839b640 1 -- 172.21.15.121:0/3739988623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b00102690 msgr2=0x7f3b00108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:25.126 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.124+0000 7f3b0839b640 1 --2- 172.21.15.121:0/3739988623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b00102690 0x7f3b00108f20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3af0009900 tx=0x7f3af002a410 comp rx=0 tx=0).stop 2024-09-15T22:38:25.126 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.124+0000 7f3b0839b640 1 -- 172.21.15.121:0/3739988623 shutdown_connections 2024-09-15T22:38:25.126 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.124+0000 7f3b0839b640 1 --2- 172.21.15.121:0/3739988623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b00102690 0x7f3b00108f20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.126 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.124+0000 7f3b0839b640 1 --2- 172.21.15.121:0/3739988623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b00101cd0 0x7f3b00102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.126 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.124+0000 7f3b0839b640 1 --2- 172.21.15.121:0/3739988623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b00100ad0 0x7f3b00100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.126 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.124+0000 7f3b0839b640 1 -- 172.21.15.121:0/3739988623 >> 172.21.15.121:0/3739988623 conn(0x7f3b000fc280 msgr2=0x7f3b000fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:25.126 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.124+0000 7f3b0839b640 1 -- 172.21.15.121:0/3739988623 shutdown_connections 2024-09-15T22:38:25.126 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.124+0000 7f3b0839b640 1 -- 172.21.15.121:0/3739988623 wait complete. 2024-09-15T22:38:25.128 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.125+0000 7f3b0839b640 1 Processor -- start 2024-09-15T22:38:25.128 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.125+0000 7f3b0839b640 1 -- start start 2024-09-15T22:38:25.128 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.125+0000 7f3b0839b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b00100ad0 0x7f3b0006aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:25.128 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.125+0000 7f3b0839b640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b00101cd0 0x7f3b0019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:25.128 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.125+0000 7f3b0839b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b00102690 0x7f3b0019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:25.128 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.125+0000 7f3b0839b640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0010c750 con 0x7f3b00101cd0 2024-09-15T22:38:25.128 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.125+0000 7f3b0839b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0010c5d0 con 0x7f3b00100ad0 2024-09-15T22:38:25.128 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.125+0000 7f3b0839b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0010c8d0 con 0x7f3b00102690 2024-09-15T22:38:25.128 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.125+0000 7f3b06137640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b00100ad0 0x7f3b0006aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:25.129 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.125+0000 7f3b06137640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b00100ad0 0x7f3b0006aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39152/0 (socket says 172.21.15.121:39152) 2024-09-15T22:38:25.129 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.125+0000 7f3b06137640 1 -- 172.21.15.121:0/426138929 learned_addr learned my addr 172.21.15.121:0/426138929 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-09-15T22:38:25.129 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.125+0000 7f3b05936640 1 --2- 172.21.15.121:0/426138929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b00101cd0 0x7f3b0019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:25.129 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.125+0000 7f3b06938640 1 --2- 172.21.15.121:0/426138929 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b00102690 0x7f3b0019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:25.129 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.125+0000 7f3b06137640 1 -- 172.21.15.121:0/426138929 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b00102690 msgr2=0x7f3b0019f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:25.129 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.125+0000 7f3b06137640 1 --2- 172.21.15.121:0/426138929 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b00102690 0x7f3b0019f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.129 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.125+0000 7f3b06137640 1 -- 172.21.15.121:0/426138929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b00101cd0 msgr2=0x7f3b0019cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:25.129 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.125+0000 7f3b06137640 1 --2- 172.21.15.121:0/426138929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b00101cd0 0x7f3b0019cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:25.129 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.125+0000 7f3b06137640 1 -- 172.21.15.121:0/426138929 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0019faa0 con 0x7f3b00100ad0 2024-09-15T22:38:25.129 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.125+0000 7f3b06938640 1 --2- 172.21.15.121:0/426138929 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b00102690 0x7f3b0019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:38:25.129 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.126+0000 7f3b06137640 1 --2- 172.21.15.121:0/426138929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b00100ad0 0x7f3b0006aad0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3af800c970 tx=0x7f3af800ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:25.129 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.126+0000 7f3aff7fe640 1 -- 172.21.15.121:0/426138929 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8007c10 con 0x7f3b00100ad0 2024-09-15T22:38:25.130 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.126+0000 7f3aff7fe640 1 -- 172.21.15.121:0/426138929 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3af8007db0 con 0x7f3b00100ad0 2024-09-15T22:38:25.130 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.126+0000 7f3aff7fe640 1 -- 172.21.15.121:0/426138929 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af800f940 con 0x7f3b00100ad0 2024-09-15T22:38:25.130 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.126+0000 7f3b0839b640 1 -- 172.21.15.121:0/426138929 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b0019fd30 con 0x7f3b00100ad0 2024-09-15T22:38:25.130 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.126+0000 7f3b0839b640 1 -- 172.21.15.121:0/426138929 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b001a01c0 con 0x7f3b00100ad0 2024-09-15T22:38:25.131 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.127+0000 7f3b0839b640 1 -- 172.21.15.121:0/426138929 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b00107e80 con 0x7f3b00100ad0 2024-09-15T22:38:25.131 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.128+0000 7f3aff7fe640 1 -- 172.21.15.121:0/426138929 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f3af800fae0 con 0x7f3b00100ad0 2024-09-15T22:38:25.131 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.128+0000 7f3aff7fe640 1 --2- 172.21.15.121:0/426138929 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3ae80a31b0 0x7f3ae80a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:25.131 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.129+0000 7f3aff7fe640 1 -- 172.21.15.121:0/426138929 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f3af80ce130 con 0x7f3b00100ad0 2024-09-15T22:38:25.131 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.129+0000 7f3b05936640 1 --2- 172.21.15.121:0/426138929 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3ae80a31b0 0x7f3ae80a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:25.131 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.129+0000 7f3b05936640 1 --2- 172.21.15.121:0/426138929 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3ae80a31b0 0x7f3ae80a5670 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3b0006b050 tx=0x7f3af4009680 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:25.134 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.133+0000 7f3aff7fe640 1 -- 172.21.15.121:0/426138929 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3af809ca70 con 0x7f3b00100ad0 2024-09-15T22:38:25.330 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:25.329+0000 7f3b0839b640 1 -- 172.21.15.121:0/426138929 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f3b00062630 con 0x7f3ae80a31b0 2024-09-15T22:38:25.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:25 smithi121 ceph-mon[34815]: purged_snaps scrub starts 2024-09-15T22:38:25.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:25 smithi121 ceph-mon[34815]: purged_snaps scrub ok 2024-09-15T22:38:25.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:25 smithi121 ceph-mon[34815]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:25.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:25 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:38:25.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:25 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:38:25.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:25 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:38:25.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:25 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:25.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:25 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:25.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:25 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:25.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:25 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:25.642 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:25 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:25.642 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:25 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:25.642 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:25 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:25.642 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:25 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:25.642 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:25 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:38:25.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:25 smithi192 ceph-mon[35427]: purged_snaps scrub starts 2024-09-15T22:38:25.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:25 smithi192 ceph-mon[35427]: purged_snaps scrub ok 2024-09-15T22:38:25.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:25 smithi192 ceph-mon[35427]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:25.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:25 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:38:25.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:25 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:38:25.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:25 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:38:25.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:25 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:25.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:25 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:25.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:25 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:25.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:25 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:25.817 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:25 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:25.817 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:25 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:25.817 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:25 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:25.817 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:25 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:25.817 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:25 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:38:25.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:25 smithi111 ceph-mon[30629]: purged_snaps scrub starts 2024-09-15T22:38:25.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:25 smithi111 ceph-mon[30629]: purged_snaps scrub ok 2024-09-15T22:38:25.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:25 smithi111 ceph-mon[30629]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:25.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:25 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:38:25.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:25 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:38:25.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:25 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:38:25.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:25 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:25.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:25 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:25.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:25 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:25.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:25 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:25.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:25 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:25.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:25 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:25.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:25 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:25.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:25 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:25.918 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:25 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:38:26.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:26 smithi121 ceph-mon[34815]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:26.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:26 smithi121 ceph-mon[34815]: Detected new or changed devices on smithi111 2024-09-15T22:38:26.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:26 smithi121 ceph-mon[34815]: Adjusting osd_memory_target on smithi111 to 17089M 2024-09-15T22:38:26.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:26 smithi121 ceph-mon[34815]: osd.0 [v2:172.21.15.111:6802/1386823446,v1:172.21.15.111:6803/1386823446] boot 2024-09-15T22:38:26.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:26 smithi121 ceph-mon[34815]: osdmap e8: 1 total, 1 up, 1 in 2024-09-15T22:38:26.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:26 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:38:26.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:26 smithi111 ceph-mon[30629]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:26.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:26 smithi111 ceph-mon[30629]: Detected new or changed devices on smithi111 2024-09-15T22:38:26.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:26 smithi111 ceph-mon[30629]: Adjusting osd_memory_target on smithi111 to 17089M 2024-09-15T22:38:26.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:26 smithi111 ceph-mon[30629]: osd.0 [v2:172.21.15.111:6802/1386823446,v1:172.21.15.111:6803/1386823446] boot 2024-09-15T22:38:26.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:26 smithi111 ceph-mon[30629]: osdmap e8: 1 total, 1 up, 1 in 2024-09-15T22:38:26.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:26 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:38:27.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:26 smithi192 ceph-mon[35427]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:27.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:26 smithi192 ceph-mon[35427]: Detected new or changed devices on smithi111 2024-09-15T22:38:27.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:26 smithi192 ceph-mon[35427]: Adjusting osd_memory_target on smithi111 to 17089M 2024-09-15T22:38:27.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:26 smithi192 ceph-mon[35427]: osd.0 [v2:172.21.15.111:6802/1386823446,v1:172.21.15.111:6803/1386823446] boot 2024-09-15T22:38:27.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:26 smithi192 ceph-mon[35427]: osdmap e8: 1 total, 1 up, 1 in 2024-09-15T22:38:27.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:26 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:38:27.852 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:27 smithi121 ceph-mon[34815]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:27.853 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:27 smithi121 ceph-mon[34815]: osdmap e9: 1 total, 1 up, 1 in 2024-09-15T22:38:27.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:27 smithi111 ceph-mon[30629]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:27.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:27 smithi111 ceph-mon[30629]: osdmap e9: 1 total, 1 up, 1 in 2024-09-15T22:38:28.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:27 smithi192 ceph-mon[35427]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:38:28.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:27 smithi192 ceph-mon[35427]: osdmap e9: 1 total, 1 up, 1 in 2024-09-15T22:38:28.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:28 smithi121 ceph-mon[34815]: from='client.? 172.21.15.121:0/4166908312' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1bdbef6e-23fa-4618-a788-66e57741a2a8"}]: dispatch 2024-09-15T22:38:28.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:28 smithi121 ceph-mon[34815]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1bdbef6e-23fa-4618-a788-66e57741a2a8"}]: dispatch 2024-09-15T22:38:28.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:28 smithi121 ceph-mon[34815]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1bdbef6e-23fa-4618-a788-66e57741a2a8"}]': finished 2024-09-15T22:38:28.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:28 smithi121 ceph-mon[34815]: osdmap e10: 2 total, 1 up, 2 in 2024-09-15T22:38:28.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:28 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:38:28.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:28 smithi121 ceph-mon[34815]: from='client.? 172.21.15.121:0/33466861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:38:28.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:28 smithi111 ceph-mon[30629]: from='client.? 172.21.15.121:0/4166908312' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1bdbef6e-23fa-4618-a788-66e57741a2a8"}]: dispatch 2024-09-15T22:38:28.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:28 smithi111 ceph-mon[30629]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1bdbef6e-23fa-4618-a788-66e57741a2a8"}]: dispatch 2024-09-15T22:38:28.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:28 smithi111 ceph-mon[30629]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1bdbef6e-23fa-4618-a788-66e57741a2a8"}]': finished 2024-09-15T22:38:28.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:28 smithi111 ceph-mon[30629]: osdmap e10: 2 total, 1 up, 2 in 2024-09-15T22:38:28.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:28 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:38:28.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:28 smithi111 ceph-mon[30629]: from='client.? 172.21.15.121:0/33466861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:38:29.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:28 smithi192 ceph-mon[35427]: from='client.? 172.21.15.121:0/4166908312' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1bdbef6e-23fa-4618-a788-66e57741a2a8"}]: dispatch 2024-09-15T22:38:29.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:28 smithi192 ceph-mon[35427]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1bdbef6e-23fa-4618-a788-66e57741a2a8"}]: dispatch 2024-09-15T22:38:29.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:28 smithi192 ceph-mon[35427]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1bdbef6e-23fa-4618-a788-66e57741a2a8"}]': finished 2024-09-15T22:38:29.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:28 smithi192 ceph-mon[35427]: osdmap e10: 2 total, 1 up, 2 in 2024-09-15T22:38:29.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:28 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:38:29.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:28 smithi192 ceph-mon[35427]: from='client.? 172.21.15.121:0/33466861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:38:29.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:29 smithi121 ceph-mon[34815]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:29.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:29 smithi111 ceph-mon[30629]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:30.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:29 smithi192 ceph-mon[35427]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:31.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:31 smithi192 ceph-mon[35427]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:31.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:31 smithi121 ceph-mon[34815]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:31.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:31 smithi111 ceph-mon[30629]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:33.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:33 smithi121 ceph-mon[34815]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:33.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:33 smithi111 ceph-mon[30629]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:34.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:33 smithi192 ceph-mon[35427]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:35.704 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:35 smithi121 ceph-mon[34815]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:35.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:35 smithi111 ceph-mon[30629]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:36.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:35 smithi192 ceph-mon[35427]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:36.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:36 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-15T22:38:36.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:36 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:36.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:36 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-15T22:38:36.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:36 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:37.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:36 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-15T22:38:37.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:36 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:37.855 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:37 smithi121 ceph-mon[34815]: Deploying daemon osd.1 on smithi121 2024-09-15T22:38:37.855 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:37 smithi121 ceph-mon[34815]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:37.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:37 smithi111 ceph-mon[30629]: Deploying daemon osd.1 on smithi121 2024-09-15T22:38:37.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:37 smithi111 ceph-mon[30629]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:38.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:37 smithi192 ceph-mon[35427]: Deploying daemon osd.1 on smithi121 2024-09-15T22:38:38.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:37 smithi192 ceph-mon[35427]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:39.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:39 smithi121 ceph-mon[34815]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:39.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:39 smithi111 ceph-mon[30629]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:40.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:39 smithi192 ceph-mon[35427]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:41.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:41 smithi121 ceph-mon[34815]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:41.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:41 smithi111 ceph-mon[30629]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:42.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:41 smithi192 ceph-mon[35427]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:42.865 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:42 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:42.865 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:42 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:42.865 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:42 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:42.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:42 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:42.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:42 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:42.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:42 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:43.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:42 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:43.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:42 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:43.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:42 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:43.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:43 smithi121 ceph-mon[34815]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:43.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:43 smithi111 ceph-mon[30629]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:44.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:43 smithi192 ceph-mon[35427]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:45.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:45 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:45.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:45 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:45.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:45 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:45.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:45 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:45.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:45 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:45.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:45 smithi192 ceph-mon[35427]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:45.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:45 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:45.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:45 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:45.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:45 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:45.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:45 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:45.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:45 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:45.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:45 smithi121 ceph-mon[34815]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:45.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:45 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:45.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:45 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:45.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:45 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:45.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:45 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:45.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:45 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:45.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:45 smithi111 ceph-mon[30629]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:46.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:46.376+0000 7f3aff7fe640 1 -- 172.21.15.121:0/426138929 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3b00062630 con 0x7f3ae80a31b0 2024-09-15T22:38:46.379 INFO:teuthology.orchestra.run.smithi121.stdout:Created osd(s) 1 on host 'smithi121' 2024-09-15T22:38:46.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:46.379+0000 7f3b0839b640 1 -- 172.21.15.121:0/426138929 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3ae80a31b0 msgr2=0x7f3ae80a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:46.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:46.379+0000 7f3b0839b640 1 --2- 172.21.15.121:0/426138929 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3ae80a31b0 0x7f3ae80a5670 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3b0006b050 tx=0x7f3af4009680 comp rx=0 tx=0).stop 2024-09-15T22:38:46.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:46.379+0000 7f3b0839b640 1 -- 172.21.15.121:0/426138929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b00100ad0 msgr2=0x7f3b0006aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:46.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:46.379+0000 7f3b0839b640 1 --2- 172.21.15.121:0/426138929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b00100ad0 0x7f3b0006aad0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3af800c970 tx=0x7f3af800ce30 comp rx=0 tx=0).stop 2024-09-15T22:38:46.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:46.379+0000 7f3b0839b640 1 -- 172.21.15.121:0/426138929 shutdown_connections 2024-09-15T22:38:46.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:46.379+0000 7f3b0839b640 1 --2- 172.21.15.121:0/426138929 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b00102690 0x7f3b0019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:46.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:46.379+0000 7f3b0839b640 1 --2- 172.21.15.121:0/426138929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b00101cd0 0x7f3b0019cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:46.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:46.379+0000 7f3b0839b640 1 --2- 172.21.15.121:0/426138929 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3ae80a31b0 0x7f3ae80a5670 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:46.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:46.379+0000 7f3b0839b640 1 --2- 172.21.15.121:0/426138929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b00100ad0 0x7f3b0006aad0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:46.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:46.379+0000 7f3b0839b640 1 -- 172.21.15.121:0/426138929 >> 172.21.15.121:0/426138929 conn(0x7f3b000fc280 msgr2=0x7f3b00105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:46.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:46.379+0000 7f3b0839b640 1 -- 172.21.15.121:0/426138929 shutdown_connections 2024-09-15T22:38:46.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:38:46.379+0000 7f3b0839b640 1 -- 172.21.15.121:0/426138929 wait complete. 2024-09-15T22:38:46.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:46.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:46.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:46.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:46.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:46.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:46.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:38:46.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:46.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:47.009 DEBUG:teuthology.orchestra.run.smithi121:osd.1> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.1.service 2024-09-15T22:38:47.011 INFO:tasks.cephadm:Deploying osd.2 on smithi192 with /dev/vg_nvme/lv_4... 2024-09-15T22:38:47.011 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-15T22:38:47.260 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:38:47.578 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:47 smithi121 ceph-mon[34815]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:47.642 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:47 smithi192 ceph-mon[35427]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:47.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:47 smithi111 ceph-mon[30629]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:48.640 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 15 22:38:48 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-1[39310]: 2024-09-15T22:38:48.511+0000 7fa96177d640 -1 osd.1 0 log_to_monitors true 2024-09-15T22:38:49.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:49 smithi121 ceph-mon[34815]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:49.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:49 smithi121 ceph-mon[34815]: from='osd.1 [v2:172.21.15.121:6800/2051512291,v1:172.21.15.121:6801/2051512291]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T22:38:49.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:49 smithi121 ceph-mon[34815]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T22:38:49.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:49 smithi192 ceph-mon[35427]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:49.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:49 smithi192 ceph-mon[35427]: from='osd.1 [v2:172.21.15.121:6800/2051512291,v1:172.21.15.121:6801/2051512291]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T22:38:49.801 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:49 smithi192 ceph-mon[35427]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T22:38:49.803 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T22:38:49.829 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi192:vg_nvme/lv_4 2024-09-15T22:38:49.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:49 smithi111 ceph-mon[30629]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:49.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:49 smithi111 ceph-mon[30629]: from='osd.1 [v2:172.21.15.121:6800/2051512291,v1:172.21.15.121:6801/2051512291]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T22:38:49.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:49 smithi111 ceph-mon[30629]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T22:38:50.095 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:38:50.487 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:50 smithi192 ceph-mon[35427]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-15T22:38:50.487 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:50 smithi192 ceph-mon[35427]: from='osd.1 [v2:172.21.15.121:6800/2051512291,v1:172.21.15.121:6801/2051512291]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-15T22:38:50.488 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:50 smithi192 ceph-mon[35427]: osdmap e11: 2 total, 1 up, 2 in 2024-09-15T22:38:50.488 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:38:50.488 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:50 smithi192 ceph-mon[35427]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-15T22:38:50.488 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:50.488 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:50.488 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:50.488 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:50.488 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:50.488 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:50.488 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:50.572 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.568+0000 7fcdbd89a640 1 -- 172.21.15.192:0/1870155308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdb8102690 msgr2=0x7fcdb8108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:50.572 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.568+0000 7fcdbd89a640 1 --2- 172.21.15.192:0/1870155308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdb8102690 0x7fcdb8108f20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009ca0 tx=0x7fcdb002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.572 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.568+0000 7fcdbd89a640 1 -- 172.21.15.192:0/1870155308 shutdown_connections 2024-09-15T22:38:50.572 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.568+0000 7fcdbd89a640 1 --2- 172.21.15.192:0/1870155308 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdb8102690 0x7fcdb8108f20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.572 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.568+0000 7fcdbd89a640 1 --2- 172.21.15.192:0/1870155308 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdb8101cd0 0x7fcdb8102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.572 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.568+0000 7fcdbd89a640 1 --2- 172.21.15.192:0/1870155308 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcdb8100ad0 0x7fcdb8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.572 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.568+0000 7fcdbd89a640 1 -- 172.21.15.192:0/1870155308 >> 172.21.15.192:0/1870155308 conn(0x7fcdb80fc280 msgr2=0x7fcdb80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:38:50.572 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.568+0000 7fcdbd89a640 1 -- 172.21.15.192:0/1870155308 shutdown_connections 2024-09-15T22:38:50.573 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.568+0000 7fcdbd89a640 1 -- 172.21.15.192:0/1870155308 wait complete. 2024-09-15T22:38:50.573 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.569+0000 7fcdbd89a640 1 Processor -- start 2024-09-15T22:38:50.573 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.569+0000 7fcdbd89a640 1 -- start start 2024-09-15T22:38:50.573 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.569+0000 7fcdbd89a640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdb8100ad0 0x7fcdb819a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:50.573 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.569+0000 7fcdbd89a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcdb8101cd0 0x7fcdb819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:50.573 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.569+0000 7fcdbd89a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdb8102690 0x7fcdb81a1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:50.573 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.569+0000 7fcdbd89a640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb810c850 con 0x7fcdb8100ad0 2024-09-15T22:38:50.573 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.569+0000 7fcdbd89a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb810c6d0 con 0x7fcdb8101cd0 2024-09-15T22:38:50.573 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.569+0000 7fcdbd89a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb810c9d0 con 0x7fcdb8102690 2024-09-15T22:38:50.573 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.569+0000 7fcdb77fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdb8102690 0x7fcdb81a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:50.573 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.570+0000 7fcdb77fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdb8102690 0x7fcdb81a1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:40098/0 (socket says 172.21.15.192:40098) 2024-09-15T22:38:50.573 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.570+0000 7fcdb77fe640 1 -- 172.21.15.192:0/1025674905 learned_addr learned my addr 172.21.15.192:0/1025674905 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-09-15T22:38:50.574 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.570+0000 7fcdae7fc640 1 --2- 172.21.15.192:0/1025674905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcdb8101cd0 0x7fcdb819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:50.574 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.570+0000 7fcdb77fe640 1 -- 172.21.15.192:0/1025674905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcdb8101cd0 msgr2=0x7fcdb819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:50.574 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.570+0000 7fcdb77fe640 1 --2- 172.21.15.192:0/1025674905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcdb8101cd0 0x7fcdb819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.574 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.570+0000 7fcdb77fe640 1 -- 172.21.15.192:0/1025674905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdb8100ad0 msgr2=0x7fcdb819a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:38:50.574 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.570+0000 7fcdb77fe640 1 --2- 172.21.15.192:0/1025674905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdb8100ad0 0x7fcdb819a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:38:50.574 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.570+0000 7fcdb77fe640 1 -- 172.21.15.192:0/1025674905 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb81a1e90 con 0x7fcdb8102690 2024-09-15T22:38:50.574 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.570+0000 7fcdb77fe640 1 --2- 172.21.15.192:0/1025674905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdb8102690 0x7fcdb81a1820 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00098d0 tx=0x7fcdb002f840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:50.574 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.570+0000 7fcdbc898640 1 -- 172.21.15.192:0/1025674905 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb002fc30 con 0x7fcdb8102690 2024-09-15T22:38:50.574 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.570+0000 7fcdbd89a640 1 -- 172.21.15.192:0/1025674905 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdb81a2120 con 0x7fcdb8102690 2024-09-15T22:38:50.574 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.570+0000 7fcdbc898640 1 -- 172.21.15.192:0/1025674905 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdb0003e50 con 0x7fcdb8102690 2024-09-15T22:38:50.574 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.570+0000 7fcdbc898640 1 -- 172.21.15.192:0/1025674905 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb00305c0 con 0x7fcdb8102690 2024-09-15T22:38:50.574 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.570+0000 7fcdbd89a640 1 -- 172.21.15.192:0/1025674905 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdb81a26e0 con 0x7fcdb8102690 2024-09-15T22:38:50.575 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.571+0000 7fcdbd89a640 1 -- 172.21.15.192:0/1025674905 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdb81a2a00 con 0x7fcdb8102690 2024-09-15T22:38:50.575 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.572+0000 7fcdbc898640 1 -- 172.21.15.192:0/1025674905 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7fcdb0003970 con 0x7fcdb8102690 2024-09-15T22:38:50.575 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.573+0000 7fcdbc898640 1 --2- 172.21.15.192:0/1025674905 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fcda40a31e0 0x7fcda40a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:38:50.576 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.574+0000 7fcdbc898640 1 -- 172.21.15.192:0/1025674905 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7fcdb00ef3f0 con 0x7fcdb8102690 2024-09-15T22:38:50.576 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.574+0000 7fcdb6ffd640 1 --2- 172.21.15.192:0/1025674905 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fcda40a31e0 0x7fcda40a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:38:50.576 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.575+0000 7fcdb6ffd640 1 --2- 172.21.15.192:0/1025674905 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fcda40a31e0 0x7fcda40a56a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8101b30 tx=0x7fcda8006c90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:38:50.579 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.578+0000 7fcdbc898640 1 -- 172.21.15.192:0/1025674905 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcdb00bdba0 con 0x7fcdb8102690 2024-09-15T22:38:50.753 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:38:50.752+0000 7fcdbd89a640 1 -- 172.21.15.192:0/1025674905 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fcdb81a2bd0 con 0x7fcda40a31e0 2024-09-15T22:38:50.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:50 smithi121 ceph-mon[34815]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-15T22:38:50.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:50 smithi121 ceph-mon[34815]: from='osd.1 [v2:172.21.15.121:6800/2051512291,v1:172.21.15.121:6801/2051512291]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-15T22:38:50.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:50 smithi121 ceph-mon[34815]: osdmap e11: 2 total, 1 up, 2 in 2024-09-15T22:38:50.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:38:50.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:50 smithi121 ceph-mon[34815]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-15T22:38:50.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:50.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:50.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:50.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:50.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:50.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:50.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:50.892 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 15 22:38:50 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-1[39310]: 2024-09-15T22:38:50.442+0000 7fa95cedd640 -1 osd.1 0 waiting for initial osdmap 2024-09-15T22:38:50.892 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 15 22:38:50 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-1[39310]: 2024-09-15T22:38:50.449+0000 7fa959517640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:38:50.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:50 smithi111 ceph-mon[30629]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-15T22:38:50.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:50 smithi111 ceph-mon[30629]: from='osd.1 [v2:172.21.15.121:6800/2051512291,v1:172.21.15.121:6801/2051512291]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-15T22:38:50.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:50 smithi111 ceph-mon[30629]: osdmap e11: 2 total, 1 up, 2 in 2024-09-15T22:38:50.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:38:50.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:50 smithi111 ceph-mon[30629]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-15T22:38:50.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:50.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:50.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:38:50.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:50.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:50.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:38:50.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:38:51.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:51 smithi192 ceph-mon[35427]: Detected new or changed devices on smithi121 2024-09-15T22:38:51.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:51 smithi192 ceph-mon[35427]: Adjusting osd_memory_target on smithi121 to 17087M 2024-09-15T22:38:51.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:51 smithi192 ceph-mon[35427]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:51.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:51 smithi192 ceph-mon[35427]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-15T22:38:51.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:51 smithi192 ceph-mon[35427]: osdmap e12: 2 total, 1 up, 2 in 2024-09-15T22:38:51.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:51 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:38:51.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:51 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:38:51.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:51 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:38:51.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:51 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:38:51.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:51 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:51.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:51 smithi121 ceph-mon[34815]: Detected new or changed devices on smithi121 2024-09-15T22:38:51.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:51 smithi121 ceph-mon[34815]: Adjusting osd_memory_target on smithi121 to 17087M 2024-09-15T22:38:51.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:51 smithi121 ceph-mon[34815]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:51.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:51 smithi121 ceph-mon[34815]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-15T22:38:51.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:51 smithi121 ceph-mon[34815]: osdmap e12: 2 total, 1 up, 2 in 2024-09-15T22:38:51.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:51 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:38:51.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:51 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:38:51.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:51 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:38:51.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:51 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:38:51.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:51 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:51.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:51 smithi111 ceph-mon[30629]: Detected new or changed devices on smithi121 2024-09-15T22:38:51.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:51 smithi111 ceph-mon[30629]: Adjusting osd_memory_target on smithi121 to 17087M 2024-09-15T22:38:51.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:51 smithi111 ceph-mon[30629]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:51.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:51 smithi111 ceph-mon[30629]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-15T22:38:51.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:51 smithi111 ceph-mon[30629]: osdmap e12: 2 total, 1 up, 2 in 2024-09-15T22:38:51.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:51 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:38:51.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:51 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:38:51.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:51 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:38:51.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:51 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:38:51.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:51 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:38:52.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:52 smithi192 ceph-mon[35427]: purged_snaps scrub starts 2024-09-15T22:38:52.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:52 smithi192 ceph-mon[35427]: purged_snaps scrub ok 2024-09-15T22:38:52.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:52 smithi192 ceph-mon[35427]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:52.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:52 smithi192 ceph-mon[35427]: osd.1 [v2:172.21.15.121:6800/2051512291,v1:172.21.15.121:6801/2051512291] boot 2024-09-15T22:38:52.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:52 smithi192 ceph-mon[35427]: osdmap e13: 2 total, 2 up, 2 in 2024-09-15T22:38:52.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:52 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:38:52.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:52 smithi121 ceph-mon[34815]: purged_snaps scrub starts 2024-09-15T22:38:52.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:52 smithi121 ceph-mon[34815]: purged_snaps scrub ok 2024-09-15T22:38:52.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:52 smithi121 ceph-mon[34815]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:52.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:52 smithi121 ceph-mon[34815]: osd.1 [v2:172.21.15.121:6800/2051512291,v1:172.21.15.121:6801/2051512291] boot 2024-09-15T22:38:52.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:52 smithi121 ceph-mon[34815]: osdmap e13: 2 total, 2 up, 2 in 2024-09-15T22:38:52.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:52 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:38:52.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:52 smithi111 ceph-mon[30629]: purged_snaps scrub starts 2024-09-15T22:38:52.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:52 smithi111 ceph-mon[30629]: purged_snaps scrub ok 2024-09-15T22:38:52.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:52 smithi111 ceph-mon[30629]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:38:52.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:52 smithi111 ceph-mon[30629]: osd.1 [v2:172.21.15.121:6800/2051512291,v1:172.21.15.121:6801/2051512291] boot 2024-09-15T22:38:52.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:52 smithi111 ceph-mon[30629]: osdmap e13: 2 total, 2 up, 2 in 2024-09-15T22:38:52.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:52 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:38:53.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:53 smithi192 ceph-mon[35427]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:53.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:53 smithi192 ceph-mon[35427]: osdmap e14: 2 total, 2 up, 2 in 2024-09-15T22:38:53.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:53 smithi192 ceph-mon[35427]: from='client.? 172.21.15.192:0/4181028309' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46735ccf-6d0e-4154-bb55-422929819bfa"}]: dispatch 2024-09-15T22:38:53.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:53 smithi192 ceph-mon[35427]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46735ccf-6d0e-4154-bb55-422929819bfa"}]: dispatch 2024-09-15T22:38:53.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:53 smithi192 ceph-mon[35427]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46735ccf-6d0e-4154-bb55-422929819bfa"}]': finished 2024-09-15T22:38:53.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:53 smithi192 ceph-mon[35427]: osdmap e15: 3 total, 2 up, 3 in 2024-09-15T22:38:53.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:53 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:38:53.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:53 smithi192 ceph-mon[35427]: from='client.? 172.21.15.192:0/3626459787' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:38:53.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:53 smithi121 ceph-mon[34815]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:53.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:53 smithi121 ceph-mon[34815]: osdmap e14: 2 total, 2 up, 2 in 2024-09-15T22:38:53.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:53 smithi121 ceph-mon[34815]: from='client.? 172.21.15.192:0/4181028309' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46735ccf-6d0e-4154-bb55-422929819bfa"}]: dispatch 2024-09-15T22:38:53.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:53 smithi121 ceph-mon[34815]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46735ccf-6d0e-4154-bb55-422929819bfa"}]: dispatch 2024-09-15T22:38:53.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:53 smithi121 ceph-mon[34815]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46735ccf-6d0e-4154-bb55-422929819bfa"}]': finished 2024-09-15T22:38:53.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:53 smithi121 ceph-mon[34815]: osdmap e15: 3 total, 2 up, 3 in 2024-09-15T22:38:53.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:53 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:38:53.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:53 smithi121 ceph-mon[34815]: from='client.? 172.21.15.192:0/3626459787' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:38:53.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:53 smithi111 ceph-mon[30629]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:38:53.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:53 smithi111 ceph-mon[30629]: osdmap e14: 2 total, 2 up, 2 in 2024-09-15T22:38:53.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:53 smithi111 ceph-mon[30629]: from='client.? 172.21.15.192:0/4181028309' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46735ccf-6d0e-4154-bb55-422929819bfa"}]: dispatch 2024-09-15T22:38:53.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:53 smithi111 ceph-mon[30629]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46735ccf-6d0e-4154-bb55-422929819bfa"}]: dispatch 2024-09-15T22:38:53.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:53 smithi111 ceph-mon[30629]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46735ccf-6d0e-4154-bb55-422929819bfa"}]': finished 2024-09-15T22:38:53.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:53 smithi111 ceph-mon[30629]: osdmap e15: 3 total, 2 up, 3 in 2024-09-15T22:38:53.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:53 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:38:53.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:53 smithi111 ceph-mon[30629]: from='client.? 172.21.15.192:0/3626459787' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:38:55.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:55 smithi111 ceph-mon[30629]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:38:55.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:55 smithi192 ceph-mon[35427]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:38:55.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:55 smithi121 ceph-mon[34815]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:38:57.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:57 smithi192 ceph-mon[35427]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:38:57.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:57 smithi121 ceph-mon[34815]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:38:57.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:57 smithi111 ceph-mon[30629]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:38:59.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:38:59 smithi192 ceph-mon[35427]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:38:59.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:38:59 smithi121 ceph-mon[34815]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:38:59.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:38:59 smithi111 ceph-mon[30629]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:01.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:01 smithi192 ceph-mon[35427]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:01.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:01 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-15T22:39:01.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:01 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:01.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:01 smithi121 ceph-mon[34815]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:01.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:01 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-15T22:39:01.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:01 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:01.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:01 smithi111 ceph-mon[30629]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:01.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:01 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-15T22:39:01.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:01 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:02.733 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:02 smithi192 ceph-mon[35427]: Deploying daemon osd.2 on smithi192 2024-09-15T22:39:02.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:02 smithi121 ceph-mon[34815]: Deploying daemon osd.2 on smithi192 2024-09-15T22:39:02.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:02 smithi111 ceph-mon[30629]: Deploying daemon osd.2 on smithi192 2024-09-15T22:39:03.684 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:03 smithi192 ceph-mon[35427]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:03.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:03 smithi121 ceph-mon[34815]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:03.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:03 smithi111 ceph-mon[30629]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:05.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:05 smithi192 ceph-mon[35427]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:05.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:05 smithi121 ceph-mon[34815]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:05.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:05 smithi111 ceph-mon[30629]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:07.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:07 smithi192 ceph-mon[35427]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:07.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:07 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:39:07.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:07 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:07.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:07 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:07.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:07 smithi121 ceph-mon[34815]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:07.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:07 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:39:07.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:07 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:07.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:07 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:07.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:07 smithi111 ceph-mon[30629]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:07.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:07 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:39:07.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:07 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:07.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:07 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:09.799 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:09 smithi192 ceph-mon[35427]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:09.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:09 smithi121 ceph-mon[34815]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:09.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:09 smithi111 ceph-mon[30629]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:11.202 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:11 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:11.202 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:11 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:11.202 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:11 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:11.202 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:11 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:39:11.202 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:11 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:11.203 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:11 smithi192 ceph-mon[35427]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:11.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:11 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:11.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:11 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:11.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:11 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:11.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:11 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:39:11.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:11 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:11.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:11 smithi121 ceph-mon[34815]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:11.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:11 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:11.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:11 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:11.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:11 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:11.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:11 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:39:11.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:11 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:11.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:11 smithi111 ceph-mon[30629]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:11.527 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:39:11.525+0000 7fcdbc898640 1 -- 172.21.15.192:0/1025674905 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcdb81a2bd0 con 0x7fcda40a31e0 2024-09-15T22:39:11.527 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 2 on host 'smithi192' 2024-09-15T22:39:11.529 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:39:11.527+0000 7fcdbd89a640 1 -- 172.21.15.192:0/1025674905 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fcda40a31e0 msgr2=0x7fcda40a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:11.529 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:39:11.527+0000 7fcdbd89a640 1 --2- 172.21.15.192:0/1025674905 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fcda40a31e0 0x7fcda40a56a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8101b30 tx=0x7fcda8006c90 comp rx=0 tx=0).stop 2024-09-15T22:39:11.529 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:39:11.527+0000 7fcdbd89a640 1 -- 172.21.15.192:0/1025674905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdb8102690 msgr2=0x7fcdb81a1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:11.529 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:39:11.527+0000 7fcdbd89a640 1 --2- 172.21.15.192:0/1025674905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdb8102690 0x7fcdb81a1820 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00098d0 tx=0x7fcdb002f840 comp rx=0 tx=0).stop 2024-09-15T22:39:11.529 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:39:11.527+0000 7fcdbd89a640 1 -- 172.21.15.192:0/1025674905 shutdown_connections 2024-09-15T22:39:11.529 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:39:11.527+0000 7fcdbd89a640 1 --2- 172.21.15.192:0/1025674905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdb8102690 0x7fcdb81a1820 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:11.529 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:39:11.527+0000 7fcdbd89a640 1 --2- 172.21.15.192:0/1025674905 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fcda40a31e0 0x7fcda40a56a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:11.529 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:39:11.527+0000 7fcdbd89a640 1 --2- 172.21.15.192:0/1025674905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcdb8101cd0 0x7fcdb819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:11.530 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:39:11.527+0000 7fcdbd89a640 1 --2- 172.21.15.192:0/1025674905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcdb8100ad0 0x7fcdb819a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:11.530 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:39:11.527+0000 7fcdbd89a640 1 -- 172.21.15.192:0/1025674905 >> 172.21.15.192:0/1025674905 conn(0x7fcdb80fc280 msgr2=0x7fcdb8069fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:11.530 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:39:11.527+0000 7fcdbd89a640 1 -- 172.21.15.192:0/1025674905 shutdown_connections 2024-09-15T22:39:11.530 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:39:11.528+0000 7fcdbd89a640 1 -- 172.21.15.192:0/1025674905 wait complete. 2024-09-15T22:39:12.114 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:12 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:39:12.114 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:12 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:12.114 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:12 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:12.145 DEBUG:teuthology.orchestra.run.smithi192:osd.2> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.2.service 2024-09-15T22:39:12.148 INFO:tasks.cephadm:Deploying osd.3 on smithi111 with /dev/vg_nvme/lv_3... 2024-09-15T22:39:12.148 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-15T22:39:12.393 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:12 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:39:12.393 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:12 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:12.393 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:12 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:12.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:12 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:39:12.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:12 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:12.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:12 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:12.515 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:39:13.204 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:13 smithi111 ceph-mon[30629]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:13.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:13 smithi192 ceph-mon[35427]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:13.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:13 smithi121 ceph-mon[34815]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:14.026 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 15 22:39:13 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-2[40429]: 2024-09-15T22:39:13.707+0000 7f5479eda640 -1 osd.2 0 log_to_monitors true 2024-09-15T22:39:14.216 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:14 smithi111 ceph-mon[30629]: from='osd.2 [v2:172.21.15.192:6800/93085639,v1:172.21.15.192:6801/93085639]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T22:39:14.216 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:14 smithi111 ceph-mon[30629]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T22:39:14.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:14 smithi192 ceph-mon[35427]: from='osd.2 [v2:172.21.15.192:6800/93085639,v1:172.21.15.192:6801/93085639]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T22:39:14.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:14 smithi192 ceph-mon[35427]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T22:39:14.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:14 smithi121 ceph-mon[34815]: from='osd.2 [v2:172.21.15.192:6800/93085639,v1:172.21.15.192:6801/93085639]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T22:39:14.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:14 smithi121 ceph-mon[34815]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T22:39:15.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:15 smithi111 ceph-mon[30629]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-15T22:39:15.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:15 smithi111 ceph-mon[30629]: osdmap e16: 3 total, 2 up, 3 in 2024-09-15T22:39:15.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:15 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:39:15.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:15 smithi111 ceph-mon[30629]: from='osd.2 [v2:172.21.15.192:6800/93085639,v1:172.21.15.192:6801/93085639]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-09-15T22:39:15.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:15 smithi111 ceph-mon[30629]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-09-15T22:39:15.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:15 smithi111 ceph-mon[30629]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:15.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:15 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:15.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:15 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:15.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:15 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:39:15.167 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:15 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:15.167 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:15 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:15.167 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:15 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:39:15.167 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:15 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:15.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:15 smithi192 ceph-mon[35427]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-15T22:39:15.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:15 smithi192 ceph-mon[35427]: osdmap e16: 3 total, 2 up, 3 in 2024-09-15T22:39:15.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:15 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:39:15.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:15 smithi192 ceph-mon[35427]: from='osd.2 [v2:172.21.15.192:6800/93085639,v1:172.21.15.192:6801/93085639]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-09-15T22:39:15.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:15 smithi192 ceph-mon[35427]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-09-15T22:39:15.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:15 smithi192 ceph-mon[35427]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:15.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:15 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:15.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:15 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:15.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:15 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:39:15.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:15 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:15.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:15 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:15.316 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:15 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:39:15.316 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:15 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:15.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:15 smithi121 ceph-mon[34815]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-15T22:39:15.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:15 smithi121 ceph-mon[34815]: osdmap e16: 3 total, 2 up, 3 in 2024-09-15T22:39:15.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:15 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:39:15.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:15 smithi121 ceph-mon[34815]: from='osd.2 [v2:172.21.15.192:6800/93085639,v1:172.21.15.192:6801/93085639]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-09-15T22:39:15.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:15 smithi121 ceph-mon[34815]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-09-15T22:39:15.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:15 smithi121 ceph-mon[34815]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:15.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:15 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:15.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:15 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:15.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:15 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:39:15.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:15 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:15.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:15 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:15.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:15 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:39:15.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:15 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:15.771 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:39:15.803 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_3 2024-09-15T22:39:16.163 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:39:16.302 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:16 smithi111 ceph-mon[30629]: Detected new or changed devices on smithi192 2024-09-15T22:39:16.303 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:16 smithi111 ceph-mon[30629]: Adjusting osd_memory_target on smithi192 to 17088M 2024-09-15T22:39:16.303 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:16 smithi111 ceph-mon[30629]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-09-15T22:39:16.303 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:16 smithi111 ceph-mon[30629]: osdmap e17: 3 total, 2 up, 3 in 2024-09-15T22:39:16.303 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:16 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:39:16.303 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:16 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:39:16.303 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:16 smithi111 ceph-mon[30629]: from='osd.2 ' entity='osd.2' 2024-09-15T22:39:16.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:16 smithi192 ceph-mon[35427]: Detected new or changed devices on smithi192 2024-09-15T22:39:16.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:16 smithi192 ceph-mon[35427]: Adjusting osd_memory_target on smithi192 to 17088M 2024-09-15T22:39:16.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:16 smithi192 ceph-mon[35427]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-09-15T22:39:16.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:16 smithi192 ceph-mon[35427]: osdmap e17: 3 total, 2 up, 3 in 2024-09-15T22:39:16.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:16 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:39:16.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:16 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:39:16.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:16 smithi192 ceph-mon[35427]: from='osd.2 ' entity='osd.2' 2024-09-15T22:39:16.316 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 15 22:39:15 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-2[40429]: 2024-09-15T22:39:15.897+0000 7f547563a640 -1 osd.2 0 waiting for initial osdmap 2024-09-15T22:39:16.316 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 15 22:39:15 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-2[40429]: 2024-09-15T22:39:15.904+0000 7f546dc6c640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:39:16.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:16 smithi121 ceph-mon[34815]: Detected new or changed devices on smithi192 2024-09-15T22:39:16.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:16 smithi121 ceph-mon[34815]: Adjusting osd_memory_target on smithi192 to 17088M 2024-09-15T22:39:16.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:16 smithi121 ceph-mon[34815]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-09-15T22:39:16.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:16 smithi121 ceph-mon[34815]: osdmap e17: 3 total, 2 up, 3 in 2024-09-15T22:39:16.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:16 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:39:16.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:16 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:39:16.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:16 smithi121 ceph-mon[34815]: from='osd.2 ' entity='osd.2' 2024-09-15T22:39:16.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.780+0000 7f3cf4a00640 1 -- 172.21.15.111:0/1922492852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3cf0078790 msgr2=0x7f3cf0079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:16.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.780+0000 7f3cf4a00640 1 --2- 172.21.15.111:0/1922492852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3cf0078790 0x7f3cf0079030 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009ca0 tx=0x7f3ce802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.780+0000 7f3cf4a00640 1 -- 172.21.15.111:0/1922492852 shutdown_connections 2024-09-15T22:39:16.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.780+0000 7f3cf4a00640 1 --2- 172.21.15.111:0/1922492852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3cf0078790 0x7f3cf0079030 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.780+0000 7f3cf4a00640 1 --2- 172.21.15.111:0/1922492852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cf0077df0 0x7f3cf0078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.780+0000 7f3cf4a00640 1 --2- 172.21.15.111:0/1922492852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cf0076ba0 0x7f3cf0076fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.780+0000 7f3cf4a00640 1 -- 172.21.15.111:0/1922492852 >> 172.21.15.111:0/1922492852 conn(0x7f3cf00ff3e0 msgr2=0x7f3cf0101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:16.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.780+0000 7f3cf4a00640 1 -- 172.21.15.111:0/1922492852 shutdown_connections 2024-09-15T22:39:16.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.780+0000 7f3cf4a00640 1 -- 172.21.15.111:0/1922492852 wait complete. 2024-09-15T22:39:16.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.781+0000 7f3cf4a00640 1 Processor -- start 2024-09-15T22:39:16.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.781+0000 7f3cf4a00640 1 -- start start 2024-09-15T22:39:16.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.782+0000 7f3cf4a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cf0076ba0 0x7f3cf019e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:16.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.782+0000 7f3cf4a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cf0077df0 0x7f3cf019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:16.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.782+0000 7f3cf4a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3cf0078790 0x7f3cf01a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:16.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.782+0000 7f3cf4a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf0113d60 con 0x7f3cf0078790 2024-09-15T22:39:16.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.782+0000 7f3cf4a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf0113be0 con 0x7f3cf0077df0 2024-09-15T22:39:16.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.782+0000 7f3cf4a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf0113ee0 con 0x7f3cf0076ba0 2024-09-15T22:39:16.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.782+0000 7f3cee800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cf0076ba0 0x7f3cf019e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:16.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.782+0000 7f3cef200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3cf0078790 0x7f3cf01a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:16.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.782+0000 7f3cee800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cf0076ba0 0x7f3cf019e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:34896/0 (socket says 172.21.15.111:34896) 2024-09-15T22:39:16.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.782+0000 7f3cee800640 1 -- 172.21.15.111:0/1162754560 learned_addr learned my addr 172.21.15.111:0/1162754560 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:39:16.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.782+0000 7f3cede00640 1 --2- 172.21.15.111:0/1162754560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cf0077df0 0x7f3cf019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:16.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.782+0000 7f3cee800640 1 -- 172.21.15.111:0/1162754560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cf0077df0 msgr2=0x7f3cf019e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:16.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.782+0000 7f3cee800640 1 --2- 172.21.15.111:0/1162754560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cf0077df0 0x7f3cf019e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.782+0000 7f3cee800640 1 -- 172.21.15.111:0/1162754560 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3cf0078790 msgr2=0x7f3cf01a5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:16.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.782+0000 7f3cee800640 1 --2- 172.21.15.111:0/1162754560 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3cf0078790 0x7f3cf01a5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:16.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.782+0000 7f3cee800640 1 -- 172.21.15.111:0/1162754560 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf01a5ef0 con 0x7f3cf0076ba0 2024-09-15T22:39:16.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.783+0000 7f3cee800640 1 --2- 172.21.15.111:0/1162754560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cf0076ba0 0x7f3cf019e2c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3ce000e9f0 tx=0x7f3ce000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:16.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.783+0000 7f3ce7400640 1 -- 172.21.15.111:0/1162754560 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce000cd80 con 0x7f3cf0076ba0 2024-09-15T22:39:16.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.783+0000 7f3cf4a00640 1 -- 172.21.15.111:0/1162754560 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cf01a61e0 con 0x7f3cf0076ba0 2024-09-15T22:39:16.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.783+0000 7f3ce7400640 1 -- 172.21.15.111:0/1162754560 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ce0005500 con 0x7f3cf0076ba0 2024-09-15T22:39:16.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.783+0000 7f3ce7400640 1 -- 172.21.15.111:0/1162754560 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce0010690 con 0x7f3cf0076ba0 2024-09-15T22:39:16.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.783+0000 7f3cf4a00640 1 -- 172.21.15.111:0/1162754560 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cf01a6720 con 0x7f3cf0076ba0 2024-09-15T22:39:16.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.784+0000 7f3cf4a00640 1 -- 172.21.15.111:0/1162754560 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cf01105d0 con 0x7f3cf0076ba0 2024-09-15T22:39:16.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.786+0000 7f3ce7400640 1 -- 172.21.15.111:0/1162754560 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 15) ==== 138300+0+0 (secure 0 0 0) 0x7f3ce00046c0 con 0x7f3cf0076ba0 2024-09-15T22:39:16.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.787+0000 7f3ce7400640 1 --2- 172.21.15.111:0/1162754560 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3cd40a31e0 0x7f3cd40a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:16.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.787+0000 7f3ce7400640 1 -- 172.21.15.111:0/1162754560 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7f3ce00ce8b0 con 0x7f3cf0076ba0 2024-09-15T22:39:16.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.787+0000 7f3cede00640 1 --2- 172.21.15.111:0/1162754560 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3cd40a31e0 0x7f3cd40a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:16.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.788+0000 7f3cede00640 1 --2- 172.21.15.111:0/1162754560 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3cd40a31e0 0x7f3cd40a56a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3cf019f7e0 tx=0x7f3cdc00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:16.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.789+0000 7f3ce7400640 1 -- 172.21.15.111:0/1162754560 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3ce009cc80 con 0x7f3cf0076ba0 2024-09-15T22:39:16.978 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:16.976+0000 7f3cf4a00640 1 -- 172.21.15.111:0/1162754560 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3cf0062630 con 0x7f3cd40a31e0 2024-09-15T22:39:17.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:17 smithi192 ceph-mon[35427]: purged_snaps scrub starts 2024-09-15T22:39:17.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:17 smithi192 ceph-mon[35427]: purged_snaps scrub ok 2024-09-15T22:39:17.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:17 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:39:17.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:17 smithi192 ceph-mon[35427]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:17.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:17 smithi192 ceph-mon[35427]: osd.2 [v2:172.21.15.192:6800/93085639,v1:172.21.15.192:6801/93085639] boot 2024-09-15T22:39:17.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:17 smithi192 ceph-mon[35427]: osdmap e18: 3 total, 3 up, 3 in 2024-09-15T22:39:17.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:17 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:39:17.316 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:17 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:39:17.316 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:17 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:39:17.316 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:17 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:17.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:17 smithi121 ceph-mon[34815]: purged_snaps scrub starts 2024-09-15T22:39:17.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:17 smithi121 ceph-mon[34815]: purged_snaps scrub ok 2024-09-15T22:39:17.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:17 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:39:17.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:17 smithi121 ceph-mon[34815]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:17.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:17 smithi121 ceph-mon[34815]: osd.2 [v2:172.21.15.192:6800/93085639,v1:172.21.15.192:6801/93085639] boot 2024-09-15T22:39:17.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:17 smithi121 ceph-mon[34815]: osdmap e18: 3 total, 3 up, 3 in 2024-09-15T22:39:17.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:17 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:39:17.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:17 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:39:17.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:17 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:39:17.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:17 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:17.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:17 smithi111 ceph-mon[30629]: purged_snaps scrub starts 2024-09-15T22:39:17.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:17 smithi111 ceph-mon[30629]: purged_snaps scrub ok 2024-09-15T22:39:17.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:17 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:39:17.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:17 smithi111 ceph-mon[30629]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:39:17.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:17 smithi111 ceph-mon[30629]: osd.2 [v2:172.21.15.192:6800/93085639,v1:172.21.15.192:6801/93085639] boot 2024-09-15T22:39:17.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:17 smithi111 ceph-mon[30629]: osdmap e18: 3 total, 3 up, 3 in 2024-09-15T22:39:17.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:17 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:39:17.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:17 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:39:17.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:17 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:39:17.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:17 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:18.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:18 smithi111 ceph-mon[30629]: from='client.24187 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:39:18.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:18 smithi192 ceph-mon[35427]: from='client.24187 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:39:18.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:18 smithi121 ceph-mon[34815]: from='client.24187 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:39:19.361 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:19 smithi111 ceph-mon[30629]: osdmap e19: 3 total, 3 up, 3 in 2024-09-15T22:39:19.361 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:19 smithi111 ceph-mon[30629]: pgmap v83: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:39:19.361 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:19 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:39:19.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:19 smithi121 ceph-mon[34815]: osdmap e19: 3 total, 3 up, 3 in 2024-09-15T22:39:19.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:19 smithi121 ceph-mon[34815]: pgmap v83: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:39:19.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:19 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:39:19.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:19 smithi192 ceph-mon[35427]: osdmap e19: 3 total, 3 up, 3 in 2024-09-15T22:39:19.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:19 smithi192 ceph-mon[35427]: pgmap v83: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:39:19.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:19 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:39:20.385 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:20 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-15T22:39:20.386 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:20 smithi111 ceph-mon[30629]: osdmap e20: 3 total, 3 up, 3 in 2024-09-15T22:39:20.386 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:20 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:39:20.386 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:20 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/2806554608' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b243af8-c958-40b1-ba24-9c7ace00c4a3"}]: dispatch 2024-09-15T22:39:20.386 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:20 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-15T22:39:20.386 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:20 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/2806554608' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b243af8-c958-40b1-ba24-9c7ace00c4a3"}]': finished 2024-09-15T22:39:20.386 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:20 smithi111 ceph-mon[30629]: osdmap e21: 4 total, 3 up, 4 in 2024-09-15T22:39:20.386 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:20 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:39:20.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:20 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-15T22:39:20.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:20 smithi121 ceph-mon[34815]: osdmap e20: 3 total, 3 up, 3 in 2024-09-15T22:39:20.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:20 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:39:20.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:20 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/2806554608' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b243af8-c958-40b1-ba24-9c7ace00c4a3"}]: dispatch 2024-09-15T22:39:20.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:20 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-15T22:39:20.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:20 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/2806554608' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b243af8-c958-40b1-ba24-9c7ace00c4a3"}]': finished 2024-09-15T22:39:20.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:20 smithi121 ceph-mon[34815]: osdmap e21: 4 total, 3 up, 4 in 2024-09-15T22:39:20.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:20 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:39:20.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:20 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-15T22:39:20.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:20 smithi192 ceph-mon[35427]: osdmap e20: 3 total, 3 up, 3 in 2024-09-15T22:39:20.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:20 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:39:20.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:20 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/2806554608' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b243af8-c958-40b1-ba24-9c7ace00c4a3"}]: dispatch 2024-09-15T22:39:20.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:20 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-15T22:39:20.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:20 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/2806554608' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b243af8-c958-40b1-ba24-9c7ace00c4a3"}]': finished 2024-09-15T22:39:20.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:20 smithi192 ceph-mon[35427]: osdmap e21: 4 total, 3 up, 4 in 2024-09-15T22:39:20.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:20 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:39:21.078 INFO:journalctl@ceph.osd.0.smithi111.stdout:Sep 15 22:39:20 smithi111 sudo[49342]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T22:39:21.078 INFO:journalctl@ceph.osd.0.smithi111.stdout:Sep 15 22:39:20 smithi111 sudo[49342]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:39:21.078 INFO:journalctl@ceph.osd.0.smithi111.stdout:Sep 15 22:39:20 smithi111 sudo[49342]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:39:21.078 INFO:journalctl@ceph.osd.0.smithi111.stdout:Sep 15 22:39:20 smithi111 sudo[49342]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:39:21.086 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:21.084+0000 7f3ce7400640 1 -- 172.21.15.111:0/1162754560 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f3ce0098bf0 con 0x7f3cf0076ba0 2024-09-15T22:39:21.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:21 smithi121 ceph-mon[34815]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:39:21.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:21 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/4045924523' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:39:21.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:21 smithi121 ceph-mon[34815]: osdmap e22: 4 total, 3 up, 4 in 2024-09-15T22:39:21.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:21 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:39:21.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:21 smithi111 ceph-mon[30629]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:39:21.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:21 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/4045924523' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:39:21.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:21 smithi111 ceph-mon[30629]: osdmap e22: 4 total, 3 up, 4 in 2024-09-15T22:39:21.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:21 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:39:21.416 INFO:journalctl@ceph.osd.0.smithi111.stdout:Sep 15 22:39:21 smithi111 sudo[49371]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T22:39:21.416 INFO:journalctl@ceph.osd.0.smithi111.stdout:Sep 15 22:39:21 smithi111 sudo[49371]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:39:21.416 INFO:journalctl@ceph.osd.0.smithi111.stdout:Sep 15 22:39:21 smithi111 sudo[49371]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:39:21.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:21 smithi192 ceph-mon[35427]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:39:21.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:21 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/4045924523' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:39:21.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:21 smithi192 ceph-mon[35427]: osdmap e22: 4 total, 3 up, 4 in 2024-09-15T22:39:21.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:21 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:39:21.915 INFO:journalctl@ceph.osd.0.smithi111.stdout:Sep 15 22:39:21 smithi111 sudo[49371]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:39:22.035 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 15 22:39:21 smithi121 sudo[43479]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T22:39:22.035 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 15 22:39:21 smithi121 sudo[43479]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:39:22.035 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 15 22:39:21 smithi121 sudo[43479]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:39:22.035 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 15 22:39:21 smithi121 sudo[43479]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:39:22.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:22 smithi121 ceph-mon[34815]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-15T22:39:22.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:22 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:22.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:22 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:22.391 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 15 22:39:22 smithi121 sudo[43482]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T22:39:22.391 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 15 22:39:22 smithi121 sudo[43482]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:39:22.391 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 15 22:39:22 smithi121 sudo[43482]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:39:22.391 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 15 22:39:22 smithi121 sudo[43482]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:39:22.417 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:22 smithi111 ceph-mon[30629]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-15T22:39:22.417 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:22 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:22.417 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:22 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:22.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:22 smithi192 ceph-mon[35427]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-15T22:39:22.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:22 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:22.562 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:22 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:22.563 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 15 22:39:22 smithi192 sudo[44082]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T22:39:22.563 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 15 22:39:22 smithi192 sudo[44082]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:39:22.563 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 15 22:39:22 smithi192 sudo[44082]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:39:22.563 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 15 22:39:22 smithi192 sudo[44082]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:39:22.814 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 15 22:39:22 smithi192 sudo[44085]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T22:39:22.815 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 15 22:39:22 smithi192 sudo[44085]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:39:22.815 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 15 22:39:22 smithi192 sudo[44085]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:39:22.815 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 15 22:39:22 smithi192 sudo[44085]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:39:23.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:23 smithi111 sudo[49781]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T22:39:23.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:23 smithi111 sudo[49781]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:39:23.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:23 smithi111 sudo[49781]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:39:23.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:23 smithi111 sudo[49781]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:39:23.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:23 smithi111 sudo[49974]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T22:39:23.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:23 smithi111 sudo[49974]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:39:23.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:23 smithi111 sudo[49974]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:39:23.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:23 smithi111 sudo[49974]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:39:24.159 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:23 smithi192 sudo[44088]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T22:39:24.160 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:23 smithi192 sudo[44088]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:39:24.160 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:23 smithi192 sudo[44088]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:39:24.160 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:23 smithi192 sudo[44088]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:39:24.421 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:24 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:24.421 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:24 smithi121 ceph-mon[34815]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:39:24.421 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:24 smithi121 sudo[43485]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T22:39:24.421 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:24 smithi121 sudo[43485]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:39:24.421 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:24 smithi121 sudo[43485]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:39:24.422 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:24 smithi121 sudo[43485]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:39:24.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:24 smithi192 sudo[44091]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T22:39:24.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:24 smithi192 sudo[44091]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:39:24.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:24 smithi192 sudo[44091]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:39:24.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:24 smithi192 sudo[44091]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:39:24.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:24 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:24.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:24 smithi192 ceph-mon[35427]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:39:24.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:24 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:24.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:24 smithi111 ceph-mon[30629]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:39:24.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:24 smithi121 sudo[43488]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T22:39:24.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:24 smithi121 sudo[43488]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:39:24.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:24 smithi121 sudo[43488]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:39:24.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:24 smithi121 sudo[43488]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:39:25.475 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:25 smithi111 ceph-mon[30629]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:39:25.475 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:25 smithi111 ceph-mon[30629]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:39:25.476 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:25 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:39:25.476 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:25 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:39:25.476 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:25 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:39:25.476 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:25 smithi111 ceph-mon[30629]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:39:25.476 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:25 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:39:25.476 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:25 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:39:25.476 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:25 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:39:25.476 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:25 smithi111 ceph-mon[30629]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:39:25.476 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:25 smithi111 ceph-mon[30629]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:39:25.476 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:25 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:39:25.476 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:25 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:39:25.476 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:25 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:39:25.476 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:25 smithi111 ceph-mon[30629]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:39:25.477 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:25 smithi111 ceph-mon[30629]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:39:25.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:25 smithi192 ceph-mon[35427]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:39:25.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:25 smithi192 ceph-mon[35427]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:39:25.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:25 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:39:25.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:25 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:39:25.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:25 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:39:25.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:25 smithi192 ceph-mon[35427]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:39:25.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:25 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:39:25.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:25 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:39:25.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:25 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:39:25.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:25 smithi192 ceph-mon[35427]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:39:25.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:25 smithi192 ceph-mon[35427]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:39:25.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:25 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:39:25.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:25 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:39:25.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:25 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:39:25.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:25 smithi192 ceph-mon[35427]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:39:25.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:25 smithi192 ceph-mon[35427]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:39:25.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:25 smithi121 ceph-mon[34815]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:39:25.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:25 smithi121 ceph-mon[34815]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:39:25.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:25 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:39:25.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:25 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:39:25.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:25 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:39:25.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:25 smithi121 ceph-mon[34815]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:39:25.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:25 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:39:25.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:25 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:39:25.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:25 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:39:25.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:25 smithi121 ceph-mon[34815]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:39:25.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:25 smithi121 ceph-mon[34815]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:39:25.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:25 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:39:25.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:25 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:39:25.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:25 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:39:25.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:25 smithi121 ceph-mon[34815]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:39:25.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:25 smithi121 ceph-mon[34815]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:39:27.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:27 smithi192 ceph-mon[35427]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:39:27.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:27 smithi121 ceph-mon[34815]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:39:27.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:27 smithi111 ceph-mon[30629]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:39:29.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:29 smithi192 ceph-mon[35427]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-09-15T22:39:29.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:29 smithi121 ceph-mon[34815]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-09-15T22:39:29.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:29 smithi111 ceph-mon[30629]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-09-15T22:39:30.719 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:30 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-15T22:39:30.719 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:30 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:30.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:30 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-15T22:39:30.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:30 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:30.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:30 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-15T22:39:30.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:30 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:31.716 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:31 smithi192 ceph-mon[35427]: Deploying daemon osd.3 on smithi111 2024-09-15T22:39:31.716 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:31 smithi192 ceph-mon[35427]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-15T22:39:31.722 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:31 smithi111 ceph-mon[30629]: Deploying daemon osd.3 on smithi111 2024-09-15T22:39:31.723 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:31 smithi111 ceph-mon[30629]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-15T22:39:31.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:31 smithi121 ceph-mon[34815]: Deploying daemon osd.3 on smithi111 2024-09-15T22:39:31.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:31 smithi121 ceph-mon[34815]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-15T22:39:33.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:33 smithi111 ceph-mon[30629]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-15T22:39:33.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:33 smithi192 ceph-mon[35427]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-15T22:39:33.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:33 smithi121 ceph-mon[34815]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-09-15T22:39:35.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:35 smithi192 ceph-mon[35427]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-15T22:39:35.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:35 smithi121 ceph-mon[34815]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-15T22:39:35.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:35 smithi111 ceph-mon[30629]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-09-15T22:39:37.720 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:37 smithi111 ceph-mon[30629]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-15T22:39:37.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:37 smithi192 ceph-mon[35427]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-15T22:39:37.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:37 smithi121 ceph-mon[34815]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-15T22:39:38.737 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:38 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:39:38.738 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:38 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:38.738 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:38 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:38.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:38 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:39:38.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:38 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:38.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:38 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:38.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:38 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:39:38.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:38 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:38.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:38 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:39.613 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:39 smithi111 ceph-mon[30629]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-15T22:39:39.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:39 smithi192 ceph-mon[35427]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-15T22:39:39.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:39 smithi121 ceph-mon[34815]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-15T22:39:41.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:41 smithi192 ceph-mon[35427]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:39:41.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:41 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:41.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:41 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:41.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:41 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:41.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:41 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:39:41.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:41 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:41.833 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:41 smithi111 ceph-mon[30629]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:39:41.833 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:41 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:41.833 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:41 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:41.833 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:41 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:41.833 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:41 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:39:41.833 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:41 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:41.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:41 smithi121 ceph-mon[34815]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:39:41.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:41 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:41.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:41 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:41.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:41 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:41.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:41 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:39:41.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:41 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:42.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:42.658+0000 7f3ce7400640 1 -- 172.21.15.111:0/1162754560 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3cf0062630 con 0x7f3cd40a31e0 2024-09-15T22:39:42.661 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 3 on host 'smithi111' 2024-09-15T22:39:42.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:42.661+0000 7f3cf4a00640 1 -- 172.21.15.111:0/1162754560 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3cd40a31e0 msgr2=0x7f3cd40a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:42.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:42.661+0000 7f3cf4a00640 1 --2- 172.21.15.111:0/1162754560 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3cd40a31e0 0x7f3cd40a56a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3cf019f7e0 tx=0x7f3cdc00b390 comp rx=0 tx=0).stop 2024-09-15T22:39:42.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:42.661+0000 7f3cf4a00640 1 -- 172.21.15.111:0/1162754560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cf0076ba0 msgr2=0x7f3cf019e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:42.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:42.661+0000 7f3cf4a00640 1 --2- 172.21.15.111:0/1162754560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cf0076ba0 0x7f3cf019e2c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3ce000e9f0 tx=0x7f3ce000edf0 comp rx=0 tx=0).stop 2024-09-15T22:39:42.663 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:42.661+0000 7f3cf4a00640 1 -- 172.21.15.111:0/1162754560 shutdown_connections 2024-09-15T22:39:42.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:42.661+0000 7f3cf4a00640 1 --2- 172.21.15.111:0/1162754560 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3cf0078790 0x7f3cf01a5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:42.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:42.661+0000 7f3cf4a00640 1 --2- 172.21.15.111:0/1162754560 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3cd40a31e0 0x7f3cd40a56a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:42.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:42.661+0000 7f3cf4a00640 1 --2- 172.21.15.111:0/1162754560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3cf0077df0 0x7f3cf019e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:42.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:42.661+0000 7f3cf4a00640 1 --2- 172.21.15.111:0/1162754560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cf0076ba0 0x7f3cf019e2c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:42.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:42.661+0000 7f3cf4a00640 1 -- 172.21.15.111:0/1162754560 >> 172.21.15.111:0/1162754560 conn(0x7f3cf00ff3e0 msgr2=0x7f3cf0100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:42.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:42.661+0000 7f3cf4a00640 1 -- 172.21.15.111:0/1162754560 shutdown_connections 2024-09-15T22:39:42.664 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:39:42.662+0000 7f3cf4a00640 1 -- 172.21.15.111:0/1162754560 wait complete. 2024-09-15T22:39:43.596 DEBUG:teuthology.orchestra.run.smithi111:osd.3> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.3.service 2024-09-15T22:39:43.602 INFO:tasks.cephadm:Deploying osd.4 on smithi121 with /dev/vg_nvme/lv_3... 2024-09-15T22:39:43.602 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-15T22:39:43.758 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:43 smithi111 ceph-mon[30629]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:39:43.758 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:43 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:39:43.758 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:43 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:43.758 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:43 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:43.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:43 smithi192 ceph-mon[35427]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:39:43.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:43 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:39:43.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:43 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:43.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:43 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:43.852 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:39:43.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:43 smithi121 ceph-mon[34815]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:39:43.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:43 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:39:43.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:43 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:43.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:43 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:44.416 INFO:journalctl@ceph.osd.3.smithi111.stdout:Sep 15 22:39:44 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-3[51379]: 2024-09-15T22:39:44.140+0000 7fb980635640 -1 osd.3 0 log_to_monitors true 2024-09-15T22:39:44.570 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:44 smithi121 ceph-mon[34815]: from='osd.3 [v2:172.21.15.111:6810/2578259041,v1:172.21.15.111:6811/2578259041]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-15T22:39:44.570 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:44 smithi121 ceph-mon[34815]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-15T22:39:44.763 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:44 smithi111 ceph-mon[30629]: from='osd.3 [v2:172.21.15.111:6810/2578259041,v1:172.21.15.111:6811/2578259041]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-15T22:39:44.764 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:44 smithi111 ceph-mon[30629]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-15T22:39:44.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:44 smithi192 ceph-mon[35427]: from='osd.3 [v2:172.21.15.111:6810/2578259041,v1:172.21.15.111:6811/2578259041]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-15T22:39:44.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:44 smithi192 ceph-mon[35427]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-15T22:39:45.585 INFO:journalctl@ceph.osd.3.smithi111.stdout:Sep 15 22:39:45 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-3[51379]: 2024-09-15T22:39:45.496+0000 7fb97a800640 -1 osd.3 0 waiting for initial osdmap 2024-09-15T22:39:45.585 INFO:journalctl@ceph.osd.3.smithi111.stdout:Sep 15 22:39:45 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-3[51379]: 2024-09-15T22:39:45.510+0000 7fb976200640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:39:45.585 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:45 smithi111 ceph-mon[30629]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:39:45.586 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:45 smithi111 ceph-mon[30629]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-15T22:39:45.586 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:45 smithi111 ceph-mon[30629]: from='osd.3 [v2:172.21.15.111:6810/2578259041,v1:172.21.15.111:6811/2578259041]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-15T22:39:45.586 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:45 smithi111 ceph-mon[30629]: osdmap e23: 4 total, 3 up, 4 in 2024-09-15T22:39:45.586 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:45 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:39:45.586 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:45 smithi111 ceph-mon[30629]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-15T22:39:45.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:45 smithi192 ceph-mon[35427]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:39:45.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:45 smithi192 ceph-mon[35427]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-15T22:39:45.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:45 smithi192 ceph-mon[35427]: from='osd.3 [v2:172.21.15.111:6810/2578259041,v1:172.21.15.111:6811/2578259041]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-15T22:39:45.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:45 smithi192 ceph-mon[35427]: osdmap e23: 4 total, 3 up, 4 in 2024-09-15T22:39:45.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:45 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:39:45.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:45 smithi192 ceph-mon[35427]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-15T22:39:45.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:45 smithi121 ceph-mon[34815]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:39:45.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:45 smithi121 ceph-mon[34815]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-15T22:39:45.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:45 smithi121 ceph-mon[34815]: from='osd.3 [v2:172.21.15.111:6810/2578259041,v1:172.21.15.111:6811/2578259041]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-15T22:39:45.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:45 smithi121 ceph-mon[34815]: osdmap e23: 4 total, 3 up, 4 in 2024-09-15T22:39:45.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:45 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:39:45.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:45 smithi121 ceph-mon[34815]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-15T22:39:46.197 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-15T22:39:46.223 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi121:vg_nvme/lv_3 2024-09-15T22:39:46.468 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:39:46.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:46 smithi192 ceph-mon[35427]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-15T22:39:46.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:46 smithi192 ceph-mon[35427]: osdmap e24: 4 total, 3 up, 4 in 2024-09-15T22:39:46.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:39:46.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:39:46.830 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:46 smithi121 ceph-mon[34815]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-15T22:39:46.831 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:46 smithi121 ceph-mon[34815]: osdmap e24: 4 total, 3 up, 4 in 2024-09-15T22:39:46.831 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:39:46.831 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:39:46.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:46 smithi111 ceph-mon[30629]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-15T22:39:46.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:46 smithi111 ceph-mon[30629]: osdmap e24: 4 total, 3 up, 4 in 2024-09-15T22:39:46.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:39:46.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:39:46.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.913+0000 7ff4d8fce640 1 -- 172.21.15.121:0/1078964356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4d4102660 msgr2=0x7ff4d4104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:46.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.913+0000 7ff4d8fce640 1 --2- 172.21.15.121:0/1078964356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4d4102660 0x7ff4d4104a50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009900 tx=0x7ff4c402a410 comp rx=0 tx=0).stop 2024-09-15T22:39:46.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.914+0000 7ff4d8fce640 1 -- 172.21.15.121:0/1078964356 shutdown_connections 2024-09-15T22:39:46.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.914+0000 7ff4d8fce640 1 --2- 172.21.15.121:0/1078964356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4d41078c0 0x7ff4d4109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:46.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.914+0000 7ff4d8fce640 1 --2- 172.21.15.121:0/1078964356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4d4104f90 0x7ff4d4107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:46.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.914+0000 7ff4d8fce640 1 --2- 172.21.15.121:0/1078964356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4d4102660 0x7ff4d4104a50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:46.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.914+0000 7ff4d8fce640 1 -- 172.21.15.121:0/1078964356 >> 172.21.15.121:0/1078964356 conn(0x7ff4d40fc280 msgr2=0x7ff4d40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:39:46.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.914+0000 7ff4d8fce640 1 -- 172.21.15.121:0/1078964356 shutdown_connections 2024-09-15T22:39:46.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.914+0000 7ff4d8fce640 1 -- 172.21.15.121:0/1078964356 wait complete. 2024-09-15T22:39:46.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.914+0000 7ff4d8fce640 1 Processor -- start 2024-09-15T22:39:46.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.914+0000 7ff4d8fce640 1 -- start start 2024-09-15T22:39:46.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.915+0000 7ff4d8fce640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4d4102660 0x7ff4d4101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:46.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.915+0000 7ff4d8fce640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4d4104f90 0x7ff4d4100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:46.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.915+0000 7ff4d8fce640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4d41078c0 0x7ff4d4100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:46.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.915+0000 7ff4d8fce640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d410d950 con 0x7ff4d41078c0 2024-09-15T22:39:46.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.915+0000 7ff4d8fce640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d410d7d0 con 0x7ff4d4102660 2024-09-15T22:39:46.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.915+0000 7ff4d8fce640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d410dad0 con 0x7ff4d4104f90 2024-09-15T22:39:46.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.915+0000 7ff4d259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4d4102660 0x7ff4d4101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:46.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.915+0000 7ff4d259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4d4102660 0x7ff4d4101fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43334/0 (socket says 172.21.15.121:43334) 2024-09-15T22:39:46.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.915+0000 7ff4d259c640 1 -- 172.21.15.121:0/3606812683 learned_addr learned my addr 172.21.15.121:0/3606812683 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-09-15T22:39:46.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.915+0000 7ff4d1d9b640 1 --2- 172.21.15.121:0/3606812683 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4d4104f90 0x7ff4d4100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:46.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.915+0000 7ff4d2d9d640 1 --2- 172.21.15.121:0/3606812683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4d41078c0 0x7ff4d4100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:46.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.915+0000 7ff4d259c640 1 -- 172.21.15.121:0/3606812683 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4d4104f90 msgr2=0x7ff4d4100610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:46.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.915+0000 7ff4d259c640 1 --2- 172.21.15.121:0/3606812683 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4d4104f90 0x7ff4d4100610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:46.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.915+0000 7ff4d259c640 1 -- 172.21.15.121:0/3606812683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4d41078c0 msgr2=0x7ff4d4100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:39:46.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.915+0000 7ff4d259c640 1 --2- 172.21.15.121:0/3606812683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4d41078c0 0x7ff4d4100b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:39:46.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.915+0000 7ff4d259c640 1 -- 172.21.15.121:0/3606812683 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d4101410 con 0x7ff4d4102660 2024-09-15T22:39:46.950 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.915+0000 7ff4d2d9d640 1 --2- 172.21.15.121:0/3606812683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4d41078c0 0x7ff4d4100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:39:46.950 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.915+0000 7ff4d259c640 1 --2- 172.21.15.121:0/3606812683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4d4102660 0x7ff4d4101fc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009230 tx=0x7ff4c4004430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:46.950 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.915+0000 7ff4c37fe640 1 -- 172.21.15.121:0/3606812683 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c403f030 con 0x7ff4d4102660 2024-09-15T22:39:46.950 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.915+0000 7ff4c37fe640 1 -- 172.21.15.121:0/3606812683 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4c4003ce0 con 0x7ff4d4102660 2024-09-15T22:39:46.950 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.915+0000 7ff4d8fce640 1 -- 172.21.15.121:0/3606812683 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d41a21c0 con 0x7ff4d4102660 2024-09-15T22:39:46.950 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.915+0000 7ff4c37fe640 1 -- 172.21.15.121:0/3606812683 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4035070 con 0x7ff4d4102660 2024-09-15T22:39:46.950 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.916+0000 7ff4d8fce640 1 -- 172.21.15.121:0/3606812683 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d41a26f0 con 0x7ff4d4102660 2024-09-15T22:39:46.950 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.917+0000 7ff4d8fce640 1 -- 172.21.15.121:0/3606812683 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4d40fdd70 con 0x7ff4d4102660 2024-09-15T22:39:46.950 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.918+0000 7ff4c37fe640 1 -- 172.21.15.121:0/3606812683 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7ff4c4042030 con 0x7ff4d4102660 2024-09-15T22:39:46.950 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.919+0000 7ff4c37fe640 1 --2- 172.21.15.121:0/3606812683 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ff4bc0a3240 0x7ff4bc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:39:46.950 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.919+0000 7ff4c37fe640 1 -- 172.21.15.121:0/3606812683 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3702+0+0 (secure 0 0 0) 0x7ff4c40ef2a0 con 0x7ff4d4102660 2024-09-15T22:39:46.950 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.919+0000 7ff4d1d9b640 1 --2- 172.21.15.121:0/3606812683 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ff4bc0a3240 0x7ff4bc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:39:46.950 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.919+0000 7ff4d1d9b640 1 --2- 172.21.15.121:0/3606812683 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ff4bc0a3240 0x7ff4bc0a5700 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4101d40 tx=0x7ff4c8009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:39:46.951 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:46.922+0000 7ff4c37fe640 1 -- 172.21.15.121:0/3606812683 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff4c40bd120 con 0x7ff4d4102660 2024-09-15T22:39:47.104 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:39:47.103+0000 7ff4d8fce640 1 -- 172.21.15.121:0/3606812683 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff4d4062630 con 0x7ff4bc0a3240 2024-09-15T22:39:47.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:47 smithi111 ceph-mon[30629]: purged_snaps scrub starts 2024-09-15T22:39:47.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:47 smithi111 ceph-mon[30629]: purged_snaps scrub ok 2024-09-15T22:39:47.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:47 smithi111 ceph-mon[30629]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:39:47.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:47 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:39:47.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:47 smithi111 ceph-mon[30629]: osd.3 [v2:172.21.15.111:6810/2578259041,v1:172.21.15.111:6811/2578259041] boot 2024-09-15T22:39:47.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:47 smithi111 ceph-mon[30629]: osdmap e25: 4 total, 4 up, 4 in 2024-09-15T22:39:47.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:47 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:39:47.667 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:47 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:39:47.667 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:47 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:39:47.667 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:47 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:47.667 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:47 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:47.667 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:47 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:47.667 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:47 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:39:47.667 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:47 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:39:47.667 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:47 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:47.667 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:47 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:47.667 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:47 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:39:47.667 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:47 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:47.775 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:47 smithi121 ceph-mon[34815]: purged_snaps scrub starts 2024-09-15T22:39:47.775 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:47 smithi121 ceph-mon[34815]: purged_snaps scrub ok 2024-09-15T22:39:47.775 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:47 smithi121 ceph-mon[34815]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:39:47.775 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:47 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:39:47.775 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:47 smithi121 ceph-mon[34815]: osd.3 [v2:172.21.15.111:6810/2578259041,v1:172.21.15.111:6811/2578259041] boot 2024-09-15T22:39:47.775 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:47 smithi121 ceph-mon[34815]: osdmap e25: 4 total, 4 up, 4 in 2024-09-15T22:39:47.776 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:47 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:39:47.776 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:47 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:39:47.776 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:47 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:39:47.776 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:47 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:47.776 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:47 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:47.776 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:47 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:47.776 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:47 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:39:47.776 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:47 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:39:47.776 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:47 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:47.776 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:47 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:47.776 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:47 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:39:47.776 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:47 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:47.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:47 smithi192 ceph-mon[35427]: purged_snaps scrub starts 2024-09-15T22:39:47.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:47 smithi192 ceph-mon[35427]: purged_snaps scrub ok 2024-09-15T22:39:47.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:47 smithi192 ceph-mon[35427]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:39:47.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:47 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:39:47.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:47 smithi192 ceph-mon[35427]: osd.3 [v2:172.21.15.111:6810/2578259041,v1:172.21.15.111:6811/2578259041] boot 2024-09-15T22:39:47.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:47 smithi192 ceph-mon[35427]: osdmap e25: 4 total, 4 up, 4 in 2024-09-15T22:39:47.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:47 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:39:47.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:47 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:39:47.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:47 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:39:47.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:47 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:47.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:47 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:47.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:47 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:47.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:47 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:39:47.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:47 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:39:47.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:47 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:47.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:47 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:47.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:47 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:39:47.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:47 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:39:48.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:48 smithi121 ceph-mon[34815]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:39:48.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:48 smithi121 ceph-mon[34815]: Detected new or changed devices on smithi111 2024-09-15T22:39:48.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:48 smithi121 ceph-mon[34815]: Adjusting osd_memory_target on smithi111 to 8544M 2024-09-15T22:39:48.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:48 smithi121 ceph-mon[34815]: osdmap e26: 4 total, 4 up, 4 in 2024-09-15T22:39:48.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:48 smithi192 ceph-mon[35427]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:39:48.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:48 smithi192 ceph-mon[35427]: Detected new or changed devices on smithi111 2024-09-15T22:39:48.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:48 smithi192 ceph-mon[35427]: Adjusting osd_memory_target on smithi111 to 8544M 2024-09-15T22:39:48.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:48 smithi192 ceph-mon[35427]: osdmap e26: 4 total, 4 up, 4 in 2024-09-15T22:39:48.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:48 smithi111 ceph-mon[30629]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:39:48.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:48 smithi111 ceph-mon[30629]: Detected new or changed devices on smithi111 2024-09-15T22:39:48.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:48 smithi111 ceph-mon[30629]: Adjusting osd_memory_target on smithi111 to 8544M 2024-09-15T22:39:48.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:48 smithi111 ceph-mon[30629]: osdmap e26: 4 total, 4 up, 4 in 2024-09-15T22:39:49.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:49 smithi192 ceph-mon[35427]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:39:49.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:49 smithi192 ceph-mon[35427]: osdmap e27: 4 total, 4 up, 4 in 2024-09-15T22:39:49.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:49 smithi192 ceph-mon[35427]: osdmap e28: 4 total, 4 up, 4 in 2024-09-15T22:39:49.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:49 smithi192 ceph-mon[35427]: from='client.? 172.21.15.121:0/144708945' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49f27541-9cbf-48e0-a466-6bb0fa5410b9"}]: dispatch 2024-09-15T22:39:49.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:49 smithi192 ceph-mon[35427]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49f27541-9cbf-48e0-a466-6bb0fa5410b9"}]: dispatch 2024-09-15T22:39:49.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:49 smithi192 ceph-mon[35427]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49f27541-9cbf-48e0-a466-6bb0fa5410b9"}]': finished 2024-09-15T22:39:49.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:49 smithi192 ceph-mon[35427]: osdmap e29: 5 total, 4 up, 5 in 2024-09-15T22:39:49.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:49 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:39:49.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:49 smithi121 ceph-mon[34815]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:39:49.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:49 smithi121 ceph-mon[34815]: osdmap e27: 4 total, 4 up, 4 in 2024-09-15T22:39:49.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:49 smithi121 ceph-mon[34815]: osdmap e28: 4 total, 4 up, 4 in 2024-09-15T22:39:49.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:49 smithi121 ceph-mon[34815]: from='client.? 172.21.15.121:0/144708945' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49f27541-9cbf-48e0-a466-6bb0fa5410b9"}]: dispatch 2024-09-15T22:39:49.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:49 smithi121 ceph-mon[34815]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49f27541-9cbf-48e0-a466-6bb0fa5410b9"}]: dispatch 2024-09-15T22:39:49.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:49 smithi121 ceph-mon[34815]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49f27541-9cbf-48e0-a466-6bb0fa5410b9"}]': finished 2024-09-15T22:39:49.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:49 smithi121 ceph-mon[34815]: osdmap e29: 5 total, 4 up, 5 in 2024-09-15T22:39:49.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:49 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:39:49.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:49 smithi111 ceph-mon[30629]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:39:49.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:49 smithi111 ceph-mon[30629]: osdmap e27: 4 total, 4 up, 4 in 2024-09-15T22:39:49.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:49 smithi111 ceph-mon[30629]: osdmap e28: 4 total, 4 up, 4 in 2024-09-15T22:39:49.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:49 smithi111 ceph-mon[30629]: from='client.? 172.21.15.121:0/144708945' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49f27541-9cbf-48e0-a466-6bb0fa5410b9"}]: dispatch 2024-09-15T22:39:49.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:49 smithi111 ceph-mon[30629]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49f27541-9cbf-48e0-a466-6bb0fa5410b9"}]: dispatch 2024-09-15T22:39:49.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:49 smithi111 ceph-mon[30629]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49f27541-9cbf-48e0-a466-6bb0fa5410b9"}]': finished 2024-09-15T22:39:49.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:49 smithi111 ceph-mon[30629]: osdmap e29: 5 total, 4 up, 5 in 2024-09-15T22:39:49.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:49 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:39:50.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:50 smithi121 ceph-mon[34815]: from='client.? 172.21.15.121:0/4166013927' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:39:50.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:50 smithi111 ceph-mon[30629]: from='client.? 172.21.15.121:0/4166013927' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:39:51.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:50 smithi192 ceph-mon[35427]: from='client.? 172.21.15.121:0/4166013927' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:39:51.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:51 smithi121 ceph-mon[34815]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:39:51.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:51 smithi111 ceph-mon[30629]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:39:52.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:51 smithi192 ceph-mon[35427]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:39:53.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:53 smithi121 ceph-mon[34815]: pgmap v110: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:39:53.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:53 smithi111 ceph-mon[30629]: pgmap v110: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:39:54.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:53 smithi192 ceph-mon[35427]: pgmap v110: 1 pgs: 1 remapped+peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:39:55.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:55 smithi121 ceph-mon[34815]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:39:55.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:55 smithi111 ceph-mon[30629]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:39:56.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:55 smithi192 ceph-mon[35427]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:39:57.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:57 smithi121 ceph-mon[34815]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-15T22:39:57.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:57 smithi111 ceph-mon[30629]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-15T22:39:58.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:57 smithi192 ceph-mon[35427]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-15T22:39:58.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:58 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-15T22:39:58.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:58 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:58.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:58 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-15T22:39:58.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:58 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:59.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:58 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-15T22:39:59.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:58 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:39:59.870 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:59 smithi121 ceph-mon[34815]: Deploying daemon osd.4 on smithi121 2024-09-15T22:39:59.870 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:39:59 smithi121 ceph-mon[34815]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-15T22:39:59.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:59 smithi111 ceph-mon[30629]: Deploying daemon osd.4 on smithi121 2024-09-15T22:39:59.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:39:59 smithi111 ceph-mon[30629]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-15T22:40:00.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:59 smithi192 ceph-mon[35427]: Deploying daemon osd.4 on smithi121 2024-09-15T22:40:00.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:39:59 smithi192 ceph-mon[35427]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-15T22:40:00.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:00 smithi121 ceph-mon[34815]: overall HEALTH_OK 2024-09-15T22:40:00.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:00 smithi111 ceph-mon[30629]: overall HEALTH_OK 2024-09-15T22:40:01.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:00 smithi192 ceph-mon[35427]: overall HEALTH_OK 2024-09-15T22:40:01.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:01 smithi121 ceph-mon[34815]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-15T22:40:01.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:01 smithi111 ceph-mon[30629]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-15T22:40:01.966 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:01 smithi192 ceph-mon[35427]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-15T22:40:03.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:03 smithi121 ceph-mon[34815]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:40:03.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:03 smithi111 ceph-mon[30629]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:40:04.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:03 smithi192 ceph-mon[35427]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:40:04.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:04 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:40:04.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:04 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:04.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:04 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:04.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:04 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:40:04.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:04 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:04.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:04 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:05.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:04 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:40:05.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:04 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:05.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:04 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:05.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:05 smithi121 ceph-mon[34815]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:40:05.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:05 smithi111 ceph-mon[30629]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:40:06.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:05 smithi192 ceph-mon[35427]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:40:07.763 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:07 smithi121 ceph-mon[34815]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:40:07.763 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:07 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:07.763 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:07 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:07.763 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:07 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:40:07.763 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:07 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:40:07.763 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:07 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:07.764 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:40:07.760+0000 7ff4c37fe640 1 -- 172.21.15.121:0/3606812683 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff4d4062630 con 0x7ff4bc0a3240 2024-09-15T22:40:07.764 INFO:teuthology.orchestra.run.smithi121.stdout:Created osd(s) 4 on host 'smithi121' 2024-09-15T22:40:07.765 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:40:07.762+0000 7ff4d8fce640 1 -- 172.21.15.121:0/3606812683 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ff4bc0a3240 msgr2=0x7ff4bc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:07.765 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:40:07.762+0000 7ff4d8fce640 1 --2- 172.21.15.121:0/3606812683 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ff4bc0a3240 0x7ff4bc0a5700 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4101d40 tx=0x7ff4c8009110 comp rx=0 tx=0).stop 2024-09-15T22:40:07.765 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:40:07.762+0000 7ff4d8fce640 1 -- 172.21.15.121:0/3606812683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4d4102660 msgr2=0x7ff4d4101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:07.765 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:40:07.762+0000 7ff4d8fce640 1 --2- 172.21.15.121:0/3606812683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4d4102660 0x7ff4d4101fc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009230 tx=0x7ff4c4004430 comp rx=0 tx=0).stop 2024-09-15T22:40:07.765 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:40:07.762+0000 7ff4d8fce640 1 -- 172.21.15.121:0/3606812683 shutdown_connections 2024-09-15T22:40:07.765 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:40:07.762+0000 7ff4d8fce640 1 --2- 172.21.15.121:0/3606812683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4d41078c0 0x7ff4d4100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.765 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:40:07.762+0000 7ff4d8fce640 1 --2- 172.21.15.121:0/3606812683 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4d4104f90 0x7ff4d4100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.765 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:40:07.763+0000 7ff4d8fce640 1 --2- 172.21.15.121:0/3606812683 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ff4bc0a3240 0x7ff4bc0a5700 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:40:07.763+0000 7ff4d8fce640 1 --2- 172.21.15.121:0/3606812683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4d4102660 0x7ff4d4101fc0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:07.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:40:07.763+0000 7ff4d8fce640 1 -- 172.21.15.121:0/3606812683 >> 172.21.15.121:0/3606812683 conn(0x7ff4d40fc280 msgr2=0x7ff4d4103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:07.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:40:07.763+0000 7ff4d8fce640 1 -- 172.21.15.121:0/3606812683 shutdown_connections 2024-09-15T22:40:07.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:40:07.763+0000 7ff4d8fce640 1 -- 172.21.15.121:0/3606812683 wait complete. 2024-09-15T22:40:07.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:07 smithi192 ceph-mon[35427]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:40:07.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:07 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:07.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:07 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:07.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:07 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:40:07.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:07 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:40:07.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:07 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:07.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:07 smithi111 ceph-mon[30629]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:40:07.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:07 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:07.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:07 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:07.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:07 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:40:07.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:07 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:40:07.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:07 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:08.353 DEBUG:teuthology.orchestra.run.smithi121:osd.4> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.4.service 2024-09-15T22:40:08.355 INFO:tasks.cephadm:Deploying osd.5 on smithi192 with /dev/vg_nvme/lv_3... 2024-09-15T22:40:08.356 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-15T22:40:08.594 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:08 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:40:08.594 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:08 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:08.594 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:08 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:08.608 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:40:08.728 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:08 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:40:08.729 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:08 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:08.729 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:08 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:08.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:08 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:40:08.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:08 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:08.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:08 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:09.777 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:09 smithi121 ceph-mon[34815]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:40:09.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:09 smithi192 ceph-mon[35427]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:40:09.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:09 smithi111 ceph-mon[30629]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:40:10.504 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 15 22:40:10 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-4[46255]: 2024-09-15T22:40:10.159+0000 7fc2bb5b8640 -1 osd.4 0 log_to_monitors true 2024-09-15T22:40:10.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:10 smithi192 ceph-mon[35427]: from='osd.4 [v2:172.21.15.121:6808/2667595915,v1:172.21.15.121:6809/2667595915]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T22:40:10.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:10 smithi192 ceph-mon[35427]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T22:40:10.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:10 smithi121 ceph-mon[34815]: from='osd.4 [v2:172.21.15.121:6808/2667595915,v1:172.21.15.121:6809/2667595915]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T22:40:10.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:10 smithi121 ceph-mon[34815]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T22:40:10.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:10 smithi111 ceph-mon[30629]: from='osd.4 [v2:172.21.15.121:6808/2667595915,v1:172.21.15.121:6809/2667595915]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T22:40:10.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:10 smithi111 ceph-mon[30629]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T22:40:11.206 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T22:40:11.234 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi192:vg_nvme/lv_3 2024-09-15T22:40:11.482 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:40:11.777 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:11 smithi192 ceph-mon[35427]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:40:11.777 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:11 smithi192 ceph-mon[35427]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-15T22:40:11.777 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:11 smithi192 ceph-mon[35427]: osdmap e30: 5 total, 4 up, 5 in 2024-09-15T22:40:11.777 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:11 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:40:11.777 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:11 smithi192 ceph-mon[35427]: from='osd.4 [v2:172.21.15.121:6808/2667595915,v1:172.21.15.121:6809/2667595915]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-15T22:40:11.777 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:11 smithi192 ceph-mon[35427]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-15T22:40:11.777 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:11 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:11.777 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:11 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:11.778 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:11 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:40:11.778 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:11 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:40:11.778 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:11 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:11.778 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:11 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:40:11.778 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:11 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:40:11.778 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:11 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:11.890 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 15 22:40:11 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-4[46255]: 2024-09-15T22:40:11.518+0000 7fc2b6d18640 -1 osd.4 0 waiting for initial osdmap 2024-09-15T22:40:11.891 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 15 22:40:11 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-4[46255]: 2024-09-15T22:40:11.524+0000 7fc2b3352640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:40:11.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:11 smithi121 ceph-mon[34815]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:40:11.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:11 smithi121 ceph-mon[34815]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-15T22:40:11.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:11 smithi121 ceph-mon[34815]: osdmap e30: 5 total, 4 up, 5 in 2024-09-15T22:40:11.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:11 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:40:11.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:11 smithi121 ceph-mon[34815]: from='osd.4 [v2:172.21.15.121:6808/2667595915,v1:172.21.15.121:6809/2667595915]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-15T22:40:11.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:11 smithi121 ceph-mon[34815]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-15T22:40:11.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:11 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:11.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:11 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:11.892 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:11 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:40:11.892 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:11 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:40:11.892 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:11 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:11.892 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:11 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:40:11.892 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:11 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:40:11.892 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:11 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:11.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:11 smithi111 ceph-mon[30629]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:40:11.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:11 smithi111 ceph-mon[30629]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-15T22:40:11.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:11 smithi111 ceph-mon[30629]: osdmap e30: 5 total, 4 up, 5 in 2024-09-15T22:40:11.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:11 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:40:11.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:11 smithi111 ceph-mon[30629]: from='osd.4 [v2:172.21.15.121:6808/2667595915,v1:172.21.15.121:6809/2667595915]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-15T22:40:11.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:11 smithi111 ceph-mon[30629]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-15T22:40:11.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:11 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:11.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:11 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:11.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:11 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:40:11.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:11 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:40:11.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:11 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:11.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:11 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:40:11.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:11 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:40:11.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:11 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:12.005 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.001+0000 7f9240e9d640 1 -- 172.21.15.192:0/130571678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f923c1026c0 msgr2=0x7f923c104ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:12.005 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.001+0000 7f9240e9d640 1 --2- 172.21.15.192:0/130571678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f923c1026c0 0x7f923c104ab0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f922c009900 tx=0x7f922c02a410 comp rx=0 tx=0).stop 2024-09-15T22:40:12.005 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.001+0000 7f9240e9d640 1 -- 172.21.15.192:0/130571678 shutdown_connections 2024-09-15T22:40:12.005 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.001+0000 7f9240e9d640 1 --2- 172.21.15.192:0/130571678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f923c107920 0x7f923c109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:12.006 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.001+0000 7f9240e9d640 1 --2- 172.21.15.192:0/130571678 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f923c104ff0 0x7f923c1073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:12.006 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.001+0000 7f9240e9d640 1 --2- 172.21.15.192:0/130571678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f923c1026c0 0x7f923c104ab0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:12.006 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.001+0000 7f9240e9d640 1 -- 172.21.15.192:0/130571678 >> 172.21.15.192:0/130571678 conn(0x7f923c0fc280 msgr2=0x7f923c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:12.006 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.001+0000 7f9240e9d640 1 -- 172.21.15.192:0/130571678 shutdown_connections 2024-09-15T22:40:12.006 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.001+0000 7f9240e9d640 1 -- 172.21.15.192:0/130571678 wait complete. 2024-09-15T22:40:12.006 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.002+0000 7f9240e9d640 1 Processor -- start 2024-09-15T22:40:12.006 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.002+0000 7f9240e9d640 1 -- start start 2024-09-15T22:40:12.006 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.002+0000 7f9240e9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f923c1026c0 0x7f923c102020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:12.006 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.002+0000 7f9240e9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f923c104ff0 0x7f923c100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:12.006 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.002+0000 7f9240e9d640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f923c107920 0x7f923c100bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:12.006 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.002+0000 7f9240e9d640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f923c10d880 con 0x7f923c107920 2024-09-15T22:40:12.007 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.002+0000 7f9240e9d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f923c10d700 con 0x7f923c1026c0 2024-09-15T22:40:12.007 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.003+0000 7f9240e9d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f923c10da00 con 0x7f923c104ff0 2024-09-15T22:40:12.007 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.003+0000 7f9239d9b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f923c104ff0 0x7f923c100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:12.007 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.003+0000 7f9239d9b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f923c104ff0 0x7f923c100670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:60862/0 (socket says 172.21.15.192:60862) 2024-09-15T22:40:12.007 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.003+0000 7f9239d9b640 1 -- 172.21.15.192:0/2151492884 learned_addr learned my addr 172.21.15.192:0/2151492884 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-09-15T22:40:12.007 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.003+0000 7f923a59c640 1 --2- 172.21.15.192:0/2151492884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f923c1026c0 0x7f923c102020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:12.007 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.003+0000 7f923ad9d640 1 --2- 172.21.15.192:0/2151492884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f923c107920 0x7f923c100bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:12.007 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.003+0000 7f9239d9b640 1 -- 172.21.15.192:0/2151492884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f923c1026c0 msgr2=0x7f923c102020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:12.008 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.003+0000 7f9239d9b640 1 --2- 172.21.15.192:0/2151492884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f923c1026c0 0x7f923c102020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:12.008 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.003+0000 7f9239d9b640 1 -- 172.21.15.192:0/2151492884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f923c107920 msgr2=0x7f923c100bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:12.008 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.003+0000 7f9239d9b640 1 --2- 172.21.15.192:0/2151492884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f923c107920 0x7f923c100bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:12.008 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.003+0000 7f9239d9b640 1 -- 172.21.15.192:0/2151492884 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f923c101470 con 0x7f923c104ff0 2024-09-15T22:40:12.008 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.003+0000 7f9239d9b640 1 --2- 172.21.15.192:0/2151492884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f923c104ff0 0x7f923c100670 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f923000d740 tx=0x7f923000dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:12.008 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.003+0000 7f922b7fe640 1 -- 172.21.15.192:0/2151492884 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9230017070 con 0x7f923c104ff0 2024-09-15T22:40:12.008 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.003+0000 7f922b7fe640 1 -- 172.21.15.192:0/2151492884 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f923000de50 con 0x7f923c104ff0 2024-09-15T22:40:12.008 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.003+0000 7f922b7fe640 1 -- 172.21.15.192:0/2151492884 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9230004e00 con 0x7f923c104ff0 2024-09-15T22:40:12.008 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.003+0000 7f9240e9d640 1 -- 172.21.15.192:0/2151492884 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f923c1a20c0 con 0x7f923c104ff0 2024-09-15T22:40:12.009 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.003+0000 7f9240e9d640 1 -- 172.21.15.192:0/2151492884 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f923c1a2580 con 0x7f923c104ff0 2024-09-15T22:40:12.009 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.005+0000 7f9240e9d640 1 -- 172.21.15.192:0/2151492884 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f923c0fdd70 con 0x7f923c104ff0 2024-09-15T22:40:12.010 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.006+0000 7f922b7fe640 1 -- 172.21.15.192:0/2151492884 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f9230004fa0 con 0x7f923c104ff0 2024-09-15T22:40:12.010 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.007+0000 7f922b7fe640 1 --2- 172.21.15.192:0/2151492884 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f92240a32c0 0x7f92240a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:12.010 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.007+0000 7f922b7fe640 1 -- 172.21.15.192:0/2151492884 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3821+0+0 (secure 0 0 0) 0x7f92300ce510 con 0x7f923c104ff0 2024-09-15T22:40:12.010 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.007+0000 7f923a59c640 1 --2- 172.21.15.192:0/2151492884 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f92240a32c0 0x7f92240a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:12.010 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.008+0000 7f923a59c640 1 --2- 172.21.15.192:0/2151492884 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f92240a32c0 0x7f92240a5780 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f922c000c00 tx=0x7f922c0315d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:12.011 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.010+0000 7f922b7fe640 1 -- 172.21.15.192:0/2151492884 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f923009c440 con 0x7f923c104ff0 2024-09-15T22:40:12.187 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:12.185+0000 7f9240e9d640 1 -- 172.21.15.192:0/2151492884 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f923c062630 con 0x7f92240a32c0 2024-09-15T22:40:12.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:12 smithi192 ceph-mon[35427]: Detected new or changed devices on smithi121 2024-09-15T22:40:12.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:12 smithi192 ceph-mon[35427]: Adjusting osd_memory_target on smithi121 to 8543M 2024-09-15T22:40:12.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:12 smithi192 ceph-mon[35427]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-15T22:40:12.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:12 smithi192 ceph-mon[35427]: osdmap e31: 5 total, 4 up, 5 in 2024-09-15T22:40:12.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:12 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:40:12.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:12 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:40:12.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:12 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:40:12.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:12 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:40:12.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:12 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:40:12.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:12 smithi121 ceph-mon[34815]: Detected new or changed devices on smithi121 2024-09-15T22:40:12.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:12 smithi121 ceph-mon[34815]: Adjusting osd_memory_target on smithi121 to 8543M 2024-09-15T22:40:12.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:12 smithi121 ceph-mon[34815]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-15T22:40:12.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:12 smithi121 ceph-mon[34815]: osdmap e31: 5 total, 4 up, 5 in 2024-09-15T22:40:12.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:12 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:40:12.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:12 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:40:12.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:12 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:40:12.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:12 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:40:12.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:12 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:40:12.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:12 smithi111 ceph-mon[30629]: Detected new or changed devices on smithi121 2024-09-15T22:40:12.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:12 smithi111 ceph-mon[30629]: Adjusting osd_memory_target on smithi121 to 8543M 2024-09-15T22:40:12.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:12 smithi111 ceph-mon[30629]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-15T22:40:12.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:12 smithi111 ceph-mon[30629]: osdmap e31: 5 total, 4 up, 5 in 2024-09-15T22:40:12.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:12 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:40:12.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:12 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:40:12.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:12 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:40:12.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:12 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:40:12.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:12 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:40:13.778 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:13 smithi192 ceph-mon[35427]: purged_snaps scrub starts 2024-09-15T22:40:13.778 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:13 smithi192 ceph-mon[35427]: purged_snaps scrub ok 2024-09-15T22:40:13.778 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:13 smithi192 ceph-mon[35427]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:40:13.778 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:13 smithi192 ceph-mon[35427]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:40:13.779 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:13 smithi192 ceph-mon[35427]: osd.4 [v2:172.21.15.121:6808/2667595915,v1:172.21.15.121:6809/2667595915] boot 2024-09-15T22:40:13.779 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:13 smithi192 ceph-mon[35427]: osdmap e32: 5 total, 5 up, 5 in 2024-09-15T22:40:13.779 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:13 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:40:13.817 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:13 smithi111 ceph-mon[30629]: purged_snaps scrub starts 2024-09-15T22:40:13.818 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:13 smithi111 ceph-mon[30629]: purged_snaps scrub ok 2024-09-15T22:40:13.818 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:13 smithi111 ceph-mon[30629]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:40:13.818 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:13 smithi111 ceph-mon[30629]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:40:13.818 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:13 smithi111 ceph-mon[30629]: osd.4 [v2:172.21.15.121:6808/2667595915,v1:172.21.15.121:6809/2667595915] boot 2024-09-15T22:40:13.818 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:13 smithi111 ceph-mon[30629]: osdmap e32: 5 total, 5 up, 5 in 2024-09-15T22:40:13.818 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:13 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:40:13.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:13 smithi121 ceph-mon[34815]: purged_snaps scrub starts 2024-09-15T22:40:13.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:13 smithi121 ceph-mon[34815]: purged_snaps scrub ok 2024-09-15T22:40:13.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:13 smithi121 ceph-mon[34815]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:40:13.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:13 smithi121 ceph-mon[34815]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:40:13.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:13 smithi121 ceph-mon[34815]: osd.4 [v2:172.21.15.121:6808/2667595915,v1:172.21.15.121:6809/2667595915] boot 2024-09-15T22:40:13.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:13 smithi121 ceph-mon[34815]: osdmap e32: 5 total, 5 up, 5 in 2024-09-15T22:40:13.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:13 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:40:14.792 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:14 smithi192 ceph-mon[35427]: osdmap e33: 5 total, 5 up, 5 in 2024-09-15T22:40:14.792 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:14 smithi192 ceph-mon[35427]: from='client.? 172.21.15.192:0/519446219' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9290e769-5470-44d3-8d24-25093af34335"}]: dispatch 2024-09-15T22:40:14.792 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:14 smithi192 ceph-mon[35427]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9290e769-5470-44d3-8d24-25093af34335"}]: dispatch 2024-09-15T22:40:14.792 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:14 smithi192 ceph-mon[35427]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9290e769-5470-44d3-8d24-25093af34335"}]': finished 2024-09-15T22:40:14.792 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:14 smithi192 ceph-mon[35427]: osdmap e34: 6 total, 5 up, 6 in 2024-09-15T22:40:14.792 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:14 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:40:14.842 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:14 smithi111 ceph-mon[30629]: osdmap e33: 5 total, 5 up, 5 in 2024-09-15T22:40:14.842 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:14 smithi111 ceph-mon[30629]: from='client.? 172.21.15.192:0/519446219' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9290e769-5470-44d3-8d24-25093af34335"}]: dispatch 2024-09-15T22:40:14.842 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:14 smithi111 ceph-mon[30629]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9290e769-5470-44d3-8d24-25093af34335"}]: dispatch 2024-09-15T22:40:14.842 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:14 smithi111 ceph-mon[30629]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9290e769-5470-44d3-8d24-25093af34335"}]': finished 2024-09-15T22:40:14.842 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:14 smithi111 ceph-mon[30629]: osdmap e34: 6 total, 5 up, 6 in 2024-09-15T22:40:14.842 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:14 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:40:14.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:14 smithi121 ceph-mon[34815]: osdmap e33: 5 total, 5 up, 5 in 2024-09-15T22:40:14.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:14 smithi121 ceph-mon[34815]: from='client.? 172.21.15.192:0/519446219' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9290e769-5470-44d3-8d24-25093af34335"}]: dispatch 2024-09-15T22:40:14.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:14 smithi121 ceph-mon[34815]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9290e769-5470-44d3-8d24-25093af34335"}]: dispatch 2024-09-15T22:40:14.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:14 smithi121 ceph-mon[34815]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9290e769-5470-44d3-8d24-25093af34335"}]': finished 2024-09-15T22:40:14.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:14 smithi121 ceph-mon[34815]: osdmap e34: 6 total, 5 up, 6 in 2024-09-15T22:40:14.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:14 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:40:15.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:15 smithi121 ceph-mon[34815]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:15.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:15 smithi121 ceph-mon[34815]: from='client.? 172.21.15.192:0/2676103238' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:40:15.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:15 smithi111 ceph-mon[30629]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:15.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:15 smithi111 ceph-mon[30629]: from='client.? 172.21.15.192:0/2676103238' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:40:16.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:15 smithi192 ceph-mon[35427]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:16.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:15 smithi192 ceph-mon[35427]: from='client.? 172.21.15.192:0/2676103238' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:40:17.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:17 smithi192 ceph-mon[35427]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:17.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:17 smithi121 ceph-mon[34815]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:17.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:17 smithi111 ceph-mon[30629]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:19.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:19 smithi121 ceph-mon[34815]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:19.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:19 smithi111 ceph-mon[30629]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:19.945 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:19 smithi192 ceph-mon[35427]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:21.754 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:21 smithi192 ceph-mon[35427]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:21.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:21 smithi121 ceph-mon[34815]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:21.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:21 smithi111 ceph-mon[30629]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:23.695 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:23 smithi192 ceph-mon[35427]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:23.696 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:23 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-15T22:40:23.696 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:23 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:40:23.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:23 smithi121 ceph-mon[34815]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:23.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:23 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-15T22:40:23.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:23 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:40:23.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:23 smithi111 ceph-mon[30629]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:23.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:23 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-15T22:40:23.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:23 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:40:24.606 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:24 smithi192 ceph-mon[35427]: Deploying daemon osd.5 on smithi192 2024-09-15T22:40:24.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:24 smithi121 ceph-mon[34815]: Deploying daemon osd.5 on smithi192 2024-09-15T22:40:24.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:24 smithi111 ceph-mon[30629]: Deploying daemon osd.5 on smithi192 2024-09-15T22:40:25.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:25 smithi192 ceph-mon[35427]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:25.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:25 smithi121 ceph-mon[34815]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:25.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:25 smithi111 ceph-mon[30629]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:27.742 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:27 smithi192 ceph-mon[35427]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:27.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:27 smithi121 ceph-mon[34815]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:27.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:27 smithi111 ceph-mon[30629]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:29.620 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:29 smithi192 ceph-mon[35427]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:29.620 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:29 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:40:29.620 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:29 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:29.620 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:29 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:29.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:29 smithi121 ceph-mon[34815]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:29.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:29 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:40:29.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:29 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:29.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:29 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:29.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:29 smithi111 ceph-mon[30629]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:29.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:29 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:40:29.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:29 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:29.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:29 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:31.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:31 smithi121 ceph-mon[34815]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:31.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:31 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:31.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:31 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:31.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:31 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:40:31.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:31 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:40:31.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:31 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:31.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:31 smithi111 ceph-mon[30629]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:31.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:31 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:31.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:31 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:31.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:31 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:40:31.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:31 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:40:31.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:31 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:32.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:31 smithi192 ceph-mon[35427]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:32.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:31 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:32.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:31 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:32.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:31 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:40:32.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:31 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:40:32.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:31 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:32.926 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:32.924+0000 7f922b7fe640 1 -- 172.21.15.192:0/2151492884 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f923c062630 con 0x7f92240a32c0 2024-09-15T22:40:32.926 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 5 on host 'smithi192' 2024-09-15T22:40:32.929 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:32.927+0000 7f9240e9d640 1 -- 172.21.15.192:0/2151492884 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f92240a32c0 msgr2=0x7f92240a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:32.929 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:32.927+0000 7f9240e9d640 1 --2- 172.21.15.192:0/2151492884 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f92240a32c0 0x7f92240a5780 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f922c000c00 tx=0x7f922c0315d0 comp rx=0 tx=0).stop 2024-09-15T22:40:32.929 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:32.927+0000 7f9240e9d640 1 -- 172.21.15.192:0/2151492884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f923c104ff0 msgr2=0x7f923c100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:32.929 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:32.927+0000 7f9240e9d640 1 --2- 172.21.15.192:0/2151492884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f923c104ff0 0x7f923c100670 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f923000d740 tx=0x7f923000dc00 comp rx=0 tx=0).stop 2024-09-15T22:40:32.929 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:32.927+0000 7f9240e9d640 1 -- 172.21.15.192:0/2151492884 shutdown_connections 2024-09-15T22:40:32.929 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:32.927+0000 7f9240e9d640 1 --2- 172.21.15.192:0/2151492884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f923c107920 0x7f923c100bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:32.929 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:32.927+0000 7f9240e9d640 1 --2- 172.21.15.192:0/2151492884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f923c104ff0 0x7f923c100670 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:32.929 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:32.927+0000 7f9240e9d640 1 --2- 172.21.15.192:0/2151492884 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f92240a32c0 0x7f92240a5780 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:32.929 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:32.927+0000 7f9240e9d640 1 --2- 172.21.15.192:0/2151492884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f923c1026c0 0x7f923c102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:32.929 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:32.927+0000 7f9240e9d640 1 -- 172.21.15.192:0/2151492884 >> 172.21.15.192:0/2151492884 conn(0x7f923c0fc280 msgr2=0x7f923c1030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:32.929 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:32.927+0000 7f9240e9d640 1 -- 172.21.15.192:0/2151492884 shutdown_connections 2024-09-15T22:40:32.930 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:40:32.927+0000 7f9240e9d640 1 -- 172.21.15.192:0/2151492884 wait complete. 2024-09-15T22:40:33.582 DEBUG:teuthology.orchestra.run.smithi192:osd.5> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.5.service 2024-09-15T22:40:33.585 INFO:tasks.cephadm:Deploying osd.6 on smithi111 with /dev/vg_nvme/lv_2... 2024-09-15T22:40:33.585 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-15T22:40:33.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:33 smithi192 ceph-mon[35427]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:33.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:33 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:40:33.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:33 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:33.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:33 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:33.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:33 smithi121 ceph-mon[34815]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:33.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:33 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:40:33.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:33 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:33.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:33 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:33.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:33 smithi111 ceph-mon[30629]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:33.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:33 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:40:33.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:33 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:33.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:33 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:33.955 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:40:35.565 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 22:40:35 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T22:40:35.249+0000 7fb4d18c5640 -1 osd.5 0 log_to_monitors true 2024-09-15T22:40:35.788 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:35 smithi111 ceph-mon[30629]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:35.788 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:35 smithi111 ceph-mon[30629]: from='osd.5 [v2:172.21.15.192:6808/1958639856,v1:172.21.15.192:6809/1958639856]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T22:40:35.789 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:35 smithi111 ceph-mon[30629]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T22:40:35.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:35 smithi121 ceph-mon[34815]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:35.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:35 smithi121 ceph-mon[34815]: from='osd.5 [v2:172.21.15.192:6808/1958639856,v1:172.21.15.192:6809/1958639856]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T22:40:35.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:35 smithi121 ceph-mon[34815]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T22:40:36.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:35 smithi192 ceph-mon[35427]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:36.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:35 smithi192 ceph-mon[35427]: from='osd.5 [v2:172.21.15.192:6808/1958639856,v1:172.21.15.192:6809/1958639856]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T22:40:36.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:35 smithi192 ceph-mon[35427]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T22:40:36.815 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 22:40:36 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T22:40:36.611+0000 7fb4cd025640 -1 osd.5 0 waiting for initial osdmap 2024-09-15T22:40:36.815 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 22:40:36 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T22:40:36.617+0000 7fb4c965f640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:40:36.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:36 smithi192 ceph-mon[35427]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-15T22:40:36.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:36 smithi192 ceph-mon[35427]: osdmap e35: 6 total, 5 up, 6 in 2024-09-15T22:40:36.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:36 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:40:36.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:36 smithi192 ceph-mon[35427]: from='osd.5 [v2:172.21.15.192:6808/1958639856,v1:172.21.15.192:6809/1958639856]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-09-15T22:40:36.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:36 smithi192 ceph-mon[35427]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-09-15T22:40:36.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:36 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:36.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:36 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:36.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:36 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:40:36.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:36 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:40:36.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:36 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:36.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:36 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:40:36.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:36 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:40:36.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:36 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:36.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:36 smithi121 ceph-mon[34815]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-15T22:40:36.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:36 smithi121 ceph-mon[34815]: osdmap e35: 6 total, 5 up, 6 in 2024-09-15T22:40:36.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:36 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:40:36.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:36 smithi121 ceph-mon[34815]: from='osd.5 [v2:172.21.15.192:6808/1958639856,v1:172.21.15.192:6809/1958639856]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-09-15T22:40:36.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:36 smithi121 ceph-mon[34815]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-09-15T22:40:36.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:36 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:36.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:36 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:36.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:36 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:40:36.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:36 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:40:36.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:36 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:36.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:36 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:40:36.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:36 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:40:36.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:36 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:36.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:36 smithi111 ceph-mon[30629]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-15T22:40:36.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:36 smithi111 ceph-mon[30629]: osdmap e35: 6 total, 5 up, 6 in 2024-09-15T22:40:36.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:36 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:40:36.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:36 smithi111 ceph-mon[30629]: from='osd.5 [v2:172.21.15.192:6808/1958639856,v1:172.21.15.192:6809/1958639856]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-09-15T22:40:36.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:36 smithi111 ceph-mon[30629]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-09-15T22:40:36.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:36 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:36.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:36 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:36.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:36 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:40:36.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:36 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:40:36.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:36 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:36.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:36 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:40:36.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:36 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:40:36.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:36 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:40:37.208 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:40:37.240 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_2 2024-09-15T22:40:37.628 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:40:37.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:37 smithi121 ceph-mon[34815]: Detected new or changed devices on smithi192 2024-09-15T22:40:37.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:37 smithi121 ceph-mon[34815]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:37.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:37 smithi121 ceph-mon[34815]: Adjusting osd_memory_target on smithi192 to 8544M 2024-09-15T22:40:37.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:37 smithi121 ceph-mon[34815]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-09-15T22:40:37.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:37 smithi121 ceph-mon[34815]: osdmap e36: 6 total, 5 up, 6 in 2024-09-15T22:40:37.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:40:37.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:40:37.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:37 smithi111 ceph-mon[30629]: Detected new or changed devices on smithi192 2024-09-15T22:40:37.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:37 smithi111 ceph-mon[30629]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:37.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:37 smithi111 ceph-mon[30629]: Adjusting osd_memory_target on smithi192 to 8544M 2024-09-15T22:40:37.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:37 smithi111 ceph-mon[30629]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-09-15T22:40:37.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:37 smithi111 ceph-mon[30629]: osdmap e36: 6 total, 5 up, 6 in 2024-09-15T22:40:37.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:37 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:40:37.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:37 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:40:38.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:37 smithi192 ceph-mon[35427]: Detected new or changed devices on smithi192 2024-09-15T22:40:38.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:37 smithi192 ceph-mon[35427]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:38.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:37 smithi192 ceph-mon[35427]: Adjusting osd_memory_target on smithi192 to 8544M 2024-09-15T22:40:38.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:37 smithi192 ceph-mon[35427]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-09-15T22:40:38.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:37 smithi192 ceph-mon[35427]: osdmap e36: 6 total, 5 up, 6 in 2024-09-15T22:40:38.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:37 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:40:38.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:37 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:40:38.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.198+0000 7f09e0c00640 1 -- 172.21.15.111:0/1791451024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09dc103e00 msgr2=0x7f09dc104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:38.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.198+0000 7f09e0c00640 1 --2- 172.21.15.111:0/1791451024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09dc103e00 0x7f09dc104200 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009900 tx=0x7f09cc02a3f0 comp rx=0 tx=0).stop 2024-09-15T22:40:38.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.199+0000 7f09e0c00640 1 -- 172.21.15.111:0/1791451024 shutdown_connections 2024-09-15T22:40:38.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.199+0000 7f09e0c00640 1 --2- 172.21.15.111:0/1791451024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09dc105920 0x7f09dc10c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:38.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.199+0000 7f09e0c00640 1 --2- 172.21.15.111:0/1791451024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09dc104f60 0x7f09dc1053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:38.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.199+0000 7f09e0c00640 1 --2- 172.21.15.111:0/1791451024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09dc103e00 0x7f09dc104200 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:38.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.199+0000 7f09e0c00640 1 -- 172.21.15.111:0/1791451024 >> 172.21.15.111:0/1791451024 conn(0x7f09dc0ff5b0 msgr2=0x7f09dc1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:40:38.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.199+0000 7f09e0c00640 1 -- 172.21.15.111:0/1791451024 shutdown_connections 2024-09-15T22:40:38.201 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.199+0000 7f09e0c00640 1 -- 172.21.15.111:0/1791451024 wait complete. 2024-09-15T22:40:38.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.200+0000 7f09e0c00640 1 Processor -- start 2024-09-15T22:40:38.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.200+0000 7f09e0c00640 1 -- start start 2024-09-15T22:40:38.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.200+0000 7f09e0c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09dc103e00 0x7f09dc19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:38.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.200+0000 7f09e0c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09dc104f60 0x7f09dc19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:38.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.200+0000 7f09e0c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09dc105920 0x7f09dc1a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:38.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.200+0000 7f09e0c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f09dc10fab0 con 0x7f09dc104f60 2024-09-15T22:40:38.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.200+0000 7f09e0c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f09dc10f930 con 0x7f09dc105920 2024-09-15T22:40:38.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.200+0000 7f09e0c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f09dc10fc30 con 0x7f09dc103e00 2024-09-15T22:40:38.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.200+0000 7f09d1400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09dc104f60 0x7f09dc19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:38.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.200+0000 7f09d9e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09dc103e00 0x7f09dc19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:38.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.200+0000 7f09d1400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09dc104f60 0x7f09dc19a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:37724/0 (socket says 172.21.15.111:37724) 2024-09-15T22:40:38.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.200+0000 7f09d9e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09dc103e00 0x7f09dc19a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:60420/0 (socket says 172.21.15.111:60420) 2024-09-15T22:40:38.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.200+0000 7f09d1400640 1 -- 172.21.15.111:0/1304956434 learned_addr learned my addr 172.21.15.111:0/1304956434 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:40:38.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.200+0000 7f09da800640 1 --2- 172.21.15.111:0/1304956434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09dc105920 0x7f09dc1a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:38.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.201+0000 7f09d1400640 1 -- 172.21.15.111:0/1304956434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09dc103e00 msgr2=0x7f09dc19a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:38.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.201+0000 7f09d1400640 1 --2- 172.21.15.111:0/1304956434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09dc103e00 0x7f09dc19a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:38.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.201+0000 7f09d1400640 1 -- 172.21.15.111:0/1304956434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09dc105920 msgr2=0x7f09dc1a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:40:38.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.201+0000 7f09d1400640 1 --2- 172.21.15.111:0/1304956434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09dc105920 0x7f09dc1a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:40:38.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.201+0000 7f09d1400640 1 -- 172.21.15.111:0/1304956434 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09dc1a1d30 con 0x7f09dc104f60 2024-09-15T22:40:38.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.201+0000 7f09d1400640 1 --2- 172.21.15.111:0/1304956434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09dc104f60 0x7f09dc19a640 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f09c400d950 tx=0x7f09c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:38.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.201+0000 7f09d3400640 1 -- 172.21.15.111:0/1304956434 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c4017070 con 0x7f09dc104f60 2024-09-15T22:40:38.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.201+0000 7f09d3400640 1 -- 172.21.15.111:0/1304956434 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c4009c30 con 0x7f09dc104f60 2024-09-15T22:40:38.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.201+0000 7f09d3400640 1 -- 172.21.15.111:0/1304956434 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c400f600 con 0x7f09dc104f60 2024-09-15T22:40:38.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.201+0000 7f09e0c00640 1 -- 172.21.15.111:0/1304956434 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09dc1a1fc0 con 0x7f09dc104f60 2024-09-15T22:40:38.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.202+0000 7f09e0c00640 1 -- 172.21.15.111:0/1304956434 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09dc1a2480 con 0x7f09dc104f60 2024-09-15T22:40:38.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.203+0000 7f09e0c00640 1 -- 172.21.15.111:0/1304956434 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09dc10b1a0 con 0x7f09dc104f60 2024-09-15T22:40:38.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.205+0000 7f09d3400640 1 -- 172.21.15.111:0/1304956434 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f09c401a070 con 0x7f09dc104f60 2024-09-15T22:40:38.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.206+0000 7f09d3400640 1 --2- 172.21.15.111:0/1304956434 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f09c00a32f0 0x7f09c00a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:40:38.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.206+0000 7f09d3400640 1 -- 172.21.15.111:0/1304956434 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4566+0+0 (secure 0 0 0) 0x7f09c40ce3b0 con 0x7f09dc104f60 2024-09-15T22:40:38.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.206+0000 7f09d9e00640 1 --2- 172.21.15.111:0/1304956434 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f09c00a32f0 0x7f09c00a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:40:38.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.207+0000 7f09d9e00640 1 --2- 172.21.15.111:0/1304956434 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f09c00a32f0 0x7f09c00a57b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f09cc0314e0 tx=0x7f09cc000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:40:38.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.209+0000 7f09d3400640 1 -- 172.21.15.111:0/1304956434 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09c409b900 con 0x7f09dc104f60 2024-09-15T22:40:38.393 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:40:38.391+0000 7f09e0c00640 1 -- 172.21.15.111:0/1304956434 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f09dc062630 con 0x7f09c00a32f0 2024-09-15T22:40:38.892 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:38 smithi121 ceph-mon[34815]: purged_snaps scrub starts 2024-09-15T22:40:38.892 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:38 smithi121 ceph-mon[34815]: purged_snaps scrub ok 2024-09-15T22:40:38.892 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:38 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:40:38.892 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:38 smithi121 ceph-mon[34815]: osd.5 [v2:172.21.15.192:6808/1958639856,v1:172.21.15.192:6809/1958639856] boot 2024-09-15T22:40:38.892 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:38 smithi121 ceph-mon[34815]: osdmap e37: 6 total, 6 up, 6 in 2024-09-15T22:40:38.892 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:38 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:40:38.892 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:38 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:40:38.892 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:38 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:40:38.892 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:38 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:40:38.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:38 smithi111 ceph-mon[30629]: purged_snaps scrub starts 2024-09-15T22:40:38.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:38 smithi111 ceph-mon[30629]: purged_snaps scrub ok 2024-09-15T22:40:38.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:38 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:40:38.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:38 smithi111 ceph-mon[30629]: osd.5 [v2:172.21.15.192:6808/1958639856,v1:172.21.15.192:6809/1958639856] boot 2024-09-15T22:40:38.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:38 smithi111 ceph-mon[30629]: osdmap e37: 6 total, 6 up, 6 in 2024-09-15T22:40:38.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:38 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:40:38.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:38 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:40:38.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:38 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:40:38.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:38 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:40:39.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:38 smithi192 ceph-mon[35427]: purged_snaps scrub starts 2024-09-15T22:40:39.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:38 smithi192 ceph-mon[35427]: purged_snaps scrub ok 2024-09-15T22:40:39.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:38 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:40:39.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:38 smithi192 ceph-mon[35427]: osd.5 [v2:172.21.15.192:6808/1958639856,v1:172.21.15.192:6809/1958639856] boot 2024-09-15T22:40:39.066 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:38 smithi192 ceph-mon[35427]: osdmap e37: 6 total, 6 up, 6 in 2024-09-15T22:40:39.066 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:38 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:40:39.066 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:38 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:40:39.066 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:38 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:40:39.066 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:38 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:40:39.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:39 smithi121 ceph-mon[34815]: from='client.14322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:40:39.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:39 smithi121 ceph-mon[34815]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:39.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:39 smithi121 ceph-mon[34815]: osdmap e38: 6 total, 6 up, 6 in 2024-09-15T22:40:39.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:39 smithi111 ceph-mon[30629]: from='client.14322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:40:39.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:39 smithi111 ceph-mon[30629]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:39.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:39 smithi111 ceph-mon[30629]: osdmap e38: 6 total, 6 up, 6 in 2024-09-15T22:40:40.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:39 smithi192 ceph-mon[35427]: from='client.14322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:40:40.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:39 smithi192 ceph-mon[35427]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:40:40.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:39 smithi192 ceph-mon[35427]: osdmap e38: 6 total, 6 up, 6 in 2024-09-15T22:40:40.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:40 smithi111 ceph-mon[30629]: osdmap e39: 6 total, 6 up, 6 in 2024-09-15T22:40:41.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:40 smithi192 ceph-mon[35427]: osdmap e39: 6 total, 6 up, 6 in 2024-09-15T22:40:41.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:40 smithi121 ceph-mon[34815]: osdmap e39: 6 total, 6 up, 6 in 2024-09-15T22:40:41.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:41 smithi111 ceph-mon[30629]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:40:41.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:41 smithi111 ceph-mon[30629]: osdmap e40: 6 total, 6 up, 6 in 2024-09-15T22:40:41.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:41 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/2708399674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18911b97-1c68-4e8f-92ec-35d2ff0b5f19"}]: dispatch 2024-09-15T22:40:41.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:41 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/2708399674' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18911b97-1c68-4e8f-92ec-35d2ff0b5f19"}]': finished 2024-09-15T22:40:41.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:41 smithi111 ceph-mon[30629]: osdmap e41: 7 total, 6 up, 7 in 2024-09-15T22:40:41.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:41 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:40:42.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:41 smithi192 ceph-mon[35427]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:40:42.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:41 smithi192 ceph-mon[35427]: osdmap e40: 6 total, 6 up, 6 in 2024-09-15T22:40:42.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:41 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/2708399674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18911b97-1c68-4e8f-92ec-35d2ff0b5f19"}]: dispatch 2024-09-15T22:40:42.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:41 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/2708399674' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18911b97-1c68-4e8f-92ec-35d2ff0b5f19"}]': finished 2024-09-15T22:40:42.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:41 smithi192 ceph-mon[35427]: osdmap e41: 7 total, 6 up, 7 in 2024-09-15T22:40:42.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:41 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:40:42.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:41 smithi121 ceph-mon[34815]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:40:42.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:41 smithi121 ceph-mon[34815]: osdmap e40: 6 total, 6 up, 6 in 2024-09-15T22:40:42.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:41 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/2708399674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18911b97-1c68-4e8f-92ec-35d2ff0b5f19"}]: dispatch 2024-09-15T22:40:42.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:41 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/2708399674' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18911b97-1c68-4e8f-92ec-35d2ff0b5f19"}]': finished 2024-09-15T22:40:42.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:41 smithi121 ceph-mon[34815]: osdmap e41: 7 total, 6 up, 7 in 2024-09-15T22:40:42.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:41 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:40:43.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:42 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/189683378' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:40:43.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:42 smithi192 ceph-mon[35427]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:40:43.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:42 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/189683378' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:40:43.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:42 smithi121 ceph-mon[34815]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:40:43.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:42 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/189683378' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:40:43.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:42 smithi111 ceph-mon[30629]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:40:45.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:45 smithi192 ceph-mon[35427]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-15T22:40:45.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:45 smithi121 ceph-mon[34815]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-15T22:40:45.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:45 smithi111 ceph-mon[30629]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-15T22:40:47.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:47 smithi192 ceph-mon[35427]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-15T22:40:47.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:47 smithi121 ceph-mon[34815]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-15T22:40:47.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:47 smithi111 ceph-mon[30629]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-15T22:40:49.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:49 smithi192 ceph-mon[35427]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-15T22:40:49.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:49 smithi121 ceph-mon[34815]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-15T22:40:49.892 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:49 smithi111 ceph-mon[30629]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-15T22:40:51.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:51 smithi192 ceph-mon[35427]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-15T22:40:51.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:51 smithi121 ceph-mon[34815]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-15T22:40:51.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:51 smithi111 ceph-mon[30629]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-15T22:40:52.728 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:52 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-15T22:40:52.728 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:52 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:40:52.728 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:52 smithi111 ceph-mon[30629]: Deploying daemon osd.6 on smithi111 2024-09-15T22:40:52.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:52 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-15T22:40:52.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:52 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:40:52.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:52 smithi192 ceph-mon[35427]: Deploying daemon osd.6 on smithi111 2024-09-15T22:40:52.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:52 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-15T22:40:52.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:52 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:40:52.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:52 smithi121 ceph-mon[34815]: Deploying daemon osd.6 on smithi111 2024-09-15T22:40:53.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:53 smithi192 ceph-mon[35427]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-15T22:40:53.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:53 smithi121 ceph-mon[34815]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-15T22:40:53.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:53 smithi111 ceph-mon[30629]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-15T22:40:55.627 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:55 smithi111 ceph-mon[30629]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:40:55.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:55 smithi192 ceph-mon[35427]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:40:55.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:55 smithi121 ceph-mon[34815]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:40:57.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:57 smithi192 ceph-mon[35427]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:40:57.839 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:57 smithi111 ceph-mon[30629]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:40:57.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:57 smithi121 ceph-mon[34815]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:40:59.643 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:40:59 smithi111 ceph-mon[30629]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:40:59.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:40:59 smithi192 ceph-mon[35427]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:40:59.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:40:59 smithi121 ceph-mon[34815]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:41:00.721 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:00 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:41:00.722 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:00 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:00.722 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:00 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:00.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:00 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:41:00.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:00 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:00.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:00 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:00.892 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:00 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:41:00.892 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:00 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:00.893 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:00 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:01.566 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:01 smithi111 ceph-mon[30629]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:41:01.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:01 smithi192 ceph-mon[35427]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:41:01.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:01 smithi121 ceph-mon[34815]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:41:03.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:03 smithi192 ceph-mon[35427]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:41:03.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:03 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:03.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:03 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:03.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:03 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:03.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:03 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:41:03.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:03 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:03.861 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:03 smithi111 ceph-mon[30629]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:41:03.861 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:03 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:03.861 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:03 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:03.861 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:03 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:03.861 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:03 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:41:03.861 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:03 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:03.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:03 smithi121 ceph-mon[34815]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:41:03.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:03 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:03.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:03 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:03.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:03 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:03.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:03 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:41:03.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:03 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:04.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:04.316+0000 7f09d3400640 1 -- 172.21.15.111:0/1304956434 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f09dc062630 con 0x7f09c00a32f0 2024-09-15T22:41:04.318 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 6 on host 'smithi111' 2024-09-15T22:41:04.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:04.318+0000 7f09e0c00640 1 -- 172.21.15.111:0/1304956434 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f09c00a32f0 msgr2=0x7f09c00a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:04.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:04.318+0000 7f09e0c00640 1 --2- 172.21.15.111:0/1304956434 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f09c00a32f0 0x7f09c00a57b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f09cc0314e0 tx=0x7f09cc000bc0 comp rx=0 tx=0).stop 2024-09-15T22:41:04.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:04.318+0000 7f09e0c00640 1 -- 172.21.15.111:0/1304956434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09dc104f60 msgr2=0x7f09dc19a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:04.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:04.318+0000 7f09e0c00640 1 --2- 172.21.15.111:0/1304956434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09dc104f60 0x7f09dc19a640 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f09c400d950 tx=0x7f09c400de10 comp rx=0 tx=0).stop 2024-09-15T22:41:04.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:04.318+0000 7f09e0c00640 1 -- 172.21.15.111:0/1304956434 shutdown_connections 2024-09-15T22:41:04.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:04.318+0000 7f09e0c00640 1 --2- 172.21.15.111:0/1304956434 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f09c00a32f0 0x7f09c00a57b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:04.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:04.318+0000 7f09e0c00640 1 --2- 172.21.15.111:0/1304956434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09dc105920 0x7f09dc1a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:04.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:04.318+0000 7f09e0c00640 1 --2- 172.21.15.111:0/1304956434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09dc104f60 0x7f09dc19a640 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:04.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:04.319+0000 7f09e0c00640 1 --2- 172.21.15.111:0/1304956434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09dc103e00 0x7f09dc19a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:04.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:04.319+0000 7f09e0c00640 1 -- 172.21.15.111:0/1304956434 >> 172.21.15.111:0/1304956434 conn(0x7f09dc0ff5b0 msgr2=0x7f09dc108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:04.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:04.319+0000 7f09e0c00640 1 -- 172.21.15.111:0/1304956434 shutdown_connections 2024-09-15T22:41:04.321 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:04.319+0000 7f09e0c00640 1 -- 172.21.15.111:0/1304956434 wait complete. 2024-09-15T22:41:04.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:04 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:41:04.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:04 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:04.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:04 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:04.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:04 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:41:04.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:04 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:04.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:04 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:04.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:04 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:41:04.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:04 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:04.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:04 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:05.148 DEBUG:teuthology.orchestra.run.smithi111:osd.6> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.6.service 2024-09-15T22:41:05.151 INFO:tasks.cephadm:Deploying osd.7 on smithi121 with /dev/vg_nvme/lv_2... 2024-09-15T22:41:05.152 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-15T22:41:05.404 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:41:05.625 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:05 smithi111 ceph-mon[30629]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:41:05.767 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:05 smithi121 ceph-mon[34815]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:41:05.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:05 smithi192 ceph-mon[35427]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:41:06.416 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 22:41:06 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T22:41:06.054+0000 7fb7b9e43640 -1 osd.6 0 log_to_monitors true 2024-09-15T22:41:06.807 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:06 smithi111 ceph-mon[30629]: from='osd.6 [v2:172.21.15.111:6818/1371543706,v1:172.21.15.111:6819/1371543706]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-15T22:41:06.807 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:06 smithi111 ceph-mon[30629]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-15T22:41:06.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:06 smithi192 ceph-mon[35427]: from='osd.6 [v2:172.21.15.111:6818/1371543706,v1:172.21.15.111:6819/1371543706]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-15T22:41:06.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:06 smithi192 ceph-mon[35427]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-15T22:41:06.843 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:06 smithi121 ceph-mon[34815]: from='osd.6 [v2:172.21.15.111:6818/1371543706,v1:172.21.15.111:6819/1371543706]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-15T22:41:06.843 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:06 smithi121 ceph-mon[34815]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-15T22:41:07.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:07 smithi192 ceph-mon[35427]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:41:07.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:07 smithi192 ceph-mon[35427]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-15T22:41:07.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:07 smithi192 ceph-mon[35427]: from='osd.6 [v2:172.21.15.111:6818/1371543706,v1:172.21.15.111:6819/1371543706]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-15T22:41:07.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:07 smithi192 ceph-mon[35427]: osdmap e42: 7 total, 6 up, 7 in 2024-09-15T22:41:07.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:07 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:41:07.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:07 smithi192 ceph-mon[35427]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-15T22:41:07.880 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:07 smithi121 ceph-mon[34815]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:41:07.880 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:07 smithi121 ceph-mon[34815]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-15T22:41:07.880 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:07 smithi121 ceph-mon[34815]: from='osd.6 [v2:172.21.15.111:6818/1371543706,v1:172.21.15.111:6819/1371543706]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-15T22:41:07.880 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:07 smithi121 ceph-mon[34815]: osdmap e42: 7 total, 6 up, 7 in 2024-09-15T22:41:07.880 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:07 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:41:07.880 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:07 smithi121 ceph-mon[34815]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-15T22:41:07.883 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-15T22:41:07.910 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi121:vg_nvme/lv_2 2024-09-15T22:41:07.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:07 smithi111 ceph-mon[30629]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:41:07.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:07 smithi111 ceph-mon[30629]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-15T22:41:07.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:07 smithi111 ceph-mon[30629]: from='osd.6 [v2:172.21.15.111:6818/1371543706,v1:172.21.15.111:6819/1371543706]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-15T22:41:07.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:07 smithi111 ceph-mon[30629]: osdmap e42: 7 total, 6 up, 7 in 2024-09-15T22:41:07.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:07 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:41:07.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:07 smithi111 ceph-mon[30629]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-15T22:41:08.154 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:41:08.541 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:08 smithi121 ceph-mon[34815]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-15T22:41:08.541 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:08 smithi121 ceph-mon[34815]: osdmap e43: 7 total, 6 up, 7 in 2024-09-15T22:41:08.541 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:08 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:41:08.541 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:08 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:41:08.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.623+0000 7fc1c77cb640 1 -- 172.21.15.121:0/2367320168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1c0101c00 msgr2=0x7fc1c0102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:08.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.623+0000 7fc1c77cb640 1 --2- 172.21.15.121:0/2367320168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1c0101c00 0x7fc1c0102000 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009900 tx=0x7fc1bc02a410 comp rx=0 tx=0).stop 2024-09-15T22:41:08.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.623+0000 7fc1c77cb640 1 -- 172.21.15.121:0/2367320168 shutdown_connections 2024-09-15T22:41:08.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.623+0000 7fc1c77cb640 1 --2- 172.21.15.121:0/2367320168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1c01037c0 0x7fc1c010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:08.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.623+0000 7fc1c77cb640 1 --2- 172.21.15.121:0/2367320168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1c0102e00 0x7fc1c0103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:08.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.623+0000 7fc1c77cb640 1 --2- 172.21.15.121:0/2367320168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1c0101c00 0x7fc1c0102000 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:08.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.623+0000 7fc1c77cb640 1 -- 172.21.15.121:0/2367320168 >> 172.21.15.121:0/2367320168 conn(0x7fc1c00fd390 msgr2=0x7fc1c00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:08.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.623+0000 7fc1c77cb640 1 -- 172.21.15.121:0/2367320168 shutdown_connections 2024-09-15T22:41:08.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.623+0000 7fc1c77cb640 1 -- 172.21.15.121:0/2367320168 wait complete. 2024-09-15T22:41:08.642 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.624+0000 7fc1c77cb640 1 Processor -- start 2024-09-15T22:41:08.642 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.624+0000 7fc1c77cb640 1 -- start start 2024-09-15T22:41:08.642 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.624+0000 7fc1c77cb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1c0101c00 0x7fc1c0075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:08.642 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.624+0000 7fc1c77cb640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1c0102e00 0x7fc1c00745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:08.642 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.624+0000 7fc1c77cb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1c01037c0 0x7fc1c0074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:08.642 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.624+0000 7fc1c77cb640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c010d960 con 0x7fc1c0102e00 2024-09-15T22:41:08.642 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.624+0000 7fc1c77cb640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c010d7e0 con 0x7fc1c01037c0 2024-09-15T22:41:08.642 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.624+0000 7fc1c77cb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c010dae0 con 0x7fc1c0101c00 2024-09-15T22:41:08.642 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.624+0000 7fc1c5d68640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1c01037c0 0x7fc1c0074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:08.642 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.624+0000 7fc1c5d68640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1c01037c0 0x7fc1c0074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43454/0 (socket says 172.21.15.121:43454) 2024-09-15T22:41:08.642 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.624+0000 7fc1c5d68640 1 -- 172.21.15.121:0/4179777049 learned_addr learned my addr 172.21.15.121:0/4179777049 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-09-15T22:41:08.643 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.624+0000 7fc1c4d66640 1 --2- 172.21.15.121:0/4179777049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1c0102e00 0x7fc1c00745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:08.643 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.624+0000 7fc1c5567640 1 --2- 172.21.15.121:0/4179777049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1c0101c00 0x7fc1c0075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:08.643 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.624+0000 7fc1c5d68640 1 -- 172.21.15.121:0/4179777049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1c0101c00 msgr2=0x7fc1c0075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:08.643 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.624+0000 7fc1c5d68640 1 --2- 172.21.15.121:0/4179777049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1c0101c00 0x7fc1c0075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:08.643 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.624+0000 7fc1c5d68640 1 -- 172.21.15.121:0/4179777049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1c0102e00 msgr2=0x7fc1c00745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:08.643 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.624+0000 7fc1c5d68640 1 --2- 172.21.15.121:0/4179777049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1c0102e00 0x7fc1c00745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:08.643 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.624+0000 7fc1c5d68640 1 -- 172.21.15.121:0/4179777049 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c00753e0 con 0x7fc1c01037c0 2024-09-15T22:41:08.643 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.625+0000 7fc1c5d68640 1 --2- 172.21.15.121:0/4179777049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1c01037c0 0x7fc1c0074b20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800b970 tx=0x7fc1b800bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:08.643 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.625+0000 7fc1b67fc640 1 -- 172.21.15.121:0/4179777049 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8019070 con 0x7fc1c01037c0 2024-09-15T22:41:08.643 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.625+0000 7fc1b67fc640 1 -- 172.21.15.121:0/4179777049 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1b8014440 con 0x7fc1c01037c0 2024-09-15T22:41:08.643 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.625+0000 7fc1c77cb640 1 -- 172.21.15.121:0/4179777049 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c01a2210 con 0x7fc1c01037c0 2024-09-15T22:41:08.643 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.625+0000 7fc1b67fc640 1 -- 172.21.15.121:0/4179777049 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b80075d0 con 0x7fc1c01037c0 2024-09-15T22:41:08.643 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.625+0000 7fc1c77cb640 1 -- 172.21.15.121:0/4179777049 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c01a2750 con 0x7fc1c01037c0 2024-09-15T22:41:08.643 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.626+0000 7fc1c77cb640 1 -- 172.21.15.121:0/4179777049 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1c01090a0 con 0x7fc1c01037c0 2024-09-15T22:41:08.644 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.627+0000 7fc1b67fc640 1 -- 172.21.15.121:0/4179777049 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fc1b801c070 con 0x7fc1c01037c0 2024-09-15T22:41:08.644 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.628+0000 7fc1b67fc640 1 --2- 172.21.15.121:0/4179777049 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc1ac0a3240 0x7fc1ac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:08.644 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.628+0000 7fc1b67fc640 1 -- 172.21.15.121:0/4179777049 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4685+0+0 (secure 0 0 0) 0x7fc1b80ce490 con 0x7fc1c01037c0 2024-09-15T22:41:08.644 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.628+0000 7fc1c5567640 1 --2- 172.21.15.121:0/4179777049 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc1ac0a3240 0x7fc1ac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:08.644 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.628+0000 7fc1c5567640 1 --2- 172.21.15.121:0/4179777049 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc1ac0a3240 0x7fc1ac0a5700 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc005ab0 tx=0x7fc1bc000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:08.644 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.631+0000 7fc1b67fc640 1 -- 172.21.15.121:0/4179777049 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc1b809b990 con 0x7fc1c01037c0 2024-09-15T22:41:08.808 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:08.806+0000 7fc1c77cb640 1 -- 172.21.15.121:0/4179777049 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc1c0062630 con 0x7fc1ac0a3240 2024-09-15T22:41:08.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:08 smithi192 ceph-mon[35427]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-15T22:41:08.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:08 smithi192 ceph-mon[35427]: osdmap e43: 7 total, 6 up, 7 in 2024-09-15T22:41:08.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:08 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:41:08.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:08 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:41:08.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:08 smithi111 ceph-mon[30629]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-15T22:41:08.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:08 smithi111 ceph-mon[30629]: osdmap e43: 7 total, 6 up, 7 in 2024-09-15T22:41:08.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:08 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:41:08.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:08 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:41:08.917 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 22:41:08 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T22:41:08.558+0000 7fb7b3c00640 -1 osd.6 0 waiting for initial osdmap 2024-09-15T22:41:08.917 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 22:41:08 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T22:41:08.592+0000 7fb7aa600640 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:41:09.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:09 smithi192 ceph-mon[35427]: purged_snaps scrub starts 2024-09-15T22:41:09.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:09 smithi192 ceph-mon[35427]: purged_snaps scrub ok 2024-09-15T22:41:09.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:09 smithi192 ceph-mon[35427]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:41:09.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:09 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:41:09.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:09 smithi192 ceph-mon[35427]: from='osd.6 ' entity='osd.6' 2024-09-15T22:41:09.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:09 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:41:09.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:09 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:41:09.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:09 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:09.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:09 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:09.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:09 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:09.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:09 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:41:09.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:09 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:41:09.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:09 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:41:09.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:09 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:09.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:09 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:09.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:09 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:41:09.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:09 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:09.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:09 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:41:09.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:09 smithi121 ceph-mon[34815]: purged_snaps scrub starts 2024-09-15T22:41:09.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:09 smithi121 ceph-mon[34815]: purged_snaps scrub ok 2024-09-15T22:41:09.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:09 smithi121 ceph-mon[34815]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:41:09.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:09 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:41:09.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:09 smithi121 ceph-mon[34815]: from='osd.6 ' entity='osd.6' 2024-09-15T22:41:09.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:09 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:41:09.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:09 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:41:09.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:09 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:09.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:09 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:09.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:09 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:09.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:09 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:41:09.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:09 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:41:09.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:09 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:41:09.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:09 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:09.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:09 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:09.892 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:09 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:41:09.892 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:09 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:09.892 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:09 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:41:09.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:09 smithi111 ceph-mon[30629]: purged_snaps scrub starts 2024-09-15T22:41:09.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:09 smithi111 ceph-mon[30629]: purged_snaps scrub ok 2024-09-15T22:41:09.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:09 smithi111 ceph-mon[30629]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:41:09.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:09 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:41:09.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:09 smithi111 ceph-mon[30629]: from='osd.6 ' entity='osd.6' 2024-09-15T22:41:09.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:09 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:41:09.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:09 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:41:09.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:09 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:09.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:09 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:09.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:09 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:09.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:09 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:41:09.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:09 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:41:09.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:09 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:41:09.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:09 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:09.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:09 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:09.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:09 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:41:09.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:09 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:09.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:09 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:41:10.836 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:10 smithi121 ceph-mon[34815]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:41:10.836 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:10 smithi121 ceph-mon[34815]: Detected new or changed devices on smithi111 2024-09-15T22:41:10.837 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:10 smithi121 ceph-mon[34815]: Adjusting osd_memory_target on smithi111 to 5696M 2024-09-15T22:41:10.837 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:10 smithi121 ceph-mon[34815]: osd.6 [v2:172.21.15.111:6818/1371543706,v1:172.21.15.111:6819/1371543706] boot 2024-09-15T22:41:10.837 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:10 smithi121 ceph-mon[34815]: osdmap e44: 7 total, 7 up, 7 in 2024-09-15T22:41:10.837 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:10 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:41:10.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:10 smithi111 ceph-mon[30629]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:41:10.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:10 smithi111 ceph-mon[30629]: Detected new or changed devices on smithi111 2024-09-15T22:41:10.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:10 smithi111 ceph-mon[30629]: Adjusting osd_memory_target on smithi111 to 5696M 2024-09-15T22:41:10.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:10 smithi111 ceph-mon[30629]: osd.6 [v2:172.21.15.111:6818/1371543706,v1:172.21.15.111:6819/1371543706] boot 2024-09-15T22:41:10.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:10 smithi111 ceph-mon[30629]: osdmap e44: 7 total, 7 up, 7 in 2024-09-15T22:41:10.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:10 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:41:11.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:10 smithi192 ceph-mon[35427]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:41:11.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:10 smithi192 ceph-mon[35427]: Detected new or changed devices on smithi111 2024-09-15T22:41:11.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:10 smithi192 ceph-mon[35427]: Adjusting osd_memory_target on smithi111 to 5696M 2024-09-15T22:41:11.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:10 smithi192 ceph-mon[35427]: osd.6 [v2:172.21.15.111:6818/1371543706,v1:172.21.15.111:6819/1371543706] boot 2024-09-15T22:41:11.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:10 smithi192 ceph-mon[35427]: osdmap e44: 7 total, 7 up, 7 in 2024-09-15T22:41:11.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:10 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:41:11.741 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:11 smithi121 ceph-mon[34815]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:41:11.741 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:11 smithi121 ceph-mon[34815]: osdmap e45: 7 total, 7 up, 7 in 2024-09-15T22:41:11.741 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:11 smithi121 ceph-mon[34815]: from='client.? 172.21.15.121:0/1085592261' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "470647c5-c00d-426b-a134-cca235902307"}]: dispatch 2024-09-15T22:41:11.741 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:11 smithi121 ceph-mon[34815]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "470647c5-c00d-426b-a134-cca235902307"}]: dispatch 2024-09-15T22:41:11.741 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:11 smithi121 ceph-mon[34815]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "470647c5-c00d-426b-a134-cca235902307"}]': finished 2024-09-15T22:41:11.741 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:11 smithi121 ceph-mon[34815]: osdmap e46: 8 total, 7 up, 8 in 2024-09-15T22:41:11.741 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:11 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:41:11.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:11 smithi111 ceph-mon[30629]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:41:11.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:11 smithi111 ceph-mon[30629]: osdmap e45: 7 total, 7 up, 7 in 2024-09-15T22:41:11.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:11 smithi111 ceph-mon[30629]: from='client.? 172.21.15.121:0/1085592261' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "470647c5-c00d-426b-a134-cca235902307"}]: dispatch 2024-09-15T22:41:11.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:11 smithi111 ceph-mon[30629]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "470647c5-c00d-426b-a134-cca235902307"}]: dispatch 2024-09-15T22:41:11.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:11 smithi111 ceph-mon[30629]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "470647c5-c00d-426b-a134-cca235902307"}]': finished 2024-09-15T22:41:11.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:11 smithi111 ceph-mon[30629]: osdmap e46: 8 total, 7 up, 8 in 2024-09-15T22:41:11.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:11 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:41:12.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:11 smithi192 ceph-mon[35427]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:41:12.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:11 smithi192 ceph-mon[35427]: osdmap e45: 7 total, 7 up, 7 in 2024-09-15T22:41:12.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:11 smithi192 ceph-mon[35427]: from='client.? 172.21.15.121:0/1085592261' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "470647c5-c00d-426b-a134-cca235902307"}]: dispatch 2024-09-15T22:41:12.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:11 smithi192 ceph-mon[35427]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "470647c5-c00d-426b-a134-cca235902307"}]: dispatch 2024-09-15T22:41:12.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:11 smithi192 ceph-mon[35427]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "470647c5-c00d-426b-a134-cca235902307"}]': finished 2024-09-15T22:41:12.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:11 smithi192 ceph-mon[35427]: osdmap e46: 8 total, 7 up, 8 in 2024-09-15T22:41:12.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:11 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:41:12.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:12 smithi192 ceph-mon[35427]: from='client.? 172.21.15.121:0/2028079206' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:41:12.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:12 smithi121 ceph-mon[34815]: from='client.? 172.21.15.121:0/2028079206' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:41:12.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:12 smithi111 ceph-mon[30629]: from='client.? 172.21.15.121:0/2028079206' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:41:13.842 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:13 smithi111 ceph-mon[30629]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:13.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:13 smithi121 ceph-mon[34815]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:14.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:13 smithi192 ceph-mon[35427]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:15.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:15 smithi121 ceph-mon[34815]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:15.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:15 smithi111 ceph-mon[30629]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:16.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:15 smithi192 ceph-mon[35427]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:17.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:17 smithi121 ceph-mon[34815]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:17.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:17 smithi111 ceph-mon[30629]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:18.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:17 smithi192 ceph-mon[35427]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:19.884 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:19 smithi121 ceph-mon[34815]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:19.884 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:19 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-15T22:41:19.884 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:19 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:19.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:19 smithi111 ceph-mon[30629]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:19.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:19 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-15T22:41:19.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:19 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:20.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:19 smithi192 ceph-mon[35427]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:20.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:19 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-15T22:41:20.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:19 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:20.682 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:20 smithi121 ceph-mon[34815]: Deploying daemon osd.7 on smithi121 2024-09-15T22:41:20.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:20 smithi111 ceph-mon[30629]: Deploying daemon osd.7 on smithi121 2024-09-15T22:41:21.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:20 smithi192 ceph-mon[35427]: Deploying daemon osd.7 on smithi121 2024-09-15T22:41:21.789 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:21 smithi121 ceph-mon[34815]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:21.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:21 smithi111 ceph-mon[30629]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:22.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:21 smithi192 ceph-mon[35427]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:23.876 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:23 smithi111 ceph-mon[30629]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:23.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:23 smithi121 ceph-mon[34815]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:24.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:23 smithi192 ceph-mon[35427]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:25.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:25 smithi121 ceph-mon[34815]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:25.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:25 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:41:25.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:25 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:25.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:25 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:25.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:25 smithi111 ceph-mon[30629]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:25.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:25 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:41:25.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:25 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:25.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:25 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:26.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:25 smithi192 ceph-mon[35427]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:26.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:25 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:41:26.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:25 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:26.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:25 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:27.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:27 smithi121 ceph-mon[34815]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:27.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:27 smithi111 ceph-mon[30629]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:28.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:27 smithi192 ceph-mon[35427]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:28.742 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:28.739+0000 7fc1b67fc640 1 -- 172.21.15.121:0/4179777049 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc1c0062630 con 0x7fc1ac0a3240 2024-09-15T22:41:28.742 INFO:teuthology.orchestra.run.smithi121.stdout:Created osd(s) 7 on host 'smithi121' 2024-09-15T22:41:28.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:28.741+0000 7fc1c77cb640 1 -- 172.21.15.121:0/4179777049 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc1ac0a3240 msgr2=0x7fc1ac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:28.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:28.741+0000 7fc1c77cb640 1 --2- 172.21.15.121:0/4179777049 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc1ac0a3240 0x7fc1ac0a5700 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc005ab0 tx=0x7fc1bc000bc0 comp rx=0 tx=0).stop 2024-09-15T22:41:28.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:28.741+0000 7fc1c77cb640 1 -- 172.21.15.121:0/4179777049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1c01037c0 msgr2=0x7fc1c0074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:28.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:28.741+0000 7fc1c77cb640 1 --2- 172.21.15.121:0/4179777049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1c01037c0 0x7fc1c0074b20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800b970 tx=0x7fc1b800bd70 comp rx=0 tx=0).stop 2024-09-15T22:41:28.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:28.741+0000 7fc1c77cb640 1 -- 172.21.15.121:0/4179777049 shutdown_connections 2024-09-15T22:41:28.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:28.741+0000 7fc1c77cb640 1 --2- 172.21.15.121:0/4179777049 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc1ac0a3240 0x7fc1ac0a5700 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:28.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:28.741+0000 7fc1c77cb640 1 --2- 172.21.15.121:0/4179777049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc1c01037c0 0x7fc1c0074b20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:28.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:28.741+0000 7fc1c77cb640 1 --2- 172.21.15.121:0/4179777049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc1c0102e00 0x7fc1c00745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:28.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:28.741+0000 7fc1c77cb640 1 --2- 172.21.15.121:0/4179777049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1c0101c00 0x7fc1c0075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:28.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:28.741+0000 7fc1c77cb640 1 -- 172.21.15.121:0/4179777049 >> 172.21.15.121:0/4179777049 conn(0x7fc1c00fd390 msgr2=0x7fc1c0106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:28.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:28.742+0000 7fc1c77cb640 1 -- 172.21.15.121:0/4179777049 shutdown_connections 2024-09-15T22:41:28.745 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:41:28.742+0000 7fc1c77cb640 1 -- 172.21.15.121:0/4179777049 wait complete. 2024-09-15T22:41:29.052 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:28 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:29.052 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:28 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:29.052 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:28 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:29.052 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:28 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:41:29.052 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:28 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:29.053 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:28 smithi121 ceph-mon[34815]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:29.053 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:28 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:41:29.053 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:28 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:29.053 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:28 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:29.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:28 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:29.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:28 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:29.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:28 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:29.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:28 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:41:29.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:28 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:29.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:28 smithi192 ceph-mon[35427]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:29.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:28 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:41:29.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:28 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:29.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:28 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:29.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:28 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:29.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:28 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:29.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:28 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:29.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:28 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:41:29.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:28 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:29.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:28 smithi111 ceph-mon[30629]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:29.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:28 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:41:29.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:28 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:29.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:28 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:29.406 DEBUG:teuthology.orchestra.run.smithi121:osd.7> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.7.service 2024-09-15T22:41:29.409 INFO:tasks.cephadm:Deploying osd.8 on smithi192 with /dev/vg_nvme/lv_2... 2024-09-15T22:41:29.409 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-15T22:41:29.664 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:41:31.811 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:31 smithi192 ceph-mon[35427]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:31.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:31 smithi192 ceph-mon[35427]: from='osd.7 [v2:172.21.15.121:6816/3501842998,v1:172.21.15.121:6817/3501842998]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T22:41:31.812 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:31 smithi192 ceph-mon[35427]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T22:41:31.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:31 smithi121 ceph-mon[34815]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:31.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:31 smithi121 ceph-mon[34815]: from='osd.7 [v2:172.21.15.121:6816/3501842998,v1:172.21.15.121:6817/3501842998]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T22:41:31.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:31 smithi121 ceph-mon[34815]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T22:41:31.891 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 22:41:31 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T22:41:31.448+0000 7f6ab3cfc640 -1 osd.7 0 log_to_monitors true 2024-09-15T22:41:31.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:31 smithi111 ceph-mon[30629]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:31.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:31 smithi111 ceph-mon[30629]: from='osd.7 [v2:172.21.15.121:6816/3501842998,v1:172.21.15.121:6817/3501842998]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T22:41:31.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:31 smithi111 ceph-mon[30629]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T22:41:32.115 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T22:41:32.141 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi192:vg_nvme/lv_2 2024-09-15T22:41:32.406 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:41:32.640 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 22:41:32 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T22:41:32.485+0000 7f6aaf45c640 -1 osd.7 0 waiting for initial osdmap 2024-09-15T22:41:32.640 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 22:41:32 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T22:41:32.494+0000 7f6aaba96640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:41:32.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:32 smithi121 ceph-mon[34815]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-15T22:41:32.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:32 smithi121 ceph-mon[34815]: osdmap e47: 8 total, 7 up, 8 in 2024-09-15T22:41:32.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:32 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:41:32.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:32 smithi121 ceph-mon[34815]: from='osd.7 [v2:172.21.15.121:6816/3501842998,v1:172.21.15.121:6817/3501842998]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-15T22:41:32.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:32 smithi121 ceph-mon[34815]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-15T22:41:32.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:32 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:32.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:32 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:32.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:32 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:41:32.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:32 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:41:32.642 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:32 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:41:32.642 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:32 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:32.642 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:32 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:32.642 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:32 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:41:32.642 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:32 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:32.791 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:32 smithi192 ceph-mon[35427]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-15T22:41:32.791 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:32 smithi192 ceph-mon[35427]: osdmap e47: 8 total, 7 up, 8 in 2024-09-15T22:41:32.791 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:32 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:41:32.791 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:32 smithi192 ceph-mon[35427]: from='osd.7 [v2:172.21.15.121:6816/3501842998,v1:172.21.15.121:6817/3501842998]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-15T22:41:32.791 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:32 smithi192 ceph-mon[35427]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-15T22:41:32.791 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:32 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:32.791 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:32 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:32.791 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:32 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:41:32.791 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:32 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:41:32.791 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:32 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:41:32.791 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:32 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:32.792 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:32 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:32.792 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:32 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:41:32.792 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:32 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:32.875 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.873+0000 7f3a316bd640 1 -- 172.21.15.192:0/557992674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a2c1037c0 msgr2=0x7f3a2c10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:32.875 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.873+0000 7f3a316bd640 1 --2- 172.21.15.192:0/557992674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a2c1037c0 0x7f3a2c10a050 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009ca0 tx=0x7f3a2402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.875 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.873+0000 7f3a316bd640 1 -- 172.21.15.192:0/557992674 shutdown_connections 2024-09-15T22:41:32.875 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.873+0000 7f3a316bd640 1 --2- 172.21.15.192:0/557992674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a2c1037c0 0x7f3a2c10a050 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.876 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.873+0000 7f3a316bd640 1 --2- 172.21.15.192:0/557992674 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a2c102e00 0x7f3a2c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.876 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.873+0000 7f3a316bd640 1 --2- 172.21.15.192:0/557992674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a2c101c00 0x7f3a2c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.876 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.873+0000 7f3a316bd640 1 -- 172.21.15.192:0/557992674 >> 172.21.15.192:0/557992674 conn(0x7f3a2c0fd390 msgr2=0x7f3a2c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:32.876 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.874+0000 7f3a316bd640 1 -- 172.21.15.192:0/557992674 shutdown_connections 2024-09-15T22:41:32.876 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.874+0000 7f3a316bd640 1 -- 172.21.15.192:0/557992674 wait complete. 2024-09-15T22:41:32.876 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.874+0000 7f3a316bd640 1 Processor -- start 2024-09-15T22:41:32.878 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.874+0000 7f3a316bd640 1 -- start start 2024-09-15T22:41:32.878 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.874+0000 7f3a316bd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a2c101c00 0x7f3a2c197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:32.878 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.874+0000 7f3a316bd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a2c102e00 0x7f3a2c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:32.878 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.874+0000 7f3a316bd640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a2c1037c0 0x7f3a2c19f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:32.878 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.874+0000 7f3a316bd640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2c10d8e0 con 0x7f3a2c1037c0 2024-09-15T22:41:32.878 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.875+0000 7f3a316bd640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2c10d760 con 0x7f3a2c102e00 2024-09-15T22:41:32.878 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.875+0000 7f3a316bd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2c10da60 con 0x7f3a2c101c00 2024-09-15T22:41:32.878 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.875+0000 7f3a2affd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a2c101c00 0x7f3a2c197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:32.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.875+0000 7f3a2affd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a2c101c00 0x7f3a2c197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:47246/0 (socket says 172.21.15.192:47246) 2024-09-15T22:41:32.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.875+0000 7f3a2affd640 1 -- 172.21.15.192:0/768100507 learned_addr learned my addr 172.21.15.192:0/768100507 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-09-15T22:41:32.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.875+0000 7f3a2a7fc640 1 --2- 172.21.15.192:0/768100507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a2c102e00 0x7f3a2c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:32.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.875+0000 7f3a2b7fe640 1 --2- 172.21.15.192:0/768100507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a2c1037c0 0x7f3a2c19f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:32.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.875+0000 7f3a2affd640 1 -- 172.21.15.192:0/768100507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a2c102e00 msgr2=0x7f3a2c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:32.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.875+0000 7f3a2affd640 1 --2- 172.21.15.192:0/768100507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a2c102e00 0x7f3a2c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.875+0000 7f3a2affd640 1 -- 172.21.15.192:0/768100507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a2c1037c0 msgr2=0x7f3a2c19f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:32.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.875+0000 7f3a2affd640 1 --2- 172.21.15.192:0/768100507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a2c1037c0 0x7f3a2c19f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:32.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.875+0000 7f3a2affd640 1 -- 172.21.15.192:0/768100507 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2c19fb00 con 0x7f3a2c101c00 2024-09-15T22:41:32.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.875+0000 7f3a2a7fc640 1 --2- 172.21.15.192:0/768100507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a2c102e00 0x7f3a2c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:41:32.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.875+0000 7f3a2b7fe640 1 --2- 172.21.15.192:0/768100507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a2c1037c0 0x7f3a2c19f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:41:32.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.875+0000 7f3a2affd640 1 --2- 172.21.15.192:0/768100507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a2c101c00 0x7f3a2c197ed0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00d770 tx=0x7f3a1c00db70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:32.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.875+0000 7f3a13fff640 1 -- 172.21.15.192:0/768100507 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c017070 con 0x7f3a2c101c00 2024-09-15T22:41:32.880 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.875+0000 7f3a316bd640 1 -- 172.21.15.192:0/768100507 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a2c19fdf0 con 0x7f3a2c101c00 2024-09-15T22:41:32.880 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.875+0000 7f3a13fff640 1 -- 172.21.15.192:0/768100507 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a1c009d00 con 0x7f3a2c101c00 2024-09-15T22:41:32.880 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.875+0000 7f3a13fff640 1 -- 172.21.15.192:0/768100507 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c003d30 con 0x7f3a2c101c00 2024-09-15T22:41:32.880 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.875+0000 7f3a316bd640 1 -- 172.21.15.192:0/768100507 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a2c1a02d0 con 0x7f3a2c101c00 2024-09-15T22:41:32.880 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.876+0000 7f3a316bd640 1 -- 172.21.15.192:0/768100507 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a2c108fb0 con 0x7f3a2c101c00 2024-09-15T22:41:32.881 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.878+0000 7f3a13fff640 1 -- 172.21.15.192:0/768100507 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f3a1c009820 con 0x7f3a2c101c00 2024-09-15T22:41:32.881 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.878+0000 7f3a13fff640 1 --2- 172.21.15.192:0/768100507 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3a140a3270 0x7f3a140a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:32.881 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.878+0000 7f3a13fff640 1 -- 172.21.15.192:0/768100507 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5117+0+0 (secure 0 0 0) 0x7f3a1c0ce640 con 0x7f3a2c101c00 2024-09-15T22:41:32.881 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.878+0000 7f3a2a7fc640 1 --2- 172.21.15.192:0/768100507 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3a140a3270 0x7f3a140a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:32.881 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.879+0000 7f3a2a7fc640 1 --2- 172.21.15.192:0/768100507 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3a140a3270 0x7f3a140a5730 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c1993f0 tx=0x7f3a20009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:32.884 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:32.883+0000 7f3a13fff640 1 -- 172.21.15.192:0/768100507 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3a1c09bfe0 con 0x7f3a2c101c00 2024-09-15T22:41:32.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:32 smithi111 ceph-mon[30629]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-15T22:41:32.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:32 smithi111 ceph-mon[30629]: osdmap e47: 8 total, 7 up, 8 in 2024-09-15T22:41:32.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:32 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:41:32.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:32 smithi111 ceph-mon[30629]: from='osd.7 [v2:172.21.15.121:6816/3501842998,v1:172.21.15.121:6817/3501842998]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-15T22:41:32.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:32 smithi111 ceph-mon[30629]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-15T22:41:32.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:32 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:32.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:32 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:32.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:32 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:41:32.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:32 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:41:32.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:32 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:41:32.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:32 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:32.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:32 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:32.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:32 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:41:32.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:32 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:33.067 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:33.065+0000 7f3a316bd640 1 -- 172.21.15.192:0/768100507 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3a2c062630 con 0x7f3a140a3270 2024-09-15T22:41:33.735 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:33 smithi192 ceph-mon[35427]: Detected new or changed devices on smithi121 2024-09-15T22:41:33.735 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:33 smithi192 ceph-mon[35427]: Adjusting osd_memory_target on smithi121 to 5695M 2024-09-15T22:41:33.735 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:33 smithi192 ceph-mon[35427]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:33.735 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:33 smithi192 ceph-mon[35427]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-15T22:41:33.735 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:33 smithi192 ceph-mon[35427]: osdmap e48: 8 total, 7 up, 8 in 2024-09-15T22:41:33.736 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:33 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:41:33.736 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:33 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:41:33.736 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:33 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:41:33.736 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:33 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:41:33.736 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:33 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:33.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:33 smithi121 ceph-mon[34815]: Detected new or changed devices on smithi121 2024-09-15T22:41:33.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:33 smithi121 ceph-mon[34815]: Adjusting osd_memory_target on smithi121 to 5695M 2024-09-15T22:41:33.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:33 smithi121 ceph-mon[34815]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:33.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:33 smithi121 ceph-mon[34815]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-15T22:41:33.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:33 smithi121 ceph-mon[34815]: osdmap e48: 8 total, 7 up, 8 in 2024-09-15T22:41:33.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:33 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:41:33.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:33 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:41:33.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:33 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:41:33.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:33 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:41:33.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:33 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:33.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:33 smithi111 ceph-mon[30629]: Detected new or changed devices on smithi121 2024-09-15T22:41:33.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:33 smithi111 ceph-mon[30629]: Adjusting osd_memory_target on smithi121 to 5695M 2024-09-15T22:41:33.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:33 smithi111 ceph-mon[30629]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:33.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:33 smithi111 ceph-mon[30629]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-15T22:41:33.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:33 smithi111 ceph-mon[30629]: osdmap e48: 8 total, 7 up, 8 in 2024-09-15T22:41:33.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:33 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:41:33.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:33 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:41:33.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:33 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:41:33.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:33 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:41:33.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:33 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:34.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:34 smithi192 ceph-mon[35427]: purged_snaps scrub starts 2024-09-15T22:41:34.732 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:34 smithi192 ceph-mon[35427]: purged_snaps scrub ok 2024-09-15T22:41:34.732 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:34 smithi192 ceph-mon[35427]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:41:34.732 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:34 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:41:34.732 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:34 smithi192 ceph-mon[35427]: osd.7 [v2:172.21.15.121:6816/3501842998,v1:172.21.15.121:6817/3501842998] boot 2024-09-15T22:41:34.732 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:34 smithi192 ceph-mon[35427]: osdmap e49: 8 total, 8 up, 8 in 2024-09-15T22:41:34.732 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:34 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:41:34.732 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:34 smithi192 ceph-mon[35427]: osdmap e50: 8 total, 8 up, 8 in 2024-09-15T22:41:34.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:34 smithi121 ceph-mon[34815]: purged_snaps scrub starts 2024-09-15T22:41:34.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:34 smithi121 ceph-mon[34815]: purged_snaps scrub ok 2024-09-15T22:41:34.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:34 smithi121 ceph-mon[34815]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:41:34.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:34 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:41:34.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:34 smithi121 ceph-mon[34815]: osd.7 [v2:172.21.15.121:6816/3501842998,v1:172.21.15.121:6817/3501842998] boot 2024-09-15T22:41:34.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:34 smithi121 ceph-mon[34815]: osdmap e49: 8 total, 8 up, 8 in 2024-09-15T22:41:34.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:34 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:41:34.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:34 smithi121 ceph-mon[34815]: osdmap e50: 8 total, 8 up, 8 in 2024-09-15T22:41:34.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:34 smithi111 ceph-mon[30629]: purged_snaps scrub starts 2024-09-15T22:41:34.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:34 smithi111 ceph-mon[30629]: purged_snaps scrub ok 2024-09-15T22:41:34.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:34 smithi111 ceph-mon[30629]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:41:34.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:34 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:41:34.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:34 smithi111 ceph-mon[30629]: osd.7 [v2:172.21.15.121:6816/3501842998,v1:172.21.15.121:6817/3501842998] boot 2024-09-15T22:41:34.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:34 smithi111 ceph-mon[30629]: osdmap e49: 8 total, 8 up, 8 in 2024-09-15T22:41:34.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:34 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:41:34.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:34 smithi111 ceph-mon[30629]: osdmap e50: 8 total, 8 up, 8 in 2024-09-15T22:41:35.750 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:35 smithi192 ceph-mon[35427]: pgmap v182: 1 pgs: 1 remapped; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:35.750 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:35 smithi192 ceph-mon[35427]: osdmap e51: 8 total, 8 up, 8 in 2024-09-15T22:41:35.750 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:35 smithi192 ceph-mon[35427]: from='client.? 172.21.15.192:0/987652903' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa2d3dd5-a768-4ada-a036-4c596b76d654"}]: dispatch 2024-09-15T22:41:35.750 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:35 smithi192 ceph-mon[35427]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa2d3dd5-a768-4ada-a036-4c596b76d654"}]: dispatch 2024-09-15T22:41:35.750 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:35 smithi192 ceph-mon[35427]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa2d3dd5-a768-4ada-a036-4c596b76d654"}]': finished 2024-09-15T22:41:35.750 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:35 smithi192 ceph-mon[35427]: osdmap e52: 9 total, 8 up, 9 in 2024-09-15T22:41:35.750 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:35 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:41:35.792 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:35 smithi121 ceph-mon[34815]: pgmap v182: 1 pgs: 1 remapped; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:35.792 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:35 smithi121 ceph-mon[34815]: osdmap e51: 8 total, 8 up, 8 in 2024-09-15T22:41:35.792 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:35 smithi121 ceph-mon[34815]: from='client.? 172.21.15.192:0/987652903' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa2d3dd5-a768-4ada-a036-4c596b76d654"}]: dispatch 2024-09-15T22:41:35.792 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:35 smithi121 ceph-mon[34815]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa2d3dd5-a768-4ada-a036-4c596b76d654"}]: dispatch 2024-09-15T22:41:35.792 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:35 smithi121 ceph-mon[34815]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa2d3dd5-a768-4ada-a036-4c596b76d654"}]': finished 2024-09-15T22:41:35.792 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:35 smithi121 ceph-mon[34815]: osdmap e52: 9 total, 8 up, 9 in 2024-09-15T22:41:35.792 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:35 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:41:35.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:35 smithi111 ceph-mon[30629]: pgmap v182: 1 pgs: 1 remapped; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:41:35.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:35 smithi111 ceph-mon[30629]: osdmap e51: 8 total, 8 up, 8 in 2024-09-15T22:41:35.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:35 smithi111 ceph-mon[30629]: from='client.? 172.21.15.192:0/987652903' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa2d3dd5-a768-4ada-a036-4c596b76d654"}]: dispatch 2024-09-15T22:41:35.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:35 smithi111 ceph-mon[30629]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa2d3dd5-a768-4ada-a036-4c596b76d654"}]: dispatch 2024-09-15T22:41:35.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:35 smithi111 ceph-mon[30629]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa2d3dd5-a768-4ada-a036-4c596b76d654"}]': finished 2024-09-15T22:41:35.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:35 smithi111 ceph-mon[30629]: osdmap e52: 9 total, 8 up, 9 in 2024-09-15T22:41:35.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:35 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:41:36.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:36 smithi121 ceph-mon[34815]: from='client.? 172.21.15.192:0/3436000348' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:41:36.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:36 smithi111 ceph-mon[30629]: from='client.? 172.21.15.192:0/3436000348' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:41:37.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:36 smithi192 ceph-mon[35427]: from='client.? 172.21.15.192:0/3436000348' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:41:37.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:37 smithi121 ceph-mon[34815]: pgmap v185: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:37.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:37 smithi121 ceph-mon[34815]: osdmap e53: 9 total, 8 up, 9 in 2024-09-15T22:41:37.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:41:37.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:37 smithi111 ceph-mon[30629]: pgmap v185: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:37.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:37 smithi111 ceph-mon[30629]: osdmap e53: 9 total, 8 up, 9 in 2024-09-15T22:41:37.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:37 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:41:38.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:37 smithi192 ceph-mon[35427]: pgmap v185: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:38.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:37 smithi192 ceph-mon[35427]: osdmap e53: 9 total, 8 up, 9 in 2024-09-15T22:41:38.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:37 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:41:38.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:38 smithi121 ceph-mon[34815]: osdmap e54: 9 total, 8 up, 9 in 2024-09-15T22:41:38.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:38 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:41:38.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:38 smithi111 ceph-mon[30629]: osdmap e54: 9 total, 8 up, 9 in 2024-09-15T22:41:38.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:38 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:41:39.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:38 smithi192 ceph-mon[35427]: osdmap e54: 9 total, 8 up, 9 in 2024-09-15T22:41:39.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:38 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:41:39.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:39 smithi121 ceph-mon[34815]: pgmap v188: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:39.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:39 smithi121 ceph-mon[34815]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-15T22:41:39.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:39 smithi111 ceph-mon[30629]: pgmap v188: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:39.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:39 smithi111 ceph-mon[30629]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-15T22:41:40.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:39 smithi192 ceph-mon[35427]: pgmap v188: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:40.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:39 smithi192 ceph-mon[35427]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-15T22:41:40.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:40 smithi192 ceph-mon[35427]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-15T22:41:40.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:40 smithi192 ceph-mon[35427]: Cluster is now healthy 2024-09-15T22:41:40.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:40 smithi121 ceph-mon[34815]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-15T22:41:40.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:40 smithi121 ceph-mon[34815]: Cluster is now healthy 2024-09-15T22:41:40.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:40 smithi111 ceph-mon[30629]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-15T22:41:40.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:40 smithi111 ceph-mon[30629]: Cluster is now healthy 2024-09-15T22:41:41.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:41 smithi192 ceph-mon[35427]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:41.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:41 smithi121 ceph-mon[34815]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:41.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:41 smithi111 ceph-mon[30629]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:43.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:43 smithi192 ceph-mon[35427]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:43.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:43 smithi111 ceph-mon[30629]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:44.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:43 smithi121 ceph-mon[34815]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:44.905 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:44 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-15T22:41:44.905 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:44 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:44.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:44 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-15T22:41:44.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:44 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:45.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:44 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-15T22:41:45.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:44 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:45.906 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:45 smithi192 ceph-mon[35427]: Deploying daemon osd.8 on smithi192 2024-09-15T22:41:45.907 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:45 smithi192 ceph-mon[35427]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:45.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:45 smithi111 ceph-mon[30629]: Deploying daemon osd.8 on smithi192 2024-09-15T22:41:45.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:45 smithi111 ceph-mon[30629]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:46.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:45 smithi121 ceph-mon[34815]: Deploying daemon osd.8 on smithi192 2024-09-15T22:41:46.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:45 smithi121 ceph-mon[34815]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:46.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:46 smithi121 ceph-mon[34815]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:47.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:46 smithi192 ceph-mon[35427]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:47.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:46 smithi111 ceph-mon[30629]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:49.720 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:49 smithi192 ceph-mon[35427]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:49.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:49 smithi121 ceph-mon[34815]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:49.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:49 smithi111 ceph-mon[30629]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:50.771 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:41:50.771 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:50.771 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:50.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:41:50.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:50.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:50.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:41:50.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:50.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:51.726 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:51 smithi192 ceph-mon[35427]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:51.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:51 smithi121 ceph-mon[34815]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:51.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:51 smithi111 ceph-mon[30629]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:53.627 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:53 smithi192 ceph-mon[35427]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:53.627 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:53 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:53.627 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:53 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:53.627 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:53 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:53.627 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:53 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:41:53.627 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:53 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:53.670 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:53.669+0000 7f3a13fff640 1 -- 172.21.15.192:0/768100507 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3a2c062630 con 0x7f3a140a3270 2024-09-15T22:41:53.670 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 8 on host 'smithi192' 2024-09-15T22:41:53.672 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:53.671+0000 7f3a316bd640 1 -- 172.21.15.192:0/768100507 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3a140a3270 msgr2=0x7f3a140a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:53.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:53.671+0000 7f3a316bd640 1 --2- 172.21.15.192:0/768100507 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3a140a3270 0x7f3a140a5730 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c1993f0 tx=0x7f3a20009110 comp rx=0 tx=0).stop 2024-09-15T22:41:53.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:53.671+0000 7f3a316bd640 1 -- 172.21.15.192:0/768100507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a2c101c00 msgr2=0x7f3a2c197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:53.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:53.671+0000 7f3a316bd640 1 --2- 172.21.15.192:0/768100507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a2c101c00 0x7f3a2c197ed0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00d770 tx=0x7f3a1c00db70 comp rx=0 tx=0).stop 2024-09-15T22:41:53.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:53.671+0000 7f3a316bd640 1 -- 172.21.15.192:0/768100507 shutdown_connections 2024-09-15T22:41:53.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:53.671+0000 7f3a316bd640 1 --2- 172.21.15.192:0/768100507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3a2c1037c0 0x7f3a2c19f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:53.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:53.671+0000 7f3a316bd640 1 --2- 172.21.15.192:0/768100507 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3a140a3270 0x7f3a140a5730 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:53.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:53.671+0000 7f3a316bd640 1 --2- 172.21.15.192:0/768100507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a2c102e00 0x7f3a2c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:53.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:53.671+0000 7f3a316bd640 1 --2- 172.21.15.192:0/768100507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a2c101c00 0x7f3a2c197ed0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:53.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:53.671+0000 7f3a316bd640 1 -- 172.21.15.192:0/768100507 >> 172.21.15.192:0/768100507 conn(0x7f3a2c0fd390 msgr2=0x7f3a2c106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:53.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:53.671+0000 7f3a316bd640 1 -- 172.21.15.192:0/768100507 shutdown_connections 2024-09-15T22:41:53.673 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:41:53.671+0000 7f3a316bd640 1 -- 172.21.15.192:0/768100507 wait complete. 2024-09-15T22:41:53.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:53 smithi121 ceph-mon[34815]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:53.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:53 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:53.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:53 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:53.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:53 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:53.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:53 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:41:53.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:53 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:53.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:53 smithi111 ceph-mon[30629]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:53.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:53 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:53.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:53 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:53.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:53 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:53.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:53 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:41:53.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:53 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:54.324 DEBUG:teuthology.orchestra.run.smithi192:osd.8> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.8.service 2024-09-15T22:41:54.327 INFO:tasks.cephadm:Deploying osd.9 on smithi111 with /dev/vg_nvme/lv_1... 2024-09-15T22:41:54.327 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-15T22:41:54.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:54 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:41:54.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:54 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:54.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:54 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:54.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:54 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:41:54.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:54 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:54.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:54 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:54.701 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:41:54.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:54 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:41:54.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:54 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:54.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:54 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:55.731 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:55 smithi192 ceph-mon[35427]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:55.847 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:55 smithi111 ceph-mon[30629]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:55.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:55 smithi121 ceph-mon[34815]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:56.462 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 22:41:56 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T22:41:56.099+0000 7f9677ec5640 -1 osd.8 0 log_to_monitors true 2024-09-15T22:41:56.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:56 smithi111 ceph-mon[30629]: from='osd.8 [v2:172.21.15.192:6816/2417771374,v1:172.21.15.192:6817/2417771374]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-15T22:41:56.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:56 smithi111 ceph-mon[30629]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-15T22:41:56.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:56 smithi192 ceph-mon[35427]: from='osd.8 [v2:172.21.15.192:6816/2417771374,v1:172.21.15.192:6817/2417771374]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-15T22:41:56.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:56 smithi192 ceph-mon[35427]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-15T22:41:56.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:56 smithi121 ceph-mon[34815]: from='osd.8 [v2:172.21.15.192:6816/2417771374,v1:172.21.15.192:6817/2417771374]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-15T22:41:56.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:56 smithi121 ceph-mon[34815]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-15T22:41:57.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:57 smithi192 ceph-mon[35427]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:57.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:57 smithi192 ceph-mon[35427]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-15T22:41:57.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:57 smithi192 ceph-mon[35427]: osdmap e55: 9 total, 8 up, 9 in 2024-09-15T22:41:57.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:41:57.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:57 smithi192 ceph-mon[35427]: from='osd.8 [v2:172.21.15.192:6816/2417771374,v1:172.21.15.192:6817/2417771374]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-09-15T22:41:57.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:57 smithi192 ceph-mon[35427]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-09-15T22:41:57.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:57.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:57.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:41:57.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:41:57.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:41:57.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:57.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:57.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:41:57.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:57.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:57 smithi121 ceph-mon[34815]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:57.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:57 smithi121 ceph-mon[34815]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-15T22:41:57.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:57 smithi121 ceph-mon[34815]: osdmap e55: 9 total, 8 up, 9 in 2024-09-15T22:41:57.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:41:57.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:57 smithi121 ceph-mon[34815]: from='osd.8 [v2:172.21.15.192:6816/2417771374,v1:172.21.15.192:6817/2417771374]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-09-15T22:41:57.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:57 smithi121 ceph-mon[34815]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-09-15T22:41:57.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:57.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:57.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:41:57.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:41:57.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:41:57.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:57.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:57.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:41:57.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:57.905 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:57 smithi111 ceph-mon[30629]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:57.905 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:57 smithi111 ceph-mon[30629]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-15T22:41:57.905 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:57 smithi111 ceph-mon[30629]: osdmap e55: 9 total, 8 up, 9 in 2024-09-15T22:41:57.905 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:41:57.905 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:57 smithi111 ceph-mon[30629]: from='osd.8 [v2:172.21.15.192:6816/2417771374,v1:172.21.15.192:6817/2417771374]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-09-15T22:41:57.905 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:57 smithi111 ceph-mon[30629]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-09-15T22:41:57.905 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:57.905 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:57.905 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:41:57.905 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:41:57.905 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:41:57.906 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:57.906 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:57.906 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:41:57.906 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:41:58.107 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:41:58.140 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi111:vg_nvme/lv_1 2024-09-15T22:41:58.508 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 22:41:58 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T22:41:58.255+0000 7f9673625640 -1 osd.8 0 waiting for initial osdmap 2024-09-15T22:41:58.508 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 22:41:58 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T22:41:58.267+0000 7f966bc57640 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:41:58.518 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:41:58.781 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:58 smithi111 ceph-mon[30629]: Detected new or changed devices on smithi192 2024-09-15T22:41:58.781 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:58 smithi111 ceph-mon[30629]: Adjusting osd_memory_target on smithi192 to 5696M 2024-09-15T22:41:58.781 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:58 smithi111 ceph-mon[30629]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-09-15T22:41:58.781 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:58 smithi111 ceph-mon[30629]: osdmap e56: 9 total, 8 up, 9 in 2024-09-15T22:41:58.781 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:58 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:41:58.781 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:58 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:41:58.781 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:58 smithi111 ceph-mon[30629]: from='osd.8 ' entity='osd.8' 2024-09-15T22:41:58.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:58 smithi192 ceph-mon[35427]: Detected new or changed devices on smithi192 2024-09-15T22:41:58.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:58 smithi192 ceph-mon[35427]: Adjusting osd_memory_target on smithi192 to 5696M 2024-09-15T22:41:58.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:58 smithi192 ceph-mon[35427]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-09-15T22:41:58.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:58 smithi192 ceph-mon[35427]: osdmap e56: 9 total, 8 up, 9 in 2024-09-15T22:41:58.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:58 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:41:58.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:58 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:41:58.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:58 smithi192 ceph-mon[35427]: from='osd.8 ' entity='osd.8' 2024-09-15T22:41:58.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:58 smithi121 ceph-mon[34815]: Detected new or changed devices on smithi192 2024-09-15T22:41:58.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:58 smithi121 ceph-mon[34815]: Adjusting osd_memory_target on smithi192 to 5696M 2024-09-15T22:41:58.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:58 smithi121 ceph-mon[34815]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-09-15T22:41:58.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:58 smithi121 ceph-mon[34815]: osdmap e56: 9 total, 8 up, 9 in 2024-09-15T22:41:58.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:58 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:41:58.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:58 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:41:58.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:58 smithi121 ceph-mon[34815]: from='osd.8 ' entity='osd.8' 2024-09-15T22:41:59.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.255+0000 7fd7d6a00640 1 -- 172.21.15.111:0/2537511938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7d01037c0 msgr2=0x7fd7d010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:59.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.255+0000 7fd7d6a00640 1 --2- 172.21.15.111:0/2537511938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7d01037c0 0x7fd7d010a050 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8009ca0 tx=0x7fd7c802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:41:59.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.255+0000 7fd7d6a00640 1 -- 172.21.15.111:0/2537511938 shutdown_connections 2024-09-15T22:41:59.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.255+0000 7fd7d6a00640 1 --2- 172.21.15.111:0/2537511938 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7d01037c0 0x7fd7d010a050 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:59.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.255+0000 7fd7d6a00640 1 --2- 172.21.15.111:0/2537511938 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7d0102e00 0x7fd7d0103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:59.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.255+0000 7fd7d6a00640 1 --2- 172.21.15.111:0/2537511938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7d0101c00 0x7fd7d0102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:59.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.255+0000 7fd7d6a00640 1 -- 172.21.15.111:0/2537511938 >> 172.21.15.111:0/2537511938 conn(0x7fd7d00fd390 msgr2=0x7fd7d00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:41:59.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.255+0000 7fd7d6a00640 1 -- 172.21.15.111:0/2537511938 shutdown_connections 2024-09-15T22:41:59.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.255+0000 7fd7d6a00640 1 -- 172.21.15.111:0/2537511938 wait complete. 2024-09-15T22:41:59.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.256+0000 7fd7d6a00640 1 Processor -- start 2024-09-15T22:41:59.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.256+0000 7fd7d6a00640 1 -- start start 2024-09-15T22:41:59.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.256+0000 7fd7d6a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7d0101c00 0x7fd7d0075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:59.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.256+0000 7fd7d6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7d0102e00 0x7fd7d00745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:59.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.256+0000 7fd7d6a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7d01037c0 0x7fd7d0074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:59.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.256+0000 7fd7d6a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d010d860 con 0x7fd7d0102e00 2024-09-15T22:41:59.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.256+0000 7fd7d6a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d010d6e0 con 0x7fd7d01037c0 2024-09-15T22:41:59.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.256+0000 7fd7d6a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d010d9e0 con 0x7fd7d0101c00 2024-09-15T22:41:59.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.256+0000 7fd7c7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7d0102e00 0x7fd7d00745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:59.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.256+0000 7fd7c7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7d0102e00 0x7fd7d00745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42010/0 (socket says 172.21.15.111:42010) 2024-09-15T22:41:59.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.256+0000 7fd7c7400640 1 -- 172.21.15.111:0/2522379826 learned_addr learned my addr 172.21.15.111:0/2522379826 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:41:59.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.256+0000 7fd7cfe00640 1 --2- 172.21.15.111:0/2522379826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7d0101c00 0x7fd7d0075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:59.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.257+0000 7fd7d4a00640 1 --2- 172.21.15.111:0/2522379826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7d01037c0 0x7fd7d0074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:59.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.257+0000 7fd7c7400640 1 -- 172.21.15.111:0/2522379826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7d0101c00 msgr2=0x7fd7d0075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:59.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.257+0000 7fd7c7400640 1 --2- 172.21.15.111:0/2522379826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7d0101c00 0x7fd7d0075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:59.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.257+0000 7fd7c7400640 1 -- 172.21.15.111:0/2522379826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7d01037c0 msgr2=0x7fd7d0074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:41:59.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.257+0000 7fd7c7400640 1 --2- 172.21.15.111:0/2522379826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7d01037c0 0x7fd7d0074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:41:59.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.257+0000 7fd7c7400640 1 -- 172.21.15.111:0/2522379826 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d00753e0 con 0x7fd7d0102e00 2024-09-15T22:41:59.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.257+0000 7fd7c7400640 1 --2- 172.21.15.111:0/2522379826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7d0102e00 0x7fd7d00745e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800e9f0 tx=0x7fd7b800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:59.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.257+0000 7fd7cd600640 1 -- 172.21.15.111:0/2522379826 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b800cd80 con 0x7fd7d0102e00 2024-09-15T22:41:59.290 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.257+0000 7fd7cd600640 1 -- 172.21.15.111:0/2522379826 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7b8010910 con 0x7fd7d0102e00 2024-09-15T22:41:59.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.257+0000 7fd7cd600640 1 -- 172.21.15.111:0/2522379826 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8018990 con 0x7fd7d0102e00 2024-09-15T22:41:59.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.257+0000 7fd7d6a00640 1 -- 172.21.15.111:0/2522379826 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d01a1fe0 con 0x7fd7d0102e00 2024-09-15T22:41:59.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.258+0000 7fd7d6a00640 1 -- 172.21.15.111:0/2522379826 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d01a2440 con 0x7fd7d0102e00 2024-09-15T22:41:59.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.259+0000 7fd7d6a00640 1 -- 172.21.15.111:0/2522379826 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7d0108fb0 con 0x7fd7d0102e00 2024-09-15T22:41:59.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.261+0000 7fd7cd600640 1 -- 172.21.15.111:0/2522379826 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fd7b8010ac0 con 0x7fd7d0102e00 2024-09-15T22:41:59.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.262+0000 7fd7cd600640 1 --2- 172.21.15.111:0/2522379826 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd7b40a3240 0x7fd7b40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:41:59.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.262+0000 7fd7cd600640 1 -- 172.21.15.111:0/2522379826 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5858+0+0 (secure 0 0 0) 0x7fd7b80cf310 con 0x7fd7d0102e00 2024-09-15T22:41:59.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.262+0000 7fd7cfe00640 1 --2- 172.21.15.111:0/2522379826 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd7b40a3240 0x7fd7b40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:41:59.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.263+0000 7fd7cfe00640 1 --2- 172.21.15.111:0/2522379826 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd7b40a3240 0x7fd7b40a5700 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0102c60 tx=0x7fd7c0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:41:59.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.266+0000 7fd7cd600640 1 -- 172.21.15.111:0/2522379826 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd7b809c2d0 con 0x7fd7d0102e00 2024-09-15T22:41:59.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:41:59.451+0000 7fd7d6a00640 1 -- 172.21.15.111:0/2522379826 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd7d0062630 con 0x7fd7b40a3240 2024-09-15T22:41:59.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:59 smithi192 ceph-mon[35427]: purged_snaps scrub starts 2024-09-15T22:41:59.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:59 smithi192 ceph-mon[35427]: purged_snaps scrub ok 2024-09-15T22:41:59.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:59 smithi192 ceph-mon[35427]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:59.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:59 smithi192 ceph-mon[35427]: osd.8 [v2:172.21.15.192:6816/2417771374,v1:172.21.15.192:6817/2417771374] boot 2024-09-15T22:41:59.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:59 smithi192 ceph-mon[35427]: osdmap e57: 9 total, 9 up, 9 in 2024-09-15T22:41:59.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:59 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:41:59.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:59 smithi192 ceph-mon[35427]: osdmap e58: 9 total, 9 up, 9 in 2024-09-15T22:41:59.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:59 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:41:59.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:59 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:41:59.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:41:59 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:59.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:59 smithi121 ceph-mon[34815]: purged_snaps scrub starts 2024-09-15T22:41:59.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:59 smithi121 ceph-mon[34815]: purged_snaps scrub ok 2024-09-15T22:41:59.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:59 smithi121 ceph-mon[34815]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:59.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:59 smithi121 ceph-mon[34815]: osd.8 [v2:172.21.15.192:6816/2417771374,v1:172.21.15.192:6817/2417771374] boot 2024-09-15T22:41:59.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:59 smithi121 ceph-mon[34815]: osdmap e57: 9 total, 9 up, 9 in 2024-09-15T22:41:59.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:59 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:41:59.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:59 smithi121 ceph-mon[34815]: osdmap e58: 9 total, 9 up, 9 in 2024-09-15T22:41:59.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:59 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:41:59.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:59 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:41:59.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:41:59 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:41:59.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:59 smithi111 ceph-mon[30629]: purged_snaps scrub starts 2024-09-15T22:41:59.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:59 smithi111 ceph-mon[30629]: purged_snaps scrub ok 2024-09-15T22:41:59.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:59 smithi111 ceph-mon[30629]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:41:59.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:59 smithi111 ceph-mon[30629]: osd.8 [v2:172.21.15.192:6816/2417771374,v1:172.21.15.192:6817/2417771374] boot 2024-09-15T22:41:59.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:59 smithi111 ceph-mon[30629]: osdmap e57: 9 total, 9 up, 9 in 2024-09-15T22:41:59.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:59 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:41:59.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:59 smithi111 ceph-mon[30629]: osdmap e58: 9 total, 9 up, 9 in 2024-09-15T22:41:59.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:59 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:41:59.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:59 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:41:59.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:41:59 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:00.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:00 smithi111 ceph-mon[30629]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:42:00.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:00 smithi111 ceph-mon[30629]: osdmap e59: 9 total, 9 up, 9 in 2024-09-15T22:42:00.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:00 smithi192 ceph-mon[35427]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:42:00.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:00 smithi192 ceph-mon[35427]: osdmap e59: 9 total, 9 up, 9 in 2024-09-15T22:42:00.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:00 smithi121 ceph-mon[34815]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:42:00.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:00 smithi121 ceph-mon[34815]: osdmap e59: 9 total, 9 up, 9 in 2024-09-15T22:42:01.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:01 smithi111 ceph-mon[30629]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 670 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:01.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:01 smithi111 ceph-mon[30629]: osdmap e60: 9 total, 9 up, 9 in 2024-09-15T22:42:01.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:01 smithi192 ceph-mon[35427]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 670 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:01.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:01 smithi192 ceph-mon[35427]: osdmap e60: 9 total, 9 up, 9 in 2024-09-15T22:42:01.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:01 smithi121 ceph-mon[34815]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 670 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:01.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:01 smithi121 ceph-mon[34815]: osdmap e60: 9 total, 9 up, 9 in 2024-09-15T22:42:03.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:03 smithi192 ceph-mon[35427]: osdmap e61: 9 total, 9 up, 9 in 2024-09-15T22:42:03.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:03 smithi192 ceph-mon[35427]: pgmap v207: 1 pgs: 1 remapped+peering; 577 KiB data, 671 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:03.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:03 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/3840274965' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71b4f5fb-1c02-461b-a257-c6e48381d768"}]: dispatch 2024-09-15T22:42:03.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:03 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/3840274965' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71b4f5fb-1c02-461b-a257-c6e48381d768"}]': finished 2024-09-15T22:42:03.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:03 smithi192 ceph-mon[35427]: osdmap e62: 10 total, 9 up, 10 in 2024-09-15T22:42:03.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:03 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:42:03.362 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:03 smithi111 ceph-mon[30629]: osdmap e61: 9 total, 9 up, 9 in 2024-09-15T22:42:03.362 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:03 smithi111 ceph-mon[30629]: pgmap v207: 1 pgs: 1 remapped+peering; 577 KiB data, 671 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:03.362 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:03 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/3840274965' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71b4f5fb-1c02-461b-a257-c6e48381d768"}]: dispatch 2024-09-15T22:42:03.362 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:03 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/3840274965' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71b4f5fb-1c02-461b-a257-c6e48381d768"}]': finished 2024-09-15T22:42:03.362 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:03 smithi111 ceph-mon[30629]: osdmap e62: 10 total, 9 up, 10 in 2024-09-15T22:42:03.362 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:03 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:42:03.393 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:03 smithi121 ceph-mon[34815]: osdmap e61: 9 total, 9 up, 9 in 2024-09-15T22:42:03.393 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:03 smithi121 ceph-mon[34815]: pgmap v207: 1 pgs: 1 remapped+peering; 577 KiB data, 671 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:03.393 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:03 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/3840274965' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "71b4f5fb-1c02-461b-a257-c6e48381d768"}]: dispatch 2024-09-15T22:42:03.393 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:03 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/3840274965' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "71b4f5fb-1c02-461b-a257-c6e48381d768"}]': finished 2024-09-15T22:42:03.393 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:03 smithi121 ceph-mon[34815]: osdmap e62: 10 total, 9 up, 10 in 2024-09-15T22:42:03.393 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:03 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:42:04.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:04 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/629301671' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:42:04.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:04 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/629301671' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:42:04.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:04 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/629301671' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:42:05.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:05 smithi192 ceph-mon[35427]: pgmap v209: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:05.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:05 smithi121 ceph-mon[34815]: pgmap v209: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:05.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:05 smithi111 ceph-mon[30629]: pgmap v209: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:07.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:07 smithi192 ceph-mon[35427]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:42:07.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:07 smithi121 ceph-mon[34815]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:42:07.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:07 smithi111 ceph-mon[30629]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:42:09.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:09 smithi111 ceph-mon[30629]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-15T22:42:09.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:09 smithi192 ceph-mon[35427]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-15T22:42:09.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:09 smithi121 ceph-mon[34815]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-15T22:42:11.707 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:11 smithi111 ceph-mon[30629]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-15T22:42:11.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:11 smithi192 ceph-mon[35427]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-15T22:42:11.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:11 smithi121 ceph-mon[34815]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-15T22:42:13.789 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:13 smithi111 ceph-mon[30629]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T22:42:13.789 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:13 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-15T22:42:13.789 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:13 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:13.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:13 smithi192 ceph-mon[35427]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T22:42:13.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:13 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-15T22:42:13.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:13 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:13.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:13 smithi121 ceph-mon[34815]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T22:42:13.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:13 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-15T22:42:13.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:13 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:14.530 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:14 smithi111 ceph-mon[30629]: Deploying daemon osd.9 on smithi111 2024-09-15T22:42:14.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:14 smithi192 ceph-mon[35427]: Deploying daemon osd.9 on smithi111 2024-09-15T22:42:14.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:14 smithi121 ceph-mon[34815]: Deploying daemon osd.9 on smithi111 2024-09-15T22:42:15.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:15 smithi111 ceph-mon[30629]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-15T22:42:15.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:15 smithi192 ceph-mon[35427]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-15T22:42:15.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:15 smithi121 ceph-mon[34815]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-15T22:42:17.564 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:17 smithi111 ceph-mon[30629]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:42:17.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:17 smithi192 ceph-mon[35427]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:42:17.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:17 smithi121 ceph-mon[34815]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:42:19.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:19 smithi192 ceph-mon[35427]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:19.843 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:19 smithi111 ceph-mon[30629]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:19.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:19 smithi121 ceph-mon[34815]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:20.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:20 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:42:20.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:20 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:20.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:20 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:20.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:20 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:42:20.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:20 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:20.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:20 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:20.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:20 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:42:20.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:20 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:20.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:20 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:21.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:21 smithi111 ceph-mon[30629]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:21.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:21 smithi192 ceph-mon[35427]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:21.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:21 smithi121 ceph-mon[34815]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:23.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:23 smithi192 ceph-mon[35427]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:23.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:23 smithi121 ceph-mon[34815]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:23.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:23 smithi111 ceph-mon[30629]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:24.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:24 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:24.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:24 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:24.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:24 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:24.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:24 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:42:24.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:24 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:24.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:24 smithi111 ceph-mon[30629]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:24.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:42:24.966+0000 7fd7cd600640 1 -- 172.21.15.111:0/2522379826 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd7d0062630 con 0x7fd7b40a3240 2024-09-15T22:42:24.969 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 9 on host 'smithi111' 2024-09-15T22:42:24.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:42:24.969+0000 7fd7d6a00640 1 -- 172.21.15.111:0/2522379826 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd7b40a3240 msgr2=0x7fd7b40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:24.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:42:24.969+0000 7fd7d6a00640 1 --2- 172.21.15.111:0/2522379826 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd7b40a3240 0x7fd7b40a5700 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0102c60 tx=0x7fd7c0005f90 comp rx=0 tx=0).stop 2024-09-15T22:42:24.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:42:24.969+0000 7fd7d6a00640 1 -- 172.21.15.111:0/2522379826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7d0102e00 msgr2=0x7fd7d00745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:24.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:42:24.969+0000 7fd7d6a00640 1 --2- 172.21.15.111:0/2522379826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7d0102e00 0x7fd7d00745e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800e9f0 tx=0x7fd7b800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:42:24.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:42:24.969+0000 7fd7d6a00640 1 -- 172.21.15.111:0/2522379826 shutdown_connections 2024-09-15T22:42:24.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:42:24.969+0000 7fd7d6a00640 1 --2- 172.21.15.111:0/2522379826 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd7b40a3240 0x7fd7b40a5700 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:24.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:42:24.969+0000 7fd7d6a00640 1 --2- 172.21.15.111:0/2522379826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7d01037c0 0x7fd7d0074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:24.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:42:24.969+0000 7fd7d6a00640 1 --2- 172.21.15.111:0/2522379826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7d0102e00 0x7fd7d00745e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:24.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:42:24.969+0000 7fd7d6a00640 1 --2- 172.21.15.111:0/2522379826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7d0101c00 0x7fd7d0075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:24.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:42:24.969+0000 7fd7d6a00640 1 -- 172.21.15.111:0/2522379826 >> 172.21.15.111:0/2522379826 conn(0x7fd7d00fd390 msgr2=0x7fd7d0106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:24.971 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:42:24.969+0000 7fd7d6a00640 1 -- 172.21.15.111:0/2522379826 shutdown_connections 2024-09-15T22:42:24.972 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:42:24.969+0000 7fd7d6a00640 1 -- 172.21.15.111:0/2522379826 wait complete. 2024-09-15T22:42:25.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:24 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:25.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:24 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:25.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:24 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:25.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:24 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:42:25.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:24 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:25.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:24 smithi192 ceph-mon[35427]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:25.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:24 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:25.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:24 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:25.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:24 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:25.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:24 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:42:25.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:24 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:25.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:24 smithi121 ceph-mon[34815]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:25.776 DEBUG:teuthology.orchestra.run.smithi111:osd.9> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.9.service 2024-09-15T22:42:25.780 INFO:tasks.cephadm:Deploying osd.10 on smithi121 with /dev/vg_nvme/lv_1... 2024-09-15T22:42:25.780 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-15T22:42:25.961 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:25 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:42:25.961 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:25 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:25.961 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:25 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:26.035 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:42:26.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:25 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:42:26.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:25 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:26.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:25 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:26.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:25 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:42:26.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:25 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:26.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:25 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:27.000 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:26 smithi111 ceph-mon[30629]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:27.000 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:26 smithi111 ceph-mon[30629]: from='osd.9 [v2:172.21.15.111:6826/3979119431,v1:172.21.15.111:6827/3979119431]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-15T22:42:27.001 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 22:42:26 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T22:42:26.716+0000 7f3f9dd61640 -1 osd.9 0 log_to_monitors true 2024-09-15T22:42:27.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:26 smithi192 ceph-mon[35427]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:27.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:26 smithi192 ceph-mon[35427]: from='osd.9 [v2:172.21.15.111:6826/3979119431,v1:172.21.15.111:6827/3979119431]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-15T22:42:27.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:26 smithi121 ceph-mon[34815]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:27.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:26 smithi121 ceph-mon[34815]: from='osd.9 [v2:172.21.15.111:6826/3979119431,v1:172.21.15.111:6827/3979119431]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-15T22:42:28.016 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:27 smithi121 ceph-mon[34815]: from='osd.9 [v2:172.21.15.111:6826/3979119431,v1:172.21.15.111:6827/3979119431]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-15T22:42:28.016 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:27 smithi121 ceph-mon[34815]: osdmap e63: 10 total, 9 up, 10 in 2024-09-15T22:42:28.017 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:27 smithi121 ceph-mon[34815]: from='osd.9 [v2:172.21.15.111:6826/3979119431,v1:172.21.15.111:6827/3979119431]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-15T22:42:28.017 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:27 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:42:28.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:27 smithi192 ceph-mon[35427]: from='osd.9 [v2:172.21.15.111:6826/3979119431,v1:172.21.15.111:6827/3979119431]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-15T22:42:28.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:27 smithi192 ceph-mon[35427]: osdmap e63: 10 total, 9 up, 10 in 2024-09-15T22:42:28.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:27 smithi192 ceph-mon[35427]: from='osd.9 [v2:172.21.15.111:6826/3979119431,v1:172.21.15.111:6827/3979119431]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-15T22:42:28.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:27 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:42:28.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:27 smithi111 ceph-mon[30629]: from='osd.9 [v2:172.21.15.111:6826/3979119431,v1:172.21.15.111:6827/3979119431]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-15T22:42:28.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:27 smithi111 ceph-mon[30629]: osdmap e63: 10 total, 9 up, 10 in 2024-09-15T22:42:28.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:27 smithi111 ceph-mon[30629]: from='osd.9 [v2:172.21.15.111:6826/3979119431,v1:172.21.15.111:6827/3979119431]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2024-09-15T22:42:28.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:27 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:42:28.685 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-15T22:42:28.713 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi121:vg_nvme/lv_1 2024-09-15T22:42:28.854 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:28 smithi111 ceph-mon[30629]: from='osd.9 [v2:172.21.15.111:6826/3979119431,v1:172.21.15.111:6827/3979119431]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-15T22:42:28.855 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:28 smithi111 ceph-mon[30629]: osdmap e64: 10 total, 9 up, 10 in 2024-09-15T22:42:28.855 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:28 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:42:28.855 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:28 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:42:28.855 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:28 smithi111 ceph-mon[30629]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:28.855 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 22:42:28 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T22:42:28.741+0000 7f3f99600640 -1 osd.9 0 waiting for initial osdmap 2024-09-15T22:42:28.855 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 22:42:28 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T22:42:28.773+0000 7f3f8ec00640 -1 osd.9 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:42:28.970 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:42:29.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:28 smithi192 ceph-mon[35427]: from='osd.9 [v2:172.21.15.111:6826/3979119431,v1:172.21.15.111:6827/3979119431]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-15T22:42:29.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:28 smithi192 ceph-mon[35427]: osdmap e64: 10 total, 9 up, 10 in 2024-09-15T22:42:29.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:28 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:42:29.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:28 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:42:29.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:28 smithi192 ceph-mon[35427]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:29.091 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:28 smithi121 ceph-mon[34815]: from='osd.9 [v2:172.21.15.111:6826/3979119431,v1:172.21.15.111:6827/3979119431]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2024-09-15T22:42:29.091 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:28 smithi121 ceph-mon[34815]: osdmap e64: 10 total, 9 up, 10 in 2024-09-15T22:42:29.091 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:28 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:42:29.091 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:28 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:42:29.091 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:28 smithi121 ceph-mon[34815]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:29.455 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.425+0000 7f1445ec5640 1 -- 172.21.15.121:0/736387930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1440102660 msgr2=0x7f1440104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:29.455 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.425+0000 7f1445ec5640 1 --2- 172.21.15.121:0/736387930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1440102660 0x7f1440104a50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1430009900 tx=0x7f143002a410 comp rx=0 tx=0).stop 2024-09-15T22:42:29.456 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.425+0000 7f1445ec5640 1 -- 172.21.15.121:0/736387930 shutdown_connections 2024-09-15T22:42:29.456 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.425+0000 7f1445ec5640 1 --2- 172.21.15.121:0/736387930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14401078c0 0x7f1440109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:29.456 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.425+0000 7f1445ec5640 1 --2- 172.21.15.121:0/736387930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1440104f90 0x7f1440107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:29.456 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.425+0000 7f1445ec5640 1 --2- 172.21.15.121:0/736387930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1440102660 0x7f1440104a50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:29.456 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.425+0000 7f1445ec5640 1 -- 172.21.15.121:0/736387930 >> 172.21.15.121:0/736387930 conn(0x7f14400fc280 msgr2=0x7f14400fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:29.456 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.425+0000 7f1445ec5640 1 -- 172.21.15.121:0/736387930 shutdown_connections 2024-09-15T22:42:29.456 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.425+0000 7f1445ec5640 1 -- 172.21.15.121:0/736387930 wait complete. 2024-09-15T22:42:29.456 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.426+0000 7f1445ec5640 1 Processor -- start 2024-09-15T22:42:29.456 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.426+0000 7f1445ec5640 1 -- start start 2024-09-15T22:42:29.456 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.426+0000 7f1445ec5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1440102660 0x7f1440197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:29.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.426+0000 7f1445ec5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1440104f90 0x7f14401984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:29.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.426+0000 7f1445ec5640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14401078c0 0x7f144019f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:29.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.426+0000 7f1445ec5640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f144010d8e0 con 0x7f14401078c0 2024-09-15T22:42:29.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.426+0000 7f1445ec5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f144010d760 con 0x7f1440104f90 2024-09-15T22:42:29.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.426+0000 7f1445ec5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f144010da60 con 0x7f1440102660 2024-09-15T22:42:29.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.426+0000 7f143effd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1440104f90 0x7f14401984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:29.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.427+0000 7f143effd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1440104f90 0x7f14401984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59254/0 (socket says 172.21.15.121:59254) 2024-09-15T22:42:29.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.427+0000 7f143effd640 1 -- 172.21.15.121:0/3906157490 learned_addr learned my addr 172.21.15.121:0/3906157490 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-09-15T22:42:29.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.427+0000 7f143f7fe640 1 --2- 172.21.15.121:0/3906157490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1440102660 0x7f1440197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:29.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.427+0000 7f143effd640 1 -- 172.21.15.121:0/3906157490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1440102660 msgr2=0x7f1440197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:29.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.427+0000 7f143effd640 1 --2- 172.21.15.121:0/3906157490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1440102660 0x7f1440197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:29.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.427+0000 7f143effd640 1 -- 172.21.15.121:0/3906157490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14401078c0 msgr2=0x7f144019f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:29.458 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.427+0000 7f143effd640 1 --2- 172.21.15.121:0/3906157490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14401078c0 0x7f144019f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:29.458 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.427+0000 7f143effd640 1 -- 172.21.15.121:0/3906157490 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144019fbc0 con 0x7f1440104f90 2024-09-15T22:42:29.458 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.427+0000 7f143effd640 1 --2- 172.21.15.121:0/3906157490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1440104f90 0x7f14401984d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f143400c970 tx=0x7f143400ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:29.458 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.427+0000 7f143cff9640 1 -- 172.21.15.121:0/3906157490 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1434007bb0 con 0x7f1440104f90 2024-09-15T22:42:29.458 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.427+0000 7f143cff9640 1 -- 172.21.15.121:0/3906157490 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1434007d50 con 0x7f1440104f90 2024-09-15T22:42:29.458 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.427+0000 7f1445ec5640 1 -- 172.21.15.121:0/3906157490 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f144019feb0 con 0x7f1440104f90 2024-09-15T22:42:29.458 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.427+0000 7f143cff9640 1 -- 172.21.15.121:0/3906157490 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143400f450 con 0x7f1440104f90 2024-09-15T22:42:29.458 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.427+0000 7f1445ec5640 1 -- 172.21.15.121:0/3906157490 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14401a03e0 con 0x7f1440104f90 2024-09-15T22:42:29.458 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.428+0000 7f1445ec5640 1 -- 172.21.15.121:0/3906157490 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14400fdd70 con 0x7f1440104f90 2024-09-15T22:42:29.458 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.429+0000 7f143cff9640 1 -- 172.21.15.121:0/3906157490 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f14340046c0 con 0x7f1440104f90 2024-09-15T22:42:29.458 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.430+0000 7f143cff9640 1 --2- 172.21.15.121:0/3906157490 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f14280a3270 0x7f14280a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:29.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.430+0000 7f143cff9640 1 -- 172.21.15.121:0/3906157490 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5981+0+0 (secure 0 0 0) 0x7f14340ced50 con 0x7f1440104f90 2024-09-15T22:42:29.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.430+0000 7f143f7fe640 1 --2- 172.21.15.121:0/3906157490 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f14280a3270 0x7f14280a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:29.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.431+0000 7f143f7fe640 1 --2- 172.21.15.121:0/3906157490 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f14280a3270 0x7f14280a5730 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1430000c00 tx=0x7f1430031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:29.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.434+0000 7f143cff9640 1 -- 172.21.15.121:0/3906157490 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f14340d2030 con 0x7f1440104f90 2024-09-15T22:42:29.610 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:29.609+0000 7f1445ec5640 1 -- 172.21.15.121:0/3906157490 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1440062630 con 0x7f14280a3270 2024-09-15T22:42:29.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:29 smithi121 ceph-mon[34815]: purged_snaps scrub starts 2024-09-15T22:42:29.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:29 smithi121 ceph-mon[34815]: purged_snaps scrub ok 2024-09-15T22:42:29.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:29 smithi121 ceph-mon[34815]: from='osd.9 [v2:172.21.15.111:6826/3979119431,v1:172.21.15.111:6827/3979119431]' entity='osd.9' 2024-09-15T22:42:29.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:29 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:42:29.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:29 smithi121 ceph-mon[34815]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:42:29.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:29 smithi121 ceph-mon[34815]: Detected new or changed devices on smithi111 2024-09-15T22:42:29.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:29 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:42:29.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:29 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:42:29.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:29 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:29.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:29 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:29.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:29 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:29.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:29 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:42:29.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:29 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:42:29.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:29 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:42:29.892 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:29 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:42:29.892 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:29 smithi121 ceph-mon[34815]: Adjusting osd_memory_target on smithi111 to 4272M 2024-09-15T22:42:29.892 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:29 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:29.892 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:29 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:29.892 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:29 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:42:29.892 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:29 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:29.892 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:29 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:42:29.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:29 smithi111 ceph-mon[30629]: purged_snaps scrub starts 2024-09-15T22:42:29.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:29 smithi111 ceph-mon[30629]: purged_snaps scrub ok 2024-09-15T22:42:29.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:29 smithi111 ceph-mon[30629]: from='osd.9 [v2:172.21.15.111:6826/3979119431,v1:172.21.15.111:6827/3979119431]' entity='osd.9' 2024-09-15T22:42:29.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:29 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:42:29.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:29 smithi111 ceph-mon[30629]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:42:29.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:29 smithi111 ceph-mon[30629]: Detected new or changed devices on smithi111 2024-09-15T22:42:29.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:29 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:42:29.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:29 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:42:29.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:29 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:29.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:29 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:29.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:29 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:29.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:29 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:42:29.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:29 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:42:29.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:29 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:42:29.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:29 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:42:29.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:29 smithi111 ceph-mon[30629]: Adjusting osd_memory_target on smithi111 to 4272M 2024-09-15T22:42:29.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:29 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:29.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:29 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:29.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:29 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:42:29.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:29 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:29.918 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:29 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:42:30.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:29 smithi192 ceph-mon[35427]: purged_snaps scrub starts 2024-09-15T22:42:30.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:29 smithi192 ceph-mon[35427]: purged_snaps scrub ok 2024-09-15T22:42:30.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:29 smithi192 ceph-mon[35427]: from='osd.9 [v2:172.21.15.111:6826/3979119431,v1:172.21.15.111:6827/3979119431]' entity='osd.9' 2024-09-15T22:42:30.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:29 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:42:30.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:29 smithi192 ceph-mon[35427]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:42:30.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:29 smithi192 ceph-mon[35427]: Detected new or changed devices on smithi111 2024-09-15T22:42:30.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:29 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:42:30.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:29 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:42:30.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:29 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:30.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:29 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:30.066 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:29 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:30.066 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:29 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:42:30.066 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:29 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:42:30.066 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:29 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:42:30.066 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:29 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:42:30.066 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:29 smithi192 ceph-mon[35427]: Adjusting osd_memory_target on smithi111 to 4272M 2024-09-15T22:42:30.066 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:29 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:30.066 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:29 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:30.066 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:29 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:42:30.066 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:29 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:30.066 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:29 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:42:30.974 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:30 smithi121 ceph-mon[34815]: osd.9 [v2:172.21.15.111:6826/3979119431,v1:172.21.15.111:6827/3979119431] boot 2024-09-15T22:42:30.975 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:30 smithi121 ceph-mon[34815]: osdmap e65: 10 total, 10 up, 10 in 2024-09-15T22:42:30.975 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:30 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:42:30.975 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:30 smithi121 ceph-mon[34815]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:31.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:30 smithi192 ceph-mon[35427]: osd.9 [v2:172.21.15.111:6826/3979119431,v1:172.21.15.111:6827/3979119431] boot 2024-09-15T22:42:31.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:30 smithi192 ceph-mon[35427]: osdmap e65: 10 total, 10 up, 10 in 2024-09-15T22:42:31.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:30 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:42:31.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:30 smithi192 ceph-mon[35427]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:31.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:30 smithi111 ceph-mon[30629]: osd.9 [v2:172.21.15.111:6826/3979119431,v1:172.21.15.111:6827/3979119431] boot 2024-09-15T22:42:31.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:30 smithi111 ceph-mon[30629]: osdmap e65: 10 total, 10 up, 10 in 2024-09-15T22:42:31.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:30 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:42:31.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:30 smithi111 ceph-mon[30629]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:42:31.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:31 smithi121 ceph-mon[34815]: osdmap e66: 10 total, 10 up, 10 in 2024-09-15T22:42:32.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:31 smithi192 ceph-mon[35427]: osdmap e66: 10 total, 10 up, 10 in 2024-09-15T22:42:32.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:31 smithi111 ceph-mon[30629]: osdmap e66: 10 total, 10 up, 10 in 2024-09-15T22:42:33.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:32 smithi192 ceph-mon[35427]: from='client.? 172.21.15.121:0/642901703' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6a39e77-fce3-4e22-bf66-a677c9f8b6bd"}]: dispatch 2024-09-15T22:42:33.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:32 smithi192 ceph-mon[35427]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6a39e77-fce3-4e22-bf66-a677c9f8b6bd"}]: dispatch 2024-09-15T22:42:33.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:32 smithi192 ceph-mon[35427]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6a39e77-fce3-4e22-bf66-a677c9f8b6bd"}]': finished 2024-09-15T22:42:33.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:32 smithi192 ceph-mon[35427]: osdmap e67: 11 total, 10 up, 11 in 2024-09-15T22:42:33.066 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:32 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:42:33.066 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:32 smithi192 ceph-mon[35427]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:33.066 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:32 smithi192 ceph-mon[35427]: from='client.? 172.21.15.121:0/585841546' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:42:33.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:32 smithi121 ceph-mon[34815]: from='client.? 172.21.15.121:0/642901703' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6a39e77-fce3-4e22-bf66-a677c9f8b6bd"}]: dispatch 2024-09-15T22:42:33.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:32 smithi121 ceph-mon[34815]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6a39e77-fce3-4e22-bf66-a677c9f8b6bd"}]: dispatch 2024-09-15T22:42:33.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:32 smithi121 ceph-mon[34815]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6a39e77-fce3-4e22-bf66-a677c9f8b6bd"}]': finished 2024-09-15T22:42:33.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:32 smithi121 ceph-mon[34815]: osdmap e67: 11 total, 10 up, 11 in 2024-09-15T22:42:33.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:32 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:42:33.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:32 smithi121 ceph-mon[34815]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:33.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:32 smithi121 ceph-mon[34815]: from='client.? 172.21.15.121:0/585841546' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:42:33.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:32 smithi111 ceph-mon[30629]: from='client.? 172.21.15.121:0/642901703' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6a39e77-fce3-4e22-bf66-a677c9f8b6bd"}]: dispatch 2024-09-15T22:42:33.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:32 smithi111 ceph-mon[30629]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6a39e77-fce3-4e22-bf66-a677c9f8b6bd"}]: dispatch 2024-09-15T22:42:33.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:32 smithi111 ceph-mon[30629]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6a39e77-fce3-4e22-bf66-a677c9f8b6bd"}]': finished 2024-09-15T22:42:33.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:32 smithi111 ceph-mon[30629]: osdmap e67: 11 total, 10 up, 11 in 2024-09-15T22:42:33.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:32 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:42:33.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:32 smithi111 ceph-mon[30629]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:33.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:32 smithi111 ceph-mon[30629]: from='client.? 172.21.15.121:0/585841546' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:42:35.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:35 smithi192 ceph-mon[35427]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:35.843 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:35 smithi121 ceph-mon[34815]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:35.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:35 smithi111 ceph-mon[30629]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:37.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:37 smithi121 ceph-mon[34815]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:37.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:37 smithi192 ceph-mon[35427]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:37.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:37 smithi111 ceph-mon[30629]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:39.703 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:39 smithi121 ceph-mon[34815]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:39.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:39 smithi192 ceph-mon[35427]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:39.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:39 smithi111 ceph-mon[30629]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:41.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:41 smithi192 ceph-mon[35427]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:41.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:41 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-15T22:42:41.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:41 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:41.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:41 smithi121 ceph-mon[34815]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:41.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:41 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-15T22:42:41.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:41 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:41.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:41 smithi111 ceph-mon[30629]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:41.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:41 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-15T22:42:41.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:41 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:42.722 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:42 smithi121 ceph-mon[34815]: Deploying daemon osd.10 on smithi121 2024-09-15T22:42:42.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:42 smithi192 ceph-mon[35427]: Deploying daemon osd.10 on smithi121 2024-09-15T22:42:42.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:42 smithi111 ceph-mon[30629]: Deploying daemon osd.10 on smithi121 2024-09-15T22:42:43.785 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:43 smithi121 ceph-mon[34815]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:43.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:43 smithi192 ceph-mon[35427]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:43.817 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:43 smithi111 ceph-mon[30629]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:45.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:45 smithi192 ceph-mon[35427]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:45.883 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:45 smithi121 ceph-mon[34815]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:45.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:45 smithi111 ceph-mon[30629]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:46.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:42:46.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:46.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:46.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:42:46.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:46.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:46.875 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:42:46.875 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:46.875 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:47.761 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:47 smithi121 ceph-mon[34815]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:47.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:47 smithi192 ceph-mon[35427]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:47.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:47 smithi111 ceph-mon[30629]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:49.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:49 smithi192 ceph-mon[35427]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:49.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:49 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:49.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:49 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:49.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:49 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:49.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:49 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:42:49.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:49 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:49.863 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:49 smithi121 ceph-mon[34815]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:49.864 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:49 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:49.864 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:49 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:49.864 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:49 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:49.864 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:49 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:42:49.864 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:49 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:49.904 INFO:teuthology.orchestra.run.smithi121.stdout:Created osd(s) 10 on host 'smithi121' 2024-09-15T22:42:49.905 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:49.902+0000 7f143cff9640 1 -- 172.21.15.121:0/3906157490 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f1440062630 con 0x7f14280a3270 2024-09-15T22:42:49.907 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:49.904+0000 7f1445ec5640 1 -- 172.21.15.121:0/3906157490 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f14280a3270 msgr2=0x7f14280a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:49.907 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:49.904+0000 7f1445ec5640 1 --2- 172.21.15.121:0/3906157490 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f14280a3270 0x7f14280a5730 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1430000c00 tx=0x7f1430031400 comp rx=0 tx=0).stop 2024-09-15T22:42:49.907 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:49.905+0000 7f1445ec5640 1 -- 172.21.15.121:0/3906157490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1440104f90 msgr2=0x7f14401984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:49.907 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:49.905+0000 7f1445ec5640 1 --2- 172.21.15.121:0/3906157490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1440104f90 0x7f14401984d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f143400c970 tx=0x7f143400ce30 comp rx=0 tx=0).stop 2024-09-15T22:42:49.907 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:49.905+0000 7f1445ec5640 1 -- 172.21.15.121:0/3906157490 shutdown_connections 2024-09-15T22:42:49.907 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:49.905+0000 7f1445ec5640 1 --2- 172.21.15.121:0/3906157490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14401078c0 0x7f144019f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:49.907 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:49.905+0000 7f1445ec5640 1 --2- 172.21.15.121:0/3906157490 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f14280a3270 0x7f14280a5730 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:49.907 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:49.905+0000 7f1445ec5640 1 --2- 172.21.15.121:0/3906157490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1440104f90 0x7f14401984d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:49.907 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:49.905+0000 7f1445ec5640 1 --2- 172.21.15.121:0/3906157490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1440102660 0x7f1440197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:49.907 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:49.905+0000 7f1445ec5640 1 -- 172.21.15.121:0/3906157490 >> 172.21.15.121:0/3906157490 conn(0x7f14400fc280 msgr2=0x7f1440102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:49.907 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:49.905+0000 7f1445ec5640 1 -- 172.21.15.121:0/3906157490 shutdown_connections 2024-09-15T22:42:49.908 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-15T22:42:49.905+0000 7f1445ec5640 1 -- 172.21.15.121:0/3906157490 wait complete. 2024-09-15T22:42:49.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:49 smithi111 ceph-mon[30629]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:49.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:49 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:49.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:49 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:49.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:49 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:49.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:49 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:42:49.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:49 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:50.519 DEBUG:teuthology.orchestra.run.smithi121:osd.10> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.10.service 2024-09-15T22:42:50.522 INFO:tasks.cephadm:Deploying osd.11 on smithi192 with /dev/vg_nvme/lv_1... 2024-09-15T22:42:50.522 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-15T22:42:50.777 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:42:50.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:42:50.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:50.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:50.832 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:42:50.832 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:50.832 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:50.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:42:50.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:50.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:51.548 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:51 smithi192 ceph-mon[35427]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:51.690 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:51 smithi121 ceph-mon[34815]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:51.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:51 smithi111 ceph-mon[30629]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:52.685 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:52 smithi121 ceph-mon[34815]: from='osd.10 [v2:172.21.15.121:6824/2897902637,v1:172.21.15.121:6825/2897902637]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T22:42:52.685 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:52 smithi121 ceph-mon[34815]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T22:42:52.685 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 22:42:52 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T22:42:52.420+0000 7f5917a19640 -1 osd.10 0 log_to_monitors true 2024-09-15T22:42:52.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:52 smithi192 ceph-mon[35427]: from='osd.10 [v2:172.21.15.121:6824/2897902637,v1:172.21.15.121:6825/2897902637]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T22:42:52.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:52 smithi192 ceph-mon[35427]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T22:42:52.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:52 smithi111 ceph-mon[30629]: from='osd.10 [v2:172.21.15.121:6824/2897902637,v1:172.21.15.121:6825/2897902637]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T22:42:52.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:52 smithi111 ceph-mon[30629]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T22:42:53.451 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T22:42:53.481 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi192:vg_nvme/lv_1 2024-09-15T22:42:53.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:53 smithi121 ceph-mon[34815]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:53.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:53 smithi121 ceph-mon[34815]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-15T22:42:53.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:53 smithi121 ceph-mon[34815]: from='osd.10 [v2:172.21.15.121:6824/2897902637,v1:172.21.15.121:6825/2897902637]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-15T22:42:53.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:53 smithi121 ceph-mon[34815]: osdmap e68: 11 total, 10 up, 11 in 2024-09-15T22:42:53.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:53 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:42:53.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:53 smithi121 ceph-mon[34815]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-15T22:42:53.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:53 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:53.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:53 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:53.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:53 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:42:53.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:53 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:42:53.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:53 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:42:53.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:53 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:42:53.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:53 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:53.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:53 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:53.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:53 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:42:53.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:53 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:53.741 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:42:53.767 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:53 smithi192 ceph-mon[35427]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:53.767 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:53 smithi192 ceph-mon[35427]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-15T22:42:53.767 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:53 smithi192 ceph-mon[35427]: from='osd.10 [v2:172.21.15.121:6824/2897902637,v1:172.21.15.121:6825/2897902637]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-15T22:42:53.767 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:53 smithi192 ceph-mon[35427]: osdmap e68: 11 total, 10 up, 11 in 2024-09-15T22:42:53.767 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:53 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:42:53.767 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:53 smithi192 ceph-mon[35427]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-15T22:42:53.767 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:53 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:53.768 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:53 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:53.768 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:53 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:42:53.768 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:53 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:42:53.768 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:53 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:42:53.768 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:53 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:42:53.768 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:53 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:53.768 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:53 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:53.768 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:53 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:42:53.768 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:53 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:53.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:53 smithi111 ceph-mon[30629]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:53.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:53 smithi111 ceph-mon[30629]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-15T22:42:53.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:53 smithi111 ceph-mon[30629]: from='osd.10 [v2:172.21.15.121:6824/2897902637,v1:172.21.15.121:6825/2897902637]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-15T22:42:53.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:53 smithi111 ceph-mon[30629]: osdmap e68: 11 total, 10 up, 11 in 2024-09-15T22:42:53.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:53 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:42:53.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:53 smithi111 ceph-mon[30629]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-15T22:42:53.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:53 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:53.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:53 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:53.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:53 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:42:53.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:53 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:42:53.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:53 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:42:53.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:53 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:42:53.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:53 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:53.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:53 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:53.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:53 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:42:53.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:53 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:42:54.276 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.245+0000 7fee7e3a2640 1 -- 172.21.15.192:0/3712527053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee78104f90 msgr2=0x7fee78107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:54.277 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.245+0000 7fee7e3a2640 1 --2- 172.21.15.192:0/3712527053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee78104f90 0x7fee78107380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fee60009900 tx=0x7fee6002a410 comp rx=0 tx=0).stop 2024-09-15T22:42:54.277 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.245+0000 7fee7e3a2640 1 -- 172.21.15.192:0/3712527053 shutdown_connections 2024-09-15T22:42:54.277 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.245+0000 7fee7e3a2640 1 --2- 172.21.15.192:0/3712527053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee781078c0 0x7fee78109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:54.277 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.245+0000 7fee7e3a2640 1 --2- 172.21.15.192:0/3712527053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee78104f90 0x7fee78107380 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:54.277 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.245+0000 7fee7e3a2640 1 --2- 172.21.15.192:0/3712527053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee78102660 0x7fee78104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:54.277 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.245+0000 7fee7e3a2640 1 -- 172.21.15.192:0/3712527053 >> 172.21.15.192:0/3712527053 conn(0x7fee780fc280 msgr2=0x7fee780fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:42:54.277 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.245+0000 7fee7e3a2640 1 -- 172.21.15.192:0/3712527053 shutdown_connections 2024-09-15T22:42:54.277 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.245+0000 7fee7e3a2640 1 -- 172.21.15.192:0/3712527053 wait complete. 2024-09-15T22:42:54.277 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.246+0000 7fee7e3a2640 1 Processor -- start 2024-09-15T22:42:54.277 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.246+0000 7fee7e3a2640 1 -- start start 2024-09-15T22:42:54.277 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.246+0000 7fee7e3a2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee78102660 0x7fee78197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:54.278 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.246+0000 7fee7e3a2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee78104f90 0x7fee781984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:54.278 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.246+0000 7fee7e3a2640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee781078c0 0x7fee7819f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:54.278 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.246+0000 7fee7e3a2640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fee7810d860 con 0x7fee781078c0 2024-09-15T22:42:54.278 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.246+0000 7fee7e3a2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fee7810d6e0 con 0x7fee78104f90 2024-09-15T22:42:54.278 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.246+0000 7fee7e3a2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fee7810d9e0 con 0x7fee78102660 2024-09-15T22:42:54.278 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.246+0000 7fee77fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee78102660 0x7fee78197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:54.278 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.246+0000 7fee77fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee78102660 0x7fee78197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:44000/0 (socket says 172.21.15.192:44000) 2024-09-15T22:42:54.278 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.246+0000 7fee77fff640 1 -- 172.21.15.192:0/2136745425 learned_addr learned my addr 172.21.15.192:0/2136745425 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-09-15T22:42:54.278 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.246+0000 7fee777fe640 1 --2- 172.21.15.192:0/2136745425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee78104f90 0x7fee781984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:54.278 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.246+0000 7fee77fff640 1 -- 172.21.15.192:0/2136745425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee78104f90 msgr2=0x7fee781984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:54.278 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.246+0000 7fee77fff640 1 --2- 172.21.15.192:0/2136745425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee78104f90 0x7fee781984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:54.278 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.247+0000 7fee77fff640 1 -- 172.21.15.192:0/2136745425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee781078c0 msgr2=0x7fee7819f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:42:54.278 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.247+0000 7fee77fff640 1 --2- 172.21.15.192:0/2136745425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee781078c0 0x7fee7819f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:42:54.279 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.247+0000 7fee77fff640 1 -- 172.21.15.192:0/2136745425 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee7819fba0 con 0x7fee78102660 2024-09-15T22:42:54.279 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.247+0000 7fee77fff640 1 --2- 172.21.15.192:0/2136745425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee78102660 0x7fee78197f70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fee6800e9f0 tx=0x7fee6800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:54.279 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.247+0000 7fee757fa640 1 -- 172.21.15.192:0/2136745425 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6800cd80 con 0x7fee78102660 2024-09-15T22:42:54.279 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.247+0000 7fee7e3a2640 1 -- 172.21.15.192:0/2136745425 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee7819fe90 con 0x7fee78102660 2024-09-15T22:42:54.279 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.247+0000 7fee757fa640 1 -- 172.21.15.192:0/2136745425 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee68005500 con 0x7fee78102660 2024-09-15T22:42:54.279 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.247+0000 7fee757fa640 1 -- 172.21.15.192:0/2136745425 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee68010690 con 0x7fee78102660 2024-09-15T22:42:54.279 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.247+0000 7fee7e3a2640 1 -- 172.21.15.192:0/2136745425 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee781a0290 con 0x7fee78102660 2024-09-15T22:42:54.279 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.248+0000 7fee7e3a2640 1 -- 172.21.15.192:0/2136745425 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee780fdd70 con 0x7fee78102660 2024-09-15T22:42:54.279 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.249+0000 7fee757fa640 1 -- 172.21.15.192:0/2136745425 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fee680046c0 con 0x7fee78102660 2024-09-15T22:42:54.279 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.250+0000 7fee757fa640 1 --2- 172.21.15.192:0/2136745425 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fee640a3270 0x7fee640a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:42:54.279 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.250+0000 7fee757fa640 1 -- 172.21.15.192:0/2136745425 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6447+0+0 (secure 0 0 0) 0x7fee680cf650 con 0x7fee78102660 2024-09-15T22:42:54.279 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.250+0000 7fee777fe640 1 --2- 172.21.15.192:0/2136745425 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fee640a3270 0x7fee640a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:42:54.279 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.251+0000 7fee777fe640 1 --2- 172.21.15.192:0/2136745425 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fee640a3270 0x7fee640a5730 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fee60005ab0 tx=0x7fee60000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:42:54.279 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.254+0000 7fee757fa640 1 -- 172.21.15.192:0/2136745425 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee6809cac0 con 0x7fee78102660 2024-09-15T22:42:54.441 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:42:54.439+0000 7fee7e3a2640 1 -- 172.21.15.192:0/2136745425 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fee78062630 con 0x7fee640a3270 2024-09-15T22:42:54.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 22:42:54 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T22:42:54.313+0000 7f5913179640 -1 osd.10 0 waiting for initial osdmap 2024-09-15T22:42:54.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 22:42:54 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T22:42:54.328+0000 7f590b7ab640 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:42:54.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:54 smithi121 ceph-mon[34815]: Detected new or changed devices on smithi121 2024-09-15T22:42:54.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:54 smithi121 ceph-mon[34815]: Adjusting osd_memory_target on smithi121 to 4271M 2024-09-15T22:42:54.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:54 smithi121 ceph-mon[34815]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-15T22:42:54.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:54 smithi121 ceph-mon[34815]: osdmap e69: 11 total, 10 up, 11 in 2024-09-15T22:42:54.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:54 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:42:54.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:54 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:42:54.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:54 smithi121 ceph-mon[34815]: osdmap e70: 11 total, 10 up, 11 in 2024-09-15T22:42:54.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:54 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:42:54.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:54 smithi121 ceph-mon[34815]: from='osd.10 ' entity='osd.10' 2024-09-15T22:42:54.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:54 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:42:54.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:54 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:42:54.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:54 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:54.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:54 smithi192 ceph-mon[35427]: Detected new or changed devices on smithi121 2024-09-15T22:42:54.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:54 smithi192 ceph-mon[35427]: Adjusting osd_memory_target on smithi121 to 4271M 2024-09-15T22:42:54.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:54 smithi192 ceph-mon[35427]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-15T22:42:54.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:54 smithi192 ceph-mon[35427]: osdmap e69: 11 total, 10 up, 11 in 2024-09-15T22:42:54.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:54 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:42:54.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:54 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:42:54.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:54 smithi192 ceph-mon[35427]: osdmap e70: 11 total, 10 up, 11 in 2024-09-15T22:42:54.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:54 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:42:54.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:54 smithi192 ceph-mon[35427]: from='osd.10 ' entity='osd.10' 2024-09-15T22:42:54.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:54 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:42:54.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:54 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:42:54.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:54 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:54.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:54 smithi111 ceph-mon[30629]: Detected new or changed devices on smithi121 2024-09-15T22:42:54.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:54 smithi111 ceph-mon[30629]: Adjusting osd_memory_target on smithi121 to 4271M 2024-09-15T22:42:54.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:54 smithi111 ceph-mon[30629]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-15T22:42:54.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:54 smithi111 ceph-mon[30629]: osdmap e69: 11 total, 10 up, 11 in 2024-09-15T22:42:54.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:54 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:42:54.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:54 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:42:54.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:54 smithi111 ceph-mon[30629]: osdmap e70: 11 total, 10 up, 11 in 2024-09-15T22:42:54.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:54 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:42:54.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:54 smithi111 ceph-mon[30629]: from='osd.10 ' entity='osd.10' 2024-09-15T22:42:54.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:54 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:42:54.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:54 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:42:54.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:54 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:42:55.801 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:55 smithi111 ceph-mon[30629]: purged_snaps scrub starts 2024-09-15T22:42:55.801 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:55 smithi111 ceph-mon[30629]: purged_snaps scrub ok 2024-09-15T22:42:55.801 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:55 smithi111 ceph-mon[30629]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:55.801 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:55 smithi111 ceph-mon[30629]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:42:55.801 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:55 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:42:55.802 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:55 smithi111 ceph-mon[30629]: osd.10 [v2:172.21.15.121:6824/2897902637,v1:172.21.15.121:6825/2897902637] boot 2024-09-15T22:42:55.802 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:55 smithi111 ceph-mon[30629]: osdmap e71: 11 total, 11 up, 11 in 2024-09-15T22:42:55.802 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:55 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:42:55.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:55 smithi192 ceph-mon[35427]: purged_snaps scrub starts 2024-09-15T22:42:55.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:55 smithi192 ceph-mon[35427]: purged_snaps scrub ok 2024-09-15T22:42:55.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:55 smithi192 ceph-mon[35427]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:55.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:55 smithi192 ceph-mon[35427]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:42:55.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:55 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:42:55.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:55 smithi192 ceph-mon[35427]: osd.10 [v2:172.21.15.121:6824/2897902637,v1:172.21.15.121:6825/2897902637] boot 2024-09-15T22:42:55.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:55 smithi192 ceph-mon[35427]: osdmap e71: 11 total, 11 up, 11 in 2024-09-15T22:42:55.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:55 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:42:55.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:55 smithi121 ceph-mon[34815]: purged_snaps scrub starts 2024-09-15T22:42:55.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:55 smithi121 ceph-mon[34815]: purged_snaps scrub ok 2024-09-15T22:42:55.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:55 smithi121 ceph-mon[34815]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:42:55.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:55 smithi121 ceph-mon[34815]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi192:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:42:55.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:55 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:42:55.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:55 smithi121 ceph-mon[34815]: osd.10 [v2:172.21.15.121:6824/2897902637,v1:172.21.15.121:6825/2897902637] boot 2024-09-15T22:42:55.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:55 smithi121 ceph-mon[34815]: osdmap e71: 11 total, 11 up, 11 in 2024-09-15T22:42:55.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:55 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:42:57.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:57 smithi192 ceph-mon[35427]: osdmap e72: 11 total, 11 up, 11 in 2024-09-15T22:42:57.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:57 smithi192 ceph-mon[35427]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:42:57.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:57 smithi192 ceph-mon[35427]: from='client.? 172.21.15.192:0/3275009173' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "411a7921-6717-4cef-b8df-d9e252edcc84"}]: dispatch 2024-09-15T22:42:57.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:57 smithi192 ceph-mon[35427]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "411a7921-6717-4cef-b8df-d9e252edcc84"}]: dispatch 2024-09-15T22:42:57.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:57 smithi192 ceph-mon[35427]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "411a7921-6717-4cef-b8df-d9e252edcc84"}]': finished 2024-09-15T22:42:57.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:57 smithi192 ceph-mon[35427]: osdmap e73: 12 total, 11 up, 12 in 2024-09-15T22:42:57.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:42:57.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:57 smithi121 ceph-mon[34815]: osdmap e72: 11 total, 11 up, 11 in 2024-09-15T22:42:57.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:57 smithi121 ceph-mon[34815]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:42:57.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:57 smithi121 ceph-mon[34815]: from='client.? 172.21.15.192:0/3275009173' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "411a7921-6717-4cef-b8df-d9e252edcc84"}]: dispatch 2024-09-15T22:42:57.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:57 smithi121 ceph-mon[34815]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "411a7921-6717-4cef-b8df-d9e252edcc84"}]: dispatch 2024-09-15T22:42:57.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:57 smithi121 ceph-mon[34815]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "411a7921-6717-4cef-b8df-d9e252edcc84"}]': finished 2024-09-15T22:42:57.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:57 smithi121 ceph-mon[34815]: osdmap e73: 12 total, 11 up, 12 in 2024-09-15T22:42:57.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:42:57.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:57 smithi111 ceph-mon[30629]: osdmap e72: 11 total, 11 up, 11 in 2024-09-15T22:42:57.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:57 smithi111 ceph-mon[30629]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:42:57.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:57 smithi111 ceph-mon[30629]: from='client.? 172.21.15.192:0/3275009173' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "411a7921-6717-4cef-b8df-d9e252edcc84"}]: dispatch 2024-09-15T22:42:57.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:57 smithi111 ceph-mon[30629]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "411a7921-6717-4cef-b8df-d9e252edcc84"}]: dispatch 2024-09-15T22:42:57.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:57 smithi111 ceph-mon[30629]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "411a7921-6717-4cef-b8df-d9e252edcc84"}]': finished 2024-09-15T22:42:57.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:57 smithi111 ceph-mon[30629]: osdmap e73: 12 total, 11 up, 12 in 2024-09-15T22:42:57.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:42:58.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:58 smithi192 ceph-mon[35427]: from='client.? 172.21.15.192:0/480424628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:42:58.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:58 smithi121 ceph-mon[34815]: from='client.? 172.21.15.192:0/480424628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:42:58.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:58 smithi111 ceph-mon[30629]: from='client.? 172.21.15.192:0/480424628' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:42:59.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:59 smithi121 ceph-mon[34815]: osdmap e74: 12 total, 11 up, 12 in 2024-09-15T22:42:59.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:59 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:42:59.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:42:59 smithi121 ceph-mon[34815]: pgmap v248: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:42:59.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:59 smithi111 ceph-mon[30629]: osdmap e74: 12 total, 11 up, 12 in 2024-09-15T22:42:59.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:59 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:42:59.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:42:59 smithi111 ceph-mon[30629]: pgmap v248: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:42:59.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:59 smithi192 ceph-mon[35427]: osdmap e74: 12 total, 11 up, 12 in 2024-09-15T22:42:59.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:59 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:42:59.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:42:59 smithi192 ceph-mon[35427]: pgmap v248: 1 pgs: 1 remapped+peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:43:00.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:00 smithi121 ceph-mon[34815]: osdmap e75: 12 total, 11 up, 12 in 2024-09-15T22:43:00.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:00 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:43:00.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:00 smithi111 ceph-mon[30629]: osdmap e75: 12 total, 11 up, 12 in 2024-09-15T22:43:00.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:00 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:43:00.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:00 smithi192 ceph-mon[35427]: osdmap e75: 12 total, 11 up, 12 in 2024-09-15T22:43:00.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:00 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:43:01.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:01 smithi121 ceph-mon[34815]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-09-15T22:43:01.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:01 smithi111 ceph-mon[30629]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-09-15T22:43:01.465 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:01 smithi192 ceph-mon[35427]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-09-15T22:43:03.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:03 smithi192 ceph-mon[35427]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:43:03.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:03 smithi121 ceph-mon[34815]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:43:03.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:03 smithi111 ceph-mon[30629]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:43:05.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:05 smithi192 ceph-mon[35427]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-15T22:43:05.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:05 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-15T22:43:05.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:05 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:05.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:05 smithi121 ceph-mon[34815]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-15T22:43:05.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:05 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-15T22:43:05.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:05 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:05.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:05 smithi111 ceph-mon[30629]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-15T22:43:05.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:05 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-15T22:43:05.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:05 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:06.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:06 smithi192 ceph-mon[35427]: Deploying daemon osd.11 on smithi192 2024-09-15T22:43:06.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:06 smithi121 ceph-mon[34815]: Deploying daemon osd.11 on smithi192 2024-09-15T22:43:06.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:06 smithi111 ceph-mon[30629]: Deploying daemon osd.11 on smithi192 2024-09-15T22:43:07.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:07 smithi192 ceph-mon[35427]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-15T22:43:07.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:07 smithi121 ceph-mon[34815]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-15T22:43:07.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:07 smithi111 ceph-mon[30629]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-15T22:43:09.713 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:09 smithi192 ceph-mon[35427]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T22:43:09.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:09 smithi121 ceph-mon[34815]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T22:43:09.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:09 smithi111 ceph-mon[30629]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T22:43:11.644 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:11 smithi192 ceph-mon[35427]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-15T22:43:11.644 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:11 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:43:11.644 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:11 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:11.644 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:11 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:11.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:11 smithi121 ceph-mon[34815]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-15T22:43:11.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:11 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:43:11.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:11 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:11.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:11 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:11.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:11 smithi111 ceph-mon[30629]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-15T22:43:11.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:11 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:43:11.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:11 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:11.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:11 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:13.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:13 smithi192 ceph-mon[35427]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:43:13.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:13 smithi121 ceph-mon[34815]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:43:13.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:13 smithi111 ceph-mon[30629]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:43:14.146 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:14.144+0000 7fee757fa640 1 -- 172.21.15.192:0/2136745425 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fee78062630 con 0x7fee640a3270 2024-09-15T22:43:14.147 INFO:teuthology.orchestra.run.smithi192.stdout:Created osd(s) 11 on host 'smithi192' 2024-09-15T22:43:14.149 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:14.147+0000 7fee7e3a2640 1 -- 172.21.15.192:0/2136745425 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fee640a3270 msgr2=0x7fee640a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:14.149 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:14.147+0000 7fee7e3a2640 1 --2- 172.21.15.192:0/2136745425 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fee640a3270 0x7fee640a5730 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fee60005ab0 tx=0x7fee60000bc0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.149 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:14.147+0000 7fee7e3a2640 1 -- 172.21.15.192:0/2136745425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee78102660 msgr2=0x7fee78197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:14.149 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:14.147+0000 7fee7e3a2640 1 --2- 172.21.15.192:0/2136745425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee78102660 0x7fee78197f70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fee6800e9f0 tx=0x7fee6800edf0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.149 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:14.147+0000 7fee7e3a2640 1 -- 172.21.15.192:0/2136745425 shutdown_connections 2024-09-15T22:43:14.149 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:14.147+0000 7fee7e3a2640 1 --2- 172.21.15.192:0/2136745425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee781078c0 0x7fee7819f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.149 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:14.147+0000 7fee7e3a2640 1 --2- 172.21.15.192:0/2136745425 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fee640a3270 0x7fee640a5730 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.149 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:14.147+0000 7fee7e3a2640 1 --2- 172.21.15.192:0/2136745425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee78104f90 0x7fee781984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.149 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:14.147+0000 7fee7e3a2640 1 --2- 172.21.15.192:0/2136745425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee78102660 0x7fee78197f70 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:14.149 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:14.147+0000 7fee7e3a2640 1 -- 172.21.15.192:0/2136745425 >> 172.21.15.192:0/2136745425 conn(0x7fee780fc280 msgr2=0x7fee78103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:14.149 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:14.147+0000 7fee7e3a2640 1 -- 172.21.15.192:0/2136745425 shutdown_connections 2024-09-15T22:43:14.150 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:14.147+0000 7fee7e3a2640 1 -- 172.21.15.192:0/2136745425 wait complete. 2024-09-15T22:43:14.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:14 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:14.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:14 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:14.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:14 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:14.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:14 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:43:14.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:14 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:14.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:14 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:43:14.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:14 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:14.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:14 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:14.795 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:14 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:14.795 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:14 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:14.795 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:14 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:14.795 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:14 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:43:14.795 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:14 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:14.795 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:14 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:43:14.795 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:14 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:14.796 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:14 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:14.822 DEBUG:teuthology.orchestra.run.smithi192:osd.11> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.11.service 2024-09-15T22:43:14.824 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-15T22:43:14.825 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-15T22:43:14.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:14 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:14.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:14 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:14.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:14 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:14.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:14 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:43:14.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:14 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:14.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:14 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:43:14.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:14 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:14.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:14 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:15.200 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:15.676 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:15 smithi192 ceph-mon[35427]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:43:15.828 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:15 smithi111 ceph-mon[30629]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:43:15.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.826+0000 7fa0c3200640 1 -- 172.21.15.111:0/3495035525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0c4074bf0 msgr2=0x7fa0c4075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:15.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.826+0000 7fa0c3200640 1 --2- 172.21.15.111:0/3495035525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0c4074bf0 0x7fa0c4075050 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0009900 tx=0x7fa0b002a480 comp rx=0 tx=0).stop 2024-09-15T22:43:15.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.826+0000 7fa0c3200640 1 -- 172.21.15.111:0/3495035525 shutdown_connections 2024-09-15T22:43:15.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.826+0000 7fa0c3200640 1 --2- 172.21.15.111:0/3495035525 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c41098e0 0x7fa0c410bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:15.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.826+0000 7fa0c3200640 1 --2- 172.21.15.111:0/3495035525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0c4074bf0 0x7fa0c4075050 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:15.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.826+0000 7fa0c3200640 1 --2- 172.21.15.111:0/3495035525 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0c40761c0 0x7fa0c4074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:15.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.826+0000 7fa0c3200640 1 -- 172.21.15.111:0/3495035525 >> 172.21.15.111:0/3495035525 conn(0x7fa0c40fd1f0 msgr2=0x7fa0c40ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:15.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.826+0000 7fa0c3200640 1 -- 172.21.15.111:0/3495035525 shutdown_connections 2024-09-15T22:43:15.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.826+0000 7fa0c3200640 1 -- 172.21.15.111:0/3495035525 wait complete. 2024-09-15T22:43:15.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.827+0000 7fa0c3200640 1 Processor -- start 2024-09-15T22:43:15.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.827+0000 7fa0c3200640 1 -- start start 2024-09-15T22:43:15.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.828+0000 7fa0c3200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0c4074bf0 0x7fa0c4199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:15.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.828+0000 7fa0c3200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c40761c0 0x7fa0c419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:15.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.828+0000 7fa0c3200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0c41098e0 0x7fa0c41a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:15.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.828+0000 7fa0c3200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c410f8f0 con 0x7fa0c41098e0 2024-09-15T22:43:15.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.828+0000 7fa0c3200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c410f770 con 0x7fa0c4074bf0 2024-09-15T22:43:15.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.828+0000 7fa0c3200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c410fa70 con 0x7fa0c40761c0 2024-09-15T22:43:15.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.828+0000 7fa0c2800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0c41098e0 0x7fa0c41a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:15.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.828+0000 7fa0c1400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c40761c0 0x7fa0c419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:15.859 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.828+0000 7fa0c1e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0c4074bf0 0x7fa0c4199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:15.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.828+0000 7fa0c2800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0c41098e0 0x7fa0c41a14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58798/0 (socket says 172.21.15.111:58798) 2024-09-15T22:43:15.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.828+0000 7fa0c1400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c40761c0 0x7fa0c419a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:41034/0 (socket says 172.21.15.111:41034) 2024-09-15T22:43:15.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.828+0000 7fa0c2800640 1 -- 172.21.15.111:0/3471691380 learned_addr learned my addr 172.21.15.111:0/3471691380 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:43:15.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.828+0000 7fa0c2800640 1 -- 172.21.15.111:0/3471691380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c40761c0 msgr2=0x7fa0c419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:15.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.828+0000 7fa0c2800640 1 --2- 172.21.15.111:0/3471691380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c40761c0 0x7fa0c419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:15.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.828+0000 7fa0c2800640 1 -- 172.21.15.111:0/3471691380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0c4074bf0 msgr2=0x7fa0c4199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:15.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.828+0000 7fa0c2800640 1 --2- 172.21.15.111:0/3471691380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0c4074bf0 0x7fa0c4199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:15.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.828+0000 7fa0c2800640 1 -- 172.21.15.111:0/3471691380 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c41a1b60 con 0x7fa0c41098e0 2024-09-15T22:43:15.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.829+0000 7fa0c2800640 1 --2- 172.21.15.111:0/3471691380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0c41098e0 0x7fa0c41a14f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00eaa0 tx=0x7fa0bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:15.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.829+0000 7fa0baa00640 1 -- 172.21.15.111:0/3471691380 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc00cdb0 con 0x7fa0c41098e0 2024-09-15T22:43:15.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.829+0000 7fa0baa00640 1 -- 172.21.15.111:0/3471691380 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0bc005020 con 0x7fa0c41098e0 2024-09-15T22:43:15.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.829+0000 7fa0c3200640 1 -- 172.21.15.111:0/3471691380 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c41a1e50 con 0x7fa0c41098e0 2024-09-15T22:43:15.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.829+0000 7fa0baa00640 1 -- 172.21.15.111:0/3471691380 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc0116b0 con 0x7fa0c41098e0 2024-09-15T22:43:15.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.829+0000 7fa0c3200640 1 -- 172.21.15.111:0/3471691380 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c41a2300 con 0x7fa0c41098e0 2024-09-15T22:43:15.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.830+0000 7fa0c3200640 1 -- 172.21.15.111:0/3471691380 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0c41086a0 con 0x7fa0c41098e0 2024-09-15T22:43:15.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.832+0000 7fa0baa00640 1 -- 172.21.15.111:0/3471691380 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fa0bc005560 con 0x7fa0c41098e0 2024-09-15T22:43:15.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.833+0000 7fa0baa00640 1 --2- 172.21.15.111:0/3471691380 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa0a80a32f0 0x7fa0a80a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:15.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.833+0000 7fa0baa00640 1 -- 172.21.15.111:0/3471691380 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6809+0+0 (secure 0 0 0) 0x7fa0bc0cf4d0 con 0x7fa0c41098e0 2024-09-15T22:43:15.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.834+0000 7fa0c1e00640 1 --2- 172.21.15.111:0/3471691380 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa0a80a32f0 0x7fa0a80a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:15.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.834+0000 7fa0c1e00640 1 --2- 172.21.15.111:0/3471691380 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa0a80a32f0 0x7fa0a80a57b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400bbe0 tx=0x7fa0b400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:15.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:15.838+0000 7fa0baa00640 1 -- 172.21.15.111:0/3471691380 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa0bc016030 con 0x7fa0c41098e0 2024-09-15T22:43:15.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:15 smithi121 ceph-mon[34815]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:43:16.014 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:16.013+0000 7fa0c3200640 1 -- 172.21.15.111:0/3471691380 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fa0c4062e90 con 0x7fa0c41098e0 2024-09-15T22:43:16.015 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:16.013+0000 7fa0baa00640 1 -- 172.21.15.111:0/3471691380 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7fa0bc09c0e0 con 0x7fa0c41098e0 2024-09-15T22:43:16.015 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:43:16.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:16.015+0000 7fa0c3200640 1 -- 172.21.15.111:0/3471691380 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa0a80a32f0 msgr2=0x7fa0a80a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:16.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:16.015+0000 7fa0c3200640 1 --2- 172.21.15.111:0/3471691380 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa0a80a32f0 0x7fa0a80a57b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400bbe0 tx=0x7fa0b400b390 comp rx=0 tx=0).stop 2024-09-15T22:43:16.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:16.015+0000 7fa0c3200640 1 -- 172.21.15.111:0/3471691380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0c41098e0 msgr2=0x7fa0c41a14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:16.017 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:16.015+0000 7fa0c3200640 1 --2- 172.21.15.111:0/3471691380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0c41098e0 0x7fa0c41a14f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc00eaa0 tx=0x7fa0bc00ef60 comp rx=0 tx=0).stop 2024-09-15T22:43:16.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:16.016+0000 7fa0c3200640 1 -- 172.21.15.111:0/3471691380 shutdown_connections 2024-09-15T22:43:16.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:16.016+0000 7fa0c3200640 1 --2- 172.21.15.111:0/3471691380 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0c41098e0 0x7fa0c41a14f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:16.016+0000 7fa0c3200640 1 --2- 172.21.15.111:0/3471691380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c40761c0 0x7fa0c419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:16.016+0000 7fa0c3200640 1 --2- 172.21.15.111:0/3471691380 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa0a80a32f0 0x7fa0a80a57b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:16.016+0000 7fa0c3200640 1 --2- 172.21.15.111:0/3471691380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0c4074bf0 0x7fa0c4199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:16.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:16.016+0000 7fa0c3200640 1 -- 172.21.15.111:0/3471691380 >> 172.21.15.111:0/3471691380 conn(0x7fa0c40fd1f0 msgr2=0x7fa0c4105870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:16.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:16.016+0000 7fa0c3200640 1 -- 172.21.15.111:0/3471691380 shutdown_connections 2024-09-15T22:43:16.018 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:16.016+0000 7fa0c3200640 1 -- 172.21.15.111:0/3471691380 wait complete. 2024-09-15T22:43:16.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:16 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/3471691380' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:43:16.881 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:16 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/3471691380' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:43:16.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:16 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/3471691380' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:43:16.920 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":75,"num_osds":12,"num_up_osds":11,"osd_up_since":1726440175,"num_in_osds":12,"osd_in_since":1726440176,"num_remapped_pgs":0} 2024-09-15T22:43:17.492 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 22:43:17 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T22:43:17.224+0000 7fe51aea7640 -1 osd.11 0 log_to_monitors true 2024-09-15T22:43:17.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:17 smithi192 ceph-mon[35427]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:43:17.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:17 smithi192 ceph-mon[35427]: from='osd.11 [v2:172.21.15.192:6824/2431464812,v1:172.21.15.192:6825/2431464812]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T22:43:17.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:17 smithi192 ceph-mon[35427]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T22:43:17.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:17 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:17.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:17 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:17.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:17 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:43:17.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:17 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:43:17.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:17 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:43:17.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:17 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:43:17.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:17 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:17.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:17 smithi121 ceph-mon[34815]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:43:17.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:17 smithi121 ceph-mon[34815]: from='osd.11 [v2:172.21.15.192:6824/2431464812,v1:172.21.15.192:6825/2431464812]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T22:43:17.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:17 smithi121 ceph-mon[34815]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T22:43:17.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:17 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:17.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:17 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:17.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:17 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:43:17.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:17 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:43:17.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:17 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:43:17.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:17 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:43:17.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:17 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:17.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:17 smithi111 ceph-mon[30629]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:43:17.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:17 smithi111 ceph-mon[30629]: from='osd.11 [v2:172.21.15.192:6824/2431464812,v1:172.21.15.192:6825/2431464812]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T22:43:17.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:17 smithi111 ceph-mon[30629]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T22:43:17.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:17 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:17.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:17 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:17.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:17 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:43:17.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:17 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:43:17.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:17 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:43:17.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:17 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:43:17.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:17 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:17.921 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-15T22:43:18.283 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:18.698 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:18 smithi111 ceph-mon[30629]: Detected new or changed devices on smithi192 2024-09-15T22:43:18.699 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:18 smithi111 ceph-mon[30629]: Adjusting osd_memory_target on smithi192 to 4272M 2024-09-15T22:43:18.699 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:18 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:18.699 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:18 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:43:18.699 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:18 smithi111 ceph-mon[30629]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-15T22:43:18.699 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:18 smithi111 ceph-mon[30629]: osdmap e76: 12 total, 11 up, 12 in 2024-09-15T22:43:18.699 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:18 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:43:18.699 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:18 smithi111 ceph-mon[30629]: from='osd.11 [v2:172.21.15.192:6824/2431464812,v1:172.21.15.192:6825/2431464812]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-09-15T22:43:18.699 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:18 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:18.699 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:18 smithi111 ceph-mon[30629]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-09-15T22:43:18.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.806+0000 7f965ee00640 1 -- 172.21.15.111:0/3345150768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9658103e00 msgr2=0x7f9658104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:18.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.806+0000 7f965ee00640 1 --2- 172.21.15.111:0/3345150768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9658103e00 0x7f9658104200 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9648009ca0 tx=0x7f964802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:43:18.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.807+0000 7f965ee00640 1 -- 172.21.15.111:0/3345150768 shutdown_connections 2024-09-15T22:43:18.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.807+0000 7f965ee00640 1 --2- 172.21.15.111:0/3345150768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9658105920 0x7f965810c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:18.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.807+0000 7f965ee00640 1 --2- 172.21.15.111:0/3345150768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9658104f60 0x7f96581053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:18.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.807+0000 7f965ee00640 1 --2- 172.21.15.111:0/3345150768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9658103e00 0x7f9658104200 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:18.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.807+0000 7f965ee00640 1 -- 172.21.15.111:0/3345150768 >> 172.21.15.111:0/3345150768 conn(0x7f96580ff5b0 msgr2=0x7f96581019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:18.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.807+0000 7f965ee00640 1 -- 172.21.15.111:0/3345150768 shutdown_connections 2024-09-15T22:43:18.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.807+0000 7f965ee00640 1 -- 172.21.15.111:0/3345150768 wait complete. 2024-09-15T22:43:18.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.808+0000 7f965ee00640 1 Processor -- start 2024-09-15T22:43:18.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.808+0000 7f965ee00640 1 -- start start 2024-09-15T22:43:18.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.808+0000 7f965ee00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9658103e00 0x7f965819a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:18.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.808+0000 7f965ee00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9658104f60 0x7f965819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:18.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.808+0000 7f965ee00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9658105920 0x7f96581a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:18.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.808+0000 7f965ee00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f965810fa90 con 0x7f9658105920 2024-09-15T22:43:18.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.808+0000 7f965ee00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f965810f910 con 0x7f9658103e00 2024-09-15T22:43:18.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.808+0000 7f965ee00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f965810fc10 con 0x7f9658104f60 2024-09-15T22:43:18.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.808+0000 7f965ce00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9658105920 0x7f96581a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:18.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.808+0000 7f965ce00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9658105920 0x7f96581a16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58810/0 (socket says 172.21.15.111:58810) 2024-09-15T22:43:18.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.808+0000 7f965ce00640 1 -- 172.21.15.111:0/2817436095 learned_addr learned my addr 172.21.15.111:0/2817436095 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:43:18.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.808+0000 7f964f400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9658104f60 0x7f965819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:18.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.808+0000 7f9657e00640 1 --2- 172.21.15.111:0/2817436095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9658103e00 0x7f965819a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:18.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.809+0000 7f965ce00640 1 -- 172.21.15.111:0/2817436095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9658104f60 msgr2=0x7f965819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:18.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.809+0000 7f965ce00640 1 --2- 172.21.15.111:0/2817436095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9658104f60 0x7f965819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:18.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.809+0000 7f965ce00640 1 -- 172.21.15.111:0/2817436095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9658103e00 msgr2=0x7f965819a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:18.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.809+0000 7f965ce00640 1 --2- 172.21.15.111:0/2817436095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9658103e00 0x7f965819a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:18.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.809+0000 7f965ce00640 1 -- 172.21.15.111:0/2817436095 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96581a1d10 con 0x7f9658105920 2024-09-15T22:43:18.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.809+0000 7f964f400640 1 --2- 172.21.15.111:0/2817436095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9658104f60 0x7f965819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:43:18.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.809+0000 7f965ce00640 1 --2- 172.21.15.111:0/2817436095 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9658105920 0x7f96581a16a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f965000d950 tx=0x7f965000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:18.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.809+0000 7f9655600640 1 -- 172.21.15.111:0/2817436095 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9650017030 con 0x7f9658105920 2024-09-15T22:43:18.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.809+0000 7f9655600640 1 -- 172.21.15.111:0/2817436095 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9650004020 con 0x7f9658105920 2024-09-15T22:43:18.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.809+0000 7f965ee00640 1 -- 172.21.15.111:0/2817436095 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96581a1fa0 con 0x7f9658105920 2024-09-15T22:43:18.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.809+0000 7f9655600640 1 -- 172.21.15.111:0/2817436095 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9650003c00 con 0x7f9658105920 2024-09-15T22:43:18.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.809+0000 7f965ee00640 1 -- 172.21.15.111:0/2817436095 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96581a2460 con 0x7f9658105920 2024-09-15T22:43:18.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.810+0000 7f965ee00640 1 -- 172.21.15.111:0/2817436095 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f965810b190 con 0x7f9658105920 2024-09-15T22:43:18.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.812+0000 7f9655600640 1 -- 172.21.15.111:0/2817436095 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f965001a070 con 0x7f9658105920 2024-09-15T22:43:18.814 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 22:43:18 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T22:43:18.571+0000 7fe516607640 -1 osd.11 0 waiting for initial osdmap 2024-09-15T22:43:18.814 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 22:43:18 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T22:43:18.588+0000 7fe512c41640 -1 osd.11 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:43:18.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:18 smithi192 ceph-mon[35427]: Detected new or changed devices on smithi192 2024-09-15T22:43:18.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:18 smithi192 ceph-mon[35427]: Adjusting osd_memory_target on smithi192 to 4272M 2024-09-15T22:43:18.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:18 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:18.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:18 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:43:18.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:18 smithi192 ceph-mon[35427]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-15T22:43:18.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:18 smithi192 ceph-mon[35427]: osdmap e76: 12 total, 11 up, 12 in 2024-09-15T22:43:18.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:18 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:43:18.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:18 smithi192 ceph-mon[35427]: from='osd.11 [v2:172.21.15.192:6824/2431464812,v1:172.21.15.192:6825/2431464812]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-09-15T22:43:18.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:18 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:18.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:18 smithi192 ceph-mon[35427]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-09-15T22:43:18.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.813+0000 7f9655600640 1 --2- 172.21.15.111:0/2817436095 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f963c0a3270 0x7f963c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:18.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.814+0000 7f9655600640 1 -- 172.21.15.111:0/2817436095 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6847+0+0 (secure 0 0 0) 0x7f96500cebd0 con 0x7f9658105920 2024-09-15T22:43:18.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.814+0000 7f9657e00640 1 --2- 172.21.15.111:0/2817436095 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f963c0a3270 0x7f963c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:18.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.815+0000 7f9657e00640 1 --2- 172.21.15.111:0/2817436095 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f963c0a3270 0x7f963c0a5730 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9648006010 tx=0x7f9648000f40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:18.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.818+0000 7f9655600640 1 -- 172.21.15.111:0/2817436095 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9650021070 con 0x7f9658105920 2024-09-15T22:43:18.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:18 smithi121 ceph-mon[34815]: Detected new or changed devices on smithi192 2024-09-15T22:43:18.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:18 smithi121 ceph-mon[34815]: Adjusting osd_memory_target on smithi192 to 4272M 2024-09-15T22:43:18.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:18 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:18.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:18 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:43:18.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:18 smithi121 ceph-mon[34815]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-15T22:43:18.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:18 smithi121 ceph-mon[34815]: osdmap e76: 12 total, 11 up, 12 in 2024-09-15T22:43:18.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:18 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:43:18.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:18 smithi121 ceph-mon[34815]: from='osd.11 [v2:172.21.15.192:6824/2431464812,v1:172.21.15.192:6825/2431464812]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-09-15T22:43:18.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:18 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:18.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:18 smithi121 ceph-mon[34815]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]: dispatch 2024-09-15T22:43:18.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.993+0000 7f965ee00640 1 -- 172.21.15.111:0/2817436095 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f9658062df0 con 0x7f9658105920 2024-09-15T22:43:18.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.994+0000 7f9655600640 1 -- 172.21.15.111:0/2817436095 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v77) ==== 74+0+133 (secure 0 0 0) 0x7f965009be10 con 0x7f9658105920 2024-09-15T22:43:18.996 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:43:18.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.997+0000 7f965ee00640 1 -- 172.21.15.111:0/2817436095 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f963c0a3270 msgr2=0x7f963c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:18.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.997+0000 7f965ee00640 1 --2- 172.21.15.111:0/2817436095 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f963c0a3270 0x7f963c0a5730 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9648006010 tx=0x7f9648000f40 comp rx=0 tx=0).stop 2024-09-15T22:43:18.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.997+0000 7f965ee00640 1 -- 172.21.15.111:0/2817436095 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9658105920 msgr2=0x7f96581a16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:18.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.997+0000 7f965ee00640 1 --2- 172.21.15.111:0/2817436095 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9658105920 0x7f96581a16a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f965000d950 tx=0x7f965000de10 comp rx=0 tx=0).stop 2024-09-15T22:43:18.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.997+0000 7f965ee00640 1 -- 172.21.15.111:0/2817436095 shutdown_connections 2024-09-15T22:43:18.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.997+0000 7f965ee00640 1 --2- 172.21.15.111:0/2817436095 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9658105920 0x7f96581a16a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:18.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.997+0000 7f965ee00640 1 --2- 172.21.15.111:0/2817436095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9658104f60 0x7f965819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:18.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.997+0000 7f965ee00640 1 --2- 172.21.15.111:0/2817436095 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f963c0a3270 0x7f963c0a5730 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:18.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.997+0000 7f965ee00640 1 --2- 172.21.15.111:0/2817436095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9658103e00 0x7f965819a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:18.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.997+0000 7f965ee00640 1 -- 172.21.15.111:0/2817436095 >> 172.21.15.111:0/2817436095 conn(0x7f96580ff5b0 msgr2=0x7f9658108360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:18.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.997+0000 7f965ee00640 1 -- 172.21.15.111:0/2817436095 shutdown_connections 2024-09-15T22:43:18.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:18.997+0000 7f965ee00640 1 -- 172.21.15.111:0/2817436095 wait complete. 2024-09-15T22:43:19.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:19 smithi192 ceph-mon[35427]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:43:19.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:19 smithi192 ceph-mon[35427]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-09-15T22:43:19.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:19 smithi192 ceph-mon[35427]: osdmap e77: 12 total, 11 up, 12 in 2024-09-15T22:43:19.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:19 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:43:19.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:19 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:43:19.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:19 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/2817436095' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:43:19.819 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:19 smithi111 ceph-mon[30629]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:43:19.819 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:19 smithi111 ceph-mon[30629]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-09-15T22:43:19.819 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:19 smithi111 ceph-mon[30629]: osdmap e77: 12 total, 11 up, 12 in 2024-09-15T22:43:19.819 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:19 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:43:19.819 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:19 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:43:19.819 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:19 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/2817436095' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:43:19.855 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":77,"num_osds":12,"num_up_osds":11,"osd_up_since":1726440175,"num_in_osds":12,"osd_in_since":1726440176,"num_remapped_pgs":0} 2024-09-15T22:43:19.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:19 smithi121 ceph-mon[34815]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:43:19.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:19 smithi121 ceph-mon[34815]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi192", "root=default"]}]': finished 2024-09-15T22:43:19.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:19 smithi121 ceph-mon[34815]: osdmap e77: 12 total, 11 up, 12 in 2024-09-15T22:43:19.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:19 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:43:19.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:19 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:43:19.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:19 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/2817436095' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:43:20.856 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-15T22:43:20.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:20 smithi121 ceph-mon[34815]: purged_snaps scrub starts 2024-09-15T22:43:20.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:20 smithi121 ceph-mon[34815]: purged_snaps scrub ok 2024-09-15T22:43:20.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:20 smithi121 ceph-mon[34815]: osd.11 [v2:172.21.15.192:6824/2431464812,v1:172.21.15.192:6825/2431464812] boot 2024-09-15T22:43:20.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:20 smithi121 ceph-mon[34815]: osdmap e78: 12 total, 12 up, 12 in 2024-09-15T22:43:20.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:20 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:43:20.910 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:20 smithi111 ceph-mon[30629]: purged_snaps scrub starts 2024-09-15T22:43:20.910 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:20 smithi111 ceph-mon[30629]: purged_snaps scrub ok 2024-09-15T22:43:20.910 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:20 smithi111 ceph-mon[30629]: osd.11 [v2:172.21.15.192:6824/2431464812,v1:172.21.15.192:6825/2431464812] boot 2024-09-15T22:43:20.910 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:20 smithi111 ceph-mon[30629]: osdmap e78: 12 total, 12 up, 12 in 2024-09-15T22:43:20.911 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:20 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:43:21.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:20 smithi192 ceph-mon[35427]: purged_snaps scrub starts 2024-09-15T22:43:21.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:20 smithi192 ceph-mon[35427]: purged_snaps scrub ok 2024-09-15T22:43:21.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:20 smithi192 ceph-mon[35427]: osd.11 [v2:172.21.15.192:6824/2431464812,v1:172.21.15.192:6825/2431464812] boot 2024-09-15T22:43:21.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:20 smithi192 ceph-mon[35427]: osdmap e78: 12 total, 12 up, 12 in 2024-09-15T22:43:21.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:20 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:43:21.231 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:21.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:21 smithi111 ceph-mon[30629]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:43:21.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:21 smithi111 ceph-mon[30629]: osdmap e79: 12 total, 12 up, 12 in 2024-09-15T22:43:21.798 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.797+0000 7f60ad200640 1 -- 172.21.15.111:0/3358832273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60a8074c50 msgr2=0x7f60a80750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:21.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.797+0000 7f60ad200640 1 --2- 172.21.15.111:0/3358832273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60a8074c50 0x7f60a80750b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6094009900 tx=0x7f609402a3f0 comp rx=0 tx=0).stop 2024-09-15T22:43:21.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.797+0000 7f60ad200640 1 -- 172.21.15.111:0/3358832273 shutdown_connections 2024-09-15T22:43:21.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.797+0000 7f60ad200640 1 --2- 172.21.15.111:0/3358832273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60a810e1e0 0x7f60a81105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:21.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.797+0000 7f60ad200640 1 --2- 172.21.15.111:0/3358832273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60a8074c50 0x7f60a80750b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:21.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.797+0000 7f60ad200640 1 --2- 172.21.15.111:0/3358832273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60a8076220 0x7f60a8074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:21.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.797+0000 7f60ad200640 1 -- 172.21.15.111:0/3358832273 >> 172.21.15.111:0/3358832273 conn(0x7f60a80fd390 msgr2=0x7f60a80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:21.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.797+0000 7f60ad200640 1 -- 172.21.15.111:0/3358832273 shutdown_connections 2024-09-15T22:43:21.799 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.797+0000 7f60ad200640 1 -- 172.21.15.111:0/3358832273 wait complete. 2024-09-15T22:43:21.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.798+0000 7f60ad200640 1 Processor -- start 2024-09-15T22:43:21.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.798+0000 7f60ad200640 1 -- start start 2024-09-15T22:43:21.800 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.798+0000 7f60ad200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60a8074c50 0x7f60a819e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:21.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.798+0000 7f60ad200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60a8076220 0x7f60a819ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:21.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.798+0000 7f60ad200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60a810e1e0 0x7f60a81a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:21.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.798+0000 7f60ad200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f60a8113ef0 con 0x7f60a810e1e0 2024-09-15T22:43:21.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.798+0000 7f60ad200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f60a8113d70 con 0x7f60a8074c50 2024-09-15T22:43:21.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.798+0000 7f60ad200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f60a8114070 con 0x7f60a8076220 2024-09-15T22:43:21.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.798+0000 7f60a5e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60a8074c50 0x7f60a819e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:21.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.799+0000 7f60a5e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60a8074c50 0x7f60a819e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:36994/0 (socket says 172.21.15.111:36994) 2024-09-15T22:43:21.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.799+0000 7f60a5e00640 1 -- 172.21.15.111:0/3376756640 learned_addr learned my addr 172.21.15.111:0/3376756640 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:43:21.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.799+0000 7f60a6800640 1 --2- 172.21.15.111:0/3376756640 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60a810e1e0 0x7f60a81a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:21.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.799+0000 7f60a5400640 1 --2- 172.21.15.111:0/3376756640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60a8076220 0x7f60a819ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:21.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.799+0000 7f60a5e00640 1 -- 172.21.15.111:0/3376756640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60a8076220 msgr2=0x7f60a819ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:21.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.799+0000 7f60a5e00640 1 --2- 172.21.15.111:0/3376756640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60a8076220 0x7f60a819ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:21.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.799+0000 7f60a5e00640 1 -- 172.21.15.111:0/3376756640 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60a810e1e0 msgr2=0x7f60a81a5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:21.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.799+0000 7f60a5e00640 1 --2- 172.21.15.111:0/3376756640 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60a810e1e0 0x7f60a81a5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:21.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.799+0000 7f60a5e00640 1 -- 172.21.15.111:0/3376756640 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a81a6150 con 0x7f60a8074c50 2024-09-15T22:43:21.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.799+0000 7f60a5e00640 1 --2- 172.21.15.111:0/3376756640 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60a8074c50 0x7f60a819e520 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f609800d770 tx=0x7f609800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:21.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.800+0000 7f609ea00640 1 -- 172.21.15.111:0/3376756640 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6098017070 con 0x7f60a8074c50 2024-09-15T22:43:21.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.800+0000 7f60ad200640 1 -- 172.21.15.111:0/3376756640 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a81a6440 con 0x7f60a8074c50 2024-09-15T22:43:21.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.800+0000 7f609ea00640 1 -- 172.21.15.111:0/3376756640 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6098009c30 con 0x7f60a8074c50 2024-09-15T22:43:21.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.800+0000 7f609ea00640 1 -- 172.21.15.111:0/3376756640 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609800f620 con 0x7f60a8074c50 2024-09-15T22:43:21.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.800+0000 7f60ad200640 1 -- 172.21.15.111:0/3376756640 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a81a69a0 con 0x7f60a8074c50 2024-09-15T22:43:21.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.801+0000 7f60ad200640 1 -- 172.21.15.111:0/3376756640 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a81043b0 con 0x7f60a8074c50 2024-09-15T22:43:21.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.803+0000 7f609ea00640 1 -- 172.21.15.111:0/3376756640 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f609801a070 con 0x7f60a8074c50 2024-09-15T22:43:21.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.804+0000 7f609ea00640 1 --2- 172.21.15.111:0/3376756640 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f608c0a32f0 0x7f608c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:21.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.804+0000 7f609ea00640 1 -- 172.21.15.111:0/3376756640 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f60980ceec0 con 0x7f60a8074c50 2024-09-15T22:43:21.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.804+0000 7f60a5400640 1 --2- 172.21.15.111:0/3376756640 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f608c0a32f0 0x7f608c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:21.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.805+0000 7f60a5400640 1 --2- 172.21.15.111:0/3376756640 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f608c0a32f0 0x7f608c0a57b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6094031440 tx=0x7f60940315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:21.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.806+0000 7f609ea00640 1 -- 172.21.15.111:0/3376756640 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f609809ba10 con 0x7f60a8074c50 2024-09-15T22:43:21.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:21 smithi121 ceph-mon[34815]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:43:21.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:21 smithi121 ceph-mon[34815]: osdmap e79: 12 total, 12 up, 12 in 2024-09-15T22:43:21.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.981+0000 7f60ad200640 1 -- 172.21.15.111:0/3376756640 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f60a802d7c0 con 0x7f60a8074c50 2024-09-15T22:43:21.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.982+0000 7f609ea00640 1 -- 172.21.15.111:0/3376756640 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) ==== 74+0+133 (secure 0 0 0) 0x7f609809fd10 con 0x7f60a8074c50 2024-09-15T22:43:21.983 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:43:21.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.984+0000 7f60ad200640 1 -- 172.21.15.111:0/3376756640 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f608c0a32f0 msgr2=0x7f608c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:21.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.984+0000 7f60ad200640 1 --2- 172.21.15.111:0/3376756640 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f608c0a32f0 0x7f608c0a57b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6094031440 tx=0x7f60940315b0 comp rx=0 tx=0).stop 2024-09-15T22:43:21.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.984+0000 7f60ad200640 1 -- 172.21.15.111:0/3376756640 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60a8074c50 msgr2=0x7f60a819e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:21.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.984+0000 7f60ad200640 1 --2- 172.21.15.111:0/3376756640 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60a8074c50 0x7f60a819e520 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f609800d770 tx=0x7f609800dc30 comp rx=0 tx=0).stop 2024-09-15T22:43:21.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.984+0000 7f60ad200640 1 -- 172.21.15.111:0/3376756640 shutdown_connections 2024-09-15T22:43:21.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.984+0000 7f60ad200640 1 --2- 172.21.15.111:0/3376756640 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60a810e1e0 0x7f60a81a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:21.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.984+0000 7f60ad200640 1 --2- 172.21.15.111:0/3376756640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60a8076220 0x7f60a819ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:21.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.984+0000 7f60ad200640 1 --2- 172.21.15.111:0/3376756640 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f608c0a32f0 0x7f608c0a57b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:21.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.984+0000 7f60ad200640 1 --2- 172.21.15.111:0/3376756640 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f60a8074c50 0x7f60a819e520 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:21.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.984+0000 7f60ad200640 1 -- 172.21.15.111:0/3376756640 >> 172.21.15.111:0/3376756640 conn(0x7f60a80fd390 msgr2=0x7f60a8101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:21.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.984+0000 7f60ad200640 1 -- 172.21.15.111:0/3376756640 shutdown_connections 2024-09-15T22:43:21.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:21.984+0000 7f60ad200640 1 -- 172.21.15.111:0/3376756640 wait complete. 2024-09-15T22:43:22.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:21 smithi192 ceph-mon[35427]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:43:22.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:21 smithi192 ceph-mon[35427]: osdmap e79: 12 total, 12 up, 12 in 2024-09-15T22:43:22.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:22 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/3376756640' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:43:22.832 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":79,"num_osds":12,"num_up_osds":12,"osd_up_since":1726440199,"num_in_osds":12,"osd_in_since":1726440176,"num_remapped_pgs":0} 2024-09-15T22:43:22.833 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:43:22.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:22 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/3376756640' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:43:23.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:22 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/3376756640' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:43:23.197 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:23.613 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:23 smithi111 ceph-mon[30629]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:23.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.721+0000 7f0854400640 1 -- 172.21.15.111:0/3140964532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f084c106f10 msgr2=0x7f084c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:23.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.721+0000 7f0854400640 1 --2- 172.21.15.111:0/3140964532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f084c106f10 0x7f084c10ea90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0844009ca0 tx=0x7f084402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:43:23.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.721+0000 7f0854400640 1 -- 172.21.15.111:0/3140964532 shutdown_connections 2024-09-15T22:43:23.724 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.721+0000 7f0854400640 1 --2- 172.21.15.111:0/3140964532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f084c106f10 0x7f084c10ea90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:23.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.721+0000 7f0854400640 1 --2- 172.21.15.111:0/3140964532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f084c102790 0x7f084c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:23.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.721+0000 7f0854400640 1 --2- 172.21.15.111:0/3140964532 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f084c101de0 0x7f084c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:23.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.721+0000 7f0854400640 1 -- 172.21.15.111:0/3140964532 >> 172.21.15.111:0/3140964532 conn(0x7f084c0fd8c0 msgr2=0x7f084c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:23.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.721+0000 7f0854400640 1 -- 172.21.15.111:0/3140964532 shutdown_connections 2024-09-15T22:43:23.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.721+0000 7f0854400640 1 -- 172.21.15.111:0/3140964532 wait complete. 2024-09-15T22:43:23.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.722+0000 7f0854400640 1 Processor -- start 2024-09-15T22:43:23.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.723+0000 7f0854400640 1 -- start start 2024-09-15T22:43:23.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.723+0000 7f0854400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f084c101de0 0x7f084c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:23.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.723+0000 7f0854400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f084c102790 0x7f084c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:23.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.723+0000 7f0854400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f084c106f10 0x7f084c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:23.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.723+0000 7f0854400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f084c111fc0 con 0x7f084c101de0 2024-09-15T22:43:23.727 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.723+0000 7f0854400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f084c111e40 con 0x7f084c106f10 2024-09-15T22:43:23.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.723+0000 7f0854400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f084c112140 con 0x7f084c102790 2024-09-15T22:43:23.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.723+0000 7f0851a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f084c101de0 0x7f084c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:23.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.723+0000 7f0851a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f084c101de0 0x7f084c19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56982/0 (socket says 172.21.15.111:56982) 2024-09-15T22:43:23.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.723+0000 7f0851a00640 1 -- 172.21.15.111:0/4017838664 learned_addr learned my addr 172.21.15.111:0/4017838664 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:43:23.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.723+0000 7f0851000640 1 --2- 172.21.15.111:0/4017838664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f084c102790 0x7f084c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:23.728 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.723+0000 7f0852400640 1 --2- 172.21.15.111:0/4017838664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f084c106f10 0x7f084c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:23.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.724+0000 7f0851a00640 1 -- 172.21.15.111:0/4017838664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f084c102790 msgr2=0x7f084c19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:23.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.724+0000 7f0851a00640 1 --2- 172.21.15.111:0/4017838664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f084c102790 0x7f084c19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:23.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.724+0000 7f0851a00640 1 -- 172.21.15.111:0/4017838664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f084c106f10 msgr2=0x7f084c19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:23.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.724+0000 7f0851a00640 1 --2- 172.21.15.111:0/4017838664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f084c106f10 0x7f084c19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:23.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.724+0000 7f0851a00640 1 -- 172.21.15.111:0/4017838664 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084c19f5c0 con 0x7f084c101de0 2024-09-15T22:43:23.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.724+0000 7f0851a00640 1 --2- 172.21.15.111:0/4017838664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f084c101de0 0x7f084c19a610 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f084000e9f0 tx=0x7f084000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:23.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.724+0000 7f084a000640 1 -- 172.21.15.111:0/4017838664 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084000cd80 con 0x7f084c101de0 2024-09-15T22:43:23.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.724+0000 7f084a000640 1 -- 172.21.15.111:0/4017838664 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0840010910 con 0x7f084c101de0 2024-09-15T22:43:23.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.724+0000 7f0854400640 1 -- 172.21.15.111:0/4017838664 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f084c19f8b0 con 0x7f084c101de0 2024-09-15T22:43:23.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.724+0000 7f084a000640 1 -- 172.21.15.111:0/4017838664 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0840018990 con 0x7f084c101de0 2024-09-15T22:43:23.729 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.724+0000 7f0854400640 1 -- 172.21.15.111:0/4017838664 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f084c1a7190 con 0x7f084c101de0 2024-09-15T22:43:23.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.726+0000 7f0854400640 1 -- 172.21.15.111:0/4017838664 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f084c1021c0 con 0x7f084c101de0 2024-09-15T22:43:23.730 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.727+0000 7f084a000640 1 -- 172.21.15.111:0/4017838664 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f0840010ac0 con 0x7f084c101de0 2024-09-15T22:43:23.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.728+0000 7f084a000640 1 --2- 172.21.15.111:0/4017838664 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f08340a3240 0x7f08340a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:23.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.729+0000 7f084a000640 1 -- 172.21.15.111:0/4017838664 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f08400cf820 con 0x7f084c101de0 2024-09-15T22:43:23.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.729+0000 7f0851000640 1 --2- 172.21.15.111:0/4017838664 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f08340a3240 0x7f08340a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:23.731 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.729+0000 7f0851000640 1 --2- 172.21.15.111:0/4017838664 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f08340a3240 0x7f08340a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f084c10eeb0 tx=0x7f0838005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:23.734 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.732+0000 7f084a000640 1 -- 172.21.15.111:0/4017838664 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f084009c2f0 con 0x7f084c101de0 2024-09-15T22:43:23.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:23 smithi121 ceph-mon[34815]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:23.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.909+0000 7f0854400640 1 -- 172.21.15.111:0/4017838664 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f084c1a7480 con 0x7f084c101de0 2024-09-15T22:43:23.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.911+0000 7f084a000640 1 -- 172.21.15.111:0/4017838664 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+18552 (secure 0 0 0) 0x7f08400a05f0 con 0x7f084c101de0 2024-09-15T22:43:23.913 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:43:23.913 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":79,"fsid":"b7614d3c-73b2-11ef-bceb-c7b262605968","created":"2024-09-15T22:35:52.571473+0000","modified":"2024-09-15T22:43:20.569352+0000","last_up_change":"2024-09-15T22:43:19.567274+0000","last_in_change":"2024-09-15T22:42:56.994246+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:39:18.954207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04f39604-be0e-4261-b316-67fa087c6666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6803","nonce":1386823446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6805","nonce":1386823446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6809","nonce":1386823446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6807","nonce":1386823446}]},"public_addr":"172.21.15.111:6803/1386823446","cluster_addr":"172.21.15.111:6805/1386823446","heartbeat_back_addr":"172.21.15.111:6809/1386823446","heartbeat_front_addr":"172.21.15.111:6807/1386823446","state":["exists","up"]},{"osd":1,"uuid":"1bdbef6e-23fa-4618-a788-66e57741a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6801","nonce":2051512291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6803","nonce":2051512291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6807","nonce":2051512291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6805","nonce":2051512291}]},"public_addr":"172.21.15.121:6801/2051512291","cluster_addr":"172.21.15.121:6803/2051512291","heartbeat_back_addr":"172.21.15.121:6807/2051512291","heartbeat_front_addr":"172.21.15.121:6805/2051512291","state":["exists","up"]},{"osd":2,"uuid":"46735ccf-6d0e-4154-bb55-422929819bfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6801","nonce":93085639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6803","nonce":93085639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6807","nonce":93085639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6805","nonce":93085639}]},"public_addr":"172.21.15.192:6801/93085639","cluster_addr":"172.21.15.192:6803/93085639","heartbeat_back_addr":"172.21.15.192:6807/93085639","heartbeat_front_addr":"172.21.15.192:6805/93085639","state":["exists","up"]},{"osd":3,"uuid":"4b243af8-c958-40b1-ba24-9c7ace00c4a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6811","nonce":2578259041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6813","nonce":2578259041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6817","nonce":2578259041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6815","nonce":2578259041}]},"public_addr":"172.21.15.111:6811/2578259041","cluster_addr":"172.21.15.111:6813/2578259041","heartbeat_back_addr":"172.21.15.111:6817/2578259041","heartbeat_front_addr":"172.21.15.111:6815/2578259041","state":["exists","up"]},{"osd":4,"uuid":"49f27541-9cbf-48e0-a466-6bb0fa5410b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6809","nonce":2667595915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6811","nonce":2667595915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6815","nonce":2667595915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6813","nonce":2667595915}]},"public_addr":"172.21.15.121:6809/2667595915","cluster_addr":"172.21.15.121:6811/2667595915","heartbeat_back_addr":"172.21.15.121:6815/2667595915","heartbeat_front_addr":"172.21.15.121:6813/2667595915","state":["exists","up"]},{"osd":5,"uuid":"9290e769-5470-44d3-8d24-25093af34335","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6809","nonce":1958639856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6811","nonce":1958639856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6815","nonce":1958639856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6813","nonce":1958639856}]},"public_addr":"172.21.15.192:6809/1958639856","cluster_addr":"172.21.15.192:6811/1958639856","heartbeat_back_addr":"172.21.15.192:6815/1958639856","heartbeat_front_addr":"172.21.15.192:6813/1958639856","state":["exists","up"]},{"osd":6,"uuid":"18911b97-1c68-4e8f-92ec-35d2ff0b5f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6819","nonce":1371543706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6821","nonce":1371543706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6825","nonce":1371543706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6823","nonce":1371543706}]},"public_addr":"172.21.15.111:6819/1371543706","cluster_addr":"172.21.15.111:6821/1371543706","heartbeat_back_addr":"172.21.15.111:6825/1371543706","heartbeat_front_addr":"172.21.15.111:6823/1371543706","state":["exists","up"]},{"osd":7,"uuid":"470647c5-c00d-426b-a134-cca235902307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6817","nonce":3501842998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6819","nonce":3501842998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6823","nonce":3501842998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6821","nonce":3501842998}]},"public_addr":"172.21.15.121:6817/3501842998","cluster_addr":"172.21.15.121:6819/3501842998","heartbeat_back_addr":"172.21.15.121:6823/3501842998","heartbeat_front_addr":"172.21.15.121:6821/3501842998","state":["exists","up"]},{"osd":8,"uuid":"fa2d3dd5-a768-4ada-a036-4c596b76d654","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6817","nonce":2417771374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6819","nonce":2417771374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6823","nonce":2417771374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6821","nonce":2417771374}]},"public_addr":"172.21.15.192:6817/2417771374","cluster_addr":"172.21.15.192:6819/2417771374","heartbeat_back_addr":"172.21.15.192:6823/2417771374","heartbeat_front_addr":"172.21.15.192:6821/2417771374","state":["exists","up"]},{"osd":9,"uuid":"71b4f5fb-1c02-461b-a257-c6e48381d768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6827","nonce":3979119431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6829","nonce":3979119431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6833","nonce":3979119431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6831","nonce":3979119431}]},"public_addr":"172.21.15.111:6827/3979119431","cluster_addr":"172.21.15.111:6829/3979119431","heartbeat_back_addr":"172.21.15.111:6833/3979119431","heartbeat_front_addr":"172.21.15.111:6831/3979119431","state":["exists","up"]},{"osd":10,"uuid":"e6a39e77-fce3-4e22-bf66-a677c9f8b6bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6825","nonce":2897902637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6827","nonce":2897902637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6831","nonce":2897902637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6829","nonce":2897902637}]},"public_addr":"172.21.15.121:6825/2897902637","cluster_addr":"172.21.15.121:6827/2897902637","heartbeat_back_addr":"172.21.15.121:6831/2897902637","heartbeat_front_addr":"172.21.15.121:6829/2897902637","state":["exists","up"]},{"osd":11,"uuid":"411a7921-6717-4cef-b8df-d9e252edcc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6825","nonce":2431464812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6827","nonce":2431464812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6831","nonce":2431464812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6829","nonce":2431464812}]},"public_addr":"172.21.15.192:6825/2431464812","cluster_addr":"172.21.15.192:6827/2431464812","heartbeat_back_addr":"172.21.15.192:6831/2431464812","heartbeat_front_addr":"172.21.15.192:6829/2431464812","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:38:23.442715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:38:49.501124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:39:14.759077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:39:45.121397+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:40:11.137612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:40:36.278567+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:07.027190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:32.439215+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:57.058702+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:42:28.741846+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:42:53.395442+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:43:18.265067+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:6801/2107775445":"2024-09-16T22:36:50.357551+0000","172.21.15.111:6800/2107775445":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/2394746540":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/4140020843":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/2114354263":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/1054173642":"2024-09-16T22:36:22.652875+0000","172.21.15.111:0/3408238004":"2024-09-16T22:36:22.652875+0000","172.21.15.111:0/2879044772":"2024-09-16T22:36:22.652875+0000","172.21.15.111:6801/3350425104":"2024-09-16T22:36:22.652875+0000","172.21.15.111:6800/3350425104":"2024-09-16T22:36:22.652875+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:43:23.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.914+0000 7f0854400640 1 -- 172.21.15.111:0/4017838664 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f08340a3240 msgr2=0x7f08340a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:23.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.914+0000 7f0854400640 1 --2- 172.21.15.111:0/4017838664 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f08340a3240 0x7f08340a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f084c10eeb0 tx=0x7f0838005f90 comp rx=0 tx=0).stop 2024-09-15T22:43:23.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.914+0000 7f0854400640 1 -- 172.21.15.111:0/4017838664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f084c101de0 msgr2=0x7f084c19a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:23.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.914+0000 7f0854400640 1 --2- 172.21.15.111:0/4017838664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f084c101de0 0x7f084c19a610 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f084000e9f0 tx=0x7f084000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:43:23.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.914+0000 7f0854400640 1 -- 172.21.15.111:0/4017838664 shutdown_connections 2024-09-15T22:43:23.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.914+0000 7f0854400640 1 --2- 172.21.15.111:0/4017838664 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f08340a3240 0x7f08340a5700 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:23.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.914+0000 7f0854400640 1 --2- 172.21.15.111:0/4017838664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f084c106f10 0x7f084c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:23.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.914+0000 7f0854400640 1 --2- 172.21.15.111:0/4017838664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f084c102790 0x7f084c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:23.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.914+0000 7f0854400640 1 --2- 172.21.15.111:0/4017838664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f084c101de0 0x7f084c19a610 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:23.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.914+0000 7f0854400640 1 -- 172.21.15.111:0/4017838664 >> 172.21.15.111:0/4017838664 conn(0x7f084c0fd8c0 msgr2=0x7f084c0ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:23.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.914+0000 7f0854400640 1 -- 172.21.15.111:0/4017838664 shutdown_connections 2024-09-15T22:43:23.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:23.914+0000 7f0854400640 1 -- 172.21.15.111:0/4017838664 wait complete. 2024-09-15T22:43:24.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:23 smithi192 ceph-mon[35427]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:24.747 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T22:39:18.954207+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-15T22:43:24.747 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-15T22:43:24.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:24 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/4017838664' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:43:24.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:24 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/4017838664' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:43:25.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:24 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/4017838664' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:43:25.111 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:25.870 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:25 smithi111 ceph-mon[30629]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:25.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.880+0000 7f5cb0400640 1 -- 172.21.15.111:0/2483498832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ca8106f10 msgr2=0x7f5ca810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:25.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.880+0000 7f5cb0400640 1 --2- 172.21.15.111:0/2483498832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ca8106f10 0x7f5ca810ea90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0009900 tx=0x7f5ca002a410 comp rx=0 tx=0).stop 2024-09-15T22:43:25.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.880+0000 7f5cb0400640 1 -- 172.21.15.111:0/2483498832 shutdown_connections 2024-09-15T22:43:25.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.880+0000 7f5cb0400640 1 --2- 172.21.15.111:0/2483498832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ca8106f10 0x7f5ca810ea90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:25.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.880+0000 7f5cb0400640 1 --2- 172.21.15.111:0/2483498832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ca8102790 0x7f5ca81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:25.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.880+0000 7f5cb0400640 1 --2- 172.21.15.111:0/2483498832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ca8101de0 0x7f5ca81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:25.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.880+0000 7f5cb0400640 1 -- 172.21.15.111:0/2483498832 >> 172.21.15.111:0/2483498832 conn(0x7f5ca80fd8c0 msgr2=0x7f5ca80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:25.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.880+0000 7f5cb0400640 1 -- 172.21.15.111:0/2483498832 shutdown_connections 2024-09-15T22:43:25.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.880+0000 7f5cb0400640 1 -- 172.21.15.111:0/2483498832 wait complete. 2024-09-15T22:43:25.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.882+0000 7f5cb0400640 1 Processor -- start 2024-09-15T22:43:25.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.882+0000 7f5cb0400640 1 -- start start 2024-09-15T22:43:25.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.882+0000 7f5cb0400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ca8101de0 0x7f5ca819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:25.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.882+0000 7f5cb0400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ca8102790 0x7f5ca819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:25.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.882+0000 7f5cb0400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ca8106f10 0x7f5ca819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:25.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.882+0000 7f5cb0400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca8111fc0 con 0x7f5ca8102790 2024-09-15T22:43:25.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.882+0000 7f5cb0400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca8111e40 con 0x7f5ca8106f10 2024-09-15T22:43:25.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.882+0000 7f5cb0400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca8112140 con 0x7f5ca8101de0 2024-09-15T22:43:25.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.882+0000 7f5cad000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ca8102790 0x7f5ca819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:25.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.882+0000 7f5cad000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ca8102790 0x7f5ca819ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56994/0 (socket says 172.21.15.111:56994) 2024-09-15T22:43:25.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.882+0000 7f5cad000640 1 -- 172.21.15.111:0/1697089475 learned_addr learned my addr 172.21.15.111:0/1697089475 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:43:25.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.882+0000 7f5cae400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ca8106f10 0x7f5ca819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:25.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.883+0000 7f5cada00640 1 --2- 172.21.15.111:0/1697089475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ca8101de0 0x7f5ca819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:25.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.883+0000 7f5cad000640 1 -- 172.21.15.111:0/1697089475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ca8101de0 msgr2=0x7f5ca819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:25.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.883+0000 7f5cad000640 1 --2- 172.21.15.111:0/1697089475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ca8101de0 0x7f5ca819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:25.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.883+0000 7f5cad000640 1 -- 172.21.15.111:0/1697089475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ca8106f10 msgr2=0x7f5ca819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:25.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.883+0000 7f5cad000640 1 --2- 172.21.15.111:0/1697089475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ca8106f10 0x7f5ca819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:25.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.883+0000 7f5cad000640 1 -- 172.21.15.111:0/1697089475 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca819f670 con 0x7f5ca8102790 2024-09-15T22:43:25.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.883+0000 7f5cad000640 1 --2- 172.21.15.111:0/1697089475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ca8102790 0x7f5ca819ac00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5c9400e9f0 tx=0x7f5c9400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:25.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.883+0000 7f5ca6000640 1 -- 172.21.15.111:0/1697089475 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9400cd80 con 0x7f5ca8102790 2024-09-15T22:43:25.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.883+0000 7f5ca6000640 1 -- 172.21.15.111:0/1697089475 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c94010910 con 0x7f5ca8102790 2024-09-15T22:43:25.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.883+0000 7f5cb0400640 1 -- 172.21.15.111:0/1697089475 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca819f960 con 0x7f5ca8102790 2024-09-15T22:43:25.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.884+0000 7f5ca6000640 1 -- 172.21.15.111:0/1697089475 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c94018990 con 0x7f5ca8102790 2024-09-15T22:43:25.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.884+0000 7f5cb0400640 1 -- 172.21.15.111:0/1697089475 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca81a7240 con 0x7f5ca8102790 2024-09-15T22:43:25.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.885+0000 7f5cb0400640 1 -- 172.21.15.111:0/1697089475 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ca81021c0 con 0x7f5ca8102790 2024-09-15T22:43:25.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.886+0000 7f5ca6000640 1 -- 172.21.15.111:0/1697089475 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f5c94010ac0 con 0x7f5ca8102790 2024-09-15T22:43:25.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.888+0000 7f5ca6000640 1 --2- 172.21.15.111:0/1697089475 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5c900a32c0 0x7f5c900a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:25.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.888+0000 7f5ca6000640 1 -- 172.21.15.111:0/1697089475 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f5c940cf6e0 con 0x7f5ca8102790 2024-09-15T22:43:25.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.888+0000 7f5cada00640 1 --2- 172.21.15.111:0/1697089475 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5c900a32c0 0x7f5c900a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:25.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:25 smithi121 ceph-mon[34815]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:25.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.889+0000 7f5cada00640 1 --2- 172.21.15.111:0/1697089475 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5c900a32c0 0x7f5c900a5780 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c005fd0 tx=0x7f5c9c009760 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:25.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:25.892+0000 7f5ca6000640 1 -- 172.21.15.111:0/1697089475 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5c9409c1b0 con 0x7f5ca8102790 2024-09-15T22:43:26.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:25 smithi192 ceph-mon[35427]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:26.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:26.066+0000 7f5cb0400640 1 -- 172.21.15.111:0/1697089475 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5ca81a74a0 con 0x7f5ca8102790 2024-09-15T22:43:26.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:26.066+0000 7f5ca6000640 1 -- 172.21.15.111:0/1697089475 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7f5c940a04b0 con 0x7f5ca8102790 2024-09-15T22:43:26.068 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-09-15T22:43:26.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:26.070+0000 7f5cb0400640 1 -- 172.21.15.111:0/1697089475 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5c900a32c0 msgr2=0x7f5c900a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:26.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:26.070+0000 7f5cb0400640 1 --2- 172.21.15.111:0/1697089475 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5c900a32c0 0x7f5c900a5780 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c005fd0 tx=0x7f5c9c009760 comp rx=0 tx=0).stop 2024-09-15T22:43:26.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:26.070+0000 7f5cb0400640 1 -- 172.21.15.111:0/1697089475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ca8102790 msgr2=0x7f5ca819ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:26.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:26.070+0000 7f5cb0400640 1 --2- 172.21.15.111:0/1697089475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ca8102790 0x7f5ca819ac00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5c9400e9f0 tx=0x7f5c9400eeb0 comp rx=0 tx=0).stop 2024-09-15T22:43:26.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:26.070+0000 7f5cb0400640 1 -- 172.21.15.111:0/1697089475 shutdown_connections 2024-09-15T22:43:26.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:26.070+0000 7f5cb0400640 1 --2- 172.21.15.111:0/1697089475 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5c900a32c0 0x7f5c900a5780 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:26.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:26.070+0000 7f5cb0400640 1 --2- 172.21.15.111:0/1697089475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ca8106f10 0x7f5ca819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:26.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:26.070+0000 7f5cb0400640 1 --2- 172.21.15.111:0/1697089475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ca8102790 0x7f5ca819ac00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:26.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:26.070+0000 7f5cb0400640 1 --2- 172.21.15.111:0/1697089475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ca8101de0 0x7f5ca819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:26.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:26.070+0000 7f5cb0400640 1 -- 172.21.15.111:0/1697089475 >> 172.21.15.111:0/1697089475 conn(0x7f5ca80fd8c0 msgr2=0x7f5ca80ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:26.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:26.070+0000 7f5cb0400640 1 -- 172.21.15.111:0/1697089475 shutdown_connections 2024-09-15T22:43:26.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:26.070+0000 7f5cb0400640 1 -- 172.21.15.111:0/1697089475 wait complete. 2024-09-15T22:43:26.833 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:26 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/1697089475' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:43:26.873 INFO:tasks.cephadm:Adding mds.a on smithi111 2024-09-15T22:43:26.873 INFO:tasks.cephadm:Adding mds.d on smithi111 2024-09-15T22:43:26.873 INFO:tasks.cephadm:Adding mds.g on smithi111 2024-09-15T22:43:26.873 INFO:tasks.cephadm:Adding mds.j on smithi111 2024-09-15T22:43:26.873 INFO:tasks.cephadm:Adding mds.b on smithi121 2024-09-15T22:43:26.873 INFO:tasks.cephadm:Adding mds.e on smithi121 2024-09-15T22:43:26.873 INFO:tasks.cephadm:Adding mds.h on smithi121 2024-09-15T22:43:26.873 INFO:tasks.cephadm:Adding mds.k on smithi121 2024-09-15T22:43:26.873 INFO:tasks.cephadm:Adding mds.c on smithi192 2024-09-15T22:43:26.874 INFO:tasks.cephadm:Adding mds.f on smithi192 2024-09-15T22:43:26.874 INFO:tasks.cephadm:Adding mds.i on smithi192 2024-09-15T22:43:26.874 INFO:tasks.cephadm:Adding mds.l on smithi192 2024-09-15T22:43:26.874 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi111=a;smithi111=d;smithi111=g;smithi111=j;smithi121=b;smithi121=e;smithi121=h;smithi121=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l' 2024-09-15T22:43:26.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:26 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/1697089475' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:43:26.903 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:26 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/1697089475' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:43:27.130 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:43:27.706 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.702+0000 7f77f0339640 1 -- 172.21.15.192:0/924228976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77e81078c0 msgr2=0x7f77e8109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:27.706 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.702+0000 7f77f0339640 1 --2- 172.21.15.192:0/924228976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77e81078c0 0x7f77e8109d10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009ca0 tx=0x7f77e002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:43:27.706 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.702+0000 7f77f0339640 1 -- 172.21.15.192:0/924228976 shutdown_connections 2024-09-15T22:43:27.706 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.702+0000 7f77f0339640 1 --2- 172.21.15.192:0/924228976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77e81078c0 0x7f77e8109d10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:27.707 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.702+0000 7f77f0339640 1 --2- 172.21.15.192:0/924228976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e8104f90 0x7f77e8107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:27.707 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.702+0000 7f77f0339640 1 --2- 172.21.15.192:0/924228976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77e8102660 0x7f77e8104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:27.707 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.702+0000 7f77f0339640 1 -- 172.21.15.192:0/924228976 >> 172.21.15.192:0/924228976 conn(0x7f77e80fc280 msgr2=0x7f77e80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:27.707 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.702+0000 7f77f0339640 1 -- 172.21.15.192:0/924228976 shutdown_connections 2024-09-15T22:43:27.707 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.702+0000 7f77f0339640 1 -- 172.21.15.192:0/924228976 wait complete. 2024-09-15T22:43:27.707 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.702+0000 7f77f0339640 1 Processor -- start 2024-09-15T22:43:27.707 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.702+0000 7f77f0339640 1 -- start start 2024-09-15T22:43:27.707 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.703+0000 7f77f0339640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77e8102660 0x7f77e8197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:27.707 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.703+0000 7f77f0339640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77e8104f90 0x7f77e8198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:27.707 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.703+0000 7f77f0339640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e81078c0 0x7f77e819f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:27.707 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.703+0000 7f77f0339640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f77e810d8f0 con 0x7f77e81078c0 2024-09-15T22:43:27.708 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.703+0000 7f77f0339640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f77e810d770 con 0x7f77e8102660 2024-09-15T22:43:27.708 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.703+0000 7f77f0339640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f77e810da70 con 0x7f77e8104f90 2024-09-15T22:43:27.708 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.703+0000 7f77ed8d4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77e8104f90 0x7f77e8198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:27.708 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.703+0000 7f77ed8d4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77e8104f90 0x7f77e8198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:54252/0 (socket says 172.21.15.192:54252) 2024-09-15T22:43:27.708 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.703+0000 7f77ed8d4640 1 -- 172.21.15.192:0/1366685252 learned_addr learned my addr 172.21.15.192:0/1366685252 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-09-15T22:43:27.708 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.703+0000 7f77ee0d5640 1 --2- 172.21.15.192:0/1366685252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77e8102660 0x7f77e8197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:27.708 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.703+0000 7f77ee8d6640 1 --2- 172.21.15.192:0/1366685252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e81078c0 0x7f77e819f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:27.708 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.703+0000 7f77ed8d4640 1 -- 172.21.15.192:0/1366685252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77e8102660 msgr2=0x7f77e8197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:27.708 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.703+0000 7f77ed8d4640 1 --2- 172.21.15.192:0/1366685252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77e8102660 0x7f77e8197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:27.708 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.703+0000 7f77ed8d4640 1 -- 172.21.15.192:0/1366685252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e81078c0 msgr2=0x7f77e819f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:27.708 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.703+0000 7f77ed8d4640 1 --2- 172.21.15.192:0/1366685252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e81078c0 0x7f77e819f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:27.709 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.703+0000 7f77ed8d4640 1 -- 172.21.15.192:0/1366685252 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e819fb80 con 0x7f77e8104f90 2024-09-15T22:43:27.709 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.703+0000 7f77ee8d6640 1 --2- 172.21.15.192:0/1366685252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e81078c0 0x7f77e819f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:27.709 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.703+0000 7f77ee0d5640 1 --2- 172.21.15.192:0/1366685252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77e8102660 0x7f77e8197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:43:27.709 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.703+0000 7f77ed8d4640 1 --2- 172.21.15.192:0/1366685252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77e8104f90 0x7f77e8198490 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00b450 tx=0x7f77dc00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:27.709 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.703+0000 7f77db7fe640 1 -- 172.21.15.192:0/1366685252 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc015020 con 0x7f77e8104f90 2024-09-15T22:43:27.709 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.704+0000 7f77db7fe640 1 -- 172.21.15.192:0/1366685252 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77dc011d70 con 0x7f77e8104f90 2024-09-15T22:43:27.709 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.704+0000 7f77f0339640 1 -- 172.21.15.192:0/1366685252 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e819fe70 con 0x7f77e8104f90 2024-09-15T22:43:27.709 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.704+0000 7f77db7fe640 1 -- 172.21.15.192:0/1366685252 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77dc00f340 con 0x7f77e8104f90 2024-09-15T22:43:27.709 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.704+0000 7f77f0339640 1 -- 172.21.15.192:0/1366685252 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e81a03a0 con 0x7f77e8104f90 2024-09-15T22:43:27.709 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.705+0000 7f77f0339640 1 -- 172.21.15.192:0/1366685252 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e80fdd70 con 0x7f77e8104f90 2024-09-15T22:43:27.710 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.706+0000 7f77db7fe640 1 -- 172.21.15.192:0/1366685252 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f77dc018020 con 0x7f77e8104f90 2024-09-15T22:43:27.710 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.706+0000 7f77db7fe640 1 --2- 172.21.15.192:0/1366685252 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f77d40a3270 0x7f77d40a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:27.710 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.707+0000 7f77db7fe640 1 -- 172.21.15.192:0/1366685252 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f77dc0cf380 con 0x7f77e8104f90 2024-09-15T22:43:27.711 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.707+0000 7f77ee0d5640 1 --2- 172.21.15.192:0/1366685252 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f77d40a3270 0x7f77d40a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:27.711 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.707+0000 7f77ee0d5640 1 --2- 172.21.15.192:0/1366685252 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f77d40a3270 0x7f77d40a5730 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f77e4005fd0 tx=0x7f77e400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:27.711 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.710+0000 7f77db7fe640 1 -- 172.21.15.192:0/1366685252 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f77dc01f020 con 0x7f77e8104f90 2024-09-15T22:43:27.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:27 smithi192 ceph-mon[35427]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:27.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:27 smithi121 ceph-mon[34815]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:27.890 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.889+0000 7f77f0339640 1 -- 172.21.15.192:0/1366685252 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi111=a;smithi111=d;smithi111=g;smithi111=j;smithi121=b;smithi121=e;smithi121=h;smithi121=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}) -- 0x7f77e8062730 con 0x7f77d40a3270 2024-09-15T22:43:27.898 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.897+0000 7f77db7fe640 1 -- 172.21.15.192:0/1366685252 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f77e8062730 con 0x7f77d40a3270 2024-09-15T22:43:27.898 INFO:teuthology.orchestra.run.smithi192.stdout:Scheduled mds.all update... 2024-09-15T22:43:27.901 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.899+0000 7f77f0339640 1 -- 172.21.15.192:0/1366685252 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f77d40a3270 msgr2=0x7f77d40a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:27.901 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.899+0000 7f77f0339640 1 --2- 172.21.15.192:0/1366685252 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f77d40a3270 0x7f77d40a5730 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f77e4005fd0 tx=0x7f77e400b390 comp rx=0 tx=0).stop 2024-09-15T22:43:27.901 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.899+0000 7f77f0339640 1 -- 172.21.15.192:0/1366685252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77e8104f90 msgr2=0x7f77e8198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:27.901 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.899+0000 7f77f0339640 1 --2- 172.21.15.192:0/1366685252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77e8104f90 0x7f77e8198490 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00b450 tx=0x7f77dc00b910 comp rx=0 tx=0).stop 2024-09-15T22:43:27.901 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.899+0000 7f77f0339640 1 -- 172.21.15.192:0/1366685252 shutdown_connections 2024-09-15T22:43:27.901 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.899+0000 7f77f0339640 1 --2- 172.21.15.192:0/1366685252 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e81078c0 0x7f77e819f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:27.901 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.899+0000 7f77f0339640 1 --2- 172.21.15.192:0/1366685252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77e8104f90 0x7f77e8198490 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:27.901 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.899+0000 7f77f0339640 1 --2- 172.21.15.192:0/1366685252 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f77d40a3270 0x7f77d40a5730 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:27.902 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.899+0000 7f77f0339640 1 --2- 172.21.15.192:0/1366685252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77e8102660 0x7f77e8197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:27.902 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.899+0000 7f77f0339640 1 -- 172.21.15.192:0/1366685252 >> 172.21.15.192:0/1366685252 conn(0x7f77e80fc280 msgr2=0x7f77e8102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:27.902 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.900+0000 7f77f0339640 1 -- 172.21.15.192:0/1366685252 shutdown_connections 2024-09-15T22:43:27.902 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T22:43:27.900+0000 7f77f0339640 1 -- 172.21.15.192:0/1366685252 wait complete. 2024-09-15T22:43:27.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:27 smithi111 ceph-mon[30629]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:28.597 DEBUG:teuthology.orchestra.run.smithi111:mds.a> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.a.service 2024-09-15T22:43:28.601 DEBUG:teuthology.orchestra.run.smithi111:mds.d> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.d.service 2024-09-15T22:43:28.646 DEBUG:teuthology.orchestra.run.smithi111:mds.g> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.g.service 2024-09-15T22:43:28.691 DEBUG:teuthology.orchestra.run.smithi111:mds.j> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.j.service 2024-09-15T22:43:28.738 DEBUG:teuthology.orchestra.run.smithi121:mds.b> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.b.service 2024-09-15T22:43:28.742 DEBUG:teuthology.orchestra.run.smithi121:mds.e> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.e.service 2024-09-15T22:43:28.792 DEBUG:teuthology.orchestra.run.smithi121:mds.h> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.h.service 2024-09-15T22:43:28.852 DEBUG:teuthology.orchestra.run.smithi121:mds.k> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.k.service 2024-09-15T22:43:28.897 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:28 smithi111 ceph-mon[30629]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi111=a;smithi111=d;smithi111=g;smithi111=j;smithi121=b;smithi121=e;smithi121=h;smithi121=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:43:28.913 DEBUG:teuthology.orchestra.run.smithi192:mds.c> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.c.service 2024-09-15T22:43:28.916 DEBUG:teuthology.orchestra.run.smithi192:mds.f> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.f.service 2024-09-15T22:43:28.964 DEBUG:teuthology.orchestra.run.smithi192:mds.i> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.i.service 2024-09-15T22:43:29.010 DEBUG:teuthology.orchestra.run.smithi192:mds.l> sudo journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.l.service 2024-09-15T22:43:29.031 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:28 smithi192 ceph-mon[35427]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi111=a;smithi111=d;smithi111=g;smithi111=j;smithi121=b;smithi121=e;smithi121=h;smithi121=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:43:29.031 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:28 smithi192 ceph-mon[35427]: Saving service mds.all spec with placement smithi111=a;smithi111=d;smithi111=g;smithi111=j;smithi121=b;smithi121=e;smithi121=h;smithi121=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2024-09-15T22:43:29.031 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:28 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:29.031 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:28 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:43:29.031 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:28 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:29.031 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:28 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:43:29.031 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:28 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:29.031 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:28 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:29.031 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:28 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:29.032 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:28 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:29.032 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:28 smithi192 ceph-mon[35427]: Deploying daemon mds.l on smithi192 2024-09-15T22:43:29.032 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:28 smithi192 ceph-mon[35427]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:29.033 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-15T22:43:29.033 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:43:29.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:28 smithi121 ceph-mon[34815]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi111=a;smithi111=d;smithi111=g;smithi111=j;smithi121=b;smithi121=e;smithi121=h;smithi121=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:43:29.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:28 smithi121 ceph-mon[34815]: Saving service mds.all spec with placement smithi111=a;smithi111=d;smithi111=g;smithi111=j;smithi121=b;smithi121=e;smithi121=h;smithi121=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2024-09-15T22:43:29.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:28 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:29.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:28 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:43:29.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:28 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:29.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:28 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:43:29.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:28 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:29.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:28 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:29.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:28 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:29.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:28 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:29.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:28 smithi121 ceph-mon[34815]: Deploying daemon mds.l on smithi192 2024-09-15T22:43:29.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:28 smithi121 ceph-mon[34815]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:29.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:28 smithi111 ceph-mon[30629]: Saving service mds.all spec with placement smithi111=a;smithi111=d;smithi111=g;smithi111=j;smithi121=b;smithi121=e;smithi121=h;smithi121=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2024-09-15T22:43:29.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:28 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:29.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:28 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:43:29.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:28 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:29.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:28 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:43:29.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:28 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:29.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:28 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:29.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:28 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:29.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:28 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:29.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:28 smithi111 ceph-mon[30629]: Deploying daemon mds.l on smithi192 2024-09-15T22:43:29.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:28 smithi111 ceph-mon[30629]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:29.395 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:29.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.926+0000 7feb85000640 1 -- 172.21.15.111:0/4279673720 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb80108f60 msgr2=0x7feb80110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:29.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.926+0000 7feb85000640 1 --2- 172.21.15.111:0/4279673720 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb80108f60 0x7feb80110ae0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feb78009900 tx=0x7feb7802a3f0 comp rx=0 tx=0).stop 2024-09-15T22:43:29.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.926+0000 7feb85000640 1 -- 172.21.15.111:0/4279673720 shutdown_connections 2024-09-15T22:43:29.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.926+0000 7feb85000640 1 --2- 172.21.15.111:0/4279673720 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb80108f60 0x7feb80110ae0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:29.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.926+0000 7feb85000640 1 --2- 172.21.15.111:0/4279673720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb801047e0 0x7feb80108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:29.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.926+0000 7feb85000640 1 --2- 172.21.15.111:0/4279673720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb80103e30 0x7feb80104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:29.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.926+0000 7feb85000640 1 -- 172.21.15.111:0/4279673720 >> 172.21.15.111:0/4279673720 conn(0x7feb800ff8d0 msgr2=0x7feb80101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:29.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.926+0000 7feb85000640 1 -- 172.21.15.111:0/4279673720 shutdown_connections 2024-09-15T22:43:29.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.926+0000 7feb85000640 1 -- 172.21.15.111:0/4279673720 wait complete. 2024-09-15T22:43:29.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.927+0000 7feb85000640 1 Processor -- start 2024-09-15T22:43:29.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.927+0000 7feb85000640 1 -- start start 2024-09-15T22:43:29.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.928+0000 7feb85000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb80103e30 0x7feb8019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:29.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.928+0000 7feb85000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb801047e0 0x7feb8019a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:29.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.928+0000 7feb85000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb80108f60 0x7feb8019eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:29.932 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.928+0000 7feb85000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7feb80114010 con 0x7feb801047e0 2024-09-15T22:43:29.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.928+0000 7feb85000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7feb80113e90 con 0x7feb80103e30 2024-09-15T22:43:29.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.928+0000 7feb85000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7feb80114190 con 0x7feb80108f60 2024-09-15T22:43:29.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.928+0000 7feb76a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb801047e0 0x7feb8019a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:29.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.928+0000 7feb76a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb801047e0 0x7feb8019a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57018/0 (socket says 172.21.15.111:57018) 2024-09-15T22:43:29.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.928+0000 7feb76a00640 1 -- 172.21.15.111:0/4229301147 learned_addr learned my addr 172.21.15.111:0/4229301147 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:43:29.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.928+0000 7feb7f400640 1 --2- 172.21.15.111:0/4229301147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb80103e30 0x7feb8019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:29.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.928+0000 7feb7fe00640 1 --2- 172.21.15.111:0/4229301147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb80108f60 0x7feb8019eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:29.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.928+0000 7feb76a00640 1 -- 172.21.15.111:0/4229301147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb80108f60 msgr2=0x7feb8019eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:29.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.928+0000 7feb76a00640 1 --2- 172.21.15.111:0/4229301147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb80108f60 0x7feb8019eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:29.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.928+0000 7feb76a00640 1 -- 172.21.15.111:0/4229301147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb80103e30 msgr2=0x7feb8019a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:29.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.928+0000 7feb76a00640 1 --2- 172.21.15.111:0/4229301147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb80103e30 0x7feb8019a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:29.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.928+0000 7feb76a00640 1 -- 172.21.15.111:0/4229301147 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb8019f380 con 0x7feb801047e0 2024-09-15T22:43:29.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.929+0000 7feb76a00640 1 --2- 172.21.15.111:0/4229301147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb801047e0 0x7feb8019a910 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feb6800db80 tx=0x7feb6800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:29.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.929+0000 7feb7cc00640 1 -- 172.21.15.111:0/4229301147 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb68017070 con 0x7feb801047e0 2024-09-15T22:43:29.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.929+0000 7feb7cc00640 1 -- 172.21.15.111:0/4229301147 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb68003b70 con 0x7feb801047e0 2024-09-15T22:43:29.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.929+0000 7feb85000640 1 -- 172.21.15.111:0/4229301147 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb8019f670 con 0x7feb801047e0 2024-09-15T22:43:29.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.929+0000 7feb7cc00640 1 -- 172.21.15.111:0/4229301147 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb680106c0 con 0x7feb801047e0 2024-09-15T22:43:29.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.929+0000 7feb85000640 1 -- 172.21.15.111:0/4229301147 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb801a6f50 con 0x7feb801047e0 2024-09-15T22:43:29.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.931+0000 7feb85000640 1 -- 172.21.15.111:0/4229301147 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb80104210 con 0x7feb801047e0 2024-09-15T22:43:29.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.932+0000 7feb7cc00640 1 -- 172.21.15.111:0/4229301147 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7feb68004020 con 0x7feb801047e0 2024-09-15T22:43:29.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.934+0000 7feb7cc00640 1 --2- 172.21.15.111:0/4229301147 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7feb640a32c0 0x7feb640a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:29.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.934+0000 7feb7cc00640 1 -- 172.21.15.111:0/4229301147 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7feb680cf160 con 0x7feb801047e0 2024-09-15T22:43:29.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.934+0000 7feb7f400640 1 --2- 172.21.15.111:0/4229301147 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7feb640a32c0 0x7feb640a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:29.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.935+0000 7feb7f400640 1 --2- 172.21.15.111:0/4229301147 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7feb640a32c0 0x7feb640a5780 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feb700095f0 tx=0x7feb70005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:29.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:29.938+0000 7feb7cc00640 1 -- 172.21.15.111:0/4229301147 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feb68020070 con 0x7feb801047e0 2024-09-15T22:43:30.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:30.148+0000 7feb85000640 1 -- 172.21.15.111:0/4229301147 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb8010c1a0 con 0x7feb801047e0 2024-09-15T22:43:30.150 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:30.149+0000 7feb7cc00640 1 -- 172.21.15.111:0/4229301147 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7feb6809c280 con 0x7feb801047e0 2024-09-15T22:43:30.150 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:43:30.150 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":1,"btime":"2024-09-15T22:35:52:571222+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-15T22:43:30.151 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 1 2024-09-15T22:43:30.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:30.151+0000 7feb85000640 1 -- 172.21.15.111:0/4229301147 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7feb640a32c0 msgr2=0x7feb640a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:30.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:30.151+0000 7feb85000640 1 --2- 172.21.15.111:0/4229301147 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7feb640a32c0 0x7feb640a5780 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feb700095f0 tx=0x7feb70005f90 comp rx=0 tx=0).stop 2024-09-15T22:43:30.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:30.151+0000 7feb85000640 1 -- 172.21.15.111:0/4229301147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb801047e0 msgr2=0x7feb8019a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:30.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:30.151+0000 7feb85000640 1 --2- 172.21.15.111:0/4229301147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb801047e0 0x7feb8019a910 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feb6800db80 tx=0x7feb6800b630 comp rx=0 tx=0).stop 2024-09-15T22:43:30.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:30.152+0000 7feb85000640 1 -- 172.21.15.111:0/4229301147 shutdown_connections 2024-09-15T22:43:30.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:30.152+0000 7feb85000640 1 --2- 172.21.15.111:0/4229301147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb80108f60 0x7feb8019eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:30.154 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:30.152+0000 7feb85000640 1 --2- 172.21.15.111:0/4229301147 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb801047e0 0x7feb8019a910 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:30.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:30.152+0000 7feb85000640 1 --2- 172.21.15.111:0/4229301147 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7feb640a32c0 0x7feb640a5780 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:30.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:30.152+0000 7feb85000640 1 --2- 172.21.15.111:0/4229301147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb80103e30 0x7feb8019a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:30.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:30.152+0000 7feb85000640 1 -- 172.21.15.111:0/4229301147 >> 172.21.15.111:0/4229301147 conn(0x7feb800ff8d0 msgr2=0x7feb80100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:30.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:30.152+0000 7feb85000640 1 -- 172.21.15.111:0/4229301147 shutdown_connections 2024-09-15T22:43:30.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:30.152+0000 7feb85000640 1 -- 172.21.15.111:0/4229301147 wait complete. 2024-09-15T22:43:30.298 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 15 22:43:30 smithi192 systemd[1]: Starting Ceph mds.l for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T22:43:30.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:30 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/4229301147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:30.559 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:30 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/4229301147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:30.559 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 15 22:43:30 smithi192 podman[65226]: 2024-09-15 22:43:30.32480527 +0000 UTC m=+0.016209793 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:43:30.559 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 15 22:43:30 smithi192 podman[65226]: 2024-09-15 22:43:30.432531998 +0000 UTC m=+0.123936563 container create 9f94d35c51ebce28eaf7c475bae58406f7bad871d7b1146301f56a3d13fcc274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-l, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-15T22:43:30.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:30 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/4229301147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:31.065 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 15 22:43:30 smithi192 podman[65226]: 2024-09-15 22:43:30.701608527 +0000 UTC m=+0.393013051 container init 9f94d35c51ebce28eaf7c475bae58406f7bad871d7b1146301f56a3d13fcc274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-15T22:43:31.065 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 15 22:43:30 smithi192 podman[65226]: 2024-09-15 22:43:30.704365823 +0000 UTC m=+0.395770351 container start 9f94d35c51ebce28eaf7c475bae58406f7bad871d7b1146301f56a3d13fcc274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-l, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-15T22:43:31.065 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 15 22:43:30 smithi192 ceph-mds[65245]: -- 172.21.15.192:0/439942247 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558771b59860 con 0x558771b36800 2024-09-15T22:43:31.065 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 15 22:43:30 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-l[65241]: starting mds.l at 2024-09-15T22:43:31.065 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 15 22:43:30 smithi192 bash[65226]: 9f94d35c51ebce28eaf7c475bae58406f7bad871d7b1146301f56a3d13fcc274 2024-09-15T22:43:31.065 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 15 22:43:30 smithi192 systemd[1]: Started Ceph mds.l for b7614d3c-73b2-11ef-bceb-c7b262605968. 2024-09-15T22:43:31.538 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:31 smithi121 ceph-mon[34815]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:31.538 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:31 smithi121 ceph-mon[34815]: mds.? [v2:172.21.15.192:6832/4061441359,v1:172.21.15.192:6833/4061441359] up:boot 2024-09-15T22:43:31.538 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:31 smithi121 ceph-mon[34815]: fsmap 1 up:standby 2024-09-15T22:43:31.538 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:31 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-15T22:43:31.538 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:31 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:31.538 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:31 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:31.539 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:31 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:31.539 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:31 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:31.539 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:31 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:31.539 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:31 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:31.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:31 smithi192 ceph-mon[35427]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:31.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:31 smithi192 ceph-mon[35427]: mds.? [v2:172.21.15.192:6832/4061441359,v1:172.21.15.192:6833/4061441359] up:boot 2024-09-15T22:43:31.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:31 smithi192 ceph-mon[35427]: fsmap 1 up:standby 2024-09-15T22:43:31.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:31 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-15T22:43:31.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:31 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:31.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:31 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:31.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:31 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:31.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:31 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:31.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:31 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:31.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:31 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:31.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:31 smithi111 ceph-mon[30629]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:31.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:31 smithi111 ceph-mon[30629]: mds.? [v2:172.21.15.192:6832/4061441359,v1:172.21.15.192:6833/4061441359] up:boot 2024-09-15T22:43:31.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:31 smithi111 ceph-mon[30629]: fsmap 1 up:standby 2024-09-15T22:43:31.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:31 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-15T22:43:31.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:31 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:31.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:31 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:31.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:31 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:31.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:31 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:31.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:31 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:31.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:31 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:32.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:32 smithi192 ceph-mon[35427]: Deploying daemon mds.k on smithi121 2024-09-15T22:43:32.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:32 smithi121 ceph-mon[34815]: Deploying daemon mds.k on smithi121 2024-09-15T22:43:32.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:32 smithi111 ceph-mon[30629]: Deploying daemon mds.k on smithi121 2024-09-15T22:43:32.935 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:43:33.023 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 15 22:43:32 smithi121 systemd[1]: Starting Ceph mds.k for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T22:43:33.305 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 15 22:43:33 smithi121 podman[64546]: 2024-09-15 22:43:32.924989197 +0000 UTC m=+0.015641465 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:43:33.305 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 15 22:43:33 smithi121 podman[64546]: 2024-09-15 22:43:33.026470047 +0000 UTC m=+0.117122377 container create c856b2b02278263463ed386b9e3426a53171d52161722e3ccf5fc3c817c47a2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-k, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True) 2024-09-15T22:43:33.305 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 15 22:43:33 smithi121 podman[64546]: 2024-09-15 22:43:33.236999695 +0000 UTC m=+0.327651965 container init c856b2b02278263463ed386b9e3426a53171d52161722e3ccf5fc3c817c47a2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-15T22:43:33.305 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 15 22:43:33 smithi121 podman[64546]: 2024-09-15 22:43:33.239753411 +0000 UTC m=+0.330405679 container start c856b2b02278263463ed386b9e3426a53171d52161722e3ccf5fc3c817c47a2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-k, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:43:33.305 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 15 22:43:33 smithi121 ceph-mds[64568]: -- 172.21.15.121:0/3845230265 <== mon.2 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564520e39860 con 0x564520e17400 2024-09-15T22:43:33.305 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 15 22:43:33 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-k[64564]: starting mds.k at 2024-09-15T22:43:33.305 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 15 22:43:33 smithi121 bash[64546]: c856b2b02278263463ed386b9e3426a53171d52161722e3ccf5fc3c817c47a2c 2024-09-15T22:43:33.308 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:33.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:33 smithi192 ceph-mon[35427]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:33.607 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:33 smithi111 ceph-mon[30629]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:33.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:33 smithi121 ceph-mon[34815]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:33.640 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 15 22:43:33 smithi121 systemd[1]: Started Ceph mds.k for b7614d3c-73b2-11ef-bceb-c7b262605968. 2024-09-15T22:43:33.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.875+0000 7fb461600640 1 -- 172.21.15.111:0/2696837114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb45c103e30 msgr2=0x7fb45c104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:33.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.875+0000 7fb461600640 1 --2- 172.21.15.111:0/2696837114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb45c103e30 0x7fb45c104210 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb44c009900 tx=0x7fb44c02a410 comp rx=0 tx=0).stop 2024-09-15T22:43:33.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.875+0000 7fb461600640 1 -- 172.21.15.111:0/2696837114 shutdown_connections 2024-09-15T22:43:33.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.875+0000 7fb461600640 1 --2- 172.21.15.111:0/2696837114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb45c108f60 0x7fb45c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:33.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.875+0000 7fb461600640 1 --2- 172.21.15.111:0/2696837114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb45c1047e0 0x7fb45c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:33.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.875+0000 7fb461600640 1 --2- 172.21.15.111:0/2696837114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb45c103e30 0x7fb45c104210 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:33.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.875+0000 7fb461600640 1 -- 172.21.15.111:0/2696837114 >> 172.21.15.111:0/2696837114 conn(0x7fb45c0ff8d0 msgr2=0x7fb45c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:33.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.875+0000 7fb461600640 1 -- 172.21.15.111:0/2696837114 shutdown_connections 2024-09-15T22:43:33.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.875+0000 7fb461600640 1 -- 172.21.15.111:0/2696837114 wait complete. 2024-09-15T22:43:33.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.876+0000 7fb461600640 1 Processor -- start 2024-09-15T22:43:33.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.876+0000 7fb461600640 1 -- start start 2024-09-15T22:43:33.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.877+0000 7fb461600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb45c103e30 0x7fb45c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:33.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.877+0000 7fb461600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb45c1047e0 0x7fb45c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:33.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.877+0000 7fb461600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb45c108f60 0x7fb45c19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:33.879 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.877+0000 7fb461600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c114010 con 0x7fb45c103e30 2024-09-15T22:43:33.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.877+0000 7fb461600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c113e90 con 0x7fb45c108f60 2024-09-15T22:43:33.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.877+0000 7fb461600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c114190 con 0x7fb45c1047e0 2024-09-15T22:43:33.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.877+0000 7fb45be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb45c103e30 0x7fb45c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:33.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.877+0000 7fb45be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb45c103e30 0x7fb45c19a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59080/0 (socket says 172.21.15.111:59080) 2024-09-15T22:43:33.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.877+0000 7fb45be00640 1 -- 172.21.15.111:0/2404827385 learned_addr learned my addr 172.21.15.111:0/2404827385 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:43:33.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.877+0000 7fb460c00640 1 --2- 172.21.15.111:0/2404827385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb45c108f60 0x7fb45c19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:33.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.877+0000 7fb453400640 1 --2- 172.21.15.111:0/2404827385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb45c1047e0 0x7fb45c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:33.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.878+0000 7fb45be00640 1 -- 172.21.15.111:0/2404827385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb45c1047e0 msgr2=0x7fb45c19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:33.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.878+0000 7fb45be00640 1 --2- 172.21.15.111:0/2404827385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb45c1047e0 0x7fb45c19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:33.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.878+0000 7fb45be00640 1 -- 172.21.15.111:0/2404827385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb45c108f60 msgr2=0x7fb45c19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:33.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.878+0000 7fb45be00640 1 --2- 172.21.15.111:0/2404827385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb45c108f60 0x7fb45c19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:33.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.878+0000 7fb45be00640 1 -- 172.21.15.111:0/2404827385 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45c19f460 con 0x7fb45c103e30 2024-09-15T22:43:33.880 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.878+0000 7fb45be00640 1 --2- 172.21.15.111:0/2404827385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb45c103e30 0x7fb45c19a4b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb44c004770 tx=0x7fb44c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:33.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.878+0000 7fb459600640 1 -- 172.21.15.111:0/2404827385 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c03f070 con 0x7fb45c103e30 2024-09-15T22:43:33.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.878+0000 7fb459600640 1 -- 172.21.15.111:0/2404827385 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb44c031ca0 con 0x7fb45c103e30 2024-09-15T22:43:33.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.878+0000 7fb461600640 1 -- 172.21.15.111:0/2404827385 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb45c19f6f0 con 0x7fb45c103e30 2024-09-15T22:43:33.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.878+0000 7fb459600640 1 -- 172.21.15.111:0/2404827385 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c035030 con 0x7fb45c103e30 2024-09-15T22:43:33.881 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.878+0000 7fb461600640 1 -- 172.21.15.111:0/2404827385 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb45c1a7030 con 0x7fb45c103e30 2024-09-15T22:43:33.882 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.879+0000 7fb461600640 1 -- 172.21.15.111:0/2404827385 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb45c104210 con 0x7fb45c103e30 2024-09-15T22:43:33.883 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.881+0000 7fb459600640 1 -- 172.21.15.111:0/2404827385 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fb44c042070 con 0x7fb45c103e30 2024-09-15T22:43:33.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.883+0000 7fb459600640 1 --2- 172.21.15.111:0/2404827385 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb4400a32c0 0x7fb4400a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:33.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.883+0000 7fb459600640 1 -- 172.21.15.111:0/2404827385 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fb44c0effe0 con 0x7fb45c103e30 2024-09-15T22:43:33.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.883+0000 7fb453400640 1 --2- 172.21.15.111:0/2404827385 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb4400a32c0 0x7fb4400a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:33.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.883+0000 7fb453400640 1 --2- 172.21.15.111:0/2404827385 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb4400a32c0 0x7fb4400a5780 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0ffd60 tx=0x7fb444005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:33.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:33.885+0000 7fb459600640 1 -- 172.21.15.111:0/2404827385 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb44c0bcbd0 con 0x7fb45c103e30 2024-09-15T22:43:34.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:34.111+0000 7fb461600640 1 -- 172.21.15.111:0/2404827385 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb45c10c140 con 0x7fb45c103e30 2024-09-15T22:43:34.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:34.112+0000 7fb459600640 1 -- 172.21.15.111:0/2404827385 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2266 (secure 0 0 0) 0x7fb44c0c0ed0 con 0x7fb45c103e30 2024-09-15T22:43:34.114 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:43:34.114 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":3,"btime":"2024-09-15T22:43:33:306238+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/4061441359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":4061441359},{"type":"v1","addr":"172.21.15.192:6833","nonce":4061441359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/4220531578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":4220531578},{"type":"v1","addr":"172.21.15.121:6833","nonce":4220531578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-15T22:43:34.114 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 3 2024-09-15T22:43:34.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:34.114+0000 7fb461600640 1 -- 172.21.15.111:0/2404827385 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb4400a32c0 msgr2=0x7fb4400a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:34.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:34.114+0000 7fb461600640 1 --2- 172.21.15.111:0/2404827385 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb4400a32c0 0x7fb4400a5780 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0ffd60 tx=0x7fb444005f90 comp rx=0 tx=0).stop 2024-09-15T22:43:34.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:34.115+0000 7fb461600640 1 -- 172.21.15.111:0/2404827385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb45c103e30 msgr2=0x7fb45c19a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:34.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:34.115+0000 7fb461600640 1 --2- 172.21.15.111:0/2404827385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb45c103e30 0x7fb45c19a4b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb44c004770 tx=0x7fb44c003c10 comp rx=0 tx=0).stop 2024-09-15T22:43:34.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:34.115+0000 7fb461600640 1 -- 172.21.15.111:0/2404827385 shutdown_connections 2024-09-15T22:43:34.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:34.115+0000 7fb461600640 1 --2- 172.21.15.111:0/2404827385 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb4400a32c0 0x7fb4400a5780 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:34.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:34.115+0000 7fb461600640 1 --2- 172.21.15.111:0/2404827385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb45c108f60 0x7fb45c19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:34.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:34.115+0000 7fb461600640 1 --2- 172.21.15.111:0/2404827385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb45c1047e0 0x7fb45c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:34.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:34.115+0000 7fb461600640 1 --2- 172.21.15.111:0/2404827385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb45c103e30 0x7fb45c19a4b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:34.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:34.115+0000 7fb461600640 1 -- 172.21.15.111:0/2404827385 >> 172.21.15.111:0/2404827385 conn(0x7fb45c0ff8d0 msgr2=0x7fb45c100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:34.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:34.115+0000 7fb461600640 1 -- 172.21.15.111:0/2404827385 shutdown_connections 2024-09-15T22:43:34.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:34.115+0000 7fb461600640 1 -- 172.21.15.111:0/2404827385 wait complete. 2024-09-15T22:43:34.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:34 smithi111 ceph-mon[30629]: mds.? [v2:172.21.15.121:6832/4220531578,v1:172.21.15.121:6833/4220531578] up:boot 2024-09-15T22:43:34.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:34 smithi111 ceph-mon[30629]: fsmap 2 up:standby 2024-09-15T22:43:34.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:34 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-15T22:43:34.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:34 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:34.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:34 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:34.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:34 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:34.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:34 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:34.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:34 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:34.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:34 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:34.417 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:34 smithi111 ceph-mon[30629]: Deploying daemon mds.j on smithi111 2024-09-15T22:43:34.417 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:34 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/2404827385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:34.625 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:34 smithi121 ceph-mon[34815]: mds.? [v2:172.21.15.121:6832/4220531578,v1:172.21.15.121:6833/4220531578] up:boot 2024-09-15T22:43:34.625 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:34 smithi121 ceph-mon[34815]: fsmap 2 up:standby 2024-09-15T22:43:34.625 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:34 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-15T22:43:34.625 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:34 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:34.625 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:34 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:34.625 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:34 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:34.625 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:34 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:34.625 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:34 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:34.626 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:34 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:34.626 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:34 smithi121 ceph-mon[34815]: Deploying daemon mds.j on smithi111 2024-09-15T22:43:34.626 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:34 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/2404827385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:34.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:34 smithi192 ceph-mon[35427]: mds.? [v2:172.21.15.121:6832/4220531578,v1:172.21.15.121:6833/4220531578] up:boot 2024-09-15T22:43:34.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:34 smithi192 ceph-mon[35427]: fsmap 2 up:standby 2024-09-15T22:43:34.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:34 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-15T22:43:34.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:34 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:34.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:34 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:34.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:34 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:34.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:34 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:34.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:34 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:34.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:34 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:34.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:34 smithi192 ceph-mon[35427]: Deploying daemon mds.j on smithi111 2024-09-15T22:43:34.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:34 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/2404827385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:35.583 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:35 smithi121 ceph-mon[34815]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:35.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:35 smithi111 ceph-mon[30629]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:35.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:35 smithi192 ceph-mon[35427]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:36.916 INFO:journalctl@ceph.mds.j.smithi111.stdout:Sep 15 22:43:36 smithi111 systemd[1]: Starting Ceph mds.j for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T22:43:37.062 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:43:37.267 INFO:journalctl@ceph.mds.j.smithi111.stdout:Sep 15 22:43:36 smithi111 podman[69637]: 2024-09-15 22:43:36.857831051 +0000 UTC m=+0.027484814 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:43:37.267 INFO:journalctl@ceph.mds.j.smithi111.stdout:Sep 15 22:43:36 smithi111 podman[69637]: 2024-09-15 22:43:36.98052182 +0000 UTC m=+0.150175566 container create 395bf4c75badffe1eb260df6b11ebe90f9a6e857c8a91c8f3c68a907c129d7e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-j, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-15T22:43:37.267 INFO:journalctl@ceph.mds.j.smithi111.stdout:Sep 15 22:43:37 smithi111 podman[69637]: 2024-09-15 22:43:37.266346861 +0000 UTC m=+0.436000604 container init 395bf4c75badffe1eb260df6b11ebe90f9a6e857c8a91c8f3c68a907c129d7e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-15T22:43:37.527 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:37.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:37 smithi111 ceph-mon[30629]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:37.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:37 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:37.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:37 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:37.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:37 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:37.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:37 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:37.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:37 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:37.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:37 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:37.666 INFO:journalctl@ceph.mds.j.smithi111.stdout:Sep 15 22:43:37 smithi111 podman[69637]: 2024-09-15 22:43:37.274542326 +0000 UTC m=+0.444196068 container start 395bf4c75badffe1eb260df6b11ebe90f9a6e857c8a91c8f3c68a907c129d7e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-j, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731) 2024-09-15T22:43:37.667 INFO:journalctl@ceph.mds.j.smithi111.stdout:Sep 15 22:43:37 smithi111 ceph-mds[69674]: -- 172.21.15.111:0/1600350735 <== mon.0 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5564fa3dfc20 con 0x5564fa3bc800 2024-09-15T22:43:37.667 INFO:journalctl@ceph.mds.j.smithi111.stdout:Sep 15 22:43:37 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-j[69665]: starting mds.j at 2024-09-15T22:43:37.667 INFO:journalctl@ceph.mds.j.smithi111.stdout:Sep 15 22:43:37 smithi111 bash[69637]: 395bf4c75badffe1eb260df6b11ebe90f9a6e857c8a91c8f3c68a907c129d7e9 2024-09-15T22:43:37.667 INFO:journalctl@ceph.mds.j.smithi111.stdout:Sep 15 22:43:37 smithi111 systemd[1]: Started Ceph mds.j for b7614d3c-73b2-11ef-bceb-c7b262605968. 2024-09-15T22:43:37.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:37 smithi192 ceph-mon[35427]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:37.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:37 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:37.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:37 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:37.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:37 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:37.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:37 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:37.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:37 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:37.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:37 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:37.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:37 smithi121 ceph-mon[34815]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:37.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:37.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:37.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:37.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:37.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:37.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:38.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.121+0000 7ff79ea00640 1 -- 172.21.15.111:0/2463887337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7981049c0 msgr2=0x7ff798108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:38.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.121+0000 7ff79ea00640 1 --2- 172.21.15.111:0/2463887337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7981049c0 0x7ff798108a10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff784009900 tx=0x7ff78402a3f0 comp rx=0 tx=0).stop 2024-09-15T22:43:38.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.122+0000 7ff79ea00640 1 -- 172.21.15.111:0/2463887337 shutdown_connections 2024-09-15T22:43:38.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.122+0000 7ff79ea00640 1 --2- 172.21.15.111:0/2463887337 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff798109140 0x7ff798110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:38.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.122+0000 7ff79ea00640 1 --2- 172.21.15.111:0/2463887337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7981049c0 0x7ff798108a10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:38.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.122+0000 7ff79ea00640 1 --2- 172.21.15.111:0/2463887337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff798104010 0x7ff7981043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:38.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.122+0000 7ff79ea00640 1 -- 172.21.15.111:0/2463887337 >> 172.21.15.111:0/2463887337 conn(0x7ff7980ffa50 msgr2=0x7ff798101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:38.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.122+0000 7ff79ea00640 1 -- 172.21.15.111:0/2463887337 shutdown_connections 2024-09-15T22:43:38.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.122+0000 7ff79ea00640 1 -- 172.21.15.111:0/2463887337 wait complete. 2024-09-15T22:43:38.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.123+0000 7ff79ea00640 1 Processor -- start 2024-09-15T22:43:38.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.123+0000 7ff79ea00640 1 -- start start 2024-09-15T22:43:38.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.123+0000 7ff79ea00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff798104010 0x7ff79819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:38.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.123+0000 7ff79ea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7981049c0 0x7ff79819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:38.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.123+0000 7ff79ea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff798109140 0x7ff79819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:38.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.123+0000 7ff79ea00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff798114190 con 0x7ff7981049c0 2024-09-15T22:43:38.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.123+0000 7ff79ea00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff798114010 con 0x7ff798109140 2024-09-15T22:43:38.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.123+0000 7ff79ea00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff798114310 con 0x7ff798104010 2024-09-15T22:43:38.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.123+0000 7ff797400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7981049c0 0x7ff79819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:38.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.124+0000 7ff797e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff798104010 0x7ff79819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:38.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.124+0000 7ff797e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff798104010 0x7ff79819a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:58178/0 (socket says 172.21.15.111:58178) 2024-09-15T22:43:38.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.124+0000 7ff797e00640 1 -- 172.21.15.111:0/3992883886 learned_addr learned my addr 172.21.15.111:0/3992883886 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:43:38.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.124+0000 7ff79ca00640 1 --2- 172.21.15.111:0/3992883886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff798109140 0x7ff79819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:38.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.124+0000 7ff797e00640 1 -- 172.21.15.111:0/3992883886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff798109140 msgr2=0x7ff79819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:38.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.124+0000 7ff797e00640 1 --2- 172.21.15.111:0/3992883886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff798109140 0x7ff79819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:38.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.124+0000 7ff797e00640 1 -- 172.21.15.111:0/3992883886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7981049c0 msgr2=0x7ff79819ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:38.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.124+0000 7ff797e00640 1 --2- 172.21.15.111:0/3992883886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7981049c0 0x7ff79819ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:38.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.124+0000 7ff797e00640 1 -- 172.21.15.111:0/3992883886 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79819f6a0 con 0x7ff798104010 2024-09-15T22:43:38.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.124+0000 7ff79ca00640 1 --2- 172.21.15.111:0/3992883886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff798109140 0x7ff79819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:43:38.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.124+0000 7ff797e00640 1 --2- 172.21.15.111:0/3992883886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff798104010 0x7ff79819a6f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff78800e9f0 tx=0x7ff78800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:38.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.125+0000 7ff794c00640 1 -- 172.21.15.111:0/3992883886 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78800cd80 con 0x7ff798104010 2024-09-15T22:43:38.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.125+0000 7ff79ea00640 1 -- 172.21.15.111:0/3992883886 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff79819f990 con 0x7ff798104010 2024-09-15T22:43:38.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.125+0000 7ff794c00640 1 -- 172.21.15.111:0/3992883886 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff788010910 con 0x7ff798104010 2024-09-15T22:43:38.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.125+0000 7ff794c00640 1 -- 172.21.15.111:0/3992883886 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788018990 con 0x7ff798104010 2024-09-15T22:43:38.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.125+0000 7ff79ea00640 1 -- 172.21.15.111:0/3992883886 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7981a7270 con 0x7ff798104010 2024-09-15T22:43:38.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.126+0000 7ff79ea00640 1 -- 172.21.15.111:0/3992883886 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7981043f0 con 0x7ff798104010 2024-09-15T22:43:38.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.128+0000 7ff794c00640 1 -- 172.21.15.111:0/3992883886 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7ff788010ac0 con 0x7ff798104010 2024-09-15T22:43:38.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.129+0000 7ff794c00640 1 --2- 172.21.15.111:0/3992883886 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ff77c0a3240 0x7ff77c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:38.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.130+0000 7ff794c00640 1 -- 172.21.15.111:0/3992883886 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7ff7880cfbb0 con 0x7ff798104010 2024-09-15T22:43:38.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.130+0000 7ff797400640 1 --2- 172.21.15.111:0/3992883886 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ff77c0a3240 0x7ff77c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:38.132 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.130+0000 7ff797400640 1 --2- 172.21.15.111:0/3992883886 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ff77c0a3240 0x7ff77c0a5700 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff784000c00 tx=0x7ff784005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:38.133 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.131+0000 7ff794c00640 1 -- 172.21.15.111:0/3992883886 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff78809c680 con 0x7ff798104010 2024-09-15T22:43:38.352 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.351+0000 7ff79ea00640 1 -- 172.21.15.111:0/3992883886 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff79819ba50 con 0x7ff798104010 2024-09-15T22:43:38.353 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.351+0000 7ff794c00640 1 -- 172.21.15.111:0/3992883886 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3091 (secure 0 0 0) 0x7ff7880a0980 con 0x7ff798104010 2024-09-15T22:43:38.353 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:43:38.353 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":4,"btime":"2024-09-15T22:43:37:497079+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/453336603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":453336603},{"type":"v1","addr":"172.21.15.111:6835","nonce":453336603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/4061441359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":4061441359},{"type":"v1","addr":"172.21.15.192:6833","nonce":4061441359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/4220531578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":4220531578},{"type":"v1","addr":"172.21.15.121:6833","nonce":4220531578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-15T22:43:38.354 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 4 2024-09-15T22:43:38.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.354+0000 7ff79ea00640 1 -- 172.21.15.111:0/3992883886 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ff77c0a3240 msgr2=0x7ff77c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:38.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.354+0000 7ff79ea00640 1 --2- 172.21.15.111:0/3992883886 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ff77c0a3240 0x7ff77c0a5700 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff784000c00 tx=0x7ff784005b50 comp rx=0 tx=0).stop 2024-09-15T22:43:38.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.354+0000 7ff79ea00640 1 -- 172.21.15.111:0/3992883886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff798104010 msgr2=0x7ff79819a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:38.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.354+0000 7ff79ea00640 1 --2- 172.21.15.111:0/3992883886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff798104010 0x7ff79819a6f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff78800e9f0 tx=0x7ff78800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:43:38.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.354+0000 7ff79ea00640 1 -- 172.21.15.111:0/3992883886 shutdown_connections 2024-09-15T22:43:38.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.354+0000 7ff79ea00640 1 --2- 172.21.15.111:0/3992883886 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ff77c0a3240 0x7ff77c0a5700 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:38.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.354+0000 7ff79ea00640 1 --2- 172.21.15.111:0/3992883886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff798109140 0x7ff79819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:38.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.354+0000 7ff79ea00640 1 --2- 172.21.15.111:0/3992883886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7981049c0 0x7ff79819ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:38.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.354+0000 7ff79ea00640 1 --2- 172.21.15.111:0/3992883886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff798104010 0x7ff79819a6f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:38.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.354+0000 7ff79ea00640 1 -- 172.21.15.111:0/3992883886 >> 172.21.15.111:0/3992883886 conn(0x7ff7980ffa50 msgr2=0x7ff798100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:38.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.354+0000 7ff79ea00640 1 -- 172.21.15.111:0/3992883886 shutdown_connections 2024-09-15T22:43:38.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:38.354+0000 7ff79ea00640 1 -- 172.21.15.111:0/3992883886 wait complete. 2024-09-15T22:43:38.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:38 smithi111 ceph-mon[30629]: Deploying daemon mds.d on smithi111 2024-09-15T22:43:38.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:38 smithi111 ceph-mon[30629]: mds.? [v2:172.21.15.111:6834/453336603,v1:172.21.15.111:6835/453336603] up:boot 2024-09-15T22:43:38.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:38 smithi111 ceph-mon[30629]: fsmap 3 up:standby 2024-09-15T22:43:38.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:38 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-15T22:43:38.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:38 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/3992883886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:38.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:38 smithi192 ceph-mon[35427]: Deploying daemon mds.d on smithi111 2024-09-15T22:43:38.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:38 smithi192 ceph-mon[35427]: mds.? [v2:172.21.15.111:6834/453336603,v1:172.21.15.111:6835/453336603] up:boot 2024-09-15T22:43:38.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:38 smithi192 ceph-mon[35427]: fsmap 3 up:standby 2024-09-15T22:43:38.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:38 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-15T22:43:38.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:38 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/3992883886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:38.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:38 smithi121 ceph-mon[34815]: Deploying daemon mds.d on smithi111 2024-09-15T22:43:38.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:38 smithi121 ceph-mon[34815]: mds.? [v2:172.21.15.111:6834/453336603,v1:172.21.15.111:6835/453336603] up:boot 2024-09-15T22:43:38.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:38 smithi121 ceph-mon[34815]: fsmap 3 up:standby 2024-09-15T22:43:38.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:38 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-15T22:43:38.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:38 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/3992883886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:39.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:39 smithi192 ceph-mon[35427]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:39.852 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:39 smithi111 ceph-mon[30629]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:39.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:39 smithi121 ceph-mon[34815]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:41.166 INFO:journalctl@ceph.mds.d.smithi111.stdout:Sep 15 22:43:40 smithi111 systemd[1]: Starting Ceph mds.d for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T22:43:41.180 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:43:41.515 INFO:journalctl@ceph.mds.d.smithi111.stdout:Sep 15 22:43:41 smithi111 podman[70047]: 2024-09-15 22:43:41.080834265 +0000 UTC m=+0.026793227 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:43:41.515 INFO:journalctl@ceph.mds.d.smithi111.stdout:Sep 15 22:43:41 smithi111 podman[70047]: 2024-09-15 22:43:41.224438614 +0000 UTC m=+0.170397573 container create 5dee87e37f44e9c3e4dec436d52d3a9018a563ad02f5590e5fb504d0a05cd84e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-d, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-09-15T22:43:41.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:41 smithi192 ceph-mon[35427]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:41.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:41 smithi121 ceph-mon[34815]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:41.891 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:41.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:41 smithi111 ceph-mon[30629]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:41.916 INFO:journalctl@ceph.mds.d.smithi111.stdout:Sep 15 22:43:41 smithi111 podman[70047]: 2024-09-15 22:43:41.602543698 +0000 UTC m=+0.548502659 container init 5dee87e37f44e9c3e4dec436d52d3a9018a563ad02f5590e5fb504d0a05cd84e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-d, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-15T22:43:41.916 INFO:journalctl@ceph.mds.d.smithi111.stdout:Sep 15 22:43:41 smithi111 podman[70047]: 2024-09-15 22:43:41.609161321 +0000 UTC m=+0.555120286 container start 5dee87e37f44e9c3e4dec436d52d3a9018a563ad02f5590e5fb504d0a05cd84e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-d, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-15T22:43:41.916 INFO:journalctl@ceph.mds.d.smithi111.stdout:Sep 15 22:43:41 smithi111 ceph-mds[70095]: -- 172.21.15.111:0/1753171177 <== mon.2 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e50ac23c20 con 0x55e50ac01400 2024-09-15T22:43:41.916 INFO:journalctl@ceph.mds.d.smithi111.stdout:Sep 15 22:43:41 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-d[70080]: starting mds.d at 2024-09-15T22:43:41.916 INFO:journalctl@ceph.mds.d.smithi111.stdout:Sep 15 22:43:41 smithi111 bash[70047]: 5dee87e37f44e9c3e4dec436d52d3a9018a563ad02f5590e5fb504d0a05cd84e 2024-09-15T22:43:41.916 INFO:journalctl@ceph.mds.d.smithi111.stdout:Sep 15 22:43:41 smithi111 systemd[1]: Started Ceph mds.d for b7614d3c-73b2-11ef-bceb-c7b262605968. 2024-09-15T22:43:42.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.448+0000 7f8ae0800640 1 -- 172.21.15.111:0/1415792646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ad8109120 msgr2=0x7f8ad8110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:42.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.448+0000 7f8ae0800640 1 --2- 172.21.15.111:0/1415792646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ad8109120 0x7f8ad8110ca0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009ca0 tx=0x7f8ad402a8b0 comp rx=0 tx=0).stop 2024-09-15T22:43:42.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.448+0000 7f8ae0800640 1 -- 172.21.15.111:0/1415792646 shutdown_connections 2024-09-15T22:43:42.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.448+0000 7f8ae0800640 1 --2- 172.21.15.111:0/1415792646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ad8109120 0x7f8ad8110ca0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:42.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.448+0000 7f8ae0800640 1 --2- 172.21.15.111:0/1415792646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ad81049a0 0x7f8ad81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:42.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.448+0000 7f8ae0800640 1 --2- 172.21.15.111:0/1415792646 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ad8103ff0 0x7f8ad81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:42.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.448+0000 7f8ae0800640 1 -- 172.21.15.111:0/1415792646 >> 172.21.15.111:0/1415792646 conn(0x7f8ad80ffa90 msgr2=0x7f8ad8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:42.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.448+0000 7f8ae0800640 1 -- 172.21.15.111:0/1415792646 shutdown_connections 2024-09-15T22:43:42.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.448+0000 7f8ae0800640 1 -- 172.21.15.111:0/1415792646 wait complete. 2024-09-15T22:43:42.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.449+0000 7f8ae0800640 1 Processor -- start 2024-09-15T22:43:42.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.449+0000 7f8ae0800640 1 -- start start 2024-09-15T22:43:42.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.450+0000 7f8ae0800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ad8103ff0 0x7f8ad819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:42.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.450+0000 7f8ae0800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ad81049a0 0x7f8ad819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:42.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.450+0000 7f8ae0800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ad8109120 0x7f8ad819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:42.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.450+0000 7f8ae0800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad8114150 con 0x7f8ad81049a0 2024-09-15T22:43:42.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.450+0000 7f8ae0800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad8113fd0 con 0x7f8ad8103ff0 2024-09-15T22:43:42.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.450+0000 7f8ae0800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad81142d0 con 0x7f8ad8109120 2024-09-15T22:43:42.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.450+0000 7f8adde00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ad8103ff0 0x7f8ad819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:42.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.450+0000 7f8adde00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ad8103ff0 0x7f8ad819a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:57828/0 (socket says 172.21.15.111:57828) 2024-09-15T22:43:42.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.450+0000 7f8add400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ad81049a0 0x7f8ad819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:42.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.450+0000 7f8adde00640 1 -- 172.21.15.111:0/1902961280 learned_addr learned my addr 172.21.15.111:0/1902961280 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:43:42.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.450+0000 7f8ade800640 1 --2- 172.21.15.111:0/1902961280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ad8109120 0x7f8ad819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:42.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.450+0000 7f8adde00640 1 -- 172.21.15.111:0/1902961280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ad8109120 msgr2=0x7f8ad819ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:42.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.450+0000 7f8adde00640 1 --2- 172.21.15.111:0/1902961280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ad8109120 0x7f8ad819ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:42.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.451+0000 7f8adde00640 1 -- 172.21.15.111:0/1902961280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ad81049a0 msgr2=0x7f8ad819aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:42.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.451+0000 7f8adde00640 1 --2- 172.21.15.111:0/1902961280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ad81049a0 0x7f8ad819aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:42.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.451+0000 7f8adde00640 1 -- 172.21.15.111:0/1902961280 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad819f560 con 0x7f8ad8103ff0 2024-09-15T22:43:42.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.451+0000 7f8ade800640 1 --2- 172.21.15.111:0/1902961280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ad8109120 0x7f8ad819ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:43:42.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.451+0000 7f8add400640 1 --2- 172.21.15.111:0/1902961280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ad81049a0 0x7f8ad819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:43:42.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.451+0000 7f8adde00640 1 --2- 172.21.15.111:0/1902961280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ad8103ff0 0x7f8ad819a5b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00d950 tx=0x7f8acc00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:42.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.451+0000 7f8ad2a00640 1 -- 172.21.15.111:0/1902961280 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc017070 con 0x7f8ad8103ff0 2024-09-15T22:43:42.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.451+0000 7f8ad2a00640 1 -- 172.21.15.111:0/1902961280 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8acc009c30 con 0x7f8ad8103ff0 2024-09-15T22:43:42.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.451+0000 7f8ae0800640 1 -- 172.21.15.111:0/1902961280 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad819f850 con 0x7f8ad8103ff0 2024-09-15T22:43:42.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.451+0000 7f8ad2a00640 1 -- 172.21.15.111:0/1902961280 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc00f6b0 con 0x7f8ad8103ff0 2024-09-15T22:43:42.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.451+0000 7f8ae0800640 1 -- 172.21.15.111:0/1902961280 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad81a7130 con 0x7f8ad8103ff0 2024-09-15T22:43:42.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.452+0000 7f8ae0800640 1 -- 172.21.15.111:0/1902961280 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ad81043d0 con 0x7f8ad8103ff0 2024-09-15T22:43:42.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.454+0000 7f8ad2a00640 1 -- 172.21.15.111:0/1902961280 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f8acc0109d0 con 0x7f8ad8103ff0 2024-09-15T22:43:42.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.455+0000 7f8ad2a00640 1 --2- 172.21.15.111:0/1902961280 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8ac00a3240 0x7f8ac00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:42.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.455+0000 7f8ad2a00640 1 -- 172.21.15.111:0/1902961280 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f8acc0cf130 con 0x7f8ad8103ff0 2024-09-15T22:43:42.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.455+0000 7f8add400640 1 --2- 172.21.15.111:0/1902961280 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8ac00a3240 0x7f8ac00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:42.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.456+0000 7f8add400640 1 --2- 172.21.15.111:0/1902961280 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8ac00a3240 0x7f8ac00a5700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8ad80ffeb0 tx=0x7f8ac8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:42.459 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.457+0000 7f8ad2a00640 1 -- 172.21.15.111:0/1902961280 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8acc009de0 con 0x7f8ad8103ff0 2024-09-15T22:43:42.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.671+0000 7f8ae0800640 1 -- 172.21.15.111:0/1902961280 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ad81a73a0 con 0x7f8ad8103ff0 2024-09-15T22:43:42.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.672+0000 7f8ad2a00640 1 -- 172.21.15.111:0/1902961280 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3919 (secure 0 0 0) 0x7f8acc09c250 con 0x7f8ad8103ff0 2024-09-15T22:43:42.674 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:43:42.674 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":5,"btime":"2024-09-15T22:43:42:514153+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/453336603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":453336603},{"type":"v1","addr":"172.21.15.111:6835","nonce":453336603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/3181832287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3181832287},{"type":"v1","addr":"172.21.15.111:6837","nonce":3181832287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/4061441359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":4061441359},{"type":"v1","addr":"172.21.15.192:6833","nonce":4061441359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/4220531578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":4220531578},{"type":"v1","addr":"172.21.15.121:6833","nonce":4220531578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-15T22:43:42.674 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 5 2024-09-15T22:43:42.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.674+0000 7f8ae0800640 1 -- 172.21.15.111:0/1902961280 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8ac00a3240 msgr2=0x7f8ac00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:42.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.674+0000 7f8ae0800640 1 --2- 172.21.15.111:0/1902961280 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8ac00a3240 0x7f8ac00a5700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8ad80ffeb0 tx=0x7f8ac8005f90 comp rx=0 tx=0).stop 2024-09-15T22:43:42.676 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.674+0000 7f8ae0800640 1 -- 172.21.15.111:0/1902961280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ad8103ff0 msgr2=0x7f8ad819a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:42.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.674+0000 7f8ae0800640 1 --2- 172.21.15.111:0/1902961280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ad8103ff0 0x7f8ad819a5b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8acc00d950 tx=0x7f8acc00de10 comp rx=0 tx=0).stop 2024-09-15T22:43:42.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.675+0000 7f8ae0800640 1 -- 172.21.15.111:0/1902961280 shutdown_connections 2024-09-15T22:43:42.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.675+0000 7f8ae0800640 1 --2- 172.21.15.111:0/1902961280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ad8109120 0x7f8ad819ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:42.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.675+0000 7f8ae0800640 1 --2- 172.21.15.111:0/1902961280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ad81049a0 0x7f8ad819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:42.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.675+0000 7f8ae0800640 1 --2- 172.21.15.111:0/1902961280 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8ac00a3240 0x7f8ac00a5700 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:42.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.675+0000 7f8ae0800640 1 --2- 172.21.15.111:0/1902961280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ad8103ff0 0x7f8ad819a5b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:42.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.675+0000 7f8ae0800640 1 -- 172.21.15.111:0/1902961280 >> 172.21.15.111:0/1902961280 conn(0x7f8ad80ffa90 msgr2=0x7f8ad8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:42.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.675+0000 7f8ae0800640 1 -- 172.21.15.111:0/1902961280 shutdown_connections 2024-09-15T22:43:42.677 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:42.675+0000 7f8ae0800640 1 -- 172.21.15.111:0/1902961280 wait complete. 2024-09-15T22:43:42.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:42 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:42.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:42 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:42.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:42 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:42.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:42 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:42.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:42 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:42.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:42 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:42.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:42 smithi121 ceph-mon[34815]: Deploying daemon mds.e on smithi121 2024-09-15T22:43:42.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:42 smithi121 ceph-mon[34815]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:42.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:42 smithi121 ceph-mon[34815]: mds.? [v2:172.21.15.111:6836/3181832287,v1:172.21.15.111:6837/3181832287] up:boot 2024-09-15T22:43:42.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:42 smithi121 ceph-mon[34815]: fsmap 4 up:standby 2024-09-15T22:43:42.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:42 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-15T22:43:42.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:42 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/1902961280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:43.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:42 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:43.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:42 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:43.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:42 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:43.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:42 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:43.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:42 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:43.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:42 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:43.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:42 smithi192 ceph-mon[35427]: Deploying daemon mds.e on smithi121 2024-09-15T22:43:43.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:42 smithi192 ceph-mon[35427]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:43.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:42 smithi192 ceph-mon[35427]: mds.? [v2:172.21.15.111:6836/3181832287,v1:172.21.15.111:6837/3181832287] up:boot 2024-09-15T22:43:43.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:42 smithi192 ceph-mon[35427]: fsmap 4 up:standby 2024-09-15T22:43:43.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:42 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-15T22:43:43.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:42 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/1902961280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:43.074 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:42 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:43.074 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:42 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:43.074 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:42 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:43.074 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:42 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:43.075 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:42 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:43.075 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:42 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:43.075 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:42 smithi111 ceph-mon[30629]: Deploying daemon mds.e on smithi121 2024-09-15T22:43:43.075 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:42 smithi111 ceph-mon[30629]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:43.075 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:42 smithi111 ceph-mon[30629]: mds.? [v2:172.21.15.111:6836/3181832287,v1:172.21.15.111:6837/3181832287] up:boot 2024-09-15T22:43:43.075 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:42 smithi111 ceph-mon[30629]: fsmap 4 up:standby 2024-09-15T22:43:43.075 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:42 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-15T22:43:43.075 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:42 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/1902961280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:43.890 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 15 22:43:43 smithi121 systemd[1]: Starting Ceph mds.e for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T22:43:44.224 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 15 22:43:43 smithi121 podman[64866]: 2024-09-15 22:43:43.853395721 +0000 UTC m=+0.015631470 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:43:44.225 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 15 22:43:43 smithi121 podman[64866]: 2024-09-15 22:43:43.970311869 +0000 UTC m=+0.132547640 container create 15264d5476bacf142b2bea78971b85db09613887f648840f0979ac06107ac043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-e, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, ceph=True, org.label-schema.build-date=20240731) 2024-09-15T22:43:44.225 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 15 22:43:44 smithi121 podman[64866]: 2024-09-15 22:43:44.197543517 +0000 UTC m=+0.359779255 container init 15264d5476bacf142b2bea78971b85db09613887f648840f0979ac06107ac043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-e, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-15T22:43:44.225 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 15 22:43:44 smithi121 podman[64866]: 2024-09-15 22:43:44.200277891 +0000 UTC m=+0.362513621 container start 15264d5476bacf142b2bea78971b85db09613887f648840f0979ac06107ac043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-e, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-15T22:43:44.640 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 15 22:43:44 smithi121 ceph-mds[64885]: -- 172.21.15.121:0/204658600 <== mon.2 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55db2ca91860 con 0x55db2ca6f000 2024-09-15T22:43:44.640 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 15 22:43:44 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-e[64881]: starting mds.e at 2024-09-15T22:43:44.640 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 15 22:43:44 smithi121 bash[64866]: 15264d5476bacf142b2bea78971b85db09613887f648840f0979ac06107ac043 2024-09-15T22:43:44.640 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 15 22:43:44 smithi121 systemd[1]: Started Ceph mds.e for b7614d3c-73b2-11ef-bceb-c7b262605968. 2024-09-15T22:43:45.365 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:43:45.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:45 smithi121 ceph-mon[34815]: mds.? [v2:172.21.15.121:6834/3371796806,v1:172.21.15.121:6835/3371796806] up:boot 2024-09-15T22:43:45.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:45 smithi121 ceph-mon[34815]: fsmap 5 up:standby 2024-09-15T22:43:45.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:45 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:43:45.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:45 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:45.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:45 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:45.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:45 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:45.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:45 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:45.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:45 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:45.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:45 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:45.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:45 smithi121 ceph-mon[34815]: Deploying daemon mds.f on smithi192 2024-09-15T22:43:45.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:45 smithi121 ceph-mon[34815]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:45.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:45 smithi192 ceph-mon[35427]: mds.? [v2:172.21.15.121:6834/3371796806,v1:172.21.15.121:6835/3371796806] up:boot 2024-09-15T22:43:45.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:45 smithi192 ceph-mon[35427]: fsmap 5 up:standby 2024-09-15T22:43:45.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:45 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:43:45.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:45 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:45.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:45 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:45.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:45 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:45.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:45 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:45.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:45 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:45.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:45 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:45.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:45 smithi192 ceph-mon[35427]: Deploying daemon mds.f on smithi192 2024-09-15T22:43:45.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:45 smithi192 ceph-mon[35427]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:45.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:45 smithi111 ceph-mon[30629]: mds.? [v2:172.21.15.121:6834/3371796806,v1:172.21.15.121:6835/3371796806] up:boot 2024-09-15T22:43:45.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:45 smithi111 ceph-mon[30629]: fsmap 5 up:standby 2024-09-15T22:43:45.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:45 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:43:45.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:45 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:45.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:45 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:45.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:45 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:45.667 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:45 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:45.667 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:45 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:45.667 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:45 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:45.667 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:45 smithi111 ceph-mon[30629]: Deploying daemon mds.f on smithi192 2024-09-15T22:43:45.667 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:45 smithi111 ceph-mon[30629]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:45.738 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:46.315 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 15 22:43:46 smithi192 systemd[1]: Starting Ceph mds.f for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T22:43:46.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.321+0000 7fc38fe00640 1 -- 172.21.15.111:0/2885854043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc390076720 msgr2=0x7fc390076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:46.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.321+0000 7fc38fe00640 1 --2- 172.21.15.111:0/2885854043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc390076720 0x7fc390076b00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc380009900 tx=0x7fc38002a3f0 comp rx=0 tx=0).stop 2024-09-15T22:43:46.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.321+0000 7fc38fe00640 1 -- 172.21.15.111:0/2885854043 shutdown_connections 2024-09-15T22:43:46.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.321+0000 7fc38fe00640 1 --2- 172.21.15.111:0/2885854043 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc390112870 0x7fc390114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:46.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.321+0000 7fc38fe00640 1 --2- 172.21.15.111:0/2885854043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc390077040 0x7fc390112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:46.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.321+0000 7fc38fe00640 1 --2- 172.21.15.111:0/2885854043 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc390076720 0x7fc390076b00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:46.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.321+0000 7fc38fe00640 1 -- 172.21.15.111:0/2885854043 >> 172.21.15.111:0/2885854043 conn(0x7fc3900ff910 msgr2=0x7fc390101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:46.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.321+0000 7fc38fe00640 1 -- 172.21.15.111:0/2885854043 shutdown_connections 2024-09-15T22:43:46.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.321+0000 7fc38fe00640 1 -- 172.21.15.111:0/2885854043 wait complete. 2024-09-15T22:43:46.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.322+0000 7fc38fe00640 1 Processor -- start 2024-09-15T22:43:46.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.322+0000 7fc38fe00640 1 -- start start 2024-09-15T22:43:46.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.323+0000 7fc38fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc390076720 0x7fc3901a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:46.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.323+0000 7fc38fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc390077040 0x7fc3901a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:46.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.323+0000 7fc38fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc390112870 0x7fc39019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:46.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.323+0000 7fc38fe00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc390118430 con 0x7fc390077040 2024-09-15T22:43:46.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.323+0000 7fc38fe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc3901182b0 con 0x7fc390076720 2024-09-15T22:43:46.356 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.323+0000 7fc38fe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc3901185b0 con 0x7fc390112870 2024-09-15T22:43:46.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.323+0000 7fc38ea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc390076720 0x7fc3901a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:46.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.323+0000 7fc38e000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc390077040 0x7fc3901a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:46.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.323+0000 7fc38ea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc390076720 0x7fc3901a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:57848/0 (socket says 172.21.15.111:57848) 2024-09-15T22:43:46.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.323+0000 7fc38e000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc390077040 0x7fc3901a4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38794/0 (socket says 172.21.15.111:38794) 2024-09-15T22:43:46.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.323+0000 7fc38ea00640 1 -- 172.21.15.111:0/1299459711 learned_addr learned my addr 172.21.15.111:0/1299459711 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:43:46.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.323+0000 7fc38f400640 1 --2- 172.21.15.111:0/1299459711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc390112870 0x7fc39019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:46.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.324+0000 7fc38ea00640 1 -- 172.21.15.111:0/1299459711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc390112870 msgr2=0x7fc39019e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:46.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.324+0000 7fc38ea00640 1 --2- 172.21.15.111:0/1299459711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc390112870 0x7fc39019e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:46.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.324+0000 7fc38ea00640 1 -- 172.21.15.111:0/1299459711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc390077040 msgr2=0x7fc3901a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:46.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.324+0000 7fc38ea00640 1 --2- 172.21.15.111:0/1299459711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc390077040 0x7fc3901a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:46.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.324+0000 7fc38ea00640 1 -- 172.21.15.111:0/1299459711 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39019ef20 con 0x7fc390076720 2024-09-15T22:43:46.357 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.324+0000 7fc38e000640 1 --2- 172.21.15.111:0/1299459711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc390077040 0x7fc3901a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:43:46.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.324+0000 7fc38ea00640 1 --2- 172.21.15.111:0/1299459711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc390076720 0x7fc3901a4600 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc380004770 tx=0x7fc380003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:46.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.324+0000 7fc387400640 1 -- 172.21.15.111:0/1299459711 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38003f070 con 0x7fc390076720 2024-09-15T22:43:46.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.324+0000 7fc38fe00640 1 -- 172.21.15.111:0/1299459711 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc39019f1b0 con 0x7fc390076720 2024-09-15T22:43:46.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.324+0000 7fc387400640 1 -- 172.21.15.111:0/1299459711 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc380031ca0 con 0x7fc390076720 2024-09-15T22:43:46.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.324+0000 7fc387400640 1 -- 172.21.15.111:0/1299459711 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc380035030 con 0x7fc390076720 2024-09-15T22:43:46.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.324+0000 7fc38fe00640 1 -- 172.21.15.111:0/1299459711 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3901ab460 con 0x7fc390076720 2024-09-15T22:43:46.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.325+0000 7fc38fe00640 1 -- 172.21.15.111:0/1299459711 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc390077e60 con 0x7fc390076720 2024-09-15T22:43:46.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.327+0000 7fc387400640 1 -- 172.21.15.111:0/1299459711 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fc380042070 con 0x7fc390076720 2024-09-15T22:43:46.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.329+0000 7fc387400640 1 --2- 172.21.15.111:0/1299459711 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc3740a32c0 0x7fc3740a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:46.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.329+0000 7fc387400640 1 -- 172.21.15.111:0/1299459711 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fc3800f02e0 con 0x7fc390076720 2024-09-15T22:43:46.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.329+0000 7fc38e000640 1 --2- 172.21.15.111:0/1299459711 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc3740a32c0 0x7fc3740a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:46.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.329+0000 7fc38e000640 1 --2- 172.21.15.111:0/1299459711 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc3740a32c0 0x7fc3740a5780 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc390079370 tx=0x7fc37c00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:46.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.331+0000 7fc387400640 1 -- 172.21.15.111:0/1299459711 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc3800bd400 con 0x7fc390076720 2024-09-15T22:43:46.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.541+0000 7fc38fe00640 1 -- 172.21.15.111:0/1299459711 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc390076b00 con 0x7fc390076720 2024-09-15T22:43:46.543 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.542+0000 7fc387400640 1 -- 172.21.15.111:0/1299459711 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4747 (secure 0 0 0) 0x7fc3800bcd00 con 0x7fc390076720 2024-09-15T22:43:46.544 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:43:46.544 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":6,"btime":"2024-09-15T22:43:44:279184+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/453336603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":453336603},{"type":"v1","addr":"172.21.15.111:6835","nonce":453336603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/3181832287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3181832287},{"type":"v1","addr":"172.21.15.111:6837","nonce":3181832287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/4061441359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":4061441359},{"type":"v1","addr":"172.21.15.192:6833","nonce":4061441359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/4220531578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":4220531578},{"type":"v1","addr":"172.21.15.121:6833","nonce":4220531578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3371796806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3371796806},{"type":"v1","addr":"172.21.15.121:6835","nonce":3371796806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-15T22:43:46.544 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 6 2024-09-15T22:43:46.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.544+0000 7fc38fe00640 1 -- 172.21.15.111:0/1299459711 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc3740a32c0 msgr2=0x7fc3740a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:46.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.544+0000 7fc38fe00640 1 --2- 172.21.15.111:0/1299459711 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc3740a32c0 0x7fc3740a5780 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc390079370 tx=0x7fc37c00b410 comp rx=0 tx=0).stop 2024-09-15T22:43:46.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.544+0000 7fc38fe00640 1 -- 172.21.15.111:0/1299459711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc390076720 msgr2=0x7fc3901a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:46.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.544+0000 7fc38fe00640 1 --2- 172.21.15.111:0/1299459711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc390076720 0x7fc3901a4600 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc380004770 tx=0x7fc380003c10 comp rx=0 tx=0).stop 2024-09-15T22:43:46.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.544+0000 7fc38fe00640 1 -- 172.21.15.111:0/1299459711 shutdown_connections 2024-09-15T22:43:46.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.544+0000 7fc38fe00640 1 --2- 172.21.15.111:0/1299459711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc390112870 0x7fc39019e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:46.546 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.544+0000 7fc38fe00640 1 --2- 172.21.15.111:0/1299459711 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc390077040 0x7fc3901a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:46.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.544+0000 7fc38fe00640 1 --2- 172.21.15.111:0/1299459711 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc3740a32c0 0x7fc3740a5780 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:46.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.544+0000 7fc38fe00640 1 --2- 172.21.15.111:0/1299459711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc390076720 0x7fc3901a4600 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:46.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.544+0000 7fc38fe00640 1 -- 172.21.15.111:0/1299459711 >> 172.21.15.111:0/1299459711 conn(0x7fc3900ff910 msgr2=0x7fc390101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:46.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.545+0000 7fc38fe00640 1 -- 172.21.15.111:0/1299459711 shutdown_connections 2024-09-15T22:43:46.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:46.545+0000 7fc38fe00640 1 -- 172.21.15.111:0/1299459711 wait complete. 2024-09-15T22:43:46.637 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 15 22:43:46 smithi192 podman[65528]: 2024-09-15 22:43:46.223435915 +0000 UTC m=+0.015437519 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:43:46.638 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 15 22:43:46 smithi192 podman[65528]: 2024-09-15 22:43:46.418070965 +0000 UTC m=+0.210072571 container create 3d5ca0b95ccd4de60f7cb3b661ae6e13504bf23d3ccf0a62f6673f1fbfb2fcb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-f, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-09-15T22:43:46.638 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 15 22:43:46 smithi192 podman[65528]: 2024-09-15 22:43:46.637160936 +0000 UTC m=+0.429162549 container init 3d5ca0b95ccd4de60f7cb3b661ae6e13504bf23d3ccf0a62f6673f1fbfb2fcb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-f, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e) 2024-09-15T22:43:47.064 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 15 22:43:46 smithi192 podman[65528]: 2024-09-15 22:43:46.640027527 +0000 UTC m=+0.432029131 container start 3d5ca0b95ccd4de60f7cb3b661ae6e13504bf23d3ccf0a62f6673f1fbfb2fcb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-15T22:43:47.065 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 15 22:43:46 smithi192 ceph-mds[65547]: -- 172.21.15.192:0/2050052264 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d2b25a3860 con 0x55d2b2580800 2024-09-15T22:43:47.065 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 15 22:43:46 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-f[65543]: starting mds.f at 2024-09-15T22:43:47.065 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 15 22:43:46 smithi192 bash[65528]: 3d5ca0b95ccd4de60f7cb3b661ae6e13504bf23d3ccf0a62f6673f1fbfb2fcb1 2024-09-15T22:43:47.065 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 15 22:43:46 smithi192 systemd[1]: Started Ceph mds.f for b7614d3c-73b2-11ef-bceb-c7b262605968. 2024-09-15T22:43:47.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:47 smithi192 ceph-mon[35427]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:47.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:47 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/1299459711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:47.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:47 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:47.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:47 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:47.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:47 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:47.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:47 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:47.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:47 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:47.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:47 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:47.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:47 smithi121 ceph-mon[34815]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:47.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:47 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/1299459711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:47.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:47 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:47.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:47 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:47.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:47 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:47.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:47 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:47.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:47 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:47.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:47 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:47.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:47 smithi111 ceph-mon[30629]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:47.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:47 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/1299459711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:47.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:47 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:47.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:47 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:47.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:47 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:47.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:47 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:47.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:47 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:47.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:47 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:48.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:48 smithi192 ceph-mon[35427]: Deploying daemon mds.c on smithi192 2024-09-15T22:43:48.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:48 smithi192 ceph-mon[35427]: mds.? [v2:172.21.15.192:6834/4080914921,v1:172.21.15.192:6835/4080914921] up:boot 2024-09-15T22:43:48.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:48 smithi192 ceph-mon[35427]: fsmap 6 up:standby 2024-09-15T22:43:48.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:48 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:43:48.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:48 smithi111 ceph-mon[30629]: Deploying daemon mds.c on smithi192 2024-09-15T22:43:48.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:48 smithi111 ceph-mon[30629]: mds.? [v2:172.21.15.192:6834/4080914921,v1:172.21.15.192:6835/4080914921] up:boot 2024-09-15T22:43:48.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:48 smithi111 ceph-mon[30629]: fsmap 6 up:standby 2024-09-15T22:43:48.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:48 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:43:48.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:48 smithi121 ceph-mon[34815]: Deploying daemon mds.c on smithi192 2024-09-15T22:43:48.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:48 smithi121 ceph-mon[34815]: mds.? [v2:172.21.15.192:6834/4080914921,v1:172.21.15.192:6835/4080914921] up:boot 2024-09-15T22:43:48.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:48 smithi121 ceph-mon[34815]: fsmap 6 up:standby 2024-09-15T22:43:48.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:48 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:43:49.013 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 15 22:43:48 smithi192 systemd[1]: Starting Ceph mds.c for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T22:43:49.287 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:43:49.292 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 15 22:43:49 smithi192 podman[65828]: 2024-09-15 22:43:48.915284684 +0000 UTC m=+0.015418290 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:43:49.292 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 15 22:43:49 smithi192 podman[65828]: 2024-09-15 22:43:49.04757983 +0000 UTC m=+0.147713427 container create 6f184d05796872f063d8b61a90784fa46aaa10ac74ed8cd3ab20b33ae56d8368 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-09-15T22:43:49.292 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 15 22:43:49 smithi192 podman[65828]: 2024-09-15 22:43:49.291726814 +0000 UTC m=+0.391860419 container init 6f184d05796872f063d8b61a90784fa46aaa10ac74ed8cd3ab20b33ae56d8368 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-c, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-15T22:43:49.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:49 smithi192 ceph-mon[35427]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:49.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:49 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:49.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:49 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:49.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:49 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:49.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:49 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:49.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:49 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:49.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:49 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:49.567 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 15 22:43:49 smithi192 podman[65828]: 2024-09-15 22:43:49.294889121 +0000 UTC m=+0.395022720 container start 6f184d05796872f063d8b61a90784fa46aaa10ac74ed8cd3ab20b33ae56d8368 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-c, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-15T22:43:49.567 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 15 22:43:49 smithi192 ceph-mds[65847]: -- 172.21.15.192:0/2859249027 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55900ddb9860 con 0x55900dd97000 2024-09-15T22:43:49.567 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 15 22:43:49 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-c[65843]: starting mds.c at 2024-09-15T22:43:49.567 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 15 22:43:49 smithi192 bash[65828]: 6f184d05796872f063d8b61a90784fa46aaa10ac74ed8cd3ab20b33ae56d8368 2024-09-15T22:43:49.567 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 15 22:43:49 smithi192 systemd[1]: Started Ceph mds.c for b7614d3c-73b2-11ef-bceb-c7b262605968. 2024-09-15T22:43:49.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:49 smithi111 ceph-mon[30629]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:49.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:49 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:49.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:49 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:49.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:49 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:49.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:49 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:49.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:49 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:49.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:49 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:49.677 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:49.889 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:49 smithi121 ceph-mon[34815]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:49.889 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:49 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:49.889 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:49 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:49.889 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:49 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:49.889 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:49 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:49.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:49 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:49.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:49 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:50.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.305+0000 7fc4faa00640 1 -- 172.21.15.111:0/4122595098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4f4101de0 msgr2=0x7fc4f41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:50.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.305+0000 7fc4faa00640 1 --2- 172.21.15.111:0/4122595098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4f4101de0 0x7fc4f41021c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009ca0 tx=0x7fc4e402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:43:50.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.306+0000 7fc4faa00640 1 -- 172.21.15.111:0/4122595098 shutdown_connections 2024-09-15T22:43:50.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.306+0000 7fc4faa00640 1 --2- 172.21.15.111:0/4122595098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4f4106f10 0x7fc4f410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:50.308 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.306+0000 7fc4faa00640 1 --2- 172.21.15.111:0/4122595098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4f4102790 0x7fc4f41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:50.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.306+0000 7fc4faa00640 1 --2- 172.21.15.111:0/4122595098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4f4101de0 0x7fc4f41021c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:50.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.306+0000 7fc4faa00640 1 -- 172.21.15.111:0/4122595098 >> 172.21.15.111:0/4122595098 conn(0x7fc4f40fd8c0 msgr2=0x7fc4f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:50.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.306+0000 7fc4faa00640 1 -- 172.21.15.111:0/4122595098 shutdown_connections 2024-09-15T22:43:50.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.306+0000 7fc4faa00640 1 -- 172.21.15.111:0/4122595098 wait complete. 2024-09-15T22:43:50.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.307+0000 7fc4faa00640 1 Processor -- start 2024-09-15T22:43:50.309 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.307+0000 7fc4faa00640 1 -- start start 2024-09-15T22:43:50.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.307+0000 7fc4faa00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4f4101de0 0x7fc4f4198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:50.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.307+0000 7fc4faa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4f4102790 0x7fc4f4198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:50.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.307+0000 7fc4faa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4f4106f10 0x7fc4f419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:50.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.307+0000 7fc4faa00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f4111fc0 con 0x7fc4f4102790 2024-09-15T22:43:50.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.307+0000 7fc4faa00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f4111e40 con 0x7fc4f4106f10 2024-09-15T22:43:50.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.307+0000 7fc4faa00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f4112140 con 0x7fc4f4101de0 2024-09-15T22:43:50.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.307+0000 7fc4f8a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4f4106f10 0x7fc4f419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:50.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.307+0000 7fc4f3400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4f4102790 0x7fc4f4198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:50.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.308+0000 7fc4f8a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4f4106f10 0x7fc4f419cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:57862/0 (socket says 172.21.15.111:57862) 2024-09-15T22:43:50.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.308+0000 7fc4f8a00640 1 -- 172.21.15.111:0/2607623749 learned_addr learned my addr 172.21.15.111:0/2607623749 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:43:50.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.308+0000 7fc4f3e00640 1 --2- 172.21.15.111:0/2607623749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4f4101de0 0x7fc4f4198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:50.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.308+0000 7fc4f8a00640 1 -- 172.21.15.111:0/2607623749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4f4101de0 msgr2=0x7fc4f4198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:50.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.308+0000 7fc4f8a00640 1 --2- 172.21.15.111:0/2607623749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4f4101de0 0x7fc4f4198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:50.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.308+0000 7fc4f8a00640 1 -- 172.21.15.111:0/2607623749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4f4102790 msgr2=0x7fc4f4198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:50.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.308+0000 7fc4f8a00640 1 --2- 172.21.15.111:0/2607623749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4f4102790 0x7fc4f4198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:50.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.308+0000 7fc4f8a00640 1 -- 172.21.15.111:0/2607623749 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f419d3f0 con 0x7fc4f4106f10 2024-09-15T22:43:50.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.308+0000 7fc4f8a00640 1 --2- 172.21.15.111:0/2607623749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4f4106f10 0x7fc4f419cd10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec00e970 tx=0x7fc4ec00ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:50.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.308+0000 7fc4f0c00640 1 -- 172.21.15.111:0/2607623749 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec00ccc0 con 0x7fc4f4106f10 2024-09-15T22:43:50.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.308+0000 7fc4f0c00640 1 -- 172.21.15.111:0/2607623749 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4ec00ce60 con 0x7fc4f4106f10 2024-09-15T22:43:50.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.308+0000 7fc4f0c00640 1 -- 172.21.15.111:0/2607623749 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec010490 con 0x7fc4f4106f10 2024-09-15T22:43:50.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.308+0000 7fc4faa00640 1 -- 172.21.15.111:0/2607623749 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f419d6e0 con 0x7fc4f4106f10 2024-09-15T22:43:50.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.309+0000 7fc4faa00640 1 -- 172.21.15.111:0/2607623749 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f41a4fc0 con 0x7fc4f4106f10 2024-09-15T22:43:50.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.310+0000 7fc4faa00640 1 -- 172.21.15.111:0/2607623749 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f41021c0 con 0x7fc4f4106f10 2024-09-15T22:43:50.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.311+0000 7fc4f0c00640 1 -- 172.21.15.111:0/2607623749 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fc4ec005020 con 0x7fc4f4106f10 2024-09-15T22:43:50.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.313+0000 7fc4f0c00640 1 --2- 172.21.15.111:0/2607623749 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc4d80a32c0 0x7fc4d80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:50.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.313+0000 7fc4f0c00640 1 -- 172.21.15.111:0/2607623749 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fc4ec0cf7f0 con 0x7fc4f4106f10 2024-09-15T22:43:50.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.313+0000 7fc4f3e00640 1 --2- 172.21.15.111:0/2607623749 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc4d80a32c0 0x7fc4d80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:50.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.313+0000 7fc4f3e00640 1 --2- 172.21.15.111:0/2607623749 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc4d80a32c0 0x7fc4d80a5780 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4006010 tx=0x7fc4e4009820 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:50.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.317+0000 7fc4f0c00640 1 -- 172.21.15.111:0/2607623749 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc4ec09c910 con 0x7fc4f4106f10 2024-09-15T22:43:50.533 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.532+0000 7fc4faa00640 1 -- 172.21.15.111:0/2607623749 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4f402d7c0 con 0x7fc4f4106f10 2024-09-15T22:43:50.534 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.533+0000 7fc4f0c00640 1 -- 172.21.15.111:0/2607623749 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6403 (secure 0 0 0) 0x7fc4ec09c210 con 0x7fc4f4106f10 2024-09-15T22:43:50.534 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:43:50.534 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":8,"btime":"2024-09-15T22:43:49:511865+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/453336603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":453336603},{"type":"v1","addr":"172.21.15.111:6835","nonce":453336603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/3181832287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3181832287},{"type":"v1","addr":"172.21.15.111:6837","nonce":3181832287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/4061441359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":4061441359},{"type":"v1","addr":"172.21.15.192:6833","nonce":4061441359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/4220531578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":4220531578},{"type":"v1","addr":"172.21.15.121:6833","nonce":4220531578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3371796806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3371796806},{"type":"v1","addr":"172.21.15.121:6835","nonce":3371796806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/4080914921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":4080914921},{"type":"v1","addr":"172.21.15.192:6835","nonce":4080914921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3995501707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3995501707},{"type":"v1","addr":"172.21.15.192:6837","nonce":3995501707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-15T22:43:50.535 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 8 2024-09-15T22:43:50.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.535+0000 7fc4faa00640 1 -- 172.21.15.111:0/2607623749 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc4d80a32c0 msgr2=0x7fc4d80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:50.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.535+0000 7fc4faa00640 1 --2- 172.21.15.111:0/2607623749 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc4d80a32c0 0x7fc4d80a5780 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4006010 tx=0x7fc4e4009820 comp rx=0 tx=0).stop 2024-09-15T22:43:50.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.535+0000 7fc4faa00640 1 -- 172.21.15.111:0/2607623749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4f4106f10 msgr2=0x7fc4f419cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:50.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.535+0000 7fc4faa00640 1 --2- 172.21.15.111:0/2607623749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4f4106f10 0x7fc4f419cd10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec00e970 tx=0x7fc4ec00ed70 comp rx=0 tx=0).stop 2024-09-15T22:43:50.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.535+0000 7fc4faa00640 1 -- 172.21.15.111:0/2607623749 shutdown_connections 2024-09-15T22:43:50.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.535+0000 7fc4faa00640 1 --2- 172.21.15.111:0/2607623749 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc4d80a32c0 0x7fc4d80a5780 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:50.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.535+0000 7fc4faa00640 1 --2- 172.21.15.111:0/2607623749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4f4106f10 0x7fc4f419cd10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:50.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.535+0000 7fc4faa00640 1 --2- 172.21.15.111:0/2607623749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4f4102790 0x7fc4f4198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:50.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.535+0000 7fc4faa00640 1 --2- 172.21.15.111:0/2607623749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4f4101de0 0x7fc4f4198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:50.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.535+0000 7fc4faa00640 1 -- 172.21.15.111:0/2607623749 >> 172.21.15.111:0/2607623749 conn(0x7fc4f40fd8c0 msgr2=0x7fc4f40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:50.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.535+0000 7fc4faa00640 1 -- 172.21.15.111:0/2607623749 shutdown_connections 2024-09-15T22:43:50.538 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:50.535+0000 7fc4faa00640 1 -- 172.21.15.111:0/2607623749 wait complete. 2024-09-15T22:43:50.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:50 smithi192 ceph-mon[35427]: Deploying daemon mds.h on smithi121 2024-09-15T22:43:50.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:50 smithi192 ceph-mon[35427]: mds.? [v2:172.21.15.192:6836/3995501707,v1:172.21.15.192:6837/3995501707] up:boot 2024-09-15T22:43:50.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:50 smithi192 ceph-mon[35427]: fsmap 7 up:standby 2024-09-15T22:43:50.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:43:50.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:50 smithi121 ceph-mon[34815]: Deploying daemon mds.h on smithi121 2024-09-15T22:43:50.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:50 smithi121 ceph-mon[34815]: mds.? [v2:172.21.15.192:6836/3995501707,v1:172.21.15.192:6837/3995501707] up:boot 2024-09-15T22:43:50.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:50 smithi121 ceph-mon[34815]: fsmap 7 up:standby 2024-09-15T22:43:50.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:43:50.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:50 smithi111 ceph-mon[30629]: Deploying daemon mds.h on smithi121 2024-09-15T22:43:50.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:50 smithi111 ceph-mon[30629]: mds.? [v2:172.21.15.192:6836/3995501707,v1:172.21.15.192:6837/3995501707] up:boot 2024-09-15T22:43:50.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:50 smithi111 ceph-mon[30629]: fsmap 7 up:standby 2024-09-15T22:43:50.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:43:51.527 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 15 22:43:51 smithi121 systemd[1]: Starting Ceph mds.h for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T22:43:51.788 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:51 smithi121 ceph-mon[34815]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:51.788 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:51 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/2607623749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:51.788 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 15 22:43:51 smithi121 podman[65166]: 2024-09-15 22:43:51.547629933 +0000 UTC m=+0.015756862 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:43:51.788 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 15 22:43:51 smithi121 podman[65166]: 2024-09-15 22:43:51.656545395 +0000 UTC m=+0.124672358 container create c64e8fa330a3b44ad3cc7f2e63a36e1c896fafe5d33ec7afb2631c715c91b2ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-15T22:43:51.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:51 smithi192 ceph-mon[35427]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:51.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:51 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/2607623749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:51.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:51 smithi111 ceph-mon[30629]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:51.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:51 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/2607623749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:52.390 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 15 22:43:51 smithi121 podman[65166]: 2024-09-15 22:43:51.917603325 +0000 UTC m=+0.385730246 container init c64e8fa330a3b44ad3cc7f2e63a36e1c896fafe5d33ec7afb2631c715c91b2ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-15T22:43:52.391 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 15 22:43:51 smithi121 podman[65166]: 2024-09-15 22:43:51.920392374 +0000 UTC m=+0.388519300 container start c64e8fa330a3b44ad3cc7f2e63a36e1c896fafe5d33ec7afb2631c715c91b2ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-h, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, ceph=True, GIT_BRANCH=HEAD) 2024-09-15T22:43:52.391 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 15 22:43:51 smithi121 ceph-mds[65185]: -- 172.21.15.121:0/299423822 <== mon.2 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5633557ad860 con 0x56335578b400 2024-09-15T22:43:52.391 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 15 22:43:51 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-h[65181]: starting mds.h at 2024-09-15T22:43:52.391 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 15 22:43:52 smithi121 bash[65166]: c64e8fa330a3b44ad3cc7f2e63a36e1c896fafe5d33ec7afb2631c715c91b2ca 2024-09-15T22:43:52.391 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 15 22:43:52 smithi121 systemd[1]: Started Ceph mds.h for b7614d3c-73b2-11ef-bceb-c7b262605968. 2024-09-15T22:43:53.286 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:43:53.342 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:53 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:53.342 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:53 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:53.342 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:53 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:53.342 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:53 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:53.342 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:53 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:53.342 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:53 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:53.342 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:53 smithi111 ceph-mon[30629]: Deploying daemon mds.i on smithi192 2024-09-15T22:43:53.342 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:53 smithi111 ceph-mon[30629]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:53.342 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:53 smithi111 ceph-mon[30629]: mds.? [v2:172.21.15.121:6836/1443772791,v1:172.21.15.121:6837/1443772791] up:boot 2024-09-15T22:43:53.342 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:53 smithi111 ceph-mon[30629]: fsmap 8 up:standby 2024-09-15T22:43:53.343 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:53 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:43:53.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:53 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:53.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:53 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:53.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:53 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:53.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:53 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:53.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:53 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:53.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:53 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:53.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:53 smithi121 ceph-mon[34815]: Deploying daemon mds.i on smithi192 2024-09-15T22:43:53.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:53 smithi121 ceph-mon[34815]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:53.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:53 smithi121 ceph-mon[34815]: mds.? [v2:172.21.15.121:6836/1443772791,v1:172.21.15.121:6837/1443772791] up:boot 2024-09-15T22:43:53.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:53 smithi121 ceph-mon[34815]: fsmap 8 up:standby 2024-09-15T22:43:53.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:53 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:43:53.529 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:53 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:53.530 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:53 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:53.530 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:53 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:53.530 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:53 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:53.530 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:53 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:53.530 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:53 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:53.530 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:53 smithi192 ceph-mon[35427]: Deploying daemon mds.i on smithi192 2024-09-15T22:43:53.530 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:53 smithi192 ceph-mon[35427]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:53.530 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:53 smithi192 ceph-mon[35427]: mds.? [v2:172.21.15.121:6836/1443772791,v1:172.21.15.121:6837/1443772791] up:boot 2024-09-15T22:43:53.530 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:53 smithi192 ceph-mon[35427]: fsmap 8 up:standby 2024-09-15T22:43:53.530 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:53 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:43:53.660 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:54.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.210+0000 7f482ac00640 1 -- 172.21.15.111:0/119335662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48241034b0 msgr2=0x7f482410b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:54.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.210+0000 7f482ac00640 1 --2- 172.21.15.111:0/119335662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48241034b0 0x7f482410b9a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4810009900 tx=0x7f481002a480 comp rx=0 tx=0).stop 2024-09-15T22:43:54.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.210+0000 7f482ac00640 1 -- 172.21.15.111:0/119335662 shutdown_connections 2024-09-15T22:43:54.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.210+0000 7f482ac00640 1 --2- 172.21.15.111:0/119335662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f482410bfc0 0x7f482410e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.210+0000 7f482ac00640 1 --2- 172.21.15.111:0/119335662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48241034b0 0x7f482410b9a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.210+0000 7f482ac00640 1 --2- 172.21.15.111:0/119335662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4824102b90 0x7f4824102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.210+0000 7f482ac00640 1 -- 172.21.15.111:0/119335662 >> 172.21.15.111:0/119335662 conn(0x7f48240fc610 msgr2=0x7f48240fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:54.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.210+0000 7f482ac00640 1 -- 172.21.15.111:0/119335662 shutdown_connections 2024-09-15T22:43:54.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.210+0000 7f482ac00640 1 -- 172.21.15.111:0/119335662 wait complete. 2024-09-15T22:43:54.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.211+0000 7f482ac00640 1 Processor -- start 2024-09-15T22:43:54.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.211+0000 7f482ac00640 1 -- start start 2024-09-15T22:43:54.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.212+0000 7f482ac00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4824102b90 0x7f48241007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:54.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.212+0000 7f482ac00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48241034b0 0x7f4824100d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:54.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.212+0000 7f482ac00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f482410bfc0 0x7f4824101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:54.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.212+0000 7f482ac00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4824111dc0 con 0x7f48241034b0 2024-09-15T22:43:54.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.212+0000 7f482ac00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4824111c40 con 0x7f4824102b90 2024-09-15T22:43:54.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.212+0000 7f482ac00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4824111f40 con 0x7f482410bfc0 2024-09-15T22:43:54.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.212+0000 7f4828e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48241034b0 0x7f4824100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:54.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.212+0000 7f4828e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48241034b0 0x7f4824100d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41068/0 (socket says 172.21.15.111:41068) 2024-09-15T22:43:54.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.212+0000 7f4828e00640 1 -- 172.21.15.111:0/49841034 learned_addr learned my addr 172.21.15.111:0/49841034 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:43:54.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.212+0000 7f4829800640 1 --2- 172.21.15.111:0/49841034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4824102b90 0x7f48241007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:54.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.212+0000 7f482a200640 1 --2- 172.21.15.111:0/49841034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f482410bfc0 0x7f4824101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:54.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.213+0000 7f4828e00640 1 -- 172.21.15.111:0/49841034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f482410bfc0 msgr2=0x7f4824101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:54.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.213+0000 7f4828e00640 1 --2- 172.21.15.111:0/49841034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f482410bfc0 0x7f4824101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.213+0000 7f4828e00640 1 -- 172.21.15.111:0/49841034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4824102b90 msgr2=0x7f48241007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:54.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.213+0000 7f4828e00640 1 --2- 172.21.15.111:0/49841034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4824102b90 0x7f48241007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.213+0000 7f4828e00640 1 -- 172.21.15.111:0/49841034 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48241a6760 con 0x7f48241034b0 2024-09-15T22:43:54.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.213+0000 7f4829800640 1 --2- 172.21.15.111:0/49841034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4824102b90 0x7f48241007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:43:54.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.213+0000 7f4828e00640 1 --2- 172.21.15.111:0/49841034 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48241034b0 0x7f4824100d30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4810006010 tx=0x7f4810004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:54.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.213+0000 7f4822000640 1 -- 172.21.15.111:0/49841034 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481003f030 con 0x7f48241034b0 2024-09-15T22:43:54.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.213+0000 7f4822000640 1 -- 172.21.15.111:0/49841034 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4810004440 con 0x7f48241034b0 2024-09-15T22:43:54.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.213+0000 7f4822000640 1 -- 172.21.15.111:0/49841034 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4810035070 con 0x7f48241034b0 2024-09-15T22:43:54.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.213+0000 7f482ac00640 1 -- 172.21.15.111:0/49841034 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48241a69f0 con 0x7f48241034b0 2024-09-15T22:43:54.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.213+0000 7f482ac00640 1 -- 172.21.15.111:0/49841034 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48241a6f20 con 0x7f48241034b0 2024-09-15T22:43:54.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.215+0000 7f482ac00640 1 -- 172.21.15.111:0/49841034 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4824102f70 con 0x7f48241034b0 2024-09-15T22:43:54.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.216+0000 7f4822000640 1 -- 172.21.15.111:0/49841034 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f4810042030 con 0x7f48241034b0 2024-09-15T22:43:54.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.217+0000 7f4822000640 1 --2- 172.21.15.111:0/49841034 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f48080a32c0 0x7f48080a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:54.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.218+0000 7f4822000640 1 -- 172.21.15.111:0/49841034 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f48100ef7e0 con 0x7f48241034b0 2024-09-15T22:43:54.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.218+0000 7f4829800640 1 --2- 172.21.15.111:0/49841034 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f48080a32c0 0x7f48080a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:54.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.218+0000 7f4829800640 1 --2- 172.21.15.111:0/49841034 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f48080a32c0 0x7f48080a5780 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4814009fd0 tx=0x7f4814009760 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:54.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.222+0000 7f4822000640 1 -- 172.21.15.111:0/49841034 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f48100bc2b0 con 0x7f48241034b0 2024-09-15T22:43:54.249 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 15 22:43:54 smithi192 systemd[1]: Starting Ceph mds.i for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T22:43:54.435 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.434+0000 7f482ac00640 1 -- 172.21.15.111:0/49841034 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4824107030 con 0x7f48241034b0 2024-09-15T22:43:54.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.434+0000 7f4822000640 1 -- 172.21.15.111:0/49841034 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7231 (secure 0 0 0) 0x7f48100b6e20 con 0x7f48241034b0 2024-09-15T22:43:54.436 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:43:54.436 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":9,"btime":"2024-09-15T22:43:52:525769+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/453336603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":453336603},{"type":"v1","addr":"172.21.15.111:6835","nonce":453336603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/3181832287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3181832287},{"type":"v1","addr":"172.21.15.111:6837","nonce":3181832287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/4061441359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":4061441359},{"type":"v1","addr":"172.21.15.192:6833","nonce":4061441359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/4220531578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":4220531578},{"type":"v1","addr":"172.21.15.121:6833","nonce":4220531578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3371796806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3371796806},{"type":"v1","addr":"172.21.15.121:6835","nonce":3371796806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/4080914921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":4080914921},{"type":"v1","addr":"172.21.15.192:6835","nonce":4080914921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3995501707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3995501707},{"type":"v1","addr":"172.21.15.192:6837","nonce":3995501707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1443772791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1443772791},{"type":"v1","addr":"172.21.15.121:6837","nonce":1443772791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-15T22:43:54.436 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 9 2024-09-15T22:43:54.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.437+0000 7f482ac00640 1 -- 172.21.15.111:0/49841034 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f48080a32c0 msgr2=0x7f48080a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:54.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.437+0000 7f482ac00640 1 --2- 172.21.15.111:0/49841034 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f48080a32c0 0x7f48080a5780 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4814009fd0 tx=0x7f4814009760 comp rx=0 tx=0).stop 2024-09-15T22:43:54.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.437+0000 7f482ac00640 1 -- 172.21.15.111:0/49841034 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48241034b0 msgr2=0x7f4824100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:54.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.437+0000 7f482ac00640 1 --2- 172.21.15.111:0/49841034 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48241034b0 0x7f4824100d30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4810006010 tx=0x7f4810004040 comp rx=0 tx=0).stop 2024-09-15T22:43:54.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.437+0000 7f482ac00640 1 -- 172.21.15.111:0/49841034 shutdown_connections 2024-09-15T22:43:54.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.437+0000 7f482ac00640 1 --2- 172.21.15.111:0/49841034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f482410bfc0 0x7f4824101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.437+0000 7f482ac00640 1 --2- 172.21.15.111:0/49841034 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48241034b0 0x7f4824100d30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.437+0000 7f482ac00640 1 --2- 172.21.15.111:0/49841034 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f48080a32c0 0x7f48080a5780 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.437+0000 7f482ac00640 1 --2- 172.21.15.111:0/49841034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4824102b90 0x7f48241007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:54.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.437+0000 7f482ac00640 1 -- 172.21.15.111:0/49841034 >> 172.21.15.111:0/49841034 conn(0x7f48240fc610 msgr2=0x7f48240fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:54.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.437+0000 7f482ac00640 1 -- 172.21.15.111:0/49841034 shutdown_connections 2024-09-15T22:43:54.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:54.437+0000 7f482ac00640 1 -- 172.21.15.111:0/49841034 wait complete. 2024-09-15T22:43:54.541 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 15 22:43:54 smithi192 podman[66130]: 2024-09-15 22:43:54.151076669 +0000 UTC m=+0.016173265 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:43:54.541 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 15 22:43:54 smithi192 podman[66130]: 2024-09-15 22:43:54.254234863 +0000 UTC m=+0.119331460 container create 6327a8c10c2d1ff4eb78e5f5766a4bd0bc7c499b82d29d4df39bf8c4c64ff044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-i, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True) 2024-09-15T22:43:54.541 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 15 22:43:54 smithi192 podman[66130]: 2024-09-15 22:43:54.540779368 +0000 UTC m=+0.405875964 container init 6327a8c10c2d1ff4eb78e5f5766a4bd0bc7c499b82d29d4df39bf8c4c64ff044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-i, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:43:54.815 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 15 22:43:54 smithi192 podman[66130]: 2024-09-15 22:43:54.543627358 +0000 UTC m=+0.408723958 container start 6327a8c10c2d1ff4eb78e5f5766a4bd0bc7c499b82d29d4df39bf8c4c64ff044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-i, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-15T22:43:54.815 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 15 22:43:54 smithi192 ceph-mds[66149]: -- 172.21.15.192:0/1260514999 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55faff835860 con 0x55faff813400 2024-09-15T22:43:54.815 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 15 22:43:54 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-i[66145]: starting mds.i at 2024-09-15T22:43:54.815 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 15 22:43:54 smithi192 bash[66130]: 6327a8c10c2d1ff4eb78e5f5766a4bd0bc7c499b82d29d4df39bf8c4c64ff044 2024-09-15T22:43:54.815 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 15 22:43:54 smithi192 systemd[1]: Started Ceph mds.i for b7614d3c-73b2-11ef-bceb-c7b262605968. 2024-09-15T22:43:55.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:55 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/49841034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:55.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:55 smithi111 ceph-mon[30629]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:55.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:55 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:55.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:55 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:55.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:55 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:55.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:55 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:55.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:55 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:55.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:55 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:55.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:55 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/49841034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:55.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:55 smithi192 ceph-mon[35427]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:55.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:55 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:55.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:55 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:55.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:55 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:55.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:55 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:55.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:55 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:55.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:55 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:55.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:55 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/49841034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:55.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:55 smithi121 ceph-mon[34815]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:55.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:55 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:55.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:55 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:55.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:55 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:55.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:55 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:55.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:55 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:55.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:55 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:56.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:56 smithi192 ceph-mon[35427]: Deploying daemon mds.g on smithi111 2024-09-15T22:43:56.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:56 smithi192 ceph-mon[35427]: mds.? [v2:172.21.15.192:6838/800136261,v1:172.21.15.192:6839/800136261] up:boot 2024-09-15T22:43:56.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:56 smithi192 ceph-mon[35427]: fsmap 9 up:standby 2024-09-15T22:43:56.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:56 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:43:56.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:56 smithi121 ceph-mon[34815]: Deploying daemon mds.g on smithi111 2024-09-15T22:43:56.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:56 smithi121 ceph-mon[34815]: mds.? [v2:172.21.15.192:6838/800136261,v1:172.21.15.192:6839/800136261] up:boot 2024-09-15T22:43:56.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:56 smithi121 ceph-mon[34815]: fsmap 9 up:standby 2024-09-15T22:43:56.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:56 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:43:56.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:56 smithi111 ceph-mon[30629]: Deploying daemon mds.g on smithi111 2024-09-15T22:43:56.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:56 smithi111 ceph-mon[30629]: mds.? [v2:172.21.15.192:6838/800136261,v1:172.21.15.192:6839/800136261] up:boot 2024-09-15T22:43:56.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:56 smithi111 ceph-mon[30629]: fsmap 9 up:standby 2024-09-15T22:43:56.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:56 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:43:57.313 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:43:57.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:57 smithi192 ceph-mon[35427]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:57.364 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:57 smithi111 ceph-mon[30629]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:57.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:57 smithi121 ceph-mon[34815]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:57.714 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:43:58.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.327+0000 7f4caa600640 1 -- 172.21.15.111:0/2403966198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ca40745c0 msgr2=0x7f4ca40749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:58.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.327+0000 7f4caa600640 1 --2- 172.21.15.111:0/2403966198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ca40745c0 0x7f4ca40749a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4c94009900 tx=0x7f4c9402a480 comp rx=0 tx=0).stop 2024-09-15T22:43:58.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.327+0000 7f4caa600640 1 -- 172.21.15.111:0/2403966198 shutdown_connections 2024-09-15T22:43:58.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.327+0000 7f4caa600640 1 --2- 172.21.15.111:0/2403966198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ca410e420 0x7f4ca4110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.327+0000 7f4caa600640 1 --2- 172.21.15.111:0/2403966198 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ca4074ee0 0x7f4ca410dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.327+0000 7f4caa600640 1 --2- 172.21.15.111:0/2403966198 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ca40745c0 0x7f4ca40749a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.327+0000 7f4caa600640 1 -- 172.21.15.111:0/2403966198 >> 172.21.15.111:0/2403966198 conn(0x7f4ca40fd8c0 msgr2=0x7f4ca40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:58.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.327+0000 7f4caa600640 1 -- 172.21.15.111:0/2403966198 shutdown_connections 2024-09-15T22:43:58.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.327+0000 7f4caa600640 1 -- 172.21.15.111:0/2403966198 wait complete. 2024-09-15T22:43:58.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.328+0000 7f4caa600640 1 Processor -- start 2024-09-15T22:43:58.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.328+0000 7f4caa600640 1 -- start start 2024-09-15T22:43:58.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.329+0000 7f4caa600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ca40745c0 0x7f4ca419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:58.332 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.329+0000 7f4caa600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ca4074ee0 0x7f4ca419ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:58.332 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.329+0000 7f4caa600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ca410e420 0x7f4ca419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:58.332 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.329+0000 7f4caa600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca4114180 con 0x7f4ca410e420 2024-09-15T22:43:58.332 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.329+0000 7f4caa600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca4114000 con 0x7f4ca40745c0 2024-09-15T22:43:58.332 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.329+0000 7f4caa600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca4114300 con 0x7f4ca4074ee0 2024-09-15T22:43:58.332 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.329+0000 7f4ca3400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ca40745c0 0x7f4ca419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:58.332 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.329+0000 7f4ca3400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ca40745c0 0x7f4ca419a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:33746/0 (socket says 172.21.15.111:33746) 2024-09-15T22:43:58.332 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.329+0000 7f4ca3400640 1 -- 172.21.15.111:0/2832408695 learned_addr learned my addr 172.21.15.111:0/2832408695 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:43:58.332 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.329+0000 7f4ca2a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ca4074ee0 0x7f4ca419ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:58.332 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.329+0000 7f4ca3e00640 1 --2- 172.21.15.111:0/2832408695 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ca410e420 0x7f4ca419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:58.332 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.330+0000 7f4ca3400640 1 -- 172.21.15.111:0/2832408695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ca4074ee0 msgr2=0x7f4ca419ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:58.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.330+0000 7f4ca3400640 1 --2- 172.21.15.111:0/2832408695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ca4074ee0 0x7f4ca419ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.330+0000 7f4ca3400640 1 -- 172.21.15.111:0/2832408695 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ca410e420 msgr2=0x7f4ca419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:58.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.330+0000 7f4ca3400640 1 --2- 172.21.15.111:0/2832408695 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ca410e420 0x7f4ca419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.330+0000 7f4ca3400640 1 -- 172.21.15.111:0/2832408695 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca419f6b0 con 0x7f4ca40745c0 2024-09-15T22:43:58.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.330+0000 7f4ca2a00640 1 --2- 172.21.15.111:0/2832408695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ca4074ee0 0x7f4ca419ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:43:58.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.330+0000 7f4ca3400640 1 --2- 172.21.15.111:0/2832408695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ca40745c0 0x7f4ca419a700 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4c94006010 tx=0x7f4c94004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:58.333 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.331+0000 7f4c9be00640 1 -- 172.21.15.111:0/2832408695 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9403f030 con 0x7f4ca40745c0 2024-09-15T22:43:58.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.331+0000 7f4c9be00640 1 -- 172.21.15.111:0/2832408695 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c94004440 con 0x7f4ca40745c0 2024-09-15T22:43:58.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.331+0000 7f4caa600640 1 -- 172.21.15.111:0/2832408695 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca419f940 con 0x7f4ca40745c0 2024-09-15T22:43:58.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.331+0000 7f4c9be00640 1 -- 172.21.15.111:0/2832408695 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c94035070 con 0x7f4ca40745c0 2024-09-15T22:43:58.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.331+0000 7f4caa600640 1 -- 172.21.15.111:0/2832408695 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca41a7280 con 0x7f4ca40745c0 2024-09-15T22:43:58.334 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.332+0000 7f4caa600640 1 -- 172.21.15.111:0/2832408695 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ca40749a0 con 0x7f4ca40745c0 2024-09-15T22:43:58.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.335+0000 7f4c9be00640 1 -- 172.21.15.111:0/2832408695 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f4c94042030 con 0x7f4ca40745c0 2024-09-15T22:43:58.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.337+0000 7f4c9be00640 1 --2- 172.21.15.111:0/2832408695 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4c880a32c0 0x7f4c880a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:43:58.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.337+0000 7f4c9be00640 1 -- 172.21.15.111:0/2832408695 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f4c940ef7e0 con 0x7f4ca40745c0 2024-09-15T22:43:58.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.337+0000 7f4ca2a00640 1 --2- 172.21.15.111:0/2832408695 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4c880a32c0 0x7f4c880a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:43:58.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.338+0000 7f4ca2a00640 1 --2- 172.21.15.111:0/2832408695 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4c880a32c0 0x7f4c880a5780 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4ca410f7d0 tx=0x7f4c90005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:43:58.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.341+0000 7f4c9be00640 1 -- 172.21.15.111:0/2832408695 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4c940bc2b0 con 0x7f4ca40745c0 2024-09-15T22:43:58.565 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.563+0000 7f4caa600640 1 -- 172.21.15.111:0/2832408695 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ca41095a0 con 0x7f4ca40745c0 2024-09-15T22:43:58.566 INFO:journalctl@ceph.mds.g.smithi111.stdout:Sep 15 22:43:58 smithi111 podman[70896]: 2024-09-15 22:43:58.49315583 +0000 UTC m=+0.146765014 container create 367ebeea7f7183a2e28ae1cdd243b2074a21d5eb89d8258f09e9259a99b0778b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-g, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-15T22:43:58.566 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.564+0000 7f4c9be00640 1 -- 172.21.15.111:0/2832408695 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8058 (secure 0 0 0) 0x7f4c940b6e20 con 0x7f4ca40745c0 2024-09-15T22:43:58.566 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:43:58.566 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":10,"btime":"2024-09-15T22:43:55:015778+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/453336603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":453336603},{"type":"v1","addr":"172.21.15.111:6835","nonce":453336603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/3181832287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3181832287},{"type":"v1","addr":"172.21.15.111:6837","nonce":3181832287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/4061441359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":4061441359},{"type":"v1","addr":"172.21.15.192:6833","nonce":4061441359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/4220531578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":4220531578},{"type":"v1","addr":"172.21.15.121:6833","nonce":4220531578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3371796806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3371796806},{"type":"v1","addr":"172.21.15.121:6835","nonce":3371796806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/4080914921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":4080914921},{"type":"v1","addr":"172.21.15.192:6835","nonce":4080914921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3995501707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3995501707},{"type":"v1","addr":"172.21.15.192:6837","nonce":3995501707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1443772791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1443772791},{"type":"v1","addr":"172.21.15.121:6837","nonce":1443772791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/800136261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":800136261},{"type":"v1","addr":"172.21.15.192:6839","nonce":800136261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-15T22:43:58.567 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 10 2024-09-15T22:43:58.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.567+0000 7f4caa600640 1 -- 172.21.15.111:0/2832408695 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4c880a32c0 msgr2=0x7f4c880a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:58.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.567+0000 7f4caa600640 1 --2- 172.21.15.111:0/2832408695 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4c880a32c0 0x7f4c880a5780 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4ca410f7d0 tx=0x7f4c90005f90 comp rx=0 tx=0).stop 2024-09-15T22:43:58.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.567+0000 7f4caa600640 1 -- 172.21.15.111:0/2832408695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ca40745c0 msgr2=0x7f4ca419a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:43:58.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.567+0000 7f4caa600640 1 --2- 172.21.15.111:0/2832408695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ca40745c0 0x7f4ca419a700 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4c94006010 tx=0x7f4c94004040 comp rx=0 tx=0).stop 2024-09-15T22:43:58.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.567+0000 7f4caa600640 1 -- 172.21.15.111:0/2832408695 shutdown_connections 2024-09-15T22:43:58.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.567+0000 7f4caa600640 1 --2- 172.21.15.111:0/2832408695 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ca410e420 0x7f4ca419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.567+0000 7f4caa600640 1 --2- 172.21.15.111:0/2832408695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ca4074ee0 0x7f4ca419ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.567+0000 7f4caa600640 1 --2- 172.21.15.111:0/2832408695 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4c880a32c0 0x7f4c880a5780 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.567+0000 7f4caa600640 1 --2- 172.21.15.111:0/2832408695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4ca40745c0 0x7f4ca419a700 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:43:58.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.567+0000 7f4caa600640 1 -- 172.21.15.111:0/2832408695 >> 172.21.15.111:0/2832408695 conn(0x7f4ca40fd8c0 msgr2=0x7f4ca40fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:43:58.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.567+0000 7f4caa600640 1 -- 172.21.15.111:0/2832408695 shutdown_connections 2024-09-15T22:43:58.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:43:58.567+0000 7f4caa600640 1 -- 172.21.15.111:0/2832408695 wait complete. 2024-09-15T22:43:59.290 INFO:journalctl@ceph.mds.g.smithi111.stdout:Sep 15 22:43:59 smithi111 podman[70896]: 2024-09-15 22:43:59.005750403 +0000 UTC m=+0.659359588 container init 367ebeea7f7183a2e28ae1cdd243b2074a21d5eb89d8258f09e9259a99b0778b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-g, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-15T22:43:59.290 INFO:journalctl@ceph.mds.g.smithi111.stdout:Sep 15 22:43:59 smithi111 podman[70896]: 2024-09-15 22:43:59.012168769 +0000 UTC m=+0.665777954 container start 367ebeea7f7183a2e28ae1cdd243b2074a21d5eb89d8258f09e9259a99b0778b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-g, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-15T22:43:59.290 INFO:journalctl@ceph.mds.g.smithi111.stdout:Sep 15 22:43:59 smithi111 ceph-mds[70930]: -- 172.21.15.111:0/448650531 <== mon.0 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558138611860 con 0x5581385ef000 2024-09-15T22:43:59.290 INFO:journalctl@ceph.mds.g.smithi111.stdout:Sep 15 22:43:59 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-g[70926]: starting mds.g at 2024-09-15T22:43:59.290 INFO:journalctl@ceph.mds.g.smithi111.stdout:Sep 15 22:43:59 smithi111 bash[70896]: 367ebeea7f7183a2e28ae1cdd243b2074a21d5eb89d8258f09e9259a99b0778b 2024-09-15T22:43:59.291 INFO:journalctl@ceph.mds.g.smithi111.stdout:Sep 15 22:43:59 smithi111 systemd[1]: Started Ceph mds.g for b7614d3c-73b2-11ef-bceb-c7b262605968. 2024-09-15T22:43:59.601 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:59 smithi111 ceph-mon[30629]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:59.601 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:59 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/2832408695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:59.601 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:59 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:59.601 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:59 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:59.601 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:59 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:59.601 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:59 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:59.601 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:59 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:59.601 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:43:59 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:59.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:59 smithi192 ceph-mon[35427]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:59.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:59 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/2832408695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:59.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:59 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:59.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:59 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:59.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:59 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:59.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:59 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:59.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:59 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:59.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:43:59 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:43:59.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:59 smithi121 ceph-mon[34815]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:43:59.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:59 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/2832408695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:43:59.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:59 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:59.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:59 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:59.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:59 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:43:59.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:59 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:43:59.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:59 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:43:59.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:43:59 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:44:00.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:00 smithi192 ceph-mon[35427]: Deploying daemon mds.a on smithi111 2024-09-15T22:44:00.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:00 smithi192 ceph-mon[35427]: mds.? [v2:172.21.15.111:6838/4190237053,v1:172.21.15.111:6839/4190237053] up:boot 2024-09-15T22:44:00.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:00 smithi192 ceph-mon[35427]: fsmap 10 up:standby 2024-09-15T22:44:00.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:00 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-15T22:44:00.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:00 smithi121 ceph-mon[34815]: Deploying daemon mds.a on smithi111 2024-09-15T22:44:00.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:00 smithi121 ceph-mon[34815]: mds.? [v2:172.21.15.111:6838/4190237053,v1:172.21.15.111:6839/4190237053] up:boot 2024-09-15T22:44:00.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:00 smithi121 ceph-mon[34815]: fsmap 10 up:standby 2024-09-15T22:44:00.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:00 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-15T22:44:00.912 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:00 smithi111 ceph-mon[30629]: Deploying daemon mds.a on smithi111 2024-09-15T22:44:00.912 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:00 smithi111 ceph-mon[30629]: mds.? [v2:172.21.15.111:6838/4190237053,v1:172.21.15.111:6839/4190237053] up:boot 2024-09-15T22:44:00.912 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:00 smithi111 ceph-mon[30629]: fsmap 10 up:standby 2024-09-15T22:44:00.912 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:00 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-15T22:44:01.645 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:44:01.697 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:01 smithi111 ceph-mon[30629]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:01.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:01 smithi192 ceph-mon[35427]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:01.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:01 smithi121 ceph-mon[34815]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:02.050 INFO:journalctl@ceph.mds.a.smithi111.stdout:Sep 15 22:44:01 smithi111 systemd[1]: Starting Ceph mds.a for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T22:44:02.321 INFO:journalctl@ceph.mds.a.smithi111.stdout:Sep 15 22:44:02 smithi111 podman[71237]: 2024-09-15 22:44:01.953500736 +0000 UTC m=+0.027905496 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:44:02.322 INFO:journalctl@ceph.mds.a.smithi111.stdout:Sep 15 22:44:02 smithi111 podman[71237]: 2024-09-15 22:44:02.067948159 +0000 UTC m=+0.142352890 container create acfd8775c53ae73aeae3eb1abadd5e8496ba587df41921fcfc2b9195ffa9d38f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD) 2024-09-15T22:44:02.322 INFO:journalctl@ceph.mds.a.smithi111.stdout:Sep 15 22:44:02 smithi111 podman[71237]: 2024-09-15 22:44:02.320548299 +0000 UTC m=+0.394953030 container init acfd8775c53ae73aeae3eb1abadd5e8496ba587df41921fcfc2b9195ffa9d38f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-a, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-15T22:44:02.539 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:02.666 INFO:journalctl@ceph.mds.a.smithi111.stdout:Sep 15 22:44:02 smithi111 podman[71237]: 2024-09-15 22:44:02.327100272 +0000 UTC m=+0.401505005 container start acfd8775c53ae73aeae3eb1abadd5e8496ba587df41921fcfc2b9195ffa9d38f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-15T22:44:02.666 INFO:journalctl@ceph.mds.a.smithi111.stdout:Sep 15 22:44:02 smithi111 ceph-mds[71275]: -- 172.21.15.111:0/1294832092 <== mon.0 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560d7307bc20 con 0x560d73059400 2024-09-15T22:44:02.666 INFO:journalctl@ceph.mds.a.smithi111.stdout:Sep 15 22:44:02 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-a[71271]: starting mds.a at 2024-09-15T22:44:02.666 INFO:journalctl@ceph.mds.a.smithi111.stdout:Sep 15 22:44:02 smithi111 bash[71237]: acfd8775c53ae73aeae3eb1abadd5e8496ba587df41921fcfc2b9195ffa9d38f 2024-09-15T22:44:02.666 INFO:journalctl@ceph.mds.a.smithi111.stdout:Sep 15 22:44:02 smithi111 systemd[1]: Started Ceph mds.a for b7614d3c-73b2-11ef-bceb-c7b262605968. 2024-09-15T22:44:03.137 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.135+0000 7fd3fae00640 1 -- 172.21.15.111:0/4144292519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3f410e3c0 msgr2=0x7fd3f4110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:03.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.135+0000 7fd3fae00640 1 --2- 172.21.15.111:0/4144292519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3f410e3c0 0x7fd3f4110810 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec009ca0 tx=0x7fd3ec02a950 comp rx=0 tx=0).stop 2024-09-15T22:44:03.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.135+0000 7fd3fae00640 1 -- 172.21.15.111:0/4144292519 shutdown_connections 2024-09-15T22:44:03.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.135+0000 7fd3fae00640 1 --2- 172.21.15.111:0/4144292519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3f410e3c0 0x7fd3f4110810 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.135+0000 7fd3fae00640 1 --2- 172.21.15.111:0/4144292519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3f4074f40 0x7fd3f410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.135+0000 7fd3fae00640 1 --2- 172.21.15.111:0/4144292519 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3f4074620 0x7fd3f4074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.135+0000 7fd3fae00640 1 -- 172.21.15.111:0/4144292519 >> 172.21.15.111:0/4144292519 conn(0x7fd3f40fd8c0 msgr2=0x7fd3f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:03.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.135+0000 7fd3fae00640 1 -- 172.21.15.111:0/4144292519 shutdown_connections 2024-09-15T22:44:03.138 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.136+0000 7fd3fae00640 1 -- 172.21.15.111:0/4144292519 wait complete. 2024-09-15T22:44:03.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.136+0000 7fd3fae00640 1 Processor -- start 2024-09-15T22:44:03.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.137+0000 7fd3fae00640 1 -- start start 2024-09-15T22:44:03.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.137+0000 7fd3fae00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3f4074620 0x7fd3f419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:03.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.137+0000 7fd3fae00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3f4074f40 0x7fd3f419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:03.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.137+0000 7fd3fae00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3f410e3c0 0x7fd3f419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:03.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.137+0000 7fd3fae00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41141c0 con 0x7fd3f4074f40 2024-09-15T22:44:03.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.137+0000 7fd3fae00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f4114040 con 0x7fd3f4074620 2024-09-15T22:44:03.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.137+0000 7fd3fae00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f4114340 con 0x7fd3f410e3c0 2024-09-15T22:44:03.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.137+0000 7fd3f3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3f4074620 0x7fd3f419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:03.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.137+0000 7fd3f3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3f4074620 0x7fd3f419a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:37676/0 (socket says 172.21.15.111:37676) 2024-09-15T22:44:03.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.137+0000 7fd3f3e00640 1 -- 172.21.15.111:0/3769652071 learned_addr learned my addr 172.21.15.111:0/3769652071 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:44:03.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.137+0000 7fd3ebe00640 1 --2- 172.21.15.111:0/3769652071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3f4074f40 0x7fd3f419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:03.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.137+0000 7fd3f8e00640 1 --2- 172.21.15.111:0/3769652071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3f410e3c0 0x7fd3f419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:03.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.137+0000 7fd3f3e00640 1 -- 172.21.15.111:0/3769652071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3f410e3c0 msgr2=0x7fd3f419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:03.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.137+0000 7fd3f3e00640 1 --2- 172.21.15.111:0/3769652071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3f410e3c0 0x7fd3f419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.137+0000 7fd3f3e00640 1 -- 172.21.15.111:0/3769652071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3f4074f40 msgr2=0x7fd3f419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:03.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.137+0000 7fd3f3e00640 1 --2- 172.21.15.111:0/3769652071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3f4074f40 0x7fd3f419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.141 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.137+0000 7fd3f3e00640 1 -- 172.21.15.111:0/3769652071 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f419f610 con 0x7fd3f4074620 2024-09-15T22:44:03.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.137+0000 7fd3f8e00640 1 --2- 172.21.15.111:0/3769652071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3f410e3c0 0x7fd3f419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:03.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.138+0000 7fd3f3e00640 1 --2- 172.21.15.111:0/3769652071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3f4074620 0x7fd3f419a660 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd3e400ea60 tx=0x7fd3e400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:03.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.138+0000 7fd3f1600640 1 -- 172.21.15.111:0/3769652071 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e400cde0 con 0x7fd3f4074620 2024-09-15T22:44:03.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.138+0000 7fd3fae00640 1 -- 172.21.15.111:0/3769652071 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3f419f900 con 0x7fd3f4074620 2024-09-15T22:44:03.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.138+0000 7fd3f1600640 1 -- 172.21.15.111:0/3769652071 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3e40035f0 con 0x7fd3f4074620 2024-09-15T22:44:03.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.138+0000 7fd3f1600640 1 -- 172.21.15.111:0/3769652071 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4010670 con 0x7fd3f4074620 2024-09-15T22:44:03.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.138+0000 7fd3fae00640 1 -- 172.21.15.111:0/3769652071 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3f41a71e0 con 0x7fd3f4074620 2024-09-15T22:44:03.142 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.139+0000 7fd3fae00640 1 -- 172.21.15.111:0/3769652071 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3f4074a00 con 0x7fd3f4074620 2024-09-15T22:44:03.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.141+0000 7fd3f1600640 1 -- 172.21.15.111:0/3769652071 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fd3e40037a0 con 0x7fd3f4074620 2024-09-15T22:44:03.143 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.142+0000 7fd3f1600640 1 --2- 172.21.15.111:0/3769652071 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd3d80a3240 0x7fd3d80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:03.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.142+0000 7fd3f1600640 1 -- 172.21.15.111:0/3769652071 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fd3e40cf8d0 con 0x7fd3f4074620 2024-09-15T22:44:03.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.142+0000 7fd3ebe00640 1 --2- 172.21.15.111:0/3769652071 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd3d80a3240 0x7fd3d80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:03.144 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.143+0000 7fd3ebe00640 1 --2- 172.21.15.111:0/3769652071 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd3d80a3240 0x7fd3d80a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4110cd0 tx=0x7fd3dc005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:03.147 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.146+0000 7fd3f1600640 1 -- 172.21.15.111:0/3769652071 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd3e409c9f0 con 0x7fd3f4074620 2024-09-15T22:44:03.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.353+0000 7fd3fae00640 1 -- 172.21.15.111:0/3769652071 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3f419b9c0 con 0x7fd3f4074620 2024-09-15T22:44:03.355 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.354+0000 7fd3f1600640 1 -- 172.21.15.111:0/3769652071 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9713 (secure 0 0 0) 0x7fd3e409c2f0 con 0x7fd3f4074620 2024-09-15T22:44:03.356 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:44:03.356 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":12,"btime":"2024-09-15T22:44:02:515641+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/453336603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":453336603},{"type":"v1","addr":"172.21.15.111:6835","nonce":453336603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/3181832287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3181832287},{"type":"v1","addr":"172.21.15.111:6837","nonce":3181832287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/4190237053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":4190237053},{"type":"v1","addr":"172.21.15.111:6839","nonce":4190237053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14502,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6841/267728828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6840","nonce":267728828},{"type":"v1","addr":"172.21.15.111:6841","nonce":267728828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/4061441359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":4061441359},{"type":"v1","addr":"172.21.15.192:6833","nonce":4061441359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/4220531578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":4220531578},{"type":"v1","addr":"172.21.15.121:6833","nonce":4220531578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3371796806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3371796806},{"type":"v1","addr":"172.21.15.121:6835","nonce":3371796806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/4080914921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":4080914921},{"type":"v1","addr":"172.21.15.192:6835","nonce":4080914921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3995501707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3995501707},{"type":"v1","addr":"172.21.15.192:6837","nonce":3995501707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1443772791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1443772791},{"type":"v1","addr":"172.21.15.121:6837","nonce":1443772791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/800136261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":800136261},{"type":"v1","addr":"172.21.15.192:6839","nonce":800136261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-15T22:44:03.356 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 12 2024-09-15T22:44:03.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.356+0000 7fd3fae00640 1 -- 172.21.15.111:0/3769652071 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd3d80a3240 msgr2=0x7fd3d80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:03.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.356+0000 7fd3fae00640 1 --2- 172.21.15.111:0/3769652071 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd3d80a3240 0x7fd3d80a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4110cd0 tx=0x7fd3dc005f90 comp rx=0 tx=0).stop 2024-09-15T22:44:03.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.356+0000 7fd3fae00640 1 -- 172.21.15.111:0/3769652071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3f4074620 msgr2=0x7fd3f419a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:03.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.356+0000 7fd3fae00640 1 --2- 172.21.15.111:0/3769652071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3f4074620 0x7fd3f419a660 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd3e400ea60 tx=0x7fd3e400ef20 comp rx=0 tx=0).stop 2024-09-15T22:44:03.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.356+0000 7fd3fae00640 1 -- 172.21.15.111:0/3769652071 shutdown_connections 2024-09-15T22:44:03.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.356+0000 7fd3fae00640 1 --2- 172.21.15.111:0/3769652071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3f410e3c0 0x7fd3f419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.358 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.356+0000 7fd3fae00640 1 --2- 172.21.15.111:0/3769652071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3f4074f40 0x7fd3f419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.356+0000 7fd3fae00640 1 --2- 172.21.15.111:0/3769652071 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd3d80a3240 0x7fd3d80a5700 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.356+0000 7fd3fae00640 1 --2- 172.21.15.111:0/3769652071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3f4074620 0x7fd3f419a660 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:03.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.356+0000 7fd3fae00640 1 -- 172.21.15.111:0/3769652071 >> 172.21.15.111:0/3769652071 conn(0x7fd3f40fd8c0 msgr2=0x7fd3f40fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:03.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.357+0000 7fd3fae00640 1 -- 172.21.15.111:0/3769652071 shutdown_connections 2024-09-15T22:44:03.359 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:03.357+0000 7fd3fae00640 1 -- 172.21.15.111:0/3769652071 wait complete. 2024-09-15T22:44:03.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:03 smithi121 ceph-mon[34815]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:03.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:03 smithi121 ceph-mon[34815]: mds.? [v2:172.21.15.111:6840/267728828,v1:172.21.15.111:6841/267728828] up:boot 2024-09-15T22:44:03.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:03 smithi121 ceph-mon[34815]: fsmap 11 up:standby 2024-09-15T22:44:03.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:03 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-15T22:44:03.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:03 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:03.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:03 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:03.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:03 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:03.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:03 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:44:03.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:03 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:44:03.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:03 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:44:03.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:03 smithi121 ceph-mon[34815]: Deploying daemon mds.b on smithi121 2024-09-15T22:44:03.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:03 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/3769652071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:44:03.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:03 smithi111 ceph-mon[30629]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:03.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:03 smithi111 ceph-mon[30629]: mds.? [v2:172.21.15.111:6840/267728828,v1:172.21.15.111:6841/267728828] up:boot 2024-09-15T22:44:03.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:03 smithi111 ceph-mon[30629]: fsmap 11 up:standby 2024-09-15T22:44:03.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:03 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-15T22:44:03.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:03 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:03.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:03 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:03.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:03 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:03.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:03 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:44:03.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:03 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:44:03.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:03 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:44:03.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:03 smithi111 ceph-mon[30629]: Deploying daemon mds.b on smithi121 2024-09-15T22:44:03.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:03 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/3769652071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:44:03.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:03 smithi192 ceph-mon[35427]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:03.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:03 smithi192 ceph-mon[35427]: mds.? [v2:172.21.15.111:6840/267728828,v1:172.21.15.111:6841/267728828] up:boot 2024-09-15T22:44:03.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:03 smithi192 ceph-mon[35427]: fsmap 11 up:standby 2024-09-15T22:44:03.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:03 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-15T22:44:03.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:03 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:03.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:03 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:03.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:03 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:03.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:03 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:44:03.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:03 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:44:03.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:03 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:44:03.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:03 smithi192 ceph-mon[35427]: Deploying daemon mds.b on smithi121 2024-09-15T22:44:03.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:03 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/3769652071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:44:04.640 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 15 22:44:04 smithi121 systemd[1]: Starting Ceph mds.b for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T22:44:04.897 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 15 22:44:04 smithi121 podman[65467]: 2024-09-15 22:44:04.546236362 +0000 UTC m=+0.015692656 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:44:04.897 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 15 22:44:04 smithi121 podman[65467]: 2024-09-15 22:44:04.650525846 +0000 UTC m=+0.119982126 container create 14529e1860a7900a453d514cb5d727f7e3e0ac88fe97f1e717c0eb5e98425874 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:44:04.897 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 15 22:44:04 smithi121 podman[65467]: 2024-09-15 22:44:04.869982441 +0000 UTC m=+0.339438722 container init 14529e1860a7900a453d514cb5d727f7e3e0ac88fe97f1e717c0eb5e98425874 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-15T22:44:04.897 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 15 22:44:04 smithi121 podman[65467]: 2024-09-15 22:44:04.872815223 +0000 UTC m=+0.342271505 container start 14529e1860a7900a453d514cb5d727f7e3e0ac88fe97f1e717c0eb5e98425874 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-b, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-15T22:44:04.897 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 15 22:44:04 smithi121 ceph-mds[65487]: -- 172.21.15.121:0/3157509961 <== mon.2 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56476c0e5860 con 0x56476c0c3000 2024-09-15T22:44:05.390 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 15 22:44:04 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-b[65483]: starting mds.b at 2024-09-15T22:44:05.390 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 15 22:44:04 smithi121 bash[65467]: 14529e1860a7900a453d514cb5d727f7e3e0ac88fe97f1e717c0eb5e98425874 2024-09-15T22:44:05.390 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 15 22:44:04 smithi121 systemd[1]: Started Ceph mds.b for b7614d3c-73b2-11ef-bceb-c7b262605968. 2024-09-15T22:44:05.796 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:05 smithi192 ceph-mon[35427]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:05.796 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:05 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:05.796 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:05 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:05.796 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:05 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:05.797 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:05 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:05.797 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:05 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:05.797 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:05 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:44:05.797 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:05 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:05.808 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:05 smithi121 ceph-mon[34815]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:05.808 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:05 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:05.808 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:05 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:05.808 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:05 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:05.808 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:05 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:05.808 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:05 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:05.808 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:05 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:44:05.808 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:05 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:05.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:05 smithi111 ceph-mon[30629]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:05.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:05 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:05.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:05 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:05.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:05 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:05.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:05 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:05.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:05 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:05.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:05 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:44:05.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:05 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:06.129 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:44:06.618 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:06.801 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:06 smithi111 ceph-mon[30629]: mds.? [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] up:boot 2024-09-15T22:44:06.801 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:06 smithi111 ceph-mon[30629]: fsmap 12 up:standby 2024-09-15T22:44:06.801 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:06 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:44:06.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:06 smithi192 ceph-mon[35427]: mds.? [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] up:boot 2024-09-15T22:44:06.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:06 smithi192 ceph-mon[35427]: fsmap 12 up:standby 2024-09-15T22:44:06.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:06 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:44:06.830 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:06 smithi121 ceph-mon[34815]: mds.? [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] up:boot 2024-09-15T22:44:06.830 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:06 smithi121 ceph-mon[34815]: fsmap 12 up:standby 2024-09-15T22:44:06.830 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:06 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:44:07.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.262+0000 7faec4200640 1 -- 172.21.15.111:0/577582763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faebc102790 msgr2=0x7faebc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:07.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.262+0000 7faec4200640 1 --2- 172.21.15.111:0/577582763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faebc102790 0x7faebc1067e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7faea8009900 tx=0x7faea802a410 comp rx=0 tx=0).stop 2024-09-15T22:44:07.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.262+0000 7faec4200640 1 -- 172.21.15.111:0/577582763 shutdown_connections 2024-09-15T22:44:07.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.262+0000 7faec4200640 1 --2- 172.21.15.111:0/577582763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faebc106f10 0x7faebc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.262+0000 7faec4200640 1 --2- 172.21.15.111:0/577582763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faebc102790 0x7faebc1067e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.262+0000 7faec4200640 1 --2- 172.21.15.111:0/577582763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faebc101de0 0x7faebc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.262+0000 7faec4200640 1 -- 172.21.15.111:0/577582763 >> 172.21.15.111:0/577582763 conn(0x7faebc0fd8c0 msgr2=0x7faebc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:07.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.263+0000 7faec4200640 1 -- 172.21.15.111:0/577582763 shutdown_connections 2024-09-15T22:44:07.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.263+0000 7faec4200640 1 -- 172.21.15.111:0/577582763 wait complete. 2024-09-15T22:44:07.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.263+0000 7faec4200640 1 Processor -- start 2024-09-15T22:44:07.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.264+0000 7faec4200640 1 -- start start 2024-09-15T22:44:07.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.264+0000 7faec4200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faebc101de0 0x7faebc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:07.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.264+0000 7faec4200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faebc102790 0x7faebc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:07.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.264+0000 7faec4200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faebc106f10 0x7faebc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:07.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.264+0000 7faec4200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faebc111fc0 con 0x7faebc102790 2024-09-15T22:44:07.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.264+0000 7faec4200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faebc111e40 con 0x7faebc101de0 2024-09-15T22:44:07.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.264+0000 7faec4200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faebc112140 con 0x7faebc106f10 2024-09-15T22:44:07.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.264+0000 7faec1800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faebc101de0 0x7faebc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:07.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.264+0000 7faec0e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faebc102790 0x7faebc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:07.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.264+0000 7faec1800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faebc101de0 0x7faebc0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:37690/0 (socket says 172.21.15.111:37690) 2024-09-15T22:44:07.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.264+0000 7faec1800640 1 -- 172.21.15.111:0/1950008204 learned_addr learned my addr 172.21.15.111:0/1950008204 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:44:07.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.264+0000 7faec2200640 1 --2- 172.21.15.111:0/1950008204 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faebc106f10 0x7faebc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:07.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.265+0000 7faec1800640 1 -- 172.21.15.111:0/1950008204 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faebc106f10 msgr2=0x7faebc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:07.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.265+0000 7faec1800640 1 --2- 172.21.15.111:0/1950008204 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faebc106f10 0x7faebc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.265+0000 7faec1800640 1 -- 172.21.15.111:0/1950008204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faebc102790 msgr2=0x7faebc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:07.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.265+0000 7faec1800640 1 --2- 172.21.15.111:0/1950008204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faebc102790 0x7faebc074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.265+0000 7faec1800640 1 -- 172.21.15.111:0/1950008204 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faebc1aade0 con 0x7faebc101de0 2024-09-15T22:44:07.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.265+0000 7faec1800640 1 --2- 172.21.15.111:0/1950008204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faebc101de0 0x7faebc0744f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faeb000e9f0 tx=0x7faeb000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:07.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.265+0000 7faeba000640 1 -- 172.21.15.111:0/1950008204 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb000cd80 con 0x7faebc101de0 2024-09-15T22:44:07.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.265+0000 7faeba000640 1 -- 172.21.15.111:0/1950008204 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faeb0005500 con 0x7faebc101de0 2024-09-15T22:44:07.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.265+0000 7faeba000640 1 -- 172.21.15.111:0/1950008204 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb0010690 con 0x7faebc101de0 2024-09-15T22:44:07.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.265+0000 7faec4200640 1 -- 172.21.15.111:0/1950008204 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faebc1aafb0 con 0x7faebc101de0 2024-09-15T22:44:07.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.265+0000 7faec4200640 1 -- 172.21.15.111:0/1950008204 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faebc1ab410 con 0x7faebc101de0 2024-09-15T22:44:07.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.266+0000 7faec4200640 1 -- 172.21.15.111:0/1950008204 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faebc1032c0 con 0x7faebc101de0 2024-09-15T22:44:07.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.268+0000 7faeba000640 1 -- 172.21.15.111:0/1950008204 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7faeb00046c0 con 0x7faebc101de0 2024-09-15T22:44:07.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.269+0000 7faeba000640 1 --2- 172.21.15.111:0/1950008204 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faea40a32c0 0x7faea40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:07.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.269+0000 7faeba000640 1 -- 172.21.15.111:0/1950008204 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7faeb00cf8d0 con 0x7faebc101de0 2024-09-15T22:44:07.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.269+0000 7faec0e00640 1 --2- 172.21.15.111:0/1950008204 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faea40a32c0 0x7faea40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:07.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.270+0000 7faec0e00640 1 --2- 172.21.15.111:0/1950008204 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faea40a32c0 0x7faea40a5780 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faea8006010 tx=0x7faea8000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:07.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.273+0000 7faeba000640 1 -- 172.21.15.111:0/1950008204 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faeb009c9f0 con 0x7faebc101de0 2024-09-15T22:44:07.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.504+0000 7faec4200640 1 -- 172.21.15.111:0/1950008204 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faebc10a150 con 0x7faebc101de0 2024-09-15T22:44:07.506 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.505+0000 7faeba000640 1 -- 172.21.15.111:0/1950008204 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10542 (secure 0 0 0) 0x7faeb009c2f0 con 0x7faebc101de0 2024-09-15T22:44:07.507 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:44:07.507 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":13,"btime":"2024-09-15T22:44:05:526639+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/453336603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":453336603},{"type":"v1","addr":"172.21.15.111:6835","nonce":453336603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/3181832287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3181832287},{"type":"v1","addr":"172.21.15.111:6837","nonce":3181832287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/4190237053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":4190237053},{"type":"v1","addr":"172.21.15.111:6839","nonce":4190237053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14502,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6841/267728828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6840","nonce":267728828},{"type":"v1","addr":"172.21.15.111:6841","nonce":267728828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/4061441359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":4061441359},{"type":"v1","addr":"172.21.15.192:6833","nonce":4061441359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/4220531578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":4220531578},{"type":"v1","addr":"172.21.15.121:6833","nonce":4220531578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24485,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3371796806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3371796806},{"type":"v1","addr":"172.21.15.121:6835","nonce":3371796806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/4080914921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":4080914921},{"type":"v1","addr":"172.21.15.192:6835","nonce":4080914921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3995501707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3995501707},{"type":"v1","addr":"172.21.15.192:6837","nonce":3995501707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24503,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/1443772791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1443772791},{"type":"v1","addr":"172.21.15.121:6837","nonce":1443772791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6839/800136261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":800136261},{"type":"v1","addr":"172.21.15.192:6839","nonce":800136261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6839/1797352097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1797352097},{"type":"v1","addr":"172.21.15.121:6839","nonce":1797352097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-15T22:44:07.507 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 13 2024-09-15T22:44:07.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.508+0000 7faec4200640 1 -- 172.21.15.111:0/1950008204 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faea40a32c0 msgr2=0x7faea40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:07.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.508+0000 7faec4200640 1 --2- 172.21.15.111:0/1950008204 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faea40a32c0 0x7faea40a5780 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faea8006010 tx=0x7faea8000bc0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.508+0000 7faec4200640 1 -- 172.21.15.111:0/1950008204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faebc101de0 msgr2=0x7faebc0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:07.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.508+0000 7faec4200640 1 --2- 172.21.15.111:0/1950008204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faebc101de0 0x7faebc0744f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faeb000e9f0 tx=0x7faeb000edf0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.508+0000 7faec4200640 1 -- 172.21.15.111:0/1950008204 shutdown_connections 2024-09-15T22:44:07.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.508+0000 7faec4200640 1 --2- 172.21.15.111:0/1950008204 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faebc106f10 0x7faebc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.508+0000 7faec4200640 1 --2- 172.21.15.111:0/1950008204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faebc102790 0x7faebc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.508+0000 7faec4200640 1 --2- 172.21.15.111:0/1950008204 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faea40a32c0 0x7faea40a5780 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.508+0000 7faec4200640 1 --2- 172.21.15.111:0/1950008204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faebc101de0 0x7faebc0744f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:07.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.508+0000 7faec4200640 1 -- 172.21.15.111:0/1950008204 >> 172.21.15.111:0/1950008204 conn(0x7faebc0fd8c0 msgr2=0x7faebc0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:07.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.509+0000 7faec4200640 1 -- 172.21.15.111:0/1950008204 shutdown_connections 2024-09-15T22:44:07.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:07.509+0000 7faec4200640 1 -- 172.21.15.111:0/1950008204 wait complete. 2024-09-15T22:44:07.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:07 smithi121 ceph-mon[34815]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:07.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:07 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:07.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:07 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:07.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:07 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:07.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:07 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:07.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:07 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/1950008204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:44:07.907 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:07 smithi111 ceph-mon[30629]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:07.907 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:07 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:07.907 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:07 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:07.907 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:07 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:07.907 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:07 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:07.908 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:07 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/1950008204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:44:08.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:07 smithi192 ceph-mon[35427]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:08.066 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:07 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:08.066 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:07 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:08.066 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:07 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:08.067 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:07 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:08.067 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:07 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/1950008204' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:44:08.350 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-15T22:44:08.742 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:09.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.289+0000 7fc481400640 1 -- 172.21.15.111:0/3040330522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc47c103ff0 msgr2=0x7fc47c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:09.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.289+0000 7fc481400640 1 --2- 172.21.15.111:0/3040330522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc47c103ff0 0x7fc47c1043d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc46c009900 tx=0x7fc46c02a410 comp rx=0 tx=0).stop 2024-09-15T22:44:09.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.289+0000 7fc481400640 1 -- 172.21.15.111:0/3040330522 shutdown_connections 2024-09-15T22:44:09.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.289+0000 7fc481400640 1 --2- 172.21.15.111:0/3040330522 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc47c109120 0x7fc47c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:09.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.289+0000 7fc481400640 1 --2- 172.21.15.111:0/3040330522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc47c1049a0 0x7fc47c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:09.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.289+0000 7fc481400640 1 --2- 172.21.15.111:0/3040330522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc47c103ff0 0x7fc47c1043d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:09.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.289+0000 7fc481400640 1 -- 172.21.15.111:0/3040330522 >> 172.21.15.111:0/3040330522 conn(0x7fc47c0ffa90 msgr2=0x7fc47c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:09.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.289+0000 7fc481400640 1 -- 172.21.15.111:0/3040330522 shutdown_connections 2024-09-15T22:44:09.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.289+0000 7fc481400640 1 -- 172.21.15.111:0/3040330522 wait complete. 2024-09-15T22:44:09.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.290+0000 7fc481400640 1 Processor -- start 2024-09-15T22:44:09.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.290+0000 7fc481400640 1 -- start start 2024-09-15T22:44:09.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.291+0000 7fc481400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc47c103ff0 0x7fc47c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:09.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.291+0000 7fc481400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc47c1049a0 0x7fc47c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:09.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.291+0000 7fc481400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc47c109120 0x7fc47c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:09.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.291+0000 7fc481400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc47c114150 con 0x7fc47c109120 2024-09-15T22:44:09.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.291+0000 7fc481400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc47c113fd0 con 0x7fc47c1049a0 2024-09-15T22:44:09.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.291+0000 7fc481400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc47c1142d0 con 0x7fc47c103ff0 2024-09-15T22:44:09.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.291+0000 7fc47a800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc47c103ff0 0x7fc47c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:09.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.291+0000 7fc47a800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc47c103ff0 0x7fc47c19a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:44256/0 (socket says 172.21.15.111:44256) 2024-09-15T22:44:09.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.291+0000 7fc479e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc47c1049a0 0x7fc47c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:09.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.291+0000 7fc47a800640 1 -- 172.21.15.111:0/309222432 learned_addr learned my addr 172.21.15.111:0/309222432 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:44:09.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.291+0000 7fc47b200640 1 --2- 172.21.15.111:0/309222432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc47c109120 0x7fc47c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:09.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.291+0000 7fc47a800640 1 -- 172.21.15.111:0/309222432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc47c1049a0 msgr2=0x7fc47c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:09.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.291+0000 7fc47a800640 1 --2- 172.21.15.111:0/309222432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc47c1049a0 0x7fc47c19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:09.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.291+0000 7fc47a800640 1 -- 172.21.15.111:0/309222432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc47c109120 msgr2=0x7fc47c19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:09.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.291+0000 7fc47a800640 1 --2- 172.21.15.111:0/309222432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc47c109120 0x7fc47c19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:09.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.291+0000 7fc47a800640 1 -- 172.21.15.111:0/309222432 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47c19f5a0 con 0x7fc47c103ff0 2024-09-15T22:44:09.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.291+0000 7fc47b200640 1 --2- 172.21.15.111:0/309222432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc47c109120 0x7fc47c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:09.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.291+0000 7fc479e00640 1 --2- 172.21.15.111:0/309222432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc47c1049a0 0x7fc47c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:44:09.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.292+0000 7fc47a800640 1 --2- 172.21.15.111:0/309222432 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc47c103ff0 0x7fc47c19a5f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc46c006010 tx=0x7fc46c004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:09.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.292+0000 7fc473400640 1 -- 172.21.15.111:0/309222432 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c03f030 con 0x7fc47c103ff0 2024-09-15T22:44:09.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.292+0000 7fc481400640 1 -- 172.21.15.111:0/309222432 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc47c19f830 con 0x7fc47c103ff0 2024-09-15T22:44:09.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.292+0000 7fc473400640 1 -- 172.21.15.111:0/309222432 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc46c004440 con 0x7fc47c103ff0 2024-09-15T22:44:09.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.292+0000 7fc473400640 1 -- 172.21.15.111:0/309222432 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c035070 con 0x7fc47c103ff0 2024-09-15T22:44:09.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.292+0000 7fc481400640 1 -- 172.21.15.111:0/309222432 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc47c1a7170 con 0x7fc47c103ff0 2024-09-15T22:44:09.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.293+0000 7fc481400640 1 -- 172.21.15.111:0/309222432 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc47c1054d0 con 0x7fc47c103ff0 2024-09-15T22:44:09.296 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.295+0000 7fc473400640 1 -- 172.21.15.111:0/309222432 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fc46c042030 con 0x7fc47c103ff0 2024-09-15T22:44:09.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.296+0000 7fc473400640 1 --2- 172.21.15.111:0/309222432 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc4600a3240 0x7fc4600a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:09.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.297+0000 7fc473400640 1 -- 172.21.15.111:0/309222432 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fc46c0efb40 con 0x7fc47c103ff0 2024-09-15T22:44:09.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.297+0000 7fc479e00640 1 --2- 172.21.15.111:0/309222432 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc4600a3240 0x7fc4600a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:09.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.297+0000 7fc479e00640 1 --2- 172.21.15.111:0/309222432 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc4600a3240 0x7fc4600a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc4680095f0 tx=0x7fc468005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:09.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.298+0000 7fc473400640 1 -- 172.21.15.111:0/309222432 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc46c0bcc60 con 0x7fc47c103ff0 2024-09-15T22:44:09.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.508+0000 7fc481400640 1 -- 172.21.15.111:0/309222432 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc47c10c300 con 0x7fc47c103ff0 2024-09-15T22:44:09.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.509+0000 7fc473400640 1 -- 172.21.15.111:0/309222432 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) ==== 84+0+65 (secure 0 0 0) 0x7fc46c0bc560 con 0x7fc47c103ff0 2024-09-15T22:44:09.511 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:44:09.511 INFO:teuthology.orchestra.run.smithi111.stdout:[ 2024-09-15T22:44:09.511 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T22:44:09.511 INFO:teuthology.orchestra.run.smithi111.stdout: "poolnum": 1, 2024-09-15T22:44:09.512 INFO:teuthology.orchestra.run.smithi111.stdout: "poolname": ".mgr" 2024-09-15T22:44:09.512 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T22:44:09.512 INFO:teuthology.orchestra.run.smithi111.stdout:] 2024-09-15T22:44:09.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.512+0000 7fc481400640 1 -- 172.21.15.111:0/309222432 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc4600a3240 msgr2=0x7fc4600a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:09.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.512+0000 7fc481400640 1 --2- 172.21.15.111:0/309222432 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc4600a3240 0x7fc4600a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc4680095f0 tx=0x7fc468005f90 comp rx=0 tx=0).stop 2024-09-15T22:44:09.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.512+0000 7fc481400640 1 -- 172.21.15.111:0/309222432 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc47c103ff0 msgr2=0x7fc47c19a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:09.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.512+0000 7fc481400640 1 --2- 172.21.15.111:0/309222432 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc47c103ff0 0x7fc47c19a5f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc46c006010 tx=0x7fc46c004040 comp rx=0 tx=0).stop 2024-09-15T22:44:09.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.512+0000 7fc481400640 1 -- 172.21.15.111:0/309222432 shutdown_connections 2024-09-15T22:44:09.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.512+0000 7fc481400640 1 --2- 172.21.15.111:0/309222432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc47c109120 0x7fc47c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:09.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.512+0000 7fc481400640 1 --2- 172.21.15.111:0/309222432 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc4600a3240 0x7fc4600a5700 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:09.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.512+0000 7fc481400640 1 --2- 172.21.15.111:0/309222432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc47c1049a0 0x7fc47c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:09.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.512+0000 7fc481400640 1 --2- 172.21.15.111:0/309222432 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc47c103ff0 0x7fc47c19a5f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:09.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.512+0000 7fc481400640 1 -- 172.21.15.111:0/309222432 >> 172.21.15.111:0/309222432 conn(0x7fc47c0ffa90 msgr2=0x7fc47c100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:09.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.512+0000 7fc481400640 1 -- 172.21.15.111:0/309222432 shutdown_connections 2024-09-15T22:44:09.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:09.512+0000 7fc481400640 1 -- 172.21.15.111:0/309222432 wait complete. 2024-09-15T22:44:09.515 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:09 smithi192 ceph-mon[35427]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:09.515 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:09 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:09.516 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:09 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:09.516 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:09 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:44:09.516 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:09 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:44:09.516 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:09 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:09.516 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:09 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:44:09.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:09 smithi121 ceph-mon[34815]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:09.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:09 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:09.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:09 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:09.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:09 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:44:09.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:09 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:44:09.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:09 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:09.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:09 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:44:09.758 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:09 smithi111 ceph-mon[30629]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:09.758 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:09 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:09.758 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:09 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:09.758 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:09 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:44:09.758 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:09 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:44:09.759 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:09 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:09.759 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:09 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:44:10.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:10 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/309222432' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-15T22:44:10.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:10 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:10.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:10 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:10.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:10 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:10.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:10 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:10.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:10 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/309222432' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-15T22:44:10.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:10 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:10.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:10 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:10.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:10 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:10.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:10 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:10.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:10 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/309222432' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-15T22:44:10.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:10 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:10.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:10 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:10.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:10 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:10.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:10 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:11.113 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-15T22:44:11.113 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-15T22:44:11.513 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:11.814 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:11 smithi111 ceph-mon[30629]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:11.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:11 smithi192 ceph-mon[35427]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:11.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:11 smithi121 ceph-mon[34815]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:12.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.121+0000 7f5af7600640 1 -- 172.21.15.111:0/635449000 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5af00745c0 msgr2=0x7f5af00749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:12.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.121+0000 7f5af7600640 1 --2- 172.21.15.111:0/635449000 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5af00745c0 0x7f5af00749a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009900 tx=0x7f5ae402a480 comp rx=0 tx=0).stop 2024-09-15T22:44:12.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.121+0000 7f5af7600640 1 -- 172.21.15.111:0/635449000 shutdown_connections 2024-09-15T22:44:12.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.121+0000 7f5af7600640 1 --2- 172.21.15.111:0/635449000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5af010e420 0x7f5af0110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.121+0000 7f5af7600640 1 --2- 172.21.15.111:0/635449000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5af0074ee0 0x7f5af010dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.121+0000 7f5af7600640 1 --2- 172.21.15.111:0/635449000 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5af00745c0 0x7f5af00749a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.121+0000 7f5af7600640 1 -- 172.21.15.111:0/635449000 >> 172.21.15.111:0/635449000 conn(0x7f5af00fd8e0 msgr2=0x7f5af00ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:12.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.121+0000 7f5af7600640 1 -- 172.21.15.111:0/635449000 shutdown_connections 2024-09-15T22:44:12.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.121+0000 7f5af7600640 1 -- 172.21.15.111:0/635449000 wait complete. 2024-09-15T22:44:12.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.122+0000 7f5af7600640 1 Processor -- start 2024-09-15T22:44:12.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.123+0000 7f5af7600640 1 -- start start 2024-09-15T22:44:12.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.123+0000 7f5af7600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5af00745c0 0x7f5af019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:12.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.123+0000 7f5af7600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5af0074ee0 0x7f5af019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:12.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.123+0000 7f5af7600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5af010e420 0x7f5af019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:12.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.123+0000 7f5af7600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01141e0 con 0x7f5af0074ee0 2024-09-15T22:44:12.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.123+0000 7f5af7600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5af0114060 con 0x7f5af010e420 2024-09-15T22:44:12.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.123+0000 7f5af7600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5af0114360 con 0x7f5af00745c0 2024-09-15T22:44:12.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.124+0000 7f5aefe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5af0074ee0 0x7f5af019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:12.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.124+0000 7f5af4c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5af00745c0 0x7f5af019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:12.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.124+0000 7f5aefe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5af0074ee0 0x7f5af019abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33738/0 (socket says 172.21.15.111:33738) 2024-09-15T22:44:12.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.124+0000 7f5aefe00640 1 -- 172.21.15.111:0/154303421 learned_addr learned my addr 172.21.15.111:0/154303421 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:44:12.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.124+0000 7f5af4c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5af00745c0 0x7f5af019a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:47128/0 (socket says 172.21.15.111:47128) 2024-09-15T22:44:12.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.124+0000 7f5af5600640 1 --2- 172.21.15.111:0/154303421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5af010e420 0x7f5af019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:12.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.124+0000 7f5af4c00640 1 -- 172.21.15.111:0/154303421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5af010e420 msgr2=0x7f5af019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:12.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.124+0000 7f5af4c00640 1 --2- 172.21.15.111:0/154303421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5af010e420 0x7f5af019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.124+0000 7f5af4c00640 1 -- 172.21.15.111:0/154303421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5af0074ee0 msgr2=0x7f5af019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:12.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.124+0000 7f5af4c00640 1 --2- 172.21.15.111:0/154303421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5af0074ee0 0x7f5af019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:12.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.124+0000 7f5af4c00640 1 -- 172.21.15.111:0/154303421 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af019f650 con 0x7f5af00745c0 2024-09-15T22:44:12.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.124+0000 7f5af4c00640 1 --2- 172.21.15.111:0/154303421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5af00745c0 0x7f5af019a6a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5ae402a9a0 tx=0x7f5ae4003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:12.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.125+0000 7f5aed600640 1 -- 172.21.15.111:0/154303421 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae403f070 con 0x7f5af00745c0 2024-09-15T22:44:12.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.125+0000 7f5aed600640 1 -- 172.21.15.111:0/154303421 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ae4031c90 con 0x7f5af00745c0 2024-09-15T22:44:12.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.125+0000 7f5aed600640 1 -- 172.21.15.111:0/154303421 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4035030 con 0x7f5af00745c0 2024-09-15T22:44:12.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.125+0000 7f5af7600640 1 -- 172.21.15.111:0/154303421 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af019f8e0 con 0x7f5af00745c0 2024-09-15T22:44:12.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.125+0000 7f5af7600640 1 -- 172.21.15.111:0/154303421 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af019fd70 con 0x7f5af00745c0 2024-09-15T22:44:12.128 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.126+0000 7f5af7600640 1 -- 172.21.15.111:0/154303421 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5af00749a0 con 0x7f5af00745c0 2024-09-15T22:44:12.129 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.128+0000 7f5aed600640 1 -- 172.21.15.111:0/154303421 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f5ae4042070 con 0x7f5af00745c0 2024-09-15T22:44:12.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.129+0000 7f5aed600640 1 --2- 172.21.15.111:0/154303421 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5ad80a32c0 0x7f5ad80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:12.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.129+0000 7f5aed600640 1 -- 172.21.15.111:0/154303421 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f5ae40efd70 con 0x7f5af00745c0 2024-09-15T22:44:12.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.130+0000 7f5aefe00640 1 --2- 172.21.15.111:0/154303421 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5ad80a32c0 0x7f5ad80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:12.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.130+0000 7f5aefe00640 1 --2- 172.21.15.111:0/154303421 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5ad80a32c0 0x7f5ad80a5780 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5af010f7b0 tx=0x7f5ae0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:12.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.134+0000 7f5aed600640 1 -- 172.21.15.111:0/154303421 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5ae40bc840 con 0x7f5af00745c0 2024-09-15T22:44:12.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:12.318+0000 7f5af7600640 1 -- 172.21.15.111:0/154303421 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5af0109600 con 0x7f5af00745c0 2024-09-15T22:44:13.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:13.154+0000 7f5aed600640 1 -- 172.21.15.111:0/154303421 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) ==== 138+0+0 (secure 0 0 0) 0x7f5ae40c0b40 con 0x7f5af00745c0 2024-09-15T22:44:13.157 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_metadata' created 2024-09-15T22:44:13.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:13.158+0000 7f5af7600640 1 -- 172.21.15.111:0/154303421 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5ad80a32c0 msgr2=0x7f5ad80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:13.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:13.158+0000 7f5af7600640 1 --2- 172.21.15.111:0/154303421 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5ad80a32c0 0x7f5ad80a5780 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5af010f7b0 tx=0x7f5ae0005f90 comp rx=0 tx=0).stop 2024-09-15T22:44:13.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:13.158+0000 7f5af7600640 1 -- 172.21.15.111:0/154303421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5af00745c0 msgr2=0x7f5af019a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:13.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:13.158+0000 7f5af7600640 1 --2- 172.21.15.111:0/154303421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5af00745c0 0x7f5af019a6a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5ae402a9a0 tx=0x7f5ae4003b20 comp rx=0 tx=0).stop 2024-09-15T22:44:13.160 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:13.159+0000 7f5af7600640 1 -- 172.21.15.111:0/154303421 shutdown_connections 2024-09-15T22:44:13.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:13.159+0000 7f5af7600640 1 --2- 172.21.15.111:0/154303421 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5ad80a32c0 0x7f5ad80a5780 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:13.159+0000 7f5af7600640 1 --2- 172.21.15.111:0/154303421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5af010e420 0x7f5af019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:13.159+0000 7f5af7600640 1 --2- 172.21.15.111:0/154303421 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5af0074ee0 0x7f5af019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:13.159+0000 7f5af7600640 1 --2- 172.21.15.111:0/154303421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5af00745c0 0x7f5af019a6a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:13.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:13.159+0000 7f5af7600640 1 -- 172.21.15.111:0/154303421 >> 172.21.15.111:0/154303421 conn(0x7f5af00fd8e0 msgr2=0x7f5af00fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:13.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:13.159+0000 7f5af7600640 1 -- 172.21.15.111:0/154303421 shutdown_connections 2024-09-15T22:44:13.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:13.159+0000 7f5af7600640 1 -- 172.21.15.111:0/154303421 wait complete. 2024-09-15T22:44:13.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:13 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:13.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:13 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:13.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:13 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/154303421' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-15T22:44:13.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:13 smithi192 ceph-mon[35427]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-15T22:44:13.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:13 smithi192 ceph-mon[35427]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:13.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:13 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:44:13.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:13 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:44:13.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:13 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:13.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:13 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:13.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:13 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:13.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:13 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/154303421' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-15T22:44:13.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:13 smithi111 ceph-mon[30629]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-15T22:44:13.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:13 smithi111 ceph-mon[30629]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:13.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:13 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:44:13.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:13 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:44:13.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:13 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:13.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:13 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:13.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:13 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:13.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:13 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/154303421' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-15T22:44:13.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:13 smithi121 ceph-mon[34815]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-15T22:44:13.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:13 smithi121 ceph-mon[34815]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:13.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:13 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:44:13.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:13 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:44:13.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:13 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:44:14.056 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-15T22:44:14.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:14 smithi111 ceph-mon[30629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-15T22:44:14.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:14 smithi111 ceph-mon[30629]: osdmap e80: 12 total, 12 up, 12 in 2024-09-15T22:44:14.434 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:14.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:14 smithi192 ceph-mon[35427]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-15T22:44:14.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:14 smithi192 ceph-mon[35427]: osdmap e80: 12 total, 12 up, 12 in 2024-09-15T22:44:14.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:14 smithi121 ceph-mon[34815]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-15T22:44:14.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:14 smithi121 ceph-mon[34815]: osdmap e80: 12 total, 12 up, 12 in 2024-09-15T22:44:14.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.996+0000 7faf65600640 1 -- 172.21.15.111:0/4098482494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf60103ff0 msgr2=0x7faf601043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:14.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.996+0000 7faf65600640 1 --2- 172.21.15.111:0/4098482494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf60103ff0 0x7faf601043d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7faf50009ca0 tx=0x7faf5002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:44:14.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.997+0000 7faf65600640 1 -- 172.21.15.111:0/4098482494 shutdown_connections 2024-09-15T22:44:14.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.997+0000 7faf65600640 1 --2- 172.21.15.111:0/4098482494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf60109120 0x7faf60110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:14.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.997+0000 7faf65600640 1 --2- 172.21.15.111:0/4098482494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf601049a0 0x7faf601089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:14.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.997+0000 7faf65600640 1 --2- 172.21.15.111:0/4098482494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf60103ff0 0x7faf601043d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:14.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.997+0000 7faf65600640 1 -- 172.21.15.111:0/4098482494 >> 172.21.15.111:0/4098482494 conn(0x7faf600ffa90 msgr2=0x7faf60101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:14.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.997+0000 7faf65600640 1 -- 172.21.15.111:0/4098482494 shutdown_connections 2024-09-15T22:44:14.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.997+0000 7faf65600640 1 -- 172.21.15.111:0/4098482494 wait complete. 2024-09-15T22:44:15.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.997+0000 7faf65600640 1 Processor -- start 2024-09-15T22:44:15.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.998+0000 7faf65600640 1 -- start start 2024-09-15T22:44:15.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.998+0000 7faf65600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf60103ff0 0x7faf6019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:15.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.998+0000 7faf65600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf601049a0 0x7faf6019ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:15.001 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.998+0000 7faf65600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf60109120 0x7faf6019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:15.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.998+0000 7faf65600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faf601141a0 con 0x7faf601049a0 2024-09-15T22:44:15.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.998+0000 7faf65600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faf60114020 con 0x7faf60109120 2024-09-15T22:44:15.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.998+0000 7faf65600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faf60114320 con 0x7faf60103ff0 2024-09-15T22:44:15.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.998+0000 7faf56000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf601049a0 0x7faf6019ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:15.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.998+0000 7faf56000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf601049a0 0x7faf6019ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33764/0 (socket says 172.21.15.111:33764) 2024-09-15T22:44:15.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.998+0000 7faf56000640 1 -- 172.21.15.111:0/788453030 learned_addr learned my addr 172.21.15.111:0/788453030 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:44:15.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.998+0000 7faf5f400640 1 --2- 172.21.15.111:0/788453030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf60109120 0x7faf6019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:15.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.998+0000 7faf5ea00640 1 --2- 172.21.15.111:0/788453030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf60103ff0 0x7faf6019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:15.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.999+0000 7faf5f400640 1 -- 172.21.15.111:0/788453030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf60103ff0 msgr2=0x7faf6019a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:15.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.999+0000 7faf5f400640 1 --2- 172.21.15.111:0/788453030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf60103ff0 0x7faf6019a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.999+0000 7faf5f400640 1 -- 172.21.15.111:0/788453030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf601049a0 msgr2=0x7faf6019ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:15.002 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.999+0000 7faf5f400640 1 --2- 172.21.15.111:0/788453030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf601049a0 0x7faf6019ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:15.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.999+0000 7faf5f400640 1 -- 172.21.15.111:0/788453030 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6019f690 con 0x7faf60109120 2024-09-15T22:44:15.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.999+0000 7faf5f400640 1 --2- 172.21.15.111:0/788453030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf60109120 0x7faf6019efb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faf5800e970 tx=0x7faf5800ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:15.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.999+0000 7faf57e00640 1 -- 172.21.15.111:0/788453030 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5800ccc0 con 0x7faf60109120 2024-09-15T22:44:15.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.999+0000 7faf65600640 1 -- 172.21.15.111:0/788453030 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf6019f980 con 0x7faf60109120 2024-09-15T22:44:15.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.999+0000 7faf57e00640 1 -- 172.21.15.111:0/788453030 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf58005020 con 0x7faf60109120 2024-09-15T22:44:15.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.999+0000 7faf57e00640 1 -- 172.21.15.111:0/788453030 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf58004a00 con 0x7faf60109120 2024-09-15T22:44:15.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:14.999+0000 7faf65600640 1 -- 172.21.15.111:0/788453030 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf601a7260 con 0x7faf60109120 2024-09-15T22:44:15.003 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:15.000+0000 7faf65600640 1 -- 172.21.15.111:0/788453030 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf601043d0 con 0x7faf60109120 2024-09-15T22:44:15.004 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:15.002+0000 7faf57e00640 1 -- 172.21.15.111:0/788453030 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7faf580051d0 con 0x7faf60109120 2024-09-15T22:44:15.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:15.004+0000 7faf57e00640 1 --2- 172.21.15.111:0/788453030 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faf440a3240 0x7faf440a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:15.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:15.004+0000 7faf57e00640 1 -- 172.21.15.111:0/788453030 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7497+0+0 (secure 0 0 0) 0x7faf580cf610 con 0x7faf60109120 2024-09-15T22:44:15.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:15.004+0000 7faf5ea00640 1 --2- 172.21.15.111:0/788453030 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faf440a3240 0x7faf440a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:15.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:15.004+0000 7faf5ea00640 1 --2- 172.21.15.111:0/788453030 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faf440a3240 0x7faf440a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faf50006010 tx=0x7faf50009820 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:15.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:15.006+0000 7faf57e00640 1 -- 172.21.15.111:0/788453030 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faf5809c650 con 0x7faf60109120 2024-09-15T22:44:15.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:15.185+0000 7faf65600640 1 -- 172.21.15.111:0/788453030 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7faf6019ba40 con 0x7faf60109120 2024-09-15T22:44:15.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:15 smithi192 ceph-mon[35427]: osdmap e81: 12 total, 12 up, 12 in 2024-09-15T22:44:15.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:15 smithi192 ceph-mon[35427]: pgmap v293: 33 pgs: 14 creating+peering, 18 unknown, 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:15.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:15 smithi121 ceph-mon[34815]: osdmap e81: 12 total, 12 up, 12 in 2024-09-15T22:44:15.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:15 smithi121 ceph-mon[34815]: pgmap v293: 33 pgs: 14 creating+peering, 18 unknown, 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:15.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:15 smithi111 ceph-mon[30629]: osdmap e81: 12 total, 12 up, 12 in 2024-09-15T22:44:15.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:15 smithi111 ceph-mon[30629]: pgmap v293: 33 pgs: 14 creating+peering, 18 unknown, 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:16.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:16.176+0000 7faf57e00640 1 -- 172.21.15.111:0/788453030 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) ==== 170+0+0 (secure 0 0 0) 0x7faf5809bf50 con 0x7faf60109120 2024-09-15T22:44:16.179 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_data' created 2024-09-15T22:44:16.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:16.182+0000 7faf65600640 1 -- 172.21.15.111:0/788453030 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faf440a3240 msgr2=0x7faf440a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:16.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:16.182+0000 7faf65600640 1 --2- 172.21.15.111:0/788453030 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faf440a3240 0x7faf440a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faf50006010 tx=0x7faf50009820 comp rx=0 tx=0).stop 2024-09-15T22:44:16.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:16.182+0000 7faf65600640 1 -- 172.21.15.111:0/788453030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf60109120 msgr2=0x7faf6019efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:16.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:16.182+0000 7faf65600640 1 --2- 172.21.15.111:0/788453030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf60109120 0x7faf6019efb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faf5800e970 tx=0x7faf5800ee30 comp rx=0 tx=0).stop 2024-09-15T22:44:16.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:16.182+0000 7faf65600640 1 -- 172.21.15.111:0/788453030 shutdown_connections 2024-09-15T22:44:16.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:16.182+0000 7faf65600640 1 --2- 172.21.15.111:0/788453030 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faf440a3240 0x7faf440a5700 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:16.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:16.182+0000 7faf65600640 1 --2- 172.21.15.111:0/788453030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf60109120 0x7faf6019efb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:16.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:16.182+0000 7faf65600640 1 --2- 172.21.15.111:0/788453030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf601049a0 0x7faf6019ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:16.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:16.182+0000 7faf65600640 1 --2- 172.21.15.111:0/788453030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf60103ff0 0x7faf6019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:16.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:16.182+0000 7faf65600640 1 -- 172.21.15.111:0/788453030 >> 172.21.15.111:0/788453030 conn(0x7faf600ffa90 msgr2=0x7faf60100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:16.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:16.182+0000 7faf65600640 1 -- 172.21.15.111:0/788453030 shutdown_connections 2024-09-15T22:44:16.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:16.182+0000 7faf65600640 1 -- 172.21.15.111:0/788453030 wait complete. 2024-09-15T22:44:16.380 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:16 smithi111 ceph-mon[30629]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-15T22:44:16.380 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:16 smithi111 ceph-mon[30629]: osdmap e82: 12 total, 12 up, 12 in 2024-09-15T22:44:16.380 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:16 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/788453030' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-15T22:44:16.380 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:16 smithi111 ceph-mon[30629]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-15T22:44:16.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:16 smithi192 ceph-mon[35427]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-15T22:44:16.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:16 smithi192 ceph-mon[35427]: osdmap e82: 12 total, 12 up, 12 in 2024-09-15T22:44:16.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:16 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/788453030' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-15T22:44:16.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:16 smithi192 ceph-mon[35427]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-15T22:44:16.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:16 smithi121 ceph-mon[34815]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-15T22:44:16.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:16 smithi121 ceph-mon[34815]: osdmap e82: 12 total, 12 up, 12 in 2024-09-15T22:44:16.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:16 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/788453030' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-15T22:44:16.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:16 smithi121 ceph-mon[34815]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-15T22:44:17.234 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-15T22:44:17.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:17 smithi121 ceph-mon[34815]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-15T22:44:17.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:17 smithi121 ceph-mon[34815]: osdmap e83: 12 total, 12 up, 12 in 2024-09-15T22:44:17.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:17 smithi121 ceph-mon[34815]: pgmap v296: 97 pgs: 13 active+clean, 14 creating+peering, 70 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:17.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:17 smithi192 ceph-mon[35427]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-15T22:44:17.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:17 smithi192 ceph-mon[35427]: osdmap e83: 12 total, 12 up, 12 in 2024-09-15T22:44:17.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:17 smithi192 ceph-mon[35427]: pgmap v296: 97 pgs: 13 active+clean, 14 creating+peering, 70 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:17.603 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:17.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:17 smithi111 ceph-mon[30629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-15T22:44:17.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:17 smithi111 ceph-mon[30629]: osdmap e83: 12 total, 12 up, 12 in 2024-09-15T22:44:17.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:17 smithi111 ceph-mon[30629]: pgmap v296: 97 pgs: 13 active+clean, 14 creating+peering, 70 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:18.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.215+0000 7f2156200640 1 -- 172.21.15.111:0/3667378990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2150108f60 msgr2=0x7f2150110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:18.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.215+0000 7f2156200640 1 --2- 172.21.15.111:0/3667378990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2150108f60 0x7f2150110ae0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2148009ca0 tx=0x7f214802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.215+0000 7f2156200640 1 -- 172.21.15.111:0/3667378990 shutdown_connections 2024-09-15T22:44:18.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.215+0000 7f2156200640 1 --2- 172.21.15.111:0/3667378990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2150108f60 0x7f2150110ae0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.215+0000 7f2156200640 1 --2- 172.21.15.111:0/3667378990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21501047e0 0x7f2150108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.215+0000 7f2156200640 1 --2- 172.21.15.111:0/3667378990 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2150103e30 0x7f2150104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.215+0000 7f2156200640 1 -- 172.21.15.111:0/3667378990 >> 172.21.15.111:0/3667378990 conn(0x7f21500ff8d0 msgr2=0x7f2150101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:18.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.216+0000 7f2156200640 1 -- 172.21.15.111:0/3667378990 shutdown_connections 2024-09-15T22:44:18.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.216+0000 7f2156200640 1 -- 172.21.15.111:0/3667378990 wait complete. 2024-09-15T22:44:18.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.216+0000 7f2156200640 1 Processor -- start 2024-09-15T22:44:18.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.217+0000 7f2156200640 1 -- start start 2024-09-15T22:44:18.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.217+0000 7f2156200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2150103e30 0x7f215019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:18.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.217+0000 7f2156200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21501047e0 0x7f215019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:18.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.217+0000 7f2156200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2150108f60 0x7f215019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:18.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.217+0000 7f2156200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2150113fc0 con 0x7f2150103e30 2024-09-15T22:44:18.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.217+0000 7f2156200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2150113e40 con 0x7f2150108f60 2024-09-15T22:44:18.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.217+0000 7f2156200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2150114140 con 0x7f21501047e0 2024-09-15T22:44:18.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.217+0000 7f2154e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2150103e30 0x7f215019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:18.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.217+0000 7f2154e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2150103e30 0x7f215019a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33786/0 (socket says 172.21.15.111:33786) 2024-09-15T22:44:18.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.217+0000 7f2154e00640 1 -- 172.21.15.111:0/82864951 learned_addr learned my addr 172.21.15.111:0/82864951 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:44:18.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.217+0000 7f2155800640 1 --2- 172.21.15.111:0/82864951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2150108f60 0x7f215019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:18.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.217+0000 7f2147e00640 1 --2- 172.21.15.111:0/82864951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21501047e0 0x7f215019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:18.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.218+0000 7f2154e00640 1 -- 172.21.15.111:0/82864951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21501047e0 msgr2=0x7f215019a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:18.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.218+0000 7f2154e00640 1 --2- 172.21.15.111:0/82864951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21501047e0 0x7f215019a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.218+0000 7f2154e00640 1 -- 172.21.15.111:0/82864951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2150108f60 msgr2=0x7f215019ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:18.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.218+0000 7f2154e00640 1 --2- 172.21.15.111:0/82864951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2150108f60 0x7f215019ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:18.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.218+0000 7f2154e00640 1 -- 172.21.15.111:0/82864951 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215019f3d0 con 0x7f2150103e30 2024-09-15T22:44:18.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.218+0000 7f2154e00640 1 --2- 172.21.15.111:0/82864951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2150103e30 0x7f215019a420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f214000ea60 tx=0x7f214000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:18.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.218+0000 7f214e000640 1 -- 172.21.15.111:0/82864951 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214000cde0 con 0x7f2150103e30 2024-09-15T22:44:18.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.218+0000 7f214e000640 1 -- 172.21.15.111:0/82864951 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21400035f0 con 0x7f2150103e30 2024-09-15T22:44:18.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.218+0000 7f2156200640 1 -- 172.21.15.111:0/82864951 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f215019f6c0 con 0x7f2150103e30 2024-09-15T22:44:18.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.218+0000 7f214e000640 1 -- 172.21.15.111:0/82864951 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2140010670 con 0x7f2150103e30 2024-09-15T22:44:18.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.218+0000 7f2156200640 1 -- 172.21.15.111:0/82864951 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21501a6fa0 con 0x7f2150103e30 2024-09-15T22:44:18.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.220+0000 7f2156200640 1 -- 172.21.15.111:0/82864951 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2150104210 con 0x7f2150103e30 2024-09-15T22:44:18.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.222+0000 7f214e000640 1 -- 172.21.15.111:0/82864951 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f21400037a0 con 0x7f2150103e30 2024-09-15T22:44:18.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.223+0000 7f214e000640 1 --2- 172.21.15.111:0/82864951 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f21340a3240 0x7f21340a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:18.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.223+0000 7f214e000640 1 -- 172.21.15.111:0/82864951 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7879+0+0 (secure 0 0 0) 0x7f21400cfba0 con 0x7f2150103e30 2024-09-15T22:44:18.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.223+0000 7f2147e00640 1 --2- 172.21.15.111:0/82864951 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f21340a3240 0x7f21340a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:18.225 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.224+0000 7f2147e00640 1 --2- 172.21.15.111:0/82864951 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f21340a3240 0x7f21340a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f215010a930 tx=0x7f2138005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:18.226 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.225+0000 7f214e000640 1 -- 172.21.15.111:0/82864951 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f214009ca60 con 0x7f2150103e30 2024-09-15T22:44:18.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:18 smithi111 ceph-mon[30629]: osdmap e84: 12 total, 12 up, 12 in 2024-09-15T22:44:18.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:18.434+0000 7f2156200640 1 -- 172.21.15.111:0/82864951 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f215019b780 con 0x7f2150103e30 2024-09-15T22:44:18.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:18 smithi192 ceph-mon[35427]: osdmap e84: 12 total, 12 up, 12 in 2024-09-15T22:44:18.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:18 smithi121 ceph-mon[34815]: osdmap e84: 12 total, 12 up, 12 in 2024-09-15T22:44:19.200 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:19.197+0000 7f214e000640 1 -- 172.21.15.111:0/82864951 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f214009c360 con 0x7f2150103e30 2024-09-15T22:44:19.200 INFO:teuthology.orchestra.run.smithi111.stderr:new fs with metadata pool 2 and data pool 3 2024-09-15T22:44:19.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:19.200+0000 7f2156200640 1 -- 172.21.15.111:0/82864951 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f21340a3240 msgr2=0x7f21340a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:19.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:19.200+0000 7f2156200640 1 --2- 172.21.15.111:0/82864951 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f21340a3240 0x7f21340a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f215010a930 tx=0x7f2138005f90 comp rx=0 tx=0).stop 2024-09-15T22:44:19.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:19.200+0000 7f2156200640 1 -- 172.21.15.111:0/82864951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2150103e30 msgr2=0x7f215019a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:19.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:19.200+0000 7f2156200640 1 --2- 172.21.15.111:0/82864951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2150103e30 0x7f215019a420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f214000ea60 tx=0x7f214000ef20 comp rx=0 tx=0).stop 2024-09-15T22:44:19.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:19.200+0000 7f2156200640 1 -- 172.21.15.111:0/82864951 shutdown_connections 2024-09-15T22:44:19.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:19.200+0000 7f2156200640 1 --2- 172.21.15.111:0/82864951 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f21340a3240 0x7f21340a5700 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:19.200+0000 7f2156200640 1 --2- 172.21.15.111:0/82864951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2150108f60 0x7f215019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:19.200+0000 7f2156200640 1 --2- 172.21.15.111:0/82864951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21501047e0 0x7f215019a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:19.200+0000 7f2156200640 1 --2- 172.21.15.111:0/82864951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2150103e30 0x7f215019a420 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:19.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:19.201+0000 7f2156200640 1 -- 172.21.15.111:0/82864951 >> 172.21.15.111:0/82864951 conn(0x7f21500ff8d0 msgr2=0x7f2150100b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:19.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:19.201+0000 7f2156200640 1 -- 172.21.15.111:0/82864951 shutdown_connections 2024-09-15T22:44:19.203 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:19.201+0000 7f2156200640 1 -- 172.21.15.111:0/82864951 wait complete. 2024-09-15T22:44:19.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:19 smithi111 ceph-mon[30629]: osdmap e85: 12 total, 12 up, 12 in 2024-09-15T22:44:19.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:19 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/82864951' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-15T22:44:19.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:19 smithi111 ceph-mon[30629]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-15T22:44:19.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:19 smithi111 ceph-mon[30629]: pgmap v299: 97 pgs: 33 active+clean, 26 creating+peering, 38 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:19.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:19 smithi192 ceph-mon[35427]: osdmap e85: 12 total, 12 up, 12 in 2024-09-15T22:44:19.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:19 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/82864951' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-15T22:44:19.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:19 smithi192 ceph-mon[35427]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-15T22:44:19.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:19 smithi192 ceph-mon[35427]: pgmap v299: 97 pgs: 33 active+clean, 26 creating+peering, 38 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:19.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:19 smithi121 ceph-mon[34815]: osdmap e85: 12 total, 12 up, 12 in 2024-09-15T22:44:19.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:19 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/82864951' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-15T22:44:19.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:19 smithi121 ceph-mon[34815]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-15T22:44:19.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:19 smithi121 ceph-mon[34815]: pgmap v299: 97 pgs: 33 active+clean, 26 creating+peering, 38 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:20.137 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:44:20.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:20 smithi111 ceph-mon[30629]: osdmap e86: 12 total, 12 up, 12 in 2024-09-15T22:44:20.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:20 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/82864951' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-15T22:44:20.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:20 smithi111 ceph-mon[30629]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-15T22:44:20.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:20 smithi111 ceph-mon[30629]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-15T22:44:20.508 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:20.544 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:20 smithi121 ceph-mon[34815]: osdmap e86: 12 total, 12 up, 12 in 2024-09-15T22:44:20.545 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:20 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/82864951' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-15T22:44:20.545 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:20 smithi121 ceph-mon[34815]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-15T22:44:20.545 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:20 smithi121 ceph-mon[34815]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-15T22:44:20.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:20 smithi192 ceph-mon[35427]: osdmap e86: 12 total, 12 up, 12 in 2024-09-15T22:44:20.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:20 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/82864951' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-15T22:44:20.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:20 smithi192 ceph-mon[35427]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-15T22:44:20.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:20 smithi192 ceph-mon[35427]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-15T22:44:20.890 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 15 22:44:20 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-b[65483]: 2024-09-15T22:44:20.542+0000 7f3d4aeb1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-15T22:44:21.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.088+0000 7f3df5600640 1 -- 172.21.15.111:0/110989568 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3df0074620 msgr2=0x7f3df0074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:21.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.088+0000 7f3df5600640 1 --2- 172.21.15.111:0/110989568 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3df0074620 0x7f3df0074a00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009ca0 tx=0x7f3de002a950 comp rx=0 tx=0).stop 2024-09-15T22:44:21.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.089+0000 7f3df5600640 1 -- 172.21.15.111:0/110989568 shutdown_connections 2024-09-15T22:44:21.091 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.089+0000 7f3df5600640 1 --2- 172.21.15.111:0/110989568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3df010e3c0 0x7f3df0110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:21.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.089+0000 7f3df5600640 1 --2- 172.21.15.111:0/110989568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3df0074f40 0x7f3df010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:21.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.089+0000 7f3df5600640 1 --2- 172.21.15.111:0/110989568 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3df0074620 0x7f3df0074a00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:21.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.089+0000 7f3df5600640 1 -- 172.21.15.111:0/110989568 >> 172.21.15.111:0/110989568 conn(0x7f3df00fd8c0 msgr2=0x7f3df00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:21.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.089+0000 7f3df5600640 1 -- 172.21.15.111:0/110989568 shutdown_connections 2024-09-15T22:44:21.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.089+0000 7f3df5600640 1 -- 172.21.15.111:0/110989568 wait complete. 2024-09-15T22:44:21.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.090+0000 7f3df5600640 1 Processor -- start 2024-09-15T22:44:21.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.090+0000 7f3df5600640 1 -- start start 2024-09-15T22:44:21.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.090+0000 7f3df5600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3df0074620 0x7f3df019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:21.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.090+0000 7f3df5600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3df0074f40 0x7f3df019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:21.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.090+0000 7f3df5600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3df010e3c0 0x7f3df019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:21.094 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.090+0000 7f3df5600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01141c0 con 0x7f3df0074620 2024-09-15T22:44:21.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.090+0000 7f3df5600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3df0114040 con 0x7f3df010e3c0 2024-09-15T22:44:21.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.090+0000 7f3df5600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3df0114340 con 0x7f3df0074f40 2024-09-15T22:44:21.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.090+0000 7f3deea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3df0074620 0x7f3df019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:21.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.090+0000 7f3deea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3df0074620 0x7f3df019a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47842/0 (socket says 172.21.15.111:47842) 2024-09-15T22:44:21.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.090+0000 7f3deea00640 1 -- 172.21.15.111:0/2852559816 learned_addr learned my addr 172.21.15.111:0/2852559816 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:44:21.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.090+0000 7f3dee000640 1 --2- 172.21.15.111:0/2852559816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3df0074f40 0x7f3df019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:21.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.090+0000 7f3def400640 1 --2- 172.21.15.111:0/2852559816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3df010e3c0 0x7f3df019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:21.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.091+0000 7f3deea00640 1 -- 172.21.15.111:0/2852559816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3df0074f40 msgr2=0x7f3df019ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:21.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.091+0000 7f3deea00640 1 --2- 172.21.15.111:0/2852559816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3df0074f40 0x7f3df019ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:21.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.091+0000 7f3deea00640 1 -- 172.21.15.111:0/2852559816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3df010e3c0 msgr2=0x7f3df019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:21.095 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.091+0000 7f3deea00640 1 --2- 172.21.15.111:0/2852559816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3df010e3c0 0x7f3df019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:21.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.091+0000 7f3deea00640 1 -- 172.21.15.111:0/2852559816 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df019f6d0 con 0x7f3df0074620 2024-09-15T22:44:21.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.091+0000 7f3deea00640 1 --2- 172.21.15.111:0/2852559816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3df0074620 0x7f3df019a720 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3de0005e30 tx=0x7f3de0003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:21.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.091+0000 7f3de7400640 1 -- 172.21.15.111:0/2852559816 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de0034040 con 0x7f3df0074620 2024-09-15T22:44:21.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.091+0000 7f3df5600640 1 -- 172.21.15.111:0/2852559816 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df019f960 con 0x7f3df0074620 2024-09-15T22:44:21.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.091+0000 7f3de7400640 1 -- 172.21.15.111:0/2852559816 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3de002fe20 con 0x7f3df0074620 2024-09-15T22:44:21.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.092+0000 7f3df5600640 1 -- 172.21.15.111:0/2852559816 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df01a72a0 con 0x7f3df0074620 2024-09-15T22:44:21.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.092+0000 7f3de7400640 1 -- 172.21.15.111:0/2852559816 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de00046b0 con 0x7f3df0074620 2024-09-15T22:44:21.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.093+0000 7f3df5600640 1 -- 172.21.15.111:0/2852559816 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3df0074a00 con 0x7f3df0074620 2024-09-15T22:44:21.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.094+0000 7f3de7400640 1 -- 172.21.15.111:0/2852559816 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f3de0037020 con 0x7f3df0074620 2024-09-15T22:44:21.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.096+0000 7f3de7400640 1 --2- 172.21.15.111:0/2852559816 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3dd40a3240 0x7f3dd40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:21.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.096+0000 7f3de7400640 1 -- 172.21.15.111:0/2852559816 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f3de00f0720 con 0x7f3df0074620 2024-09-15T22:44:21.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.096+0000 7f3dee000640 1 --2- 172.21.15.111:0/2852559816 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3dd40a3240 0x7f3dd40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:21.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.097+0000 7f3dee000640 1 --2- 172.21.15.111:0/2852559816 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3dd40a3240 0x7f3dd40a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3df00ffe70 tx=0x7f3ddc005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:21.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.100+0000 7f3de7400640 1 -- 172.21.15.111:0/2852559816 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3de00bd4f0 con 0x7f3df0074620 2024-09-15T22:44:21.281 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.279+0000 7f3df5600640 1 -- 172.21.15.111:0/2852559816 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3df0109540 con 0x7f3df0074620 2024-09-15T22:44:21.282 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:21 smithi111 ceph-mon[30629]: mds.? [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] up:active 2024-09-15T22:44:21.282 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:21 smithi111 ceph-mon[30629]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:44:21.282 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:21 smithi111 ceph-mon[30629]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:44:21.282 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:21 smithi111 ceph-mon[30629]: pgmap v301: 97 pgs: 66 active+clean, 12 creating+peering, 19 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:21.282 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.281+0000 7f3de7400640 1 -- 172.21.15.111:0/2852559816 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22363 (secure 0 0 0) 0x7f3de00bcdf0 con 0x7f3df0074620 2024-09-15T22:44:21.284 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:44:21.284 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":86,"fsid":"b7614d3c-73b2-11ef-bceb-c7b262605968","created":"2024-09-15T22:35:52.571473+0000","modified":"2024-09-15T22:44:18.436245+0000","last_up_change":"2024-09-15T22:43:19.567274+0000","last_in_change":"2024-09-15T22:42:56.994246+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:39:18.954207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:44:12.846418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:44:15.685946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04f39604-be0e-4261-b316-67fa087c6666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6803","nonce":1386823446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6805","nonce":1386823446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6809","nonce":1386823446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6807","nonce":1386823446}]},"public_addr":"172.21.15.111:6803/1386823446","cluster_addr":"172.21.15.111:6805/1386823446","heartbeat_back_addr":"172.21.15.111:6809/1386823446","heartbeat_front_addr":"172.21.15.111:6807/1386823446","state":["exists","up"]},{"osd":1,"uuid":"1bdbef6e-23fa-4618-a788-66e57741a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6801","nonce":2051512291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6803","nonce":2051512291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6807","nonce":2051512291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6805","nonce":2051512291}]},"public_addr":"172.21.15.121:6801/2051512291","cluster_addr":"172.21.15.121:6803/2051512291","heartbeat_back_addr":"172.21.15.121:6807/2051512291","heartbeat_front_addr":"172.21.15.121:6805/2051512291","state":["exists","up"]},{"osd":2,"uuid":"46735ccf-6d0e-4154-bb55-422929819bfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6801","nonce":93085639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6803","nonce":93085639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6807","nonce":93085639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6805","nonce":93085639}]},"public_addr":"172.21.15.192:6801/93085639","cluster_addr":"172.21.15.192:6803/93085639","heartbeat_back_addr":"172.21.15.192:6807/93085639","heartbeat_front_addr":"172.21.15.192:6805/93085639","state":["exists","up"]},{"osd":3,"uuid":"4b243af8-c958-40b1-ba24-9c7ace00c4a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6811","nonce":2578259041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6813","nonce":2578259041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6817","nonce":2578259041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6815","nonce":2578259041}]},"public_addr":"172.21.15.111:6811/2578259041","cluster_addr":"172.21.15.111:6813/2578259041","heartbeat_back_addr":"172.21.15.111:6817/2578259041","heartbeat_front_addr":"172.21.15.111:6815/2578259041","state":["exists","up"]},{"osd":4,"uuid":"49f27541-9cbf-48e0-a466-6bb0fa5410b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6809","nonce":2667595915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6811","nonce":2667595915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6815","nonce":2667595915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6813","nonce":2667595915}]},"public_addr":"172.21.15.121:6809/2667595915","cluster_addr":"172.21.15.121:6811/2667595915","heartbeat_back_addr":"172.21.15.121:6815/2667595915","heartbeat_front_addr":"172.21.15.121:6813/2667595915","state":["exists","up"]},{"osd":5,"uuid":"9290e769-5470-44d3-8d24-25093af34335","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6809","nonce":1958639856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6811","nonce":1958639856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6815","nonce":1958639856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6813","nonce":1958639856}]},"public_addr":"172.21.15.192:6809/1958639856","cluster_addr":"172.21.15.192:6811/1958639856","heartbeat_back_addr":"172.21.15.192:6815/1958639856","heartbeat_front_addr":"172.21.15.192:6813/1958639856","state":["exists","up"]},{"osd":6,"uuid":"18911b97-1c68-4e8f-92ec-35d2ff0b5f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6819","nonce":1371543706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6821","nonce":1371543706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6825","nonce":1371543706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6823","nonce":1371543706}]},"public_addr":"172.21.15.111:6819/1371543706","cluster_addr":"172.21.15.111:6821/1371543706","heartbeat_back_addr":"172.21.15.111:6825/1371543706","heartbeat_front_addr":"172.21.15.111:6823/1371543706","state":["exists","up"]},{"osd":7,"uuid":"470647c5-c00d-426b-a134-cca235902307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6817","nonce":3501842998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6819","nonce":3501842998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6823","nonce":3501842998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6821","nonce":3501842998}]},"public_addr":"172.21.15.121:6817/3501842998","cluster_addr":"172.21.15.121:6819/3501842998","heartbeat_back_addr":"172.21.15.121:6823/3501842998","heartbeat_front_addr":"172.21.15.121:6821/3501842998","state":["exists","up"]},{"osd":8,"uuid":"fa2d3dd5-a768-4ada-a036-4c596b76d654","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6817","nonce":2417771374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6819","nonce":2417771374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6823","nonce":2417771374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6821","nonce":2417771374}]},"public_addr":"172.21.15.192:6817/2417771374","cluster_addr":"172.21.15.192:6819/2417771374","heartbeat_back_addr":"172.21.15.192:6823/2417771374","heartbeat_front_addr":"172.21.15.192:6821/2417771374","state":["exists","up"]},{"osd":9,"uuid":"71b4f5fb-1c02-461b-a257-c6e48381d768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6827","nonce":3979119431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6829","nonce":3979119431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6833","nonce":3979119431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6831","nonce":3979119431}]},"public_addr":"172.21.15.111:6827/3979119431","cluster_addr":"172.21.15.111:6829/3979119431","heartbeat_back_addr":"172.21.15.111:6833/3979119431","heartbeat_front_addr":"172.21.15.111:6831/3979119431","state":["exists","up"]},{"osd":10,"uuid":"e6a39e77-fce3-4e22-bf66-a677c9f8b6bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6825","nonce":2897902637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6827","nonce":2897902637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6831","nonce":2897902637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6829","nonce":2897902637}]},"public_addr":"172.21.15.121:6825/2897902637","cluster_addr":"172.21.15.121:6827/2897902637","heartbeat_back_addr":"172.21.15.121:6831/2897902637","heartbeat_front_addr":"172.21.15.121:6829/2897902637","state":["exists","up"]},{"osd":11,"uuid":"411a7921-6717-4cef-b8df-d9e252edcc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6825","nonce":2431464812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6827","nonce":2431464812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6831","nonce":2431464812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6829","nonce":2431464812}]},"public_addr":"172.21.15.192:6825/2431464812","cluster_addr":"172.21.15.192:6827/2431464812","heartbeat_back_addr":"172.21.15.192:6831/2431464812","heartbeat_front_addr":"172.21.15.192:6829/2431464812","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:38:23.442715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:38:49.501124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:39:14.759077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:39:45.121397+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:40:11.137612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:40:36.278567+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:07.027190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:32.439215+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:57.058702+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:42:28.741846+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:42:53.395442+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:43:18.265067+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:6801/2107775445":"2024-09-16T22:36:50.357551+0000","172.21.15.111:6800/2107775445":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/2394746540":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/4140020843":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/2114354263":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/1054173642":"2024-09-16T22:36:22.652875+0000","172.21.15.111:0/3408238004":"2024-09-16T22:36:22.652875+0000","172.21.15.111:0/2879044772":"2024-09-16T22:36:22.652875+0000","172.21.15.111:6801/3350425104":"2024-09-16T22:36:22.652875+0000","172.21.15.111:6800/3350425104":"2024-09-16T22:36:22.652875+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:44:21.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.283+0000 7f3df5600640 1 -- 172.21.15.111:0/2852559816 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3dd40a3240 msgr2=0x7f3dd40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:21.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.283+0000 7f3df5600640 1 --2- 172.21.15.111:0/2852559816 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3dd40a3240 0x7f3dd40a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3df00ffe70 tx=0x7f3ddc005f90 comp rx=0 tx=0).stop 2024-09-15T22:44:21.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.283+0000 7f3df5600640 1 -- 172.21.15.111:0/2852559816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3df0074620 msgr2=0x7f3df019a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:21.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.283+0000 7f3df5600640 1 --2- 172.21.15.111:0/2852559816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3df0074620 0x7f3df019a720 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3de0005e30 tx=0x7f3de0003c90 comp rx=0 tx=0).stop 2024-09-15T22:44:21.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.283+0000 7f3df5600640 1 -- 172.21.15.111:0/2852559816 shutdown_connections 2024-09-15T22:44:21.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.283+0000 7f3df5600640 1 --2- 172.21.15.111:0/2852559816 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3dd40a3240 0x7f3dd40a5700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:21.285 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.283+0000 7f3df5600640 1 --2- 172.21.15.111:0/2852559816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3df010e3c0 0x7f3df019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:21.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.283+0000 7f3df5600640 1 --2- 172.21.15.111:0/2852559816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3df0074f40 0x7f3df019ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:21.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.283+0000 7f3df5600640 1 --2- 172.21.15.111:0/2852559816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3df0074620 0x7f3df019a720 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:21.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.283+0000 7f3df5600640 1 -- 172.21.15.111:0/2852559816 >> 172.21.15.111:0/2852559816 conn(0x7f3df00fd8c0 msgr2=0x7f3df00fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:21.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.284+0000 7f3df5600640 1 -- 172.21.15.111:0/2852559816 shutdown_connections 2024-09-15T22:44:21.286 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:21.284+0000 7f3df5600640 1 -- 172.21.15.111:0/2852559816 wait complete. 2024-09-15T22:44:21.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:21 smithi192 ceph-mon[35427]: mds.? [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] up:active 2024-09-15T22:44:21.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:21 smithi192 ceph-mon[35427]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:44:21.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:21 smithi192 ceph-mon[35427]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:44:21.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:21 smithi192 ceph-mon[35427]: pgmap v301: 97 pgs: 66 active+clean, 12 creating+peering, 19 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:21.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:21 smithi121 ceph-mon[34815]: mds.? [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] up:active 2024-09-15T22:44:21.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:21 smithi121 ceph-mon[34815]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:44:21.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:21 smithi121 ceph-mon[34815]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:44:21.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:21 smithi121 ceph-mon[34815]: pgmap v301: 97 pgs: 66 active+clean, 12 creating+peering, 19 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:44:22.025 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:44:22.403 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:22.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:22 smithi111 ceph-mon[30629]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-15T22:44:22.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:22 smithi111 ceph-mon[30629]: Cluster is now healthy 2024-09-15T22:44:22.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:22 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/2852559816' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:44:22.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:22 smithi192 ceph-mon[35427]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-15T22:44:22.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:22 smithi192 ceph-mon[35427]: Cluster is now healthy 2024-09-15T22:44:22.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:22 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/2852559816' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:44:22.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:22 smithi121 ceph-mon[34815]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-15T22:44:22.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:22 smithi121 ceph-mon[34815]: Cluster is now healthy 2024-09-15T22:44:22.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:22 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/2852559816' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:44:23.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:22.999+0000 7f4633c00640 1 -- 172.21.15.111:0/2296175577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f462c101de0 msgr2=0x7f462c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:23.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:22.999+0000 7f4633c00640 1 --2- 172.21.15.111:0/2296175577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f462c101de0 0x7f462c1021c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f461c009ca0 tx=0x7f461c02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:44:23.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.000+0000 7f4633c00640 1 -- 172.21.15.111:0/2296175577 shutdown_connections 2024-09-15T22:44:23.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.000+0000 7f4633c00640 1 --2- 172.21.15.111:0/2296175577 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f462c106f10 0x7f462c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:23.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.000+0000 7f4633c00640 1 --2- 172.21.15.111:0/2296175577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f462c102790 0x7f462c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:23.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.000+0000 7f4633c00640 1 --2- 172.21.15.111:0/2296175577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f462c101de0 0x7f462c1021c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:23.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.000+0000 7f4633c00640 1 -- 172.21.15.111:0/2296175577 >> 172.21.15.111:0/2296175577 conn(0x7f462c0fd8c0 msgr2=0x7f462c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:23.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.000+0000 7f4633c00640 1 -- 172.21.15.111:0/2296175577 shutdown_connections 2024-09-15T22:44:23.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.000+0000 7f4633c00640 1 -- 172.21.15.111:0/2296175577 wait complete. 2024-09-15T22:44:23.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.001+0000 7f4633c00640 1 Processor -- start 2024-09-15T22:44:23.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.001+0000 7f4633c00640 1 -- start start 2024-09-15T22:44:23.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.001+0000 7f4633c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f462c101de0 0x7f462c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:23.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.001+0000 7f4633c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f462c102790 0x7f462c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:23.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.001+0000 7f4633c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f462c106f10 0x7f462c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:23.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.001+0000 7f4633c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f462c111f30 con 0x7f462c101de0 2024-09-15T22:44:23.006 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.001+0000 7f4633c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f462c111db0 con 0x7f462c106f10 2024-09-15T22:44:23.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.001+0000 7f4633c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1120b0 con 0x7f462c102790 2024-09-15T22:44:23.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.001+0000 7f4631200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f462c101de0 0x7f462c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:23.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.001+0000 7f4631200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f462c101de0 0x7f462c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47860/0 (socket says 172.21.15.111:47860) 2024-09-15T22:44:23.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.001+0000 7f4631c00640 1 --2- 172.21.15.111:0/781239066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f462c106f10 0x7f462c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:23.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.001+0000 7f4631200640 1 -- 172.21.15.111:0/781239066 learned_addr learned my addr 172.21.15.111:0/781239066 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:44:23.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.001+0000 7f462be00640 1 --2- 172.21.15.111:0/781239066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f462c102790 0x7f462c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:23.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.002+0000 7f4631c00640 1 -- 172.21.15.111:0/781239066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f462c102790 msgr2=0x7f462c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:23.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.002+0000 7f4631c00640 1 --2- 172.21.15.111:0/781239066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f462c102790 0x7f462c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:23.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.002+0000 7f4631c00640 1 -- 172.21.15.111:0/781239066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f462c101de0 msgr2=0x7f462c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:23.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.002+0000 7f4631c00640 1 --2- 172.21.15.111:0/781239066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f462c101de0 0x7f462c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:23.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.002+0000 7f4631c00640 1 -- 172.21.15.111:0/781239066 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462c1a6970 con 0x7f462c106f10 2024-09-15T22:44:23.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.002+0000 7f4631200640 1 --2- 172.21.15.111:0/781239066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f462c101de0 0x7f462c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:44:23.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.002+0000 7f4631c00640 1 --2- 172.21.15.111:0/781239066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f462c106f10 0x7f462c074f70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f462400e970 tx=0x7f462400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:23.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.002+0000 7f4629600640 1 -- 172.21.15.111:0/781239066 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462400ccc0 con 0x7f462c106f10 2024-09-15T22:44:23.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.002+0000 7f4629600640 1 -- 172.21.15.111:0/781239066 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4624005020 con 0x7f462c106f10 2024-09-15T22:44:23.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.002+0000 7f4629600640 1 -- 172.21.15.111:0/781239066 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4624004a00 con 0x7f462c106f10 2024-09-15T22:44:23.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.002+0000 7f4633c00640 1 -- 172.21.15.111:0/781239066 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f462c1a6c60 con 0x7f462c106f10 2024-09-15T22:44:23.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.003+0000 7f4633c00640 1 -- 172.21.15.111:0/781239066 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f462c1a7150 con 0x7f462c106f10 2024-09-15T22:44:23.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.004+0000 7f4633c00640 1 -- 172.21.15.111:0/781239066 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f462c1021c0 con 0x7f462c106f10 2024-09-15T22:44:23.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.006+0000 7f4629600640 1 -- 172.21.15.111:0/781239066 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f46240051d0 con 0x7f462c106f10 2024-09-15T22:44:23.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.007+0000 7f4629600640 1 --2- 172.21.15.111:0/781239066 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f46100a3240 0x7f46100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:23.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.007+0000 7f4629600640 1 -- 172.21.15.111:0/781239066 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f46240cfa40 con 0x7f462c106f10 2024-09-15T22:44:23.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.007+0000 7f4631200640 1 --2- 172.21.15.111:0/781239066 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f46100a3240 0x7f46100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:23.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.008+0000 7f4631200640 1 --2- 172.21.15.111:0/781239066 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f46100a3240 0x7f46100a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f461c031f80 tx=0x7f461c004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:23.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.009+0000 7f4629600640 1 -- 172.21.15.111:0/781239066 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f462409c810 con 0x7f462c106f10 2024-09-15T22:44:23.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.185+0000 7f4633c00640 1 -- 172.21.15.111:0/781239066 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f462c10a120 con 0x7f462c106f10 2024-09-15T22:44:23.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.187+0000 7f4629600640 1 -- 172.21.15.111:0/781239066 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22363 (secure 0 0 0) 0x7f462409c110 con 0x7f462c106f10 2024-09-15T22:44:23.188 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:44:23.189 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":86,"fsid":"b7614d3c-73b2-11ef-bceb-c7b262605968","created":"2024-09-15T22:35:52.571473+0000","modified":"2024-09-15T22:44:18.436245+0000","last_up_change":"2024-09-15T22:43:19.567274+0000","last_in_change":"2024-09-15T22:42:56.994246+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:39:18.954207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:44:12.846418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:44:15.685946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04f39604-be0e-4261-b316-67fa087c6666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6803","nonce":1386823446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6805","nonce":1386823446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6809","nonce":1386823446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6807","nonce":1386823446}]},"public_addr":"172.21.15.111:6803/1386823446","cluster_addr":"172.21.15.111:6805/1386823446","heartbeat_back_addr":"172.21.15.111:6809/1386823446","heartbeat_front_addr":"172.21.15.111:6807/1386823446","state":["exists","up"]},{"osd":1,"uuid":"1bdbef6e-23fa-4618-a788-66e57741a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6801","nonce":2051512291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6803","nonce":2051512291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6807","nonce":2051512291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6805","nonce":2051512291}]},"public_addr":"172.21.15.121:6801/2051512291","cluster_addr":"172.21.15.121:6803/2051512291","heartbeat_back_addr":"172.21.15.121:6807/2051512291","heartbeat_front_addr":"172.21.15.121:6805/2051512291","state":["exists","up"]},{"osd":2,"uuid":"46735ccf-6d0e-4154-bb55-422929819bfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6801","nonce":93085639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6803","nonce":93085639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6807","nonce":93085639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6805","nonce":93085639}]},"public_addr":"172.21.15.192:6801/93085639","cluster_addr":"172.21.15.192:6803/93085639","heartbeat_back_addr":"172.21.15.192:6807/93085639","heartbeat_front_addr":"172.21.15.192:6805/93085639","state":["exists","up"]},{"osd":3,"uuid":"4b243af8-c958-40b1-ba24-9c7ace00c4a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6811","nonce":2578259041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6813","nonce":2578259041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6817","nonce":2578259041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6815","nonce":2578259041}]},"public_addr":"172.21.15.111:6811/2578259041","cluster_addr":"172.21.15.111:6813/2578259041","heartbeat_back_addr":"172.21.15.111:6817/2578259041","heartbeat_front_addr":"172.21.15.111:6815/2578259041","state":["exists","up"]},{"osd":4,"uuid":"49f27541-9cbf-48e0-a466-6bb0fa5410b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6809","nonce":2667595915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6811","nonce":2667595915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6815","nonce":2667595915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6813","nonce":2667595915}]},"public_addr":"172.21.15.121:6809/2667595915","cluster_addr":"172.21.15.121:6811/2667595915","heartbeat_back_addr":"172.21.15.121:6815/2667595915","heartbeat_front_addr":"172.21.15.121:6813/2667595915","state":["exists","up"]},{"osd":5,"uuid":"9290e769-5470-44d3-8d24-25093af34335","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6809","nonce":1958639856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6811","nonce":1958639856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6815","nonce":1958639856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6813","nonce":1958639856}]},"public_addr":"172.21.15.192:6809/1958639856","cluster_addr":"172.21.15.192:6811/1958639856","heartbeat_back_addr":"172.21.15.192:6815/1958639856","heartbeat_front_addr":"172.21.15.192:6813/1958639856","state":["exists","up"]},{"osd":6,"uuid":"18911b97-1c68-4e8f-92ec-35d2ff0b5f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6819","nonce":1371543706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6821","nonce":1371543706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6825","nonce":1371543706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6823","nonce":1371543706}]},"public_addr":"172.21.15.111:6819/1371543706","cluster_addr":"172.21.15.111:6821/1371543706","heartbeat_back_addr":"172.21.15.111:6825/1371543706","heartbeat_front_addr":"172.21.15.111:6823/1371543706","state":["exists","up"]},{"osd":7,"uuid":"470647c5-c00d-426b-a134-cca235902307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6817","nonce":3501842998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6819","nonce":3501842998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6823","nonce":3501842998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6821","nonce":3501842998}]},"public_addr":"172.21.15.121:6817/3501842998","cluster_addr":"172.21.15.121:6819/3501842998","heartbeat_back_addr":"172.21.15.121:6823/3501842998","heartbeat_front_addr":"172.21.15.121:6821/3501842998","state":["exists","up"]},{"osd":8,"uuid":"fa2d3dd5-a768-4ada-a036-4c596b76d654","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6817","nonce":2417771374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6819","nonce":2417771374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6823","nonce":2417771374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6821","nonce":2417771374}]},"public_addr":"172.21.15.192:6817/2417771374","cluster_addr":"172.21.15.192:6819/2417771374","heartbeat_back_addr":"172.21.15.192:6823/2417771374","heartbeat_front_addr":"172.21.15.192:6821/2417771374","state":["exists","up"]},{"osd":9,"uuid":"71b4f5fb-1c02-461b-a257-c6e48381d768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6827","nonce":3979119431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6829","nonce":3979119431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6833","nonce":3979119431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6831","nonce":3979119431}]},"public_addr":"172.21.15.111:6827/3979119431","cluster_addr":"172.21.15.111:6829/3979119431","heartbeat_back_addr":"172.21.15.111:6833/3979119431","heartbeat_front_addr":"172.21.15.111:6831/3979119431","state":["exists","up"]},{"osd":10,"uuid":"e6a39e77-fce3-4e22-bf66-a677c9f8b6bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6825","nonce":2897902637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6827","nonce":2897902637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6831","nonce":2897902637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6829","nonce":2897902637}]},"public_addr":"172.21.15.121:6825/2897902637","cluster_addr":"172.21.15.121:6827/2897902637","heartbeat_back_addr":"172.21.15.121:6831/2897902637","heartbeat_front_addr":"172.21.15.121:6829/2897902637","state":["exists","up"]},{"osd":11,"uuid":"411a7921-6717-4cef-b8df-d9e252edcc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6825","nonce":2431464812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6827","nonce":2431464812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6831","nonce":2431464812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6829","nonce":2431464812}]},"public_addr":"172.21.15.192:6825/2431464812","cluster_addr":"172.21.15.192:6827/2431464812","heartbeat_back_addr":"172.21.15.192:6831/2431464812","heartbeat_front_addr":"172.21.15.192:6829/2431464812","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:38:23.442715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:38:49.501124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:39:14.759077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:39:45.121397+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:40:11.137612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:40:36.278567+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:07.027190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:32.439215+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:57.058702+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:42:28.741846+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:42:53.395442+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:43:18.265067+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:6801/2107775445":"2024-09-16T22:36:50.357551+0000","172.21.15.111:6800/2107775445":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/2394746540":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/4140020843":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/2114354263":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/1054173642":"2024-09-16T22:36:22.652875+0000","172.21.15.111:0/3408238004":"2024-09-16T22:36:22.652875+0000","172.21.15.111:0/2879044772":"2024-09-16T22:36:22.652875+0000","172.21.15.111:6801/3350425104":"2024-09-16T22:36:22.652875+0000","172.21.15.111:6800/3350425104":"2024-09-16T22:36:22.652875+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:44:23.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.190+0000 7f4633c00640 1 -- 172.21.15.111:0/781239066 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f46100a3240 msgr2=0x7f46100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:23.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.190+0000 7f4633c00640 1 --2- 172.21.15.111:0/781239066 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f46100a3240 0x7f46100a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f461c031f80 tx=0x7f461c004a40 comp rx=0 tx=0).stop 2024-09-15T22:44:23.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.190+0000 7f4633c00640 1 -- 172.21.15.111:0/781239066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f462c106f10 msgr2=0x7f462c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:23.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.190+0000 7f4633c00640 1 --2- 172.21.15.111:0/781239066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f462c106f10 0x7f462c074f70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f462400e970 tx=0x7f462400ee30 comp rx=0 tx=0).stop 2024-09-15T22:44:23.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.190+0000 7f4633c00640 1 -- 172.21.15.111:0/781239066 shutdown_connections 2024-09-15T22:44:23.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.190+0000 7f4633c00640 1 --2- 172.21.15.111:0/781239066 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f46100a3240 0x7f46100a5700 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:23.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.190+0000 7f4633c00640 1 --2- 172.21.15.111:0/781239066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f462c106f10 0x7f462c074f70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:23.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.190+0000 7f4633c00640 1 --2- 172.21.15.111:0/781239066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f462c102790 0x7f462c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:23.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.190+0000 7f4633c00640 1 --2- 172.21.15.111:0/781239066 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f462c101de0 0x7f462c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:23.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.190+0000 7f4633c00640 1 -- 172.21.15.111:0/781239066 >> 172.21.15.111:0/781239066 conn(0x7f462c0fd8c0 msgr2=0x7f462c0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:23.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.190+0000 7f4633c00640 1 -- 172.21.15.111:0/781239066 shutdown_connections 2024-09-15T22:44:23.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:23.190+0000 7f4633c00640 1 -- 172.21.15.111:0/781239066 wait complete. 2024-09-15T22:44:23.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:23 smithi192 ceph-mon[35427]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2024-09-15T22:44:23.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:23 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/781239066' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:44:23.587 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:23 smithi111 ceph-mon[30629]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2024-09-15T22:44:23.587 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:23 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/781239066' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:44:23.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:23 smithi121 ceph-mon[34815]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2024-09-15T22:44:23.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:23 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/781239066' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:44:23.992 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-15T22:44:24.359 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:24.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.934+0000 7f0f30600640 1 -- 172.21.15.111:0/2702958292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f28106f10 msgr2=0x7f0f2810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:24.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.934+0000 7f0f30600640 1 --2- 172.21.15.111:0/2702958292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f28106f10 0x7f0f2810ea90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009ca0 tx=0x7f0f2002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.935+0000 7f0f30600640 1 -- 172.21.15.111:0/2702958292 shutdown_connections 2024-09-15T22:44:24.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.935+0000 7f0f30600640 1 --2- 172.21.15.111:0/2702958292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f28106f10 0x7f0f2810ea90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.935+0000 7f0f30600640 1 --2- 172.21.15.111:0/2702958292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f28102790 0x7f0f281067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.935+0000 7f0f30600640 1 --2- 172.21.15.111:0/2702958292 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f28101de0 0x7f0f281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.935+0000 7f0f30600640 1 -- 172.21.15.111:0/2702958292 >> 172.21.15.111:0/2702958292 conn(0x7f0f280fd8c0 msgr2=0x7f0f280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:24.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.935+0000 7f0f30600640 1 -- 172.21.15.111:0/2702958292 shutdown_connections 2024-09-15T22:44:24.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.935+0000 7f0f30600640 1 -- 172.21.15.111:0/2702958292 wait complete. 2024-09-15T22:44:24.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.936+0000 7f0f30600640 1 Processor -- start 2024-09-15T22:44:24.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.936+0000 7f0f30600640 1 -- start start 2024-09-15T22:44:24.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.937+0000 7f0f30600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f28101de0 0x7f0f281983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:24.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.937+0000 7f0f30600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f28102790 0x7f0f28198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:24.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.937+0000 7f0f30600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f28106f10 0x7f0f2819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:24.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.937+0000 7f0f30600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0f28111fc0 con 0x7f0f28101de0 2024-09-15T22:44:24.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.937+0000 7f0f30600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0f28111e40 con 0x7f0f28106f10 2024-09-15T22:44:24.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.937+0000 7f0f30600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0f28112140 con 0x7f0f28102790 2024-09-15T22:44:24.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.937+0000 7f0f2dc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f28101de0 0x7f0f281983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:24.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.937+0000 7f0f2dc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f28101de0 0x7f0f281983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47872/0 (socket says 172.21.15.111:47872) 2024-09-15T22:44:24.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.937+0000 7f0f2dc00640 1 -- 172.21.15.111:0/1310854293 learned_addr learned my addr 172.21.15.111:0/1310854293 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:44:24.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.937+0000 7f0f2d200640 1 --2- 172.21.15.111:0/1310854293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f28102790 0x7f0f28198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:24.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.937+0000 7f0f2e600640 1 --2- 172.21.15.111:0/1310854293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f28106f10 0x7f0f2819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:24.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.937+0000 7f0f2dc00640 1 -- 172.21.15.111:0/1310854293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f28102790 msgr2=0x7f0f28198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:24.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.937+0000 7f0f2dc00640 1 --2- 172.21.15.111:0/1310854293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f28102790 0x7f0f28198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.937+0000 7f0f2dc00640 1 -- 172.21.15.111:0/1310854293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f28106f10 msgr2=0x7f0f2819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:24.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.937+0000 7f0f2dc00640 1 --2- 172.21.15.111:0/1310854293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f28106f10 0x7f0f2819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:24.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.937+0000 7f0f2dc00640 1 -- 172.21.15.111:0/1310854293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2819d370 con 0x7f0f28101de0 2024-09-15T22:44:24.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.937+0000 7f0f2d200640 1 --2- 172.21.15.111:0/1310854293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f28102790 0x7f0f28198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:44:24.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.938+0000 7f0f2dc00640 1 --2- 172.21.15.111:0/1310854293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f28101de0 0x7f0f281983c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c00e9f0 tx=0x7f0f1c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:24.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.938+0000 7f0f26000640 1 -- 172.21.15.111:0/1310854293 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c00cd80 con 0x7f0f28101de0 2024-09-15T22:44:24.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.938+0000 7f0f30600640 1 -- 172.21.15.111:0/1310854293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f2819d660 con 0x7f0f28101de0 2024-09-15T22:44:24.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.938+0000 7f0f26000640 1 -- 172.21.15.111:0/1310854293 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f1c010910 con 0x7f0f28101de0 2024-09-15T22:44:24.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.938+0000 7f0f26000640 1 -- 172.21.15.111:0/1310854293 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c018990 con 0x7f0f28101de0 2024-09-15T22:44:24.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.938+0000 7f0f30600640 1 -- 172.21.15.111:0/1310854293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f281a4f40 con 0x7f0f28101de0 2024-09-15T22:44:24.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.939+0000 7f0f30600640 1 -- 172.21.15.111:0/1310854293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f281021c0 con 0x7f0f28101de0 2024-09-15T22:44:24.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.941+0000 7f0f26000640 1 -- 172.21.15.111:0/1310854293 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f0f1c010ac0 con 0x7f0f28101de0 2024-09-15T22:44:24.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.942+0000 7f0f26000640 1 --2- 172.21.15.111:0/1310854293 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0f100a3240 0x7f0f100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:24.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.942+0000 7f0f26000640 1 -- 172.21.15.111:0/1310854293 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f0f1c0cfb70 con 0x7f0f28101de0 2024-09-15T22:44:24.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.942+0000 7f0f2d200640 1 --2- 172.21.15.111:0/1310854293 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0f100a3240 0x7f0f100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:24.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.943+0000 7f0f2d200640 1 --2- 172.21.15.111:0/1310854293 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0f100a3240 0x7f0f100a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0f280fdce0 tx=0x7f0f14005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:24.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:24.946+0000 7f0f26000640 1 -- 172.21.15.111:0/1310854293 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0f1c09c2f0 con 0x7f0f28101de0 2024-09-15T22:44:25.155 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:25.155+0000 7f0f30600640 1 -- 172.21.15.111:0/1310854293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f0f2802d7c0 con 0x7f0f28101de0 2024-09-15T22:44:25.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:25.507+0000 7f0f26000640 1 -- 172.21.15.111:0/1310854293 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f0f1c0a05f0 con 0x7f0f28101de0 2024-09-15T22:44:25.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:25.510+0000 7f0f30600640 1 -- 172.21.15.111:0/1310854293 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0f100a3240 msgr2=0x7f0f100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:25.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:25.510+0000 7f0f30600640 1 --2- 172.21.15.111:0/1310854293 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0f100a3240 0x7f0f100a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0f280fdce0 tx=0x7f0f14005f90 comp rx=0 tx=0).stop 2024-09-15T22:44:25.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:25.510+0000 7f0f30600640 1 -- 172.21.15.111:0/1310854293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f28101de0 msgr2=0x7f0f281983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:25.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:25.510+0000 7f0f30600640 1 --2- 172.21.15.111:0/1310854293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f28101de0 0x7f0f281983c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0f1c00e9f0 tx=0x7f0f1c00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:44:25.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:25.510+0000 7f0f30600640 1 -- 172.21.15.111:0/1310854293 shutdown_connections 2024-09-15T22:44:25.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:25.510+0000 7f0f30600640 1 --2- 172.21.15.111:0/1310854293 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0f100a3240 0x7f0f100a5700 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:25.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:25.510+0000 7f0f30600640 1 --2- 172.21.15.111:0/1310854293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f28106f10 0x7f0f2819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:25.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:25.510+0000 7f0f30600640 1 --2- 172.21.15.111:0/1310854293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f28102790 0x7f0f28198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:25.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:25.510+0000 7f0f30600640 1 --2- 172.21.15.111:0/1310854293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f28101de0 0x7f0f281983c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:25.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:25.510+0000 7f0f30600640 1 -- 172.21.15.111:0/1310854293 >> 172.21.15.111:0/1310854293 conn(0x7f0f280fd8c0 msgr2=0x7f0f280feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:25.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:25.510+0000 7f0f30600640 1 -- 172.21.15.111:0/1310854293 shutdown_connections 2024-09-15T22:44:25.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:25.510+0000 7f0f30600640 1 -- 172.21.15.111:0/1310854293 wait complete. 2024-09-15T22:44:25.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:25 smithi192 ceph-mon[35427]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-15T22:44:25.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:25 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/1310854293' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-15T22:44:25.887 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:25 smithi111 ceph-mon[30629]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-15T22:44:25.887 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:25 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/1310854293' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-15T22:44:25.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:25 smithi121 ceph-mon[34815]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-15T22:44:25.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:25 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/1310854293' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-15T22:44:26.434 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-15T22:44:26.434 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-15T22:44:26.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:26 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/1310854293' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-15T22:44:26.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:26 smithi111 ceph-mon[30629]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:44:26.806 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:26.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:26 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/1310854293' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-15T22:44:26.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:26 smithi192 ceph-mon[35427]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:44:26.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:26 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/1310854293' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-15T22:44:26.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:26 smithi121 ceph-mon[34815]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:44:27.310 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.310+0000 7f6ca5400640 1 -- 172.21.15.111:0/3060615300 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ca0102360 msgr2=0x7f6ca010e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.310+0000 7f6ca5400640 1 --2- 172.21.15.111:0/3060615300 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ca0102360 0x7f6ca010e890 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6c98009cd0 tx=0x7f6c9802a8f0 comp rx=0 tx=0).stop 2024-09-15T22:44:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.310+0000 7f6ca5400640 1 -- 172.21.15.111:0/3060615300 shutdown_connections 2024-09-15T22:44:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.310+0000 7f6ca5400640 1 --2- 172.21.15.111:0/3060615300 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ca0102360 0x7f6ca010e890 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.310+0000 7f6ca5400640 1 --2- 172.21.15.111:0/3060615300 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ca01019c0 0x7f6ca0101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.310+0000 7f6ca5400640 1 --2- 172.21.15.111:0/3060615300 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ca01079c0 0x7f6ca0107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.310+0000 7f6ca5400640 1 -- 172.21.15.111:0/3060615300 >> 172.21.15.111:0/3060615300 conn(0x7f6ca00fd740 msgr2=0x7f6ca00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.310+0000 7f6ca5400640 1 -- 172.21.15.111:0/3060615300 shutdown_connections 2024-09-15T22:44:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.310+0000 7f6ca5400640 1 -- 172.21.15.111:0/3060615300 wait complete. 2024-09-15T22:44:27.312 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.311+0000 7f6ca5400640 1 Processor -- start 2024-09-15T22:44:27.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.311+0000 7f6ca5400640 1 -- start start 2024-09-15T22:44:27.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.311+0000 7f6ca5400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ca01019c0 0x7f6ca019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:27.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.311+0000 7f6ca5400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ca0102360 0x7f6ca019a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:27.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.311+0000 7f6ca5400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ca01079c0 0x7f6ca019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:27.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.311+0000 7f6ca5400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca0111e10 con 0x7f6ca0102360 2024-09-15T22:44:27.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.311+0000 7f6ca5400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca0111c90 con 0x7f6ca01079c0 2024-09-15T22:44:27.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.311+0000 7f6ca5400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca0111f90 con 0x7f6ca01019c0 2024-09-15T22:44:27.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.312+0000 7f6c97400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ca0102360 0x7f6ca019a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:27.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.312+0000 7f6c9fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ca01019c0 0x7f6ca019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:27.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.312+0000 7f6c97400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ca0102360 0x7f6ca019a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47890/0 (socket says 172.21.15.111:47890) 2024-09-15T22:44:27.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.312+0000 7f6c97400640 1 -- 172.21.15.111:0/4221553637 learned_addr learned my addr 172.21.15.111:0/4221553637 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:44:27.313 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.312+0000 7f6ca4a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ca01079c0 0x7f6ca019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:27.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.312+0000 7f6c9fe00640 1 -- 172.21.15.111:0/4221553637 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ca01079c0 msgr2=0x7f6ca019ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:27.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.312+0000 7f6c9fe00640 1 --2- 172.21.15.111:0/4221553637 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ca01079c0 0x7f6ca019ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:27.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.312+0000 7f6c9fe00640 1 -- 172.21.15.111:0/4221553637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ca0102360 msgr2=0x7f6ca019a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:27.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.312+0000 7f6c9fe00640 1 --2- 172.21.15.111:0/4221553637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ca0102360 0x7f6ca019a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:27.314 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.312+0000 7f6c9fe00640 1 -- 172.21.15.111:0/4221553637 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca019f3f0 con 0x7f6ca01019c0 2024-09-15T22:44:27.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.312+0000 7f6c9fe00640 1 --2- 172.21.15.111:0/4221553637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ca01019c0 0x7f6ca019a440 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000e9f0 tx=0x7f6c9000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:27.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.313+0000 7f6c9d600640 1 -- 172.21.15.111:0/4221553637 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9000ccc0 con 0x7f6ca01019c0 2024-09-15T22:44:27.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.313+0000 7f6c9d600640 1 -- 172.21.15.111:0/4221553637 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c90005020 con 0x7f6ca01019c0 2024-09-15T22:44:27.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.313+0000 7f6ca5400640 1 -- 172.21.15.111:0/4221553637 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca019f6e0 con 0x7f6ca01019c0 2024-09-15T22:44:27.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.313+0000 7f6c9d600640 1 -- 172.21.15.111:0/4221553637 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c900049e0 con 0x7f6ca01019c0 2024-09-15T22:44:27.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.313+0000 7f6ca5400640 1 -- 172.21.15.111:0/4221553637 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca01a6fc0 con 0x7f6ca01019c0 2024-09-15T22:44:27.315 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.314+0000 7f6ca5400640 1 -- 172.21.15.111:0/4221553637 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ca0101e20 con 0x7f6ca01019c0 2024-09-15T22:44:27.316 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.316+0000 7f6c9d600640 1 -- 172.21.15.111:0/4221553637 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f6c90005560 con 0x7f6ca01019c0 2024-09-15T22:44:27.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.317+0000 7f6c9d600640 1 --2- 172.21.15.111:0/4221553637 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6c840a3240 0x7f6c840a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:27.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.317+0000 7f6c9d600640 1 -- 172.21.15.111:0/4221553637 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f6c900cf820 con 0x7f6ca01019c0 2024-09-15T22:44:27.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.317+0000 7f6c97400640 1 --2- 172.21.15.111:0/4221553637 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6c840a3240 0x7f6c840a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:27.318 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.318+0000 7f6c97400640 1 --2- 172.21.15.111:0/4221553637 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6c840a3240 0x7f6c840a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6ca010ecb0 tx=0x7f6c88005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:27.320 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.319+0000 7f6c9d600640 1 -- 172.21.15.111:0/4221553637 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6c900051d0 con 0x7f6ca01019c0 2024-09-15T22:44:27.530 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:27.529+0000 7f6ca5400640 1 -- 172.21.15.111:0/4221553637 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f6ca019b7a0 con 0x7f6ca01019c0 2024-09-15T22:44:27.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:27 smithi192 ceph-mon[35427]: pgmap v304: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-15T22:44:27.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:27 smithi121 ceph-mon[34815]: pgmap v304: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-15T22:44:27.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:27 smithi111 ceph-mon[30629]: pgmap v304: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-15T22:44:28.532 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:28.530+0000 7f6c9d600640 1 -- 172.21.15.111:0/4221553637 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f6ca019b7a0 con 0x7f6ca01019c0 2024-09-15T22:44:28.535 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:28.533+0000 7f6ca5400640 1 -- 172.21.15.111:0/4221553637 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6c840a3240 msgr2=0x7f6c840a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:28.535 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:28.533+0000 7f6ca5400640 1 --2- 172.21.15.111:0/4221553637 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6c840a3240 0x7f6c840a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6ca010ecb0 tx=0x7f6c88005f90 comp rx=0 tx=0).stop 2024-09-15T22:44:28.535 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:28.533+0000 7f6ca5400640 1 -- 172.21.15.111:0/4221553637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ca01019c0 msgr2=0x7f6ca019a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:28.535 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:28.533+0000 7f6ca5400640 1 --2- 172.21.15.111:0/4221553637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ca01019c0 0x7f6ca019a440 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000e9f0 tx=0x7f6c9000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.536 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:28.533+0000 7f6ca5400640 1 -- 172.21.15.111:0/4221553637 shutdown_connections 2024-09-15T22:44:28.536 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:28.533+0000 7f6ca5400640 1 --2- 172.21.15.111:0/4221553637 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6c840a3240 0x7f6c840a5700 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.536 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:28.533+0000 7f6ca5400640 1 --2- 172.21.15.111:0/4221553637 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6ca01079c0 0x7f6ca019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.536 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:28.533+0000 7f6ca5400640 1 --2- 172.21.15.111:0/4221553637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6ca0102360 0x7f6ca019a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.536 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:28.533+0000 7f6ca5400640 1 --2- 172.21.15.111:0/4221553637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6ca01019c0 0x7f6ca019a440 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:28.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:28.533+0000 7f6ca5400640 1 -- 172.21.15.111:0/4221553637 >> 172.21.15.111:0/4221553637 conn(0x7f6ca00fd740 msgr2=0x7f6ca00feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:28.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:28.533+0000 7f6ca5400640 1 -- 172.21.15.111:0/4221553637 shutdown_connections 2024-09-15T22:44:28.537 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:28.533+0000 7f6ca5400640 1 -- 172.21.15.111:0/4221553637 wait complete. 2024-09-15T22:44:28.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:28 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/4221553637' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-15T22:44:28.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:28 smithi192 ceph-mon[35427]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-15T22:44:28.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:28 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/4221553637' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-15T22:44:28.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:28 smithi121 ceph-mon[34815]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-15T22:44:28.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:28 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/4221553637' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-15T22:44:28.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:28 smithi111 ceph-mon[30629]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-15T22:44:29.464 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-15T22:44:29.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:29 smithi192 ceph-mon[35427]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-09-15T22:44:29.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:29 smithi192 ceph-mon[35427]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-15T22:44:29.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:29 smithi192 ceph-mon[35427]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-15T22:44:29.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:29 smithi192 ceph-mon[35427]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-15T22:44:29.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:29 smithi192 ceph-mon[35427]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:44:29.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:29 smithi192 ceph-mon[35427]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-15T22:44:29.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:29 smithi192 ceph-mon[35427]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-15T22:44:29.837 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:29.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:29 smithi121 ceph-mon[34815]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-09-15T22:44:29.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:29 smithi121 ceph-mon[34815]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-15T22:44:29.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:29 smithi121 ceph-mon[34815]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-15T22:44:29.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:29 smithi121 ceph-mon[34815]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-15T22:44:29.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:29 smithi121 ceph-mon[34815]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:44:29.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:29 smithi121 ceph-mon[34815]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-15T22:44:29.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:29 smithi121 ceph-mon[34815]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-15T22:44:29.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:29 smithi111 ceph-mon[30629]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-09-15T22:44:29.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:29 smithi111 ceph-mon[30629]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-15T22:44:29.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:29 smithi111 ceph-mon[30629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-15T22:44:29.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:29 smithi111 ceph-mon[30629]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-15T22:44:29.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:29 smithi111 ceph-mon[30629]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:44:29.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:29 smithi111 ceph-mon[30629]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-15T22:44:29.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:29 smithi111 ceph-mon[30629]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-15T22:44:30.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.423+0000 7fb088400640 1 -- 172.21.15.111:0/4174027642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb080112a30 msgr2=0x7fb080114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:30.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.423+0000 7fb088400640 1 --2- 172.21.15.111:0/4174027642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb080112a30 0x7fb080114e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb078009ca0 tx=0x7fb07802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.424+0000 7fb088400640 1 -- 172.21.15.111:0/4174027642 shutdown_connections 2024-09-15T22:44:30.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.424+0000 7fb088400640 1 --2- 172.21.15.111:0/4174027642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb080112a30 0x7fb080114e20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.424+0000 7fb088400640 1 --2- 172.21.15.111:0/4174027642 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb080077040 0x7fb0801124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.424+0000 7fb088400640 1 --2- 172.21.15.111:0/4174027642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb080076720 0x7fb080076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.424+0000 7fb088400640 1 -- 172.21.15.111:0/4174027642 >> 172.21.15.111:0/4174027642 conn(0x7fb0800ffa90 msgr2=0x7fb080101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:30.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.424+0000 7fb088400640 1 -- 172.21.15.111:0/4174027642 shutdown_connections 2024-09-15T22:44:30.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.424+0000 7fb088400640 1 -- 172.21.15.111:0/4174027642 wait complete. 2024-09-15T22:44:30.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.425+0000 7fb088400640 1 Processor -- start 2024-09-15T22:44:30.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.425+0000 7fb088400640 1 -- start start 2024-09-15T22:44:30.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.425+0000 7fb088400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb080076720 0x7fb0801a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:30.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.425+0000 7fb088400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb080077040 0x7fb0801a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:30.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.425+0000 7fb088400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb080112a30 0x7fb08019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:30.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.425+0000 7fb088400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb0801185d0 con 0x7fb080112a30 2024-09-15T22:44:30.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.425+0000 7fb088400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb080118450 con 0x7fb080077040 2024-09-15T22:44:30.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.425+0000 7fb088400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb080118750 con 0x7fb080076720 2024-09-15T22:44:30.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.426+0000 7fb086400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb080112a30 0x7fb08019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:30.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.426+0000 7fb086400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb080112a30 0x7fb08019e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47898/0 (socket says 172.21.15.111:47898) 2024-09-15T22:44:30.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.426+0000 7fb086400640 1 -- 172.21.15.111:0/4284546749 learned_addr learned my addr 172.21.15.111:0/4284546749 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:44:30.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.426+0000 7fb085a00640 1 --2- 172.21.15.111:0/4284546749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb080076720 0x7fb0801a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:30.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.426+0000 7fb085000640 1 --2- 172.21.15.111:0/4284546749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb080077040 0x7fb0801a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:30.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.426+0000 7fb086400640 1 -- 172.21.15.111:0/4284546749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb080076720 msgr2=0x7fb0801a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:30.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.426+0000 7fb086400640 1 --2- 172.21.15.111:0/4284546749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb080076720 0x7fb0801a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.426+0000 7fb086400640 1 -- 172.21.15.111:0/4284546749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb080077040 msgr2=0x7fb0801a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:30.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.426+0000 7fb086400640 1 --2- 172.21.15.111:0/4284546749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb080077040 0x7fb0801a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:30.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.426+0000 7fb086400640 1 -- 172.21.15.111:0/4284546749 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08019efd0 con 0x7fb080112a30 2024-09-15T22:44:30.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.426+0000 7fb086400640 1 --2- 172.21.15.111:0/4284546749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb080112a30 0x7fb08019e8a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb078005e30 tx=0x7fb078003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:30.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.427+0000 7fb07e000640 1 -- 172.21.15.111:0/4284546749 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078031650 con 0x7fb080112a30 2024-09-15T22:44:30.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.427+0000 7fb07e000640 1 -- 172.21.15.111:0/4284546749 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0780317f0 con 0x7fb080112a30 2024-09-15T22:44:30.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.427+0000 7fb088400640 1 -- 172.21.15.111:0/4284546749 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb08019f260 con 0x7fb080112a30 2024-09-15T22:44:30.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.427+0000 7fb07e000640 1 -- 172.21.15.111:0/4284546749 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07802f970 con 0x7fb080112a30 2024-09-15T22:44:30.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.427+0000 7fb088400640 1 -- 172.21.15.111:0/4284546749 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0801ab5b0 con 0x7fb080112a30 2024-09-15T22:44:30.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.428+0000 7fb088400640 1 -- 172.21.15.111:0/4284546749 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb080076b00 con 0x7fb080112a30 2024-09-15T22:44:30.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.430+0000 7fb07e000640 1 -- 172.21.15.111:0/4284546749 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7fb07802fb30 con 0x7fb080112a30 2024-09-15T22:44:30.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.432+0000 7fb07e000640 1 --2- 172.21.15.111:0/4284546749 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb0680a32c0 0x7fb0680a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:30.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.432+0000 7fb07e000640 1 -- 172.21.15.111:0/4284546749 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fb0780f0a00 con 0x7fb080112a30 2024-09-15T22:44:30.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.432+0000 7fb085a00640 1 --2- 172.21.15.111:0/4284546749 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb0680a32c0 0x7fb0680a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:30.457 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.432+0000 7fb085a00640 1 --2- 172.21.15.111:0/4284546749 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb0680a32c0 0x7fb0680a5780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb08019f9e0 tx=0x7fb074005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:30.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.436+0000 7fb07e000640 1 -- 172.21.15.111:0/4284546749 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb0780319a0 con 0x7fb080112a30 2024-09-15T22:44:30.651 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:30.649+0000 7fb088400640 1 -- 172.21.15.111:0/4284546749 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fb08010dbf0 con 0x7fb080112a30 2024-09-15T22:44:30.651 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:30 smithi111 ceph-mon[30629]: mds.? [v2:172.21.15.192:6838/800136261,v1:172.21.15.192:6839/800136261] up:active 2024-09-15T22:44:30.651 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:30 smithi111 ceph-mon[30629]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-15T22:44:30.651 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:30 smithi111 ceph-mon[30629]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-15T22:44:30.652 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:30 smithi111 ceph-mon[30629]: Cluster is now healthy 2024-09-15T22:44:30.652 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:30 smithi111 ceph-mon[30629]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-15T22:44:30.652 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:30 smithi111 ceph-mon[30629]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-15T22:44:30.652 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:30 smithi111 ceph-mon[30629]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-15T22:44:30.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:30 smithi192 ceph-mon[35427]: mds.? [v2:172.21.15.192:6838/800136261,v1:172.21.15.192:6839/800136261] up:active 2024-09-15T22:44:30.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:30 smithi192 ceph-mon[35427]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-15T22:44:30.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:30 smithi192 ceph-mon[35427]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-15T22:44:30.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:30 smithi192 ceph-mon[35427]: Cluster is now healthy 2024-09-15T22:44:30.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:30 smithi192 ceph-mon[35427]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-15T22:44:30.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:30 smithi192 ceph-mon[35427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-15T22:44:30.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:30 smithi192 ceph-mon[35427]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-15T22:44:30.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:30 smithi121 ceph-mon[34815]: mds.? [v2:172.21.15.192:6838/800136261,v1:172.21.15.192:6839/800136261] up:active 2024-09-15T22:44:30.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:30 smithi121 ceph-mon[34815]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-15T22:44:30.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:30 smithi121 ceph-mon[34815]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-15T22:44:30.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:30 smithi121 ceph-mon[34815]: Cluster is now healthy 2024-09-15T22:44:30.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:30 smithi121 ceph-mon[34815]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-15T22:44:30.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:30 smithi121 ceph-mon[34815]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-15T22:44:30.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:30 smithi121 ceph-mon[34815]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-15T22:44:31.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:31.557+0000 7fb07e000640 1 -- 172.21.15.111:0/4284546749 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7fb0780bd7d0 con 0x7fb080112a30 2024-09-15T22:44:31.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:31.561+0000 7fb088400640 1 -- 172.21.15.111:0/4284546749 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb0680a32c0 msgr2=0x7fb0680a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:31.563 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:31.561+0000 7fb088400640 1 --2- 172.21.15.111:0/4284546749 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb0680a32c0 0x7fb0680a5780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb08019f9e0 tx=0x7fb074005f90 comp rx=0 tx=0).stop 2024-09-15T22:44:31.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:31.562+0000 7fb088400640 1 -- 172.21.15.111:0/4284546749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb080112a30 msgr2=0x7fb08019e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:31.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:31.562+0000 7fb088400640 1 --2- 172.21.15.111:0/4284546749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb080112a30 0x7fb08019e8a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb078005e30 tx=0x7fb078003f10 comp rx=0 tx=0).stop 2024-09-15T22:44:31.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:31.562+0000 7fb088400640 1 -- 172.21.15.111:0/4284546749 shutdown_connections 2024-09-15T22:44:31.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:31.562+0000 7fb088400640 1 --2- 172.21.15.111:0/4284546749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb080112a30 0x7fb08019e8a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:31.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:31.562+0000 7fb088400640 1 --2- 172.21.15.111:0/4284546749 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb0680a32c0 0x7fb0680a5780 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:31.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:31.562+0000 7fb088400640 1 --2- 172.21.15.111:0/4284546749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb080077040 0x7fb0801a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:31.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:31.562+0000 7fb088400640 1 --2- 172.21.15.111:0/4284546749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb080076720 0x7fb0801a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:31.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:31.562+0000 7fb088400640 1 -- 172.21.15.111:0/4284546749 >> 172.21.15.111:0/4284546749 conn(0x7fb0800ffa90 msgr2=0x7fb080101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:31.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:31.562+0000 7fb088400640 1 -- 172.21.15.111:0/4284546749 shutdown_connections 2024-09-15T22:44:31.564 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:31.562+0000 7fb088400640 1 -- 172.21.15.111:0/4284546749 wait complete. 2024-09-15T22:44:31.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:31 smithi192 ceph-mon[35427]: pgmap v306: 97 pgs: 97 active+clean; 580 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-15T22:44:31.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:31 smithi192 ceph-mon[35427]: mds.? [v2:172.21.15.121:6836/1443772791,v1:172.21.15.121:6837/1443772791] up:active 2024-09-15T22:44:31.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:31 smithi192 ceph-mon[35427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-15T22:44:31.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:31 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/4284546749' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-15T22:44:31.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:31 smithi121 ceph-mon[34815]: pgmap v306: 97 pgs: 97 active+clean; 580 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-15T22:44:31.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:31 smithi121 ceph-mon[34815]: mds.? [v2:172.21.15.121:6836/1443772791,v1:172.21.15.121:6837/1443772791] up:active 2024-09-15T22:44:31.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:31 smithi121 ceph-mon[34815]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-15T22:44:31.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:31 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/4284546749' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-15T22:44:31.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:31 smithi111 ceph-mon[30629]: pgmap v306: 97 pgs: 97 active+clean; 580 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-15T22:44:31.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:31 smithi111 ceph-mon[30629]: mds.? [v2:172.21.15.121:6836/1443772791,v1:172.21.15.121:6837/1443772791] up:active 2024-09-15T22:44:31.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:31 smithi111 ceph-mon[30629]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-15T22:44:31.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:31 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/4284546749' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-15T22:44:32.478 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-15T22:44:32.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:32 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/4284546749' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-15T22:44:32.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:32 smithi192 ceph-mon[35427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-15T22:44:32.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:32 smithi192 ceph-mon[35427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:44:32.850 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:32.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:32 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/4284546749' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-15T22:44:32.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:32 smithi121 ceph-mon[34815]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-15T22:44:32.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:32 smithi121 ceph-mon[34815]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:44:32.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:32 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/4284546749' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-15T22:44:32.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:32 smithi111 ceph-mon[30629]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-15T22:44:32.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:32 smithi111 ceph-mon[30629]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:44:33.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.431+0000 7efccea00640 1 -- 172.21.15.111:0/55155750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcc8103ff0 msgr2=0x7efcc81043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:33.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.431+0000 7efccea00640 1 --2- 172.21.15.111:0/55155750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcc8103ff0 0x7efcc81043d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efcb8009900 tx=0x7efcb802a410 comp rx=0 tx=0).stop 2024-09-15T22:44:33.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.431+0000 7efccea00640 1 -- 172.21.15.111:0/55155750 shutdown_connections 2024-09-15T22:44:33.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.431+0000 7efccea00640 1 --2- 172.21.15.111:0/55155750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcc8109120 0x7efcc8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:33.433 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.431+0000 7efccea00640 1 --2- 172.21.15.111:0/55155750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcc81049a0 0x7efcc81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:33.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.431+0000 7efccea00640 1 --2- 172.21.15.111:0/55155750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcc8103ff0 0x7efcc81043d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:33.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.431+0000 7efccea00640 1 -- 172.21.15.111:0/55155750 >> 172.21.15.111:0/55155750 conn(0x7efcc80ffa90 msgr2=0x7efcc8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:33.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.431+0000 7efccea00640 1 -- 172.21.15.111:0/55155750 shutdown_connections 2024-09-15T22:44:33.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.431+0000 7efccea00640 1 -- 172.21.15.111:0/55155750 wait complete. 2024-09-15T22:44:33.435 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.432+0000 7efccea00640 1 Processor -- start 2024-09-15T22:44:33.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.432+0000 7efccea00640 1 -- start start 2024-09-15T22:44:33.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.433+0000 7efccea00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcc8103ff0 0x7efcc819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:33.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.433+0000 7efccea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcc81049a0 0x7efcc819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:33.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.433+0000 7efccea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcc8109120 0x7efcc819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:33.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.433+0000 7efccea00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efcc8114150 con 0x7efcc8109120 2024-09-15T22:44:33.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.433+0000 7efccea00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efcc8113fd0 con 0x7efcc81049a0 2024-09-15T22:44:33.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.433+0000 7efccea00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efcc81142d0 con 0x7efcc8103ff0 2024-09-15T22:44:33.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.433+0000 7efcc7e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcc8103ff0 0x7efcc819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:33.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.433+0000 7efcc7e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcc8103ff0 0x7efcc819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:57770/0 (socket says 172.21.15.111:57770) 2024-09-15T22:44:33.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.433+0000 7efcc7e00640 1 -- 172.21.15.111:0/1632068495 learned_addr learned my addr 172.21.15.111:0/1632068495 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:44:33.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.433+0000 7efccca00640 1 --2- 172.21.15.111:0/1632068495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcc8109120 0x7efcc819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:33.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.433+0000 7efcc7400640 1 --2- 172.21.15.111:0/1632068495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcc81049a0 0x7efcc819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:33.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.433+0000 7efcc7e00640 1 -- 172.21.15.111:0/1632068495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcc81049a0 msgr2=0x7efcc819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:33.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.433+0000 7efcc7e00640 1 --2- 172.21.15.111:0/1632068495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcc81049a0 0x7efcc819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:33.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.433+0000 7efcc7e00640 1 -- 172.21.15.111:0/1632068495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcc8109120 msgr2=0x7efcc819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:33.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.433+0000 7efcc7e00640 1 --2- 172.21.15.111:0/1632068495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcc8109120 0x7efcc819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:33.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.434+0000 7efcc7e00640 1 -- 172.21.15.111:0/1632068495 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc819f5a0 con 0x7efcc8103ff0 2024-09-15T22:44:33.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.434+0000 7efcc7e00640 1 --2- 172.21.15.111:0/1632068495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcc8103ff0 0x7efcc819a5f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efcb8006010 tx=0x7efcb8004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:33.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.434+0000 7efcc4c00640 1 -- 172.21.15.111:0/1632068495 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb803f030 con 0x7efcc8103ff0 2024-09-15T22:44:33.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.434+0000 7efcc4c00640 1 -- 172.21.15.111:0/1632068495 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb8004440 con 0x7efcc8103ff0 2024-09-15T22:44:33.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.434+0000 7efccea00640 1 -- 172.21.15.111:0/1632068495 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc819f830 con 0x7efcc8103ff0 2024-09-15T22:44:33.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.434+0000 7efcc4c00640 1 -- 172.21.15.111:0/1632068495 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb8035070 con 0x7efcc8103ff0 2024-09-15T22:44:33.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.434+0000 7efccea00640 1 -- 172.21.15.111:0/1632068495 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc81a7170 con 0x7efcc8103ff0 2024-09-15T22:44:33.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.435+0000 7efccea00640 1 -- 172.21.15.111:0/1632068495 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcc81054d0 con 0x7efcc8103ff0 2024-09-15T22:44:33.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.437+0000 7efcc4c00640 1 -- 172.21.15.111:0/1632068495 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7efcb8042030 con 0x7efcc8103ff0 2024-09-15T22:44:33.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.438+0000 7efcc4c00640 1 --2- 172.21.15.111:0/1632068495 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7efcac0a3240 0x7efcac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:33.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.438+0000 7efcc4c00640 1 -- 172.21.15.111:0/1632068495 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7efcb80efbb0 con 0x7efcc8103ff0 2024-09-15T22:44:33.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.438+0000 7efcc7400640 1 --2- 172.21.15.111:0/1632068495 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7efcac0a3240 0x7efcac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:33.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.439+0000 7efcc7400640 1 --2- 172.21.15.111:0/1632068495 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7efcac0a3240 0x7efcac0a5700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efcc80ffeb0 tx=0x7efcb4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:33.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.442+0000 7efcc4c00640 1 -- 172.21.15.111:0/1632068495 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efcb80bc330 con 0x7efcc8103ff0 2024-09-15T22:44:33.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:33.653+0000 7efccea00640 1 -- 172.21.15.111:0/1632068495 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7efcc81043d0 con 0x7efcc8103ff0 2024-09-15T22:44:33.655 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:33 smithi111 ceph-mon[30629]: pgmap v307: 97 pgs: 97 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-15T22:44:33.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:33 smithi121 ceph-mon[34815]: pgmap v307: 97 pgs: 97 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-15T22:44:33.954 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:33 smithi192 ceph-mon[35427]: pgmap v307: 97 pgs: 97 active+clean; 581 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-15T22:44:34.576 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:34.575+0000 7efcc4c00640 1 -- 172.21.15.111:0/1632068495 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7efcb80c0630 con 0x7efcc8103ff0 2024-09-15T22:44:34.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:34.577+0000 7efccea00640 1 -- 172.21.15.111:0/1632068495 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7efcac0a3240 msgr2=0x7efcac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:34.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:34.577+0000 7efccea00640 1 --2- 172.21.15.111:0/1632068495 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7efcac0a3240 0x7efcac0a5700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efcc80ffeb0 tx=0x7efcb4005f90 comp rx=0 tx=0).stop 2024-09-15T22:44:34.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:34.577+0000 7efccea00640 1 -- 172.21.15.111:0/1632068495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcc8103ff0 msgr2=0x7efcc819a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:34.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:34.577+0000 7efccea00640 1 --2- 172.21.15.111:0/1632068495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcc8103ff0 0x7efcc819a5f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efcb8006010 tx=0x7efcb8004040 comp rx=0 tx=0).stop 2024-09-15T22:44:34.579 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:34.577+0000 7efccea00640 1 -- 172.21.15.111:0/1632068495 shutdown_connections 2024-09-15T22:44:34.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:34.577+0000 7efccea00640 1 --2- 172.21.15.111:0/1632068495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcc8109120 0x7efcc819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:34.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:34.577+0000 7efccea00640 1 --2- 172.21.15.111:0/1632068495 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7efcac0a3240 0x7efcac0a5700 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:34.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:34.577+0000 7efccea00640 1 --2- 172.21.15.111:0/1632068495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcc81049a0 0x7efcc819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:34.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:34.577+0000 7efccea00640 1 --2- 172.21.15.111:0/1632068495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcc8103ff0 0x7efcc819a5f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:34.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:34.577+0000 7efccea00640 1 -- 172.21.15.111:0/1632068495 >> 172.21.15.111:0/1632068495 conn(0x7efcc80ffa90 msgr2=0x7efcc8100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:34.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:34.578+0000 7efccea00640 1 -- 172.21.15.111:0/1632068495 shutdown_connections 2024-09-15T22:44:34.580 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:34.578+0000 7efccea00640 1 -- 172.21.15.111:0/1632068495 wait complete. 2024-09-15T22:44:34.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:34 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/1632068495' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-15T22:44:34.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:34 smithi121 ceph-mon[34815]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-15T22:44:34.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:34 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/1632068495' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-15T22:44:34.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:34 smithi111 ceph-mon[30629]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-15T22:44:35.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:34 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/1632068495' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-15T22:44:35.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:34 smithi192 ceph-mon[35427]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-15T22:44:35.425 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-15T22:44:35.425 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-15T22:44:35.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:35 smithi111 ceph-mon[30629]: pgmap v308: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-15T22:44:35.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:35 smithi111 ceph-mon[30629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-15T22:44:35.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:35 smithi111 ceph-mon[30629]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:44:35.795 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:35.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:35 smithi121 ceph-mon[34815]: pgmap v308: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-15T22:44:35.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:35 smithi121 ceph-mon[34815]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-15T22:44:35.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:35 smithi121 ceph-mon[34815]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:44:36.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:35 smithi192 ceph-mon[35427]: pgmap v308: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-09-15T22:44:36.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:35 smithi192 ceph-mon[35427]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-15T22:44:36.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:35 smithi192 ceph-mon[35427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:44:36.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.396+0000 7f6553e00640 1 -- 172.21.15.111:0/598098311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6554108f60 msgr2=0x7f6554110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:36.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.396+0000 7f6553e00640 1 --2- 172.21.15.111:0/598098311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6554108f60 0x7f6554110ae0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f654c009ca0 tx=0x7f654c02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:44:36.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.396+0000 7f6553e00640 1 -- 172.21.15.111:0/598098311 shutdown_connections 2024-09-15T22:44:36.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.396+0000 7f6553e00640 1 --2- 172.21.15.111:0/598098311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6554108f60 0x7f6554110ae0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:36.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.396+0000 7f6553e00640 1 --2- 172.21.15.111:0/598098311 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65541047e0 0x7f6554108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:36.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.396+0000 7f6553e00640 1 --2- 172.21.15.111:0/598098311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6554103e30 0x7f6554104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:36.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.396+0000 7f6553e00640 1 -- 172.21.15.111:0/598098311 >> 172.21.15.111:0/598098311 conn(0x7f65540ff910 msgr2=0x7f6554101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:36.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.396+0000 7f6553e00640 1 -- 172.21.15.111:0/598098311 shutdown_connections 2024-09-15T22:44:36.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.396+0000 7f6553e00640 1 -- 172.21.15.111:0/598098311 wait complete. 2024-09-15T22:44:36.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.397+0000 7f6553e00640 1 Processor -- start 2024-09-15T22:44:36.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.398+0000 7f6553e00640 1 -- start start 2024-09-15T22:44:36.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.398+0000 7f6553e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6554103e30 0x7f655419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:36.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.398+0000 7f6553e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65541047e0 0x7f655419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:36.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.398+0000 7f6553e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6554108f60 0x7f655419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:36.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.398+0000 7f6553e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6554114010 con 0x7f6554103e30 2024-09-15T22:44:36.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.398+0000 7f6553e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6554113e90 con 0x7f6554108f60 2024-09-15T22:44:36.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.398+0000 7f6553e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6554114190 con 0x7f65541047e0 2024-09-15T22:44:36.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.398+0000 7f6552000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65541047e0 0x7f655419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:36.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.398+0000 7f6552a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6554103e30 0x7f655419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:36.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.398+0000 7f6553400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6554108f60 0x7f655419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:36.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.398+0000 7f6552000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65541047e0 0x7f655419a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:57792/0 (socket says 172.21.15.111:57792) 2024-09-15T22:44:36.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.398+0000 7f6552000640 1 -- 172.21.15.111:0/3274229771 learned_addr learned my addr 172.21.15.111:0/3274229771 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:44:36.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.399+0000 7f6552000640 1 -- 172.21.15.111:0/3274229771 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6554108f60 msgr2=0x7f655419ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:36.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.399+0000 7f6552000640 1 --2- 172.21.15.111:0/3274229771 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6554108f60 0x7f655419ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:36.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.399+0000 7f6552000640 1 -- 172.21.15.111:0/3274229771 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6554103e30 msgr2=0x7f655419a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:36.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.399+0000 7f6552000640 1 --2- 172.21.15.111:0/3274229771 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6554103e30 0x7f655419a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:36.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.399+0000 7f6552000640 1 -- 172.21.15.111:0/3274229771 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655419f3b0 con 0x7f65541047e0 2024-09-15T22:44:36.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.399+0000 7f6552000640 1 --2- 172.21.15.111:0/3274229771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65541047e0 0x7f655419a940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f654000e9f0 tx=0x7f654000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:36.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.399+0000 7f654b400640 1 -- 172.21.15.111:0/3274229771 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654000cd80 con 0x7f65541047e0 2024-09-15T22:44:36.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.399+0000 7f6553e00640 1 -- 172.21.15.111:0/3274229771 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f655419f6a0 con 0x7f65541047e0 2024-09-15T22:44:36.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.399+0000 7f654b400640 1 -- 172.21.15.111:0/3274229771 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6540010910 con 0x7f65541047e0 2024-09-15T22:44:36.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.399+0000 7f654b400640 1 -- 172.21.15.111:0/3274229771 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65400188f0 con 0x7f65541047e0 2024-09-15T22:44:36.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.399+0000 7f6553e00640 1 -- 172.21.15.111:0/3274229771 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65541a6f80 con 0x7f65541047e0 2024-09-15T22:44:36.407 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.400+0000 7f6553e00640 1 -- 172.21.15.111:0/3274229771 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6554105310 con 0x7f65541047e0 2024-09-15T22:44:36.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.403+0000 7f654b400640 1 -- 172.21.15.111:0/3274229771 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f6540010ac0 con 0x7f65541047e0 2024-09-15T22:44:36.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.404+0000 7f654b400640 1 --2- 172.21.15.111:0/3274229771 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f65380a32c0 0x7f65380a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:36.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.404+0000 7f654b400640 1 -- 172.21.15.111:0/3274229771 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f65400cfed0 con 0x7f65541047e0 2024-09-15T22:44:36.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.404+0000 7f6552a00640 1 --2- 172.21.15.111:0/3274229771 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f65380a32c0 0x7f65380a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:36.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.405+0000 7f6552a00640 1 --2- 172.21.15.111:0/3274229771 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f65380a32c0 0x7f65380a5780 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6544009bd0 tx=0x7f6544005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:36.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.406+0000 7f654b400640 1 -- 172.21.15.111:0/3274229771 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f654009c650 con 0x7f65541047e0 2024-09-15T22:44:36.608 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.604+0000 7f6553e00640 1 -- 172.21.15.111:0/3274229771 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6554062630 con 0x7f65380a32c0 2024-09-15T22:44:36.633 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.631+0000 7f654b400640 1 -- 172.21.15.111:0/3274229771 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6554062630 con 0x7f65380a32c0 2024-09-15T22:44:36.636 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.634+0000 7f6553e00640 1 -- 172.21.15.111:0/3274229771 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f65380a32c0 msgr2=0x7f65380a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:36.636 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.634+0000 7f6553e00640 1 --2- 172.21.15.111:0/3274229771 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f65380a32c0 0x7f65380a5780 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6544009bd0 tx=0x7f6544005f90 comp rx=0 tx=0).stop 2024-09-15T22:44:36.636 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.634+0000 7f6553e00640 1 -- 172.21.15.111:0/3274229771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65541047e0 msgr2=0x7f655419a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:36.637 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.634+0000 7f6553e00640 1 --2- 172.21.15.111:0/3274229771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65541047e0 0x7f655419a940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f654000e9f0 tx=0x7f654000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:44:36.637 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.634+0000 7f6553e00640 1 -- 172.21.15.111:0/3274229771 shutdown_connections 2024-09-15T22:44:36.637 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.634+0000 7f6553e00640 1 --2- 172.21.15.111:0/3274229771 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f65380a32c0 0x7f65380a5780 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:36.637 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.634+0000 7f6553e00640 1 --2- 172.21.15.111:0/3274229771 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6554108f60 0x7f655419ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:36.637 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.634+0000 7f6553e00640 1 --2- 172.21.15.111:0/3274229771 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65541047e0 0x7f655419a940 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:36.637 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.634+0000 7f6553e00640 1 --2- 172.21.15.111:0/3274229771 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6554103e30 0x7f655419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:36.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.634+0000 7f6553e00640 1 -- 172.21.15.111:0/3274229771 >> 172.21.15.111:0/3274229771 conn(0x7f65540ff910 msgr2=0x7f6554100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:36.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.634+0000 7f6553e00640 1 -- 172.21.15.111:0/3274229771 shutdown_connections 2024-09-15T22:44:36.638 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:36.634+0000 7f6553e00640 1 -- 172.21.15.111:0/3274229771 wait complete. 2024-09-15T22:44:36.915 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:44:36 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:44:36.612+0000 7f296d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:44:36.916 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:44:36 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:44:36.612+0000 7f296d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:44:36.916 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:44:36 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:44:36.612+0000 7f296d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:44:36.916 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:44:36 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:44:36.612+0000 7f296d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:44:36.916 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:44:36 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:44:36.612+0000 7f296d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:44:37.528 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-15T22:44:37.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:37 smithi121 ceph-mon[34815]: pgmap v309: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-15T22:44:37.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:37 smithi121 ceph-mon[34815]: from='client.24613 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:44:37.893 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:37.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:37 smithi111 ceph-mon[30629]: pgmap v309: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-15T22:44:37.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:37 smithi111 ceph-mon[30629]: from='client.24613 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:44:38.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:37 smithi192 ceph-mon[35427]: pgmap v309: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-15T22:44:38.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:37 smithi192 ceph-mon[35427]: from='client.24613 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:44:38.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.481+0000 7f3883200640 1 -- 172.21.15.111:0/2832295451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3884074e30 msgr2=0x7f388410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:38.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.481+0000 7f3883200640 1 --2- 172.21.15.111:0/2832295451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3884074e30 0x7f388410dd40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3870009900 tx=0x7f387002a480 comp rx=0 tx=0).stop 2024-09-15T22:44:38.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.481+0000 7f3883200640 1 -- 172.21.15.111:0/2832295451 shutdown_connections 2024-09-15T22:44:38.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.481+0000 7f3883200640 1 --2- 172.21.15.111:0/2832295451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f388410e280 0x7f3884110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:38.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.481+0000 7f3883200640 1 --2- 172.21.15.111:0/2832295451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3884074e30 0x7f388410dd40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:38.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.481+0000 7f3883200640 1 --2- 172.21.15.111:0/2832295451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3884074510 0x7f38840748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:38.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.481+0000 7f3883200640 1 -- 172.21.15.111:0/2832295451 >> 172.21.15.111:0/2832295451 conn(0x7f38840fd740 msgr2=0x7f38840ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:38.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.481+0000 7f3883200640 1 -- 172.21.15.111:0/2832295451 shutdown_connections 2024-09-15T22:44:38.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.481+0000 7f3883200640 1 -- 172.21.15.111:0/2832295451 wait complete. 2024-09-15T22:44:38.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.482+0000 7f3883200640 1 Processor -- start 2024-09-15T22:44:38.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.482+0000 7f3883200640 1 -- start start 2024-09-15T22:44:38.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.482+0000 7f3883200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3884074510 0x7f388419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:38.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.482+0000 7f3883200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3884074e30 0x7f388419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:38.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.482+0000 7f3883200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f388410e280 0x7f388419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:38.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.482+0000 7f3883200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3884114000 con 0x7f388410e280 2024-09-15T22:44:38.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.482+0000 7f3883200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3884113e80 con 0x7f3884074510 2024-09-15T22:44:38.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.482+0000 7f3883200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3884114180 con 0x7f3884074e30 2024-09-15T22:44:38.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.483+0000 7f3882800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f388410e280 0x7f388419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:38.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.483+0000 7f3881e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3884074510 0x7f388419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:38.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.483+0000 7f3881400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3884074e30 0x7f388419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:38.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.483+0000 7f3881e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3884074510 0x7f388419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:38668/0 (socket says 172.21.15.111:38668) 2024-09-15T22:44:38.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.483+0000 7f3882800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f388410e280 0x7f388419edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48346/0 (socket says 172.21.15.111:48346) 2024-09-15T22:44:38.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.483+0000 7f3881e00640 1 -- 172.21.15.111:0/1058115454 learned_addr learned my addr 172.21.15.111:0/1058115454 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:44:38.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.483+0000 7f3881e00640 1 -- 172.21.15.111:0/1058115454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3884074e30 msgr2=0x7f388419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:38.486 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.483+0000 7f3881e00640 1 --2- 172.21.15.111:0/1058115454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3884074e30 0x7f388419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:38.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.483+0000 7f3881e00640 1 -- 172.21.15.111:0/1058115454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f388410e280 msgr2=0x7f388419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:38.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.483+0000 7f3881e00640 1 --2- 172.21.15.111:0/1058115454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f388410e280 0x7f388419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:38.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.483+0000 7f3881e00640 1 -- 172.21.15.111:0/1058115454 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388419f490 con 0x7f3884074510 2024-09-15T22:44:38.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.483+0000 7f3881e00640 1 --2- 172.21.15.111:0/1058115454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3884074510 0x7f388419a4e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f387400e9f0 tx=0x7f387400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:38.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.484+0000 7f387aa00640 1 -- 172.21.15.111:0/1058115454 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387400cd80 con 0x7f3884074510 2024-09-15T22:44:38.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.484+0000 7f3883200640 1 -- 172.21.15.111:0/1058115454 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388419f780 con 0x7f3884074510 2024-09-15T22:44:38.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.484+0000 7f387aa00640 1 -- 172.21.15.111:0/1058115454 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3874005500 con 0x7f3884074510 2024-09-15T22:44:38.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.484+0000 7f387aa00640 1 -- 172.21.15.111:0/1058115454 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3874010690 con 0x7f3884074510 2024-09-15T22:44:38.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.484+0000 7f3883200640 1 -- 172.21.15.111:0/1058115454 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38841a7060 con 0x7f3884074510 2024-09-15T22:44:38.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.485+0000 7f3883200640 1 -- 172.21.15.111:0/1058115454 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38840748f0 con 0x7f3884074510 2024-09-15T22:44:38.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.487+0000 7f387aa00640 1 -- 172.21.15.111:0/1058115454 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) ==== 138356+0+0 (secure 0 0 0) 0x7f38740046c0 con 0x7f3884074510 2024-09-15T22:44:38.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.488+0000 7f387aa00640 1 --2- 172.21.15.111:0/1058115454 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f38680a3240 0x7f38680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:38.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.488+0000 7f387aa00640 1 -- 172.21.15.111:0/1058115454 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f38740cfe70 con 0x7f3884074510 2024-09-15T22:44:38.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.488+0000 7f3881400640 1 --2- 172.21.15.111:0/1058115454 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f38680a3240 0x7f38680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:38.490 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.489+0000 7f3881400640 1 --2- 172.21.15.111:0/1058115454 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f38680a3240 0x7f38680a5700 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3870000c00 tx=0x7f3870005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:38.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.490+0000 7f387aa00640 1 -- 172.21.15.111:0/1058115454 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f38740541b0 con 0x7f3884074510 2024-09-15T22:44:38.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.594+0000 7f387aa00640 1 -- 172.21.15.111:0/1058115454 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f3874098bb0 con 0x7f3884074510 2024-09-15T22:44:38.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.697+0000 7f3883200640 1 -- 172.21.15.111:0/1058115454 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3884109470 con 0x7f38680a3240 2024-09-15T22:44:38.722 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.721+0000 7f387aa00640 1 -- 172.21.15.111:0/1058115454 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3870004280 con 0x7f38680a3240 2024-09-15T22:44:38.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.723+0000 7f3883200640 1 -- 172.21.15.111:0/1058115454 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f38680a3240 msgr2=0x7f38680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:38.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.723+0000 7f3883200640 1 --2- 172.21.15.111:0/1058115454 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f38680a3240 0x7f38680a5700 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3870000c00 tx=0x7f3870005b50 comp rx=0 tx=0).stop 2024-09-15T22:44:38.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.723+0000 7f3883200640 1 -- 172.21.15.111:0/1058115454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3884074510 msgr2=0x7f388419a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:38.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.723+0000 7f3883200640 1 --2- 172.21.15.111:0/1058115454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3884074510 0x7f388419a4e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f387400e9f0 tx=0x7f387400edf0 comp rx=0 tx=0).stop 2024-09-15T22:44:38.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.723+0000 7f3883200640 1 -- 172.21.15.111:0/1058115454 shutdown_connections 2024-09-15T22:44:38.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.723+0000 7f3883200640 1 --2- 172.21.15.111:0/1058115454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f388410e280 0x7f388419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:38.725 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.723+0000 7f3883200640 1 --2- 172.21.15.111:0/1058115454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3884074e30 0x7f388419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:38.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.723+0000 7f3883200640 1 --2- 172.21.15.111:0/1058115454 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f38680a3240 0x7f38680a5700 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:38.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.723+0000 7f3883200640 1 --2- 172.21.15.111:0/1058115454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3884074510 0x7f388419a4e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:38.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.723+0000 7f3883200640 1 -- 172.21.15.111:0/1058115454 >> 172.21.15.111:0/1058115454 conn(0x7f38840fd740 msgr2=0x7f38840fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:38.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.723+0000 7f3883200640 1 -- 172.21.15.111:0/1058115454 shutdown_connections 2024-09-15T22:44:38.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:38.724+0000 7f3883200640 1 -- 172.21.15.111:0/1058115454 wait complete. 2024-09-15T22:44:39.675 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-15T22:44:39.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:39 smithi121 ceph-mon[34815]: pgmap v310: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-15T22:44:39.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:39 smithi121 ceph-mon[34815]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-15T22:44:39.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:39 smithi111 ceph-mon[30629]: pgmap v310: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-15T22:44:39.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:39 smithi111 ceph-mon[30629]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-15T22:44:40.042 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:40.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:39 smithi192 ceph-mon[35427]: pgmap v310: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-15T22:44:40.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:39 smithi192 ceph-mon[35427]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-15T22:44:40.642 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.638+0000 7fdae1400640 1 -- 172.21.15.111:0/81049002 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdadc1047c0 msgr2=0x7fdadc108810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:40.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.638+0000 7fdae1400640 1 --2- 172.21.15.111:0/81049002 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdadc1047c0 0x7fdadc108810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009900 tx=0x7fdac802a410 comp rx=0 tx=0).stop 2024-09-15T22:44:40.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.638+0000 7fdae1400640 1 -- 172.21.15.111:0/81049002 shutdown_connections 2024-09-15T22:44:40.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.638+0000 7fdae1400640 1 --2- 172.21.15.111:0/81049002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdadc108f40 0x7fdadc110ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:40.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.638+0000 7fdae1400640 1 --2- 172.21.15.111:0/81049002 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdadc1047c0 0x7fdadc108810 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:40.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.638+0000 7fdae1400640 1 --2- 172.21.15.111:0/81049002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdadc103e10 0x7fdadc1041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:40.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.638+0000 7fdae1400640 1 -- 172.21.15.111:0/81049002 >> 172.21.15.111:0/81049002 conn(0x7fdadc0ff870 msgr2=0x7fdadc101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:40.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.638+0000 7fdae1400640 1 -- 172.21.15.111:0/81049002 shutdown_connections 2024-09-15T22:44:40.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.638+0000 7fdae1400640 1 -- 172.21.15.111:0/81049002 wait complete. 2024-09-15T22:44:40.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.639+0000 7fdae1400640 1 Processor -- start 2024-09-15T22:44:40.643 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.639+0000 7fdae1400640 1 -- start start 2024-09-15T22:44:40.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.639+0000 7fdae1400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdadc103e10 0x7fdadc19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:40.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.639+0000 7fdae1400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdadc1047c0 0x7fdadc19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:40.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.639+0000 7fdae1400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdadc108f40 0x7fdadc19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:40.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.639+0000 7fdae1400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc113fb0 con 0x7fdadc103e10 2024-09-15T22:44:40.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.639+0000 7fdae1400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc113e30 con 0x7fdadc1047c0 2024-09-15T22:44:40.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.639+0000 7fdae1400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdadc114130 con 0x7fdadc108f40 2024-09-15T22:44:40.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.640+0000 7fdadbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdadc103e10 0x7fdadc19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:40.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.640+0000 7fdadb400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdadc1047c0 0x7fdadc19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:40.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.640+0000 7fdadbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdadc103e10 0x7fdadc19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48358/0 (socket says 172.21.15.111:48358) 2024-09-15T22:44:40.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.640+0000 7fdadbe00640 1 -- 172.21.15.111:0/3680234363 learned_addr learned my addr 172.21.15.111:0/3680234363 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:44:40.644 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.640+0000 7fdae0a00640 1 --2- 172.21.15.111:0/3680234363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdadc108f40 0x7fdadc19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:40.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.640+0000 7fdadbe00640 1 -- 172.21.15.111:0/3680234363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdadc108f40 msgr2=0x7fdadc19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:40.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.640+0000 7fdadbe00640 1 --2- 172.21.15.111:0/3680234363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdadc108f40 0x7fdadc19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:40.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.640+0000 7fdadbe00640 1 -- 172.21.15.111:0/3680234363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdadc1047c0 msgr2=0x7fdadc19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:40.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.640+0000 7fdadbe00640 1 --2- 172.21.15.111:0/3680234363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdadc1047c0 0x7fdadc19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:40.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.640+0000 7fdadbe00640 1 -- 172.21.15.111:0/3680234363 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdadc19f410 con 0x7fdadc103e10 2024-09-15T22:44:40.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.640+0000 7fdadbe00640 1 --2- 172.21.15.111:0/3680234363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdadc103e10 0x7fdadc19a460 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00d950 tx=0x7fdacc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:40.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.641+0000 7fdad8c00640 1 -- 172.21.15.111:0/3680234363 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc017030 con 0x7fdadc103e10 2024-09-15T22:44:40.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.641+0000 7fdad8c00640 1 -- 172.21.15.111:0/3680234363 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdacc004020 con 0x7fdadc103e10 2024-09-15T22:44:40.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.641+0000 7fdae1400640 1 -- 172.21.15.111:0/3680234363 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdadc19f700 con 0x7fdadc103e10 2024-09-15T22:44:40.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.641+0000 7fdad8c00640 1 -- 172.21.15.111:0/3680234363 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdacc0047a0 con 0x7fdadc103e10 2024-09-15T22:44:40.645 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.641+0000 7fdae1400640 1 -- 172.21.15.111:0/3680234363 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdadc1a6fe0 con 0x7fdadc103e10 2024-09-15T22:44:40.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.642+0000 7fdae1400640 1 -- 172.21.15.111:0/3680234363 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdadc1041f0 con 0x7fdadc103e10 2024-09-15T22:44:40.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.644+0000 7fdad8c00640 1 -- 172.21.15.111:0/3680234363 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fdacc0041d0 con 0x7fdadc103e10 2024-09-15T22:44:40.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.645+0000 7fdad8c00640 1 --2- 172.21.15.111:0/3680234363 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fdac00a3350 0x7fdac00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:40.646 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.645+0000 7fdad8c00640 1 -- 172.21.15.111:0/3680234363 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fdacc0cf2d0 con 0x7fdadc103e10 2024-09-15T22:44:40.647 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.645+0000 7fdadb400640 1 --2- 172.21.15.111:0/3680234363 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fdac00a3350 0x7fdac00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:40.647 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.646+0000 7fdadb400640 1 --2- 172.21.15.111:0/3680234363 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fdac00a3350 0x7fdac00a5810 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdac8000c00 tx=0x7fdac8005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:40.651 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.649+0000 7fdad8c00640 1 -- 172.21.15.111:0/3680234363 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdacc09bf40 con 0x7fdadc103e10 2024-09-15T22:44:40.853 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.851+0000 7fdae1400640 1 -- 172.21.15.111:0/3680234363 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fdadc062630 con 0x7fdac00a3350 2024-09-15T22:44:40.853 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:40 smithi111 ceph-mon[30629]: from='client.24629 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:44:40.858 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.856+0000 7fdad8c00640 1 -- 172.21.15.111:0/3680234363 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fdadc062630 con 0x7fdac00a3350 2024-09-15T22:44:40.858 INFO:teuthology.orchestra.run.smithi111.stdout:/volumes/qa/sv_0/5299f0e1-6d44-4a9e-9c38-c3d1cc5d8569 2024-09-15T22:44:40.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.858+0000 7fdae1400640 1 -- 172.21.15.111:0/3680234363 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fdac00a3350 msgr2=0x7fdac00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:40.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.858+0000 7fdae1400640 1 --2- 172.21.15.111:0/3680234363 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fdac00a3350 0x7fdac00a5810 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdac8000c00 tx=0x7fdac8005b50 comp rx=0 tx=0).stop 2024-09-15T22:44:40.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.858+0000 7fdae1400640 1 -- 172.21.15.111:0/3680234363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdadc103e10 msgr2=0x7fdadc19a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:40.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.858+0000 7fdae1400640 1 --2- 172.21.15.111:0/3680234363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdadc103e10 0x7fdadc19a460 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00d950 tx=0x7fdacc00de10 comp rx=0 tx=0).stop 2024-09-15T22:44:40.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.858+0000 7fdae1400640 1 -- 172.21.15.111:0/3680234363 shutdown_connections 2024-09-15T22:44:40.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.858+0000 7fdae1400640 1 --2- 172.21.15.111:0/3680234363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdadc108f40 0x7fdadc19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:40.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.858+0000 7fdae1400640 1 --2- 172.21.15.111:0/3680234363 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fdac00a3350 0x7fdac00a5810 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:40.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.858+0000 7fdae1400640 1 --2- 172.21.15.111:0/3680234363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdadc1047c0 0x7fdadc19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:40.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.858+0000 7fdae1400640 1 --2- 172.21.15.111:0/3680234363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdadc103e10 0x7fdadc19a460 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:40.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.858+0000 7fdae1400640 1 -- 172.21.15.111:0/3680234363 >> 172.21.15.111:0/3680234363 conn(0x7fdadc0ff870 msgr2=0x7fdadc100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:40.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.859+0000 7fdae1400640 1 -- 172.21.15.111:0/3680234363 shutdown_connections 2024-09-15T22:44:40.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:40.859+0000 7fdae1400640 1 -- 172.21.15.111:0/3680234363 wait complete. 2024-09-15T22:44:40.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:40 smithi121 ceph-mon[34815]: from='client.24629 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:44:41.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:40 smithi192 ceph-mon[35427]: from='client.24629 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:44:41.758 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-15T22:44:41.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:41 smithi121 ceph-mon[34815]: pgmap v311: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-09-15T22:44:41.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:41 smithi111 ceph-mon[30629]: pgmap v311: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-09-15T22:44:42.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:41 smithi192 ceph-mon[35427]: pgmap v311: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-09-15T22:44:42.124 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:42.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.713+0000 7f8c14000640 1 -- 172.21.15.111:0/134976152 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c0c100a80 msgr2=0x7f8c0c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:42.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.713+0000 7f8c14000640 1 --2- 172.21.15.111:0/134976152 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c0c100a80 0x7f8c0c100ee0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8009930 tx=0x7f8bf802a420 comp rx=0 tx=0).stop 2024-09-15T22:44:42.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.713+0000 7f8c14000640 1 -- 172.21.15.111:0/134976152 shutdown_connections 2024-09-15T22:44:42.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.713+0000 7f8c14000640 1 --2- 172.21.15.111:0/134976152 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c0c101420 0x7f8c0c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:42.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.713+0000 7f8c14000640 1 --2- 172.21.15.111:0/134976152 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c0c100a80 0x7f8c0c100ee0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:42.741 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.713+0000 7f8c14000640 1 --2- 172.21.15.111:0/134976152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c0c106a80 0x7f8c0c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:42.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.713+0000 7f8c14000640 1 -- 172.21.15.111:0/134976152 >> 172.21.15.111:0/134976152 conn(0x7f8c0c0fc7a0 msgr2=0x7f8c0c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:42.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.713+0000 7f8c14000640 1 -- 172.21.15.111:0/134976152 shutdown_connections 2024-09-15T22:44:42.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.713+0000 7f8c14000640 1 -- 172.21.15.111:0/134976152 wait complete. 2024-09-15T22:44:42.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.714+0000 7f8c14000640 1 Processor -- start 2024-09-15T22:44:42.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.714+0000 7f8c14000640 1 -- start start 2024-09-15T22:44:42.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.715+0000 7f8c14000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c0c100a80 0x7f8c0c110aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:42.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.715+0000 7f8c14000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c0c101420 0x7f8c0c110fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:42.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.715+0000 7f8c14000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c0c106a80 0x7f8c0c111520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:42.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.715+0000 7f8c14000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c06ac40 con 0x7f8c0c100a80 2024-09-15T22:44:42.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.715+0000 7f8c14000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c06aac0 con 0x7f8c0c101420 2024-09-15T22:44:42.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.715+0000 7f8c14000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c06adc0 con 0x7f8c0c106a80 2024-09-15T22:44:42.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.715+0000 7f8c11600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c0c100a80 0x7f8c0c110aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:42.742 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.715+0000 7f8c11600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c0c100a80 0x7f8c0c110aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:49314/0 (socket says 172.21.15.111:49314) 2024-09-15T22:44:42.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.715+0000 7f8c11600640 1 -- 172.21.15.111:0/2287071067 learned_addr learned my addr 172.21.15.111:0/2287071067 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:44:42.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.715+0000 7f8c12000640 1 --2- 172.21.15.111:0/2287071067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c0c106a80 0x7f8c0c111520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:42.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.715+0000 7f8c10c00640 1 --2- 172.21.15.111:0/2287071067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c0c101420 0x7f8c0c110fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:42.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.715+0000 7f8c11600640 1 -- 172.21.15.111:0/2287071067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c0c106a80 msgr2=0x7f8c0c111520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:42.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.715+0000 7f8c11600640 1 --2- 172.21.15.111:0/2287071067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c0c106a80 0x7f8c0c111520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:42.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.715+0000 7f8c11600640 1 -- 172.21.15.111:0/2287071067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c0c101420 msgr2=0x7f8c0c110fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:42.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.715+0000 7f8c11600640 1 --2- 172.21.15.111:0/2287071067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c0c101420 0x7f8c0c110fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:42.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.715+0000 7f8c11600640 1 -- 172.21.15.111:0/2287071067 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0c1aae50 con 0x7f8c0c100a80 2024-09-15T22:44:42.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.715+0000 7f8c12000640 1 --2- 172.21.15.111:0/2287071067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c0c106a80 0x7f8c0c111520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:42.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.716+0000 7f8c11600640 1 --2- 172.21.15.111:0/2287071067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c0c100a80 0x7f8c0c110aa0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000e9f0 tx=0x7f8c0000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:42.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.716+0000 7f8c0a000640 1 -- 172.21.15.111:0/2287071067 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0000cd80 con 0x7f8c0c100a80 2024-09-15T22:44:42.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.716+0000 7f8c0a000640 1 -- 172.21.15.111:0/2287071067 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c00005500 con 0x7f8c0c100a80 2024-09-15T22:44:42.743 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.716+0000 7f8c0a000640 1 -- 172.21.15.111:0/2287071067 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c00010690 con 0x7f8c0c100a80 2024-09-15T22:44:42.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.716+0000 7f8c14000640 1 -- 172.21.15.111:0/2287071067 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c0c1ab080 con 0x7f8c0c100a80 2024-09-15T22:44:42.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.716+0000 7f8c14000640 1 -- 172.21.15.111:0/2287071067 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c0c1ab540 con 0x7f8c0c100a80 2024-09-15T22:44:42.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.717+0000 7f8c14000640 1 -- 172.21.15.111:0/2287071067 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c0c1021c0 con 0x7f8c0c100a80 2024-09-15T22:44:42.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.719+0000 7f8c0a000640 1 -- 172.21.15.111:0/2287071067 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f8c000046c0 con 0x7f8c0c100a80 2024-09-15T22:44:42.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.720+0000 7f8c0a000640 1 --2- 172.21.15.111:0/2287071067 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8bf40a3350 0x7f8bf40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:42.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.720+0000 7f8c0a000640 1 -- 172.21.15.111:0/2287071067 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f8c000cfec0 con 0x7f8c0c100a80 2024-09-15T22:44:42.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.720+0000 7f8c10c00640 1 --2- 172.21.15.111:0/2287071067 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8bf40a3350 0x7f8bf40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:42.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.721+0000 7f8c10c00640 1 --2- 172.21.15.111:0/2287071067 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8bf40a3350 0x7f8bf40a5810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8006010 tx=0x7f8bf8005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:42.744 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.724+0000 7f8c0a000640 1 -- 172.21.15.111:0/2287071067 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8c0009cb30 con 0x7f8c0c100a80 2024-09-15T22:44:42.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:42 smithi121 ceph-mon[34815]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:44:42.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:42 smithi111 ceph-mon[30629]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:44:42.925 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.924+0000 7f8c14000640 1 -- 172.21.15.111:0/2287071067 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f8c0c0fd290 con 0x7f8bf40a3350 2024-09-15T22:44:42.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.948+0000 7f8c0a000640 1 -- 172.21.15.111:0/2287071067 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8c0c0fd290 con 0x7f8bf40a3350 2024-09-15T22:44:42.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.950+0000 7f8c14000640 1 -- 172.21.15.111:0/2287071067 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8bf40a3350 msgr2=0x7f8bf40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:42.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.950+0000 7f8c14000640 1 --2- 172.21.15.111:0/2287071067 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8bf40a3350 0x7f8bf40a5810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8006010 tx=0x7f8bf8005a20 comp rx=0 tx=0).stop 2024-09-15T22:44:42.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.950+0000 7f8c14000640 1 -- 172.21.15.111:0/2287071067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c0c100a80 msgr2=0x7f8c0c110aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:42.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.950+0000 7f8c14000640 1 --2- 172.21.15.111:0/2287071067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c0c100a80 0x7f8c0c110aa0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000e9f0 tx=0x7f8c0000edf0 comp rx=0 tx=0).stop 2024-09-15T22:44:42.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.950+0000 7f8c14000640 1 -- 172.21.15.111:0/2287071067 shutdown_connections 2024-09-15T22:44:42.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.950+0000 7f8c14000640 1 --2- 172.21.15.111:0/2287071067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c0c106a80 0x7f8c0c111520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:42.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.950+0000 7f8c14000640 1 --2- 172.21.15.111:0/2287071067 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8bf40a3350 0x7f8bf40a5810 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:42.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.950+0000 7f8c14000640 1 --2- 172.21.15.111:0/2287071067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c0c101420 0x7f8c0c110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:42.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.950+0000 7f8c14000640 1 --2- 172.21.15.111:0/2287071067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c0c100a80 0x7f8c0c110aa0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:42.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.950+0000 7f8c14000640 1 -- 172.21.15.111:0/2287071067 >> 172.21.15.111:0/2287071067 conn(0x7f8c0c0fc7a0 msgr2=0x7f8c0c0fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:42.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.951+0000 7f8c14000640 1 -- 172.21.15.111:0/2287071067 shutdown_connections 2024-09-15T22:44:42.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:42.951+0000 7f8c14000640 1 -- 172.21.15.111:0/2287071067 wait complete. 2024-09-15T22:44:43.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:42 smithi192 ceph-mon[35427]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:44:43.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:43 smithi192 ceph-mon[35427]: pgmap v312: 97 pgs: 97 active+clean; 607 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-09-15T22:44:43.849 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-15T22:44:43.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:43 smithi121 ceph-mon[34815]: pgmap v312: 97 pgs: 97 active+clean; 607 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-09-15T22:44:43.897 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:43 smithi111 ceph-mon[30629]: pgmap v312: 97 pgs: 97 active+clean; 607 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-09-15T22:44:44.218 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:44.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:44 smithi111 ceph-mon[30629]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:44:44.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.831+0000 7f666e200640 1 -- 172.21.15.111:0/1471828310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6668106f00 msgr2=0x7f666810ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:44.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.831+0000 7f666e200640 1 --2- 172.21.15.111:0/1471828310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6668106f00 0x7f666810ea80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6660009ca0 tx=0x7f666002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:44:44.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.831+0000 7f666e200640 1 -- 172.21.15.111:0/1471828310 shutdown_connections 2024-09-15T22:44:44.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.831+0000 7f666e200640 1 --2- 172.21.15.111:0/1471828310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6668106f00 0x7f666810ea80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:44.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.831+0000 7f666e200640 1 --2- 172.21.15.111:0/1471828310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6668102780 0x7f66681067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:44.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.831+0000 7f666e200640 1 --2- 172.21.15.111:0/1471828310 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6668101dd0 0x7f66681021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:44.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.831+0000 7f666e200640 1 -- 172.21.15.111:0/1471828310 >> 172.21.15.111:0/1471828310 conn(0x7f66680fd8b0 msgr2=0x7f66680ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:44.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.831+0000 7f666e200640 1 -- 172.21.15.111:0/1471828310 shutdown_connections 2024-09-15T22:44:44.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.831+0000 7f666e200640 1 -- 172.21.15.111:0/1471828310 wait complete. 2024-09-15T22:44:44.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.832+0000 7f666e200640 1 Processor -- start 2024-09-15T22:44:44.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.832+0000 7f666e200640 1 -- start start 2024-09-15T22:44:44.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.833+0000 7f666e200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6668101dd0 0x7f666819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:44.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.833+0000 7f666e200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6668102780 0x7f666819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:44.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.833+0000 7f666e200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6668106f00 0x7f666819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:44.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.833+0000 7f666e200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6668111fb0 con 0x7f6668102780 2024-09-15T22:44:44.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.833+0000 7f666e200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6668111e30 con 0x7f6668101dd0 2024-09-15T22:44:44.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.833+0000 7f666e200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6668112130 con 0x7f6668106f00 2024-09-15T22:44:44.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.833+0000 7f6666a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6668102780 0x7f666819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:44.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.833+0000 7f6666a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6668102780 0x7f666819ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:49332/0 (socket says 172.21.15.111:49332) 2024-09-15T22:44:44.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.833+0000 7f6666a00640 1 -- 172.21.15.111:0/3897353691 learned_addr learned my addr 172.21.15.111:0/3897353691 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:44:44.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.833+0000 7f6667400640 1 --2- 172.21.15.111:0/3897353691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6668101dd0 0x7f666819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:44.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.833+0000 7f6667e00640 1 --2- 172.21.15.111:0/3897353691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6668106f00 0x7f666819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:44.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.833+0000 7f6666a00640 1 -- 172.21.15.111:0/3897353691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6668106f00 msgr2=0x7f666819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:44.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.833+0000 7f6666a00640 1 --2- 172.21.15.111:0/3897353691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6668106f00 0x7f666819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:44.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.833+0000 7f6666a00640 1 -- 172.21.15.111:0/3897353691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6668101dd0 msgr2=0x7f666819a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:44.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.833+0000 7f6666a00640 1 --2- 172.21.15.111:0/3897353691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6668101dd0 0x7f666819a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:44.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.833+0000 7f6666a00640 1 -- 172.21.15.111:0/3897353691 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666819f5c0 con 0x7f6668102780 2024-09-15T22:44:44.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.834+0000 7f6666a00640 1 --2- 172.21.15.111:0/3897353691 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6668102780 0x7f666819ab50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f665400ea60 tx=0x7f665400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:44.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.834+0000 7f665fe00640 1 -- 172.21.15.111:0/3897353691 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665400cde0 con 0x7f6668102780 2024-09-15T22:44:44.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.834+0000 7f665fe00640 1 -- 172.21.15.111:0/3897353691 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66540035f0 con 0x7f6668102780 2024-09-15T22:44:44.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.834+0000 7f665fe00640 1 -- 172.21.15.111:0/3897353691 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6654010670 con 0x7f6668102780 2024-09-15T22:44:44.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.834+0000 7f666e200640 1 -- 172.21.15.111:0/3897353691 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f666819f8b0 con 0x7f6668102780 2024-09-15T22:44:44.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.834+0000 7f666e200640 1 -- 172.21.15.111:0/3897353691 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66681a7190 con 0x7f6668102780 2024-09-15T22:44:44.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.835+0000 7f666e200640 1 -- 172.21.15.111:0/3897353691 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66681021b0 con 0x7f6668102780 2024-09-15T22:44:44.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.840+0000 7f665fe00640 1 -- 172.21.15.111:0/3897353691 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f66540037a0 con 0x7f6668102780 2024-09-15T22:44:44.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.842+0000 7f665fe00640 1 --2- 172.21.15.111:0/3897353691 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f664c0a3420 0x7f664c0a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:44.843 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.842+0000 7f665fe00640 1 -- 172.21.15.111:0/3897353691 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f66540cfb60 con 0x7f6668102780 2024-09-15T22:44:44.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.842+0000 7f6667400640 1 --2- 172.21.15.111:0/3897353691 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f664c0a3420 0x7f664c0a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:44.844 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.843+0000 7f6667400640 1 --2- 172.21.15.111:0/3897353691 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f664c0a3420 0x7f664c0a58e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6658009bd0 tx=0x7f6658005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:44.848 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:44.846+0000 7f665fe00640 1 -- 172.21.15.111:0/3897353691 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f665409c7d0 con 0x7f6668102780 2024-09-15T22:44:44.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:44 smithi121 ceph-mon[34815]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:44:45.057 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:45.056+0000 7f666e200640 1 -- 172.21.15.111:0/3897353691 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f66680febe0 con 0x7f664c0a3420 2024-09-15T22:44:45.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:45.062+0000 7f665fe00640 1 -- 172.21.15.111:0/3897353691 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f66680febe0 con 0x7f664c0a3420 2024-09-15T22:44:45.063 INFO:teuthology.orchestra.run.smithi111.stdout:/volumes/qa/sv_1/507a8ade-25bd-410d-99a3-61a663cb7a66 2024-09-15T22:44:45.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:44 smithi192 ceph-mon[35427]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:44:45.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:45.064+0000 7f666e200640 1 -- 172.21.15.111:0/3897353691 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f664c0a3420 msgr2=0x7f664c0a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:45.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:45.064+0000 7f666e200640 1 --2- 172.21.15.111:0/3897353691 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f664c0a3420 0x7f664c0a58e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6658009bd0 tx=0x7f6658005f90 comp rx=0 tx=0).stop 2024-09-15T22:44:45.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:45.064+0000 7f666e200640 1 -- 172.21.15.111:0/3897353691 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6668102780 msgr2=0x7f666819ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:45.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:45.064+0000 7f666e200640 1 --2- 172.21.15.111:0/3897353691 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6668102780 0x7f666819ab50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f665400ea60 tx=0x7f665400ef20 comp rx=0 tx=0).stop 2024-09-15T22:44:45.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:45.064+0000 7f666e200640 1 -- 172.21.15.111:0/3897353691 shutdown_connections 2024-09-15T22:44:45.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:45.064+0000 7f666e200640 1 --2- 172.21.15.111:0/3897353691 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6668106f00 0x7f666819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:45.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:45.064+0000 7f666e200640 1 --2- 172.21.15.111:0/3897353691 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6668102780 0x7f666819ab50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:45.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:45.064+0000 7f666e200640 1 --2- 172.21.15.111:0/3897353691 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f664c0a3420 0x7f664c0a58e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:45.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:45.064+0000 7f666e200640 1 --2- 172.21.15.111:0/3897353691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6668101dd0 0x7f666819a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:45.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:45.064+0000 7f666e200640 1 -- 172.21.15.111:0/3897353691 >> 172.21.15.111:0/3897353691 conn(0x7f66680fd8b0 msgr2=0x7f66680ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:45.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:45.065+0000 7f666e200640 1 -- 172.21.15.111:0/3897353691 shutdown_connections 2024-09-15T22:44:45.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:45.065+0000 7f666e200640 1 -- 172.21.15.111:0/3897353691 wait complete. 2024-09-15T22:44:45.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:45 smithi121 ceph-mon[34815]: pgmap v313: 97 pgs: 97 active+clean; 607 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-15T22:44:45.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:45 smithi111 ceph-mon[30629]: pgmap v313: 97 pgs: 97 active+clean; 607 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-15T22:44:46.045 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:44:46.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:45 smithi192 ceph-mon[35427]: pgmap v313: 97 pgs: 97 active+clean; 607 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-15T22:44:46.415 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:46.873 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:46 smithi111 ceph-mon[30629]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:44:46.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:46 smithi121 ceph-mon[34815]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:44:47.021 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.981+0000 7f121ae00640 1 -- 172.21.15.111:0/1363301976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1214074590 msgr2=0x7f1214074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:47.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.981+0000 7f121ae00640 1 --2- 172.21.15.111:0/1363301976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1214074590 0x7f1214074970 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1204009900 tx=0x7f120402a480 comp rx=0 tx=0).stop 2024-09-15T22:44:47.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.981+0000 7f121ae00640 1 -- 172.21.15.111:0/1363301976 shutdown_connections 2024-09-15T22:44:47.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.981+0000 7f121ae00640 1 --2- 172.21.15.111:0/1363301976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1214112b90 0x7f1214114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:47.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.981+0000 7f121ae00640 1 --2- 172.21.15.111:0/1363301976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1214074eb0 0x7f1214112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:47.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.981+0000 7f121ae00640 1 --2- 172.21.15.111:0/1363301976 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1214074590 0x7f1214074970 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:47.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.981+0000 7f121ae00640 1 -- 172.21.15.111:0/1363301976 >> 172.21.15.111:0/1363301976 conn(0x7f12140fd8c0 msgr2=0x7f12140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:47.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.981+0000 7f121ae00640 1 -- 172.21.15.111:0/1363301976 shutdown_connections 2024-09-15T22:44:47.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.981+0000 7f121ae00640 1 -- 172.21.15.111:0/1363301976 wait complete. 2024-09-15T22:44:47.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.982+0000 7f121ae00640 1 Processor -- start 2024-09-15T22:44:47.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.982+0000 7f121ae00640 1 -- start start 2024-09-15T22:44:47.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.983+0000 7f121ae00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1214074590 0x7f12141a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:47.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.983+0000 7f121ae00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1214074eb0 0x7f12141a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:47.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.983+0000 7f121ae00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1214112b90 0x7f121419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:47.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.983+0000 7f121ae00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f12141185c0 con 0x7f1214074eb0 2024-09-15T22:44:47.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.983+0000 7f121ae00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1214118440 con 0x7f1214112b90 2024-09-15T22:44:47.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.983+0000 7f121ae00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1214118740 con 0x7f1214074590 2024-09-15T22:44:47.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.983+0000 7f1213400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1214074eb0 0x7f12141a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:47.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.983+0000 7f1213e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1214074590 0x7f12141a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:47.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.983+0000 7f1213400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1214074eb0 0x7f12141a4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:49346/0 (socket says 172.21.15.111:49346) 2024-09-15T22:44:47.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.983+0000 7f1213400640 1 -- 172.21.15.111:0/576711545 learned_addr learned my addr 172.21.15.111:0/576711545 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:44:47.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.983+0000 7f1213e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1214074590 0x7f12141a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:45250/0 (socket says 172.21.15.111:45250) 2024-09-15T22:44:47.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.983+0000 7f1218e00640 1 --2- 172.21.15.111:0/576711545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1214112b90 0x7f121419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:47.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.984+0000 7f1213400640 1 -- 172.21.15.111:0/576711545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1214074590 msgr2=0x7f12141a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:47.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.984+0000 7f1213400640 1 --2- 172.21.15.111:0/576711545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1214074590 0x7f12141a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:47.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.984+0000 7f1213400640 1 -- 172.21.15.111:0/576711545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1214112b90 msgr2=0x7f121419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:47.023 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.984+0000 7f1213400640 1 --2- 172.21.15.111:0/576711545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1214112b90 0x7f121419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:47.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.984+0000 7f1213400640 1 -- 172.21.15.111:0/576711545 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121419efe0 con 0x7f1214074eb0 2024-09-15T22:44:47.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.984+0000 7f1218e00640 1 --2- 172.21.15.111:0/576711545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1214112b90 0x7f121419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:44:47.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.984+0000 7f1213400640 1 --2- 172.21.15.111:0/576711545 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1214074eb0 0x7f12141a4c30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f120000e9f0 tx=0x7f120000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:47.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.984+0000 7f1210c00640 1 -- 172.21.15.111:0/576711545 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120000cd80 con 0x7f1214074eb0 2024-09-15T22:44:47.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.984+0000 7f1210c00640 1 -- 172.21.15.111:0/576711545 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1200010910 con 0x7f1214074eb0 2024-09-15T22:44:47.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.984+0000 7f121ae00640 1 -- 172.21.15.111:0/576711545 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f121419f2d0 con 0x7f1214074eb0 2024-09-15T22:44:47.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.984+0000 7f1210c00640 1 -- 172.21.15.111:0/576711545 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1200018990 con 0x7f1214074eb0 2024-09-15T22:44:47.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.984+0000 7f121ae00640 1 -- 172.21.15.111:0/576711545 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12141ab5c0 con 0x7f1214074eb0 2024-09-15T22:44:47.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.985+0000 7f121ae00640 1 -- 172.21.15.111:0/576711545 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1214074970 con 0x7f1214074eb0 2024-09-15T22:44:47.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.988+0000 7f1210c00640 1 -- 172.21.15.111:0/576711545 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f1200010ac0 con 0x7f1214074eb0 2024-09-15T22:44:47.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.989+0000 7f1210c00640 1 --2- 172.21.15.111:0/576711545 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f11f80a32d0 0x7f11f80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:47.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.989+0000 7f1210c00640 1 -- 172.21.15.111:0/576711545 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f12000d0110 con 0x7f1214074eb0 2024-09-15T22:44:47.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.989+0000 7f1213e00640 1 --2- 172.21.15.111:0/576711545 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f11f80a32d0 0x7f11f80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:47.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.990+0000 7f1213e00640 1 --2- 172.21.15.111:0/576711545 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f11f80a32d0 0x7f11f80a5790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1204006010 tx=0x7f1204000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:47.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:46.990+0000 7f1210c00640 1 -- 172.21.15.111:0/576711545 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f120009cd80 con 0x7f1214074eb0 2024-09-15T22:44:47.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:46 smithi192 ceph-mon[35427]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:44:47.204 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:47.202+0000 7f121ae00640 1 -- 172.21.15.111:0/576711545 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12141ab8b0 con 0x7f1214074eb0 2024-09-15T22:44:47.205 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:47.203+0000 7f1210c00640 1 -- 172.21.15.111:0/576711545 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11931 (secure 0 0 0) 0x7f120009c680 con 0x7f1214074eb0 2024-09-15T22:44:47.205 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:44:47.205 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":25,"btime":"2024-09-15T22:44:34:567598+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/453336603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":453336603},{"type":"v1","addr":"172.21.15.111:6835","nonce":453336603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/3181832287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3181832287},{"type":"v1","addr":"172.21.15.111:6837","nonce":3181832287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/4190237053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":4190237053},{"type":"v1","addr":"172.21.15.111:6839","nonce":4190237053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14502,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6841/267728828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6840","nonce":267728828},{"type":"v1","addr":"172.21.15.111:6841","nonce":267728828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/4061441359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":4061441359},{"type":"v1","addr":"172.21.15.192:6833","nonce":4061441359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/4220531578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":4220531578},{"type":"v1","addr":"172.21.15.121:6833","nonce":4220531578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:44:18.436394+0000","modified":"2024-09-15T22:44:33.655164+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24514,"mds_2":24503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3371796806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3371796806},{"type":"v1","addr":"172.21.15.121:6835","nonce":3371796806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/4080914921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":4080914921},{"type":"v1","addr":"172.21.15.192:6835","nonce":4080914921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/3995501707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3995501707},{"type":"v1","addr":"172.21.15.192:6837","nonce":3995501707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.121:6837/1443772791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1443772791},{"type":"v1","addr":"172.21.15.121:6837","nonce":1443772791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6839/800136261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":800136261},{"type":"v1","addr":"172.21.15.192:6839","nonce":800136261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.121:6839/1797352097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1797352097},{"type":"v1","addr":"172.21.15.121:6839","nonce":1797352097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545,24514,24503]},"id":1}]} 2024-09-15T22:44:47.205 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 25 2024-09-15T22:44:47.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:47.206+0000 7f121ae00640 1 -- 172.21.15.111:0/576711545 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f11f80a32d0 msgr2=0x7f11f80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:47.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:47.206+0000 7f121ae00640 1 --2- 172.21.15.111:0/576711545 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f11f80a32d0 0x7f11f80a5790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1204006010 tx=0x7f1204000bc0 comp rx=0 tx=0).stop 2024-09-15T22:44:47.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:47.206+0000 7f121ae00640 1 -- 172.21.15.111:0/576711545 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1214074eb0 msgr2=0x7f12141a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:47.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:47.206+0000 7f121ae00640 1 --2- 172.21.15.111:0/576711545 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1214074eb0 0x7f12141a4c30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f120000e9f0 tx=0x7f120000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:44:47.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:47.206+0000 7f121ae00640 1 -- 172.21.15.111:0/576711545 shutdown_connections 2024-09-15T22:44:47.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:47.206+0000 7f121ae00640 1 --2- 172.21.15.111:0/576711545 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f11f80a32d0 0x7f11f80a5790 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:47.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:47.206+0000 7f121ae00640 1 --2- 172.21.15.111:0/576711545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1214112b90 0x7f121419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:47.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:47.206+0000 7f121ae00640 1 --2- 172.21.15.111:0/576711545 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1214074eb0 0x7f12141a4c30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:47.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:47.206+0000 7f121ae00640 1 --2- 172.21.15.111:0/576711545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1214074590 0x7f12141a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:47.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:47.206+0000 7f121ae00640 1 -- 172.21.15.111:0/576711545 >> 172.21.15.111:0/576711545 conn(0x7f12140fd8c0 msgr2=0x7f12140fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:47.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:47.206+0000 7f121ae00640 1 -- 172.21.15.111:0/576711545 shutdown_connections 2024-09-15T22:44:47.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:47.206+0000 7f121ae00640 1 -- 172.21.15.111:0/576711545 wait complete. 2024-09-15T22:44:47.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:47 smithi121 ceph-mon[34815]: pgmap v314: 97 pgs: 97 active+clean; 629 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-09-15T22:44:47.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:47 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/576711545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:44:47.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:47 smithi111 ceph-mon[30629]: pgmap v314: 97 pgs: 97 active+clean; 629 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-09-15T22:44:47.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:47 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/576711545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:44:48.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:47 smithi192 ceph-mon[35427]: pgmap v314: 97 pgs: 97 active+clean; 629 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-09-15T22:44:48.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:47 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/576711545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:44:48.080 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:44:48.449 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:49.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.079+0000 7f3001000640 1 -- 172.21.15.111:0/3516119975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ffc102790 msgr2=0x7f2ffc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:49.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.079+0000 7f3001000640 1 --2- 172.21.15.111:0/3516119975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ffc102790 0x7f2ffc1067e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8009900 tx=0x7f2fe802a410 comp rx=0 tx=0).stop 2024-09-15T22:44:49.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.079+0000 7f3001000640 1 -- 172.21.15.111:0/3516119975 shutdown_connections 2024-09-15T22:44:49.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.079+0000 7f3001000640 1 --2- 172.21.15.111:0/3516119975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ffc106f10 0x7f2ffc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:49.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.079+0000 7f3001000640 1 --2- 172.21.15.111:0/3516119975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ffc102790 0x7f2ffc1067e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:49.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.079+0000 7f3001000640 1 --2- 172.21.15.111:0/3516119975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ffc101de0 0x7f2ffc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:49.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.079+0000 7f3001000640 1 -- 172.21.15.111:0/3516119975 >> 172.21.15.111:0/3516119975 conn(0x7f2ffc0fd8c0 msgr2=0x7f2ffc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:49.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.079+0000 7f3001000640 1 -- 172.21.15.111:0/3516119975 shutdown_connections 2024-09-15T22:44:49.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.079+0000 7f3001000640 1 -- 172.21.15.111:0/3516119975 wait complete. 2024-09-15T22:44:49.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.080+0000 7f3001000640 1 Processor -- start 2024-09-15T22:44:49.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.080+0000 7f3001000640 1 -- start start 2024-09-15T22:44:49.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.081+0000 7f3001000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ffc101de0 0x7f2ffc19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:49.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.081+0000 7f3001000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ffc102790 0x7f2ffc19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:49.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.081+0000 7f3001000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ffc106f10 0x7f2ffc19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:49.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.081+0000 7f3001000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc111fc0 con 0x7f2ffc101de0 2024-09-15T22:44:49.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.081+0000 7f3001000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc111e40 con 0x7f2ffc102790 2024-09-15T22:44:49.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.081+0000 7f3001000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc112140 con 0x7f2ffc106f10 2024-09-15T22:44:49.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.081+0000 7f2ff9400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ffc102790 0x7f2ffc19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:49.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.081+0000 7f2ff9e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ffc101de0 0x7f2ffc19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:49.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.081+0000 7f2ff9400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ffc102790 0x7f2ffc19ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:51436/0 (socket says 172.21.15.111:51436) 2024-09-15T22:44:49.098 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.081+0000 7f2ff9400640 1 -- 172.21.15.111:0/1114551133 learned_addr learned my addr 172.21.15.111:0/1114551133 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:44:49.099 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.081+0000 7f2ffa800640 1 --2- 172.21.15.111:0/1114551133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ffc106f10 0x7f2ffc19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:49.099 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.081+0000 7f2ff9400640 1 -- 172.21.15.111:0/1114551133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ffc106f10 msgr2=0x7f2ffc19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:49.099 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.081+0000 7f2ff9400640 1 --2- 172.21.15.111:0/1114551133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ffc106f10 0x7f2ffc19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:49.099 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.082+0000 7f2ff9400640 1 -- 172.21.15.111:0/1114551133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ffc101de0 msgr2=0x7f2ffc19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:49.099 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.082+0000 7f2ff9400640 1 --2- 172.21.15.111:0/1114551133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ffc101de0 0x7f2ffc19a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:49.099 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.082+0000 7f2ff9400640 1 -- 172.21.15.111:0/1114551133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ffc19f5e0 con 0x7f2ffc102790 2024-09-15T22:44:49.099 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.082+0000 7f2ff9e00640 1 --2- 172.21.15.111:0/1114551133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ffc101de0 0x7f2ffc19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:44:49.099 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.082+0000 7f2ff9400640 1 --2- 172.21.15.111:0/1114551133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ffc102790 0x7f2ffc19ab70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8004770 tx=0x7f2fe8003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:49.099 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.082+0000 7f2ff2a00640 1 -- 172.21.15.111:0/1114551133 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe803f070 con 0x7f2ffc102790 2024-09-15T22:44:49.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.082+0000 7f2ff2a00640 1 -- 172.21.15.111:0/1114551133 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fe8031ca0 con 0x7f2ffc102790 2024-09-15T22:44:49.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.082+0000 7f3001000640 1 -- 172.21.15.111:0/1114551133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ffc19f870 con 0x7f2ffc102790 2024-09-15T22:44:49.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.082+0000 7f2ff2a00640 1 -- 172.21.15.111:0/1114551133 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8035030 con 0x7f2ffc102790 2024-09-15T22:44:49.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.082+0000 7f3001000640 1 -- 172.21.15.111:0/1114551133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ffc1a71b0 con 0x7f2ffc102790 2024-09-15T22:44:49.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.083+0000 7f3001000640 1 -- 172.21.15.111:0/1114551133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ffc1021c0 con 0x7f2ffc102790 2024-09-15T22:44:49.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.085+0000 7f2ff2a00640 1 -- 172.21.15.111:0/1114551133 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f2fe8042070 con 0x7f2ffc102790 2024-09-15T22:44:49.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.086+0000 7f2ff2a00640 1 --2- 172.21.15.111:0/1114551133 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2fe00a3350 0x7f2fe00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:49.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.086+0000 7f2ff2a00640 1 -- 172.21.15.111:0/1114551133 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f2fe80f0100 con 0x7f2ffc102790 2024-09-15T22:44:49.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.087+0000 7f2ff9e00640 1 --2- 172.21.15.111:0/1114551133 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2fe00a3350 0x7f2fe00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:49.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.087+0000 7f2ff9e00640 1 --2- 172.21.15.111:0/1114551133 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2fe00a3350 0x7f2fe00a5810 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2fec009fd0 tx=0x7f2fec009760 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:49.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.089+0000 7f2ff2a00640 1 -- 172.21.15.111:0/1114551133 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2fe80bc830 con 0x7f2ffc102790 2024-09-15T22:44:49.271 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.269+0000 7f3001000640 1 -- 172.21.15.111:0/1114551133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ffc1a73b0 con 0x7f2ffc102790 2024-09-15T22:44:49.273 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.271+0000 7f2ff2a00640 1 -- 172.21.15.111:0/1114551133 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22363 (secure 0 0 0) 0x7f2fe80c0b30 con 0x7f2ffc102790 2024-09-15T22:44:49.273 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:44:49.273 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":86,"fsid":"b7614d3c-73b2-11ef-bceb-c7b262605968","created":"2024-09-15T22:35:52.571473+0000","modified":"2024-09-15T22:44:18.436245+0000","last_up_change":"2024-09-15T22:43:19.567274+0000","last_in_change":"2024-09-15T22:42:56.994246+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:39:18.954207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:44:12.846418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:44:15.685946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04f39604-be0e-4261-b316-67fa087c6666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6803","nonce":1386823446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6805","nonce":1386823446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6809","nonce":1386823446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6807","nonce":1386823446}]},"public_addr":"172.21.15.111:6803/1386823446","cluster_addr":"172.21.15.111:6805/1386823446","heartbeat_back_addr":"172.21.15.111:6809/1386823446","heartbeat_front_addr":"172.21.15.111:6807/1386823446","state":["exists","up"]},{"osd":1,"uuid":"1bdbef6e-23fa-4618-a788-66e57741a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6801","nonce":2051512291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6803","nonce":2051512291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6807","nonce":2051512291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6805","nonce":2051512291}]},"public_addr":"172.21.15.121:6801/2051512291","cluster_addr":"172.21.15.121:6803/2051512291","heartbeat_back_addr":"172.21.15.121:6807/2051512291","heartbeat_front_addr":"172.21.15.121:6805/2051512291","state":["exists","up"]},{"osd":2,"uuid":"46735ccf-6d0e-4154-bb55-422929819bfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6801","nonce":93085639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6803","nonce":93085639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6807","nonce":93085639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6805","nonce":93085639}]},"public_addr":"172.21.15.192:6801/93085639","cluster_addr":"172.21.15.192:6803/93085639","heartbeat_back_addr":"172.21.15.192:6807/93085639","heartbeat_front_addr":"172.21.15.192:6805/93085639","state":["exists","up"]},{"osd":3,"uuid":"4b243af8-c958-40b1-ba24-9c7ace00c4a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6811","nonce":2578259041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6813","nonce":2578259041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6817","nonce":2578259041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6815","nonce":2578259041}]},"public_addr":"172.21.15.111:6811/2578259041","cluster_addr":"172.21.15.111:6813/2578259041","heartbeat_back_addr":"172.21.15.111:6817/2578259041","heartbeat_front_addr":"172.21.15.111:6815/2578259041","state":["exists","up"]},{"osd":4,"uuid":"49f27541-9cbf-48e0-a466-6bb0fa5410b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6809","nonce":2667595915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6811","nonce":2667595915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6815","nonce":2667595915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6813","nonce":2667595915}]},"public_addr":"172.21.15.121:6809/2667595915","cluster_addr":"172.21.15.121:6811/2667595915","heartbeat_back_addr":"172.21.15.121:6815/2667595915","heartbeat_front_addr":"172.21.15.121:6813/2667595915","state":["exists","up"]},{"osd":5,"uuid":"9290e769-5470-44d3-8d24-25093af34335","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6809","nonce":1958639856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6811","nonce":1958639856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6815","nonce":1958639856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6813","nonce":1958639856}]},"public_addr":"172.21.15.192:6809/1958639856","cluster_addr":"172.21.15.192:6811/1958639856","heartbeat_back_addr":"172.21.15.192:6815/1958639856","heartbeat_front_addr":"172.21.15.192:6813/1958639856","state":["exists","up"]},{"osd":6,"uuid":"18911b97-1c68-4e8f-92ec-35d2ff0b5f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6819","nonce":1371543706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6821","nonce":1371543706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6825","nonce":1371543706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6823","nonce":1371543706}]},"public_addr":"172.21.15.111:6819/1371543706","cluster_addr":"172.21.15.111:6821/1371543706","heartbeat_back_addr":"172.21.15.111:6825/1371543706","heartbeat_front_addr":"172.21.15.111:6823/1371543706","state":["exists","up"]},{"osd":7,"uuid":"470647c5-c00d-426b-a134-cca235902307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6817","nonce":3501842998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6819","nonce":3501842998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6823","nonce":3501842998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6821","nonce":3501842998}]},"public_addr":"172.21.15.121:6817/3501842998","cluster_addr":"172.21.15.121:6819/3501842998","heartbeat_back_addr":"172.21.15.121:6823/3501842998","heartbeat_front_addr":"172.21.15.121:6821/3501842998","state":["exists","up"]},{"osd":8,"uuid":"fa2d3dd5-a768-4ada-a036-4c596b76d654","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6817","nonce":2417771374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6819","nonce":2417771374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6823","nonce":2417771374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6821","nonce":2417771374}]},"public_addr":"172.21.15.192:6817/2417771374","cluster_addr":"172.21.15.192:6819/2417771374","heartbeat_back_addr":"172.21.15.192:6823/2417771374","heartbeat_front_addr":"172.21.15.192:6821/2417771374","state":["exists","up"]},{"osd":9,"uuid":"71b4f5fb-1c02-461b-a257-c6e48381d768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6827","nonce":3979119431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6829","nonce":3979119431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6833","nonce":3979119431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6831","nonce":3979119431}]},"public_addr":"172.21.15.111:6827/3979119431","cluster_addr":"172.21.15.111:6829/3979119431","heartbeat_back_addr":"172.21.15.111:6833/3979119431","heartbeat_front_addr":"172.21.15.111:6831/3979119431","state":["exists","up"]},{"osd":10,"uuid":"e6a39e77-fce3-4e22-bf66-a677c9f8b6bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6825","nonce":2897902637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6827","nonce":2897902637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6831","nonce":2897902637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6829","nonce":2897902637}]},"public_addr":"172.21.15.121:6825/2897902637","cluster_addr":"172.21.15.121:6827/2897902637","heartbeat_back_addr":"172.21.15.121:6831/2897902637","heartbeat_front_addr":"172.21.15.121:6829/2897902637","state":["exists","up"]},{"osd":11,"uuid":"411a7921-6717-4cef-b8df-d9e252edcc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6825","nonce":2431464812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6827","nonce":2431464812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6831","nonce":2431464812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6829","nonce":2431464812}]},"public_addr":"172.21.15.192:6825/2431464812","cluster_addr":"172.21.15.192:6827/2431464812","heartbeat_back_addr":"172.21.15.192:6831/2431464812","heartbeat_front_addr":"172.21.15.192:6829/2431464812","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:38:23.442715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:38:49.501124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:39:14.759077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:39:45.121397+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:40:11.137612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:40:36.278567+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:07.027190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:32.439215+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:57.058702+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:42:28.741846+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:42:53.395442+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:43:18.265067+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:6801/2107775445":"2024-09-16T22:36:50.357551+0000","172.21.15.111:6800/2107775445":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/2394746540":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/4140020843":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/2114354263":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/1054173642":"2024-09-16T22:36:22.652875+0000","172.21.15.111:0/3408238004":"2024-09-16T22:36:22.652875+0000","172.21.15.111:0/2879044772":"2024-09-16T22:36:22.652875+0000","172.21.15.111:6801/3350425104":"2024-09-16T22:36:22.652875+0000","172.21.15.111:6800/3350425104":"2024-09-16T22:36:22.652875+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:44:49.275 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.273+0000 7f3001000640 1 -- 172.21.15.111:0/1114551133 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2fe00a3350 msgr2=0x7f2fe00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:49.275 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.273+0000 7f3001000640 1 --2- 172.21.15.111:0/1114551133 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2fe00a3350 0x7f2fe00a5810 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2fec009fd0 tx=0x7f2fec009760 comp rx=0 tx=0).stop 2024-09-15T22:44:49.275 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.273+0000 7f3001000640 1 -- 172.21.15.111:0/1114551133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ffc102790 msgr2=0x7f2ffc19ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:49.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.274+0000 7f3001000640 1 --2- 172.21.15.111:0/1114551133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ffc102790 0x7f2ffc19ab70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8004770 tx=0x7f2fe8003c10 comp rx=0 tx=0).stop 2024-09-15T22:44:49.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.274+0000 7f3001000640 1 -- 172.21.15.111:0/1114551133 shutdown_connections 2024-09-15T22:44:49.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.274+0000 7f3001000640 1 --2- 172.21.15.111:0/1114551133 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ffc106f10 0x7f2ffc19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:49.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.274+0000 7f3001000640 1 --2- 172.21.15.111:0/1114551133 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2fe00a3350 0x7f2fe00a5810 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:49.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.274+0000 7f3001000640 1 --2- 172.21.15.111:0/1114551133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ffc102790 0x7f2ffc19ab70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:49.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.274+0000 7f3001000640 1 --2- 172.21.15.111:0/1114551133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ffc101de0 0x7f2ffc19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:49.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.274+0000 7f3001000640 1 -- 172.21.15.111:0/1114551133 >> 172.21.15.111:0/1114551133 conn(0x7f2ffc0fd8c0 msgr2=0x7f2ffc0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:49.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.274+0000 7f3001000640 1 -- 172.21.15.111:0/1114551133 shutdown_connections 2024-09-15T22:44:49.276 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:49.274+0000 7f3001000640 1 -- 172.21.15.111:0/1114551133 wait complete. 2024-09-15T22:44:49.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:49 smithi121 ceph-mon[34815]: pgmap v315: 97 pgs: 97 active+clean; 629 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.1 KiB/s wr, 7 op/s 2024-09-15T22:44:49.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:49 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/1114551133' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:44:49.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:49 smithi111 ceph-mon[30629]: pgmap v315: 97 pgs: 97 active+clean; 629 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.1 KiB/s wr, 7 op/s 2024-09-15T22:44:49.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:49 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/1114551133' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:44:50.031 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-15T22:44:50.032 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T22:44:50.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:49 smithi192 ceph-mon[35427]: pgmap v315: 97 pgs: 97 active+clean; 629 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.1 KiB/s wr, 7 op/s 2024-09-15T22:44:50.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:49 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/1114551133' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:44:50.400 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:50.871 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-15T22:44:50.871 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-15T22:44:50.871 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-09-15T22:44:50.871 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-15T22:44:50.871 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-15T22:44:50.871 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-15T22:44:50.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-15T22:44:50.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-15T22:44:50.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-09-15T22:44:50.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-15T22:44:50.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-15T22:44:50.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-15T22:44:50.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.978+0000 7f7c18a00640 1 -- 172.21.15.111:0/556091697 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c14112870 msgr2=0x7f7c14114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:50.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.978+0000 7f7c18a00640 1 --2- 172.21.15.111:0/556091697 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c14112870 0x7f7c14114c60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c009ca0 tx=0x7f7c0c02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:44:50.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.978+0000 7f7c18a00640 1 -- 172.21.15.111:0/556091697 shutdown_connections 2024-09-15T22:44:50.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.978+0000 7f7c18a00640 1 --2- 172.21.15.111:0/556091697 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c14112870 0x7f7c14114c60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:50.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.978+0000 7f7c18a00640 1 --2- 172.21.15.111:0/556091697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c14077040 0x7f7c14112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:50.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.978+0000 7f7c18a00640 1 --2- 172.21.15.111:0/556091697 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c14076720 0x7f7c14076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:50.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.978+0000 7f7c18a00640 1 -- 172.21.15.111:0/556091697 >> 172.21.15.111:0/556091697 conn(0x7f7c140ff8d0 msgr2=0x7f7c14101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:50.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.978+0000 7f7c18a00640 1 -- 172.21.15.111:0/556091697 shutdown_connections 2024-09-15T22:44:50.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.978+0000 7f7c18a00640 1 -- 172.21.15.111:0/556091697 wait complete. 2024-09-15T22:44:50.982 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.979+0000 7f7c18a00640 1 Processor -- start 2024-09-15T22:44:50.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.979+0000 7f7c18a00640 1 -- start start 2024-09-15T22:44:50.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.980+0000 7f7c18a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c14076720 0x7f7c141a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:50.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.980+0000 7f7c18a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c14077040 0x7f7c141a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:50.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.980+0000 7f7c18a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c14112870 0x7f7c1419e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:50.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.980+0000 7f7c18a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7c14118400 con 0x7f7c14076720 2024-09-15T22:44:50.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.980+0000 7f7c18a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7c14118280 con 0x7f7c14112870 2024-09-15T22:44:50.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.980+0000 7f7c18a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7c14118580 con 0x7f7c14077040 2024-09-15T22:44:50.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.980+0000 7f7c13400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c14076720 0x7f7c141a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:50.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.980+0000 7f7c0aa00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c14077040 0x7f7c141a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:50.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.980+0000 7f7c13400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c14076720 0x7f7c141a45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56088/0 (socket says 172.21.15.111:56088) 2024-09-15T22:44:50.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.980+0000 7f7c13400640 1 -- 172.21.15.111:0/699251975 learned_addr learned my addr 172.21.15.111:0/699251975 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:44:50.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.980+0000 7f7c13e00640 1 --2- 172.21.15.111:0/699251975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c14112870 0x7f7c1419e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:50.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.981+0000 7f7c0aa00640 1 -- 172.21.15.111:0/699251975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c14112870 msgr2=0x7f7c1419e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:50.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.981+0000 7f7c0aa00640 1 --2- 172.21.15.111:0/699251975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c14112870 0x7f7c1419e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:50.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.981+0000 7f7c0aa00640 1 -- 172.21.15.111:0/699251975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c14076720 msgr2=0x7f7c141a45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:50.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.981+0000 7f7c0aa00640 1 --2- 172.21.15.111:0/699251975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c14076720 0x7f7c141a45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:50.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.981+0000 7f7c0aa00640 1 -- 172.21.15.111:0/699251975 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c1419ef20 con 0x7f7c14077040 2024-09-15T22:44:50.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.981+0000 7f7c13400640 1 --2- 172.21.15.111:0/699251975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c14076720 0x7f7c141a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:44:50.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.981+0000 7f7c0aa00640 1 --2- 172.21.15.111:0/699251975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c14077040 0x7f7c141a4b20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc00e9f0 tx=0x7f7bfc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:50.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.981+0000 7f7c10c00640 1 -- 172.21.15.111:0/699251975 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc00cd80 con 0x7f7c14077040 2024-09-15T22:44:50.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.981+0000 7f7c18a00640 1 -- 172.21.15.111:0/699251975 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c1419f210 con 0x7f7c14077040 2024-09-15T22:44:50.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.981+0000 7f7c10c00640 1 -- 172.21.15.111:0/699251975 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bfc0035f0 con 0x7f7c14077040 2024-09-15T22:44:50.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.981+0000 7f7c10c00640 1 -- 172.21.15.111:0/699251975 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc010670 con 0x7f7c14077040 2024-09-15T22:44:50.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.981+0000 7f7c18a00640 1 -- 172.21.15.111:0/699251975 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c141ab560 con 0x7f7c14077040 2024-09-15T22:44:50.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.982+0000 7f7c18a00640 1 -- 172.21.15.111:0/699251975 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c14077e60 con 0x7f7c14077040 2024-09-15T22:44:50.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.984+0000 7f7c10c00640 1 -- 172.21.15.111:0/699251975 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f7bfc0037a0 con 0x7f7c14077040 2024-09-15T22:44:50.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.985+0000 7f7c10c00640 1 --2- 172.21.15.111:0/699251975 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7bf80a32d0 0x7f7bf80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:50.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.986+0000 7f7c10c00640 1 -- 172.21.15.111:0/699251975 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8153+0+0 (secure 0 0 0) 0x7f7bfc0cffb0 con 0x7f7c14077040 2024-09-15T22:44:50.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.986+0000 7f7c13400640 1 --2- 172.21.15.111:0/699251975 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7bf80a32d0 0x7f7bf80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:50.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.986+0000 7f7c13400640 1 --2- 172.21.15.111:0/699251975 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7bf80a32d0 0x7f7bf80a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7c04011550 tx=0x7f7c04005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:50.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:50.987+0000 7f7c10c00640 1 -- 172.21.15.111:0/699251975 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7bfc09cb70 con 0x7f7c14077040 2024-09-15T22:44:51.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-09-15T22:44:51.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-09-15T22:44:51.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-09-15T22:44:51.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-15T22:44:51.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-15T22:44:51.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-15T22:44:51.162 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:51.161+0000 7f7c18a00640 1 -- 172.21.15.111:0/699251975 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7c14062630 con 0x7f7bf80a32d0 2024-09-15T22:44:51.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:51.169+0000 7f7c10c00640 1 -- 172.21.15.111:0/699251975 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258702 (secure 0 0 0) 0x7f7c14062630 con 0x7f7bf80a32d0 2024-09-15T22:44:51.175 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:44:51.176 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-09-15T22:44:51.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:51.173+0000 7f7c18a00640 1 -- 172.21.15.111:0/699251975 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7bf80a32d0 msgr2=0x7f7bf80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:51.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:51.173+0000 7f7c18a00640 1 --2- 172.21.15.111:0/699251975 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7bf80a32d0 0x7f7bf80a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7c04011550 tx=0x7f7c04005f90 comp rx=0 tx=0).stop 2024-09-15T22:44:51.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:51.173+0000 7f7c18a00640 1 -- 172.21.15.111:0/699251975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c14077040 msgr2=0x7f7c141a4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:51.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:51.173+0000 7f7c18a00640 1 --2- 172.21.15.111:0/699251975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c14077040 0x7f7c141a4b20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc00e9f0 tx=0x7f7bfc00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:44:51.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:51.173+0000 7f7c18a00640 1 -- 172.21.15.111:0/699251975 shutdown_connections 2024-09-15T22:44:51.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:51.173+0000 7f7c18a00640 1 --2- 172.21.15.111:0/699251975 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7bf80a32d0 0x7f7bf80a5790 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:51.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:51.173+0000 7f7c18a00640 1 --2- 172.21.15.111:0/699251975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c14112870 0x7f7c1419e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:51.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:51.173+0000 7f7c18a00640 1 --2- 172.21.15.111:0/699251975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c14077040 0x7f7c141a4b20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:51.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:51.173+0000 7f7c18a00640 1 --2- 172.21.15.111:0/699251975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c14076720 0x7f7c141a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:51.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:51.173+0000 7f7c18a00640 1 -- 172.21.15.111:0/699251975 >> 172.21.15.111:0/699251975 conn(0x7f7c140ff8d0 msgr2=0x7f7c14101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:51.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:51.173+0000 7f7c18a00640 1 -- 172.21.15.111:0/699251975 shutdown_connections 2024-09-15T22:44:51.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:51.173+0000 7f7c18a00640 1 -- 172.21.15.111:0/699251975 wait complete. 2024-09-15T22:44:51.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:51 smithi121 ceph-mon[34815]: pgmap v316: 97 pgs: 97 active+clean; 629 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-09-15T22:44:51.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:51 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-15T22:44:51.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:51 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-15T22:44:51.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:51 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-15T22:44:51.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:51 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-15T22:44:51.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:51 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-15T22:44:51.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:51 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-15T22:44:51.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:51 smithi121 ceph-mon[34815]: osdmap e87: 12 total, 12 up, 12 in 2024-09-15T22:44:51.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:51 smithi111 ceph-mon[30629]: pgmap v316: 97 pgs: 97 active+clean; 629 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-09-15T22:44:51.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:51 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-15T22:44:51.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:51 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-15T22:44:51.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:51 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-15T22:44:51.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:51 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-15T22:44:51.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:51 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-15T22:44:51.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:51 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-15T22:44:51.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:51 smithi111 ceph-mon[30629]: osdmap e87: 12 total, 12 up, 12 in 2024-09-15T22:44:51.986 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-09-15T22:44:50.629744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":648465,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":263,"num_read_kb":236,"num_write":338,"num_write_kb":3042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":272,"ondisk_log_size":272,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374764,"kb_used_data":13036,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567060,"statfs":{"total":1151940427776,"available":1151556669440,"internally_reserved":0,"allocated":13348864,"data_stored":7379865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":52482,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":94,"num_write":31,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.172192"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.217652+0000","last_change":"2024-09-15T22:44:17.222326+0000","last_active":"2024-09-15T22:44:19.217652+0000","last_peered":"2024-09-15T22:44:19.217652+0000","last_clean":"2024-09-15T22:44:19.217652+0000","last_became_active":"2024-09-15T22:44:17.221903+0000","last_became_peered":"2024-09-15T22:44:17.221903+0000","last_unstale":"2024-09-15T22:44:19.217652+0000","last_undegraded":"2024-09-15T22:44:19.217652+0000","last_fullsized":"2024-09-15T22:44:19.217652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:28.303194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.212887+0000","last_change":"2024-09-15T22:44:17.220515+0000","last_active":"2024-09-15T22:44:19.212887+0000","last_peered":"2024-09-15T22:44:19.212887+0000","last_clean":"2024-09-15T22:44:19.212887+0000","last_became_active":"2024-09-15T22:44:17.218794+0000","last_became_peered":"2024-09-15T22:44:17.218794+0000","last_unstale":"2024-09-15T22:44:19.212887+0000","last_undegraded":"2024-09-15T22:44:19.212887+0000","last_fullsized":"2024-09-15T22:44:19.212887+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:58:17.892809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.213811+0000","last_change":"2024-09-15T22:44:17.213183+0000","last_active":"2024-09-15T22:44:19.213811+0000","last_peered":"2024-09-15T22:44:19.213811+0000","last_clean":"2024-09-15T22:44:19.213811+0000","last_became_active":"2024-09-15T22:44:17.212888+0000","last_became_peered":"2024-09-15T22:44:17.212888+0000","last_unstale":"2024-09-15T22:44:19.213811+0000","last_undegraded":"2024-09-15T22:44:19.213811+0000","last_fullsized":"2024-09-15T22:44:19.213811+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:32.224099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.214564+0000","last_change":"2024-09-15T22:44:17.203869+0000","last_active":"2024-09-15T22:44:19.214564+0000","last_peered":"2024-09-15T22:44:19.214564+0000","last_clean":"2024-09-15T22:44:19.214564+0000","last_became_active":"2024-09-15T22:44:17.203529+0000","last_became_peered":"2024-09-15T22:44:17.203529+0000","last_unstale":"2024-09-15T22:44:19.214564+0000","last_undegraded":"2024-09-15T22:44:19.214564+0000","last_fullsized":"2024-09-15T22:44:19.214564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:22:58.539513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.210896+0000","last_change":"2024-09-15T22:44:17.209521+0000","last_active":"2024-09-15T22:44:19.210896+0000","last_peered":"2024-09-15T22:44:19.210896+0000","last_clean":"2024-09-15T22:44:19.210896+0000","last_became_active":"2024-09-15T22:44:17.209203+0000","last_became_peered":"2024-09-15T22:44:17.209203+0000","last_unstale":"2024-09-15T22:44:19.210896+0000","last_undegraded":"2024-09-15T22:44:19.210896+0000","last_fullsized":"2024-09-15T22:44:19.210896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:34:28.418483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.210130+0000","last_change":"2024-09-15T22:44:17.202857+0000","last_active":"2024-09-15T22:44:19.210130+0000","last_peered":"2024-09-15T22:44:19.210130+0000","last_clean":"2024-09-15T22:44:19.210130+0000","last_became_active":"2024-09-15T22:44:17.202530+0000","last_became_peered":"2024-09-15T22:44:17.202530+0000","last_unstale":"2024-09-15T22:44:19.210130+0000","last_undegraded":"2024-09-15T22:44:19.210130+0000","last_fullsized":"2024-09-15T22:44:19.210130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:16:06.947769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.210951+0000","last_change":"2024-09-15T22:44:17.203317+0000","last_active":"2024-09-15T22:44:19.210951+0000","last_peered":"2024-09-15T22:44:19.210951+0000","last_clean":"2024-09-15T22:44:19.210951+0000","last_became_active":"2024-09-15T22:44:17.202975+0000","last_became_peered":"2024-09-15T22:44:17.202975+0000","last_unstale":"2024-09-15T22:44:19.210951+0000","last_undegraded":"2024-09-15T22:44:19.210951+0000","last_fullsized":"2024-09-15T22:44:19.210951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:51.292052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.211311+0000","last_change":"2024-09-15T22:44:17.203583+0000","last_active":"2024-09-15T22:44:19.211311+0000","last_peered":"2024-09-15T22:44:19.211311+0000","last_clean":"2024-09-15T22:44:19.211311+0000","last_became_active":"2024-09-15T22:44:17.203251+0000","last_became_peered":"2024-09-15T22:44:17.203251+0000","last_unstale":"2024-09-15T22:44:19.211311+0000","last_undegraded":"2024-09-15T22:44:19.211311+0000","last_fullsized":"2024-09-15T22:44:19.211311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:53:17.378024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.196788+0000","last_change":"2024-09-15T22:44:17.209643+0000","last_active":"2024-09-15T22:44:19.196788+0000","last_peered":"2024-09-15T22:44:19.196788+0000","last_clean":"2024-09-15T22:44:19.196788+0000","last_became_active":"2024-09-15T22:44:17.208709+0000","last_became_peered":"2024-09-15T22:44:17.208709+0000","last_unstale":"2024-09-15T22:44:19.196788+0000","last_undegraded":"2024-09-15T22:44:19.196788+0000","last_fullsized":"2024-09-15T22:44:19.196788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:38:38.305676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.212031+0000","last_change":"2024-09-15T22:44:17.201830+0000","last_active":"2024-09-15T22:44:19.212031+0000","last_peered":"2024-09-15T22:44:19.212031+0000","last_clean":"2024-09-15T22:44:19.212031+0000","last_became_active":"2024-09-15T22:44:17.200844+0000","last_became_peered":"2024-09-15T22:44:17.200844+0000","last_unstale":"2024-09-15T22:44:19.212031+0000","last_undegraded":"2024-09-15T22:44:19.212031+0000","last_fullsized":"2024-09-15T22:44:19.212031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:38:53.681454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.212758+0000","last_change":"2024-09-15T22:44:17.208106+0000","last_active":"2024-09-15T22:44:19.212758+0000","last_peered":"2024-09-15T22:44:19.212758+0000","last_clean":"2024-09-15T22:44:19.212758+0000","last_became_active":"2024-09-15T22:44:17.207774+0000","last_became_peered":"2024-09-15T22:44:17.207774+0000","last_unstale":"2024-09-15T22:44:19.212758+0000","last_undegraded":"2024-09-15T22:44:19.212758+0000","last_fullsized":"2024-09-15T22:44:19.212758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:05:15.876692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.214745+0000","last_change":"2024-09-15T22:44:17.202857+0000","last_active":"2024-09-15T22:44:19.214745+0000","last_peered":"2024-09-15T22:44:19.214745+0000","last_clean":"2024-09-15T22:44:19.214745+0000","last_became_active":"2024-09-15T22:44:17.202520+0000","last_became_peered":"2024-09-15T22:44:17.202520+0000","last_unstale":"2024-09-15T22:44:19.214745+0000","last_undegraded":"2024-09-15T22:44:19.214745+0000","last_fullsized":"2024-09-15T22:44:19.214745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:11:19.111086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.213091+0000","last_change":"2024-09-15T22:44:17.201776+0000","last_active":"2024-09-15T22:44:19.213091+0000","last_peered":"2024-09-15T22:44:19.213091+0000","last_clean":"2024-09-15T22:44:19.213091+0000","last_became_active":"2024-09-15T22:44:17.201508+0000","last_became_peered":"2024-09-15T22:44:17.201508+0000","last_unstale":"2024-09-15T22:44:19.213091+0000","last_undegraded":"2024-09-15T22:44:19.213091+0000","last_fullsized":"2024-09-15T22:44:19.213091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:22:48.104776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.211824+0000","last_change":"2024-09-15T22:44:17.221005+0000","last_active":"2024-09-15T22:44:19.211824+0000","last_peered":"2024-09-15T22:44:19.211824+0000","last_clean":"2024-09-15T22:44:19.211824+0000","last_became_active":"2024-09-15T22:44:17.218455+0000","last_became_peered":"2024-09-15T22:44:17.218455+0000","last_unstale":"2024-09-15T22:44:19.211824+0000","last_undegraded":"2024-09-15T22:44:19.211824+0000","last_fullsized":"2024-09-15T22:44:19.211824+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:40.227843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.212170+0000","last_change":"2024-09-15T22:44:17.208125+0000","last_active":"2024-09-15T22:44:19.212170+0000","last_peered":"2024-09-15T22:44:19.212170+0000","last_clean":"2024-09-15T22:44:19.212170+0000","last_became_active":"2024-09-15T22:44:17.207762+0000","last_became_peered":"2024-09-15T22:44:17.207762+0000","last_unstale":"2024-09-15T22:44:19.212170+0000","last_undegraded":"2024-09-15T22:44:19.212170+0000","last_fullsized":"2024-09-15T22:44:19.212170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:21:45.181921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.196130+0000","last_change":"2024-09-15T22:44:17.212058+0000","last_active":"2024-09-15T22:44:19.196130+0000","last_peered":"2024-09-15T22:44:19.196130+0000","last_clean":"2024-09-15T22:44:19.196130+0000","last_became_active":"2024-09-15T22:44:17.211789+0000","last_became_peered":"2024-09-15T22:44:17.211789+0000","last_unstale":"2024-09-15T22:44:19.196130+0000","last_undegraded":"2024-09-15T22:44:19.196130+0000","last_fullsized":"2024-09-15T22:44:19.196130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:14.647374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.217987+0000","last_change":"2024-09-15T22:44:17.225139+0000","last_active":"2024-09-15T22:44:19.217987+0000","last_peered":"2024-09-15T22:44:19.217987+0000","last_clean":"2024-09-15T22:44:19.217987+0000","last_became_active":"2024-09-15T22:44:17.224791+0000","last_became_peered":"2024-09-15T22:44:17.224791+0000","last_unstale":"2024-09-15T22:44:19.217987+0000","last_undegraded":"2024-09-15T22:44:19.217987+0000","last_fullsized":"2024-09-15T22:44:19.217987+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:56.457155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.193654+0000","last_change":"2024-09-15T22:44:17.208604+0000","last_active":"2024-09-15T22:44:19.193654+0000","last_peered":"2024-09-15T22:44:19.193654+0000","last_clean":"2024-09-15T22:44:19.193654+0000","last_became_active":"2024-09-15T22:44:17.202231+0000","last_became_peered":"2024-09-15T22:44:17.202231+0000","last_unstale":"2024-09-15T22:44:19.193654+0000","last_undegraded":"2024-09-15T22:44:19.193654+0000","last_fullsized":"2024-09-15T22:44:19.193654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:39.572881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.212060+0000","last_change":"2024-09-15T22:44:17.211837+0000","last_active":"2024-09-15T22:44:19.212060+0000","last_peered":"2024-09-15T22:44:19.212060+0000","last_clean":"2024-09-15T22:44:19.212060+0000","last_became_active":"2024-09-15T22:44:17.211460+0000","last_became_peered":"2024-09-15T22:44:17.211460+0000","last_unstale":"2024-09-15T22:44:19.212060+0000","last_undegraded":"2024-09-15T22:44:19.212060+0000","last_fullsized":"2024-09-15T22:44:19.212060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:21:51.820134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.217414+0000","last_change":"2024-09-15T22:44:17.206350+0000","last_active":"2024-09-15T22:44:19.217414+0000","last_peered":"2024-09-15T22:44:19.217414+0000","last_clean":"2024-09-15T22:44:19.217414+0000","last_became_active":"2024-09-15T22:44:17.204943+0000","last_became_peered":"2024-09-15T22:44:17.204943+0000","last_unstale":"2024-09-15T22:44:19.217414+0000","last_undegraded":"2024-09-15T22:44:19.217414+0000","last_fullsized":"2024-09-15T22:44:19.217414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:42:59.918220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.213222+0000","last_change":"2024-09-15T22:44:17.223390+0000","last_active":"2024-09-15T22:44:19.213222+0000","last_peered":"2024-09-15T22:44:19.213222+0000","last_clean":"2024-09-15T22:44:19.213222+0000","last_became_active":"2024-09-15T22:44:17.223008+0000","last_became_peered":"2024-09-15T22:44:17.223008+0000","last_unstale":"2024-09-15T22:44:19.213222+0000","last_undegraded":"2024-09-15T22:44:19.213222+0000","last_fullsized":"2024-09-15T22:44:19.213222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:31.545336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.196491+0000","last_change":"2024-09-15T22:44:17.207567+0000","last_active":"2024-09-15T22:44:19.196491+0000","last_peered":"2024-09-15T22:44:19.196491+0000","last_clean":"2024-09-15T22:44:19.196491+0000","last_became_active":"2024-09-15T22:44:17.207245+0000","last_became_peered":"2024-09-15T22:44:17.207245+0000","last_unstale":"2024-09-15T22:44:19.196491+0000","last_undegraded":"2024-09-15T22:44:19.196491+0000","last_fullsized":"2024-09-15T22:44:19.196491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:35.169205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.211896+0000","last_change":"2024-09-15T22:44:17.209449+0000","last_active":"2024-09-15T22:44:19.211896+0000","last_peered":"2024-09-15T22:44:19.211896+0000","last_clean":"2024-09-15T22:44:19.211896+0000","last_became_active":"2024-09-15T22:44:17.209012+0000","last_became_peered":"2024-09-15T22:44:17.209012+0000","last_unstale":"2024-09-15T22:44:19.211896+0000","last_undegraded":"2024-09-15T22:44:19.211896+0000","last_fullsized":"2024-09-15T22:44:19.211896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:59.550776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.196130+0000","last_change":"2024-09-15T22:44:17.214184+0000","last_active":"2024-09-15T22:44:19.196130+0000","last_peered":"2024-09-15T22:44:19.196130+0000","last_clean":"2024-09-15T22:44:19.196130+0000","last_became_active":"2024-09-15T22:44:17.213884+0000","last_became_peered":"2024-09-15T22:44:17.213884+0000","last_unstale":"2024-09-15T22:44:19.196130+0000","last_undegraded":"2024-09-15T22:44:19.196130+0000","last_fullsized":"2024-09-15T22:44:19.196130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:33.336473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.213894+0000","last_change":"2024-09-15T22:44:17.203904+0000","last_active":"2024-09-15T22:44:19.213894+0000","last_peered":"2024-09-15T22:44:19.213894+0000","last_clean":"2024-09-15T22:44:19.213894+0000","last_became_active":"2024-09-15T22:44:17.203484+0000","last_became_peered":"2024-09-15T22:44:17.203484+0000","last_unstale":"2024-09-15T22:44:19.213894+0000","last_undegraded":"2024-09-15T22:44:19.213894+0000","last_fullsized":"2024-09-15T22:44:19.213894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:33:09.906869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.212578+0000","last_change":"2024-09-15T22:44:17.209356+0000","last_active":"2024-09-15T22:44:19.212578+0000","last_peered":"2024-09-15T22:44:19.212578+0000","last_clean":"2024-09-15T22:44:19.212578+0000","last_became_active":"2024-09-15T22:44:17.207229+0000","last_became_peered":"2024-09-15T22:44:17.207229+0000","last_unstale":"2024-09-15T22:44:19.212578+0000","last_undegraded":"2024-09-15T22:44:19.212578+0000","last_fullsized":"2024-09-15T22:44:19.212578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:32:28.405429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.217516+0000","last_change":"2024-09-15T22:44:17.202505+0000","last_active":"2024-09-15T22:44:19.217516+0000","last_peered":"2024-09-15T22:44:19.217516+0000","last_clean":"2024-09-15T22:44:19.217516+0000","last_became_active":"2024-09-15T22:44:17.202154+0000","last_became_peered":"2024-09-15T22:44:17.202154+0000","last_unstale":"2024-09-15T22:44:19.217516+0000","last_undegraded":"2024-09-15T22:44:19.217516+0000","last_fullsized":"2024-09-15T22:44:19.217516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:40:05.680982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.212271+0000","last_change":"2024-09-15T22:44:17.222767+0000","last_active":"2024-09-15T22:44:19.212271+0000","last_peered":"2024-09-15T22:44:19.212271+0000","last_clean":"2024-09-15T22:44:19.212271+0000","last_became_active":"2024-09-15T22:44:17.221655+0000","last_became_peered":"2024-09-15T22:44:17.221655+0000","last_unstale":"2024-09-15T22:44:19.212271+0000","last_undegraded":"2024-09-15T22:44:19.212271+0000","last_fullsized":"2024-09-15T22:44:19.212271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:22.457576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.214543+0000","last_change":"2024-09-15T22:44:17.217485+0000","last_active":"2024-09-15T22:44:19.214543+0000","last_peered":"2024-09-15T22:44:19.214543+0000","last_clean":"2024-09-15T22:44:19.214543+0000","last_became_active":"2024-09-15T22:44:17.217138+0000","last_became_peered":"2024-09-15T22:44:17.217138+0000","last_unstale":"2024-09-15T22:44:19.214543+0000","last_undegraded":"2024-09-15T22:44:19.214543+0000","last_fullsized":"2024-09-15T22:44:19.214543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:41:40.606593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.211724+0000","last_change":"2024-09-15T22:44:17.217996+0000","last_active":"2024-09-15T22:44:19.211724+0000","last_peered":"2024-09-15T22:44:19.211724+0000","last_clean":"2024-09-15T22:44:19.211724+0000","last_became_active":"2024-09-15T22:44:17.217467+0000","last_became_peered":"2024-09-15T22:44:17.217467+0000","last_unstale":"2024-09-15T22:44:19.211724+0000","last_undegraded":"2024-09-15T22:44:19.211724+0000","last_fullsized":"2024-09-15T22:44:19.211724+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:25:45.150979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.211590+0000","last_change":"2024-09-15T22:44:17.202840+0000","last_active":"2024-09-15T22:44:19.211590+0000","last_peered":"2024-09-15T22:44:19.211590+0000","last_clean":"2024-09-15T22:44:19.211590+0000","last_became_active":"2024-09-15T22:44:17.202481+0000","last_became_peered":"2024-09-15T22:44:17.202481+0000","last_unstale":"2024-09-15T22:44:19.211590+0000","last_undegraded":"2024-09-15T22:44:19.211590+0000","last_fullsized":"2024-09-15T22:44:19.211590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:31:09.360211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.217492+0000","last_change":"2024-09-15T22:44:17.209387+0000","last_active":"2024-09-15T22:44:19.217492+0000","last_peered":"2024-09-15T22:44:19.217492+0000","last_clean":"2024-09-15T22:44:19.217492+0000","last_became_active":"2024-09-15T22:44:17.209100+0000","last_became_peered":"2024-09-15T22:44:17.209100+0000","last_unstale":"2024-09-15T22:44:19.217492+0000","last_undegraded":"2024-09-15T22:44:19.217492+0000","last_fullsized":"2024-09-15T22:44:19.217492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:57:47.942784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.211418+0000","last_change":"2024-09-15T22:44:17.203626+0000","last_active":"2024-09-15T22:44:19.211418+0000","last_peered":"2024-09-15T22:44:19.211418+0000","last_clean":"2024-09-15T22:44:19.211418+0000","last_became_active":"2024-09-15T22:44:17.203313+0000","last_became_peered":"2024-09-15T22:44:17.203313+0000","last_unstale":"2024-09-15T22:44:19.211418+0000","last_undegraded":"2024-09-15T22:44:19.211418+0000","last_fullsized":"2024-09-15T22:44:19.211418+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:42:53.765397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:47.596337+0000","last_change":"2024-09-15T22:44:14.172460+0000","last_active":"2024-09-15T22:44:47.596337+0000","last_peered":"2024-09-15T22:44:47.596337+0000","last_clean":"2024-09-15T22:44:47.596337+0000","last_became_active":"2024-09-15T22:44:14.171960+0000","last_became_peered":"2024-09-15T22:44:14.171960+0000","last_unstale":"2024-09-15T22:44:47.596337+0000","last_undegraded":"2024-09-15T22:44:47.596337+0000","last_fullsized":"2024-09-15T22:44:47.596337+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:27:55.116610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.196495+0000","last_change":"2024-09-15T22:44:17.210054+0000","last_active":"2024-09-15T22:44:19.196495+0000","last_peered":"2024-09-15T22:44:19.196495+0000","last_clean":"2024-09-15T22:44:19.196495+0000","last_became_active":"2024-09-15T22:44:17.209607+0000","last_became_peered":"2024-09-15T22:44:17.209607+0000","last_unstale":"2024-09-15T22:44:19.196495+0000","last_undegraded":"2024-09-15T22:44:19.196495+0000","last_fullsized":"2024-09-15T22:44:19.196495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:13.867965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:29.555182+0000","last_change":"2024-09-15T22:44:14.167537+0000","last_active":"2024-09-15T22:44:29.555182+0000","last_peered":"2024-09-15T22:44:29.555182+0000","last_clean":"2024-09-15T22:44:29.555182+0000","last_became_active":"2024-09-15T22:44:14.167173+0000","last_became_peered":"2024-09-15T22:44:14.167173+0000","last_unstale":"2024-09-15T22:44:29.555182+0000","last_undegraded":"2024-09-15T22:44:29.555182+0000","last_fullsized":"2024-09-15T22:44:29.555182+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:45:56.539218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:29.554959+0000","last_change":"2024-09-15T22:44:14.166676+0000","last_active":"2024-09-15T22:44:29.554959+0000","last_peered":"2024-09-15T22:44:29.554959+0000","last_clean":"2024-09-15T22:44:29.554959+0000","last_became_active":"2024-09-15T22:44:14.166342+0000","last_became_peered":"2024-09-15T22:44:14.166342+0000","last_unstale":"2024-09-15T22:44:29.554959+0000","last_undegraded":"2024-09-15T22:44:29.554959+0000","last_fullsized":"2024-09-15T22:44:29.554959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:41.861253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.211425+0000","last_change":"2024-09-15T22:44:17.211312+0000","last_active":"2024-09-15T22:44:19.211425+0000","last_peered":"2024-09-15T22:44:19.211425+0000","last_clean":"2024-09-15T22:44:19.211425+0000","last_became_active":"2024-09-15T22:44:17.208771+0000","last_became_peered":"2024-09-15T22:44:17.208771+0000","last_unstale":"2024-09-15T22:44:19.211425+0000","last_undegraded":"2024-09-15T22:44:19.211425+0000","last_fullsized":"2024-09-15T22:44:19.211425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:59:51.250489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.196460+0000","last_change":"2024-09-15T22:44:14.167263+0000","last_active":"2024-09-15T22:44:19.196460+0000","last_peered":"2024-09-15T22:44:19.196460+0000","last_clean":"2024-09-15T22:44:19.196460+0000","last_became_active":"2024-09-15T22:44:14.166942+0000","last_became_peered":"2024-09-15T22:44:14.166942+0000","last_unstale":"2024-09-15T22:44:19.196460+0000","last_undegraded":"2024-09-15T22:44:19.196460+0000","last_fullsized":"2024-09-15T22:44:19.196460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:47.320374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.193297+0000","last_change":"2024-09-15T22:44:17.210986+0000","last_active":"2024-09-15T22:44:19.193297+0000","last_peered":"2024-09-15T22:44:19.193297+0000","last_clean":"2024-09-15T22:44:19.193297+0000","last_became_active":"2024-09-15T22:44:17.210618+0000","last_became_peered":"2024-09-15T22:44:17.210618+0000","last_unstale":"2024-09-15T22:44:19.193297+0000","last_undegraded":"2024-09-15T22:44:19.193297+0000","last_fullsized":"2024-09-15T22:44:19.193297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:58:04.961630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:48.597053+0000","last_change":"2024-09-15T22:44:14.164770+0000","last_active":"2024-09-15T22:44:48.597053+0000","last_peered":"2024-09-15T22:44:48.597053+0000","last_clean":"2024-09-15T22:44:48.597053+0000","last_became_active":"2024-09-15T22:44:14.164411+0000","last_became_peered":"2024-09-15T22:44:14.164411+0000","last_unstale":"2024-09-15T22:44:48.597053+0000","last_undegraded":"2024-09-15T22:44:48.597053+0000","last_fullsized":"2024-09-15T22:44:48.597053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:26:47.623520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.196692+0000","last_change":"2024-09-15T22:44:17.204049+0000","last_active":"2024-09-15T22:44:19.196692+0000","last_peered":"2024-09-15T22:44:19.196692+0000","last_clean":"2024-09-15T22:44:19.196692+0000","last_became_active":"2024-09-15T22:44:17.203738+0000","last_became_peered":"2024-09-15T22:44:17.203738+0000","last_unstale":"2024-09-15T22:44:19.196692+0000","last_undegraded":"2024-09-15T22:44:19.196692+0000","last_fullsized":"2024-09-15T22:44:19.196692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:56.347792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:28.544197+0000","last_change":"2024-09-15T22:44:14.167228+0000","last_active":"2024-09-15T22:44:28.544197+0000","last_peered":"2024-09-15T22:44:28.544197+0000","last_clean":"2024-09-15T22:44:28.544197+0000","last_became_active":"2024-09-15T22:44:14.166662+0000","last_became_peered":"2024-09-15T22:44:14.166662+0000","last_unstale":"2024-09-15T22:44:28.544197+0000","last_undegraded":"2024-09-15T22:44:28.544197+0000","last_fullsized":"2024-09-15T22:44:28.544197+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:31:17.160718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.213604+0000","last_change":"2024-09-15T22:44:17.212549+0000","last_active":"2024-09-15T22:44:19.213604+0000","last_peered":"2024-09-15T22:44:19.213604+0000","last_clean":"2024-09-15T22:44:19.213604+0000","last_became_active":"2024-09-15T22:44:17.212123+0000","last_became_peered":"2024-09-15T22:44:17.212123+0000","last_unstale":"2024-09-15T22:44:19.213604+0000","last_undegraded":"2024-09-15T22:44:19.213604+0000","last_fullsized":"2024-09-15T22:44:19.213604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:32:45.241199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:28.544533+0000","last_change":"2024-09-15T22:44:14.679664+0000","last_active":"2024-09-15T22:44:28.544533+0000","last_peered":"2024-09-15T22:44:28.544533+0000","last_clean":"2024-09-15T22:44:28.544533+0000","last_became_active":"2024-09-15T22:44:14.679256+0000","last_became_peered":"2024-09-15T22:44:14.679256+0000","last_unstale":"2024-09-15T22:44:28.544533+0000","last_undegraded":"2024-09-15T22:44:28.544533+0000","last_fullsized":"2024-09-15T22:44:28.544533+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:33:41.512488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.196305+0000","last_change":"2024-09-15T22:44:17.200448+0000","last_active":"2024-09-15T22:44:19.196305+0000","last_peered":"2024-09-15T22:44:19.196305+0000","last_clean":"2024-09-15T22:44:19.196305+0000","last_became_active":"2024-09-15T22:44:17.200114+0000","last_became_peered":"2024-09-15T22:44:17.200114+0000","last_unstale":"2024-09-15T22:44:19.196305+0000","last_undegraded":"2024-09-15T22:44:19.196305+0000","last_fullsized":"2024-09-15T22:44:19.196305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:36.403644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'26","reported_seq":93,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:49.599637+0000","last_change":"2024-09-15T22:44:14.167067+0000","last_active":"2024-09-15T22:44:49.599637+0000","last_peered":"2024-09-15T22:44:49.599637+0000","last_clean":"2024-09-15T22:44:49.599637+0000","last_became_active":"2024-09-15T22:44:14.166613+0000","last_became_peered":"2024-09-15T22:44:14.166613+0000","last_unstale":"2024-09-15T22:44:49.599637+0000","last_undegraded":"2024-09-15T22:44:49.599637+0000","last_fullsized":"2024-09-15T22:44:49.599637+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:15.360717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53860,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":70,"num_write":27,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.211865+0000","last_change":"2024-09-15T22:44:17.202705+0000","last_active":"2024-09-15T22:44:19.211865+0000","last_peered":"2024-09-15T22:44:19.211865+0000","last_clean":"2024-09-15T22:44:19.211865+0000","last_became_active":"2024-09-15T22:44:17.202346+0000","last_became_peered":"2024-09-15T22:44:17.202346+0000","last_unstale":"2024-09-15T22:44:19.211865+0000","last_undegraded":"2024-09-15T22:44:19.211865+0000","last_fullsized":"2024-09-15T22:44:19.211865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:31.193746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:29.555212+0000","last_change":"2024-09-15T22:44:14.169255+0000","last_active":"2024-09-15T22:44:29.555212+0000","last_peered":"2024-09-15T22:44:29.555212+0000","last_clean":"2024-09-15T22:44:29.555212+0000","last_became_active":"2024-09-15T22:44:14.168774+0000","last_became_peered":"2024-09-15T22:44:14.168774+0000","last_unstale":"2024-09-15T22:44:29.555212+0000","last_undegraded":"2024-09-15T22:44:29.555212+0000","last_fullsized":"2024-09-15T22:44:29.555212+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:48.858831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.210812+0000","last_change":"2024-09-15T22:44:17.218635+0000","last_active":"2024-09-15T22:44:19.210812+0000","last_peered":"2024-09-15T22:44:19.210812+0000","last_clean":"2024-09-15T22:44:19.210812+0000","last_became_active":"2024-09-15T22:44:17.218287+0000","last_became_peered":"2024-09-15T22:44:17.218287+0000","last_unstale":"2024-09-15T22:44:19.210812+0000","last_undegraded":"2024-09-15T22:44:19.210812+0000","last_fullsized":"2024-09-15T22:44:19.210812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:58.264382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:28.544054+0000","last_change":"2024-09-15T22:44:14.166580+0000","last_active":"2024-09-15T22:44:28.544054+0000","last_peered":"2024-09-15T22:44:28.544054+0000","last_clean":"2024-09-15T22:44:28.544054+0000","last_became_active":"2024-09-15T22:44:14.164955+0000","last_became_peered":"2024-09-15T22:44:14.164955+0000","last_unstale":"2024-09-15T22:44:28.544054+0000","last_undegraded":"2024-09-15T22:44:28.544054+0000","last_fullsized":"2024-09-15T22:44:28.544054+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:14.215662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.211342+0000","last_change":"2024-09-15T22:44:17.205731+0000","last_active":"2024-09-15T22:44:19.211342+0000","last_peered":"2024-09-15T22:44:19.211342+0000","last_clean":"2024-09-15T22:44:19.211342+0000","last_became_active":"2024-09-15T22:44:17.205361+0000","last_became_peered":"2024-09-15T22:44:17.205361+0000","last_unstale":"2024-09-15T22:44:19.211342+0000","last_undegraded":"2024-09-15T22:44:19.211342+0000","last_fullsized":"2024-09-15T22:44:19.211342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:25.430166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:46.594288+0000","last_change":"2024-09-15T22:44:14.166471+0000","last_active":"2024-09-15T22:44:46.594288+0000","last_peered":"2024-09-15T22:44:46.594288+0000","last_clean":"2024-09-15T22:44:46.594288+0000","last_became_active":"2024-09-15T22:44:14.165969+0000","last_became_peered":"2024-09-15T22:44:14.165969+0000","last_unstale":"2024-09-15T22:44:46.594288+0000","last_undegraded":"2024-09-15T22:44:46.594288+0000","last_fullsized":"2024-09-15T22:44:46.594288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:57:40.108575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.211070+0000","last_change":"2024-09-15T22:44:17.201488+0000","last_active":"2024-09-15T22:44:19.211070+0000","last_peered":"2024-09-15T22:44:19.211070+0000","last_clean":"2024-09-15T22:44:19.211070+0000","last_became_active":"2024-09-15T22:44:17.201101+0000","last_became_peered":"2024-09-15T22:44:17.201101+0000","last_unstale":"2024-09-15T22:44:19.211070+0000","last_undegraded":"2024-09-15T22:44:19.211070+0000","last_fullsized":"2024-09-15T22:44:19.211070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:41.758433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:29.555225+0000","last_change":"2024-09-15T22:44:14.680734+0000","last_active":"2024-09-15T22:44:29.555225+0000","last_peered":"2024-09-15T22:44:29.555225+0000","last_clean":"2024-09-15T22:44:29.555225+0000","last_became_active":"2024-09-15T22:44:14.680362+0000","last_became_peered":"2024-09-15T22:44:14.680362+0000","last_unstale":"2024-09-15T22:44:29.555225+0000","last_undegraded":"2024-09-15T22:44:29.555225+0000","last_fullsized":"2024-09-15T22:44:29.555225+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:22:37.086300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.196218+0000","last_change":"2024-09-15T22:44:17.199944+0000","last_active":"2024-09-15T22:44:19.196218+0000","last_peered":"2024-09-15T22:44:19.196218+0000","last_clean":"2024-09-15T22:44:19.196218+0000","last_became_active":"2024-09-15T22:44:17.199586+0000","last_became_peered":"2024-09-15T22:44:17.199586+0000","last_unstale":"2024-09-15T22:44:19.196218+0000","last_undegraded":"2024-09-15T22:44:19.196218+0000","last_fullsized":"2024-09-15T22:44:19.196218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:42:22.178939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:31.549790+0000","last_change":"2024-09-15T22:44:14.169077+0000","last_active":"2024-09-15T22:44:31.549790+0000","last_peered":"2024-09-15T22:44:31.549790+0000","last_clean":"2024-09-15T22:44:31.549790+0000","last_became_active":"2024-09-15T22:44:14.168675+0000","last_became_peered":"2024-09-15T22:44:14.168675+0000","last_unstale":"2024-09-15T22:44:31.549790+0000","last_undegraded":"2024-09-15T22:44:31.549790+0000","last_fullsized":"2024-09-15T22:44:31.549790+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:01.892217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.217871+0000","last_change":"2024-09-15T22:44:17.208587+0000","last_active":"2024-09-15T22:44:19.217871+0000","last_peered":"2024-09-15T22:44:19.217871+0000","last_clean":"2024-09-15T22:44:19.217871+0000","last_became_active":"2024-09-15T22:44:17.208260+0000","last_became_peered":"2024-09-15T22:44:17.208260+0000","last_unstale":"2024-09-15T22:44:19.217871+0000","last_undegraded":"2024-09-15T22:44:19.217871+0000","last_fullsized":"2024-09-15T22:44:19.217871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:01:04.154149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.216128+0000","last_change":"2024-09-15T22:44:14.168223+0000","last_active":"2024-09-15T22:44:19.216128+0000","last_peered":"2024-09-15T22:44:19.216128+0000","last_clean":"2024-09-15T22:44:19.216128+0000","last_became_active":"2024-09-15T22:44:14.167830+0000","last_became_peered":"2024-09-15T22:44:14.167830+0000","last_unstale":"2024-09-15T22:44:19.216128+0000","last_undegraded":"2024-09-15T22:44:19.216128+0000","last_fullsized":"2024-09-15T22:44:19.216128+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:54:38.064606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.213111+0000","last_change":"2024-09-15T22:44:17.209696+0000","last_active":"2024-09-15T22:44:19.213111+0000","last_peered":"2024-09-15T22:44:19.213111+0000","last_clean":"2024-09-15T22:44:19.213111+0000","last_became_active":"2024-09-15T22:44:17.208667+0000","last_became_peered":"2024-09-15T22:44:17.208667+0000","last_unstale":"2024-09-15T22:44:19.213111+0000","last_undegraded":"2024-09-15T22:44:19.213111+0000","last_fullsized":"2024-09-15T22:44:19.213111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:04.653022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.215695+0000","last_change":"2024-09-15T22:44:14.169728+0000","last_active":"2024-09-15T22:44:19.215695+0000","last_peered":"2024-09-15T22:44:19.215695+0000","last_clean":"2024-09-15T22:44:19.215695+0000","last_became_active":"2024-09-15T22:44:14.169354+0000","last_became_peered":"2024-09-15T22:44:14.169354+0000","last_unstale":"2024-09-15T22:44:19.215695+0000","last_undegraded":"2024-09-15T22:44:19.215695+0000","last_fullsized":"2024-09-15T22:44:19.215695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:20.273852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.502524+0000","last_change":"2024-09-15T22:44:17.210593+0000","last_active":"2024-09-15T22:44:19.502524+0000","last_peered":"2024-09-15T22:44:19.502524+0000","last_clean":"2024-09-15T22:44:19.502524+0000","last_became_active":"2024-09-15T22:44:17.210210+0000","last_became_peered":"2024-09-15T22:44:17.210210+0000","last_unstale":"2024-09-15T22:44:19.502524+0000","last_undegraded":"2024-09-15T22:44:19.502524+0000","last_fullsized":"2024-09-15T22:44:19.502524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:56.817301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:29.554668+0000","last_change":"2024-09-15T22:44:14.679757+0000","last_active":"2024-09-15T22:44:29.554668+0000","last_peered":"2024-09-15T22:44:29.554668+0000","last_clean":"2024-09-15T22:44:29.554668+0000","last_became_active":"2024-09-15T22:44:14.679284+0000","last_became_peered":"2024-09-15T22:44:14.679284+0000","last_unstale":"2024-09-15T22:44:29.554668+0000","last_undegraded":"2024-09-15T22:44:29.554668+0000","last_fullsized":"2024-09-15T22:44:29.554668+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:47:21.368373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.212866+0000","last_change":"2024-09-15T22:44:17.207115+0000","last_active":"2024-09-15T22:44:19.212866+0000","last_peered":"2024-09-15T22:44:19.212866+0000","last_clean":"2024-09-15T22:44:19.212866+0000","last_became_active":"2024-09-15T22:44:17.206800+0000","last_became_peered":"2024-09-15T22:44:17.206800+0000","last_unstale":"2024-09-15T22:44:19.212866+0000","last_undegraded":"2024-09-15T22:44:19.212866+0000","last_fullsized":"2024-09-15T22:44:19.212866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:49:17.497725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":335,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.211217+0000","last_change":"2024-09-15T22:42:59.117839+0000","last_active":"2024-09-15T22:44:19.211217+0000","last_peered":"2024-09-15T22:44:19.211217+0000","last_clean":"2024-09-15T22:44:19.211217+0000","last_became_active":"2024-09-15T22:42:59.117434+0000","last_became_peered":"2024-09-15T22:42:59.117434+0000","last_unstale":"2024-09-15T22:44:19.211217+0000","last_undegraded":"2024-09-15T22:44:19.211217+0000","last_fullsized":"2024-09-15T22:44:19.211217+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:39:19.064511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:39:19.064511+0000","last_clean_scrub_stamp":"2024-09-15T22:39:19.064511+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:38.974950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:31.569007+0000","last_change":"2024-09-15T22:44:14.170422+0000","last_active":"2024-09-15T22:44:31.569007+0000","last_peered":"2024-09-15T22:44:31.569007+0000","last_clean":"2024-09-15T22:44:31.569007+0000","last_became_active":"2024-09-15T22:44:14.169152+0000","last_became_peered":"2024-09-15T22:44:14.169152+0000","last_unstale":"2024-09-15T22:44:31.569007+0000","last_undegraded":"2024-09-15T22:44:31.569007+0000","last_fullsized":"2024-09-15T22:44:31.569007+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:10:00.890434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.212511+0000","last_change":"2024-09-15T22:44:17.212972+0000","last_active":"2024-09-15T22:44:19.212511+0000","last_peered":"2024-09-15T22:44:19.212511+0000","last_clean":"2024-09-15T22:44:19.212511+0000","last_became_active":"2024-09-15T22:44:17.212546+0000","last_became_peered":"2024-09-15T22:44:17.212546+0000","last_unstale":"2024-09-15T22:44:19.212511+0000","last_undegraded":"2024-09-15T22:44:19.212511+0000","last_fullsized":"2024-09-15T22:44:19.212511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:36.322855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:29.554822+0000","last_change":"2024-09-15T22:44:14.171827+0000","last_active":"2024-09-15T22:44:29.554822+0000","last_peered":"2024-09-15T22:44:29.554822+0000","last_clean":"2024-09-15T22:44:29.554822+0000","last_became_active":"2024-09-15T22:44:14.171447+0000","last_became_peered":"2024-09-15T22:44:14.171447+0000","last_unstale":"2024-09-15T22:44:29.554822+0000","last_undegraded":"2024-09-15T22:44:29.554822+0000","last_fullsized":"2024-09-15T22:44:29.554822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:40:49.869819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.211310+0000","last_change":"2024-09-15T22:44:17.208666+0000","last_active":"2024-09-15T22:44:19.211310+0000","last_peered":"2024-09-15T22:44:19.211310+0000","last_clean":"2024-09-15T22:44:19.211310+0000","last_became_active":"2024-09-15T22:44:17.208338+0000","last_became_peered":"2024-09-15T22:44:17.208338+0000","last_unstale":"2024-09-15T22:44:19.211310+0000","last_undegraded":"2024-09-15T22:44:19.211310+0000","last_fullsized":"2024-09-15T22:44:19.211310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:23.250307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:48.596489+0000","last_change":"2024-09-15T22:44:14.679708+0000","last_active":"2024-09-15T22:44:48.596489+0000","last_peered":"2024-09-15T22:44:48.596489+0000","last_clean":"2024-09-15T22:44:48.596489+0000","last_became_active":"2024-09-15T22:44:14.679192+0000","last_became_peered":"2024-09-15T22:44:14.679192+0000","last_unstale":"2024-09-15T22:44:48.596489+0000","last_undegraded":"2024-09-15T22:44:48.596489+0000","last_fullsized":"2024-09-15T22:44:48.596489+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:25:21.192842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.196209+0000","last_change":"2024-09-15T22:44:17.202716+0000","last_active":"2024-09-15T22:44:19.196209+0000","last_peered":"2024-09-15T22:44:19.196209+0000","last_clean":"2024-09-15T22:44:19.196209+0000","last_became_active":"2024-09-15T22:44:17.202412+0000","last_became_peered":"2024-09-15T22:44:17.202412+0000","last_unstale":"2024-09-15T22:44:19.196209+0000","last_undegraded":"2024-09-15T22:44:19.196209+0000","last_fullsized":"2024-09-15T22:44:19.196209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:18:59.180862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.213083+0000","last_change":"2024-09-15T22:44:14.169202+0000","last_active":"2024-09-15T22:44:19.213083+0000","last_peered":"2024-09-15T22:44:19.213083+0000","last_clean":"2024-09-15T22:44:19.213083+0000","last_became_active":"2024-09-15T22:44:14.168866+0000","last_became_peered":"2024-09-15T22:44:14.168866+0000","last_unstale":"2024-09-15T22:44:19.213083+0000","last_undegraded":"2024-09-15T22:44:19.213083+0000","last_fullsized":"2024-09-15T22:44:19.213083+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:28:57.057050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.210406+0000","last_change":"2024-09-15T22:44:17.218341+0000","last_active":"2024-09-15T22:44:19.210406+0000","last_peered":"2024-09-15T22:44:19.210406+0000","last_clean":"2024-09-15T22:44:19.210406+0000","last_became_active":"2024-09-15T22:44:17.218004+0000","last_became_peered":"2024-09-15T22:44:17.218004+0000","last_unstale":"2024-09-15T22:44:19.210406+0000","last_undegraded":"2024-09-15T22:44:19.210406+0000","last_fullsized":"2024-09-15T22:44:19.210406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:56:54.416938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.193395+0000","last_change":"2024-09-15T22:44:14.681514+0000","last_active":"2024-09-15T22:44:19.193395+0000","last_peered":"2024-09-15T22:44:19.193395+0000","last_clean":"2024-09-15T22:44:19.193395+0000","last_became_active":"2024-09-15T22:44:14.679895+0000","last_became_peered":"2024-09-15T22:44:14.679895+0000","last_unstale":"2024-09-15T22:44:19.193395+0000","last_undegraded":"2024-09-15T22:44:19.193395+0000","last_fullsized":"2024-09-15T22:44:19.193395+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:13:22.059389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:20.545393+0000","last_change":"2024-09-15T22:44:17.210094+0000","last_active":"2024-09-15T22:44:20.545393+0000","last_peered":"2024-09-15T22:44:20.545393+0000","last_clean":"2024-09-15T22:44:20.545393+0000","last_became_active":"2024-09-15T22:44:17.209784+0000","last_became_peered":"2024-09-15T22:44:17.209784+0000","last_unstale":"2024-09-15T22:44:20.545393+0000","last_undegraded":"2024-09-15T22:44:20.545393+0000","last_fullsized":"2024-09-15T22:44:20.545393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:02.357936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:28.545395+0000","last_change":"2024-09-15T22:44:14.680558+0000","last_active":"2024-09-15T22:44:28.545395+0000","last_peered":"2024-09-15T22:44:28.545395+0000","last_clean":"2024-09-15T22:44:28.545395+0000","last_became_active":"2024-09-15T22:44:14.680183+0000","last_became_peered":"2024-09-15T22:44:14.680183+0000","last_unstale":"2024-09-15T22:44:28.545395+0000","last_undegraded":"2024-09-15T22:44:28.545395+0000","last_fullsized":"2024-09-15T22:44:28.545395+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:44.327412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.214624+0000","last_change":"2024-09-15T22:44:17.210661+0000","last_active":"2024-09-15T22:44:19.214624+0000","last_peered":"2024-09-15T22:44:19.214624+0000","last_clean":"2024-09-15T22:44:19.214624+0000","last_became_active":"2024-09-15T22:44:17.210279+0000","last_became_peered":"2024-09-15T22:44:17.210279+0000","last_unstale":"2024-09-15T22:44:19.214624+0000","last_undegraded":"2024-09-15T22:44:19.214624+0000","last_fullsized":"2024-09-15T22:44:19.214624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:58:45.430753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'8","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:43.209636+0000","last_change":"2024-09-15T22:44:14.164859+0000","last_active":"2024-09-15T22:44:43.209636+0000","last_peered":"2024-09-15T22:44:43.209636+0000","last_clean":"2024-09-15T22:44:43.209636+0000","last_became_active":"2024-09-15T22:44:14.163843+0000","last_became_peered":"2024-09-15T22:44:14.163843+0000","last_unstale":"2024-09-15T22:44:43.209636+0000","last_undegraded":"2024-09-15T22:44:43.209636+0000","last_fullsized":"2024-09-15T22:44:43.209636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:01.322732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.214067+0000","last_change":"2024-09-15T22:44:17.211891+0000","last_active":"2024-09-15T22:44:19.214067+0000","last_peered":"2024-09-15T22:44:19.214067+0000","last_clean":"2024-09-15T22:44:19.214067+0000","last_became_active":"2024-09-15T22:44:17.211515+0000","last_became_peered":"2024-09-15T22:44:17.211515+0000","last_unstale":"2024-09-15T22:44:19.214067+0000","last_undegraded":"2024-09-15T22:44:19.214067+0000","last_fullsized":"2024-09-15T22:44:19.214067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:05:58.223408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:28.543513+0000","last_change":"2024-09-15T22:44:14.167623+0000","last_active":"2024-09-15T22:44:28.543513+0000","last_peered":"2024-09-15T22:44:28.543513+0000","last_clean":"2024-09-15T22:44:28.543513+0000","last_became_active":"2024-09-15T22:44:14.167280+0000","last_became_peered":"2024-09-15T22:44:14.167280+0000","last_unstale":"2024-09-15T22:44:28.543513+0000","last_undegraded":"2024-09-15T22:44:28.543513+0000","last_fullsized":"2024-09-15T22:44:28.543513+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:00:07.609473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.213082+0000","last_change":"2024-09-15T22:44:17.215630+0000","last_active":"2024-09-15T22:44:19.213082+0000","last_peered":"2024-09-15T22:44:19.213082+0000","last_clean":"2024-09-15T22:44:19.213082+0000","last_became_active":"2024-09-15T22:44:17.215328+0000","last_became_peered":"2024-09-15T22:44:17.215328+0000","last_unstale":"2024-09-15T22:44:19.213082+0000","last_undegraded":"2024-09-15T22:44:19.213082+0000","last_fullsized":"2024-09-15T22:44:19.213082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:13:54.545954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:28.545096+0000","last_change":"2024-09-15T22:44:14.680231+0000","last_active":"2024-09-15T22:44:28.545096+0000","last_peered":"2024-09-15T22:44:28.545096+0000","last_clean":"2024-09-15T22:44:28.545096+0000","last_became_active":"2024-09-15T22:44:14.679858+0000","last_became_peered":"2024-09-15T22:44:14.679858+0000","last_unstale":"2024-09-15T22:44:28.545096+0000","last_undegraded":"2024-09-15T22:44:28.545096+0000","last_fullsized":"2024-09-15T22:44:28.545096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:45:58.992336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.210843+0000","last_change":"2024-09-15T22:44:17.208072+0000","last_active":"2024-09-15T22:44:19.210843+0000","last_peered":"2024-09-15T22:44:19.210843+0000","last_clean":"2024-09-15T22:44:19.210843+0000","last_became_active":"2024-09-15T22:44:17.207580+0000","last_became_peered":"2024-09-15T22:44:17.207580+0000","last_unstale":"2024-09-15T22:44:19.210843+0000","last_undegraded":"2024-09-15T22:44:19.210843+0000","last_fullsized":"2024-09-15T22:44:19.210843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:48.973412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:28.545395+0000","last_change":"2024-09-15T22:44:14.168841+0000","last_active":"2024-09-15T22:44:28.545395+0000","last_peered":"2024-09-15T22:44:28.545395+0000","last_clean":"2024-09-15T22:44:28.545395+0000","last_became_active":"2024-09-15T22:44:14.168359+0000","last_became_peered":"2024-09-15T22:44:14.168359+0000","last_unstale":"2024-09-15T22:44:28.545395+0000","last_undegraded":"2024-09-15T22:44:28.545395+0000","last_fullsized":"2024-09-15T22:44:28.545395+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:09:29.752679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.217654+0000","last_change":"2024-09-15T22:44:17.218227+0000","last_active":"2024-09-15T22:44:19.217654+0000","last_peered":"2024-09-15T22:44:19.217654+0000","last_clean":"2024-09-15T22:44:19.217654+0000","last_became_active":"2024-09-15T22:44:17.217846+0000","last_became_peered":"2024-09-15T22:44:17.217846+0000","last_unstale":"2024-09-15T22:44:19.217654+0000","last_undegraded":"2024-09-15T22:44:19.217654+0000","last_fullsized":"2024-09-15T22:44:19.217654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:07.601525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.211941+0000","last_change":"2024-09-15T22:44:14.165322+0000","last_active":"2024-09-15T22:44:19.211941+0000","last_peered":"2024-09-15T22:44:19.211941+0000","last_clean":"2024-09-15T22:44:19.211941+0000","last_became_active":"2024-09-15T22:44:14.164981+0000","last_became_peered":"2024-09-15T22:44:14.164981+0000","last_unstale":"2024-09-15T22:44:19.211941+0000","last_undegraded":"2024-09-15T22:44:19.211941+0000","last_fullsized":"2024-09-15T22:44:19.211941+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:34.812514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.212912+0000","last_change":"2024-09-15T22:44:17.222414+0000","last_active":"2024-09-15T22:44:19.212912+0000","last_peered":"2024-09-15T22:44:19.212912+0000","last_clean":"2024-09-15T22:44:19.212912+0000","last_became_active":"2024-09-15T22:44:17.222115+0000","last_became_peered":"2024-09-15T22:44:17.222115+0000","last_unstale":"2024-09-15T22:44:19.212912+0000","last_undegraded":"2024-09-15T22:44:19.212912+0000","last_fullsized":"2024-09-15T22:44:19.212912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:22:53.791587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:28.544510+0000","last_change":"2024-09-15T22:44:14.166605+0000","last_active":"2024-09-15T22:44:28.544510+0000","last_peered":"2024-09-15T22:44:28.544510+0000","last_clean":"2024-09-15T22:44:28.544510+0000","last_became_active":"2024-09-15T22:44:14.165667+0000","last_became_peered":"2024-09-15T22:44:14.165667+0000","last_unstale":"2024-09-15T22:44:28.544510+0000","last_undegraded":"2024-09-15T22:44:28.544510+0000","last_fullsized":"2024-09-15T22:44:28.544510+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:27.060540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.212549+0000","last_change":"2024-09-15T22:44:17.208143+0000","last_active":"2024-09-15T22:44:19.212549+0000","last_peered":"2024-09-15T22:44:19.212549+0000","last_clean":"2024-09-15T22:44:19.212549+0000","last_became_active":"2024-09-15T22:44:17.207807+0000","last_became_peered":"2024-09-15T22:44:17.207807+0000","last_unstale":"2024-09-15T22:44:19.212549+0000","last_undegraded":"2024-09-15T22:44:19.212549+0000","last_fullsized":"2024-09-15T22:44:19.212549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:19:59.752854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.196642+0000","last_change":"2024-09-15T22:44:17.201752+0000","last_active":"2024-09-15T22:44:19.196642+0000","last_peered":"2024-09-15T22:44:19.196642+0000","last_clean":"2024-09-15T22:44:19.196642+0000","last_became_active":"2024-09-15T22:44:17.201378+0000","last_became_peered":"2024-09-15T22:44:17.201378+0000","last_unstale":"2024-09-15T22:44:19.196642+0000","last_undegraded":"2024-09-15T22:44:19.196642+0000","last_fullsized":"2024-09-15T22:44:19.196642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:42:53.752795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.215941+0000","last_change":"2024-09-15T22:44:14.168024+0000","last_active":"2024-09-15T22:44:19.215941+0000","last_peered":"2024-09-15T22:44:19.215941+0000","last_clean":"2024-09-15T22:44:19.215941+0000","last_became_active":"2024-09-15T22:44:14.167639+0000","last_became_peered":"2024-09-15T22:44:14.167639+0000","last_unstale":"2024-09-15T22:44:19.215941+0000","last_undegraded":"2024-09-15T22:44:19.215941+0000","last_fullsized":"2024-09-15T22:44:19.215941+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:57.867895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.196294+0000","last_change":"2024-09-15T22:44:17.202731+0000","last_active":"2024-09-15T22:44:19.196294+0000","last_peered":"2024-09-15T22:44:19.196294+0000","last_clean":"2024-09-15T22:44:19.196294+0000","last_became_active":"2024-09-15T22:44:17.202410+0000","last_became_peered":"2024-09-15T22:44:17.202410+0000","last_unstale":"2024-09-15T22:44:19.196294+0000","last_undegraded":"2024-09-15T22:44:19.196294+0000","last_fullsized":"2024-09-15T22:44:19.196294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:34.613686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:29.554912+0000","last_change":"2024-09-15T22:44:14.166865+0000","last_active":"2024-09-15T22:44:29.554912+0000","last_peered":"2024-09-15T22:44:29.554912+0000","last_clean":"2024-09-15T22:44:29.554912+0000","last_became_active":"2024-09-15T22:44:14.166014+0000","last_became_peered":"2024-09-15T22:44:14.166014+0000","last_unstale":"2024-09-15T22:44:29.554912+0000","last_undegraded":"2024-09-15T22:44:29.554912+0000","last_fullsized":"2024-09-15T22:44:29.554912+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:44.527777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.213466+0000","last_change":"2024-09-15T22:44:17.221028+0000","last_active":"2024-09-15T22:44:19.213466+0000","last_peered":"2024-09-15T22:44:19.213466+0000","last_clean":"2024-09-15T22:44:19.213466+0000","last_became_active":"2024-09-15T22:44:17.220853+0000","last_became_peered":"2024-09-15T22:44:17.220853+0000","last_unstale":"2024-09-15T22:44:19.213466+0000","last_undegraded":"2024-09-15T22:44:19.213466+0000","last_fullsized":"2024-09-15T22:44:19.213466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:10:50.535946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.216457+0000","last_change":"2024-09-15T22:44:14.679506+0000","last_active":"2024-09-15T22:44:19.216457+0000","last_peered":"2024-09-15T22:44:19.216457+0000","last_clean":"2024-09-15T22:44:19.216457+0000","last_became_active":"2024-09-15T22:44:14.679164+0000","last_became_peered":"2024-09-15T22:44:14.679164+0000","last_unstale":"2024-09-15T22:44:19.216457+0000","last_undegraded":"2024-09-15T22:44:19.216457+0000","last_fullsized":"2024-09-15T22:44:19.216457+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:57.786804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.212650+0000","last_change":"2024-09-15T22:44:17.203867+0000","last_active":"2024-09-15T22:44:19.212650+0000","last_peered":"2024-09-15T22:44:19.212650+0000","last_clean":"2024-09-15T22:44:19.212650+0000","last_became_active":"2024-09-15T22:44:17.203450+0000","last_became_peered":"2024-09-15T22:44:17.203450+0000","last_unstale":"2024-09-15T22:44:19.212650+0000","last_undegraded":"2024-09-15T22:44:19.212650+0000","last_fullsized":"2024-09-15T22:44:19.212650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:17.210989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:29.555918+0000","last_change":"2024-09-15T22:44:14.164846+0000","last_active":"2024-09-15T22:44:29.555918+0000","last_peered":"2024-09-15T22:44:29.555918+0000","last_clean":"2024-09-15T22:44:29.555918+0000","last_became_active":"2024-09-15T22:44:14.164507+0000","last_became_peered":"2024-09-15T22:44:14.164507+0000","last_unstale":"2024-09-15T22:44:29.555918+0000","last_undegraded":"2024-09-15T22:44:29.555918+0000","last_fullsized":"2024-09-15T22:44:29.555918+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:55.234677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":57863,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":137,"num_read_kb":127,"num_write":155,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":385024,"data_stored":165075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133,"ondisk_log_size":133,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449108,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":454164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678041,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32768,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712384,"statfs":{"total":95995035648,"available":95961481216,"internally_reserved":0,"allocated":1622016,"data_stored":1099505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874271,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":454400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135907,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":454320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397546,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1544192,"data_stored":1046252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561070,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":937984,"data_stored":453789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790005,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1556480,"data_stored":1045936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953528,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":454226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182463,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":454100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":504928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":503185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":958464,"data_stored":455060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":49603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":51346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":55438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:44:51.990 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T22:44:52.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:51 smithi192 ceph-mon[35427]: pgmap v316: 97 pgs: 97 active+clean; 629 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2024-09-15T22:44:52.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:51 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-09-15T22:44:52.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:51 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-09-15T22:44:52.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:51 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-15T22:44:52.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:51 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-15T22:44:52.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:51 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-15T22:44:52.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:51 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-15T22:44:52.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:51 smithi192 ceph-mon[35427]: osdmap e87: 12 total, 12 up, 12 in 2024-09-15T22:44:52.357 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:52.758 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:52 smithi111 ceph-mon[30629]: from='client.24661 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:44:52.759 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:52 smithi111 ceph-mon[30629]: osdmap e88: 12 total, 12 up, 12 in 2024-09-15T22:44:53.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.045+0000 7f366cc00640 1 -- 172.21.15.111:0/4096212409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3668074590 msgr2=0x7f3668074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:53.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.045+0000 7f366cc00640 1 --2- 172.21.15.111:0/4096212409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3668074590 0x7f3668074970 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3658009860 tx=0x7f365802a3d0 comp rx=0 tx=0).stop 2024-09-15T22:44:53.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.045+0000 7f366cc00640 1 -- 172.21.15.111:0/4096212409 shutdown_connections 2024-09-15T22:44:53.047 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.045+0000 7f366cc00640 1 --2- 172.21.15.111:0/4096212409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3668112b90 0x7f3668114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:53.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.045+0000 7f366cc00640 1 --2- 172.21.15.111:0/4096212409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3668074eb0 0x7f3668112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:53.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.045+0000 7f366cc00640 1 --2- 172.21.15.111:0/4096212409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3668074590 0x7f3668074970 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:53.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.045+0000 7f366cc00640 1 -- 172.21.15.111:0/4096212409 >> 172.21.15.111:0/4096212409 conn(0x7f36680fd8c0 msgr2=0x7f36680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:53.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.045+0000 7f366cc00640 1 -- 172.21.15.111:0/4096212409 shutdown_connections 2024-09-15T22:44:53.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.045+0000 7f366cc00640 1 -- 172.21.15.111:0/4096212409 wait complete. 2024-09-15T22:44:53.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.046+0000 7f366cc00640 1 Processor -- start 2024-09-15T22:44:53.048 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.046+0000 7f366cc00640 1 -- start start 2024-09-15T22:44:53.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.046+0000 7f366cc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3668074590 0x7f36681a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:53.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.046+0000 7f366cc00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3668074eb0 0x7f36681a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:53.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.046+0000 7f366cc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3668112b90 0x7f366819e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:53.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.046+0000 7f366cc00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3668118550 con 0x7f3668074590 2024-09-15T22:44:53.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.046+0000 7f366cc00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f36681183d0 con 0x7f3668074eb0 2024-09-15T22:44:53.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.046+0000 7f366cc00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f36681186d0 con 0x7f3668112b90 2024-09-15T22:44:53.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.047+0000 7f3665e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3668074590 0x7f36681a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:53.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.047+0000 7f3665e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3668074590 0x7f36681a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56110/0 (socket says 172.21.15.111:56110) 2024-09-15T22:44:53.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.047+0000 7f3665e00640 1 -- 172.21.15.111:0/1750032226 learned_addr learned my addr 172.21.15.111:0/1750032226 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:44:53.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.047+0000 7f3666800640 1 --2- 172.21.15.111:0/1750032226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3668112b90 0x7f366819e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:53.050 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.047+0000 7f365fe00640 1 --2- 172.21.15.111:0/1750032226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3668074eb0 0x7f36681a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:53.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.047+0000 7f3666800640 1 -- 172.21.15.111:0/1750032226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3668074eb0 msgr2=0x7f36681a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:53.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.047+0000 7f3666800640 1 --2- 172.21.15.111:0/1750032226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3668074eb0 0x7f36681a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:53.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.047+0000 7f3666800640 1 -- 172.21.15.111:0/1750032226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3668074590 msgr2=0x7f36681a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:53.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.047+0000 7f3666800640 1 --2- 172.21.15.111:0/1750032226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3668074590 0x7f36681a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:53.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.047+0000 7f3666800640 1 -- 172.21.15.111:0/1750032226 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366819eff0 con 0x7f3668112b90 2024-09-15T22:44:53.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.047+0000 7f3666800640 1 --2- 172.21.15.111:0/1750032226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3668112b90 0x7f366819e8c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3660005fd0 tx=0x7f366000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:53.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.048+0000 7f365ea00640 1 -- 172.21.15.111:0/1750032226 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366000cdf0 con 0x7f3668112b90 2024-09-15T22:44:53.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.048+0000 7f366cc00640 1 -- 172.21.15.111:0/1750032226 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366819f2e0 con 0x7f3668112b90 2024-09-15T22:44:53.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.048+0000 7f365ea00640 1 -- 172.21.15.111:0/1750032226 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3660005020 con 0x7f3668112b90 2024-09-15T22:44:53.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.048+0000 7f365ea00640 1 -- 172.21.15.111:0/1750032226 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36600106d0 con 0x7f3668112b90 2024-09-15T22:44:53.051 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.048+0000 7f366cc00640 1 -- 172.21.15.111:0/1750032226 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36681ab5d0 con 0x7f3668112b90 2024-09-15T22:44:53.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.049+0000 7f366cc00640 1 -- 172.21.15.111:0/1750032226 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3668074970 con 0x7f3668112b90 2024-09-15T22:44:53.052 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.051+0000 7f365ea00640 1 -- 172.21.15.111:0/1750032226 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f36600054d0 con 0x7f3668112b90 2024-09-15T22:44:53.053 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.052+0000 7f365ea00640 1 --2- 172.21.15.111:0/1750032226 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f364c0a32d0 0x7f364c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:53.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.052+0000 7f365ea00640 1 -- 172.21.15.111:0/1750032226 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f36600d0280 con 0x7f3668112b90 2024-09-15T22:44:53.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.052+0000 7f3665e00640 1 --2- 172.21.15.111:0/1750032226 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f364c0a32d0 0x7f364c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:53.054 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.053+0000 7f3665e00640 1 --2- 172.21.15.111:0/1750032226 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f364c0a32d0 0x7f364c0a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f36580095f0 tx=0x7f36580058b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:53.055 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.054+0000 7f365ea00640 1 -- 172.21.15.111:0/1750032226 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f366009ce40 con 0x7f3668112b90 2024-09-15T22:44:53.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:52 smithi192 ceph-mon[35427]: from='client.24661 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:44:53.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:52 smithi192 ceph-mon[35427]: osdmap e88: 12 total, 12 up, 12 in 2024-09-15T22:44:53.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:52 smithi121 ceph-mon[34815]: from='client.24661 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:44:53.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:52 smithi121 ceph-mon[34815]: osdmap e88: 12 total, 12 up, 12 in 2024-09-15T22:44:53.228 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.227+0000 7f366cc00640 1 -- 172.21.15.111:0/1750032226 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3668062630 con 0x7f364c0a32d0 2024-09-15T22:44:53.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.234+0000 7f365ea00640 1 -- 172.21.15.111:0/1750032226 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258705 (secure 0 0 0) 0x7f3668062630 con 0x7f364c0a32d0 2024-09-15T22:44:53.238 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:44:53.239 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-09-15T22:44:53.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.238+0000 7f366cc00640 1 -- 172.21.15.111:0/1750032226 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f364c0a32d0 msgr2=0x7f364c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:53.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.238+0000 7f366cc00640 1 --2- 172.21.15.111:0/1750032226 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f364c0a32d0 0x7f364c0a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f36580095f0 tx=0x7f36580058b0 comp rx=0 tx=0).stop 2024-09-15T22:44:53.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.238+0000 7f366cc00640 1 -- 172.21.15.111:0/1750032226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3668112b90 msgr2=0x7f366819e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:53.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.238+0000 7f366cc00640 1 --2- 172.21.15.111:0/1750032226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3668112b90 0x7f366819e8c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3660005fd0 tx=0x7f366000ef10 comp rx=0 tx=0).stop 2024-09-15T22:44:53.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.238+0000 7f366cc00640 1 -- 172.21.15.111:0/1750032226 shutdown_connections 2024-09-15T22:44:53.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.238+0000 7f366cc00640 1 --2- 172.21.15.111:0/1750032226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3668112b90 0x7f366819e8c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:53.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.238+0000 7f366cc00640 1 --2- 172.21.15.111:0/1750032226 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f364c0a32d0 0x7f364c0a5790 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:53.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.238+0000 7f366cc00640 1 --2- 172.21.15.111:0/1750032226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3668074eb0 0x7f36681a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:53.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.238+0000 7f366cc00640 1 --2- 172.21.15.111:0/1750032226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3668074590 0x7f36681a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:53.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.238+0000 7f366cc00640 1 -- 172.21.15.111:0/1750032226 >> 172.21.15.111:0/1750032226 conn(0x7f36680fd8c0 msgr2=0x7f36680ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:53.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.238+0000 7f366cc00640 1 -- 172.21.15.111:0/1750032226 shutdown_connections 2024-09-15T22:44:53.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:53.238+0000 7f366cc00640 1 -- 172.21.15.111:0/1750032226 wait complete. 2024-09-15T22:44:53.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:53 smithi111 ceph-mon[30629]: pgmap v319: 97 pgs: 97 active+clean; 633 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-09-15T22:44:54.029 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-09-15T22:44:52.460582+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":648465,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":263,"num_read_kb":236,"num_write":342,"num_write_kb":3042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374800,"kb_used_data":13064,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567024,"statfs":{"total":1151940427776,"available":1151556632576,"internally_reserved":0,"allocated":13377536,"data_stored":7401176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27030,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":53,"num_read_kb":58,"num_write":19,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001957"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.217652+0000","last_change":"2024-09-15T22:44:17.222326+0000","last_active":"2024-09-15T22:44:19.217652+0000","last_peered":"2024-09-15T22:44:19.217652+0000","last_clean":"2024-09-15T22:44:19.217652+0000","last_became_active":"2024-09-15T22:44:17.221903+0000","last_became_peered":"2024-09-15T22:44:17.221903+0000","last_unstale":"2024-09-15T22:44:19.217652+0000","last_undegraded":"2024-09-15T22:44:19.217652+0000","last_fullsized":"2024-09-15T22:44:19.217652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:28.303194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:44:50.631629+0000","last_change":"2024-09-15T22:44:17.220515+0000","last_active":"2024-09-15T22:44:50.631629+0000","last_peered":"2024-09-15T22:44:50.631629+0000","last_clean":"2024-09-15T22:44:50.631629+0000","last_became_active":"2024-09-15T22:44:17.218794+0000","last_became_peered":"2024-09-15T22:44:17.218794+0000","last_unstale":"2024-09-15T22:44:50.631629+0000","last_undegraded":"2024-09-15T22:44:50.631629+0000","last_fullsized":"2024-09-15T22:44:50.631629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:58:17.892809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:44:50.631627+0000","last_change":"2024-09-15T22:44:17.213183+0000","last_active":"2024-09-15T22:44:50.631627+0000","last_peered":"2024-09-15T22:44:50.631627+0000","last_clean":"2024-09-15T22:44:50.631627+0000","last_became_active":"2024-09-15T22:44:17.212888+0000","last_became_peered":"2024-09-15T22:44:17.212888+0000","last_unstale":"2024-09-15T22:44:50.631627+0000","last_undegraded":"2024-09-15T22:44:50.631627+0000","last_fullsized":"2024-09-15T22:44:50.631627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:32.224099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.214564+0000","last_change":"2024-09-15T22:44:17.203869+0000","last_active":"2024-09-15T22:44:19.214564+0000","last_peered":"2024-09-15T22:44:19.214564+0000","last_clean":"2024-09-15T22:44:19.214564+0000","last_became_active":"2024-09-15T22:44:17.203529+0000","last_became_peered":"2024-09-15T22:44:17.203529+0000","last_unstale":"2024-09-15T22:44:19.214564+0000","last_undegraded":"2024-09-15T22:44:19.214564+0000","last_fullsized":"2024-09-15T22:44:19.214564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:22:58.539513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.210896+0000","last_change":"2024-09-15T22:44:17.209521+0000","last_active":"2024-09-15T22:44:19.210896+0000","last_peered":"2024-09-15T22:44:19.210896+0000","last_clean":"2024-09-15T22:44:19.210896+0000","last_became_active":"2024-09-15T22:44:17.209203+0000","last_became_peered":"2024-09-15T22:44:17.209203+0000","last_unstale":"2024-09-15T22:44:19.210896+0000","last_undegraded":"2024-09-15T22:44:19.210896+0000","last_fullsized":"2024-09-15T22:44:19.210896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:34:28.418483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.210130+0000","last_change":"2024-09-15T22:44:17.202857+0000","last_active":"2024-09-15T22:44:19.210130+0000","last_peered":"2024-09-15T22:44:19.210130+0000","last_clean":"2024-09-15T22:44:19.210130+0000","last_became_active":"2024-09-15T22:44:17.202530+0000","last_became_peered":"2024-09-15T22:44:17.202530+0000","last_unstale":"2024-09-15T22:44:19.210130+0000","last_undegraded":"2024-09-15T22:44:19.210130+0000","last_fullsized":"2024-09-15T22:44:19.210130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:16:06.947769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.210951+0000","last_change":"2024-09-15T22:44:17.203317+0000","last_active":"2024-09-15T22:44:19.210951+0000","last_peered":"2024-09-15T22:44:19.210951+0000","last_clean":"2024-09-15T22:44:19.210951+0000","last_became_active":"2024-09-15T22:44:17.202975+0000","last_became_peered":"2024-09-15T22:44:17.202975+0000","last_unstale":"2024-09-15T22:44:19.210951+0000","last_undegraded":"2024-09-15T22:44:19.210951+0000","last_fullsized":"2024-09-15T22:44:19.210951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:51.292052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.211311+0000","last_change":"2024-09-15T22:44:17.203583+0000","last_active":"2024-09-15T22:44:19.211311+0000","last_peered":"2024-09-15T22:44:19.211311+0000","last_clean":"2024-09-15T22:44:19.211311+0000","last_became_active":"2024-09-15T22:44:17.203251+0000","last_became_peered":"2024-09-15T22:44:17.203251+0000","last_unstale":"2024-09-15T22:44:19.211311+0000","last_undegraded":"2024-09-15T22:44:19.211311+0000","last_fullsized":"2024-09-15T22:44:19.211311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:53:17.378024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.196788+0000","last_change":"2024-09-15T22:44:17.209643+0000","last_active":"2024-09-15T22:44:19.196788+0000","last_peered":"2024-09-15T22:44:19.196788+0000","last_clean":"2024-09-15T22:44:19.196788+0000","last_became_active":"2024-09-15T22:44:17.208709+0000","last_became_peered":"2024-09-15T22:44:17.208709+0000","last_unstale":"2024-09-15T22:44:19.196788+0000","last_undegraded":"2024-09-15T22:44:19.196788+0000","last_fullsized":"2024-09-15T22:44:19.196788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:38:38.305676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.212031+0000","last_change":"2024-09-15T22:44:17.201830+0000","last_active":"2024-09-15T22:44:19.212031+0000","last_peered":"2024-09-15T22:44:19.212031+0000","last_clean":"2024-09-15T22:44:19.212031+0000","last_became_active":"2024-09-15T22:44:17.200844+0000","last_became_peered":"2024-09-15T22:44:17.200844+0000","last_unstale":"2024-09-15T22:44:19.212031+0000","last_undegraded":"2024-09-15T22:44:19.212031+0000","last_fullsized":"2024-09-15T22:44:19.212031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:38:53.681454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.212758+0000","last_change":"2024-09-15T22:44:17.208106+0000","last_active":"2024-09-15T22:44:19.212758+0000","last_peered":"2024-09-15T22:44:19.212758+0000","last_clean":"2024-09-15T22:44:19.212758+0000","last_became_active":"2024-09-15T22:44:17.207774+0000","last_became_peered":"2024-09-15T22:44:17.207774+0000","last_unstale":"2024-09-15T22:44:19.212758+0000","last_undegraded":"2024-09-15T22:44:19.212758+0000","last_fullsized":"2024-09-15T22:44:19.212758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:05:15.876692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.214745+0000","last_change":"2024-09-15T22:44:17.202857+0000","last_active":"2024-09-15T22:44:19.214745+0000","last_peered":"2024-09-15T22:44:19.214745+0000","last_clean":"2024-09-15T22:44:19.214745+0000","last_became_active":"2024-09-15T22:44:17.202520+0000","last_became_peered":"2024-09-15T22:44:17.202520+0000","last_unstale":"2024-09-15T22:44:19.214745+0000","last_undegraded":"2024-09-15T22:44:19.214745+0000","last_fullsized":"2024-09-15T22:44:19.214745+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:11:19.111086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.213091+0000","last_change":"2024-09-15T22:44:17.201776+0000","last_active":"2024-09-15T22:44:19.213091+0000","last_peered":"2024-09-15T22:44:19.213091+0000","last_clean":"2024-09-15T22:44:19.213091+0000","last_became_active":"2024-09-15T22:44:17.201508+0000","last_became_peered":"2024-09-15T22:44:17.201508+0000","last_unstale":"2024-09-15T22:44:19.213091+0000","last_undegraded":"2024-09-15T22:44:19.213091+0000","last_fullsized":"2024-09-15T22:44:19.213091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:22:48.104776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:44:50.631430+0000","last_change":"2024-09-15T22:44:17.221005+0000","last_active":"2024-09-15T22:44:50.631430+0000","last_peered":"2024-09-15T22:44:50.631430+0000","last_clean":"2024-09-15T22:44:50.631430+0000","last_became_active":"2024-09-15T22:44:17.218455+0000","last_became_peered":"2024-09-15T22:44:17.218455+0000","last_unstale":"2024-09-15T22:44:50.631430+0000","last_undegraded":"2024-09-15T22:44:50.631430+0000","last_fullsized":"2024-09-15T22:44:50.631430+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:40.227843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:44:50.631492+0000","last_change":"2024-09-15T22:44:17.208125+0000","last_active":"2024-09-15T22:44:50.631492+0000","last_peered":"2024-09-15T22:44:50.631492+0000","last_clean":"2024-09-15T22:44:50.631492+0000","last_became_active":"2024-09-15T22:44:17.207762+0000","last_became_peered":"2024-09-15T22:44:17.207762+0000","last_unstale":"2024-09-15T22:44:50.631492+0000","last_undegraded":"2024-09-15T22:44:50.631492+0000","last_fullsized":"2024-09-15T22:44:50.631492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:21:45.181921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.196130+0000","last_change":"2024-09-15T22:44:17.212058+0000","last_active":"2024-09-15T22:44:19.196130+0000","last_peered":"2024-09-15T22:44:19.196130+0000","last_clean":"2024-09-15T22:44:19.196130+0000","last_became_active":"2024-09-15T22:44:17.211789+0000","last_became_peered":"2024-09-15T22:44:17.211789+0000","last_unstale":"2024-09-15T22:44:19.196130+0000","last_undegraded":"2024-09-15T22:44:19.196130+0000","last_fullsized":"2024-09-15T22:44:19.196130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:14.647374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.217987+0000","last_change":"2024-09-15T22:44:17.225139+0000","last_active":"2024-09-15T22:44:19.217987+0000","last_peered":"2024-09-15T22:44:19.217987+0000","last_clean":"2024-09-15T22:44:19.217987+0000","last_became_active":"2024-09-15T22:44:17.224791+0000","last_became_peered":"2024-09-15T22:44:17.224791+0000","last_unstale":"2024-09-15T22:44:19.217987+0000","last_undegraded":"2024-09-15T22:44:19.217987+0000","last_fullsized":"2024-09-15T22:44:19.217987+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:56.457155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:44:50.635615+0000","last_change":"2024-09-15T22:44:17.208604+0000","last_active":"2024-09-15T22:44:50.635615+0000","last_peered":"2024-09-15T22:44:50.635615+0000","last_clean":"2024-09-15T22:44:50.635615+0000","last_became_active":"2024-09-15T22:44:17.202231+0000","last_became_peered":"2024-09-15T22:44:17.202231+0000","last_unstale":"2024-09-15T22:44:50.635615+0000","last_undegraded":"2024-09-15T22:44:50.635615+0000","last_fullsized":"2024-09-15T22:44:50.635615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:39.572881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635576+0000","last_change":"2024-09-15T22:44:17.211837+0000","last_active":"2024-09-15T22:44:51.635576+0000","last_peered":"2024-09-15T22:44:51.635576+0000","last_clean":"2024-09-15T22:44:51.635576+0000","last_became_active":"2024-09-15T22:44:17.211460+0000","last_became_peered":"2024-09-15T22:44:17.211460+0000","last_unstale":"2024-09-15T22:44:51.635576+0000","last_undegraded":"2024-09-15T22:44:51.635576+0000","last_fullsized":"2024-09-15T22:44:51.635576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:21:51.820134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.217414+0000","last_change":"2024-09-15T22:44:17.206350+0000","last_active":"2024-09-15T22:44:19.217414+0000","last_peered":"2024-09-15T22:44:19.217414+0000","last_clean":"2024-09-15T22:44:19.217414+0000","last_became_active":"2024-09-15T22:44:17.204943+0000","last_became_peered":"2024-09-15T22:44:17.204943+0000","last_unstale":"2024-09-15T22:44:19.217414+0000","last_undegraded":"2024-09-15T22:44:19.217414+0000","last_fullsized":"2024-09-15T22:44:19.217414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:42:59.918220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.213222+0000","last_change":"2024-09-15T22:44:17.223390+0000","last_active":"2024-09-15T22:44:19.213222+0000","last_peered":"2024-09-15T22:44:19.213222+0000","last_clean":"2024-09-15T22:44:19.213222+0000","last_became_active":"2024-09-15T22:44:17.223008+0000","last_became_peered":"2024-09-15T22:44:17.223008+0000","last_unstale":"2024-09-15T22:44:19.213222+0000","last_undegraded":"2024-09-15T22:44:19.213222+0000","last_fullsized":"2024-09-15T22:44:19.213222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:31.545336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.196491+0000","last_change":"2024-09-15T22:44:17.207567+0000","last_active":"2024-09-15T22:44:19.196491+0000","last_peered":"2024-09-15T22:44:19.196491+0000","last_clean":"2024-09-15T22:44:19.196491+0000","last_became_active":"2024-09-15T22:44:17.207245+0000","last_became_peered":"2024-09-15T22:44:17.207245+0000","last_unstale":"2024-09-15T22:44:19.196491+0000","last_undegraded":"2024-09-15T22:44:19.196491+0000","last_fullsized":"2024-09-15T22:44:19.196491+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:35.169205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.211896+0000","last_change":"2024-09-15T22:44:17.209449+0000","last_active":"2024-09-15T22:44:19.211896+0000","last_peered":"2024-09-15T22:44:19.211896+0000","last_clean":"2024-09-15T22:44:19.211896+0000","last_became_active":"2024-09-15T22:44:17.209012+0000","last_became_peered":"2024-09-15T22:44:17.209012+0000","last_unstale":"2024-09-15T22:44:19.211896+0000","last_undegraded":"2024-09-15T22:44:19.211896+0000","last_fullsized":"2024-09-15T22:44:19.211896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:59.550776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.196130+0000","last_change":"2024-09-15T22:44:17.214184+0000","last_active":"2024-09-15T22:44:19.196130+0000","last_peered":"2024-09-15T22:44:19.196130+0000","last_clean":"2024-09-15T22:44:19.196130+0000","last_became_active":"2024-09-15T22:44:17.213884+0000","last_became_peered":"2024-09-15T22:44:17.213884+0000","last_unstale":"2024-09-15T22:44:19.196130+0000","last_undegraded":"2024-09-15T22:44:19.196130+0000","last_fullsized":"2024-09-15T22:44:19.196130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:33.336473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.213894+0000","last_change":"2024-09-15T22:44:17.203904+0000","last_active":"2024-09-15T22:44:19.213894+0000","last_peered":"2024-09-15T22:44:19.213894+0000","last_clean":"2024-09-15T22:44:19.213894+0000","last_became_active":"2024-09-15T22:44:17.203484+0000","last_became_peered":"2024-09-15T22:44:17.203484+0000","last_unstale":"2024-09-15T22:44:19.213894+0000","last_undegraded":"2024-09-15T22:44:19.213894+0000","last_fullsized":"2024-09-15T22:44:19.213894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:33:09.906869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.636131+0000","last_change":"2024-09-15T22:44:17.209356+0000","last_active":"2024-09-15T22:44:51.636131+0000","last_peered":"2024-09-15T22:44:51.636131+0000","last_clean":"2024-09-15T22:44:51.636131+0000","last_became_active":"2024-09-15T22:44:17.207229+0000","last_became_peered":"2024-09-15T22:44:17.207229+0000","last_unstale":"2024-09-15T22:44:51.636131+0000","last_undegraded":"2024-09-15T22:44:51.636131+0000","last_fullsized":"2024-09-15T22:44:51.636131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:32:28.405429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.217516+0000","last_change":"2024-09-15T22:44:17.202505+0000","last_active":"2024-09-15T22:44:19.217516+0000","last_peered":"2024-09-15T22:44:19.217516+0000","last_clean":"2024-09-15T22:44:19.217516+0000","last_became_active":"2024-09-15T22:44:17.202154+0000","last_became_peered":"2024-09-15T22:44:17.202154+0000","last_unstale":"2024-09-15T22:44:19.217516+0000","last_undegraded":"2024-09-15T22:44:19.217516+0000","last_fullsized":"2024-09-15T22:44:19.217516+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:40:05.680982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.212271+0000","last_change":"2024-09-15T22:44:17.222767+0000","last_active":"2024-09-15T22:44:19.212271+0000","last_peered":"2024-09-15T22:44:19.212271+0000","last_clean":"2024-09-15T22:44:19.212271+0000","last_became_active":"2024-09-15T22:44:17.221655+0000","last_became_peered":"2024-09-15T22:44:17.221655+0000","last_unstale":"2024-09-15T22:44:19.212271+0000","last_undegraded":"2024-09-15T22:44:19.212271+0000","last_fullsized":"2024-09-15T22:44:19.212271+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:22.457576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:44:50.631615+0000","last_change":"2024-09-15T22:44:17.217485+0000","last_active":"2024-09-15T22:44:50.631615+0000","last_peered":"2024-09-15T22:44:50.631615+0000","last_clean":"2024-09-15T22:44:50.631615+0000","last_became_active":"2024-09-15T22:44:17.217138+0000","last_became_peered":"2024-09-15T22:44:17.217138+0000","last_unstale":"2024-09-15T22:44:50.631615+0000","last_undegraded":"2024-09-15T22:44:50.631615+0000","last_fullsized":"2024-09-15T22:44:50.631615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:41:40.606593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635434+0000","last_change":"2024-09-15T22:44:17.217996+0000","last_active":"2024-09-15T22:44:51.635434+0000","last_peered":"2024-09-15T22:44:51.635434+0000","last_clean":"2024-09-15T22:44:51.635434+0000","last_became_active":"2024-09-15T22:44:17.217467+0000","last_became_peered":"2024-09-15T22:44:17.217467+0000","last_unstale":"2024-09-15T22:44:51.635434+0000","last_undegraded":"2024-09-15T22:44:51.635434+0000","last_fullsized":"2024-09-15T22:44:51.635434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:25:45.150979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.636279+0000","last_change":"2024-09-15T22:44:17.202840+0000","last_active":"2024-09-15T22:44:51.636279+0000","last_peered":"2024-09-15T22:44:51.636279+0000","last_clean":"2024-09-15T22:44:51.636279+0000","last_became_active":"2024-09-15T22:44:17.202481+0000","last_became_peered":"2024-09-15T22:44:17.202481+0000","last_unstale":"2024-09-15T22:44:51.636279+0000","last_undegraded":"2024-09-15T22:44:51.636279+0000","last_fullsized":"2024-09-15T22:44:51.636279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:31:09.360211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.217492+0000","last_change":"2024-09-15T22:44:17.209387+0000","last_active":"2024-09-15T22:44:19.217492+0000","last_peered":"2024-09-15T22:44:19.217492+0000","last_clean":"2024-09-15T22:44:19.217492+0000","last_became_active":"2024-09-15T22:44:17.209100+0000","last_became_peered":"2024-09-15T22:44:17.209100+0000","last_unstale":"2024-09-15T22:44:19.217492+0000","last_undegraded":"2024-09-15T22:44:19.217492+0000","last_fullsized":"2024-09-15T22:44:19.217492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:57:47.942784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.211418+0000","last_change":"2024-09-15T22:44:17.203626+0000","last_active":"2024-09-15T22:44:19.211418+0000","last_peered":"2024-09-15T22:44:19.211418+0000","last_clean":"2024-09-15T22:44:19.211418+0000","last_became_active":"2024-09-15T22:44:17.203313+0000","last_became_peered":"2024-09-15T22:44:17.203313+0000","last_unstale":"2024-09-15T22:44:19.211418+0000","last_undegraded":"2024-09-15T22:44:19.211418+0000","last_fullsized":"2024-09-15T22:44:19.211418+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:42:53.765397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:47.596337+0000","last_change":"2024-09-15T22:44:14.172460+0000","last_active":"2024-09-15T22:44:47.596337+0000","last_peered":"2024-09-15T22:44:47.596337+0000","last_clean":"2024-09-15T22:44:47.596337+0000","last_became_active":"2024-09-15T22:44:14.171960+0000","last_became_peered":"2024-09-15T22:44:14.171960+0000","last_unstale":"2024-09-15T22:44:47.596337+0000","last_undegraded":"2024-09-15T22:44:47.596337+0000","last_fullsized":"2024-09-15T22:44:47.596337+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:27:55.116610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.196495+0000","last_change":"2024-09-15T22:44:17.210054+0000","last_active":"2024-09-15T22:44:19.196495+0000","last_peered":"2024-09-15T22:44:19.196495+0000","last_clean":"2024-09-15T22:44:19.196495+0000","last_became_active":"2024-09-15T22:44:17.209607+0000","last_became_peered":"2024-09-15T22:44:17.209607+0000","last_unstale":"2024-09-15T22:44:19.196495+0000","last_undegraded":"2024-09-15T22:44:19.196495+0000","last_fullsized":"2024-09-15T22:44:19.196495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:13.867965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:29.555182+0000","last_change":"2024-09-15T22:44:14.167537+0000","last_active":"2024-09-15T22:44:29.555182+0000","last_peered":"2024-09-15T22:44:29.555182+0000","last_clean":"2024-09-15T22:44:29.555182+0000","last_became_active":"2024-09-15T22:44:14.167173+0000","last_became_peered":"2024-09-15T22:44:14.167173+0000","last_unstale":"2024-09-15T22:44:29.555182+0000","last_undegraded":"2024-09-15T22:44:29.555182+0000","last_fullsized":"2024-09-15T22:44:29.555182+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:45:56.539218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:29.554959+0000","last_change":"2024-09-15T22:44:14.166676+0000","last_active":"2024-09-15T22:44:29.554959+0000","last_peered":"2024-09-15T22:44:29.554959+0000","last_clean":"2024-09-15T22:44:29.554959+0000","last_became_active":"2024-09-15T22:44:14.166342+0000","last_became_peered":"2024-09-15T22:44:14.166342+0000","last_unstale":"2024-09-15T22:44:29.554959+0000","last_undegraded":"2024-09-15T22:44:29.554959+0000","last_fullsized":"2024-09-15T22:44:29.554959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:41.861253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.211425+0000","last_change":"2024-09-15T22:44:17.211312+0000","last_active":"2024-09-15T22:44:19.211425+0000","last_peered":"2024-09-15T22:44:19.211425+0000","last_clean":"2024-09-15T22:44:19.211425+0000","last_became_active":"2024-09-15T22:44:17.208771+0000","last_became_peered":"2024-09-15T22:44:17.208771+0000","last_unstale":"2024-09-15T22:44:19.211425+0000","last_undegraded":"2024-09-15T22:44:19.211425+0000","last_fullsized":"2024-09-15T22:44:19.211425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:59:51.250489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.196460+0000","last_change":"2024-09-15T22:44:14.167263+0000","last_active":"2024-09-15T22:44:19.196460+0000","last_peered":"2024-09-15T22:44:19.196460+0000","last_clean":"2024-09-15T22:44:19.196460+0000","last_became_active":"2024-09-15T22:44:14.166942+0000","last_became_peered":"2024-09-15T22:44:14.166942+0000","last_unstale":"2024-09-15T22:44:19.196460+0000","last_undegraded":"2024-09-15T22:44:19.196460+0000","last_fullsized":"2024-09-15T22:44:19.196460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:47.320374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:44:50.635227+0000","last_change":"2024-09-15T22:44:17.210986+0000","last_active":"2024-09-15T22:44:50.635227+0000","last_peered":"2024-09-15T22:44:50.635227+0000","last_clean":"2024-09-15T22:44:50.635227+0000","last_became_active":"2024-09-15T22:44:17.210618+0000","last_became_peered":"2024-09-15T22:44:17.210618+0000","last_unstale":"2024-09-15T22:44:50.635227+0000","last_undegraded":"2024-09-15T22:44:50.635227+0000","last_fullsized":"2024-09-15T22:44:50.635227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:58:04.961630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:48.597053+0000","last_change":"2024-09-15T22:44:14.164770+0000","last_active":"2024-09-15T22:44:48.597053+0000","last_peered":"2024-09-15T22:44:48.597053+0000","last_clean":"2024-09-15T22:44:48.597053+0000","last_became_active":"2024-09-15T22:44:14.164411+0000","last_became_peered":"2024-09-15T22:44:14.164411+0000","last_unstale":"2024-09-15T22:44:48.597053+0000","last_undegraded":"2024-09-15T22:44:48.597053+0000","last_fullsized":"2024-09-15T22:44:48.597053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:26:47.623520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.196692+0000","last_change":"2024-09-15T22:44:17.204049+0000","last_active":"2024-09-15T22:44:19.196692+0000","last_peered":"2024-09-15T22:44:19.196692+0000","last_clean":"2024-09-15T22:44:19.196692+0000","last_became_active":"2024-09-15T22:44:17.203738+0000","last_became_peered":"2024-09-15T22:44:17.203738+0000","last_unstale":"2024-09-15T22:44:19.196692+0000","last_undegraded":"2024-09-15T22:44:19.196692+0000","last_fullsized":"2024-09-15T22:44:19.196692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:56.347792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:28.544197+0000","last_change":"2024-09-15T22:44:14.167228+0000","last_active":"2024-09-15T22:44:28.544197+0000","last_peered":"2024-09-15T22:44:28.544197+0000","last_clean":"2024-09-15T22:44:28.544197+0000","last_became_active":"2024-09-15T22:44:14.166662+0000","last_became_peered":"2024-09-15T22:44:14.166662+0000","last_unstale":"2024-09-15T22:44:28.544197+0000","last_undegraded":"2024-09-15T22:44:28.544197+0000","last_fullsized":"2024-09-15T22:44:28.544197+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:31:17.160718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:44:50.632153+0000","last_change":"2024-09-15T22:44:17.212549+0000","last_active":"2024-09-15T22:44:50.632153+0000","last_peered":"2024-09-15T22:44:50.632153+0000","last_clean":"2024-09-15T22:44:50.632153+0000","last_became_active":"2024-09-15T22:44:17.212123+0000","last_became_peered":"2024-09-15T22:44:17.212123+0000","last_unstale":"2024-09-15T22:44:50.632153+0000","last_undegraded":"2024-09-15T22:44:50.632153+0000","last_fullsized":"2024-09-15T22:44:50.632153+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:32:45.241199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:28.544533+0000","last_change":"2024-09-15T22:44:14.679664+0000","last_active":"2024-09-15T22:44:28.544533+0000","last_peered":"2024-09-15T22:44:28.544533+0000","last_clean":"2024-09-15T22:44:28.544533+0000","last_became_active":"2024-09-15T22:44:14.679256+0000","last_became_peered":"2024-09-15T22:44:14.679256+0000","last_unstale":"2024-09-15T22:44:28.544533+0000","last_undegraded":"2024-09-15T22:44:28.544533+0000","last_fullsized":"2024-09-15T22:44:28.544533+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:33:41.512488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.196305+0000","last_change":"2024-09-15T22:44:17.200448+0000","last_active":"2024-09-15T22:44:19.196305+0000","last_peered":"2024-09-15T22:44:19.196305+0000","last_clean":"2024-09-15T22:44:19.196305+0000","last_became_active":"2024-09-15T22:44:17.200114+0000","last_became_peered":"2024-09-15T22:44:17.200114+0000","last_unstale":"2024-09-15T22:44:19.196305+0000","last_undegraded":"2024-09-15T22:44:19.196305+0000","last_fullsized":"2024-09-15T22:44:19.196305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:36.403644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'26","reported_seq":93,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:49.599637+0000","last_change":"2024-09-15T22:44:14.167067+0000","last_active":"2024-09-15T22:44:49.599637+0000","last_peered":"2024-09-15T22:44:49.599637+0000","last_clean":"2024-09-15T22:44:49.599637+0000","last_became_active":"2024-09-15T22:44:14.166613+0000","last_became_peered":"2024-09-15T22:44:14.166613+0000","last_unstale":"2024-09-15T22:44:49.599637+0000","last_undegraded":"2024-09-15T22:44:49.599637+0000","last_fullsized":"2024-09-15T22:44:49.599637+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:15.360717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53860,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":70,"num_write":27,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.211865+0000","last_change":"2024-09-15T22:44:17.202705+0000","last_active":"2024-09-15T22:44:19.211865+0000","last_peered":"2024-09-15T22:44:19.211865+0000","last_clean":"2024-09-15T22:44:19.211865+0000","last_became_active":"2024-09-15T22:44:17.202346+0000","last_became_peered":"2024-09-15T22:44:17.202346+0000","last_unstale":"2024-09-15T22:44:19.211865+0000","last_undegraded":"2024-09-15T22:44:19.211865+0000","last_fullsized":"2024-09-15T22:44:19.211865+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:31.193746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.637052+0000","last_change":"2024-09-15T22:44:14.169255+0000","last_active":"2024-09-15T22:44:51.637052+0000","last_peered":"2024-09-15T22:44:51.637052+0000","last_clean":"2024-09-15T22:44:51.637052+0000","last_became_active":"2024-09-15T22:44:14.168774+0000","last_became_peered":"2024-09-15T22:44:14.168774+0000","last_unstale":"2024-09-15T22:44:51.637052+0000","last_undegraded":"2024-09-15T22:44:51.637052+0000","last_fullsized":"2024-09-15T22:44:51.637052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:48.858831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.210812+0000","last_change":"2024-09-15T22:44:17.218635+0000","last_active":"2024-09-15T22:44:19.210812+0000","last_peered":"2024-09-15T22:44:19.210812+0000","last_clean":"2024-09-15T22:44:19.210812+0000","last_became_active":"2024-09-15T22:44:17.218287+0000","last_became_peered":"2024-09-15T22:44:17.218287+0000","last_unstale":"2024-09-15T22:44:19.210812+0000","last_undegraded":"2024-09-15T22:44:19.210812+0000","last_fullsized":"2024-09-15T22:44:19.210812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:58.264382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:28.544054+0000","last_change":"2024-09-15T22:44:14.166580+0000","last_active":"2024-09-15T22:44:28.544054+0000","last_peered":"2024-09-15T22:44:28.544054+0000","last_clean":"2024-09-15T22:44:28.544054+0000","last_became_active":"2024-09-15T22:44:14.164955+0000","last_became_peered":"2024-09-15T22:44:14.164955+0000","last_unstale":"2024-09-15T22:44:28.544054+0000","last_undegraded":"2024-09-15T22:44:28.544054+0000","last_fullsized":"2024-09-15T22:44:28.544054+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:14.215662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.211342+0000","last_change":"2024-09-15T22:44:17.205731+0000","last_active":"2024-09-15T22:44:19.211342+0000","last_peered":"2024-09-15T22:44:19.211342+0000","last_clean":"2024-09-15T22:44:19.211342+0000","last_became_active":"2024-09-15T22:44:17.205361+0000","last_became_peered":"2024-09-15T22:44:17.205361+0000","last_unstale":"2024-09-15T22:44:19.211342+0000","last_undegraded":"2024-09-15T22:44:19.211342+0000","last_fullsized":"2024-09-15T22:44:19.211342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:25.430166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:46.594288+0000","last_change":"2024-09-15T22:44:14.166471+0000","last_active":"2024-09-15T22:44:46.594288+0000","last_peered":"2024-09-15T22:44:46.594288+0000","last_clean":"2024-09-15T22:44:46.594288+0000","last_became_active":"2024-09-15T22:44:14.165969+0000","last_became_peered":"2024-09-15T22:44:14.165969+0000","last_unstale":"2024-09-15T22:44:46.594288+0000","last_undegraded":"2024-09-15T22:44:46.594288+0000","last_fullsized":"2024-09-15T22:44:46.594288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:57:40.108575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.211070+0000","last_change":"2024-09-15T22:44:17.201488+0000","last_active":"2024-09-15T22:44:19.211070+0000","last_peered":"2024-09-15T22:44:19.211070+0000","last_clean":"2024-09-15T22:44:19.211070+0000","last_became_active":"2024-09-15T22:44:17.201101+0000","last_became_peered":"2024-09-15T22:44:17.201101+0000","last_unstale":"2024-09-15T22:44:19.211070+0000","last_undegraded":"2024-09-15T22:44:19.211070+0000","last_fullsized":"2024-09-15T22:44:19.211070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:41.758433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:29.555225+0000","last_change":"2024-09-15T22:44:14.680734+0000","last_active":"2024-09-15T22:44:29.555225+0000","last_peered":"2024-09-15T22:44:29.555225+0000","last_clean":"2024-09-15T22:44:29.555225+0000","last_became_active":"2024-09-15T22:44:14.680362+0000","last_became_peered":"2024-09-15T22:44:14.680362+0000","last_unstale":"2024-09-15T22:44:29.555225+0000","last_undegraded":"2024-09-15T22:44:29.555225+0000","last_fullsized":"2024-09-15T22:44:29.555225+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:22:37.086300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.196218+0000","last_change":"2024-09-15T22:44:17.199944+0000","last_active":"2024-09-15T22:44:19.196218+0000","last_peered":"2024-09-15T22:44:19.196218+0000","last_clean":"2024-09-15T22:44:19.196218+0000","last_became_active":"2024-09-15T22:44:17.199586+0000","last_became_peered":"2024-09-15T22:44:17.199586+0000","last_unstale":"2024-09-15T22:44:19.196218+0000","last_undegraded":"2024-09-15T22:44:19.196218+0000","last_fullsized":"2024-09-15T22:44:19.196218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:42:22.178939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:31.549790+0000","last_change":"2024-09-15T22:44:14.169077+0000","last_active":"2024-09-15T22:44:31.549790+0000","last_peered":"2024-09-15T22:44:31.549790+0000","last_clean":"2024-09-15T22:44:31.549790+0000","last_became_active":"2024-09-15T22:44:14.168675+0000","last_became_peered":"2024-09-15T22:44:14.168675+0000","last_unstale":"2024-09-15T22:44:31.549790+0000","last_undegraded":"2024-09-15T22:44:31.549790+0000","last_fullsized":"2024-09-15T22:44:31.549790+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:01.892217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.217871+0000","last_change":"2024-09-15T22:44:17.208587+0000","last_active":"2024-09-15T22:44:19.217871+0000","last_peered":"2024-09-15T22:44:19.217871+0000","last_clean":"2024-09-15T22:44:19.217871+0000","last_became_active":"2024-09-15T22:44:17.208260+0000","last_became_peered":"2024-09-15T22:44:17.208260+0000","last_unstale":"2024-09-15T22:44:19.217871+0000","last_undegraded":"2024-09-15T22:44:19.217871+0000","last_fullsized":"2024-09-15T22:44:19.217871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:01:04.154149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.216128+0000","last_change":"2024-09-15T22:44:14.168223+0000","last_active":"2024-09-15T22:44:19.216128+0000","last_peered":"2024-09-15T22:44:19.216128+0000","last_clean":"2024-09-15T22:44:19.216128+0000","last_became_active":"2024-09-15T22:44:14.167830+0000","last_became_peered":"2024-09-15T22:44:14.167830+0000","last_unstale":"2024-09-15T22:44:19.216128+0000","last_undegraded":"2024-09-15T22:44:19.216128+0000","last_fullsized":"2024-09-15T22:44:19.216128+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:54:38.064606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.213111+0000","last_change":"2024-09-15T22:44:17.209696+0000","last_active":"2024-09-15T22:44:19.213111+0000","last_peered":"2024-09-15T22:44:19.213111+0000","last_clean":"2024-09-15T22:44:19.213111+0000","last_became_active":"2024-09-15T22:44:17.208667+0000","last_became_peered":"2024-09-15T22:44:17.208667+0000","last_unstale":"2024-09-15T22:44:19.213111+0000","last_undegraded":"2024-09-15T22:44:19.213111+0000","last_fullsized":"2024-09-15T22:44:19.213111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:04.653022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.215695+0000","last_change":"2024-09-15T22:44:14.169728+0000","last_active":"2024-09-15T22:44:19.215695+0000","last_peered":"2024-09-15T22:44:19.215695+0000","last_clean":"2024-09-15T22:44:19.215695+0000","last_became_active":"2024-09-15T22:44:14.169354+0000","last_became_peered":"2024-09-15T22:44:14.169354+0000","last_unstale":"2024-09-15T22:44:19.215695+0000","last_undegraded":"2024-09-15T22:44:19.215695+0000","last_fullsized":"2024-09-15T22:44:19.215695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:20.273852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.502524+0000","last_change":"2024-09-15T22:44:17.210593+0000","last_active":"2024-09-15T22:44:19.502524+0000","last_peered":"2024-09-15T22:44:19.502524+0000","last_clean":"2024-09-15T22:44:19.502524+0000","last_became_active":"2024-09-15T22:44:17.210210+0000","last_became_peered":"2024-09-15T22:44:17.210210+0000","last_unstale":"2024-09-15T22:44:19.502524+0000","last_undegraded":"2024-09-15T22:44:19.502524+0000","last_fullsized":"2024-09-15T22:44:19.502524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:56.817301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635928+0000","last_change":"2024-09-15T22:44:14.679757+0000","last_active":"2024-09-15T22:44:51.635928+0000","last_peered":"2024-09-15T22:44:51.635928+0000","last_clean":"2024-09-15T22:44:51.635928+0000","last_became_active":"2024-09-15T22:44:14.679284+0000","last_became_peered":"2024-09-15T22:44:14.679284+0000","last_unstale":"2024-09-15T22:44:51.635928+0000","last_undegraded":"2024-09-15T22:44:51.635928+0000","last_fullsized":"2024-09-15T22:44:51.635928+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:47:21.368373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.212866+0000","last_change":"2024-09-15T22:44:17.207115+0000","last_active":"2024-09-15T22:44:19.212866+0000","last_peered":"2024-09-15T22:44:19.212866+0000","last_clean":"2024-09-15T22:44:19.212866+0000","last_became_active":"2024-09-15T22:44:17.206800+0000","last_became_peered":"2024-09-15T22:44:17.206800+0000","last_unstale":"2024-09-15T22:44:19.212866+0000","last_undegraded":"2024-09-15T22:44:19.212866+0000","last_fullsized":"2024-09-15T22:44:19.212866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:49:17.497725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":335,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.211217+0000","last_change":"2024-09-15T22:42:59.117839+0000","last_active":"2024-09-15T22:44:19.211217+0000","last_peered":"2024-09-15T22:44:19.211217+0000","last_clean":"2024-09-15T22:44:19.211217+0000","last_became_active":"2024-09-15T22:42:59.117434+0000","last_became_peered":"2024-09-15T22:42:59.117434+0000","last_unstale":"2024-09-15T22:44:19.211217+0000","last_undegraded":"2024-09-15T22:44:19.211217+0000","last_fullsized":"2024-09-15T22:44:19.211217+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:39:19.064511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:39:19.064511+0000","last_clean_scrub_stamp":"2024-09-15T22:39:19.064511+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:38.974950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:31.569007+0000","last_change":"2024-09-15T22:44:14.170422+0000","last_active":"2024-09-15T22:44:31.569007+0000","last_peered":"2024-09-15T22:44:31.569007+0000","last_clean":"2024-09-15T22:44:31.569007+0000","last_became_active":"2024-09-15T22:44:14.169152+0000","last_became_peered":"2024-09-15T22:44:14.169152+0000","last_unstale":"2024-09-15T22:44:31.569007+0000","last_undegraded":"2024-09-15T22:44:31.569007+0000","last_fullsized":"2024-09-15T22:44:31.569007+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:10:00.890434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.212511+0000","last_change":"2024-09-15T22:44:17.212972+0000","last_active":"2024-09-15T22:44:19.212511+0000","last_peered":"2024-09-15T22:44:19.212511+0000","last_clean":"2024-09-15T22:44:19.212511+0000","last_became_active":"2024-09-15T22:44:17.212546+0000","last_became_peered":"2024-09-15T22:44:17.212546+0000","last_unstale":"2024-09-15T22:44:19.212511+0000","last_undegraded":"2024-09-15T22:44:19.212511+0000","last_fullsized":"2024-09-15T22:44:19.212511+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:36.322855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:29.554822+0000","last_change":"2024-09-15T22:44:14.171827+0000","last_active":"2024-09-15T22:44:29.554822+0000","last_peered":"2024-09-15T22:44:29.554822+0000","last_clean":"2024-09-15T22:44:29.554822+0000","last_became_active":"2024-09-15T22:44:14.171447+0000","last_became_peered":"2024-09-15T22:44:14.171447+0000","last_unstale":"2024-09-15T22:44:29.554822+0000","last_undegraded":"2024-09-15T22:44:29.554822+0000","last_fullsized":"2024-09-15T22:44:29.554822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:40:49.869819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.211310+0000","last_change":"2024-09-15T22:44:17.208666+0000","last_active":"2024-09-15T22:44:19.211310+0000","last_peered":"2024-09-15T22:44:19.211310+0000","last_clean":"2024-09-15T22:44:19.211310+0000","last_became_active":"2024-09-15T22:44:17.208338+0000","last_became_peered":"2024-09-15T22:44:17.208338+0000","last_unstale":"2024-09-15T22:44:19.211310+0000","last_undegraded":"2024-09-15T22:44:19.211310+0000","last_fullsized":"2024-09-15T22:44:19.211310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:23.250307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":45,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:48.596489+0000","last_change":"2024-09-15T22:44:14.679708+0000","last_active":"2024-09-15T22:44:48.596489+0000","last_peered":"2024-09-15T22:44:48.596489+0000","last_clean":"2024-09-15T22:44:48.596489+0000","last_became_active":"2024-09-15T22:44:14.679192+0000","last_became_peered":"2024-09-15T22:44:14.679192+0000","last_unstale":"2024-09-15T22:44:48.596489+0000","last_undegraded":"2024-09-15T22:44:48.596489+0000","last_fullsized":"2024-09-15T22:44:48.596489+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:25:21.192842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.196209+0000","last_change":"2024-09-15T22:44:17.202716+0000","last_active":"2024-09-15T22:44:19.196209+0000","last_peered":"2024-09-15T22:44:19.196209+0000","last_clean":"2024-09-15T22:44:19.196209+0000","last_became_active":"2024-09-15T22:44:17.202412+0000","last_became_peered":"2024-09-15T22:44:17.202412+0000","last_unstale":"2024-09-15T22:44:19.196209+0000","last_undegraded":"2024-09-15T22:44:19.196209+0000","last_fullsized":"2024-09-15T22:44:19.196209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:18:59.180862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.213083+0000","last_change":"2024-09-15T22:44:14.169202+0000","last_active":"2024-09-15T22:44:19.213083+0000","last_peered":"2024-09-15T22:44:19.213083+0000","last_clean":"2024-09-15T22:44:19.213083+0000","last_became_active":"2024-09-15T22:44:14.168866+0000","last_became_peered":"2024-09-15T22:44:14.168866+0000","last_unstale":"2024-09-15T22:44:19.213083+0000","last_undegraded":"2024-09-15T22:44:19.213083+0000","last_fullsized":"2024-09-15T22:44:19.213083+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:28:57.057050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.210406+0000","last_change":"2024-09-15T22:44:17.218341+0000","last_active":"2024-09-15T22:44:19.210406+0000","last_peered":"2024-09-15T22:44:19.210406+0000","last_clean":"2024-09-15T22:44:19.210406+0000","last_became_active":"2024-09-15T22:44:17.218004+0000","last_became_peered":"2024-09-15T22:44:17.218004+0000","last_unstale":"2024-09-15T22:44:19.210406+0000","last_undegraded":"2024-09-15T22:44:19.210406+0000","last_fullsized":"2024-09-15T22:44:19.210406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:56:54.416938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:44:50.635482+0000","last_change":"2024-09-15T22:44:14.681514+0000","last_active":"2024-09-15T22:44:50.635482+0000","last_peered":"2024-09-15T22:44:50.635482+0000","last_clean":"2024-09-15T22:44:50.635482+0000","last_became_active":"2024-09-15T22:44:14.679895+0000","last_became_peered":"2024-09-15T22:44:14.679895+0000","last_unstale":"2024-09-15T22:44:50.635482+0000","last_undegraded":"2024-09-15T22:44:50.635482+0000","last_fullsized":"2024-09-15T22:44:50.635482+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:13:22.059389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:20.545393+0000","last_change":"2024-09-15T22:44:17.210094+0000","last_active":"2024-09-15T22:44:20.545393+0000","last_peered":"2024-09-15T22:44:20.545393+0000","last_clean":"2024-09-15T22:44:20.545393+0000","last_became_active":"2024-09-15T22:44:17.209784+0000","last_became_peered":"2024-09-15T22:44:17.209784+0000","last_unstale":"2024-09-15T22:44:20.545393+0000","last_undegraded":"2024-09-15T22:44:20.545393+0000","last_fullsized":"2024-09-15T22:44:20.545393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:02.357936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:28.545395+0000","last_change":"2024-09-15T22:44:14.680558+0000","last_active":"2024-09-15T22:44:28.545395+0000","last_peered":"2024-09-15T22:44:28.545395+0000","last_clean":"2024-09-15T22:44:28.545395+0000","last_became_active":"2024-09-15T22:44:14.680183+0000","last_became_peered":"2024-09-15T22:44:14.680183+0000","last_unstale":"2024-09-15T22:44:28.545395+0000","last_undegraded":"2024-09-15T22:44:28.545395+0000","last_fullsized":"2024-09-15T22:44:28.545395+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:44.327412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.214624+0000","last_change":"2024-09-15T22:44:17.210661+0000","last_active":"2024-09-15T22:44:19.214624+0000","last_peered":"2024-09-15T22:44:19.214624+0000","last_clean":"2024-09-15T22:44:19.214624+0000","last_became_active":"2024-09-15T22:44:17.210279+0000","last_became_peered":"2024-09-15T22:44:17.210279+0000","last_unstale":"2024-09-15T22:44:19.214624+0000","last_undegraded":"2024-09-15T22:44:19.214624+0000","last_fullsized":"2024-09-15T22:44:19.214624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:58:45.430753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'10","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:44:50.635620+0000","last_change":"2024-09-15T22:44:14.164859+0000","last_active":"2024-09-15T22:44:50.635620+0000","last_peered":"2024-09-15T22:44:50.635620+0000","last_clean":"2024-09-15T22:44:50.635620+0000","last_became_active":"2024-09-15T22:44:14.163843+0000","last_became_peered":"2024-09-15T22:44:14.163843+0000","last_unstale":"2024-09-15T22:44:50.635620+0000","last_undegraded":"2024-09-15T22:44:50.635620+0000","last_fullsized":"2024-09-15T22:44:50.635620+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:01.322732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.214067+0000","last_change":"2024-09-15T22:44:17.211891+0000","last_active":"2024-09-15T22:44:19.214067+0000","last_peered":"2024-09-15T22:44:19.214067+0000","last_clean":"2024-09-15T22:44:19.214067+0000","last_became_active":"2024-09-15T22:44:17.211515+0000","last_became_peered":"2024-09-15T22:44:17.211515+0000","last_unstale":"2024-09-15T22:44:19.214067+0000","last_undegraded":"2024-09-15T22:44:19.214067+0000","last_fullsized":"2024-09-15T22:44:19.214067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:05:58.223408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:28.543513+0000","last_change":"2024-09-15T22:44:14.167623+0000","last_active":"2024-09-15T22:44:28.543513+0000","last_peered":"2024-09-15T22:44:28.543513+0000","last_clean":"2024-09-15T22:44:28.543513+0000","last_became_active":"2024-09-15T22:44:14.167280+0000","last_became_peered":"2024-09-15T22:44:14.167280+0000","last_unstale":"2024-09-15T22:44:28.543513+0000","last_undegraded":"2024-09-15T22:44:28.543513+0000","last_fullsized":"2024-09-15T22:44:28.543513+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:00:07.609473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:44:50.631730+0000","last_change":"2024-09-15T22:44:17.215630+0000","last_active":"2024-09-15T22:44:50.631730+0000","last_peered":"2024-09-15T22:44:50.631730+0000","last_clean":"2024-09-15T22:44:50.631730+0000","last_became_active":"2024-09-15T22:44:17.215328+0000","last_became_peered":"2024-09-15T22:44:17.215328+0000","last_unstale":"2024-09-15T22:44:50.631730+0000","last_undegraded":"2024-09-15T22:44:50.631730+0000","last_fullsized":"2024-09-15T22:44:50.631730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:13:54.545954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:28.545096+0000","last_change":"2024-09-15T22:44:14.680231+0000","last_active":"2024-09-15T22:44:28.545096+0000","last_peered":"2024-09-15T22:44:28.545096+0000","last_clean":"2024-09-15T22:44:28.545096+0000","last_became_active":"2024-09-15T22:44:14.679858+0000","last_became_peered":"2024-09-15T22:44:14.679858+0000","last_unstale":"2024-09-15T22:44:28.545096+0000","last_undegraded":"2024-09-15T22:44:28.545096+0000","last_fullsized":"2024-09-15T22:44:28.545096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:45:58.992336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.210843+0000","last_change":"2024-09-15T22:44:17.208072+0000","last_active":"2024-09-15T22:44:19.210843+0000","last_peered":"2024-09-15T22:44:19.210843+0000","last_clean":"2024-09-15T22:44:19.210843+0000","last_became_active":"2024-09-15T22:44:17.207580+0000","last_became_peered":"2024-09-15T22:44:17.207580+0000","last_unstale":"2024-09-15T22:44:19.210843+0000","last_undegraded":"2024-09-15T22:44:19.210843+0000","last_fullsized":"2024-09-15T22:44:19.210843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:48.973412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:28.545395+0000","last_change":"2024-09-15T22:44:14.168841+0000","last_active":"2024-09-15T22:44:28.545395+0000","last_peered":"2024-09-15T22:44:28.545395+0000","last_clean":"2024-09-15T22:44:28.545395+0000","last_became_active":"2024-09-15T22:44:14.168359+0000","last_became_peered":"2024-09-15T22:44:14.168359+0000","last_unstale":"2024-09-15T22:44:28.545395+0000","last_undegraded":"2024-09-15T22:44:28.545395+0000","last_fullsized":"2024-09-15T22:44:28.545395+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:09:29.752679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.217654+0000","last_change":"2024-09-15T22:44:17.218227+0000","last_active":"2024-09-15T22:44:19.217654+0000","last_peered":"2024-09-15T22:44:19.217654+0000","last_clean":"2024-09-15T22:44:19.217654+0000","last_became_active":"2024-09-15T22:44:17.217846+0000","last_became_peered":"2024-09-15T22:44:17.217846+0000","last_unstale":"2024-09-15T22:44:19.217654+0000","last_undegraded":"2024-09-15T22:44:19.217654+0000","last_fullsized":"2024-09-15T22:44:19.217654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:07.601525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635631+0000","last_change":"2024-09-15T22:44:14.165322+0000","last_active":"2024-09-15T22:44:51.635631+0000","last_peered":"2024-09-15T22:44:51.635631+0000","last_clean":"2024-09-15T22:44:51.635631+0000","last_became_active":"2024-09-15T22:44:14.164981+0000","last_became_peered":"2024-09-15T22:44:14.164981+0000","last_unstale":"2024-09-15T22:44:51.635631+0000","last_undegraded":"2024-09-15T22:44:51.635631+0000","last_fullsized":"2024-09-15T22:44:51.635631+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:34.812514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.212912+0000","last_change":"2024-09-15T22:44:17.222414+0000","last_active":"2024-09-15T22:44:19.212912+0000","last_peered":"2024-09-15T22:44:19.212912+0000","last_clean":"2024-09-15T22:44:19.212912+0000","last_became_active":"2024-09-15T22:44:17.222115+0000","last_became_peered":"2024-09-15T22:44:17.222115+0000","last_unstale":"2024-09-15T22:44:19.212912+0000","last_undegraded":"2024-09-15T22:44:19.212912+0000","last_fullsized":"2024-09-15T22:44:19.212912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:22:53.791587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:28.544510+0000","last_change":"2024-09-15T22:44:14.166605+0000","last_active":"2024-09-15T22:44:28.544510+0000","last_peered":"2024-09-15T22:44:28.544510+0000","last_clean":"2024-09-15T22:44:28.544510+0000","last_became_active":"2024-09-15T22:44:14.165667+0000","last_became_peered":"2024-09-15T22:44:14.165667+0000","last_unstale":"2024-09-15T22:44:28.544510+0000","last_undegraded":"2024-09-15T22:44:28.544510+0000","last_fullsized":"2024-09-15T22:44:28.544510+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:27.060540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.212549+0000","last_change":"2024-09-15T22:44:17.208143+0000","last_active":"2024-09-15T22:44:19.212549+0000","last_peered":"2024-09-15T22:44:19.212549+0000","last_clean":"2024-09-15T22:44:19.212549+0000","last_became_active":"2024-09-15T22:44:17.207807+0000","last_became_peered":"2024-09-15T22:44:17.207807+0000","last_unstale":"2024-09-15T22:44:19.212549+0000","last_undegraded":"2024-09-15T22:44:19.212549+0000","last_fullsized":"2024-09-15T22:44:19.212549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:19:59.752854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.196642+0000","last_change":"2024-09-15T22:44:17.201752+0000","last_active":"2024-09-15T22:44:19.196642+0000","last_peered":"2024-09-15T22:44:19.196642+0000","last_clean":"2024-09-15T22:44:19.196642+0000","last_became_active":"2024-09-15T22:44:17.201378+0000","last_became_peered":"2024-09-15T22:44:17.201378+0000","last_unstale":"2024-09-15T22:44:19.196642+0000","last_undegraded":"2024-09-15T22:44:19.196642+0000","last_fullsized":"2024-09-15T22:44:19.196642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:42:53.752795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.215941+0000","last_change":"2024-09-15T22:44:14.168024+0000","last_active":"2024-09-15T22:44:19.215941+0000","last_peered":"2024-09-15T22:44:19.215941+0000","last_clean":"2024-09-15T22:44:19.215941+0000","last_became_active":"2024-09-15T22:44:14.167639+0000","last_became_peered":"2024-09-15T22:44:14.167639+0000","last_unstale":"2024-09-15T22:44:19.215941+0000","last_undegraded":"2024-09-15T22:44:19.215941+0000","last_fullsized":"2024-09-15T22:44:19.215941+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:57.867895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.196294+0000","last_change":"2024-09-15T22:44:17.202731+0000","last_active":"2024-09-15T22:44:19.196294+0000","last_peered":"2024-09-15T22:44:19.196294+0000","last_clean":"2024-09-15T22:44:19.196294+0000","last_became_active":"2024-09-15T22:44:17.202410+0000","last_became_peered":"2024-09-15T22:44:17.202410+0000","last_unstale":"2024-09-15T22:44:19.196294+0000","last_undegraded":"2024-09-15T22:44:19.196294+0000","last_fullsized":"2024-09-15T22:44:19.196294+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:34.613686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:29.554912+0000","last_change":"2024-09-15T22:44:14.166865+0000","last_active":"2024-09-15T22:44:29.554912+0000","last_peered":"2024-09-15T22:44:29.554912+0000","last_clean":"2024-09-15T22:44:29.554912+0000","last_became_active":"2024-09-15T22:44:14.166014+0000","last_became_peered":"2024-09-15T22:44:14.166014+0000","last_unstale":"2024-09-15T22:44:29.554912+0000","last_undegraded":"2024-09-15T22:44:29.554912+0000","last_fullsized":"2024-09-15T22:44:29.554912+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:44.527777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:44:50.632064+0000","last_change":"2024-09-15T22:44:17.221028+0000","last_active":"2024-09-15T22:44:50.632064+0000","last_peered":"2024-09-15T22:44:50.632064+0000","last_clean":"2024-09-15T22:44:50.632064+0000","last_became_active":"2024-09-15T22:44:17.220853+0000","last_became_peered":"2024-09-15T22:44:17.220853+0000","last_unstale":"2024-09-15T22:44:50.632064+0000","last_undegraded":"2024-09-15T22:44:50.632064+0000","last_fullsized":"2024-09-15T22:44:50.632064+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:10:50.535946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.216457+0000","last_change":"2024-09-15T22:44:14.679506+0000","last_active":"2024-09-15T22:44:19.216457+0000","last_peered":"2024-09-15T22:44:19.216457+0000","last_clean":"2024-09-15T22:44:19.216457+0000","last_became_active":"2024-09-15T22:44:14.679164+0000","last_became_peered":"2024-09-15T22:44:14.679164+0000","last_unstale":"2024-09-15T22:44:19.216457+0000","last_undegraded":"2024-09-15T22:44:19.216457+0000","last_fullsized":"2024-09-15T22:44:19.216457+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:57.786804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:44:19.212650+0000","last_change":"2024-09-15T22:44:17.203867+0000","last_active":"2024-09-15T22:44:19.212650+0000","last_peered":"2024-09-15T22:44:19.212650+0000","last_clean":"2024-09-15T22:44:19.212650+0000","last_became_active":"2024-09-15T22:44:17.203450+0000","last_became_peered":"2024-09-15T22:44:17.203450+0000","last_unstale":"2024-09-15T22:44:19.212650+0000","last_undegraded":"2024-09-15T22:44:19.212650+0000","last_fullsized":"2024-09-15T22:44:19.212650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:17.210989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-15T22:44:50.635662+0000","last_change":"2024-09-15T22:44:14.164846+0000","last_active":"2024-09-15T22:44:50.635662+0000","last_peered":"2024-09-15T22:44:50.635662+0000","last_clean":"2024-09-15T22:44:50.635662+0000","last_became_active":"2024-09-15T22:44:14.164507+0000","last_became_peered":"2024-09-15T22:44:14.164507+0000","last_unstale":"2024-09-15T22:44:50.635662+0000","last_undegraded":"2024-09-15T22:44:50.635662+0000","last_fullsized":"2024-09-15T22:44:50.635662+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:55.234677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":57863,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":137,"num_read_kb":127,"num_write":159,"num_write_kb":102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":168317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449108,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":454164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678041,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32768,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712384,"statfs":{"total":95995035648,"available":95961481216,"internally_reserved":0,"allocated":1622016,"data_stored":1099505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874272,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":454400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135907,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":454320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397546,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28532,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716620,"statfs":{"total":95995035648,"available":95965818880,"internally_reserved":0,"allocated":1544192,"data_stored":1046252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561070,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36260,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708892,"statfs":{"total":95995035648,"available":95957905408,"internally_reserved":0,"allocated":937984,"data_stored":453789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790005,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1556480,"data_stored":1045936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953528,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":942080,"data_stored":454226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182463,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":454100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1028096,"data_stored":517712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":503185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":970752,"data_stored":463587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":49603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":54588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":55438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:44:54.033 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-15T22:44:54.034 INFO:tasks.cephadm:Setting up client nodes... 2024-09-15T22:44:54.035 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-15T22:44:54.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:53 smithi192 ceph-mon[35427]: pgmap v319: 97 pgs: 97 active+clean; 633 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-09-15T22:44:54.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:53 smithi121 ceph-mon[34815]: pgmap v319: 97 pgs: 97 active+clean; 633 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-09-15T22:44:54.402 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:54.885 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:54 smithi111 ceph-mon[30629]: from='client.24667 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:44:55.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.003+0000 7fb8ca800640 1 -- 172.21.15.111:0/2763273570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8c4103e00 msgr2=0x7fb8c4104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:55.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.003+0000 7fb8ca800640 1 --2- 172.21.15.111:0/2763273570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8c4103e00 0x7fb8c4104200 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4009900 tx=0x7fb8b402a410 comp rx=0 tx=0).stop 2024-09-15T22:44:55.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.003+0000 7fb8ca800640 1 -- 172.21.15.111:0/2763273570 shutdown_connections 2024-09-15T22:44:55.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.003+0000 7fb8ca800640 1 --2- 172.21.15.111:0/2763273570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8c4105920 0x7fb8c410c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:55.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.003+0000 7fb8ca800640 1 --2- 172.21.15.111:0/2763273570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8c4104f60 0x7fb8c41053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:55.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.003+0000 7fb8ca800640 1 --2- 172.21.15.111:0/2763273570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8c4103e00 0x7fb8c4104200 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:55.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.003+0000 7fb8ca800640 1 -- 172.21.15.111:0/2763273570 >> 172.21.15.111:0/2763273570 conn(0x7fb8c40ff5b0 msgr2=0x7fb8c41019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:55.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.003+0000 7fb8ca800640 1 -- 172.21.15.111:0/2763273570 shutdown_connections 2024-09-15T22:44:55.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.003+0000 7fb8ca800640 1 -- 172.21.15.111:0/2763273570 wait complete. 2024-09-15T22:44:55.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.004+0000 7fb8ca800640 1 Processor -- start 2024-09-15T22:44:55.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.004+0000 7fb8ca800640 1 -- start start 2024-09-15T22:44:55.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.004+0000 7fb8ca800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8c4103e00 0x7fb8c419a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:55.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.004+0000 7fb8ca800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8c4104f60 0x7fb8c419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:55.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.004+0000 7fb8ca800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8c4105920 0x7fb8c41a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:55.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.004+0000 7fb8ca800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c410fb00 con 0x7fb8c4104f60 2024-09-15T22:44:55.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.004+0000 7fb8ca800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c410f980 con 0x7fb8c4105920 2024-09-15T22:44:55.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.004+0000 7fb8ca800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c410fc80 con 0x7fb8c4103e00 2024-09-15T22:44:55.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.005+0000 7fb8c3400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8c4103e00 0x7fb8c419a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:55.007 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.005+0000 7fb8baa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8c4104f60 0x7fb8c419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:55.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.005+0000 7fb8c3400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8c4103e00 0x7fb8c419a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:49228/0 (socket says 172.21.15.111:49228) 2024-09-15T22:44:55.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.005+0000 7fb8c3400640 1 -- 172.21.15.111:0/4045069891 learned_addr learned my addr 172.21.15.111:0/4045069891 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:44:55.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.005+0000 7fb8c3e00640 1 --2- 172.21.15.111:0/4045069891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8c4105920 0x7fb8c41a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:55.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.005+0000 7fb8c3400640 1 -- 172.21.15.111:0/4045069891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8c4105920 msgr2=0x7fb8c41a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:55.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.005+0000 7fb8c3400640 1 --2- 172.21.15.111:0/4045069891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8c4105920 0x7fb8c41a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:55.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.005+0000 7fb8c3400640 1 -- 172.21.15.111:0/4045069891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8c4104f60 msgr2=0x7fb8c419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:55.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.005+0000 7fb8c3400640 1 --2- 172.21.15.111:0/4045069891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8c4104f60 0x7fb8c419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:55.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.005+0000 7fb8c3400640 1 -- 172.21.15.111:0/4045069891 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c41a1cf0 con 0x7fb8c4103e00 2024-09-15T22:44:55.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.005+0000 7fb8baa00640 1 --2- 172.21.15.111:0/4045069891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8c4104f60 0x7fb8c419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:44:55.008 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.006+0000 7fb8c3400640 1 --2- 172.21.15.111:0/4045069891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8c4103e00 0x7fb8c419a0c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4031440 tx=0x7fb8b4032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:55.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.006+0000 7fb8c0c00640 1 -- 172.21.15.111:0/4045069891 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b403f030 con 0x7fb8c4103e00 2024-09-15T22:44:55.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.006+0000 7fb8c0c00640 1 -- 172.21.15.111:0/4045069891 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8b4003ce0 con 0x7fb8c4103e00 2024-09-15T22:44:55.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.006+0000 7fb8ca800640 1 -- 172.21.15.111:0/4045069891 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8c41a1f80 con 0x7fb8c4103e00 2024-09-15T22:44:55.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.006+0000 7fb8c0c00640 1 -- 172.21.15.111:0/4045069891 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4035070 con 0x7fb8c4103e00 2024-09-15T22:44:55.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.006+0000 7fb8ca800640 1 -- 172.21.15.111:0/4045069891 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8c41a2380 con 0x7fb8c4103e00 2024-09-15T22:44:55.009 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.007+0000 7fb8ca800640 1 -- 172.21.15.111:0/4045069891 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8c410b1a0 con 0x7fb8c4103e00 2024-09-15T22:44:55.010 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.009+0000 7fb8c0c00640 1 -- 172.21.15.111:0/4045069891 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fb8b4042030 con 0x7fb8c4103e00 2024-09-15T22:44:55.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.010+0000 7fb8c0c00640 1 --2- 172.21.15.111:0/4045069891 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb8a80a3300 0x7fb8a80a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:55.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.010+0000 7fb8c0c00640 1 -- 172.21.15.111:0/4045069891 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fb8b40efff0 con 0x7fb8c4103e00 2024-09-15T22:44:55.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.011+0000 7fb8baa00640 1 --2- 172.21.15.111:0/4045069891 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb8a80a3300 0x7fb8a80a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:55.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.011+0000 7fb8baa00640 1 --2- 172.21.15.111:0/4045069891 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb8a80a3300 0x7fb8a80a57c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb8c419b5e0 tx=0x7fb8ac00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:55.014 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.013+0000 7fb8c0c00640 1 -- 172.21.15.111:0/4045069891 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb8b40bc5c0 con 0x7fb8c4103e00 2024-09-15T22:44:55.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:54 smithi192 ceph-mon[35427]: from='client.24667 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:44:55.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:54 smithi121 ceph-mon[34815]: from='client.24667 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:44:55.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.254+0000 7fb8ca800640 1 -- 172.21.15.111:0/4045069891 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fb8c402d7c0 con 0x7fb8c4103e00 2024-09-15T22:44:55.262 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.261+0000 7fb8c0c00640 1 -- 172.21.15.111:0/4045069891 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fb8b40c08c0 con 0x7fb8c4103e00 2024-09-15T22:44:55.263 INFO:teuthology.orchestra.run.smithi111.stdout:[client.0] 2024-09-15T22:44:55.263 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBnY+dm9tZRDxAAnhhN1Vp0i6e4qmcTA+l3xQ== 2024-09-15T22:44:55.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.263+0000 7fb8ca800640 1 -- 172.21.15.111:0/4045069891 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb8a80a3300 msgr2=0x7fb8a80a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:55.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.263+0000 7fb8ca800640 1 --2- 172.21.15.111:0/4045069891 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb8a80a3300 0x7fb8a80a57c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb8c419b5e0 tx=0x7fb8ac00b410 comp rx=0 tx=0).stop 2024-09-15T22:44:55.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.263+0000 7fb8ca800640 1 -- 172.21.15.111:0/4045069891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8c4103e00 msgr2=0x7fb8c419a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:55.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.263+0000 7fb8ca800640 1 --2- 172.21.15.111:0/4045069891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8c4103e00 0x7fb8c419a0c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4031440 tx=0x7fb8b4032ce0 comp rx=0 tx=0).stop 2024-09-15T22:44:55.265 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.263+0000 7fb8ca800640 1 -- 172.21.15.111:0/4045069891 shutdown_connections 2024-09-15T22:44:55.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.263+0000 7fb8ca800640 1 --2- 172.21.15.111:0/4045069891 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb8a80a3300 0x7fb8a80a57c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:55.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.263+0000 7fb8ca800640 1 --2- 172.21.15.111:0/4045069891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb8c4105920 0x7fb8c41a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:55.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.263+0000 7fb8ca800640 1 --2- 172.21.15.111:0/4045069891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8c4104f60 0x7fb8c419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:55.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.263+0000 7fb8ca800640 1 --2- 172.21.15.111:0/4045069891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8c4103e00 0x7fb8c419a0c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:55.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.263+0000 7fb8ca800640 1 -- 172.21.15.111:0/4045069891 >> 172.21.15.111:0/4045069891 conn(0x7fb8c40ff5b0 msgr2=0x7fb8c4108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:55.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.264+0000 7fb8ca800640 1 -- 172.21.15.111:0/4045069891 shutdown_connections 2024-09-15T22:44:55.266 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:55.264+0000 7fb8ca800640 1 -- 172.21.15.111:0/4045069891 wait complete. 2024-09-15T22:44:55.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:55 smithi111 ceph-mon[30629]: pgmap v320: 97 pgs: 97 active+clean; 633 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 639 B/s wr, 4 op/s; 126 B/s, 0 objects/s recovering 2024-09-15T22:44:55.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:55 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/4045069891' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-15T22:44:55.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:55 smithi111 ceph-mon[30629]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-15T22:44:55.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:55 smithi111 ceph-mon[30629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-15T22:44:56.001 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-15T22:44:56.001 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-15T22:44:56.002 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-15T22:44:56.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:55 smithi192 ceph-mon[35427]: pgmap v320: 97 pgs: 97 active+clean; 633 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 639 B/s wr, 4 op/s; 126 B/s, 0 objects/s recovering 2024-09-15T22:44:56.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:55 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/4045069891' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-15T22:44:56.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:55 smithi192 ceph-mon[35427]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-15T22:44:56.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:55 smithi192 ceph-mon[35427]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-15T22:44:56.080 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-15T22:44:56.080 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-15T22:44:56.080 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-15T22:44:56.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:55 smithi121 ceph-mon[34815]: pgmap v320: 97 pgs: 97 active+clean; 633 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 639 B/s wr, 4 op/s; 126 B/s, 0 objects/s recovering 2024-09-15T22:44:56.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:55 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/4045069891' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-15T22:44:56.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:55 smithi121 ceph-mon[34815]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-15T22:44:56.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:55 smithi121 ceph-mon[34815]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-15T22:44:56.485 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:57.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.057+0000 7f27bc400640 1 -- 172.21.15.111:0/112098345 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27b4101de0 msgr2=0x7f27b41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:57.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.057+0000 7f27bc400640 1 --2- 172.21.15.111:0/112098345 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27b4101de0 0x7f27b41021c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f27a8009900 tx=0x7f27a802a410 comp rx=0 tx=0).stop 2024-09-15T22:44:57.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.058+0000 7f27bc400640 1 -- 172.21.15.111:0/112098345 shutdown_connections 2024-09-15T22:44:57.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.058+0000 7f27bc400640 1 --2- 172.21.15.111:0/112098345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27b4106f10 0x7f27b410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:57.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.058+0000 7f27bc400640 1 --2- 172.21.15.111:0/112098345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27b4102790 0x7f27b41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:57.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.058+0000 7f27bc400640 1 --2- 172.21.15.111:0/112098345 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27b4101de0 0x7f27b41021c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:57.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.058+0000 7f27bc400640 1 -- 172.21.15.111:0/112098345 >> 172.21.15.111:0/112098345 conn(0x7f27b40fd8c0 msgr2=0x7f27b40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:57.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.058+0000 7f27bc400640 1 -- 172.21.15.111:0/112098345 shutdown_connections 2024-09-15T22:44:57.060 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.058+0000 7f27bc400640 1 -- 172.21.15.111:0/112098345 wait complete. 2024-09-15T22:44:57.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.059+0000 7f27bc400640 1 Processor -- start 2024-09-15T22:44:57.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.059+0000 7f27bc400640 1 -- start start 2024-09-15T22:44:57.061 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.059+0000 7f27bc400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27b4101de0 0x7f27b419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.059+0000 7f27bc400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27b4102790 0x7f27b419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.059+0000 7f27bc400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27b4106f10 0x7f27b419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.059+0000 7f27bc400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f27b4111f70 con 0x7f27b4106f10 2024-09-15T22:44:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.059+0000 7f27bc400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f27b4111df0 con 0x7f27b4102790 2024-09-15T22:44:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.059+0000 7f27bc400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f27b41120f0 con 0x7f27b4101de0 2024-09-15T22:44:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.059+0000 7f27ba400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27b4106f10 0x7f27b419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.059+0000 7f27b9000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27b4102790 0x7f27b419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.059+0000 7f27ba400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27b4106f10 0x7f27b419ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56142/0 (socket says 172.21.15.111:56142) 2024-09-15T22:44:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.059+0000 7f27ba400640 1 -- 172.21.15.111:0/3344970058 learned_addr learned my addr 172.21.15.111:0/3344970058 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:44:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.059+0000 7f27b9a00640 1 --2- 172.21.15.111:0/3344970058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27b4101de0 0x7f27b419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.060+0000 7f27b9000640 1 -- 172.21.15.111:0/3344970058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27b4101de0 msgr2=0x7f27b419a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.060+0000 7f27b9000640 1 --2- 172.21.15.111:0/3344970058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27b4101de0 0x7f27b419a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.060+0000 7f27b9000640 1 -- 172.21.15.111:0/3344970058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27b4106f10 msgr2=0x7f27b419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:57.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.060+0000 7f27b9000640 1 --2- 172.21.15.111:0/3344970058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27b4106f10 0x7f27b419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:57.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.060+0000 7f27b9000640 1 -- 172.21.15.111:0/3344970058 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b419f600 con 0x7f27b4102790 2024-09-15T22:44:57.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.060+0000 7f27ba400640 1 --2- 172.21.15.111:0/3344970058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27b4106f10 0x7f27b419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:44:57.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.060+0000 7f27b9000640 1 --2- 172.21.15.111:0/3344970058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27b4102790 0x7f27b419ac30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f27a000e9f0 tx=0x7f27a000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:57.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.060+0000 7f27b2000640 1 -- 172.21.15.111:0/3344970058 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a000ccc0 con 0x7f27b4102790 2024-09-15T22:44:57.063 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.060+0000 7f27b2000640 1 -- 172.21.15.111:0/3344970058 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27a0005020 con 0x7f27b4102790 2024-09-15T22:44:57.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.060+0000 7f27b2000640 1 -- 172.21.15.111:0/3344970058 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a00049e0 con 0x7f27b4102790 2024-09-15T22:44:57.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.060+0000 7f27bc400640 1 -- 172.21.15.111:0/3344970058 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b419f8f0 con 0x7f27b4102790 2024-09-15T22:44:57.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.061+0000 7f27bc400640 1 -- 172.21.15.111:0/3344970058 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b41a71d0 con 0x7f27b4102790 2024-09-15T22:44:57.064 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.062+0000 7f27bc400640 1 -- 172.21.15.111:0/3344970058 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27b41021c0 con 0x7f27b4102790 2024-09-15T22:44:57.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.064+0000 7f27b2000640 1 -- 172.21.15.111:0/3344970058 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f27a0005560 con 0x7f27b4102790 2024-09-15T22:44:57.066 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.065+0000 7f27b2000640 1 --2- 172.21.15.111:0/3344970058 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f279c0a32d0 0x7f279c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:57.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.065+0000 7f27b2000640 1 -- 172.21.15.111:0/3344970058 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f27a00cfcb0 con 0x7f27b4102790 2024-09-15T22:44:57.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.065+0000 7f27b9a00640 1 --2- 172.21.15.111:0/3344970058 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f279c0a32d0 0x7f279c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:57.067 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.066+0000 7f27b9a00640 1 --2- 172.21.15.111:0/3344970058 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f279c0a32d0 0x7f279c0a5790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f27a8009230 tx=0x7f27a80058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:57.068 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.067+0000 7f27b2000640 1 -- 172.21.15.111:0/3344970058 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f27a00051d0 con 0x7f27b4102790 2024-09-15T22:44:57.294 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.292+0000 7f27bc400640 1 -- 172.21.15.111:0/3344970058 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f27b419ba50 con 0x7f27b4102790 2024-09-15T22:44:57.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.297+0000 7f27b2000640 1 -- 172.21.15.111:0/3344970058 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264058 (secure 0 0 0) 0x7f27a009c870 con 0x7f27b4102790 2024-09-15T22:44:57.303 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:44:57.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.302+0000 7f27bc400640 1 -- 172.21.15.111:0/3344970058 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f279c0a32d0 msgr2=0x7f279c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:57.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.302+0000 7f27bc400640 1 --2- 172.21.15.111:0/3344970058 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f279c0a32d0 0x7f279c0a5790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f27a8009230 tx=0x7f27a80058d0 comp rx=0 tx=0).stop 2024-09-15T22:44:57.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.302+0000 7f27bc400640 1 -- 172.21.15.111:0/3344970058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27b4102790 msgr2=0x7f27b419ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:57.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.302+0000 7f27bc400640 1 --2- 172.21.15.111:0/3344970058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27b4102790 0x7f27b419ac30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f27a000e9f0 tx=0x7f27a000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:44:57.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.302+0000 7f27bc400640 1 -- 172.21.15.111:0/3344970058 shutdown_connections 2024-09-15T22:44:57.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.302+0000 7f27bc400640 1 --2- 172.21.15.111:0/3344970058 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f27b4106f10 0x7f27b419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:57.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.302+0000 7f27bc400640 1 --2- 172.21.15.111:0/3344970058 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f279c0a32d0 0x7f279c0a5790 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:57.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.302+0000 7f27bc400640 1 --2- 172.21.15.111:0/3344970058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27b4102790 0x7f27b419ac30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:57.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.302+0000 7f27bc400640 1 --2- 172.21.15.111:0/3344970058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27b4101de0 0x7f27b419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:57.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.302+0000 7f27bc400640 1 -- 172.21.15.111:0/3344970058 >> 172.21.15.111:0/3344970058 conn(0x7f27b40fd8c0 msgr2=0x7f27b40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:57.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.302+0000 7f27bc400640 1 -- 172.21.15.111:0/3344970058 shutdown_connections 2024-09-15T22:44:57.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:57.302+0000 7f27bc400640 1 -- 172.21.15.111:0/3344970058 wait complete. 2024-09-15T22:44:57.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:57 smithi111 ceph-mon[30629]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 8 op/s; 141 B/s, 0 objects/s recovering 2024-09-15T22:44:57.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:57 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/3344970058' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-15T22:44:58.059 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":17,"flags":0,"active_gid":14148,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":3125866964},{"type":"v1","addr":"172.21.15.111:6801","nonce":3125866964}]},"active_addr":"172.21.15.111:6801/3125866964","active_change":"2024-09-15T22:36:50.357676+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.111:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.111:0","nonce":1734278339}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.111:0","nonce":1342823489}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.111:0","nonce":738939510}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.111:0","nonce":3843671844}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.111:0","nonce":4231083903}]}]} 2024-09-15T22:44:58.064 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-15T22:44:58.064 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-15T22:44:58.065 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:44:58.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:57 smithi192 ceph-mon[35427]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 8 op/s; 141 B/s, 0 objects/s recovering 2024-09-15T22:44:58.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:57 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/3344970058' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-15T22:44:58.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:57 smithi121 ceph-mon[34815]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 8 op/s; 141 B/s, 0 objects/s recovering 2024-09-15T22:44:58.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:57 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/3344970058' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-15T22:44:58.430 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:44:59.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.988+0000 7f34e6400640 1 -- 172.21.15.111:0/1218031131 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34e01049a0 msgr2=0x7f34e01089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:59.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.988+0000 7f34e6400640 1 --2- 172.21.15.111:0/1218031131 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34e01049a0 0x7f34e01089f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f34c8009900 tx=0x7f34c802a3f0 comp rx=0 tx=0).stop 2024-09-15T22:44:59.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.988+0000 7f34e6400640 1 -- 172.21.15.111:0/1218031131 shutdown_connections 2024-09-15T22:44:59.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.988+0000 7f34e6400640 1 --2- 172.21.15.111:0/1218031131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34e0109120 0x7f34e0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:59.025 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.988+0000 7f34e6400640 1 --2- 172.21.15.111:0/1218031131 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34e01049a0 0x7f34e01089f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:59.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.988+0000 7f34e6400640 1 --2- 172.21.15.111:0/1218031131 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34e0103ff0 0x7f34e01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:59.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.988+0000 7f34e6400640 1 -- 172.21.15.111:0/1218031131 >> 172.21.15.111:0/1218031131 conn(0x7f34e00ffa90 msgr2=0x7f34e0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:59.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.988+0000 7f34e6400640 1 -- 172.21.15.111:0/1218031131 shutdown_connections 2024-09-15T22:44:59.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.988+0000 7f34e6400640 1 -- 172.21.15.111:0/1218031131 wait complete. 2024-09-15T22:44:59.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.989+0000 7f34e6400640 1 Processor -- start 2024-09-15T22:44:59.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.989+0000 7f34e6400640 1 -- start start 2024-09-15T22:44:59.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.990+0000 7f34e6400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34e0103ff0 0x7f34e019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:59.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.990+0000 7f34e6400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34e01049a0 0x7f34e019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:59.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.990+0000 7f34e6400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34e0109120 0x7f34e019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:59.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.990+0000 7f34e6400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f34e0114170 con 0x7f34e0109120 2024-09-15T22:44:59.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.990+0000 7f34e6400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f34e0113ff0 con 0x7f34e01049a0 2024-09-15T22:44:59.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.990+0000 7f34e6400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f34e01142f0 con 0x7f34e0103ff0 2024-09-15T22:44:59.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.990+0000 7f34df400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34e0103ff0 0x7f34e019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:59.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.990+0000 7f34df400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34e0103ff0 0x7f34e019a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:49246/0 (socket says 172.21.15.111:49246) 2024-09-15T22:44:59.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.990+0000 7f34dfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34e0109120 0x7f34e019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:59.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.990+0000 7f34df400640 1 -- 172.21.15.111:0/2180811154 learned_addr learned my addr 172.21.15.111:0/2180811154 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:44:59.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.990+0000 7f34d7e00640 1 --2- 172.21.15.111:0/2180811154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34e01049a0 0x7f34e019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:59.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.990+0000 7f34df400640 1 -- 172.21.15.111:0/2180811154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34e01049a0 msgr2=0x7f34e019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:59.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.990+0000 7f34df400640 1 --2- 172.21.15.111:0/2180811154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34e01049a0 0x7f34e019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:59.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.990+0000 7f34df400640 1 -- 172.21.15.111:0/2180811154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34e0109120 msgr2=0x7f34e019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:59.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.990+0000 7f34df400640 1 --2- 172.21.15.111:0/2180811154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34e0109120 0x7f34e019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:59.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.990+0000 7f34df400640 1 -- 172.21.15.111:0/2180811154 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e019f600 con 0x7f34e0103ff0 2024-09-15T22:44:59.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.990+0000 7f34dfe00640 1 --2- 172.21.15.111:0/2180811154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34e0109120 0x7f34e019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:44:59.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.991+0000 7f34df400640 1 --2- 172.21.15.111:0/2180811154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34e0103ff0 0x7f34e019a650 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f34d000d950 tx=0x7f34d000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:59.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.991+0000 7f34dcc00640 1 -- 172.21.15.111:0/2180811154 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d0017030 con 0x7f34e0103ff0 2024-09-15T22:44:59.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.991+0000 7f34e6400640 1 -- 172.21.15.111:0/2180811154 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e019f8f0 con 0x7f34e0103ff0 2024-09-15T22:44:59.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.991+0000 7f34dcc00640 1 -- 172.21.15.111:0/2180811154 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34d0004020 con 0x7f34e0103ff0 2024-09-15T22:44:59.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.991+0000 7f34dcc00640 1 -- 172.21.15.111:0/2180811154 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d00047a0 con 0x7f34e0103ff0 2024-09-15T22:44:59.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.991+0000 7f34e6400640 1 -- 172.21.15.111:0/2180811154 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e01a71d0 con 0x7f34e0103ff0 2024-09-15T22:44:59.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.992+0000 7f34e6400640 1 -- 172.21.15.111:0/2180811154 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34e01043d0 con 0x7f34e0103ff0 2024-09-15T22:44:59.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.994+0000 7f34dcc00640 1 -- 172.21.15.111:0/2180811154 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f34d00041d0 con 0x7f34e0103ff0 2024-09-15T22:44:59.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.995+0000 7f34dcc00640 1 --2- 172.21.15.111:0/2180811154 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f34c40a32d0 0x7f34c40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:44:59.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.995+0000 7f34dcc00640 1 -- 172.21.15.111:0/2180811154 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f34d00cf3d0 con 0x7f34e0103ff0 2024-09-15T22:44:59.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.995+0000 7f34d7e00640 1 --2- 172.21.15.111:0/2180811154 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f34c40a32d0 0x7f34c40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:44:59.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.996+0000 7f34d7e00640 1 --2- 172.21.15.111:0/2180811154 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f34c40a32d0 0x7f34c40a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f34c8000c00 tx=0x7f34c8005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:44:59.029 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:58.999+0000 7f34dcc00640 1 -- 172.21.15.111:0/2180811154 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f34d009bf90 con 0x7f34e0103ff0 2024-09-15T22:44:59.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:59.179+0000 7f34e6400640 1 -- 172.21.15.111:0/2180811154 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34e010c3c0 con 0x7f34e0103ff0 2024-09-15T22:44:59.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:59.181+0000 7f34dcc00640 1 -- 172.21.15.111:0/2180811154 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22701 (secure 0 0 0) 0x7f34d009b890 con 0x7f34e0103ff0 2024-09-15T22:44:59.183 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:44:59.183 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":88,"fsid":"b7614d3c-73b2-11ef-bceb-c7b262605968","created":"2024-09-15T22:35:52.571473+0000","modified":"2024-09-15T22:44:51.627977+0000","last_up_change":"2024-09-15T22:43:19.567274+0000","last_in_change":"2024-09-15T22:42:56.994246+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:39:18.954207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:44:12.846418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:44:15.685946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04f39604-be0e-4261-b316-67fa087c6666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6803","nonce":1386823446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6805","nonce":1386823446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6809","nonce":1386823446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6807","nonce":1386823446}]},"public_addr":"172.21.15.111:6803/1386823446","cluster_addr":"172.21.15.111:6805/1386823446","heartbeat_back_addr":"172.21.15.111:6809/1386823446","heartbeat_front_addr":"172.21.15.111:6807/1386823446","state":["exists","up"]},{"osd":1,"uuid":"1bdbef6e-23fa-4618-a788-66e57741a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6801","nonce":2051512291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6803","nonce":2051512291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6807","nonce":2051512291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6805","nonce":2051512291}]},"public_addr":"172.21.15.121:6801/2051512291","cluster_addr":"172.21.15.121:6803/2051512291","heartbeat_back_addr":"172.21.15.121:6807/2051512291","heartbeat_front_addr":"172.21.15.121:6805/2051512291","state":["exists","up"]},{"osd":2,"uuid":"46735ccf-6d0e-4154-bb55-422929819bfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6801","nonce":93085639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6803","nonce":93085639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6807","nonce":93085639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6805","nonce":93085639}]},"public_addr":"172.21.15.192:6801/93085639","cluster_addr":"172.21.15.192:6803/93085639","heartbeat_back_addr":"172.21.15.192:6807/93085639","heartbeat_front_addr":"172.21.15.192:6805/93085639","state":["exists","up"]},{"osd":3,"uuid":"4b243af8-c958-40b1-ba24-9c7ace00c4a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6811","nonce":2578259041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6813","nonce":2578259041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6817","nonce":2578259041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6815","nonce":2578259041}]},"public_addr":"172.21.15.111:6811/2578259041","cluster_addr":"172.21.15.111:6813/2578259041","heartbeat_back_addr":"172.21.15.111:6817/2578259041","heartbeat_front_addr":"172.21.15.111:6815/2578259041","state":["exists","up"]},{"osd":4,"uuid":"49f27541-9cbf-48e0-a466-6bb0fa5410b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6809","nonce":2667595915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6811","nonce":2667595915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6815","nonce":2667595915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6813","nonce":2667595915}]},"public_addr":"172.21.15.121:6809/2667595915","cluster_addr":"172.21.15.121:6811/2667595915","heartbeat_back_addr":"172.21.15.121:6815/2667595915","heartbeat_front_addr":"172.21.15.121:6813/2667595915","state":["exists","up"]},{"osd":5,"uuid":"9290e769-5470-44d3-8d24-25093af34335","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6809","nonce":1958639856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6811","nonce":1958639856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6815","nonce":1958639856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6813","nonce":1958639856}]},"public_addr":"172.21.15.192:6809/1958639856","cluster_addr":"172.21.15.192:6811/1958639856","heartbeat_back_addr":"172.21.15.192:6815/1958639856","heartbeat_front_addr":"172.21.15.192:6813/1958639856","state":["exists","up"]},{"osd":6,"uuid":"18911b97-1c68-4e8f-92ec-35d2ff0b5f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6819","nonce":1371543706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6821","nonce":1371543706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6825","nonce":1371543706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6823","nonce":1371543706}]},"public_addr":"172.21.15.111:6819/1371543706","cluster_addr":"172.21.15.111:6821/1371543706","heartbeat_back_addr":"172.21.15.111:6825/1371543706","heartbeat_front_addr":"172.21.15.111:6823/1371543706","state":["exists","up"]},{"osd":7,"uuid":"470647c5-c00d-426b-a134-cca235902307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6817","nonce":3501842998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6819","nonce":3501842998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6823","nonce":3501842998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6821","nonce":3501842998}]},"public_addr":"172.21.15.121:6817/3501842998","cluster_addr":"172.21.15.121:6819/3501842998","heartbeat_back_addr":"172.21.15.121:6823/3501842998","heartbeat_front_addr":"172.21.15.121:6821/3501842998","state":["exists","up"]},{"osd":8,"uuid":"fa2d3dd5-a768-4ada-a036-4c596b76d654","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6817","nonce":2417771374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6819","nonce":2417771374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6823","nonce":2417771374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6821","nonce":2417771374}]},"public_addr":"172.21.15.192:6817/2417771374","cluster_addr":"172.21.15.192:6819/2417771374","heartbeat_back_addr":"172.21.15.192:6823/2417771374","heartbeat_front_addr":"172.21.15.192:6821/2417771374","state":["exists","up"]},{"osd":9,"uuid":"71b4f5fb-1c02-461b-a257-c6e48381d768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6827","nonce":3979119431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6829","nonce":3979119431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6833","nonce":3979119431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6831","nonce":3979119431}]},"public_addr":"172.21.15.111:6827/3979119431","cluster_addr":"172.21.15.111:6829/3979119431","heartbeat_back_addr":"172.21.15.111:6833/3979119431","heartbeat_front_addr":"172.21.15.111:6831/3979119431","state":["exists","up"]},{"osd":10,"uuid":"e6a39e77-fce3-4e22-bf66-a677c9f8b6bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6825","nonce":2897902637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6827","nonce":2897902637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6831","nonce":2897902637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6829","nonce":2897902637}]},"public_addr":"172.21.15.121:6825/2897902637","cluster_addr":"172.21.15.121:6827/2897902637","heartbeat_back_addr":"172.21.15.121:6831/2897902637","heartbeat_front_addr":"172.21.15.121:6829/2897902637","state":["exists","up"]},{"osd":11,"uuid":"411a7921-6717-4cef-b8df-d9e252edcc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6825","nonce":2431464812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6827","nonce":2431464812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6831","nonce":2431464812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6829","nonce":2431464812}]},"public_addr":"172.21.15.192:6825/2431464812","cluster_addr":"172.21.15.192:6827/2431464812","heartbeat_back_addr":"172.21.15.192:6831/2431464812","heartbeat_front_addr":"172.21.15.192:6829/2431464812","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:38:23.442715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:38:49.501124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:39:14.759077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:39:45.121397+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:40:11.137612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:40:36.278567+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:07.027190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:32.439215+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:57.058702+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:42:28.741846+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:42:53.395442+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:43:18.265067+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:6801/2107775445":"2024-09-16T22:36:50.357551+0000","172.21.15.111:6800/2107775445":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/2394746540":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/4140020843":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/2114354263":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/1054173642":"2024-09-16T22:36:22.652875+0000","172.21.15.111:0/3408238004":"2024-09-16T22:36:22.652875+0000","172.21.15.111:0/2879044772":"2024-09-16T22:36:22.652875+0000","172.21.15.111:6801/3350425104":"2024-09-16T22:36:22.652875+0000","172.21.15.111:6800/3350425104":"2024-09-16T22:36:22.652875+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:44:59.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:59.183+0000 7f34e6400640 1 -- 172.21.15.111:0/2180811154 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f34c40a32d0 msgr2=0x7f34c40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:59.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:59.183+0000 7f34e6400640 1 --2- 172.21.15.111:0/2180811154 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f34c40a32d0 0x7f34c40a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f34c8000c00 tx=0x7f34c8005b50 comp rx=0 tx=0).stop 2024-09-15T22:44:59.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:59.183+0000 7f34e6400640 1 -- 172.21.15.111:0/2180811154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34e0103ff0 msgr2=0x7f34e019a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:44:59.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:59.183+0000 7f34e6400640 1 --2- 172.21.15.111:0/2180811154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34e0103ff0 0x7f34e019a650 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f34d000d950 tx=0x7f34d000de10 comp rx=0 tx=0).stop 2024-09-15T22:44:59.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:59.183+0000 7f34e6400640 1 -- 172.21.15.111:0/2180811154 shutdown_connections 2024-09-15T22:44:59.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:59.183+0000 7f34e6400640 1 --2- 172.21.15.111:0/2180811154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34e0109120 0x7f34e019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:59.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:59.183+0000 7f34e6400640 1 --2- 172.21.15.111:0/2180811154 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f34c40a32d0 0x7f34c40a5790 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:59.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:59.183+0000 7f34e6400640 1 --2- 172.21.15.111:0/2180811154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34e01049a0 0x7f34e019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:59.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:59.183+0000 7f34e6400640 1 --2- 172.21.15.111:0/2180811154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34e0103ff0 0x7f34e019a650 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:44:59.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:59.183+0000 7f34e6400640 1 -- 172.21.15.111:0/2180811154 >> 172.21.15.111:0/2180811154 conn(0x7f34e00ffa90 msgr2=0x7f34e0100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:44:59.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:59.183+0000 7f34e6400640 1 -- 172.21.15.111:0/2180811154 shutdown_connections 2024-09-15T22:44:59.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:44:59.183+0000 7f34e6400640 1 -- 172.21.15.111:0/2180811154 wait complete. 2024-09-15T22:44:59.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:59 smithi111 ceph-mon[30629]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s; 141 B/s, 0 objects/s recovering 2024-09-15T22:44:59.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:44:59 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/2180811154' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:45:00.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:59 smithi192 ceph-mon[35427]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s; 141 B/s, 0 objects/s recovering 2024-09-15T22:45:00.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:44:59 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/2180811154' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:45:00.100 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-15T22:45:00.101 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:45:00.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:59 smithi121 ceph-mon[34815]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.9 KiB/s wr, 6 op/s; 141 B/s, 0 objects/s recovering 2024-09-15T22:45:00.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:44:59 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/2180811154' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:45:00.467 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:01.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.055+0000 7f457ce00640 1 -- 172.21.15.111:0/952547275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4578109120 msgr2=0x7f4578110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:01.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.055+0000 7f457ce00640 1 --2- 172.21.15.111:0/952547275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4578109120 0x7f4578110ca0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4570009ca0 tx=0x7f457002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.055+0000 7f457ce00640 1 -- 172.21.15.111:0/952547275 shutdown_connections 2024-09-15T22:45:01.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.055+0000 7f457ce00640 1 --2- 172.21.15.111:0/952547275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4578109120 0x7f4578110ca0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.055+0000 7f457ce00640 1 --2- 172.21.15.111:0/952547275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45781049a0 0x7f45781089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.073 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.055+0000 7f457ce00640 1 --2- 172.21.15.111:0/952547275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4578103ff0 0x7f45781043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.055+0000 7f457ce00640 1 -- 172.21.15.111:0/952547275 >> 172.21.15.111:0/952547275 conn(0x7f45780ffa90 msgr2=0x7f4578101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:01.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.055+0000 7f457ce00640 1 -- 172.21.15.111:0/952547275 shutdown_connections 2024-09-15T22:45:01.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.056+0000 7f457ce00640 1 -- 172.21.15.111:0/952547275 wait complete. 2024-09-15T22:45:01.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.056+0000 7f457ce00640 1 Processor -- start 2024-09-15T22:45:01.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.056+0000 7f457ce00640 1 -- start start 2024-09-15T22:45:01.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.057+0000 7f457ce00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4578103ff0 0x7f457819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:01.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.057+0000 7f457ce00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45781049a0 0x7f457819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:01.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.057+0000 7f457ce00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4578109120 0x7f457819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:01.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.057+0000 7f457ce00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4578114130 con 0x7f4578103ff0 2024-09-15T22:45:01.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.057+0000 7f457ce00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4578113fb0 con 0x7f45781049a0 2024-09-15T22:45:01.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.057+0000 7f457ce00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f45781142b0 con 0x7f4578109120 2024-09-15T22:45:01.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.057+0000 7f4575e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4578103ff0 0x7f457819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:01.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.057+0000 7f4575e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4578103ff0 0x7f457819a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58600/0 (socket says 172.21.15.111:58600) 2024-09-15T22:45:01.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.057+0000 7f4575e00640 1 -- 172.21.15.111:0/115752398 learned_addr learned my addr 172.21.15.111:0/115752398 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:01.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.057+0000 7f4575400640 1 --2- 172.21.15.111:0/115752398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45781049a0 0x7f457819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:01.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.057+0000 7f4576800640 1 --2- 172.21.15.111:0/115752398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4578109120 0x7f457819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:01.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.057+0000 7f4575e00640 1 -- 172.21.15.111:0/115752398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4578109120 msgr2=0x7f457819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:01.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.057+0000 7f4575e00640 1 --2- 172.21.15.111:0/115752398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4578109120 0x7f457819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.057+0000 7f4575e00640 1 -- 172.21.15.111:0/115752398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45781049a0 msgr2=0x7f457819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:01.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.057+0000 7f4575e00640 1 --2- 172.21.15.111:0/115752398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45781049a0 0x7f457819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.057+0000 7f4575e00640 1 -- 172.21.15.111:0/115752398 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457819f5c0 con 0x7f4578103ff0 2024-09-15T22:45:01.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.058+0000 7f4575e00640 1 --2- 172.21.15.111:0/115752398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4578103ff0 0x7f457819a610 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f456800e9f0 tx=0x7f456800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:01.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.058+0000 7f456ea00640 1 -- 172.21.15.111:0/115752398 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456800cd80 con 0x7f4578103ff0 2024-09-15T22:45:01.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.058+0000 7f456ea00640 1 -- 172.21.15.111:0/115752398 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4568010910 con 0x7f4578103ff0 2024-09-15T22:45:01.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.058+0000 7f456ea00640 1 -- 172.21.15.111:0/115752398 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4568018990 con 0x7f4578103ff0 2024-09-15T22:45:01.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.058+0000 7f457ce00640 1 -- 172.21.15.111:0/115752398 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f457819f8b0 con 0x7f4578103ff0 2024-09-15T22:45:01.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.058+0000 7f457ce00640 1 -- 172.21.15.111:0/115752398 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45781a7190 con 0x7f4578103ff0 2024-09-15T22:45:01.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.059+0000 7f457ce00640 1 -- 172.21.15.111:0/115752398 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45781043d0 con 0x7f4578103ff0 2024-09-15T22:45:01.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.061+0000 7f456ea00640 1 -- 172.21.15.111:0/115752398 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f4568010ac0 con 0x7f4578103ff0 2024-09-15T22:45:01.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.062+0000 7f456ea00640 1 --2- 172.21.15.111:0/115752398 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f455c0a3350 0x7f455c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:01.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.062+0000 7f456ea00640 1 -- 172.21.15.111:0/115752398 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f45680d0130 con 0x7f4578103ff0 2024-09-15T22:45:01.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.063+0000 7f4575400640 1 --2- 172.21.15.111:0/115752398 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f455c0a3350 0x7f455c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:01.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.063+0000 7f4575400640 1 --2- 172.21.15.111:0/115752398 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f455c0a3350 0x7f455c0a5810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f45640095f0 tx=0x7f4564005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:01.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.065+0000 7f456ea00640 1 -- 172.21.15.111:0/115752398 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f456809ccf0 con 0x7f4578103ff0 2024-09-15T22:45:01.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.241+0000 7f457ce00640 1 -- 172.21.15.111:0/115752398 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45781a7390 con 0x7f4578103ff0 2024-09-15T22:45:01.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.242+0000 7f456ea00640 1 -- 172.21.15.111:0/115752398 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22701 (secure 0 0 0) 0x7f456809c5f0 con 0x7f4578103ff0 2024-09-15T22:45:01.244 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:45:01.244 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":88,"fsid":"b7614d3c-73b2-11ef-bceb-c7b262605968","created":"2024-09-15T22:35:52.571473+0000","modified":"2024-09-15T22:44:51.627977+0000","last_up_change":"2024-09-15T22:43:19.567274+0000","last_in_change":"2024-09-15T22:42:56.994246+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:39:18.954207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:44:12.846418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:44:15.685946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04f39604-be0e-4261-b316-67fa087c6666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6803","nonce":1386823446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6805","nonce":1386823446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6809","nonce":1386823446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6807","nonce":1386823446}]},"public_addr":"172.21.15.111:6803/1386823446","cluster_addr":"172.21.15.111:6805/1386823446","heartbeat_back_addr":"172.21.15.111:6809/1386823446","heartbeat_front_addr":"172.21.15.111:6807/1386823446","state":["exists","up"]},{"osd":1,"uuid":"1bdbef6e-23fa-4618-a788-66e57741a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6801","nonce":2051512291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6803","nonce":2051512291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6807","nonce":2051512291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6805","nonce":2051512291}]},"public_addr":"172.21.15.121:6801/2051512291","cluster_addr":"172.21.15.121:6803/2051512291","heartbeat_back_addr":"172.21.15.121:6807/2051512291","heartbeat_front_addr":"172.21.15.121:6805/2051512291","state":["exists","up"]},{"osd":2,"uuid":"46735ccf-6d0e-4154-bb55-422929819bfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6801","nonce":93085639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6803","nonce":93085639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6807","nonce":93085639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6805","nonce":93085639}]},"public_addr":"172.21.15.192:6801/93085639","cluster_addr":"172.21.15.192:6803/93085639","heartbeat_back_addr":"172.21.15.192:6807/93085639","heartbeat_front_addr":"172.21.15.192:6805/93085639","state":["exists","up"]},{"osd":3,"uuid":"4b243af8-c958-40b1-ba24-9c7ace00c4a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6811","nonce":2578259041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6813","nonce":2578259041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6817","nonce":2578259041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6815","nonce":2578259041}]},"public_addr":"172.21.15.111:6811/2578259041","cluster_addr":"172.21.15.111:6813/2578259041","heartbeat_back_addr":"172.21.15.111:6817/2578259041","heartbeat_front_addr":"172.21.15.111:6815/2578259041","state":["exists","up"]},{"osd":4,"uuid":"49f27541-9cbf-48e0-a466-6bb0fa5410b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6809","nonce":2667595915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6811","nonce":2667595915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6815","nonce":2667595915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6813","nonce":2667595915}]},"public_addr":"172.21.15.121:6809/2667595915","cluster_addr":"172.21.15.121:6811/2667595915","heartbeat_back_addr":"172.21.15.121:6815/2667595915","heartbeat_front_addr":"172.21.15.121:6813/2667595915","state":["exists","up"]},{"osd":5,"uuid":"9290e769-5470-44d3-8d24-25093af34335","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6809","nonce":1958639856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6811","nonce":1958639856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6815","nonce":1958639856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6813","nonce":1958639856}]},"public_addr":"172.21.15.192:6809/1958639856","cluster_addr":"172.21.15.192:6811/1958639856","heartbeat_back_addr":"172.21.15.192:6815/1958639856","heartbeat_front_addr":"172.21.15.192:6813/1958639856","state":["exists","up"]},{"osd":6,"uuid":"18911b97-1c68-4e8f-92ec-35d2ff0b5f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6819","nonce":1371543706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6821","nonce":1371543706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6825","nonce":1371543706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6823","nonce":1371543706}]},"public_addr":"172.21.15.111:6819/1371543706","cluster_addr":"172.21.15.111:6821/1371543706","heartbeat_back_addr":"172.21.15.111:6825/1371543706","heartbeat_front_addr":"172.21.15.111:6823/1371543706","state":["exists","up"]},{"osd":7,"uuid":"470647c5-c00d-426b-a134-cca235902307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6817","nonce":3501842998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6819","nonce":3501842998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6823","nonce":3501842998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6821","nonce":3501842998}]},"public_addr":"172.21.15.121:6817/3501842998","cluster_addr":"172.21.15.121:6819/3501842998","heartbeat_back_addr":"172.21.15.121:6823/3501842998","heartbeat_front_addr":"172.21.15.121:6821/3501842998","state":["exists","up"]},{"osd":8,"uuid":"fa2d3dd5-a768-4ada-a036-4c596b76d654","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6817","nonce":2417771374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6819","nonce":2417771374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6823","nonce":2417771374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6821","nonce":2417771374}]},"public_addr":"172.21.15.192:6817/2417771374","cluster_addr":"172.21.15.192:6819/2417771374","heartbeat_back_addr":"172.21.15.192:6823/2417771374","heartbeat_front_addr":"172.21.15.192:6821/2417771374","state":["exists","up"]},{"osd":9,"uuid":"71b4f5fb-1c02-461b-a257-c6e48381d768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6827","nonce":3979119431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6829","nonce":3979119431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6833","nonce":3979119431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6831","nonce":3979119431}]},"public_addr":"172.21.15.111:6827/3979119431","cluster_addr":"172.21.15.111:6829/3979119431","heartbeat_back_addr":"172.21.15.111:6833/3979119431","heartbeat_front_addr":"172.21.15.111:6831/3979119431","state":["exists","up"]},{"osd":10,"uuid":"e6a39e77-fce3-4e22-bf66-a677c9f8b6bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6825","nonce":2897902637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6827","nonce":2897902637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6831","nonce":2897902637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6829","nonce":2897902637}]},"public_addr":"172.21.15.121:6825/2897902637","cluster_addr":"172.21.15.121:6827/2897902637","heartbeat_back_addr":"172.21.15.121:6831/2897902637","heartbeat_front_addr":"172.21.15.121:6829/2897902637","state":["exists","up"]},{"osd":11,"uuid":"411a7921-6717-4cef-b8df-d9e252edcc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6825","nonce":2431464812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6827","nonce":2431464812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6831","nonce":2431464812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6829","nonce":2431464812}]},"public_addr":"172.21.15.192:6825/2431464812","cluster_addr":"172.21.15.192:6827/2431464812","heartbeat_back_addr":"172.21.15.192:6831/2431464812","heartbeat_front_addr":"172.21.15.192:6829/2431464812","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:38:23.442715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:38:49.501124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:39:14.759077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:39:45.121397+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:40:11.137612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:40:36.278567+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:07.027190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:32.439215+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:57.058702+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:42:28.741846+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:42:53.395442+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:43:18.265067+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:6801/2107775445":"2024-09-16T22:36:50.357551+0000","172.21.15.111:6800/2107775445":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/2394746540":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/4140020843":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/2114354263":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/1054173642":"2024-09-16T22:36:22.652875+0000","172.21.15.111:0/3408238004":"2024-09-16T22:36:22.652875+0000","172.21.15.111:0/2879044772":"2024-09-16T22:36:22.652875+0000","172.21.15.111:6801/3350425104":"2024-09-16T22:36:22.652875+0000","172.21.15.111:6800/3350425104":"2024-09-16T22:36:22.652875+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:45:01.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.246+0000 7f457ce00640 1 -- 172.21.15.111:0/115752398 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f455c0a3350 msgr2=0x7f455c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:01.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.246+0000 7f457ce00640 1 --2- 172.21.15.111:0/115752398 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f455c0a3350 0x7f455c0a5810 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f45640095f0 tx=0x7f4564005f90 comp rx=0 tx=0).stop 2024-09-15T22:45:01.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.246+0000 7f457ce00640 1 -- 172.21.15.111:0/115752398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4578103ff0 msgr2=0x7f457819a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:01.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.246+0000 7f457ce00640 1 --2- 172.21.15.111:0/115752398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4578103ff0 0x7f457819a610 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f456800e9f0 tx=0x7f456800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.246+0000 7f457ce00640 1 -- 172.21.15.111:0/115752398 shutdown_connections 2024-09-15T22:45:01.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.246+0000 7f457ce00640 1 --2- 172.21.15.111:0/115752398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4578109120 0x7f457819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.246+0000 7f457ce00640 1 --2- 172.21.15.111:0/115752398 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f455c0a3350 0x7f455c0a5810 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.246+0000 7f457ce00640 1 --2- 172.21.15.111:0/115752398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45781049a0 0x7f457819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.246+0000 7f457ce00640 1 --2- 172.21.15.111:0/115752398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4578103ff0 0x7f457819a610 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:01.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.246+0000 7f457ce00640 1 -- 172.21.15.111:0/115752398 >> 172.21.15.111:0/115752398 conn(0x7f45780ffa90 msgr2=0x7f45781005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:01.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.246+0000 7f457ce00640 1 -- 172.21.15.111:0/115752398 shutdown_connections 2024-09-15T22:45:01.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:01.246+0000 7f457ce00640 1 -- 172.21.15.111:0/115752398 wait complete. 2024-09-15T22:45:01.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:01 smithi111 ceph-mon[30629]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 1.0 KiB/s wr, 4 op/s; 115 B/s, 0 objects/s recovering 2024-09-15T22:45:01.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:01 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/115752398' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:45:02.042 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-15T22:45:02.042 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-15T22:45:02.043 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-15T22:45:02.043 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-15T22:45:02.043 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-15T22:45:02.044 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-15T22:45:02.044 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-15T22:45:02.045 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-15T22:45:02.045 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-15T22:45:02.046 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-15T22:45:02.047 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-15T22:45:02.048 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-15T22:45:02.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:01 smithi192 ceph-mon[35427]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 1.0 KiB/s wr, 4 op/s; 115 B/s, 0 objects/s recovering 2024-09-15T22:45:02.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:01 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/115752398' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:45:02.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:01 smithi121 ceph-mon[34815]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 1.0 KiB/s wr, 4 op/s; 115 B/s, 0 objects/s recovering 2024-09-15T22:45:02.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:01 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/115752398' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:45:03.016 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:03.124 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:03.224 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:03.224 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:03.224 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:03.228 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:03.561 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:03.562 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:03.562 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:03.565 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:03.817 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:03 smithi111 ceph-mon[30629]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 945 B/s wr, 5 op/s; 104 B/s, 0 objects/s recovering 2024-09-15T22:45:03.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.926+0000 7f3884400640 1 -- 172.21.15.111:0/411356816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f387c074f40 msgr2=0x7f387c10de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:03.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.926+0000 7f3884400640 1 --2- 172.21.15.111:0/411356816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f387c074f40 0x7f387c10de80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3868009900 tx=0x7f386802a480 comp rx=0 tx=0).stop 2024-09-15T22:45:03.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.926+0000 7f3884400640 1 -- 172.21.15.111:0/411356816 shutdown_connections 2024-09-15T22:45:03.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.926+0000 7f3884400640 1 --2- 172.21.15.111:0/411356816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f387c10e3c0 0x7f387c110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:03.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.926+0000 7f3884400640 1 --2- 172.21.15.111:0/411356816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f387c074f40 0x7f387c10de80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:03.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.926+0000 7f3884400640 1 --2- 172.21.15.111:0/411356816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f387c074620 0x7f387c074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:03.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.926+0000 7f3884400640 1 -- 172.21.15.111:0/411356816 >> 172.21.15.111:0/411356816 conn(0x7f387c0fd8c0 msgr2=0x7f387c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:03.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.926+0000 7f3884400640 1 -- 172.21.15.111:0/411356816 shutdown_connections 2024-09-15T22:45:03.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.926+0000 7f3884400640 1 -- 172.21.15.111:0/411356816 wait complete. 2024-09-15T22:45:03.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.927+0000 7f3884400640 1 Processor -- start 2024-09-15T22:45:03.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.927+0000 7f3884400640 1 -- start start 2024-09-15T22:45:03.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.928+0000 7f3884400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f387c074620 0x7f387c19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:03.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.928+0000 7f3884400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f387c074f40 0x7f387c19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:03.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.928+0000 7f3884400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f387c10e3c0 0x7f387c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:03.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.928+0000 7f3884400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f387c114160 con 0x7f387c10e3c0 2024-09-15T22:45:03.934 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.928+0000 7f3884400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f387c113fe0 con 0x7f387c074620 2024-09-15T22:45:03.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.928+0000 7f3884400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f387c1142e0 con 0x7f387c074f40 2024-09-15T22:45:03.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.928+0000 7f3882400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f387c10e3c0 0x7f387c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:03.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.928+0000 7f3882400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f387c10e3c0 0x7f387c19efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58620/0 (socket says 172.21.15.111:58620) 2024-09-15T22:45:03.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.928+0000 7f3882400640 1 -- 172.21.15.111:0/1482526293 learned_addr learned my addr 172.21.15.111:0/1482526293 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:03.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.928+0000 7f3881a00640 1 --2- 172.21.15.111:0/1482526293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f387c074620 0x7f387c19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:03.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.928+0000 7f3881000640 1 --2- 172.21.15.111:0/1482526293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f387c074f40 0x7f387c19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:03.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.928+0000 7f3882400640 1 -- 172.21.15.111:0/1482526293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f387c074f40 msgr2=0x7f387c19ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:03.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.928+0000 7f3882400640 1 --2- 172.21.15.111:0/1482526293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f387c074f40 0x7f387c19ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:03.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.928+0000 7f3882400640 1 -- 172.21.15.111:0/1482526293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f387c074620 msgr2=0x7f387c19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:03.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.928+0000 7f3882400640 1 --2- 172.21.15.111:0/1482526293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f387c074620 0x7f387c19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:03.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.928+0000 7f3882400640 1 -- 172.21.15.111:0/1482526293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f387c19f6b0 con 0x7f387c10e3c0 2024-09-15T22:45:03.935 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.929+0000 7f3882400640 1 --2- 172.21.15.111:0/1482526293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f387c10e3c0 0x7f387c19efd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f387400eb10 tx=0x7f387400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:03.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.929+0000 7f387a000640 1 -- 172.21.15.111:0/1482526293 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3874004b20 con 0x7f387c10e3c0 2024-09-15T22:45:03.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.929+0000 7f3884400640 1 -- 172.21.15.111:0/1482526293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f387c19f9a0 con 0x7f387c10e3c0 2024-09-15T22:45:03.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.929+0000 7f387a000640 1 -- 172.21.15.111:0/1482526293 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3874004cc0 con 0x7f387c10e3c0 2024-09-15T22:45:03.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.929+0000 7f387a000640 1 -- 172.21.15.111:0/1482526293 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3874010700 con 0x7f387c10e3c0 2024-09-15T22:45:03.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.929+0000 7f3884400640 1 -- 172.21.15.111:0/1482526293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f387c1a7280 con 0x7f387c10e3c0 2024-09-15T22:45:03.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.930+0000 7f3884400640 1 -- 172.21.15.111:0/1482526293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f387c062900 con 0x7f387c10e3c0 2024-09-15T22:45:03.936 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.932+0000 7f387a000640 1 -- 172.21.15.111:0/1482526293 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f3874005020 con 0x7f387c10e3c0 2024-09-15T22:45:03.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.933+0000 7f387a000640 1 --2- 172.21.15.111:0/1482526293 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f38640a32d0 0x7f38640a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:03.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.933+0000 7f387a000640 1 -- 172.21.15.111:0/1482526293 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f38740d0090 con 0x7f387c10e3c0 2024-09-15T22:45:03.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.933+0000 7f3881a00640 1 --2- 172.21.15.111:0/1482526293 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f38640a32d0 0x7f38640a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:03.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.934+0000 7f387a000640 1 --2- 172.21.15.111:0/1482526293 >> [v2:172.21.15.121:6808/2667595915,v1:172.21.15.121:6809/2667595915] conn(0x7f38640acd80 0x7f38640af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:03.937 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.934+0000 7f387a000640 1 -- 172.21.15.111:0/1482526293 --> [v2:172.21.15.121:6808/2667595915,v1:172.21.15.121:6809/2667595915] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3874019e50 con 0x7f38640acd80 2024-09-15T22:45:03.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.934+0000 7f387a000640 1 -- 172.21.15.111:0/1482526293 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f38740d7050 con 0x7f387c10e3c0 2024-09-15T22:45:03.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.934+0000 7f3881a00640 1 --2- 172.21.15.111:0/1482526293 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f38640a32d0 0x7f38640a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3870009bd0 tx=0x7f3870005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:03.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.934+0000 7f3881000640 1 --2- 172.21.15.111:0/1482526293 >> [v2:172.21.15.121:6808/2667595915,v1:172.21.15.121:6809/2667595915] conn(0x7f38640acd80 0x7f38640af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:03.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.935+0000 7f3881000640 1 --2- 172.21.15.111:0/1482526293 >> [v2:172.21.15.121:6808/2667595915,v1:172.21.15.121:6809/2667595915] conn(0x7f38640acd80 0x7f38640af220 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:03.938 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.935+0000 7f387a000640 1 -- 172.21.15.111:0/1482526293 <== osd.4 v2:172.21.15.121:6808/2667595915 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3874019e50 con 0x7f38640acd80 2024-09-15T22:45:03.951 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.950+0000 7f3884400640 1 -- 172.21.15.111:0/1482526293 --> [v2:172.21.15.121:6808/2667595915,v1:172.21.15.121:6809/2667595915] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f387c062d30 con 0x7f38640acd80 2024-09-15T22:45:03.952 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.951+0000 7f387a000640 1 -- 172.21.15.111:0/1482526293 <== osd.4 v2:172.21.15.121:6808/2667595915 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f387c062d30 con 0x7f38640acd80 2024-09-15T22:45:03.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.951+0000 7f3884400640 1 -- 172.21.15.111:0/1482526293 >> [v2:172.21.15.121:6808/2667595915,v1:172.21.15.121:6809/2667595915] conn(0x7f38640acd80 msgr2=0x7f38640af220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:03.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.951+0000 7f3884400640 1 --2- 172.21.15.111:0/1482526293 >> [v2:172.21.15.121:6808/2667595915,v1:172.21.15.121:6809/2667595915] conn(0x7f38640acd80 0x7f38640af220 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:03.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.951+0000 7f3884400640 1 -- 172.21.15.111:0/1482526293 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f38640a32d0 msgr2=0x7f38640a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:03.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.951+0000 7f3884400640 1 --2- 172.21.15.111:0/1482526293 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f38640a32d0 0x7f38640a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3870009bd0 tx=0x7f3870005f90 comp rx=0 tx=0).stop 2024-09-15T22:45:03.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.951+0000 7f3884400640 1 -- 172.21.15.111:0/1482526293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f387c10e3c0 msgr2=0x7f387c19efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:03.953 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.951+0000 7f3884400640 1 --2- 172.21.15.111:0/1482526293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f387c10e3c0 0x7f387c19efd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f387400eb10 tx=0x7f387400c2d0 comp rx=0 tx=0).stop 2024-09-15T22:45:03.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.951+0000 7f3882400640 1 -- 172.21.15.111:0/1482526293 reap_dead start 2024-09-15T22:45:03.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.951+0000 7f3884400640 1 -- 172.21.15.111:0/1482526293 shutdown_connections 2024-09-15T22:45:03.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.951+0000 7f3884400640 1 -- 172.21.15.111:0/1482526293 >> 172.21.15.111:0/1482526293 conn(0x7f387c0fd8c0 msgr2=0x7f387c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:03.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.951+0000 7f3884400640 1 -- 172.21.15.111:0/1482526293 shutdown_connections 2024-09-15T22:45:03.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:03.951+0000 7f3884400640 1 -- 172.21.15.111:0/1482526293 wait complete. 2024-09-15T22:45:04.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:03 smithi192 ceph-mon[35427]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 945 B/s wr, 5 op/s; 104 B/s, 0 objects/s recovering 2024-09-15T22:45:04.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:03 smithi121 ceph-mon[34815]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 945 B/s wr, 5 op/s; 104 B/s, 0 objects/s recovering 2024-09-15T22:45:04.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.211+0000 7f9e30a00640 1 -- 172.21.15.111:0/1355317920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e2c1049c0 msgr2=0x7f9e2c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:04.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.211+0000 7f9e30a00640 1 --2- 172.21.15.111:0/1355317920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e2c1049c0 0x7f9e2c108a10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9e18009900 tx=0x7f9e1802a3f0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.211+0000 7f9e30a00640 1 -- 172.21.15.111:0/1355317920 shutdown_connections 2024-09-15T22:45:04.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.211+0000 7f9e30a00640 1 --2- 172.21.15.111:0/1355317920 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e2c109140 0x7f9e2c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.211+0000 7f9e30a00640 1 --2- 172.21.15.111:0/1355317920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e2c1049c0 0x7f9e2c108a10 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.211+0000 7f9e30a00640 1 --2- 172.21.15.111:0/1355317920 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e2c104010 0x7f9e2c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.211+0000 7f9e30a00640 1 -- 172.21.15.111:0/1355317920 >> 172.21.15.111:0/1355317920 conn(0x7f9e2c0ffa50 msgr2=0x7f9e2c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:04.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.211+0000 7f9e30a00640 1 -- 172.21.15.111:0/1355317920 shutdown_connections 2024-09-15T22:45:04.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.211+0000 7f9e30a00640 1 -- 172.21.15.111:0/1355317920 wait complete. 2024-09-15T22:45:04.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.213+0000 7f9e30a00640 1 Processor -- start 2024-09-15T22:45:04.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.213+0000 7f9e30a00640 1 -- start start 2024-09-15T22:45:04.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.213+0000 7f9e30a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e2c104010 0x7f9e2c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:04.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.213+0000 7f9e30a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e2c1049c0 0x7f9e2c19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:04.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.213+0000 7f9e30a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e2c109140 0x7f9e2c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:04.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.213+0000 7f9e30a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c114190 con 0x7f9e2c104010 2024-09-15T22:45:04.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.213+0000 7f9e30a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c114010 con 0x7f9e2c1049c0 2024-09-15T22:45:04.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.213+0000 7f9e30a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2c114310 con 0x7f9e2c109140 2024-09-15T22:45:04.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.214+0000 7f9e29e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e2c104010 0x7f9e2c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:04.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.214+0000 7f9e29e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e2c104010 0x7f9e2c19a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58628/0 (socket says 172.21.15.111:58628) 2024-09-15T22:45:04.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.214+0000 7f9e29400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e2c1049c0 0x7f9e2c19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:04.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.214+0000 7f9e29e00640 1 -- 172.21.15.111:0/658512383 learned_addr learned my addr 172.21.15.111:0/658512383 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:04.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.214+0000 7f9e2a800640 1 --2- 172.21.15.111:0/658512383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e2c109140 0x7f9e2c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:04.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.214+0000 7f9e29400640 1 -- 172.21.15.111:0/658512383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e2c109140 msgr2=0x7f9e2c19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:04.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.214+0000 7f9e29400640 1 --2- 172.21.15.111:0/658512383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e2c109140 0x7f9e2c19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.214+0000 7f9e29400640 1 -- 172.21.15.111:0/658512383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e2c104010 msgr2=0x7f9e2c19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:04.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.214+0000 7f9e29400640 1 --2- 172.21.15.111:0/658512383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e2c104010 0x7f9e2c19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.214+0000 7f9e29400640 1 -- 172.21.15.111:0/658512383 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e2c19f6a0 con 0x7f9e2c1049c0 2024-09-15T22:45:04.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.214+0000 7f9e29400640 1 --2- 172.21.15.111:0/658512383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e2c1049c0 0x7f9e2c19ac30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9e1802a910 tx=0x7f9e18003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:04.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.215+0000 7f9e22a00640 1 -- 172.21.15.111:0/658512383 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1803f070 con 0x7f9e2c1049c0 2024-09-15T22:45:04.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.215+0000 7f9e30a00640 1 -- 172.21.15.111:0/658512383 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e2c19f930 con 0x7f9e2c1049c0 2024-09-15T22:45:04.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.215+0000 7f9e30a00640 1 -- 172.21.15.111:0/658512383 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e2c1a7270 con 0x7f9e2c1049c0 2024-09-15T22:45:04.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.215+0000 7f9e22a00640 1 -- 172.21.15.111:0/658512383 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e18031c90 con 0x7f9e2c1049c0 2024-09-15T22:45:04.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.215+0000 7f9e22a00640 1 -- 172.21.15.111:0/658512383 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e18035030 con 0x7f9e2c1049c0 2024-09-15T22:45:04.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.216+0000 7f9e30a00640 1 -- 172.21.15.111:0/658512383 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9e0c001030 con 0x7f9e2c1049c0 2024-09-15T22:45:04.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.218+0000 7f9e22a00640 1 -- 172.21.15.111:0/658512383 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f9e18042070 con 0x7f9e2c1049c0 2024-09-15T22:45:04.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.220+0000 7f9e22a00640 1 --2- 172.21.15.111:0/658512383 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9e100a3350 0x7f9e100a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:04.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.220+0000 7f9e22a00640 1 -- 172.21.15.111:0/658512383 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f9e180f0370 con 0x7f9e2c1049c0 2024-09-15T22:45:04.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.221+0000 7f9e29e00640 1 --2- 172.21.15.111:0/658512383 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9e100a3350 0x7f9e100a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:04.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.221+0000 7f9e22a00640 1 --2- 172.21.15.111:0/658512383 >> [v2:172.21.15.111:6818/1371543706,v1:172.21.15.111:6819/1371543706] conn(0x7f9e100ace00 0x7f9e100af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:04.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.221+0000 7f9e22a00640 1 -- 172.21.15.111:0/658512383 --> [v2:172.21.15.111:6818/1371543706,v1:172.21.15.111:6819/1371543706] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9e100af970 con 0x7f9e100ace00 2024-09-15T22:45:04.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.221+0000 7f9e22a00640 1 -- 172.21.15.111:0/658512383 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f9e180f0720 con 0x7f9e2c1049c0 2024-09-15T22:45:04.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.221+0000 7f9e2a800640 1 --2- 172.21.15.111:0/658512383 >> [v2:172.21.15.111:6818/1371543706,v1:172.21.15.111:6819/1371543706] conn(0x7f9e100ace00 0x7f9e100af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:04.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.221+0000 7f9e29e00640 1 --2- 172.21.15.111:0/658512383 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9e100a3350 0x7f9e100a5810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c009c90 tx=0x7f9e1c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:04.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.222+0000 7f9e2a800640 1 --2- 172.21.15.111:0/658512383 >> [v2:172.21.15.111:6818/1371543706,v1:172.21.15.111:6819/1371543706] conn(0x7f9e100ace00 0x7f9e100af2a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:04.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.223+0000 7f9e22a00640 1 -- 172.21.15.111:0/658512383 <== osd.6 v2:172.21.15.111:6818/1371543706 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9e100af970 con 0x7f9e100ace00 2024-09-15T22:45:04.239 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.237+0000 7f9e30a00640 1 -- 172.21.15.111:0/658512383 --> [v2:172.21.15.111:6818/1371543706,v1:172.21.15.111:6819/1371543706] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9e0c002dd0 con 0x7f9e100ace00 2024-09-15T22:45:04.240 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.238+0000 7f9e22a00640 1 -- 172.21.15.111:0/658512383 <== osd.6 v2:172.21.15.111:6818/1371543706 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9e0c002dd0 con 0x7f9e100ace00 2024-09-15T22:45:04.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.238+0000 7f9e30a00640 1 -- 172.21.15.111:0/658512383 >> [v2:172.21.15.111:6818/1371543706,v1:172.21.15.111:6819/1371543706] conn(0x7f9e100ace00 msgr2=0x7f9e100af2a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:04.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.238+0000 7f9e30a00640 1 --2- 172.21.15.111:0/658512383 >> [v2:172.21.15.111:6818/1371543706,v1:172.21.15.111:6819/1371543706] conn(0x7f9e100ace00 0x7f9e100af2a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:04.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.238+0000 7f9e30a00640 1 -- 172.21.15.111:0/658512383 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9e100a3350 msgr2=0x7f9e100a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:04.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.238+0000 7f9e30a00640 1 --2- 172.21.15.111:0/658512383 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9e100a3350 0x7f9e100a5810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c009c90 tx=0x7f9e1c005f90 comp rx=0 tx=0).stop 2024-09-15T22:45:04.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.238+0000 7f9e30a00640 1 -- 172.21.15.111:0/658512383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e2c1049c0 msgr2=0x7f9e2c19ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:04.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.238+0000 7f9e30a00640 1 --2- 172.21.15.111:0/658512383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e2c1049c0 0x7f9e2c19ac30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9e1802a910 tx=0x7f9e18003b20 comp rx=0 tx=0).stop 2024-09-15T22:45:04.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.238+0000 7f9e2a800640 1 -- 172.21.15.111:0/658512383 reap_dead start 2024-09-15T22:45:04.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.238+0000 7f9e30a00640 1 -- 172.21.15.111:0/658512383 shutdown_connections 2024-09-15T22:45:04.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.238+0000 7f9e30a00640 1 -- 172.21.15.111:0/658512383 >> 172.21.15.111:0/658512383 conn(0x7f9e2c0ffa50 msgr2=0x7f9e2c100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:04.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.238+0000 7f9e30a00640 1 -- 172.21.15.111:0/658512383 shutdown_connections 2024-09-15T22:45:04.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:04.238+0000 7f9e30a00640 1 -- 172.21.15.111:0/658512383 wait complete. 2024-09-15T22:45:05.361 INFO:teuthology.orchestra.run.smithi111.stdout:137438953532 2024-09-15T22:45:05.362 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-15T22:45:05.889 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:05 smithi111 ceph-mon[30629]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 6 op/s; 94 B/s, 0 objects/s recovering 2024-09-15T22:45:05.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:05 smithi121 ceph-mon[34815]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 6 op/s; 94 B/s, 0 objects/s recovering 2024-09-15T22:45:06.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:05 smithi192 ceph-mon[35427]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 6 op/s; 94 B/s, 0 objects/s recovering 2024-09-15T22:45:06.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.099+0000 7f980ca00640 1 -- 172.21.15.111:0/2983995524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9808106f10 msgr2=0x7f980810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:06.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.099+0000 7f980ca00640 1 --2- 172.21.15.111:0/2983995524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9808106f10 0x7f980810ea90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9800009ca0 tx=0x7f980002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.099+0000 7f980ca00640 1 -- 172.21.15.111:0/2983995524 shutdown_connections 2024-09-15T22:45:06.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.099+0000 7f980ca00640 1 --2- 172.21.15.111:0/2983995524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9808106f10 0x7f980810ea90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.099+0000 7f980ca00640 1 --2- 172.21.15.111:0/2983995524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9808102790 0x7f98081067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.099+0000 7f980ca00640 1 --2- 172.21.15.111:0/2983995524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9808101de0 0x7f98081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.099+0000 7f980ca00640 1 -- 172.21.15.111:0/2983995524 >> 172.21.15.111:0/2983995524 conn(0x7f98080fd8c0 msgr2=0x7f98080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:06.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.099+0000 7f980ca00640 1 -- 172.21.15.111:0/2983995524 shutdown_connections 2024-09-15T22:45:06.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.099+0000 7f980ca00640 1 -- 172.21.15.111:0/2983995524 wait complete. 2024-09-15T22:45:06.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.100+0000 7f980ca00640 1 Processor -- start 2024-09-15T22:45:06.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.100+0000 7f980ca00640 1 -- start start 2024-09-15T22:45:06.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.100+0000 7f980ca00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9808101de0 0x7f980819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:06.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.100+0000 7f980ca00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9808102790 0x7f980819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:06.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.100+0000 7f980ca00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9808106f10 0x7f980819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:06.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.100+0000 7f980ca00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9808111f70 con 0x7f9808106f10 2024-09-15T22:45:06.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.100+0000 7f980ca00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9808111df0 con 0x7f9808102790 2024-09-15T22:45:06.103 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.100+0000 7f980ca00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f98081120f0 con 0x7f9808101de0 2024-09-15T22:45:06.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.101+0000 7f9805e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9808101de0 0x7f980819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:06.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.101+0000 7f9805e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9808101de0 0x7f980819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:47844/0 (socket says 172.21.15.111:47844) 2024-09-15T22:45:06.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.101+0000 7f9805e00640 1 -- 172.21.15.111:0/2257691977 learned_addr learned my addr 172.21.15.111:0/2257691977 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:06.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.101+0000 7f9806800640 1 --2- 172.21.15.111:0/2257691977 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9808106f10 0x7f980819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:06.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.101+0000 7f97fd400640 1 --2- 172.21.15.111:0/2257691977 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9808102790 0x7f980819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:06.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.101+0000 7f9805e00640 1 -- 172.21.15.111:0/2257691977 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9808102790 msgr2=0x7f980819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:06.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.101+0000 7f9805e00640 1 --2- 172.21.15.111:0/2257691977 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9808102790 0x7f980819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.101+0000 7f9805e00640 1 -- 172.21.15.111:0/2257691977 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9808106f10 msgr2=0x7f980819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:06.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.101+0000 7f9805e00640 1 --2- 172.21.15.111:0/2257691977 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9808106f10 0x7f980819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.101+0000 7f9805e00640 1 -- 172.21.15.111:0/2257691977 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f980819f5a0 con 0x7f9808101de0 2024-09-15T22:45:06.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.101+0000 7f9805e00640 1 --2- 172.21.15.111:0/2257691977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9808101de0 0x7f980819a5f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f97f800e9f0 tx=0x7f97f800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:06.104 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.102+0000 7f97ff400640 1 -- 172.21.15.111:0/2257691977 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f800cd80 con 0x7f9808101de0 2024-09-15T22:45:06.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.102+0000 7f980ca00640 1 -- 172.21.15.111:0/2257691977 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f980819f890 con 0x7f9808101de0 2024-09-15T22:45:06.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.102+0000 7f97ff400640 1 -- 172.21.15.111:0/2257691977 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97f8005500 con 0x7f9808101de0 2024-09-15T22:45:06.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.102+0000 7f97ff400640 1 -- 172.21.15.111:0/2257691977 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f8010690 con 0x7f9808101de0 2024-09-15T22:45:06.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.102+0000 7f980ca00640 1 -- 172.21.15.111:0/2257691977 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98081a7170 con 0x7f9808101de0 2024-09-15T22:45:06.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.103+0000 7f980ca00640 1 -- 172.21.15.111:0/2257691977 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f980810e150 con 0x7f9808101de0 2024-09-15T22:45:06.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.105+0000 7f97ff400640 1 -- 172.21.15.111:0/2257691977 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f97f80046c0 con 0x7f9808101de0 2024-09-15T22:45:06.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.106+0000 7f97ff400640 1 --2- 172.21.15.111:0/2257691977 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f97ec0a32d0 0x7f97ec0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:06.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.106+0000 7f97ff400640 1 -- 172.21.15.111:0/2257691977 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f97f80d02e0 con 0x7f9808101de0 2024-09-15T22:45:06.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.106+0000 7f97fd400640 1 --2- 172.21.15.111:0/2257691977 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f97ec0a32d0 0x7f97ec0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:06.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.107+0000 7f97ff400640 1 --2- 172.21.15.111:0/2257691977 >> [v2:172.21.15.111:6802/1386823446,v1:172.21.15.111:6803/1386823446] conn(0x7f97ec0acd80 0x7f97ec0af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:06.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.107+0000 7f97ff400640 1 -- 172.21.15.111:0/2257691977 --> [v2:172.21.15.111:6802/1386823446,v1:172.21.15.111:6803/1386823446] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f97f8005350 con 0x7f97ec0acd80 2024-09-15T22:45:06.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.107+0000 7f97ff400640 1 -- 172.21.15.111:0/2257691977 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f97f80d0690 con 0x7f9808101de0 2024-09-15T22:45:06.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.107+0000 7f9806800640 1 --2- 172.21.15.111:0/2257691977 >> [v2:172.21.15.111:6802/1386823446,v1:172.21.15.111:6803/1386823446] conn(0x7f97ec0acd80 0x7f97ec0af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:06.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.107+0000 7f97fd400640 1 --2- 172.21.15.111:0/2257691977 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f97ec0a32d0 0x7f97ec0a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9808076480 tx=0x7f97f000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:06.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.107+0000 7f9806800640 1 --2- 172.21.15.111:0/2257691977 >> [v2:172.21.15.111:6802/1386823446,v1:172.21.15.111:6803/1386823446] conn(0x7f97ec0acd80 0x7f97ec0af220 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:06.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.108+0000 7f97ff400640 1 -- 172.21.15.111:0/2257691977 <== osd.0 v2:172.21.15.111:6802/1386823446 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f97f8005350 con 0x7f97ec0acd80 2024-09-15T22:45:06.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.122+0000 7f980ca00640 1 -- 172.21.15.111:0/2257691977 --> [v2:172.21.15.111:6802/1386823446,v1:172.21.15.111:6803/1386823446] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f98081032c0 con 0x7f97ec0acd80 2024-09-15T22:45:06.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.123+0000 7f97ff400640 1 -- 172.21.15.111:0/2257691977 <== osd.0 v2:172.21.15.111:6802/1386823446 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f98081032c0 con 0x7f97ec0acd80 2024-09-15T22:45:06.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.123+0000 7f980ca00640 1 -- 172.21.15.111:0/2257691977 >> [v2:172.21.15.111:6802/1386823446,v1:172.21.15.111:6803/1386823446] conn(0x7f97ec0acd80 msgr2=0x7f97ec0af220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:06.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.123+0000 7f980ca00640 1 --2- 172.21.15.111:0/2257691977 >> [v2:172.21.15.111:6802/1386823446,v1:172.21.15.111:6803/1386823446] conn(0x7f97ec0acd80 0x7f97ec0af220 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.123+0000 7f980ca00640 1 -- 172.21.15.111:0/2257691977 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f97ec0a32d0 msgr2=0x7f97ec0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:06.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.123+0000 7f980ca00640 1 --2- 172.21.15.111:0/2257691977 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f97ec0a32d0 0x7f97ec0a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9808076480 tx=0x7f97f000b410 comp rx=0 tx=0).stop 2024-09-15T22:45:06.125 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.123+0000 7f980ca00640 1 -- 172.21.15.111:0/2257691977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9808101de0 msgr2=0x7f980819a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:06.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.123+0000 7f980ca00640 1 --2- 172.21.15.111:0/2257691977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9808101de0 0x7f980819a5f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f97f800e9f0 tx=0x7f97f800edf0 comp rx=0 tx=0).stop 2024-09-15T22:45:06.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.123+0000 7f9806800640 1 -- 172.21.15.111:0/2257691977 reap_dead start 2024-09-15T22:45:06.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.123+0000 7f980ca00640 1 -- 172.21.15.111:0/2257691977 shutdown_connections 2024-09-15T22:45:06.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.123+0000 7f980ca00640 1 -- 172.21.15.111:0/2257691977 >> 172.21.15.111:0/2257691977 conn(0x7f98080fd8c0 msgr2=0x7f98080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:06.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.124+0000 7f980ca00640 1 -- 172.21.15.111:0/2257691977 shutdown_connections 2024-09-15T22:45:06.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:06.124+0000 7f980ca00640 1 -- 172.21.15.111:0/2257691977 wait complete. 2024-09-15T22:45:07.402 INFO:teuthology.orchestra.run.smithi111.stdout:34359738450 2024-09-15T22:45:07.403 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-15T22:45:07.514 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:07.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.858+0000 7fe177a00640 1 -- 172.21.15.111:0/1625653461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe170074f40 msgr2=0x7fe17010de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:07.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.858+0000 7fe177a00640 1 --2- 172.21.15.111:0/1625653461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe170074f40 0x7fe17010de80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe158009900 tx=0x7fe15802a480 comp rx=0 tx=0).stop 2024-09-15T22:45:07.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.858+0000 7fe177a00640 1 -- 172.21.15.111:0/1625653461 shutdown_connections 2024-09-15T22:45:07.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.858+0000 7fe177a00640 1 --2- 172.21.15.111:0/1625653461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe17010e3c0 0x7fe170110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:07.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.858+0000 7fe177a00640 1 --2- 172.21.15.111:0/1625653461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe170074f40 0x7fe17010de80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:07.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.858+0000 7fe177a00640 1 --2- 172.21.15.111:0/1625653461 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe170074620 0x7fe170074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:07.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.858+0000 7fe177a00640 1 -- 172.21.15.111:0/1625653461 >> 172.21.15.111:0/1625653461 conn(0x7fe1700fd8c0 msgr2=0x7fe1700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:07.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.858+0000 7fe177a00640 1 -- 172.21.15.111:0/1625653461 shutdown_connections 2024-09-15T22:45:07.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.859+0000 7fe177a00640 1 -- 172.21.15.111:0/1625653461 wait complete. 2024-09-15T22:45:07.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.859+0000 7fe177a00640 1 Processor -- start 2024-09-15T22:45:07.862 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.859+0000 7fe177a00640 1 -- start start 2024-09-15T22:45:07.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.860+0000 7fe177a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe170074620 0x7fe17019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:07.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.860+0000 7fe177a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe170074f40 0x7fe17019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:07.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.860+0000 7fe177a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe17010e3c0 0x7fe17019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:07.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.860+0000 7fe177a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe170114190 con 0x7fe170074f40 2024-09-15T22:45:07.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.860+0000 7fe177a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe170114010 con 0x7fe170074620 2024-09-15T22:45:07.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.860+0000 7fe177a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe170114310 con 0x7fe17010e3c0 2024-09-15T22:45:07.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.860+0000 7fe167e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe170074f40 0x7fe17019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:07.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.860+0000 7fe167e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe170074f40 0x7fe17019ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58662/0 (socket says 172.21.15.111:58662) 2024-09-15T22:45:07.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.860+0000 7fe175000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe170074620 0x7fe17019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:07.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.860+0000 7fe167e00640 1 -- 172.21.15.111:0/1856966355 learned_addr learned my addr 172.21.15.111:0/1856966355 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:07.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.860+0000 7fe175a00640 1 --2- 172.21.15.111:0/1856966355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe17010e3c0 0x7fe17019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:07.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.861+0000 7fe175000640 1 -- 172.21.15.111:0/1856966355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe17010e3c0 msgr2=0x7fe17019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:07.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.861+0000 7fe175000640 1 --2- 172.21.15.111:0/1856966355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe17010e3c0 0x7fe17019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:07.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.861+0000 7fe175000640 1 -- 172.21.15.111:0/1856966355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe170074f40 msgr2=0x7fe17019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:07.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.861+0000 7fe175000640 1 --2- 172.21.15.111:0/1856966355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe170074f40 0x7fe17019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:07.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.861+0000 7fe175000640 1 -- 172.21.15.111:0/1856966355 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe17019f6c0 con 0x7fe170074620 2024-09-15T22:45:07.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.861+0000 7fe167e00640 1 --2- 172.21.15.111:0/1856966355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe170074f40 0x7fe17019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:07.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.861+0000 7fe175000640 1 --2- 172.21.15.111:0/1856966355 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe170074620 0x7fe17019a710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe16000e9f0 tx=0x7fe16000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:07.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.861+0000 7fe16e000640 1 -- 172.21.15.111:0/1856966355 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16000ccc0 con 0x7fe170074620 2024-09-15T22:45:07.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.861+0000 7fe16e000640 1 -- 172.21.15.111:0/1856966355 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe160010910 con 0x7fe170074620 2024-09-15T22:45:07.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.861+0000 7fe16e000640 1 -- 172.21.15.111:0/1856966355 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160018960 con 0x7fe170074620 2024-09-15T22:45:07.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.861+0000 7fe177a00640 1 -- 172.21.15.111:0/1856966355 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe17019f9b0 con 0x7fe170074620 2024-09-15T22:45:07.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.862+0000 7fe177a00640 1 -- 172.21.15.111:0/1856966355 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1701a7290 con 0x7fe170074620 2024-09-15T22:45:07.865 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.862+0000 7fe177a00640 1 -- 172.21.15.111:0/1856966355 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe170062900 con 0x7fe170074620 2024-09-15T22:45:07.866 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.864+0000 7fe16e000640 1 -- 172.21.15.111:0/1856966355 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fe160010ac0 con 0x7fe170074620 2024-09-15T22:45:07.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.866+0000 7fe16e000640 1 --2- 172.21.15.111:0/1856966355 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fe1540a3350 0x7fe1540a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:07.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.866+0000 7fe16e000640 1 -- 172.21.15.111:0/1856966355 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fe1600d05c0 con 0x7fe170074620 2024-09-15T22:45:07.867 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.866+0000 7fe167e00640 1 --2- 172.21.15.111:0/1856966355 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fe1540a3350 0x7fe1540a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:07.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.866+0000 7fe16e000640 1 --2- 172.21.15.111:0/1856966355 >> [v2:172.21.15.111:6810/2578259041,v1:172.21.15.111:6811/2578259041] conn(0x7fe1540ace00 0x7fe1540af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:07.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.866+0000 7fe16e000640 1 -- 172.21.15.111:0/1856966355 --> [v2:172.21.15.111:6810/2578259041,v1:172.21.15.111:6811/2578259041] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe160028ea0 con 0x7fe1540ace00 2024-09-15T22:45:07.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.866+0000 7fe16e000640 1 -- 172.21.15.111:0/1856966355 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fe1600d0970 con 0x7fe170074620 2024-09-15T22:45:07.868 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.866+0000 7fe175a00640 1 --2- 172.21.15.111:0/1856966355 >> [v2:172.21.15.111:6810/2578259041,v1:172.21.15.111:6811/2578259041] conn(0x7fe1540ace00 0x7fe1540af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:07.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.867+0000 7fe167e00640 1 --2- 172.21.15.111:0/1856966355 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fe1540a3350 0x7fe1540a5810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe1580096c0 tx=0x7fe1580046c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:07.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.867+0000 7fe175a00640 1 --2- 172.21.15.111:0/1856966355 >> [v2:172.21.15.111:6810/2578259041,v1:172.21.15.111:6811/2578259041] conn(0x7fe1540ace00 0x7fe1540af2a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:07.869 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.868+0000 7fe16e000640 1 -- 172.21.15.111:0/1856966355 <== osd.3 v2:172.21.15.111:6810/2578259041 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe160028ea0 con 0x7fe1540ace00 2024-09-15T22:45:07.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.882+0000 7fe177a00640 1 -- 172.21.15.111:0/1856966355 --> [v2:172.21.15.111:6810/2578259041,v1:172.21.15.111:6811/2578259041] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe170062d00 con 0x7fe1540ace00 2024-09-15T22:45:07.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.882+0000 7fe16e000640 1 -- 172.21.15.111:0/1856966355 <== osd.3 v2:172.21.15.111:6810/2578259041 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe170062d00 con 0x7fe1540ace00 2024-09-15T22:45:07.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.883+0000 7fe177a00640 1 -- 172.21.15.111:0/1856966355 >> [v2:172.21.15.111:6810/2578259041,v1:172.21.15.111:6811/2578259041] conn(0x7fe1540ace00 msgr2=0x7fe1540af2a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:07.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.883+0000 7fe177a00640 1 --2- 172.21.15.111:0/1856966355 >> [v2:172.21.15.111:6810/2578259041,v1:172.21.15.111:6811/2578259041] conn(0x7fe1540ace00 0x7fe1540af2a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:07.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.883+0000 7fe177a00640 1 -- 172.21.15.111:0/1856966355 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fe1540a3350 msgr2=0x7fe1540a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:07.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.883+0000 7fe177a00640 1 --2- 172.21.15.111:0/1856966355 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fe1540a3350 0x7fe1540a5810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe1580096c0 tx=0x7fe1580046c0 comp rx=0 tx=0).stop 2024-09-15T22:45:07.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.883+0000 7fe177a00640 1 -- 172.21.15.111:0/1856966355 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe170074620 msgr2=0x7fe17019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:07.885 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:07 smithi111 ceph-mon[30629]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 7 op/s; 9 B/s, 0 objects/s recovering 2024-09-15T22:45:07.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.883+0000 7fe177a00640 1 --2- 172.21.15.111:0/1856966355 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe170074620 0x7fe17019a710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe16000e9f0 tx=0x7fe16000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:45:07.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.883+0000 7fe175a00640 1 -- 172.21.15.111:0/1856966355 reap_dead start 2024-09-15T22:45:07.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.883+0000 7fe177a00640 1 -- 172.21.15.111:0/1856966355 shutdown_connections 2024-09-15T22:45:07.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.883+0000 7fe177a00640 1 -- 172.21.15.111:0/1856966355 >> 172.21.15.111:0/1856966355 conn(0x7fe1700fd8c0 msgr2=0x7fe1700fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:07.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.883+0000 7fe177a00640 1 -- 172.21.15.111:0/1856966355 shutdown_connections 2024-09-15T22:45:07.886 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:07.883+0000 7fe177a00640 1 -- 172.21.15.111:0/1856966355 wait complete. 2024-09-15T22:45:08.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:07 smithi192 ceph-mon[35427]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 7 op/s; 9 B/s, 0 objects/s recovering 2024-09-15T22:45:08.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:07 smithi121 ceph-mon[34815]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 7 op/s; 9 B/s, 0 objects/s recovering 2024-09-15T22:45:08.206 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.204+0000 7fb4d7400640 1 -- 172.21.15.111:0/1404865605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4d0102790 msgr2=0x7fb4d01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:08.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.204+0000 7fb4d7400640 1 --2- 172.21.15.111:0/1404865605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4d0102790 0x7fb4d01067e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8009900 tx=0x7fb4b802a3f0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.205+0000 7fb4d7400640 1 -- 172.21.15.111:0/1404865605 shutdown_connections 2024-09-15T22:45:08.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.205+0000 7fb4d7400640 1 --2- 172.21.15.111:0/1404865605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4d0106f10 0x7fb4d010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.205+0000 7fb4d7400640 1 --2- 172.21.15.111:0/1404865605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4d0102790 0x7fb4d01067e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.205+0000 7fb4d7400640 1 --2- 172.21.15.111:0/1404865605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4d0101de0 0x7fb4d01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.205+0000 7fb4d7400640 1 -- 172.21.15.111:0/1404865605 >> 172.21.15.111:0/1404865605 conn(0x7fb4d00fd8c0 msgr2=0x7fb4d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:08.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.205+0000 7fb4d7400640 1 -- 172.21.15.111:0/1404865605 shutdown_connections 2024-09-15T22:45:08.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.205+0000 7fb4d7400640 1 -- 172.21.15.111:0/1404865605 wait complete. 2024-09-15T22:45:08.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.206+0000 7fb4d7400640 1 Processor -- start 2024-09-15T22:45:08.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.206+0000 7fb4d7400640 1 -- start start 2024-09-15T22:45:08.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.206+0000 7fb4d7400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4d0101de0 0x7fb4d01983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:08.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.206+0000 7fb4d7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4d0102790 0x7fb4d0198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:08.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.206+0000 7fb4d7400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4d0106f10 0x7fb4d019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:08.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.206+0000 7fb4d7400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d0111f70 con 0x7fb4d0102790 2024-09-15T22:45:08.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.206+0000 7fb4d7400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d0111df0 con 0x7fb4d0101de0 2024-09-15T22:45:08.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.206+0000 7fb4d7400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d01120f0 con 0x7fb4d0106f10 2024-09-15T22:45:08.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.206+0000 7fb4d4a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4d0101de0 0x7fb4d01983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:08.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.206+0000 7fb4c7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4d0102790 0x7fb4d0198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:08.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.206+0000 7fb4d4a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4d0101de0 0x7fb4d01983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:60652/0 (socket says 172.21.15.111:60652) 2024-09-15T22:45:08.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.206+0000 7fb4d4a00640 1 -- 172.21.15.111:0/3378235197 learned_addr learned my addr 172.21.15.111:0/3378235197 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:08.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.206+0000 7fb4d5400640 1 --2- 172.21.15.111:0/3378235197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4d0106f10 0x7fb4d019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:08.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.207+0000 7fb4d4a00640 1 -- 172.21.15.111:0/3378235197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4d0106f10 msgr2=0x7fb4d019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:08.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.207+0000 7fb4d4a00640 1 --2- 172.21.15.111:0/3378235197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4d0106f10 0x7fb4d019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.207+0000 7fb4d4a00640 1 -- 172.21.15.111:0/3378235197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4d0102790 msgr2=0x7fb4d0198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:08.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.207+0000 7fb4d4a00640 1 --2- 172.21.15.111:0/3378235197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb4d0102790 0x7fb4d0198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.207+0000 7fb4d4a00640 1 -- 172.21.15.111:0/3378235197 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d019d3a0 con 0x7fb4d0101de0 2024-09-15T22:45:08.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.207+0000 7fb4d4a00640 1 --2- 172.21.15.111:0/3378235197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4d0101de0 0x7fb4d01983f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000ea60 tx=0x7fb4c000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:08.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.207+0000 7fb4ce000640 1 -- 172.21.15.111:0/3378235197 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c000cde0 con 0x7fb4d0101de0 2024-09-15T22:45:08.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.207+0000 7fb4ce000640 1 -- 172.21.15.111:0/3378235197 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4c00035f0 con 0x7fb4d0101de0 2024-09-15T22:45:08.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.207+0000 7fb4ce000640 1 -- 172.21.15.111:0/3378235197 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0010670 con 0x7fb4d0101de0 2024-09-15T22:45:08.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.207+0000 7fb4d7400640 1 -- 172.21.15.111:0/3378235197 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d019d690 con 0x7fb4d0101de0 2024-09-15T22:45:08.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.208+0000 7fb4d7400640 1 -- 172.21.15.111:0/3378235197 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d01a4f70 con 0x7fb4d0101de0 2024-09-15T22:45:08.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.208+0000 7fb4d7400640 1 -- 172.21.15.111:0/3378235197 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb4d0062980 con 0x7fb4d0101de0 2024-09-15T22:45:08.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.210+0000 7fb4ce000640 1 -- 172.21.15.111:0/3378235197 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fb4c00037a0 con 0x7fb4d0101de0 2024-09-15T22:45:08.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.212+0000 7fb4ce000640 1 --2- 172.21.15.111:0/3378235197 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb4b40a3350 0x7fb4b40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:08.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.212+0000 7fb4ce000640 1 -- 172.21.15.111:0/3378235197 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fb4c00cffd0 con 0x7fb4d0101de0 2024-09-15T22:45:08.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.212+0000 7fb4c7e00640 1 --2- 172.21.15.111:0/3378235197 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb4b40a3350 0x7fb4b40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:08.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.212+0000 7fb4ce000640 1 --2- 172.21.15.111:0/3378235197 >> [v2:172.21.15.192:6824/2431464812,v1:172.21.15.192:6825/2431464812] conn(0x7fb4b40ace00 0x7fb4b40af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:08.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.212+0000 7fb4ce000640 1 -- 172.21.15.111:0/3378235197 --> [v2:172.21.15.192:6824/2431464812,v1:172.21.15.192:6825/2431464812] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb4c001aec0 con 0x7fb4b40ace00 2024-09-15T22:45:08.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.212+0000 7fb4ce000640 1 -- 172.21.15.111:0/3378235197 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fb4c00d0380 con 0x7fb4d0101de0 2024-09-15T22:45:08.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.213+0000 7fb4d5400640 1 --2- 172.21.15.111:0/3378235197 >> [v2:172.21.15.192:6824/2431464812,v1:172.21.15.192:6825/2431464812] conn(0x7fb4b40ace00 0x7fb4b40af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:08.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.213+0000 7fb4c7e00640 1 --2- 172.21.15.111:0/3378235197 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb4b40a3350 0x7fb4b40a5810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8009230 tx=0x7fb4b80058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:08.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.213+0000 7fb4d5400640 1 --2- 172.21.15.111:0/3378235197 >> [v2:172.21.15.192:6824/2431464812,v1:172.21.15.192:6825/2431464812] conn(0x7fb4b40ace00 0x7fb4b40af2a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:08.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.214+0000 7fb4ce000640 1 -- 172.21.15.111:0/3378235197 <== osd.11 v2:172.21.15.192:6824/2431464812 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb4c001aec0 con 0x7fb4b40ace00 2024-09-15T22:45:08.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.228+0000 7fb4d7400640 1 -- 172.21.15.111:0/3378235197 --> [v2:172.21.15.192:6824/2431464812,v1:172.21.15.192:6825/2431464812] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb4d0062e50 con 0x7fb4b40ace00 2024-09-15T22:45:08.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.229+0000 7fb4ce000640 1 -- 172.21.15.111:0/3378235197 <== osd.11 v2:172.21.15.192:6824/2431464812 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb4d0062e50 con 0x7fb4b40ace00 2024-09-15T22:45:08.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.229+0000 7fb4d7400640 1 -- 172.21.15.111:0/3378235197 >> [v2:172.21.15.192:6824/2431464812,v1:172.21.15.192:6825/2431464812] conn(0x7fb4b40ace00 msgr2=0x7fb4b40af2a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:08.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.229+0000 7fb4d7400640 1 --2- 172.21.15.111:0/3378235197 >> [v2:172.21.15.192:6824/2431464812,v1:172.21.15.192:6825/2431464812] conn(0x7fb4b40ace00 0x7fb4b40af2a0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.229+0000 7fb4d7400640 1 -- 172.21.15.111:0/3378235197 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb4b40a3350 msgr2=0x7fb4b40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:08.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.229+0000 7fb4d7400640 1 --2- 172.21.15.111:0/3378235197 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb4b40a3350 0x7fb4b40a5810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8009230 tx=0x7fb4b80058d0 comp rx=0 tx=0).stop 2024-09-15T22:45:08.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.229+0000 7fb4d7400640 1 -- 172.21.15.111:0/3378235197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4d0101de0 msgr2=0x7fb4d01983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:08.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.229+0000 7fb4d7400640 1 --2- 172.21.15.111:0/3378235197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4d0101de0 0x7fb4d01983f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000ea60 tx=0x7fb4c000ef20 comp rx=0 tx=0).stop 2024-09-15T22:45:08.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.229+0000 7fb4d5400640 1 -- 172.21.15.111:0/3378235197 reap_dead start 2024-09-15T22:45:08.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.229+0000 7fb4d7400640 1 -- 172.21.15.111:0/3378235197 shutdown_connections 2024-09-15T22:45:08.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.229+0000 7fb4d7400640 1 -- 172.21.15.111:0/3378235197 >> 172.21.15.111:0/3378235197 conn(0x7fb4d00fd8c0 msgr2=0x7fb4d00fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:08.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.229+0000 7fb4d7400640 1 -- 172.21.15.111:0/3378235197 shutdown_connections 2024-09-15T22:45:08.233 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:08.229+0000 7fb4d7400640 1 -- 172.21.15.111:0/3378235197 wait complete. 2024-09-15T22:45:09.361 INFO:teuthology.orchestra.run.smithi111.stdout:188978561073 2024-09-15T22:45:09.361 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-15T22:45:09.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.814+0000 7fc040e00640 1 -- 172.21.15.111:0/4202901004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc03c1049c0 msgr2=0x7fc03c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:09.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.814+0000 7fc040e00640 1 --2- 172.21.15.111:0/4202901004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc03c1049c0 0x7fc03c108a10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc028009900 tx=0x7fc02802a400 comp rx=0 tx=0).stop 2024-09-15T22:45:09.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.814+0000 7fc040e00640 1 -- 172.21.15.111:0/4202901004 shutdown_connections 2024-09-15T22:45:09.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.814+0000 7fc040e00640 1 --2- 172.21.15.111:0/4202901004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc03c109140 0x7fc03c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:09.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.814+0000 7fc040e00640 1 --2- 172.21.15.111:0/4202901004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc03c1049c0 0x7fc03c108a10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:09.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.814+0000 7fc040e00640 1 --2- 172.21.15.111:0/4202901004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc03c104010 0x7fc03c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:09.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.814+0000 7fc040e00640 1 -- 172.21.15.111:0/4202901004 >> 172.21.15.111:0/4202901004 conn(0x7fc03c0ffa50 msgr2=0x7fc03c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:09.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.814+0000 7fc040e00640 1 -- 172.21.15.111:0/4202901004 shutdown_connections 2024-09-15T22:45:09.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.814+0000 7fc040e00640 1 -- 172.21.15.111:0/4202901004 wait complete. 2024-09-15T22:45:09.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.815+0000 7fc040e00640 1 Processor -- start 2024-09-15T22:45:09.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.816+0000 7fc040e00640 1 -- start start 2024-09-15T22:45:09.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.816+0000 7fc040e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc03c104010 0x7fc03c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:09.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.816+0000 7fc040e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc03c1049c0 0x7fc03c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:09.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.816+0000 7fc040e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc03c109140 0x7fc03c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:09.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.816+0000 7fc040e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c114140 con 0x7fc03c109140 2024-09-15T22:45:09.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.816+0000 7fc040e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c113fc0 con 0x7fc03c1049c0 2024-09-15T22:45:09.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.816+0000 7fc040e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc03c1142c0 con 0x7fc03c104010 2024-09-15T22:45:09.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.816+0000 7fc039400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc03c1049c0 0x7fc03c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:09.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.816+0000 7fc039400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc03c1049c0 0x7fc03c19ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:60682/0 (socket says 172.21.15.111:60682) 2024-09-15T22:45:09.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.816+0000 7fc039400640 1 -- 172.21.15.111:0/2204185425 learned_addr learned my addr 172.21.15.111:0/2204185425 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:09.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.816+0000 7fc03a800640 1 --2- 172.21.15.111:0/2204185425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc03c109140 0x7fc03c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:09.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.816+0000 7fc039e00640 1 --2- 172.21.15.111:0/2204185425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc03c104010 0x7fc03c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:09.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.817+0000 7fc039400640 1 -- 172.21.15.111:0/2204185425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc03c104010 msgr2=0x7fc03c19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:09.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.817+0000 7fc039400640 1 --2- 172.21.15.111:0/2204185425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc03c104010 0x7fc03c19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:09.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.817+0000 7fc039400640 1 -- 172.21.15.111:0/2204185425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc03c109140 msgr2=0x7fc03c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:09.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.817+0000 7fc039400640 1 --2- 172.21.15.111:0/2204185425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc03c109140 0x7fc03c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:09.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.817+0000 7fc039400640 1 -- 172.21.15.111:0/2204185425 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03c19f5f0 con 0x7fc03c1049c0 2024-09-15T22:45:09.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.817+0000 7fc039400640 1 --2- 172.21.15.111:0/2204185425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc03c1049c0 0x7fc03c19ab80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc028006010 tx=0x7fc028004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:09.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.817+0000 7fc032a00640 1 -- 172.21.15.111:0/2204185425 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02803f030 con 0x7fc03c1049c0 2024-09-15T22:45:09.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.817+0000 7fc040e00640 1 -- 172.21.15.111:0/2204185425 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc03c19f880 con 0x7fc03c1049c0 2024-09-15T22:45:09.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.817+0000 7fc032a00640 1 -- 172.21.15.111:0/2204185425 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc028004440 con 0x7fc03c1049c0 2024-09-15T22:45:09.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.817+0000 7fc032a00640 1 -- 172.21.15.111:0/2204185425 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc028035070 con 0x7fc03c1049c0 2024-09-15T22:45:09.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.818+0000 7fc040e00640 1 -- 172.21.15.111:0/2204185425 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc03c1a71c0 con 0x7fc03c1049c0 2024-09-15T22:45:09.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.819+0000 7fc040e00640 1 -- 172.21.15.111:0/2204185425 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc03c062980 con 0x7fc03c1049c0 2024-09-15T22:45:09.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.820+0000 7fc032a00640 1 -- 172.21.15.111:0/2204185425 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fc028042030 con 0x7fc03c1049c0 2024-09-15T22:45:09.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.822+0000 7fc032a00640 1 --2- 172.21.15.111:0/2204185425 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc0200a32d0 0x7fc0200a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:09.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.822+0000 7fc032a00640 1 -- 172.21.15.111:0/2204185425 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fc0280f0100 con 0x7fc03c1049c0 2024-09-15T22:45:09.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.822+0000 7fc039e00640 1 --2- 172.21.15.111:0/2204185425 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc0200a32d0 0x7fc0200a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:09.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.822+0000 7fc032a00640 1 --2- 172.21.15.111:0/2204185425 >> [v2:172.21.15.121:6800/2051512291,v1:172.21.15.121:6801/2051512291] conn(0x7fc0200acd80 0x7fc0200af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:09.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.822+0000 7fc032a00640 1 -- 172.21.15.111:0/2204185425 --> [v2:172.21.15.121:6800/2051512291,v1:172.21.15.121:6801/2051512291] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc0200af8f0 con 0x7fc0200acd80 2024-09-15T22:45:09.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.822+0000 7fc032a00640 1 -- 172.21.15.111:0/2204185425 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fc0280f04b0 con 0x7fc03c1049c0 2024-09-15T22:45:09.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.823+0000 7fc039e00640 1 --2- 172.21.15.111:0/2204185425 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc0200a32d0 0x7fc0200a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009bd0 tx=0x7fc02c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:09.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.823+0000 7fc03a800640 1 --2- 172.21.15.111:0/2204185425 >> [v2:172.21.15.121:6800/2051512291,v1:172.21.15.121:6801/2051512291] conn(0x7fc0200acd80 0x7fc0200af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:09.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.823+0000 7fc03a800640 1 --2- 172.21.15.111:0/2204185425 >> [v2:172.21.15.121:6800/2051512291,v1:172.21.15.121:6801/2051512291] conn(0x7fc0200acd80 0x7fc0200af220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:09.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.824+0000 7fc032a00640 1 -- 172.21.15.111:0/2204185425 <== osd.1 v2:172.21.15.121:6800/2051512291 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc0200af8f0 con 0x7fc0200acd80 2024-09-15T22:45:09.839 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.838+0000 7fc040e00640 1 -- 172.21.15.111:0/2204185425 --> [v2:172.21.15.121:6800/2051512291,v1:172.21.15.121:6801/2051512291] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc03c062d80 con 0x7fc0200acd80 2024-09-15T22:45:09.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.839+0000 7fc032a00640 1 -- 172.21.15.111:0/2204185425 <== osd.1 v2:172.21.15.121:6800/2051512291 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc03c062d80 con 0x7fc0200acd80 2024-09-15T22:45:09.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.839+0000 7fc040e00640 1 -- 172.21.15.111:0/2204185425 >> [v2:172.21.15.121:6800/2051512291,v1:172.21.15.121:6801/2051512291] conn(0x7fc0200acd80 msgr2=0x7fc0200af220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:09.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.839+0000 7fc040e00640 1 --2- 172.21.15.111:0/2204185425 >> [v2:172.21.15.121:6800/2051512291,v1:172.21.15.121:6801/2051512291] conn(0x7fc0200acd80 0x7fc0200af220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:09.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.839+0000 7fc040e00640 1 -- 172.21.15.111:0/2204185425 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc0200a32d0 msgr2=0x7fc0200a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:09.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.839+0000 7fc040e00640 1 --2- 172.21.15.111:0/2204185425 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc0200a32d0 0x7fc0200a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009bd0 tx=0x7fc02c005f90 comp rx=0 tx=0).stop 2024-09-15T22:45:09.842 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.839+0000 7fc040e00640 1 -- 172.21.15.111:0/2204185425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc03c1049c0 msgr2=0x7fc03c19ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:09.842 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.839+0000 7fc040e00640 1 --2- 172.21.15.111:0/2204185425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc03c1049c0 0x7fc03c19ab80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc028006010 tx=0x7fc028004040 comp rx=0 tx=0).stop 2024-09-15T22:45:09.842 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.839+0000 7fc03a800640 1 -- 172.21.15.111:0/2204185425 reap_dead start 2024-09-15T22:45:09.842 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.839+0000 7fc040e00640 1 -- 172.21.15.111:0/2204185425 shutdown_connections 2024-09-15T22:45:09.842 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.839+0000 7fc040e00640 1 -- 172.21.15.111:0/2204185425 >> 172.21.15.111:0/2204185425 conn(0x7fc03c0ffa50 msgr2=0x7fc03c100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:09.842 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.840+0000 7fc040e00640 1 -- 172.21.15.111:0/2204185425 shutdown_connections 2024-09-15T22:45:09.842 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.840+0000 7fc040e00640 1 -- 172.21.15.111:0/2204185425 wait complete. 2024-09-15T22:45:09.977 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:09 smithi111 ceph-mon[30629]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-15T22:45:10.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:09 smithi192 ceph-mon[35427]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-15T22:45:10.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:09 smithi121 ceph-mon[34815]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-15T22:45:10.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.086+0000 7f9be0c00640 1 -- 172.21.15.111:0/3524869360 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bdc076720 msgr2=0x7f9bdc076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:10.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.086+0000 7f9be0c00640 1 --2- 172.21.15.111:0/3524869360 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bdc076720 0x7f9bdc076b00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc009900 tx=0x7f9bcc02a410 comp rx=0 tx=0).stop 2024-09-15T22:45:10.336 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.086+0000 7f9be0c00640 1 -- 172.21.15.111:0/3524869360 shutdown_connections 2024-09-15T22:45:10.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.086+0000 7f9be0c00640 1 --2- 172.21.15.111:0/3524869360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bdc112a30 0x7f9bdc114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.086+0000 7f9be0c00640 1 --2- 172.21.15.111:0/3524869360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bdc077040 0x7f9bdc1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.086+0000 7f9be0c00640 1 --2- 172.21.15.111:0/3524869360 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bdc076720 0x7f9bdc076b00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.087+0000 7f9be0c00640 1 -- 172.21.15.111:0/3524869360 >> 172.21.15.111:0/3524869360 conn(0x7f9bdc0ffa90 msgr2=0x7f9bdc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:10.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.087+0000 7f9be0c00640 1 -- 172.21.15.111:0/3524869360 shutdown_connections 2024-09-15T22:45:10.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.087+0000 7f9be0c00640 1 -- 172.21.15.111:0/3524869360 wait complete. 2024-09-15T22:45:10.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.088+0000 7f9be0c00640 1 Processor -- start 2024-09-15T22:45:10.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.088+0000 7f9be0c00640 1 -- start start 2024-09-15T22:45:10.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.088+0000 7f9be0c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bdc076720 0x7f9bdc1a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:10.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.088+0000 7f9be0c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bdc077040 0x7f9bdc1a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:10.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.088+0000 7f9be0c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bdc112a30 0x7f9bdc19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:10.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.088+0000 7f9be0c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9bdc118610 con 0x7f9bdc112a30 2024-09-15T22:45:10.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.089+0000 7f9be0c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9bdc118490 con 0x7f9bdc076720 2024-09-15T22:45:10.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.089+0000 7f9be0c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9bdc118790 con 0x7f9bdc077040 2024-09-15T22:45:10.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.089+0000 7f9bd9e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bdc076720 0x7f9bdc1a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:10.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.089+0000 7f9bd9e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bdc076720 0x7f9bdc1a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:60702/0 (socket says 172.21.15.111:60702) 2024-09-15T22:45:10.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.089+0000 7f9bd9e00640 1 -- 172.21.15.111:0/2462180255 learned_addr learned my addr 172.21.15.111:0/2462180255 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:10.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.089+0000 7f9bda800640 1 --2- 172.21.15.111:0/2462180255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bdc112a30 0x7f9bdc19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:10.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.089+0000 7f9bd3e00640 1 --2- 172.21.15.111:0/2462180255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bdc077040 0x7f9bdc1a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:10.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.089+0000 7f9bd9e00640 1 -- 172.21.15.111:0/2462180255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bdc077040 msgr2=0x7f9bdc1a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:10.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.089+0000 7f9bd9e00640 1 --2- 172.21.15.111:0/2462180255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bdc077040 0x7f9bdc1a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.089+0000 7f9bd9e00640 1 -- 172.21.15.111:0/2462180255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bdc112a30 msgr2=0x7f9bdc19e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:10.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.089+0000 7f9bd9e00640 1 --2- 172.21.15.111:0/2462180255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bdc112a30 0x7f9bdc19e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.089+0000 7f9bd9e00640 1 -- 172.21.15.111:0/2462180255 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bdc19efd0 con 0x7f9bdc076720 2024-09-15T22:45:10.338 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.089+0000 7f9bda800640 1 --2- 172.21.15.111:0/2462180255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bdc112a30 0x7f9bdc19e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:10.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.089+0000 7f9bd3e00640 1 --2- 172.21.15.111:0/2462180255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bdc077040 0x7f9bdc1a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:10.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.089+0000 7f9bd9e00640 1 --2- 172.21.15.111:0/2462180255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bdc076720 0x7f9bdc1a46e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc004770 tx=0x7f9bcc003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:10.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.090+0000 7f9bd2a00640 1 -- 172.21.15.111:0/2462180255 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc03f070 con 0x7f9bdc076720 2024-09-15T22:45:10.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.090+0000 7f9bd2a00640 1 -- 172.21.15.111:0/2462180255 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bcc031ca0 con 0x7f9bdc076720 2024-09-15T22:45:10.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.090+0000 7f9be0c00640 1 -- 172.21.15.111:0/2462180255 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bdc19f260 con 0x7f9bdc076720 2024-09-15T22:45:10.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.090+0000 7f9bd2a00640 1 -- 172.21.15.111:0/2462180255 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc035030 con 0x7f9bdc076720 2024-09-15T22:45:10.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.090+0000 7f9be0c00640 1 -- 172.21.15.111:0/2462180255 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bdc1ab5b0 con 0x7f9bdc076720 2024-09-15T22:45:10.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.091+0000 7f9be0c00640 1 -- 172.21.15.111:0/2462180255 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9bdc114ed0 con 0x7f9bdc076720 2024-09-15T22:45:10.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.093+0000 7f9bd2a00640 1 -- 172.21.15.111:0/2462180255 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f9bcc042070 con 0x7f9bdc076720 2024-09-15T22:45:10.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.094+0000 7f9bd2a00640 1 --2- 172.21.15.111:0/2462180255 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9bc00a32d0 0x7f9bc00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:10.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.094+0000 7f9bd2a00640 1 -- 172.21.15.111:0/2462180255 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f9bcc0f0610 con 0x7f9bdc076720 2024-09-15T22:45:10.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.094+0000 7f9bd3e00640 1 --2- 172.21.15.111:0/2462180255 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9bc00a32d0 0x7f9bc00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:10.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.095+0000 7f9bd2a00640 1 --2- 172.21.15.111:0/2462180255 >> [v2:172.21.15.121:6824/2897902637,v1:172.21.15.121:6825/2897902637] conn(0x7f9bc00acd80 0x7f9bc00af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:10.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.095+0000 7f9bd2a00640 1 -- 172.21.15.111:0/2462180255 --> [v2:172.21.15.121:6824/2897902637,v1:172.21.15.121:6825/2897902637] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9bcc038e90 con 0x7f9bc00acd80 2024-09-15T22:45:10.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.095+0000 7f9bd2a00640 1 -- 172.21.15.111:0/2462180255 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f9bcc0f09c0 con 0x7f9bdc076720 2024-09-15T22:45:10.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.095+0000 7f9bda800640 1 --2- 172.21.15.111:0/2462180255 >> [v2:172.21.15.121:6824/2897902637,v1:172.21.15.121:6825/2897902637] conn(0x7f9bc00acd80 0x7f9bc00af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:10.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.095+0000 7f9bd3e00640 1 --2- 172.21.15.111:0/2462180255 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9bc00a32d0 0x7f9bc00a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc0797b0 tx=0x7f9bc4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:10.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.095+0000 7f9bda800640 1 --2- 172.21.15.111:0/2462180255 >> [v2:172.21.15.121:6824/2897902637,v1:172.21.15.121:6825/2897902637] conn(0x7f9bc00acd80 0x7f9bc00af220 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:10.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.096+0000 7f9bd2a00640 1 -- 172.21.15.111:0/2462180255 <== osd.10 v2:172.21.15.121:6824/2897902637 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9bcc038e90 con 0x7f9bc00acd80 2024-09-15T22:45:10.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.110+0000 7f9be0c00640 1 -- 172.21.15.111:0/2462180255 --> [v2:172.21.15.121:6824/2897902637,v1:172.21.15.121:6825/2897902637] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9bdc077e60 con 0x7f9bc00acd80 2024-09-15T22:45:10.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.111+0000 7f9bd2a00640 1 -- 172.21.15.111:0/2462180255 <== osd.10 v2:172.21.15.121:6824/2897902637 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9bdc077e60 con 0x7f9bc00acd80 2024-09-15T22:45:10.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.111+0000 7f9be0c00640 1 -- 172.21.15.111:0/2462180255 >> [v2:172.21.15.121:6824/2897902637,v1:172.21.15.121:6825/2897902637] conn(0x7f9bc00acd80 msgr2=0x7f9bc00af220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:10.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.111+0000 7f9be0c00640 1 --2- 172.21.15.111:0/2462180255 >> [v2:172.21.15.121:6824/2897902637,v1:172.21.15.121:6825/2897902637] conn(0x7f9bc00acd80 0x7f9bc00af220 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:10.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.112+0000 7f9be0c00640 1 -- 172.21.15.111:0/2462180255 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9bc00a32d0 msgr2=0x7f9bc00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:10.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.112+0000 7f9be0c00640 1 --2- 172.21.15.111:0/2462180255 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9bc00a32d0 0x7f9bc00a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc0797b0 tx=0x7f9bc4005f90 comp rx=0 tx=0).stop 2024-09-15T22:45:10.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.112+0000 7f9be0c00640 1 -- 172.21.15.111:0/2462180255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bdc076720 msgr2=0x7f9bdc1a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:10.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.112+0000 7f9be0c00640 1 --2- 172.21.15.111:0/2462180255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bdc076720 0x7f9bdc1a46e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc004770 tx=0x7f9bcc003c10 comp rx=0 tx=0).stop 2024-09-15T22:45:10.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.112+0000 7f9bda800640 1 -- 172.21.15.111:0/2462180255 reap_dead start 2024-09-15T22:45:10.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.112+0000 7f9be0c00640 1 -- 172.21.15.111:0/2462180255 shutdown_connections 2024-09-15T22:45:10.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.112+0000 7f9be0c00640 1 -- 172.21.15.111:0/2462180255 >> 172.21.15.111:0/2462180255 conn(0x7f9bdc0ffa90 msgr2=0x7f9bdc101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:10.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.112+0000 7f9be0c00640 1 -- 172.21.15.111:0/2462180255 shutdown_connections 2024-09-15T22:45:10.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:10.112+0000 7f9be0c00640 1 -- 172.21.15.111:0/2462180255 wait complete. 2024-09-15T22:45:11.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.227+0000 7f1e8a400640 1 -- 172.21.15.111:0/3557193534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e841129d0 msgr2=0x7f1e84114dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:11.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.227+0000 7f1e8a400640 1 --2- 172.21.15.111:0/3557193534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e841129d0 0x7f1e84114dc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009ca0 tx=0x7f1e7c02a950 comp rx=0 tx=0).stop 2024-09-15T22:45:11.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.227+0000 7f1e8a400640 1 -- 172.21.15.111:0/3557193534 shutdown_connections 2024-09-15T22:45:11.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.227+0000 7f1e8a400640 1 --2- 172.21.15.111:0/3557193534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e841129d0 0x7f1e84114dc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:11.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.227+0000 7f1e8a400640 1 --2- 172.21.15.111:0/3557193534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e84074eb0 0x7f1e84112490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:11.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.227+0000 7f1e8a400640 1 --2- 172.21.15.111:0/3557193534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e84074590 0x7f1e84074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:11.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.227+0000 7f1e8a400640 1 -- 172.21.15.111:0/3557193534 >> 172.21.15.111:0/3557193534 conn(0x7f1e840fd6e0 msgr2=0x7f1e840ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:11.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.228+0000 7f1e8a400640 1 -- 172.21.15.111:0/3557193534 shutdown_connections 2024-09-15T22:45:11.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.228+0000 7f1e8a400640 1 -- 172.21.15.111:0/3557193534 wait complete. 2024-09-15T22:45:11.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.228+0000 7f1e8a400640 1 Processor -- start 2024-09-15T22:45:11.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.229+0000 7f1e8a400640 1 -- start start 2024-09-15T22:45:11.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.229+0000 7f1e8a400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e84074590 0x7f1e841a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:11.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.229+0000 7f1e8a400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e84074eb0 0x7f1e841a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:11.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.229+0000 7f1e8a400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e841129d0 0x7f1e8419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:11.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.229+0000 7f1e8a400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1e841183f0 con 0x7f1e84074590 2024-09-15T22:45:11.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.229+0000 7f1e8a400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1e84118270 con 0x7f1e841129d0 2024-09-15T22:45:11.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.229+0000 7f1e8a400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1e84118570 con 0x7f1e84074eb0 2024-09-15T22:45:11.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.229+0000 7f1e89000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e84074590 0x7f1e841a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:11.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.229+0000 7f1e89000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e84074590 0x7f1e841a45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39958/0 (socket says 172.21.15.111:39958) 2024-09-15T22:45:11.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.229+0000 7f1e89000640 1 -- 172.21.15.111:0/3939616631 learned_addr learned my addr 172.21.15.111:0/3939616631 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:11.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.229+0000 7f1e89a00640 1 --2- 172.21.15.111:0/3939616631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e841129d0 0x7f1e8419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:11.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.229+0000 7f1e7be00640 1 --2- 172.21.15.111:0/3939616631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e84074eb0 0x7f1e841a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:11.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.230+0000 7f1e89000640 1 -- 172.21.15.111:0/3939616631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e84074eb0 msgr2=0x7f1e841a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:11.233 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.230+0000 7f1e89000640 1 --2- 172.21.15.111:0/3939616631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e84074eb0 0x7f1e841a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:11.233 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.230+0000 7f1e89000640 1 -- 172.21.15.111:0/3939616631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e841129d0 msgr2=0x7f1e8419e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:11.233 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.230+0000 7f1e89000640 1 --2- 172.21.15.111:0/3939616631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e841129d0 0x7f1e8419e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:11.233 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.230+0000 7f1e89000640 1 -- 172.21.15.111:0/3939616631 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8419ef30 con 0x7f1e84074590 2024-09-15T22:45:11.233 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.230+0000 7f1e89000640 1 --2- 172.21.15.111:0/3939616631 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e84074590 0x7f1e841a45f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1e7400e9f0 tx=0x7f1e7400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:11.234 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.230+0000 7f1e82000640 1 -- 172.21.15.111:0/3939616631 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7400cd80 con 0x7f1e84074590 2024-09-15T22:45:11.234 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.231+0000 7f1e8a400640 1 -- 172.21.15.111:0/3939616631 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e8419f220 con 0x7f1e84074590 2024-09-15T22:45:11.234 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.231+0000 7f1e82000640 1 -- 172.21.15.111:0/3939616631 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e74005500 con 0x7f1e84074590 2024-09-15T22:45:11.234 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.231+0000 7f1e82000640 1 -- 172.21.15.111:0/3939616631 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e74010690 con 0x7f1e84074590 2024-09-15T22:45:11.234 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.231+0000 7f1e8a400640 1 -- 172.21.15.111:0/3939616631 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e841ab3a0 con 0x7f1e84074590 2024-09-15T22:45:11.234 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.232+0000 7f1e8a400640 1 -- 172.21.15.111:0/3939616631 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1e84062940 con 0x7f1e84074590 2024-09-15T22:45:11.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.234+0000 7f1e82000640 1 -- 172.21.15.111:0/3939616631 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f1e740046c0 con 0x7f1e84074590 2024-09-15T22:45:11.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.235+0000 7f1e82000640 1 --2- 172.21.15.111:0/3939616631 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1e680a32d0 0x7f1e680a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:11.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.235+0000 7f1e82000640 1 -- 172.21.15.111:0/3939616631 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f1e740d02e0 con 0x7f1e84074590 2024-09-15T22:45:11.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.235+0000 7f1e7be00640 1 --2- 172.21.15.111:0/3939616631 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1e680a32d0 0x7f1e680a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:11.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.235+0000 7f1e82000640 1 --2- 172.21.15.111:0/3939616631 >> [v2:172.21.15.121:6816/3501842998,v1:172.21.15.121:6817/3501842998] conn(0x7f1e680acd80 0x7f1e680af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:11.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.235+0000 7f1e82000640 1 -- 172.21.15.111:0/3939616631 --> [v2:172.21.15.121:6816/3501842998,v1:172.21.15.121:6817/3501842998] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1e74005350 con 0x7f1e680acd80 2024-09-15T22:45:11.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.235+0000 7f1e82000640 1 -- 172.21.15.111:0/3939616631 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f1e740d0690 con 0x7f1e84074590 2024-09-15T22:45:11.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.236+0000 7f1e7be00640 1 --2- 172.21.15.111:0/3939616631 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1e680a32d0 0x7f1e680a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1e8419ff10 tx=0x7f1e6c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:11.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.236+0000 7f1e89a00640 1 --2- 172.21.15.111:0/3939616631 >> [v2:172.21.15.121:6816/3501842998,v1:172.21.15.121:6817/3501842998] conn(0x7f1e680acd80 0x7f1e680af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:11.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.237+0000 7f1e89a00640 1 --2- 172.21.15.111:0/3939616631 >> [v2:172.21.15.121:6816/3501842998,v1:172.21.15.121:6817/3501842998] conn(0x7f1e680acd80 0x7f1e680af220 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:11.239 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.237+0000 7f1e82000640 1 -- 172.21.15.111:0/3939616631 <== osd.7 v2:172.21.15.121:6816/3501842998 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1e74005350 con 0x7f1e680acd80 2024-09-15T22:45:11.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.252+0000 7f1e8a400640 1 -- 172.21.15.111:0/3939616631 --> [v2:172.21.15.121:6816/3501842998,v1:172.21.15.121:6817/3501842998] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1e84062dc0 con 0x7f1e680acd80 2024-09-15T22:45:11.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.252+0000 7f1e82000640 1 -- 172.21.15.111:0/3939616631 <== osd.7 v2:172.21.15.121:6816/3501842998 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1e84062dc0 con 0x7f1e680acd80 2024-09-15T22:45:11.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.253+0000 7f1e8a400640 1 -- 172.21.15.111:0/3939616631 >> [v2:172.21.15.121:6816/3501842998,v1:172.21.15.121:6817/3501842998] conn(0x7f1e680acd80 msgr2=0x7f1e680af220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:11.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.253+0000 7f1e8a400640 1 --2- 172.21.15.111:0/3939616631 >> [v2:172.21.15.121:6816/3501842998,v1:172.21.15.121:6817/3501842998] conn(0x7f1e680acd80 0x7f1e680af220 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:11.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.253+0000 7f1e8a400640 1 -- 172.21.15.111:0/3939616631 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1e680a32d0 msgr2=0x7f1e680a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:11.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.253+0000 7f1e8a400640 1 --2- 172.21.15.111:0/3939616631 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1e680a32d0 0x7f1e680a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1e8419ff10 tx=0x7f1e6c005f90 comp rx=0 tx=0).stop 2024-09-15T22:45:11.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.253+0000 7f1e8a400640 1 -- 172.21.15.111:0/3939616631 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e84074590 msgr2=0x7f1e841a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:11.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.253+0000 7f1e8a400640 1 --2- 172.21.15.111:0/3939616631 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e84074590 0x7f1e841a45f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1e7400e9f0 tx=0x7f1e7400edf0 comp rx=0 tx=0).stop 2024-09-15T22:45:11.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.253+0000 7f1e89a00640 1 -- 172.21.15.111:0/3939616631 reap_dead start 2024-09-15T22:45:11.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.253+0000 7f1e8a400640 1 -- 172.21.15.111:0/3939616631 shutdown_connections 2024-09-15T22:45:11.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.253+0000 7f1e8a400640 1 -- 172.21.15.111:0/3939616631 >> 172.21.15.111:0/3939616631 conn(0x7f1e840fd6e0 msgr2=0x7f1e840fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:11.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.253+0000 7f1e8a400640 1 -- 172.21.15.111:0/3939616631 shutdown_connections 2024-09-15T22:45:11.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:11.253+0000 7f1e8a400640 1 -- 172.21.15.111:0/3939616631 wait complete. 2024-09-15T22:45:11.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.138+0000 7f2b58a00640 1 -- 172.21.15.111:0/1536123754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b54108f60 msgr2=0x7f2b54110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:11.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.138+0000 7f2b58a00640 1 --2- 172.21.15.111:0/1536123754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b54108f60 0x7f2b54110ae0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c009ca0 tx=0x7f2b4c02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:45:11.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.138+0000 7f2b58a00640 1 -- 172.21.15.111:0/1536123754 shutdown_connections 2024-09-15T22:45:11.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.138+0000 7f2b58a00640 1 --2- 172.21.15.111:0/1536123754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b54108f60 0x7f2b54110ae0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:11.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.138+0000 7f2b58a00640 1 --2- 172.21.15.111:0/1536123754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b541047e0 0x7f2b54108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:11.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.138+0000 7f2b58a00640 1 --2- 172.21.15.111:0/1536123754 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b54103e30 0x7f2b54104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:11.510 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.138+0000 7f2b58a00640 1 -- 172.21.15.111:0/1536123754 >> 172.21.15.111:0/1536123754 conn(0x7f2b540ff8d0 msgr2=0x7f2b54101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:11.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.138+0000 7f2b58a00640 1 -- 172.21.15.111:0/1536123754 shutdown_connections 2024-09-15T22:45:11.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.138+0000 7f2b58a00640 1 -- 172.21.15.111:0/1536123754 wait complete. 2024-09-15T22:45:11.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.139+0000 7f2b58a00640 1 Processor -- start 2024-09-15T22:45:11.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.139+0000 7f2b58a00640 1 -- start start 2024-09-15T22:45:11.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.139+0000 7f2b58a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b54103e30 0x7f2b5419a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:11.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.139+0000 7f2b58a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b541047e0 0x7f2b5419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:11.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.139+0000 7f2b58a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b54108f60 0x7f2b5419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:11.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.139+0000 7f2b58a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2b54113fc0 con 0x7f2b541047e0 2024-09-15T22:45:11.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.139+0000 7f2b58a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2b54113e40 con 0x7f2b54103e30 2024-09-15T22:45:11.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.139+0000 7f2b58a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2b54114140 con 0x7f2b54108f60 2024-09-15T22:45:11.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.140+0000 7f2b4aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b541047e0 0x7f2b5419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:11.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.140+0000 7f2b4aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b541047e0 0x7f2b5419a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58684/0 (socket says 172.21.15.111:58684) 2024-09-15T22:45:11.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.140+0000 7f2b4aa00640 1 -- 172.21.15.111:0/26671676 learned_addr learned my addr 172.21.15.111:0/26671676 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:11.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.140+0000 7f2b53400640 1 --2- 172.21.15.111:0/26671676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b54103e30 0x7f2b5419a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:11.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.140+0000 7f2b53e00640 1 --2- 172.21.15.111:0/26671676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b54108f60 0x7f2b5419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:11.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.140+0000 7f2b4aa00640 1 -- 172.21.15.111:0/26671676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b54108f60 msgr2=0x7f2b5419ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:11.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.140+0000 7f2b4aa00640 1 --2- 172.21.15.111:0/26671676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b54108f60 0x7f2b5419ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:11.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.140+0000 7f2b4aa00640 1 -- 172.21.15.111:0/26671676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b54103e30 msgr2=0x7f2b5419a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:11.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.140+0000 7f2b4aa00640 1 --2- 172.21.15.111:0/26671676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b54103e30 0x7f2b5419a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:11.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.140+0000 7f2b4aa00640 1 -- 172.21.15.111:0/26671676 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b5419f390 con 0x7f2b541047e0 2024-09-15T22:45:11.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.140+0000 7f2b4aa00640 1 --2- 172.21.15.111:0/26671676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b541047e0 0x7f2b5419a920 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c00d950 tx=0x7f2b3c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:11.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.141+0000 7f2b50c00640 1 -- 172.21.15.111:0/26671676 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c017030 con 0x7f2b541047e0 2024-09-15T22:45:11.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.141+0000 7f2b50c00640 1 -- 172.21.15.111:0/26671676 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b3c004020 con 0x7f2b541047e0 2024-09-15T22:45:11.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.141+0000 7f2b50c00640 1 -- 172.21.15.111:0/26671676 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c0047a0 con 0x7f2b541047e0 2024-09-15T22:45:11.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.141+0000 7f2b58a00640 1 -- 172.21.15.111:0/26671676 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b5419f680 con 0x7f2b541047e0 2024-09-15T22:45:11.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.141+0000 7f2b58a00640 1 -- 172.21.15.111:0/26671676 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b541a6f60 con 0x7f2b541047e0 2024-09-15T22:45:11.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.142+0000 7f2b58a00640 1 -- 172.21.15.111:0/26671676 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2b54062980 con 0x7f2b541047e0 2024-09-15T22:45:11.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.144+0000 7f2b50c00640 1 -- 172.21.15.111:0/26671676 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f2b3c0041d0 con 0x7f2b541047e0 2024-09-15T22:45:11.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.145+0000 7f2b50c00640 1 --2- 172.21.15.111:0/26671676 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2b380a3350 0x7f2b380a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:11.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.145+0000 7f2b50c00640 1 -- 172.21.15.111:0/26671676 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f2b3c0cf9d0 con 0x7f2b541047e0 2024-09-15T22:45:11.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.145+0000 7f2b53400640 1 --2- 172.21.15.111:0/26671676 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2b380a3350 0x7f2b380a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:11.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.145+0000 7f2b50c00640 1 --2- 172.21.15.111:0/26671676 >> [v2:172.21.15.192:6800/93085639,v1:172.21.15.192:6801/93085639] conn(0x7f2b380ace00 0x7f2b380af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:11.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.145+0000 7f2b50c00640 1 -- 172.21.15.111:0/26671676 --> [v2:172.21.15.192:6800/93085639,v1:172.21.15.192:6801/93085639] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2b3c011e50 con 0x7f2b380ace00 2024-09-15T22:45:11.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.145+0000 7f2b50c00640 1 -- 172.21.15.111:0/26671676 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f2b3c0cfd80 con 0x7f2b541047e0 2024-09-15T22:45:11.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.146+0000 7f2b53400640 1 --2- 172.21.15.111:0/26671676 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2b380a3350 0x7f2b380a5810 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2b440095f0 tx=0x7f2b44005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:11.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.146+0000 7f2b53e00640 1 --2- 172.21.15.111:0/26671676 >> [v2:172.21.15.192:6800/93085639,v1:172.21.15.192:6801/93085639] conn(0x7f2b380ace00 0x7f2b380af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:11.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.146+0000 7f2b53e00640 1 --2- 172.21.15.111:0/26671676 >> [v2:172.21.15.192:6800/93085639,v1:172.21.15.192:6801/93085639] conn(0x7f2b380ace00 0x7f2b380af2a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:11.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.147+0000 7f2b50c00640 1 -- 172.21.15.111:0/26671676 <== osd.2 v2:172.21.15.192:6800/93085639 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2b3c011e50 con 0x7f2b380ace00 2024-09-15T22:45:11.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.161+0000 7f2b58a00640 1 -- 172.21.15.111:0/26671676 --> [v2:172.21.15.192:6800/93085639,v1:172.21.15.192:6801/93085639] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2b54062e50 con 0x7f2b380ace00 2024-09-15T22:45:11.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.162+0000 7f2b50c00640 1 -- 172.21.15.111:0/26671676 <== osd.2 v2:172.21.15.192:6800/93085639 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2b54062e50 con 0x7f2b380ace00 2024-09-15T22:45:11.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.162+0000 7f2b58a00640 1 -- 172.21.15.111:0/26671676 >> [v2:172.21.15.192:6800/93085639,v1:172.21.15.192:6801/93085639] conn(0x7f2b380ace00 msgr2=0x7f2b380af2a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:11.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.162+0000 7f2b58a00640 1 --2- 172.21.15.111:0/26671676 >> [v2:172.21.15.192:6800/93085639,v1:172.21.15.192:6801/93085639] conn(0x7f2b380ace00 0x7f2b380af2a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:11.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.163+0000 7f2b58a00640 1 -- 172.21.15.111:0/26671676 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2b380a3350 msgr2=0x7f2b380a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:11.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.163+0000 7f2b58a00640 1 --2- 172.21.15.111:0/26671676 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2b380a3350 0x7f2b380a5810 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2b440095f0 tx=0x7f2b44005f90 comp rx=0 tx=0).stop 2024-09-15T22:45:11.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.163+0000 7f2b58a00640 1 -- 172.21.15.111:0/26671676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b541047e0 msgr2=0x7f2b5419a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:11.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.163+0000 7f2b58a00640 1 --2- 172.21.15.111:0/26671676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b541047e0 0x7f2b5419a920 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c00d950 tx=0x7f2b3c00de10 comp rx=0 tx=0).stop 2024-09-15T22:45:11.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.163+0000 7f2b53e00640 1 -- 172.21.15.111:0/26671676 reap_dead start 2024-09-15T22:45:11.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.163+0000 7f2b58a00640 1 -- 172.21.15.111:0/26671676 shutdown_connections 2024-09-15T22:45:11.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.163+0000 7f2b58a00640 1 -- 172.21.15.111:0/26671676 >> 172.21.15.111:0/26671676 conn(0x7f2b540ff8d0 msgr2=0x7f2b54100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:11.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.163+0000 7f2b58a00640 1 -- 172.21.15.111:0/26671676 shutdown_connections 2024-09-15T22:45:11.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:09.163+0000 7f2b58a00640 1 -- 172.21.15.111:0/26671676 wait complete. 2024-09-15T22:45:11.889 INFO:teuthology.orchestra.run.smithi111.stdout:107374182468 2024-09-15T22:45:11.890 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-15T22:45:11.938 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:11 smithi111 ceph-mon[30629]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-15T22:45:12.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:11 smithi192 ceph-mon[35427]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-15T22:45:12.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:11 smithi121 ceph-mon[34815]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 170 B/s wr, 7 op/s 2024-09-15T22:45:12.311 INFO:teuthology.orchestra.run.smithi111.stdout:55834574926 2024-09-15T22:45:12.311 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-15T22:45:12.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.988+0000 7f3417200640 1 -- 172.21.15.111:0/3295844364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3418103e30 msgr2=0x7f3418104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:12.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.988+0000 7f3417200640 1 --2- 172.21.15.111:0/3295844364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3418103e30 0x7f3418104210 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3408009900 tx=0x7f340802a3f0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.988+0000 7f3417200640 1 -- 172.21.15.111:0/3295844364 shutdown_connections 2024-09-15T22:45:12.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.988+0000 7f3417200640 1 --2- 172.21.15.111:0/3295844364 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3418108f60 0x7f3418110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.988+0000 7f3417200640 1 --2- 172.21.15.111:0/3295844364 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34181047e0 0x7f3418108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.988+0000 7f3417200640 1 --2- 172.21.15.111:0/3295844364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3418103e30 0x7f3418104210 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.988+0000 7f3417200640 1 -- 172.21.15.111:0/3295844364 >> 172.21.15.111:0/3295844364 conn(0x7f34180ff8d0 msgr2=0x7f3418101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:12.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.988+0000 7f3417200640 1 -- 172.21.15.111:0/3295844364 shutdown_connections 2024-09-15T22:45:12.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.988+0000 7f3417200640 1 -- 172.21.15.111:0/3295844364 wait complete. 2024-09-15T22:45:12.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.989+0000 7f3417200640 1 Processor -- start 2024-09-15T22:45:12.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.990+0000 7f3417200640 1 -- start start 2024-09-15T22:45:12.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.990+0000 7f3417200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3418103e30 0x7f341819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:12.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.990+0000 7f3417200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34181047e0 0x7f341819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:12.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.990+0000 7f3417200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3418108f60 0x7f341819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:12.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.990+0000 7f3417200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3418113fd0 con 0x7f3418103e30 2024-09-15T22:45:12.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.990+0000 7f3417200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3418113e50 con 0x7f34181047e0 2024-09-15T22:45:12.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.990+0000 7f3417200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3418114150 con 0x7f3418108f60 2024-09-15T22:45:12.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.990+0000 7f3415e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3418103e30 0x7f341819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:12.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.990+0000 7f3415e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3418103e30 0x7f341819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39978/0 (socket says 172.21.15.111:39978) 2024-09-15T22:45:12.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.990+0000 7f3415400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34181047e0 0x7f341819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:12.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.990+0000 7f3415e00640 1 -- 172.21.15.111:0/4265825655 learned_addr learned my addr 172.21.15.111:0/4265825655 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:12.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.990+0000 7f3416800640 1 --2- 172.21.15.111:0/4265825655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3418108f60 0x7f341819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:12.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.991+0000 7f3415400640 1 -- 172.21.15.111:0/4265825655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3418108f60 msgr2=0x7f341819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:12.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.991+0000 7f3415400640 1 --2- 172.21.15.111:0/4265825655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3418108f60 0x7f341819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.991+0000 7f3415400640 1 -- 172.21.15.111:0/4265825655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3418103e30 msgr2=0x7f341819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:12.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.991+0000 7f3415400640 1 --2- 172.21.15.111:0/4265825655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3418103e30 0x7f341819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:12.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.991+0000 7f3415400640 1 -- 172.21.15.111:0/4265825655 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f341819f420 con 0x7f34181047e0 2024-09-15T22:45:12.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.991+0000 7f3415400640 1 --2- 172.21.15.111:0/4265825655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34181047e0 0x7f341819a9b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f340400d6d0 tx=0x7f340400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:12.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.991+0000 7f340ea00640 1 -- 172.21.15.111:0/4265825655 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3404017070 con 0x7f34181047e0 2024-09-15T22:45:12.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.991+0000 7f3417200640 1 -- 172.21.15.111:0/4265825655 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f341819f710 con 0x7f34181047e0 2024-09-15T22:45:12.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.991+0000 7f340ea00640 1 -- 172.21.15.111:0/4265825655 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3404009820 con 0x7f34181047e0 2024-09-15T22:45:12.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.991+0000 7f340ea00640 1 -- 172.21.15.111:0/4265825655 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3404004cd0 con 0x7f34181047e0 2024-09-15T22:45:12.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.992+0000 7f3417200640 1 -- 172.21.15.111:0/4265825655 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34181a6ff0 con 0x7f34181047e0 2024-09-15T22:45:12.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.992+0000 7f3417200640 1 -- 172.21.15.111:0/4265825655 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3418062940 con 0x7f34181047e0 2024-09-15T22:45:12.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.994+0000 7f340ea00640 1 -- 172.21.15.111:0/4265825655 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f34040099d0 con 0x7f34181047e0 2024-09-15T22:45:12.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.995+0000 7f340ea00640 1 --2- 172.21.15.111:0/4265825655 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f33fc0a3350 0x7f33fc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:12.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.996+0000 7f340ea00640 1 -- 172.21.15.111:0/4265825655 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f34040cf5f0 con 0x7f34181047e0 2024-09-15T22:45:12.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.996+0000 7f3415e00640 1 --2- 172.21.15.111:0/4265825655 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f33fc0a3350 0x7f33fc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:12.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.996+0000 7f340ea00640 1 --2- 172.21.15.111:0/4265825655 >> [v2:172.21.15.111:6826/3979119431,v1:172.21.15.111:6827/3979119431] conn(0x7f33fc0ace00 0x7f33fc0af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:12.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.996+0000 7f340ea00640 1 -- 172.21.15.111:0/4265825655 --> [v2:172.21.15.111:6826/3979119431,v1:172.21.15.111:6827/3979119431] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3404047ea0 con 0x7f33fc0ace00 2024-09-15T22:45:12.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.996+0000 7f340ea00640 1 -- 172.21.15.111:0/4265825655 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f34040cf9a0 con 0x7f34181047e0 2024-09-15T22:45:12.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.996+0000 7f3416800640 1 --2- 172.21.15.111:0/4265825655 >> [v2:172.21.15.111:6826/3979119431,v1:172.21.15.111:6827/3979119431] conn(0x7f33fc0ace00 0x7f33fc0af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:12.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.997+0000 7f3415e00640 1 --2- 172.21.15.111:0/4265825655 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f33fc0a3350 0x7f33fc0a5810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f34080096c0 tx=0x7f34080046c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:12.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.997+0000 7f3416800640 1 --2- 172.21.15.111:0/4265825655 >> [v2:172.21.15.111:6826/3979119431,v1:172.21.15.111:6827/3979119431] conn(0x7f33fc0ace00 0x7f33fc0af2a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:12.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:12.998+0000 7f340ea00640 1 -- 172.21.15.111:0/4265825655 <== osd.9 v2:172.21.15.111:6826/3979119431 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3404047ea0 con 0x7f33fc0ace00 2024-09-15T22:45:13.011 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:13.022 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:13.020+0000 7f3417200640 1 -- 172.21.15.111:0/4265825655 --> [v2:172.21.15.111:6826/3979119431,v1:172.21.15.111:6827/3979119431] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3418062d40 con 0x7f33fc0ace00 2024-09-15T22:45:13.024 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:13.022+0000 7f340ea00640 1 -- 172.21.15.111:0/4265825655 <== osd.9 v2:172.21.15.111:6826/3979119431 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3418062d40 con 0x7f33fc0ace00 2024-09-15T22:45:13.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:13.024+0000 7f3403e00640 1 -- 172.21.15.111:0/4265825655 >> [v2:172.21.15.111:6826/3979119431,v1:172.21.15.111:6827/3979119431] conn(0x7f33fc0ace00 msgr2=0x7f33fc0af2a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:13.026 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:13.024+0000 7f3403e00640 1 --2- 172.21.15.111:0/4265825655 >> [v2:172.21.15.111:6826/3979119431,v1:172.21.15.111:6827/3979119431] conn(0x7f33fc0ace00 0x7f33fc0af2a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:13.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:13.025+0000 7f3403e00640 1 -- 172.21.15.111:0/4265825655 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f33fc0a3350 msgr2=0x7f33fc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:13.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:13.025+0000 7f3403e00640 1 --2- 172.21.15.111:0/4265825655 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f33fc0a3350 0x7f33fc0a5810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f34080096c0 tx=0x7f34080046c0 comp rx=0 tx=0).stop 2024-09-15T22:45:13.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:13.025+0000 7f3403e00640 1 -- 172.21.15.111:0/4265825655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34181047e0 msgr2=0x7f341819a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:13.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:13.025+0000 7f3403e00640 1 --2- 172.21.15.111:0/4265825655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34181047e0 0x7f341819a9b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f340400d6d0 tx=0x7f340400db90 comp rx=0 tx=0).stop 2024-09-15T22:45:13.027 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:13.026+0000 7f3416800640 1 -- 172.21.15.111:0/4265825655 reap_dead start 2024-09-15T22:45:13.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:13.026+0000 7f3403e00640 1 -- 172.21.15.111:0/4265825655 shutdown_connections 2024-09-15T22:45:13.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:13.026+0000 7f3403e00640 1 -- 172.21.15.111:0/4265825655 >> 172.21.15.111:0/4265825655 conn(0x7f34180ff8d0 msgr2=0x7f3418100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:13.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:13.026+0000 7f3403e00640 1 -- 172.21.15.111:0/4265825655 shutdown_connections 2024-09-15T22:45:13.028 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:13.026+0000 7f3403e00640 1 -- 172.21.15.111:0/4265825655 wait complete. 2024-09-15T22:45:13.923 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:13 smithi121 ceph-mon[34815]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-15T22:45:13.923 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:13 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:45:14.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:13 smithi192 ceph-mon[35427]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-15T22:45:14.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:13 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:45:14.143 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:13 smithi111 ceph-mon[30629]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-15T22:45:14.143 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:13 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:45:14.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.409+0000 7fbae8400640 1 -- 172.21.15.111:0/2062481258 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbae0104010 msgr2=0x7fbae01043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:14.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.409+0000 7fbae8400640 1 --2- 172.21.15.111:0/2062481258 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbae0104010 0x7fbae01043f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbad4009900 tx=0x7fbad402a3f0 comp rx=0 tx=0).stop 2024-09-15T22:45:14.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.409+0000 7fbae8400640 1 -- 172.21.15.111:0/2062481258 shutdown_connections 2024-09-15T22:45:14.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.409+0000 7fbae8400640 1 --2- 172.21.15.111:0/2062481258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbae0109140 0x7fbae0110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:14.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.409+0000 7fbae8400640 1 --2- 172.21.15.111:0/2062481258 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbae01049c0 0x7fbae0108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:14.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.409+0000 7fbae8400640 1 --2- 172.21.15.111:0/2062481258 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbae0104010 0x7fbae01043f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:14.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.409+0000 7fbae8400640 1 -- 172.21.15.111:0/2062481258 >> 172.21.15.111:0/2062481258 conn(0x7fbae00ffa50 msgr2=0x7fbae0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:14.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.409+0000 7fbae8400640 1 -- 172.21.15.111:0/2062481258 shutdown_connections 2024-09-15T22:45:14.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.409+0000 7fbae8400640 1 -- 172.21.15.111:0/2062481258 wait complete. 2024-09-15T22:45:14.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.410+0000 7fbae8400640 1 Processor -- start 2024-09-15T22:45:14.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.410+0000 7fbae8400640 1 -- start start 2024-09-15T22:45:14.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.411+0000 7fbae8400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbae0104010 0x7fbae019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:14.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.411+0000 7fbae8400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbae01049c0 0x7fbae019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:14.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.411+0000 7fbae8400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbae0109140 0x7fbae019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:14.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.411+0000 7fbae8400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbae0114190 con 0x7fbae0104010 2024-09-15T22:45:14.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.411+0000 7fbae8400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbae0114010 con 0x7fbae01049c0 2024-09-15T22:45:14.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.411+0000 7fbae8400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbae0114310 con 0x7fbae0109140 2024-09-15T22:45:14.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.411+0000 7fbae5a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbae0104010 0x7fbae019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:14.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.411+0000 7fbae5000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbae01049c0 0x7fbae019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:14.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.411+0000 7fbae5000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbae01049c0 0x7fbae019ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:35326/0 (socket says 172.21.15.111:35326) 2024-09-15T22:45:14.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.411+0000 7fbae5000640 1 -- 172.21.15.111:0/4066975274 learned_addr learned my addr 172.21.15.111:0/4066975274 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:14.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.411+0000 7fbae5a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbae0104010 0x7fbae019a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39994/0 (socket says 172.21.15.111:39994) 2024-09-15T22:45:14.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.411+0000 7fbae6400640 1 --2- 172.21.15.111:0/4066975274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbae0109140 0x7fbae019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:14.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.411+0000 7fbae5000640 1 -- 172.21.15.111:0/4066975274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbae0109140 msgr2=0x7fbae019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:14.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.411+0000 7fbae5000640 1 --2- 172.21.15.111:0/4066975274 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbae0109140 0x7fbae019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:14.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.411+0000 7fbae5000640 1 -- 172.21.15.111:0/4066975274 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbae0104010 msgr2=0x7fbae019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:14.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.411+0000 7fbae5000640 1 --2- 172.21.15.111:0/4066975274 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbae0104010 0x7fbae019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:14.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.411+0000 7fbae5000640 1 -- 172.21.15.111:0/4066975274 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae019f5c0 con 0x7fbae01049c0 2024-09-15T22:45:14.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.412+0000 7fbae5000640 1 --2- 172.21.15.111:0/4066975274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbae01049c0 0x7fbae019ab50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00ea60 tx=0x7fbacc00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:14.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.412+0000 7fbade000640 1 -- 172.21.15.111:0/4066975274 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc00cde0 con 0x7fbae01049c0 2024-09-15T22:45:14.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.412+0000 7fbade000640 1 -- 172.21.15.111:0/4066975274 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbacc0035f0 con 0x7fbae01049c0 2024-09-15T22:45:14.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.412+0000 7fbae8400640 1 -- 172.21.15.111:0/4066975274 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae019f8b0 con 0x7fbae01049c0 2024-09-15T22:45:14.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.412+0000 7fbade000640 1 -- 172.21.15.111:0/4066975274 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbacc010690 con 0x7fbae01049c0 2024-09-15T22:45:14.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.412+0000 7fbae8400640 1 -- 172.21.15.111:0/4066975274 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae01a7190 con 0x7fbae01049c0 2024-09-15T22:45:14.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.413+0000 7fbae8400640 1 -- 172.21.15.111:0/4066975274 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbae0062980 con 0x7fbae01049c0 2024-09-15T22:45:14.416 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.415+0000 7fbade000640 1 -- 172.21.15.111:0/4066975274 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fbacc0037a0 con 0x7fbae01049c0 2024-09-15T22:45:14.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.416+0000 7fbade000640 1 --2- 172.21.15.111:0/4066975274 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fbac80a3350 0x7fbac80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:14.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.417+0000 7fbade000640 1 -- 172.21.15.111:0/4066975274 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fbacc0d0130 con 0x7fbae01049c0 2024-09-15T22:45:14.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.417+0000 7fbae5a00640 1 --2- 172.21.15.111:0/4066975274 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fbac80a3350 0x7fbac80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:14.418 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.417+0000 7fbade000640 1 --2- 172.21.15.111:0/4066975274 >> [v2:172.21.15.192:6816/2417771374,v1:172.21.15.192:6817/2417771374] conn(0x7fbac80ace00 0x7fbac80af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:14.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.417+0000 7fbade000640 1 -- 172.21.15.111:0/4066975274 --> [v2:172.21.15.192:6816/2417771374,v1:172.21.15.192:6817/2417771374] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbacc011e50 con 0x7fbac80ace00 2024-09-15T22:45:14.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.417+0000 7fbade000640 1 -- 172.21.15.111:0/4066975274 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fbacc0d04e0 con 0x7fbae01049c0 2024-09-15T22:45:14.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.417+0000 7fbae6400640 1 --2- 172.21.15.111:0/4066975274 >> [v2:172.21.15.192:6816/2417771374,v1:172.21.15.192:6817/2417771374] conn(0x7fbac80ace00 0x7fbac80af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:14.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.417+0000 7fbae5a00640 1 --2- 172.21.15.111:0/4066975274 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fbac80a3350 0x7fbac80a5810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbad4009230 tx=0x7fbad40058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:14.419 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.418+0000 7fbae6400640 1 --2- 172.21.15.111:0/4066975274 >> [v2:172.21.15.192:6816/2417771374,v1:172.21.15.192:6817/2417771374] conn(0x7fbac80ace00 0x7fbac80af2a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:14.420 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.419+0000 7fbade000640 1 -- 172.21.15.111:0/4066975274 <== osd.8 v2:172.21.15.192:6816/2417771374 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbacc011e50 con 0x7fbac80ace00 2024-09-15T22:45:14.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.433+0000 7fbae8400640 1 -- 172.21.15.111:0/4066975274 --> [v2:172.21.15.192:6816/2417771374,v1:172.21.15.192:6817/2417771374] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbae0062d80 con 0x7fbac80ace00 2024-09-15T22:45:14.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.433+0000 7fbade000640 1 -- 172.21.15.111:0/4066975274 <== osd.8 v2:172.21.15.192:6816/2417771374 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbae0062d80 con 0x7fbac80ace00 2024-09-15T22:45:14.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.433+0000 7fbae8400640 1 -- 172.21.15.111:0/4066975274 >> [v2:172.21.15.192:6816/2417771374,v1:172.21.15.192:6817/2417771374] conn(0x7fbac80ace00 msgr2=0x7fbac80af2a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:14.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.433+0000 7fbae8400640 1 --2- 172.21.15.111:0/4066975274 >> [v2:172.21.15.192:6816/2417771374,v1:172.21.15.192:6817/2417771374] conn(0x7fbac80ace00 0x7fbac80af2a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:14.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.434+0000 7fbae8400640 1 -- 172.21.15.111:0/4066975274 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fbac80a3350 msgr2=0x7fbac80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:14.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.434+0000 7fbae8400640 1 --2- 172.21.15.111:0/4066975274 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fbac80a3350 0x7fbac80a5810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbad4009230 tx=0x7fbad40058d0 comp rx=0 tx=0).stop 2024-09-15T22:45:14.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.434+0000 7fbae8400640 1 -- 172.21.15.111:0/4066975274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbae01049c0 msgr2=0x7fbae019ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:14.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.434+0000 7fbae8400640 1 --2- 172.21.15.111:0/4066975274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbae01049c0 0x7fbae019ab50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00ea60 tx=0x7fbacc00ef20 comp rx=0 tx=0).stop 2024-09-15T22:45:14.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.434+0000 7fbae6400640 1 -- 172.21.15.111:0/4066975274 reap_dead start 2024-09-15T22:45:14.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.434+0000 7fbae8400640 1 -- 172.21.15.111:0/4066975274 shutdown_connections 2024-09-15T22:45:14.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.434+0000 7fbae8400640 1 -- 172.21.15.111:0/4066975274 >> 172.21.15.111:0/4066975274 conn(0x7fbae00ffa50 msgr2=0x7fbae0100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:14.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.434+0000 7fbae8400640 1 -- 172.21.15.111:0/4066975274 shutdown_connections 2024-09-15T22:45:14.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:14.434+0000 7fbae8400640 1 -- 172.21.15.111:0/4066975274 wait complete. 2024-09-15T22:45:14.776 INFO:teuthology.orchestra.run.smithi111.stdout:210453397551 2024-09-15T22:45:14.777 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-15T22:45:15.027 INFO:teuthology.orchestra.run.smithi111.stdout:279172874277 2024-09-15T22:45:15.028 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-15T22:45:15.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:15 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:45:15.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:15 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:45:15.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:15 smithi111 ceph-mon[30629]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-15T22:45:15.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:15 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:45:15.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:15 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:45:15.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:15 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:45:15.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:15 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:45:15.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:15 smithi192 ceph-mon[35427]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-15T22:45:15.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:15 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:45:15.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:15 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:45:15.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:15 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:45:15.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:15 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:45:15.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:15 smithi121 ceph-mon[34815]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-15T22:45:15.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:15 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:45:15.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:15 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:45:15.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.938+0000 7f0774a00640 1 -- 172.21.15.111:0/2474942261 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0770074f40 msgr2=0x7f077010dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:15.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.938+0000 7f0774a00640 1 --2- 172.21.15.111:0/2474942261 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0770074f40 0x7f077010dcd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0758009860 tx=0x7f075802a3d0 comp rx=0 tx=0).stop 2024-09-15T22:45:15.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.938+0000 7f0774a00640 1 -- 172.21.15.111:0/2474942261 shutdown_connections 2024-09-15T22:45:15.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.938+0000 7f0774a00640 1 --2- 172.21.15.111:0/2474942261 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f077010e210 0x7f0770110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:15.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.938+0000 7f0774a00640 1 --2- 172.21.15.111:0/2474942261 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0770074f40 0x7f077010dcd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:15.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.938+0000 7f0774a00640 1 --2- 172.21.15.111:0/2474942261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0770074620 0x7f0770074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:15.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.938+0000 7f0774a00640 1 -- 172.21.15.111:0/2474942261 >> 172.21.15.111:0/2474942261 conn(0x7f07700fd710 msgr2=0x7f07700ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:15.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.938+0000 7f0774a00640 1 -- 172.21.15.111:0/2474942261 shutdown_connections 2024-09-15T22:45:15.941 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.938+0000 7f0774a00640 1 -- 172.21.15.111:0/2474942261 wait complete. 2024-09-15T22:45:15.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.939+0000 7f0774a00640 1 Processor -- start 2024-09-15T22:45:15.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.939+0000 7f0774a00640 1 -- start start 2024-09-15T22:45:15.942 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.939+0000 7f0774a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0770074620 0x7f077019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:15.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.939+0000 7f0774a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0770074f40 0x7f077019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:15.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.939+0000 7f0774a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f077010e210 0x7f077019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:15.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.939+0000 7f0774a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0770113fc0 con 0x7f077010e210 2024-09-15T22:45:15.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.939+0000 7f0774a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0770113e40 con 0x7f0770074620 2024-09-15T22:45:15.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.939+0000 7f0774a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0770114140 con 0x7f0770074f40 2024-09-15T22:45:15.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.940+0000 7f076fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f077010e210 0x7f077019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:15.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.940+0000 7f076fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f077010e210 0x7f077019ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40008/0 (socket says 172.21.15.111:40008) 2024-09-15T22:45:15.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.940+0000 7f076fe00640 1 -- 172.21.15.111:0/2727718822 learned_addr learned my addr 172.21.15.111:0/2727718822 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:15.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.940+0000 7f0767e00640 1 --2- 172.21.15.111:0/2727718822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0770074f40 0x7f077019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:15.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.940+0000 7f076f400640 1 --2- 172.21.15.111:0/2727718822 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0770074620 0x7f077019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:15.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.940+0000 7f076f400640 1 -- 172.21.15.111:0/2727718822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0770074f40 msgr2=0x7f077019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:15.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.940+0000 7f076f400640 1 --2- 172.21.15.111:0/2727718822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0770074f40 0x7f077019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:15.943 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.940+0000 7f076f400640 1 -- 172.21.15.111:0/2727718822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f077010e210 msgr2=0x7f077019ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:15.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.940+0000 7f076f400640 1 --2- 172.21.15.111:0/2727718822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f077010e210 0x7f077019ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:15.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.940+0000 7f076f400640 1 -- 172.21.15.111:0/2727718822 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077019f560 con 0x7f0770074620 2024-09-15T22:45:15.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.940+0000 7f076fe00640 1 --2- 172.21.15.111:0/2727718822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f077010e210 0x7f077019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:15.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.941+0000 7f076f400640 1 --2- 172.21.15.111:0/2727718822 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0770074620 0x7f077019a5b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0760005fd0 tx=0x7f076000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:15.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.941+0000 7f076cc00640 1 -- 172.21.15.111:0/2727718822 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076000cd00 con 0x7f0770074620 2024-09-15T22:45:15.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.941+0000 7f076cc00640 1 -- 172.21.15.111:0/2727718822 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07600035f0 con 0x7f0770074620 2024-09-15T22:45:15.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.941+0000 7f076cc00640 1 -- 172.21.15.111:0/2727718822 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0760010670 con 0x7f0770074620 2024-09-15T22:45:15.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.942+0000 7f0774a00640 1 -- 172.21.15.111:0/2727718822 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f077019f850 con 0x7f0770074620 2024-09-15T22:45:15.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.942+0000 7f0774a00640 1 -- 172.21.15.111:0/2727718822 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07701a7130 con 0x7f0770074620 2024-09-15T22:45:15.945 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.942+0000 7f0774a00640 1 -- 172.21.15.111:0/2727718822 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0770062900 con 0x7f0770074620 2024-09-15T22:45:15.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.945+0000 7f076cc00640 1 -- 172.21.15.111:0/2727718822 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f07600037a0 con 0x7f0770074620 2024-09-15T22:45:15.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.946+0000 7f076cc00640 1 --2- 172.21.15.111:0/2727718822 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f07540a3350 0x7f07540a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:15.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.946+0000 7f076cc00640 1 -- 172.21.15.111:0/2727718822 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f07600cfe30 con 0x7f0770074620 2024-09-15T22:45:15.948 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.947+0000 7f0767e00640 1 --2- 172.21.15.111:0/2727718822 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f07540a3350 0x7f07540a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:15.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.947+0000 7f076cc00640 1 --2- 172.21.15.111:0/2727718822 >> [v2:172.21.15.192:6808/1958639856,v1:172.21.15.192:6809/1958639856] conn(0x7f07540ace00 0x7f07540af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:15.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.947+0000 7f076cc00640 1 -- 172.21.15.111:0/2727718822 --> [v2:172.21.15.192:6808/1958639856,v1:172.21.15.192:6809/1958639856] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f07540af970 con 0x7f07540ace00 2024-09-15T22:45:15.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.947+0000 7f076cc00640 1 -- 172.21.15.111:0/2727718822 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f07600d01e0 con 0x7f0770074620 2024-09-15T22:45:15.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.947+0000 7f076fe00640 1 --2- 172.21.15.111:0/2727718822 >> [v2:172.21.15.192:6808/1958639856,v1:172.21.15.192:6809/1958639856] conn(0x7f07540ace00 0x7f07540af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:15.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.947+0000 7f0767e00640 1 --2- 172.21.15.111:0/2727718822 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f07540a3350 0x7f07540a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f077010f660 tx=0x7f07580094d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:15.949 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.948+0000 7f076fe00640 1 --2- 172.21.15.111:0/2727718822 >> [v2:172.21.15.192:6808/1958639856,v1:172.21.15.192:6809/1958639856] conn(0x7f07540ace00 0x7f07540af2a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:15.950 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.948+0000 7f076cc00640 1 -- 172.21.15.111:0/2727718822 <== osd.5 v2:172.21.15.192:6808/1958639856 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f07540af970 con 0x7f07540ace00 2024-09-15T22:45:15.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.963+0000 7f0774a00640 1 -- 172.21.15.111:0/2727718822 --> [v2:172.21.15.192:6808/1958639856,v1:172.21.15.192:6809/1958639856] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0770062d50 con 0x7f07540ace00 2024-09-15T22:45:15.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.963+0000 7f076cc00640 1 -- 172.21.15.111:0/2727718822 <== osd.5 v2:172.21.15.192:6808/1958639856 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0770062d50 con 0x7f07540ace00 2024-09-15T22:45:15.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.964+0000 7f0774a00640 1 -- 172.21.15.111:0/2727718822 >> [v2:172.21.15.192:6808/1958639856,v1:172.21.15.192:6809/1958639856] conn(0x7f07540ace00 msgr2=0x7f07540af2a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:15.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.964+0000 7f0774a00640 1 --2- 172.21.15.111:0/2727718822 >> [v2:172.21.15.192:6808/1958639856,v1:172.21.15.192:6809/1958639856] conn(0x7f07540ace00 0x7f07540af2a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:15.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.964+0000 7f0774a00640 1 -- 172.21.15.111:0/2727718822 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f07540a3350 msgr2=0x7f07540a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:15.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.964+0000 7f0774a00640 1 --2- 172.21.15.111:0/2727718822 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f07540a3350 0x7f07540a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f077010f660 tx=0x7f07580094d0 comp rx=0 tx=0).stop 2024-09-15T22:45:15.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.964+0000 7f0774a00640 1 -- 172.21.15.111:0/2727718822 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0770074620 msgr2=0x7f077019a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:15.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.964+0000 7f0774a00640 1 --2- 172.21.15.111:0/2727718822 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0770074620 0x7f077019a5b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0760005fd0 tx=0x7f076000ee50 comp rx=0 tx=0).stop 2024-09-15T22:45:15.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.964+0000 7f076fe00640 1 -- 172.21.15.111:0/2727718822 reap_dead start 2024-09-15T22:45:15.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.964+0000 7f0774a00640 1 -- 172.21.15.111:0/2727718822 shutdown_connections 2024-09-15T22:45:15.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.964+0000 7f0774a00640 1 -- 172.21.15.111:0/2727718822 >> 172.21.15.111:0/2727718822 conn(0x7f07700fd710 msgr2=0x7f07700fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:15.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.964+0000 7f0774a00640 1 -- 172.21.15.111:0/2727718822 shutdown_connections 2024-09-15T22:45:15.966 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:15.964+0000 7f0774a00640 1 -- 172.21.15.111:0/2727718822 wait complete. 2024-09-15T22:45:16.281 INFO:teuthology.orchestra.run.smithi111.stdout:77309411400 2024-09-15T22:45:16.282 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-15T22:45:16.613 INFO:teuthology.orchestra.run.smithi111.stdout:335007449112 2024-09-15T22:45:16.614 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-15T22:45:16.971 INFO:teuthology.orchestra.run.smithi111.stdout:244813135913 2024-09-15T22:45:16.972 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-15T22:45:17.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:17 smithi111 ceph-mon[30629]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-15T22:45:17.754 INFO:teuthology.orchestra.run.smithi111.stdout:158913790011 2024-09-15T22:45:17.754 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-15T22:45:17.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:17 smithi192 ceph-mon[35427]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-15T22:45:17.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:17 smithi121 ceph-mon[34815]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-15T22:45:18.234 INFO:teuthology.orchestra.run.smithi111.stdout:304942678045 2024-09-15T22:45:18.235 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-15T22:45:18.277 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:18.480 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:18.507 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:18.508 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:18.508 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:18.519 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:19.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.448+0000 7f3e71600640 1 -- 172.21.15.111:0/3491829511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e6c1019d0 msgr2=0x7f3e6c101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:19.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.448+0000 7f3e71600640 1 --2- 172.21.15.111:0/3491829511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e6c1019d0 0x7f3e6c101e30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3e58009930 tx=0x7f3e5802a440 comp rx=0 tx=0).stop 2024-09-15T22:45:19.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.448+0000 7f3e71600640 1 -- 172.21.15.111:0/3491829511 shutdown_connections 2024-09-15T22:45:19.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.448+0000 7f3e71600640 1 --2- 172.21.15.111:0/3491829511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e6c102370 0x7f3e6c10e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:19.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.448+0000 7f3e71600640 1 --2- 172.21.15.111:0/3491829511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e6c1019d0 0x7f3e6c101e30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:19.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.448+0000 7f3e71600640 1 --2- 172.21.15.111:0/3491829511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e6c1079d0 0x7f3e6c107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:19.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.448+0000 7f3e71600640 1 -- 172.21.15.111:0/3491829511 >> 172.21.15.111:0/3491829511 conn(0x7f3e6c0fd710 msgr2=0x7f3e6c0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:19.450 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.448+0000 7f3e71600640 1 -- 172.21.15.111:0/3491829511 shutdown_connections 2024-09-15T22:45:19.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.448+0000 7f3e71600640 1 -- 172.21.15.111:0/3491829511 wait complete. 2024-09-15T22:45:19.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.450+0000 7f3e71600640 1 Processor -- start 2024-09-15T22:45:19.451 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.450+0000 7f3e71600640 1 -- start start 2024-09-15T22:45:19.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.450+0000 7f3e71600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e6c1019d0 0x7f3e6c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:19.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.450+0000 7f3e71600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e6c102370 0x7f3e6c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:19.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.450+0000 7f3e71600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e6c1079d0 0x7f3e6c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:19.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.450+0000 7f3e71600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c111e20 con 0x7f3e6c1079d0 2024-09-15T22:45:19.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.450+0000 7f3e71600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c111ca0 con 0x7f3e6c102370 2024-09-15T22:45:19.452 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.450+0000 7f3e71600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c111fa0 con 0x7f3e6c1019d0 2024-09-15T22:45:19.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.450+0000 7f3e70c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e6c1079d0 0x7f3e6c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:19.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.450+0000 7f3e70c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e6c1079d0 0x7f3e6c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40052/0 (socket says 172.21.15.111:40052) 2024-09-15T22:45:19.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.450+0000 7f3e6be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e6c1019d0 0x7f3e6c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:19.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.451+0000 7f3e70c00640 1 -- 172.21.15.111:0/3001369919 learned_addr learned my addr 172.21.15.111:0/3001369919 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:19.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.450+0000 7f3e6b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e6c102370 0x7f3e6c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:19.453 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.451+0000 7f3e6be00640 1 -- 172.21.15.111:0/3001369919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e6c102370 msgr2=0x7f3e6c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:19.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.451+0000 7f3e6be00640 1 --2- 172.21.15.111:0/3001369919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e6c102370 0x7f3e6c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:19.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.451+0000 7f3e6be00640 1 -- 172.21.15.111:0/3001369919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e6c1079d0 msgr2=0x7f3e6c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:19.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.451+0000 7f3e6be00640 1 --2- 172.21.15.111:0/3001369919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e6c1079d0 0x7f3e6c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:19.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.451+0000 7f3e6be00640 1 -- 172.21.15.111:0/3001369919 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6c1a67e0 con 0x7f3e6c1019d0 2024-09-15T22:45:19.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.451+0000 7f3e6be00640 1 --2- 172.21.15.111:0/3001369919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e6c1019d0 0x7f3e6c0744f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00e9f0 tx=0x7f3e5c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:19.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.451+0000 7f3e68c00640 1 -- 172.21.15.111:0/3001369919 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c00ccc0 con 0x7f3e6c1019d0 2024-09-15T22:45:19.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.452+0000 7f3e68c00640 1 -- 172.21.15.111:0/3001369919 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e5c010910 con 0x7f3e6c1019d0 2024-09-15T22:45:19.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.452+0000 7f3e71600640 1 -- 172.21.15.111:0/3001369919 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e6c1a6ad0 con 0x7f3e6c1019d0 2024-09-15T22:45:19.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.452+0000 7f3e68c00640 1 -- 172.21.15.111:0/3001369919 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c018960 con 0x7f3e6c1019d0 2024-09-15T22:45:19.454 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.452+0000 7f3e71600640 1 -- 172.21.15.111:0/3001369919 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e6c1a7030 con 0x7f3e6c1019d0 2024-09-15T22:45:19.455 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.453+0000 7f3e71600640 1 -- 172.21.15.111:0/3001369919 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e6c101e30 con 0x7f3e6c1019d0 2024-09-15T22:45:19.456 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.455+0000 7f3e68c00640 1 -- 172.21.15.111:0/3001369919 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f3e5c010ac0 con 0x7f3e6c1019d0 2024-09-15T22:45:19.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.456+0000 7f3e68c00640 1 --2- 172.21.15.111:0/3001369919 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3e500a3350 0x7f3e500a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:19.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.456+0000 7f3e68c00640 1 -- 172.21.15.111:0/3001369919 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f3e5c0cfd30 con 0x7f3e6c1019d0 2024-09-15T22:45:19.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.457+0000 7f3e6b400640 1 --2- 172.21.15.111:0/3001369919 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3e500a3350 0x7f3e500a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:19.458 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.457+0000 7f3e6b400640 1 --2- 172.21.15.111:0/3001369919 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3e500a3350 0x7f3e500a5810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3e58006010 tx=0x7f3e58000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:19.461 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.460+0000 7f3e68c00640 1 -- 172.21.15.111:0/3001369919 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3e5c09c2a0 con 0x7f3e6c1019d0 2024-09-15T22:45:19.653 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.651+0000 7f3e71600640 1 -- 172.21.15.111:0/3001369919 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f3e6c109f40 con 0x7f3e6c1019d0 2024-09-15T22:45:19.653 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:19 smithi111 ceph-mon[30629]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:45:19.653 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:19 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/1344887746' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-15T22:45:19.654 INFO:teuthology.orchestra.run.smithi111.stdout:77309411402 2024-09-15T22:45:19.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.652+0000 7f3e68c00640 1 -- 172.21.15.111:0/3001369919 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3e5c0a05a0 con 0x7f3e6c1019d0 2024-09-15T22:45:19.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.655+0000 7f3e71600640 1 -- 172.21.15.111:0/3001369919 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3e500a3350 msgr2=0x7f3e500a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:19.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.655+0000 7f3e71600640 1 --2- 172.21.15.111:0/3001369919 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3e500a3350 0x7f3e500a5810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3e58006010 tx=0x7f3e58000bc0 comp rx=0 tx=0).stop 2024-09-15T22:45:19.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.655+0000 7f3e71600640 1 -- 172.21.15.111:0/3001369919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e6c1019d0 msgr2=0x7f3e6c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:19.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.655+0000 7f3e71600640 1 --2- 172.21.15.111:0/3001369919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e6c1019d0 0x7f3e6c0744f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00e9f0 tx=0x7f3e5c00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:45:19.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.655+0000 7f3e71600640 1 -- 172.21.15.111:0/3001369919 shutdown_connections 2024-09-15T22:45:19.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.655+0000 7f3e71600640 1 --2- 172.21.15.111:0/3001369919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e6c1079d0 0x7f3e6c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:19.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.655+0000 7f3e71600640 1 --2- 172.21.15.111:0/3001369919 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3e500a3350 0x7f3e500a5810 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:19.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.655+0000 7f3e71600640 1 --2- 172.21.15.111:0/3001369919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e6c102370 0x7f3e6c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:19.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.655+0000 7f3e71600640 1 --2- 172.21.15.111:0/3001369919 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e6c1019d0 0x7f3e6c0744f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:19.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.655+0000 7f3e71600640 1 -- 172.21.15.111:0/3001369919 >> 172.21.15.111:0/3001369919 conn(0x7f3e6c0fd710 msgr2=0x7f3e6c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:19.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.655+0000 7f3e71600640 1 -- 172.21.15.111:0/3001369919 shutdown_connections 2024-09-15T22:45:19.657 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.655+0000 7f3e71600640 1 -- 172.21.15.111:0/3001369919 wait complete. 2024-09-15T22:45:19.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:19 smithi192 ceph-mon[35427]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:45:19.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:19 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/1344887746' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-15T22:45:19.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:19 smithi121 ceph-mon[34815]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:45:19.891 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:19 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/1344887746' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-15T22:45:19.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.958+0000 7f0912800640 1 -- 172.21.15.111:0/1526776587 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f090c074620 msgr2=0x7f090c074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:19.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.958+0000 7f0912800640 1 --2- 172.21.15.111:0/1526776587 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f090c074620 0x7f090c074a00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009900 tx=0x7f08fc02a480 comp rx=0 tx=0).stop 2024-09-15T22:45:19.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.958+0000 7f0912800640 1 -- 172.21.15.111:0/1526776587 shutdown_connections 2024-09-15T22:45:19.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.958+0000 7f0912800640 1 --2- 172.21.15.111:0/1526776587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f090c10e3c0 0x7f090c110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:19.960 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.958+0000 7f0912800640 1 --2- 172.21.15.111:0/1526776587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f090c074f40 0x7f090c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:19.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.958+0000 7f0912800640 1 --2- 172.21.15.111:0/1526776587 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f090c074620 0x7f090c074a00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:19.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.958+0000 7f0912800640 1 -- 172.21.15.111:0/1526776587 >> 172.21.15.111:0/1526776587 conn(0x7f090c0fd8c0 msgr2=0x7f090c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:19.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.959+0000 7f0912800640 1 -- 172.21.15.111:0/1526776587 shutdown_connections 2024-09-15T22:45:19.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.959+0000 7f0912800640 1 -- 172.21.15.111:0/1526776587 wait complete. 2024-09-15T22:45:19.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.960+0000 7f0912800640 1 Processor -- start 2024-09-15T22:45:19.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.960+0000 7f0912800640 1 -- start start 2024-09-15T22:45:19.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.960+0000 7f0912800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f090c074620 0x7f090c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:19.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.960+0000 7f0912800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f090c074f40 0x7f090c19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:19.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.960+0000 7f0912800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f090c10e3c0 0x7f090c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:19.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.960+0000 7f0912800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f090c114170 con 0x7f090c10e3c0 2024-09-15T22:45:19.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.960+0000 7f0912800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f090c113ff0 con 0x7f090c074f40 2024-09-15T22:45:19.962 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.960+0000 7f0912800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f090c1142f0 con 0x7f090c074620 2024-09-15T22:45:19.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.961+0000 7f090b400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f090c074620 0x7f090c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:19.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.961+0000 7f090aa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f090c074f40 0x7f090c19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:19.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.961+0000 7f090be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f090c10e3c0 0x7f090c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:19.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.961+0000 7f090aa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f090c074f40 0x7f090c19abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:35408/0 (socket says 172.21.15.111:35408) 2024-09-15T22:45:19.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.961+0000 7f090b400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f090c074620 0x7f090c19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:36684/0 (socket says 172.21.15.111:36684) 2024-09-15T22:45:19.963 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.961+0000 7f090aa00640 1 -- 172.21.15.111:0/1725006449 learned_addr learned my addr 172.21.15.111:0/1725006449 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:19.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.961+0000 7f090aa00640 1 -- 172.21.15.111:0/1725006449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f090c074620 msgr2=0x7f090c19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:19.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.961+0000 7f090aa00640 1 --2- 172.21.15.111:0/1725006449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f090c074620 0x7f090c19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:19.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.961+0000 7f090aa00640 1 -- 172.21.15.111:0/1725006449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f090c10e3c0 msgr2=0x7f090c19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:19.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.961+0000 7f090aa00640 1 --2- 172.21.15.111:0/1725006449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f090c10e3c0 0x7f090c19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:19.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.961+0000 7f090aa00640 1 -- 172.21.15.111:0/1725006449 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f090c19f640 con 0x7f090c074f40 2024-09-15T22:45:19.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.961+0000 7f090aa00640 1 --2- 172.21.15.111:0/1725006449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f090c074f40 0x7f090c19abd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f08f800e9f0 tx=0x7f08f800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:19.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.962+0000 7f0903e00640 1 -- 172.21.15.111:0/1725006449 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f800ccc0 con 0x7f090c074f40 2024-09-15T22:45:19.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.962+0000 7f0903e00640 1 -- 172.21.15.111:0/1725006449 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08f800ce60 con 0x7f090c074f40 2024-09-15T22:45:19.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.962+0000 7f0912800640 1 -- 172.21.15.111:0/1725006449 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f090c19f930 con 0x7f090c074f40 2024-09-15T22:45:19.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.962+0000 7f0903e00640 1 -- 172.21.15.111:0/1725006449 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f8010690 con 0x7f090c074f40 2024-09-15T22:45:19.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.962+0000 7f0912800640 1 -- 172.21.15.111:0/1725006449 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f090c1a7210 con 0x7f090c074f40 2024-09-15T22:45:19.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.963+0000 7f0912800640 1 -- 172.21.15.111:0/1725006449 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f090c074a00 con 0x7f090c074f40 2024-09-15T22:45:19.967 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.965+0000 7f0903e00640 1 -- 172.21.15.111:0/1725006449 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f08f80046c0 con 0x7f090c074f40 2024-09-15T22:45:19.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.967+0000 7f0903e00640 1 --2- 172.21.15.111:0/1725006449 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f08f00a32d0 0x7f08f00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:19.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.967+0000 7f0903e00640 1 -- 172.21.15.111:0/1725006449 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f08f80d0120 con 0x7f090c074f40 2024-09-15T22:45:19.968 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.967+0000 7f090b400640 1 --2- 172.21.15.111:0/1725006449 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f08f00a32d0 0x7f08f00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:19.969 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.967+0000 7f090b400640 1 --2- 172.21.15.111:0/1725006449 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f08f00a32d0 0x7f08f00a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009230 tx=0x7f08fc0058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:19.970 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.968+0000 7f0903e00640 1 -- 172.21.15.111:0/1725006449 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f08f809cce0 con 0x7f090c074f40 2024-09-15T22:45:20.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:20.147+0000 7f0912800640 1 -- 172.21.15.111:0/1725006449 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f090c19b9f0 con 0x7f090c074f40 2024-09-15T22:45:20.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:20.148+0000 7f0903e00640 1 -- 172.21.15.111:0/1725006449 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f090c19b9f0 con 0x7f090c074f40 2024-09-15T22:45:20.150 INFO:teuthology.orchestra.run.smithi111.stdout:55834574928 2024-09-15T22:45:20.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:20.150+0000 7f0912800640 1 -- 172.21.15.111:0/1725006449 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f08f00a32d0 msgr2=0x7f08f00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:20.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:20.150+0000 7f0912800640 1 --2- 172.21.15.111:0/1725006449 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f08f00a32d0 0x7f08f00a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009230 tx=0x7f08fc0058d0 comp rx=0 tx=0).stop 2024-09-15T22:45:20.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:20.150+0000 7f0912800640 1 -- 172.21.15.111:0/1725006449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f090c074f40 msgr2=0x7f090c19abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:20.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:20.150+0000 7f0912800640 1 --2- 172.21.15.111:0/1725006449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f090c074f40 0x7f090c19abd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f08f800e9f0 tx=0x7f08f800edf0 comp rx=0 tx=0).stop 2024-09-15T22:45:20.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:20.150+0000 7f0912800640 1 -- 172.21.15.111:0/1725006449 shutdown_connections 2024-09-15T22:45:20.152 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:20.150+0000 7f0912800640 1 --2- 172.21.15.111:0/1725006449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f090c10e3c0 0x7f090c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:20.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:20.150+0000 7f0912800640 1 --2- 172.21.15.111:0/1725006449 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f08f00a32d0 0x7f08f00a5790 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:20.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:20.150+0000 7f0912800640 1 --2- 172.21.15.111:0/1725006449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f090c074f40 0x7f090c19abd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:20.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:20.150+0000 7f0912800640 1 --2- 172.21.15.111:0/1725006449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f090c074620 0x7f090c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:20.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:20.150+0000 7f0912800640 1 -- 172.21.15.111:0/1725006449 >> 172.21.15.111:0/1725006449 conn(0x7f090c0fd8c0 msgr2=0x7f090c0ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:20.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:20.150+0000 7f0912800640 1 -- 172.21.15.111:0/1725006449 shutdown_connections 2024-09-15T22:45:20.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:20.150+0000 7f0912800640 1 -- 172.21.15.111:0/1725006449 wait complete. 2024-09-15T22:45:20.789 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:20 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/3001369919' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-15T22:45:20.790 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:20 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/1725006449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-15T22:45:20.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:20 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/3001369919' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-15T22:45:20.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:20 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/1725006449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-15T22:45:20.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:20 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/3001369919' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-15T22:45:20.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:20 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/1725006449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-15T22:45:21.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.308+0000 7fb2db200640 1 -- 172.21.15.111:0/2411453958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2dc1047e0 msgr2=0x7fb2dc108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:21.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.308+0000 7fb2db200640 1 --2- 172.21.15.111:0/2411453958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2dc1047e0 0x7fb2dc108830 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009900 tx=0x7fb2c802a410 comp rx=0 tx=0).stop 2024-09-15T22:45:21.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.308+0000 7fb2db200640 1 -- 172.21.15.111:0/2411453958 shutdown_connections 2024-09-15T22:45:21.369 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.308+0000 7fb2db200640 1 --2- 172.21.15.111:0/2411453958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2dc108f60 0x7fb2dc110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:21.369 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.308+0000 7fb2db200640 1 --2- 172.21.15.111:0/2411453958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2dc1047e0 0x7fb2dc108830 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:21.369 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.308+0000 7fb2db200640 1 --2- 172.21.15.111:0/2411453958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2dc103e30 0x7fb2dc104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:21.369 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.308+0000 7fb2db200640 1 -- 172.21.15.111:0/2411453958 >> 172.21.15.111:0/2411453958 conn(0x7fb2dc0ff8d0 msgr2=0x7fb2dc101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:21.369 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.308+0000 7fb2db200640 1 -- 172.21.15.111:0/2411453958 shutdown_connections 2024-09-15T22:45:21.369 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.308+0000 7fb2db200640 1 -- 172.21.15.111:0/2411453958 wait complete. 2024-09-15T22:45:21.369 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.309+0000 7fb2db200640 1 Processor -- start 2024-09-15T22:45:21.369 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.309+0000 7fb2db200640 1 -- start start 2024-09-15T22:45:21.369 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.309+0000 7fb2db200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2dc103e30 0x7fb2dc19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:21.369 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.309+0000 7fb2db200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2dc1047e0 0x7fb2dc19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:21.369 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.310+0000 7fb2db200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2dc108f60 0x7fb2dc19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:21.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.310+0000 7fb2db200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc113fc0 con 0x7fb2dc108f60 2024-09-15T22:45:21.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.310+0000 7fb2db200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc113e40 con 0x7fb2dc103e30 2024-09-15T22:45:21.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.310+0000 7fb2db200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc114140 con 0x7fb2dc1047e0 2024-09-15T22:45:21.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.310+0000 7fb2d9400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2dc1047e0 0x7fb2dc19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:21.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.310+0000 7fb2da800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2dc108f60 0x7fb2dc19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:21.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.310+0000 7fb2d9400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2dc1047e0 0x7fb2dc19a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:36640/0 (socket says 172.21.15.111:36640) 2024-09-15T22:45:21.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.310+0000 7fb2d9400640 1 -- 172.21.15.111:0/1344887746 learned_addr learned my addr 172.21.15.111:0/1344887746 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:21.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.310+0000 7fb2d9e00640 1 --2- 172.21.15.111:0/1344887746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2dc103e30 0x7fb2dc19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:21.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.310+0000 7fb2d9400640 1 -- 172.21.15.111:0/1344887746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2dc103e30 msgr2=0x7fb2dc19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:21.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.310+0000 7fb2d9400640 1 --2- 172.21.15.111:0/1344887746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2dc103e30 0x7fb2dc19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:21.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.310+0000 7fb2d9400640 1 -- 172.21.15.111:0/1344887746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2dc108f60 msgr2=0x7fb2dc19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:21.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.310+0000 7fb2d9400640 1 --2- 172.21.15.111:0/1344887746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2dc108f60 0x7fb2dc19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:21.370 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.310+0000 7fb2d9400640 1 -- 172.21.15.111:0/1344887746 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2dc19f410 con 0x7fb2dc1047e0 2024-09-15T22:45:21.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.310+0000 7fb2da800640 1 --2- 172.21.15.111:0/1344887746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2dc108f60 0x7fb2dc19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:21.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.311+0000 7fb2d9400640 1 --2- 172.21.15.111:0/1344887746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2dc1047e0 0x7fb2dc19a9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8006010 tx=0x7fb2c8004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:21.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.311+0000 7fb2d2a00640 1 -- 172.21.15.111:0/1344887746 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c803f030 con 0x7fb2dc1047e0 2024-09-15T22:45:21.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.311+0000 7fb2db200640 1 -- 172.21.15.111:0/1344887746 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2dc19f6a0 con 0x7fb2dc1047e0 2024-09-15T22:45:21.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.311+0000 7fb2d2a00640 1 -- 172.21.15.111:0/1344887746 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2c8004440 con 0x7fb2dc1047e0 2024-09-15T22:45:21.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.311+0000 7fb2d2a00640 1 -- 172.21.15.111:0/1344887746 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8035070 con 0x7fb2dc1047e0 2024-09-15T22:45:21.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.311+0000 7fb2db200640 1 -- 172.21.15.111:0/1344887746 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2dc1a6fe0 con 0x7fb2dc1047e0 2024-09-15T22:45:21.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.312+0000 7fb2db200640 1 -- 172.21.15.111:0/1344887746 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2dc104210 con 0x7fb2dc1047e0 2024-09-15T22:45:21.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.314+0000 7fb2d2a00640 1 -- 172.21.15.111:0/1344887746 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fb2c8042030 con 0x7fb2dc1047e0 2024-09-15T22:45:21.371 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.315+0000 7fb2d2a00640 1 --2- 172.21.15.111:0/1344887746 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb2c00a32d0 0x7fb2c00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:21.372 INFO:teuthology.orchestra.run.smithi111.stdout:137438953534 2024-09-15T22:45:21.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.315+0000 7fb2d2a00640 1 -- 172.21.15.111:0/1344887746 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fb2c80effe0 con 0x7fb2dc1047e0 2024-09-15T22:45:21.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.315+0000 7fb2d9e00640 1 --2- 172.21.15.111:0/1344887746 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb2c00a32d0 0x7fb2c00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:21.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.316+0000 7fb2d9e00640 1 --2- 172.21.15.111:0/1344887746 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb2c00a32d0 0x7fb2c00a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009bd0 tx=0x7fb2cc005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:21.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.318+0000 7fb2d2a00640 1 -- 172.21.15.111:0/1344887746 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb2c80bc590 con 0x7fb2dc1047e0 2024-09-15T22:45:21.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.507+0000 7fb2db200640 1 -- 172.21.15.111:0/1344887746 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fb2dc10c1a0 con 0x7fb2dc1047e0 2024-09-15T22:45:21.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.507+0000 7fb2d2a00640 1 -- 172.21.15.111:0/1344887746 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb2c80c0890 con 0x7fb2dc1047e0 2024-09-15T22:45:21.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.510+0000 7fb2db200640 1 -- 172.21.15.111:0/1344887746 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb2c00a32d0 msgr2=0x7fb2c00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:21.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.510+0000 7fb2db200640 1 --2- 172.21.15.111:0/1344887746 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb2c00a32d0 0x7fb2c00a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009bd0 tx=0x7fb2cc005f90 comp rx=0 tx=0).stop 2024-09-15T22:45:21.372 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.510+0000 7fb2db200640 1 -- 172.21.15.111:0/1344887746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2dc1047e0 msgr2=0x7fb2dc19a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:21.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.510+0000 7fb2db200640 1 --2- 172.21.15.111:0/1344887746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2dc1047e0 0x7fb2dc19a9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8006010 tx=0x7fb2c8004040 comp rx=0 tx=0).stop 2024-09-15T22:45:21.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.510+0000 7fb2db200640 1 -- 172.21.15.111:0/1344887746 shutdown_connections 2024-09-15T22:45:21.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.510+0000 7fb2db200640 1 --2- 172.21.15.111:0/1344887746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2dc108f60 0x7fb2dc19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:21.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.510+0000 7fb2db200640 1 --2- 172.21.15.111:0/1344887746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2dc1047e0 0x7fb2dc19a9a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:21.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.510+0000 7fb2db200640 1 --2- 172.21.15.111:0/1344887746 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb2c00a32d0 0x7fb2c00a5790 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:21.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.510+0000 7fb2db200640 1 --2- 172.21.15.111:0/1344887746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb2dc103e30 0x7fb2dc19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:21.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.510+0000 7fb2db200640 1 -- 172.21.15.111:0/1344887746 >> 172.21.15.111:0/1344887746 conn(0x7fb2dc0ff8d0 msgr2=0x7fb2dc100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:21.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.510+0000 7fb2db200640 1 -- 172.21.15.111:0/1344887746 shutdown_connections 2024-09-15T22:45:21.373 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:19.510+0000 7fb2db200640 1 -- 172.21.15.111:0/1344887746 wait complete. 2024-09-15T22:45:21.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:21 smithi111 ceph-mon[30629]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:45:21.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:21 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/2491595714' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-15T22:45:21.782 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:21.782 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:21.783 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:21.783 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:21.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:21 smithi192 ceph-mon[35427]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:45:21.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:21 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/2491595714' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-15T22:45:21.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:21 smithi121 ceph-mon[34815]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-15T22:45:21.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:21 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/2491595714' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-15T22:45:21.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.346+0000 7fa751000640 1 -- 172.21.15.111:0/4131380578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa74c109120 msgr2=0x7fa74c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:21.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.346+0000 7fa751000640 1 --2- 172.21.15.111:0/4131380578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa74c109120 0x7fa74c110ca0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa744009ca0 tx=0x7fa74402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:45:21.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.347+0000 7fa751000640 1 -- 172.21.15.111:0/4131380578 shutdown_connections 2024-09-15T22:45:21.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.347+0000 7fa751000640 1 --2- 172.21.15.111:0/4131380578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa74c109120 0x7fa74c110ca0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:21.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.347+0000 7fa751000640 1 --2- 172.21.15.111:0/4131380578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa74c1049a0 0x7fa74c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:21.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.347+0000 7fa751000640 1 --2- 172.21.15.111:0/4131380578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa74c103ff0 0x7fa74c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:21.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.347+0000 7fa751000640 1 -- 172.21.15.111:0/4131380578 >> 172.21.15.111:0/4131380578 conn(0x7fa74c0ffa90 msgr2=0x7fa74c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:21.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.347+0000 7fa751000640 1 -- 172.21.15.111:0/4131380578 shutdown_connections 2024-09-15T22:45:21.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.347+0000 7fa751000640 1 -- 172.21.15.111:0/4131380578 wait complete. 2024-09-15T22:45:21.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.348+0000 7fa751000640 1 Processor -- start 2024-09-15T22:45:21.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.348+0000 7fa751000640 1 -- start start 2024-09-15T22:45:21.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.348+0000 7fa751000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa74c103ff0 0x7fa74c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:21.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.348+0000 7fa751000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa74c1049a0 0x7fa74c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:21.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.348+0000 7fa751000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa74c109120 0x7fa74c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:21.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.348+0000 7fa751000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c114130 con 0x7fa74c109120 2024-09-15T22:45:21.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.348+0000 7fa751000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c113fb0 con 0x7fa74c1049a0 2024-09-15T22:45:21.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.348+0000 7fa751000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa74c1142b0 con 0x7fa74c103ff0 2024-09-15T22:45:21.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.348+0000 7fa74a800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa74c109120 0x7fa74c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:21.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.348+0000 7fa749400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa74c1049a0 0x7fa74c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:21.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.349+0000 7fa74a800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa74c109120 0x7fa74c19eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60642/0 (socket says 172.21.15.111:60642) 2024-09-15T22:45:21.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.349+0000 7fa74a800640 1 -- 172.21.15.111:0/3565483100 learned_addr learned my addr 172.21.15.111:0/3565483100 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:21.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.349+0000 7fa749e00640 1 --2- 172.21.15.111:0/3565483100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa74c103ff0 0x7fa74c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:21.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.349+0000 7fa749400640 1 -- 172.21.15.111:0/3565483100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa74c103ff0 msgr2=0x7fa74c19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:21.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.349+0000 7fa749400640 1 --2- 172.21.15.111:0/3565483100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa74c103ff0 0x7fa74c19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:21.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.349+0000 7fa749400640 1 -- 172.21.15.111:0/3565483100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa74c109120 msgr2=0x7fa74c19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:21.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.349+0000 7fa749400640 1 --2- 172.21.15.111:0/3565483100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa74c109120 0x7fa74c19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:21.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.349+0000 7fa749400640 1 -- 172.21.15.111:0/3565483100 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa74c19f5c0 con 0x7fa74c1049a0 2024-09-15T22:45:21.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.349+0000 7fa749400640 1 --2- 172.21.15.111:0/3565483100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa74c1049a0 0x7fa74c19ab50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa73800e9f0 tx=0x7fa73800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:21.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.349+0000 7fa742a00640 1 -- 172.21.15.111:0/3565483100 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73800ccc0 con 0x7fa74c1049a0 2024-09-15T22:45:21.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.350+0000 7fa742a00640 1 -- 172.21.15.111:0/3565483100 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa738010910 con 0x7fa74c1049a0 2024-09-15T22:45:21.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.350+0000 7fa751000640 1 -- 172.21.15.111:0/3565483100 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa74c19f8b0 con 0x7fa74c1049a0 2024-09-15T22:45:21.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.350+0000 7fa742a00640 1 -- 172.21.15.111:0/3565483100 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa738018960 con 0x7fa74c1049a0 2024-09-15T22:45:21.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.350+0000 7fa751000640 1 -- 172.21.15.111:0/3565483100 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa74c1a7190 con 0x7fa74c1049a0 2024-09-15T22:45:21.916 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.351+0000 7fa751000640 1 -- 172.21.15.111:0/3565483100 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa74c1043d0 con 0x7fa74c1049a0 2024-09-15T22:45:21.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.353+0000 7fa742a00640 1 -- 172.21.15.111:0/3565483100 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fa738010ac0 con 0x7fa74c1049a0 2024-09-15T22:45:21.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.354+0000 7fa742a00640 1 --2- 172.21.15.111:0/3565483100 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa7300a3350 0x7fa7300a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:21.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.354+0000 7fa742a00640 1 -- 172.21.15.111:0/3565483100 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fa7380d01b0 con 0x7fa74c1049a0 2024-09-15T22:45:21.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.354+0000 7fa749e00640 1 --2- 172.21.15.111:0/3565483100 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa7300a3350 0x7fa7300a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:21.917 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.355+0000 7fa749e00640 1 --2- 172.21.15.111:0/3565483100 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa7300a3350 0x7fa7300a5810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa73c009bd0 tx=0x7fa73c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:21.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.356+0000 7fa742a00640 1 -- 172.21.15.111:0/3565483100 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa73809c720 con 0x7fa74c1049a0 2024-09-15T22:45:21.918 INFO:teuthology.orchestra.run.smithi111.stdout:188978561076 2024-09-15T22:45:21.918 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.546+0000 7fa751000640 1 -- 172.21.15.111:0/3565483100 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fa74c02d7c0 con 0x7fa74c1049a0 2024-09-15T22:45:21.919 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.546+0000 7fa742a00640 1 -- 172.21.15.111:0/3565483100 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa7380a0a20 con 0x7fa74c1049a0 2024-09-15T22:45:21.919 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.548+0000 7fa751000640 1 -- 172.21.15.111:0/3565483100 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa7300a3350 msgr2=0x7fa7300a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:21.919 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.548+0000 7fa751000640 1 --2- 172.21.15.111:0/3565483100 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa7300a3350 0x7fa7300a5810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa73c009bd0 tx=0x7fa73c005f90 comp rx=0 tx=0).stop 2024-09-15T22:45:21.919 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.548+0000 7fa751000640 1 -- 172.21.15.111:0/3565483100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa74c1049a0 msgr2=0x7fa74c19ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:21.919 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.548+0000 7fa751000640 1 --2- 172.21.15.111:0/3565483100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa74c1049a0 0x7fa74c19ab50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa73800e9f0 tx=0x7fa73800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:45:21.919 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.549+0000 7fa751000640 1 -- 172.21.15.111:0/3565483100 shutdown_connections 2024-09-15T22:45:21.920 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.549+0000 7fa751000640 1 --2- 172.21.15.111:0/3565483100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa74c109120 0x7fa74c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:21.920 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.549+0000 7fa751000640 1 --2- 172.21.15.111:0/3565483100 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa7300a3350 0x7fa7300a5810 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:21.920 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.549+0000 7fa751000640 1 --2- 172.21.15.111:0/3565483100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa74c1049a0 0x7fa74c19ab50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:21.920 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.549+0000 7fa751000640 1 --2- 172.21.15.111:0/3565483100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa74c103ff0 0x7fa74c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:21.920 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.549+0000 7fa751000640 1 -- 172.21.15.111:0/3565483100 >> 172.21.15.111:0/3565483100 conn(0x7fa74c0ffa90 msgr2=0x7fa74c100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:21.920 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.549+0000 7fa751000640 1 -- 172.21.15.111:0/3565483100 shutdown_connections 2024-09-15T22:45:21.921 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.549+0000 7fa751000640 1 -- 172.21.15.111:0/3565483100 wait complete. 2024-09-15T22:45:22.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:22 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/3565483100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-15T22:45:22.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:22 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/3565483100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-15T22:45:22.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:22 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/3565483100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-15T22:45:23.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.644+0000 7ffb11800640 1 -- 172.21.15.111:0/373501883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb0c102360 msgr2=0x7ffb0c10e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:23.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.644+0000 7ffb11800640 1 --2- 172.21.15.111:0/373501883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb0c102360 0x7ffb0c10e890 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffb04009cd0 tx=0x7ffb0402a8f0 comp rx=0 tx=0).stop 2024-09-15T22:45:23.670 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.644+0000 7ffb11800640 1 -- 172.21.15.111:0/373501883 shutdown_connections 2024-09-15T22:45:23.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.644+0000 7ffb11800640 1 --2- 172.21.15.111:0/373501883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb0c102360 0x7ffb0c10e890 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:23.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.644+0000 7ffb11800640 1 --2- 172.21.15.111:0/373501883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb0c1019c0 0x7ffb0c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:23.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.644+0000 7ffb11800640 1 --2- 172.21.15.111:0/373501883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb0c1079c0 0x7ffb0c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:23.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.644+0000 7ffb11800640 1 -- 172.21.15.111:0/373501883 >> 172.21.15.111:0/373501883 conn(0x7ffb0c0fd6e0 msgr2=0x7ffb0c0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:23.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.644+0000 7ffb11800640 1 -- 172.21.15.111:0/373501883 shutdown_connections 2024-09-15T22:45:23.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.644+0000 7ffb11800640 1 -- 172.21.15.111:0/373501883 wait complete. 2024-09-15T22:45:23.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.645+0000 7ffb11800640 1 Processor -- start 2024-09-15T22:45:23.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.645+0000 7ffb11800640 1 -- start start 2024-09-15T22:45:23.671 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.646+0000 7ffb11800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb0c1019c0 0x7ffb0c19a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:23.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.646+0000 7ffb11800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb0c102360 0x7ffb0c19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:23.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.646+0000 7ffb11800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb0c1079c0 0x7ffb0c19eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:23.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.646+0000 7ffb11800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0c111dc0 con 0x7ffb0c102360 2024-09-15T22:45:23.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.646+0000 7ffb11800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0c111c40 con 0x7ffb0c1079c0 2024-09-15T22:45:23.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.646+0000 7ffb11800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0c111f40 con 0x7ffb0c1019c0 2024-09-15T22:45:23.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.646+0000 7ffb0b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb0c102360 0x7ffb0c19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:23.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.646+0000 7ffb0b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb0c102360 0x7ffb0c19a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60670/0 (socket says 172.21.15.111:60670) 2024-09-15T22:45:23.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.646+0000 7ffb0b400640 1 -- 172.21.15.111:0/238445149 learned_addr learned my addr 172.21.15.111:0/238445149 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:23.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.646+0000 7ffb10e00640 1 --2- 172.21.15.111:0/238445149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb0c1079c0 0x7ffb0c19eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:23.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.646+0000 7ffb0be00640 1 --2- 172.21.15.111:0/238445149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb0c1019c0 0x7ffb0c19a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:23.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.646+0000 7ffb0b400640 1 -- 172.21.15.111:0/238445149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb0c1019c0 msgr2=0x7ffb0c19a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:23.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.646+0000 7ffb0b400640 1 --2- 172.21.15.111:0/238445149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb0c1019c0 0x7ffb0c19a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:23.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.646+0000 7ffb0b400640 1 -- 172.21.15.111:0/238445149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb0c1079c0 msgr2=0x7ffb0c19eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:23.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.646+0000 7ffb0b400640 1 --2- 172.21.15.111:0/238445149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb0c1079c0 0x7ffb0c19eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:23.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.646+0000 7ffb0b400640 1 -- 172.21.15.111:0/238445149 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb0c19f380 con 0x7ffb0c102360 2024-09-15T22:45:23.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.647+0000 7ffb0b400640 1 --2- 172.21.15.111:0/238445149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb0c102360 0x7ffb0c19a910 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800e9f0 tx=0x7ffaf800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:23.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.647+0000 7ffb08c00640 1 -- 172.21.15.111:0/238445149 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf800cd80 con 0x7ffb0c102360 2024-09-15T22:45:23.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.647+0000 7ffb08c00640 1 -- 172.21.15.111:0/238445149 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaf8010910 con 0x7ffb0c102360 2024-09-15T22:45:23.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.647+0000 7ffb11800640 1 -- 172.21.15.111:0/238445149 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb0c19f670 con 0x7ffb0c102360 2024-09-15T22:45:23.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.647+0000 7ffb08c00640 1 -- 172.21.15.111:0/238445149 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8018990 con 0x7ffb0c102360 2024-09-15T22:45:23.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.647+0000 7ffb11800640 1 -- 172.21.15.111:0/238445149 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb0c1a6f50 con 0x7ffb0c102360 2024-09-15T22:45:23.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.648+0000 7ffb11800640 1 -- 172.21.15.111:0/238445149 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb0c101e20 con 0x7ffb0c102360 2024-09-15T22:45:23.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.651+0000 7ffb08c00640 1 -- 172.21.15.111:0/238445149 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ffaf8010ac0 con 0x7ffb0c102360 2024-09-15T22:45:23.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.652+0000 7ffb08c00640 1 --2- 172.21.15.111:0/238445149 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ffaf00a32d0 0x7ffaf00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:23.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.652+0000 7ffb08c00640 1 -- 172.21.15.111:0/238445149 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7ffaf80cfdb0 con 0x7ffb0c102360 2024-09-15T22:45:23.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.652+0000 7ffb0be00640 1 --2- 172.21.15.111:0/238445149 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ffaf00a32d0 0x7ffaf00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:23.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.653+0000 7ffb0be00640 1 --2- 172.21.15.111:0/238445149 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ffaf00a32d0 0x7ffaf00a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009b50 tx=0x7ffafc005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:23.674 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.656+0000 7ffb08c00640 1 -- 172.21.15.111:0/238445149 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffaf80d4070 con 0x7ffb0c102360 2024-09-15T22:45:23.802 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:23 smithi111 ceph-mon[30629]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:45:23.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:23 smithi192 ceph-mon[35427]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:45:23.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.859+0000 7ffb11800640 1 -- 172.21.15.111:0/238445149 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ffb0c1a7180 con 0x7ffb0c102360 2024-09-15T22:45:23.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.860+0000 7ffb08c00640 1 -- 172.21.15.111:0/238445149 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffaf809c320 con 0x7ffb0c102360 2024-09-15T22:45:23.861 INFO:teuthology.orchestra.run.smithi111.stdout:34359738454 2024-09-15T22:45:23.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.862+0000 7ffb11800640 1 -- 172.21.15.111:0/238445149 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ffaf00a32d0 msgr2=0x7ffaf00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:23.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.862+0000 7ffb11800640 1 --2- 172.21.15.111:0/238445149 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ffaf00a32d0 0x7ffaf00a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009b50 tx=0x7ffafc005f90 comp rx=0 tx=0).stop 2024-09-15T22:45:23.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.862+0000 7ffb11800640 1 -- 172.21.15.111:0/238445149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb0c102360 msgr2=0x7ffb0c19a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:23.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.862+0000 7ffb11800640 1 --2- 172.21.15.111:0/238445149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb0c102360 0x7ffb0c19a910 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800e9f0 tx=0x7ffaf800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:45:23.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.862+0000 7ffb11800640 1 -- 172.21.15.111:0/238445149 shutdown_connections 2024-09-15T22:45:23.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.862+0000 7ffb11800640 1 --2- 172.21.15.111:0/238445149 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ffaf00a32d0 0x7ffaf00a5790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:23.863 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.862+0000 7ffb11800640 1 --2- 172.21.15.111:0/238445149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb0c1079c0 0x7ffb0c19eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:23.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.862+0000 7ffb11800640 1 --2- 172.21.15.111:0/238445149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffb0c102360 0x7ffb0c19a910 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:23.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.862+0000 7ffb11800640 1 --2- 172.21.15.111:0/238445149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb0c1019c0 0x7ffb0c19a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:23.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.862+0000 7ffb11800640 1 -- 172.21.15.111:0/238445149 >> 172.21.15.111:0/238445149 conn(0x7ffb0c0fd6e0 msgr2=0x7ffb0c0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:23.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.863+0000 7ffb11800640 1 -- 172.21.15.111:0/238445149 shutdown_connections 2024-09-15T22:45:23.864 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:23.863+0000 7ffb11800640 1 -- 172.21.15.111:0/238445149 wait complete. 2024-09-15T22:45:23.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:23 smithi121 ceph-mon[34815]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:45:24.713 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:24 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/238445149' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-15T22:45:24.715 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953532 got 137438953534 for osd.4 2024-09-15T22:45:24.716 DEBUG:teuthology.parallel:result is None 2024-09-15T22:45:24.816 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:24 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/238445149' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-15T22:45:24.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:24 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/238445149' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-15T22:45:25.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.434+0000 7f5553e00640 1 -- 172.21.15.111:0/1931224022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f554c106f10 msgr2=0x7f554c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:25.074 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.434+0000 7f5553e00640 1 --2- 172.21.15.111:0/1931224022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f554c106f10 0x7f554c10ea90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5544009ca0 tx=0x7f554402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:45:25.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.434+0000 7f5553e00640 1 -- 172.21.15.111:0/1931224022 shutdown_connections 2024-09-15T22:45:25.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.434+0000 7f5553e00640 1 --2- 172.21.15.111:0/1931224022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f554c106f10 0x7f554c10ea90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:25.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.434+0000 7f5553e00640 1 --2- 172.21.15.111:0/1931224022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f554c102790 0x7f554c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:25.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.434+0000 7f5553e00640 1 --2- 172.21.15.111:0/1931224022 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f554c101de0 0x7f554c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:25.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.434+0000 7f5553e00640 1 -- 172.21.15.111:0/1931224022 >> 172.21.15.111:0/1931224022 conn(0x7f554c0fd8c0 msgr2=0x7f554c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:25.075 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.434+0000 7f5553e00640 1 -- 172.21.15.111:0/1931224022 shutdown_connections 2024-09-15T22:45:25.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.434+0000 7f5553e00640 1 -- 172.21.15.111:0/1931224022 wait complete. 2024-09-15T22:45:25.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.435+0000 7f5553e00640 1 Processor -- start 2024-09-15T22:45:25.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.435+0000 7f5553e00640 1 -- start start 2024-09-15T22:45:25.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.436+0000 7f5553e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f554c101de0 0x7f554c1161e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:25.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.436+0000 7f5553e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f554c102790 0x7f554c116720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:25.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.436+0000 7f5553e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f554c106f10 0x7f554c11bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:25.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.436+0000 7f5553e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f554c076140 con 0x7f554c102790 2024-09-15T22:45:25.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.436+0000 7f5553e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f554c075fc0 con 0x7f554c106f10 2024-09-15T22:45:25.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.436+0000 7f5553e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f554c0762c0 con 0x7f554c101de0 2024-09-15T22:45:25.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.436+0000 7f5551400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f554c101de0 0x7f554c1161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:25.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.436+0000 7f5551400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f554c101de0 0x7f554c1161e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:43472/0 (socket says 172.21.15.111:43472) 2024-09-15T22:45:25.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.436+0000 7f5551400640 1 -- 172.21.15.111:0/2233421882 learned_addr learned my addr 172.21.15.111:0/2233421882 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:25.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.436+0000 7f5550a00640 1 --2- 172.21.15.111:0/2233421882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f554c102790 0x7f554c116720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:25.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.436+0000 7f5551e00640 1 --2- 172.21.15.111:0/2233421882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f554c106f10 0x7f554c11bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:25.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.436+0000 7f5551400640 1 -- 172.21.15.111:0/2233421882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f554c106f10 msgr2=0x7f554c11bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:25.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.436+0000 7f5551400640 1 --2- 172.21.15.111:0/2233421882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f554c106f10 0x7f554c11bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:25.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.436+0000 7f5551400640 1 -- 172.21.15.111:0/2233421882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f554c102790 msgr2=0x7f554c116720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:25.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.436+0000 7f5551400640 1 --2- 172.21.15.111:0/2233421882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f554c102790 0x7f554c116720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:25.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.436+0000 7f5551400640 1 -- 172.21.15.111:0/2233421882 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554c11c7c0 con 0x7f554c101de0 2024-09-15T22:45:25.077 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.437+0000 7f5551400640 1 --2- 172.21.15.111:0/2233421882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f554c101de0 0x7f554c1161e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f554000e9f0 tx=0x7f554000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:25.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.437+0000 7f554a000640 1 -- 172.21.15.111:0/2233421882 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554000cd80 con 0x7f554c101de0 2024-09-15T22:45:25.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.437+0000 7f5553e00640 1 -- 172.21.15.111:0/2233421882 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f554c11cab0 con 0x7f554c101de0 2024-09-15T22:45:25.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.437+0000 7f554a000640 1 -- 172.21.15.111:0/2233421882 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5540010910 con 0x7f554c101de0 2024-09-15T22:45:25.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.437+0000 7f554a000640 1 -- 172.21.15.111:0/2233421882 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55400188f0 con 0x7f554c101de0 2024-09-15T22:45:25.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.437+0000 7f5553e00640 1 -- 172.21.15.111:0/2233421882 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f554c1114d0 con 0x7f554c101de0 2024-09-15T22:45:25.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.438+0000 7f5553e00640 1 -- 172.21.15.111:0/2233421882 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f554c1032c0 con 0x7f554c101de0 2024-09-15T22:45:25.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.440+0000 7f554a000640 1 -- 172.21.15.111:0/2233421882 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f5540010ac0 con 0x7f554c101de0 2024-09-15T22:45:25.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.441+0000 7f554a000640 1 --2- 172.21.15.111:0/2233421882 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f55340a32d0 0x7f55340a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:25.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.441+0000 7f554a000640 1 -- 172.21.15.111:0/2233421882 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f55400d0230 con 0x7f554c101de0 2024-09-15T22:45:25.078 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.441+0000 7f5550a00640 1 --2- 172.21.15.111:0/2233421882 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f55340a32d0 0x7f55340a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:25.078 INFO:teuthology.orchestra.run.smithi111.stdout:279172874279 2024-09-15T22:45:25.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.442+0000 7f5550a00640 1 --2- 172.21.15.111:0/2233421882 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f55340a32d0 0x7f55340a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f554c112ac0 tx=0x7f5538005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:25.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.443+0000 7f554a000640 1 -- 172.21.15.111:0/2233421882 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f554009c7a0 con 0x7f554c101de0 2024-09-15T22:45:25.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.636+0000 7f5553e00640 1 -- 172.21.15.111:0/2233421882 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f554c111700 con 0x7f554c101de0 2024-09-15T22:45:25.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.637+0000 7f554a000640 1 -- 172.21.15.111:0/2233421882 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f55400a0aa0 con 0x7f554c101de0 2024-09-15T22:45:25.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.639+0000 7f5553e00640 1 -- 172.21.15.111:0/2233421882 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f55340a32d0 msgr2=0x7f55340a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:25.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.639+0000 7f5553e00640 1 --2- 172.21.15.111:0/2233421882 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f55340a32d0 0x7f55340a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f554c112ac0 tx=0x7f5538005f90 comp rx=0 tx=0).stop 2024-09-15T22:45:25.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.639+0000 7f5553e00640 1 -- 172.21.15.111:0/2233421882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f554c101de0 msgr2=0x7f554c1161e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:25.079 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.639+0000 7f5553e00640 1 --2- 172.21.15.111:0/2233421882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f554c101de0 0x7f554c1161e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f554000e9f0 tx=0x7f554000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:45:25.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.639+0000 7f5553e00640 1 -- 172.21.15.111:0/2233421882 shutdown_connections 2024-09-15T22:45:25.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.639+0000 7f5553e00640 1 --2- 172.21.15.111:0/2233421882 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f55340a32d0 0x7f55340a5790 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:25.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.639+0000 7f5553e00640 1 --2- 172.21.15.111:0/2233421882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f554c106f10 0x7f554c11bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:25.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.639+0000 7f5553e00640 1 --2- 172.21.15.111:0/2233421882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f554c102790 0x7f554c116720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:25.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.639+0000 7f5553e00640 1 --2- 172.21.15.111:0/2233421882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f554c101de0 0x7f554c1161e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:25.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.639+0000 7f5553e00640 1 -- 172.21.15.111:0/2233421882 >> 172.21.15.111:0/2233421882 conn(0x7f554c0fd8c0 msgr2=0x7f554c0fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:25.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.639+0000 7f5553e00640 1 -- 172.21.15.111:0/2233421882 shutdown_connections 2024-09-15T22:45:25.080 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.639+0000 7f5553e00640 1 -- 172.21.15.111:0/2233421882 wait complete. 2024-09-15T22:45:25.797 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574926 got 55834574928 for osd.1 2024-09-15T22:45:25.798 DEBUG:teuthology.parallel:result is None 2024-09-15T22:45:25.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:25 smithi192 ceph-mon[35427]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:45:25.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:25 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/2233421882' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-15T22:45:25.815 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:25 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/2744184174' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-15T22:45:25.881 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:25 smithi111 ceph-mon[30629]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:45:25.881 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:25 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/2233421882' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-15T22:45:25.881 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:25 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/2744184174' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-15T22:45:25.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:25 smithi121 ceph-mon[34815]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:45:25.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:25 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/2233421882' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-15T22:45:25.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:25 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/2744184174' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-15T22:45:25.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.558+0000 7f5f20400640 1 -- 172.21.15.111:0/3689518207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f18104010 msgr2=0x7f5f181043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:25.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.558+0000 7f5f20400640 1 --2- 172.21.15.111:0/3689518207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f18104010 0x7f5f181043f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009900 tx=0x7f5f0c02a410 comp rx=0 tx=0).stop 2024-09-15T22:45:25.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.559+0000 7f5f20400640 1 -- 172.21.15.111:0/3689518207 shutdown_connections 2024-09-15T22:45:25.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.559+0000 7f5f20400640 1 --2- 172.21.15.111:0/3689518207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f18109140 0x7f5f18110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:25.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.559+0000 7f5f20400640 1 --2- 172.21.15.111:0/3689518207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f181049c0 0x7f5f18108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:25.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.559+0000 7f5f20400640 1 --2- 172.21.15.111:0/3689518207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f18104010 0x7f5f181043f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:25.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.559+0000 7f5f20400640 1 -- 172.21.15.111:0/3689518207 >> 172.21.15.111:0/3689518207 conn(0x7f5f180ffa50 msgr2=0x7f5f18101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:25.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.559+0000 7f5f20400640 1 -- 172.21.15.111:0/3689518207 shutdown_connections 2024-09-15T22:45:25.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.559+0000 7f5f20400640 1 -- 172.21.15.111:0/3689518207 wait complete. 2024-09-15T22:45:25.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.560+0000 7f5f20400640 1 Processor -- start 2024-09-15T22:45:25.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.560+0000 7f5f20400640 1 -- start start 2024-09-15T22:45:25.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.560+0000 7f5f20400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f18104010 0x7f5f1819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:25.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.560+0000 7f5f20400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f181049c0 0x7f5f1819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:25.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.560+0000 7f5f20400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f18109140 0x7f5f1819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:25.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.560+0000 7f5f20400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181141b0 con 0x7f5f181049c0 2024-09-15T22:45:25.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.560+0000 7f5f20400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18114030 con 0x7f5f18109140 2024-09-15T22:45:25.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.560+0000 7f5f20400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18114330 con 0x7f5f18104010 2024-09-15T22:45:25.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.560+0000 7f5f1d000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f181049c0 0x7f5f1819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:25.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.560+0000 7f5f1d000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f181049c0 0x7f5f1819ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60694/0 (socket says 172.21.15.111:60694) 2024-09-15T22:45:25.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.560+0000 7f5f1d000640 1 -- 172.21.15.111:0/2744184174 learned_addr learned my addr 172.21.15.111:0/2744184174 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:25.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.560+0000 7f5f1e400640 1 --2- 172.21.15.111:0/2744184174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f18109140 0x7f5f1819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:25.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.560+0000 7f5f1da00640 1 --2- 172.21.15.111:0/2744184174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f18104010 0x7f5f1819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:25.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.561+0000 7f5f1d000640 1 -- 172.21.15.111:0/2744184174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f18104010 msgr2=0x7f5f1819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:25.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.561+0000 7f5f1d000640 1 --2- 172.21.15.111:0/2744184174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f18104010 0x7f5f1819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:25.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.561+0000 7f5f1d000640 1 -- 172.21.15.111:0/2744184174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f18109140 msgr2=0x7f5f1819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:25.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.561+0000 7f5f1d000640 1 --2- 172.21.15.111:0/2744184174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f18109140 0x7f5f1819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:25.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.561+0000 7f5f1d000640 1 -- 172.21.15.111:0/2744184174 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1819f5e0 con 0x7f5f181049c0 2024-09-15T22:45:25.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.561+0000 7f5f1da00640 1 --2- 172.21.15.111:0/2744184174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f18104010 0x7f5f1819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:45:25.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.561+0000 7f5f1e400640 1 --2- 172.21.15.111:0/2744184174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f18109140 0x7f5f1819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:25.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.561+0000 7f5f1d000640 1 --2- 172.21.15.111:0/2744184174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f181049c0 0x7f5f1819ab70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400d770 tx=0x7f5f0400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:25.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.561+0000 7f5f16000640 1 -- 172.21.15.111:0/2744184174 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f04017070 con 0x7f5f181049c0 2024-09-15T22:45:25.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.561+0000 7f5f16000640 1 -- 172.21.15.111:0/2744184174 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f04005500 con 0x7f5f181049c0 2024-09-15T22:45:25.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.561+0000 7f5f20400640 1 -- 172.21.15.111:0/2744184174 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f1819f8d0 con 0x7f5f181049c0 2024-09-15T22:45:25.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.561+0000 7f5f16000640 1 -- 172.21.15.111:0/2744184174 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f04004e00 con 0x7f5f181049c0 2024-09-15T22:45:25.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.561+0000 7f5f20400640 1 -- 172.21.15.111:0/2744184174 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f181a71b0 con 0x7f5f181049c0 2024-09-15T22:45:25.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.563+0000 7f5f20400640 1 -- 172.21.15.111:0/2744184174 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f181043f0 con 0x7f5f181049c0 2024-09-15T22:45:25.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.564+0000 7f5f16000640 1 -- 172.21.15.111:0/2744184174 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f5f04004fa0 con 0x7f5f181049c0 2024-09-15T22:45:25.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.566+0000 7f5f16000640 1 --2- 172.21.15.111:0/2744184174 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5f000a32d0 0x7f5f000a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:25.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.566+0000 7f5f16000640 1 -- 172.21.15.111:0/2744184174 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f5f040cf520 con 0x7f5f181049c0 2024-09-15T22:45:25.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.566+0000 7f5f1da00640 1 --2- 172.21.15.111:0/2744184174 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5f000a32d0 0x7f5f000a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:25.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.567+0000 7f5f1da00640 1 --2- 172.21.15.111:0/2744184174 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5f000a32d0 0x7f5f000a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c000c00 tx=0x7f5f0c005a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:25.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.570+0000 7f5f16000640 1 -- 172.21.15.111:0/2744184174 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5f0409c0e0 con 0x7f5f181049c0 2024-09-15T22:45:25.991 INFO:teuthology.orchestra.run.smithi111.stdout:107374182471 2024-09-15T22:45:25.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.778+0000 7f5f20400640 1 -- 172.21.15.111:0/2744184174 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f5f1810bd90 con 0x7f5f181049c0 2024-09-15T22:45:25.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.778+0000 7f5f16000640 1 -- 172.21.15.111:0/2744184174 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5f0409b9e0 con 0x7f5f181049c0 2024-09-15T22:45:25.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.780+0000 7f5f20400640 1 -- 172.21.15.111:0/2744184174 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5f000a32d0 msgr2=0x7f5f000a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:25.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.780+0000 7f5f20400640 1 --2- 172.21.15.111:0/2744184174 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5f000a32d0 0x7f5f000a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c000c00 tx=0x7f5f0c005a40 comp rx=0 tx=0).stop 2024-09-15T22:45:25.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.780+0000 7f5f20400640 1 -- 172.21.15.111:0/2744184174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f181049c0 msgr2=0x7f5f1819ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:25.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.780+0000 7f5f20400640 1 --2- 172.21.15.111:0/2744184174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f181049c0 0x7f5f1819ab70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400d770 tx=0x7f5f0400dc30 comp rx=0 tx=0).stop 2024-09-15T22:45:25.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.781+0000 7f5f20400640 1 -- 172.21.15.111:0/2744184174 shutdown_connections 2024-09-15T22:45:25.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.781+0000 7f5f20400640 1 --2- 172.21.15.111:0/2744184174 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5f000a32d0 0x7f5f000a5790 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:25.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.781+0000 7f5f20400640 1 --2- 172.21.15.111:0/2744184174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f18109140 0x7f5f1819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:25.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.781+0000 7f5f20400640 1 --2- 172.21.15.111:0/2744184174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f181049c0 0x7f5f1819ab70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:25.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.781+0000 7f5f20400640 1 --2- 172.21.15.111:0/2744184174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f18104010 0x7f5f1819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:25.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.781+0000 7f5f20400640 1 -- 172.21.15.111:0/2744184174 >> 172.21.15.111:0/2744184174 conn(0x7f5f180ffa50 msgr2=0x7f5f18100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:25.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.781+0000 7f5f20400640 1 -- 172.21.15.111:0/2744184174 shutdown_connections 2024-09-15T22:45:25.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:24.781+0000 7f5f20400640 1 -- 172.21.15.111:0/2744184174 wait complete. 2024-09-15T22:45:26.357 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411400 got 77309411402 for osd.2 2024-09-15T22:45:26.357 DEBUG:teuthology.parallel:result is None 2024-09-15T22:45:26.760 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561073 got 188978561076 for osd.6 2024-09-15T22:45:26.760 DEBUG:teuthology.parallel:result is None 2024-09-15T22:45:26.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.192+0000 7f849f000640 1 -- 172.21.15.111:0/2699535855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84981049a0 msgr2=0x7f84981089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:26.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.192+0000 7f849f000640 1 --2- 172.21.15.111:0/2699535855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84981049a0 0x7f84981089f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8480009900 tx=0x7f848002a410 comp rx=0 tx=0).stop 2024-09-15T22:45:26.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.192+0000 7f849f000640 1 -- 172.21.15.111:0/2699535855 shutdown_connections 2024-09-15T22:45:26.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.192+0000 7f849f000640 1 --2- 172.21.15.111:0/2699535855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8498109120 0x7f8498110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:26.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.192+0000 7f849f000640 1 --2- 172.21.15.111:0/2699535855 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84981049a0 0x7f84981089f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:26.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.192+0000 7f849f000640 1 --2- 172.21.15.111:0/2699535855 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8498103ff0 0x7f84981043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:26.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.192+0000 7f849f000640 1 -- 172.21.15.111:0/2699535855 >> 172.21.15.111:0/2699535855 conn(0x7f84980ffa90 msgr2=0x7f8498101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:26.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.192+0000 7f849f000640 1 -- 172.21.15.111:0/2699535855 shutdown_connections 2024-09-15T22:45:26.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.192+0000 7f849f000640 1 -- 172.21.15.111:0/2699535855 wait complete. 2024-09-15T22:45:26.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.193+0000 7f849f000640 1 Processor -- start 2024-09-15T22:45:26.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.193+0000 7f849f000640 1 -- start start 2024-09-15T22:45:26.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.193+0000 7f849f000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8498103ff0 0x7f849819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:26.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.193+0000 7f849f000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84981049a0 0x7f849819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:26.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.193+0000 7f849f000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8498109120 0x7f849819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:26.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.193+0000 7f849f000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8498114190 con 0x7f84981049a0 2024-09-15T22:45:26.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.193+0000 7f849f000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8498114010 con 0x7f8498109120 2024-09-15T22:45:26.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.193+0000 7f849f000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8498114310 con 0x7f8498103ff0 2024-09-15T22:45:26.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.193+0000 7f848fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84981049a0 0x7f849819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:26.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.193+0000 7f848fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84981049a0 0x7f849819abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60628/0 (socket says 172.21.15.111:60628) 2024-09-15T22:45:26.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.193+0000 7f848fe00640 1 -- 172.21.15.111:0/2491595714 learned_addr learned my addr 172.21.15.111:0/2491595714 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:26.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.194+0000 7f8497e00640 1 --2- 172.21.15.111:0/2491595714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8498103ff0 0x7f849819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:26.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.194+0000 7f849d000640 1 --2- 172.21.15.111:0/2491595714 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8498109120 0x7f849819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:26.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.194+0000 7f848fe00640 1 -- 172.21.15.111:0/2491595714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8498103ff0 msgr2=0x7f849819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:26.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.194+0000 7f848fe00640 1 --2- 172.21.15.111:0/2491595714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8498103ff0 0x7f849819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:26.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.194+0000 7f848fe00640 1 -- 172.21.15.111:0/2491595714 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8498109120 msgr2=0x7f849819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:26.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.194+0000 7f848fe00640 1 --2- 172.21.15.111:0/2491595714 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8498109120 0x7f849819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:26.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.194+0000 7f848fe00640 1 -- 172.21.15.111:0/2491595714 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849819f620 con 0x7f84981049a0 2024-09-15T22:45:26.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.194+0000 7f848fe00640 1 --2- 172.21.15.111:0/2491595714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84981049a0 0x7f849819abb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8480004770 tx=0x7f8480003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:26.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.194+0000 7f8495600640 1 -- 172.21.15.111:0/2491595714 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848003f070 con 0x7f84981049a0 2024-09-15T22:45:26.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.194+0000 7f8495600640 1 -- 172.21.15.111:0/2491595714 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8480031ca0 con 0x7f84981049a0 2024-09-15T22:45:26.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.194+0000 7f8495600640 1 -- 172.21.15.111:0/2491595714 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8480035030 con 0x7f84981049a0 2024-09-15T22:45:26.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.194+0000 7f849f000640 1 -- 172.21.15.111:0/2491595714 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f849819f8b0 con 0x7f84981049a0 2024-09-15T22:45:26.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.195+0000 7f849f000640 1 -- 172.21.15.111:0/2491595714 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84981a71f0 con 0x7f84981049a0 2024-09-15T22:45:26.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.196+0000 7f849f000640 1 -- 172.21.15.111:0/2491595714 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84981054d0 con 0x7f84981049a0 2024-09-15T22:45:26.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.197+0000 7f8495600640 1 -- 172.21.15.111:0/2491595714 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f8480042070 con 0x7f84981049a0 2024-09-15T22:45:26.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.199+0000 7f8495600640 1 --2- 172.21.15.111:0/2491595714 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f847c0a32d0 0x7f847c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:26.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.199+0000 7f8495600640 1 -- 172.21.15.111:0/2491595714 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f84800f01d0 con 0x7f84981049a0 2024-09-15T22:45:26.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.199+0000 7f8497e00640 1 --2- 172.21.15.111:0/2491595714 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f847c0a32d0 0x7f847c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:26.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.200+0000 7f8497e00640 1 --2- 172.21.15.111:0/2491595714 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f847c0a32d0 0x7f847c0a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8488005d00 tx=0x7f848800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:26.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.203+0000 7f8495600640 1 -- 172.21.15.111:0/2491595714 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f84800bcd90 con 0x7f84981049a0 2024-09-15T22:45:26.828 INFO:teuthology.orchestra.run.smithi111.stdout:210453397553 2024-09-15T22:45:26.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.395+0000 7f849f000640 1 -- 172.21.15.111:0/2491595714 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f849810c360 con 0x7f84981049a0 2024-09-15T22:45:26.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.396+0000 7f8495600640 1 -- 172.21.15.111:0/2491595714 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f84800bc690 con 0x7f84981049a0 2024-09-15T22:45:26.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.399+0000 7f849f000640 1 -- 172.21.15.111:0/2491595714 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f847c0a32d0 msgr2=0x7f847c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:26.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.399+0000 7f849f000640 1 --2- 172.21.15.111:0/2491595714 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f847c0a32d0 0x7f847c0a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8488005d00 tx=0x7f848800b410 comp rx=0 tx=0).stop 2024-09-15T22:45:26.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.399+0000 7f849f000640 1 -- 172.21.15.111:0/2491595714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84981049a0 msgr2=0x7f849819abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:26.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.399+0000 7f849f000640 1 --2- 172.21.15.111:0/2491595714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84981049a0 0x7f849819abb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8480004770 tx=0x7f8480003c10 comp rx=0 tx=0).stop 2024-09-15T22:45:26.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.399+0000 7f849f000640 1 -- 172.21.15.111:0/2491595714 shutdown_connections 2024-09-15T22:45:26.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.399+0000 7f849f000640 1 --2- 172.21.15.111:0/2491595714 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f847c0a32d0 0x7f847c0a5790 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:26.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.399+0000 7f849f000640 1 --2- 172.21.15.111:0/2491595714 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8498109120 0x7f849819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:26.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.399+0000 7f849f000640 1 --2- 172.21.15.111:0/2491595714 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84981049a0 0x7f849819abb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:26.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.399+0000 7f849f000640 1 --2- 172.21.15.111:0/2491595714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8498103ff0 0x7f849819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:26.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.399+0000 7f849f000640 1 -- 172.21.15.111:0/2491595714 >> 172.21.15.111:0/2491595714 conn(0x7f84980ffa90 msgr2=0x7f8498100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:26.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.400+0000 7f849f000640 1 -- 172.21.15.111:0/2491595714 shutdown_connections 2024-09-15T22:45:26.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:21.400+0000 7f849f000640 1 -- 172.21.15.111:0/2491595714 wait complete. 2024-09-15T22:45:27.578 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397551 got 210453397553 for osd.7 2024-09-15T22:45:27.578 DEBUG:teuthology.parallel:result is None 2024-09-15T22:45:27.802 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:27 smithi111 ceph-mon[30629]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:45:27.812 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:27.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:27 smithi192 ceph-mon[35427]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:45:27.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:27 smithi121 ceph-mon[34815]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:45:28.287 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.286+0000 7f3b51000640 1 -- 172.21.15.111:0/727946106 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b4c1049a0 msgr2=0x7f3b4c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:28.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.286+0000 7f3b51000640 1 --2- 172.21.15.111:0/727946106 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b4c1049a0 0x7f3b4c1089f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009860 tx=0x7f3b3802a340 comp rx=0 tx=0).stop 2024-09-15T22:45:28.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.286+0000 7f3b51000640 1 -- 172.21.15.111:0/727946106 shutdown_connections 2024-09-15T22:45:28.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.286+0000 7f3b51000640 1 --2- 172.21.15.111:0/727946106 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b4c109120 0x7f3b4c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:28.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.286+0000 7f3b51000640 1 --2- 172.21.15.111:0/727946106 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b4c1049a0 0x7f3b4c1089f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:28.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.286+0000 7f3b51000640 1 --2- 172.21.15.111:0/727946106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b4c103ff0 0x7f3b4c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:28.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.286+0000 7f3b51000640 1 -- 172.21.15.111:0/727946106 >> 172.21.15.111:0/727946106 conn(0x7f3b4c0ffa90 msgr2=0x7f3b4c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:28.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.286+0000 7f3b51000640 1 -- 172.21.15.111:0/727946106 shutdown_connections 2024-09-15T22:45:28.288 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.286+0000 7f3b51000640 1 -- 172.21.15.111:0/727946106 wait complete. 2024-09-15T22:45:28.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.287+0000 7f3b51000640 1 Processor -- start 2024-09-15T22:45:28.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.287+0000 7f3b51000640 1 -- start start 2024-09-15T22:45:28.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.288+0000 7f3b51000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b4c103ff0 0x7f3b4c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:28.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.288+0000 7f3b51000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b4c1049a0 0x7f3b4c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:28.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.288+0000 7f3b51000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b4c109120 0x7f3b4c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:28.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.288+0000 7f3b51000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c114120 con 0x7f3b4c103ff0 2024-09-15T22:45:28.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.288+0000 7f3b51000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c113fa0 con 0x7f3b4c1049a0 2024-09-15T22:45:28.289 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.288+0000 7f3b51000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c1142a0 con 0x7f3b4c109120 2024-09-15T22:45:28.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.288+0000 7f3b49e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b4c103ff0 0x7f3b4c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:28.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.288+0000 7f3b49e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b4c103ff0 0x7f3b4c19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60714/0 (socket says 172.21.15.111:60714) 2024-09-15T22:45:28.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.288+0000 7f3b49e00640 1 -- 172.21.15.111:0/2977003546 learned_addr learned my addr 172.21.15.111:0/2977003546 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:28.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.288+0000 7f3b4a800640 1 --2- 172.21.15.111:0/2977003546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b4c109120 0x7f3b4c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:28.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.288+0000 7f3b49400640 1 --2- 172.21.15.111:0/2977003546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b4c1049a0 0x7f3b4c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:28.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.288+0000 7f3b49e00640 1 -- 172.21.15.111:0/2977003546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b4c109120 msgr2=0x7f3b4c19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:28.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.288+0000 7f3b49e00640 1 --2- 172.21.15.111:0/2977003546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b4c109120 0x7f3b4c19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:28.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.288+0000 7f3b49e00640 1 -- 172.21.15.111:0/2977003546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b4c1049a0 msgr2=0x7f3b4c19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:28.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.288+0000 7f3b49e00640 1 --2- 172.21.15.111:0/2977003546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b4c1049a0 0x7f3b4c19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:28.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.288+0000 7f3b49e00640 1 -- 172.21.15.111:0/2977003546 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4c19f5d0 con 0x7f3b4c103ff0 2024-09-15T22:45:28.291 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.288+0000 7f3b4a800640 1 --2- 172.21.15.111:0/2977003546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b4c109120 0x7f3b4c19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:28.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.288+0000 7f3b49400640 1 --2- 172.21.15.111:0/2977003546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b4c1049a0 0x7f3b4c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:28.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.289+0000 7f3b49e00640 1 --2- 172.21.15.111:0/2977003546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b4c103ff0 0x7f3b4c19a620 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c00e9f0 tx=0x7f3b3c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:28.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.289+0000 7f3b42a00640 1 -- 172.21.15.111:0/2977003546 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c00cd80 con 0x7f3b4c103ff0 2024-09-15T22:45:28.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.289+0000 7f3b51000640 1 -- 172.21.15.111:0/2977003546 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4c19f8c0 con 0x7f3b4c103ff0 2024-09-15T22:45:28.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.289+0000 7f3b42a00640 1 -- 172.21.15.111:0/2977003546 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b3c010910 con 0x7f3b4c103ff0 2024-09-15T22:45:28.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.289+0000 7f3b42a00640 1 -- 172.21.15.111:0/2977003546 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c0188f0 con 0x7f3b4c103ff0 2024-09-15T22:45:28.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.289+0000 7f3b51000640 1 -- 172.21.15.111:0/2977003546 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b4c1a71a0 con 0x7f3b4c103ff0 2024-09-15T22:45:28.292 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.290+0000 7f3b51000640 1 -- 172.21.15.111:0/2977003546 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b4c1043d0 con 0x7f3b4c103ff0 2024-09-15T22:45:28.293 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.292+0000 7f3b42a00640 1 -- 172.21.15.111:0/2977003546 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f3b3c010ac0 con 0x7f3b4c103ff0 2024-09-15T22:45:28.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.293+0000 7f3b42a00640 1 --2- 172.21.15.111:0/2977003546 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3b300a32d0 0x7f3b300a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:28.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.294+0000 7f3b42a00640 1 -- 172.21.15.111:0/2977003546 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f3b3c0cfe30 con 0x7f3b4c103ff0 2024-09-15T22:45:28.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.294+0000 7f3b49400640 1 --2- 172.21.15.111:0/2977003546 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3b300a32d0 0x7f3b300a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:28.295 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.294+0000 7f3b49400640 1 --2- 172.21.15.111:0/2977003546 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3b300a32d0 0x7f3b300a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3b3802a930 tx=0x7f3b380058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:28.299 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.297+0000 7f3b42a00640 1 -- 172.21.15.111:0/2977003546 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3b3c09c3a0 con 0x7f3b4c103ff0 2024-09-15T22:45:28.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.478+0000 7f3b51000640 1 -- 172.21.15.111:0/2977003546 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f3b4c10bdd0 con 0x7f3b4c103ff0 2024-09-15T22:45:28.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.479+0000 7f3b42a00640 1 -- 172.21.15.111:0/2977003546 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3b3c0a06a0 con 0x7f3b4c103ff0 2024-09-15T22:45:28.480 INFO:teuthology.orchestra.run.smithi111.stdout:158913790013 2024-09-15T22:45:28.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.481+0000 7f3b51000640 1 -- 172.21.15.111:0/2977003546 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3b300a32d0 msgr2=0x7f3b300a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:28.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.481+0000 7f3b51000640 1 --2- 172.21.15.111:0/2977003546 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3b300a32d0 0x7f3b300a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3b3802a930 tx=0x7f3b380058d0 comp rx=0 tx=0).stop 2024-09-15T22:45:28.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.481+0000 7f3b51000640 1 -- 172.21.15.111:0/2977003546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b4c103ff0 msgr2=0x7f3b4c19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:28.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.481+0000 7f3b51000640 1 --2- 172.21.15.111:0/2977003546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b4c103ff0 0x7f3b4c19a620 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c00e9f0 tx=0x7f3b3c00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:45:28.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.481+0000 7f3b51000640 1 -- 172.21.15.111:0/2977003546 shutdown_connections 2024-09-15T22:45:28.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.481+0000 7f3b51000640 1 --2- 172.21.15.111:0/2977003546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b4c109120 0x7f3b4c19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:28.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.481+0000 7f3b51000640 1 --2- 172.21.15.111:0/2977003546 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3b300a32d0 0x7f3b300a5790 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:28.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.481+0000 7f3b51000640 1 --2- 172.21.15.111:0/2977003546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b4c1049a0 0x7f3b4c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:28.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.481+0000 7f3b51000640 1 --2- 172.21.15.111:0/2977003546 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b4c103ff0 0x7f3b4c19a620 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:28.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.481+0000 7f3b51000640 1 -- 172.21.15.111:0/2977003546 >> 172.21.15.111:0/2977003546 conn(0x7f3b4c0ffa90 msgr2=0x7f3b4c100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:28.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.482+0000 7f3b51000640 1 -- 172.21.15.111:0/2977003546 shutdown_connections 2024-09-15T22:45:28.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:28.482+0000 7f3b51000640 1 -- 172.21.15.111:0/2977003546 wait complete. 2024-09-15T22:45:28.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:28 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/2977003546' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-15T22:45:28.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:28 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/2977003546' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-15T22:45:28.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:28 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/2977003546' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-15T22:45:29.335 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874277 got 279172874279 for osd.9 2024-09-15T22:45:29.336 DEBUG:teuthology.parallel:result is None 2024-09-15T22:45:29.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.690+0000 7f4bf8a00640 1 -- 172.21.15.111:0/1061726327 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf41049a0 msgr2=0x7f4bf41089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:29.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.690+0000 7f4bf8a00640 1 --2- 172.21.15.111:0/1061726327 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf41049a0 0x7f4bf41089f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009900 tx=0x7f4be002a410 comp rx=0 tx=0).stop 2024-09-15T22:45:29.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.690+0000 7f4bf8a00640 1 -- 172.21.15.111:0/1061726327 shutdown_connections 2024-09-15T22:45:29.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.690+0000 7f4bf8a00640 1 --2- 172.21.15.111:0/1061726327 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bf4109120 0x7f4bf4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:29.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.690+0000 7f4bf8a00640 1 --2- 172.21.15.111:0/1061726327 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf41049a0 0x7f4bf41089f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:29.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.690+0000 7f4bf8a00640 1 --2- 172.21.15.111:0/1061726327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bf4103ff0 0x7f4bf41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:29.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.690+0000 7f4bf8a00640 1 -- 172.21.15.111:0/1061726327 >> 172.21.15.111:0/1061726327 conn(0x7f4bf40ffa90 msgr2=0x7f4bf4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:29.693 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.690+0000 7f4bf8a00640 1 -- 172.21.15.111:0/1061726327 shutdown_connections 2024-09-15T22:45:29.694 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.691+0000 7f4bf8a00640 1 -- 172.21.15.111:0/1061726327 wait complete. 2024-09-15T22:45:29.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.691+0000 7f4bf8a00640 1 Processor -- start 2024-09-15T22:45:29.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.692+0000 7f4bf8a00640 1 -- start start 2024-09-15T22:45:29.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.692+0000 7f4bf8a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bf4103ff0 0x7f4bf419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:29.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.692+0000 7f4bf8a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bf41049a0 0x7f4bf419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:29.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.692+0000 7f4bf8a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf4109120 0x7f4bf419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:29.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.692+0000 7f4bf8a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf4114140 con 0x7f4bf41049a0 2024-09-15T22:45:29.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.692+0000 7f4bf8a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf4113fc0 con 0x7f4bf4103ff0 2024-09-15T22:45:29.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.692+0000 7f4bf8a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf41142c0 con 0x7f4bf4109120 2024-09-15T22:45:29.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.692+0000 7f4bf1e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bf4103ff0 0x7f4bf419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:29.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.692+0000 7f4bf1400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bf41049a0 0x7f4bf419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:29.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.692+0000 7f4bf1e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bf4103ff0 0x7f4bf419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:56452/0 (socket says 172.21.15.111:56452) 2024-09-15T22:45:29.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.692+0000 7f4bf1e00640 1 -- 172.21.15.111:0/3998941557 learned_addr learned my addr 172.21.15.111:0/3998941557 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:29.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.693+0000 7f4bf1e00640 1 -- 172.21.15.111:0/3998941557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf4109120 msgr2=0x7f4bf419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:29.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.693+0000 7f4bf1e00640 1 --2- 172.21.15.111:0/3998941557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf4109120 0x7f4bf419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:29.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.693+0000 7f4bf1e00640 1 -- 172.21.15.111:0/3998941557 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bf41049a0 msgr2=0x7f4bf419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:29.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.693+0000 7f4bf1e00640 1 --2- 172.21.15.111:0/3998941557 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bf41049a0 0x7f4bf419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:29.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.693+0000 7f4bf1e00640 1 -- 172.21.15.111:0/3998941557 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf419f5d0 con 0x7f4bf4103ff0 2024-09-15T22:45:29.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.693+0000 7f4bf1e00640 1 --2- 172.21.15.111:0/3998941557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bf4103ff0 0x7f4bf419a620 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4be400e9f0 tx=0x7f4be400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:29.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.693+0000 7f4beaa00640 1 -- 172.21.15.111:0/3998941557 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be400cd80 con 0x7f4bf4103ff0 2024-09-15T22:45:29.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.693+0000 7f4beaa00640 1 -- 172.21.15.111:0/3998941557 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4be4005500 con 0x7f4bf4103ff0 2024-09-15T22:45:29.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.693+0000 7f4beaa00640 1 -- 172.21.15.111:0/3998941557 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be4010690 con 0x7f4bf4103ff0 2024-09-15T22:45:29.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.693+0000 7f4bf8a00640 1 -- 172.21.15.111:0/3998941557 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bf419f8c0 con 0x7f4bf4103ff0 2024-09-15T22:45:29.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.694+0000 7f4bf8a00640 1 -- 172.21.15.111:0/3998941557 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bf41a71a0 con 0x7f4bf4103ff0 2024-09-15T22:45:29.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.695+0000 7f4bf8a00640 1 -- 172.21.15.111:0/3998941557 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bf41043d0 con 0x7f4bf4103ff0 2024-09-15T22:45:29.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.697+0000 7f4beaa00640 1 -- 172.21.15.111:0/3998941557 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f4be40046c0 con 0x7f4bf4103ff0 2024-09-15T22:45:29.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.698+0000 7f4beaa00640 1 --2- 172.21.15.111:0/3998941557 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4bd80a3350 0x7f4bd80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:29.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.699+0000 7f4beaa00640 1 -- 172.21.15.111:0/3998941557 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f4be40cfe50 con 0x7f4bf4103ff0 2024-09-15T22:45:29.700 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.699+0000 7f4bf1400640 1 --2- 172.21.15.111:0/3998941557 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4bd80a3350 0x7f4bd80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:29.701 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.700+0000 7f4bf1400640 1 --2- 172.21.15.111:0/3998941557 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4bd80a3350 0x7f4bd80a5810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009230 tx=0x7f4be00058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:29.704 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.703+0000 7f4beaa00640 1 -- 172.21.15.111:0/3998941557 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4be409ca90 con 0x7f4bf4103ff0 2024-09-15T22:45:29.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:29 smithi192 ceph-mon[35427]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:29.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:29 smithi121 ceph-mon[34815]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:29.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.894+0000 7f4bf8a00640 1 -- 172.21.15.111:0/3998941557 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f4bf419b980 con 0x7f4bf4103ff0 2024-09-15T22:45:29.896 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:29 smithi111 ceph-mon[30629]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:29.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.895+0000 7f4beaa00640 1 -- 172.21.15.111:0/3998941557 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f4be409c390 con 0x7f4bf4103ff0 2024-09-15T22:45:29.896 INFO:teuthology.orchestra.run.smithi111.stdout:335007449116 2024-09-15T22:45:29.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.897+0000 7f4bf8a00640 1 -- 172.21.15.111:0/3998941557 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4bd80a3350 msgr2=0x7f4bd80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:29.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.897+0000 7f4bf8a00640 1 --2- 172.21.15.111:0/3998941557 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4bd80a3350 0x7f4bd80a5810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009230 tx=0x7f4be00058d0 comp rx=0 tx=0).stop 2024-09-15T22:45:29.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.897+0000 7f4bf8a00640 1 -- 172.21.15.111:0/3998941557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bf4103ff0 msgr2=0x7f4bf419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:29.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.897+0000 7f4bf8a00640 1 --2- 172.21.15.111:0/3998941557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bf4103ff0 0x7f4bf419a620 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4be400e9f0 tx=0x7f4be400edf0 comp rx=0 tx=0).stop 2024-09-15T22:45:29.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.897+0000 7f4bf8a00640 1 -- 172.21.15.111:0/3998941557 shutdown_connections 2024-09-15T22:45:29.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.897+0000 7f4bf8a00640 1 --2- 172.21.15.111:0/3998941557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf4109120 0x7f4bf419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:29.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.897+0000 7f4bf8a00640 1 --2- 172.21.15.111:0/3998941557 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bf41049a0 0x7f4bf419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:29.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.897+0000 7f4bf8a00640 1 --2- 172.21.15.111:0/3998941557 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4bd80a3350 0x7f4bd80a5810 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:29.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.897+0000 7f4bf8a00640 1 --2- 172.21.15.111:0/3998941557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bf4103ff0 0x7f4bf419a620 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:29.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.897+0000 7f4bf8a00640 1 -- 172.21.15.111:0/3998941557 >> 172.21.15.111:0/3998941557 conn(0x7f4bf40ffa90 msgr2=0x7f4bf4100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:29.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.897+0000 7f4bf8a00640 1 -- 172.21.15.111:0/3998941557 shutdown_connections 2024-09-15T22:45:29.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:29.897+0000 7f4bf8a00640 1 -- 172.21.15.111:0/3998941557 wait complete. 2024-09-15T22:45:30.170 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790011 got 158913790013 for osd.5 2024-09-15T22:45:30.170 DEBUG:teuthology.parallel:result is None 2024-09-15T22:45:30.776 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:30 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/3998941557' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-15T22:45:30.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:30 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/3998941557' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-15T22:45:31.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:30 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/3998941557' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-15T22:45:31.209 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738450 got 34359738454 for osd.0 2024-09-15T22:45:31.210 DEBUG:teuthology.parallel:result is None 2024-09-15T22:45:31.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.887+0000 7fbeab200640 1 -- 172.21.15.111:0/538543462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbea41047e0 msgr2=0x7fbea4108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:31.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.887+0000 7fbeab200640 1 --2- 172.21.15.111:0/538543462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbea41047e0 0x7fbea4108830 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009900 tx=0x7fbe9402a3f0 comp rx=0 tx=0).stop 2024-09-15T22:45:31.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.887+0000 7fbeab200640 1 -- 172.21.15.111:0/538543462 shutdown_connections 2024-09-15T22:45:31.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.887+0000 7fbeab200640 1 --2- 172.21.15.111:0/538543462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbea4108f60 0x7fbea4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:31.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.887+0000 7fbeab200640 1 --2- 172.21.15.111:0/538543462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbea41047e0 0x7fbea4108830 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:31.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.887+0000 7fbeab200640 1 --2- 172.21.15.111:0/538543462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbea4103e30 0x7fbea4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:31.339 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.887+0000 7fbeab200640 1 -- 172.21.15.111:0/538543462 >> 172.21.15.111:0/538543462 conn(0x7fbea40ff8d0 msgr2=0x7fbea4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:31.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.887+0000 7fbeab200640 1 -- 172.21.15.111:0/538543462 shutdown_connections 2024-09-15T22:45:31.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.887+0000 7fbeab200640 1 -- 172.21.15.111:0/538543462 wait complete. 2024-09-15T22:45:31.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.888+0000 7fbeab200640 1 Processor -- start 2024-09-15T22:45:31.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.888+0000 7fbeab200640 1 -- start start 2024-09-15T22:45:31.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.889+0000 7fbeab200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbea4103e30 0x7fbea419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:31.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.889+0000 7fbeab200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbea41047e0 0x7fbea419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:31.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.889+0000 7fbeab200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbea4108f60 0x7fbea419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:31.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.889+0000 7fbeab200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbea4114010 con 0x7fbea41047e0 2024-09-15T22:45:31.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.889+0000 7fbeab200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbea4113e90 con 0x7fbea4108f60 2024-09-15T22:45:31.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.889+0000 7fbeab200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbea4114190 con 0x7fbea4103e30 2024-09-15T22:45:31.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.889+0000 7fbea9400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbea41047e0 0x7fbea419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:31.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.889+0000 7fbea9400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbea41047e0 0x7fbea419a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:49138/0 (socket says 172.21.15.111:49138) 2024-09-15T22:45:31.340 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.889+0000 7fbea9400640 1 -- 172.21.15.111:0/1308521905 learned_addr learned my addr 172.21.15.111:0/1308521905 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:31.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.889+0000 7fbeaa800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbea4108f60 0x7fbea419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:31.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.889+0000 7fbea9e00640 1 --2- 172.21.15.111:0/1308521905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbea4103e30 0x7fbea419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:31.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.889+0000 7fbeaa800640 1 -- 172.21.15.111:0/1308521905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbea4103e30 msgr2=0x7fbea419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:31.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.889+0000 7fbeaa800640 1 --2- 172.21.15.111:0/1308521905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbea4103e30 0x7fbea419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:31.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.889+0000 7fbeaa800640 1 -- 172.21.15.111:0/1308521905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbea41047e0 msgr2=0x7fbea419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:31.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.889+0000 7fbeaa800640 1 --2- 172.21.15.111:0/1308521905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbea41047e0 0x7fbea419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:31.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.889+0000 7fbeaa800640 1 -- 172.21.15.111:0/1308521905 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea419f440 con 0x7fbea4108f60 2024-09-15T22:45:31.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.889+0000 7fbea9400640 1 --2- 172.21.15.111:0/1308521905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbea41047e0 0x7fbea419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:31.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.890+0000 7fbeaa800640 1 --2- 172.21.15.111:0/1308521905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbea4108f60 0x7fbea419ed60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009fd0 tx=0x7fbea000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:31.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.890+0000 7fbe9ea00640 1 -- 172.21.15.111:0/1308521905 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea000ce20 con 0x7fbea4108f60 2024-09-15T22:45:31.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.890+0000 7fbeab200640 1 -- 172.21.15.111:0/1308521905 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea419f730 con 0x7fbea4108f60 2024-09-15T22:45:31.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.890+0000 7fbe9ea00640 1 -- 172.21.15.111:0/1308521905 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbea0005020 con 0x7fbea4108f60 2024-09-15T22:45:31.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.890+0000 7fbe9ea00640 1 -- 172.21.15.111:0/1308521905 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0010720 con 0x7fbea4108f60 2024-09-15T22:45:31.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.890+0000 7fbeab200640 1 -- 172.21.15.111:0/1308521905 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea41a7010 con 0x7fbea4108f60 2024-09-15T22:45:31.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.891+0000 7fbeab200640 1 -- 172.21.15.111:0/1308521905 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbea4104210 con 0x7fbea4108f60 2024-09-15T22:45:31.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.893+0000 7fbe9ea00640 1 -- 172.21.15.111:0/1308521905 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fbea0005540 con 0x7fbea4108f60 2024-09-15T22:45:31.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.894+0000 7fbe9ea00640 1 --2- 172.21.15.111:0/1308521905 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fbe8c0a32d0 0x7fbe8c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:31.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.894+0000 7fbe9ea00640 1 -- 172.21.15.111:0/1308521905 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fbea00d0120 con 0x7fbea4108f60 2024-09-15T22:45:31.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.895+0000 7fbea9e00640 1 --2- 172.21.15.111:0/1308521905 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fbe8c0a32d0 0x7fbe8c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:31.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.895+0000 7fbea9e00640 1 --2- 172.21.15.111:0/1308521905 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fbe8c0a32d0 0x7fbe8c0a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009bd0 tx=0x7fbe98005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:31.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:30.898+0000 7fbe9ea00640 1 -- 172.21.15.111:0/1308521905 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbea009cce0 con 0x7fbea4108f60 2024-09-15T22:45:31.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:31.079+0000 7fbeab200640 1 -- 172.21.15.111:0/1308521905 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fbea410bc10 con 0x7fbea4108f60 2024-09-15T22:45:31.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:31.080+0000 7fbe9ea00640 1 -- 172.21.15.111:0/1308521905 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbea009c5e0 con 0x7fbea4108f60 2024-09-15T22:45:31.342 INFO:teuthology.orchestra.run.smithi111.stdout:244813135916 2024-09-15T22:45:31.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:31.083+0000 7fbeab200640 1 -- 172.21.15.111:0/1308521905 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fbe8c0a32d0 msgr2=0x7fbe8c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:31.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:31.083+0000 7fbeab200640 1 --2- 172.21.15.111:0/1308521905 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fbe8c0a32d0 0x7fbe8c0a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009bd0 tx=0x7fbe98005f90 comp rx=0 tx=0).stop 2024-09-15T22:45:31.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:31.083+0000 7fbeab200640 1 -- 172.21.15.111:0/1308521905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbea4108f60 msgr2=0x7fbea419ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:31.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:31.083+0000 7fbeab200640 1 --2- 172.21.15.111:0/1308521905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbea4108f60 0x7fbea419ed60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009fd0 tx=0x7fbea000ef10 comp rx=0 tx=0).stop 2024-09-15T22:45:31.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:31.083+0000 7fbeab200640 1 -- 172.21.15.111:0/1308521905 shutdown_connections 2024-09-15T22:45:31.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:31.083+0000 7fbeab200640 1 --2- 172.21.15.111:0/1308521905 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fbe8c0a32d0 0x7fbe8c0a5790 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:31.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:31.083+0000 7fbeab200640 1 --2- 172.21.15.111:0/1308521905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbea4108f60 0x7fbea419ed60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:31.343 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:31.083+0000 7fbeab200640 1 --2- 172.21.15.111:0/1308521905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbea41047e0 0x7fbea419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:31.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:31.083+0000 7fbeab200640 1 --2- 172.21.15.111:0/1308521905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbea4103e30 0x7fbea419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:31.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:31.083+0000 7fbeab200640 1 -- 172.21.15.111:0/1308521905 >> 172.21.15.111:0/1308521905 conn(0x7fbea40ff8d0 msgr2=0x7fbea4100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:31.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:31.083+0000 7fbeab200640 1 -- 172.21.15.111:0/1308521905 shutdown_connections 2024-09-15T22:45:31.344 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:31.083+0000 7fbeab200640 1 -- 172.21.15.111:0/1308521905 wait complete. 2024-09-15T22:45:31.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:31 smithi111 ceph-mon[30629]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:45:31.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:31 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/1308521905' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-15T22:45:31.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:31 smithi121 ceph-mon[34815]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:45:31.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:31 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/1308521905' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-15T22:45:31.965 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:31 smithi192 ceph-mon[35427]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:45:31.966 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:31 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/1308521905' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-15T22:45:32.013 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135913 got 244813135916 for osd.8 2024-09-15T22:45:32.013 DEBUG:teuthology.parallel:result is None 2024-09-15T22:45:32.821 INFO:tasks.cephadm.ceph_manager.ceph:need seq 335007449112 got 335007449116 for osd.11 2024-09-15T22:45:32.821 DEBUG:teuthology.parallel:result is None 2024-09-15T22:45:33.164 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:33.195 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182468 got 107374182471 for osd.3 2024-09-15T22:45:33.195 DEBUG:teuthology.parallel:result is None 2024-09-15T22:45:33.742 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:33 smithi111 ceph-mon[30629]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:45:33.742 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:33 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:45:33.742 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:33 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:45:33.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.744+0000 7fec5a400640 1 -- 172.21.15.111:0/4267053476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec54109140 msgr2=0x7fec54110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:33.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.744+0000 7fec5a400640 1 --2- 172.21.15.111:0/4267053476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec54109140 0x7fec54110cc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fec4c009ca0 tx=0x7fec4c02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:45:33.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.744+0000 7fec5a400640 1 -- 172.21.15.111:0/4267053476 shutdown_connections 2024-09-15T22:45:33.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.744+0000 7fec5a400640 1 --2- 172.21.15.111:0/4267053476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec54109140 0x7fec54110cc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:33.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.744+0000 7fec5a400640 1 --2- 172.21.15.111:0/4267053476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec541049c0 0x7fec54108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:33.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.744+0000 7fec5a400640 1 --2- 172.21.15.111:0/4267053476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec54104010 0x7fec541043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:33.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.744+0000 7fec5a400640 1 -- 172.21.15.111:0/4267053476 >> 172.21.15.111:0/4267053476 conn(0x7fec540ffa50 msgr2=0x7fec54101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:33.746 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.744+0000 7fec5a400640 1 -- 172.21.15.111:0/4267053476 shutdown_connections 2024-09-15T22:45:33.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.745+0000 7fec5a400640 1 -- 172.21.15.111:0/4267053476 wait complete. 2024-09-15T22:45:33.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.746+0000 7fec5a400640 1 Processor -- start 2024-09-15T22:45:33.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.746+0000 7fec5a400640 1 -- start start 2024-09-15T22:45:33.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.746+0000 7fec5a400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec54104010 0x7fec5419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:33.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.746+0000 7fec5a400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec541049c0 0x7fec5419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:33.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.746+0000 7fec5a400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec54109140 0x7fec5419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:33.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.746+0000 7fec5a400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fec54114150 con 0x7fec54109140 2024-09-15T22:45:33.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.746+0000 7fec5a400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fec54113fd0 con 0x7fec54104010 2024-09-15T22:45:33.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.746+0000 7fec5a400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fec541142d0 con 0x7fec541049c0 2024-09-15T22:45:33.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.747+0000 7fec52a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec541049c0 0x7fec5419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:33.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.747+0000 7fec52a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec541049c0 0x7fec5419ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:33328/0 (socket says 172.21.15.111:33328) 2024-09-15T22:45:33.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.747+0000 7fec52a00640 1 -- 172.21.15.111:0/3294229608 learned_addr learned my addr 172.21.15.111:0/3294229608 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:33.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.747+0000 7fec53400640 1 --2- 172.21.15.111:0/3294229608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec54104010 0x7fec5419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:33.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.747+0000 7fec53e00640 1 --2- 172.21.15.111:0/3294229608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec54109140 0x7fec5419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:33.749 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.747+0000 7fec52a00640 1 -- 172.21.15.111:0/3294229608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec54104010 msgr2=0x7fec5419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:33.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.747+0000 7fec52a00640 1 --2- 172.21.15.111:0/3294229608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec54104010 0x7fec5419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:33.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.747+0000 7fec52a00640 1 -- 172.21.15.111:0/3294229608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec54109140 msgr2=0x7fec5419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:33.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.747+0000 7fec52a00640 1 --2- 172.21.15.111:0/3294229608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec54109140 0x7fec5419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:33.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.747+0000 7fec52a00640 1 -- 172.21.15.111:0/3294229608 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5419f5e0 con 0x7fec541049c0 2024-09-15T22:45:33.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.747+0000 7fec52a00640 1 --2- 172.21.15.111:0/3294229608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec541049c0 0x7fec5419ab70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fec4400ea60 tx=0x7fec4400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:33.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.747+0000 7fec4be00640 1 -- 172.21.15.111:0/3294229608 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4400cde0 con 0x7fec541049c0 2024-09-15T22:45:33.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.748+0000 7fec4be00640 1 -- 172.21.15.111:0/3294229608 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec440035f0 con 0x7fec541049c0 2024-09-15T22:45:33.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.748+0000 7fec4be00640 1 -- 172.21.15.111:0/3294229608 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec44010670 con 0x7fec541049c0 2024-09-15T22:45:33.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.748+0000 7fec5a400640 1 -- 172.21.15.111:0/3294229608 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec5419f8d0 con 0x7fec541049c0 2024-09-15T22:45:33.750 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.748+0000 7fec5a400640 1 -- 172.21.15.111:0/3294229608 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec541a71b0 con 0x7fec541049c0 2024-09-15T22:45:33.751 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.749+0000 7fec5a400640 1 -- 172.21.15.111:0/3294229608 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec541043f0 con 0x7fec541049c0 2024-09-15T22:45:33.752 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.751+0000 7fec4be00640 1 -- 172.21.15.111:0/3294229608 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fec440037a0 con 0x7fec541049c0 2024-09-15T22:45:33.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.752+0000 7fec4be00640 1 --2- 172.21.15.111:0/3294229608 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fec380a3350 0x7fec380a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:33.753 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.752+0000 7fec4be00640 1 -- 172.21.15.111:0/3294229608 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fec440cfcb0 con 0x7fec541049c0 2024-09-15T22:45:33.754 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.752+0000 7fec53400640 1 --2- 172.21.15.111:0/3294229608 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fec380a3350 0x7fec380a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:33.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.753+0000 7fec53400640 1 --2- 172.21.15.111:0/3294229608 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fec380a3350 0x7fec380a5810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009bd0 tx=0x7fec3c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:33.757 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.756+0000 7fec4be00640 1 -- 172.21.15.111:0/3294229608 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fec4409c870 con 0x7fec541049c0 2024-09-15T22:45:33.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:33 smithi121 ceph-mon[34815]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:45:33.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:33 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:45:33.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:33 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:45:33.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.942+0000 7fec5a400640 1 -- 172.21.15.111:0/3294229608 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fec5402d7c0 con 0x7fec541049c0 2024-09-15T22:45:33.944 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.942+0000 7fec4be00640 1 -- 172.21.15.111:0/3294229608 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fec4409c170 con 0x7fec541049c0 2024-09-15T22:45:33.945 INFO:teuthology.orchestra.run.smithi111.stdout:304942678050 2024-09-15T22:45:33.946 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.945+0000 7fec5a400640 1 -- 172.21.15.111:0/3294229608 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fec380a3350 msgr2=0x7fec380a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:33.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.945+0000 7fec5a400640 1 --2- 172.21.15.111:0/3294229608 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fec380a3350 0x7fec380a5810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009bd0 tx=0x7fec3c005f90 comp rx=0 tx=0).stop 2024-09-15T22:45:33.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.945+0000 7fec5a400640 1 -- 172.21.15.111:0/3294229608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec541049c0 msgr2=0x7fec5419ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:33.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.945+0000 7fec5a400640 1 --2- 172.21.15.111:0/3294229608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec541049c0 0x7fec5419ab70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fec4400ea60 tx=0x7fec4400ef20 comp rx=0 tx=0).stop 2024-09-15T22:45:33.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.945+0000 7fec5a400640 1 -- 172.21.15.111:0/3294229608 shutdown_connections 2024-09-15T22:45:33.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.945+0000 7fec5a400640 1 --2- 172.21.15.111:0/3294229608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec54109140 0x7fec5419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:33.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.945+0000 7fec5a400640 1 --2- 172.21.15.111:0/3294229608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fec541049c0 0x7fec5419ab70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:33.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.945+0000 7fec5a400640 1 --2- 172.21.15.111:0/3294229608 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fec380a3350 0x7fec380a5810 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:33.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.945+0000 7fec5a400640 1 --2- 172.21.15.111:0/3294229608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec54104010 0x7fec5419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:33.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.945+0000 7fec5a400640 1 -- 172.21.15.111:0/3294229608 >> 172.21.15.111:0/3294229608 conn(0x7fec540ffa50 msgr2=0x7fec54100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:33.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.945+0000 7fec5a400640 1 -- 172.21.15.111:0/3294229608 shutdown_connections 2024-09-15T22:45:33.947 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:33.945+0000 7fec5a400640 1 -- 172.21.15.111:0/3294229608 wait complete. 2024-09-15T22:45:34.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:33 smithi192 ceph-mon[35427]: pgmap v339: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:45:34.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:33 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:45:34.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:33 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:45:34.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:34 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/3294229608' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-15T22:45:34.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:34 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:45:34.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:34 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:45:34.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:34 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:45:34.885 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678045 got 304942678050 for osd.10 2024-09-15T22:45:34.885 DEBUG:teuthology.parallel:result is None 2024-09-15T22:45:34.885 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-15T22:45:34.886 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T22:45:34.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:34 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/3294229608' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-15T22:45:34.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:34 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:45:34.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:34 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:45:34.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:34 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:45:35.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:34 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/3294229608' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-15T22:45:35.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:34 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:45:35.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:34 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:45:35.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:34 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:45:35.253 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:35.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:35 smithi111 ceph-mon[30629]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:45:35.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.813+0000 7f569be00640 1 -- 172.21.15.111:0/1713513936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f569c103e30 msgr2=0x7f569c104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:35.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.813+0000 7f569be00640 1 --2- 172.21.15.111:0/1713513936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f569c103e30 0x7f569c104210 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f568c009900 tx=0x7f568c02a410 comp rx=0 tx=0).stop 2024-09-15T22:45:35.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.814+0000 7f569be00640 1 -- 172.21.15.111:0/1713513936 shutdown_connections 2024-09-15T22:45:35.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.814+0000 7f569be00640 1 --2- 172.21.15.111:0/1713513936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f569c108f60 0x7f569c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:35.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.814+0000 7f569be00640 1 --2- 172.21.15.111:0/1713513936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f569c1047e0 0x7f569c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:35.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.814+0000 7f569be00640 1 --2- 172.21.15.111:0/1713513936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f569c103e30 0x7f569c104210 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:35.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.814+0000 7f569be00640 1 -- 172.21.15.111:0/1713513936 >> 172.21.15.111:0/1713513936 conn(0x7f569c0ff8d0 msgr2=0x7f569c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:35.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.814+0000 7f569be00640 1 -- 172.21.15.111:0/1713513936 shutdown_connections 2024-09-15T22:45:35.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.814+0000 7f569be00640 1 -- 172.21.15.111:0/1713513936 wait complete. 2024-09-15T22:45:35.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.815+0000 7f569be00640 1 Processor -- start 2024-09-15T22:45:35.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.815+0000 7f569be00640 1 -- start start 2024-09-15T22:45:35.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.815+0000 7f569be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f569c103e30 0x7f569c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:35.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.815+0000 7f569be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f569c1047e0 0x7f569c19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:35.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.815+0000 7f569be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f569c108f60 0x7f569c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:35.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.815+0000 7f569be00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f569c113fc0 con 0x7f569c103e30 2024-09-15T22:45:35.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.815+0000 7f569be00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f569c113e40 con 0x7f569c108f60 2024-09-15T22:45:35.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.815+0000 7f569be00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f569c114140 con 0x7f569c1047e0 2024-09-15T22:45:35.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.815+0000 7f569aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f569c103e30 0x7f569c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:35.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.815+0000 7f569aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f569c103e30 0x7f569c19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:49170/0 (socket says 172.21.15.111:49170) 2024-09-15T22:45:35.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.815+0000 7f569aa00640 1 -- 172.21.15.111:0/4046799931 learned_addr learned my addr 172.21.15.111:0/4046799931 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:35.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.815+0000 7f569b400640 1 --2- 172.21.15.111:0/4046799931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f569c108f60 0x7f569c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:35.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.816+0000 7f5692000640 1 --2- 172.21.15.111:0/4046799931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f569c1047e0 0x7f569c19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:35.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.816+0000 7f569aa00640 1 -- 172.21.15.111:0/4046799931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f569c1047e0 msgr2=0x7f569c19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:35.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.816+0000 7f569aa00640 1 --2- 172.21.15.111:0/4046799931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f569c1047e0 0x7f569c19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:35.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.816+0000 7f569aa00640 1 -- 172.21.15.111:0/4046799931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f569c108f60 msgr2=0x7f569c19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:35.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.816+0000 7f569aa00640 1 --2- 172.21.15.111:0/4046799931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f569c108f60 0x7f569c19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:35.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.816+0000 7f569aa00640 1 -- 172.21.15.111:0/4046799931 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569c19f430 con 0x7f569c103e30 2024-09-15T22:45:35.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.816+0000 7f569b400640 1 --2- 172.21.15.111:0/4046799931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f569c108f60 0x7f569c19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:35.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.816+0000 7f569aa00640 1 --2- 172.21.15.111:0/4046799931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f569c103e30 0x7f569c19a480 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f568c006010 tx=0x7f568c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:35.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.816+0000 7f5693e00640 1 -- 172.21.15.111:0/4046799931 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c03f030 con 0x7f569c103e30 2024-09-15T22:45:35.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.817+0000 7f569be00640 1 -- 172.21.15.111:0/4046799931 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f569c19f6c0 con 0x7f569c103e30 2024-09-15T22:45:35.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.817+0000 7f5693e00640 1 -- 172.21.15.111:0/4046799931 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f568c004440 con 0x7f569c103e30 2024-09-15T22:45:35.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.817+0000 7f5693e00640 1 -- 172.21.15.111:0/4046799931 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c035070 con 0x7f569c103e30 2024-09-15T22:45:35.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.817+0000 7f569be00640 1 -- 172.21.15.111:0/4046799931 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f569c1a7000 con 0x7f569c103e30 2024-09-15T22:45:35.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.818+0000 7f569be00640 1 -- 172.21.15.111:0/4046799931 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f569c104210 con 0x7f569c103e30 2024-09-15T22:45:35.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.819+0000 7f5693e00640 1 -- 172.21.15.111:0/4046799931 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f568c042030 con 0x7f569c103e30 2024-09-15T22:45:35.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.821+0000 7f5693e00640 1 --2- 172.21.15.111:0/4046799931 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f56800a3350 0x7f56800a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:35.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.821+0000 7f5693e00640 1 -- 172.21.15.111:0/4046799931 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f568c0efe10 con 0x7f569c103e30 2024-09-15T22:45:35.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.821+0000 7f5692000640 1 --2- 172.21.15.111:0/4046799931 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f56800a3350 0x7f56800a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:35.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.822+0000 7f5692000640 1 --2- 172.21.15.111:0/4046799931 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f56800a3350 0x7f56800a5810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f569c0ffcf0 tx=0x7f5684005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:35.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.825+0000 7f5693e00640 1 -- 172.21.15.111:0/4046799931 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f568c0f3030 con 0x7f569c103e30 2024-09-15T22:45:35.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:35 smithi121 ceph-mon[34815]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:45:35.996 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:35.995+0000 7f569be00640 1 -- 172.21.15.111:0/4046799931 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f569c062630 con 0x7f56800a3350 2024-09-15T22:45:36.005 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:36.003+0000 7f5693e00640 1 -- 172.21.15.111:0/4046799931 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258730 (secure 0 0 0) 0x7f569c062630 con 0x7f56800a3350 2024-09-15T22:45:36.008 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:45:36.009 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-09-15T22:45:36.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:36.007+0000 7f569be00640 1 -- 172.21.15.111:0/4046799931 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f56800a3350 msgr2=0x7f56800a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:36.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:36.007+0000 7f569be00640 1 --2- 172.21.15.111:0/4046799931 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f56800a3350 0x7f56800a5810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f569c0ffcf0 tx=0x7f5684005f90 comp rx=0 tx=0).stop 2024-09-15T22:45:36.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:36.007+0000 7f569be00640 1 -- 172.21.15.111:0/4046799931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f569c103e30 msgr2=0x7f569c19a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:36.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:36.007+0000 7f569be00640 1 --2- 172.21.15.111:0/4046799931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f569c103e30 0x7f569c19a480 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f568c006010 tx=0x7f568c004040 comp rx=0 tx=0).stop 2024-09-15T22:45:36.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:36.007+0000 7f569be00640 1 -- 172.21.15.111:0/4046799931 shutdown_connections 2024-09-15T22:45:36.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:36.007+0000 7f569be00640 1 --2- 172.21.15.111:0/4046799931 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f56800a3350 0x7f56800a5810 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:36.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:36.007+0000 7f569be00640 1 --2- 172.21.15.111:0/4046799931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f569c108f60 0x7f569c19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:36.011 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:36.007+0000 7f569be00640 1 --2- 172.21.15.111:0/4046799931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f569c1047e0 0x7f569c19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:36.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:36.007+0000 7f569be00640 1 --2- 172.21.15.111:0/4046799931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f569c103e30 0x7f569c19a480 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:36.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:36.007+0000 7f569be00640 1 -- 172.21.15.111:0/4046799931 >> 172.21.15.111:0/4046799931 conn(0x7f569c0ff8d0 msgr2=0x7f569c100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:36.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:36.007+0000 7f569be00640 1 -- 172.21.15.111:0/4046799931 shutdown_connections 2024-09-15T22:45:36.012 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:36.007+0000 7f569be00640 1 -- 172.21.15.111:0/4046799931 wait complete. 2024-09-15T22:45:36.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:35 smithi192 ceph-mon[35427]: pgmap v340: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:45:36.892 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2024-09-15T22:45:34.473149+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656915,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":508,"num_read_kb":368,"num_write":349,"num_write_kb":3054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181868,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375160,"kb_used_data":13368,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566664,"statfs":{"total":1151940427776,"available":1151556263936,"internally_reserved":0,"allocated":13688832,"data_stored":7616769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004827"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.633452+0000","last_change":"2024-09-15T22:44:17.222326+0000","last_active":"2024-09-15T22:44:51.633452+0000","last_peered":"2024-09-15T22:44:51.633452+0000","last_clean":"2024-09-15T22:44:51.633452+0000","last_became_active":"2024-09-15T22:44:17.221903+0000","last_became_peered":"2024-09-15T22:44:17.221903+0000","last_unstale":"2024-09-15T22:44:51.633452+0000","last_undegraded":"2024-09-15T22:44:51.633452+0000","last_fullsized":"2024-09-15T22:44:51.633452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:28.303194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.640206+0000","last_change":"2024-09-15T22:44:17.220515+0000","last_active":"2024-09-15T22:44:51.640206+0000","last_peered":"2024-09-15T22:44:51.640206+0000","last_clean":"2024-09-15T22:44:51.640206+0000","last_became_active":"2024-09-15T22:44:17.218794+0000","last_became_peered":"2024-09-15T22:44:17.218794+0000","last_unstale":"2024-09-15T22:44:51.640206+0000","last_undegraded":"2024-09-15T22:44:51.640206+0000","last_fullsized":"2024-09-15T22:44:51.640206+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:58:17.892809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.640270+0000","last_change":"2024-09-15T22:44:17.213183+0000","last_active":"2024-09-15T22:44:51.640270+0000","last_peered":"2024-09-15T22:44:51.640270+0000","last_clean":"2024-09-15T22:44:51.640270+0000","last_became_active":"2024-09-15T22:44:17.212888+0000","last_became_peered":"2024-09-15T22:44:17.212888+0000","last_unstale":"2024-09-15T22:44:51.640270+0000","last_undegraded":"2024-09-15T22:44:51.640270+0000","last_fullsized":"2024-09-15T22:44:51.640270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:32.224099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.795036+0000","last_change":"2024-09-15T22:44:17.203869+0000","last_active":"2024-09-15T22:44:51.795036+0000","last_peered":"2024-09-15T22:44:51.795036+0000","last_clean":"2024-09-15T22:44:51.795036+0000","last_became_active":"2024-09-15T22:44:17.203529+0000","last_became_peered":"2024-09-15T22:44:17.203529+0000","last_unstale":"2024-09-15T22:44:51.795036+0000","last_undegraded":"2024-09-15T22:44:51.795036+0000","last_fullsized":"2024-09-15T22:44:51.795036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:22:58.539513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635166+0000","last_change":"2024-09-15T22:44:17.209521+0000","last_active":"2024-09-15T22:44:51.635166+0000","last_peered":"2024-09-15T22:44:51.635166+0000","last_clean":"2024-09-15T22:44:51.635166+0000","last_became_active":"2024-09-15T22:44:17.209203+0000","last_became_peered":"2024-09-15T22:44:17.209203+0000","last_unstale":"2024-09-15T22:44:51.635166+0000","last_undegraded":"2024-09-15T22:44:51.635166+0000","last_fullsized":"2024-09-15T22:44:51.635166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:34:28.418483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635469+0000","last_change":"2024-09-15T22:44:17.202857+0000","last_active":"2024-09-15T22:44:51.635469+0000","last_peered":"2024-09-15T22:44:51.635469+0000","last_clean":"2024-09-15T22:44:51.635469+0000","last_became_active":"2024-09-15T22:44:17.202530+0000","last_became_peered":"2024-09-15T22:44:17.202530+0000","last_unstale":"2024-09-15T22:44:51.635469+0000","last_undegraded":"2024-09-15T22:44:51.635469+0000","last_fullsized":"2024-09-15T22:44:51.635469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:16:06.947769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635314+0000","last_change":"2024-09-15T22:44:17.203317+0000","last_active":"2024-09-15T22:44:51.635314+0000","last_peered":"2024-09-15T22:44:51.635314+0000","last_clean":"2024-09-15T22:44:51.635314+0000","last_became_active":"2024-09-15T22:44:17.202975+0000","last_became_peered":"2024-09-15T22:44:17.202975+0000","last_unstale":"2024-09-15T22:44:51.635314+0000","last_undegraded":"2024-09-15T22:44:51.635314+0000","last_fullsized":"2024-09-15T22:44:51.635314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:51.292052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.638186+0000","last_change":"2024-09-15T22:44:17.203583+0000","last_active":"2024-09-15T22:44:51.638186+0000","last_peered":"2024-09-15T22:44:51.638186+0000","last_clean":"2024-09-15T22:44:51.638186+0000","last_became_active":"2024-09-15T22:44:17.203251+0000","last_became_peered":"2024-09-15T22:44:17.203251+0000","last_unstale":"2024-09-15T22:44:51.638186+0000","last_undegraded":"2024-09-15T22:44:51.638186+0000","last_fullsized":"2024-09-15T22:44:51.638186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:53:17.378024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.636870+0000","last_change":"2024-09-15T22:44:17.209643+0000","last_active":"2024-09-15T22:44:51.636870+0000","last_peered":"2024-09-15T22:44:51.636870+0000","last_clean":"2024-09-15T22:44:51.636870+0000","last_became_active":"2024-09-15T22:44:17.208709+0000","last_became_peered":"2024-09-15T22:44:17.208709+0000","last_unstale":"2024-09-15T22:44:51.636870+0000","last_undegraded":"2024-09-15T22:44:51.636870+0000","last_fullsized":"2024-09-15T22:44:51.636870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:38:38.305676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.639787+0000","last_change":"2024-09-15T22:44:51.639787+0000","last_active":"2024-09-15T22:44:51.639787+0000","last_peered":"2024-09-15T22:44:51.639787+0000","last_clean":"2024-09-15T22:44:51.639787+0000","last_became_active":"2024-09-15T22:44:51.639300+0000","last_became_peered":"2024-09-15T22:44:51.639300+0000","last_unstale":"2024-09-15T22:44:51.639787+0000","last_undegraded":"2024-09-15T22:44:51.639787+0000","last_fullsized":"2024-09-15T22:44:51.639787+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:52.775861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635522+0000","last_change":"2024-09-15T22:44:17.208106+0000","last_active":"2024-09-15T22:44:51.635522+0000","last_peered":"2024-09-15T22:44:51.635522+0000","last_clean":"2024-09-15T22:44:51.635522+0000","last_became_active":"2024-09-15T22:44:17.207774+0000","last_became_peered":"2024-09-15T22:44:17.207774+0000","last_unstale":"2024-09-15T22:44:51.635522+0000","last_undegraded":"2024-09-15T22:44:51.635522+0000","last_fullsized":"2024-09-15T22:44:51.635522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:05:15.876692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.795040+0000","last_change":"2024-09-15T22:44:17.202857+0000","last_active":"2024-09-15T22:44:51.795040+0000","last_peered":"2024-09-15T22:44:51.795040+0000","last_clean":"2024-09-15T22:44:51.795040+0000","last_became_active":"2024-09-15T22:44:17.202520+0000","last_became_peered":"2024-09-15T22:44:17.202520+0000","last_unstale":"2024-09-15T22:44:51.795040+0000","last_undegraded":"2024-09-15T22:44:51.795040+0000","last_fullsized":"2024-09-15T22:44:51.795040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:11:19.111086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.632664+0000","last_change":"2024-09-15T22:44:17.201776+0000","last_active":"2024-09-15T22:44:51.632664+0000","last_peered":"2024-09-15T22:44:51.632664+0000","last_clean":"2024-09-15T22:44:51.632664+0000","last_became_active":"2024-09-15T22:44:17.201508+0000","last_became_peered":"2024-09-15T22:44:17.201508+0000","last_unstale":"2024-09-15T22:44:51.632664+0000","last_undegraded":"2024-09-15T22:44:51.632664+0000","last_fullsized":"2024-09-15T22:44:51.632664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:22:48.104776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.639416+0000","last_change":"2024-09-15T22:44:17.221005+0000","last_active":"2024-09-15T22:44:51.639416+0000","last_peered":"2024-09-15T22:44:51.639416+0000","last_clean":"2024-09-15T22:44:51.639416+0000","last_became_active":"2024-09-15T22:44:17.218455+0000","last_became_peered":"2024-09-15T22:44:17.218455+0000","last_unstale":"2024-09-15T22:44:51.639416+0000","last_undegraded":"2024-09-15T22:44:51.639416+0000","last_fullsized":"2024-09-15T22:44:51.639416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:40.227843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.639776+0000","last_change":"2024-09-15T22:44:17.208125+0000","last_active":"2024-09-15T22:44:51.639776+0000","last_peered":"2024-09-15T22:44:51.639776+0000","last_clean":"2024-09-15T22:44:51.639776+0000","last_became_active":"2024-09-15T22:44:17.207762+0000","last_became_peered":"2024-09-15T22:44:17.207762+0000","last_unstale":"2024-09-15T22:44:51.639776+0000","last_undegraded":"2024-09-15T22:44:51.639776+0000","last_fullsized":"2024-09-15T22:44:51.639776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:21:45.181921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635521+0000","last_change":"2024-09-15T22:44:17.212058+0000","last_active":"2024-09-15T22:44:51.635521+0000","last_peered":"2024-09-15T22:44:51.635521+0000","last_clean":"2024-09-15T22:44:51.635521+0000","last_became_active":"2024-09-15T22:44:17.211789+0000","last_became_peered":"2024-09-15T22:44:17.211789+0000","last_unstale":"2024-09-15T22:44:51.635521+0000","last_undegraded":"2024-09-15T22:44:51.635521+0000","last_fullsized":"2024-09-15T22:44:51.635521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:14.647374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.633547+0000","last_change":"2024-09-15T22:44:17.225139+0000","last_active":"2024-09-15T22:44:51.633547+0000","last_peered":"2024-09-15T22:44:51.633547+0000","last_clean":"2024-09-15T22:44:51.633547+0000","last_became_active":"2024-09-15T22:44:17.224791+0000","last_became_peered":"2024-09-15T22:44:17.224791+0000","last_unstale":"2024-09-15T22:44:51.633547+0000","last_undegraded":"2024-09-15T22:44:51.633547+0000","last_fullsized":"2024-09-15T22:44:51.633547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:56.457155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.632502+0000","last_change":"2024-09-15T22:44:17.208604+0000","last_active":"2024-09-15T22:44:51.632502+0000","last_peered":"2024-09-15T22:44:51.632502+0000","last_clean":"2024-09-15T22:44:51.632502+0000","last_became_active":"2024-09-15T22:44:17.202231+0000","last_became_peered":"2024-09-15T22:44:17.202231+0000","last_unstale":"2024-09-15T22:44:51.632502+0000","last_undegraded":"2024-09-15T22:44:51.632502+0000","last_fullsized":"2024-09-15T22:44:51.632502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:39.572881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635576+0000","last_change":"2024-09-15T22:44:17.211837+0000","last_active":"2024-09-15T22:44:51.635576+0000","last_peered":"2024-09-15T22:44:51.635576+0000","last_clean":"2024-09-15T22:44:51.635576+0000","last_became_active":"2024-09-15T22:44:17.211460+0000","last_became_peered":"2024-09-15T22:44:17.211460+0000","last_unstale":"2024-09-15T22:44:51.635576+0000","last_undegraded":"2024-09-15T22:44:51.635576+0000","last_fullsized":"2024-09-15T22:44:51.635576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:21:51.820134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.633047+0000","last_change":"2024-09-15T22:44:17.206350+0000","last_active":"2024-09-15T22:44:51.633047+0000","last_peered":"2024-09-15T22:44:51.633047+0000","last_clean":"2024-09-15T22:44:51.633047+0000","last_became_active":"2024-09-15T22:44:17.204943+0000","last_became_peered":"2024-09-15T22:44:17.204943+0000","last_unstale":"2024-09-15T22:44:51.633047+0000","last_undegraded":"2024-09-15T22:44:51.633047+0000","last_fullsized":"2024-09-15T22:44:51.633047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:42:59.918220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.632678+0000","last_change":"2024-09-15T22:44:17.223390+0000","last_active":"2024-09-15T22:44:51.632678+0000","last_peered":"2024-09-15T22:44:51.632678+0000","last_clean":"2024-09-15T22:44:51.632678+0000","last_became_active":"2024-09-15T22:44:17.223008+0000","last_became_peered":"2024-09-15T22:44:17.223008+0000","last_unstale":"2024-09-15T22:44:51.632678+0000","last_undegraded":"2024-09-15T22:44:51.632678+0000","last_fullsized":"2024-09-15T22:44:51.632678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:31.545336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.636324+0000","last_change":"2024-09-15T22:44:17.207567+0000","last_active":"2024-09-15T22:44:51.636324+0000","last_peered":"2024-09-15T22:44:51.636324+0000","last_clean":"2024-09-15T22:44:51.636324+0000","last_became_active":"2024-09-15T22:44:17.207245+0000","last_became_peered":"2024-09-15T22:44:17.207245+0000","last_unstale":"2024-09-15T22:44:51.636324+0000","last_undegraded":"2024-09-15T22:44:51.636324+0000","last_fullsized":"2024-09-15T22:44:51.636324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:35.169205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.638414+0000","last_change":"2024-09-15T22:44:17.209449+0000","last_active":"2024-09-15T22:44:51.638414+0000","last_peered":"2024-09-15T22:44:51.638414+0000","last_clean":"2024-09-15T22:44:51.638414+0000","last_became_active":"2024-09-15T22:44:17.209012+0000","last_became_peered":"2024-09-15T22:44:17.209012+0000","last_unstale":"2024-09-15T22:44:51.638414+0000","last_undegraded":"2024-09-15T22:44:51.638414+0000","last_fullsized":"2024-09-15T22:44:51.638414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:59.550776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.640005+0000","last_change":"2024-09-15T22:44:17.214184+0000","last_active":"2024-09-15T22:44:51.640005+0000","last_peered":"2024-09-15T22:44:51.640005+0000","last_clean":"2024-09-15T22:44:51.640005+0000","last_became_active":"2024-09-15T22:44:17.213884+0000","last_became_peered":"2024-09-15T22:44:17.213884+0000","last_unstale":"2024-09-15T22:44:51.640005+0000","last_undegraded":"2024-09-15T22:44:51.640005+0000","last_fullsized":"2024-09-15T22:44:51.640005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:33.336473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.639279+0000","last_change":"2024-09-15T22:44:17.203904+0000","last_active":"2024-09-15T22:44:51.639279+0000","last_peered":"2024-09-15T22:44:51.639279+0000","last_clean":"2024-09-15T22:44:51.639279+0000","last_became_active":"2024-09-15T22:44:17.203484+0000","last_became_peered":"2024-09-15T22:44:17.203484+0000","last_unstale":"2024-09-15T22:44:51.639279+0000","last_undegraded":"2024-09-15T22:44:51.639279+0000","last_fullsized":"2024-09-15T22:44:51.639279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:33:09.906869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.636131+0000","last_change":"2024-09-15T22:44:17.209356+0000","last_active":"2024-09-15T22:44:51.636131+0000","last_peered":"2024-09-15T22:44:51.636131+0000","last_clean":"2024-09-15T22:44:51.636131+0000","last_became_active":"2024-09-15T22:44:17.207229+0000","last_became_peered":"2024-09-15T22:44:17.207229+0000","last_unstale":"2024-09-15T22:44:51.636131+0000","last_undegraded":"2024-09-15T22:44:51.636131+0000","last_fullsized":"2024-09-15T22:44:51.636131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:32:28.405429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.633207+0000","last_change":"2024-09-15T22:44:17.202505+0000","last_active":"2024-09-15T22:44:51.633207+0000","last_peered":"2024-09-15T22:44:51.633207+0000","last_clean":"2024-09-15T22:44:51.633207+0000","last_became_active":"2024-09-15T22:44:17.202154+0000","last_became_peered":"2024-09-15T22:44:17.202154+0000","last_unstale":"2024-09-15T22:44:51.633207+0000","last_undegraded":"2024-09-15T22:44:51.633207+0000","last_fullsized":"2024-09-15T22:44:51.633207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:40:05.680982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.632632+0000","last_change":"2024-09-15T22:44:17.222767+0000","last_active":"2024-09-15T22:44:51.632632+0000","last_peered":"2024-09-15T22:44:51.632632+0000","last_clean":"2024-09-15T22:44:51.632632+0000","last_became_active":"2024-09-15T22:44:17.221655+0000","last_became_peered":"2024-09-15T22:44:17.221655+0000","last_unstale":"2024-09-15T22:44:51.632632+0000","last_undegraded":"2024-09-15T22:44:51.632632+0000","last_fullsized":"2024-09-15T22:44:51.632632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:22.457576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.639825+0000","last_change":"2024-09-15T22:44:17.217485+0000","last_active":"2024-09-15T22:44:51.639825+0000","last_peered":"2024-09-15T22:44:51.639825+0000","last_clean":"2024-09-15T22:44:51.639825+0000","last_became_active":"2024-09-15T22:44:17.217138+0000","last_became_peered":"2024-09-15T22:44:17.217138+0000","last_unstale":"2024-09-15T22:44:51.639825+0000","last_undegraded":"2024-09-15T22:44:51.639825+0000","last_fullsized":"2024-09-15T22:44:51.639825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:41:40.606593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635434+0000","last_change":"2024-09-15T22:44:17.217996+0000","last_active":"2024-09-15T22:44:51.635434+0000","last_peered":"2024-09-15T22:44:51.635434+0000","last_clean":"2024-09-15T22:44:51.635434+0000","last_became_active":"2024-09-15T22:44:17.217467+0000","last_became_peered":"2024-09-15T22:44:17.217467+0000","last_unstale":"2024-09-15T22:44:51.635434+0000","last_undegraded":"2024-09-15T22:44:51.635434+0000","last_fullsized":"2024-09-15T22:44:51.635434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:25:45.150979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.636279+0000","last_change":"2024-09-15T22:44:17.202840+0000","last_active":"2024-09-15T22:44:51.636279+0000","last_peered":"2024-09-15T22:44:51.636279+0000","last_clean":"2024-09-15T22:44:51.636279+0000","last_became_active":"2024-09-15T22:44:17.202481+0000","last_became_peered":"2024-09-15T22:44:17.202481+0000","last_unstale":"2024-09-15T22:44:51.636279+0000","last_undegraded":"2024-09-15T22:44:51.636279+0000","last_fullsized":"2024-09-15T22:44:51.636279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:31:09.360211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.632973+0000","last_change":"2024-09-15T22:44:17.209387+0000","last_active":"2024-09-15T22:44:51.632973+0000","last_peered":"2024-09-15T22:44:51.632973+0000","last_clean":"2024-09-15T22:44:51.632973+0000","last_became_active":"2024-09-15T22:44:17.209100+0000","last_became_peered":"2024-09-15T22:44:17.209100+0000","last_unstale":"2024-09-15T22:44:51.632973+0000","last_undegraded":"2024-09-15T22:44:51.632973+0000","last_fullsized":"2024-09-15T22:44:51.632973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:57:47.942784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635137+0000","last_change":"2024-09-15T22:44:17.203626+0000","last_active":"2024-09-15T22:44:51.635137+0000","last_peered":"2024-09-15T22:44:51.635137+0000","last_clean":"2024-09-15T22:44:51.635137+0000","last_became_active":"2024-09-15T22:44:17.203313+0000","last_became_peered":"2024-09-15T22:44:17.203313+0000","last_unstale":"2024-09-15T22:44:51.635137+0000","last_undegraded":"2024-09-15T22:44:51.635137+0000","last_fullsized":"2024-09-15T22:44:51.635137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:42:53.765397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":109,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:45:31.859623+0000","last_change":"2024-09-15T22:44:51.800707+0000","last_active":"2024-09-15T22:45:31.859623+0000","last_peered":"2024-09-15T22:45:31.859623+0000","last_clean":"2024-09-15T22:45:31.859623+0000","last_became_active":"2024-09-15T22:44:51.796445+0000","last_became_peered":"2024-09-15T22:44:51.796445+0000","last_unstale":"2024-09-15T22:45:31.859623+0000","last_undegraded":"2024-09-15T22:45:31.859623+0000","last_fullsized":"2024-09-15T22:45:31.859623+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:13:09.444747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":60,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.638512+0000","last_change":"2024-09-15T22:44:17.210054+0000","last_active":"2024-09-15T22:44:51.638512+0000","last_peered":"2024-09-15T22:44:51.638512+0000","last_clean":"2024-09-15T22:44:51.638512+0000","last_became_active":"2024-09-15T22:44:17.209607+0000","last_became_peered":"2024-09-15T22:44:17.209607+0000","last_unstale":"2024-09-15T22:44:51.638512+0000","last_undegraded":"2024-09-15T22:44:51.638512+0000","last_fullsized":"2024-09-15T22:44:51.638512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:13.867965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.636335+0000","last_change":"2024-09-15T22:44:14.167537+0000","last_active":"2024-09-15T22:44:51.636335+0000","last_peered":"2024-09-15T22:44:51.636335+0000","last_clean":"2024-09-15T22:44:51.636335+0000","last_became_active":"2024-09-15T22:44:14.167173+0000","last_became_peered":"2024-09-15T22:44:14.167173+0000","last_unstale":"2024-09-15T22:44:51.636335+0000","last_undegraded":"2024-09-15T22:44:51.636335+0000","last_fullsized":"2024-09-15T22:44:51.636335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:45:56.539218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.632329+0000","last_change":"2024-09-15T22:44:14.166676+0000","last_active":"2024-09-15T22:44:51.632329+0000","last_peered":"2024-09-15T22:44:51.632329+0000","last_clean":"2024-09-15T22:44:51.632329+0000","last_became_active":"2024-09-15T22:44:14.166342+0000","last_became_peered":"2024-09-15T22:44:14.166342+0000","last_unstale":"2024-09-15T22:44:51.632329+0000","last_undegraded":"2024-09-15T22:44:51.632329+0000","last_fullsized":"2024-09-15T22:44:51.632329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:41.861253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.638293+0000","last_change":"2024-09-15T22:44:17.211312+0000","last_active":"2024-09-15T22:44:51.638293+0000","last_peered":"2024-09-15T22:44:51.638293+0000","last_clean":"2024-09-15T22:44:51.638293+0000","last_became_active":"2024-09-15T22:44:17.208771+0000","last_became_peered":"2024-09-15T22:44:17.208771+0000","last_unstale":"2024-09-15T22:44:51.638293+0000","last_undegraded":"2024-09-15T22:44:51.638293+0000","last_fullsized":"2024-09-15T22:44:51.638293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:59:51.250489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.640372+0000","last_change":"2024-09-15T22:44:14.167263+0000","last_active":"2024-09-15T22:44:51.640372+0000","last_peered":"2024-09-15T22:44:51.640372+0000","last_clean":"2024-09-15T22:44:51.640372+0000","last_became_active":"2024-09-15T22:44:14.166942+0000","last_became_peered":"2024-09-15T22:44:14.166942+0000","last_unstale":"2024-09-15T22:44:51.640372+0000","last_undegraded":"2024-09-15T22:44:51.640372+0000","last_fullsized":"2024-09-15T22:44:51.640372+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:47.320374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.631899+0000","last_change":"2024-09-15T22:44:17.210986+0000","last_active":"2024-09-15T22:44:51.631899+0000","last_peered":"2024-09-15T22:44:51.631899+0000","last_clean":"2024-09-15T22:44:51.631899+0000","last_became_active":"2024-09-15T22:44:17.210618+0000","last_became_peered":"2024-09-15T22:44:17.210618+0000","last_unstale":"2024-09-15T22:44:51.631899+0000","last_undegraded":"2024-09-15T22:44:51.631899+0000","last_fullsized":"2024-09-15T22:44:51.631899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:58:04.961630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":88,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:45:28.854042+0000","last_change":"2024-09-15T22:44:14.164770+0000","last_active":"2024-09-15T22:45:28.854042+0000","last_peered":"2024-09-15T22:45:28.854042+0000","last_clean":"2024-09-15T22:45:28.854042+0000","last_became_active":"2024-09-15T22:44:14.164411+0000","last_became_peered":"2024-09-15T22:44:14.164411+0000","last_unstale":"2024-09-15T22:45:28.854042+0000","last_undegraded":"2024-09-15T22:45:28.854042+0000","last_fullsized":"2024-09-15T22:45:28.854042+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:26:47.623520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":59,"num_read_kb":59,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.636419+0000","last_change":"2024-09-15T22:44:17.204049+0000","last_active":"2024-09-15T22:44:51.636419+0000","last_peered":"2024-09-15T22:44:51.636419+0000","last_clean":"2024-09-15T22:44:51.636419+0000","last_became_active":"2024-09-15T22:44:17.203738+0000","last_became_peered":"2024-09-15T22:44:17.203738+0000","last_unstale":"2024-09-15T22:44:51.636419+0000","last_undegraded":"2024-09-15T22:44:51.636419+0000","last_fullsized":"2024-09-15T22:44:51.636419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:56.347792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635667+0000","last_change":"2024-09-15T22:44:14.167228+0000","last_active":"2024-09-15T22:44:51.635667+0000","last_peered":"2024-09-15T22:44:51.635667+0000","last_clean":"2024-09-15T22:44:51.635667+0000","last_became_active":"2024-09-15T22:44:14.166662+0000","last_became_peered":"2024-09-15T22:44:14.166662+0000","last_unstale":"2024-09-15T22:44:51.635667+0000","last_undegraded":"2024-09-15T22:44:51.635667+0000","last_fullsized":"2024-09-15T22:44:51.635667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:31:17.160718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.640531+0000","last_change":"2024-09-15T22:44:17.212549+0000","last_active":"2024-09-15T22:44:51.640531+0000","last_peered":"2024-09-15T22:44:51.640531+0000","last_clean":"2024-09-15T22:44:51.640531+0000","last_became_active":"2024-09-15T22:44:17.212123+0000","last_became_peered":"2024-09-15T22:44:17.212123+0000","last_unstale":"2024-09-15T22:44:51.640531+0000","last_undegraded":"2024-09-15T22:44:51.640531+0000","last_fullsized":"2024-09-15T22:44:51.640531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:32:45.241199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.640041+0000","last_change":"2024-09-15T22:44:14.679664+0000","last_active":"2024-09-15T22:44:51.640041+0000","last_peered":"2024-09-15T22:44:51.640041+0000","last_clean":"2024-09-15T22:44:51.640041+0000","last_became_active":"2024-09-15T22:44:14.679256+0000","last_became_peered":"2024-09-15T22:44:14.679256+0000","last_unstale":"2024-09-15T22:44:51.640041+0000","last_undegraded":"2024-09-15T22:44:51.640041+0000","last_fullsized":"2024-09-15T22:44:51.640041+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:33:41.512488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635773+0000","last_change":"2024-09-15T22:44:17.200448+0000","last_active":"2024-09-15T22:44:51.635773+0000","last_peered":"2024-09-15T22:44:51.635773+0000","last_clean":"2024-09-15T22:44:51.635773+0000","last_became_active":"2024-09-15T22:44:17.200114+0000","last_became_peered":"2024-09-15T22:44:17.200114+0000","last_unstale":"2024-09-15T22:44:51.635773+0000","last_undegraded":"2024-09-15T22:44:51.635773+0000","last_fullsized":"2024-09-15T22:44:51.635773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:36.403644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'31","reported_seq":185,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:45:29.855960+0000","last_change":"2024-09-15T22:44:14.167067+0000","last_active":"2024-09-15T22:45:29.855960+0000","last_peered":"2024-09-15T22:45:29.855960+0000","last_clean":"2024-09-15T22:45:29.855960+0000","last_became_active":"2024-09-15T22:44:14.166613+0000","last_became_peered":"2024-09-15T22:44:14.166613+0000","last_unstale":"2024-09-15T22:45:29.855960+0000","last_undegraded":"2024-09-15T22:45:29.855960+0000","last_fullsized":"2024-09-15T22:45:29.855960+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:15.360717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62310,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":128,"num_read_kb":124,"num_write":32,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635625+0000","last_change":"2024-09-15T22:44:17.202705+0000","last_active":"2024-09-15T22:44:51.635625+0000","last_peered":"2024-09-15T22:44:51.635625+0000","last_clean":"2024-09-15T22:44:51.635625+0000","last_became_active":"2024-09-15T22:44:17.202346+0000","last_became_peered":"2024-09-15T22:44:17.202346+0000","last_unstale":"2024-09-15T22:44:51.635625+0000","last_undegraded":"2024-09-15T22:44:51.635625+0000","last_fullsized":"2024-09-15T22:44:51.635625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:31.193746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.637052+0000","last_change":"2024-09-15T22:44:14.169255+0000","last_active":"2024-09-15T22:44:51.637052+0000","last_peered":"2024-09-15T22:44:51.637052+0000","last_clean":"2024-09-15T22:44:51.637052+0000","last_became_active":"2024-09-15T22:44:14.168774+0000","last_became_peered":"2024-09-15T22:44:14.168774+0000","last_unstale":"2024-09-15T22:44:51.637052+0000","last_undegraded":"2024-09-15T22:44:51.637052+0000","last_fullsized":"2024-09-15T22:44:51.637052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:48.858831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.632337+0000","last_change":"2024-09-15T22:44:17.218635+0000","last_active":"2024-09-15T22:44:51.632337+0000","last_peered":"2024-09-15T22:44:51.632337+0000","last_clean":"2024-09-15T22:44:51.632337+0000","last_became_active":"2024-09-15T22:44:17.218287+0000","last_became_peered":"2024-09-15T22:44:17.218287+0000","last_unstale":"2024-09-15T22:44:51.632337+0000","last_undegraded":"2024-09-15T22:44:51.632337+0000","last_fullsized":"2024-09-15T22:44:51.632337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:58.264382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.639773+0000","last_change":"2024-09-15T22:44:14.166580+0000","last_active":"2024-09-15T22:44:51.639773+0000","last_peered":"2024-09-15T22:44:51.639773+0000","last_clean":"2024-09-15T22:44:51.639773+0000","last_became_active":"2024-09-15T22:44:14.164955+0000","last_became_peered":"2024-09-15T22:44:14.164955+0000","last_unstale":"2024-09-15T22:44:51.639773+0000","last_undegraded":"2024-09-15T22:44:51.639773+0000","last_fullsized":"2024-09-15T22:44:51.639773+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:14.215662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635544+0000","last_change":"2024-09-15T22:44:17.205731+0000","last_active":"2024-09-15T22:44:51.635544+0000","last_peered":"2024-09-15T22:44:51.635544+0000","last_clean":"2024-09-15T22:44:51.635544+0000","last_became_active":"2024-09-15T22:44:17.205361+0000","last_became_peered":"2024-09-15T22:44:17.205361+0000","last_unstale":"2024-09-15T22:44:51.635544+0000","last_undegraded":"2024-09-15T22:44:51.635544+0000","last_fullsized":"2024-09-15T22:44:51.635544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:25.430166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:45:31.662304+0000","last_change":"2024-09-15T22:44:14.166471+0000","last_active":"2024-09-15T22:45:31.662304+0000","last_peered":"2024-09-15T22:45:31.662304+0000","last_clean":"2024-09-15T22:45:31.662304+0000","last_became_active":"2024-09-15T22:44:14.165969+0000","last_became_peered":"2024-09-15T22:44:14.165969+0000","last_unstale":"2024-09-15T22:45:31.662304+0000","last_undegraded":"2024-09-15T22:45:31.662304+0000","last_fullsized":"2024-09-15T22:45:31.662304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:57:40.108575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635288+0000","last_change":"2024-09-15T22:44:17.201488+0000","last_active":"2024-09-15T22:44:51.635288+0000","last_peered":"2024-09-15T22:44:51.635288+0000","last_clean":"2024-09-15T22:44:51.635288+0000","last_became_active":"2024-09-15T22:44:17.201101+0000","last_became_peered":"2024-09-15T22:44:17.201101+0000","last_unstale":"2024-09-15T22:44:51.635288+0000","last_undegraded":"2024-09-15T22:44:51.635288+0000","last_fullsized":"2024-09-15T22:44:51.635288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:41.758433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635006+0000","last_change":"2024-09-15T22:44:14.680734+0000","last_active":"2024-09-15T22:44:51.635006+0000","last_peered":"2024-09-15T22:44:51.635006+0000","last_clean":"2024-09-15T22:44:51.635006+0000","last_became_active":"2024-09-15T22:44:14.680362+0000","last_became_peered":"2024-09-15T22:44:14.680362+0000","last_unstale":"2024-09-15T22:44:51.635006+0000","last_undegraded":"2024-09-15T22:44:51.635006+0000","last_fullsized":"2024-09-15T22:44:51.635006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:22:37.086300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.638580+0000","last_change":"2024-09-15T22:44:17.199944+0000","last_active":"2024-09-15T22:44:51.638580+0000","last_peered":"2024-09-15T22:44:51.638580+0000","last_clean":"2024-09-15T22:44:51.638580+0000","last_became_active":"2024-09-15T22:44:17.199586+0000","last_became_peered":"2024-09-15T22:44:17.199586+0000","last_unstale":"2024-09-15T22:44:51.638580+0000","last_undegraded":"2024-09-15T22:44:51.638580+0000","last_fullsized":"2024-09-15T22:44:51.638580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:42:22.178939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635879+0000","last_change":"2024-09-15T22:44:14.169077+0000","last_active":"2024-09-15T22:44:51.635879+0000","last_peered":"2024-09-15T22:44:51.635879+0000","last_clean":"2024-09-15T22:44:51.635879+0000","last_became_active":"2024-09-15T22:44:14.168675+0000","last_became_peered":"2024-09-15T22:44:14.168675+0000","last_unstale":"2024-09-15T22:44:51.635879+0000","last_undegraded":"2024-09-15T22:44:51.635879+0000","last_fullsized":"2024-09-15T22:44:51.635879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:01.892217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.633178+0000","last_change":"2024-09-15T22:44:17.208587+0000","last_active":"2024-09-15T22:44:51.633178+0000","last_peered":"2024-09-15T22:44:51.633178+0000","last_clean":"2024-09-15T22:44:51.633178+0000","last_became_active":"2024-09-15T22:44:17.208260+0000","last_became_peered":"2024-09-15T22:44:17.208260+0000","last_unstale":"2024-09-15T22:44:51.633178+0000","last_undegraded":"2024-09-15T22:44:51.633178+0000","last_fullsized":"2024-09-15T22:44:51.633178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:01:04.154149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.632308+0000","last_change":"2024-09-15T22:44:14.168223+0000","last_active":"2024-09-15T22:44:51.632308+0000","last_peered":"2024-09-15T22:44:51.632308+0000","last_clean":"2024-09-15T22:44:51.632308+0000","last_became_active":"2024-09-15T22:44:14.167830+0000","last_became_peered":"2024-09-15T22:44:14.167830+0000","last_unstale":"2024-09-15T22:44:51.632308+0000","last_undegraded":"2024-09-15T22:44:51.632308+0000","last_fullsized":"2024-09-15T22:44:51.632308+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:54:38.064606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635822+0000","last_change":"2024-09-15T22:44:17.209696+0000","last_active":"2024-09-15T22:44:51.635822+0000","last_peered":"2024-09-15T22:44:51.635822+0000","last_clean":"2024-09-15T22:44:51.635822+0000","last_became_active":"2024-09-15T22:44:17.208667+0000","last_became_peered":"2024-09-15T22:44:17.208667+0000","last_unstale":"2024-09-15T22:44:51.635822+0000","last_undegraded":"2024-09-15T22:44:51.635822+0000","last_fullsized":"2024-09-15T22:44:51.635822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:04.653022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.639263+0000","last_change":"2024-09-15T22:44:14.169728+0000","last_active":"2024-09-15T22:44:51.639263+0000","last_peered":"2024-09-15T22:44:51.639263+0000","last_clean":"2024-09-15T22:44:51.639263+0000","last_became_active":"2024-09-15T22:44:14.169354+0000","last_became_peered":"2024-09-15T22:44:14.169354+0000","last_unstale":"2024-09-15T22:44:51.639263+0000","last_undegraded":"2024-09-15T22:44:51.639263+0000","last_fullsized":"2024-09-15T22:44:51.639263+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:20.273852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.644709+0000","last_change":"2024-09-15T22:44:51.644709+0000","last_active":"2024-09-15T22:44:51.644709+0000","last_peered":"2024-09-15T22:44:51.644709+0000","last_clean":"2024-09-15T22:44:51.644709+0000","last_became_active":"2024-09-15T22:44:51.641165+0000","last_became_peered":"2024-09-15T22:44:51.641165+0000","last_unstale":"2024-09-15T22:44:51.644709+0000","last_undegraded":"2024-09-15T22:44:51.644709+0000","last_fullsized":"2024-09-15T22:44:51.644709+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:36.253167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635928+0000","last_change":"2024-09-15T22:44:14.679757+0000","last_active":"2024-09-15T22:44:51.635928+0000","last_peered":"2024-09-15T22:44:51.635928+0000","last_clean":"2024-09-15T22:44:51.635928+0000","last_became_active":"2024-09-15T22:44:14.679284+0000","last_became_peered":"2024-09-15T22:44:14.679284+0000","last_unstale":"2024-09-15T22:44:51.635928+0000","last_undegraded":"2024-09-15T22:44:51.635928+0000","last_fullsized":"2024-09-15T22:44:51.635928+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:47:21.368373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.636324+0000","last_change":"2024-09-15T22:44:17.207115+0000","last_active":"2024-09-15T22:44:51.636324+0000","last_peered":"2024-09-15T22:44:51.636324+0000","last_clean":"2024-09-15T22:44:51.636324+0000","last_became_active":"2024-09-15T22:44:17.206800+0000","last_became_peered":"2024-09-15T22:44:17.206800+0000","last_unstale":"2024-09-15T22:44:51.636324+0000","last_undegraded":"2024-09-15T22:44:51.636324+0000","last_fullsized":"2024-09-15T22:44:51.636324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:49:17.497725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":339,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635886+0000","last_change":"2024-09-15T22:42:59.117839+0000","last_active":"2024-09-15T22:44:51.635886+0000","last_peered":"2024-09-15T22:44:51.635886+0000","last_clean":"2024-09-15T22:44:51.635886+0000","last_became_active":"2024-09-15T22:42:59.117434+0000","last_became_peered":"2024-09-15T22:42:59.117434+0000","last_unstale":"2024-09-15T22:44:51.635886+0000","last_undegraded":"2024-09-15T22:44:51.635886+0000","last_fullsized":"2024-09-15T22:44:51.635886+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:39:19.064511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:39:19.064511+0000","last_clean_scrub_stamp":"2024-09-15T22:39:19.064511+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:38.974950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.795272+0000","last_change":"2024-09-15T22:44:14.170422+0000","last_active":"2024-09-15T22:44:51.795272+0000","last_peered":"2024-09-15T22:44:51.795272+0000","last_clean":"2024-09-15T22:44:51.795272+0000","last_became_active":"2024-09-15T22:44:14.169152+0000","last_became_peered":"2024-09-15T22:44:14.169152+0000","last_unstale":"2024-09-15T22:44:51.795272+0000","last_undegraded":"2024-09-15T22:44:51.795272+0000","last_fullsized":"2024-09-15T22:44:51.795272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:10:00.890434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.638676+0000","last_change":"2024-09-15T22:44:17.212972+0000","last_active":"2024-09-15T22:44:51.638676+0000","last_peered":"2024-09-15T22:44:51.638676+0000","last_clean":"2024-09-15T22:44:51.638676+0000","last_became_active":"2024-09-15T22:44:17.212546+0000","last_became_peered":"2024-09-15T22:44:17.212546+0000","last_unstale":"2024-09-15T22:44:51.638676+0000","last_undegraded":"2024-09-15T22:44:51.638676+0000","last_fullsized":"2024-09-15T22:44:51.638676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:36.322855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635707+0000","last_change":"2024-09-15T22:44:14.171827+0000","last_active":"2024-09-15T22:44:51.635707+0000","last_peered":"2024-09-15T22:44:51.635707+0000","last_clean":"2024-09-15T22:44:51.635707+0000","last_became_active":"2024-09-15T22:44:14.171447+0000","last_became_peered":"2024-09-15T22:44:14.171447+0000","last_unstale":"2024-09-15T22:44:51.635707+0000","last_undegraded":"2024-09-15T22:44:51.635707+0000","last_fullsized":"2024-09-15T22:44:51.635707+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:40:49.869819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.632269+0000","last_change":"2024-09-15T22:44:17.208666+0000","last_active":"2024-09-15T22:44:51.632269+0000","last_peered":"2024-09-15T22:44:51.632269+0000","last_clean":"2024-09-15T22:44:51.632269+0000","last_became_active":"2024-09-15T22:44:17.208338+0000","last_became_peered":"2024-09-15T22:44:17.208338+0000","last_unstale":"2024-09-15T22:44:51.632269+0000","last_undegraded":"2024-09-15T22:44:51.632269+0000","last_fullsized":"2024-09-15T22:44:51.632269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:23.250307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":89,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:45:28.656749+0000","last_change":"2024-09-15T22:44:14.679708+0000","last_active":"2024-09-15T22:45:28.656749+0000","last_peered":"2024-09-15T22:45:28.656749+0000","last_clean":"2024-09-15T22:45:28.656749+0000","last_became_active":"2024-09-15T22:44:14.679192+0000","last_became_peered":"2024-09-15T22:44:14.679192+0000","last_unstale":"2024-09-15T22:45:28.656749+0000","last_undegraded":"2024-09-15T22:45:28.656749+0000","last_fullsized":"2024-09-15T22:45:28.656749+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:25:21.192842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":59,"num_read_kb":59,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.640007+0000","last_change":"2024-09-15T22:44:17.202716+0000","last_active":"2024-09-15T22:44:51.640007+0000","last_peered":"2024-09-15T22:44:51.640007+0000","last_clean":"2024-09-15T22:44:51.640007+0000","last_became_active":"2024-09-15T22:44:17.202412+0000","last_became_peered":"2024-09-15T22:44:17.202412+0000","last_unstale":"2024-09-15T22:44:51.640007+0000","last_undegraded":"2024-09-15T22:44:51.640007+0000","last_fullsized":"2024-09-15T22:44:51.640007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:18:59.180862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.636482+0000","last_change":"2024-09-15T22:44:14.169202+0000","last_active":"2024-09-15T22:44:51.636482+0000","last_peered":"2024-09-15T22:44:51.636482+0000","last_clean":"2024-09-15T22:44:51.636482+0000","last_became_active":"2024-09-15T22:44:14.168866+0000","last_became_peered":"2024-09-15T22:44:14.168866+0000","last_unstale":"2024-09-15T22:44:51.636482+0000","last_undegraded":"2024-09-15T22:44:51.636482+0000","last_fullsized":"2024-09-15T22:44:51.636482+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:28:57.057050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.641455+0000","last_change":"2024-09-15T22:44:51.641455+0000","last_active":"2024-09-15T22:44:51.641455+0000","last_peered":"2024-09-15T22:44:51.641455+0000","last_clean":"2024-09-15T22:44:51.641455+0000","last_became_active":"2024-09-15T22:44:51.641037+0000","last_became_peered":"2024-09-15T22:44:51.641037+0000","last_unstale":"2024-09-15T22:44:51.641455+0000","last_undegraded":"2024-09-15T22:44:51.641455+0000","last_fullsized":"2024-09-15T22:44:51.641455+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:43:34.251236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.632255+0000","last_change":"2024-09-15T22:44:14.681514+0000","last_active":"2024-09-15T22:44:51.632255+0000","last_peered":"2024-09-15T22:44:51.632255+0000","last_clean":"2024-09-15T22:44:51.632255+0000","last_became_active":"2024-09-15T22:44:14.679895+0000","last_became_peered":"2024-09-15T22:44:14.679895+0000","last_unstale":"2024-09-15T22:44:51.632255+0000","last_undegraded":"2024-09-15T22:44:51.632255+0000","last_fullsized":"2024-09-15T22:44:51.632255+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:13:22.059389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.636468+0000","last_change":"2024-09-15T22:44:17.210094+0000","last_active":"2024-09-15T22:44:51.636468+0000","last_peered":"2024-09-15T22:44:51.636468+0000","last_clean":"2024-09-15T22:44:51.636468+0000","last_became_active":"2024-09-15T22:44:17.209784+0000","last_became_peered":"2024-09-15T22:44:17.209784+0000","last_unstale":"2024-09-15T22:44:51.636468+0000","last_undegraded":"2024-09-15T22:44:51.636468+0000","last_fullsized":"2024-09-15T22:44:51.636468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:02.357936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635847+0000","last_change":"2024-09-15T22:44:14.680558+0000","last_active":"2024-09-15T22:44:51.635847+0000","last_peered":"2024-09-15T22:44:51.635847+0000","last_clean":"2024-09-15T22:44:51.635847+0000","last_became_active":"2024-09-15T22:44:14.680183+0000","last_became_peered":"2024-09-15T22:44:14.680183+0000","last_unstale":"2024-09-15T22:44:51.635847+0000","last_undegraded":"2024-09-15T22:44:51.635847+0000","last_fullsized":"2024-09-15T22:44:51.635847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:44.327412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.632674+0000","last_change":"2024-09-15T22:44:17.210661+0000","last_active":"2024-09-15T22:44:51.632674+0000","last_peered":"2024-09-15T22:44:51.632674+0000","last_clean":"2024-09-15T22:44:51.632674+0000","last_became_active":"2024-09-15T22:44:17.210279+0000","last_became_peered":"2024-09-15T22:44:17.210279+0000","last_unstale":"2024-09-15T22:44:51.632674+0000","last_undegraded":"2024-09-15T22:44:51.632674+0000","last_fullsized":"2024-09-15T22:44:51.632674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:58:45.430753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'11","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.632511+0000","last_change":"2024-09-15T22:44:14.164859+0000","last_active":"2024-09-15T22:44:51.632511+0000","last_peered":"2024-09-15T22:44:51.632511+0000","last_clean":"2024-09-15T22:44:51.632511+0000","last_became_active":"2024-09-15T22:44:14.163843+0000","last_became_peered":"2024-09-15T22:44:14.163843+0000","last_unstale":"2024-09-15T22:44:51.632511+0000","last_undegraded":"2024-09-15T22:44:51.632511+0000","last_fullsized":"2024-09-15T22:44:51.632511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:01.322732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.639355+0000","last_change":"2024-09-15T22:44:17.211891+0000","last_active":"2024-09-15T22:44:51.639355+0000","last_peered":"2024-09-15T22:44:51.639355+0000","last_clean":"2024-09-15T22:44:51.639355+0000","last_became_active":"2024-09-15T22:44:17.211515+0000","last_became_peered":"2024-09-15T22:44:17.211515+0000","last_unstale":"2024-09-15T22:44:51.639355+0000","last_undegraded":"2024-09-15T22:44:51.639355+0000","last_fullsized":"2024-09-15T22:44:51.639355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:05:58.223408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.795107+0000","last_change":"2024-09-15T22:44:14.167623+0000","last_active":"2024-09-15T22:44:51.795107+0000","last_peered":"2024-09-15T22:44:51.795107+0000","last_clean":"2024-09-15T22:44:51.795107+0000","last_became_active":"2024-09-15T22:44:14.167280+0000","last_became_peered":"2024-09-15T22:44:14.167280+0000","last_unstale":"2024-09-15T22:44:51.795107+0000","last_undegraded":"2024-09-15T22:44:51.795107+0000","last_fullsized":"2024-09-15T22:44:51.795107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:00:07.609473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.640290+0000","last_change":"2024-09-15T22:44:17.215630+0000","last_active":"2024-09-15T22:44:51.640290+0000","last_peered":"2024-09-15T22:44:51.640290+0000","last_clean":"2024-09-15T22:44:51.640290+0000","last_became_active":"2024-09-15T22:44:17.215328+0000","last_became_peered":"2024-09-15T22:44:17.215328+0000","last_unstale":"2024-09-15T22:44:51.640290+0000","last_undegraded":"2024-09-15T22:44:51.640290+0000","last_fullsized":"2024-09-15T22:44:51.640290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:13:54.545954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.644603+0000","last_change":"2024-09-15T22:44:51.644603+0000","last_active":"2024-09-15T22:44:51.644603+0000","last_peered":"2024-09-15T22:44:51.644603+0000","last_clean":"2024-09-15T22:44:51.644603+0000","last_became_active":"2024-09-15T22:44:51.641126+0000","last_became_peered":"2024-09-15T22:44:51.641126+0000","last_unstale":"2024-09-15T22:44:51.644603+0000","last_undegraded":"2024-09-15T22:44:51.644603+0000","last_fullsized":"2024-09-15T22:44:51.644603+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:04:23.985487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635000+0000","last_change":"2024-09-15T22:44:17.208072+0000","last_active":"2024-09-15T22:44:51.635000+0000","last_peered":"2024-09-15T22:44:51.635000+0000","last_clean":"2024-09-15T22:44:51.635000+0000","last_became_active":"2024-09-15T22:44:17.207580+0000","last_became_peered":"2024-09-15T22:44:17.207580+0000","last_unstale":"2024-09-15T22:44:51.635000+0000","last_undegraded":"2024-09-15T22:44:51.635000+0000","last_fullsized":"2024-09-15T22:44:51.635000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:48.973412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.638567+0000","last_change":"2024-09-15T22:44:14.168841+0000","last_active":"2024-09-15T22:44:51.638567+0000","last_peered":"2024-09-15T22:44:51.638567+0000","last_clean":"2024-09-15T22:44:51.638567+0000","last_became_active":"2024-09-15T22:44:14.168359+0000","last_became_peered":"2024-09-15T22:44:14.168359+0000","last_unstale":"2024-09-15T22:44:51.638567+0000","last_undegraded":"2024-09-15T22:44:51.638567+0000","last_fullsized":"2024-09-15T22:44:51.638567+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:09:29.752679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.633044+0000","last_change":"2024-09-15T22:44:17.218227+0000","last_active":"2024-09-15T22:44:51.633044+0000","last_peered":"2024-09-15T22:44:51.633044+0000","last_clean":"2024-09-15T22:44:51.633044+0000","last_became_active":"2024-09-15T22:44:17.217846+0000","last_became_peered":"2024-09-15T22:44:17.217846+0000","last_unstale":"2024-09-15T22:44:51.633044+0000","last_undegraded":"2024-09-15T22:44:51.633044+0000","last_fullsized":"2024-09-15T22:44:51.633044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:07.601525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635631+0000","last_change":"2024-09-15T22:44:14.165322+0000","last_active":"2024-09-15T22:44:51.635631+0000","last_peered":"2024-09-15T22:44:51.635631+0000","last_clean":"2024-09-15T22:44:51.635631+0000","last_became_active":"2024-09-15T22:44:14.164981+0000","last_became_peered":"2024-09-15T22:44:14.164981+0000","last_unstale":"2024-09-15T22:44:51.635631+0000","last_undegraded":"2024-09-15T22:44:51.635631+0000","last_fullsized":"2024-09-15T22:44:51.635631+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:34.812514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.638751+0000","last_change":"2024-09-15T22:44:17.222414+0000","last_active":"2024-09-15T22:44:51.638751+0000","last_peered":"2024-09-15T22:44:51.638751+0000","last_clean":"2024-09-15T22:44:51.638751+0000","last_became_active":"2024-09-15T22:44:17.222115+0000","last_became_peered":"2024-09-15T22:44:17.222115+0000","last_unstale":"2024-09-15T22:44:51.638751+0000","last_undegraded":"2024-09-15T22:44:51.638751+0000","last_fullsized":"2024-09-15T22:44:51.638751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:22:53.791587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.644655+0000","last_change":"2024-09-15T22:44:51.644117+0000","last_active":"2024-09-15T22:44:51.644655+0000","last_peered":"2024-09-15T22:44:51.644655+0000","last_clean":"2024-09-15T22:44:51.644655+0000","last_became_active":"2024-09-15T22:44:51.638270+0000","last_became_peered":"2024-09-15T22:44:51.638270+0000","last_unstale":"2024-09-15T22:44:51.644655+0000","last_undegraded":"2024-09-15T22:44:51.644655+0000","last_fullsized":"2024-09-15T22:44:51.644655+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:01.496067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635668+0000","last_change":"2024-09-15T22:44:17.208143+0000","last_active":"2024-09-15T22:44:51.635668+0000","last_peered":"2024-09-15T22:44:51.635668+0000","last_clean":"2024-09-15T22:44:51.635668+0000","last_became_active":"2024-09-15T22:44:17.207807+0000","last_became_peered":"2024-09-15T22:44:17.207807+0000","last_unstale":"2024-09-15T22:44:51.635668+0000","last_undegraded":"2024-09-15T22:44:51.635668+0000","last_fullsized":"2024-09-15T22:44:51.635668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:19:59.752854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.638931+0000","last_change":"2024-09-15T22:44:17.201752+0000","last_active":"2024-09-15T22:44:51.638931+0000","last_peered":"2024-09-15T22:44:51.638931+0000","last_clean":"2024-09-15T22:44:51.638931+0000","last_became_active":"2024-09-15T22:44:17.201378+0000","last_became_peered":"2024-09-15T22:44:17.201378+0000","last_unstale":"2024-09-15T22:44:51.638931+0000","last_undegraded":"2024-09-15T22:44:51.638931+0000","last_fullsized":"2024-09-15T22:44:51.638931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:42:53.752795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635480+0000","last_change":"2024-09-15T22:44:14.168024+0000","last_active":"2024-09-15T22:44:51.635480+0000","last_peered":"2024-09-15T22:44:51.635480+0000","last_clean":"2024-09-15T22:44:51.635480+0000","last_became_active":"2024-09-15T22:44:14.167639+0000","last_became_peered":"2024-09-15T22:44:14.167639+0000","last_unstale":"2024-09-15T22:44:51.635480+0000","last_undegraded":"2024-09-15T22:44:51.635480+0000","last_fullsized":"2024-09-15T22:44:51.635480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:57.867895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635640+0000","last_change":"2024-09-15T22:44:17.202731+0000","last_active":"2024-09-15T22:44:51.635640+0000","last_peered":"2024-09-15T22:44:51.635640+0000","last_clean":"2024-09-15T22:44:51.635640+0000","last_became_active":"2024-09-15T22:44:17.202410+0000","last_became_peered":"2024-09-15T22:44:17.202410+0000","last_unstale":"2024-09-15T22:44:51.635640+0000","last_undegraded":"2024-09-15T22:44:51.635640+0000","last_fullsized":"2024-09-15T22:44:51.635640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:34.613686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635735+0000","last_change":"2024-09-15T22:44:14.166865+0000","last_active":"2024-09-15T22:44:51.635735+0000","last_peered":"2024-09-15T22:44:51.635735+0000","last_clean":"2024-09-15T22:44:51.635735+0000","last_became_active":"2024-09-15T22:44:14.166014+0000","last_became_peered":"2024-09-15T22:44:14.166014+0000","last_unstale":"2024-09-15T22:44:51.635735+0000","last_undegraded":"2024-09-15T22:44:51.635735+0000","last_fullsized":"2024-09-15T22:44:51.635735+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:44.527777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.640513+0000","last_change":"2024-09-15T22:44:17.221028+0000","last_active":"2024-09-15T22:44:51.640513+0000","last_peered":"2024-09-15T22:44:51.640513+0000","last_clean":"2024-09-15T22:44:51.640513+0000","last_became_active":"2024-09-15T22:44:17.220853+0000","last_became_peered":"2024-09-15T22:44:17.220853+0000","last_unstale":"2024-09-15T22:44:51.640513+0000","last_undegraded":"2024-09-15T22:44:51.640513+0000","last_fullsized":"2024-09-15T22:44:51.640513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:10:50.535946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.639205+0000","last_change":"2024-09-15T22:44:14.679506+0000","last_active":"2024-09-15T22:44:51.639205+0000","last_peered":"2024-09-15T22:44:51.639205+0000","last_clean":"2024-09-15T22:44:51.639205+0000","last_became_active":"2024-09-15T22:44:14.679164+0000","last_became_peered":"2024-09-15T22:44:14.679164+0000","last_unstale":"2024-09-15T22:44:51.639205+0000","last_undegraded":"2024-09-15T22:44:51.639205+0000","last_fullsized":"2024-09-15T22:44:51.639205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:57.786804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.639078+0000","last_change":"2024-09-15T22:44:17.203867+0000","last_active":"2024-09-15T22:44:51.639078+0000","last_peered":"2024-09-15T22:44:51.639078+0000","last_clean":"2024-09-15T22:44:51.639078+0000","last_became_active":"2024-09-15T22:44:17.203450+0000","last_became_peered":"2024-09-15T22:44:17.203450+0000","last_unstale":"2024-09-15T22:44:51.639078+0000","last_undegraded":"2024-09-15T22:44:51.639078+0000","last_fullsized":"2024-09-15T22:44:51.639078+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:17.210989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.632786+0000","last_change":"2024-09-15T22:44:14.164846+0000","last_active":"2024-09-15T22:44:51.632786+0000","last_peered":"2024-09-15T22:44:51.632786+0000","last_clean":"2024-09-15T22:44:51.632786+0000","last_became_active":"2024-09-15T22:44:14.164507+0000","last_became_peered":"2024-09-15T22:44:14.164507+0000","last_unstale":"2024-09-15T22:44:51.632786+0000","last_undegraded":"2024-09-15T22:44:51.632786+0000","last_fullsized":"2024-09-15T22:44:51.632786+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:55.234677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66313,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":382,"num_read_kb":259,"num_write":166,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1015,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":198939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449117,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":958464,"data_stored":471084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678050,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32804,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712348,"statfs":{"total":95995035648,"available":95961444352,"internally_reserved":0,"allocated":1658880,"data_stored":1124875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874281,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":978944,"data_stored":471437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135917,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":974848,"data_stored":471240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397555,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1568768,"data_stored":1063172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561079,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":962560,"data_stored":470709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790015,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1581056,"data_stored":1062856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953538,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":974848,"data_stored":472161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":470903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411405,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":533540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":532812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":983040,"data_stored":471980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":63888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:45:36.896 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T22:45:37.268 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:37.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:37 smithi111 ceph-mon[30629]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:45:37.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:37 smithi111 ceph-mon[30629]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:45:37.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:37 smithi121 ceph-mon[34815]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:45:37.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:37 smithi121 ceph-mon[34815]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:45:37.907 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.873+0000 7f10fd800640 1 -- 172.21.15.111:0/120997216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10f8106f20 msgr2=0x7f10f810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:37.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.873+0000 7f10fd800640 1 --2- 172.21.15.111:0/120997216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10f8106f20 0x7f10f810eaa0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009ca0 tx=0x7f10f002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:45:37.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.873+0000 7f10fd800640 1 -- 172.21.15.111:0/120997216 shutdown_connections 2024-09-15T22:45:37.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.873+0000 7f10fd800640 1 --2- 172.21.15.111:0/120997216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10f8106f20 0x7f10f810eaa0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:37.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.873+0000 7f10fd800640 1 --2- 172.21.15.111:0/120997216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10f81027a0 0x7f10f81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:37.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.873+0000 7f10fd800640 1 --2- 172.21.15.111:0/120997216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10f8101df0 0x7f10f81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:37.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.873+0000 7f10fd800640 1 -- 172.21.15.111:0/120997216 >> 172.21.15.111:0/120997216 conn(0x7f10f80fd8b0 msgr2=0x7f10f80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:37.908 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.873+0000 7f10fd800640 1 -- 172.21.15.111:0/120997216 shutdown_connections 2024-09-15T22:45:37.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.873+0000 7f10fd800640 1 -- 172.21.15.111:0/120997216 wait complete. 2024-09-15T22:45:37.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.874+0000 7f10fd800640 1 Processor -- start 2024-09-15T22:45:37.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.874+0000 7f10fd800640 1 -- start start 2024-09-15T22:45:37.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.875+0000 7f10fd800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10f8101df0 0x7f10f8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:37.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.875+0000 7f10fd800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10f81027a0 0x7f10f81988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:37.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.875+0000 7f10fd800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10f8106f20 0x7f10f819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:37.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.875+0000 7f10fd800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f10f8111f80 con 0x7f10f8101df0 2024-09-15T22:45:37.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.875+0000 7f10fd800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f10f8111e00 con 0x7f10f81027a0 2024-09-15T22:45:37.909 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.875+0000 7f10fd800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f10f8112100 con 0x7f10f8106f20 2024-09-15T22:45:37.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.875+0000 7f10f6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10f8101df0 0x7f10f8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:37.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.875+0000 7f10f6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10f8101df0 0x7f10f8198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:49176/0 (socket says 172.21.15.111:49176) 2024-09-15T22:45:37.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.875+0000 7f10f6a00640 1 -- 172.21.15.111:0/3441032065 learned_addr learned my addr 172.21.15.111:0/3441032065 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:37.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.875+0000 7f10f6000640 1 --2- 172.21.15.111:0/3441032065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10f81027a0 0x7f10f81988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:37.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.875+0000 7f10f7400640 1 --2- 172.21.15.111:0/3441032065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10f8106f20 0x7f10f819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:37.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.875+0000 7f10f6000640 1 -- 172.21.15.111:0/3441032065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10f8106f20 msgr2=0x7f10f819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:37.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.875+0000 7f10f6000640 1 --2- 172.21.15.111:0/3441032065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10f8106f20 0x7f10f819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:37.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.875+0000 7f10f6000640 1 -- 172.21.15.111:0/3441032065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10f8101df0 msgr2=0x7f10f8198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:37.910 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.875+0000 7f10f6000640 1 --2- 172.21.15.111:0/3441032065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10f8101df0 0x7f10f8198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:37.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.875+0000 7f10f6000640 1 -- 172.21.15.111:0/3441032065 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f819d330 con 0x7f10f81027a0 2024-09-15T22:45:37.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.875+0000 7f10f6a00640 1 --2- 172.21.15.111:0/3441032065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10f8101df0 0x7f10f8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:37.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.876+0000 7f10f6000640 1 --2- 172.21.15.111:0/3441032065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10f81027a0 0x7f10f81988c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f10e400e9f0 tx=0x7f10e400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:37.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.876+0000 7f10ef400640 1 -- 172.21.15.111:0/3441032065 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e400ccc0 con 0x7f10f81027a0 2024-09-15T22:45:37.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.876+0000 7f10ef400640 1 -- 172.21.15.111:0/3441032065 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10e4010910 con 0x7f10f81027a0 2024-09-15T22:45:37.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.876+0000 7f10ef400640 1 -- 172.21.15.111:0/3441032065 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e4018960 con 0x7f10f81027a0 2024-09-15T22:45:37.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.876+0000 7f10fd800640 1 -- 172.21.15.111:0/3441032065 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10f819d620 con 0x7f10f81027a0 2024-09-15T22:45:37.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.876+0000 7f10fd800640 1 -- 172.21.15.111:0/3441032065 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10f81a4f00 con 0x7f10f81027a0 2024-09-15T22:45:37.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.877+0000 7f10fd800640 1 -- 172.21.15.111:0/3441032065 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10f81021d0 con 0x7f10f81027a0 2024-09-15T22:45:37.911 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.879+0000 7f10ef400640 1 -- 172.21.15.111:0/3441032065 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f10e4010ac0 con 0x7f10f81027a0 2024-09-15T22:45:37.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.881+0000 7f10ef400640 1 --2- 172.21.15.111:0/3441032065 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f10dc0a3350 0x7f10dc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:37.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.881+0000 7f10ef400640 1 -- 172.21.15.111:0/3441032065 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f10e40cfd50 con 0x7f10f81027a0 2024-09-15T22:45:37.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.881+0000 7f10f6a00640 1 --2- 172.21.15.111:0/3441032065 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f10dc0a3350 0x7f10dc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:37.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.881+0000 7f10f6a00640 1 --2- 172.21.15.111:0/3441032065 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f10dc0a3350 0x7f10dc0a5810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f10e8011550 tx=0x7f10e8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:37.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:37.884+0000 7f10ef400640 1 -- 172.21.15.111:0/3441032065 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f10e409c2c0 con 0x7f10f81027a0 2024-09-15T22:45:38.059 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:38.057+0000 7f10fd800640 1 -- 172.21.15.111:0/3441032065 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f10f80febc0 con 0x7f10dc0a3350 2024-09-15T22:45:38.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:37 smithi192 ceph-mon[35427]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:45:38.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:37 smithi192 ceph-mon[35427]: pgmap v341: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:45:38.065 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:38.064+0000 7f10ef400640 1 -- 172.21.15.111:0/3441032065 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258730 (secure 0 0 0) 0x7f10f80febc0 con 0x7f10dc0a3350 2024-09-15T22:45:38.067 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:45:38.068 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2024-09-15T22:45:38.070 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:38.067+0000 7f10fd800640 1 -- 172.21.15.111:0/3441032065 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f10dc0a3350 msgr2=0x7f10dc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:38.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:38.067+0000 7f10fd800640 1 --2- 172.21.15.111:0/3441032065 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f10dc0a3350 0x7f10dc0a5810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f10e8011550 tx=0x7f10e8005f90 comp rx=0 tx=0).stop 2024-09-15T22:45:38.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:38.067+0000 7f10fd800640 1 -- 172.21.15.111:0/3441032065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10f81027a0 msgr2=0x7f10f81988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:38.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:38.067+0000 7f10fd800640 1 --2- 172.21.15.111:0/3441032065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10f81027a0 0x7f10f81988c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f10e400e9f0 tx=0x7f10e400eeb0 comp rx=0 tx=0).stop 2024-09-15T22:45:38.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:38.067+0000 7f10fd800640 1 -- 172.21.15.111:0/3441032065 shutdown_connections 2024-09-15T22:45:38.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:38.067+0000 7f10fd800640 1 --2- 172.21.15.111:0/3441032065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10f8106f20 0x7f10f819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:38.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:38.067+0000 7f10fd800640 1 --2- 172.21.15.111:0/3441032065 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f10dc0a3350 0x7f10dc0a5810 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:38.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:38.067+0000 7f10fd800640 1 --2- 172.21.15.111:0/3441032065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10f81027a0 0x7f10f81988c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:38.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:38.067+0000 7f10fd800640 1 --2- 172.21.15.111:0/3441032065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10f8101df0 0x7f10f8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:38.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:38.067+0000 7f10fd800640 1 -- 172.21.15.111:0/3441032065 >> 172.21.15.111:0/3441032065 conn(0x7f10f80fd8b0 msgr2=0x7f10f80ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:38.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:38.068+0000 7f10fd800640 1 -- 172.21.15.111:0/3441032065 shutdown_connections 2024-09-15T22:45:38.071 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:38.068+0000 7f10fd800640 1 -- 172.21.15.111:0/3441032065 wait complete. 2024-09-15T22:45:38.814 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-09-15T22:45:36.473815+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656915,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":528,"num_read_kb":383,"num_write":349,"num_write_kb":3054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181868,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375160,"kb_used_data":13368,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566664,"statfs":{"total":1151940427776,"available":1151556263936,"internally_reserved":0,"allocated":13688832,"data_stored":7616769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004976"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.633452+0000","last_change":"2024-09-15T22:44:17.222326+0000","last_active":"2024-09-15T22:44:51.633452+0000","last_peered":"2024-09-15T22:44:51.633452+0000","last_clean":"2024-09-15T22:44:51.633452+0000","last_became_active":"2024-09-15T22:44:17.221903+0000","last_became_peered":"2024-09-15T22:44:17.221903+0000","last_unstale":"2024-09-15T22:44:51.633452+0000","last_undegraded":"2024-09-15T22:44:51.633452+0000","last_fullsized":"2024-09-15T22:44:51.633452+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:28.303194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.640206+0000","last_change":"2024-09-15T22:44:17.220515+0000","last_active":"2024-09-15T22:44:51.640206+0000","last_peered":"2024-09-15T22:44:51.640206+0000","last_clean":"2024-09-15T22:44:51.640206+0000","last_became_active":"2024-09-15T22:44:17.218794+0000","last_became_peered":"2024-09-15T22:44:17.218794+0000","last_unstale":"2024-09-15T22:44:51.640206+0000","last_undegraded":"2024-09-15T22:44:51.640206+0000","last_fullsized":"2024-09-15T22:44:51.640206+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:58:17.892809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.640270+0000","last_change":"2024-09-15T22:44:17.213183+0000","last_active":"2024-09-15T22:44:51.640270+0000","last_peered":"2024-09-15T22:44:51.640270+0000","last_clean":"2024-09-15T22:44:51.640270+0000","last_became_active":"2024-09-15T22:44:17.212888+0000","last_became_peered":"2024-09-15T22:44:17.212888+0000","last_unstale":"2024-09-15T22:44:51.640270+0000","last_undegraded":"2024-09-15T22:44:51.640270+0000","last_fullsized":"2024-09-15T22:44:51.640270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:55:32.224099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.795036+0000","last_change":"2024-09-15T22:44:17.203869+0000","last_active":"2024-09-15T22:44:51.795036+0000","last_peered":"2024-09-15T22:44:51.795036+0000","last_clean":"2024-09-15T22:44:51.795036+0000","last_became_active":"2024-09-15T22:44:17.203529+0000","last_became_peered":"2024-09-15T22:44:17.203529+0000","last_unstale":"2024-09-15T22:44:51.795036+0000","last_undegraded":"2024-09-15T22:44:51.795036+0000","last_fullsized":"2024-09-15T22:44:51.795036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:22:58.539513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635166+0000","last_change":"2024-09-15T22:44:17.209521+0000","last_active":"2024-09-15T22:44:51.635166+0000","last_peered":"2024-09-15T22:44:51.635166+0000","last_clean":"2024-09-15T22:44:51.635166+0000","last_became_active":"2024-09-15T22:44:17.209203+0000","last_became_peered":"2024-09-15T22:44:17.209203+0000","last_unstale":"2024-09-15T22:44:51.635166+0000","last_undegraded":"2024-09-15T22:44:51.635166+0000","last_fullsized":"2024-09-15T22:44:51.635166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:34:28.418483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635469+0000","last_change":"2024-09-15T22:44:17.202857+0000","last_active":"2024-09-15T22:44:51.635469+0000","last_peered":"2024-09-15T22:44:51.635469+0000","last_clean":"2024-09-15T22:44:51.635469+0000","last_became_active":"2024-09-15T22:44:17.202530+0000","last_became_peered":"2024-09-15T22:44:17.202530+0000","last_unstale":"2024-09-15T22:44:51.635469+0000","last_undegraded":"2024-09-15T22:44:51.635469+0000","last_fullsized":"2024-09-15T22:44:51.635469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:16:06.947769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635314+0000","last_change":"2024-09-15T22:44:17.203317+0000","last_active":"2024-09-15T22:44:51.635314+0000","last_peered":"2024-09-15T22:44:51.635314+0000","last_clean":"2024-09-15T22:44:51.635314+0000","last_became_active":"2024-09-15T22:44:17.202975+0000","last_became_peered":"2024-09-15T22:44:17.202975+0000","last_unstale":"2024-09-15T22:44:51.635314+0000","last_undegraded":"2024-09-15T22:44:51.635314+0000","last_fullsized":"2024-09-15T22:44:51.635314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:51.292052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.638186+0000","last_change":"2024-09-15T22:44:17.203583+0000","last_active":"2024-09-15T22:44:51.638186+0000","last_peered":"2024-09-15T22:44:51.638186+0000","last_clean":"2024-09-15T22:44:51.638186+0000","last_became_active":"2024-09-15T22:44:17.203251+0000","last_became_peered":"2024-09-15T22:44:17.203251+0000","last_unstale":"2024-09-15T22:44:51.638186+0000","last_undegraded":"2024-09-15T22:44:51.638186+0000","last_fullsized":"2024-09-15T22:44:51.638186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:53:17.378024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.636870+0000","last_change":"2024-09-15T22:44:17.209643+0000","last_active":"2024-09-15T22:44:51.636870+0000","last_peered":"2024-09-15T22:44:51.636870+0000","last_clean":"2024-09-15T22:44:51.636870+0000","last_became_active":"2024-09-15T22:44:17.208709+0000","last_became_peered":"2024-09-15T22:44:17.208709+0000","last_unstale":"2024-09-15T22:44:51.636870+0000","last_undegraded":"2024-09-15T22:44:51.636870+0000","last_fullsized":"2024-09-15T22:44:51.636870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:38:38.305676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.639787+0000","last_change":"2024-09-15T22:44:51.639787+0000","last_active":"2024-09-15T22:44:51.639787+0000","last_peered":"2024-09-15T22:44:51.639787+0000","last_clean":"2024-09-15T22:44:51.639787+0000","last_became_active":"2024-09-15T22:44:51.639300+0000","last_became_peered":"2024-09-15T22:44:51.639300+0000","last_unstale":"2024-09-15T22:44:51.639787+0000","last_undegraded":"2024-09-15T22:44:51.639787+0000","last_fullsized":"2024-09-15T22:44:51.639787+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:18:52.775861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635522+0000","last_change":"2024-09-15T22:44:17.208106+0000","last_active":"2024-09-15T22:44:51.635522+0000","last_peered":"2024-09-15T22:44:51.635522+0000","last_clean":"2024-09-15T22:44:51.635522+0000","last_became_active":"2024-09-15T22:44:17.207774+0000","last_became_peered":"2024-09-15T22:44:17.207774+0000","last_unstale":"2024-09-15T22:44:51.635522+0000","last_undegraded":"2024-09-15T22:44:51.635522+0000","last_fullsized":"2024-09-15T22:44:51.635522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:05:15.876692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.795040+0000","last_change":"2024-09-15T22:44:17.202857+0000","last_active":"2024-09-15T22:44:51.795040+0000","last_peered":"2024-09-15T22:44:51.795040+0000","last_clean":"2024-09-15T22:44:51.795040+0000","last_became_active":"2024-09-15T22:44:17.202520+0000","last_became_peered":"2024-09-15T22:44:17.202520+0000","last_unstale":"2024-09-15T22:44:51.795040+0000","last_undegraded":"2024-09-15T22:44:51.795040+0000","last_fullsized":"2024-09-15T22:44:51.795040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:11:19.111086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.632664+0000","last_change":"2024-09-15T22:44:17.201776+0000","last_active":"2024-09-15T22:44:51.632664+0000","last_peered":"2024-09-15T22:44:51.632664+0000","last_clean":"2024-09-15T22:44:51.632664+0000","last_became_active":"2024-09-15T22:44:17.201508+0000","last_became_peered":"2024-09-15T22:44:17.201508+0000","last_unstale":"2024-09-15T22:44:51.632664+0000","last_undegraded":"2024-09-15T22:44:51.632664+0000","last_fullsized":"2024-09-15T22:44:51.632664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:22:48.104776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.639416+0000","last_change":"2024-09-15T22:44:17.221005+0000","last_active":"2024-09-15T22:44:51.639416+0000","last_peered":"2024-09-15T22:44:51.639416+0000","last_clean":"2024-09-15T22:44:51.639416+0000","last_became_active":"2024-09-15T22:44:17.218455+0000","last_became_peered":"2024-09-15T22:44:17.218455+0000","last_unstale":"2024-09-15T22:44:51.639416+0000","last_undegraded":"2024-09-15T22:44:51.639416+0000","last_fullsized":"2024-09-15T22:44:51.639416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:40.227843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.639776+0000","last_change":"2024-09-15T22:44:17.208125+0000","last_active":"2024-09-15T22:44:51.639776+0000","last_peered":"2024-09-15T22:44:51.639776+0000","last_clean":"2024-09-15T22:44:51.639776+0000","last_became_active":"2024-09-15T22:44:17.207762+0000","last_became_peered":"2024-09-15T22:44:17.207762+0000","last_unstale":"2024-09-15T22:44:51.639776+0000","last_undegraded":"2024-09-15T22:44:51.639776+0000","last_fullsized":"2024-09-15T22:44:51.639776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:21:45.181921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635521+0000","last_change":"2024-09-15T22:44:17.212058+0000","last_active":"2024-09-15T22:44:51.635521+0000","last_peered":"2024-09-15T22:44:51.635521+0000","last_clean":"2024-09-15T22:44:51.635521+0000","last_became_active":"2024-09-15T22:44:17.211789+0000","last_became_peered":"2024-09-15T22:44:17.211789+0000","last_unstale":"2024-09-15T22:44:51.635521+0000","last_undegraded":"2024-09-15T22:44:51.635521+0000","last_fullsized":"2024-09-15T22:44:51.635521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:52:14.647374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.633547+0000","last_change":"2024-09-15T22:44:17.225139+0000","last_active":"2024-09-15T22:44:51.633547+0000","last_peered":"2024-09-15T22:44:51.633547+0000","last_clean":"2024-09-15T22:44:51.633547+0000","last_became_active":"2024-09-15T22:44:17.224791+0000","last_became_peered":"2024-09-15T22:44:17.224791+0000","last_unstale":"2024-09-15T22:44:51.633547+0000","last_undegraded":"2024-09-15T22:44:51.633547+0000","last_fullsized":"2024-09-15T22:44:51.633547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:30:56.457155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.632502+0000","last_change":"2024-09-15T22:44:17.208604+0000","last_active":"2024-09-15T22:44:51.632502+0000","last_peered":"2024-09-15T22:44:51.632502+0000","last_clean":"2024-09-15T22:44:51.632502+0000","last_became_active":"2024-09-15T22:44:17.202231+0000","last_became_peered":"2024-09-15T22:44:17.202231+0000","last_unstale":"2024-09-15T22:44:51.632502+0000","last_undegraded":"2024-09-15T22:44:51.632502+0000","last_fullsized":"2024-09-15T22:44:51.632502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:39.572881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635576+0000","last_change":"2024-09-15T22:44:17.211837+0000","last_active":"2024-09-15T22:44:51.635576+0000","last_peered":"2024-09-15T22:44:51.635576+0000","last_clean":"2024-09-15T22:44:51.635576+0000","last_became_active":"2024-09-15T22:44:17.211460+0000","last_became_peered":"2024-09-15T22:44:17.211460+0000","last_unstale":"2024-09-15T22:44:51.635576+0000","last_undegraded":"2024-09-15T22:44:51.635576+0000","last_fullsized":"2024-09-15T22:44:51.635576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:21:51.820134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.633047+0000","last_change":"2024-09-15T22:44:17.206350+0000","last_active":"2024-09-15T22:44:51.633047+0000","last_peered":"2024-09-15T22:44:51.633047+0000","last_clean":"2024-09-15T22:44:51.633047+0000","last_became_active":"2024-09-15T22:44:17.204943+0000","last_became_peered":"2024-09-15T22:44:17.204943+0000","last_unstale":"2024-09-15T22:44:51.633047+0000","last_undegraded":"2024-09-15T22:44:51.633047+0000","last_fullsized":"2024-09-15T22:44:51.633047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:42:59.918220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.632678+0000","last_change":"2024-09-15T22:44:17.223390+0000","last_active":"2024-09-15T22:44:51.632678+0000","last_peered":"2024-09-15T22:44:51.632678+0000","last_clean":"2024-09-15T22:44:51.632678+0000","last_became_active":"2024-09-15T22:44:17.223008+0000","last_became_peered":"2024-09-15T22:44:17.223008+0000","last_unstale":"2024-09-15T22:44:51.632678+0000","last_undegraded":"2024-09-15T22:44:51.632678+0000","last_fullsized":"2024-09-15T22:44:51.632678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:31.545336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.636324+0000","last_change":"2024-09-15T22:44:17.207567+0000","last_active":"2024-09-15T22:44:51.636324+0000","last_peered":"2024-09-15T22:44:51.636324+0000","last_clean":"2024-09-15T22:44:51.636324+0000","last_became_active":"2024-09-15T22:44:17.207245+0000","last_became_peered":"2024-09-15T22:44:17.207245+0000","last_unstale":"2024-09-15T22:44:51.636324+0000","last_undegraded":"2024-09-15T22:44:51.636324+0000","last_fullsized":"2024-09-15T22:44:51.636324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:35.169205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.638414+0000","last_change":"2024-09-15T22:44:17.209449+0000","last_active":"2024-09-15T22:44:51.638414+0000","last_peered":"2024-09-15T22:44:51.638414+0000","last_clean":"2024-09-15T22:44:51.638414+0000","last_became_active":"2024-09-15T22:44:17.209012+0000","last_became_peered":"2024-09-15T22:44:17.209012+0000","last_unstale":"2024-09-15T22:44:51.638414+0000","last_undegraded":"2024-09-15T22:44:51.638414+0000","last_fullsized":"2024-09-15T22:44:51.638414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:59.550776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.640005+0000","last_change":"2024-09-15T22:44:17.214184+0000","last_active":"2024-09-15T22:44:51.640005+0000","last_peered":"2024-09-15T22:44:51.640005+0000","last_clean":"2024-09-15T22:44:51.640005+0000","last_became_active":"2024-09-15T22:44:17.213884+0000","last_became_peered":"2024-09-15T22:44:17.213884+0000","last_unstale":"2024-09-15T22:44:51.640005+0000","last_undegraded":"2024-09-15T22:44:51.640005+0000","last_fullsized":"2024-09-15T22:44:51.640005+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:33.336473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.639279+0000","last_change":"2024-09-15T22:44:17.203904+0000","last_active":"2024-09-15T22:44:51.639279+0000","last_peered":"2024-09-15T22:44:51.639279+0000","last_clean":"2024-09-15T22:44:51.639279+0000","last_became_active":"2024-09-15T22:44:17.203484+0000","last_became_peered":"2024-09-15T22:44:17.203484+0000","last_unstale":"2024-09-15T22:44:51.639279+0000","last_undegraded":"2024-09-15T22:44:51.639279+0000","last_fullsized":"2024-09-15T22:44:51.639279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:33:09.906869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.636131+0000","last_change":"2024-09-15T22:44:17.209356+0000","last_active":"2024-09-15T22:44:51.636131+0000","last_peered":"2024-09-15T22:44:51.636131+0000","last_clean":"2024-09-15T22:44:51.636131+0000","last_became_active":"2024-09-15T22:44:17.207229+0000","last_became_peered":"2024-09-15T22:44:17.207229+0000","last_unstale":"2024-09-15T22:44:51.636131+0000","last_undegraded":"2024-09-15T22:44:51.636131+0000","last_fullsized":"2024-09-15T22:44:51.636131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:32:28.405429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.633207+0000","last_change":"2024-09-15T22:44:17.202505+0000","last_active":"2024-09-15T22:44:51.633207+0000","last_peered":"2024-09-15T22:44:51.633207+0000","last_clean":"2024-09-15T22:44:51.633207+0000","last_became_active":"2024-09-15T22:44:17.202154+0000","last_became_peered":"2024-09-15T22:44:17.202154+0000","last_unstale":"2024-09-15T22:44:51.633207+0000","last_undegraded":"2024-09-15T22:44:51.633207+0000","last_fullsized":"2024-09-15T22:44:51.633207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:40:05.680982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.632632+0000","last_change":"2024-09-15T22:44:17.222767+0000","last_active":"2024-09-15T22:44:51.632632+0000","last_peered":"2024-09-15T22:44:51.632632+0000","last_clean":"2024-09-15T22:44:51.632632+0000","last_became_active":"2024-09-15T22:44:17.221655+0000","last_became_peered":"2024-09-15T22:44:17.221655+0000","last_unstale":"2024-09-15T22:44:51.632632+0000","last_undegraded":"2024-09-15T22:44:51.632632+0000","last_fullsized":"2024-09-15T22:44:51.632632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:22.457576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.639825+0000","last_change":"2024-09-15T22:44:17.217485+0000","last_active":"2024-09-15T22:44:51.639825+0000","last_peered":"2024-09-15T22:44:51.639825+0000","last_clean":"2024-09-15T22:44:51.639825+0000","last_became_active":"2024-09-15T22:44:17.217138+0000","last_became_peered":"2024-09-15T22:44:17.217138+0000","last_unstale":"2024-09-15T22:44:51.639825+0000","last_undegraded":"2024-09-15T22:44:51.639825+0000","last_fullsized":"2024-09-15T22:44:51.639825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:41:40.606593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635434+0000","last_change":"2024-09-15T22:44:17.217996+0000","last_active":"2024-09-15T22:44:51.635434+0000","last_peered":"2024-09-15T22:44:51.635434+0000","last_clean":"2024-09-15T22:44:51.635434+0000","last_became_active":"2024-09-15T22:44:17.217467+0000","last_became_peered":"2024-09-15T22:44:17.217467+0000","last_unstale":"2024-09-15T22:44:51.635434+0000","last_undegraded":"2024-09-15T22:44:51.635434+0000","last_fullsized":"2024-09-15T22:44:51.635434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:25:45.150979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.636279+0000","last_change":"2024-09-15T22:44:17.202840+0000","last_active":"2024-09-15T22:44:51.636279+0000","last_peered":"2024-09-15T22:44:51.636279+0000","last_clean":"2024-09-15T22:44:51.636279+0000","last_became_active":"2024-09-15T22:44:17.202481+0000","last_became_peered":"2024-09-15T22:44:17.202481+0000","last_unstale":"2024-09-15T22:44:51.636279+0000","last_undegraded":"2024-09-15T22:44:51.636279+0000","last_fullsized":"2024-09-15T22:44:51.636279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:31:09.360211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.632973+0000","last_change":"2024-09-15T22:44:17.209387+0000","last_active":"2024-09-15T22:44:51.632973+0000","last_peered":"2024-09-15T22:44:51.632973+0000","last_clean":"2024-09-15T22:44:51.632973+0000","last_became_active":"2024-09-15T22:44:17.209100+0000","last_became_peered":"2024-09-15T22:44:17.209100+0000","last_unstale":"2024-09-15T22:44:51.632973+0000","last_undegraded":"2024-09-15T22:44:51.632973+0000","last_fullsized":"2024-09-15T22:44:51.632973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:57:47.942784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635137+0000","last_change":"2024-09-15T22:44:17.203626+0000","last_active":"2024-09-15T22:44:51.635137+0000","last_peered":"2024-09-15T22:44:51.635137+0000","last_clean":"2024-09-15T22:44:51.635137+0000","last_became_active":"2024-09-15T22:44:17.203313+0000","last_became_peered":"2024-09-15T22:44:17.203313+0000","last_unstale":"2024-09-15T22:44:51.635137+0000","last_undegraded":"2024-09-15T22:44:51.635137+0000","last_fullsized":"2024-09-15T22:44:51.635137+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:42:53.765397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":109,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:45:31.859623+0000","last_change":"2024-09-15T22:44:51.800707+0000","last_active":"2024-09-15T22:45:31.859623+0000","last_peered":"2024-09-15T22:45:31.859623+0000","last_clean":"2024-09-15T22:45:31.859623+0000","last_became_active":"2024-09-15T22:44:51.796445+0000","last_became_peered":"2024-09-15T22:44:51.796445+0000","last_unstale":"2024-09-15T22:45:31.859623+0000","last_undegraded":"2024-09-15T22:45:31.859623+0000","last_fullsized":"2024-09-15T22:45:31.859623+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:13:09.444747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":60,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.638512+0000","last_change":"2024-09-15T22:44:17.210054+0000","last_active":"2024-09-15T22:44:51.638512+0000","last_peered":"2024-09-15T22:44:51.638512+0000","last_clean":"2024-09-15T22:44:51.638512+0000","last_became_active":"2024-09-15T22:44:17.209607+0000","last_became_peered":"2024-09-15T22:44:17.209607+0000","last_unstale":"2024-09-15T22:44:51.638512+0000","last_undegraded":"2024-09-15T22:44:51.638512+0000","last_fullsized":"2024-09-15T22:44:51.638512+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:13.867965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.636335+0000","last_change":"2024-09-15T22:44:14.167537+0000","last_active":"2024-09-15T22:44:51.636335+0000","last_peered":"2024-09-15T22:44:51.636335+0000","last_clean":"2024-09-15T22:44:51.636335+0000","last_became_active":"2024-09-15T22:44:14.167173+0000","last_became_peered":"2024-09-15T22:44:14.167173+0000","last_unstale":"2024-09-15T22:44:51.636335+0000","last_undegraded":"2024-09-15T22:44:51.636335+0000","last_fullsized":"2024-09-15T22:44:51.636335+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:45:56.539218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.632329+0000","last_change":"2024-09-15T22:44:14.166676+0000","last_active":"2024-09-15T22:44:51.632329+0000","last_peered":"2024-09-15T22:44:51.632329+0000","last_clean":"2024-09-15T22:44:51.632329+0000","last_became_active":"2024-09-15T22:44:14.166342+0000","last_became_peered":"2024-09-15T22:44:14.166342+0000","last_unstale":"2024-09-15T22:44:51.632329+0000","last_undegraded":"2024-09-15T22:44:51.632329+0000","last_fullsized":"2024-09-15T22:44:51.632329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:41.861253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.638293+0000","last_change":"2024-09-15T22:44:17.211312+0000","last_active":"2024-09-15T22:44:51.638293+0000","last_peered":"2024-09-15T22:44:51.638293+0000","last_clean":"2024-09-15T22:44:51.638293+0000","last_became_active":"2024-09-15T22:44:17.208771+0000","last_became_peered":"2024-09-15T22:44:17.208771+0000","last_unstale":"2024-09-15T22:44:51.638293+0000","last_undegraded":"2024-09-15T22:44:51.638293+0000","last_fullsized":"2024-09-15T22:44:51.638293+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:59:51.250489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.640372+0000","last_change":"2024-09-15T22:44:14.167263+0000","last_active":"2024-09-15T22:44:51.640372+0000","last_peered":"2024-09-15T22:44:51.640372+0000","last_clean":"2024-09-15T22:44:51.640372+0000","last_became_active":"2024-09-15T22:44:14.166942+0000","last_became_peered":"2024-09-15T22:44:14.166942+0000","last_unstale":"2024-09-15T22:44:51.640372+0000","last_undegraded":"2024-09-15T22:44:51.640372+0000","last_fullsized":"2024-09-15T22:44:51.640372+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:47.320374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.631899+0000","last_change":"2024-09-15T22:44:17.210986+0000","last_active":"2024-09-15T22:44:51.631899+0000","last_peered":"2024-09-15T22:44:51.631899+0000","last_clean":"2024-09-15T22:44:51.631899+0000","last_became_active":"2024-09-15T22:44:17.210618+0000","last_became_peered":"2024-09-15T22:44:17.210618+0000","last_unstale":"2024-09-15T22:44:51.631899+0000","last_undegraded":"2024-09-15T22:44:51.631899+0000","last_fullsized":"2024-09-15T22:44:51.631899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:58:04.961630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:45:33.861514+0000","last_change":"2024-09-15T22:44:14.164770+0000","last_active":"2024-09-15T22:45:33.861514+0000","last_peered":"2024-09-15T22:45:33.861514+0000","last_clean":"2024-09-15T22:45:33.861514+0000","last_became_active":"2024-09-15T22:44:14.164411+0000","last_became_peered":"2024-09-15T22:44:14.164411+0000","last_unstale":"2024-09-15T22:45:33.861514+0000","last_undegraded":"2024-09-15T22:45:33.861514+0000","last_fullsized":"2024-09-15T22:45:33.861514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:26:47.623520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":64,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.636419+0000","last_change":"2024-09-15T22:44:17.204049+0000","last_active":"2024-09-15T22:44:51.636419+0000","last_peered":"2024-09-15T22:44:51.636419+0000","last_clean":"2024-09-15T22:44:51.636419+0000","last_became_active":"2024-09-15T22:44:17.203738+0000","last_became_peered":"2024-09-15T22:44:17.203738+0000","last_unstale":"2024-09-15T22:44:51.636419+0000","last_undegraded":"2024-09-15T22:44:51.636419+0000","last_fullsized":"2024-09-15T22:44:51.636419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:00:56.347792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635667+0000","last_change":"2024-09-15T22:44:14.167228+0000","last_active":"2024-09-15T22:44:51.635667+0000","last_peered":"2024-09-15T22:44:51.635667+0000","last_clean":"2024-09-15T22:44:51.635667+0000","last_became_active":"2024-09-15T22:44:14.166662+0000","last_became_peered":"2024-09-15T22:44:14.166662+0000","last_unstale":"2024-09-15T22:44:51.635667+0000","last_undegraded":"2024-09-15T22:44:51.635667+0000","last_fullsized":"2024-09-15T22:44:51.635667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:31:17.160718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.640531+0000","last_change":"2024-09-15T22:44:17.212549+0000","last_active":"2024-09-15T22:44:51.640531+0000","last_peered":"2024-09-15T22:44:51.640531+0000","last_clean":"2024-09-15T22:44:51.640531+0000","last_became_active":"2024-09-15T22:44:17.212123+0000","last_became_peered":"2024-09-15T22:44:17.212123+0000","last_unstale":"2024-09-15T22:44:51.640531+0000","last_undegraded":"2024-09-15T22:44:51.640531+0000","last_fullsized":"2024-09-15T22:44:51.640531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:32:45.241199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.640041+0000","last_change":"2024-09-15T22:44:14.679664+0000","last_active":"2024-09-15T22:44:51.640041+0000","last_peered":"2024-09-15T22:44:51.640041+0000","last_clean":"2024-09-15T22:44:51.640041+0000","last_became_active":"2024-09-15T22:44:14.679256+0000","last_became_peered":"2024-09-15T22:44:14.679256+0000","last_unstale":"2024-09-15T22:44:51.640041+0000","last_undegraded":"2024-09-15T22:44:51.640041+0000","last_fullsized":"2024-09-15T22:44:51.640041+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:33:41.512488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635773+0000","last_change":"2024-09-15T22:44:17.200448+0000","last_active":"2024-09-15T22:44:51.635773+0000","last_peered":"2024-09-15T22:44:51.635773+0000","last_clean":"2024-09-15T22:44:51.635773+0000","last_became_active":"2024-09-15T22:44:17.200114+0000","last_became_peered":"2024-09-15T22:44:17.200114+0000","last_unstale":"2024-09-15T22:44:51.635773+0000","last_undegraded":"2024-09-15T22:44:51.635773+0000","last_fullsized":"2024-09-15T22:44:51.635773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:36.403644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"88'31","reported_seq":195,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:45:34.862924+0000","last_change":"2024-09-15T22:44:14.167067+0000","last_active":"2024-09-15T22:45:34.862924+0000","last_peered":"2024-09-15T22:45:34.862924+0000","last_clean":"2024-09-15T22:45:34.862924+0000","last_became_active":"2024-09-15T22:44:14.166613+0000","last_became_peered":"2024-09-15T22:44:14.166613+0000","last_unstale":"2024-09-15T22:45:34.862924+0000","last_undegraded":"2024-09-15T22:45:34.862924+0000","last_fullsized":"2024-09-15T22:45:34.862924+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:35:15.360717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62310,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":138,"num_read_kb":129,"num_write":32,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635625+0000","last_change":"2024-09-15T22:44:17.202705+0000","last_active":"2024-09-15T22:44:51.635625+0000","last_peered":"2024-09-15T22:44:51.635625+0000","last_clean":"2024-09-15T22:44:51.635625+0000","last_became_active":"2024-09-15T22:44:17.202346+0000","last_became_peered":"2024-09-15T22:44:17.202346+0000","last_unstale":"2024-09-15T22:44:51.635625+0000","last_undegraded":"2024-09-15T22:44:51.635625+0000","last_fullsized":"2024-09-15T22:44:51.635625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:35:31.193746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.637052+0000","last_change":"2024-09-15T22:44:14.169255+0000","last_active":"2024-09-15T22:44:51.637052+0000","last_peered":"2024-09-15T22:44:51.637052+0000","last_clean":"2024-09-15T22:44:51.637052+0000","last_became_active":"2024-09-15T22:44:14.168774+0000","last_became_peered":"2024-09-15T22:44:14.168774+0000","last_unstale":"2024-09-15T22:44:51.637052+0000","last_undegraded":"2024-09-15T22:44:51.637052+0000","last_fullsized":"2024-09-15T22:44:51.637052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:48.858831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.632337+0000","last_change":"2024-09-15T22:44:17.218635+0000","last_active":"2024-09-15T22:44:51.632337+0000","last_peered":"2024-09-15T22:44:51.632337+0000","last_clean":"2024-09-15T22:44:51.632337+0000","last_became_active":"2024-09-15T22:44:17.218287+0000","last_became_peered":"2024-09-15T22:44:17.218287+0000","last_unstale":"2024-09-15T22:44:51.632337+0000","last_undegraded":"2024-09-15T22:44:51.632337+0000","last_fullsized":"2024-09-15T22:44:51.632337+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:41:58.264382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.639773+0000","last_change":"2024-09-15T22:44:14.166580+0000","last_active":"2024-09-15T22:44:51.639773+0000","last_peered":"2024-09-15T22:44:51.639773+0000","last_clean":"2024-09-15T22:44:51.639773+0000","last_became_active":"2024-09-15T22:44:14.164955+0000","last_became_peered":"2024-09-15T22:44:14.164955+0000","last_unstale":"2024-09-15T22:44:51.639773+0000","last_undegraded":"2024-09-15T22:44:51.639773+0000","last_fullsized":"2024-09-15T22:44:51.639773+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:16:14.215662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635544+0000","last_change":"2024-09-15T22:44:17.205731+0000","last_active":"2024-09-15T22:44:51.635544+0000","last_peered":"2024-09-15T22:44:51.635544+0000","last_clean":"2024-09-15T22:44:51.635544+0000","last_became_active":"2024-09-15T22:44:17.205361+0000","last_became_peered":"2024-09-15T22:44:17.205361+0000","last_unstale":"2024-09-15T22:44:51.635544+0000","last_undegraded":"2024-09-15T22:44:51.635544+0000","last_fullsized":"2024-09-15T22:44:51.635544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:25.430166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:45:31.662304+0000","last_change":"2024-09-15T22:44:14.166471+0000","last_active":"2024-09-15T22:45:31.662304+0000","last_peered":"2024-09-15T22:45:31.662304+0000","last_clean":"2024-09-15T22:45:31.662304+0000","last_became_active":"2024-09-15T22:44:14.165969+0000","last_became_peered":"2024-09-15T22:44:14.165969+0000","last_unstale":"2024-09-15T22:45:31.662304+0000","last_undegraded":"2024-09-15T22:45:31.662304+0000","last_fullsized":"2024-09-15T22:45:31.662304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:57:40.108575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635288+0000","last_change":"2024-09-15T22:44:17.201488+0000","last_active":"2024-09-15T22:44:51.635288+0000","last_peered":"2024-09-15T22:44:51.635288+0000","last_clean":"2024-09-15T22:44:51.635288+0000","last_became_active":"2024-09-15T22:44:17.201101+0000","last_became_peered":"2024-09-15T22:44:17.201101+0000","last_unstale":"2024-09-15T22:44:51.635288+0000","last_undegraded":"2024-09-15T22:44:51.635288+0000","last_fullsized":"2024-09-15T22:44:51.635288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:41.758433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635006+0000","last_change":"2024-09-15T22:44:14.680734+0000","last_active":"2024-09-15T22:44:51.635006+0000","last_peered":"2024-09-15T22:44:51.635006+0000","last_clean":"2024-09-15T22:44:51.635006+0000","last_became_active":"2024-09-15T22:44:14.680362+0000","last_became_peered":"2024-09-15T22:44:14.680362+0000","last_unstale":"2024-09-15T22:44:51.635006+0000","last_undegraded":"2024-09-15T22:44:51.635006+0000","last_fullsized":"2024-09-15T22:44:51.635006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:22:37.086300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.638580+0000","last_change":"2024-09-15T22:44:17.199944+0000","last_active":"2024-09-15T22:44:51.638580+0000","last_peered":"2024-09-15T22:44:51.638580+0000","last_clean":"2024-09-15T22:44:51.638580+0000","last_became_active":"2024-09-15T22:44:17.199586+0000","last_became_peered":"2024-09-15T22:44:17.199586+0000","last_unstale":"2024-09-15T22:44:51.638580+0000","last_undegraded":"2024-09-15T22:44:51.638580+0000","last_fullsized":"2024-09-15T22:44:51.638580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:42:22.178939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635879+0000","last_change":"2024-09-15T22:44:14.169077+0000","last_active":"2024-09-15T22:44:51.635879+0000","last_peered":"2024-09-15T22:44:51.635879+0000","last_clean":"2024-09-15T22:44:51.635879+0000","last_became_active":"2024-09-15T22:44:14.168675+0000","last_became_peered":"2024-09-15T22:44:14.168675+0000","last_unstale":"2024-09-15T22:44:51.635879+0000","last_undegraded":"2024-09-15T22:44:51.635879+0000","last_fullsized":"2024-09-15T22:44:51.635879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:01.892217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.633178+0000","last_change":"2024-09-15T22:44:17.208587+0000","last_active":"2024-09-15T22:44:51.633178+0000","last_peered":"2024-09-15T22:44:51.633178+0000","last_clean":"2024-09-15T22:44:51.633178+0000","last_became_active":"2024-09-15T22:44:17.208260+0000","last_became_peered":"2024-09-15T22:44:17.208260+0000","last_unstale":"2024-09-15T22:44:51.633178+0000","last_undegraded":"2024-09-15T22:44:51.633178+0000","last_fullsized":"2024-09-15T22:44:51.633178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:01:04.154149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"86'2","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.632308+0000","last_change":"2024-09-15T22:44:14.168223+0000","last_active":"2024-09-15T22:44:51.632308+0000","last_peered":"2024-09-15T22:44:51.632308+0000","last_clean":"2024-09-15T22:44:51.632308+0000","last_became_active":"2024-09-15T22:44:14.167830+0000","last_became_peered":"2024-09-15T22:44:14.167830+0000","last_unstale":"2024-09-15T22:44:51.632308+0000","last_undegraded":"2024-09-15T22:44:51.632308+0000","last_fullsized":"2024-09-15T22:44:51.632308+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:54:38.064606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635822+0000","last_change":"2024-09-15T22:44:17.209696+0000","last_active":"2024-09-15T22:44:51.635822+0000","last_peered":"2024-09-15T22:44:51.635822+0000","last_clean":"2024-09-15T22:44:51.635822+0000","last_became_active":"2024-09-15T22:44:17.208667+0000","last_became_peered":"2024-09-15T22:44:17.208667+0000","last_unstale":"2024-09-15T22:44:51.635822+0000","last_undegraded":"2024-09-15T22:44:51.635822+0000","last_fullsized":"2024-09-15T22:44:51.635822+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:04.653022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'1","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.639263+0000","last_change":"2024-09-15T22:44:14.169728+0000","last_active":"2024-09-15T22:44:51.639263+0000","last_peered":"2024-09-15T22:44:51.639263+0000","last_clean":"2024-09-15T22:44:51.639263+0000","last_became_active":"2024-09-15T22:44:14.169354+0000","last_became_peered":"2024-09-15T22:44:14.169354+0000","last_unstale":"2024-09-15T22:44:51.639263+0000","last_undegraded":"2024-09-15T22:44:51.639263+0000","last_fullsized":"2024-09-15T22:44:51.639263+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:20.273852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.644709+0000","last_change":"2024-09-15T22:44:51.644709+0000","last_active":"2024-09-15T22:44:51.644709+0000","last_peered":"2024-09-15T22:44:51.644709+0000","last_clean":"2024-09-15T22:44:51.644709+0000","last_became_active":"2024-09-15T22:44:51.641165+0000","last_became_peered":"2024-09-15T22:44:51.641165+0000","last_unstale":"2024-09-15T22:44:51.644709+0000","last_undegraded":"2024-09-15T22:44:51.644709+0000","last_fullsized":"2024-09-15T22:44:51.644709+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:23:36.253167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635928+0000","last_change":"2024-09-15T22:44:14.679757+0000","last_active":"2024-09-15T22:44:51.635928+0000","last_peered":"2024-09-15T22:44:51.635928+0000","last_clean":"2024-09-15T22:44:51.635928+0000","last_became_active":"2024-09-15T22:44:14.679284+0000","last_became_peered":"2024-09-15T22:44:14.679284+0000","last_unstale":"2024-09-15T22:44:51.635928+0000","last_undegraded":"2024-09-15T22:44:51.635928+0000","last_fullsized":"2024-09-15T22:44:51.635928+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:47:21.368373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.636324+0000","last_change":"2024-09-15T22:44:17.207115+0000","last_active":"2024-09-15T22:44:51.636324+0000","last_peered":"2024-09-15T22:44:51.636324+0000","last_clean":"2024-09-15T22:44:51.636324+0000","last_became_active":"2024-09-15T22:44:17.206800+0000","last_became_peered":"2024-09-15T22:44:17.206800+0000","last_unstale":"2024-09-15T22:44:51.636324+0000","last_undegraded":"2024-09-15T22:44:51.636324+0000","last_fullsized":"2024-09-15T22:44:51.636324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:49:17.497725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":339,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635886+0000","last_change":"2024-09-15T22:42:59.117839+0000","last_active":"2024-09-15T22:44:51.635886+0000","last_peered":"2024-09-15T22:44:51.635886+0000","last_clean":"2024-09-15T22:44:51.635886+0000","last_became_active":"2024-09-15T22:42:59.117434+0000","last_became_peered":"2024-09-15T22:42:59.117434+0000","last_unstale":"2024-09-15T22:44:51.635886+0000","last_undegraded":"2024-09-15T22:44:51.635886+0000","last_fullsized":"2024-09-15T22:44:51.635886+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:39:19.064511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:39:19.064511+0000","last_clean_scrub_stamp":"2024-09-15T22:39:19.064511+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:38.974950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.795272+0000","last_change":"2024-09-15T22:44:14.170422+0000","last_active":"2024-09-15T22:44:51.795272+0000","last_peered":"2024-09-15T22:44:51.795272+0000","last_clean":"2024-09-15T22:44:51.795272+0000","last_became_active":"2024-09-15T22:44:14.169152+0000","last_became_peered":"2024-09-15T22:44:14.169152+0000","last_unstale":"2024-09-15T22:44:51.795272+0000","last_undegraded":"2024-09-15T22:44:51.795272+0000","last_fullsized":"2024-09-15T22:44:51.795272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:10:00.890434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.638676+0000","last_change":"2024-09-15T22:44:17.212972+0000","last_active":"2024-09-15T22:44:51.638676+0000","last_peered":"2024-09-15T22:44:51.638676+0000","last_clean":"2024-09-15T22:44:51.638676+0000","last_became_active":"2024-09-15T22:44:17.212546+0000","last_became_peered":"2024-09-15T22:44:17.212546+0000","last_unstale":"2024-09-15T22:44:51.638676+0000","last_undegraded":"2024-09-15T22:44:51.638676+0000","last_fullsized":"2024-09-15T22:44:51.638676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:35:36.322855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635707+0000","last_change":"2024-09-15T22:44:14.171827+0000","last_active":"2024-09-15T22:44:51.635707+0000","last_peered":"2024-09-15T22:44:51.635707+0000","last_clean":"2024-09-15T22:44:51.635707+0000","last_became_active":"2024-09-15T22:44:14.171447+0000","last_became_peered":"2024-09-15T22:44:14.171447+0000","last_unstale":"2024-09-15T22:44:51.635707+0000","last_undegraded":"2024-09-15T22:44:51.635707+0000","last_fullsized":"2024-09-15T22:44:51.635707+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:40:49.869819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.632269+0000","last_change":"2024-09-15T22:44:17.208666+0000","last_active":"2024-09-15T22:44:51.632269+0000","last_peered":"2024-09-15T22:44:51.632269+0000","last_clean":"2024-09-15T22:44:51.632269+0000","last_became_active":"2024-09-15T22:44:17.208338+0000","last_became_peered":"2024-09-15T22:44:17.208338+0000","last_unstale":"2024-09-15T22:44:51.632269+0000","last_undegraded":"2024-09-15T22:44:51.632269+0000","last_fullsized":"2024-09-15T22:44:51.632269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:10:23.250307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":94,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:45:33.664489+0000","last_change":"2024-09-15T22:44:14.679708+0000","last_active":"2024-09-15T22:45:33.664489+0000","last_peered":"2024-09-15T22:45:33.664489+0000","last_clean":"2024-09-15T22:45:33.664489+0000","last_became_active":"2024-09-15T22:44:14.679192+0000","last_became_peered":"2024-09-15T22:44:14.679192+0000","last_unstale":"2024-09-15T22:45:33.664489+0000","last_undegraded":"2024-09-15T22:45:33.664489+0000","last_fullsized":"2024-09-15T22:45:33.664489+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:25:21.192842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":64,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.640007+0000","last_change":"2024-09-15T22:44:17.202716+0000","last_active":"2024-09-15T22:44:51.640007+0000","last_peered":"2024-09-15T22:44:51.640007+0000","last_clean":"2024-09-15T22:44:51.640007+0000","last_became_active":"2024-09-15T22:44:17.202412+0000","last_became_peered":"2024-09-15T22:44:17.202412+0000","last_unstale":"2024-09-15T22:44:51.640007+0000","last_undegraded":"2024-09-15T22:44:51.640007+0000","last_fullsized":"2024-09-15T22:44:51.640007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:18:59.180862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.636482+0000","last_change":"2024-09-15T22:44:14.169202+0000","last_active":"2024-09-15T22:44:51.636482+0000","last_peered":"2024-09-15T22:44:51.636482+0000","last_clean":"2024-09-15T22:44:51.636482+0000","last_became_active":"2024-09-15T22:44:14.168866+0000","last_became_peered":"2024-09-15T22:44:14.168866+0000","last_unstale":"2024-09-15T22:44:51.636482+0000","last_undegraded":"2024-09-15T22:44:51.636482+0000","last_fullsized":"2024-09-15T22:44:51.636482+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:28:57.057050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.641455+0000","last_change":"2024-09-15T22:44:51.641455+0000","last_active":"2024-09-15T22:44:51.641455+0000","last_peered":"2024-09-15T22:44:51.641455+0000","last_clean":"2024-09-15T22:44:51.641455+0000","last_became_active":"2024-09-15T22:44:51.641037+0000","last_became_peered":"2024-09-15T22:44:51.641037+0000","last_unstale":"2024-09-15T22:44:51.641455+0000","last_undegraded":"2024-09-15T22:44:51.641455+0000","last_fullsized":"2024-09-15T22:44:51.641455+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:43:34.251236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.632255+0000","last_change":"2024-09-15T22:44:14.681514+0000","last_active":"2024-09-15T22:44:51.632255+0000","last_peered":"2024-09-15T22:44:51.632255+0000","last_clean":"2024-09-15T22:44:51.632255+0000","last_became_active":"2024-09-15T22:44:14.679895+0000","last_became_peered":"2024-09-15T22:44:14.679895+0000","last_unstale":"2024-09-15T22:44:51.632255+0000","last_undegraded":"2024-09-15T22:44:51.632255+0000","last_fullsized":"2024-09-15T22:44:51.632255+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:13:22.059389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.636468+0000","last_change":"2024-09-15T22:44:17.210094+0000","last_active":"2024-09-15T22:44:51.636468+0000","last_peered":"2024-09-15T22:44:51.636468+0000","last_clean":"2024-09-15T22:44:51.636468+0000","last_became_active":"2024-09-15T22:44:17.209784+0000","last_became_peered":"2024-09-15T22:44:17.209784+0000","last_unstale":"2024-09-15T22:44:51.636468+0000","last_undegraded":"2024-09-15T22:44:51.636468+0000","last_fullsized":"2024-09-15T22:44:51.636468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:12:02.357936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635847+0000","last_change":"2024-09-15T22:44:14.680558+0000","last_active":"2024-09-15T22:44:51.635847+0000","last_peered":"2024-09-15T22:44:51.635847+0000","last_clean":"2024-09-15T22:44:51.635847+0000","last_became_active":"2024-09-15T22:44:14.680183+0000","last_became_peered":"2024-09-15T22:44:14.680183+0000","last_unstale":"2024-09-15T22:44:51.635847+0000","last_undegraded":"2024-09-15T22:44:51.635847+0000","last_fullsized":"2024-09-15T22:44:51.635847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:44.327412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.632674+0000","last_change":"2024-09-15T22:44:17.210661+0000","last_active":"2024-09-15T22:44:51.632674+0000","last_peered":"2024-09-15T22:44:51.632674+0000","last_clean":"2024-09-15T22:44:51.632674+0000","last_became_active":"2024-09-15T22:44:17.210279+0000","last_became_peered":"2024-09-15T22:44:17.210279+0000","last_unstale":"2024-09-15T22:44:51.632674+0000","last_undegraded":"2024-09-15T22:44:51.632674+0000","last_fullsized":"2024-09-15T22:44:51.632674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:58:45.430753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'11","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.632511+0000","last_change":"2024-09-15T22:44:14.164859+0000","last_active":"2024-09-15T22:44:51.632511+0000","last_peered":"2024-09-15T22:44:51.632511+0000","last_clean":"2024-09-15T22:44:51.632511+0000","last_became_active":"2024-09-15T22:44:14.163843+0000","last_became_peered":"2024-09-15T22:44:14.163843+0000","last_unstale":"2024-09-15T22:44:51.632511+0000","last_undegraded":"2024-09-15T22:44:51.632511+0000","last_fullsized":"2024-09-15T22:44:51.632511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:01.322732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.639355+0000","last_change":"2024-09-15T22:44:17.211891+0000","last_active":"2024-09-15T22:44:51.639355+0000","last_peered":"2024-09-15T22:44:51.639355+0000","last_clean":"2024-09-15T22:44:51.639355+0000","last_became_active":"2024-09-15T22:44:17.211515+0000","last_became_peered":"2024-09-15T22:44:17.211515+0000","last_unstale":"2024-09-15T22:44:51.639355+0000","last_undegraded":"2024-09-15T22:44:51.639355+0000","last_fullsized":"2024-09-15T22:44:51.639355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:05:58.223408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.795107+0000","last_change":"2024-09-15T22:44:14.167623+0000","last_active":"2024-09-15T22:44:51.795107+0000","last_peered":"2024-09-15T22:44:51.795107+0000","last_clean":"2024-09-15T22:44:51.795107+0000","last_became_active":"2024-09-15T22:44:14.167280+0000","last_became_peered":"2024-09-15T22:44:14.167280+0000","last_unstale":"2024-09-15T22:44:51.795107+0000","last_undegraded":"2024-09-15T22:44:51.795107+0000","last_fullsized":"2024-09-15T22:44:51.795107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:00:07.609473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.640290+0000","last_change":"2024-09-15T22:44:17.215630+0000","last_active":"2024-09-15T22:44:51.640290+0000","last_peered":"2024-09-15T22:44:51.640290+0000","last_clean":"2024-09-15T22:44:51.640290+0000","last_became_active":"2024-09-15T22:44:17.215328+0000","last_became_peered":"2024-09-15T22:44:17.215328+0000","last_unstale":"2024-09-15T22:44:51.640290+0000","last_undegraded":"2024-09-15T22:44:51.640290+0000","last_fullsized":"2024-09-15T22:44:51.640290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:13:54.545954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.644603+0000","last_change":"2024-09-15T22:44:51.644603+0000","last_active":"2024-09-15T22:44:51.644603+0000","last_peered":"2024-09-15T22:44:51.644603+0000","last_clean":"2024-09-15T22:44:51.644603+0000","last_became_active":"2024-09-15T22:44:51.641126+0000","last_became_peered":"2024-09-15T22:44:51.641126+0000","last_unstale":"2024-09-15T22:44:51.644603+0000","last_undegraded":"2024-09-15T22:44:51.644603+0000","last_fullsized":"2024-09-15T22:44:51.644603+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:04:23.985487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635000+0000","last_change":"2024-09-15T22:44:17.208072+0000","last_active":"2024-09-15T22:44:51.635000+0000","last_peered":"2024-09-15T22:44:51.635000+0000","last_clean":"2024-09-15T22:44:51.635000+0000","last_became_active":"2024-09-15T22:44:17.207580+0000","last_became_peered":"2024-09-15T22:44:17.207580+0000","last_unstale":"2024-09-15T22:44:51.635000+0000","last_undegraded":"2024-09-15T22:44:51.635000+0000","last_fullsized":"2024-09-15T22:44:51.635000+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:48.973412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.638567+0000","last_change":"2024-09-15T22:44:14.168841+0000","last_active":"2024-09-15T22:44:51.638567+0000","last_peered":"2024-09-15T22:44:51.638567+0000","last_clean":"2024-09-15T22:44:51.638567+0000","last_became_active":"2024-09-15T22:44:14.168359+0000","last_became_peered":"2024-09-15T22:44:14.168359+0000","last_unstale":"2024-09-15T22:44:51.638567+0000","last_undegraded":"2024-09-15T22:44:51.638567+0000","last_fullsized":"2024-09-15T22:44:51.638567+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:09:29.752679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.633044+0000","last_change":"2024-09-15T22:44:17.218227+0000","last_active":"2024-09-15T22:44:51.633044+0000","last_peered":"2024-09-15T22:44:51.633044+0000","last_clean":"2024-09-15T22:44:51.633044+0000","last_became_active":"2024-09-15T22:44:17.217846+0000","last_became_peered":"2024-09-15T22:44:17.217846+0000","last_unstale":"2024-09-15T22:44:51.633044+0000","last_undegraded":"2024-09-15T22:44:51.633044+0000","last_fullsized":"2024-09-15T22:44:51.633044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:07.601525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635631+0000","last_change":"2024-09-15T22:44:14.165322+0000","last_active":"2024-09-15T22:44:51.635631+0000","last_peered":"2024-09-15T22:44:51.635631+0000","last_clean":"2024-09-15T22:44:51.635631+0000","last_became_active":"2024-09-15T22:44:14.164981+0000","last_became_peered":"2024-09-15T22:44:14.164981+0000","last_unstale":"2024-09-15T22:44:51.635631+0000","last_undegraded":"2024-09-15T22:44:51.635631+0000","last_fullsized":"2024-09-15T22:44:51.635631+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:32:34.812514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.638751+0000","last_change":"2024-09-15T22:44:17.222414+0000","last_active":"2024-09-15T22:44:51.638751+0000","last_peered":"2024-09-15T22:44:51.638751+0000","last_clean":"2024-09-15T22:44:51.638751+0000","last_became_active":"2024-09-15T22:44:17.222115+0000","last_became_peered":"2024-09-15T22:44:17.222115+0000","last_unstale":"2024-09-15T22:44:51.638751+0000","last_undegraded":"2024-09-15T22:44:51.638751+0000","last_fullsized":"2024-09-15T22:44:51.638751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:22:53.791587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.644655+0000","last_change":"2024-09-15T22:44:51.644117+0000","last_active":"2024-09-15T22:44:51.644655+0000","last_peered":"2024-09-15T22:44:51.644655+0000","last_clean":"2024-09-15T22:44:51.644655+0000","last_became_active":"2024-09-15T22:44:51.638270+0000","last_became_peered":"2024-09-15T22:44:51.638270+0000","last_unstale":"2024-09-15T22:44:51.644655+0000","last_undegraded":"2024-09-15T22:44:51.644655+0000","last_fullsized":"2024-09-15T22:44:51.644655+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:01.496067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635668+0000","last_change":"2024-09-15T22:44:17.208143+0000","last_active":"2024-09-15T22:44:51.635668+0000","last_peered":"2024-09-15T22:44:51.635668+0000","last_clean":"2024-09-15T22:44:51.635668+0000","last_became_active":"2024-09-15T22:44:17.207807+0000","last_became_peered":"2024-09-15T22:44:17.207807+0000","last_unstale":"2024-09-15T22:44:51.635668+0000","last_undegraded":"2024-09-15T22:44:51.635668+0000","last_fullsized":"2024-09-15T22:44:51.635668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:19:59.752854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.638931+0000","last_change":"2024-09-15T22:44:17.201752+0000","last_active":"2024-09-15T22:44:51.638931+0000","last_peered":"2024-09-15T22:44:51.638931+0000","last_clean":"2024-09-15T22:44:51.638931+0000","last_became_active":"2024-09-15T22:44:17.201378+0000","last_became_peered":"2024-09-15T22:44:17.201378+0000","last_unstale":"2024-09-15T22:44:51.638931+0000","last_undegraded":"2024-09-15T22:44:51.638931+0000","last_fullsized":"2024-09-15T22:44:51.638931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:42:53.752795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"86'2","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635480+0000","last_change":"2024-09-15T22:44:14.168024+0000","last_active":"2024-09-15T22:44:51.635480+0000","last_peered":"2024-09-15T22:44:51.635480+0000","last_clean":"2024-09-15T22:44:51.635480+0000","last_became_active":"2024-09-15T22:44:14.167639+0000","last_became_peered":"2024-09-15T22:44:14.167639+0000","last_unstale":"2024-09-15T22:44:51.635480+0000","last_undegraded":"2024-09-15T22:44:51.635480+0000","last_fullsized":"2024-09-15T22:44:51.635480+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:31:57.867895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635640+0000","last_change":"2024-09-15T22:44:17.202731+0000","last_active":"2024-09-15T22:44:51.635640+0000","last_peered":"2024-09-15T22:44:51.635640+0000","last_clean":"2024-09-15T22:44:51.635640+0000","last_became_active":"2024-09-15T22:44:17.202410+0000","last_became_peered":"2024-09-15T22:44:17.202410+0000","last_unstale":"2024-09-15T22:44:51.635640+0000","last_undegraded":"2024-09-15T22:44:51.635640+0000","last_fullsized":"2024-09-15T22:44:51.635640+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:34.613686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.635735+0000","last_change":"2024-09-15T22:44:14.166865+0000","last_active":"2024-09-15T22:44:51.635735+0000","last_peered":"2024-09-15T22:44:51.635735+0000","last_clean":"2024-09-15T22:44:51.635735+0000","last_became_active":"2024-09-15T22:44:14.166014+0000","last_became_peered":"2024-09-15T22:44:14.166014+0000","last_unstale":"2024-09-15T22:44:51.635735+0000","last_undegraded":"2024-09-15T22:44:51.635735+0000","last_fullsized":"2024-09-15T22:44:51.635735+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:08:44.527777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.640513+0000","last_change":"2024-09-15T22:44:17.221028+0000","last_active":"2024-09-15T22:44:51.640513+0000","last_peered":"2024-09-15T22:44:51.640513+0000","last_clean":"2024-09-15T22:44:51.640513+0000","last_became_active":"2024-09-15T22:44:17.220853+0000","last_became_peered":"2024-09-15T22:44:17.220853+0000","last_unstale":"2024-09-15T22:44:51.640513+0000","last_undegraded":"2024-09-15T22:44:51.640513+0000","last_fullsized":"2024-09-15T22:44:51.640513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:10:50.535946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.639205+0000","last_change":"2024-09-15T22:44:14.679506+0000","last_active":"2024-09-15T22:44:51.639205+0000","last_peered":"2024-09-15T22:44:51.639205+0000","last_clean":"2024-09-15T22:44:51.639205+0000","last_became_active":"2024-09-15T22:44:14.679164+0000","last_became_peered":"2024-09-15T22:44:14.679164+0000","last_unstale":"2024-09-15T22:44:51.639205+0000","last_undegraded":"2024-09-15T22:44:51.639205+0000","last_fullsized":"2024-09-15T22:44:51.639205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:57.786804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.639078+0000","last_change":"2024-09-15T22:44:17.203867+0000","last_active":"2024-09-15T22:44:51.639078+0000","last_peered":"2024-09-15T22:44:51.639078+0000","last_clean":"2024-09-15T22:44:51.639078+0000","last_became_active":"2024-09-15T22:44:17.203450+0000","last_became_peered":"2024-09-15T22:44:17.203450+0000","last_unstale":"2024-09-15T22:44:51.639078+0000","last_undegraded":"2024-09-15T22:44:51.639078+0000","last_fullsized":"2024-09-15T22:44:51.639078+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:16.165171+0000","last_clean_scrub_stamp":"2024-09-15T22:44:16.165171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:05:17.210989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:44:51.632786+0000","last_change":"2024-09-15T22:44:14.164846+0000","last_active":"2024-09-15T22:44:51.632786+0000","last_peered":"2024-09-15T22:44:51.632786+0000","last_clean":"2024-09-15T22:44:51.632786+0000","last_became_active":"2024-09-15T22:44:14.164507+0000","last_became_peered":"2024-09-15T22:44:14.164507+0000","last_unstale":"2024-09-15T22:44:51.632786+0000","last_undegraded":"2024-09-15T22:44:51.632786+0000","last_fullsized":"2024-09-15T22:44:51.632786+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:44:13.144741+0000","last_clean_scrub_stamp":"2024-09-15T22:44:13.144741+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:55.234677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66313,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":402,"num_read_kb":274,"num_write":166,"num_write_kb":114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1015,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":198939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449118,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":958464,"data_stored":471084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678051,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32804,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712348,"statfs":{"total":95995035648,"available":95961444352,"internally_reserved":0,"allocated":1658880,"data_stored":1124875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874282,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":978944,"data_stored":471437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135917,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":974848,"data_stored":471240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397556,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1568768,"data_stored":1063172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561080,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":962560,"data_stored":470709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790015,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1581056,"data_stored":1062856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953538,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":974848,"data_stored":472161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":470903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411405,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":533540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":532812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":983040,"data_stored":471980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":63888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:45:38.818 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-15T22:45:38.818 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-15T22:45:38.818 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-15T22:45:38.818 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-15T22:45:39.184 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:39.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.811+0000 7f3f98a00640 1 -- 172.21.15.111:0/1472037563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f94103e50 msgr2=0x7f3f94104230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:39.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.811+0000 7f3f98a00640 1 --2- 172.21.15.111:0/1472037563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f94103e50 0x7f3f94104230 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3f84009ca0 tx=0x7f3f8402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:45:39.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.811+0000 7f3f98a00640 1 -- 172.21.15.111:0/1472037563 shutdown_connections 2024-09-15T22:45:39.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.811+0000 7f3f98a00640 1 --2- 172.21.15.111:0/1472037563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f94109090 0x7f3f94110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:39.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.811+0000 7f3f98a00640 1 --2- 172.21.15.111:0/1472037563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f94104800 0x7f3f94108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:39.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.811+0000 7f3f98a00640 1 --2- 172.21.15.111:0/1472037563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f94103e50 0x7f3f94104230 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:39.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.811+0000 7f3f98a00640 1 -- 172.21.15.111:0/1472037563 >> 172.21.15.111:0/1472037563 conn(0x7f3f940ff8b0 msgr2=0x7f3f94101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:39.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.811+0000 7f3f98a00640 1 -- 172.21.15.111:0/1472037563 shutdown_connections 2024-09-15T22:45:39.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.812+0000 7f3f98a00640 1 -- 172.21.15.111:0/1472037563 wait complete. 2024-09-15T22:45:39.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.812+0000 7f3f98a00640 1 Processor -- start 2024-09-15T22:45:39.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.812+0000 7f3f98a00640 1 -- start start 2024-09-15T22:45:39.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.813+0000 7f3f98a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f94103e50 0x7f3f9419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:39.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.813+0000 7f3f98a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f94104800 0x7f3f9419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:39.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.813+0000 7f3f98a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f94109090 0x7f3f9419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:39.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.813+0000 7f3f98a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3f94114100 con 0x7f3f94104800 2024-09-15T22:45:39.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.813+0000 7f3f98a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3f94113f80 con 0x7f3f94109090 2024-09-15T22:45:39.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.813+0000 7f3f98a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3f94114280 con 0x7f3f94103e50 2024-09-15T22:45:39.816 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.813+0000 7f3f91e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f94104800 0x7f3f9419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:39.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.813+0000 7f3f91e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f94104800 0x7f3f9419ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:49194/0 (socket says 172.21.15.111:49194) 2024-09-15T22:45:39.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.813+0000 7f3f91e00640 1 -- 172.21.15.111:0/3355649107 learned_addr learned my addr 172.21.15.111:0/3355649107 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:39.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.813+0000 7f3f93200640 1 --2- 172.21.15.111:0/3355649107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f94109090 0x7f3f9419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:39.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.813+0000 7f3f92800640 1 --2- 172.21.15.111:0/3355649107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f94103e50 0x7f3f9419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:39.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.814+0000 7f3f91e00640 1 -- 172.21.15.111:0/3355649107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f94103e50 msgr2=0x7f3f9419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:39.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.814+0000 7f3f91e00640 1 --2- 172.21.15.111:0/3355649107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f94103e50 0x7f3f9419a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:39.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.814+0000 7f3f91e00640 1 -- 172.21.15.111:0/3355649107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f94109090 msgr2=0x7f3f9419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:39.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.814+0000 7f3f91e00640 1 --2- 172.21.15.111:0/3355649107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f94109090 0x7f3f9419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:39.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.814+0000 7f3f91e00640 1 -- 172.21.15.111:0/3355649107 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f9419f570 con 0x7f3f94104800 2024-09-15T22:45:39.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.814+0000 7f3f93200640 1 --2- 172.21.15.111:0/3355649107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f94109090 0x7f3f9419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:39.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.814+0000 7f3f91e00640 1 --2- 172.21.15.111:0/3355649107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f94104800 0x7f3f9419ab00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000e9f0 tx=0x7f3f8000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:39.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.814+0000 7f3f8b400640 1 -- 172.21.15.111:0/3355649107 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8000cd80 con 0x7f3f94104800 2024-09-15T22:45:39.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.814+0000 7f3f8b400640 1 -- 172.21.15.111:0/3355649107 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f80005500 con 0x7f3f94104800 2024-09-15T22:45:39.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.814+0000 7f3f98a00640 1 -- 172.21.15.111:0/3355649107 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f9419f860 con 0x7f3f94104800 2024-09-15T22:45:39.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.814+0000 7f3f8b400640 1 -- 172.21.15.111:0/3355649107 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80010690 con 0x7f3f94104800 2024-09-15T22:45:39.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.815+0000 7f3f98a00640 1 -- 172.21.15.111:0/3355649107 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f941a7140 con 0x7f3f94104800 2024-09-15T22:45:39.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.816+0000 7f3f98a00640 1 -- 172.21.15.111:0/3355649107 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f94105330 con 0x7f3f94104800 2024-09-15T22:45:39.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.817+0000 7f3f8b400640 1 -- 172.21.15.111:0/3355649107 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f3f800046c0 con 0x7f3f94104800 2024-09-15T22:45:39.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.819+0000 7f3f8b400640 1 --2- 172.21.15.111:0/3355649107 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3f780a32d0 0x7f3f780a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:39.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.820+0000 7f3f8b400640 1 -- 172.21.15.111:0/3355649107 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f3f800cfe80 con 0x7f3f94104800 2024-09-15T22:45:39.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.820+0000 7f3f92800640 1 --2- 172.21.15.111:0/3355649107 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3f780a32d0 0x7f3f780a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:39.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.820+0000 7f3f92800640 1 --2- 172.21.15.111:0/3355649107 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3f780a32d0 0x7f3f780a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3f84006010 tx=0x7f3f84009820 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:39.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:39.824+0000 7f3f8b400640 1 -- 172.21.15.111:0/3355649107 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3f8009ca40 con 0x7f3f94104800 2024-09-15T22:45:39.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:39 smithi121 ceph-mon[34815]: from='client.24851 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:45:39.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:39 smithi121 ceph-mon[34815]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:39.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:39 smithi111 ceph-mon[30629]: from='client.24851 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:45:39.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:39 smithi111 ceph-mon[30629]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:40.037 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:40.035+0000 7f3f98a00640 1 -- 172.21.15.111:0/3355649107 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3f9410c270 con 0x7f3f94104800 2024-09-15T22:45:40.037 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:40.035+0000 7f3f8b400640 1 -- 172.21.15.111:0/3355649107 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f3f8009c340 con 0x7f3f94104800 2024-09-15T22:45:40.037 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:45:40.037 INFO:teuthology.orchestra.run.smithi111.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-15T22:45:40.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:40.037+0000 7f3f98a00640 1 -- 172.21.15.111:0/3355649107 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3f780a32d0 msgr2=0x7f3f780a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:40.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:40.037+0000 7f3f98a00640 1 --2- 172.21.15.111:0/3355649107 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3f780a32d0 0x7f3f780a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3f84006010 tx=0x7f3f84009820 comp rx=0 tx=0).stop 2024-09-15T22:45:40.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:40.037+0000 7f3f98a00640 1 -- 172.21.15.111:0/3355649107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f94104800 msgr2=0x7f3f9419ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:40.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:40.037+0000 7f3f98a00640 1 --2- 172.21.15.111:0/3355649107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f94104800 0x7f3f9419ab00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000e9f0 tx=0x7f3f8000edf0 comp rx=0 tx=0).stop 2024-09-15T22:45:40.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:40.038+0000 7f3f98a00640 1 -- 172.21.15.111:0/3355649107 shutdown_connections 2024-09-15T22:45:40.039 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:40.038+0000 7f3f98a00640 1 --2- 172.21.15.111:0/3355649107 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3f780a32d0 0x7f3f780a5790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:40.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:40.038+0000 7f3f98a00640 1 --2- 172.21.15.111:0/3355649107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3f94109090 0x7f3f9419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:40.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:40.038+0000 7f3f98a00640 1 --2- 172.21.15.111:0/3355649107 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f94104800 0x7f3f9419ab00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:40.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:40.038+0000 7f3f98a00640 1 --2- 172.21.15.111:0/3355649107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f94103e50 0x7f3f9419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:40.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:40.038+0000 7f3f98a00640 1 -- 172.21.15.111:0/3355649107 >> 172.21.15.111:0/3355649107 conn(0x7f3f940ff8b0 msgr2=0x7f3f94100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:40.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:40.038+0000 7f3f98a00640 1 -- 172.21.15.111:0/3355649107 shutdown_connections 2024-09-15T22:45:40.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:40.038+0000 7f3f98a00640 1 -- 172.21.15.111:0/3355649107 wait complete. 2024-09-15T22:45:40.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:39 smithi192 ceph-mon[35427]: from='client.24851 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:45:40.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:39 smithi192 ceph-mon[35427]: pgmap v342: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:40.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:40 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/3355649107' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-15T22:45:40.901 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:40 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/3355649107' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-15T22:45:40.938 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-15T22:45:40.938 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:45:41.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:40 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/3355649107' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-15T22:45:41.304 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:41.766 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:41 smithi111 ceph-mon[30629]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:45:41.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:41 smithi121 ceph-mon[34815]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:45:41.912 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.875+0000 7f2de9c00640 1 -- 172.21.15.111:0/2519734883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2de4074f40 msgr2=0x7f2de410de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:41.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.875+0000 7f2de9c00640 1 --2- 172.21.15.111:0/2519734883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2de4074f40 0x7f2de410de80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0009900 tx=0x7f2dd002a480 comp rx=0 tx=0).stop 2024-09-15T22:45:41.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.875+0000 7f2de9c00640 1 -- 172.21.15.111:0/2519734883 shutdown_connections 2024-09-15T22:45:41.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.875+0000 7f2de9c00640 1 --2- 172.21.15.111:0/2519734883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2de410e3c0 0x7f2de4110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:41.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.875+0000 7f2de9c00640 1 --2- 172.21.15.111:0/2519734883 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2de4074f40 0x7f2de410de80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:41.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.875+0000 7f2de9c00640 1 --2- 172.21.15.111:0/2519734883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2de4074620 0x7f2de4074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:41.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.875+0000 7f2de9c00640 1 -- 172.21.15.111:0/2519734883 >> 172.21.15.111:0/2519734883 conn(0x7f2de40fd8c0 msgr2=0x7f2de40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:41.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.876+0000 7f2de9c00640 1 -- 172.21.15.111:0/2519734883 shutdown_connections 2024-09-15T22:45:41.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.876+0000 7f2de9c00640 1 -- 172.21.15.111:0/2519734883 wait complete. 2024-09-15T22:45:41.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.876+0000 7f2de9c00640 1 Processor -- start 2024-09-15T22:45:41.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.877+0000 7f2de9c00640 1 -- start start 2024-09-15T22:45:41.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.877+0000 7f2de9c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2de4074620 0x7f2de419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:41.913 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.877+0000 7f2de9c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2de4074f40 0x7f2de419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:41.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.877+0000 7f2de9c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2de410e3c0 0x7f2de419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:41.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.877+0000 7f2de9c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2de41141b0 con 0x7f2de4074f40 2024-09-15T22:45:41.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.877+0000 7f2de9c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2de4114030 con 0x7f2de410e3c0 2024-09-15T22:45:41.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.877+0000 7f2de9c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2de4114330 con 0x7f2de4074620 2024-09-15T22:45:41.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.877+0000 7f2de2000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2de4074f40 0x7f2de419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:41.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.877+0000 7f2de2000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2de4074f40 0x7f2de419abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45598/0 (socket says 172.21.15.111:45598) 2024-09-15T22:45:41.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.877+0000 7f2de2000640 1 -- 172.21.15.111:0/642769904 learned_addr learned my addr 172.21.15.111:0/642769904 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:41.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.877+0000 7f2de2a00640 1 --2- 172.21.15.111:0/642769904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2de4074620 0x7f2de419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:41.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.877+0000 7f2de3400640 1 --2- 172.21.15.111:0/642769904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2de410e3c0 0x7f2de419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:41.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.878+0000 7f2de2000640 1 -- 172.21.15.111:0/642769904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2de4074620 msgr2=0x7f2de419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:41.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.878+0000 7f2de2000640 1 --2- 172.21.15.111:0/642769904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2de4074620 0x7f2de419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:41.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.878+0000 7f2de2000640 1 -- 172.21.15.111:0/642769904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2de410e3c0 msgr2=0x7f2de419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:41.914 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.878+0000 7f2de2000640 1 --2- 172.21.15.111:0/642769904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2de410e3c0 0x7f2de419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:41.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.878+0000 7f2de2000640 1 -- 172.21.15.111:0/642769904 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de419f620 con 0x7f2de4074f40 2024-09-15T22:45:41.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.878+0000 7f2de2000640 1 --2- 172.21.15.111:0/642769904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2de4074f40 0x7f2de419abb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0004770 tx=0x7f2dd0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:41.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.878+0000 7f2ddb400640 1 -- 172.21.15.111:0/642769904 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd003f070 con 0x7f2de4074f40 2024-09-15T22:45:41.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.878+0000 7f2ddb400640 1 -- 172.21.15.111:0/642769904 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dd0031ca0 con 0x7f2de4074f40 2024-09-15T22:45:41.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.878+0000 7f2ddb400640 1 -- 172.21.15.111:0/642769904 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0035030 con 0x7f2de4074f40 2024-09-15T22:45:41.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.878+0000 7f2de9c00640 1 -- 172.21.15.111:0/642769904 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de419f8b0 con 0x7f2de4074f40 2024-09-15T22:45:41.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.878+0000 7f2de9c00640 1 -- 172.21.15.111:0/642769904 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de419fd40 con 0x7f2de4074f40 2024-09-15T22:45:41.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.879+0000 7f2de9c00640 1 -- 172.21.15.111:0/642769904 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2de4074a00 con 0x7f2de4074f40 2024-09-15T22:45:41.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.881+0000 7f2ddb400640 1 -- 172.21.15.111:0/642769904 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f2dd0042070 con 0x7f2de4074f40 2024-09-15T22:45:41.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.883+0000 7f2ddb400640 1 --2- 172.21.15.111:0/642769904 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2dc80a3350 0x7f2dc80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:41.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.883+0000 7f2ddb400640 1 -- 172.21.15.111:0/642769904 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f2dd00f05f0 con 0x7f2de4074f40 2024-09-15T22:45:41.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.883+0000 7f2de2a00640 1 --2- 172.21.15.111:0/642769904 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2dc80a3350 0x7f2dc80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:41.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.883+0000 7f2de2a00640 1 --2- 172.21.15.111:0/642769904 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2dc80a3350 0x7f2dc80a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4009bd0 tx=0x7f2dd4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:41.915 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:41.884+0000 7f2ddb400640 1 -- 172.21.15.111:0/642769904 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2dd00bcc20 con 0x7f2de4074f40 2024-09-15T22:45:42.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:41 smithi192 ceph-mon[35427]: pgmap v343: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:45:42.096 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:42.094+0000 7f2de9c00640 1 -- 172.21.15.111:0/642769904 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2de402d7c0 con 0x7f2de4074f40 2024-09-15T22:45:42.097 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:42.095+0000 7f2ddb400640 1 -- 172.21.15.111:0/642769904 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11931 (secure 0 0 0) 0x7f2dd00c0f20 con 0x7f2de4074f40 2024-09-15T22:45:42.097 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:45:42.098 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":25,"btime":"2024-09-15T22:44:34:567598+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/453336603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":453336603},{"type":"v1","addr":"172.21.15.111:6835","nonce":453336603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/3181832287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3181832287},{"type":"v1","addr":"172.21.15.111:6837","nonce":3181832287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/4190237053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":4190237053},{"type":"v1","addr":"172.21.15.111:6839","nonce":4190237053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14502,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6841/267728828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6840","nonce":267728828},{"type":"v1","addr":"172.21.15.111:6841","nonce":267728828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/4061441359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":4061441359},{"type":"v1","addr":"172.21.15.192:6833","nonce":4061441359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/4220531578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":4220531578},{"type":"v1","addr":"172.21.15.121:6833","nonce":4220531578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:44:18.436394+0000","modified":"2024-09-15T22:44:33.655164+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24514,"mds_2":24503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3371796806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3371796806},{"type":"v1","addr":"172.21.15.121:6835","nonce":3371796806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/4080914921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":4080914921},{"type":"v1","addr":"172.21.15.192:6835","nonce":4080914921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/3995501707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3995501707},{"type":"v1","addr":"172.21.15.192:6837","nonce":3995501707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.121:6837/1443772791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1443772791},{"type":"v1","addr":"172.21.15.121:6837","nonce":1443772791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6839/800136261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":800136261},{"type":"v1","addr":"172.21.15.192:6839","nonce":800136261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.121:6839/1797352097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1797352097},{"type":"v1","addr":"172.21.15.121:6839","nonce":1797352097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545,24514,24503]},"id":1}]} 2024-09-15T22:45:42.098 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 25 2024-09-15T22:45:42.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:42.098+0000 7f2de9c00640 1 -- 172.21.15.111:0/642769904 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2dc80a3350 msgr2=0x7f2dc80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:42.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:42.098+0000 7f2de9c00640 1 --2- 172.21.15.111:0/642769904 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2dc80a3350 0x7f2dc80a5810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4009bd0 tx=0x7f2dd4005f90 comp rx=0 tx=0).stop 2024-09-15T22:45:42.100 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:42.098+0000 7f2de9c00640 1 -- 172.21.15.111:0/642769904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2de4074f40 msgr2=0x7f2de419abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:42.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:42.098+0000 7f2de9c00640 1 --2- 172.21.15.111:0/642769904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2de4074f40 0x7f2de419abb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0004770 tx=0x7f2dd0003c10 comp rx=0 tx=0).stop 2024-09-15T22:45:42.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:42.098+0000 7f2de9c00640 1 -- 172.21.15.111:0/642769904 shutdown_connections 2024-09-15T22:45:42.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:42.098+0000 7f2de9c00640 1 --2- 172.21.15.111:0/642769904 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2dc80a3350 0x7f2dc80a5810 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:42.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:42.098+0000 7f2de9c00640 1 --2- 172.21.15.111:0/642769904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2de410e3c0 0x7f2de419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:42.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:42.098+0000 7f2de9c00640 1 --2- 172.21.15.111:0/642769904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2de4074f40 0x7f2de419abb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:42.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:42.098+0000 7f2de9c00640 1 --2- 172.21.15.111:0/642769904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2de4074620 0x7f2de419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:42.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:42.098+0000 7f2de9c00640 1 -- 172.21.15.111:0/642769904 >> 172.21.15.111:0/642769904 conn(0x7f2de40fd8c0 msgr2=0x7f2de40fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:42.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:42.098+0000 7f2de9c00640 1 -- 172.21.15.111:0/642769904 shutdown_connections 2024-09-15T22:45:42.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:42.098+0000 7f2de9c00640 1 -- 172.21.15.111:0/642769904 wait complete. 2024-09-15T22:45:42.825 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:42 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/642769904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:45:42.862 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:45:42.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:42 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/642769904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:45:43.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:42 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/642769904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:45:43.226 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:43.769 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:43 smithi111 ceph-mon[30629]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:45:43.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.768+0000 7f3187200640 1 -- 172.21.15.111:0/466549253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3188103e30 msgr2=0x7f3188104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:43.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.768+0000 7f3187200640 1 --2- 172.21.15.111:0/466549253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3188103e30 0x7f3188104210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3178009900 tx=0x7f317802a3f0 comp rx=0 tx=0).stop 2024-09-15T22:45:43.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.768+0000 7f3187200640 1 -- 172.21.15.111:0/466549253 shutdown_connections 2024-09-15T22:45:43.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.768+0000 7f3187200640 1 --2- 172.21.15.111:0/466549253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3188108f60 0x7f3188110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:43.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.768+0000 7f3187200640 1 --2- 172.21.15.111:0/466549253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31881047e0 0x7f3188108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:43.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.768+0000 7f3187200640 1 --2- 172.21.15.111:0/466549253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3188103e30 0x7f3188104210 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:43.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.768+0000 7f3187200640 1 -- 172.21.15.111:0/466549253 >> 172.21.15.111:0/466549253 conn(0x7f31880ff8d0 msgr2=0x7f3188101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:43.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.768+0000 7f3187200640 1 -- 172.21.15.111:0/466549253 shutdown_connections 2024-09-15T22:45:43.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.768+0000 7f3187200640 1 -- 172.21.15.111:0/466549253 wait complete. 2024-09-15T22:45:43.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.769+0000 7f3187200640 1 Processor -- start 2024-09-15T22:45:43.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.769+0000 7f3187200640 1 -- start start 2024-09-15T22:45:43.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.770+0000 7f3187200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3188103e30 0x7f318819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:43.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.770+0000 7f3187200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31881047e0 0x7f318819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:43.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.770+0000 7f3187200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3188108f60 0x7f318819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:43.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.770+0000 7f3187200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3188114010 con 0x7f3188103e30 2024-09-15T22:45:43.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.770+0000 7f3187200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3188113e90 con 0x7f31881047e0 2024-09-15T22:45:43.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.770+0000 7f3187200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3188114190 con 0x7f3188108f60 2024-09-15T22:45:43.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.770+0000 7f3185e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3188103e30 0x7f318819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:43.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.770+0000 7f3185e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3188103e30 0x7f318819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45624/0 (socket says 172.21.15.111:45624) 2024-09-15T22:45:43.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.770+0000 7f3185e00640 1 -- 172.21.15.111:0/383926873 learned_addr learned my addr 172.21.15.111:0/383926873 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:43.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.770+0000 7f3186800640 1 --2- 172.21.15.111:0/383926873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3188108f60 0x7f318819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:43.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.770+0000 7f3185400640 1 --2- 172.21.15.111:0/383926873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31881047e0 0x7f318819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:43.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.770+0000 7f3186800640 1 -- 172.21.15.111:0/383926873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31881047e0 msgr2=0x7f318819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:43.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.770+0000 7f3186800640 1 --2- 172.21.15.111:0/383926873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31881047e0 0x7f318819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:43.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.770+0000 7f3186800640 1 -- 172.21.15.111:0/383926873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3188103e30 msgr2=0x7f318819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:43.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.770+0000 7f3186800640 1 --2- 172.21.15.111:0/383926873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3188103e30 0x7f318819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:43.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.770+0000 7f3186800640 1 -- 172.21.15.111:0/383926873 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318819f440 con 0x7f3188108f60 2024-09-15T22:45:43.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.770+0000 7f3185400640 1 --2- 172.21.15.111:0/383926873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31881047e0 0x7f318819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:45:43.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.771+0000 7f3186800640 1 --2- 172.21.15.111:0/383926873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3188108f60 0x7f318819ed60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3180009fd0 tx=0x7f318000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:43.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.771+0000 7f317ea00640 1 -- 172.21.15.111:0/383926873 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318000ce20 con 0x7f3188108f60 2024-09-15T22:45:43.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.771+0000 7f3187200640 1 -- 172.21.15.111:0/383926873 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f318819f730 con 0x7f3188108f60 2024-09-15T22:45:43.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.771+0000 7f317ea00640 1 -- 172.21.15.111:0/383926873 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3180005020 con 0x7f3188108f60 2024-09-15T22:45:43.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.771+0000 7f317ea00640 1 -- 172.21.15.111:0/383926873 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3180010720 con 0x7f3188108f60 2024-09-15T22:45:43.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.771+0000 7f3187200640 1 -- 172.21.15.111:0/383926873 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31881a7010 con 0x7f3188108f60 2024-09-15T22:45:43.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.772+0000 7f3187200640 1 -- 172.21.15.111:0/383926873 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3188104210 con 0x7f3188108f60 2024-09-15T22:45:43.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.774+0000 7f317ea00640 1 -- 172.21.15.111:0/383926873 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f3180005540 con 0x7f3188108f60 2024-09-15T22:45:43.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.776+0000 7f317ea00640 1 --2- 172.21.15.111:0/383926873 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f316c0a32d0 0x7f316c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:43.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.776+0000 7f317ea00640 1 -- 172.21.15.111:0/383926873 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f31800d02e0 con 0x7f3188108f60 2024-09-15T22:45:43.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.776+0000 7f3185e00640 1 --2- 172.21.15.111:0/383926873 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f316c0a32d0 0x7f316c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:43.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.776+0000 7f3185e00640 1 --2- 172.21.15.111:0/383926873 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f316c0a32d0 0x7f316c0a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f31780096c0 tx=0x7f31780046c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:43.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.777+0000 7f317ea00640 1 -- 172.21.15.111:0/383926873 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f318009cea0 con 0x7f3188108f60 2024-09-15T22:45:43.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:43 smithi121 ceph-mon[34815]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:45:43.986 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.985+0000 7f3187200640 1 -- 172.21.15.111:0/383926873 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f318802d7c0 con 0x7f3188108f60 2024-09-15T22:45:43.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.986+0000 7f317ea00640 1 -- 172.21.15.111:0/383926873 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11931 (secure 0 0 0) 0x7f318009c7a0 con 0x7f3188108f60 2024-09-15T22:45:43.988 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:45:43.988 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":25,"btime":"2024-09-15T22:44:34:567598+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/453336603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":453336603},{"type":"v1","addr":"172.21.15.111:6835","nonce":453336603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/3181832287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3181832287},{"type":"v1","addr":"172.21.15.111:6837","nonce":3181832287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/4190237053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":4190237053},{"type":"v1","addr":"172.21.15.111:6839","nonce":4190237053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14502,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6841/267728828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6840","nonce":267728828},{"type":"v1","addr":"172.21.15.111:6841","nonce":267728828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/4061441359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":4061441359},{"type":"v1","addr":"172.21.15.192:6833","nonce":4061441359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/4220531578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":4220531578},{"type":"v1","addr":"172.21.15.121:6833","nonce":4220531578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:44:18.436394+0000","modified":"2024-09-15T22:44:33.655164+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24514,"mds_2":24503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3371796806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3371796806},{"type":"v1","addr":"172.21.15.121:6835","nonce":3371796806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/4080914921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":4080914921},{"type":"v1","addr":"172.21.15.192:6835","nonce":4080914921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/3995501707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3995501707},{"type":"v1","addr":"172.21.15.192:6837","nonce":3995501707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.121:6837/1443772791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1443772791},{"type":"v1","addr":"172.21.15.121:6837","nonce":1443772791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6839/800136261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":800136261},{"type":"v1","addr":"172.21.15.192:6839","nonce":800136261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.121:6839/1797352097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1797352097},{"type":"v1","addr":"172.21.15.121:6839","nonce":1797352097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545,24514,24503]},"id":1}]} 2024-09-15T22:45:43.989 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 25 2024-09-15T22:45:43.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.988+0000 7f3187200640 1 -- 172.21.15.111:0/383926873 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f316c0a32d0 msgr2=0x7f316c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:43.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.988+0000 7f3187200640 1 --2- 172.21.15.111:0/383926873 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f316c0a32d0 0x7f316c0a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f31780096c0 tx=0x7f31780046c0 comp rx=0 tx=0).stop 2024-09-15T22:45:43.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.988+0000 7f3187200640 1 -- 172.21.15.111:0/383926873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3188108f60 msgr2=0x7f318819ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:43.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.988+0000 7f3187200640 1 --2- 172.21.15.111:0/383926873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3188108f60 0x7f318819ed60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3180009fd0 tx=0x7f318000ef10 comp rx=0 tx=0).stop 2024-09-15T22:45:43.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.988+0000 7f3187200640 1 -- 172.21.15.111:0/383926873 shutdown_connections 2024-09-15T22:45:43.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.988+0000 7f3187200640 1 --2- 172.21.15.111:0/383926873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3188108f60 0x7f318819ed60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:43.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.988+0000 7f3187200640 1 --2- 172.21.15.111:0/383926873 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f316c0a32d0 0x7f316c0a5790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:43.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.988+0000 7f3187200640 1 --2- 172.21.15.111:0/383926873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31881047e0 0x7f318819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:43.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.988+0000 7f3187200640 1 --2- 172.21.15.111:0/383926873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3188103e30 0x7f318819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:43.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.988+0000 7f3187200640 1 -- 172.21.15.111:0/383926873 >> 172.21.15.111:0/383926873 conn(0x7f31880ff8d0 msgr2=0x7f3188100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:43.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.989+0000 7f3187200640 1 -- 172.21.15.111:0/383926873 shutdown_connections 2024-09-15T22:45:43.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:43.989+0000 7f3187200640 1 -- 172.21.15.111:0/383926873 wait complete. 2024-09-15T22:45:44.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:43 smithi192 ceph-mon[35427]: pgmap v344: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:45:44.857 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:45:44.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:44 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/383926873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:45:44.908 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:44 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/383926873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:45:45.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:44 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/383926873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:45:45.237 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:45.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:45 smithi111 ceph-mon[30629]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:45:45.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.793+0000 7ff288400640 1 -- 172.21.15.111:0/1067797644 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff280074eb0 msgr2=0x7ff280112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:45.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.793+0000 7ff288400640 1 --2- 172.21.15.111:0/1067797644 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff280074eb0 0x7ff280112650 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009900 tx=0x7ff26c02a480 comp rx=0 tx=0).stop 2024-09-15T22:45:45.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.793+0000 7ff288400640 1 -- 172.21.15.111:0/1067797644 shutdown_connections 2024-09-15T22:45:45.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.793+0000 7ff288400640 1 --2- 172.21.15.111:0/1067797644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff280112b90 0x7ff280114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:45.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.793+0000 7ff288400640 1 --2- 172.21.15.111:0/1067797644 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff280074eb0 0x7ff280112650 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:45.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.793+0000 7ff288400640 1 --2- 172.21.15.111:0/1067797644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff280074590 0x7ff280074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:45.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.793+0000 7ff288400640 1 -- 172.21.15.111:0/1067797644 >> 172.21.15.111:0/1067797644 conn(0x7ff2800fd8c0 msgr2=0x7ff2800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:45.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.793+0000 7ff288400640 1 -- 172.21.15.111:0/1067797644 shutdown_connections 2024-09-15T22:45:45.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.793+0000 7ff288400640 1 -- 172.21.15.111:0/1067797644 wait complete. 2024-09-15T22:45:45.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.794+0000 7ff288400640 1 Processor -- start 2024-09-15T22:45:45.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.794+0000 7ff288400640 1 -- start start 2024-09-15T22:45:45.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.794+0000 7ff288400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff280074590 0x7ff2801a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:45.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.794+0000 7ff288400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff280074eb0 0x7ff2801a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:45.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.794+0000 7ff288400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff280112b90 0x7ff28019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:45.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.794+0000 7ff288400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff280118590 con 0x7ff280074590 2024-09-15T22:45:45.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.794+0000 7ff288400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff280118410 con 0x7ff280074eb0 2024-09-15T22:45:45.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.794+0000 7ff288400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff280118710 con 0x7ff280112b90 2024-09-15T22:45:45.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.794+0000 7ff285a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff280074590 0x7ff2801a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:45.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.794+0000 7ff285a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff280074590 0x7ff2801a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45650/0 (socket says 172.21.15.111:45650) 2024-09-15T22:45:45.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.794+0000 7ff285a00640 1 -- 172.21.15.111:0/596279271 learned_addr learned my addr 172.21.15.111:0/596279271 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:45.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.795+0000 7ff285000640 1 --2- 172.21.15.111:0/596279271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff280074eb0 0x7ff2801a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:45.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.795+0000 7ff286400640 1 --2- 172.21.15.111:0/596279271 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff280112b90 0x7ff28019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:45.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.795+0000 7ff285a00640 1 -- 172.21.15.111:0/596279271 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff280112b90 msgr2=0x7ff28019e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:45.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.795+0000 7ff285a00640 1 --2- 172.21.15.111:0/596279271 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff280112b90 0x7ff28019e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:45.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.795+0000 7ff285a00640 1 -- 172.21.15.111:0/596279271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff280074eb0 msgr2=0x7ff2801a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:45.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.795+0000 7ff285a00640 1 --2- 172.21.15.111:0/596279271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff280074eb0 0x7ff2801a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:45.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.795+0000 7ff285a00640 1 -- 172.21.15.111:0/596279271 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28019f000 con 0x7ff280074590 2024-09-15T22:45:45.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.795+0000 7ff285a00640 1 --2- 172.21.15.111:0/596279271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff280074590 0x7ff2801a4710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff27400e9f0 tx=0x7ff27400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:45.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.795+0000 7ff27e000640 1 -- 172.21.15.111:0/596279271 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27400cd80 con 0x7ff280074590 2024-09-15T22:45:45.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.795+0000 7ff27e000640 1 -- 172.21.15.111:0/596279271 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff274010910 con 0x7ff280074590 2024-09-15T22:45:45.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.795+0000 7ff288400640 1 -- 172.21.15.111:0/596279271 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff28019f2f0 con 0x7ff280074590 2024-09-15T22:45:45.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.795+0000 7ff27e000640 1 -- 172.21.15.111:0/596279271 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff274018990 con 0x7ff280074590 2024-09-15T22:45:45.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.796+0000 7ff288400640 1 -- 172.21.15.111:0/596279271 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2801ab5e0 con 0x7ff280074590 2024-09-15T22:45:45.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.797+0000 7ff288400640 1 -- 172.21.15.111:0/596279271 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff280074970 con 0x7ff280074590 2024-09-15T22:45:45.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.799+0000 7ff27e000640 1 -- 172.21.15.111:0/596279271 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7ff274010ac0 con 0x7ff280074590 2024-09-15T22:45:45.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.801+0000 7ff27e000640 1 --2- 172.21.15.111:0/596279271 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ff2680a3350 0x7ff2680a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:45.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.801+0000 7ff285000640 1 --2- 172.21.15.111:0/596279271 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ff2680a3350 0x7ff2680a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:45.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.801+0000 7ff27e000640 1 -- 172.21.15.111:0/596279271 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7ff2740cfd10 con 0x7ff280074590 2024-09-15T22:45:45.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.801+0000 7ff285000640 1 --2- 172.21.15.111:0/596279271 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ff2680a3350 0x7ff2680a5810 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff26c000c00 tx=0x7ff26c005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:45.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.805+0000 7ff27e000640 1 -- 172.21.15.111:0/596279271 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2740d5020 con 0x7ff280074590 2024-09-15T22:45:45.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:45 smithi121 ceph-mon[34815]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:45:45.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.982+0000 7ff288400640 1 -- 172.21.15.111:0/596279271 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff280104a40 con 0x7ff280074590 2024-09-15T22:45:45.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.983+0000 7ff27e000640 1 -- 172.21.15.111:0/596279271 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22701 (secure 0 0 0) 0x7ff27409c280 con 0x7ff280074590 2024-09-15T22:45:45.985 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:45:45.985 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":88,"fsid":"b7614d3c-73b2-11ef-bceb-c7b262605968","created":"2024-09-15T22:35:52.571473+0000","modified":"2024-09-15T22:44:51.627977+0000","last_up_change":"2024-09-15T22:43:19.567274+0000","last_in_change":"2024-09-15T22:42:56.994246+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:39:18.954207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:44:12.846418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:44:15.685946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04f39604-be0e-4261-b316-67fa087c6666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6803","nonce":1386823446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6805","nonce":1386823446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6809","nonce":1386823446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6807","nonce":1386823446}]},"public_addr":"172.21.15.111:6803/1386823446","cluster_addr":"172.21.15.111:6805/1386823446","heartbeat_back_addr":"172.21.15.111:6809/1386823446","heartbeat_front_addr":"172.21.15.111:6807/1386823446","state":["exists","up"]},{"osd":1,"uuid":"1bdbef6e-23fa-4618-a788-66e57741a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6801","nonce":2051512291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6803","nonce":2051512291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6807","nonce":2051512291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6805","nonce":2051512291}]},"public_addr":"172.21.15.121:6801/2051512291","cluster_addr":"172.21.15.121:6803/2051512291","heartbeat_back_addr":"172.21.15.121:6807/2051512291","heartbeat_front_addr":"172.21.15.121:6805/2051512291","state":["exists","up"]},{"osd":2,"uuid":"46735ccf-6d0e-4154-bb55-422929819bfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6801","nonce":93085639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6803","nonce":93085639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6807","nonce":93085639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6805","nonce":93085639}]},"public_addr":"172.21.15.192:6801/93085639","cluster_addr":"172.21.15.192:6803/93085639","heartbeat_back_addr":"172.21.15.192:6807/93085639","heartbeat_front_addr":"172.21.15.192:6805/93085639","state":["exists","up"]},{"osd":3,"uuid":"4b243af8-c958-40b1-ba24-9c7ace00c4a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6811","nonce":2578259041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6813","nonce":2578259041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6817","nonce":2578259041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6815","nonce":2578259041}]},"public_addr":"172.21.15.111:6811/2578259041","cluster_addr":"172.21.15.111:6813/2578259041","heartbeat_back_addr":"172.21.15.111:6817/2578259041","heartbeat_front_addr":"172.21.15.111:6815/2578259041","state":["exists","up"]},{"osd":4,"uuid":"49f27541-9cbf-48e0-a466-6bb0fa5410b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6809","nonce":2667595915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6811","nonce":2667595915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6815","nonce":2667595915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6813","nonce":2667595915}]},"public_addr":"172.21.15.121:6809/2667595915","cluster_addr":"172.21.15.121:6811/2667595915","heartbeat_back_addr":"172.21.15.121:6815/2667595915","heartbeat_front_addr":"172.21.15.121:6813/2667595915","state":["exists","up"]},{"osd":5,"uuid":"9290e769-5470-44d3-8d24-25093af34335","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6809","nonce":1958639856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6811","nonce":1958639856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6815","nonce":1958639856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6813","nonce":1958639856}]},"public_addr":"172.21.15.192:6809/1958639856","cluster_addr":"172.21.15.192:6811/1958639856","heartbeat_back_addr":"172.21.15.192:6815/1958639856","heartbeat_front_addr":"172.21.15.192:6813/1958639856","state":["exists","up"]},{"osd":6,"uuid":"18911b97-1c68-4e8f-92ec-35d2ff0b5f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6819","nonce":1371543706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6821","nonce":1371543706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6825","nonce":1371543706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6823","nonce":1371543706}]},"public_addr":"172.21.15.111:6819/1371543706","cluster_addr":"172.21.15.111:6821/1371543706","heartbeat_back_addr":"172.21.15.111:6825/1371543706","heartbeat_front_addr":"172.21.15.111:6823/1371543706","state":["exists","up"]},{"osd":7,"uuid":"470647c5-c00d-426b-a134-cca235902307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6817","nonce":3501842998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6819","nonce":3501842998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6823","nonce":3501842998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6821","nonce":3501842998}]},"public_addr":"172.21.15.121:6817/3501842998","cluster_addr":"172.21.15.121:6819/3501842998","heartbeat_back_addr":"172.21.15.121:6823/3501842998","heartbeat_front_addr":"172.21.15.121:6821/3501842998","state":["exists","up"]},{"osd":8,"uuid":"fa2d3dd5-a768-4ada-a036-4c596b76d654","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6817","nonce":2417771374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6819","nonce":2417771374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6823","nonce":2417771374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6821","nonce":2417771374}]},"public_addr":"172.21.15.192:6817/2417771374","cluster_addr":"172.21.15.192:6819/2417771374","heartbeat_back_addr":"172.21.15.192:6823/2417771374","heartbeat_front_addr":"172.21.15.192:6821/2417771374","state":["exists","up"]},{"osd":9,"uuid":"71b4f5fb-1c02-461b-a257-c6e48381d768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6827","nonce":3979119431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6829","nonce":3979119431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6833","nonce":3979119431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6831","nonce":3979119431}]},"public_addr":"172.21.15.111:6827/3979119431","cluster_addr":"172.21.15.111:6829/3979119431","heartbeat_back_addr":"172.21.15.111:6833/3979119431","heartbeat_front_addr":"172.21.15.111:6831/3979119431","state":["exists","up"]},{"osd":10,"uuid":"e6a39e77-fce3-4e22-bf66-a677c9f8b6bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6825","nonce":2897902637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6827","nonce":2897902637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6831","nonce":2897902637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6829","nonce":2897902637}]},"public_addr":"172.21.15.121:6825/2897902637","cluster_addr":"172.21.15.121:6827/2897902637","heartbeat_back_addr":"172.21.15.121:6831/2897902637","heartbeat_front_addr":"172.21.15.121:6829/2897902637","state":["exists","up"]},{"osd":11,"uuid":"411a7921-6717-4cef-b8df-d9e252edcc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6825","nonce":2431464812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6827","nonce":2431464812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6831","nonce":2431464812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6829","nonce":2431464812}]},"public_addr":"172.21.15.192:6825/2431464812","cluster_addr":"172.21.15.192:6827/2431464812","heartbeat_back_addr":"172.21.15.192:6831/2431464812","heartbeat_front_addr":"172.21.15.192:6829/2431464812","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:38:23.442715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:38:49.501124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:39:14.759077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:39:45.121397+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:40:11.137612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:40:36.278567+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:07.027190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:32.439215+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:57.058702+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:42:28.741846+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:42:53.395442+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:43:18.265067+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:6801/2107775445":"2024-09-16T22:36:50.357551+0000","172.21.15.111:6800/2107775445":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/2394746540":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/4140020843":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/2114354263":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/1054173642":"2024-09-16T22:36:22.652875+0000","172.21.15.111:0/3408238004":"2024-09-16T22:36:22.652875+0000","172.21.15.111:0/2879044772":"2024-09-16T22:36:22.652875+0000","172.21.15.111:6801/3350425104":"2024-09-16T22:36:22.652875+0000","172.21.15.111:6800/3350425104":"2024-09-16T22:36:22.652875+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:45:45.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.986+0000 7ff288400640 1 -- 172.21.15.111:0/596279271 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ff2680a3350 msgr2=0x7ff2680a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:45.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.986+0000 7ff288400640 1 --2- 172.21.15.111:0/596279271 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ff2680a3350 0x7ff2680a5810 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff26c000c00 tx=0x7ff26c005b50 comp rx=0 tx=0).stop 2024-09-15T22:45:45.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.986+0000 7ff288400640 1 -- 172.21.15.111:0/596279271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff280074590 msgr2=0x7ff2801a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:45.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.986+0000 7ff288400640 1 --2- 172.21.15.111:0/596279271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff280074590 0x7ff2801a4710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff27400e9f0 tx=0x7ff27400eeb0 comp rx=0 tx=0).stop 2024-09-15T22:45:45.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.986+0000 7ff288400640 1 -- 172.21.15.111:0/596279271 shutdown_connections 2024-09-15T22:45:45.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.986+0000 7ff288400640 1 --2- 172.21.15.111:0/596279271 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff280112b90 0x7ff28019e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:45.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.986+0000 7ff288400640 1 --2- 172.21.15.111:0/596279271 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ff2680a3350 0x7ff2680a5810 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:45.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.986+0000 7ff288400640 1 --2- 172.21.15.111:0/596279271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff280074eb0 0x7ff2801a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:45.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.986+0000 7ff288400640 1 --2- 172.21.15.111:0/596279271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff280074590 0x7ff2801a4710 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:45.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.986+0000 7ff288400640 1 -- 172.21.15.111:0/596279271 >> 172.21.15.111:0/596279271 conn(0x7ff2800fd8c0 msgr2=0x7ff2800ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:45.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.986+0000 7ff288400640 1 -- 172.21.15.111:0/596279271 shutdown_connections 2024-09-15T22:45:45.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:45.986+0000 7ff288400640 1 -- 172.21.15.111:0/596279271 wait complete. 2024-09-15T22:45:46.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:45 smithi192 ceph-mon[35427]: pgmap v345: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:45:46.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:46 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/596279271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:45:46.822 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:45:46.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:46 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/596279271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:45:47.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:46 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/596279271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:45:47.188 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:47.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.769+0000 7fbb22600640 1 -- 172.21.15.111:0/1647740931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb1c076720 msgr2=0x7fbb1c076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:47.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.769+0000 7fbb22600640 1 --2- 172.21.15.111:0/1647740931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb1c076720 0x7fbb1c076b00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbb10009900 tx=0x7fbb1002a410 comp rx=0 tx=0).stop 2024-09-15T22:45:47.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.769+0000 7fbb22600640 1 -- 172.21.15.111:0/1647740931 shutdown_connections 2024-09-15T22:45:47.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.769+0000 7fbb22600640 1 --2- 172.21.15.111:0/1647740931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb1c112890 0x7fbb1c114c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.769+0000 7fbb22600640 1 --2- 172.21.15.111:0/1647740931 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb1c077040 0x7fbb1c112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.769+0000 7fbb22600640 1 --2- 172.21.15.111:0/1647740931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb1c076720 0x7fbb1c076b00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.769+0000 7fbb22600640 1 -- 172.21.15.111:0/1647740931 >> 172.21.15.111:0/1647740931 conn(0x7fbb1c0ff8b0 msgr2=0x7fbb1c101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:47.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.769+0000 7fbb22600640 1 -- 172.21.15.111:0/1647740931 shutdown_connections 2024-09-15T22:45:47.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.769+0000 7fbb22600640 1 -- 172.21.15.111:0/1647740931 wait complete. 2024-09-15T22:45:47.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.770+0000 7fbb22600640 1 Processor -- start 2024-09-15T22:45:47.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.770+0000 7fbb22600640 1 -- start start 2024-09-15T22:45:47.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.770+0000 7fbb22600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb1c076720 0x7fbb1c1a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:47.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.770+0000 7fbb22600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb1c077040 0x7fbb1c1a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:47.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.770+0000 7fbb22600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb1c112890 0x7fbb1c19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:47.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.770+0000 7fbb22600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c118570 con 0x7fbb1c112890 2024-09-15T22:45:47.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.770+0000 7fbb22600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c1183f0 con 0x7fbb1c077040 2024-09-15T22:45:47.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.770+0000 7fbb22600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c1186f0 con 0x7fbb1c076720 2024-09-15T22:45:47.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.771+0000 7fbb21200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb1c076720 0x7fbb1c1a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:47.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.771+0000 7fbb21200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb1c076720 0x7fbb1c1a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:50802/0 (socket says 172.21.15.111:50802) 2024-09-15T22:45:47.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.771+0000 7fbb21200640 1 -- 172.21.15.111:0/1708072093 learned_addr learned my addr 172.21.15.111:0/1708072093 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:47.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.771+0000 7fbb21c00640 1 --2- 172.21.15.111:0/1708072093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb1c112890 0x7fbb1c19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:47.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.771+0000 7fbb1be00640 1 --2- 172.21.15.111:0/1708072093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb1c077040 0x7fbb1c1a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:47.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.771+0000 7fbb21c00640 1 -- 172.21.15.111:0/1708072093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb1c076720 msgr2=0x7fbb1c1a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:47.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.771+0000 7fbb21c00640 1 --2- 172.21.15.111:0/1708072093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb1c076720 0x7fbb1c1a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.771+0000 7fbb21c00640 1 -- 172.21.15.111:0/1708072093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb1c077040 msgr2=0x7fbb1c1a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:47.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.771+0000 7fbb21c00640 1 --2- 172.21.15.111:0/1708072093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb1c077040 0x7fbb1c1a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.771+0000 7fbb21c00640 1 -- 172.21.15.111:0/1708072093 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1c19efe0 con 0x7fbb1c112890 2024-09-15T22:45:47.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.771+0000 7fbb21200640 1 --2- 172.21.15.111:0/1708072093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb1c076720 0x7fbb1c1a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:47.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.771+0000 7fbb21c00640 1 --2- 172.21.15.111:0/1708072093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb1c112890 0x7fbb1c19e840 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c07aa10 tx=0x7fbb1400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:47.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.772+0000 7fbb19600640 1 -- 172.21.15.111:0/1708072093 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1400cdf0 con 0x7fbb1c112890 2024-09-15T22:45:47.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.772+0000 7fbb19600640 1 -- 172.21.15.111:0/1708072093 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb14005020 con 0x7fbb1c112890 2024-09-15T22:45:47.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.772+0000 7fbb19600640 1 -- 172.21.15.111:0/1708072093 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14010670 con 0x7fbb1c112890 2024-09-15T22:45:47.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.772+0000 7fbb22600640 1 -- 172.21.15.111:0/1708072093 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb1c19f2d0 con 0x7fbb1c112890 2024-09-15T22:45:47.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.772+0000 7fbb22600640 1 -- 172.21.15.111:0/1708072093 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb1c1ab520 con 0x7fbb1c112890 2024-09-15T22:45:47.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.773+0000 7fbb22600640 1 -- 172.21.15.111:0/1708072093 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb1c077e60 con 0x7fbb1c112890 2024-09-15T22:45:47.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.775+0000 7fbb19600640 1 -- 172.21.15.111:0/1708072093 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7fbb140054d0 con 0x7fbb1c112890 2024-09-15T22:45:47.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.776+0000 7fbb19600640 1 --2- 172.21.15.111:0/1708072093 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fbb040a3350 0x7fbb040a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:47.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.776+0000 7fbb19600640 1 -- 172.21.15.111:0/1708072093 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fbb140d0060 con 0x7fbb1c112890 2024-09-15T22:45:47.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.776+0000 7fbb21200640 1 --2- 172.21.15.111:0/1708072093 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fbb040a3350 0x7fbb040a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:47.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.777+0000 7fbb21200640 1 --2- 172.21.15.111:0/1708072093 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fbb040a3350 0x7fbb040a5810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbb100096c0 tx=0x7fbb100046c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:47.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.780+0000 7fbb19600640 1 -- 172.21.15.111:0/1708072093 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbb1409cc20 con 0x7fbb1c112890 2024-09-15T22:45:47.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:47 smithi121 ceph-mon[34815]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:45:47.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:47 smithi111 ceph-mon[30629]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:45:47.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.993+0000 7fbb22600640 1 -- 172.21.15.111:0/1708072093 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb1c1ab750 con 0x7fbb1c112890 2024-09-15T22:45:47.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.994+0000 7fbb19600640 1 -- 172.21.15.111:0/1708072093 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11931 (secure 0 0 0) 0x7fbb1409c520 con 0x7fbb1c112890 2024-09-15T22:45:47.996 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:45:47.996 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":25,"btime":"2024-09-15T22:44:34:567598+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/453336603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":453336603},{"type":"v1","addr":"172.21.15.111:6835","nonce":453336603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/3181832287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3181832287},{"type":"v1","addr":"172.21.15.111:6837","nonce":3181832287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/4190237053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":4190237053},{"type":"v1","addr":"172.21.15.111:6839","nonce":4190237053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14502,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6841/267728828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6840","nonce":267728828},{"type":"v1","addr":"172.21.15.111:6841","nonce":267728828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/4061441359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":4061441359},{"type":"v1","addr":"172.21.15.192:6833","nonce":4061441359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/4220531578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":4220531578},{"type":"v1","addr":"172.21.15.121:6833","nonce":4220531578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:44:18.436394+0000","modified":"2024-09-15T22:44:33.655164+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24514,"mds_2":24503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3371796806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3371796806},{"type":"v1","addr":"172.21.15.121:6835","nonce":3371796806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/4080914921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":4080914921},{"type":"v1","addr":"172.21.15.192:6835","nonce":4080914921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/3995501707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3995501707},{"type":"v1","addr":"172.21.15.192:6837","nonce":3995501707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.121:6837/1443772791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1443772791},{"type":"v1","addr":"172.21.15.121:6837","nonce":1443772791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6839/800136261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":800136261},{"type":"v1","addr":"172.21.15.192:6839","nonce":800136261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.121:6839/1797352097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1797352097},{"type":"v1","addr":"172.21.15.121:6839","nonce":1797352097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545,24514,24503]},"id":1}]} 2024-09-15T22:45:47.997 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 25 2024-09-15T22:45:47.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.996+0000 7fbb22600640 1 -- 172.21.15.111:0/1708072093 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fbb040a3350 msgr2=0x7fbb040a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:47.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.996+0000 7fbb22600640 1 --2- 172.21.15.111:0/1708072093 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fbb040a3350 0x7fbb040a5810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbb100096c0 tx=0x7fbb100046c0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.996+0000 7fbb22600640 1 -- 172.21.15.111:0/1708072093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb1c112890 msgr2=0x7fbb1c19e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:47.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.996+0000 7fbb22600640 1 --2- 172.21.15.111:0/1708072093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb1c112890 0x7fbb1c19e840 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c07aa10 tx=0x7fbb1400ef10 comp rx=0 tx=0).stop 2024-09-15T22:45:47.998 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.996+0000 7fbb22600640 1 -- 172.21.15.111:0/1708072093 shutdown_connections 2024-09-15T22:45:47.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.996+0000 7fbb22600640 1 --2- 172.21.15.111:0/1708072093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb1c112890 0x7fbb1c19e840 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.996+0000 7fbb22600640 1 --2- 172.21.15.111:0/1708072093 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fbb040a3350 0x7fbb040a5810 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.996+0000 7fbb22600640 1 --2- 172.21.15.111:0/1708072093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbb1c077040 0x7fbb1c1a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.996+0000 7fbb22600640 1 --2- 172.21.15.111:0/1708072093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb1c076720 0x7fbb1c1a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:47.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.996+0000 7fbb22600640 1 -- 172.21.15.111:0/1708072093 >> 172.21.15.111:0/1708072093 conn(0x7fbb1c0ff8b0 msgr2=0x7fbb1c101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:47.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.997+0000 7fbb22600640 1 -- 172.21.15.111:0/1708072093 shutdown_connections 2024-09-15T22:45:47.999 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:47.997+0000 7fbb22600640 1 -- 172.21.15.111:0/1708072093 wait complete. 2024-09-15T22:45:48.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:47 smithi192 ceph-mon[35427]: pgmap v346: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:45:48.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:48 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/1708072093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:45:48.829 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-15T22:44:18.436394+0000', 'modified': '2024-09-15T22:44:33.655164+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24545, 'mds_1': 24514, 'mds_2': 24503}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24485': {'gid': 24485, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.121:6835/3371796806', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 3371796806}, {'type': 'v1', 'addr': '172.21.15.121:6835', 'nonce': 3371796806}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.192:6835/4080914921', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6834', 'nonce': 4080914921}, {'type': 'v1', 'addr': '172.21.15.192:6835', 'nonce': 4080914921}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.192:6837/3995501707', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6836', 'nonce': 3995501707}, {'type': 'v1', 'addr': '172.21.15.192:6837', 'nonce': 3995501707}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24503': {'gid': 24503, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.121:6837/1443772791', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6836', 'nonce': 1443772791}, {'type': 'v1', 'addr': '172.21.15.121:6837', 'nonce': 1443772791}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24514': {'gid': 24514, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.192:6839/800136261', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6838', 'nonce': 800136261}, {'type': 'v1', 'addr': '172.21.15.192:6839', 'nonce': 800136261}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24545': {'gid': 24545, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.121:6839/1797352097', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6838', 'nonce': 1797352097}, {'type': 'v1', 'addr': '172.21.15.121:6839', 'nonce': 1797352097}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24545, 'qdb_cluster': [24545, 24514, 24503]} 2024-09-15T22:45:48.829 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-15T22:45:48.829 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-15T22:45:48.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:48 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/1708072093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:45:49.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:48 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/1708072093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:45:49.196 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:49.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.753+0000 7f953fe00640 1 -- 172.21.15.111:0/2186037676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9538101de0 msgr2=0x7f95381021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:49.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.753+0000 7f953fe00640 1 --2- 172.21.15.111:0/2186037676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9538101de0 0x7f95381021c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f952c009900 tx=0x7f952c02a410 comp rx=0 tx=0).stop 2024-09-15T22:45:49.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.753+0000 7f953fe00640 1 -- 172.21.15.111:0/2186037676 shutdown_connections 2024-09-15T22:45:49.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.753+0000 7f953fe00640 1 --2- 172.21.15.111:0/2186037676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9538106f10 0x7f953810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.753+0000 7f953fe00640 1 --2- 172.21.15.111:0/2186037676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9538102790 0x7f95381067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.753+0000 7f953fe00640 1 --2- 172.21.15.111:0/2186037676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9538101de0 0x7f95381021c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.753+0000 7f953fe00640 1 -- 172.21.15.111:0/2186037676 >> 172.21.15.111:0/2186037676 conn(0x7f95380fd8c0 msgr2=0x7f95380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:49.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.753+0000 7f953fe00640 1 -- 172.21.15.111:0/2186037676 shutdown_connections 2024-09-15T22:45:49.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.753+0000 7f953fe00640 1 -- 172.21.15.111:0/2186037676 wait complete. 2024-09-15T22:45:49.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.754+0000 7f953fe00640 1 Processor -- start 2024-09-15T22:45:49.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.754+0000 7f953fe00640 1 -- start start 2024-09-15T22:45:49.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.754+0000 7f953fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9538101de0 0x7f9538198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:49.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.755+0000 7f953fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9538102790 0x7f9538198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:49.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.755+0000 7f953fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9538106f10 0x7f953819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:49.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.755+0000 7f953fe00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9538111fc0 con 0x7f9538101de0 2024-09-15T22:45:49.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.755+0000 7f953fe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9538111e40 con 0x7f9538102790 2024-09-15T22:45:49.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.755+0000 7f953fe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9538112140 con 0x7f9538106f10 2024-09-15T22:45:49.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.755+0000 7f953d400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9538101de0 0x7f9538198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:49.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.755+0000 7f953d400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9538101de0 0x7f9538198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45686/0 (socket says 172.21.15.111:45686) 2024-09-15T22:45:49.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.755+0000 7f953d400640 1 -- 172.21.15.111:0/2513375829 learned_addr learned my addr 172.21.15.111:0/2513375829 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:49.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.755+0000 7f953de00640 1 --2- 172.21.15.111:0/2513375829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9538106f10 0x7f953819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:49.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.755+0000 7f953ca00640 1 --2- 172.21.15.111:0/2513375829 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9538102790 0x7f9538198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:49.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.755+0000 7f953d400640 1 -- 172.21.15.111:0/2513375829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9538106f10 msgr2=0x7f953819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:49.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.755+0000 7f953d400640 1 --2- 172.21.15.111:0/2513375829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9538106f10 0x7f953819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.755+0000 7f953d400640 1 -- 172.21.15.111:0/2513375829 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9538102790 msgr2=0x7f9538198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:49.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.755+0000 7f953d400640 1 --2- 172.21.15.111:0/2513375829 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9538102790 0x7f9538198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.755+0000 7f953d400640 1 -- 172.21.15.111:0/2513375829 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f953819d3f0 con 0x7f9538101de0 2024-09-15T22:45:49.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.756+0000 7f953d400640 1 --2- 172.21.15.111:0/2513375829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9538101de0 0x7f9538198440 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f952c004770 tx=0x7f952c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:49.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.756+0000 7f9536000640 1 -- 172.21.15.111:0/2513375829 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c03f070 con 0x7f9538101de0 2024-09-15T22:45:49.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.756+0000 7f9536000640 1 -- 172.21.15.111:0/2513375829 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f952c031ca0 con 0x7f9538101de0 2024-09-15T22:45:49.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.756+0000 7f953fe00640 1 -- 172.21.15.111:0/2513375829 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f953819d680 con 0x7f9538101de0 2024-09-15T22:45:49.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.756+0000 7f9536000640 1 -- 172.21.15.111:0/2513375829 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952c035030 con 0x7f9538101de0 2024-09-15T22:45:49.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.756+0000 7f953fe00640 1 -- 172.21.15.111:0/2513375829 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95381a4fc0 con 0x7f9538101de0 2024-09-15T22:45:49.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.759+0000 7f9536000640 1 -- 172.21.15.111:0/2513375829 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) ==== 138407+0+0 (secure 0 0 0) 0x7f952c042070 con 0x7f9538101de0 2024-09-15T22:45:49.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.759+0000 7f95429cf740 1 Processor -- start 2024-09-15T22:45:49.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.760+0000 7f95429cf740 1 -- start start 2024-09-15T22:45:49.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.760+0000 7f95429cf740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56445f294cd0 0x56445f2950b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:49.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.760+0000 7f95429cf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56445f28f7e0 0x56445f28fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:49.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.760+0000 7f95429cf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56445f290160 0x56445f29bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:49.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.760+0000 7f95429cf740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56445f1f40b0 con 0x56445f290160 2024-09-15T22:45:49.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.760+0000 7f95429cf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56445f19bab0 con 0x56445f28f7e0 2024-09-15T22:45:49.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.760+0000 7f95429cf740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56445f1f0070 con 0x56445f294cd0 2024-09-15T22:45:49.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.760+0000 7f953de00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56445f294cd0 0x56445f2950b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:49.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.760+0000 7f953d400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56445f28f7e0 0x56445f28fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:49.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.760+0000 7f953de00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56445f294cd0 0x56445f2950b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:50832/0 (socket says 172.21.15.111:50832) 2024-09-15T22:45:49.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.760+0000 7f953de00640 1 -- 172.21.15.111:0/3711551832 learned_addr learned my addr 172.21.15.111:0/3711551832 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:49.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.761+0000 7f953ca00640 1 --2- 172.21.15.111:0/3711551832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56445f290160 0x56445f29bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:49.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.761+0000 7f953de00640 1 -- 172.21.15.111:0/3711551832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56445f28f7e0 msgr2=0x56445f28fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:49.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.761+0000 7f953de00640 1 --2- 172.21.15.111:0/3711551832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56445f28f7e0 0x56445f28fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.761+0000 7f953de00640 1 -- 172.21.15.111:0/3711551832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56445f290160 msgr2=0x56445f29bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:49.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.761+0000 7f953de00640 1 --2- 172.21.15.111:0/3711551832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56445f290160 0x56445f29bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.761+0000 7f953de00640 1 -- 172.21.15.111:0/3711551832 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56445f1a84a0 con 0x56445f294cd0 2024-09-15T22:45:49.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.761+0000 7f953de00640 1 --2- 172.21.15.111:0/3711551832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56445f294cd0 0x56445f2950b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9530009f70 tx=0x7f9530010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75b9074d363a85e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:49.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.761+0000 7f9536000640 1 --2- 172.21.15.111:0/2513375829 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f95200abb60 0x7f95200ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:49.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.761+0000 7f952a600640 1 -- 172.21.15.111:0/3711551832 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530016070 con 0x56445f294cd0 2024-09-15T22:45:49.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.761+0000 7f952a600640 1 -- 172.21.15.111:0/3711551832 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9530004d20 con 0x56445f294cd0 2024-09-15T22:45:49.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.761+0000 7f952a600640 1 -- 172.21.15.111:0/3711551832 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95300128f0 con 0x56445f294cd0 2024-09-15T22:45:49.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.761+0000 7f953d400640 1 --2- 172.21.15.111:0/2513375829 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f95200abb60 0x7f95200ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:49.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.762+0000 7f9536000640 1 -- 172.21.15.111:0/2513375829 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f952c0f00e0 con 0x7f9538101de0 2024-09-15T22:45:49.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.762+0000 7f95429cf740 1 -- 172.21.15.111:0/3711551832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56445f294cd0 msgr2=0x56445f2950b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:49.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.762+0000 7f95429cf740 1 --2- 172.21.15.111:0/3711551832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56445f294cd0 0x56445f2950b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9530009f70 tx=0x7f9530010800 comp rx=0 tx=0).stop 2024-09-15T22:45:49.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.762+0000 7f953d400640 1 --2- 172.21.15.111:0/2513375829 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f95200abb60 0x7f95200ae020 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x56445f28b0b0 tx=0x7f952c0f3c40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:49.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.762+0000 7f95429cf740 1 -- 172.21.15.111:0/3711551832 shutdown_connections 2024-09-15T22:45:49.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.762+0000 7f95429cf740 1 --2- 172.21.15.111:0/3711551832 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56445f290160 0x56445f29bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.762+0000 7f95429cf740 1 --2- 172.21.15.111:0/3711551832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56445f28f7e0 0x56445f28fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.762+0000 7f95429cf740 1 --2- 172.21.15.111:0/3711551832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56445f294cd0 0x56445f2950b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.762+0000 7f95429cf740 1 -- 172.21.15.111:0/3711551832 >> 172.21.15.111:0/3711551832 conn(0x56445f28a250 msgr2=0x56445f28a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:49.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.762+0000 7f95429cf740 1 -- 172.21.15.111:0/3711551832 shutdown_connections 2024-09-15T22:45:49.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.762+0000 7f95429cf740 1 -- 172.21.15.111:0/3711551832 wait complete. 2024-09-15T22:45:49.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.763+0000 7f95429cf740 1 -- start start 2024-09-15T22:45:49.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.763+0000 7f95429cf740 1 Processor -- start 2024-09-15T22:45:49.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.763+0000 7f95429cf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56445f28f7e0 0x56445f298fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:49.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.763+0000 7f95429cf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56445f290160 0x56445f299520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:49.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.763+0000 7f95429cf740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56445f294cd0 0x56445f299a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:49.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.763+0000 7f95429cf740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56445f29c1a0 con 0x56445f28f7e0 2024-09-15T22:45:49.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.763+0000 7f95429cf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56445f17c920 con 0x56445f290160 2024-09-15T22:45:49.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.763+0000 7f95429cf740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56445f29c320 con 0x56445f294cd0 2024-09-15T22:45:49.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.763+0000 7f95429cf740 10 client.0 _ll_register_callbacks cb 0x56445f1a08a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:49.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.764+0000 7f953de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56445f28f7e0 0x56445f298fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:49.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.764+0000 7f953de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56445f28f7e0 0x56445f298fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45698/0 (socket says 172.21.15.111:45698) 2024-09-15T22:45:49.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.764+0000 7f953de00640 1 -- 172.21.15.111:0/1754672941 learned_addr learned my addr 172.21.15.111:0/1754672941 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:49.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.764+0000 7f953ca00640 1 --2- 172.21.15.111:0/1754672941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56445f290160 0x56445f299520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:49.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.764+0000 7f953de00640 1 --2- 172.21.15.111:0/1754672941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56445f294cd0 0x56445f299a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:49.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.764+0000 7f953de00640 1 -- 172.21.15.111:0/1754672941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56445f294cd0 msgr2=0x56445f299a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:49.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.764+0000 7f953de00640 1 --2- 172.21.15.111:0/1754672941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56445f294cd0 0x56445f299a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.764+0000 7f953de00640 1 -- 172.21.15.111:0/1754672941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56445f290160 msgr2=0x56445f299520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:49.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.764+0000 7f953de00640 1 --2- 172.21.15.111:0/1754672941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56445f290160 0x56445f299520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.764+0000 7f953de00640 1 -- 172.21.15.111:0/1754672941 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56445ef652b0 con 0x56445f28f7e0 2024-09-15T22:45:49.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.764+0000 7f953ca00640 1 --2- 172.21.15.111:0/1754672941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56445f290160 0x56445f299520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:45:49.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.764+0000 7f953de00640 1 --2- 172.21.15.111:0/1754672941 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56445f28f7e0 0x56445f298fe0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9530005b40 tx=0x7f9530005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6e9df646d230fe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:49.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.764+0000 7f951ea00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:45:49.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.764+0000 7f951ea00640 1 -- 172.21.15.111:0/1754672941 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530016050 con 0x56445f28f7e0 2024-09-15T22:45:49.789 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.764+0000 7f95429cf740 1 -- 172.21.15.111:0/1754672941 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9530004d20 con 0x56445f28f7e0 2024-09-15T22:45:49.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.764+0000 7f951ea00640 1 -- 172.21.15.111:0/1754672941 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f9530005510 con 0x56445f28f7e0 2024-09-15T22:45:49.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.765+0000 7f951ea00640 1 -- 172.21.15.111:0/1754672941 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9530031050 con 0x56445f28f7e0 2024-09-15T22:45:49.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.765+0000 7f95429cf740 10 client.14856 fetch_fsmap learned FSMap version 25 2024-09-15T22:45:49.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.765+0000 7f95429cf740 1 -- 172.21.15.111:0/1754672941 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56445f2a7f30 con 0x56445f28f7e0 2024-09-15T22:45:49.790 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.765+0000 7f951ea00640 1 -- 172.21.15.111:0/1754672941 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9530004d20 con 0x56445f28f7e0 2024-09-15T22:45:49.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.765+0000 7f951ea00640 1 -- 172.21.15.111:0/1754672941 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f953000f040 con 0x56445f28f7e0 2024-09-15T22:45:49.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.765+0000 7f95429cf740 10 client.14856 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:45:49.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.765+0000 7f95429cf740 10 client.14856 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-15T22:45:49.791 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.765+0000 7f95429cf740 20 client.14856 populate_metadata read hostname 'smithi111' 2024-09-15T22:45:49.792 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.765+0000 7f95429cf740 1 --2- 172.21.15.111:0/1754672941 >> [v2:172.21.15.121:6836/1443772791,v1:172.21.15.121:6837/1443772791] conn(0x56445f2aa130 0x56445f2ab5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:49.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.765+0000 7f95429cf740 4 client.14856 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:45:49.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.765+0000 7f95429cf740 1 -- 172.21.15.111:0/1754672941 --> [v2:172.21.15.121:6836/1443772791,v1:172.21.15.121:6837/1443772791] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56445f201620 con 0x56445f2aa130 2024-09-15T22:45:49.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.766+0000 7f953ca00640 1 --2- 172.21.15.111:0/1754672941 >> [v2:172.21.15.121:6836/1443772791,v1:172.21.15.121:6837/1443772791] conn(0x56445f2aa130 0x56445f2ab5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:49.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.766+0000 7f953ca00640 1 --2- 172.21.15.111:0/1754672941 >> [v2:172.21.15.121:6836/1443772791,v1:172.21.15.121:6837/1443772791] conn(0x56445f2aa130 0x56445f2ab5a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ee09510b932d92e server_cookie=1c6e1ccf1092e14c in_seq=0 out_seq=0 2024-09-15T22:45:49.793 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.766+0000 7f951ea00640 10 client.14856 ms_handle_connect on v2:172.21.15.121:6836/1443772791 2024-09-15T22:45:49.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.767+0000 7f951ea00640 1 -- 172.21.15.111:0/1754672941 <== mds.2 v2:172.21.15.121:6836/1443772791 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56445f201620 con 0x56445f2aa130 2024-09-15T22:45:49.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.767+0000 7f951ea00640 10 client.14856 handle_command_reply: tid=0 2024-09-15T22:45:49.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.767+0000 7f95429cf740 1 client.14856 shutdown 2024-09-15T22:45:49.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.768+0000 7f95429cf740 1 -- 172.21.15.111:0/1754672941 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56445f28f7e0 msgr2=0x56445f298fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:49.794 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.768+0000 7f95429cf740 1 --2- 172.21.15.111:0/1754672941 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56445f28f7e0 0x56445f298fe0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9530005b40 tx=0x7f9530005970 comp rx=0 tx=0).stop 2024-09-15T22:45:49.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.768+0000 7f95429cf740 1 -- 172.21.15.111:0/1754672941 shutdown_connections 2024-09-15T22:45:49.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.768+0000 7f95429cf740 1 --2- 172.21.15.111:0/1754672941 >> [v2:172.21.15.121:6836/1443772791,v1:172.21.15.121:6837/1443772791] conn(0x56445f2aa130 0x56445f2ab5a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.768+0000 7f95429cf740 1 --2- 172.21.15.111:0/1754672941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56445f294cd0 0x56445f299a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.768+0000 7f95429cf740 1 --2- 172.21.15.111:0/1754672941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56445f290160 0x56445f299520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.768+0000 7f95429cf740 1 --2- 172.21.15.111:0/1754672941 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56445f28f7e0 0x56445f298fe0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.795 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.768+0000 7f951ea00640 0 client.14856 ms_handle_reset on v2:172.21.15.121:6836/1443772791 2024-09-15T22:45:49.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.768+0000 7f95429cf740 1 -- 172.21.15.111:0/1754672941 >> 172.21.15.111:0/1754672941 conn(0x56445f28a250 msgr2=0x56445f289a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:49.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.768+0000 7f95429cf740 1 -- 172.21.15.111:0/1754672941 shutdown_connections 2024-09-15T22:45:49.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.768+0000 7f95429cf740 1 -- 172.21.15.111:0/1754672941 wait complete. 2024-09-15T22:45:49.796 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.768+0000 7f95429cf740 20 client.14856 trim_cache size 0 max 16384 2024-09-15T22:45:49.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.786+0000 7f95429cf740 1 Processor -- start 2024-09-15T22:45:49.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.786+0000 7f95429cf740 1 -- start start 2024-09-15T22:45:49.801 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.787+0000 7f95429cf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56445f28f7e0 0x56445f28a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:49.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.787+0000 7f95429cf740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56445f290160 0x56445f296980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:49.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.787+0000 7f95429cf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56445f294cd0 0x56445f2978d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:49.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.787+0000 7f95429cf740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56445f2a98e0 con 0x56445f294cd0 2024-09-15T22:45:49.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.787+0000 7f95429cf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56445f2a9fb0 con 0x56445f28f7e0 2024-09-15T22:45:49.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.787+0000 7f95429cf740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56445f2aa680 con 0x56445f290160 2024-09-15T22:45:49.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.787+0000 7f953de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56445f294cd0 0x56445f2978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:49.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.787+0000 7f953de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56445f294cd0 0x56445f2978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45708/0 (socket says 172.21.15.111:45708) 2024-09-15T22:45:49.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.787+0000 7f953de00640 1 -- 172.21.15.111:0/3001279332 learned_addr learned my addr 172.21.15.111:0/3001279332 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:49.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.787+0000 7f953ca00640 1 --2- 172.21.15.111:0/3001279332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56445f290160 0x56445f296980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:49.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.787+0000 7f953de00640 1 --2- 172.21.15.111:0/3001279332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56445f28f7e0 0x56445f28a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:49.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.787+0000 7f953de00640 1 -- 172.21.15.111:0/3001279332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56445f290160 msgr2=0x56445f296980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:49.802 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.787+0000 7f953de00640 1 --2- 172.21.15.111:0/3001279332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56445f290160 0x56445f296980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.787+0000 7f953de00640 1 -- 172.21.15.111:0/3001279332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56445f28f7e0 msgr2=0x56445f28a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:49.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.787+0000 7f953de00640 1 --2- 172.21.15.111:0/3001279332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56445f28f7e0 0x56445f28a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.787+0000 7f953de00640 1 -- 172.21.15.111:0/3001279332 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56445f297550 con 0x56445f294cd0 2024-09-15T22:45:49.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.788+0000 7f953de00640 1 --2- 172.21.15.111:0/3001279332 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56445f294cd0 0x56445f2978d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f953001a910 tx=0x7f9530005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e989ae8cf775a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:49.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.788+0000 7f9529c00640 1 -- 172.21.15.111:0/3001279332 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530016070 con 0x56445f294cd0 2024-09-15T22:45:49.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.788+0000 7f9529c00640 1 -- 172.21.15.111:0/3001279332 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9530025040 con 0x56445f294cd0 2024-09-15T22:45:49.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.788+0000 7f9529c00640 1 -- 172.21.15.111:0/3001279332 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530012830 con 0x56445f294cd0 2024-09-15T22:45:49.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.788+0000 7f95429cf740 1 -- 172.21.15.111:0/3001279332 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56445f294cd0 msgr2=0x56445f2978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:49.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.788+0000 7f95429cf740 1 --2- 172.21.15.111:0/3001279332 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56445f294cd0 0x56445f2978d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f953001a910 tx=0x7f9530005fd0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.789+0000 7f95429cf740 1 -- 172.21.15.111:0/3001279332 shutdown_connections 2024-09-15T22:45:49.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.789+0000 7f95429cf740 1 --2- 172.21.15.111:0/3001279332 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56445f294cd0 0x56445f2978d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.789+0000 7f95429cf740 1 --2- 172.21.15.111:0/3001279332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56445f290160 0x56445f296980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.789+0000 7f95429cf740 1 --2- 172.21.15.111:0/3001279332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56445f28f7e0 0x56445f28a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.789+0000 7f95429cf740 1 -- 172.21.15.111:0/3001279332 >> 172.21.15.111:0/3001279332 conn(0x56445f28a250 msgr2=0x56445f293240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:49.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.789+0000 7f95429cf740 1 -- 172.21.15.111:0/3001279332 shutdown_connections 2024-09-15T22:45:49.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.789+0000 7f95429cf740 1 -- 172.21.15.111:0/3001279332 wait complete. 2024-09-15T22:45:49.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.789+0000 7f95429cf740 1 -- start start 2024-09-15T22:45:49.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.789+0000 7f95429cf740 1 Processor -- start 2024-09-15T22:45:49.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.790+0000 7f95429cf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56445f28f7e0 0x56445f2ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:49.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.790+0000 7f95429cf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56445f290160 0x56445f2ae930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:49.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.790+0000 7f95429cf740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56445f294cd0 0x56445f2aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:49.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.790+0000 7f95429cf740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56445f2ab420 con 0x56445f28f7e0 2024-09-15T22:45:49.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.790+0000 7f95429cf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56445f2aad50 con 0x56445f290160 2024-09-15T22:45:49.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.790+0000 7f95429cf740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56445f2abaf0 con 0x56445f294cd0 2024-09-15T22:45:49.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.790+0000 7f95429cf740 10 client.0 _ll_register_callbacks cb 0x7f9530004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:49.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.790+0000 7f953de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56445f28f7e0 0x56445f2ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:49.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.790+0000 7f953de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56445f28f7e0 0x56445f2ae3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45724/0 (socket says 172.21.15.111:45724) 2024-09-15T22:45:49.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.790+0000 7f953de00640 1 -- 172.21.15.111:0/462304320 learned_addr learned my addr 172.21.15.111:0/462304320 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:49.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.790+0000 7f953ca00640 1 --2- 172.21.15.111:0/462304320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56445f290160 0x56445f2ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:49.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.790+0000 7f953de00640 1 --2- 172.21.15.111:0/462304320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56445f294cd0 0x56445f2aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:49.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.790+0000 7f953de00640 1 -- 172.21.15.111:0/462304320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56445f294cd0 msgr2=0x56445f2aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:49.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.790+0000 7f953de00640 1 --2- 172.21.15.111:0/462304320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56445f294cd0 0x56445f2aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.790+0000 7f953de00640 1 -- 172.21.15.111:0/462304320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56445f290160 msgr2=0x56445f2ae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:49.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.790+0000 7f953de00640 1 --2- 172.21.15.111:0/462304320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56445f290160 0x56445f2ae930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.790+0000 7f953de00640 1 -- 172.21.15.111:0/462304320 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56445f2b3440 con 0x56445f28f7e0 2024-09-15T22:45:49.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.790+0000 7f953ca00640 1 --2- 172.21.15.111:0/462304320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56445f290160 0x56445f2ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:45:49.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.791+0000 7f953de00640 1 --2- 172.21.15.111:0/462304320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56445f28f7e0 0x56445f2ae3f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9530012500 tx=0x7f9530009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25152a920ce7ad8a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:49.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.791+0000 7f951ea00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:45:49.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.791+0000 7f951ea00640 1 -- 172.21.15.111:0/462304320 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530016070 con 0x56445f28f7e0 2024-09-15T22:45:49.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.791+0000 7f95429cf740 1 -- 172.21.15.111:0/462304320 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9530025040 con 0x56445f28f7e0 2024-09-15T22:45:49.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.791+0000 7f951ea00640 1 -- 172.21.15.111:0/462304320 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f9530005aa0 con 0x56445f28f7e0 2024-09-15T22:45:49.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.791+0000 7f951ea00640 1 -- 172.21.15.111:0/462304320 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9530004af0 con 0x56445f28f7e0 2024-09-15T22:45:49.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.791+0000 7f95429cf740 10 client.14862 fetch_fsmap learned FSMap version 25 2024-09-15T22:45:49.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.791+0000 7f95429cf740 1 -- 172.21.15.111:0/462304320 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56445f3149c0 con 0x56445f28f7e0 2024-09-15T22:45:49.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.791+0000 7f951ea00640 1 -- 172.21.15.111:0/462304320 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9530025040 con 0x56445f28f7e0 2024-09-15T22:45:49.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.792+0000 7f951ea00640 1 -- 172.21.15.111:0/462304320 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f953002abd0 con 0x56445f28f7e0 2024-09-15T22:45:49.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.792+0000 7f95429cf740 10 client.14862 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:45:49.806 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.792+0000 7f95429cf740 10 client.14862 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-15T22:45:49.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.792+0000 7f95429cf740 20 client.14862 populate_metadata read hostname 'smithi111' 2024-09-15T22:45:49.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.792+0000 7f95429cf740 1 --2- 172.21.15.111:0/462304320 >> [v2:172.21.15.121:6836/1443772791,v1:172.21.15.121:6837/1443772791] conn(0x56445f319220 0x56445f317f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:49.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.792+0000 7f95429cf740 4 client.14862 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:45:49.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.792+0000 7f95429cf740 1 -- 172.21.15.111:0/462304320 --> [v2:172.21.15.121:6836/1443772791,v1:172.21.15.121:6837/1443772791] -- command(tid 0: {"prefix": "status"}) -- 0x56445f1fbb30 con 0x56445f319220 2024-09-15T22:45:49.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.792+0000 7f953ca00640 1 --2- 172.21.15.111:0/462304320 >> [v2:172.21.15.121:6836/1443772791,v1:172.21.15.121:6837/1443772791] conn(0x56445f319220 0x56445f317f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:49.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.792+0000 7f951ea00640 10 client.14862 ms_handle_connect on v2:172.21.15.121:6836/1443772791 2024-09-15T22:45:49.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.792+0000 7f953ca00640 1 --2- 172.21.15.111:0/462304320 >> [v2:172.21.15.121:6836/1443772791,v1:172.21.15.121:6837/1443772791] conn(0x56445f319220 0x56445f317f30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fc0cdbd0f0375217 server_cookie=42dd8dbb3392f43d in_seq=0 out_seq=0 2024-09-15T22:45:49.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.793+0000 7f951ea00640 1 -- 172.21.15.111:0/462304320 <== mds.2 v2:172.21.15.121:6836/1443772791 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56445f1fbb30 con 0x56445f319220 2024-09-15T22:45:49.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.793+0000 7f951ea00640 10 client.14862 handle_command_reply: tid=0 2024-09-15T22:45:49.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.793+0000 7f95429cf740 1 client.14862 shutdown 2024-09-15T22:45:49.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.793+0000 7f95429cf740 1 -- 172.21.15.111:0/462304320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56445f28f7e0 msgr2=0x56445f2ae3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:49.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.793+0000 7f95429cf740 1 --2- 172.21.15.111:0/462304320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56445f28f7e0 0x56445f2ae3f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9530012500 tx=0x7f9530009a00 comp rx=0 tx=0).stop 2024-09-15T22:45:49.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.793+0000 7f95429cf740 1 -- 172.21.15.111:0/462304320 shutdown_connections 2024-09-15T22:45:49.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.793+0000 7f95429cf740 1 --2- 172.21.15.111:0/462304320 >> [v2:172.21.15.121:6836/1443772791,v1:172.21.15.121:6837/1443772791] conn(0x56445f319220 0x56445f317f30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.793+0000 7f95429cf740 1 --2- 172.21.15.111:0/462304320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56445f294cd0 0x56445f2aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.793+0000 7f95429cf740 1 --2- 172.21.15.111:0/462304320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56445f290160 0x56445f2ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.793+0000 7f95429cf740 1 --2- 172.21.15.111:0/462304320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56445f28f7e0 0x56445f2ae3f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.793+0000 7f951ea00640 0 client.14862 ms_handle_reset on v2:172.21.15.121:6836/1443772791 2024-09-15T22:45:49.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.793+0000 7f95429cf740 1 -- 172.21.15.111:0/462304320 >> 172.21.15.111:0/462304320 conn(0x56445f28a250 msgr2=0x56445f297760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:49.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.793+0000 7f95429cf740 1 -- 172.21.15.111:0/462304320 shutdown_connections 2024-09-15T22:45:49.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.793+0000 7f95429cf740 1 -- 172.21.15.111:0/462304320 wait complete. 2024-09-15T22:45:49.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.793+0000 7f95429cf740 20 client.14862 trim_cache size 0 max 16384 2024-09-15T22:45:49.809 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-15T22:45:49.809 INFO:teuthology.orchestra.run.smithi111.stdout: "cluster_fsid": "b7614d3c-73b2-11ef-bceb-c7b262605968", 2024-09-15T22:45:49.810 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": 2, 2024-09-15T22:45:49.810 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 24503, 2024-09-15T22:45:49.810 INFO:teuthology.orchestra.run.smithi111.stdout: "want_state": "up:active", 2024-09-15T22:45:49.810 INFO:teuthology.orchestra.run.smithi111.stdout: "state": "up:active", 2024-09-15T22:45:49.810 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs", 2024-09-15T22:45:49.810 INFO:teuthology.orchestra.run.smithi111.stdout: "rank_uptime": 80.250197947999993, 2024-09-15T22:45:49.810 INFO:teuthology.orchestra.run.smithi111.stdout: "mdsmap_epoch": 25, 2024-09-15T22:45:49.810 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch": 88, 2024-09-15T22:45:49.811 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch_barrier": 86, 2024-09-15T22:45:49.811 INFO:teuthology.orchestra.run.smithi111.stdout: "uptime": 117.848292818 2024-09-15T22:45:49.811 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-15T22:45:49.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.794+0000 7f953fe00640 1 -- 172.21.15.111:0/2513375829 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f95200abb60 msgr2=0x7f95200ae020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:49.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.794+0000 7f953fe00640 1 --2- 172.21.15.111:0/2513375829 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f95200abb60 0x7f95200ae020 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x56445f28b0b0 tx=0x7f952c0f3c40 comp rx=0 tx=0).stop 2024-09-15T22:45:49.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.794+0000 7f953fe00640 1 -- 172.21.15.111:0/2513375829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9538101de0 msgr2=0x7f9538198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:49.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.794+0000 7f953fe00640 1 --2- 172.21.15.111:0/2513375829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9538101de0 0x7f9538198440 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f952c004770 tx=0x7f952c003c10 comp rx=0 tx=0).stop 2024-09-15T22:45:49.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.794+0000 7f953fe00640 1 -- 172.21.15.111:0/2513375829 shutdown_connections 2024-09-15T22:45:49.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.794+0000 7f953fe00640 1 --2- 172.21.15.111:0/2513375829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9538106f10 0x7f953819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.794+0000 7f953fe00640 1 --2- 172.21.15.111:0/2513375829 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f95200abb60 0x7f95200ae020 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.794+0000 7f953fe00640 1 --2- 172.21.15.111:0/2513375829 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9538102790 0x7f9538198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.794+0000 7f953fe00640 1 --2- 172.21.15.111:0/2513375829 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9538101de0 0x7f9538198440 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:49.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.794+0000 7f953fe00640 1 -- 172.21.15.111:0/2513375829 >> 172.21.15.111:0/2513375829 conn(0x7f95380fd8c0 msgr2=0x7f95380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:49.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.794+0000 7f953fe00640 1 -- 172.21.15.111:0/2513375829 shutdown_connections 2024-09-15T22:45:49.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:49.794+0000 7f953fe00640 1 -- 172.21.15.111:0/2513375829 wait complete. 2024-09-15T22:45:49.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:49 smithi121 ceph-mon[34815]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:49.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:49 smithi111 ceph-mon[30629]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:50.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:49 smithi192 ceph-mon[35427]: pgmap v347: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:45:50.754 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-15T22:45:51.120 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:51.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.768+0000 7f3056a00640 1 -- 172.21.15.111:0/513038592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30501047e0 msgr2=0x7f3050108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:51.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.768+0000 7f3056a00640 1 --2- 172.21.15.111:0/513038592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30501047e0 0x7f3050108830 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f303c009900 tx=0x7f303c02a410 comp rx=0 tx=0).stop 2024-09-15T22:45:51.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.768+0000 7f3056a00640 1 -- 172.21.15.111:0/513038592 shutdown_connections 2024-09-15T22:45:51.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.768+0000 7f3056a00640 1 --2- 172.21.15.111:0/513038592 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3050108f60 0x7f3050110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.768+0000 7f3056a00640 1 --2- 172.21.15.111:0/513038592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30501047e0 0x7f3050108830 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.768+0000 7f3056a00640 1 --2- 172.21.15.111:0/513038592 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3050103e30 0x7f3050104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.768+0000 7f3056a00640 1 -- 172.21.15.111:0/513038592 >> 172.21.15.111:0/513038592 conn(0x7f30500ff910 msgr2=0x7f3050101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:51.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.768+0000 7f3056a00640 1 -- 172.21.15.111:0/513038592 shutdown_connections 2024-09-15T22:45:51.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.769+0000 7f3056a00640 1 -- 172.21.15.111:0/513038592 wait complete. 2024-09-15T22:45:51.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.769+0000 7f3056a00640 1 Processor -- start 2024-09-15T22:45:51.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.769+0000 7f3056a00640 1 -- start start 2024-09-15T22:45:51.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.770+0000 7f3056a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3050103e30 0x7f305019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:51.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.770+0000 7f3056a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30501047e0 0x7f305019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:51.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.770+0000 7f3056a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3050108f60 0x7f305019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:51.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.770+0000 7f3056a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3050114010 con 0x7f3050108f60 2024-09-15T22:45:51.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.770+0000 7f3056a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3050113e90 con 0x7f3050103e30 2024-09-15T22:45:51.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.770+0000 7f3056a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3050114190 con 0x7f30501047e0 2024-09-15T22:45:51.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.770+0000 7f3056000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3050108f60 0x7f305019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:51.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.770+0000 7f3054c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30501047e0 0x7f305019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:51.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.770+0000 7f3056000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3050108f60 0x7f305019ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:35068/0 (socket says 172.21.15.111:35068) 2024-09-15T22:45:51.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.770+0000 7f3056000640 1 -- 172.21.15.111:0/2256131247 learned_addr learned my addr 172.21.15.111:0/2256131247 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:51.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.770+0000 7f3055600640 1 --2- 172.21.15.111:0/2256131247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3050103e30 0x7f305019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:51.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.771+0000 7f3055600640 1 -- 172.21.15.111:0/2256131247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30501047e0 msgr2=0x7f305019a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:51.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.771+0000 7f3055600640 1 --2- 172.21.15.111:0/2256131247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30501047e0 0x7f305019a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.771+0000 7f3055600640 1 -- 172.21.15.111:0/2256131247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3050108f60 msgr2=0x7f305019ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:51.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.771+0000 7f3055600640 1 --2- 172.21.15.111:0/2256131247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3050108f60 0x7f305019ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.771+0000 7f3055600640 1 -- 172.21.15.111:0/2256131247 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305019f420 con 0x7f3050103e30 2024-09-15T22:45:51.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.771+0000 7f3055600640 1 --2- 172.21.15.111:0/2256131247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3050103e30 0x7f305019a470 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f304400e9f0 tx=0x7f304400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:51.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.771+0000 7f304e000640 1 -- 172.21.15.111:0/2256131247 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304400ccc0 con 0x7f3050103e30 2024-09-15T22:45:51.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.771+0000 7f3056a00640 1 -- 172.21.15.111:0/2256131247 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f305019f710 con 0x7f3050103e30 2024-09-15T22:45:51.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.771+0000 7f304e000640 1 -- 172.21.15.111:0/2256131247 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3044005500 con 0x7f3050103e30 2024-09-15T22:45:51.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.771+0000 7f304e000640 1 -- 172.21.15.111:0/2256131247 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3044010690 con 0x7f3050103e30 2024-09-15T22:45:51.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.771+0000 7f3056a00640 1 -- 172.21.15.111:0/2256131247 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30501a6ff0 con 0x7f3050103e30 2024-09-15T22:45:51.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.774+0000 7f304e000640 1 -- 172.21.15.111:0/2256131247 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f30440046c0 con 0x7f3050103e30 2024-09-15T22:45:51.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.774+0000 7f305aa0e740 1 Processor -- start 2024-09-15T22:45:51.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.774+0000 7f305aa0e740 1 -- start start 2024-09-15T22:45:51.777 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.775+0000 7f305aa0e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3871444e0 0x55d3871448c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:51.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.775+0000 7f305aa0e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d387144e00 0x55d38714f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:51.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.775+0000 7f305aa0e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d38714f6c0 0x55d387151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:51.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.775+0000 7f305aa0e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d3870aa0b0 con 0x55d387144e00 2024-09-15T22:45:51.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.775+0000 7f305aa0e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d387051ab0 con 0x55d3871444e0 2024-09-15T22:45:51.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.775+0000 7f305aa0e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d3870a6070 con 0x55d38714f6c0 2024-09-15T22:45:51.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.775+0000 7f3055600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d387144e00 0x55d38714f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:51.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.775+0000 7f3056000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3871444e0 0x55d3871448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:51.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.775+0000 7f3056000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3871444e0 0x55d3871448c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:45762/0 (socket says 172.21.15.111:45762) 2024-09-15T22:45:51.778 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.775+0000 7f3056000640 1 -- 172.21.15.111:0/1676784892 learned_addr learned my addr 172.21.15.111:0/1676784892 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:51.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.775+0000 7f3054c00640 1 --2- 172.21.15.111:0/1676784892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d38714f6c0 0x55d387151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:51.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.776+0000 7f3056000640 1 -- 172.21.15.111:0/1676784892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d38714f6c0 msgr2=0x55d387151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:51.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.776+0000 7f3056000640 1 --2- 172.21.15.111:0/1676784892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d38714f6c0 0x55d387151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.776+0000 7f3056000640 1 -- 172.21.15.111:0/1676784892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d387144e00 msgr2=0x55d38714f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:51.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.776+0000 7f3056000640 1 --2- 172.21.15.111:0/1676784892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d387144e00 0x55d38714f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.776+0000 7f3056000640 1 -- 172.21.15.111:0/1676784892 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d38705e4a0 con 0x55d3871444e0 2024-09-15T22:45:51.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.776+0000 7f304e000640 1 --2- 172.21.15.111:0/2256131247 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f30380a3240 0x7f30380a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:51.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.776+0000 7f3056000640 1 --2- 172.21.15.111:0/1676784892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3871444e0 0x55d3871448c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3048009fb0 tx=0x7f3048011870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af329367860f5351 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:51.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.776+0000 7f304e000640 1 -- 172.21.15.111:0/2256131247 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f30440cfc50 con 0x7f3050103e30 2024-09-15T22:45:51.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.776+0000 7f3055600640 1 --2- 172.21.15.111:0/2256131247 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f30380a3240 0x7f30380a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:51.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.776+0000 7f3042600640 1 -- 172.21.15.111:0/1676784892 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30480127d0 con 0x55d3871444e0 2024-09-15T22:45:51.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.776+0000 7f3042600640 1 -- 172.21.15.111:0/1676784892 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3048012e50 con 0x55d3871444e0 2024-09-15T22:45:51.779 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.776+0000 7f3042600640 1 -- 172.21.15.111:0/1676784892 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30480193a0 con 0x55d3871444e0 2024-09-15T22:45:51.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.777+0000 7f305aa0e740 1 -- 172.21.15.111:0/1676784892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3871444e0 msgr2=0x55d3871448c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:51.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.777+0000 7f3055600640 1 --2- 172.21.15.111:0/2256131247 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f30380a3240 0x7f30380a5700 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f304409c200 tx=0x7f30440d3690 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:51.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.777+0000 7f305aa0e740 1 --2- 172.21.15.111:0/1676784892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3871444e0 0x55d3871448c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3048009fb0 tx=0x7f3048011870 comp rx=0 tx=0).stop 2024-09-15T22:45:51.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.777+0000 7f305aa0e740 1 -- 172.21.15.111:0/1676784892 shutdown_connections 2024-09-15T22:45:51.782 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.777+0000 7f305aa0e740 1 --2- 172.21.15.111:0/1676784892 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d38714f6c0 0x55d387151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.777+0000 7f305aa0e740 1 --2- 172.21.15.111:0/1676784892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d387144e00 0x55d38714f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.777+0000 7f305aa0e740 1 --2- 172.21.15.111:0/1676784892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3871444e0 0x55d3871448c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.777+0000 7f305aa0e740 1 -- 172.21.15.111:0/1676784892 >> 172.21.15.111:0/1676784892 conn(0x55d387140a70 msgr2=0x55d387140e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:51.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.777+0000 7f305aa0e740 1 -- 172.21.15.111:0/1676784892 shutdown_connections 2024-09-15T22:45:51.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.777+0000 7f305aa0e740 1 -- 172.21.15.111:0/1676784892 wait complete. 2024-09-15T22:45:51.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.778+0000 7f305aa0e740 1 -- start start 2024-09-15T22:45:51.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.778+0000 7f305aa0e740 1 Processor -- start 2024-09-15T22:45:51.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.778+0000 7f305aa0e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3871444e0 0x55d38714c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:51.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.778+0000 7f305aa0e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d387144e00 0x55d38714caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:51.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.778+0000 7f305aa0e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d38714f6c0 0x55d38714cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:51.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.778+0000 7f305aa0e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d387152330 con 0x55d38714f6c0 2024-09-15T22:45:51.783 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.778+0000 7f305aa0e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d387032920 con 0x55d3871444e0 2024-09-15T22:45:51.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.778+0000 7f305aa0e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d3871524b0 con 0x55d387144e00 2024-09-15T22:45:51.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.778+0000 7f305aa0e740 10 client.0 _ll_register_callbacks cb 0x55d3870568a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:51.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.778+0000 7f3054c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d387144e00 0x55d38714caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:51.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.779+0000 7f3054c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d387144e00 0x55d38714caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:48576/0 (socket says 172.21.15.111:48576) 2024-09-15T22:45:51.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.779+0000 7f3054c00640 1 -- 172.21.15.111:0/2657176363 learned_addr learned my addr 172.21.15.111:0/2657176363 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:51.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.779+0000 7f3056000640 1 --2- 172.21.15.111:0/2657176363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3871444e0 0x55d38714c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:51.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.779+0000 7f3056000640 1 --2- 172.21.15.111:0/2657176363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d38714f6c0 0x55d38714cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:51.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.779+0000 7f3054c00640 1 -- 172.21.15.111:0/2657176363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3871444e0 msgr2=0x55d38714c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:51.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.779+0000 7f3054c00640 1 --2- 172.21.15.111:0/2657176363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3871444e0 0x55d38714c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.779+0000 7f3054c00640 1 -- 172.21.15.111:0/2657176363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d38714f6c0 msgr2=0x55d38714cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:51.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.779+0000 7f3054c00640 1 --2- 172.21.15.111:0/2657176363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d38714f6c0 0x55d38714cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.779+0000 7f3054c00640 1 -- 172.21.15.111:0/2657176363 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d386e1b2b0 con 0x55d387144e00 2024-09-15T22:45:51.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.779+0000 7f3054c00640 1 --2- 172.21.15.111:0/2657176363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d387144e00 0x55d38714caa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f303c009900 tx=0x7f303c004420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed4e32399c51b6ac server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:51.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.779+0000 7f3036a00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:45:51.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.780+0000 7f3036a00640 1 -- 172.21.15.111:0/2657176363 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c035040 con 0x55d387144e00 2024-09-15T22:45:51.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.780+0000 7f305aa0e740 1 -- 172.21.15.111:0/2657176363 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3048012e50 con 0x55d387144e00 2024-09-15T22:45:51.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.780+0000 7f3036a00640 1 -- 172.21.15.111:0/2657176363 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f303c005450 con 0x55d387144e00 2024-09-15T22:45:51.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.780+0000 7f3036a00640 1 -- 172.21.15.111:0/2657176363 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f303c005650 con 0x55d387144e00 2024-09-15T22:45:51.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.780+0000 7f305aa0e740 10 client.24904 fetch_fsmap learned FSMap version 25 2024-09-15T22:45:51.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.780+0000 7f305aa0e740 1 -- 172.21.15.111:0/2657176363 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d38715dfe0 con 0x55d387144e00 2024-09-15T22:45:51.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.780+0000 7f3036a00640 1 -- 172.21.15.111:0/2657176363 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3048012e50 con 0x55d387144e00 2024-09-15T22:45:51.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.780+0000 7f3036a00640 1 -- 172.21.15.111:0/2657176363 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f303c003bc0 con 0x55d387144e00 2024-09-15T22:45:51.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.780+0000 7f305aa0e740 10 client.24904 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:45:51.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.781+0000 7f305aa0e740 10 client.24904 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-15T22:45:51.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.781+0000 7f305aa0e740 20 client.24904 populate_metadata read hostname 'smithi111' 2024-09-15T22:45:51.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.781+0000 7f305aa0e740 1 --2- 172.21.15.111:0/2657176363 >> [v2:172.21.15.192:6838/800136261,v1:172.21.15.192:6839/800136261] conn(0x55d387160590 0x55d387161a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:51.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.781+0000 7f305aa0e740 4 client.24904 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:45:51.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.781+0000 7f305aa0e740 1 -- 172.21.15.111:0/2657176363 --> [v2:172.21.15.192:6838/800136261,v1:172.21.15.192:6839/800136261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3870b7620 con 0x55d387160590 2024-09-15T22:45:51.786 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.781+0000 7f3056000640 1 --2- 172.21.15.111:0/2657176363 >> [v2:172.21.15.192:6838/800136261,v1:172.21.15.192:6839/800136261] conn(0x55d387160590 0x55d387161a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:51.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.781+0000 7f3056000640 1 --2- 172.21.15.111:0/2657176363 >> [v2:172.21.15.192:6838/800136261,v1:172.21.15.192:6839/800136261] conn(0x55d387160590 0x55d387161a00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dee290de5da57fd server_cookie=dd80b6ca64c1e9bb in_seq=0 out_seq=0 2024-09-15T22:45:51.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.781+0000 7f3036a00640 10 client.24904 ms_handle_connect on v2:172.21.15.192:6838/800136261 2024-09-15T22:45:51.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.782+0000 7f3036a00640 1 -- 172.21.15.111:0/2657176363 <== mds.1 v2:172.21.15.192:6838/800136261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d3870b7620 con 0x55d387160590 2024-09-15T22:45:51.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.782+0000 7f3036a00640 10 client.24904 handle_command_reply: tid=0 2024-09-15T22:45:51.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.782+0000 7f305aa0e740 1 client.24904 shutdown 2024-09-15T22:45:51.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.783+0000 7f305aa0e740 1 -- 172.21.15.111:0/2657176363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d387144e00 msgr2=0x55d38714caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:51.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.783+0000 7f305aa0e740 1 --2- 172.21.15.111:0/2657176363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d387144e00 0x55d38714caa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f303c009900 tx=0x7f303c004420 comp rx=0 tx=0).stop 2024-09-15T22:45:51.787 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.783+0000 7f305aa0e740 1 -- 172.21.15.111:0/2657176363 shutdown_connections 2024-09-15T22:45:51.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.783+0000 7f305aa0e740 1 --2- 172.21.15.111:0/2657176363 >> [v2:172.21.15.192:6838/800136261,v1:172.21.15.192:6839/800136261] conn(0x55d387160590 0x55d387161a00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.783+0000 7f305aa0e740 1 --2- 172.21.15.111:0/2657176363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d38714f6c0 0x55d38714cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.783+0000 7f305aa0e740 1 --2- 172.21.15.111:0/2657176363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d387144e00 0x55d38714caa0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.783+0000 7f3036a00640 0 client.24904 ms_handle_reset on v2:172.21.15.192:6838/800136261 2024-09-15T22:45:51.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.783+0000 7f305aa0e740 1 --2- 172.21.15.111:0/2657176363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d3871444e0 0x55d38714c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.783+0000 7f305aa0e740 1 -- 172.21.15.111:0/2657176363 >> 172.21.15.111:0/2657176363 conn(0x55d387140a70 msgr2=0x55d38713fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:51.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.783+0000 7f305aa0e740 1 -- 172.21.15.111:0/2657176363 shutdown_connections 2024-09-15T22:45:51.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.783+0000 7f305aa0e740 1 -- 172.21.15.111:0/2657176363 wait complete. 2024-09-15T22:45:51.788 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.783+0000 7f305aa0e740 20 client.24904 trim_cache size 0 max 16384 2024-09-15T22:45:51.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.801+0000 7f305aa0e740 1 Processor -- start 2024-09-15T22:45:51.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.801+0000 7f305aa0e740 1 -- start start 2024-09-15T22:45:51.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.802+0000 7f305aa0e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d3871444e0 0x55d38714ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:51.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.802+0000 7f305aa0e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d387144e00 0x55d38714b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:51.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.802+0000 7f305aa0e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d38714f6c0 0x55d3871497c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:51.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.802+0000 7f305aa0e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d38715fd40 con 0x55d3871444e0 2024-09-15T22:45:51.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.802+0000 7f305aa0e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d387160410 con 0x55d38714f6c0 2024-09-15T22:45:51.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.802+0000 7f305aa0e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d387160ae0 con 0x55d387144e00 2024-09-15T22:45:51.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.802+0000 7f3056000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d3871444e0 0x55d38714ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:51.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.802+0000 7f3056000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d3871444e0 0x55d38714ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:35076/0 (socket says 172.21.15.111:35076) 2024-09-15T22:45:51.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.802+0000 7f3056000640 1 -- 172.21.15.111:0/676816115 learned_addr learned my addr 172.21.15.111:0/676816115 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:51.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.802+0000 7f3054c00640 1 --2- 172.21.15.111:0/676816115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d387144e00 0x55d38714b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:51.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.802+0000 7f3056000640 1 -- 172.21.15.111:0/676816115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d387144e00 msgr2=0x55d38714b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:51.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.802+0000 7f3056000640 1 --2- 172.21.15.111:0/676816115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d387144e00 0x55d38714b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.802+0000 7f3056000640 1 -- 172.21.15.111:0/676816115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d38714f6c0 msgr2=0x55d3871497c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:51.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.802+0000 7f3056000640 1 --2- 172.21.15.111:0/676816115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d38714f6c0 0x55d3871497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.802+0000 7f3056000640 1 -- 172.21.15.111:0/676816115 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d387147350 con 0x55d3871444e0 2024-09-15T22:45:51.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.802+0000 7f3054c00640 1 --2- 172.21.15.111:0/676816115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d387144e00 0x55d38714b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:51.805 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.803+0000 7f3056000640 1 --2- 172.21.15.111:0/676816115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d3871444e0 0x55d38714ae30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f304801aac0 tx=0x7f304801aa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406a4445ffa8d6e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:51.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.803+0000 7f3041c00640 1 -- 172.21.15.111:0/676816115 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3048035070 con 0x55d3871444e0 2024-09-15T22:45:51.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.803+0000 7f3041c00640 1 -- 172.21.15.111:0/676816115 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3048030070 con 0x55d3871444e0 2024-09-15T22:45:51.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.803+0000 7f3041c00640 1 -- 172.21.15.111:0/676816115 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30480128c0 con 0x55d3871444e0 2024-09-15T22:45:51.809 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.803+0000 7f305aa0e740 1 -- 172.21.15.111:0/676816115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d3871444e0 msgr2=0x55d38714ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:51.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.803+0000 7f305aa0e740 1 --2- 172.21.15.111:0/676816115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d3871444e0 0x55d38714ae30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f304801aac0 tx=0x7f304801aa50 comp rx=0 tx=0).stop 2024-09-15T22:45:51.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.804+0000 7f305aa0e740 1 -- 172.21.15.111:0/676816115 shutdown_connections 2024-09-15T22:45:51.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.804+0000 7f305aa0e740 1 --2- 172.21.15.111:0/676816115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d38714f6c0 0x55d3871497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.804+0000 7f305aa0e740 1 --2- 172.21.15.111:0/676816115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d387144e00 0x55d38714b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.804+0000 7f305aa0e740 1 --2- 172.21.15.111:0/676816115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d3871444e0 0x55d38714ae30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.804+0000 7f305aa0e740 1 -- 172.21.15.111:0/676816115 >> 172.21.15.111:0/676816115 conn(0x55d387140a70 msgr2=0x55d387147860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:51.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.804+0000 7f305aa0e740 1 -- 172.21.15.111:0/676816115 shutdown_connections 2024-09-15T22:45:51.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.804+0000 7f305aa0e740 1 -- 172.21.15.111:0/676816115 wait complete. 2024-09-15T22:45:51.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.804+0000 7f305aa0e740 1 -- start start 2024-09-15T22:45:51.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.804+0000 7f305aa0e740 1 Processor -- start 2024-09-15T22:45:51.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.805+0000 7f305aa0e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d3871444e0 0x55d387163930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:51.810 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.805+0000 7f305aa0e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d387144e00 0x55d387163e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:51.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.805+0000 7f305aa0e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d38714f6c0 0x55d3871643b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:51.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.805+0000 7f305aa0e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d387161880 con 0x55d3871444e0 2024-09-15T22:45:51.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.805+0000 7f305aa0e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d3871611b0 con 0x55d387144e00 2024-09-15T22:45:51.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.805+0000 7f305aa0e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d387161f50 con 0x55d38714f6c0 2024-09-15T22:45:51.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.805+0000 7f305aa0e740 10 client.0 _ll_register_callbacks cb 0x7f303c0040b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:51.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.805+0000 7f3056000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d3871444e0 0x55d387163930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:51.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.805+0000 7f3056000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d3871444e0 0x55d387163930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:35086/0 (socket says 172.21.15.111:35086) 2024-09-15T22:45:51.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.805+0000 7f3056000640 1 -- 172.21.15.111:0/2807374945 learned_addr learned my addr 172.21.15.111:0/2807374945 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:51.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.805+0000 7f3054c00640 1 --2- 172.21.15.111:0/2807374945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d387144e00 0x55d387163e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:51.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.805+0000 7f3056000640 1 --2- 172.21.15.111:0/2807374945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d38714f6c0 0x55d3871643b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:51.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.806+0000 7f3056000640 1 -- 172.21.15.111:0/2807374945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d38714f6c0 msgr2=0x55d3871643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:51.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.806+0000 7f3056000640 1 --2- 172.21.15.111:0/2807374945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d38714f6c0 0x55d3871643b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.811 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.806+0000 7f3056000640 1 -- 172.21.15.111:0/2807374945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d387144e00 msgr2=0x55d387163e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:51.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.806+0000 7f3056000640 1 --2- 172.21.15.111:0/2807374945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d387144e00 0x55d387163e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.806+0000 7f3056000640 1 -- 172.21.15.111:0/2807374945 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d387168120 con 0x55d3871444e0 2024-09-15T22:45:51.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.806+0000 7f3054c00640 1 --2- 172.21.15.111:0/2807374945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d387144e00 0x55d387163e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:51.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.806+0000 7f3056000640 1 --2- 172.21.15.111:0/2807374945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d3871444e0 0x55d387163930 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f30480096c0 tx=0x7f304801a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f55a43c41d3ee9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:51.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.806+0000 7f3036a00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:45:51.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.806+0000 7f3036a00640 1 -- 172.21.15.111:0/2807374945 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3048035040 con 0x55d3871444e0 2024-09-15T22:45:51.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.806+0000 7f3036a00640 1 -- 172.21.15.111:0/2807374945 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f304801a5f0 con 0x55d3871444e0 2024-09-15T22:45:51.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.806+0000 7f305aa0e740 1 -- 172.21.15.111:0/2807374945 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3048030070 con 0x55d3871444e0 2024-09-15T22:45:51.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.806+0000 7f3036a00640 1 -- 172.21.15.111:0/2807374945 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3048004a00 con 0x55d3871444e0 2024-09-15T22:45:51.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.806+0000 7f305aa0e740 10 client.14880 fetch_fsmap learned FSMap version 25 2024-09-15T22:45:51.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.807+0000 7f305aa0e740 1 -- 172.21.15.111:0/2807374945 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d3871caf00 con 0x55d3871444e0 2024-09-15T22:45:51.812 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.807+0000 7f3036a00640 1 -- 172.21.15.111:0/2807374945 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3048030070 con 0x55d3871444e0 2024-09-15T22:45:51.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.807+0000 7f3036a00640 1 -- 172.21.15.111:0/2807374945 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3048026040 con 0x55d3871444e0 2024-09-15T22:45:51.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.807+0000 7f305aa0e740 10 client.14880 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:45:51.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.807+0000 7f305aa0e740 10 client.14880 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-15T22:45:51.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.807+0000 7f305aa0e740 20 client.14880 populate_metadata read hostname 'smithi111' 2024-09-15T22:45:51.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.807+0000 7f305aa0e740 1 --2- 172.21.15.111:0/2807374945 >> [v2:172.21.15.192:6838/800136261,v1:172.21.15.192:6839/800136261] conn(0x55d3871cf430 0x55d3871ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:51.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.807+0000 7f305aa0e740 4 client.14880 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:45:51.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.807+0000 7f305aa0e740 1 -- 172.21.15.111:0/2807374945 --> [v2:172.21.15.192:6838/800136261,v1:172.21.15.192:6839/800136261] -- command(tid 0: {"prefix": "status"}) -- 0x55d3870b1b30 con 0x55d3871cf430 2024-09-15T22:45:51.813 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.808+0000 7f3054c00640 1 --2- 172.21.15.111:0/2807374945 >> [v2:172.21.15.192:6838/800136261,v1:172.21.15.192:6839/800136261] conn(0x55d3871cf430 0x55d3871ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:51.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.808+0000 7f3054c00640 1 --2- 172.21.15.111:0/2807374945 >> [v2:172.21.15.192:6838/800136261,v1:172.21.15.192:6839/800136261] conn(0x55d3871cf430 0x55d3871ce140 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f3b11b4b922d526d server_cookie=62a1421f0774a760 in_seq=0 out_seq=0 2024-09-15T22:45:51.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.808+0000 7f3036a00640 10 client.14880 ms_handle_connect on v2:172.21.15.192:6838/800136261 2024-09-15T22:45:51.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.808+0000 7f3036a00640 1 -- 172.21.15.111:0/2807374945 <== mds.1 v2:172.21.15.192:6838/800136261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d3870b1b30 con 0x55d3871cf430 2024-09-15T22:45:51.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.808+0000 7f3036a00640 10 client.14880 handle_command_reply: tid=0 2024-09-15T22:45:51.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.809+0000 7f305aa0e740 1 client.14880 shutdown 2024-09-15T22:45:51.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.809+0000 7f305aa0e740 1 -- 172.21.15.111:0/2807374945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d3871444e0 msgr2=0x55d387163930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:51.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.809+0000 7f305aa0e740 1 --2- 172.21.15.111:0/2807374945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d3871444e0 0x55d387163930 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f30480096c0 tx=0x7f304801a9e0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.809+0000 7f305aa0e740 1 -- 172.21.15.111:0/2807374945 shutdown_connections 2024-09-15T22:45:51.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.809+0000 7f305aa0e740 1 --2- 172.21.15.111:0/2807374945 >> [v2:172.21.15.192:6838/800136261,v1:172.21.15.192:6839/800136261] conn(0x55d3871cf430 0x55d3871ce140 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.809+0000 7f305aa0e740 1 --2- 172.21.15.111:0/2807374945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d38714f6c0 0x55d3871643b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.809+0000 7f305aa0e740 1 --2- 172.21.15.111:0/2807374945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d387144e00 0x55d387163e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.809+0000 7f305aa0e740 1 --2- 172.21.15.111:0/2807374945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d3871444e0 0x55d387163930 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.809+0000 7f3036a00640 0 client.14880 ms_handle_reset on v2:172.21.15.192:6838/800136261 2024-09-15T22:45:51.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.809+0000 7f305aa0e740 1 -- 172.21.15.111:0/2807374945 >> 172.21.15.111:0/2807374945 conn(0x55d387140a70 msgr2=0x55d38714aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:51.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.809+0000 7f305aa0e740 1 -- 172.21.15.111:0/2807374945 shutdown_connections 2024-09-15T22:45:51.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.809+0000 7f305aa0e740 1 -- 172.21.15.111:0/2807374945 wait complete. 2024-09-15T22:45:51.815 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.809+0000 7f305aa0e740 20 client.14880 trim_cache size 0 max 16384 2024-09-15T22:45:51.816 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-15T22:45:51.816 INFO:teuthology.orchestra.run.smithi111.stdout: "cluster_fsid": "b7614d3c-73b2-11ef-bceb-c7b262605968", 2024-09-15T22:45:51.816 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": 1, 2024-09-15T22:45:51.816 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 24514, 2024-09-15T22:45:51.816 INFO:teuthology.orchestra.run.smithi111.stdout: "want_state": "up:active", 2024-09-15T22:45:51.816 INFO:teuthology.orchestra.run.smithi111.stdout: "state": "up:active", 2024-09-15T22:45:51.816 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs", 2024-09-15T22:45:51.816 INFO:teuthology.orchestra.run.smithi111.stdout: "rank_uptime": 83.274533665999996, 2024-09-15T22:45:51.817 INFO:teuthology.orchestra.run.smithi111.stdout: "mdsmap_epoch": 25, 2024-09-15T22:45:51.817 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch": 88, 2024-09-15T22:45:51.817 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch_barrier": 86, 2024-09-15T22:45:51.817 INFO:teuthology.orchestra.run.smithi111.stdout: "uptime": 117.239560377 2024-09-15T22:45:51.817 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-15T22:45:51.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.810+0000 7f3056a00640 1 -- 172.21.15.111:0/2256131247 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f30380a3240 msgr2=0x7f30380a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:51.817 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.810+0000 7f3056a00640 1 --2- 172.21.15.111:0/2256131247 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f30380a3240 0x7f30380a5700 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f304409c200 tx=0x7f30440d3690 comp rx=0 tx=0).stop 2024-09-15T22:45:51.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.810+0000 7f3056a00640 1 -- 172.21.15.111:0/2256131247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3050103e30 msgr2=0x7f305019a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:51.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.810+0000 7f3056a00640 1 --2- 172.21.15.111:0/2256131247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3050103e30 0x7f305019a470 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f304400e9f0 tx=0x7f304400eeb0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.810+0000 7f3056a00640 1 -- 172.21.15.111:0/2256131247 shutdown_connections 2024-09-15T22:45:51.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.810+0000 7f3056a00640 1 --2- 172.21.15.111:0/2256131247 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3050108f60 0x7f305019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.810+0000 7f3056a00640 1 --2- 172.21.15.111:0/2256131247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30501047e0 0x7f305019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.810+0000 7f3056a00640 1 --2- 172.21.15.111:0/2256131247 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f30380a3240 0x7f30380a5700 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.810+0000 7f3056a00640 1 --2- 172.21.15.111:0/2256131247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3050103e30 0x7f305019a470 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:51.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.810+0000 7f3056a00640 1 -- 172.21.15.111:0/2256131247 >> 172.21.15.111:0/2256131247 conn(0x7f30500ff910 msgr2=0x7f3050100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:51.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.810+0000 7f3056a00640 1 -- 172.21.15.111:0/2256131247 shutdown_connections 2024-09-15T22:45:51.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:51.810+0000 7f3056a00640 1 -- 172.21.15.111:0/2256131247 wait complete. 2024-09-15T22:45:51.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:51 smithi121 ceph-mon[34815]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:45:51.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:51 smithi121 ceph-mon[34815]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-15T22:45:51.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:51 smithi111 ceph-mon[30629]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:45:51.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:51 smithi111 ceph-mon[30629]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-15T22:45:52.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:51 smithi192 ceph-mon[35427]: pgmap v348: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:45:52.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:51 smithi192 ceph-mon[35427]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-15T22:45:52.796 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-15T22:45:53.162 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:53.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.778+0000 7f744f200640 1 -- 172.21.15.111:0/2364451103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74501079c0 msgr2=0x7f7450107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:53.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.778+0000 7f744f200640 1 --2- 172.21.15.111:0/2364451103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74501079c0 0x7f7450107da0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7440009cd0 tx=0x7f744002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.778+0000 7f744f200640 1 -- 172.21.15.111:0/2364451103 shutdown_connections 2024-09-15T22:45:53.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.778+0000 7f744f200640 1 --2- 172.21.15.111:0/2364451103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7450102360 0x7f745010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.778+0000 7f744f200640 1 --2- 172.21.15.111:0/2364451103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74501019c0 0x7f7450101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.778+0000 7f744f200640 1 --2- 172.21.15.111:0/2364451103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74501079c0 0x7f7450107da0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.778+0000 7f744f200640 1 -- 172.21.15.111:0/2364451103 >> 172.21.15.111:0/2364451103 conn(0x7f74500fd740 msgr2=0x7f74500ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:53.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.778+0000 7f744f200640 1 -- 172.21.15.111:0/2364451103 shutdown_connections 2024-09-15T22:45:53.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.778+0000 7f744f200640 1 -- 172.21.15.111:0/2364451103 wait complete. 2024-09-15T22:45:53.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.779+0000 7f744f200640 1 Processor -- start 2024-09-15T22:45:53.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.779+0000 7f744f200640 1 -- start start 2024-09-15T22:45:53.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.779+0000 7f744f200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74501019c0 0x7f74501088f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.780+0000 7f744f200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7450102360 0x7f7450108e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.780+0000 7f744f200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74501079c0 0x7f7450109370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.780+0000 7f744f200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7450111dc0 con 0x7f74501019c0 2024-09-15T22:45:53.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.780+0000 7f744f200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7450111c40 con 0x7f7450102360 2024-09-15T22:45:53.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.780+0000 7f744f200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7450111f40 con 0x7f74501079c0 2024-09-15T22:45:53.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.780+0000 7f744de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74501019c0 0x7f74501088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.780+0000 7f744de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74501019c0 0x7f74501088f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:35108/0 (socket says 172.21.15.111:35108) 2024-09-15T22:45:53.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.780+0000 7f744de00640 1 -- 172.21.15.111:0/2544893011 learned_addr learned my addr 172.21.15.111:0/2544893011 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:53.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.780+0000 7f744e800640 1 --2- 172.21.15.111:0/2544893011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74501079c0 0x7f7450109370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.780+0000 7f7445400640 1 --2- 172.21.15.111:0/2544893011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7450102360 0x7f7450108e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.780+0000 7f744de00640 1 -- 172.21.15.111:0/2544893011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74501079c0 msgr2=0x7f7450109370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:53.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.780+0000 7f744de00640 1 --2- 172.21.15.111:0/2544893011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74501079c0 0x7f7450109370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.780+0000 7f744de00640 1 -- 172.21.15.111:0/2544893011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7450102360 msgr2=0x7f7450108e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:53.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.780+0000 7f744de00640 1 --2- 172.21.15.111:0/2544893011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7450102360 0x7f7450108e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.780+0000 7f744de00640 1 -- 172.21.15.111:0/2544893011 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74501aabc0 con 0x7f74501019c0 2024-09-15T22:45:53.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.780+0000 7f744e800640 1 --2- 172.21.15.111:0/2544893011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74501079c0 0x7f7450109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:53.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.781+0000 7f744de00640 1 --2- 172.21.15.111:0/2544893011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74501019c0 0x7f74501088f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7440006010 tx=0x7f7440031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:53.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.781+0000 7f7447400640 1 -- 172.21.15.111:0/2544893011 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7440003e00 con 0x7f74501019c0 2024-09-15T22:45:53.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.781+0000 7f7447400640 1 -- 172.21.15.111:0/2544893011 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7440004520 con 0x7f74501019c0 2024-09-15T22:45:53.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.781+0000 7f7447400640 1 -- 172.21.15.111:0/2544893011 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7440030df0 con 0x7f74501019c0 2024-09-15T22:45:53.823 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.781+0000 7f744f200640 1 -- 172.21.15.111:0/2544893011 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74501aae50 con 0x7f74501019c0 2024-09-15T22:45:53.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.781+0000 7f744f200640 1 -- 172.21.15.111:0/2544893011 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74501ab300 con 0x7f74501019c0 2024-09-15T22:45:53.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.784+0000 7f7447400640 1 -- 172.21.15.111:0/2544893011 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f74400046d0 con 0x7f74501019c0 2024-09-15T22:45:53.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.784+0000 7f74576c2740 1 Processor -- start 2024-09-15T22:45:53.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.784+0000 7f74576c2740 1 -- start start 2024-09-15T22:45:53.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.785+0000 7f74576c2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e36c4e0 0x55a28e36c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.785+0000 7f74576c2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28e36ce00 0x55a28e377180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.785+0000 7f74576c2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28e3776c0 0x55a28e379ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.785+0000 7f74576c2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a28e2d20b0 con 0x55a28e36c4e0 2024-09-15T22:45:53.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.785+0000 7f74576c2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a28e279ab0 con 0x55a28e3776c0 2024-09-15T22:45:53.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.785+0000 7f74576c2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a28e2ce070 con 0x55a28e36ce00 2024-09-15T22:45:53.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.785+0000 7f744e800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e36c4e0 0x55a28e36c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.785+0000 7f744e800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e36c4e0 0x55a28e36c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:35114/0 (socket says 172.21.15.111:35114) 2024-09-15T22:45:53.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.785+0000 7f744e800640 1 -- 172.21.15.111:0/1148338758 learned_addr learned my addr 172.21.15.111:0/1148338758 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:53.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.785+0000 7f7445400640 1 --2- 172.21.15.111:0/1148338758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28e3776c0 0x55a28e379ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.785+0000 7f744de00640 1 --2- 172.21.15.111:0/1148338758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28e36ce00 0x55a28e377180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.786+0000 7f7445400640 1 -- 172.21.15.111:0/1148338758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28e36ce00 msgr2=0x55a28e377180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:53.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.786+0000 7f7445400640 1 --2- 172.21.15.111:0/1148338758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28e36ce00 0x55a28e377180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.786+0000 7f7445400640 1 -- 172.21.15.111:0/1148338758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e36c4e0 msgr2=0x55a28e36c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:53.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.786+0000 7f7445400640 1 --2- 172.21.15.111:0/1148338758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e36c4e0 0x55a28e36c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.786+0000 7f7445400640 1 -- 172.21.15.111:0/1148338758 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a28e2864a0 con 0x55a28e3776c0 2024-09-15T22:45:53.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.786+0000 7f744e800640 1 --2- 172.21.15.111:0/1148338758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e36c4e0 0x55a28e36c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:53.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.786+0000 7f7447400640 1 --2- 172.21.15.111:0/2544893011 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f74340a3240 0x7f74340a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.786+0000 7f7445400640 1 --2- 172.21.15.111:0/1148338758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28e3776c0 0x55a28e379ab0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f745010a140 tx=0x7f7438011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38bfa21f775bb91d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:53.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.786+0000 7f7447400640 1 -- 172.21.15.111:0/2544893011 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f744002e030 con 0x7f74501019c0 2024-09-15T22:45:53.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.786+0000 7f744e800640 1 --2- 172.21.15.111:0/2544893011 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f74340a3240 0x7f74340a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.786+0000 7f743f000640 1 -- 172.21.15.111:0/1148338758 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7438004230 con 0x55a28e3776c0 2024-09-15T22:45:53.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.786+0000 7f743f000640 1 -- 172.21.15.111:0/1148338758 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7438009e10 con 0x55a28e3776c0 2024-09-15T22:45:53.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.786+0000 7f743f000640 1 -- 172.21.15.111:0/1148338758 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7438004d70 con 0x55a28e3776c0 2024-09-15T22:45:53.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.787+0000 7f74576c2740 1 -- 172.21.15.111:0/1148338758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28e3776c0 msgr2=0x55a28e379ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:53.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.787+0000 7f74576c2740 1 --2- 172.21.15.111:0/1148338758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28e3776c0 0x55a28e379ab0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f745010a140 tx=0x7f7438011810 comp rx=0 tx=0).stop 2024-09-15T22:45:53.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.787+0000 7f744e800640 1 --2- 172.21.15.111:0/2544893011 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f74340a3240 0x7f74340a5700 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7448005f60 tx=0x7f7448005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:53.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.787+0000 7f74576c2740 1 -- 172.21.15.111:0/1148338758 shutdown_connections 2024-09-15T22:45:53.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.787+0000 7f74576c2740 1 --2- 172.21.15.111:0/1148338758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28e3776c0 0x55a28e379ab0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.787+0000 7f74576c2740 1 --2- 172.21.15.111:0/1148338758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28e36ce00 0x55a28e377180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.787+0000 7f74576c2740 1 --2- 172.21.15.111:0/1148338758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e36c4e0 0x55a28e36c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.787+0000 7f74576c2740 1 -- 172.21.15.111:0/1148338758 >> 172.21.15.111:0/1148338758 conn(0x55a28e368a70 msgr2=0x55a28e368e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:53.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.787+0000 7f74576c2740 1 -- 172.21.15.111:0/1148338758 shutdown_connections 2024-09-15T22:45:53.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.787+0000 7f74576c2740 1 -- 172.21.15.111:0/1148338758 wait complete. 2024-09-15T22:45:53.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.788+0000 7f74576c2740 1 -- start start 2024-09-15T22:45:53.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.788+0000 7f74576c2740 1 Processor -- start 2024-09-15T22:45:53.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.788+0000 7f74576c2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28e36c4e0 0x55a28e374590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.788+0000 7f74576c2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28e36ce00 0x55a28e374ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.788+0000 7f74576c2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e3776c0 0x55a28e375010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.788+0000 7f74576c2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a28e37a330 con 0x55a28e3776c0 2024-09-15T22:45:53.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.788+0000 7f74576c2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a28e25a920 con 0x55a28e36ce00 2024-09-15T22:45:53.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.788+0000 7f74576c2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a28e37a4b0 con 0x55a28e36c4e0 2024-09-15T22:45:53.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.788+0000 7f74576c2740 10 client.0 _ll_register_callbacks cb 0x55a28e27e8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:53.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.789+0000 7f744e800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e3776c0 0x55a28e375010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.789+0000 7f744e800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e3776c0 0x55a28e375010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:35122/0 (socket says 172.21.15.111:35122) 2024-09-15T22:45:53.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.789+0000 7f744e800640 1 -- 172.21.15.111:0/3637842375 learned_addr learned my addr 172.21.15.111:0/3637842375 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:53.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.789+0000 7f744de00640 1 --2- 172.21.15.111:0/3637842375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28e36c4e0 0x55a28e374590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.827 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.789+0000 7f7445400640 1 --2- 172.21.15.111:0/3637842375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28e36ce00 0x55a28e374ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.789+0000 7f744de00640 1 -- 172.21.15.111:0/3637842375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28e36ce00 msgr2=0x55a28e374ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:53.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.789+0000 7f744de00640 1 --2- 172.21.15.111:0/3637842375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28e36ce00 0x55a28e374ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.789+0000 7f744de00640 1 -- 172.21.15.111:0/3637842375 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e3776c0 msgr2=0x55a28e375010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:53.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.789+0000 7f744de00640 1 --2- 172.21.15.111:0/3637842375 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e3776c0 0x55a28e375010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.789+0000 7f744de00640 1 -- 172.21.15.111:0/3637842375 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a28e0432b0 con 0x55a28e36c4e0 2024-09-15T22:45:53.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.789+0000 7f7445400640 1 --2- 172.21.15.111:0/3637842375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28e36ce00 0x55a28e374ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:53.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.789+0000 7f744e800640 1 --2- 172.21.15.111:0/3637842375 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e3776c0 0x55a28e375010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:53.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.789+0000 7f744de00640 1 --2- 172.21.15.111:0/3637842375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28e36c4e0 0x55a28e374590 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55a28e3698d0 tx=0x7f7440005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3cf4a010ccdb383 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:53.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.789+0000 7f7433400640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:45:53.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.790+0000 7f7433400640 1 -- 172.21.15.111:0/3637842375 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74400f3e60 con 0x55a28e36c4e0 2024-09-15T22:45:53.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.790+0000 7f7433400640 1 -- 172.21.15.111:0/3637842375 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f74400305a0 con 0x55a28e36c4e0 2024-09-15T22:45:53.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.790+0000 7f74576c2740 1 -- 172.21.15.111:0/3637842375 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7438009e10 con 0x55a28e36c4e0 2024-09-15T22:45:53.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.790+0000 7f7433400640 1 -- 172.21.15.111:0/3637842375 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f74400f6020 con 0x55a28e36c4e0 2024-09-15T22:45:53.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.790+0000 7f74576c2740 10 client.24922 fetch_fsmap learned FSMap version 25 2024-09-15T22:45:53.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.790+0000 7f74576c2740 1 -- 172.21.15.111:0/3637842375 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a28e37a380 con 0x55a28e36c4e0 2024-09-15T22:45:53.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.790+0000 7f7433400640 1 -- 172.21.15.111:0/3637842375 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7438009e10 con 0x55a28e36c4e0 2024-09-15T22:45:53.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.790+0000 7f7433400640 1 -- 172.21.15.111:0/3637842375 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f74400f0ce0 con 0x55a28e36c4e0 2024-09-15T22:45:53.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.790+0000 7f74576c2740 10 client.24922 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:45:53.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.790+0000 7f74576c2740 10 client.24922 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:45:53.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.790+0000 7f74576c2740 20 client.24922 populate_metadata read hostname 'smithi111' 2024-09-15T22:45:53.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.790+0000 7f74576c2740 1 --2- 172.21.15.111:0/3637842375 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a28e3885d0 0x55a28e389a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.790+0000 7f74576c2740 4 client.24922 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:45:53.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.790+0000 7f74576c2740 1 -- 172.21.15.111:0/3637842375 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a28e2df620 con 0x55a28e3885d0 2024-09-15T22:45:53.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.791+0000 7f7445400640 1 --2- 172.21.15.111:0/3637842375 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a28e3885d0 0x55a28e389a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.791+0000 7f7445400640 1 --2- 172.21.15.111:0/3637842375 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a28e3885d0 0x55a28e389a40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cf64bd288005a5c server_cookie=645cdfeb896febc4 in_seq=0 out_seq=0 2024-09-15T22:45:53.829 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.791+0000 7f7433400640 10 client.24922 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:45:53.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.792+0000 7f7433400640 1 -- 172.21.15.111:0/3637842375 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a28e2df620 con 0x55a28e3885d0 2024-09-15T22:45:53.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.792+0000 7f7433400640 10 client.24922 handle_command_reply: tid=0 2024-09-15T22:45:53.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.792+0000 7f74576c2740 1 client.24922 shutdown 2024-09-15T22:45:53.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.792+0000 7f74576c2740 1 -- 172.21.15.111:0/3637842375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28e36c4e0 msgr2=0x55a28e374590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:53.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.792+0000 7f74576c2740 1 --2- 172.21.15.111:0/3637842375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28e36c4e0 0x55a28e374590 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55a28e3698d0 tx=0x7f7440005f80 comp rx=0 tx=0).stop 2024-09-15T22:45:53.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.792+0000 7f74576c2740 1 -- 172.21.15.111:0/3637842375 shutdown_connections 2024-09-15T22:45:53.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.792+0000 7f74576c2740 1 --2- 172.21.15.111:0/3637842375 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a28e3885d0 0x55a28e389a40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.792+0000 7f74576c2740 1 --2- 172.21.15.111:0/3637842375 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e3776c0 0x55a28e375010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.830 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.793+0000 7f74576c2740 1 --2- 172.21.15.111:0/3637842375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28e36ce00 0x55a28e374ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.793+0000 7f74576c2740 1 --2- 172.21.15.111:0/3637842375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28e36c4e0 0x55a28e374590 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.793+0000 7f7433400640 0 client.24922 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:45:53.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.793+0000 7f74576c2740 1 -- 172.21.15.111:0/3637842375 >> 172.21.15.111:0/3637842375 conn(0x55a28e368a70 msgr2=0x55a28e36e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:53.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.793+0000 7f74576c2740 1 -- 172.21.15.111:0/3637842375 shutdown_connections 2024-09-15T22:45:53.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.793+0000 7f74576c2740 1 -- 172.21.15.111:0/3637842375 wait complete. 2024-09-15T22:45:53.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.793+0000 7f74576c2740 20 client.24922 trim_cache size 0 max 16384 2024-09-15T22:45:53.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.811+0000 7f74576c2740 1 Processor -- start 2024-09-15T22:45:53.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.811+0000 7f74576c2740 1 -- start start 2024-09-15T22:45:53.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.812+0000 7f74576c2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28e36c4e0 0x55a28e374ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.812+0000 7f74576c2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e36ce00 0x55a28e375020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.812+0000 7f74576c2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28e3776c0 0x55a28e3717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.812+0000 7f74576c2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a28e387d80 con 0x55a28e36ce00 2024-09-15T22:45:53.831 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.812+0000 7f74576c2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a28e388450 con 0x55a28e3776c0 2024-09-15T22:45:53.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.812+0000 7f74576c2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a28e388b20 con 0x55a28e36c4e0 2024-09-15T22:45:53.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.812+0000 7f7445400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e36ce00 0x55a28e375020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.812+0000 7f7445400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e36ce00 0x55a28e375020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:35134/0 (socket says 172.21.15.111:35134) 2024-09-15T22:45:53.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.812+0000 7f7445400640 1 -- 172.21.15.111:0/3794463456 learned_addr learned my addr 172.21.15.111:0/3794463456 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:53.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.812+0000 7f744de00640 1 --2- 172.21.15.111:0/3794463456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28e36c4e0 0x55a28e374ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.812+0000 7f744e800640 1 --2- 172.21.15.111:0/3794463456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28e3776c0 0x55a28e3717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.812+0000 7f7445400640 1 -- 172.21.15.111:0/3794463456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28e36c4e0 msgr2=0x55a28e374ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:53.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.812+0000 7f7445400640 1 --2- 172.21.15.111:0/3794463456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28e36c4e0 0x55a28e374ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.812+0000 7f7445400640 1 -- 172.21.15.111:0/3794463456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28e3776c0 msgr2=0x55a28e3717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:53.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.812+0000 7f7445400640 1 --2- 172.21.15.111:0/3794463456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28e3776c0 0x55a28e3717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.812+0000 7f7445400640 1 -- 172.21.15.111:0/3794463456 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a28e36c310 con 0x55a28e36ce00 2024-09-15T22:45:53.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.813+0000 7f7445400640 1 --2- 172.21.15.111:0/3794463456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e36ce00 0x55a28e375020 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7438009b20 tx=0x7f7438004880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328b2e6d54fe2b4c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:53.832 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.813+0000 7f743e600640 1 -- 172.21.15.111:0/3794463456 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743801e040 con 0x55a28e36ce00 2024-09-15T22:45:53.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.813+0000 7f743e600640 1 -- 172.21.15.111:0/3794463456 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7438029070 con 0x55a28e36ce00 2024-09-15T22:45:53.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.813+0000 7f743e600640 1 -- 172.21.15.111:0/3794463456 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743802f690 con 0x55a28e36ce00 2024-09-15T22:45:53.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.814+0000 7f74576c2740 1 -- 172.21.15.111:0/3794463456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e36ce00 msgr2=0x55a28e375020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:53.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.814+0000 7f74576c2740 1 --2- 172.21.15.111:0/3794463456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e36ce00 0x55a28e375020 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7438009b20 tx=0x7f7438004880 comp rx=0 tx=0).stop 2024-09-15T22:45:53.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.814+0000 7f74576c2740 1 -- 172.21.15.111:0/3794463456 shutdown_connections 2024-09-15T22:45:53.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.814+0000 7f74576c2740 1 --2- 172.21.15.111:0/3794463456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28e3776c0 0x55a28e3717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.814+0000 7f74576c2740 1 --2- 172.21.15.111:0/3794463456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e36ce00 0x55a28e375020 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.814+0000 7f74576c2740 1 --2- 172.21.15.111:0/3794463456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28e36c4e0 0x55a28e374ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.814+0000 7f74576c2740 1 -- 172.21.15.111:0/3794463456 >> 172.21.15.111:0/3794463456 conn(0x55a28e368a70 msgr2=0x55a28e372e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:53.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.814+0000 7f74576c2740 1 -- 172.21.15.111:0/3794463456 shutdown_connections 2024-09-15T22:45:53.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.814+0000 7f74576c2740 1 -- 172.21.15.111:0/3794463456 wait complete. 2024-09-15T22:45:53.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.814+0000 7f74576c2740 1 -- start start 2024-09-15T22:45:53.833 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.814+0000 7f74576c2740 1 Processor -- start 2024-09-15T22:45:53.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.815+0000 7f74576c2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28e36c4e0 0x55a28e392280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.815+0000 7f74576c2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e36ce00 0x55a28e38b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.815+0000 7f74576c2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28e3776c0 0x55a28e38baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.815+0000 7f74576c2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a28e3898c0 con 0x55a28e36ce00 2024-09-15T22:45:53.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.815+0000 7f74576c2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a28e3891f0 con 0x55a28e3776c0 2024-09-15T22:45:53.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.815+0000 7f74576c2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a28e389f90 con 0x55a28e36c4e0 2024-09-15T22:45:53.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.815+0000 7f74576c2740 10 client.0 _ll_register_callbacks cb 0x7f7440009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:45:53.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.815+0000 7f7445400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e36ce00 0x55a28e38b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.815+0000 7f7445400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e36ce00 0x55a28e38b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:35148/0 (socket says 172.21.15.111:35148) 2024-09-15T22:45:53.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.815+0000 7f7445400640 1 -- 172.21.15.111:0/1968170356 learned_addr learned my addr 172.21.15.111:0/1968170356 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:53.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.815+0000 7f744e800640 1 --2- 172.21.15.111:0/1968170356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28e3776c0 0x55a28e38baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.815+0000 7f744de00640 1 --2- 172.21.15.111:0/1968170356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28e36c4e0 0x55a28e392280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.834 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.815+0000 7f7445400640 1 -- 172.21.15.111:0/1968170356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28e36c4e0 msgr2=0x55a28e392280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:53.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.815+0000 7f7445400640 1 --2- 172.21.15.111:0/1968170356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28e36c4e0 0x55a28e392280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.815+0000 7f7445400640 1 -- 172.21.15.111:0/1968170356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28e3776c0 msgr2=0x55a28e38baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:45:53.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.815+0000 7f7445400640 1 --2- 172.21.15.111:0/1968170356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28e3776c0 0x55a28e38baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.815+0000 7f7445400640 1 -- 172.21.15.111:0/1968170356 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a28e38c190 con 0x55a28e36ce00 2024-09-15T22:45:53.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.816+0000 7f7445400640 1 --2- 172.21.15.111:0/1968170356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e36ce00 0x55a28e38b5b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7438009990 tx=0x7f7438005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ee8664cdb946eb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:53.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.816+0000 7f7433400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:45:53.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.816+0000 7f7433400640 1 -- 172.21.15.111:0/1968170356 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7438004350 con 0x55a28e36ce00 2024-09-15T22:45:53.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.816+0000 7f7433400640 1 -- 172.21.15.111:0/1968170356 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f74380044f0 con 0x55a28e36ce00 2024-09-15T22:45:53.835 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.816+0000 7f74576c2740 1 -- 172.21.15.111:0/1968170356 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7438029070 con 0x55a28e36ce00 2024-09-15T22:45:53.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.816+0000 7f7433400640 1 -- 172.21.15.111:0/1968170356 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f74380035f0 con 0x55a28e36ce00 2024-09-15T22:45:53.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.816+0000 7f74576c2740 10 client.14898 fetch_fsmap learned FSMap version 25 2024-09-15T22:45:53.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.816+0000 7f74576c2740 1 -- 172.21.15.111:0/1968170356 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a28e3f2ce0 con 0x55a28e36ce00 2024-09-15T22:45:53.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.817+0000 7f7433400640 1 -- 172.21.15.111:0/1968170356 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7438029070 con 0x55a28e36ce00 2024-09-15T22:45:53.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.817+0000 7f7433400640 1 -- 172.21.15.111:0/1968170356 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7438018070 con 0x55a28e36ce00 2024-09-15T22:45:53.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.817+0000 7f74576c2740 10 client.14898 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:45:53.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.817+0000 7f74576c2740 10 client.14898 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:45:53.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.817+0000 7f74576c2740 20 client.14898 populate_metadata read hostname 'smithi111' 2024-09-15T22:45:53.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.817+0000 7f74576c2740 1 --2- 172.21.15.111:0/1968170356 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a28e3f70c0 0x55a28e3f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:53.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.817+0000 7f74576c2740 4 client.14898 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:45:53.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.817+0000 7f74576c2740 1 -- 172.21.15.111:0/1968170356 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "status"}) -- 0x55a28e2d9b30 con 0x55a28e3f70c0 2024-09-15T22:45:53.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.817+0000 7f744de00640 1 --2- 172.21.15.111:0/1968170356 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a28e3f70c0 0x55a28e3f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:53.836 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.818+0000 7f744de00640 1 --2- 172.21.15.111:0/1968170356 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a28e3f70c0 0x55a28e3f5dd0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf973474e1906e3f server_cookie=982564fb9d7e4ace in_seq=0 out_seq=0 2024-09-15T22:45:53.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.818+0000 7f7433400640 10 client.14898 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:45:53.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.818+0000 7f7433400640 1 -- 172.21.15.111:0/1968170356 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a28e2d9b30 con 0x55a28e3f70c0 2024-09-15T22:45:53.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.818+0000 7f7433400640 10 client.14898 handle_command_reply: tid=0 2024-09-15T22:45:53.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.818+0000 7f74576c2740 1 client.14898 shutdown 2024-09-15T22:45:53.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.818+0000 7f74576c2740 1 -- 172.21.15.111:0/1968170356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e36ce00 msgr2=0x55a28e38b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:53.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.818+0000 7f74576c2740 1 --2- 172.21.15.111:0/1968170356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e36ce00 0x55a28e38b5b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7438009990 tx=0x7f7438005fb0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.818+0000 7f74576c2740 1 -- 172.21.15.111:0/1968170356 shutdown_connections 2024-09-15T22:45:53.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.818+0000 7f74576c2740 1 --2- 172.21.15.111:0/1968170356 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a28e3f70c0 0x55a28e3f5dd0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.818+0000 7f74576c2740 1 --2- 172.21.15.111:0/1968170356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28e3776c0 0x55a28e38baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.818+0000 7f74576c2740 1 --2- 172.21.15.111:0/1968170356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28e36ce00 0x55a28e38b5b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.818+0000 7f74576c2740 1 --2- 172.21.15.111:0/1968170356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28e36c4e0 0x55a28e392280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.818+0000 7f7433400640 0 client.14898 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:45:53.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.818+0000 7f74576c2740 1 -- 172.21.15.111:0/1968170356 >> 172.21.15.111:0/1968170356 conn(0x55a28e368a70 msgr2=0x55a28e382700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:53.837 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.818+0000 7f74576c2740 1 -- 172.21.15.111:0/1968170356 shutdown_connections 2024-09-15T22:45:53.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.818+0000 7f74576c2740 1 -- 172.21.15.111:0/1968170356 wait complete. 2024-09-15T22:45:53.838 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.819+0000 7f74576c2740 20 client.14898 trim_cache size 0 max 16384 2024-09-15T22:45:53.838 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-15T22:45:53.838 INFO:teuthology.orchestra.run.smithi111.stdout: "cluster_fsid": "b7614d3c-73b2-11ef-bceb-c7b262605968", 2024-09-15T22:45:53.838 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": 0, 2024-09-15T22:45:53.838 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 24545, 2024-09-15T22:45:53.838 INFO:teuthology.orchestra.run.smithi111.stdout: "want_state": "up:active", 2024-09-15T22:45:53.838 INFO:teuthology.orchestra.run.smithi111.stdout: "state": "up:active", 2024-09-15T22:45:53.839 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs", 2024-09-15T22:45:53.839 INFO:teuthology.orchestra.run.smithi111.stdout: "rank_uptime": 94.617552548000006, 2024-09-15T22:45:53.839 INFO:teuthology.orchestra.run.smithi111.stdout: "mdsmap_epoch": 25, 2024-09-15T22:45:53.839 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch": 88, 2024-09-15T22:45:53.839 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch_barrier": 86, 2024-09-15T22:45:53.839 INFO:teuthology.orchestra.run.smithi111.stdout: "uptime": 108.920436303 2024-09-15T22:45:53.839 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-15T22:45:53.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.819+0000 7f744f200640 1 -- 172.21.15.111:0/2544893011 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f74340a3240 msgr2=0x7f74340a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:53.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.819+0000 7f744f200640 1 --2- 172.21.15.111:0/2544893011 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f74340a3240 0x7f74340a5700 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7448005f60 tx=0x7f7448005f90 comp rx=0 tx=0).stop 2024-09-15T22:45:53.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.819+0000 7f744f200640 1 -- 172.21.15.111:0/2544893011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74501019c0 msgr2=0x7f74501088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:53.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.819+0000 7f744f200640 1 --2- 172.21.15.111:0/2544893011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74501019c0 0x7f74501088f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7440006010 tx=0x7f7440031e60 comp rx=0 tx=0).stop 2024-09-15T22:45:53.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.819+0000 7f744f200640 1 -- 172.21.15.111:0/2544893011 shutdown_connections 2024-09-15T22:45:53.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.819+0000 7f744f200640 1 --2- 172.21.15.111:0/2544893011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74501079c0 0x7f7450109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.819+0000 7f744f200640 1 --2- 172.21.15.111:0/2544893011 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f74340a3240 0x7f74340a5700 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.840 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.819+0000 7f744f200640 1 --2- 172.21.15.111:0/2544893011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7450102360 0x7f7450108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.819+0000 7f744f200640 1 --2- 172.21.15.111:0/2544893011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74501019c0 0x7f74501088f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:53.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.819+0000 7f744f200640 1 -- 172.21.15.111:0/2544893011 >> 172.21.15.111:0/2544893011 conn(0x7f74500fd740 msgr2=0x7f745010cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:53.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.819+0000 7f744f200640 1 -- 172.21.15.111:0/2544893011 shutdown_connections 2024-09-15T22:45:53.841 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:53.819+0000 7f744f200640 1 -- 172.21.15.111:0/2544893011 wait complete. 2024-09-15T22:45:53.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:53 smithi121 ceph-mon[34815]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-15T22:45:53.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:53 smithi111 ceph-mon[30629]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-15T22:45:54.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:53 smithi192 ceph-mon[35427]: pgmap v349: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-09-15T22:45:54.810 INFO:tasks.cephadm:Setup complete, yielding 2024-09-15T22:45:54.810 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-15T22:45:55.501 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi111.front.sepia.ceph.com 2024-09-15T22:45:55.502 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-15T22:45:55.873 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:55.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:55 smithi121 ceph-mon[34815]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-15T22:45:55.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:55 smithi111 ceph-mon[30629]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-15T22:45:56.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:55 smithi192 ceph-mon[35427]: pgmap v350: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-15T22:45:56.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.478+0000 7f3137e00640 1 -- 172.21.15.111:0/3439420311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3138105810 msgr2=0x7f313810c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:56.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.478+0000 7f3137e00640 1 --2- 172.21.15.111:0/3439420311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3138105810 0x7f313810c0a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3130009ca0 tx=0x7f313002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:45:56.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.478+0000 7f3137e00640 1 -- 172.21.15.111:0/3439420311 shutdown_connections 2024-09-15T22:45:56.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.478+0000 7f3137e00640 1 --2- 172.21.15.111:0/3439420311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3138105810 0x7f313810c0a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:56.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.478+0000 7f3137e00640 1 --2- 172.21.15.111:0/3439420311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3138104e50 0x7f31381052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:56.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.478+0000 7f3137e00640 1 --2- 172.21.15.111:0/3439420311 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3138103c50 0x7f3138104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:56.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.478+0000 7f3137e00640 1 -- 172.21.15.111:0/3439420311 >> 172.21.15.111:0/3439420311 conn(0x7f31380ff3e0 msgr2=0x7f3138101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:56.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.478+0000 7f3137e00640 1 -- 172.21.15.111:0/3439420311 shutdown_connections 2024-09-15T22:45:56.480 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.478+0000 7f3137e00640 1 -- 172.21.15.111:0/3439420311 wait complete. 2024-09-15T22:45:56.481 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.479+0000 7f3137e00640 1 Processor -- start 2024-09-15T22:45:56.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.479+0000 7f3137e00640 1 -- start start 2024-09-15T22:45:56.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.480+0000 7f3137e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3138103c50 0x7f3138199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:56.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.480+0000 7f3137e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3138104e50 0x7f313819a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:56.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.480+0000 7f3137e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3138105810 0x7f31381a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:56.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.480+0000 7f3137e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f313810f8b0 con 0x7f3138103c50 2024-09-15T22:45:56.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.480+0000 7f3137e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f313810f730 con 0x7f3138105810 2024-09-15T22:45:56.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.480+0000 7f3137e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f313810fa30 con 0x7f3138104e50 2024-09-15T22:45:56.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.480+0000 7f3136a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3138103c50 0x7f3138199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:56.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.480+0000 7f3136a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3138103c50 0x7f3138199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:35170/0 (socket says 172.21.15.111:35170) 2024-09-15T22:45:56.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.480+0000 7f3136a00640 1 -- 172.21.15.111:0/4274524163 learned_addr learned my addr 172.21.15.111:0/4274524163 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:56.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.480+0000 7f312e000640 1 --2- 172.21.15.111:0/4274524163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3138104e50 0x7f313819a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:56.482 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.480+0000 7f3137400640 1 --2- 172.21.15.111:0/4274524163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3138105810 0x7f31381a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:56.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.480+0000 7f3136a00640 1 -- 172.21.15.111:0/4274524163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3138104e50 msgr2=0x7f313819a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:56.483 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.481+0000 7f3136a00640 1 --2- 172.21.15.111:0/4274524163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3138104e50 0x7f313819a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:56.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.481+0000 7f3136a00640 1 -- 172.21.15.111:0/4274524163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3138105810 msgr2=0x7f31381a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:56.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.481+0000 7f3136a00640 1 --2- 172.21.15.111:0/4274524163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3138105810 0x7f31381a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:56.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.481+0000 7f3136a00640 1 -- 172.21.15.111:0/4274524163 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31381a1ab0 con 0x7f3138103c50 2024-09-15T22:45:56.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.481+0000 7f3136a00640 1 --2- 172.21.15.111:0/4274524163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3138103c50 0x7f3138199e80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f312800e9c0 tx=0x7f312800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:56.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.481+0000 7f312fe00640 1 -- 172.21.15.111:0/4274524163 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312800cd80 con 0x7f3138103c50 2024-09-15T22:45:56.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.481+0000 7f312fe00640 1 -- 172.21.15.111:0/4274524163 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31280035f0 con 0x7f3138103c50 2024-09-15T22:45:56.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.481+0000 7f3137e00640 1 -- 172.21.15.111:0/4274524163 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31381a1da0 con 0x7f3138103c50 2024-09-15T22:45:56.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.481+0000 7f312fe00640 1 -- 172.21.15.111:0/4274524163 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3128010690 con 0x7f3138103c50 2024-09-15T22:45:56.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.481+0000 7f3137e00640 1 -- 172.21.15.111:0/4274524163 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31381a2250 con 0x7f3138103c50 2024-09-15T22:45:56.484 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.482+0000 7f3137e00640 1 -- 172.21.15.111:0/4274524163 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f313810b000 con 0x7f3138103c50 2024-09-15T22:45:56.485 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.484+0000 7f312fe00640 1 -- 172.21.15.111:0/4274524163 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f31280037a0 con 0x7f3138103c50 2024-09-15T22:45:56.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.485+0000 7f312fe00640 1 --2- 172.21.15.111:0/4274524163 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f311c0a32f0 0x7f311c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:56.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.486+0000 7f312fe00640 1 -- 172.21.15.111:0/4274524163 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f31280cfc10 con 0x7f3138103c50 2024-09-15T22:45:56.487 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.486+0000 7f312e000640 1 --2- 172.21.15.111:0/4274524163 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f311c0a32f0 0x7f311c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:56.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.486+0000 7f312e000640 1 --2- 172.21.15.111:0/4274524163 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f311c0a32f0 0x7f311c0a57b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f313819b3a0 tx=0x7f3120005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:56.491 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.490+0000 7f312fe00640 1 -- 172.21.15.111:0/4274524163 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f312809c930 con 0x7f3138103c50 2024-09-15T22:45:56.686 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.684+0000 7f3137e00640 1 -- 172.21.15.111:0/4274524163 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f3138062630 con 0x7f311c0a32f0 2024-09-15T22:45:56.687 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.686+0000 7f312fe00640 1 -- 172.21.15.111:0/4274524163 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f3138062630 con 0x7f311c0a32f0 2024-09-15T22:45:56.687 INFO:teuthology.orchestra.run.smithi111.stdout:Backend: cephadm 2024-09-15T22:45:56.687 INFO:teuthology.orchestra.run.smithi111.stdout:Available: Yes 2024-09-15T22:45:56.687 INFO:teuthology.orchestra.run.smithi111.stdout:Paused: No 2024-09-15T22:45:56.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.688+0000 7f3137e00640 1 -- 172.21.15.111:0/4274524163 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f311c0a32f0 msgr2=0x7f311c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:56.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.688+0000 7f3137e00640 1 --2- 172.21.15.111:0/4274524163 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f311c0a32f0 0x7f311c0a57b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f313819b3a0 tx=0x7f3120005f90 comp rx=0 tx=0).stop 2024-09-15T22:45:56.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.688+0000 7f3137e00640 1 -- 172.21.15.111:0/4274524163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3138103c50 msgr2=0x7f3138199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:56.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.688+0000 7f3137e00640 1 --2- 172.21.15.111:0/4274524163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3138103c50 0x7f3138199e80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f312800e9c0 tx=0x7f312800ee80 comp rx=0 tx=0).stop 2024-09-15T22:45:56.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.688+0000 7f3137e00640 1 -- 172.21.15.111:0/4274524163 shutdown_connections 2024-09-15T22:45:56.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.688+0000 7f3137e00640 1 --2- 172.21.15.111:0/4274524163 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f311c0a32f0 0x7f311c0a57b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:56.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.688+0000 7f3137e00640 1 --2- 172.21.15.111:0/4274524163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3138105810 0x7f31381a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:56.690 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.688+0000 7f3137e00640 1 --2- 172.21.15.111:0/4274524163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3138104e50 0x7f313819a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:56.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.688+0000 7f3137e00640 1 --2- 172.21.15.111:0/4274524163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3138103c50 0x7f3138199e80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:56.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.688+0000 7f3137e00640 1 -- 172.21.15.111:0/4274524163 >> 172.21.15.111:0/4274524163 conn(0x7f31380ff3e0 msgr2=0x7f31381081d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:56.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.688+0000 7f3137e00640 1 -- 172.21.15.111:0/4274524163 shutdown_connections 2024-09-15T22:45:56.691 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:56.688+0000 7f3137e00640 1 -- 172.21.15.111:0/4274524163 wait complete. 2024-09-15T22:45:57.486 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-15T22:45:57.852 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:45:57.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:57 smithi111 ceph-mon[30629]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-15T22:45:57.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:57 smithi111 ceph-mon[30629]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:45:58.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:57 smithi192 ceph-mon[35427]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-15T22:45:58.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:57 smithi192 ceph-mon[35427]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:45:58.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:57 smithi121 ceph-mon[34815]: pgmap v351: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-15T22:45:58.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:57 smithi121 ceph-mon[34815]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:45:58.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.459+0000 7f7c12200640 1 -- 172.21.15.111:0/1021700111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c0c102680 msgr2=0x7f7c0c104a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:58.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.459+0000 7f7c12200640 1 --2- 172.21.15.111:0/1021700111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c0c102680 0x7f7c0c104a70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc009900 tx=0x7f7bfc02a3f0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.460+0000 7f7c12200640 1 -- 172.21.15.111:0/1021700111 shutdown_connections 2024-09-15T22:45:58.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.460+0000 7f7c12200640 1 --2- 172.21.15.111:0/1021700111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c0c1078e0 0x7f7c0c109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.460+0000 7f7c12200640 1 --2- 172.21.15.111:0/1021700111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c0c104fb0 0x7f7c0c1073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.460+0000 7f7c12200640 1 --2- 172.21.15.111:0/1021700111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c0c102680 0x7f7c0c104a70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.460+0000 7f7c12200640 1 -- 172.21.15.111:0/1021700111 >> 172.21.15.111:0/1021700111 conn(0x7f7c0c0fc280 msgr2=0x7f7c0c0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:58.501 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.460+0000 7f7c12200640 1 -- 172.21.15.111:0/1021700111 shutdown_connections 2024-09-15T22:45:58.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.460+0000 7f7c12200640 1 -- 172.21.15.111:0/1021700111 wait complete. 2024-09-15T22:45:58.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.460+0000 7f7c12200640 1 Processor -- start 2024-09-15T22:45:58.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.461+0000 7f7c12200640 1 -- start start 2024-09-15T22:45:58.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.461+0000 7f7c12200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c0c102680 0x7f7c0c19a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:58.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.461+0000 7f7c12200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c0c104fb0 0x7f7c0c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:58.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.461+0000 7f7c12200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c0c1078e0 0x7f7c0c1a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:58.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.461+0000 7f7c12200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c10d8e0 con 0x7f7c0c104fb0 2024-09-15T22:45:58.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.461+0000 7f7c12200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c10d760 con 0x7f7c0c1078e0 2024-09-15T22:45:58.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.461+0000 7f7c12200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c10da60 con 0x7f7c0c102680 2024-09-15T22:45:58.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.461+0000 7f7c0aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c0c104fb0 0x7f7c0c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:58.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.461+0000 7f7c0b400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c0c102680 0x7f7c0c19a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:58.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.461+0000 7f7c0b400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c0c102680 0x7f7c0c19a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:48672/0 (socket says 172.21.15.111:48672) 2024-09-15T22:45:58.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.461+0000 7f7c0b400640 1 -- 172.21.15.111:0/1872189099 learned_addr learned my addr 172.21.15.111:0/1872189099 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:45:58.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.461+0000 7f7c0be00640 1 --2- 172.21.15.111:0/1872189099 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c0c1078e0 0x7f7c0c1a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:58.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.462+0000 7f7c0b400640 1 -- 172.21.15.111:0/1872189099 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c0c1078e0 msgr2=0x7f7c0c1a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:58.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.462+0000 7f7c0b400640 1 --2- 172.21.15.111:0/1872189099 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c0c1078e0 0x7f7c0c1a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.462+0000 7f7c0b400640 1 -- 172.21.15.111:0/1872189099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c0c104fb0 msgr2=0x7f7c0c19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:58.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.462+0000 7f7c0b400640 1 --2- 172.21.15.111:0/1872189099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c0c104fb0 0x7f7c0c19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.462+0000 7f7c0b400640 1 -- 172.21.15.111:0/1872189099 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0c1a1d40 con 0x7f7c0c102680 2024-09-15T22:45:58.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.462+0000 7f7c0aa00640 1 --2- 172.21.15.111:0/1872189099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c0c104fb0 0x7f7c0c19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:45:58.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.462+0000 7f7c0b400640 1 --2- 172.21.15.111:0/1872189099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c0c102680 0x7f7c0c19a110 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc031440 tx=0x7f7bfc032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:58.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.462+0000 7f7c03e00640 1 -- 172.21.15.111:0/1872189099 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc03f030 con 0x7f7c0c102680 2024-09-15T22:45:58.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.462+0000 7f7c03e00640 1 -- 172.21.15.111:0/1872189099 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bfc003ce0 con 0x7f7c0c102680 2024-09-15T22:45:58.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.462+0000 7f7c12200640 1 -- 172.21.15.111:0/1872189099 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c0c1a1fd0 con 0x7f7c0c102680 2024-09-15T22:45:58.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.462+0000 7f7c03e00640 1 -- 172.21.15.111:0/1872189099 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc035070 con 0x7f7c0c102680 2024-09-15T22:45:58.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.463+0000 7f7c12200640 1 -- 172.21.15.111:0/1872189099 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c0c1a23d0 con 0x7f7c0c102680 2024-09-15T22:45:58.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.464+0000 7f7c12200640 1 -- 172.21.15.111:0/1872189099 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c0c109430 con 0x7f7c0c102680 2024-09-15T22:45:58.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.466+0000 7f7c03e00640 1 -- 172.21.15.111:0/1872189099 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f7bfc042030 con 0x7f7c0c102680 2024-09-15T22:45:58.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.467+0000 7f7c03e00640 1 --2- 172.21.15.111:0/1872189099 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7bf00a3270 0x7f7bf00a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:45:58.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.467+0000 7f7c03e00640 1 -- 172.21.15.111:0/1872189099 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f7bfc0f0260 con 0x7f7c0c102680 2024-09-15T22:45:58.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.467+0000 7f7c0aa00640 1 --2- 172.21.15.111:0/1872189099 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7bf00a3270 0x7f7bf00a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:45:58.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.468+0000 7f7c0aa00640 1 --2- 172.21.15.111:0/1872189099 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7bf00a3270 0x7f7bf00a5730 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c19b630 tx=0x7f7bf800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:45:58.504 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.469+0000 7f7c03e00640 1 -- 172.21.15.111:0/1872189099 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7bfc0bc930 con 0x7f7c0c102680 2024-09-15T22:45:58.656 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.654+0000 7f7c12200640 1 -- 172.21.15.111:0/1872189099 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f7c0c062630 con 0x7f7bf00a3270 2024-09-15T22:45:58.668 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.666+0000 7f7c03e00640 1 -- 172.21.15.111:0/1872189099 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7f7c0c062630 con 0x7f7bf00a3270 2024-09-15T22:45:58.668 INFO:teuthology.orchestra.run.smithi111.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-15T22:45:58.669 INFO:teuthology.orchestra.run.smithi111.stdout:mds.a smithi111 running (116s) 25s ago 116s 12.7M - 19.1.1-326-g8921c8ef f1455bd0219c acfd8775c53a 2024-09-15T22:45:58.669 INFO:teuthology.orchestra.run.smithi111.stdout:mds.b smithi121 running (114s) 44s ago 113s 25.5M - 19.1.1-326-g8921c8ef f1455bd0219c 14529e1860a7 2024-09-15T22:45:58.669 INFO:teuthology.orchestra.run.smithi111.stdout:mds.c smithi192 running (2m) 44s ago 2m 17.7M - 19.1.1-326-g8921c8ef f1455bd0219c 6f184d057968 2024-09-15T22:45:58.669 INFO:teuthology.orchestra.run.smithi111.stdout:mds.d smithi111 running (2m) 25s ago 2m 12.7M - 19.1.1-326-g8921c8ef f1455bd0219c 5dee87e37f44 2024-09-15T22:45:58.669 INFO:teuthology.orchestra.run.smithi111.stdout:mds.e smithi121 running (2m) 44s ago 2m 18.3M - 19.1.1-326-g8921c8ef f1455bd0219c 15264d5476ba 2024-09-15T22:45:58.669 INFO:teuthology.orchestra.run.smithi111.stdout:mds.f smithi192 running (2m) 44s ago 2m 16.9M - 19.1.1-326-g8921c8ef f1455bd0219c 3d5ca0b95ccd 2024-09-15T22:45:58.669 INFO:teuthology.orchestra.run.smithi111.stdout:mds.g smithi111 running (2m) 25s ago 119s 12.6M - 19.1.1-326-g8921c8ef f1455bd0219c 367ebeea7f71 2024-09-15T22:45:58.669 INFO:teuthology.orchestra.run.smithi111.stdout:mds.h smithi121 running (2m) 44s ago 2m 17.8M - 19.1.1-326-g8921c8ef f1455bd0219c c64e8fa330a3 2024-09-15T22:45:58.669 INFO:teuthology.orchestra.run.smithi111.stdout:mds.i smithi192 running (2m) 44s ago 2m 20.1M - 19.1.1-326-g8921c8ef f1455bd0219c 6327a8c10c2d 2024-09-15T22:45:58.669 INFO:teuthology.orchestra.run.smithi111.stdout:mds.j smithi111 running (2m) 25s ago 2m 12.7M - 19.1.1-326-g8921c8ef f1455bd0219c 395bf4c75bad 2024-09-15T22:45:58.669 INFO:teuthology.orchestra.run.smithi111.stdout:mds.k smithi121 running (2m) 44s ago 2m 14.4M - 19.1.1-326-g8921c8ef f1455bd0219c c856b2b02278 2024-09-15T22:45:58.669 INFO:teuthology.orchestra.run.smithi111.stdout:mds.l smithi192 running (2m) 44s ago 2m 14.5M - 19.1.1-326-g8921c8ef f1455bd0219c 9f94d35c51eb 2024-09-15T22:45:58.669 INFO:teuthology.orchestra.run.smithi111.stdout:mgr.x smithi111 *:9283,8765 running (9m) 25s ago 9m 508M - 19.1.1-326-g8921c8ef f1455bd0219c 4fb6ded66cc6 2024-09-15T22:45:58.670 INFO:teuthology.orchestra.run.smithi111.stdout:mgr.y smithi121 *:8443,8765 running (8m) 44s ago 8m 443M - 19.1.1-326-g8921c8ef f1455bd0219c cca4847b6242 2024-09-15T22:45:58.670 INFO:teuthology.orchestra.run.smithi111.stdout:mgr.z smithi192 *:8443,8765 running (8m) 44s ago 8m 443M - 19.1.1-326-g8921c8ef f1455bd0219c 983cb48dfec3 2024-09-15T22:45:58.670 INFO:teuthology.orchestra.run.smithi111.stdout:mon.a smithi111 running (10m) 25s ago 10m 47.7M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 200de39e07ab 2024-09-15T22:45:58.670 INFO:teuthology.orchestra.run.smithi111.stdout:mon.b smithi121 running (8m) 44s ago 8m 41.6M 2048M 19.1.1-326-g8921c8ef f1455bd0219c d9e180790a27 2024-09-15T22:45:58.670 INFO:teuthology.orchestra.run.smithi111.stdout:mon.c smithi192 running (8m) 44s ago 8m 43.6M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 2e1cba46ba4c 2024-09-15T22:45:58.670 INFO:teuthology.orchestra.run.smithi111.stdout:osd.0 smithi111 running (7m) 25s ago 7m 54.1M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 3834fe2a83b1 2024-09-15T22:45:58.670 INFO:teuthology.orchestra.run.smithi111.stdout:osd.1 smithi121 running (7m) 44s ago 7m 48.7M 4271M 19.1.1-326-g8921c8ef f1455bd0219c d6f2291033d8 2024-09-15T22:45:58.670 INFO:teuthology.orchestra.run.smithi111.stdout:osd.2 smithi192 running (6m) 44s ago 6m 59.5M 4272M 19.1.1-326-g8921c8ef f1455bd0219c da1156454396 2024-09-15T22:45:58.670 INFO:teuthology.orchestra.run.smithi111.stdout:osd.3 smithi111 running (6m) 25s ago 6m 45.1M 4272M 19.1.1-326-g8921c8ef f1455bd0219c c1ea6241f8c3 2024-09-15T22:45:58.670 INFO:teuthology.orchestra.run.smithi111.stdout:osd.4 smithi121 running (5m) 44s ago 5m 46.2M 4271M 19.1.1-326-g8921c8ef f1455bd0219c 77125fb77304 2024-09-15T22:45:58.670 INFO:teuthology.orchestra.run.smithi111.stdout:osd.5 smithi192 running (5m) 44s ago 5m 46.8M 4272M 19.1.1-326-g8921c8ef f1455bd0219c b2b27757ef15 2024-09-15T22:45:58.670 INFO:teuthology.orchestra.run.smithi111.stdout:osd.6 smithi111 running (4m) 25s ago 4m 51.6M 4272M 19.1.1-326-g8921c8ef f1455bd0219c c6b5a294c7cb 2024-09-15T22:45:58.670 INFO:teuthology.orchestra.run.smithi111.stdout:osd.7 smithi121 running (4m) 44s ago 4m 48.8M 4271M 19.1.1-326-g8921c8ef f1455bd0219c 2deb1842a0f8 2024-09-15T22:45:58.670 INFO:teuthology.orchestra.run.smithi111.stdout:osd.8 smithi192 running (4m) 44s ago 4m 52.2M 4272M 19.1.1-326-g8921c8ef f1455bd0219c f19a0a1213db 2024-09-15T22:45:58.671 INFO:teuthology.orchestra.run.smithi111.stdout:osd.9 smithi111 running (3m) 25s ago 3m 48.7M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 0d16a3d36636 2024-09-15T22:45:58.671 INFO:teuthology.orchestra.run.smithi111.stdout:osd.10 smithi121 running (3m) 44s ago 3m 57.1M 4271M 19.1.1-326-g8921c8ef f1455bd0219c ea22f0a023e9 2024-09-15T22:45:58.671 INFO:teuthology.orchestra.run.smithi111.stdout:osd.11 smithi192 running (2m) 44s ago 2m 46.5M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 2804b28dbf60 2024-09-15T22:45:58.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.669+0000 7f7c12200640 1 -- 172.21.15.111:0/1872189099 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7bf00a3270 msgr2=0x7f7bf00a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:58.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.669+0000 7f7c12200640 1 --2- 172.21.15.111:0/1872189099 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7bf00a3270 0x7f7bf00a5730 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c19b630 tx=0x7f7bf800b390 comp rx=0 tx=0).stop 2024-09-15T22:45:58.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.669+0000 7f7c12200640 1 -- 172.21.15.111:0/1872189099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c0c102680 msgr2=0x7f7c0c19a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:45:58.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.669+0000 7f7c12200640 1 --2- 172.21.15.111:0/1872189099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c0c102680 0x7f7c0c19a110 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc031440 tx=0x7f7bfc032ce0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.669+0000 7f7c12200640 1 -- 172.21.15.111:0/1872189099 shutdown_connections 2024-09-15T22:45:58.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.669+0000 7f7c12200640 1 --2- 172.21.15.111:0/1872189099 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7bf00a3270 0x7f7bf00a5730 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.669+0000 7f7c12200640 1 --2- 172.21.15.111:0/1872189099 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c0c1078e0 0x7f7c0c1a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.669+0000 7f7c12200640 1 --2- 172.21.15.111:0/1872189099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7c0c104fb0 0x7f7c0c19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.669+0000 7f7c12200640 1 --2- 172.21.15.111:0/1872189099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c0c102680 0x7f7c0c19a110 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:45:58.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.669+0000 7f7c12200640 1 -- 172.21.15.111:0/1872189099 >> 172.21.15.111:0/1872189099 conn(0x7f7c0c0fc280 msgr2=0x7f7c0c101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:45:58.672 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.669+0000 7f7c12200640 1 -- 172.21.15.111:0/1872189099 shutdown_connections 2024-09-15T22:45:58.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:45:58.669+0000 7f7c12200640 1 -- 172.21.15.111:0/1872189099 wait complete. 2024-09-15T22:45:59.612 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-15T22:45:59.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:59 smithi111 ceph-mon[30629]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-15T22:45:59.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:45:59 smithi111 ceph-mon[30629]: from='client.24940 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:45:59.978 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:46:00.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:59 smithi192 ceph-mon[35427]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-15T22:46:00.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:45:59 smithi192 ceph-mon[35427]: from='client.24940 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:46:00.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:59 smithi121 ceph-mon[34815]: pgmap v352: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-15T22:46:00.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:45:59 smithi121 ceph-mon[34815]: from='client.24940 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:46:00.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.543+0000 7f51fb200640 1 -- 172.21.15.111:0/2428322179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51fc104e50 msgr2=0x7f51fc1052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:00.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.543+0000 7f51fb200640 1 --2- 172.21.15.111:0/2428322179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51fc104e50 0x7f51fc1052d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f51e8009860 tx=0x7f51e802a340 comp rx=0 tx=0).stop 2024-09-15T22:46:00.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.543+0000 7f51fb200640 1 -- 172.21.15.111:0/2428322179 shutdown_connections 2024-09-15T22:46:00.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.543+0000 7f51fb200640 1 --2- 172.21.15.111:0/2428322179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51fc105810 0x7f51fc10c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:00.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.543+0000 7f51fb200640 1 --2- 172.21.15.111:0/2428322179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51fc104e50 0x7f51fc1052d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:00.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.543+0000 7f51fb200640 1 --2- 172.21.15.111:0/2428322179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51fc103c50 0x7f51fc104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:00.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.543+0000 7f51fb200640 1 -- 172.21.15.111:0/2428322179 >> 172.21.15.111:0/2428322179 conn(0x7f51fc0ff3e0 msgr2=0x7f51fc101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:00.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.543+0000 7f51fb200640 1 -- 172.21.15.111:0/2428322179 shutdown_connections 2024-09-15T22:46:00.545 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.543+0000 7f51fb200640 1 -- 172.21.15.111:0/2428322179 wait complete. 2024-09-15T22:46:00.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.544+0000 7f51fb200640 1 Processor -- start 2024-09-15T22:46:00.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.544+0000 7f51fb200640 1 -- start start 2024-09-15T22:46:00.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.544+0000 7f51fb200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51fc103c50 0x7f51fc199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:00.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.544+0000 7f51fb200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51fc104e50 0x7f51fc19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:00.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.544+0000 7f51fb200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51fc105810 0x7f51fc1a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:00.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.544+0000 7f51fb200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc10f8b0 con 0x7f51fc104e50 2024-09-15T22:46:00.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.544+0000 7f51fb200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc10f730 con 0x7f51fc105810 2024-09-15T22:46:00.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.544+0000 7f51fb200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f51fc10fa30 con 0x7f51fc103c50 2024-09-15T22:46:00.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.545+0000 7f51f9400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51fc104e50 0x7f51fc19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:00.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.545+0000 7f51f9e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51fc103c50 0x7f51fc199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:00.547 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.545+0000 7f51f9400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51fc104e50 0x7f51fc19a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:35198/0 (socket says 172.21.15.111:35198) 2024-09-15T22:46:00.548 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.545+0000 7f51f9400640 1 -- 172.21.15.111:0/1885846358 learned_addr learned my addr 172.21.15.111:0/1885846358 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:00.548 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.545+0000 7f51f9e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51fc103c50 0x7f51fc199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:48692/0 (socket says 172.21.15.111:48692) 2024-09-15T22:46:00.548 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.545+0000 7f51fa800640 1 --2- 172.21.15.111:0/1885846358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51fc105810 0x7f51fc1a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:00.548 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.545+0000 7f51f9400640 1 -- 172.21.15.111:0/1885846358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51fc103c50 msgr2=0x7f51fc199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:00.548 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.545+0000 7f51f9400640 1 --2- 172.21.15.111:0/1885846358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51fc103c50 0x7f51fc199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:00.548 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.545+0000 7f51f9400640 1 -- 172.21.15.111:0/1885846358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51fc105810 msgr2=0x7f51fc1a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:00.548 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.545+0000 7f51f9400640 1 --2- 172.21.15.111:0/1885846358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51fc105810 0x7f51fc1a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:00.548 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.545+0000 7f51f9400640 1 -- 172.21.15.111:0/1885846358 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51fc1a1b30 con 0x7f51fc104e50 2024-09-15T22:46:00.548 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.545+0000 7f51fa800640 1 --2- 172.21.15.111:0/1885846358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51fc105810 0x7f51fc1a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:46:00.548 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.545+0000 7f51f9e00640 1 --2- 172.21.15.111:0/1885846358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51fc103c50 0x7f51fc199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:46:00.548 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.545+0000 7f51f9400640 1 --2- 172.21.15.111:0/1885846358 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51fc104e50 0x7f51fc19a440 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f51e8005940 tx=0x7f51e8033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:00.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.546+0000 7f51f2a00640 1 -- 172.21.15.111:0/1885846358 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e803f020 con 0x7f51fc104e50 2024-09-15T22:46:00.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.546+0000 7f51f2a00640 1 -- 172.21.15.111:0/1885846358 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51e8030850 con 0x7f51fc104e50 2024-09-15T22:46:00.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.546+0000 7f51f2a00640 1 -- 172.21.15.111:0/1885846358 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e8037040 con 0x7f51fc104e50 2024-09-15T22:46:00.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.546+0000 7f51fb200640 1 -- 172.21.15.111:0/1885846358 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51fc1a1dc0 con 0x7f51fc104e50 2024-09-15T22:46:00.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.546+0000 7f51fb200640 1 -- 172.21.15.111:0/1885846358 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51fc1a21f0 con 0x7f51fc104e50 2024-09-15T22:46:00.549 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.547+0000 7f51fb200640 1 -- 172.21.15.111:0/1885846358 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51fc10b040 con 0x7f51fc104e50 2024-09-15T22:46:00.550 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.549+0000 7f51f2a00640 1 -- 172.21.15.111:0/1885846358 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f51e8042020 con 0x7f51fc104e50 2024-09-15T22:46:00.551 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.550+0000 7f51f2a00640 1 --2- 172.21.15.111:0/1885846358 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f51e00a32f0 0x7f51e00a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:00.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.550+0000 7f51f2a00640 1 -- 172.21.15.111:0/1885846358 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f51e80efc10 con 0x7f51fc104e50 2024-09-15T22:46:00.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.550+0000 7f51f9e00640 1 --2- 172.21.15.111:0/1885846358 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f51e00a32f0 0x7f51e00a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:00.552 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.551+0000 7f51f9e00640 1 --2- 172.21.15.111:0/1885846358 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f51e00a32f0 0x7f51e00a57b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0096c0 tx=0x7f51ec009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:00.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.555+0000 7f51f2a00640 1 -- 172.21.15.111:0/1885846358 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f51e8049020 con 0x7f51fc104e50 2024-09-15T22:46:00.739 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.737+0000 7f51fb200640 1 -- 172.21.15.111:0/1885846358 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f51fc062630 con 0x7f51e00a32f0 2024-09-15T22:46:00.745 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.743+0000 7f51f2a00640 1 -- 172.21.15.111:0/1885846358 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f51fc062630 con 0x7f51e00a32f0 2024-09-15T22:46:00.745 INFO:teuthology.orchestra.run.smithi111.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-15T22:46:00.745 INFO:teuthology.orchestra.run.smithi111.stdout:mds.all 12/12 46s ago 2m smithi111=a;smithi111=d;smithi111=g;smithi111=j;smithi121=b;smithi121=e;smithi121=h;smithi121=k;smithi192=c;smithi192=f;smithi192=i;smithi192=l;count:12 2024-09-15T22:46:00.745 INFO:teuthology.orchestra.run.smithi111.stdout:mgr 3/3 46s ago 8m smithi111=x;smithi121=y;smithi192=z;count:3 2024-09-15T22:46:00.745 INFO:teuthology.orchestra.run.smithi111.stdout:mon 3/3 46s ago 8m smithi111:172.21.15.111=a;smithi121:172.21.15.121=b;smithi192:172.21.15.192=c;count:3 2024-09-15T22:46:00.745 INFO:teuthology.orchestra.run.smithi111.stdout:osd 12 46s ago - 2024-09-15T22:46:00.747 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.745+0000 7f51fb200640 1 -- 172.21.15.111:0/1885846358 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f51e00a32f0 msgr2=0x7f51e00a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:00.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.745+0000 7f51fb200640 1 --2- 172.21.15.111:0/1885846358 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f51e00a32f0 0x7f51e00a57b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0096c0 tx=0x7f51ec009260 comp rx=0 tx=0).stop 2024-09-15T22:46:00.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.745+0000 7f51fb200640 1 -- 172.21.15.111:0/1885846358 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51fc104e50 msgr2=0x7f51fc19a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:00.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.745+0000 7f51fb200640 1 --2- 172.21.15.111:0/1885846358 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51fc104e50 0x7f51fc19a440 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f51e8005940 tx=0x7f51e8033d90 comp rx=0 tx=0).stop 2024-09-15T22:46:00.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.745+0000 7f51fb200640 1 -- 172.21.15.111:0/1885846358 shutdown_connections 2024-09-15T22:46:00.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.745+0000 7f51fb200640 1 --2- 172.21.15.111:0/1885846358 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f51e00a32f0 0x7f51e00a57b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:00.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.745+0000 7f51fb200640 1 --2- 172.21.15.111:0/1885846358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51fc105810 0x7f51fc1a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:00.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.745+0000 7f51fb200640 1 --2- 172.21.15.111:0/1885846358 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51fc104e50 0x7f51fc19a440 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:00.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.745+0000 7f51fb200640 1 --2- 172.21.15.111:0/1885846358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51fc103c50 0x7f51fc199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:00.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.745+0000 7f51fb200640 1 -- 172.21.15.111:0/1885846358 >> 172.21.15.111:0/1885846358 conn(0x7f51fc0ff3e0 msgr2=0x7f51fc108210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:00.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.746+0000 7f51fb200640 1 -- 172.21.15.111:0/1885846358 shutdown_connections 2024-09-15T22:46:00.748 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:00.746+0000 7f51fb200640 1 -- 172.21.15.111:0/1885846358 wait complete. 2024-09-15T22:46:01.616 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-15T22:46:01.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:01 smithi111 ceph-mon[30629]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-15T22:46:01.985 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:46:02.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:01 smithi192 ceph-mon[35427]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-15T22:46:02.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:01 smithi121 ceph-mon[34815]: pgmap v353: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-15T22:46:02.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.588+0000 7f82cd000640 1 -- 172.21.15.111:0/1329681968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82c8104f60 msgr2=0x7f82c81053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:02.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.588+0000 7f82cd000640 1 --2- 172.21.15.111:0/1329681968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82c8104f60 0x7f82c81053e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009900 tx=0x7f82b002a410 comp rx=0 tx=0).stop 2024-09-15T22:46:02.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.588+0000 7f82cd000640 1 -- 172.21.15.111:0/1329681968 shutdown_connections 2024-09-15T22:46:02.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.588+0000 7f82cd000640 1 --2- 172.21.15.111:0/1329681968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82c8105920 0x7f82c810c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:02.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.588+0000 7f82cd000640 1 --2- 172.21.15.111:0/1329681968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82c8104f60 0x7f82c81053e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:02.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.588+0000 7f82cd000640 1 --2- 172.21.15.111:0/1329681968 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82c8103e00 0x7f82c8104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:02.590 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.588+0000 7f82cd000640 1 -- 172.21.15.111:0/1329681968 >> 172.21.15.111:0/1329681968 conn(0x7f82c80ff5b0 msgr2=0x7f82c81019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:02.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.588+0000 7f82cd000640 1 -- 172.21.15.111:0/1329681968 shutdown_connections 2024-09-15T22:46:02.591 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.588+0000 7f82cd000640 1 -- 172.21.15.111:0/1329681968 wait complete. 2024-09-15T22:46:02.592 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.589+0000 7f82cd000640 1 Processor -- start 2024-09-15T22:46:02.592 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.589+0000 7f82cd000640 1 -- start start 2024-09-15T22:46:02.592 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.589+0000 7f82cd000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82c8103e00 0x7f82c819a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:02.592 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.589+0000 7f82cd000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82c8104f60 0x7f82c819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:02.592 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.589+0000 7f82cd000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82c8105920 0x7f82c81a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:02.592 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.589+0000 7f82cd000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f82c810fb00 con 0x7f82c8104f60 2024-09-15T22:46:02.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.589+0000 7f82cd000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f82c810f980 con 0x7f82c8103e00 2024-09-15T22:46:02.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.589+0000 7f82cd000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f82c810fc80 con 0x7f82c8105920 2024-09-15T22:46:02.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.590+0000 7f82c5e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82c8103e00 0x7f82c819a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:02.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.590+0000 7f82c5e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82c8103e00 0x7f82c819a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:41408/0 (socket says 172.21.15.111:41408) 2024-09-15T22:46:02.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.590+0000 7f82c5e00640 1 -- 172.21.15.111:0/3298335861 learned_addr learned my addr 172.21.15.111:0/3298335861 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:02.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.590+0000 7f82bd400640 1 --2- 172.21.15.111:0/3298335861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82c8104f60 0x7f82c819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:02.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.590+0000 7f82c6800640 1 --2- 172.21.15.111:0/3298335861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82c8105920 0x7f82c81a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:02.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.590+0000 7f82c5e00640 1 -- 172.21.15.111:0/3298335861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82c8105920 msgr2=0x7f82c81a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:02.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.590+0000 7f82c5e00640 1 --2- 172.21.15.111:0/3298335861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82c8105920 0x7f82c81a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:02.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.590+0000 7f82c5e00640 1 -- 172.21.15.111:0/3298335861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82c8104f60 msgr2=0x7f82c819a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:02.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.590+0000 7f82c5e00640 1 --2- 172.21.15.111:0/3298335861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82c8104f60 0x7f82c819a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:02.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.590+0000 7f82c5e00640 1 -- 172.21.15.111:0/3298335861 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c81a1d00 con 0x7f82c8103e00 2024-09-15T22:46:02.593 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.590+0000 7f82c5e00640 1 --2- 172.21.15.111:0/3298335861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82c8103e00 0x7f82c819a0d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f82b800db80 tx=0x7f82b800b630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:02.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.591+0000 7f82bf400640 1 -- 172.21.15.111:0/3298335861 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b8017070 con 0x7f82c8103e00 2024-09-15T22:46:02.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.591+0000 7f82bf400640 1 -- 172.21.15.111:0/3298335861 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82b8003b70 con 0x7f82c8103e00 2024-09-15T22:46:02.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.591+0000 7f82cd000640 1 -- 172.21.15.111:0/3298335861 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c81a1ff0 con 0x7f82c8103e00 2024-09-15T22:46:02.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.591+0000 7f82bf400640 1 -- 172.21.15.111:0/3298335861 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82b80106c0 con 0x7f82c8103e00 2024-09-15T22:46:02.594 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.591+0000 7f82cd000640 1 -- 172.21.15.111:0/3298335861 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c81a2440 con 0x7f82c8103e00 2024-09-15T22:46:02.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.592+0000 7f82cd000640 1 -- 172.21.15.111:0/3298335861 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82c810b1a0 con 0x7f82c8103e00 2024-09-15T22:46:02.595 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.594+0000 7f82bf400640 1 -- 172.21.15.111:0/3298335861 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f82b8004020 con 0x7f82c8103e00 2024-09-15T22:46:02.596 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.595+0000 7f82bf400640 1 --2- 172.21.15.111:0/3298335861 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f82ac0a3270 0x7f82ac0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:02.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.595+0000 7f82bf400640 1 -- 172.21.15.111:0/3298335861 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f82b80cf9d0 con 0x7f82c8103e00 2024-09-15T22:46:02.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.595+0000 7f82bd400640 1 --2- 172.21.15.111:0/3298335861 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f82ac0a3270 0x7f82ac0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:02.597 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.596+0000 7f82bd400640 1 --2- 172.21.15.111:0/3298335861 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f82ac0a3270 0x7f82ac0a5730 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f82b002aa00 tx=0x7f82b00315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:02.599 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.597+0000 7f82bf400640 1 -- 172.21.15.111:0/3298335861 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f82b8020070 con 0x7f82c8103e00 2024-09-15T22:46:02.780 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.778+0000 7f82cd000640 1 -- 172.21.15.111:0/3298335861 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f82c8062630 con 0x7f82ac0a3270 2024-09-15T22:46:02.780 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:02 smithi111 ceph-mon[30629]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:46:02.781 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.780+0000 7f82bf400640 1 -- 172.21.15.111:0/3298335861 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f82c8062630 con 0x7f82ac0a3270 2024-09-15T22:46:02.781 INFO:teuthology.orchestra.run.smithi111.stdout:HOST ADDR LABELS STATUS 2024-09-15T22:46:02.781 INFO:teuthology.orchestra.run.smithi111.stdout:smithi111 172.21.15.111 2024-09-15T22:46:02.782 INFO:teuthology.orchestra.run.smithi111.stdout:smithi121 172.21.15.121 2024-09-15T22:46:02.782 INFO:teuthology.orchestra.run.smithi111.stdout:smithi192 172.21.15.192 2024-09-15T22:46:02.782 INFO:teuthology.orchestra.run.smithi111.stdout:3 hosts in cluster 2024-09-15T22:46:02.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.782+0000 7f82cd000640 1 -- 172.21.15.111:0/3298335861 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f82ac0a3270 msgr2=0x7f82ac0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:02.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.782+0000 7f82cd000640 1 --2- 172.21.15.111:0/3298335861 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f82ac0a3270 0x7f82ac0a5730 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f82b002aa00 tx=0x7f82b00315b0 comp rx=0 tx=0).stop 2024-09-15T22:46:02.784 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.782+0000 7f82cd000640 1 -- 172.21.15.111:0/3298335861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82c8103e00 msgr2=0x7f82c819a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:02.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.782+0000 7f82cd000640 1 --2- 172.21.15.111:0/3298335861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82c8103e00 0x7f82c819a0d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f82b800db80 tx=0x7f82b800b630 comp rx=0 tx=0).stop 2024-09-15T22:46:02.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.783+0000 7f82cd000640 1 -- 172.21.15.111:0/3298335861 shutdown_connections 2024-09-15T22:46:02.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.783+0000 7f82cd000640 1 --2- 172.21.15.111:0/3298335861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82c8105920 0x7f82c81a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:02.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.783+0000 7f82cd000640 1 --2- 172.21.15.111:0/3298335861 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82c8104f60 0x7f82c819a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:02.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.783+0000 7f82cd000640 1 --2- 172.21.15.111:0/3298335861 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f82ac0a3270 0x7f82ac0a5730 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:02.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.783+0000 7f82cd000640 1 --2- 172.21.15.111:0/3298335861 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82c8103e00 0x7f82c819a0d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:02.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.783+0000 7f82cd000640 1 -- 172.21.15.111:0/3298335861 >> 172.21.15.111:0/3298335861 conn(0x7f82c80ff5b0 msgr2=0x7f82c8108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:02.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.783+0000 7f82cd000640 1 -- 172.21.15.111:0/3298335861 shutdown_connections 2024-09-15T22:46:02.785 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:02.783+0000 7f82cd000640 1 -- 172.21.15.111:0/3298335861 wait complete. 2024-09-15T22:46:03.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:02 smithi192 ceph-mon[35427]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:46:03.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:02 smithi121 ceph-mon[34815]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:46:03.641 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-15T22:46:03.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:03 smithi111 ceph-mon[30629]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-15T22:46:04.008 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:46:04.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:03 smithi192 ceph-mon[35427]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-15T22:46:04.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:03 smithi121 ceph-mon[34815]: pgmap v354: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-15T22:46:04.610 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.608+0000 7fde64c00640 1 -- 172.21.15.111:0/3927514091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde60102760 msgr2=0x7fde60104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:04.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.608+0000 7fde64c00640 1 --2- 172.21.15.111:0/3927514091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde60102760 0x7fde60104b50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fde50009900 tx=0x7fde5002a410 comp rx=0 tx=0).stop 2024-09-15T22:46:04.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.608+0000 7fde64c00640 1 -- 172.21.15.111:0/3927514091 shutdown_connections 2024-09-15T22:46:04.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.608+0000 7fde64c00640 1 --2- 172.21.15.111:0/3927514091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde601079c0 0x7fde60109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:04.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.608+0000 7fde64c00640 1 --2- 172.21.15.111:0/3927514091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde60105090 0x7fde60107480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:04.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.608+0000 7fde64c00640 1 --2- 172.21.15.111:0/3927514091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde60102760 0x7fde60104b50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:04.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.608+0000 7fde64c00640 1 -- 172.21.15.111:0/3927514091 >> 172.21.15.111:0/3927514091 conn(0x7fde600fc2a0 msgr2=0x7fde600fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:04.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.609+0000 7fde64c00640 1 -- 172.21.15.111:0/3927514091 shutdown_connections 2024-09-15T22:46:04.611 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.609+0000 7fde64c00640 1 -- 172.21.15.111:0/3927514091 wait complete. 2024-09-15T22:46:04.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.610+0000 7fde64c00640 1 Processor -- start 2024-09-15T22:46:04.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.610+0000 7fde64c00640 1 -- start start 2024-09-15T22:46:04.612 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.610+0000 7fde64c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde60102760 0x7fde6019a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:04.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.610+0000 7fde64c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde60105090 0x7fde6019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:04.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.610+0000 7fde64c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde601079c0 0x7fde601a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:04.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.610+0000 7fde64c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fde6010d900 con 0x7fde60105090 2024-09-15T22:46:04.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.610+0000 7fde64c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fde6010d780 con 0x7fde601079c0 2024-09-15T22:46:04.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.610+0000 7fde64c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fde6010da80 con 0x7fde60102760 2024-09-15T22:46:04.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.610+0000 7fde5d400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde60105090 0x7fde6019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:04.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.610+0000 7fde5e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde601079c0 0x7fde601a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:04.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.610+0000 7fde5d400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde60105090 0x7fde6019a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36544/0 (socket says 172.21.15.111:36544) 2024-09-15T22:46:04.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.610+0000 7fde5d400640 1 -- 172.21.15.111:0/2289009046 learned_addr learned my addr 172.21.15.111:0/2289009046 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:04.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.610+0000 7fde5de00640 1 --2- 172.21.15.111:0/2289009046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde60102760 0x7fde6019a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:04.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.611+0000 7fde5e800640 1 -- 172.21.15.111:0/2289009046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde60102760 msgr2=0x7fde6019a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:04.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.611+0000 7fde5e800640 1 --2- 172.21.15.111:0/2289009046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde60102760 0x7fde6019a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:04.613 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.611+0000 7fde5e800640 1 -- 172.21.15.111:0/2289009046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde60105090 msgr2=0x7fde6019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:04.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.611+0000 7fde5e800640 1 --2- 172.21.15.111:0/2289009046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde60105090 0x7fde6019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:04.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.611+0000 7fde5e800640 1 -- 172.21.15.111:0/2289009046 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde601a1d80 con 0x7fde601079c0 2024-09-15T22:46:04.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.611+0000 7fde5e800640 1 --2- 172.21.15.111:0/2289009046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde601079c0 0x7fde601a1710 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fde5800e940 tx=0x7fde5800ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:04.614 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.611+0000 7fde56a00640 1 -- 172.21.15.111:0/2289009046 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde58004d20 con 0x7fde601079c0 2024-09-15T22:46:04.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.611+0000 7fde56a00640 1 -- 172.21.15.111:0/2289009046 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde58005020 con 0x7fde601079c0 2024-09-15T22:46:04.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.611+0000 7fde64c00640 1 -- 172.21.15.111:0/2289009046 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde601a2070 con 0x7fde601079c0 2024-09-15T22:46:04.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.611+0000 7fde56a00640 1 -- 172.21.15.111:0/2289009046 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde580116b0 con 0x7fde601079c0 2024-09-15T22:46:04.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.612+0000 7fde64c00640 1 -- 172.21.15.111:0/2289009046 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde601a2560 con 0x7fde601079c0 2024-09-15T22:46:04.615 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.612+0000 7fde64c00640 1 -- 172.21.15.111:0/2289009046 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde601094b0 con 0x7fde601079c0 2024-09-15T22:46:04.616 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.614+0000 7fde56a00640 1 -- 172.21.15.111:0/2289009046 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fde580054d0 con 0x7fde601079c0 2024-09-15T22:46:04.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.616+0000 7fde56a00640 1 --2- 172.21.15.111:0/2289009046 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fde440a3270 0x7fde440a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:04.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.616+0000 7fde56a00640 1 -- 172.21.15.111:0/2289009046 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fde580cfa30 con 0x7fde601079c0 2024-09-15T22:46:04.617 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.616+0000 7fde5de00640 1 --2- 172.21.15.111:0/2289009046 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fde440a3270 0x7fde440a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:04.618 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.616+0000 7fde5de00640 1 --2- 172.21.15.111:0/2289009046 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fde440a3270 0x7fde440a5730 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fde50031440 tx=0x7fde500315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:04.619 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.618+0000 7fde56a00640 1 -- 172.21.15.111:0/2289009046 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fde58016030 con 0x7fde601079c0 2024-09-15T22:46:04.803 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.801+0000 7fde64c00640 1 -- 172.21.15.111:0/2289009046 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fde60062630 con 0x7fde440a3270 2024-09-15T22:46:04.803 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:04 smithi111 ceph-mon[30629]: from='client.24956 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:46:04.804 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.803+0000 7fde56a00640 1 -- 172.21.15.111:0/2289009046 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fde60062630 con 0x7fde440a3270 2024-09-15T22:46:04.804 INFO:teuthology.orchestra.run.smithi111.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-15T22:46:04.805 INFO:teuthology.orchestra.run.smithi111.stdout:smithi111 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400VZ400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-15T22:46:04.805 INFO:teuthology.orchestra.run.smithi111.stdout:smithi121 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400ZF400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-15T22:46:04.805 INFO:teuthology.orchestra.run.smithi111.stdout:smithi192 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500NN400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-15T22:46:04.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.805+0000 7fde64c00640 1 -- 172.21.15.111:0/2289009046 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fde440a3270 msgr2=0x7fde440a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:04.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.805+0000 7fde64c00640 1 --2- 172.21.15.111:0/2289009046 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fde440a3270 0x7fde440a5730 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fde50031440 tx=0x7fde500315b0 comp rx=0 tx=0).stop 2024-09-15T22:46:04.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.805+0000 7fde64c00640 1 -- 172.21.15.111:0/2289009046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde601079c0 msgr2=0x7fde601a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:04.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.805+0000 7fde64c00640 1 --2- 172.21.15.111:0/2289009046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde601079c0 0x7fde601a1710 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fde5800e940 tx=0x7fde5800ee00 comp rx=0 tx=0).stop 2024-09-15T22:46:04.807 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.805+0000 7fde64c00640 1 -- 172.21.15.111:0/2289009046 shutdown_connections 2024-09-15T22:46:04.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.805+0000 7fde64c00640 1 --2- 172.21.15.111:0/2289009046 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fde440a3270 0x7fde440a5730 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:04.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.805+0000 7fde64c00640 1 --2- 172.21.15.111:0/2289009046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fde601079c0 0x7fde601a1710 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:04.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.805+0000 7fde64c00640 1 --2- 172.21.15.111:0/2289009046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde60105090 0x7fde6019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:04.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.805+0000 7fde64c00640 1 --2- 172.21.15.111:0/2289009046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde60102760 0x7fde6019a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:04.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.805+0000 7fde64c00640 1 -- 172.21.15.111:0/2289009046 >> 172.21.15.111:0/2289009046 conn(0x7fde600fc2a0 msgr2=0x7fde60101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:04.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.805+0000 7fde64c00640 1 -- 172.21.15.111:0/2289009046 shutdown_connections 2024-09-15T22:46:04.808 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:04.806+0000 7fde64c00640 1 -- 172.21.15.111:0/2289009046 wait complete. 2024-09-15T22:46:05.041 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:04 smithi121 ceph-mon[34815]: from='client.24956 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:46:05.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:04 smithi192 ceph-mon[35427]: from='client.24956 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:46:05.700 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-15T22:46:05.712 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi111.front.sepia.ceph.com 2024-09-15T22:46:05.712 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-15T22:46:05.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:05 smithi111 ceph-mon[30629]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-15T22:46:06.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:05 smithi192 ceph-mon[35427]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-15T22:46:06.079 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:46:06.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:05 smithi121 ceph-mon[34815]: pgmap v355: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-15T22:46:06.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.648+0000 7fafc1e00640 1 -- 172.21.15.111:0/657604355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafbc104f60 msgr2=0x7fafbc1053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:06.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.648+0000 7fafc1e00640 1 --2- 172.21.15.111:0/657604355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafbc104f60 0x7fafbc1053e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009900 tx=0x7fafa402a410 comp rx=0 tx=0).stop 2024-09-15T22:46:06.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.648+0000 7fafc1e00640 1 -- 172.21.15.111:0/657604355 shutdown_connections 2024-09-15T22:46:06.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.648+0000 7fafc1e00640 1 --2- 172.21.15.111:0/657604355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafbc105920 0x7fafbc10c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:06.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.648+0000 7fafc1e00640 1 --2- 172.21.15.111:0/657604355 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafbc104f60 0x7fafbc1053e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:06.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.648+0000 7fafc1e00640 1 --2- 172.21.15.111:0/657604355 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafbc103e00 0x7fafbc104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:06.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.648+0000 7fafc1e00640 1 -- 172.21.15.111:0/657604355 >> 172.21.15.111:0/657604355 conn(0x7fafbc0ff5b0 msgr2=0x7fafbc1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:06.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.648+0000 7fafc1e00640 1 -- 172.21.15.111:0/657604355 shutdown_connections 2024-09-15T22:46:06.650 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.648+0000 7fafc1e00640 1 -- 172.21.15.111:0/657604355 wait complete. 2024-09-15T22:46:06.651 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.649+0000 7fafc1e00640 1 Processor -- start 2024-09-15T22:46:06.651 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.649+0000 7fafc1e00640 1 -- start start 2024-09-15T22:46:06.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.650+0000 7fafc1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafbc103e00 0x7fafbc19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:06.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.650+0000 7fafc1e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafbc104f60 0x7fafbc19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:06.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.650+0000 7fafc1e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafbc105920 0x7fafbc1a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:06.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.650+0000 7fafc1e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc10fb00 con 0x7fafbc103e00 2024-09-15T22:46:06.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.650+0000 7fafc1e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc10f980 con 0x7fafbc105920 2024-09-15T22:46:06.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.650+0000 7fafc1e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc10fc80 con 0x7fafbc104f60 2024-09-15T22:46:06.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.650+0000 7fafbaa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafbc103e00 0x7fafbc19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:06.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.650+0000 7fafbaa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafbc103e00 0x7fafbc19a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36556/0 (socket says 172.21.15.111:36556) 2024-09-15T22:46:06.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.650+0000 7fafbaa00640 1 -- 172.21.15.111:0/2057627566 learned_addr learned my addr 172.21.15.111:0/2057627566 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:06.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.650+0000 7fafbb400640 1 --2- 172.21.15.111:0/2057627566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafbc105920 0x7fafbc1a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:06.652 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.650+0000 7fafb2000640 1 --2- 172.21.15.111:0/2057627566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafbc104f60 0x7fafbc19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:06.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.650+0000 7fafbaa00640 1 -- 172.21.15.111:0/2057627566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafbc104f60 msgr2=0x7fafbc19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:06.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.650+0000 7fafbaa00640 1 --2- 172.21.15.111:0/2057627566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafbc104f60 0x7fafbc19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:06.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.650+0000 7fafbaa00640 1 -- 172.21.15.111:0/2057627566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafbc105920 msgr2=0x7fafbc1a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:06.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.650+0000 7fafbaa00640 1 --2- 172.21.15.111:0/2057627566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafbc105920 0x7fafbc1a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:06.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.650+0000 7fafbaa00640 1 -- 172.21.15.111:0/2057627566 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafbc1a1d30 con 0x7fafbc103e00 2024-09-15T22:46:06.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.651+0000 7fafbaa00640 1 --2- 172.21.15.111:0/2057627566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafbc103e00 0x7fafbc19a100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fafac00e9f0 tx=0x7fafac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:06.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.651+0000 7fafc0a00640 1 -- 172.21.15.111:0/2057627566 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac00cd80 con 0x7fafbc103e00 2024-09-15T22:46:06.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.651+0000 7fafc0a00640 1 -- 172.21.15.111:0/2057627566 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafac010910 con 0x7fafbc103e00 2024-09-15T22:46:06.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.651+0000 7fafc1e00640 1 -- 172.21.15.111:0/2057627566 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafbc1a1fc0 con 0x7fafbc103e00 2024-09-15T22:46:06.654 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.651+0000 7fafc0a00640 1 -- 172.21.15.111:0/2057627566 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac018990 con 0x7fafbc103e00 2024-09-15T22:46:06.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.651+0000 7fafc1e00640 1 -- 172.21.15.111:0/2057627566 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafbc1a24c0 con 0x7fafbc103e00 2024-09-15T22:46:06.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.652+0000 7fafc1e00640 1 -- 172.21.15.111:0/2057627566 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafbc10b120 con 0x7fafbc103e00 2024-09-15T22:46:06.655 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.654+0000 7fafc0a00640 1 -- 172.21.15.111:0/2057627566 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fafac010ac0 con 0x7fafbc103e00 2024-09-15T22:46:06.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.656+0000 7fafc0a00640 1 --2- 172.21.15.111:0/2057627566 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fafa00a32f0 0x7fafa00a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:06.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.657+0000 7fafc0a00640 1 -- 172.21.15.111:0/2057627566 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fafac0cff20 con 0x7fafbc103e00 2024-09-15T22:46:06.658 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.657+0000 7fafb2000640 1 --2- 172.21.15.111:0/2057627566 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fafa00a32f0 0x7fafa00a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:06.659 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.657+0000 7fafb2000640 1 --2- 172.21.15.111:0/2057627566 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fafa00a32f0 0x7fafa00a57b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fafa402aa00 tx=0x7fafa40315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:06.660 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.659+0000 7fafc0a00640 1 -- 172.21.15.111:0/2057627566 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fafac09c5f0 con 0x7fafbc103e00 2024-09-15T22:46:06.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.870+0000 7fafc1e00640 1 -- 172.21.15.111:0/2057627566 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fafbc062e20 con 0x7fafbc103e00 2024-09-15T22:46:06.872 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:06 smithi111 ceph-mon[30629]: from='client.24962 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:46:06.872 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.871+0000 7fafc0a00640 1 -- 172.21.15.111:0/2057627566 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3100 (secure 0 0 0) 0x7fafac0a08f0 con 0x7fafbc103e00 2024-09-15T22:46:06.873 INFO:teuthology.orchestra.run.smithi111.stdout:e25 2024-09-15T22:46:06.873 INFO:teuthology.orchestra.run.smithi111.stdout:btime 2024-09-15T22:44:34:567598+0000 2024-09-15T22:46:06.873 INFO:teuthology.orchestra.run.smithi111.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-15T22:46:06.873 INFO:teuthology.orchestra.run.smithi111.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-15T22:46:06.873 INFO:teuthology.orchestra.run.smithi111.stdout:legacy client fscid: 1 2024-09-15T22:46:06.873 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:46:06.873 INFO:teuthology.orchestra.run.smithi111.stdout:Filesystem 'cephfs' (1) 2024-09-15T22:46:06.873 INFO:teuthology.orchestra.run.smithi111.stdout:fs_name cephfs 2024-09-15T22:46:06.873 INFO:teuthology.orchestra.run.smithi111.stdout:epoch 25 2024-09-15T22:46:06.874 INFO:teuthology.orchestra.run.smithi111.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-15T22:46:06.874 INFO:teuthology.orchestra.run.smithi111.stdout:created 2024-09-15T22:44:18.436394+0000 2024-09-15T22:46:06.874 INFO:teuthology.orchestra.run.smithi111.stdout:modified 2024-09-15T22:44:33.655164+0000 2024-09-15T22:46:06.874 INFO:teuthology.orchestra.run.smithi111.stdout:tableserver 0 2024-09-15T22:46:06.874 INFO:teuthology.orchestra.run.smithi111.stdout:root 0 2024-09-15T22:46:06.874 INFO:teuthology.orchestra.run.smithi111.stdout:session_timeout 300 2024-09-15T22:46:06.874 INFO:teuthology.orchestra.run.smithi111.stdout:session_autoclose 300 2024-09-15T22:46:06.874 INFO:teuthology.orchestra.run.smithi111.stdout:max_file_size 1099511627776 2024-09-15T22:46:06.874 INFO:teuthology.orchestra.run.smithi111.stdout:max_xattr_size 65536 2024-09-15T22:46:06.874 INFO:teuthology.orchestra.run.smithi111.stdout:required_client_features {} 2024-09-15T22:46:06.874 INFO:teuthology.orchestra.run.smithi111.stdout:last_failure 0 2024-09-15T22:46:06.874 INFO:teuthology.orchestra.run.smithi111.stdout:last_failure_osd_epoch 0 2024-09-15T22:46:06.875 INFO:teuthology.orchestra.run.smithi111.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-15T22:46:06.875 INFO:teuthology.orchestra.run.smithi111.stdout:max_mds 3 2024-09-15T22:46:06.875 INFO:teuthology.orchestra.run.smithi111.stdout:in 0,1,2 2024-09-15T22:46:06.875 INFO:teuthology.orchestra.run.smithi111.stdout:up {0=24545,1=24514,2=24503} 2024-09-15T22:46:06.875 INFO:teuthology.orchestra.run.smithi111.stdout:failed 2024-09-15T22:46:06.875 INFO:teuthology.orchestra.run.smithi111.stdout:damaged 2024-09-15T22:46:06.875 INFO:teuthology.orchestra.run.smithi111.stdout:stopped 2024-09-15T22:46:06.875 INFO:teuthology.orchestra.run.smithi111.stdout:data_pools [3] 2024-09-15T22:46:06.875 INFO:teuthology.orchestra.run.smithi111.stdout:metadata_pool 2 2024-09-15T22:46:06.875 INFO:teuthology.orchestra.run.smithi111.stdout:inline_data disabled 2024-09-15T22:46:06.875 INFO:teuthology.orchestra.run.smithi111.stdout:balancer 2024-09-15T22:46:06.875 INFO:teuthology.orchestra.run.smithi111.stdout:bal_rank_mask -1 2024-09-15T22:46:06.875 INFO:teuthology.orchestra.run.smithi111.stdout:standby_count_wanted 0 2024-09-15T22:46:06.875 INFO:teuthology.orchestra.run.smithi111.stdout:qdb_cluster leader: 24545 members: 24545,24514,24503 2024-09-15T22:46:06.875 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.b{0:24545} state up:active seq 5 addr [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:46:06.876 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.c{0:24496} state up:standby-replay seq 1 addr [v2:172.21.15.192:6836/3995501707,v1:172.21.15.192:6837/3995501707] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:46:06.876 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.i{1:24514} state up:active seq 10 addr [v2:172.21.15.192:6838/800136261,v1:172.21.15.192:6839/800136261] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:46:06.876 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.f{1:24490} state up:standby-replay seq 1 addr [v2:172.21.15.192:6834/4080914921,v1:172.21.15.192:6835/4080914921] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:46:06.876 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.h{2:24503} state up:active seq 11 addr [v2:172.21.15.121:6836/1443772791,v1:172.21.15.121:6837/1443772791] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:46:06.876 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.e{2:24485} state up:standby-replay seq 1 addr [v2:172.21.15.121:6834/3371796806,v1:172.21.15.121:6835/3371796806] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:46:06.876 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:46:06.876 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:46:06.876 INFO:teuthology.orchestra.run.smithi111.stdout:Standby daemons: 2024-09-15T22:46:06.876 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:46:06.876 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.j{-1:14442} state up:standby seq 1 addr [v2:172.21.15.111:6834/453336603,v1:172.21.15.111:6835/453336603] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:46:06.876 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.d{-1:14454} state up:standby seq 1 addr [v2:172.21.15.111:6836/3181832287,v1:172.21.15.111:6837/3181832287] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:46:06.876 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.g{-1:14496} state up:standby seq 1 addr [v2:172.21.15.111:6838/4190237053,v1:172.21.15.111:6839/4190237053] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:46:06.876 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.a{-1:14502} state up:standby seq 1 addr [v2:172.21.15.111:6840/267728828,v1:172.21.15.111:6841/267728828] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:46:06.876 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.l{-1:24448} state up:standby seq 1 addr [v2:172.21.15.192:6832/4061441359,v1:172.21.15.192:6833/4061441359] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:46:06.877 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.k{-1:24449} state up:standby seq 1 addr [v2:172.21.15.121:6832/4220531578,v1:172.21.15.121:6833/4220531578] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:46:06.877 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 25 2024-09-15T22:46:06.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.873+0000 7fafc1e00640 1 -- 172.21.15.111:0/2057627566 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fafa00a32f0 msgr2=0x7fafa00a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:06.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.873+0000 7fafc1e00640 1 --2- 172.21.15.111:0/2057627566 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fafa00a32f0 0x7fafa00a57b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fafa402aa00 tx=0x7fafa40315b0 comp rx=0 tx=0).stop 2024-09-15T22:46:06.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.873+0000 7fafc1e00640 1 -- 172.21.15.111:0/2057627566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafbc103e00 msgr2=0x7fafbc19a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:06.877 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.873+0000 7fafc1e00640 1 --2- 172.21.15.111:0/2057627566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafbc103e00 0x7fafbc19a100 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fafac00e9f0 tx=0x7fafac00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:46:06.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.873+0000 7fafc1e00640 1 -- 172.21.15.111:0/2057627566 shutdown_connections 2024-09-15T22:46:06.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.873+0000 7fafc1e00640 1 --2- 172.21.15.111:0/2057627566 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fafa00a32f0 0x7fafa00a57b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:06.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.873+0000 7fafc1e00640 1 --2- 172.21.15.111:0/2057627566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafbc105920 0x7fafbc1a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:06.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.873+0000 7fafc1e00640 1 --2- 172.21.15.111:0/2057627566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafbc104f60 0x7fafbc19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:06.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.873+0000 7fafc1e00640 1 --2- 172.21.15.111:0/2057627566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fafbc103e00 0x7fafbc19a100 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:06.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.873+0000 7fafc1e00640 1 -- 172.21.15.111:0/2057627566 >> 172.21.15.111:0/2057627566 conn(0x7fafbc0ff5b0 msgr2=0x7fafbc1082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:06.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.873+0000 7fafc1e00640 1 -- 172.21.15.111:0/2057627566 shutdown_connections 2024-09-15T22:46:06.878 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:06.873+0000 7fafc1e00640 1 -- 172.21.15.111:0/2057627566 wait complete. 2024-09-15T22:46:07.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:06 smithi192 ceph-mon[35427]: from='client.24962 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:46:07.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:06 smithi121 ceph-mon[34815]: from='client.24962 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:46:07.664 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-15T22:46:07.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:07 smithi111 ceph-mon[30629]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-15T22:46:07.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:07 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/2057627566' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-15T22:46:08.030 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:46:08.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:07 smithi192 ceph-mon[35427]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-15T22:46:08.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:07 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/2057627566' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-15T22:46:08.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:07 smithi121 ceph-mon[34815]: pgmap v356: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-15T22:46:08.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:07 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/2057627566' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-15T22:46:08.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.565+0000 7f440b000640 1 -- 172.21.15.111:0/116287302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4404105810 msgr2=0x7f440410c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:08.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.565+0000 7f440b000640 1 --2- 172.21.15.111:0/116287302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4404105810 0x7f440410c0a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009ca0 tx=0x7f43fc02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:46:08.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.565+0000 7f440b000640 1 -- 172.21.15.111:0/116287302 shutdown_connections 2024-09-15T22:46:08.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.565+0000 7f440b000640 1 --2- 172.21.15.111:0/116287302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4404105810 0x7f440410c0a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:08.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.565+0000 7f440b000640 1 --2- 172.21.15.111:0/116287302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4404104e50 0x7f44041052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:08.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.565+0000 7f440b000640 1 --2- 172.21.15.111:0/116287302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4404103c50 0x7f4404104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:08.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.565+0000 7f440b000640 1 -- 172.21.15.111:0/116287302 >> 172.21.15.111:0/116287302 conn(0x7f44040ff3e0 msgr2=0x7f4404101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:08.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.565+0000 7f440b000640 1 -- 172.21.15.111:0/116287302 shutdown_connections 2024-09-15T22:46:08.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.565+0000 7f440b000640 1 -- 172.21.15.111:0/116287302 wait complete. 2024-09-15T22:46:08.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.566+0000 7f440b000640 1 Processor -- start 2024-09-15T22:46:08.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.566+0000 7f440b000640 1 -- start start 2024-09-15T22:46:08.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.566+0000 7f440b000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4404103c50 0x7f4404199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:08.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.566+0000 7f440b000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4404104e50 0x7f440419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:08.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.566+0000 7f440b000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4404105810 0x7f44041a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:08.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.566+0000 7f440b000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f440410f920 con 0x7f4404103c50 2024-09-15T22:46:08.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.566+0000 7f440b000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f440410f7a0 con 0x7f4404104e50 2024-09-15T22:46:08.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.566+0000 7f440b000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f440410faa0 con 0x7f4404105810 2024-09-15T22:46:08.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.566+0000 7f4409c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4404103c50 0x7f4404199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:08.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.566+0000 7f4409c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4404103c50 0x7f4404199ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36564/0 (socket says 172.21.15.111:36564) 2024-09-15T22:46:08.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.566+0000 7f4409c00640 1 -- 172.21.15.111:0/3830446164 learned_addr learned my addr 172.21.15.111:0/3830446164 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:08.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.566+0000 7f4409200640 1 --2- 172.21.15.111:0/3830446164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4404104e50 0x7f440419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:08.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.567+0000 7f440a600640 1 --2- 172.21.15.111:0/3830446164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4404105810 0x7f44041a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:08.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.567+0000 7f4409c00640 1 -- 172.21.15.111:0/3830446164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4404105810 msgr2=0x7f44041a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:08.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.567+0000 7f4409c00640 1 --2- 172.21.15.111:0/3830446164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4404105810 0x7f44041a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:08.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.567+0000 7f4409c00640 1 -- 172.21.15.111:0/3830446164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4404104e50 msgr2=0x7f440419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:08.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.567+0000 7f4409c00640 1 --2- 172.21.15.111:0/3830446164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4404104e50 0x7f440419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:08.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.567+0000 7f4409c00640 1 -- 172.21.15.111:0/3830446164 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44041a1b20 con 0x7f4404103c50 2024-09-15T22:46:08.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.567+0000 7f4409c00640 1 --2- 172.21.15.111:0/3830446164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4404103c50 0x7f4404199ef0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f43f800d950 tx=0x7f43f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:08.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.567+0000 7f4402000640 1 -- 172.21.15.111:0/3830446164 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f8017070 con 0x7f4404103c50 2024-09-15T22:46:08.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.567+0000 7f4402000640 1 -- 172.21.15.111:0/3830446164 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43f8009c30 con 0x7f4404103c50 2024-09-15T22:46:08.576 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.567+0000 7f440b000640 1 -- 172.21.15.111:0/3830446164 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44041a1e10 con 0x7f4404103c50 2024-09-15T22:46:08.576 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.567+0000 7f4402000640 1 -- 172.21.15.111:0/3830446164 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f800f600 con 0x7f4404103c50 2024-09-15T22:46:08.576 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.568+0000 7f440b000640 1 -- 172.21.15.111:0/3830446164 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44041a2260 con 0x7f4404103c50 2024-09-15T22:46:08.576 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.569+0000 7f440b000640 1 -- 172.21.15.111:0/3830446164 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f440410af80 con 0x7f4404103c50 2024-09-15T22:46:08.576 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.570+0000 7f4402000640 1 -- 172.21.15.111:0/3830446164 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f43f801a070 con 0x7f4404103c50 2024-09-15T22:46:08.576 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.572+0000 7f4402000640 1 --2- 172.21.15.111:0/3830446164 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f43ec0a32f0 0x7f43ec0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:08.576 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.572+0000 7f4402000640 1 -- 172.21.15.111:0/3830446164 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f43f80cf250 con 0x7f4404103c50 2024-09-15T22:46:08.577 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.572+0000 7f4409200640 1 --2- 172.21.15.111:0/3830446164 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f43ec0a32f0 0x7f43ec0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:08.577 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.573+0000 7f4409200640 1 --2- 172.21.15.111:0/3830446164 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f43ec0a32f0 0x7f43ec0a57b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f440419b410 tx=0x7f43f000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:08.577 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.576+0000 7f4402000640 1 -- 172.21.15.111:0/3830446164 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f43f809b920 con 0x7f4404103c50 2024-09-15T22:46:08.755 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.753+0000 7f440b000640 1 -- 172.21.15.111:0/3830446164 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f440402d7c0 con 0x7f4404103c50 2024-09-15T22:46:08.756 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.754+0000 7f4402000640 1 -- 172.21.15.111:0/3830446164 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) ==== 56+0+5262 (secure 0 0 0) 0x7f43f809fc20 con 0x7f4404103c50 2024-09-15T22:46:08.756 INFO:teuthology.orchestra.run.smithi111.stdout:epoch 88 2024-09-15T22:46:08.756 INFO:teuthology.orchestra.run.smithi111.stdout:fsid b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T22:46:08.756 INFO:teuthology.orchestra.run.smithi111.stdout:created 2024-09-15T22:35:52.571473+0000 2024-09-15T22:46:08.757 INFO:teuthology.orchestra.run.smithi111.stdout:modified 2024-09-15T22:44:51.627977+0000 2024-09-15T22:46:08.757 INFO:teuthology.orchestra.run.smithi111.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-15T22:46:08.757 INFO:teuthology.orchestra.run.smithi111.stdout:crush_version 26 2024-09-15T22:46:08.757 INFO:teuthology.orchestra.run.smithi111.stdout:full_ratio 0.95 2024-09-15T22:46:08.757 INFO:teuthology.orchestra.run.smithi111.stdout:backfillfull_ratio 0.9 2024-09-15T22:46:08.757 INFO:teuthology.orchestra.run.smithi111.stdout:nearfull_ratio 0.85 2024-09-15T22:46:08.757 INFO:teuthology.orchestra.run.smithi111.stdout:require_min_compat_client luminous 2024-09-15T22:46:08.757 INFO:teuthology.orchestra.run.smithi111.stdout:min_compat_client luminous 2024-09-15T22:46:08.757 INFO:teuthology.orchestra.run.smithi111.stdout:require_osd_release squid 2024-09-15T22:46:08.757 INFO:teuthology.orchestra.run.smithi111.stdout:stretch_mode_enabled false 2024-09-15T22:46:08.757 INFO:teuthology.orchestra.run.smithi111.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-15T22:46:08.757 INFO:teuthology.orchestra.run.smithi111.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-15T22:46:08.757 INFO:teuthology.orchestra.run.smithi111.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-15T22:46:08.757 INFO:teuthology.orchestra.run.smithi111.stdout:max_osd 12 2024-09-15T22:46:08.758 INFO:teuthology.orchestra.run.smithi111.stdout:osd.0 up in weight 1 up_from 8 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6802/1386823446,v1:172.21.15.111:6803/1386823446] [v2:172.21.15.111:6804/1386823446,v1:172.21.15.111:6805/1386823446] exists,up 04f39604-be0e-4261-b316-67fa087c6666 2024-09-15T22:46:08.758 INFO:teuthology.orchestra.run.smithi111.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6800/2051512291,v1:172.21.15.121:6801/2051512291] [v2:172.21.15.121:6802/2051512291,v1:172.21.15.121:6803/2051512291] exists,up 1bdbef6e-23fa-4618-a788-66e57741a2a8 2024-09-15T22:46:08.758 INFO:teuthology.orchestra.run.smithi111.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6800/93085639,v1:172.21.15.192:6801/93085639] [v2:172.21.15.192:6802/93085639,v1:172.21.15.192:6803/93085639] exists,up 46735ccf-6d0e-4154-bb55-422929819bfa 2024-09-15T22:46:08.758 INFO:teuthology.orchestra.run.smithi111.stdout:osd.3 up in weight 1 up_from 25 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6810/2578259041,v1:172.21.15.111:6811/2578259041] [v2:172.21.15.111:6812/2578259041,v1:172.21.15.111:6813/2578259041] exists,up 4b243af8-c958-40b1-ba24-9c7ace00c4a3 2024-09-15T22:46:08.758 INFO:teuthology.orchestra.run.smithi111.stdout:osd.4 up in weight 1 up_from 32 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6808/2667595915,v1:172.21.15.121:6809/2667595915] [v2:172.21.15.121:6810/2667595915,v1:172.21.15.121:6811/2667595915] exists,up 49f27541-9cbf-48e0-a466-6bb0fa5410b9 2024-09-15T22:46:08.758 INFO:teuthology.orchestra.run.smithi111.stdout:osd.5 up in weight 1 up_from 37 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6808/1958639856,v1:172.21.15.192:6809/1958639856] [v2:172.21.15.192:6810/1958639856,v1:172.21.15.192:6811/1958639856] exists,up 9290e769-5470-44d3-8d24-25093af34335 2024-09-15T22:46:08.758 INFO:teuthology.orchestra.run.smithi111.stdout:osd.6 up in weight 1 up_from 44 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6818/1371543706,v1:172.21.15.111:6819/1371543706] [v2:172.21.15.111:6820/1371543706,v1:172.21.15.111:6821/1371543706] exists,up 18911b97-1c68-4e8f-92ec-35d2ff0b5f19 2024-09-15T22:46:08.758 INFO:teuthology.orchestra.run.smithi111.stdout:osd.7 up in weight 1 up_from 49 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6816/3501842998,v1:172.21.15.121:6817/3501842998] [v2:172.21.15.121:6818/3501842998,v1:172.21.15.121:6819/3501842998] exists,up 470647c5-c00d-426b-a134-cca235902307 2024-09-15T22:46:08.758 INFO:teuthology.orchestra.run.smithi111.stdout:osd.8 up in weight 1 up_from 57 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6816/2417771374,v1:172.21.15.192:6817/2417771374] [v2:172.21.15.192:6818/2417771374,v1:172.21.15.192:6819/2417771374] exists,up fa2d3dd5-a768-4ada-a036-4c596b76d654 2024-09-15T22:46:08.758 INFO:teuthology.orchestra.run.smithi111.stdout:osd.9 up in weight 1 up_from 65 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6826/3979119431,v1:172.21.15.111:6827/3979119431] [v2:172.21.15.111:6828/3979119431,v1:172.21.15.111:6829/3979119431] exists,up 71b4f5fb-1c02-461b-a257-c6e48381d768 2024-09-15T22:46:08.758 INFO:teuthology.orchestra.run.smithi111.stdout:osd.10 up in weight 1 up_from 71 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6824/2897902637,v1:172.21.15.121:6825/2897902637] [v2:172.21.15.121:6826/2897902637,v1:172.21.15.121:6827/2897902637] exists,up e6a39e77-fce3-4e22-bf66-a677c9f8b6bd 2024-09-15T22:46:08.758 INFO:teuthology.orchestra.run.smithi111.stdout:osd.11 up in weight 1 up_from 78 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6824/2431464812,v1:172.21.15.192:6825/2431464812] [v2:172.21.15.192:6826/2431464812,v1:172.21.15.192:6827/2431464812] exists,up 411a7921-6717-4cef-b8df-d9e252edcc84 2024-09-15T22:46:08.758 INFO:teuthology.orchestra.run.smithi111.stdout:pg_upmap_items 2.8 [2,11] 2024-09-15T22:46:08.759 INFO:teuthology.orchestra.run.smithi111.stdout:pg_upmap_items 2.b [8,11] 2024-09-15T22:46:08.759 INFO:teuthology.orchestra.run.smithi111.stdout:pg_upmap_items 2.1b [2,4] 2024-09-15T22:46:08.759 INFO:teuthology.orchestra.run.smithi111.stdout:pg_upmap_items 3.3 [4,9] 2024-09-15T22:46:08.759 INFO:teuthology.orchestra.run.smithi111.stdout:pg_upmap_items 3.c [3,9] 2024-09-15T22:46:08.759 INFO:teuthology.orchestra.run.smithi111.stdout:pg_upmap_items 3.36 [4,9] 2024-09-15T22:46:08.759 INFO:teuthology.orchestra.run.smithi111.stdout:blocklist 172.21.15.111:6801/2107775445 expires 2024-09-16T22:36:50.357551+0000 2024-09-15T22:46:08.759 INFO:teuthology.orchestra.run.smithi111.stdout:blocklist 172.21.15.111:6800/2107775445 expires 2024-09-16T22:36:50.357551+0000 2024-09-15T22:46:08.759 INFO:teuthology.orchestra.run.smithi111.stdout:blocklist 172.21.15.111:0/2394746540 expires 2024-09-16T22:36:50.357551+0000 2024-09-15T22:46:08.759 INFO:teuthology.orchestra.run.smithi111.stdout:blocklist 172.21.15.111:0/4140020843 expires 2024-09-16T22:36:50.357551+0000 2024-09-15T22:46:08.759 INFO:teuthology.orchestra.run.smithi111.stdout:blocklist 172.21.15.111:0/2114354263 expires 2024-09-16T22:36:50.357551+0000 2024-09-15T22:46:08.759 INFO:teuthology.orchestra.run.smithi111.stdout:blocklist 172.21.15.111:0/1054173642 expires 2024-09-16T22:36:22.652875+0000 2024-09-15T22:46:08.759 INFO:teuthology.orchestra.run.smithi111.stdout:blocklist 172.21.15.111:0/3408238004 expires 2024-09-16T22:36:22.652875+0000 2024-09-15T22:46:08.759 INFO:teuthology.orchestra.run.smithi111.stdout:blocklist 172.21.15.111:0/2879044772 expires 2024-09-16T22:36:22.652875+0000 2024-09-15T22:46:08.759 INFO:teuthology.orchestra.run.smithi111.stdout:blocklist 172.21.15.111:6801/3350425104 expires 2024-09-16T22:36:22.652875+0000 2024-09-15T22:46:08.759 INFO:teuthology.orchestra.run.smithi111.stdout:blocklist 172.21.15.111:6800/3350425104 expires 2024-09-16T22:36:22.652875+0000 2024-09-15T22:46:08.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.757+0000 7f440b000640 1 -- 172.21.15.111:0/3830446164 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f43ec0a32f0 msgr2=0x7f43ec0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:08.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.757+0000 7f440b000640 1 --2- 172.21.15.111:0/3830446164 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f43ec0a32f0 0x7f43ec0a57b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f440419b410 tx=0x7f43f000b410 comp rx=0 tx=0).stop 2024-09-15T22:46:08.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.757+0000 7f440b000640 1 -- 172.21.15.111:0/3830446164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4404103c50 msgr2=0x7f4404199ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:08.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.757+0000 7f440b000640 1 --2- 172.21.15.111:0/3830446164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4404103c50 0x7f4404199ef0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f43f800d950 tx=0x7f43f800de10 comp rx=0 tx=0).stop 2024-09-15T22:46:08.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.757+0000 7f440b000640 1 -- 172.21.15.111:0/3830446164 shutdown_connections 2024-09-15T22:46:08.760 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.757+0000 7f440b000640 1 --2- 172.21.15.111:0/3830446164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4404105810 0x7f44041a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:08.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.757+0000 7f440b000640 1 --2- 172.21.15.111:0/3830446164 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f43ec0a32f0 0x7f43ec0a57b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:08.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.757+0000 7f440b000640 1 --2- 172.21.15.111:0/3830446164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4404104e50 0x7f440419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:08.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.757+0000 7f440b000640 1 --2- 172.21.15.111:0/3830446164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4404103c50 0x7f4404199ef0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:08.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.757+0000 7f440b000640 1 -- 172.21.15.111:0/3830446164 >> 172.21.15.111:0/3830446164 conn(0x7f44040ff3e0 msgr2=0x7f4404108150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:08.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.757+0000 7f440b000640 1 -- 172.21.15.111:0/3830446164 shutdown_connections 2024-09-15T22:46:08.761 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:08.757+0000 7f440b000640 1 -- 172.21.15.111:0/3830446164 wait complete. 2024-09-15T22:46:09.614 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-15T22:46:09.626 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:46:09.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:09 smithi111 ceph-mon[30629]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:46:09.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:09 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/3830446164' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-15T22:46:09.991 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:46:10.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:09 smithi192 ceph-mon[35427]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:46:10.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:09 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/3830446164' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-15T22:46:10.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:09 smithi121 ceph-mon[34815]: pgmap v357: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:46:10.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:09 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/3830446164' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-15T22:46:10.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.550+0000 7f8aa7c00640 1 -- 172.21.15.111:0/3514646451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8aa0106f10 msgr2=0x7f8aa010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:10.555 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.550+0000 7f8aa7c00640 1 --2- 172.21.15.111:0/3514646451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8aa0106f10 0x7f8aa010ea90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009ca0 tx=0x7f8a9802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:46:10.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.550+0000 7f8aa7c00640 1 -- 172.21.15.111:0/3514646451 shutdown_connections 2024-09-15T22:46:10.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.550+0000 7f8aa7c00640 1 --2- 172.21.15.111:0/3514646451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8aa0106f10 0x7f8aa010ea90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:10.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.550+0000 7f8aa7c00640 1 --2- 172.21.15.111:0/3514646451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8aa0102790 0x7f8aa01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:10.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.550+0000 7f8aa7c00640 1 --2- 172.21.15.111:0/3514646451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8aa0101de0 0x7f8aa01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:10.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.550+0000 7f8aa7c00640 1 -- 172.21.15.111:0/3514646451 >> 172.21.15.111:0/3514646451 conn(0x7f8aa00fd8c0 msgr2=0x7f8aa00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:10.556 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.550+0000 7f8aa7c00640 1 -- 172.21.15.111:0/3514646451 shutdown_connections 2024-09-15T22:46:10.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.550+0000 7f8aa7c00640 1 -- 172.21.15.111:0/3514646451 wait complete. 2024-09-15T22:46:10.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.551+0000 7f8aa7c00640 1 Processor -- start 2024-09-15T22:46:10.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.551+0000 7f8aa7c00640 1 -- start start 2024-09-15T22:46:10.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.552+0000 7f8aa7c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8aa0101de0 0x7f8aa019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:10.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.552+0000 7f8aa7c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8aa0102790 0x7f8aa019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:10.557 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.552+0000 7f8aa7c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8aa0106f10 0x7f8aa019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:10.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.552+0000 7f8aa7c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa0111fc0 con 0x7f8aa0101de0 2024-09-15T22:46:10.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.552+0000 7f8aa7c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa0111e40 con 0x7f8aa0106f10 2024-09-15T22:46:10.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.552+0000 7f8aa7c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa0112140 con 0x7f8aa0102790 2024-09-15T22:46:10.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.552+0000 7f8aa5200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8aa0101de0 0x7f8aa019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:10.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.552+0000 7f8aa5200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8aa0101de0 0x7f8aa019a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36580/0 (socket says 172.21.15.111:36580) 2024-09-15T22:46:10.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.552+0000 7f8aa5200640 1 -- 172.21.15.111:0/1498571049 learned_addr learned my addr 172.21.15.111:0/1498571049 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:10.558 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.552+0000 7f8aa5c00640 1 --2- 172.21.15.111:0/1498571049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8aa0106f10 0x7f8aa019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:10.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.552+0000 7f8a9fe00640 1 --2- 172.21.15.111:0/1498571049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8aa0102790 0x7f8aa019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:10.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.552+0000 7f8aa5200640 1 -- 172.21.15.111:0/1498571049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8aa0102790 msgr2=0x7f8aa019ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:10.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.552+0000 7f8aa5200640 1 --2- 172.21.15.111:0/1498571049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8aa0102790 0x7f8aa019ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:10.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.552+0000 7f8aa5200640 1 -- 172.21.15.111:0/1498571049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8aa0106f10 msgr2=0x7f8aa019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:10.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.552+0000 7f8aa5200640 1 --2- 172.21.15.111:0/1498571049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8aa0106f10 0x7f8aa019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:10.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.552+0000 7f8aa5200640 1 -- 172.21.15.111:0/1498571049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa019f5b0 con 0x7f8aa0101de0 2024-09-15T22:46:10.559 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.553+0000 7f8aa5200640 1 --2- 172.21.15.111:0/1498571049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8aa0101de0 0x7f8aa019a600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400e9f0 tx=0x7f8a9400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:10.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.553+0000 7f8a9d600640 1 -- 172.21.15.111:0/1498571049 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9400cd80 con 0x7f8aa0101de0 2024-09-15T22:46:10.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.553+0000 7f8a9d600640 1 -- 172.21.15.111:0/1498571049 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a94005500 con 0x7f8aa0101de0 2024-09-15T22:46:10.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.553+0000 7f8aa7c00640 1 -- 172.21.15.111:0/1498571049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa019f8a0 con 0x7f8aa0101de0 2024-09-15T22:46:10.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.553+0000 7f8aa7c00640 1 -- 172.21.15.111:0/1498571049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa01a7180 con 0x7f8aa0101de0 2024-09-15T22:46:10.560 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.553+0000 7f8a9d600640 1 -- 172.21.15.111:0/1498571049 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a94010690 con 0x7f8aa0101de0 2024-09-15T22:46:10.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.554+0000 7f8aa7c00640 1 -- 172.21.15.111:0/1498571049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a80005290 con 0x7f8aa0101de0 2024-09-15T22:46:10.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.556+0000 7f8a9d600640 1 -- 172.21.15.111:0/1498571049 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f8a940046c0 con 0x7f8aa0101de0 2024-09-15T22:46:10.561 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.558+0000 7f8a9d600640 1 --2- 172.21.15.111:0/1498571049 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8a880a32c0 0x7f8a880a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:10.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.558+0000 7f8a9d600640 1 -- 172.21.15.111:0/1498571049 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f8a940cfe70 con 0x7f8aa0101de0 2024-09-15T22:46:10.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.558+0000 7f8a9fe00640 1 --2- 172.21.15.111:0/1498571049 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8a880a32c0 0x7f8a880a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:10.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.558+0000 7f8a9fe00640 1 --2- 172.21.15.111:0/1498571049 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8a880a32c0 0x7f8a880a5780 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8aa010eeb0 tx=0x7f8a8c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:10.562 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.560+0000 7f8a9d600640 1 -- 172.21.15.111:0/1498571049 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8a9409cb90 con 0x7f8aa0101de0 2024-09-15T22:46:10.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.770+0000 7f8aa7c00640 1 -- 172.21.15.111:0/1498571049 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a80005580 con 0x7f8aa0101de0 2024-09-15T22:46:10.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.771+0000 7f8a9d600640 1 -- 172.21.15.111:0/1498571049 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11931 (secure 0 0 0) 0x7f8a9409c490 con 0x7f8aa0101de0 2024-09-15T22:46:10.773 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:46:10.773 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":25,"btime":"2024-09-15T22:44:34:567598+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/453336603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":453336603},{"type":"v1","addr":"172.21.15.111:6835","nonce":453336603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/3181832287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3181832287},{"type":"v1","addr":"172.21.15.111:6837","nonce":3181832287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/4190237053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":4190237053},{"type":"v1","addr":"172.21.15.111:6839","nonce":4190237053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14502,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6841/267728828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6840","nonce":267728828},{"type":"v1","addr":"172.21.15.111:6841","nonce":267728828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/4061441359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":4061441359},{"type":"v1","addr":"172.21.15.192:6833","nonce":4061441359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/4220531578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":4220531578},{"type":"v1","addr":"172.21.15.121:6833","nonce":4220531578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:44:18.436394+0000","modified":"2024-09-15T22:44:33.655164+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24514,"mds_2":24503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3371796806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3371796806},{"type":"v1","addr":"172.21.15.121:6835","nonce":3371796806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/4080914921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":4080914921},{"type":"v1","addr":"172.21.15.192:6835","nonce":4080914921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/3995501707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3995501707},{"type":"v1","addr":"172.21.15.192:6837","nonce":3995501707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.121:6837/1443772791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1443772791},{"type":"v1","addr":"172.21.15.121:6837","nonce":1443772791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6839/800136261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":800136261},{"type":"v1","addr":"172.21.15.192:6839","nonce":800136261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.121:6839/1797352097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1797352097},{"type":"v1","addr":"172.21.15.121:6839","nonce":1797352097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545,24514,24503]},"id":1}]} 2024-09-15T22:46:10.773 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 25 2024-09-15T22:46:10.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.773+0000 7f8aa7c00640 1 -- 172.21.15.111:0/1498571049 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8a880a32c0 msgr2=0x7f8a880a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:10.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.773+0000 7f8aa7c00640 1 --2- 172.21.15.111:0/1498571049 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8a880a32c0 0x7f8a880a5780 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8aa010eeb0 tx=0x7f8a8c00b390 comp rx=0 tx=0).stop 2024-09-15T22:46:10.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.773+0000 7f8aa7c00640 1 -- 172.21.15.111:0/1498571049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8aa0101de0 msgr2=0x7f8aa019a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:10.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.773+0000 7f8aa7c00640 1 --2- 172.21.15.111:0/1498571049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8aa0101de0 0x7f8aa019a600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400e9f0 tx=0x7f8a9400edf0 comp rx=0 tx=0).stop 2024-09-15T22:46:10.775 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.774+0000 7f8aa7c00640 1 -- 172.21.15.111:0/1498571049 shutdown_connections 2024-09-15T22:46:10.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.774+0000 7f8aa7c00640 1 --2- 172.21.15.111:0/1498571049 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8a880a32c0 0x7f8a880a5780 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:10.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.774+0000 7f8aa7c00640 1 --2- 172.21.15.111:0/1498571049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8aa0106f10 0x7f8aa019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:10.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.774+0000 7f8aa7c00640 1 --2- 172.21.15.111:0/1498571049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8aa0102790 0x7f8aa019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:10.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.774+0000 7f8aa7c00640 1 --2- 172.21.15.111:0/1498571049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8aa0101de0 0x7f8aa019a600 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:10.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.774+0000 7f8aa7c00640 1 -- 172.21.15.111:0/1498571049 >> 172.21.15.111:0/1498571049 conn(0x7f8aa00fd8c0 msgr2=0x7f8aa00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:10.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.774+0000 7f8aa7c00640 1 -- 172.21.15.111:0/1498571049 shutdown_connections 2024-09-15T22:46:10.776 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:10.774+0000 7f8aa7c00640 1 -- 172.21.15.111:0/1498571049 wait complete. 2024-09-15T22:46:11.655 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:46:11.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:11 smithi111 ceph-mon[30629]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:46:11.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:11 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/1498571049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:46:12.021 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:46:12.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:11 smithi192 ceph-mon[35427]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:46:12.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:11 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/1498571049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:46:12.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:11 smithi121 ceph-mon[34815]: pgmap v358: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:46:12.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:11 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/1498571049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:46:12.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.635+0000 7f44fe000640 1 -- 172.21.15.111:0/1673798710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44f8074f40 msgr2=0x7f44f810de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:12.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.635+0000 7f44fe000640 1 --2- 172.21.15.111:0/1673798710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44f8074f40 0x7f44f810de80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f44e4009900 tx=0x7f44e402a480 comp rx=0 tx=0).stop 2024-09-15T22:46:12.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.635+0000 7f44fe000640 1 -- 172.21.15.111:0/1673798710 shutdown_connections 2024-09-15T22:46:12.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.635+0000 7f44fe000640 1 --2- 172.21.15.111:0/1673798710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44f810e3c0 0x7f44f8110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:12.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.635+0000 7f44fe000640 1 --2- 172.21.15.111:0/1673798710 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44f8074f40 0x7f44f810de80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:12.679 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.635+0000 7f44fe000640 1 --2- 172.21.15.111:0/1673798710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44f8074620 0x7f44f8074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:12.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.635+0000 7f44fe000640 1 -- 172.21.15.111:0/1673798710 >> 172.21.15.111:0/1673798710 conn(0x7f44f80fd8c0 msgr2=0x7f44f80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:12.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.635+0000 7f44fe000640 1 -- 172.21.15.111:0/1673798710 shutdown_connections 2024-09-15T22:46:12.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.635+0000 7f44fe000640 1 -- 172.21.15.111:0/1673798710 wait complete. 2024-09-15T22:46:12.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.636+0000 7f44fe000640 1 Processor -- start 2024-09-15T22:46:12.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.636+0000 7f44fe000640 1 -- start start 2024-09-15T22:46:12.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.636+0000 7f44fe000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44f8074620 0x7f44f819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:12.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.637+0000 7f44fe000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44f8074f40 0x7f44f819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:12.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.637+0000 7f44fe000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44f810e3c0 0x7f44f819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:12.680 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.637+0000 7f44fe000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f44f81141b0 con 0x7f44f8074620 2024-09-15T22:46:12.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.637+0000 7f44fe000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f44f8114030 con 0x7f44f8074f40 2024-09-15T22:46:12.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.637+0000 7f44fe000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f44f8114330 con 0x7f44f810e3c0 2024-09-15T22:46:12.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.637+0000 7f44f7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44f8074620 0x7f44f819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:12.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.637+0000 7f44f7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44f8074620 0x7f44f819a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44450/0 (socket says 172.21.15.111:44450) 2024-09-15T22:46:12.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.637+0000 7f44f7400640 1 -- 172.21.15.111:0/2388970048 learned_addr learned my addr 172.21.15.111:0/2388970048 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:12.681 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.637+0000 7f44f7e00640 1 --2- 172.21.15.111:0/2388970048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44f810e3c0 0x7f44f819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:12.682 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.637+0000 7f44f6a00640 1 --2- 172.21.15.111:0/2388970048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44f8074f40 0x7f44f819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:12.682 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.637+0000 7f44f7400640 1 -- 172.21.15.111:0/2388970048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44f810e3c0 msgr2=0x7f44f819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:12.682 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.637+0000 7f44f7400640 1 --2- 172.21.15.111:0/2388970048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44f810e3c0 0x7f44f819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:12.682 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.637+0000 7f44f7400640 1 -- 172.21.15.111:0/2388970048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44f8074f40 msgr2=0x7f44f819abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:12.682 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.637+0000 7f44f7400640 1 --2- 172.21.15.111:0/2388970048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44f8074f40 0x7f44f819abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:12.682 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.637+0000 7f44f7400640 1 -- 172.21.15.111:0/2388970048 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f819f660 con 0x7f44f8074620 2024-09-15T22:46:12.682 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.637+0000 7f44f6a00640 1 --2- 172.21.15.111:0/2388970048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44f8074f40 0x7f44f819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:46:12.682 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.637+0000 7f44f7400640 1 --2- 172.21.15.111:0/2388970048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44f8074620 0x7f44f819a6b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f44e800e9f0 tx=0x7f44e800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:12.682 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.638+0000 7f44efe00640 1 -- 172.21.15.111:0/2388970048 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e800cd80 con 0x7f44f8074620 2024-09-15T22:46:12.682 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.638+0000 7f44efe00640 1 -- 172.21.15.111:0/2388970048 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44e8005500 con 0x7f44f8074620 2024-09-15T22:46:12.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.638+0000 7f44efe00640 1 -- 172.21.15.111:0/2388970048 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e8010690 con 0x7f44f8074620 2024-09-15T22:46:12.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.638+0000 7f44fe000640 1 -- 172.21.15.111:0/2388970048 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f819f950 con 0x7f44f8074620 2024-09-15T22:46:12.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.638+0000 7f44fe000640 1 -- 172.21.15.111:0/2388970048 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f81a7230 con 0x7f44f8074620 2024-09-15T22:46:12.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.639+0000 7f44fe000640 1 -- 172.21.15.111:0/2388970048 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44f8074a00 con 0x7f44f8074620 2024-09-15T22:46:12.683 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.641+0000 7f44efe00640 1 -- 172.21.15.111:0/2388970048 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f44e80046c0 con 0x7f44f8074620 2024-09-15T22:46:12.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.642+0000 7f44efe00640 1 --2- 172.21.15.111:0/2388970048 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f44dc0a32c0 0x7f44dc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:12.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.642+0000 7f44efe00640 1 -- 172.21.15.111:0/2388970048 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f44e80cfd40 con 0x7f44f8074620 2024-09-15T22:46:12.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.642+0000 7f44f6a00640 1 --2- 172.21.15.111:0/2388970048 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f44dc0a32c0 0x7f44dc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:12.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.643+0000 7f44f6a00640 1 --2- 172.21.15.111:0/2388970048 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f44dc0a32c0 0x7f44dc0a5780 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f44e4000c00 tx=0x7f44e4005a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:12.684 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.646+0000 7f44efe00640 1 -- 172.21.15.111:0/2388970048 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f44e809ca60 con 0x7f44f8074620 2024-09-15T22:46:12.856 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.855+0000 7f44fe000640 1 -- 172.21.15.111:0/2388970048 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44f802d7c0 con 0x7f44f8074620 2024-09-15T22:46:12.857 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.855+0000 7f44efe00640 1 -- 172.21.15.111:0/2388970048 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11931 (secure 0 0 0) 0x7f44e809c360 con 0x7f44f8074620 2024-09-15T22:46:12.857 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:46:12.858 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":25,"btime":"2024-09-15T22:44:34:567598+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/453336603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":453336603},{"type":"v1","addr":"172.21.15.111:6835","nonce":453336603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/3181832287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3181832287},{"type":"v1","addr":"172.21.15.111:6837","nonce":3181832287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/4190237053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":4190237053},{"type":"v1","addr":"172.21.15.111:6839","nonce":4190237053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14502,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6841/267728828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6840","nonce":267728828},{"type":"v1","addr":"172.21.15.111:6841","nonce":267728828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/4061441359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":4061441359},{"type":"v1","addr":"172.21.15.192:6833","nonce":4061441359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/4220531578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":4220531578},{"type":"v1","addr":"172.21.15.121:6833","nonce":4220531578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:44:18.436394+0000","modified":"2024-09-15T22:44:33.655164+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24514,"mds_2":24503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3371796806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3371796806},{"type":"v1","addr":"172.21.15.121:6835","nonce":3371796806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/4080914921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":4080914921},{"type":"v1","addr":"172.21.15.192:6835","nonce":4080914921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24496":{"gid":24496,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/3995501707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":3995501707},{"type":"v1","addr":"172.21.15.192:6837","nonce":3995501707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.121:6837/1443772791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1443772791},{"type":"v1","addr":"172.21.15.121:6837","nonce":1443772791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6839/800136261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":800136261},{"type":"v1","addr":"172.21.15.192:6839","nonce":800136261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.121:6839/1797352097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1797352097},{"type":"v1","addr":"172.21.15.121:6839","nonce":1797352097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545,24514,24503]},"id":1}]} 2024-09-15T22:46:12.858 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 25 2024-09-15T22:46:12.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.858+0000 7f44fe000640 1 -- 172.21.15.111:0/2388970048 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f44dc0a32c0 msgr2=0x7f44dc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:12.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.858+0000 7f44fe000640 1 --2- 172.21.15.111:0/2388970048 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f44dc0a32c0 0x7f44dc0a5780 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f44e4000c00 tx=0x7f44e4005a40 comp rx=0 tx=0).stop 2024-09-15T22:46:12.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.858+0000 7f44fe000640 1 -- 172.21.15.111:0/2388970048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44f8074620 msgr2=0x7f44f819a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:12.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.858+0000 7f44fe000640 1 --2- 172.21.15.111:0/2388970048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44f8074620 0x7f44f819a6b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f44e800e9f0 tx=0x7f44e800edf0 comp rx=0 tx=0).stop 2024-09-15T22:46:12.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.858+0000 7f44fe000640 1 -- 172.21.15.111:0/2388970048 shutdown_connections 2024-09-15T22:46:12.860 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.859+0000 7f44fe000640 1 --2- 172.21.15.111:0/2388970048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44f810e3c0 0x7f44f819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:12.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.859+0000 7f44fe000640 1 --2- 172.21.15.111:0/2388970048 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f44dc0a32c0 0x7f44dc0a5780 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:12.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.859+0000 7f44fe000640 1 --2- 172.21.15.111:0/2388970048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f44f8074f40 0x7f44f819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:12.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.859+0000 7f44fe000640 1 --2- 172.21.15.111:0/2388970048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f44f8074620 0x7f44f819a6b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:12.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.859+0000 7f44fe000640 1 -- 172.21.15.111:0/2388970048 >> 172.21.15.111:0/2388970048 conn(0x7f44f80fd8c0 msgr2=0x7f44f80fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:12.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.859+0000 7f44fe000640 1 -- 172.21.15.111:0/2388970048 shutdown_connections 2024-09-15T22:46:12.861 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:12.859+0000 7f44fe000640 1 -- 172.21.15.111:0/2388970048 wait complete. 2024-09-15T22:46:13.671 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:46:13.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:13 smithi111 ceph-mon[30629]: pgmap v359: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:46:13.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:13 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/2388970048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:46:14.039 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:46:14.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:13 smithi192 ceph-mon[35427]: pgmap v359: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:46:14.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:13 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/2388970048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:46:14.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:13 smithi121 ceph-mon[34815]: pgmap v359: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:46:14.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:13 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/2388970048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:46:14.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.567+0000 7f7d97e00640 1 -- 172.21.15.111:0/2378358929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d981079c0 msgr2=0x7f7d98107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:14.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.567+0000 7f7d97e00640 1 --2- 172.21.15.111:0/2378358929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d981079c0 0x7f7d98107da0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7d88009930 tx=0x7f7d8802a420 comp rx=0 tx=0).stop 2024-09-15T22:46:14.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.567+0000 7f7d97e00640 1 -- 172.21.15.111:0/2378358929 shutdown_connections 2024-09-15T22:46:14.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.567+0000 7f7d97e00640 1 --2- 172.21.15.111:0/2378358929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d98102360 0x7f7d9810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:14.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.567+0000 7f7d97e00640 1 --2- 172.21.15.111:0/2378358929 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d981019c0 0x7f7d98101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:14.569 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.567+0000 7f7d97e00640 1 --2- 172.21.15.111:0/2378358929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d981079c0 0x7f7d98107da0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:14.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.567+0000 7f7d97e00640 1 -- 172.21.15.111:0/2378358929 >> 172.21.15.111:0/2378358929 conn(0x7f7d980fd740 msgr2=0x7f7d980ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:14.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.567+0000 7f7d97e00640 1 -- 172.21.15.111:0/2378358929 shutdown_connections 2024-09-15T22:46:14.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.567+0000 7f7d97e00640 1 -- 172.21.15.111:0/2378358929 wait complete. 2024-09-15T22:46:14.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.569+0000 7f7d97e00640 1 Processor -- start 2024-09-15T22:46:14.570 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.569+0000 7f7d97e00640 1 -- start start 2024-09-15T22:46:14.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.569+0000 7f7d97e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d981019c0 0x7f7d981982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:14.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.569+0000 7f7d97e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d98102360 0x7f7d98198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:14.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.569+0000 7f7d97e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d981079c0 0x7f7d9819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:14.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.569+0000 7f7d97e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7d98111d90 con 0x7f7d981019c0 2024-09-15T22:46:14.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.569+0000 7f7d97e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7d98111c10 con 0x7f7d98102360 2024-09-15T22:46:14.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.569+0000 7f7d97e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7d98111f10 con 0x7f7d981079c0 2024-09-15T22:46:14.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.570+0000 7f7d96a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d981019c0 0x7f7d981982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:14.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.570+0000 7f7d96a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d981019c0 0x7f7d981982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44464/0 (socket says 172.21.15.111:44464) 2024-09-15T22:46:14.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.570+0000 7f7d96a00640 1 -- 172.21.15.111:0/1236227286 learned_addr learned my addr 172.21.15.111:0/1236227286 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:14.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.570+0000 7f7d96000640 1 --2- 172.21.15.111:0/1236227286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d98102360 0x7f7d98198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:14.572 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.570+0000 7f7d97400640 1 --2- 172.21.15.111:0/1236227286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d981079c0 0x7f7d9819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:14.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.570+0000 7f7d96a00640 1 -- 172.21.15.111:0/1236227286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d981079c0 msgr2=0x7f7d9819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:14.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.570+0000 7f7d96a00640 1 --2- 172.21.15.111:0/1236227286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d981079c0 0x7f7d9819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:14.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.570+0000 7f7d96a00640 1 -- 172.21.15.111:0/1236227286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d98102360 msgr2=0x7f7d98198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:14.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.570+0000 7f7d96a00640 1 --2- 172.21.15.111:0/1236227286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d98102360 0x7f7d98198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:14.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.570+0000 7f7d96a00640 1 -- 172.21.15.111:0/1236227286 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9819d2a0 con 0x7f7d981019c0 2024-09-15T22:46:14.573 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.571+0000 7f7d96a00640 1 --2- 172.21.15.111:0/1236227286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d981019c0 0x7f7d981982f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7d88009230 tx=0x7f7d88004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:14.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.571+0000 7f7d8f400640 1 -- 172.21.15.111:0/1236227286 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8803f030 con 0x7f7d981019c0 2024-09-15T22:46:14.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.571+0000 7f7d8f400640 1 -- 172.21.15.111:0/1236227286 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d88003ad0 con 0x7f7d981019c0 2024-09-15T22:46:14.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.571+0000 7f7d97e00640 1 -- 172.21.15.111:0/1236227286 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d9819d530 con 0x7f7d981019c0 2024-09-15T22:46:14.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.571+0000 7f7d8f400640 1 -- 172.21.15.111:0/1236227286 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d88035070 con 0x7f7d981019c0 2024-09-15T22:46:14.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.571+0000 7f7d97e00640 1 -- 172.21.15.111:0/1236227286 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d981a4e70 con 0x7f7d981019c0 2024-09-15T22:46:14.574 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.572+0000 7f7d97e00640 1 -- 172.21.15.111:0/1236227286 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d98103100 con 0x7f7d981019c0 2024-09-15T22:46:14.575 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.574+0000 7f7d8f400640 1 -- 172.21.15.111:0/1236227286 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f7d88042030 con 0x7f7d981019c0 2024-09-15T22:46:14.577 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.575+0000 7f7d8f400640 1 --2- 172.21.15.111:0/1236227286 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7d7c0a3240 0x7f7d7c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:14.577 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.575+0000 7f7d8f400640 1 -- 172.21.15.111:0/1236227286 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f7d880efb50 con 0x7f7d981019c0 2024-09-15T22:46:14.577 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.575+0000 7f7d96000640 1 --2- 172.21.15.111:0/1236227286 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7d7c0a3240 0x7f7d7c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:14.578 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.576+0000 7f7d96000640 1 --2- 172.21.15.111:0/1236227286 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7d7c0a3240 0x7f7d7c0a5700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7d980ffde0 tx=0x7f7d8400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:14.581 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.579+0000 7f7d8f400640 1 -- 172.21.15.111:0/1236227286 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7d880bc220 con 0x7f7d981019c0 2024-09-15T22:46:14.767 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.765+0000 7f7d97e00640 1 -- 172.21.15.111:0/1236227286 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d98109f30 con 0x7f7d981019c0 2024-09-15T22:46:14.768 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.766+0000 7f7d8f400640 1 -- 172.21.15.111:0/1236227286 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22701 (secure 0 0 0) 0x7f7d880c0520 con 0x7f7d981019c0 2024-09-15T22:46:14.768 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T22:46:14.769 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":88,"fsid":"b7614d3c-73b2-11ef-bceb-c7b262605968","created":"2024-09-15T22:35:52.571473+0000","modified":"2024-09-15T22:44:51.627977+0000","last_up_change":"2024-09-15T22:43:19.567274+0000","last_in_change":"2024-09-15T22:42:56.994246+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:39:18.954207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:44:12.846418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:44:15.685946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04f39604-be0e-4261-b316-67fa087c6666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6803","nonce":1386823446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6805","nonce":1386823446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6809","nonce":1386823446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6807","nonce":1386823446}]},"public_addr":"172.21.15.111:6803/1386823446","cluster_addr":"172.21.15.111:6805/1386823446","heartbeat_back_addr":"172.21.15.111:6809/1386823446","heartbeat_front_addr":"172.21.15.111:6807/1386823446","state":["exists","up"]},{"osd":1,"uuid":"1bdbef6e-23fa-4618-a788-66e57741a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6801","nonce":2051512291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6803","nonce":2051512291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6807","nonce":2051512291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6805","nonce":2051512291}]},"public_addr":"172.21.15.121:6801/2051512291","cluster_addr":"172.21.15.121:6803/2051512291","heartbeat_back_addr":"172.21.15.121:6807/2051512291","heartbeat_front_addr":"172.21.15.121:6805/2051512291","state":["exists","up"]},{"osd":2,"uuid":"46735ccf-6d0e-4154-bb55-422929819bfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6801","nonce":93085639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6803","nonce":93085639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6807","nonce":93085639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6805","nonce":93085639}]},"public_addr":"172.21.15.192:6801/93085639","cluster_addr":"172.21.15.192:6803/93085639","heartbeat_back_addr":"172.21.15.192:6807/93085639","heartbeat_front_addr":"172.21.15.192:6805/93085639","state":["exists","up"]},{"osd":3,"uuid":"4b243af8-c958-40b1-ba24-9c7ace00c4a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6811","nonce":2578259041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6813","nonce":2578259041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6817","nonce":2578259041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6815","nonce":2578259041}]},"public_addr":"172.21.15.111:6811/2578259041","cluster_addr":"172.21.15.111:6813/2578259041","heartbeat_back_addr":"172.21.15.111:6817/2578259041","heartbeat_front_addr":"172.21.15.111:6815/2578259041","state":["exists","up"]},{"osd":4,"uuid":"49f27541-9cbf-48e0-a466-6bb0fa5410b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6809","nonce":2667595915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6811","nonce":2667595915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6815","nonce":2667595915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6813","nonce":2667595915}]},"public_addr":"172.21.15.121:6809/2667595915","cluster_addr":"172.21.15.121:6811/2667595915","heartbeat_back_addr":"172.21.15.121:6815/2667595915","heartbeat_front_addr":"172.21.15.121:6813/2667595915","state":["exists","up"]},{"osd":5,"uuid":"9290e769-5470-44d3-8d24-25093af34335","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6809","nonce":1958639856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6811","nonce":1958639856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6815","nonce":1958639856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6813","nonce":1958639856}]},"public_addr":"172.21.15.192:6809/1958639856","cluster_addr":"172.21.15.192:6811/1958639856","heartbeat_back_addr":"172.21.15.192:6815/1958639856","heartbeat_front_addr":"172.21.15.192:6813/1958639856","state":["exists","up"]},{"osd":6,"uuid":"18911b97-1c68-4e8f-92ec-35d2ff0b5f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6819","nonce":1371543706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6821","nonce":1371543706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6825","nonce":1371543706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6823","nonce":1371543706}]},"public_addr":"172.21.15.111:6819/1371543706","cluster_addr":"172.21.15.111:6821/1371543706","heartbeat_back_addr":"172.21.15.111:6825/1371543706","heartbeat_front_addr":"172.21.15.111:6823/1371543706","state":["exists","up"]},{"osd":7,"uuid":"470647c5-c00d-426b-a134-cca235902307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6817","nonce":3501842998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6819","nonce":3501842998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6823","nonce":3501842998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6821","nonce":3501842998}]},"public_addr":"172.21.15.121:6817/3501842998","cluster_addr":"172.21.15.121:6819/3501842998","heartbeat_back_addr":"172.21.15.121:6823/3501842998","heartbeat_front_addr":"172.21.15.121:6821/3501842998","state":["exists","up"]},{"osd":8,"uuid":"fa2d3dd5-a768-4ada-a036-4c596b76d654","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6817","nonce":2417771374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6819","nonce":2417771374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6823","nonce":2417771374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6821","nonce":2417771374}]},"public_addr":"172.21.15.192:6817/2417771374","cluster_addr":"172.21.15.192:6819/2417771374","heartbeat_back_addr":"172.21.15.192:6823/2417771374","heartbeat_front_addr":"172.21.15.192:6821/2417771374","state":["exists","up"]},{"osd":9,"uuid":"71b4f5fb-1c02-461b-a257-c6e48381d768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6827","nonce":3979119431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6829","nonce":3979119431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6833","nonce":3979119431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6831","nonce":3979119431}]},"public_addr":"172.21.15.111:6827/3979119431","cluster_addr":"172.21.15.111:6829/3979119431","heartbeat_back_addr":"172.21.15.111:6833/3979119431","heartbeat_front_addr":"172.21.15.111:6831/3979119431","state":["exists","up"]},{"osd":10,"uuid":"e6a39e77-fce3-4e22-bf66-a677c9f8b6bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6825","nonce":2897902637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6827","nonce":2897902637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6831","nonce":2897902637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6829","nonce":2897902637}]},"public_addr":"172.21.15.121:6825/2897902637","cluster_addr":"172.21.15.121:6827/2897902637","heartbeat_back_addr":"172.21.15.121:6831/2897902637","heartbeat_front_addr":"172.21.15.121:6829/2897902637","state":["exists","up"]},{"osd":11,"uuid":"411a7921-6717-4cef-b8df-d9e252edcc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6825","nonce":2431464812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6827","nonce":2431464812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6831","nonce":2431464812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6829","nonce":2431464812}]},"public_addr":"172.21.15.192:6825/2431464812","cluster_addr":"172.21.15.192:6827/2431464812","heartbeat_back_addr":"172.21.15.192:6831/2431464812","heartbeat_front_addr":"172.21.15.192:6829/2431464812","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:38:23.442715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:38:49.501124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:39:14.759077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:39:45.121397+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:40:11.137612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:40:36.278567+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:07.027190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:32.439215+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:57.058702+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:42:28.741846+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:42:53.395442+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:43:18.265067+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:6801/2107775445":"2024-09-16T22:36:50.357551+0000","172.21.15.111:6800/2107775445":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/2394746540":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/4140020843":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/2114354263":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/1054173642":"2024-09-16T22:36:22.652875+0000","172.21.15.111:0/3408238004":"2024-09-16T22:36:22.652875+0000","172.21.15.111:0/2879044772":"2024-09-16T22:36:22.652875+0000","172.21.15.111:6801/3350425104":"2024-09-16T22:36:22.652875+0000","172.21.15.111:6800/3350425104":"2024-09-16T22:36:22.652875+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:46:14.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.769+0000 7f7d97e00640 1 -- 172.21.15.111:0/1236227286 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7d7c0a3240 msgr2=0x7f7d7c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:14.770 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.769+0000 7f7d97e00640 1 --2- 172.21.15.111:0/1236227286 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7d7c0a3240 0x7f7d7c0a5700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7d980ffde0 tx=0x7f7d8400b410 comp rx=0 tx=0).stop 2024-09-15T22:46:14.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.769+0000 7f7d97e00640 1 -- 172.21.15.111:0/1236227286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d981019c0 msgr2=0x7f7d981982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:14.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.769+0000 7f7d97e00640 1 --2- 172.21.15.111:0/1236227286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d981019c0 0x7f7d981982f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7d88009230 tx=0x7f7d88004270 comp rx=0 tx=0).stop 2024-09-15T22:46:14.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.769+0000 7f7d97e00640 1 -- 172.21.15.111:0/1236227286 shutdown_connections 2024-09-15T22:46:14.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.769+0000 7f7d97e00640 1 --2- 172.21.15.111:0/1236227286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d981079c0 0x7f7d9819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:14.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.769+0000 7f7d97e00640 1 --2- 172.21.15.111:0/1236227286 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7d7c0a3240 0x7f7d7c0a5700 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:14.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.769+0000 7f7d97e00640 1 --2- 172.21.15.111:0/1236227286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d98102360 0x7f7d98198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:14.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.769+0000 7f7d97e00640 1 --2- 172.21.15.111:0/1236227286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7d981019c0 0x7f7d981982f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:14.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.769+0000 7f7d97e00640 1 -- 172.21.15.111:0/1236227286 >> 172.21.15.111:0/1236227286 conn(0x7f7d980fd740 msgr2=0x7f7d980fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:14.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.769+0000 7f7d97e00640 1 -- 172.21.15.111:0/1236227286 shutdown_connections 2024-09-15T22:46:14.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:14.769+0000 7f7d97e00640 1 -- 172.21.15.111:0/1236227286 wait complete. 2024-09-15T22:46:15.472 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-15T22:44:18.436394+0000', 'modified': '2024-09-15T22:44:33.655164+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24545, 'mds_1': 24514, 'mds_2': 24503}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24485': {'gid': 24485, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.121:6835/3371796806', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 3371796806}, {'type': 'v1', 'addr': '172.21.15.121:6835', 'nonce': 3371796806}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.192:6835/4080914921', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6834', 'nonce': 4080914921}, {'type': 'v1', 'addr': '172.21.15.192:6835', 'nonce': 4080914921}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24496': {'gid': 24496, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.192:6837/3995501707', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6836', 'nonce': 3995501707}, {'type': 'v1', 'addr': '172.21.15.192:6837', 'nonce': 3995501707}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24503': {'gid': 24503, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.121:6837/1443772791', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6836', 'nonce': 1443772791}, {'type': 'v1', 'addr': '172.21.15.121:6837', 'nonce': 1443772791}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24514': {'gid': 24514, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.192:6839/800136261', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6838', 'nonce': 800136261}, {'type': 'v1', 'addr': '172.21.15.192:6839', 'nonce': 800136261}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24545': {'gid': 24545, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.121:6839/1797352097', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6838', 'nonce': 1797352097}, {'type': 'v1', 'addr': '172.21.15.121:6839', 'nonce': 1797352097}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24545, 'qdb_cluster': [24545, 24514, 24503]} 2024-09-15T22:46:15.472 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-15T22:46:15.472 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-15T22:46:15.827 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:15 smithi111 ceph-mon[30629]: pgmap v360: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:46:15.827 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:15 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/1236227286' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:46:15.858 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:46:16.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:15 smithi192 ceph-mon[35427]: pgmap v360: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:46:16.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:15 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/1236227286' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:46:16.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:15 smithi121 ceph-mon[34815]: pgmap v360: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:46:16.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:15 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/1236227286' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:46:16.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.394+0000 7f8abba00640 1 -- 172.21.15.111:0/4070518555 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ab4106f10 msgr2=0x7f8ab410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:16.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.394+0000 7f8abba00640 1 --2- 172.21.15.111:0/4070518555 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ab4106f10 0x7f8ab410ea90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009900 tx=0x7f8aac02a410 comp rx=0 tx=0).stop 2024-09-15T22:46:16.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.394+0000 7f8abba00640 1 -- 172.21.15.111:0/4070518555 shutdown_connections 2024-09-15T22:46:16.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.394+0000 7f8abba00640 1 --2- 172.21.15.111:0/4070518555 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ab4106f10 0x7f8ab410ea90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.394+0000 7f8abba00640 1 --2- 172.21.15.111:0/4070518555 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ab4102790 0x7f8ab41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.394+0000 7f8abba00640 1 --2- 172.21.15.111:0/4070518555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab4101de0 0x7f8ab41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.394+0000 7f8abba00640 1 -- 172.21.15.111:0/4070518555 >> 172.21.15.111:0/4070518555 conn(0x7f8ab40fd8c0 msgr2=0x7f8ab40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:16.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.394+0000 7f8abba00640 1 -- 172.21.15.111:0/4070518555 shutdown_connections 2024-09-15T22:46:16.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.394+0000 7f8abba00640 1 -- 172.21.15.111:0/4070518555 wait complete. 2024-09-15T22:46:16.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.395+0000 7f8abba00640 1 Processor -- start 2024-09-15T22:46:16.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.395+0000 7f8abba00640 1 -- start start 2024-09-15T22:46:16.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.395+0000 7f8abba00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab4101de0 0x7f8ab419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:16.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.395+0000 7f8abba00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ab4102790 0x7f8ab419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:16.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.395+0000 7f8abba00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ab4106f10 0x7f8ab419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:16.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.395+0000 7f8abba00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab4111fc0 con 0x7f8ab4106f10 2024-09-15T22:46:16.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.395+0000 7f8abba00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab4111e40 con 0x7f8ab4102790 2024-09-15T22:46:16.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.395+0000 7f8abba00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab4112140 con 0x7f8ab4101de0 2024-09-15T22:46:16.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.396+0000 7f8ab9000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab4101de0 0x7f8ab419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:16.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.396+0000 7f8ab9000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab4101de0 0x7f8ab419a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:36378/0 (socket says 172.21.15.111:36378) 2024-09-15T22:46:16.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.396+0000 7f8ab9000640 1 -- 172.21.15.111:0/2986755551 learned_addr learned my addr 172.21.15.111:0/2986755551 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:16.398 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.396+0000 7f8ab9a00640 1 --2- 172.21.15.111:0/2986755551 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ab4106f10 0x7f8ab419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:16.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.396+0000 7f8ab3e00640 1 --2- 172.21.15.111:0/2986755551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ab4102790 0x7f8ab419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:16.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.396+0000 7f8ab9000640 1 -- 172.21.15.111:0/2986755551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ab4102790 msgr2=0x7f8ab419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:16.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.396+0000 7f8ab9000640 1 --2- 172.21.15.111:0/2986755551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ab4102790 0x7f8ab419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.396+0000 7f8ab9000640 1 -- 172.21.15.111:0/2986755551 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ab4106f10 msgr2=0x7f8ab419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:16.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.396+0000 7f8ab9000640 1 --2- 172.21.15.111:0/2986755551 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ab4106f10 0x7f8ab419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.396+0000 7f8ab9000640 1 -- 172.21.15.111:0/2986755551 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab419f610 con 0x7f8ab4101de0 2024-09-15T22:46:16.399 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.396+0000 7f8ab9000640 1 --2- 172.21.15.111:0/2986755551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab4101de0 0x7f8ab419a660 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400eb90 tx=0x7f8aa400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:16.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.397+0000 7f8ab1600640 1 -- 172.21.15.111:0/2986755551 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4004d20 con 0x7f8ab4101de0 2024-09-15T22:46:16.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.397+0000 7f8ab1600640 1 -- 172.21.15.111:0/2986755551 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aa400ee90 con 0x7f8ab4101de0 2024-09-15T22:46:16.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.397+0000 7f8abba00640 1 -- 172.21.15.111:0/2986755551 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab419f900 con 0x7f8ab4101de0 2024-09-15T22:46:16.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.397+0000 7f8ab1600640 1 -- 172.21.15.111:0/2986755551 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4010830 con 0x7f8ab4101de0 2024-09-15T22:46:16.400 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.397+0000 7f8abba00640 1 -- 172.21.15.111:0/2986755551 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab41a71e0 con 0x7f8ab4101de0 2024-09-15T22:46:16.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.400+0000 7f8abe5ea740 1 Processor -- start 2024-09-15T22:46:16.401 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.400+0000 7f8ab1600640 1 -- 172.21.15.111:0/2986755551 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f8aa4005020 con 0x7f8ab4101de0 2024-09-15T22:46:16.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.400+0000 7f8abe5ea740 1 -- start start 2024-09-15T22:46:16.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.400+0000 7f8abe5ea740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a706d3d330 0x55a706d3d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:16.402 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.401+0000 7f8abe5ea740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a706d3dc50 0x55a706d47fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:16.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.401+0000 7f8abe5ea740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a706d48510 0x55a706d4a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:16.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.401+0000 7f8abe5ea740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a706ca30b0 con 0x55a706d3dc50 2024-09-15T22:46:16.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.401+0000 7f8abe5ea740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a706c4aab0 con 0x55a706d48510 2024-09-15T22:46:16.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.401+0000 7f8abe5ea740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a706c9f070 con 0x55a706d3d330 2024-09-15T22:46:16.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.401+0000 7f8ab9a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a706d3d330 0x55a706d3d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:16.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.401+0000 7f8ab9000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a706d3dc50 0x55a706d47fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:16.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.401+0000 7f8ab9a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a706d3d330 0x55a706d3d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:36388/0 (socket says 172.21.15.111:36388) 2024-09-15T22:46:16.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.401+0000 7f8ab9a00640 1 -- 172.21.15.111:0/722439905 learned_addr learned my addr 172.21.15.111:0/722439905 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:16.403 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.401+0000 7f8ab3e00640 1 --2- 172.21.15.111:0/722439905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a706d48510 0x55a706d4a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:16.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.401+0000 7f8ab9a00640 1 -- 172.21.15.111:0/722439905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a706d48510 msgr2=0x55a706d4a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:16.404 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.401+0000 7f8ab9a00640 1 --2- 172.21.15.111:0/722439905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a706d48510 0x55a706d4a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.401+0000 7f8ab9a00640 1 -- 172.21.15.111:0/722439905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a706d3dc50 msgr2=0x55a706d47fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:16.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.401+0000 7f8ab9a00640 1 --2- 172.21.15.111:0/722439905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a706d3dc50 0x55a706d47fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.401+0000 7f8ab9a00640 1 -- 172.21.15.111:0/722439905 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a706c574a0 con 0x55a706d3d330 2024-09-15T22:46:16.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.402+0000 7f8ab9a00640 1 --2- 172.21.15.111:0/722439905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a706d3d330 0x55a706d3d710 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8aac02aa00 tx=0x7f8aac005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f36efa242818863e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:16.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.402+0000 7f8ab1600640 1 --2- 172.21.15.111:0/2986755551 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8a980a3240 0x7f8a980a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:16.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.402+0000 7f8aaa000640 1 -- 172.21.15.111:0/722439905 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac040070 con 0x55a706d3d330 2024-09-15T22:46:16.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.402+0000 7f8ab1600640 1 -- 172.21.15.111:0/2986755551 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f8aa40cff50 con 0x7f8ab4101de0 2024-09-15T22:46:16.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.402+0000 7f8aaa000640 1 -- 172.21.15.111:0/722439905 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aac004400 con 0x55a706d3d330 2024-09-15T22:46:16.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.402+0000 7f8aaa000640 1 -- 172.21.15.111:0/722439905 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac035040 con 0x55a706d3d330 2024-09-15T22:46:16.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.402+0000 7f8ab9000640 1 --2- 172.21.15.111:0/2986755551 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8a980a3240 0x7f8a980a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:16.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.403+0000 7f8abe5ea740 1 -- 172.21.15.111:0/722439905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a706d3d330 msgr2=0x55a706d3d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:16.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.403+0000 7f8ab9000640 1 --2- 172.21.15.111:0/2986755551 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8a980a3240 0x7f8a980a5700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4098f40 tx=0x7f8aa40d3990 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:16.405 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.403+0000 7f8abe5ea740 1 --2- 172.21.15.111:0/722439905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a706d3d330 0x55a706d3d710 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8aac02aa00 tx=0x7f8aac005c00 comp rx=0 tx=0).stop 2024-09-15T22:46:16.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.403+0000 7f8abe5ea740 1 -- 172.21.15.111:0/722439905 shutdown_connections 2024-09-15T22:46:16.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.403+0000 7f8abe5ea740 1 --2- 172.21.15.111:0/722439905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a706d48510 0x55a706d4a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.403+0000 7f8abe5ea740 1 --2- 172.21.15.111:0/722439905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a706d3dc50 0x55a706d47fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.403+0000 7f8abe5ea740 1 --2- 172.21.15.111:0/722439905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a706d3d330 0x55a706d3d710 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.403+0000 7f8abe5ea740 1 -- 172.21.15.111:0/722439905 >> 172.21.15.111:0/722439905 conn(0x55a706d398c0 msgr2=0x55a706d39ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:16.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.403+0000 7f8abe5ea740 1 -- 172.21.15.111:0/722439905 shutdown_connections 2024-09-15T22:46:16.406 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.403+0000 7f8abe5ea740 1 -- 172.21.15.111:0/722439905 wait complete. 2024-09-15T22:46:16.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.404+0000 7f8abe5ea740 1 -- start start 2024-09-15T22:46:16.408 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.404+0000 7f8abe5ea740 1 Processor -- start 2024-09-15T22:46:16.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.404+0000 7f8abe5ea740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a706d3d330 0x55a706d455c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:16.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.404+0000 7f8abe5ea740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a706d3dc50 0x55a706d45b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:16.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.404+0000 7f8abe5ea740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a706d48510 0x55a706d46040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:16.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.404+0000 7f8abe5ea740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a706d4b060 con 0x55a706d48510 2024-09-15T22:46:16.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.404+0000 7f8abe5ea740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a706c2b920 con 0x55a706d3dc50 2024-09-15T22:46:16.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.404+0000 7f8abe5ea740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a706d4b1e0 con 0x55a706d3d330 2024-09-15T22:46:16.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.404+0000 7f8abe5ea740 10 client.0 _ll_register_callbacks cb 0x55a706c4f8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:16.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.405+0000 7f8ab3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a706d3dc50 0x55a706d45b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:16.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.405+0000 7f8ab9a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a706d3d330 0x55a706d455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:16.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.405+0000 7f8ab3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a706d3dc50 0x55a706d45b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:60962/0 (socket says 172.21.15.111:60962) 2024-09-15T22:46:16.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.405+0000 7f8ab9a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a706d3d330 0x55a706d455c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:36400/0 (socket says 172.21.15.111:36400) 2024-09-15T22:46:16.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.405+0000 7f8ab3e00640 1 -- 172.21.15.111:0/1873498349 learned_addr learned my addr 172.21.15.111:0/1873498349 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:16.409 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.405+0000 7f8ab9a00640 1 --2- 172.21.15.111:0/1873498349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a706d48510 0x55a706d46040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:16.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.405+0000 7f8ab3e00640 1 -- 172.21.15.111:0/1873498349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a706d3d330 msgr2=0x55a706d455c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:16.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.405+0000 7f8ab3e00640 1 --2- 172.21.15.111:0/1873498349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a706d3d330 0x55a706d455c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.405+0000 7f8ab3e00640 1 -- 172.21.15.111:0/1873498349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a706d48510 msgr2=0x55a706d46040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:16.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.405+0000 7f8ab3e00640 1 --2- 172.21.15.111:0/1873498349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a706d48510 0x55a706d46040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.405+0000 7f8ab3e00640 1 -- 172.21.15.111:0/1873498349 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a706a142b0 con 0x55a706d3dc50 2024-09-15T22:46:16.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.405+0000 7f8ab9a00640 1 --2- 172.21.15.111:0/1873498349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a706d48510 0x55a706d46040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:46:16.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.405+0000 7f8ab3e00640 1 --2- 172.21.15.111:0/1873498349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a706d3dc50 0x55a706d45b00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000bf00 tx=0x7f8aa0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6135c42e9d2b281 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:16.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.405+0000 7f8a9ea00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:46:16.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.405+0000 7f8a9ea00640 1 -- 172.21.15.111:0/1873498349 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0004020 con 0x55a706d3dc50 2024-09-15T22:46:16.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.406+0000 7f8abe5ea740 1 -- 172.21.15.111:0/1873498349 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8aac004400 con 0x55a706d3dc50 2024-09-15T22:46:16.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.406+0000 7f8a9ea00640 1 -- 172.21.15.111:0/1873498349 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f8aa00041c0 con 0x55a706d3dc50 2024-09-15T22:46:16.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.406+0000 7f8a9ea00640 1 -- 172.21.15.111:0/1873498349 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8aa00043c0 con 0x55a706d3dc50 2024-09-15T22:46:16.410 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.406+0000 7f8abe5ea740 10 client.25004 fetch_fsmap learned FSMap version 25 2024-09-15T22:46:16.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.406+0000 7f8abe5ea740 1 -- 172.21.15.111:0/1873498349 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a706d56eb0 con 0x55a706d3dc50 2024-09-15T22:46:16.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.406+0000 7f8a9ea00640 1 -- 172.21.15.111:0/1873498349 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aac004400 con 0x55a706d3dc50 2024-09-15T22:46:16.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.406+0000 7f8a9ea00640 1 -- 172.21.15.111:0/1873498349 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8aa0023020 con 0x55a706d3dc50 2024-09-15T22:46:16.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.406+0000 7f8abe5ea740 10 client.25004 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:46:16.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.406+0000 7f8abe5ea740 10 client.25004 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-15T22:46:16.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.406+0000 7f8abe5ea740 20 client.25004 populate_metadata read hostname 'smithi111' 2024-09-15T22:46:16.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.406+0000 7f8abe5ea740 1 --2- 172.21.15.111:0/1873498349 >> [v2:172.21.15.121:6836/1443772791,v1:172.21.15.121:6837/1443772791] conn(0x55a706d594b0 0x55a706d5a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:16.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.407+0000 7f8abe5ea740 4 client.25004 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:46:16.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.407+0000 7f8abe5ea740 1 -- 172.21.15.111:0/1873498349 --> [v2:172.21.15.121:6836/1443772791,v1:172.21.15.121:6837/1443772791] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a706cb0620 con 0x55a706d594b0 2024-09-15T22:46:16.411 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.407+0000 7f8ab9a00640 1 --2- 172.21.15.111:0/1873498349 >> [v2:172.21.15.121:6836/1443772791,v1:172.21.15.121:6837/1443772791] conn(0x55a706d594b0 0x55a706d5a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:16.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.407+0000 7f8ab9a00640 1 --2- 172.21.15.111:0/1873498349 >> [v2:172.21.15.121:6836/1443772791,v1:172.21.15.121:6837/1443772791] conn(0x55a706d594b0 0x55a706d5a920 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c01176a074e5dcb server_cookie=da0c0f333346edf in_seq=0 out_seq=0 2024-09-15T22:46:16.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.407+0000 7f8a9ea00640 10 client.25004 ms_handle_connect on v2:172.21.15.121:6836/1443772791 2024-09-15T22:46:16.412 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.408+0000 7f8a9ea00640 1 -- 172.21.15.111:0/1873498349 <== mds.2 v2:172.21.15.121:6836/1443772791 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a706cb0620 con 0x55a706d594b0 2024-09-15T22:46:16.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.408+0000 7f8a9ea00640 10 client.25004 handle_command_reply: tid=0 2024-09-15T22:46:16.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.408+0000 7f8abe5ea740 1 client.25004 shutdown 2024-09-15T22:46:16.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.408+0000 7f8abe5ea740 1 -- 172.21.15.111:0/1873498349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a706d3dc50 msgr2=0x55a706d45b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:16.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.408+0000 7f8abe5ea740 1 --2- 172.21.15.111:0/1873498349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a706d3dc50 0x55a706d45b00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000bf00 tx=0x7f8aa0011810 comp rx=0 tx=0).stop 2024-09-15T22:46:16.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.408+0000 7f8abe5ea740 1 -- 172.21.15.111:0/1873498349 shutdown_connections 2024-09-15T22:46:16.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.408+0000 7f8abe5ea740 1 --2- 172.21.15.111:0/1873498349 >> [v2:172.21.15.121:6836/1443772791,v1:172.21.15.121:6837/1443772791] conn(0x55a706d594b0 0x55a706d5a920 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.408+0000 7f8abe5ea740 1 --2- 172.21.15.111:0/1873498349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a706d48510 0x55a706d46040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.409+0000 7f8abe5ea740 1 --2- 172.21.15.111:0/1873498349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a706d3dc50 0x55a706d45b00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.409+0000 7f8abe5ea740 1 --2- 172.21.15.111:0/1873498349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a706d3d330 0x55a706d455c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.409+0000 7f8a9ea00640 0 client.25004 ms_handle_reset on v2:172.21.15.121:6836/1443772791 2024-09-15T22:46:16.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.409+0000 7f8abe5ea740 1 -- 172.21.15.111:0/1873498349 >> 172.21.15.111:0/1873498349 conn(0x55a706d398c0 msgr2=0x55a706d3f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:16.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.409+0000 7f8abe5ea740 1 -- 172.21.15.111:0/1873498349 shutdown_connections 2024-09-15T22:46:16.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.409+0000 7f8abe5ea740 1 -- 172.21.15.111:0/1873498349 wait complete. 2024-09-15T22:46:16.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.409+0000 7f8abe5ea740 20 client.25004 trim_cache size 0 max 16384 2024-09-15T22:46:16.428 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.427+0000 7f8abe5ea740 1 Processor -- start 2024-09-15T22:46:16.428 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.427+0000 7f8abe5ea740 1 -- start start 2024-09-15T22:46:16.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.427+0000 7f8abe5ea740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a706d3d330 0x55a706d45e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:16.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.427+0000 7f8abe5ea740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a706d3dc50 0x55a706d42860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:16.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.427+0000 7f8abe5ea740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a706d48510 0x55a706d42da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:16.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.427+0000 7f8abe5ea740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a706d58c60 con 0x55a706d48510 2024-09-15T22:46:16.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.427+0000 7f8abe5ea740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a706d59330 con 0x55a706d3d330 2024-09-15T22:46:16.429 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.427+0000 7f8abe5ea740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a706d59a00 con 0x55a706d3dc50 2024-09-15T22:46:16.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.427+0000 7f8ab9a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a706d48510 0x55a706d42da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:16.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.427+0000 7f8ab9a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a706d3d330 0x55a706d45e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:16.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.427+0000 7f8ab9a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a706d3d330 0x55a706d45e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:60964/0 (socket says 172.21.15.111:60964) 2024-09-15T22:46:16.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.427+0000 7f8ab9a00640 1 -- 172.21.15.111:0/2923137222 learned_addr learned my addr 172.21.15.111:0/2923137222 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:16.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.427+0000 7f8ab3e00640 1 --2- 172.21.15.111:0/2923137222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a706d3dc50 0x55a706d42860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:16.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.428+0000 7f8ab9a00640 1 -- 172.21.15.111:0/2923137222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a706d3dc50 msgr2=0x55a706d42860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:16.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.428+0000 7f8ab9a00640 1 --2- 172.21.15.111:0/2923137222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a706d3dc50 0x55a706d42860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.428+0000 7f8ab9a00640 1 -- 172.21.15.111:0/2923137222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a706d48510 msgr2=0x55a706d42da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:16.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.428+0000 7f8ab9a00640 1 --2- 172.21.15.111:0/2923137222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a706d48510 0x55a706d42da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.428+0000 7f8ab9a00640 1 -- 172.21.15.111:0/2923137222 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a706d48310 con 0x55a706d3d330 2024-09-15T22:46:16.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.428+0000 7f8ab9a00640 1 --2- 172.21.15.111:0/2923137222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a706d3d330 0x55a706d45e90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8aac031520 tx=0x7f8aac0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35290f1260958cad server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:16.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.428+0000 7f8aa9600640 1 -- 172.21.15.111:0/2923137222 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac035040 con 0x55a706d3d330 2024-09-15T22:46:16.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.428+0000 7f8aa9600640 1 -- 172.21.15.111:0/2923137222 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aac032e70 con 0x55a706d3d330 2024-09-15T22:46:16.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.428+0000 7f8aa9600640 1 -- 172.21.15.111:0/2923137222 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac040040 con 0x55a706d3d330 2024-09-15T22:46:16.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.429+0000 7f8abe5ea740 1 -- 172.21.15.111:0/2923137222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a706d3d330 msgr2=0x55a706d45e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:16.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.429+0000 7f8abe5ea740 1 --2- 172.21.15.111:0/2923137222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a706d3d330 0x55a706d45e90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8aac031520 tx=0x7f8aac0046c0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.429+0000 7f8abe5ea740 1 -- 172.21.15.111:0/2923137222 shutdown_connections 2024-09-15T22:46:16.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.429+0000 7f8abe5ea740 1 --2- 172.21.15.111:0/2923137222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a706d48510 0x55a706d42da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.429+0000 7f8abe5ea740 1 --2- 172.21.15.111:0/2923137222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a706d3dc50 0x55a706d42860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.429+0000 7f8abe5ea740 1 --2- 172.21.15.111:0/2923137222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a706d3d330 0x55a706d45e90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.429+0000 7f8abe5ea740 1 -- 172.21.15.111:0/2923137222 >> 172.21.15.111:0/2923137222 conn(0x55a706d398c0 msgr2=0x55a706d39ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:16.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.429+0000 7f8abe5ea740 1 -- 172.21.15.111:0/2923137222 shutdown_connections 2024-09-15T22:46:16.432 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.429+0000 7f8abe5ea740 1 -- 172.21.15.111:0/2923137222 wait complete. 2024-09-15T22:46:16.434 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.429+0000 7f8abe5ea740 1 -- start start 2024-09-15T22:46:16.435 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.430+0000 7f8abe5ea740 1 Processor -- start 2024-09-15T22:46:16.435 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.430+0000 7f8abe5ea740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a706d3d330 0x55a706d41fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:16.435 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.430+0000 7f8abe5ea740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a706d3dc50 0x55a706d424f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:16.435 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.430+0000 7f8abe5ea740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a706d48510 0x55a706d61be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:16.435 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.430+0000 7f8abe5ea740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a706d5a7a0 con 0x55a706d3dc50 2024-09-15T22:46:16.435 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.430+0000 7f8abe5ea740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a706d5a0d0 con 0x55a706d48510 2024-09-15T22:46:16.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.430+0000 7f8abe5ea740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a706d5ae70 con 0x55a706d3d330 2024-09-15T22:46:16.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.430+0000 7f8abe5ea740 10 client.0 _ll_register_callbacks cb 0x7f8aa000b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:16.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.430+0000 7f8ab9a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a706d48510 0x55a706d61be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:16.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.430+0000 7f8ab9a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a706d48510 0x55a706d61be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:60970/0 (socket says 172.21.15.111:60970) 2024-09-15T22:46:16.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.430+0000 7f8ab9a00640 1 -- 172.21.15.111:0/3064852343 learned_addr learned my addr 172.21.15.111:0/3064852343 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:16.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.430+0000 7f8ab3e00640 1 --2- 172.21.15.111:0/3064852343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a706d3dc50 0x55a706d424f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:16.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.430+0000 7f8ab9a00640 1 --2- 172.21.15.111:0/3064852343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a706d3d330 0x55a706d41fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:16.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.431+0000 7f8ab9a00640 1 -- 172.21.15.111:0/3064852343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a706d48510 msgr2=0x55a706d61be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:16.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.431+0000 7f8ab9a00640 1 --2- 172.21.15.111:0/3064852343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a706d48510 0x55a706d61be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.431+0000 7f8ab9a00640 1 -- 172.21.15.111:0/3064852343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a706d3dc50 msgr2=0x55a706d424f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:16.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.431+0000 7f8ab9a00640 1 --2- 172.21.15.111:0/3064852343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a706d3dc50 0x55a706d424f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.436 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.431+0000 7f8ab9a00640 1 -- 172.21.15.111:0/3064852343 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a706d621a0 con 0x55a706d3d330 2024-09-15T22:46:16.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.431+0000 7f8ab3e00640 1 --2- 172.21.15.111:0/3064852343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a706d3dc50 0x55a706d424f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:46:16.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.431+0000 7f8ab9a00640 1 --2- 172.21.15.111:0/3064852343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a706d3d330 0x55a706d41fb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009900 tx=0x7f8aac004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4229c49c7fdd15b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:16.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.431+0000 7f8a9ea00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:46:16.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.432+0000 7f8a9ea00640 1 -- 172.21.15.111:0/3064852343 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac058030 con 0x55a706d3d330 2024-09-15T22:46:16.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.432+0000 7f8abe5ea740 1 -- 172.21.15.111:0/3064852343 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a706caab30 con 0x55a706d3d330 2024-09-15T22:46:16.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.432+0000 7f8a9ea00640 1 -- 172.21.15.111:0/3064852343 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f8aac031c90 con 0x55a706d3d330 2024-09-15T22:46:16.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.432+0000 7f8a9ea00640 1 -- 172.21.15.111:0/3064852343 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8aac040070 con 0x55a706d3d330 2024-09-15T22:46:16.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.432+0000 7f8abe5ea740 10 client.25006 fetch_fsmap learned FSMap version 25 2024-09-15T22:46:16.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.432+0000 7f8abe5ea740 1 -- 172.21.15.111:0/3064852343 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a706d634a0 con 0x55a706d3d330 2024-09-15T22:46:16.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.432+0000 7f8a9ea00640 1 -- 172.21.15.111:0/3064852343 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a706caab30 con 0x55a706d3d330 2024-09-15T22:46:16.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.432+0000 7f8a9ea00640 1 -- 172.21.15.111:0/3064852343 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f8aac032370 con 0x55a706d3d330 2024-09-15T22:46:16.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.432+0000 7f8abe5ea740 10 client.25006 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:46:16.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.432+0000 7f8abe5ea740 10 client.25006 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-15T22:46:16.437 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.432+0000 7f8abe5ea740 20 client.25006 populate_metadata read hostname 'smithi111' 2024-09-15T22:46:16.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.432+0000 7f8abe5ea740 1 --2- 172.21.15.111:0/3064852343 >> [v2:172.21.15.121:6836/1443772791,v1:172.21.15.121:6837/1443772791] conn(0x55a706dc7bb0 0x55a706dc68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:16.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.432+0000 7f8abe5ea740 4 client.25006 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:46:16.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.432+0000 7f8abe5ea740 1 -- 172.21.15.111:0/3064852343 --> [v2:172.21.15.121:6836/1443772791,v1:172.21.15.121:6837/1443772791] -- command(tid 0: {"prefix": "status"}) -- 0x55a706c26f80 con 0x55a706dc7bb0 2024-09-15T22:46:16.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.433+0000 7f8ab3e00640 1 --2- 172.21.15.111:0/3064852343 >> [v2:172.21.15.121:6836/1443772791,v1:172.21.15.121:6837/1443772791] conn(0x55a706dc7bb0 0x55a706dc68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:16.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.433+0000 7f8ab3e00640 1 --2- 172.21.15.111:0/3064852343 >> [v2:172.21.15.121:6836/1443772791,v1:172.21.15.121:6837/1443772791] conn(0x55a706dc7bb0 0x55a706dc68c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c140ddf60e5861d4 server_cookie=1743c758689a4299 in_seq=0 out_seq=0 2024-09-15T22:46:16.438 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.433+0000 7f8a9ea00640 10 client.25006 ms_handle_connect on v2:172.21.15.121:6836/1443772791 2024-09-15T22:46:16.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.433+0000 7f8a9ea00640 1 -- 172.21.15.111:0/3064852343 <== mds.2 v2:172.21.15.121:6836/1443772791 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a706c26f80 con 0x55a706dc7bb0 2024-09-15T22:46:16.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.433+0000 7f8a9ea00640 10 client.25006 handle_command_reply: tid=0 2024-09-15T22:46:16.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.433+0000 7f8abe5ea740 1 client.25006 shutdown 2024-09-15T22:46:16.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.434+0000 7f8abe5ea740 1 -- 172.21.15.111:0/3064852343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a706d3d330 msgr2=0x55a706d41fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:16.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.434+0000 7f8abe5ea740 1 --2- 172.21.15.111:0/3064852343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a706d3d330 0x55a706d41fb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009900 tx=0x7f8aac004610 comp rx=0 tx=0).stop 2024-09-15T22:46:16.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.434+0000 7f8abe5ea740 1 -- 172.21.15.111:0/3064852343 shutdown_connections 2024-09-15T22:46:16.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.434+0000 7f8abe5ea740 1 --2- 172.21.15.111:0/3064852343 >> [v2:172.21.15.121:6836/1443772791,v1:172.21.15.121:6837/1443772791] conn(0x55a706dc7bb0 0x55a706dc68c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.439 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.434+0000 7f8abe5ea740 1 --2- 172.21.15.111:0/3064852343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a706d48510 0x55a706d61be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.434+0000 7f8abe5ea740 1 --2- 172.21.15.111:0/3064852343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a706d3dc50 0x55a706d424f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.434+0000 7f8abe5ea740 1 --2- 172.21.15.111:0/3064852343 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a706d3d330 0x55a706d41fb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.434+0000 7f8a9ea00640 0 client.25006 ms_handle_reset on v2:172.21.15.121:6836/1443772791 2024-09-15T22:46:16.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.434+0000 7f8abe5ea740 1 -- 172.21.15.111:0/3064852343 >> 172.21.15.111:0/3064852343 conn(0x55a706d398c0 msgr2=0x55a706d43eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:16.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.434+0000 7f8abe5ea740 1 -- 172.21.15.111:0/3064852343 shutdown_connections 2024-09-15T22:46:16.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.434+0000 7f8abe5ea740 1 -- 172.21.15.111:0/3064852343 wait complete. 2024-09-15T22:46:16.440 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.434+0000 7f8abe5ea740 20 client.25006 trim_cache size 0 max 16384 2024-09-15T22:46:16.440 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-15T22:46:16.440 INFO:teuthology.orchestra.run.smithi111.stdout: "cluster_fsid": "b7614d3c-73b2-11ef-bceb-c7b262605968", 2024-09-15T22:46:16.440 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": 2, 2024-09-15T22:46:16.441 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 24503, 2024-09-15T22:46:16.441 INFO:teuthology.orchestra.run.smithi111.stdout: "want_state": "up:active", 2024-09-15T22:46:16.441 INFO:teuthology.orchestra.run.smithi111.stdout: "state": "up:active", 2024-09-15T22:46:16.441 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs", 2024-09-15T22:46:16.441 INFO:teuthology.orchestra.run.smithi111.stdout: "rank_uptime": 106.89090396500001, 2024-09-15T22:46:16.441 INFO:teuthology.orchestra.run.smithi111.stdout: "mdsmap_epoch": 25, 2024-09-15T22:46:16.441 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch": 88, 2024-09-15T22:46:16.441 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch_barrier": 86, 2024-09-15T22:46:16.441 INFO:teuthology.orchestra.run.smithi111.stdout: "uptime": 144.48899950500001 2024-09-15T22:46:16.441 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-15T22:46:16.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.435+0000 7f8abba00640 1 -- 172.21.15.111:0/2986755551 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8a980a3240 msgr2=0x7f8a980a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:16.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.435+0000 7f8abba00640 1 --2- 172.21.15.111:0/2986755551 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8a980a3240 0x7f8a980a5700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4098f40 tx=0x7f8aa40d3990 comp rx=0 tx=0).stop 2024-09-15T22:46:16.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.435+0000 7f8abba00640 1 -- 172.21.15.111:0/2986755551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab4101de0 msgr2=0x7f8ab419a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:16.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.435+0000 7f8abba00640 1 --2- 172.21.15.111:0/2986755551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab4101de0 0x7f8ab419a660 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400eb90 tx=0x7f8aa400c6a0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.435+0000 7f8abba00640 1 -- 172.21.15.111:0/2986755551 shutdown_connections 2024-09-15T22:46:16.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.435+0000 7f8abba00640 1 --2- 172.21.15.111:0/2986755551 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ab4106f10 0x7f8ab419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.435+0000 7f8abba00640 1 --2- 172.21.15.111:0/2986755551 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8a980a3240 0x7f8a980a5700 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.435+0000 7f8abba00640 1 --2- 172.21.15.111:0/2986755551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ab4102790 0x7f8ab419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.442 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.435+0000 7f8abba00640 1 --2- 172.21.15.111:0/2986755551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab4101de0 0x7f8ab419a660 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:16.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.435+0000 7f8abba00640 1 -- 172.21.15.111:0/2986755551 >> 172.21.15.111:0/2986755551 conn(0x7f8ab40fd8c0 msgr2=0x7f8ab40fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:16.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.435+0000 7f8abba00640 1 -- 172.21.15.111:0/2986755551 shutdown_connections 2024-09-15T22:46:16.443 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:16.435+0000 7f8abba00640 1 -- 172.21.15.111:0/2986755551 wait complete. 2024-09-15T22:46:17.271 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-15T22:46:17.638 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:46:17.949 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:17 smithi111 ceph-mon[30629]: pgmap v361: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:46:18.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:17 smithi192 ceph-mon[35427]: pgmap v361: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:46:18.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:17 smithi121 ceph-mon[34815]: pgmap v361: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:46:18.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.205+0000 7f9ed5e00640 1 -- 172.21.15.111:0/2917360652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ed0102360 msgr2=0x7f9ed010e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:18.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.205+0000 7f9ed5e00640 1 --2- 172.21.15.111:0/2917360652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ed0102360 0x7f9ed010e890 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8009930 tx=0x7f9ec802a420 comp rx=0 tx=0).stop 2024-09-15T22:46:18.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.205+0000 7f9ed5e00640 1 -- 172.21.15.111:0/2917360652 shutdown_connections 2024-09-15T22:46:18.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.205+0000 7f9ed5e00640 1 --2- 172.21.15.111:0/2917360652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ed0102360 0x7f9ed010e890 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.205+0000 7f9ed5e00640 1 --2- 172.21.15.111:0/2917360652 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ed01019c0 0x7f9ed0101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.205+0000 7f9ed5e00640 1 --2- 172.21.15.111:0/2917360652 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ed01079c0 0x7f9ed0107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.205+0000 7f9ed5e00640 1 -- 172.21.15.111:0/2917360652 >> 172.21.15.111:0/2917360652 conn(0x7f9ed00fd740 msgr2=0x7f9ed00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:18.207 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.205+0000 7f9ed5e00640 1 -- 172.21.15.111:0/2917360652 shutdown_connections 2024-09-15T22:46:18.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.205+0000 7f9ed5e00640 1 -- 172.21.15.111:0/2917360652 wait complete. 2024-09-15T22:46:18.208 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.206+0000 7f9ed5e00640 1 Processor -- start 2024-09-15T22:46:18.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.206+0000 7f9ed5e00640 1 -- start start 2024-09-15T22:46:18.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.207+0000 7f9ed5e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ed01019c0 0x7f9ed019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:18.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.207+0000 7f9ed5e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ed0102360 0x7f9ed019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:18.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.207+0000 7f9ed5e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ed01079c0 0x7f9ed019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:18.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.207+0000 7f9ed5e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed0111dc0 con 0x7f9ed01079c0 2024-09-15T22:46:18.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.207+0000 7f9ed5e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed0111c40 con 0x7f9ed01019c0 2024-09-15T22:46:18.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.207+0000 7f9ed5e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed0111f40 con 0x7f9ed0102360 2024-09-15T22:46:18.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.207+0000 7f9ed5400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ed01079c0 0x7f9ed019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:18.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.207+0000 7f9ed5400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ed01079c0 0x7f9ed019ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44518/0 (socket says 172.21.15.111:44518) 2024-09-15T22:46:18.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.207+0000 7f9ed5400640 1 -- 172.21.15.111:0/315156635 learned_addr learned my addr 172.21.15.111:0/315156635 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:18.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.207+0000 7f9ed4a00640 1 --2- 172.21.15.111:0/315156635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ed01019c0 0x7f9ed019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:18.209 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.207+0000 7f9ecfe00640 1 --2- 172.21.15.111:0/315156635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ed0102360 0x7f9ed019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:18.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.207+0000 7f9ed4a00640 1 -- 172.21.15.111:0/315156635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ed0102360 msgr2=0x7f9ed019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:18.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.207+0000 7f9ed4a00640 1 --2- 172.21.15.111:0/315156635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ed0102360 0x7f9ed019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.210 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.207+0000 7f9ed4a00640 1 -- 172.21.15.111:0/315156635 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ed01079c0 msgr2=0x7f9ed019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:18.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.207+0000 7f9ed4a00640 1 --2- 172.21.15.111:0/315156635 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ed01079c0 0x7f9ed019ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.207+0000 7f9ed4a00640 1 -- 172.21.15.111:0/315156635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed019f440 con 0x7f9ed01019c0 2024-09-15T22:46:18.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.207+0000 7f9ed5400640 1 --2- 172.21.15.111:0/315156635 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ed01079c0 0x7f9ed019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:46:18.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.208+0000 7f9ed4a00640 1 --2- 172.21.15.111:0/315156635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ed01019c0 0x7f9ed019a490 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400eb00 tx=0x7f9ec400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:18.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.208+0000 7f9ecd600640 1 -- 172.21.15.111:0/315156635 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4010430 con 0x7f9ed01019c0 2024-09-15T22:46:18.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.208+0000 7f9ed5e00640 1 -- 172.21.15.111:0/315156635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ed019f730 con 0x7f9ed01019c0 2024-09-15T22:46:18.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.208+0000 7f9ecd600640 1 -- 172.21.15.111:0/315156635 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ec4010ab0 con 0x7f9ed01019c0 2024-09-15T22:46:18.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.208+0000 7f9ecd600640 1 -- 172.21.15.111:0/315156635 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4017bb0 con 0x7f9ed01019c0 2024-09-15T22:46:18.211 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.208+0000 7f9ed5e00640 1 -- 172.21.15.111:0/315156635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ed01a7010 con 0x7f9ed01019c0 2024-09-15T22:46:18.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.211+0000 7f9ed9e82740 1 Processor -- start 2024-09-15T22:46:18.212 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.211+0000 7f9ed9e82740 1 -- start start 2024-09-15T22:46:18.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.211+0000 7f9ecd600640 1 -- 172.21.15.111:0/315156635 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f9ec4010c60 con 0x7f9ed01019c0 2024-09-15T22:46:18.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.211+0000 7f9ed9e82740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1865a64e0 0x55e1865a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:18.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.211+0000 7f9ed9e82740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1865a6e00 0x55e1865b1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:18.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.211+0000 7f9ed9e82740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e1865b16c0 0x55e1865b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:18.213 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.211+0000 7f9ed9e82740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e18650c0b0 con 0x55e1865a6e00 2024-09-15T22:46:18.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.211+0000 7f9ed9e82740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e1864b3ab0 con 0x55e1865a64e0 2024-09-15T22:46:18.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.211+0000 7f9ed9e82740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55e186508070 con 0x55e1865b16c0 2024-09-15T22:46:18.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.213+0000 7f9ed5400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1865a64e0 0x55e1865a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:18.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.213+0000 7f9ed4a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1865a6e00 0x55e1865b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:18.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.213+0000 7f9ed5400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1865a64e0 0x55e1865a68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:60994/0 (socket says 172.21.15.111:60994) 2024-09-15T22:46:18.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.213+0000 7f9ed5400640 1 -- 172.21.15.111:0/3756798332 learned_addr learned my addr 172.21.15.111:0/3756798332 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:18.214 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.213+0000 7f9ecfe00640 1 --2- 172.21.15.111:0/3756798332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e1865b16c0 0x55e1865b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:18.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.213+0000 7f9ed5400640 1 -- 172.21.15.111:0/3756798332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e1865b16c0 msgr2=0x55e1865b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:18.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.213+0000 7f9ed5400640 1 --2- 172.21.15.111:0/3756798332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e1865b16c0 0x55e1865b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.213+0000 7f9ed5400640 1 -- 172.21.15.111:0/3756798332 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1865a6e00 msgr2=0x55e1865b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:18.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.213+0000 7f9ed5400640 1 --2- 172.21.15.111:0/3756798332 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1865a6e00 0x55e1865b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.213+0000 7f9ed5400640 1 -- 172.21.15.111:0/3756798332 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1864c04a0 con 0x55e1865a64e0 2024-09-15T22:46:18.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.213+0000 7f9ed5400640 1 --2- 172.21.15.111:0/3756798332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1865a64e0 0x55e1865a68c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8005c00 tx=0x7f9ec8005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e991bb81971739a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:18.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.213+0000 7f9ec1c00640 1 -- 172.21.15.111:0/3756798332 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8035040 con 0x55e1865a64e0 2024-09-15T22:46:18.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.213+0000 7f9ec1c00640 1 -- 172.21.15.111:0/3756798332 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ec80318f0 con 0x55e1865a64e0 2024-09-15T22:46:18.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.213+0000 7f9ec1c00640 1 -- 172.21.15.111:0/3756798332 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec803c440 con 0x55e1865a64e0 2024-09-15T22:46:18.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.213+0000 7f9ecd600640 1 --2- 172.21.15.111:0/315156635 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9eb80a3240 0x7f9eb80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:18.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.213+0000 7f9ed9e82740 1 -- 172.21.15.111:0/3756798332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1865a64e0 msgr2=0x55e1865a68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:18.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.213+0000 7f9ed9e82740 1 --2- 172.21.15.111:0/3756798332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1865a64e0 0x55e1865a68c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8005c00 tx=0x7f9ec8005d50 comp rx=0 tx=0).stop 2024-09-15T22:46:18.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.214+0000 7f9ed4a00640 1 --2- 172.21.15.111:0/315156635 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9eb80a3240 0x7f9eb80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:18.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.214+0000 7f9ecd600640 1 -- 172.21.15.111:0/315156635 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f9ec40d0180 con 0x7f9ed01019c0 2024-09-15T22:46:18.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.214+0000 7f9ed9e82740 1 -- 172.21.15.111:0/3756798332 shutdown_connections 2024-09-15T22:46:18.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.214+0000 7f9ed9e82740 1 --2- 172.21.15.111:0/3756798332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e1865b16c0 0x55e1865b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.216 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.214+0000 7f9ed9e82740 1 --2- 172.21.15.111:0/3756798332 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1865a6e00 0x55e1865b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.214+0000 7f9ed9e82740 1 --2- 172.21.15.111:0/3756798332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1865a64e0 0x55e1865a68c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.214+0000 7f9ed9e82740 1 -- 172.21.15.111:0/3756798332 >> 172.21.15.111:0/3756798332 conn(0x55e1865a2a70 msgr2=0x55e1865a2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:18.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.214+0000 7f9ed9e82740 1 -- 172.21.15.111:0/3756798332 shutdown_connections 2024-09-15T22:46:18.217 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.214+0000 7f9ed9e82740 1 -- 172.21.15.111:0/3756798332 wait complete. 2024-09-15T22:46:18.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.214+0000 7f9ed4a00640 1 --2- 172.21.15.111:0/315156635 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9eb80a3240 0x7f9eb80a5700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9ec40a4f60 tx=0x7f9ec40d3c70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:18.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.214+0000 7f9ed9e82740 1 -- start start 2024-09-15T22:46:18.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.215+0000 7f9ed9e82740 1 Processor -- start 2024-09-15T22:46:18.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.215+0000 7f9ed9e82740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e1865a64e0 0x55e1865ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:18.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.215+0000 7f9ed9e82740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1865a6e00 0x55e1865aea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:18.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.215+0000 7f9ed9e82740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1865b16c0 0x55e1865aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:18.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.215+0000 7f9ed9e82740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e1865b4330 con 0x55e1865a6e00 2024-09-15T22:46:18.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.215+0000 7f9ed9e82740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e186494920 con 0x55e1865b16c0 2024-09-15T22:46:18.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.215+0000 7f9ed9e82740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55e1865b44b0 con 0x55e1865a64e0 2024-09-15T22:46:18.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.215+0000 7f9ed9e82740 10 client.0 _ll_register_callbacks cb 0x55e1864b88a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:18.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.215+0000 7f9ecfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1865a6e00 0x55e1865aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:18.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.215+0000 7f9ed5400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e1865a64e0 0x55e1865ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:18.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.215+0000 7f9ecfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1865a6e00 0x55e1865aea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44542/0 (socket says 172.21.15.111:44542) 2024-09-15T22:46:18.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.215+0000 7f9ecfe00640 1 -- 172.21.15.111:0/2255291888 learned_addr learned my addr 172.21.15.111:0/2255291888 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:18.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.216+0000 7f9ed5400640 1 --2- 172.21.15.111:0/2255291888 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1865b16c0 0x55e1865aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:18.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.216+0000 7f9ecfe00640 1 -- 172.21.15.111:0/2255291888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e1865a64e0 msgr2=0x55e1865ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:18.220 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.216+0000 7f9ecfe00640 1 --2- 172.21.15.111:0/2255291888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e1865a64e0 0x55e1865ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.216+0000 7f9ecfe00640 1 -- 172.21.15.111:0/2255291888 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1865b16c0 msgr2=0x55e1865aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:18.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.216+0000 7f9ecfe00640 1 --2- 172.21.15.111:0/2255291888 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1865b16c0 0x55e1865aefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.216+0000 7f9ecfe00640 1 -- 172.21.15.111:0/2255291888 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e18627d2b0 con 0x55e1865a6e00 2024-09-15T22:46:18.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.216+0000 7f9ed5400640 1 --2- 172.21.15.111:0/2255291888 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1865b16c0 0x55e1865aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:46:18.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.216+0000 7f9ecfe00640 1 --2- 172.21.15.111:0/2255291888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1865a6e00 0x55e1865aea80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc012710 tx=0x7f9ebc012bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f441d5524b34ae08 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:18.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.216+0000 7f9eb6000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:46:18.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.216+0000 7f9eb6000640 1 -- 172.21.15.111:0/2255291888 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc00f920 con 0x55e1865a6e00 2024-09-15T22:46:18.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.217+0000 7f9ed9e82740 1 -- 172.21.15.111:0/2255291888 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ec80318f0 con 0x55e1865a6e00 2024-09-15T22:46:18.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.217+0000 7f9eb6000640 1 -- 172.21.15.111:0/2255291888 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f9ebc00fac0 con 0x55e1865a6e00 2024-09-15T22:46:18.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.217+0000 7f9eb6000640 1 -- 172.21.15.111:0/2255291888 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9ebc00fcc0 con 0x55e1865a6e00 2024-09-15T22:46:18.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.217+0000 7f9ed9e82740 10 client.14988 fetch_fsmap learned FSMap version 25 2024-09-15T22:46:18.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.217+0000 7f9ed9e82740 1 -- 172.21.15.111:0/2255291888 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e1865c8830 con 0x55e1865a6e00 2024-09-15T22:46:18.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.217+0000 7f9eb6000640 1 -- 172.21.15.111:0/2255291888 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ec80318f0 con 0x55e1865a6e00 2024-09-15T22:46:18.221 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.217+0000 7f9eb6000640 1 -- 172.21.15.111:0/2255291888 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9ebc01c420 con 0x55e1865a6e00 2024-09-15T22:46:18.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.217+0000 7f9ed9e82740 10 client.14988 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:46:18.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.217+0000 7f9ed9e82740 10 client.14988 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-15T22:46:18.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.217+0000 7f9ed9e82740 20 client.14988 populate_metadata read hostname 'smithi111' 2024-09-15T22:46:18.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.217+0000 7f9ed9e82740 1 --2- 172.21.15.111:0/2255291888 >> [v2:172.21.15.192:6838/800136261,v1:172.21.15.192:6839/800136261] conn(0x55e1865cae10 0x55e1865cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:18.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.218+0000 7f9ed9e82740 4 client.14988 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:46:18.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.218+0000 7f9ed9e82740 1 -- 172.21.15.111:0/2255291888 --> [v2:172.21.15.192:6838/800136261,v1:172.21.15.192:6839/800136261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e186519620 con 0x55e1865cae10 2024-09-15T22:46:18.222 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.218+0000 7f9ed5400640 1 --2- 172.21.15.111:0/2255291888 >> [v2:172.21.15.192:6838/800136261,v1:172.21.15.192:6839/800136261] conn(0x55e1865cae10 0x55e1865cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:18.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.218+0000 7f9ed5400640 1 --2- 172.21.15.111:0/2255291888 >> [v2:172.21.15.192:6838/800136261,v1:172.21.15.192:6839/800136261] conn(0x55e1865cae10 0x55e1865cc280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=430390d932283510 server_cookie=539889b8b892d6d3 in_seq=0 out_seq=0 2024-09-15T22:46:18.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.218+0000 7f9eb6000640 10 client.14988 ms_handle_connect on v2:172.21.15.192:6838/800136261 2024-09-15T22:46:18.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.219+0000 7f9eb6000640 1 -- 172.21.15.111:0/2255291888 <== mds.1 v2:172.21.15.192:6838/800136261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e186519620 con 0x55e1865cae10 2024-09-15T22:46:18.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.219+0000 7f9eb6000640 10 client.14988 handle_command_reply: tid=0 2024-09-15T22:46:18.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.219+0000 7f9ed9e82740 1 client.14988 shutdown 2024-09-15T22:46:18.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.220+0000 7f9ed9e82740 1 -- 172.21.15.111:0/2255291888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1865a6e00 msgr2=0x55e1865aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:18.223 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.220+0000 7f9ed9e82740 1 --2- 172.21.15.111:0/2255291888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1865a6e00 0x55e1865aea80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc012710 tx=0x7f9ebc012bd0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.220+0000 7f9ed9e82740 1 -- 172.21.15.111:0/2255291888 shutdown_connections 2024-09-15T22:46:18.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.220+0000 7f9ed9e82740 1 --2- 172.21.15.111:0/2255291888 >> [v2:172.21.15.192:6838/800136261,v1:172.21.15.192:6839/800136261] conn(0x55e1865cae10 0x55e1865cc280 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.220+0000 7f9ed9e82740 1 --2- 172.21.15.111:0/2255291888 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1865b16c0 0x55e1865aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.220+0000 7f9ed9e82740 1 --2- 172.21.15.111:0/2255291888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1865a6e00 0x55e1865aea80 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.220+0000 7f9ed9e82740 1 --2- 172.21.15.111:0/2255291888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e1865a64e0 0x55e1865ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.220+0000 7f9eb6000640 0 client.14988 ms_handle_reset on v2:172.21.15.192:6838/800136261 2024-09-15T22:46:18.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.220+0000 7f9ed9e82740 1 -- 172.21.15.111:0/2255291888 >> 172.21.15.111:0/2255291888 conn(0x55e1865a2a70 msgr2=0x55e1865a87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:18.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.220+0000 7f9ed9e82740 1 -- 172.21.15.111:0/2255291888 shutdown_connections 2024-09-15T22:46:18.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.220+0000 7f9ed9e82740 1 -- 172.21.15.111:0/2255291888 wait complete. 2024-09-15T22:46:18.224 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.220+0000 7f9ed9e82740 20 client.14988 trim_cache size 0 max 16384 2024-09-15T22:46:18.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.238+0000 7f9ed9e82740 1 Processor -- start 2024-09-15T22:46:18.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.238+0000 7f9ed9e82740 1 -- start start 2024-09-15T22:46:18.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.239+0000 7f9ed9e82740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e1865a64e0 0x55e1865cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:18.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.239+0000 7f9ed9e82740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1865a6e00 0x55e1865aee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:18.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.239+0000 7f9ed9e82740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1865b16c0 0x55e1865ab7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:18.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.239+0000 7f9ed9e82740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e1865ca5c0 con 0x55e1865a6e00 2024-09-15T22:46:18.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.239+0000 7f9ed9e82740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e1865cac90 con 0x55e1865b16c0 2024-09-15T22:46:18.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.239+0000 7f9ed9e82740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55e1865cb360 con 0x55e1865a64e0 2024-09-15T22:46:18.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.239+0000 7f9ecfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1865a6e00 0x55e1865aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:18.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.239+0000 7f9ecfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1865a6e00 0x55e1865aee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44558/0 (socket says 172.21.15.111:44558) 2024-09-15T22:46:18.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.239+0000 7f9ecfe00640 1 -- 172.21.15.111:0/209616486 learned_addr learned my addr 172.21.15.111:0/209616486 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:18.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.239+0000 7f9ed5400640 1 --2- 172.21.15.111:0/209616486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e1865a64e0 0x55e1865cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:18.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.239+0000 7f9ecfe00640 1 -- 172.21.15.111:0/209616486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e1865a64e0 msgr2=0x55e1865cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:18.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.239+0000 7f9ecfe00640 1 --2- 172.21.15.111:0/209616486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e1865a64e0 0x55e1865cc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.239+0000 7f9ecfe00640 1 -- 172.21.15.111:0/209616486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1865b16c0 msgr2=0x55e1865ab7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:18.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.239+0000 7f9ed5400640 1 --2- 172.21.15.111:0/209616486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1865b16c0 0x55e1865ab7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:18.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.239+0000 7f9ecfe00640 1 --2- 172.21.15.111:0/209616486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1865b16c0 0x55e1865ab7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.239+0000 7f9ecfe00640 1 -- 172.21.15.111:0/209616486 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1865a6c30 con 0x55e1865a6e00 2024-09-15T22:46:18.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.239+0000 7f9ed5400640 1 --2- 172.21.15.111:0/209616486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1865b16c0 0x55e1865ab7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:46:18.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.240+0000 7f9ecfe00640 1 --2- 172.21.15.111:0/209616486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1865a6e00 0x55e1865aee10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc012710 tx=0x7f9ebc009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a717fbe194a6d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:18.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.240+0000 7f9ec1200640 1 -- 172.21.15.111:0/209616486 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc02b030 con 0x55e1865a6e00 2024-09-15T22:46:18.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.240+0000 7f9ec1200640 1 -- 172.21.15.111:0/209616486 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ebc005510 con 0x55e1865a6e00 2024-09-15T22:46:18.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.240+0000 7f9ec1200640 1 -- 172.21.15.111:0/209616486 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc01ce60 con 0x55e1865a6e00 2024-09-15T22:46:18.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.240+0000 7f9ed9e82740 1 -- 172.21.15.111:0/209616486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1865a6e00 msgr2=0x55e1865aee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:18.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.240+0000 7f9ed9e82740 1 --2- 172.21.15.111:0/209616486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1865a6e00 0x55e1865aee10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc012710 tx=0x7f9ebc009fd0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.240+0000 7f9ed9e82740 1 -- 172.21.15.111:0/209616486 shutdown_connections 2024-09-15T22:46:18.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.240+0000 7f9ed9e82740 1 --2- 172.21.15.111:0/209616486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1865b16c0 0x55e1865ab7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.240+0000 7f9ed9e82740 1 --2- 172.21.15.111:0/209616486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1865a6e00 0x55e1865aee10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.240+0000 7f9ed9e82740 1 --2- 172.21.15.111:0/209616486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e1865a64e0 0x55e1865cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.241+0000 7f9ed9e82740 1 -- 172.21.15.111:0/209616486 >> 172.21.15.111:0/209616486 conn(0x55e1865a2a70 msgr2=0x55e1865ace10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:18.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.241+0000 7f9ed9e82740 1 -- 172.21.15.111:0/209616486 shutdown_connections 2024-09-15T22:46:18.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.241+0000 7f9ed9e82740 1 -- 172.21.15.111:0/209616486 wait complete. 2024-09-15T22:46:18.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.241+0000 7f9ed9e82740 1 -- start start 2024-09-15T22:46:18.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.241+0000 7f9ed9e82740 1 Processor -- start 2024-09-15T22:46:18.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.242+0000 7f9ed9e82740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e1865a64e0 0x55e1865ab740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:18.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.242+0000 7f9ed9e82740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1865a6e00 0x55e1865abc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:18.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.242+0000 7f9ed9e82740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1865b16c0 0x55e1865ac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:18.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.242+0000 7f9ed9e82740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e1865cc100 con 0x55e1865b16c0 2024-09-15T22:46:18.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.242+0000 7f9ed9e82740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e1865cba30 con 0x55e1865a6e00 2024-09-15T22:46:18.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.242+0000 7f9ed9e82740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55e1865cc7d0 con 0x55e1865a64e0 2024-09-15T22:46:18.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.242+0000 7f9ed9e82740 10 client.0 _ll_register_callbacks cb 0x7f9ebc009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:18.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.242+0000 7f9ed5400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e1865a64e0 0x55e1865ab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:18.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.242+0000 7f9ed5400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e1865a64e0 0x55e1865ab740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:36466/0 (socket says 172.21.15.111:36466) 2024-09-15T22:46:18.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.242+0000 7f9ecfe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1865a6e00 0x55e1865abc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:18.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.242+0000 7f9ed5400640 1 -- 172.21.15.111:0/1424627904 learned_addr learned my addr 172.21.15.111:0/1424627904 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:18.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.242+0000 7f9ed5400640 1 --2- 172.21.15.111:0/1424627904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1865b16c0 0x55e1865ac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:18.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.242+0000 7f9ecfe00640 1 -- 172.21.15.111:0/1424627904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e1865a64e0 msgr2=0x55e1865ab740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:18.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.242+0000 7f9ecfe00640 1 --2- 172.21.15.111:0/1424627904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e1865a64e0 0x55e1865ab740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.242+0000 7f9ecfe00640 1 -- 172.21.15.111:0/1424627904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1865b16c0 msgr2=0x55e1865ac1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:18.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.242+0000 7f9ecfe00640 1 --2- 172.21.15.111:0/1424627904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1865b16c0 0x55e1865ac1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.242+0000 7f9ecfe00640 1 -- 172.21.15.111:0/1424627904 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1865d2e70 con 0x55e1865a6e00 2024-09-15T22:46:18.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.243+0000 7f9ecfe00640 1 --2- 172.21.15.111:0/1424627904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1865a6e00 0x55e1865abc80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc012c10 tx=0x7f9ebc005cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43f223fe0c6aadb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:18.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.243+0000 7f9eb6000640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:46:18.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.243+0000 7f9eb6000640 1 -- 172.21.15.111:0/1424627904 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc02b070 con 0x55e1865a6e00 2024-09-15T22:46:18.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.243+0000 7f9eb6000640 1 -- 172.21.15.111:0/1424627904 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f9ebc0050c0 con 0x55e1865a6e00 2024-09-15T22:46:18.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.243+0000 7f9ed9e82740 1 -- 172.21.15.111:0/1424627904 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ebc005510 con 0x55e1865a6e00 2024-09-15T22:46:18.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.243+0000 7f9eb6000640 1 -- 172.21.15.111:0/1424627904 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9ebc01c630 con 0x55e1865a6e00 2024-09-15T22:46:18.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.243+0000 7f9ed9e82740 10 client.25025 fetch_fsmap learned FSMap version 25 2024-09-15T22:46:18.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.243+0000 7f9ed9e82740 1 -- 172.21.15.111:0/1424627904 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e186635900 con 0x55e1865a6e00 2024-09-15T22:46:18.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.243+0000 7f9eb6000640 1 -- 172.21.15.111:0/1424627904 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ebc005510 con 0x55e1865a6e00 2024-09-15T22:46:18.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.244+0000 7f9eb6000640 1 -- 172.21.15.111:0/1424627904 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9ebc010430 con 0x55e1865a6e00 2024-09-15T22:46:18.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.244+0000 7f9ed9e82740 10 client.25025 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:46:18.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.244+0000 7f9ed9e82740 10 client.25025 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-15T22:46:18.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.244+0000 7f9ed9e82740 20 client.25025 populate_metadata read hostname 'smithi111' 2024-09-15T22:46:18.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.244+0000 7f9ed9e82740 1 --2- 172.21.15.111:0/1424627904 >> [v2:172.21.15.192:6838/800136261,v1:172.21.15.192:6839/800136261] conn(0x55e18663a200 0x55e186638f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:18.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.244+0000 7f9ed9e82740 4 client.25025 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:46:18.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.244+0000 7f9ed9e82740 1 -- 172.21.15.111:0/1424627904 --> [v2:172.21.15.192:6838/800136261,v1:172.21.15.192:6839/800136261] -- command(tid 0: {"prefix": "status"}) -- 0x55e186513b30 con 0x55e18663a200 2024-09-15T22:46:18.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.244+0000 7f9ed5400640 1 --2- 172.21.15.111:0/1424627904 >> [v2:172.21.15.192:6838/800136261,v1:172.21.15.192:6839/800136261] conn(0x55e18663a200 0x55e186638f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:18.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.246+0000 7f9ed5400640 1 --2- 172.21.15.111:0/1424627904 >> [v2:172.21.15.192:6838/800136261,v1:172.21.15.192:6839/800136261] conn(0x55e18663a200 0x55e186638f10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6b614c03540779e7 server_cookie=acadbfc1d35c7265 in_seq=0 out_seq=0 2024-09-15T22:46:18.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.246+0000 7f9eb6000640 10 client.25025 ms_handle_connect on v2:172.21.15.192:6838/800136261 2024-09-15T22:46:18.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.246+0000 7f9eb6000640 1 -- 172.21.15.111:0/1424627904 <== mds.1 v2:172.21.15.192:6838/800136261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e186513b30 con 0x55e18663a200 2024-09-15T22:46:18.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.246+0000 7f9eb6000640 10 client.25025 handle_command_reply: tid=0 2024-09-15T22:46:18.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.246+0000 7f9ed9e82740 1 client.25025 shutdown 2024-09-15T22:46:18.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.246+0000 7f9ed9e82740 1 -- 172.21.15.111:0/1424627904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1865a6e00 msgr2=0x55e1865abc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:18.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.246+0000 7f9ed9e82740 1 --2- 172.21.15.111:0/1424627904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1865a6e00 0x55e1865abc80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc012c10 tx=0x7f9ebc005cc0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.246+0000 7f9ed9e82740 1 -- 172.21.15.111:0/1424627904 shutdown_connections 2024-09-15T22:46:18.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.246+0000 7f9ed9e82740 1 --2- 172.21.15.111:0/1424627904 >> [v2:172.21.15.192:6838/800136261,v1:172.21.15.192:6839/800136261] conn(0x55e18663a200 0x55e186638f10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.246+0000 7f9ed9e82740 1 --2- 172.21.15.111:0/1424627904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e1865b16c0 0x55e1865ac1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.246+0000 7f9ed9e82740 1 --2- 172.21.15.111:0/1424627904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1865a6e00 0x55e1865abc80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.246+0000 7f9ed9e82740 1 --2- 172.21.15.111:0/1424627904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e1865a64e0 0x55e1865ab740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.246+0000 7f9eb6000640 0 client.25025 ms_handle_reset on v2:172.21.15.192:6838/800136261 2024-09-15T22:46:18.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.246+0000 7f9ed9e82740 1 -- 172.21.15.111:0/1424627904 >> 172.21.15.111:0/1424627904 conn(0x55e1865a2a70 msgr2=0x55e1865a1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:18.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.246+0000 7f9ed9e82740 1 -- 172.21.15.111:0/1424627904 shutdown_connections 2024-09-15T22:46:18.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.246+0000 7f9ed9e82740 1 -- 172.21.15.111:0/1424627904 wait complete. 2024-09-15T22:46:18.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.246+0000 7f9ed9e82740 20 client.25025 trim_cache size 0 max 16384 2024-09-15T22:46:18.252 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-15T22:46:18.252 INFO:teuthology.orchestra.run.smithi111.stdout: "cluster_fsid": "b7614d3c-73b2-11ef-bceb-c7b262605968", 2024-09-15T22:46:18.253 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": 1, 2024-09-15T22:46:18.253 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 24514, 2024-09-15T22:46:18.253 INFO:teuthology.orchestra.run.smithi111.stdout: "want_state": "up:active", 2024-09-15T22:46:18.253 INFO:teuthology.orchestra.run.smithi111.stdout: "state": "up:active", 2024-09-15T22:46:18.253 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs", 2024-09-15T22:46:18.253 INFO:teuthology.orchestra.run.smithi111.stdout: "rank_uptime": 109.711170322, 2024-09-15T22:46:18.253 INFO:teuthology.orchestra.run.smithi111.stdout: "mdsmap_epoch": 25, 2024-09-15T22:46:18.253 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch": 88, 2024-09-15T22:46:18.253 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch_barrier": 86, 2024-09-15T22:46:18.253 INFO:teuthology.orchestra.run.smithi111.stdout: "uptime": 143.67619827199999 2024-09-15T22:46:18.253 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-15T22:46:18.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.246+0000 7f9ed5e00640 1 -- 172.21.15.111:0/315156635 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9eb80a3240 msgr2=0x7f9eb80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:18.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.246+0000 7f9ed5e00640 1 --2- 172.21.15.111:0/315156635 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9eb80a3240 0x7f9eb80a5700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9ec40a4f60 tx=0x7f9ec40d3c70 comp rx=0 tx=0).stop 2024-09-15T22:46:18.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.246+0000 7f9ed5e00640 1 -- 172.21.15.111:0/315156635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ed01019c0 msgr2=0x7f9ed019a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:18.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.246+0000 7f9ed5e00640 1 --2- 172.21.15.111:0/315156635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ed01019c0 0x7f9ed019a490 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400eb00 tx=0x7f9ec400c2d0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.246+0000 7f9ed5e00640 1 -- 172.21.15.111:0/315156635 shutdown_connections 2024-09-15T22:46:18.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.246+0000 7f9ed5e00640 1 --2- 172.21.15.111:0/315156635 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ed01079c0 0x7f9ed019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.246+0000 7f9ed5e00640 1 --2- 172.21.15.111:0/315156635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ed0102360 0x7f9ed019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.246+0000 7f9ed5e00640 1 --2- 172.21.15.111:0/315156635 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9eb80a3240 0x7f9eb80a5700 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.246+0000 7f9ed5e00640 1 --2- 172.21.15.111:0/315156635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ed01019c0 0x7f9ed019a490 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:18.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.246+0000 7f9ed5e00640 1 -- 172.21.15.111:0/315156635 >> 172.21.15.111:0/315156635 conn(0x7f9ed00fd740 msgr2=0x7f9ed00fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:18.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.247+0000 7f9ed5e00640 1 -- 172.21.15.111:0/315156635 shutdown_connections 2024-09-15T22:46:18.255 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:18.247+0000 7f9ed5e00640 1 -- 172.21.15.111:0/315156635 wait complete. 2024-09-15T22:46:19.179 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-15T22:46:19.545 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:46:20.034 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:19 smithi111 ceph-mon[30629]: pgmap v362: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:46:20.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:19 smithi192 ceph-mon[35427]: pgmap v362: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:46:20.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:19 smithi121 ceph-mon[34815]: pgmap v362: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:46:20.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.145+0000 7fa6c7e00640 1 -- 172.21.15.111:0/3463459994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6c8103e30 msgr2=0x7fa6c8104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:20.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.145+0000 7fa6c7e00640 1 --2- 172.21.15.111:0/3463459994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6c8103e30 0x7fa6c8104210 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8009ca0 tx=0x7fa6b802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.145+0000 7fa6c7e00640 1 -- 172.21.15.111:0/3463459994 shutdown_connections 2024-09-15T22:46:20.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.145+0000 7fa6c7e00640 1 --2- 172.21.15.111:0/3463459994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6c8108f60 0x7fa6c8110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.145+0000 7fa6c7e00640 1 --2- 172.21.15.111:0/3463459994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6c81047e0 0x7fa6c8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.145+0000 7fa6c7e00640 1 --2- 172.21.15.111:0/3463459994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6c8103e30 0x7fa6c8104210 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.145+0000 7fa6c7e00640 1 -- 172.21.15.111:0/3463459994 >> 172.21.15.111:0/3463459994 conn(0x7fa6c80ff8d0 msgr2=0x7fa6c8101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:20.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.145+0000 7fa6c7e00640 1 -- 172.21.15.111:0/3463459994 shutdown_connections 2024-09-15T22:46:20.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.145+0000 7fa6c7e00640 1 -- 172.21.15.111:0/3463459994 wait complete. 2024-09-15T22:46:20.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.146+0000 7fa6c7e00640 1 Processor -- start 2024-09-15T22:46:20.229 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.146+0000 7fa6c7e00640 1 -- start start 2024-09-15T22:46:20.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.146+0000 7fa6c7e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6c8103e30 0x7fa6c819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:20.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.146+0000 7fa6c7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6c81047e0 0x7fa6c819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:20.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.146+0000 7fa6c7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6c8108f60 0x7fa6c819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:20.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.146+0000 7fa6c7e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c8114010 con 0x7fa6c81047e0 2024-09-15T22:46:20.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.146+0000 7fa6c7e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c8113e90 con 0x7fa6c8108f60 2024-09-15T22:46:20.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.146+0000 7fa6c7e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c8114190 con 0x7fa6c8103e30 2024-09-15T22:46:20.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.147+0000 7fa6be000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6c81047e0 0x7fa6c819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:20.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.147+0000 7fa6c6a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6c8103e30 0x7fa6c819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:20.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.147+0000 7fa6c6a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6c8103e30 0x7fa6c819a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:36476/0 (socket says 172.21.15.111:36476) 2024-09-15T22:46:20.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.147+0000 7fa6be000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6c81047e0 0x7fa6c819a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44580/0 (socket says 172.21.15.111:44580) 2024-09-15T22:46:20.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.147+0000 7fa6c6a00640 1 -- 172.21.15.111:0/3075640816 learned_addr learned my addr 172.21.15.111:0/3075640816 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:20.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.147+0000 7fa6c7400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6c8108f60 0x7fa6c819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:20.230 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.147+0000 7fa6c7400640 1 -- 172.21.15.111:0/3075640816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6c8103e30 msgr2=0x7fa6c819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:20.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.147+0000 7fa6c7400640 1 --2- 172.21.15.111:0/3075640816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6c8103e30 0x7fa6c819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.147+0000 7fa6c7400640 1 -- 172.21.15.111:0/3075640816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6c81047e0 msgr2=0x7fa6c819a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:20.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.147+0000 7fa6c7400640 1 --2- 172.21.15.111:0/3075640816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6c81047e0 0x7fa6c819a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.147+0000 7fa6c7400640 1 -- 172.21.15.111:0/3075640816 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c819f410 con 0x7fa6c8108f60 2024-09-15T22:46:20.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.147+0000 7fa6c6a00640 1 --2- 172.21.15.111:0/3075640816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6c8103e30 0x7fa6c819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:46:20.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.147+0000 7fa6be000640 1 --2- 172.21.15.111:0/3075640816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6c81047e0 0x7fa6c819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:46:20.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.147+0000 7fa6c7400640 1 --2- 172.21.15.111:0/3075640816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6c8108f60 0x7fa6c819ed30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000d680 tx=0x7fa6c000db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:20.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.148+0000 7fa6cca00640 1 -- 172.21.15.111:0/3075640816 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0017070 con 0x7fa6c8108f60 2024-09-15T22:46:20.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.148+0000 7fa6cca00640 1 -- 172.21.15.111:0/3075640816 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6c0009820 con 0x7fa6c8108f60 2024-09-15T22:46:20.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.148+0000 7fa6c7e00640 1 -- 172.21.15.111:0/3075640816 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c819f700 con 0x7fa6c8108f60 2024-09-15T22:46:20.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.148+0000 7fa6cca00640 1 -- 172.21.15.111:0/3075640816 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0004e00 con 0x7fa6c8108f60 2024-09-15T22:46:20.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.148+0000 7fa6c7e00640 1 -- 172.21.15.111:0/3075640816 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c81a6fe0 con 0x7fa6c8108f60 2024-09-15T22:46:20.231 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.151+0000 7fa6d0a4c740 1 Processor -- start 2024-09-15T22:46:20.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.151+0000 7fa6cca00640 1 -- 172.21.15.111:0/3075640816 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fa6c00099d0 con 0x7fa6c8108f60 2024-09-15T22:46:20.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.151+0000 7fa6d0a4c740 1 -- start start 2024-09-15T22:46:20.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.151+0000 7fa6d0a4c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ce390b4e0 0x561ce390b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:20.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.151+0000 7fa6d0a4c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ce390be00 0x561ce3916180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:20.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.151+0000 7fa6d0a4c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ce39166c0 0x561ce3918ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:20.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.151+0000 7fa6d0a4c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561ce38710b0 con 0x561ce390b4e0 2024-09-15T22:46:20.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.151+0000 7fa6d0a4c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561ce3818ab0 con 0x561ce390be00 2024-09-15T22:46:20.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.151+0000 7fa6d0a4c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561ce386d070 con 0x561ce39166c0 2024-09-15T22:46:20.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.152+0000 7fa6be000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ce390b4e0 0x561ce390b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:20.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.152+0000 7fa6be000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ce390b4e0 0x561ce390b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44594/0 (socket says 172.21.15.111:44594) 2024-09-15T22:46:20.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.152+0000 7fa6be000640 1 -- 172.21.15.111:0/3002943510 learned_addr learned my addr 172.21.15.111:0/3002943510 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:20.232 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.152+0000 7fa6c6a00640 1 --2- 172.21.15.111:0/3002943510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ce390be00 0x561ce3916180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:20.233 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.152+0000 7fa6be000640 1 --2- 172.21.15.111:0/3002943510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ce39166c0 0x561ce3918ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:20.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.152+0000 7fa6be000640 1 -- 172.21.15.111:0/3002943510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ce39166c0 msgr2=0x561ce3918ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:20.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.152+0000 7fa6be000640 1 --2- 172.21.15.111:0/3002943510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ce39166c0 0x561ce3918ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.152+0000 7fa6be000640 1 -- 172.21.15.111:0/3002943510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ce390be00 msgr2=0x561ce3916180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:20.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.152+0000 7fa6be000640 1 --2- 172.21.15.111:0/3002943510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ce390be00 0x561ce3916180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.152+0000 7fa6be000640 1 -- 172.21.15.111:0/3002943510 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ce38254a0 con 0x561ce390b4e0 2024-09-15T22:46:20.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.152+0000 7fa6c6a00640 1 --2- 172.21.15.111:0/3002943510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ce390be00 0x561ce3916180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:46:20.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.152+0000 7fa6cca00640 1 --2- 172.21.15.111:0/3075640816 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa6ac0a3240 0x7fa6ac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:20.235 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.152+0000 7fa6cca00640 1 -- 172.21.15.111:0/3075640816 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fa6c00cf080 con 0x7fa6c8108f60 2024-09-15T22:46:20.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.152+0000 7fa6be000640 1 --2- 172.21.15.111:0/3002943510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ce390b4e0 0x561ce390b8c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0011bc0 tx=0x7fa6b0009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8f1bebdaaa456e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:20.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.153+0000 7fa6c6a00640 1 --2- 172.21.15.111:0/3075640816 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa6ac0a3240 0x7fa6ac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:20.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.153+0000 7fa6b7e00640 1 -- 172.21.15.111:0/3002943510 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b000fc50 con 0x561ce390b4e0 2024-09-15T22:46:20.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.153+0000 7fa6b7e00640 1 -- 172.21.15.111:0/3002943510 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6b000fdf0 con 0x561ce390b4e0 2024-09-15T22:46:20.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.153+0000 7fa6b7e00640 1 -- 172.21.15.111:0/3002943510 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b000e640 con 0x561ce390b4e0 2024-09-15T22:46:20.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.153+0000 7fa6d0a4c740 1 -- 172.21.15.111:0/3002943510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ce390b4e0 msgr2=0x561ce390b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:20.236 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.153+0000 7fa6c6a00640 1 --2- 172.21.15.111:0/3075640816 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa6ac0a3240 0x7fa6ac0a5700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x561ce39088d0 tx=0x7fa6b8005d80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:20.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.153+0000 7fa6d0a4c740 1 --2- 172.21.15.111:0/3002943510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ce390b4e0 0x561ce390b8c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0011bc0 tx=0x7fa6b0009b70 comp rx=0 tx=0).stop 2024-09-15T22:46:20.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.154+0000 7fa6d0a4c740 1 -- 172.21.15.111:0/3002943510 shutdown_connections 2024-09-15T22:46:20.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.154+0000 7fa6d0a4c740 1 --2- 172.21.15.111:0/3002943510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ce39166c0 0x561ce3918ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.154+0000 7fa6d0a4c740 1 --2- 172.21.15.111:0/3002943510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ce390be00 0x561ce3916180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.154+0000 7fa6d0a4c740 1 --2- 172.21.15.111:0/3002943510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ce390b4e0 0x561ce390b8c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.154+0000 7fa6d0a4c740 1 -- 172.21.15.111:0/3002943510 >> 172.21.15.111:0/3002943510 conn(0x561ce3907a70 msgr2=0x561ce3907e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:20.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.154+0000 7fa6d0a4c740 1 -- 172.21.15.111:0/3002943510 shutdown_connections 2024-09-15T22:46:20.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.154+0000 7fa6d0a4c740 1 -- 172.21.15.111:0/3002943510 wait complete. 2024-09-15T22:46:20.237 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.154+0000 7fa6d0a4c740 1 -- start start 2024-09-15T22:46:20.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.154+0000 7fa6d0a4c740 1 Processor -- start 2024-09-15T22:46:20.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.155+0000 7fa6d0a4c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ce390b4e0 0x561ce3913520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:20.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.155+0000 7fa6d0a4c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ce390be00 0x561ce3913a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:20.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.155+0000 7fa6d0a4c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ce39166c0 0x561ce3913fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:20.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.155+0000 7fa6d0a4c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561ce3919330 con 0x561ce390b4e0 2024-09-15T22:46:20.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.155+0000 7fa6d0a4c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561ce37f9920 con 0x561ce390be00 2024-09-15T22:46:20.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.155+0000 7fa6d0a4c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561ce39194b0 con 0x561ce39166c0 2024-09-15T22:46:20.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.155+0000 7fa6d0a4c740 10 client.0 _ll_register_callbacks cb 0x561ce381d8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:20.238 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.155+0000 7fa6c6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ce390b4e0 0x561ce3913520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:20.239 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.155+0000 7fa6be000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ce390be00 0x561ce3913a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:20.239 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.155+0000 7fa6c6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ce390b4e0 0x561ce3913520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44608/0 (socket says 172.21.15.111:44608) 2024-09-15T22:46:20.239 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.155+0000 7fa6c6a00640 1 -- 172.21.15.111:0/2877134424 learned_addr learned my addr 172.21.15.111:0/2877134424 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:20.239 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.155+0000 7fa6c7400640 1 --2- 172.21.15.111:0/2877134424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ce39166c0 0x561ce3913fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:20.239 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.155+0000 7fa6c6a00640 1 -- 172.21.15.111:0/2877134424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ce39166c0 msgr2=0x561ce3913fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:20.239 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.155+0000 7fa6c6a00640 1 --2- 172.21.15.111:0/2877134424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ce39166c0 0x561ce3913fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.239 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.155+0000 7fa6c6a00640 1 -- 172.21.15.111:0/2877134424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ce390be00 msgr2=0x561ce3913a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:20.239 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.155+0000 7fa6c6a00640 1 --2- 172.21.15.111:0/2877134424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ce390be00 0x561ce3913a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.239 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.155+0000 7fa6c6a00640 1 -- 172.21.15.111:0/2877134424 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ce35e22b0 con 0x561ce390b4e0 2024-09-15T22:46:20.239 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.155+0000 7fa6be000640 1 --2- 172.21.15.111:0/2877134424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ce390be00 0x561ce3913a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:46:20.239 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.156+0000 7fa6c6a00640 1 --2- 172.21.15.111:0/2877134424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ce390b4e0 0x561ce3913520 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80098c0 tx=0x7fa6b802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0773eaaa7c9796 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:20.240 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.156+0000 7fa6b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:46:20.240 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.156+0000 7fa6b4c00640 1 -- 172.21.15.111:0/2877134424 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8004250 con 0x561ce390b4e0 2024-09-15T22:46:20.240 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.156+0000 7fa6b4c00640 1 -- 172.21.15.111:0/2877134424 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fa6b80043f0 con 0x561ce390b4e0 2024-09-15T22:46:20.240 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.156+0000 7fa6d0a4c740 1 -- 172.21.15.111:0/2877134424 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6b000fdf0 con 0x561ce390b4e0 2024-09-15T22:46:20.240 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.156+0000 7fa6b4c00640 1 -- 172.21.15.111:0/2877134424 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa6b80045f0 con 0x561ce390b4e0 2024-09-15T22:46:20.240 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.156+0000 7fa6d0a4c740 10 client.15006 fetch_fsmap learned FSMap version 25 2024-09-15T22:46:20.240 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.156+0000 7fa6d0a4c740 1 -- 172.21.15.111:0/2877134424 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561ce3924f90 con 0x561ce390b4e0 2024-09-15T22:46:20.240 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.157+0000 7fa6b4c00640 1 -- 172.21.15.111:0/2877134424 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6b000fdf0 con 0x561ce390b4e0 2024-09-15T22:46:20.240 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.157+0000 7fa6b4c00640 1 -- 172.21.15.111:0/2877134424 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa6b8052020 con 0x561ce390b4e0 2024-09-15T22:46:20.240 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.157+0000 7fa6d0a4c740 10 client.15006 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:46:20.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.157+0000 7fa6d0a4c740 10 client.15006 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:46:20.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.157+0000 7fa6d0a4c740 20 client.15006 populate_metadata read hostname 'smithi111' 2024-09-15T22:46:20.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.157+0000 7fa6d0a4c740 1 --2- 172.21.15.111:0/2877134424 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561ce39276f0 0x561ce3928b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:20.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.157+0000 7fa6d0a4c740 4 client.15006 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:46:20.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.157+0000 7fa6d0a4c740 1 -- 172.21.15.111:0/2877134424 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ce387e620 con 0x561ce39276f0 2024-09-15T22:46:20.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.157+0000 7fa6be000640 1 --2- 172.21.15.111:0/2877134424 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561ce39276f0 0x561ce3928b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:20.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.158+0000 7fa6be000640 1 --2- 172.21.15.111:0/2877134424 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561ce39276f0 0x561ce3928b60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82249fa2dc489d82 server_cookie=9ccd4ecce2ae1aa4 in_seq=0 out_seq=0 2024-09-15T22:46:20.241 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.158+0000 7fa6b4c00640 10 client.15006 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:46:20.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.159+0000 7fa6b4c00640 1 -- 172.21.15.111:0/2877134424 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ce387e620 con 0x561ce39276f0 2024-09-15T22:46:20.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.159+0000 7fa6b4c00640 10 client.15006 handle_command_reply: tid=0 2024-09-15T22:46:20.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.159+0000 7fa6d0a4c740 1 client.15006 shutdown 2024-09-15T22:46:20.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.159+0000 7fa6d0a4c740 1 -- 172.21.15.111:0/2877134424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ce390b4e0 msgr2=0x561ce3913520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:20.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.159+0000 7fa6d0a4c740 1 --2- 172.21.15.111:0/2877134424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ce390b4e0 0x561ce3913520 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80098c0 tx=0x7fa6b802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.159+0000 7fa6d0a4c740 1 -- 172.21.15.111:0/2877134424 shutdown_connections 2024-09-15T22:46:20.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.159+0000 7fa6d0a4c740 1 --2- 172.21.15.111:0/2877134424 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561ce39276f0 0x561ce3928b60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.242 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.159+0000 7fa6d0a4c740 1 --2- 172.21.15.111:0/2877134424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ce39166c0 0x561ce3913fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.159+0000 7fa6d0a4c740 1 --2- 172.21.15.111:0/2877134424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ce390be00 0x561ce3913a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.159+0000 7fa6d0a4c740 1 --2- 172.21.15.111:0/2877134424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ce390b4e0 0x561ce3913520 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.159+0000 7fa6d0a4c740 1 -- 172.21.15.111:0/2877134424 >> 172.21.15.111:0/2877134424 conn(0x561ce3907a70 msgr2=0x561ce390d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:20.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.159+0000 7fa6b4c00640 0 client.15006 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:46:20.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.159+0000 7fa6d0a4c740 1 -- 172.21.15.111:0/2877134424 shutdown_connections 2024-09-15T22:46:20.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.159+0000 7fa6d0a4c740 1 -- 172.21.15.111:0/2877134424 wait complete. 2024-09-15T22:46:20.243 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.159+0000 7fa6d0a4c740 20 client.15006 trim_cache size 0 max 16384 2024-09-15T22:46:20.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.177+0000 7fa6d0a4c740 1 Processor -- start 2024-09-15T22:46:20.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.178+0000 7fa6d0a4c740 1 -- start start 2024-09-15T22:46:20.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.178+0000 7fa6d0a4c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ce390b4e0 0x561ce3927dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:20.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.178+0000 7fa6d0a4c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ce390be00 0x561ce3913df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:20.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.178+0000 7fa6d0a4c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ce39166c0 0x561ce3910780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:20.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.178+0000 7fa6d0a4c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561ce3926ea0 con 0x561ce39166c0 2024-09-15T22:46:20.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.178+0000 7fa6d0a4c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561ce3927570 con 0x561ce390be00 2024-09-15T22:46:20.244 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.178+0000 7fa6d0a4c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561ce3927c40 con 0x561ce390b4e0 2024-09-15T22:46:20.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.178+0000 7fa6c6a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ce390b4e0 0x561ce3927dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:20.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.178+0000 7fa6c6a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ce390b4e0 0x561ce3927dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:36512/0 (socket says 172.21.15.111:36512) 2024-09-15T22:46:20.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.178+0000 7fa6c6a00640 1 -- 172.21.15.111:0/1957022051 learned_addr learned my addr 172.21.15.111:0/1957022051 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:20.245 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.178+0000 7fa6c7400640 1 --2- 172.21.15.111:0/1957022051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ce39166c0 0x561ce3910780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:20.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.178+0000 7fa6be000640 1 --2- 172.21.15.111:0/1957022051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ce390be00 0x561ce3913df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:20.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.178+0000 7fa6c6a00640 1 -- 172.21.15.111:0/1957022051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ce390be00 msgr2=0x561ce3913df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:20.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.178+0000 7fa6c6a00640 1 --2- 172.21.15.111:0/1957022051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ce390be00 0x561ce3913df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.178+0000 7fa6c6a00640 1 -- 172.21.15.111:0/1957022051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ce39166c0 msgr2=0x561ce3910780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:20.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.178+0000 7fa6c6a00640 1 --2- 172.21.15.111:0/1957022051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ce39166c0 0x561ce3910780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.178+0000 7fa6c6a00640 1 -- 172.21.15.111:0/1957022051 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ce39078a0 con 0x561ce390b4e0 2024-09-15T22:46:20.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.178+0000 7fa6c7400640 1 --2- 172.21.15.111:0/1957022051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ce39166c0 0x561ce3910780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:46:20.246 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.179+0000 7fa6be000640 1 --2- 172.21.15.111:0/1957022051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ce390be00 0x561ce3913df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:46:20.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.179+0000 7fa6c6a00640 1 --2- 172.21.15.111:0/1957022051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ce390b4e0 0x561ce3927dc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8000f80 tx=0x7fa6b8003970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb5f815988d0045a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:20.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.179+0000 7fa6b7400640 1 -- 172.21.15.111:0/1957022051 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8004250 con 0x561ce390b4e0 2024-09-15T22:46:20.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.179+0000 7fa6b7400640 1 -- 172.21.15.111:0/1957022051 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6b8036040 con 0x561ce390b4e0 2024-09-15T22:46:20.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.179+0000 7fa6b7400640 1 -- 172.21.15.111:0/1957022051 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b802fd50 con 0x561ce390b4e0 2024-09-15T22:46:20.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.180+0000 7fa6d0a4c740 1 -- 172.21.15.111:0/1957022051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ce390b4e0 msgr2=0x561ce3927dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:20.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.180+0000 7fa6d0a4c740 1 --2- 172.21.15.111:0/1957022051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ce390b4e0 0x561ce3927dc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8000f80 tx=0x7fa6b8003970 comp rx=0 tx=0).stop 2024-09-15T22:46:20.247 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.180+0000 7fa6d0a4c740 1 -- 172.21.15.111:0/1957022051 shutdown_connections 2024-09-15T22:46:20.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.180+0000 7fa6d0a4c740 1 --2- 172.21.15.111:0/1957022051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ce39166c0 0x561ce3910780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.180+0000 7fa6d0a4c740 1 --2- 172.21.15.111:0/1957022051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ce390be00 0x561ce3913df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.180+0000 7fa6d0a4c740 1 --2- 172.21.15.111:0/1957022051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ce390b4e0 0x561ce3927dc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.180+0000 7fa6d0a4c740 1 -- 172.21.15.111:0/1957022051 >> 172.21.15.111:0/1957022051 conn(0x561ce3907a70 msgr2=0x561ce3911df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:20.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.180+0000 7fa6d0a4c740 1 -- 172.21.15.111:0/1957022051 shutdown_connections 2024-09-15T22:46:20.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.180+0000 7fa6d0a4c740 1 -- 172.21.15.111:0/1957022051 wait complete. 2024-09-15T22:46:20.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.180+0000 7fa6d0a4c740 1 -- start start 2024-09-15T22:46:20.248 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.181+0000 7fa6d0a4c740 1 Processor -- start 2024-09-15T22:46:20.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.181+0000 7fa6d0a4c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ce390b4e0 0x561ce392a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:20.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.181+0000 7fa6d0a4c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ce390be00 0x561ce392a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:20.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.181+0000 7fa6d0a4c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ce39166c0 0x561ce398fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:20.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.181+0000 7fa6d0a4c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561ce39289e0 con 0x561ce390be00 2024-09-15T22:46:20.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.181+0000 7fa6d0a4c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561ce3928310 con 0x561ce39166c0 2024-09-15T22:46:20.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.181+0000 7fa6d0a4c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561ce39290b0 con 0x561ce390b4e0 2024-09-15T22:46:20.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.181+0000 7fa6d0a4c740 10 client.0 _ll_register_callbacks cb 0x7fa6b8003e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:20.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.181+0000 7fa6be000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ce390be00 0x561ce392a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:20.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.181+0000 7fa6be000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ce390be00 0x561ce392a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44632/0 (socket says 172.21.15.111:44632) 2024-09-15T22:46:20.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.181+0000 7fa6be000640 1 -- 172.21.15.111:0/2409098236 learned_addr learned my addr 172.21.15.111:0/2409098236 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:20.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.181+0000 7fa6c6a00640 1 --2- 172.21.15.111:0/2409098236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ce390b4e0 0x561ce392a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:20.249 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.181+0000 7fa6c7400640 1 --2- 172.21.15.111:0/2409098236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ce39166c0 0x561ce398fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:20.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.182+0000 7fa6be000640 1 -- 172.21.15.111:0/2409098236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ce390b4e0 msgr2=0x561ce392a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:20.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.182+0000 7fa6be000640 1 --2- 172.21.15.111:0/2409098236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ce390b4e0 0x561ce392a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.182+0000 7fa6be000640 1 -- 172.21.15.111:0/2409098236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ce39166c0 msgr2=0x561ce398fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:20.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.182+0000 7fa6be000640 1 --2- 172.21.15.111:0/2409098236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ce39166c0 0x561ce398fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.182+0000 7fa6be000640 1 -- 172.21.15.111:0/2409098236 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ce3908700 con 0x561ce390be00 2024-09-15T22:46:20.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.182+0000 7fa6be000640 1 --2- 172.21.15.111:0/2409098236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ce390be00 0x561ce392a6b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0010010 tx=0x7fa6b001fe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5cab7a008936009 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:20.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.182+0000 7fa6b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:46:20.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.182+0000 7fa6b4c00640 1 -- 172.21.15.111:0/2409098236 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0005650 con 0x561ce390be00 2024-09-15T22:46:20.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.182+0000 7fa6b4c00640 1 -- 172.21.15.111:0/2409098236 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fa6b000fd60 con 0x561ce390be00 2024-09-15T22:46:20.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.182+0000 7fa6d0a4c740 1 -- 172.21.15.111:0/2409098236 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6b8036040 con 0x561ce390be00 2024-09-15T22:46:20.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.183+0000 7fa6b4c00640 1 -- 172.21.15.111:0/2409098236 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa6b001f440 con 0x561ce390be00 2024-09-15T22:46:20.250 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.183+0000 7fa6d0a4c740 10 client.15012 fetch_fsmap learned FSMap version 25 2024-09-15T22:46:20.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.183+0000 7fa6d0a4c740 1 -- 172.21.15.111:0/2409098236 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561ce3991310 con 0x561ce390be00 2024-09-15T22:46:20.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.183+0000 7fa6b4c00640 1 -- 172.21.15.111:0/2409098236 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6b8036040 con 0x561ce390be00 2024-09-15T22:46:20.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.183+0000 7fa6b4c00640 1 -- 172.21.15.111:0/2409098236 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa6b001d020 con 0x561ce390be00 2024-09-15T22:46:20.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.183+0000 7fa6d0a4c740 10 client.15012 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:46:20.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.183+0000 7fa6d0a4c740 10 client.15012 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:46:20.251 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.183+0000 7fa6d0a4c740 20 client.15012 populate_metadata read hostname 'smithi111' 2024-09-15T22:46:20.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.183+0000 7fa6d0a4c740 1 --2- 172.21.15.111:0/2409098236 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561ce3995c60 0x561ce3994970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:20.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.183+0000 7fa6d0a4c740 4 client.15012 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:46:20.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.183+0000 7fa6d0a4c740 1 -- 172.21.15.111:0/2409098236 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "status"}) -- 0x561ce3878b30 con 0x561ce3995c60 2024-09-15T22:46:20.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.184+0000 7fa6c6a00640 1 --2- 172.21.15.111:0/2409098236 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561ce3995c60 0x561ce3994970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:20.252 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.184+0000 7fa6c6a00640 1 --2- 172.21.15.111:0/2409098236 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561ce3995c60 0x561ce3994970 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92efaaa79d9d6d2e server_cookie=988261194d6285da in_seq=0 out_seq=0 2024-09-15T22:46:20.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.184+0000 7fa6b4c00640 10 client.15012 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:46:20.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.184+0000 7fa6b4c00640 1 -- 172.21.15.111:0/2409098236 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561ce3878b30 con 0x561ce3995c60 2024-09-15T22:46:20.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.184+0000 7fa6b4c00640 10 client.15012 handle_command_reply: tid=0 2024-09-15T22:46:20.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.184+0000 7fa6d0a4c740 1 client.15012 shutdown 2024-09-15T22:46:20.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.184+0000 7fa6d0a4c740 1 -- 172.21.15.111:0/2409098236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ce390be00 msgr2=0x561ce392a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:20.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.184+0000 7fa6d0a4c740 1 --2- 172.21.15.111:0/2409098236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ce390be00 0x561ce392a6b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0010010 tx=0x7fa6b001fe20 comp rx=0 tx=0).stop 2024-09-15T22:46:20.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.185+0000 7fa6d0a4c740 1 -- 172.21.15.111:0/2409098236 shutdown_connections 2024-09-15T22:46:20.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.185+0000 7fa6d0a4c740 1 --2- 172.21.15.111:0/2409098236 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561ce3995c60 0x561ce3994970 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.185+0000 7fa6d0a4c740 1 --2- 172.21.15.111:0/2409098236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ce39166c0 0x561ce398fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.185+0000 7fa6b4c00640 0 client.15012 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:46:20.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.185+0000 7fa6d0a4c740 1 --2- 172.21.15.111:0/2409098236 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561ce390be00 0x561ce392a6b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.185+0000 7fa6d0a4c740 1 --2- 172.21.15.111:0/2409098236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561ce390b4e0 0x561ce392a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.253 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.185+0000 7fa6d0a4c740 1 -- 172.21.15.111:0/2409098236 >> 172.21.15.111:0/2409098236 conn(0x561ce3907a70 msgr2=0x561ce3906bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:20.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.185+0000 7fa6d0a4c740 1 -- 172.21.15.111:0/2409098236 shutdown_connections 2024-09-15T22:46:20.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.185+0000 7fa6d0a4c740 1 -- 172.21.15.111:0/2409098236 wait complete. 2024-09-15T22:46:20.254 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.185+0000 7fa6d0a4c740 20 client.15012 trim_cache size 0 max 16384 2024-09-15T22:46:20.254 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-15T22:46:20.254 INFO:teuthology.orchestra.run.smithi111.stdout: "cluster_fsid": "b7614d3c-73b2-11ef-bceb-c7b262605968", 2024-09-15T22:46:20.255 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": 0, 2024-09-15T22:46:20.255 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 24545, 2024-09-15T22:46:20.255 INFO:teuthology.orchestra.run.smithi111.stdout: "want_state": "up:active", 2024-09-15T22:46:20.255 INFO:teuthology.orchestra.run.smithi111.stdout: "state": "up:active", 2024-09-15T22:46:20.255 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs", 2024-09-15T22:46:20.255 INFO:teuthology.orchestra.run.smithi111.stdout: "rank_uptime": 120.98292347899999, 2024-09-15T22:46:20.255 INFO:teuthology.orchestra.run.smithi111.stdout: "mdsmap_epoch": 25, 2024-09-15T22:46:20.255 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch": 88, 2024-09-15T22:46:20.255 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch_barrier": 86, 2024-09-15T22:46:20.255 INFO:teuthology.orchestra.run.smithi111.stdout: "uptime": 135.285809048 2024-09-15T22:46:20.255 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-15T22:46:20.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.185+0000 7fa6c7e00640 1 -- 172.21.15.111:0/3075640816 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa6ac0a3240 msgr2=0x7fa6ac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:20.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.185+0000 7fa6c7e00640 1 --2- 172.21.15.111:0/3075640816 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa6ac0a3240 0x7fa6ac0a5700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x561ce39088d0 tx=0x7fa6b8005d80 comp rx=0 tx=0).stop 2024-09-15T22:46:20.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.186+0000 7fa6c7e00640 1 -- 172.21.15.111:0/3075640816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6c8108f60 msgr2=0x7fa6c819ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:20.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.186+0000 7fa6c7e00640 1 --2- 172.21.15.111:0/3075640816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6c8108f60 0x7fa6c819ed30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000d680 tx=0x7fa6c000db40 comp rx=0 tx=0).stop 2024-09-15T22:46:20.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.186+0000 7fa6c7e00640 1 -- 172.21.15.111:0/3075640816 shutdown_connections 2024-09-15T22:46:20.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.186+0000 7fa6c7e00640 1 --2- 172.21.15.111:0/3075640816 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa6ac0a3240 0x7fa6ac0a5700 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.256 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.186+0000 7fa6c7e00640 1 --2- 172.21.15.111:0/3075640816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6c8108f60 0x7fa6c819ed30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.257 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.186+0000 7fa6c7e00640 1 --2- 172.21.15.111:0/3075640816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6c81047e0 0x7fa6c819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.257 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.186+0000 7fa6c7e00640 1 --2- 172.21.15.111:0/3075640816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6c8103e30 0x7fa6c819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:20.257 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.186+0000 7fa6c7e00640 1 -- 172.21.15.111:0/3075640816 >> 172.21.15.111:0/3075640816 conn(0x7fa6c80ff8d0 msgr2=0x7fa6c8100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:20.257 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.186+0000 7fa6c7e00640 1 -- 172.21.15.111:0/3075640816 shutdown_connections 2024-09-15T22:46:20.257 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:20.186+0000 7fa6c7e00640 1 -- 172.21.15.111:0/3075640816 wait complete. 2024-09-15T22:46:21.069 INFO:teuthology.run_tasks:Running task sequential... 2024-09-15T22:46:21.078 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-15T22:46:21.079 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-15T22:46:21.079 INFO:teuthology.task.print:Enabling mgr modules 2024-09-15T22:46:21.079 INFO:teuthology.run_tasks:Running task kclient... 2024-09-15T22:46:21.110 INFO:tasks.kclient:Mounting kernel clients... 2024-09-15T22:46:21.111 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-15T22:46:21.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:46:21.111 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2024-09-15T22:46:21.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:46:21.148 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link delete ceph-brx 2024-09-15T22:46:21.247 INFO:teuthology.orchestra.run.smithi111.stderr:Cannot find device "ceph-brx" 2024-09-15T22:46:21.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T22:46:21.250 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-15T22:46:21.250 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-15T22:46:21.250 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-15T22:46:21.250 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-15T22:46:21.616 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:46:21.952 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:21 smithi111 ceph-mon[30629]: pgmap v363: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:46:22.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:21 smithi192 ceph-mon[35427]: pgmap v363: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:46:22.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:21 smithi121 ceph-mon[34815]: pgmap v363: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:46:22.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.186+0000 7fd490e00640 1 -- 172.21.15.111:0/3232381067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd48c1047e0 msgr2=0x7fd48c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:22.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.186+0000 7fd490e00640 1 --2- 172.21.15.111:0/3232381067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd48c1047e0 0x7fd48c108830 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd474009900 tx=0x7fd47402a410 comp rx=0 tx=0).stop 2024-09-15T22:46:22.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.186+0000 7fd490e00640 1 -- 172.21.15.111:0/3232381067 shutdown_connections 2024-09-15T22:46:22.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.186+0000 7fd490e00640 1 --2- 172.21.15.111:0/3232381067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd48c108f60 0x7fd48c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:22.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.186+0000 7fd490e00640 1 --2- 172.21.15.111:0/3232381067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd48c1047e0 0x7fd48c108830 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:22.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.186+0000 7fd490e00640 1 --2- 172.21.15.111:0/3232381067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd48c103e30 0x7fd48c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:22.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.186+0000 7fd490e00640 1 -- 172.21.15.111:0/3232381067 >> 172.21.15.111:0/3232381067 conn(0x7fd48c0ff8d0 msgr2=0x7fd48c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:22.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.187+0000 7fd490e00640 1 -- 172.21.15.111:0/3232381067 shutdown_connections 2024-09-15T22:46:22.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.187+0000 7fd490e00640 1 -- 172.21.15.111:0/3232381067 wait complete. 2024-09-15T22:46:22.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.187+0000 7fd490e00640 1 Processor -- start 2024-09-15T22:46:22.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.188+0000 7fd490e00640 1 -- start start 2024-09-15T22:46:22.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.188+0000 7fd490e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd48c103e30 0x7fd48c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:22.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.188+0000 7fd490e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd48c1047e0 0x7fd48c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:22.190 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.188+0000 7fd490e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd48c108f60 0x7fd48c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:22.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.188+0000 7fd490e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c113fc0 con 0x7fd48c108f60 2024-09-15T22:46:22.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.188+0000 7fd490e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c113e40 con 0x7fd48c1047e0 2024-09-15T22:46:22.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.188+0000 7fd490e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c114140 con 0x7fd48c103e30 2024-09-15T22:46:22.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.188+0000 7fd48be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd48c108f60 0x7fd48c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:22.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.188+0000 7fd48be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd48c108f60 0x7fd48c19ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46844/0 (socket says 172.21.15.111:46844) 2024-09-15T22:46:22.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.188+0000 7fd48be00640 1 -- 172.21.15.111:0/689007930 learned_addr learned my addr 172.21.15.111:0/689007930 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:22.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.188+0000 7fd48b400640 1 --2- 172.21.15.111:0/689007930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd48c103e30 0x7fd48c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:22.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.188+0000 7fd483e00640 1 --2- 172.21.15.111:0/689007930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd48c1047e0 0x7fd48c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:22.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.189+0000 7fd48be00640 1 -- 172.21.15.111:0/689007930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd48c103e30 msgr2=0x7fd48c19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:22.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.189+0000 7fd48be00640 1 --2- 172.21.15.111:0/689007930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd48c103e30 0x7fd48c19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:22.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.189+0000 7fd48be00640 1 -- 172.21.15.111:0/689007930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd48c1047e0 msgr2=0x7fd48c19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:22.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.189+0000 7fd48be00640 1 --2- 172.21.15.111:0/689007930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd48c1047e0 0x7fd48c19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:22.191 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.189+0000 7fd48be00640 1 -- 172.21.15.111:0/689007930 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c19f450 con 0x7fd48c108f60 2024-09-15T22:46:22.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.189+0000 7fd48be00640 1 --2- 172.21.15.111:0/689007930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd48c108f60 0x7fd48c19ed70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd484009fd0 tx=0x7fd48400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:22.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.189+0000 7fd488c00640 1 -- 172.21.15.111:0/689007930 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484017070 con 0x7fd48c108f60 2024-09-15T22:46:22.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.189+0000 7fd488c00640 1 -- 172.21.15.111:0/689007930 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd484003e50 con 0x7fd48c108f60 2024-09-15T22:46:22.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.189+0000 7fd490e00640 1 -- 172.21.15.111:0/689007930 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48c19f740 con 0x7fd48c108f60 2024-09-15T22:46:22.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.189+0000 7fd488c00640 1 -- 172.21.15.111:0/689007930 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4840106c0 con 0x7fd48c108f60 2024-09-15T22:46:22.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.189+0000 7fd490e00640 1 -- 172.21.15.111:0/689007930 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48c1a7020 con 0x7fd48c108f60 2024-09-15T22:46:22.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.190+0000 7fd490e00640 1 -- 172.21.15.111:0/689007930 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd48c104210 con 0x7fd48c108f60 2024-09-15T22:46:22.193 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.192+0000 7fd488c00640 1 -- 172.21.15.111:0/689007930 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7fd484003970 con 0x7fd48c108f60 2024-09-15T22:46:22.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.193+0000 7fd488c00640 1 --2- 172.21.15.111:0/689007930 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd4700a32c0 0x7fd4700a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:22.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.194+0000 7fd488c00640 1 -- 172.21.15.111:0/689007930 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fd4840cf500 con 0x7fd48c108f60 2024-09-15T22:46:22.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.194+0000 7fd48b400640 1 --2- 172.21.15.111:0/689007930 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd4700a32c0 0x7fd4700a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:22.195 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.194+0000 7fd48b400640 1 --2- 172.21.15.111:0/689007930 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd4700a32c0 0x7fd4700a5780 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009bd0 tx=0x7fd47c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:22.199 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.198+0000 7fd488c00640 1 -- 172.21.15.111:0/689007930 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd48409bbd0 con 0x7fd48c108f60 2024-09-15T22:46:22.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.409+0000 7fd490e00640 1 -- 172.21.15.111:0/689007930 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd48c10c1a0 con 0x7fd48c108f60 2024-09-15T22:46:22.413 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.410+0000 7fd488c00640 1 -- 172.21.15.111:0/689007930 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7fd48409fed0 con 0x7fd48c108f60 2024-09-15T22:46:22.413 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-15T22:46:22.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.412+0000 7fd490e00640 1 -- 172.21.15.111:0/689007930 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd4700a32c0 msgr2=0x7fd4700a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:22.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.412+0000 7fd490e00640 1 --2- 172.21.15.111:0/689007930 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd4700a32c0 0x7fd4700a5780 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009bd0 tx=0x7fd47c005f90 comp rx=0 tx=0).stop 2024-09-15T22:46:22.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.412+0000 7fd490e00640 1 -- 172.21.15.111:0/689007930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd48c108f60 msgr2=0x7fd48c19ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:22.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.412+0000 7fd490e00640 1 --2- 172.21.15.111:0/689007930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd48c108f60 0x7fd48c19ed70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd484009fd0 tx=0x7fd48400b260 comp rx=0 tx=0).stop 2024-09-15T22:46:22.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.412+0000 7fd490e00640 1 -- 172.21.15.111:0/689007930 shutdown_connections 2024-09-15T22:46:22.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.412+0000 7fd490e00640 1 --2- 172.21.15.111:0/689007930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd48c108f60 0x7fd48c19ed70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:22.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.412+0000 7fd490e00640 1 --2- 172.21.15.111:0/689007930 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd4700a32c0 0x7fd4700a5780 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:22.414 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.412+0000 7fd490e00640 1 --2- 172.21.15.111:0/689007930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd48c1047e0 0x7fd48c19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:22.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.412+0000 7fd490e00640 1 --2- 172.21.15.111:0/689007930 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd48c103e30 0x7fd48c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:22.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.412+0000 7fd490e00640 1 -- 172.21.15.111:0/689007930 >> 172.21.15.111:0/689007930 conn(0x7fd48c0ff8d0 msgr2=0x7fd48c100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:22.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.413+0000 7fd490e00640 1 -- 172.21.15.111:0/689007930 shutdown_connections 2024-09-15T22:46:22.415 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:22.413+0000 7fd490e00640 1 -- 172.21.15.111:0/689007930 wait complete. 2024-09-15T22:46:23.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:22 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/689007930' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-15T22:46:23.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:22 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/689007930' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-15T22:46:23.147 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:22 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/689007930' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-15T22:46:23.186 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-15T22:46:23.186 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-15T22:46:23.187 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2024-09-15T22:46:23.187 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-15T22:46:23.187 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-15T22:46:23.187 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-15T22:46:23.187 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-15T22:46:23.187 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-15T22:46:23.188 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-15T22:46:23.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:46:23.188 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2024-09-15T22:46:23.224 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-15T22:46:23.224 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-15T22:46:23.224 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2024-09-15T22:46:23.224 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-15T22:46:23.224 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2024-09-15T22:46:23.225 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-15T22:46:23.225 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T22:46:23.225 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2024-09-15T22:46:23.225 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2024-09-15T22:46:23.225 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T22:46:23.225 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2024-09-15T22:46:23.225 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2024-09-15T22:46:23.225 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T22:46:23.225 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2024-09-15T22:46:23.225 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-15T22:46:23.225 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2024-09-15T22:46:23.226 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-15T22:46:23.226 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42106sec preferred_lft 42106sec 2024-09-15T22:46:23.226 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link noprefixroute 2024-09-15T22:46:23.226 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2024-09-15T22:46:23.229 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-15T22:46:23.229 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:46:23.229 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-09-15T22:46:23.229 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add name ceph-brx type bridge 2024-09-15T22:46:23.229 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip addr flush dev ceph-brx 2024-09-15T22:46:23.229 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set ceph-brx up 2024-09-15T22:46:23.229 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-15T22:46:23.229 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-09-15T22:46:23.342 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:46:23.689 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:46:23.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:46:23.698 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-15T22:46:23.754 INFO:teuthology.orchestra.run.smithi111.stdout:1 2024-09-15T22:46:23.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:46:23.758 DEBUG:teuthology.orchestra.run.smithi111:> route 2024-09-15T22:46:23.825 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2024-09-15T22:46:23.825 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-15T22:46:23.825 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-15T22:46:23.825 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-15T22:46:23.825 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-15T22:46:23.825 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:46:23.826 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-09-15T22:46:23.826 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-15T22:46:23.826 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-15T22:46:23.826 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-15T22:46:23.826 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-09-15T22:46:23.903 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:46:23.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:23 smithi111 ceph-mon[30629]: pgmap v364: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:46:24.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:23 smithi192 ceph-mon[35427]: pgmap v364: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:46:24.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:23 smithi121 ceph-mon[34815]: pgmap v364: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:46:24.431 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:46:24.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:46:24.438 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2024-09-15T22:46:24.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:46:24.475 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list-id 2024-09-15T22:46:24.554 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:46:24.554 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-09-15T22:46:24.554 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-15T22:46:24.554 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-15T22:46:24.554 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-09-15T22:46:24.673 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:46:24.726 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:46:24.735 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-15T22:46:24.735 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:46:24.735 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-09-15T22:46:24.735 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-15T22:46:24.735 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-15T22:46:24.736 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-15T22:46:24.736 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-15T22:46:24.736 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-15T22:46:24.736 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-09-15T22:46:24.814 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:46:24.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:46:24.991 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:46:24.991 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2024-09-15T22:46:24.991 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2024-09-15T22:46:24.991 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set dev brx.0 master ceph-brx 2024-09-15T22:46:24.991 DEBUG:teuthology.orchestra.run.smithi111:> ') 2024-09-15T22:46:25.069 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:46:25.149 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:46:25.157 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T22:46:25.157 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-15T22:46:25.193 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-15T22:46:25.193 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T22:46:25.193 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-15T22:46:25.268 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-15T22:46:25.268 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-15T22:46:25.268 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/507a8ade-25bd-410d-99a3-61a663cb7a66 2024-09-15T22:46:25.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:46:25.269 DEBUG:teuthology.orchestra.run.smithi111:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/507a8ade-25bd-410d-99a3-61a663cb7a66 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-09-15T22:46:25.416 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-09-15T22:46:25.416 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-09-15T22:46:25.416 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2024-09-15T22:46:26.008 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:25 smithi111 ceph-mon[30629]: pgmap v365: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:46:26.023 INFO:teuthology.orchestra.run.smithi111.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-09-15T22:46:26.023 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-09-15T22:46:26.023 INFO:teuthology.orchestra.run.smithi111.stdout:invalid new device string format 2024-09-15T22:46:26.023 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: resolved to: "172.21.15.111:6789,172.21.15.121:6789,172.21.15.192:6789" 2024-09-15T22:46:26.023 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.111:6789,172.21.15.121:6789,172.21.15.192:6789:/volumes/qa/sv_1/507a8ade-25bd-410d-99a3-61a663cb7a66 2024-09-15T22:46:26.023 INFO:teuthology.orchestra.run.smithi111.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=b7614d3c-73b2-11ef-bceb-c7b262605968" will pass to kernel 2024-09-15T22:46:26.026 INFO:teuthology.orchestra.run.smithi111.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-15T22:46:26.026 INFO:teuthology.orchestra.run.smithi111.stdout: You just mounted a file system that supports labels which does not 2024-09-15T22:46:26.026 INFO:teuthology.orchestra.run.smithi111.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-15T22:46:26.026 INFO:teuthology.orchestra.run.smithi111.stdout: applications will generate AVC messages and not be allowed access to 2024-09-15T22:46:26.026 INFO:teuthology.orchestra.run.smithi111.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-15T22:46:26.029 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-15T22:46:26.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:46:26.029 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-15T22:46:26.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:25 smithi192 ceph-mon[35427]: pgmap v365: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:46:26.088 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T22:46:26.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:25 smithi121 ceph-mon[34815]: pgmap v365: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:46:26.170 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T22:46:26.174 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:46:26.181 INFO:teuthology.orchestra.run.smithi111.stdout:client15021 2024-09-15T22:46:26.182 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-15T22:46:26.182 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T22:46:26.218 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T22:46:26.337 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T22:46:26.341 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:46:26.349 INFO:teuthology.orchestra.run.smithi111.stdout:client15021 2024-09-15T22:46:26.349 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T22:46:26.427 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T22:46:26.430 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:46:26.438 INFO:teuthology.orchestra.run.smithi111.stdout:client15021 2024-09-15T22:46:26.439 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968.client15021/status') 2024-09-15T22:46:26.518 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:46:26.539 INFO:teuthology.orchestra.run.smithi111.stdout:instance: client.15021 (3)192.168.0.1:0/3542530795 2024-09-15T22:46:26.539 INFO:teuthology.orchestra.run.smithi111.stdout:blocklisted: false 2024-09-15T22:46:26.540 INFO:teuthology.orchestra.run.smithi111.stderr:0+1 records in 2024-09-15T22:46:26.540 INFO:teuthology.orchestra.run.smithi111.stderr:0+1 records out 2024-09-15T22:46:26.540 INFO:teuthology.orchestra.run.smithi111.stderr:70 bytes copied, 7.1648e-05 s, 977 kB/s 2024-09-15T22:46:26.542 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:46:26.550 INFO:teuthology.run_tasks:Running task exec... 2024-09-15T22:46:27.019 INFO:teuthology.task.exec:Executing custom commands... 2024-09-15T22:46:27.019 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi111.front.sepia.ceph.com 2024-09-15T22:46:27.020 DEBUG:teuthology.orchestra.run.smithi111:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-15T22:46:27.177 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.172+0000 7f7039000640 1 -- 172.21.15.111:0/1347384155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7034101b40 msgr2=0x7f7034101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:27.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.172+0000 7f7039000640 1 --2- 172.21.15.111:0/1347384155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7034101b40 0x7f7034101f40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7024009900 tx=0x7f702402a410 comp rx=0 tx=0).stop 2024-09-15T22:46:27.178 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.172+0000 7f7039000640 1 -- 172.21.15.111:0/1347384155 shutdown_connections 2024-09-15T22:46:27.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.172+0000 7f7039000640 1 --2- 172.21.15.111:0/1347384155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70341036f0 0x7f7034109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:27.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.172+0000 7f7039000640 1 --2- 172.21.15.111:0/1347384155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7034102d30 0x7f70341031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:27.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.172+0000 7f7039000640 1 --2- 172.21.15.111:0/1347384155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7034101b40 0x7f7034101f40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:27.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.172+0000 7f7039000640 1 -- 172.21.15.111:0/1347384155 >> 172.21.15.111:0/1347384155 conn(0x7f70340fd2e0 msgr2=0x7f70340ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:27.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.172+0000 7f7039000640 1 -- 172.21.15.111:0/1347384155 shutdown_connections 2024-09-15T22:46:27.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.172+0000 7f7039000640 1 -- 172.21.15.111:0/1347384155 wait complete. 2024-09-15T22:46:27.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.173+0000 7f7039000640 1 Processor -- start 2024-09-15T22:46:27.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.173+0000 7f7039000640 1 -- start start 2024-09-15T22:46:27.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.174+0000 7f7039000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7034101b40 0x7f7034075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:27.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.174+0000 7f7039000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7034102d30 0x7f70340745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:27.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.174+0000 7f7039000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70341036f0 0x7f7034074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:27.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.174+0000 7f7039000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f703410d7a0 con 0x7f7034102d30 2024-09-15T22:46:27.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.174+0000 7f7039000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f703410d620 con 0x7f7034101b40 2024-09-15T22:46:27.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.174+0000 7f7039000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f703410d920 con 0x7f70341036f0 2024-09-15T22:46:27.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.174+0000 7f7029400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7034102d30 0x7f70340745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:27.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.174+0000 7f7029400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7034102d30 0x7f70340745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46860/0 (socket says 172.21.15.111:46860) 2024-09-15T22:46:27.180 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.174+0000 7f7029400640 1 -- 172.21.15.111:0/1990170896 learned_addr learned my addr 172.21.15.111:0/1990170896 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:27.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.174+0000 7f7031e00640 1 --2- 172.21.15.111:0/1990170896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7034101b40 0x7f7034075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:27.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.174+0000 7f7032800640 1 --2- 172.21.15.111:0/1990170896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70341036f0 0x7f7034074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:27.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.174+0000 7f7029400640 1 -- 172.21.15.111:0/1990170896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70341036f0 msgr2=0x7f7034074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:27.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.174+0000 7f7029400640 1 --2- 172.21.15.111:0/1990170896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70341036f0 0x7f7034074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:27.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.174+0000 7f7029400640 1 -- 172.21.15.111:0/1990170896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7034101b40 msgr2=0x7f7034075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:27.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.174+0000 7f7029400640 1 --2- 172.21.15.111:0/1990170896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7034101b40 0x7f7034075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:27.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.174+0000 7f7029400640 1 -- 172.21.15.111:0/1990170896 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70340753b0 con 0x7f7034102d30 2024-09-15T22:46:27.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.175+0000 7f7029400640 1 --2- 172.21.15.111:0/1990170896 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7034102d30 0x7f70340745e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f701c00e9f0 tx=0x7f701c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:27.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.175+0000 7f702b400640 1 -- 172.21.15.111:0/1990170896 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c00cd80 con 0x7f7034102d30 2024-09-15T22:46:27.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.175+0000 7f702b400640 1 -- 172.21.15.111:0/1990170896 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f701c010910 con 0x7f7034102d30 2024-09-15T22:46:27.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.175+0000 7f7039000640 1 -- 172.21.15.111:0/1990170896 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70341a1ec0 con 0x7f7034102d30 2024-09-15T22:46:27.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.175+0000 7f702b400640 1 -- 172.21.15.111:0/1990170896 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c018990 con 0x7f7034102d30 2024-09-15T22:46:27.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.175+0000 7f7039000640 1 -- 172.21.15.111:0/1990170896 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70341a2320 con 0x7f7034102d30 2024-09-15T22:46:27.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.176+0000 7f7039000640 1 -- 172.21.15.111:0/1990170896 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7034108f00 con 0x7f7034102d30 2024-09-15T22:46:27.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.178+0000 7f702b400640 1 -- 172.21.15.111:0/1990170896 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f701c010ac0 con 0x7f7034102d30 2024-09-15T22:46:27.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.179+0000 7f702b400640 1 --2- 172.21.15.111:0/1990170896 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f70180a32f0 0x7f70180a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:27.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.179+0000 7f702b400640 1 -- 172.21.15.111:0/1990170896 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f701c0cfba0 con 0x7f7034102d30 2024-09-15T22:46:27.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.179+0000 7f7031e00640 1 --2- 172.21.15.111:0/1990170896 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f70180a32f0 0x7f70180a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:27.183 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.180+0000 7f7031e00640 1 --2- 172.21.15.111:0/1990170896 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f70180a32f0 0x7f70180a57b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7024005ab0 tx=0x7f7024000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:27.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.183+0000 7f702b400640 1 -- 172.21.15.111:0/1990170896 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f701c09c270 con 0x7f7034102d30 2024-09-15T22:46:27.396 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.395+0000 7f7039000640 1 -- 172.21.15.111:0/1990170896 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f7034062ef0 con 0x7f7034102d30 2024-09-15T22:46:27.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.709+0000 7f702b400640 1 -- 172.21.15.111:0/1990170896 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f701c0a0570 con 0x7f7034102d30 2024-09-15T22:46:27.716 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.713+0000 7f7039000640 1 -- 172.21.15.111:0/1990170896 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f70180a32f0 msgr2=0x7f70180a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:27.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.713+0000 7f7039000640 1 --2- 172.21.15.111:0/1990170896 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f70180a32f0 0x7f70180a57b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7024005ab0 tx=0x7f7024000bc0 comp rx=0 tx=0).stop 2024-09-15T22:46:27.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.713+0000 7f7039000640 1 -- 172.21.15.111:0/1990170896 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7034102d30 msgr2=0x7f70340745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:27.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.713+0000 7f7039000640 1 --2- 172.21.15.111:0/1990170896 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7034102d30 0x7f70340745e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f701c00e9f0 tx=0x7f701c00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:46:27.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.713+0000 7f7039000640 1 -- 172.21.15.111:0/1990170896 shutdown_connections 2024-09-15T22:46:27.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.713+0000 7f7039000640 1 --2- 172.21.15.111:0/1990170896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70341036f0 0x7f7034074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:27.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.713+0000 7f7039000640 1 --2- 172.21.15.111:0/1990170896 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7034102d30 0x7f70340745e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:27.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.713+0000 7f7039000640 1 --2- 172.21.15.111:0/1990170896 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f70180a32f0 0x7f70180a57b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:27.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.713+0000 7f7039000640 1 --2- 172.21.15.111:0/1990170896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7034101b40 0x7f7034075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:27.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.713+0000 7f7039000640 1 -- 172.21.15.111:0/1990170896 >> 172.21.15.111:0/1990170896 conn(0x7f70340fd2e0 msgr2=0x7f70341060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:27.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.713+0000 7f7039000640 1 -- 172.21.15.111:0/1990170896 shutdown_connections 2024-09-15T22:46:27.717 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.713+0000 7f7039000640 1 -- 172.21.15.111:0/1990170896 wait complete. 2024-09-15T22:46:27.731 DEBUG:teuthology.orchestra.run.smithi111:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-15T22:46:27.884 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.883+0000 7f8fbee00640 1 -- 172.21.15.111:0/2648732466 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8fb8101b60 msgr2=0x7f8fb8101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:27.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.883+0000 7f8fbee00640 1 --2- 172.21.15.111:0/2648732466 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8fb8101b60 0x7f8fb8101f60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8009900 tx=0x7f8fa802a410 comp rx=0 tx=0).stop 2024-09-15T22:46:27.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.883+0000 7f8fbee00640 1 -- 172.21.15.111:0/2648732466 shutdown_connections 2024-09-15T22:46:27.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.883+0000 7f8fbee00640 1 --2- 172.21.15.111:0/2648732466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fb8103710 0x7f8fb8109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:27.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.883+0000 7f8fbee00640 1 --2- 172.21.15.111:0/2648732466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fb8102d50 0x7f8fb81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:27.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.883+0000 7f8fbee00640 1 --2- 172.21.15.111:0/2648732466 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8fb8101b60 0x7f8fb8101f60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:27.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.883+0000 7f8fbee00640 1 -- 172.21.15.111:0/2648732466 >> 172.21.15.111:0/2648732466 conn(0x7f8fb80fd2e0 msgr2=0x7f8fb80ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:27.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.883+0000 7f8fbee00640 1 -- 172.21.15.111:0/2648732466 shutdown_connections 2024-09-15T22:46:27.885 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.883+0000 7f8fbee00640 1 -- 172.21.15.111:0/2648732466 wait complete. 2024-09-15T22:46:27.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.884+0000 7f8fbee00640 1 Processor -- start 2024-09-15T22:46:27.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.884+0000 7f8fbee00640 1 -- start start 2024-09-15T22:46:27.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.884+0000 7f8fbee00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fb8101b60 0x7f8fb8197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:27.887 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.884+0000 7f8fbee00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fb8102d50 0x7f8fb8198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:27.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.884+0000 7f8fbee00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8fb8103710 0x7f8fb819f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:27.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.884+0000 7f8fbee00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb810d7e0 con 0x7f8fb8103710 2024-09-15T22:46:27.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.884+0000 7f8fbee00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb810d660 con 0x7f8fb8102d50 2024-09-15T22:46:27.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.884+0000 7f8fbee00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb810d960 con 0x7f8fb8101b60 2024-09-15T22:46:27.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.884+0000 7f8fafe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fb8102d50 0x7f8fb8198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:27.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.884+0000 7f8fbce00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8fb8103710 0x7f8fb819f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:27.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.884+0000 7f8fafe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fb8102d50 0x7f8fb8198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:60442/0 (socket says 172.21.15.111:60442) 2024-09-15T22:46:27.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.884+0000 7f8fafe00640 1 -- 172.21.15.111:0/2295050888 learned_addr learned my addr 172.21.15.111:0/2295050888 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:27.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.885+0000 7f8fb7e00640 1 --2- 172.21.15.111:0/2295050888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fb8101b60 0x7f8fb8197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:27.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.885+0000 7f8fafe00640 1 -- 172.21.15.111:0/2295050888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fb8101b60 msgr2=0x7f8fb8197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:27.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.885+0000 7f8fafe00640 1 --2- 172.21.15.111:0/2295050888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fb8101b60 0x7f8fb8197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:27.888 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.885+0000 7f8fafe00640 1 -- 172.21.15.111:0/2295050888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8fb8103710 msgr2=0x7f8fb819f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:27.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.885+0000 7f8fafe00640 1 --2- 172.21.15.111:0/2295050888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8fb8103710 0x7f8fb819f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:27.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.885+0000 7f8fafe00640 1 -- 172.21.15.111:0/2295050888 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb819f9f0 con 0x7f8fb8102d50 2024-09-15T22:46:27.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.885+0000 7f8fbce00640 1 --2- 172.21.15.111:0/2295050888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8fb8103710 0x7f8fb819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:46:27.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.885+0000 7f8fafe00640 1 --2- 172.21.15.111:0/2295050888 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fb8102d50 0x7f8fb8198310 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000e9f0 tx=0x7f8fa000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:27.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.885+0000 7f8fb5600640 1 -- 172.21.15.111:0/2295050888 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa000cd80 con 0x7f8fb8102d50 2024-09-15T22:46:27.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.885+0000 7f8fbee00640 1 -- 172.21.15.111:0/2295050888 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb819fce0 con 0x7f8fb8102d50 2024-09-15T22:46:27.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.885+0000 7f8fb5600640 1 -- 172.21.15.111:0/2295050888 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fa0010910 con 0x7f8fb8102d50 2024-09-15T22:46:27.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.885+0000 7f8fb5600640 1 -- 172.21.15.111:0/2295050888 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa00188f0 con 0x7f8fb8102d50 2024-09-15T22:46:27.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.885+0000 7f8fbee00640 1 -- 172.21.15.111:0/2295050888 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb81a01d0 con 0x7f8fb8102d50 2024-09-15T22:46:27.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.886+0000 7f8fbee00640 1 -- 172.21.15.111:0/2295050888 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb8108f00 con 0x7f8fb8102d50 2024-09-15T22:46:27.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.888+0000 7f8fb5600640 1 -- 172.21.15.111:0/2295050888 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 138356+0+0 (secure 0 0 0) 0x7f8fa0010ac0 con 0x7f8fb8102d50 2024-09-15T22:46:27.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.889+0000 7f8fb5600640 1 --2- 172.21.15.111:0/2295050888 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8f9c0a32f0 0x7f8f9c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:27.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.890+0000 7f8fb5600640 1 -- 172.21.15.111:0/2295050888 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f8fa00cffe0 con 0x7f8fb8102d50 2024-09-15T22:46:27.891 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.890+0000 7f8fb7e00640 1 --2- 172.21.15.111:0/2295050888 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8f9c0a32f0 0x7f8f9c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:27.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.890+0000 7f8fb7e00640 1 --2- 172.21.15.111:0/2295050888 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8f9c0a32f0 0x7f8f9c0a57b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8005ab0 tx=0x7f8fa8000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:27.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:27.892+0000 7f8fb5600640 1 -- 172.21.15.111:0/2295050888 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8fa009c6b0 con 0x7f8fb8102d50 2024-09-15T22:46:28.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:27 smithi192 ceph-mon[35427]: pgmap v366: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:46:28.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:27 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/1990170896' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-15T22:46:28.092 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:28.091+0000 7f8fbee00640 1 -- 172.21.15.111:0/2295050888 --> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f8fb8062790 con 0x7f8f9c0a32f0 2024-09-15T22:46:28.099 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:27 smithi111 ceph-mon[30629]: pgmap v366: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:46:28.099 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:27 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/1990170896' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-15T22:46:28.112 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:28.111+0000 7f8fb5600640 1 -- 172.21.15.111:0/2295050888 <== mgr.14148 v2:172.21.15.111:6800/3125866964 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f8fb8062790 con 0x7f8f9c0a32f0 2024-09-15T22:46:28.113 INFO:teuthology.orchestra.run.smithi111.stdout:{} 2024-09-15T22:46:28.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:28.113+0000 7f8fbee00640 1 -- 172.21.15.111:0/2295050888 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8f9c0a32f0 msgr2=0x7f8f9c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:28.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:28.113+0000 7f8fbee00640 1 --2- 172.21.15.111:0/2295050888 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8f9c0a32f0 0x7f8f9c0a57b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8005ab0 tx=0x7f8fa8000bc0 comp rx=0 tx=0).stop 2024-09-15T22:46:28.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:28.113+0000 7f8fbee00640 1 -- 172.21.15.111:0/2295050888 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fb8102d50 msgr2=0x7f8fb8198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:28.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:28.113+0000 7f8fbee00640 1 --2- 172.21.15.111:0/2295050888 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fb8102d50 0x7f8fb8198310 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000e9f0 tx=0x7f8fa000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:46:28.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:28.113+0000 7f8fbee00640 1 -- 172.21.15.111:0/2295050888 shutdown_connections 2024-09-15T22:46:28.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:28.113+0000 7f8fbee00640 1 --2- 172.21.15.111:0/2295050888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8fb8103710 0x7f8fb819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:28.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:28.113+0000 7f8fbee00640 1 --2- 172.21.15.111:0/2295050888 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8f9c0a32f0 0x7f8f9c0a57b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:28.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:28.113+0000 7f8fbee00640 1 --2- 172.21.15.111:0/2295050888 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fb8102d50 0x7f8fb8198310 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:28.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:28.113+0000 7f8fbee00640 1 --2- 172.21.15.111:0/2295050888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fb8101b60 0x7f8fb8197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:28.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:28.113+0000 7f8fbee00640 1 -- 172.21.15.111:0/2295050888 >> 172.21.15.111:0/2295050888 conn(0x7f8fb80fd2e0 msgr2=0x7f8fb81060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:28.115 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:28.113+0000 7f8fbee00640 1 -- 172.21.15.111:0/2295050888 shutdown_connections 2024-09-15T22:46:28.116 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T22:46:28.113+0000 7f8fbee00640 1 -- 172.21.15.111:0/2295050888 wait complete. 2024-09-15T22:46:28.131 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-15T22:46:28.142 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-15T22:46:28.200 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi111.front.sepia.ceph.com 2024-09-15T22:46:28.200 DEBUG:teuthology.orchestra.run.smithi111:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-15T22:46:28.201 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:27 smithi121 ceph-mon[34815]: pgmap v366: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:46:28.201 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:27 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/1990170896' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-15T22:46:28.205 INFO:teuthology.run_tasks:Running task workunit... 2024-09-15T22:46:28.220 INFO:tasks.workunit:Pulling workunits from ref ad35db06e8924912f01e32b35576297c0f1352b4 2024-09-15T22:46:28.220 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-15T22:46:28.220 INFO:tasks.workunit:timeout=3h 2024-09-15T22:46:28.220 INFO:tasks.workunit:cleanup=True 2024-09-15T22:46:28.221 DEBUG:teuthology.orchestra.run.smithi111:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-15T22:46:28.258 INFO:teuthology.orchestra.run.smithi111.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-15T22:46:28.258 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-15T22:46:28.258 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-15T22:46:28.259 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:46:28.259 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-15T22:46:28.259 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2024-09-15 22:44:42.929389964 +0000 2024-09-15T22:46:28.259 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2024-09-15 22:44:42.929389964 +0000 2024-09-15T22:46:28.259 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2024-09-15 22:46:26.081925171 +0000 2024-09-15T22:46:28.259 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2024-09-15 22:44:42.929389964 +0000 2024-09-15T22:46:28.259 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-15T22:46:28.259 DEBUG:teuthology.orchestra.run.smithi111:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-15T22:46:28.351 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:46:28 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:46:28.096+0000 7f296d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:46:28.351 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:46:28 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:46:28.096+0000 7f296d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:46:28.351 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:46:28 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:46:28.096+0000 7f296d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:46:28.351 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:46:28 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:46:28.097+0000 7f296d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:46:28.351 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 22:46:28 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T22:46:28.097+0000 7f296d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:46:28.370 DEBUG:teuthology.orchestra.run.smithi111:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ad35db06e8924912f01e32b35576297c0f1352b4 2024-09-15T22:46:28.408 INFO:tasks.workunit.client.0.smithi111.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-15T22:46:29.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:28 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/1990170896' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-15T22:46:29.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:28 smithi192 ceph-mon[35427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:46:29.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:28 smithi192 ceph-mon[35427]: from='client.25061 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:46:29.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:28 smithi192 ceph-mon[35427]: pgmap v367: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:46:29.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:28 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/1990170896' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-15T22:46:29.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:28 smithi121 ceph-mon[34815]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:46:29.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:28 smithi121 ceph-mon[34815]: from='client.25061 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:46:29.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:28 smithi121 ceph-mon[34815]: pgmap v367: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:46:29.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:28 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/1990170896' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-15T22:46:29.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:28 smithi111 ceph-mon[30629]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:46:29.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:28 smithi111 ceph-mon[30629]: from='client.25061 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:46:29.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:28 smithi111 ceph-mon[30629]: pgmap v367: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:46:30.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:29 smithi192 ceph-mon[35427]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-15T22:46:30.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:29 smithi121 ceph-mon[34815]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-15T22:46:30.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:29 smithi111 ceph-mon[30629]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-15T22:46:31.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:30 smithi192 ceph-mon[35427]: pgmap v368: 97 pgs: 97 active+clean; 643 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-15T22:46:31.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:30 smithi121 ceph-mon[34815]: pgmap v368: 97 pgs: 97 active+clean; 643 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-15T22:46:31.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:30 smithi111 ceph-mon[30629]: pgmap v368: 97 pgs: 97 active+clean; 643 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-15T22:46:33.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:33 smithi192 ceph-mon[35427]: pgmap v369: 97 pgs: 97 active+clean; 655 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-15T22:46:33.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:33 smithi121 ceph-mon[34815]: pgmap v369: 97 pgs: 97 active+clean; 655 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-15T22:46:33.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:33 smithi111 ceph-mon[30629]: pgmap v369: 97 pgs: 97 active+clean; 655 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-15T22:46:34.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:34 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:46:34.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:34 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:46:34.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:34 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:46:35.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:35 smithi192 ceph-mon[35427]: pgmap v370: 97 pgs: 97 active+clean; 655 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-15T22:46:35.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:35 smithi121 ceph-mon[34815]: pgmap v370: 97 pgs: 97 active+clean; 655 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-15T22:46:35.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:35 smithi111 ceph-mon[30629]: pgmap v370: 97 pgs: 97 active+clean; 655 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-15T22:46:37.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:36 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:46:37.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:36 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:46:37.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:36 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:46:37.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:36 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:46:37.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:36 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:46:37.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:36 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:46:37.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:36 smithi192 ceph-mon[35427]: pgmap v371: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-15T22:46:37.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:36 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:46:37.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:36 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:46:37.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:36 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:46:37.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:36 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:46:37.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:36 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:46:37.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:36 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:46:37.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:36 smithi121 ceph-mon[34815]: pgmap v371: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-15T22:46:37.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:36 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:46:37.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:36 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:46:37.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:36 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:46:37.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:36 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:46:37.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:36 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:46:37.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:36 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:46:37.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:36 smithi111 ceph-mon[30629]: pgmap v371: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-15T22:46:38.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:37 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:46:38.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:37 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:46:38.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:37 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:46:38.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:46:38.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:46:38.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:46:38.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:37 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:46:38.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:37 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:46:38.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:37 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:46:39.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:38 smithi192 ceph-mon[35427]: pgmap v372: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-15T22:46:39.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:38 smithi121 ceph-mon[34815]: pgmap v372: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-15T22:46:39.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:38 smithi111 ceph-mon[30629]: pgmap v372: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-09-15T22:46:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.403+0000 7f183ee00640 1 -- 172.21.15.111:0/4174515959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1838101580 msgr2=0x7f1838101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.403+0000 7f183ee00640 1 --2- 172.21.15.111:0/4174515959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1838101580 0x7f1838101a00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1824009900 tx=0x7f182402a410 comp rx=0 tx=0).stop 2024-09-15T22:46:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.403+0000 7f183ee00640 1 -- 172.21.15.111:0/4174515959 shutdown_connections 2024-09-15T22:46:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.403+0000 7f183ee00640 1 --2- 172.21.15.111:0/4174515959 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1838101f40 0x7f1838109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.403+0000 7f183ee00640 1 --2- 172.21.15.111:0/4174515959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1838101580 0x7f1838101a00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.403+0000 7f183ee00640 1 --2- 172.21.15.111:0/4174515959 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1838103050 0x7f1838103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.403+0000 7f183ee00640 1 -- 172.21.15.111:0/4174515959 >> 172.21.15.111:0/4174515959 conn(0x7f18380fd0d0 msgr2=0x7f18380ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.404+0000 7f183ee00640 1 -- 172.21.15.111:0/4174515959 shutdown_connections 2024-09-15T22:46:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.404+0000 7f183ee00640 1 -- 172.21.15.111:0/4174515959 wait complete. 2024-09-15T22:46:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.404+0000 7f183ee00640 1 Processor -- start 2024-09-15T22:46:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.405+0000 7f183ee00640 1 -- start start 2024-09-15T22:46:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.405+0000 7f183ee00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1838101580 0x7f1838197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.405+0000 7f183ee00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1838101f40 0x7f1838198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.405+0000 7f183ee00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1838103050 0x7f183819f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.405+0000 7f183ee00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f183810d6d0 con 0x7f1838103050 2024-09-15T22:46:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.405+0000 7f183ee00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f183810d550 con 0x7f1838101f40 2024-09-15T22:46:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.405+0000 7f183ee00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f183810d850 con 0x7f1838101580 2024-09-15T22:46:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.405+0000 7f183e400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1838103050 0x7f183819f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.405+0000 7f183e400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1838103050 0x7f183819f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55430/0 (socket says 172.21.15.111:55430) 2024-09-15T22:46:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.405+0000 7f183e400640 1 -- 172.21.15.111:0/1925959060 learned_addr learned my addr 172.21.15.111:0/1925959060 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.405+0000 7f183da00640 1 --2- 172.21.15.111:0/1925959060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1838101580 0x7f1838197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.405+0000 7f183d000640 1 --2- 172.21.15.111:0/1925959060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1838101f40 0x7f1838198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.406+0000 7f183e400640 1 -- 172.21.15.111:0/1925959060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1838101580 msgr2=0x7f1838197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.406+0000 7f183e400640 1 --2- 172.21.15.111:0/1925959060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1838101580 0x7f1838197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.406+0000 7f183e400640 1 -- 172.21.15.111:0/1925959060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1838101f40 msgr2=0x7f1838198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.406+0000 7f183e400640 1 --2- 172.21.15.111:0/1925959060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1838101f40 0x7f1838198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.406+0000 7f183e400640 1 -- 172.21.15.111:0/1925959060 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f183819fa00 con 0x7f1838103050 2024-09-15T22:46:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.406+0000 7f183e400640 1 --2- 172.21.15.111:0/1925959060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1838103050 0x7f183819f390 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f183000eb10 tx=0x7f183000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.406+0000 7f1836000640 1 -- 172.21.15.111:0/1925959060 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1830004b20 con 0x7f1838103050 2024-09-15T22:46:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.406+0000 7f1836000640 1 -- 172.21.15.111:0/1925959060 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1830004cc0 con 0x7f1838103050 2024-09-15T22:46:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.406+0000 7f183ee00640 1 -- 172.21.15.111:0/1925959060 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f183819fcf0 con 0x7f1838103050 2024-09-15T22:46:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.406+0000 7f1836000640 1 -- 172.21.15.111:0/1925959060 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1830010760 con 0x7f1838103050 2024-09-15T22:46:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.406+0000 7f183ee00640 1 -- 172.21.15.111:0/1925959060 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18381a01c0 con 0x7f1838103050 2024-09-15T22:46:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.410+0000 7f184320a740 1 Processor -- start 2024-09-15T22:46:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.411+0000 7f1836000640 1 -- 172.21.15.111:0/1925959060 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f1830005020 con 0x7f1838103050 2024-09-15T22:46:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.411+0000 7f184320a740 1 -- start start 2024-09-15T22:46:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.411+0000 7f184320a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1670 0x5597c1bf1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.411+0000 7f184320a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597c1bf1fb0 0x5597c1bf6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.411+0000 7f184320a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597c1bf7040 0x5597c1bf9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.411+0000 7f184320a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5597c1b52100 con 0x5597c1bf1670 2024-09-15T22:46:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.411+0000 7f184320a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5597c1afa1b0 con 0x5597c1bf7040 2024-09-15T22:46:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.411+0000 7f184320a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5597c1b4e0c0 con 0x5597c1bf1fb0 2024-09-15T22:46:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.411+0000 7f183d000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1670 0x5597c1bf1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.411+0000 7f183d000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1670 0x5597c1bf1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55432/0 (socket says 172.21.15.111:55432) 2024-09-15T22:46:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.411+0000 7f183d000640 1 -- 172.21.15.111:0/524965603 learned_addr learned my addr 172.21.15.111:0/524965603 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.412+0000 7f183da00640 1 --2- 172.21.15.111:0/524965603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597c1bf1fb0 0x5597c1bf6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.412+0000 7f183d000640 1 --2- 172.21.15.111:0/524965603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597c1bf7040 0x5597c1bf9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.412+0000 7f183d000640 1 -- 172.21.15.111:0/524965603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597c1bf1fb0 msgr2=0x5597c1bf6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.412+0000 7f183d000640 1 --2- 172.21.15.111:0/524965603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597c1bf1fb0 0x5597c1bf6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.412+0000 7f183d000640 1 -- 172.21.15.111:0/524965603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597c1bf7040 msgr2=0x5597c1bf9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.412+0000 7f183d000640 1 --2- 172.21.15.111:0/524965603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597c1bf7040 0x5597c1bf9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.412+0000 7f183d000640 1 -- 172.21.15.111:0/524965603 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597c1865210 con 0x5597c1bf1670 2024-09-15T22:46:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.412+0000 7f183d000640 1 --2- 172.21.15.111:0/524965603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1670 0x5597c1bf1a70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1824009900 tx=0x7f1824005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3d5d6d7a8ec90b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.412+0000 7f1836000640 1 --2- 172.21.15.111:0/1925959060 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f18200a32d0 0x7f18200a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.413+0000 7f182aa00640 1 -- 172.21.15.111:0/524965603 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1824035070 con 0x5597c1bf1670 2024-09-15T22:46:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.413+0000 7f182aa00640 1 -- 172.21.15.111:0/524965603 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1824005030 con 0x5597c1bf1670 2024-09-15T22:46:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.413+0000 7f182aa00640 1 -- 172.21.15.111:0/524965603 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18240049e0 con 0x5597c1bf1670 2024-09-15T22:46:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.413+0000 7f1836000640 1 -- 172.21.15.111:0/1925959060 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f18300cfff0 con 0x7f1838103050 2024-09-15T22:46:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.413+0000 7f183da00640 1 --2- 172.21.15.111:0/1925959060 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f18200a32d0 0x7f18200a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.413+0000 7f184320a740 1 -- 172.21.15.111:0/524965603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1670 msgr2=0x5597c1bf1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.413+0000 7f184320a740 1 --2- 172.21.15.111:0/524965603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1670 0x5597c1bf1a70 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1824009900 tx=0x7f1824005fd0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.414+0000 7f183da00640 1 --2- 172.21.15.111:0/1925959060 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f18200a32d0 0x7f18200a5790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x5597c1bf29e0 tx=0x7f182c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.414+0000 7f184320a740 1 -- 172.21.15.111:0/524965603 shutdown_connections 2024-09-15T22:46:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.414+0000 7f184320a740 1 --2- 172.21.15.111:0/524965603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597c1bf7040 0x5597c1bf9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.414+0000 7f184320a740 1 --2- 172.21.15.111:0/524965603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597c1bf1fb0 0x5597c1bf6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.414+0000 7f184320a740 1 --2- 172.21.15.111:0/524965603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1670 0x5597c1bf1a70 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.414+0000 7f184320a740 1 -- 172.21.15.111:0/524965603 >> 172.21.15.111:0/524965603 conn(0x5597c1be88d0 msgr2=0x5597c1be8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.414+0000 7f184320a740 1 -- 172.21.15.111:0/524965603 shutdown_connections 2024-09-15T22:46:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.414+0000 7f184320a740 1 -- 172.21.15.111:0/524965603 wait complete. 2024-09-15T22:46:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.414+0000 7f184320a740 1 -- start start 2024-09-15T22:46:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.414+0000 7f184320a740 1 Processor -- start 2024-09-15T22:46:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.415+0000 7f184320a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1670 0x5597c1bfb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.415+0000 7f184320a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597c1bf1fb0 0x5597c1bfbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.415+0000 7f184320a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597c1bf7040 0x5597c1c02d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.415+0000 7f184320a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5597c1bf9fb0 con 0x5597c1bf1670 2024-09-15T22:46:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.415+0000 7f184320a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5597c1adb090 con 0x5597c1bf7040 2024-09-15T22:46:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.415+0000 7f184320a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5597c1bfa130 con 0x5597c1bf1fb0 2024-09-15T22:46:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.415+0000 7f184320a740 10 client.0 _ll_register_callbacks cb 0x5597c1b2ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.415+0000 7f183da00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1670 0x5597c1bfb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.420 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.415+0000 7f183da00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1670 0x5597c1bfb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55442/0 (socket says 172.21.15.111:55442) 2024-09-15T22:46:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.415+0000 7f183da00640 1 -- 172.21.15.111:0/2828993852 learned_addr learned my addr 172.21.15.111:0/2828993852 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.415+0000 7f183d000640 1 --2- 172.21.15.111:0/2828993852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597c1bf1fb0 0x5597c1bfbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.415+0000 7f183e400640 1 --2- 172.21.15.111:0/2828993852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597c1bf7040 0x5597c1c02d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.416+0000 7f183da00640 1 -- 172.21.15.111:0/2828993852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597c1bf1fb0 msgr2=0x5597c1bfbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.416+0000 7f183da00640 1 --2- 172.21.15.111:0/2828993852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597c1bf1fb0 0x5597c1bfbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.416+0000 7f183da00640 1 -- 172.21.15.111:0/2828993852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597c1bf7040 msgr2=0x5597c1c02d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.416+0000 7f183da00640 1 --2- 172.21.15.111:0/2828993852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597c1bf7040 0x5597c1c02d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.416+0000 7f183da00640 1 -- 172.21.15.111:0/2828993852 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597c1931620 con 0x5597c1bf1670 2024-09-15T22:46:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.416+0000 7f183da00640 1 --2- 172.21.15.111:0/2828993852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1670 0x5597c1bfb7b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f182c0058a0 tx=0x7f182c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f86c962f60be2b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.416+0000 7f181f400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:46:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.416+0000 7f181f400640 1 -- 172.21.15.111:0/2828993852 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182c01f630 con 0x5597c1bf1670 2024-09-15T22:46:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.416+0000 7f184320a740 1 -- 172.21.15.111:0/2828993852 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1824005030 con 0x5597c1bf1670 2024-09-15T22:46:41.421 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.416+0000 7f181f400640 1 -- 172.21.15.111:0/2828993852 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f182c01f7d0 con 0x5597c1bf1670 2024-09-15T22:46:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.417+0000 7f181f400640 1 -- 172.21.15.111:0/2828993852 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f182c01f9d0 con 0x5597c1bf1670 2024-09-15T22:46:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.417+0000 7f184320a740 10 client.15051 fetch_fsmap learned FSMap version 26 2024-09-15T22:46:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.417+0000 7f184320a740 1 -- 172.21.15.111:0/2828993852 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5597c1c05c40 con 0x5597c1bf1670 2024-09-15T22:46:41.422 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.417+0000 7f181f400640 1 -- 172.21.15.111:0/2828993852 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1824005030 con 0x5597c1bf1670 2024-09-15T22:46:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.417+0000 7f181f400640 1 -- 172.21.15.111:0/2828993852 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f182c027070 con 0x5597c1bf1670 2024-09-15T22:46:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.417+0000 7f184320a740 10 client.15051 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:46:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.417+0000 7f184320a740 10 client.15051 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:46:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.417+0000 7f184320a740 20 client.15051 populate_metadata read hostname 'smithi111' 2024-09-15T22:46:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.417+0000 7f184320a740 1 --2- 172.21.15.111:0/2828993852 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5597c1c072c0 0x5597c1c09730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.417+0000 7f184320a740 4 client.15051 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:46:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.417+0000 7f184320a740 1 -- 172.21.15.111:0/2828993852 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597c1b5f830 con 0x5597c1c072c0 2024-09-15T22:46:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.417+0000 7f183d000640 1 --2- 172.21.15.111:0/2828993852 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5597c1c072c0 0x5597c1c09730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.418+0000 7f183d000640 1 --2- 172.21.15.111:0/2828993852 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5597c1c072c0 0x5597c1c09730 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=920dea7157003aab server_cookie=232d0020b2135246 in_seq=0 out_seq=0 2024-09-15T22:46:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.418+0000 7f181f400640 10 client.15051 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:46:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.419+0000 7f181f400640 1 -- 172.21.15.111:0/2828993852 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5597c1b5f830 con 0x5597c1c072c0 2024-09-15T22:46:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.419+0000 7f181f400640 10 client.15051 handle_command_reply: tid=0 2024-09-15T22:46:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.419+0000 7f184320a740 1 client.15051 shutdown 2024-09-15T22:46:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.419+0000 7f184320a740 1 -- 172.21.15.111:0/2828993852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1670 msgr2=0x5597c1bfb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.419+0000 7f184320a740 1 --2- 172.21.15.111:0/2828993852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1670 0x5597c1bfb7b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f182c0058a0 tx=0x7f182c004730 comp rx=0 tx=0).stop 2024-09-15T22:46:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.419+0000 7f184320a740 1 -- 172.21.15.111:0/2828993852 shutdown_connections 2024-09-15T22:46:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.419+0000 7f184320a740 1 --2- 172.21.15.111:0/2828993852 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5597c1c072c0 0x5597c1c09730 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.419+0000 7f184320a740 1 --2- 172.21.15.111:0/2828993852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597c1bf7040 0x5597c1c02d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.419+0000 7f184320a740 1 --2- 172.21.15.111:0/2828993852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597c1bf1fb0 0x5597c1bfbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.419+0000 7f181f400640 0 client.15051 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:46:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.419+0000 7f184320a740 1 --2- 172.21.15.111:0/2828993852 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1670 0x5597c1bfb7b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.419+0000 7f184320a740 1 -- 172.21.15.111:0/2828993852 >> 172.21.15.111:0/2828993852 conn(0x5597c1be88d0 msgr2=0x5597c1bea0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.419+0000 7f184320a740 1 -- 172.21.15.111:0/2828993852 shutdown_connections 2024-09-15T22:46:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.419+0000 7f184320a740 1 -- 172.21.15.111:0/2828993852 wait complete. 2024-09-15T22:46:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.419+0000 7f184320a740 20 client.15051 trim_cache size 0 max 16384 2024-09-15T22:46:41.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.434+0000 7f184320a740 1 Processor -- start 2024-09-15T22:46:41.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.434+0000 7f184320a740 1 -- start start 2024-09-15T22:46:41.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.434+0000 7f184320a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597c1bf1670 0x5597c1bfbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.434+0000 7f184320a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1fb0 0x5597c1c02c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.434+0000 7f184320a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597c1bf7040 0x5597c1be9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.434+0000 7f184320a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f18381a9430 con 0x5597c1bf1fb0 2024-09-15T22:46:41.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.434+0000 7f184320a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f18381a6d00 con 0x5597c1bf7040 2024-09-15T22:46:41.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.434+0000 7f184320a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f18381a4b70 con 0x5597c1bf1670 2024-09-15T22:46:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.435+0000 7f183d000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1fb0 0x5597c1c02c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.435+0000 7f183d000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1fb0 0x5597c1c02c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55456/0 (socket says 172.21.15.111:55456) 2024-09-15T22:46:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.435+0000 7f183d000640 1 -- 172.21.15.111:0/1490672149 learned_addr learned my addr 172.21.15.111:0/1490672149 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.435+0000 7f183da00640 1 --2- 172.21.15.111:0/1490672149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597c1bf1670 0x5597c1bfbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.435+0000 7f183e400640 1 --2- 172.21.15.111:0/1490672149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597c1bf7040 0x5597c1be9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.435+0000 7f183d000640 1 -- 172.21.15.111:0/1490672149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597c1bf1670 msgr2=0x5597c1bfbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.435+0000 7f183d000640 1 --2- 172.21.15.111:0/1490672149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597c1bf1670 0x5597c1bfbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.435+0000 7f183d000640 1 -- 172.21.15.111:0/1490672149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597c1bf7040 msgr2=0x5597c1be9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.435+0000 7f183d000640 1 --2- 172.21.15.111:0/1490672149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597c1bf7040 0x5597c1be9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.435+0000 7f183d000640 1 -- 172.21.15.111:0/1490672149 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597c1be86f0 con 0x5597c1bf1fb0 2024-09-15T22:46:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.435+0000 7f183d000640 1 --2- 172.21.15.111:0/1490672149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1fb0 0x5597c1c02c50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1824000c00 tx=0x7f1824005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3faaf79ce6c295 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.436+0000 7f182a000640 1 -- 172.21.15.111:0/1490672149 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1824035070 con 0x5597c1bf1fb0 2024-09-15T22:46:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.436+0000 7f182a000640 1 -- 172.21.15.111:0/1490672149 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1824004780 con 0x5597c1bf1fb0 2024-09-15T22:46:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.436+0000 7f182a000640 1 -- 172.21.15.111:0/1490672149 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f182403d6b0 con 0x5597c1bf1fb0 2024-09-15T22:46:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.436+0000 7f184320a740 1 -- 172.21.15.111:0/1490672149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1fb0 msgr2=0x5597c1c02c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.436+0000 7f184320a740 1 --2- 172.21.15.111:0/1490672149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1fb0 0x5597c1c02c50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1824000c00 tx=0x7f1824005dd0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.436+0000 7f184320a740 1 -- 172.21.15.111:0/1490672149 shutdown_connections 2024-09-15T22:46:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.436+0000 7f184320a740 1 --2- 172.21.15.111:0/1490672149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597c1bf7040 0x5597c1be9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.436+0000 7f184320a740 1 --2- 172.21.15.111:0/1490672149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1fb0 0x5597c1c02c50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.436+0000 7f184320a740 1 --2- 172.21.15.111:0/1490672149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597c1bf1670 0x5597c1bfbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.436+0000 7f184320a740 1 -- 172.21.15.111:0/1490672149 >> 172.21.15.111:0/1490672149 conn(0x5597c1be88d0 msgr2=0x5597c1be9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:41.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.436+0000 7f184320a740 1 -- 172.21.15.111:0/1490672149 shutdown_connections 2024-09-15T22:46:41.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.436+0000 7f184320a740 1 -- 172.21.15.111:0/1490672149 wait complete. 2024-09-15T22:46:41.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.437+0000 7f184320a740 1 -- start start 2024-09-15T22:46:41.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.437+0000 7f184320a740 1 Processor -- start 2024-09-15T22:46:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.437+0000 7f184320a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597c1bf1670 0x5597c1c00160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.437+0000 7f184320a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1fb0 0x5597c1c006a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.437+0000 7f184320a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597c1bf7040 0x5597c1c016b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.437+0000 7f184320a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f18381a51c0 con 0x5597c1bf1fb0 2024-09-15T22:46:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.437+0000 7f184320a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f18381a6630 con 0x5597c1bf1670 2024-09-15T22:46:41.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.437+0000 7f184320a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f18381a5890 con 0x5597c1bf7040 2024-09-15T22:46:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.437+0000 7f184320a740 10 client.0 _ll_register_callbacks cb 0x7f182c009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.438+0000 7f183d000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1fb0 0x5597c1c006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.438+0000 7f183da00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597c1bf1670 0x5597c1c00160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.438+0000 7f183d000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1fb0 0x5597c1c006a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55470/0 (socket says 172.21.15.111:55470) 2024-09-15T22:46:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.438+0000 7f183da00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597c1bf1670 0x5597c1c00160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:45498/0 (socket says 172.21.15.111:45498) 2024-09-15T22:46:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.438+0000 7f183d000640 1 -- 172.21.15.111:0/3600690678 learned_addr learned my addr 172.21.15.111:0/3600690678 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.438+0000 7f183e400640 1 --2- 172.21.15.111:0/3600690678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597c1bf7040 0x5597c1c016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.438+0000 7f183d000640 1 -- 172.21.15.111:0/3600690678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597c1bf7040 msgr2=0x5597c1c016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.438+0000 7f183d000640 1 --2- 172.21.15.111:0/3600690678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597c1bf7040 0x5597c1c016b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.438+0000 7f183d000640 1 -- 172.21.15.111:0/3600690678 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597c1bf1670 msgr2=0x5597c1c00160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.438+0000 7f183d000640 1 --2- 172.21.15.111:0/3600690678 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597c1bf1670 0x5597c1c00160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.438+0000 7f183d000640 1 -- 172.21.15.111:0/3600690678 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597c1c01cc0 con 0x5597c1bf1fb0 2024-09-15T22:46:41.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.438+0000 7f183d000640 1 --2- 172.21.15.111:0/3600690678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1fb0 0x5597c1c006a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1824009900 tx=0x7f18240319c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da43b83a3edca0c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.438+0000 7f181f400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:46:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.438+0000 7f181f400640 1 -- 172.21.15.111:0/3600690678 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1824035070 con 0x5597c1bf1fb0 2024-09-15T22:46:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.439+0000 7f181f400640 1 -- 172.21.15.111:0/3600690678 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f1824005260 con 0x5597c1bf1fb0 2024-09-15T22:46:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.439+0000 7f184320a740 1 -- 172.21.15.111:0/3600690678 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1824004780 con 0x5597c1bf1fb0 2024-09-15T22:46:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.439+0000 7f181f400640 1 -- 172.21.15.111:0/3600690678 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f18240057b0 con 0x5597c1bf1fb0 2024-09-15T22:46:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.439+0000 7f184320a740 10 client.15057 fetch_fsmap learned FSMap version 26 2024-09-15T22:46:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.439+0000 7f184320a740 1 -- 172.21.15.111:0/3600690678 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5597c1c02fd0 con 0x5597c1bf1fb0 2024-09-15T22:46:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.439+0000 7f181f400640 1 -- 172.21.15.111:0/3600690678 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1824004780 con 0x5597c1bf1fb0 2024-09-15T22:46:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.439+0000 7f181f400640 1 -- 172.21.15.111:0/3600690678 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f182403d6d0 con 0x5597c1bf1fb0 2024-09-15T22:46:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.439+0000 7f184320a740 10 client.15057 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:46:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.439+0000 7f184320a740 10 client.15057 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:46:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.439+0000 7f184320a740 20 client.15057 populate_metadata read hostname 'smithi111' 2024-09-15T22:46:41.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.439+0000 7f184320a740 1 --2- 172.21.15.111:0/3600690678 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5597c1c72200 0x5597c1c745f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.439+0000 7f184320a740 4 client.15057 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:46:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.439+0000 7f184320a740 1 -- 172.21.15.111:0/3600690678 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5597c1b59b80 con 0x5597c1c72200 2024-09-15T22:46:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.440+0000 7f183da00640 1 --2- 172.21.15.111:0/3600690678 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5597c1c72200 0x5597c1c745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.440+0000 7f181f400640 10 client.15057 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:46:41.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.440+0000 7f183da00640 1 --2- 172.21.15.111:0/3600690678 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5597c1c72200 0x5597c1c745f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37feeeeaa3f40dab server_cookie=fc25854816117be in_seq=0 out_seq=0 2024-09-15T22:46:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.450+0000 7f181f400640 1 -- 172.21.15.111:0/3600690678 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5597c1b59b80 con 0x5597c1c72200 2024-09-15T22:46:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.450+0000 7f181f400640 10 client.15057 handle_command_reply: tid=0 2024-09-15T22:46:41.452 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.450+0000 7f184320a740 1 client.15057 shutdown 2024-09-15T22:46:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.451+0000 7f184320a740 1 -- 172.21.15.111:0/3600690678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1fb0 msgr2=0x5597c1c006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.451+0000 7f184320a740 1 --2- 172.21.15.111:0/3600690678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1fb0 0x5597c1c006a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1824009900 tx=0x7f18240319c0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.451+0000 7f184320a740 1 -- 172.21.15.111:0/3600690678 shutdown_connections 2024-09-15T22:46:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.451+0000 7f184320a740 1 --2- 172.21.15.111:0/3600690678 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5597c1c72200 0x5597c1c745f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.451+0000 7f184320a740 1 --2- 172.21.15.111:0/3600690678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5597c1bf7040 0x5597c1c016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.451+0000 7f184320a740 1 --2- 172.21.15.111:0/3600690678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5597c1bf1fb0 0x5597c1c006a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.451+0000 7f184320a740 1 --2- 172.21.15.111:0/3600690678 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5597c1bf1670 0x5597c1c00160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.451+0000 7f181f400640 0 client.15057 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:46:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.451+0000 7f184320a740 1 -- 172.21.15.111:0/3600690678 >> 172.21.15.111:0/3600690678 conn(0x5597c1be88d0 msgr2=0x5597c1be71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.451+0000 7f184320a740 1 -- 172.21.15.111:0/3600690678 shutdown_connections 2024-09-15T22:46:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.451+0000 7f184320a740 1 -- 172.21.15.111:0/3600690678 wait complete. 2024-09-15T22:46:41.453 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.451+0000 7f184320a740 20 client.15057 trim_cache size 0 max 16384 2024-09-15T22:46:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:46:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:46:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:46:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:46:41.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.452+0000 7f183ee00640 1 -- 172.21.15.111:0/1925959060 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f18200a32d0 msgr2=0x7f18200a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.452+0000 7f183ee00640 1 --2- 172.21.15.111:0/1925959060 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f18200a32d0 0x7f18200a5790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x5597c1bf29e0 tx=0x7f182c005f90 comp rx=0 tx=0).stop 2024-09-15T22:46:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.452+0000 7f183ee00640 1 -- 172.21.15.111:0/1925959060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1838103050 msgr2=0x7f183819f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.452+0000 7f183ee00640 1 --2- 172.21.15.111:0/1925959060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1838103050 0x7f183819f390 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f183000eb10 tx=0x7f183000c2d0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.452+0000 7f183ee00640 1 -- 172.21.15.111:0/1925959060 shutdown_connections 2024-09-15T22:46:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.452+0000 7f183ee00640 1 --2- 172.21.15.111:0/1925959060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1838103050 0x7f183819f390 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.452+0000 7f183ee00640 1 --2- 172.21.15.111:0/1925959060 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f18200a32d0 0x7f18200a5790 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.452+0000 7f183ee00640 1 --2- 172.21.15.111:0/1925959060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1838101f40 0x7f1838198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.452+0000 7f183ee00640 1 --2- 172.21.15.111:0/1925959060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1838101580 0x7f1838197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.452+0000 7f183ee00640 1 -- 172.21.15.111:0/1925959060 >> 172.21.15.111:0/1925959060 conn(0x7f18380fd0d0 msgr2=0x7f1838105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.452+0000 7f183ee00640 1 -- 172.21.15.111:0/1925959060 shutdown_connections 2024-09-15T22:46:41.455 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:41.452+0000 7f183ee00640 1 -- 172.21.15.111:0/1925959060 wait complete. 2024-09-15T22:46:41.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:41 smithi192 ceph-mon[35427]: pgmap v373: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-15T22:46:41.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:41 smithi121 ceph-mon[34815]: pgmap v373: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-15T22:46:41.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:41 smithi111 ceph-mon[30629]: pgmap v373: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-15T22:46:43.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:43 smithi192 ceph-mon[35427]: pgmap v374: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-09-15T22:46:43.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:43 smithi121 ceph-mon[34815]: pgmap v374: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-09-15T22:46:43.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:43 smithi111 ceph-mon[30629]: pgmap v374: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-09-15T22:46:45.666 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:45 smithi111 ceph-mon[30629]: pgmap v375: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-15T22:46:45.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:45 smithi192 ceph-mon[35427]: pgmap v375: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-15T22:46:45.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:45 smithi121 ceph-mon[34815]: pgmap v375: 97 pgs: 97 active+clean; 657 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-15T22:46:47.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:47 smithi192 ceph-mon[35427]: pgmap v376: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-15T22:46:47.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:47 smithi121 ceph-mon[34815]: pgmap v376: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-15T22:46:47.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:47 smithi111 ceph-mon[30629]: pgmap v376: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-15T22:46:49.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:49 smithi192 ceph-mon[35427]: pgmap v377: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-15T22:46:49.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:49 smithi121 ceph-mon[34815]: pgmap v377: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-15T22:46:49.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:49 smithi111 ceph-mon[30629]: pgmap v377: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-09-15T22:46:51.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:51 smithi192 ceph-mon[35427]: pgmap v378: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-09-15T22:46:51.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:51 smithi121 ceph-mon[34815]: pgmap v378: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-09-15T22:46:51.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:51 smithi111 ceph-mon[30629]: pgmap v378: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-09-15T22:46:53.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:53 smithi121 ceph-mon[34815]: pgmap v379: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-09-15T22:46:53.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:53 smithi111 ceph-mon[30629]: pgmap v379: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-09-15T22:46:54.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:53 smithi192 ceph-mon[35427]: pgmap v379: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-09-15T22:46:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.568+0000 7f897dc00640 1 -- 172.21.15.111:0/1238004837 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8978103b70 msgr2=0x7f8978103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.568+0000 7f897dc00640 1 --2- 172.21.15.111:0/1238004837 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8978103b70 0x7f8978103f70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8968009900 tx=0x7f896802a3f0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.568+0000 7f897dc00640 1 -- 172.21.15.111:0/1238004837 shutdown_connections 2024-09-15T22:46:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.568+0000 7f897dc00640 1 --2- 172.21.15.111:0/1238004837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8978105720 0x7f897810bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.568+0000 7f897dc00640 1 --2- 172.21.15.111:0/1238004837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8978104d60 0x7f89781051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.568+0000 7f897dc00640 1 --2- 172.21.15.111:0/1238004837 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8978103b70 0x7f8978103f70 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.568+0000 7f897dc00640 1 -- 172.21.15.111:0/1238004837 >> 172.21.15.111:0/1238004837 conn(0x7f89780ff2f0 msgr2=0x7f8978101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.568+0000 7f897dc00640 1 -- 172.21.15.111:0/1238004837 shutdown_connections 2024-09-15T22:46:54.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.568+0000 7f897dc00640 1 -- 172.21.15.111:0/1238004837 wait complete. 2024-09-15T22:46:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.569+0000 7f897dc00640 1 Processor -- start 2024-09-15T22:46:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.569+0000 7f897dc00640 1 -- start start 2024-09-15T22:46:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.570+0000 7f897dc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8978103b70 0x7f8978199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.570+0000 7f897dc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8978104d60 0x7f897819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.570+0000 7f897dc00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8978105720 0x7f89781a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.570+0000 7f897dc00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f897810f840 con 0x7f8978104d60 2024-09-15T22:46:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.570+0000 7f897dc00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f897810f6c0 con 0x7f8978105720 2024-09-15T22:46:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.570+0000 7f897dc00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f897810f9c0 con 0x7f8978103b70 2024-09-15T22:46:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.570+0000 7f8977400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8978104d60 0x7f897819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.570+0000 7f8977400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8978104d60 0x7f897819a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53442/0 (socket says 172.21.15.111:53442) 2024-09-15T22:46:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.570+0000 7f8977400640 1 -- 172.21.15.111:0/395870089 learned_addr learned my addr 172.21.15.111:0/395870089 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:54.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.570+0000 7f8977e00640 1 --2- 172.21.15.111:0/395870089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8978103b70 0x7f8978199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.570+0000 7f897d200640 1 --2- 172.21.15.111:0/395870089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8978105720 0x7f89781a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.570+0000 7f8977400640 1 -- 172.21.15.111:0/395870089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8978103b70 msgr2=0x7f8978199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.570+0000 7f8977400640 1 --2- 172.21.15.111:0/395870089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8978103b70 0x7f8978199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.570+0000 7f8977400640 1 -- 172.21.15.111:0/395870089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8978105720 msgr2=0x7f89781a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.570+0000 7f8977400640 1 --2- 172.21.15.111:0/395870089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8978105720 0x7f89781a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.571+0000 7f8977400640 1 -- 172.21.15.111:0/395870089 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89781a1ac0 con 0x7f8978104d60 2024-09-15T22:46:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.571+0000 7f8977400640 1 --2- 172.21.15.111:0/395870089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8978104d60 0x7f897819a3e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f896400e9f0 tx=0x7f896400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.571+0000 7f8974c00640 1 -- 172.21.15.111:0/395870089 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896400cd80 con 0x7f8978104d60 2024-09-15T22:46:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.571+0000 7f8974c00640 1 -- 172.21.15.111:0/395870089 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8964005500 con 0x7f8978104d60 2024-09-15T22:46:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.571+0000 7f8974c00640 1 -- 172.21.15.111:0/395870089 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8964010690 con 0x7f8978104d60 2024-09-15T22:46:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.571+0000 7f897dc00640 1 -- 172.21.15.111:0/395870089 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89781a1db0 con 0x7f8978104d60 2024-09-15T22:46:54.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.571+0000 7f897dc00640 1 -- 172.21.15.111:0/395870089 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89781a2270 con 0x7f8978104d60 2024-09-15T22:46:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.574+0000 7f89820c9740 1 Processor -- start 2024-09-15T22:46:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.575+0000 7f89820c9740 1 -- start start 2024-09-15T22:46:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.575+0000 7f8974c00640 1 -- 172.21.15.111:0/395870089 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f89640046c0 con 0x7f8978104d60 2024-09-15T22:46:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.575+0000 7f89820c9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a814a87260 0x55a814a87660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.575+0000 7f89820c9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a814a87ba0 0x55a814a8c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.575+0000 7f89820c9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a814a8cc30 0x55a814a8f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.575+0000 7f89820c9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a8149ec100 con 0x55a814a87260 2024-09-15T22:46:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.575+0000 7f89820c9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a8149941b0 con 0x55a814a87ba0 2024-09-15T22:46:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.575+0000 7f89820c9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a8149e80c0 con 0x55a814a8cc30 2024-09-15T22:46:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.575+0000 7f897d200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a814a87260 0x55a814a87660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.575+0000 7f8977e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a814a87ba0 0x55a814a8c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.575+0000 7f8977e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a814a87ba0 0x55a814a8c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:41228/0 (socket says 172.21.15.111:41228) 2024-09-15T22:46:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.575+0000 7f8977e00640 1 -- 172.21.15.111:0/3514340054 learned_addr learned my addr 172.21.15.111:0/3514340054 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.575+0000 7f897d200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a814a87260 0x55a814a87660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53456/0 (socket says 172.21.15.111:53456) 2024-09-15T22:46:54.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.576+0000 7f8977400640 1 --2- 172.21.15.111:0/3514340054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a814a8cc30 0x55a814a8f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.576+0000 7f8977e00640 1 -- 172.21.15.111:0/3514340054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a814a8cc30 msgr2=0x55a814a8f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.576+0000 7f8977e00640 1 --2- 172.21.15.111:0/3514340054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a814a8cc30 0x55a814a8f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.576+0000 7f8977e00640 1 -- 172.21.15.111:0/3514340054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a814a87260 msgr2=0x55a814a87660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.576+0000 7f8977e00640 1 --2- 172.21.15.111:0/3514340054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a814a87260 0x55a814a87660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.576+0000 7f8977e00640 1 -- 172.21.15.111:0/3514340054 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8146ff210 con 0x55a814a87ba0 2024-09-15T22:46:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.576+0000 7f8977400640 1 --2- 172.21.15.111:0/3514340054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a814a8cc30 0x55a814a8f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:46:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.576+0000 7f8977e00640 1 --2- 172.21.15.111:0/3514340054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a814a87ba0 0x55a814a8c660 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8968006010 tx=0x7f8968004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b7812c342518ff8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.576+0000 7f8974c00640 1 --2- 172.21.15.111:0/395870089 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f895c0a3350 0x7f895c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.576+0000 7f896d600640 1 -- 172.21.15.111:0/3514340054 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8968035040 con 0x55a814a87ba0 2024-09-15T22:46:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.576+0000 7f896d600640 1 -- 172.21.15.111:0/3514340054 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8968004430 con 0x55a814a87ba0 2024-09-15T22:46:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.576+0000 7f896d600640 1 -- 172.21.15.111:0/3514340054 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89680316b0 con 0x55a814a87ba0 2024-09-15T22:46:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.577+0000 7f8974c00640 1 -- 172.21.15.111:0/395870089 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f89640cfd70 con 0x7f8978104d60 2024-09-15T22:46:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.577+0000 7f897d200640 1 --2- 172.21.15.111:0/395870089 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f895c0a3350 0x7f895c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.577+0000 7f89820c9740 1 -- 172.21.15.111:0/3514340054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a814a87ba0 msgr2=0x55a814a8c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.577+0000 7f89820c9740 1 --2- 172.21.15.111:0/3514340054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a814a87ba0 0x55a814a8c660 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8968006010 tx=0x7f8968004040 comp rx=0 tx=0).stop 2024-09-15T22:46:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.577+0000 7f897d200640 1 --2- 172.21.15.111:0/395870089 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f895c0a3350 0x7f895c0a5810 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8970005fd0 tx=0x7f8970009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.577+0000 7f89820c9740 1 -- 172.21.15.111:0/3514340054 shutdown_connections 2024-09-15T22:46:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.577+0000 7f89820c9740 1 --2- 172.21.15.111:0/3514340054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a814a8cc30 0x55a814a8f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.577+0000 7f89820c9740 1 --2- 172.21.15.111:0/3514340054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a814a87ba0 0x55a814a8c660 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.577+0000 7f89820c9740 1 --2- 172.21.15.111:0/3514340054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a814a87260 0x55a814a87660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.577+0000 7f89820c9740 1 -- 172.21.15.111:0/3514340054 >> 172.21.15.111:0/3514340054 conn(0x55a814a828d0 msgr2=0x55a814a82cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.578+0000 7f89820c9740 1 -- 172.21.15.111:0/3514340054 shutdown_connections 2024-09-15T22:46:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.578+0000 7f89820c9740 1 -- 172.21.15.111:0/3514340054 wait complete. 2024-09-15T22:46:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.578+0000 7f89820c9740 1 -- start start 2024-09-15T22:46:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.578+0000 7f89820c9740 1 Processor -- start 2024-09-15T22:46:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.579+0000 7f89820c9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a814a87260 0x55a814a99c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.579+0000 7f89820c9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a814a87ba0 0x55a814a9a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.579+0000 7f89820c9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a814a8cc30 0x55a814aa11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.579+0000 7f89820c9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a814a8fba0 con 0x55a814a8cc30 2024-09-15T22:46:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.579+0000 7f89820c9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a814975090 con 0x55a814a87260 2024-09-15T22:46:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.579+0000 7f89820c9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a814a8fd20 con 0x55a814a87ba0 2024-09-15T22:46:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.579+0000 7f89820c9740 10 client.0 _ll_register_callbacks cb 0x55a8149c9f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.579+0000 7f897d200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a814a8cc30 0x55a814aa11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.579+0000 7f897d200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a814a8cc30 0x55a814aa11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53458/0 (socket says 172.21.15.111:53458) 2024-09-15T22:46:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.579+0000 7f897d200640 1 -- 172.21.15.111:0/3160023570 learned_addr learned my addr 172.21.15.111:0/3160023570 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.579+0000 7f8977e00640 1 --2- 172.21.15.111:0/3160023570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a814a87260 0x55a814a99c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.579+0000 7f8977400640 1 --2- 172.21.15.111:0/3160023570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a814a87ba0 0x55a814a9a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.579+0000 7f897d200640 1 -- 172.21.15.111:0/3160023570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a814a87ba0 msgr2=0x55a814a9a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.579+0000 7f897d200640 1 --2- 172.21.15.111:0/3160023570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a814a87ba0 0x55a814a9a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.579+0000 7f897d200640 1 -- 172.21.15.111:0/3160023570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a814a87260 msgr2=0x55a814a99c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.579+0000 7f897d200640 1 --2- 172.21.15.111:0/3160023570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a814a87260 0x55a814a99c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.579+0000 7f897d200640 1 -- 172.21.15.111:0/3160023570 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8147cb620 con 0x55a814a8cc30 2024-09-15T22:46:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.580+0000 7f897d200640 1 --2- 172.21.15.111:0/3160023570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a814a8cc30 0x55a814aa11a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8970003f90 tx=0x7f8970004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ddd7336f3fae05f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.580+0000 7f8962000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:46:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.580+0000 7f8962000640 1 -- 172.21.15.111:0/3160023570 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897000f8b0 con 0x55a814a8cc30 2024-09-15T22:46:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.580+0000 7f89820c9740 1 -- 172.21.15.111:0/3160023570 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8968004430 con 0x55a814a8cc30 2024-09-15T22:46:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.580+0000 7f8962000640 1 -- 172.21.15.111:0/3160023570 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f897000fa50 con 0x55a814a8cc30 2024-09-15T22:46:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.580+0000 7f8962000640 1 -- 172.21.15.111:0/3160023570 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f897000fc50 con 0x55a814a8cc30 2024-09-15T22:46:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.580+0000 7f89820c9740 10 client.15069 fetch_fsmap learned FSMap version 26 2024-09-15T22:46:54.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.580+0000 7f89820c9740 1 -- 172.21.15.111:0/3160023570 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a814aa40b0 con 0x55a814a8cc30 2024-09-15T22:46:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.580+0000 7f8962000640 1 -- 172.21.15.111:0/3160023570 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8968004430 con 0x55a814a8cc30 2024-09-15T22:46:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.581+0000 7f8962000640 1 -- 172.21.15.111:0/3160023570 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f8970026070 con 0x55a814a8cc30 2024-09-15T22:46:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.581+0000 7f89820c9740 10 client.15069 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:46:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.581+0000 7f89820c9740 10 client.15069 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:46:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.581+0000 7f89820c9740 20 client.15069 populate_metadata read hostname 'smithi111' 2024-09-15T22:46:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.581+0000 7f89820c9740 1 --2- 172.21.15.111:0/3160023570 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a814aa5750 0x55a814aa7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.581+0000 7f89820c9740 4 client.15069 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:46:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.581+0000 7f89820c9740 1 -- 172.21.15.111:0/3160023570 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8149f9830 con 0x55a814aa5750 2024-09-15T22:46:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.581+0000 7f8977e00640 1 --2- 172.21.15.111:0/3160023570 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a814aa5750 0x55a814aa7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.582+0000 7f8977e00640 1 --2- 172.21.15.111:0/3160023570 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a814aa5750 0x55a814aa7b60 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3afc70313a62829d server_cookie=3fc15b6d5fae3ad9 in_seq=0 out_seq=0 2024-09-15T22:46:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.582+0000 7f8962000640 10 client.15069 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:46:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.582+0000 7f8962000640 1 -- 172.21.15.111:0/3160023570 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a8149f9830 con 0x55a814aa5750 2024-09-15T22:46:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.583+0000 7f8962000640 10 client.15069 handle_command_reply: tid=0 2024-09-15T22:46:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.583+0000 7f89820c9740 1 client.15069 shutdown 2024-09-15T22:46:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.583+0000 7f89820c9740 1 -- 172.21.15.111:0/3160023570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a814a8cc30 msgr2=0x55a814aa11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.583+0000 7f89820c9740 1 --2- 172.21.15.111:0/3160023570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a814a8cc30 0x55a814aa11a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8970003f90 tx=0x7f8970004ab0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.583+0000 7f89820c9740 1 -- 172.21.15.111:0/3160023570 shutdown_connections 2024-09-15T22:46:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.583+0000 7f89820c9740 1 --2- 172.21.15.111:0/3160023570 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a814aa5750 0x55a814aa7b60 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.583+0000 7f89820c9740 1 --2- 172.21.15.111:0/3160023570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a814a8cc30 0x55a814aa11a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.583+0000 7f89820c9740 1 --2- 172.21.15.111:0/3160023570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a814a87ba0 0x55a814a9a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.583+0000 7f8962000640 0 client.15069 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:46:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.583+0000 7f89820c9740 1 --2- 172.21.15.111:0/3160023570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a814a87260 0x55a814a99c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.583+0000 7f89820c9740 1 -- 172.21.15.111:0/3160023570 >> 172.21.15.111:0/3160023570 conn(0x55a814a828d0 msgr2=0x55a814a840e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.583+0000 7f89820c9740 1 -- 172.21.15.111:0/3160023570 shutdown_connections 2024-09-15T22:46:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.583+0000 7f89820c9740 1 -- 172.21.15.111:0/3160023570 wait complete. 2024-09-15T22:46:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.583+0000 7f89820c9740 20 client.15069 trim_cache size 0 max 16384 2024-09-15T22:46:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.598+0000 7f89820c9740 1 Processor -- start 2024-09-15T22:46:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.598+0000 7f89820c9740 1 -- start start 2024-09-15T22:46:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.598+0000 7f89820c9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a814a87260 0x55a814a83780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.598+0000 7f89820c9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a814a87ba0 0x55a814aa1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.598+0000 7f89820c9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a814a8cc30 0x55a814a83cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.598+0000 7f89820c9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f89781a93f0 con 0x55a814a87260 2024-09-15T22:46:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.598+0000 7f89820c9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f89781a6cc0 con 0x55a814a8cc30 2024-09-15T22:46:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.598+0000 7f89820c9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f89781a4b30 con 0x55a814a87ba0 2024-09-15T22:46:54.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.599+0000 7f8977e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a814a87260 0x55a814a83780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:54.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.599+0000 7f8977e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a814a87260 0x55a814a83780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53468/0 (socket says 172.21.15.111:53468) 2024-09-15T22:46:54.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.599+0000 7f8977e00640 1 -- 172.21.15.111:0/901232973 learned_addr learned my addr 172.21.15.111:0/901232973 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:54.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.599+0000 7f897d200640 1 --2- 172.21.15.111:0/901232973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a814a8cc30 0x55a814a83cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:54.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.599+0000 7f8977400640 1 --2- 172.21.15.111:0/901232973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a814a87ba0 0x55a814aa1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:54.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.599+0000 7f897d200640 1 -- 172.21.15.111:0/901232973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a814a87ba0 msgr2=0x55a814aa1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:54.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.599+0000 7f897d200640 1 --2- 172.21.15.111:0/901232973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a814a87ba0 0x55a814aa1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.599+0000 7f897d200640 1 -- 172.21.15.111:0/901232973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a814a87260 msgr2=0x55a814a83780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:54.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.599+0000 7f897d200640 1 --2- 172.21.15.111:0/901232973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a814a87260 0x55a814a83780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.599+0000 7f897d200640 1 -- 172.21.15.111:0/901232973 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a814a99020 con 0x55a814a8cc30 2024-09-15T22:46:54.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.599+0000 7f8977e00640 1 --2- 172.21.15.111:0/901232973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a814a87260 0x55a814a83780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:46:54.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.599+0000 7f897d200640 1 --2- 172.21.15.111:0/901232973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a814a8cc30 0x55a814a83cc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8970011f30 tx=0x7f8970000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a511bc194a63e0bc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:54.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.600+0000 7f896cc00640 1 -- 172.21.15.111:0/901232973 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8970037070 con 0x55a814a8cc30 2024-09-15T22:46:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.600+0000 7f896cc00640 1 -- 172.21.15.111:0/901232973 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f897000fdf0 con 0x55a814a8cc30 2024-09-15T22:46:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.600+0000 7f896cc00640 1 -- 172.21.15.111:0/901232973 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8970005020 con 0x55a814a8cc30 2024-09-15T22:46:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.600+0000 7f89820c9740 1 -- 172.21.15.111:0/901232973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a814a8cc30 msgr2=0x55a814a83cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.600+0000 7f89820c9740 1 --2- 172.21.15.111:0/901232973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a814a8cc30 0x55a814a83cc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8970011f30 tx=0x7f8970000f40 comp rx=0 tx=0).stop 2024-09-15T22:46:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.600+0000 7f89820c9740 1 -- 172.21.15.111:0/901232973 shutdown_connections 2024-09-15T22:46:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.600+0000 7f89820c9740 1 --2- 172.21.15.111:0/901232973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a814a8cc30 0x55a814a83cc0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.600+0000 7f89820c9740 1 --2- 172.21.15.111:0/901232973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a814a87ba0 0x55a814aa1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.600+0000 7f89820c9740 1 --2- 172.21.15.111:0/901232973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a814a87260 0x55a814a83780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.600+0000 7f89820c9740 1 -- 172.21.15.111:0/901232973 >> 172.21.15.111:0/901232973 conn(0x55a814a828d0 msgr2=0x55a814a99c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.600+0000 7f89820c9740 1 -- 172.21.15.111:0/901232973 shutdown_connections 2024-09-15T22:46:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.600+0000 7f89820c9740 1 -- 172.21.15.111:0/901232973 wait complete. 2024-09-15T22:46:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.601+0000 7f89820c9740 1 -- start start 2024-09-15T22:46:54.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.601+0000 7f89820c9740 1 Processor -- start 2024-09-15T22:46:54.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.601+0000 7f89820c9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a814a87260 0x55a814a9e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:54.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.601+0000 7f89820c9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a814a87ba0 0x55a814a9e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:54.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.601+0000 7f89820c9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a814a8cc30 0x55a814a9f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.601+0000 7f89820c9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f89781a5180 con 0x55a814a87260 2024-09-15T22:46:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.601+0000 7f89820c9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f89781a65f0 con 0x55a814a87ba0 2024-09-15T22:46:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.601+0000 7f89820c9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f89781a5850 con 0x55a814a8cc30 2024-09-15T22:46:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.601+0000 7f89820c9740 10 client.0 _ll_register_callbacks cb 0x7f8970009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:46:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.602+0000 7f8977e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a814a87260 0x55a814a9e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.602+0000 7f8977400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a814a87ba0 0x55a814a9e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.602+0000 7f8977400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a814a87ba0 0x55a814a9e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:41260/0 (socket says 172.21.15.111:41260) 2024-09-15T22:46:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.602+0000 7f8977400640 1 -- 172.21.15.111:0/2964385271 learned_addr learned my addr 172.21.15.111:0/2964385271 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:46:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.602+0000 7f897d200640 1 --2- 172.21.15.111:0/2964385271 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a814a8cc30 0x55a814a9f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.602+0000 7f8977400640 1 -- 172.21.15.111:0/2964385271 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a814a8cc30 msgr2=0x55a814a9f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.602+0000 7f8977400640 1 --2- 172.21.15.111:0/2964385271 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a814a8cc30 0x55a814a9f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.602+0000 7f8977400640 1 -- 172.21.15.111:0/2964385271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a814a87260 msgr2=0x55a814a9e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:46:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.602+0000 7f8977400640 1 --2- 172.21.15.111:0/2964385271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a814a87260 0x55a814a9e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.602+0000 7f8977400640 1 -- 172.21.15.111:0/2964385271 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a814a9ff90 con 0x55a814a87ba0 2024-09-15T22:46:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.602+0000 7f8977400640 1 --2- 172.21.15.111:0/2964385271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a814a87ba0 0x55a814a9e9c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f89640097d0 tx=0x7f896409c140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cad3ba67f99a7a9a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:46:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.602+0000 7f8962000640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:46:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.602+0000 7f8962000640 1 -- 172.21.15.111:0/2964385271 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896409c2b0 con 0x55a814a87ba0 2024-09-15T22:46:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.603+0000 7f8962000640 1 -- 172.21.15.111:0/2964385271 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f89640bc0b0 con 0x55a814a87ba0 2024-09-15T22:46:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.603+0000 7f89820c9740 1 -- 172.21.15.111:0/2964385271 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f897000fdf0 con 0x55a814a87ba0 2024-09-15T22:46:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.603+0000 7f8962000640 1 -- 172.21.15.111:0/2964385271 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f89640a0660 con 0x55a814a87ba0 2024-09-15T22:46:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.603+0000 7f89820c9740 10 client.25103 fetch_fsmap learned FSMap version 26 2024-09-15T22:46:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.603+0000 7f89820c9740 1 -- 172.21.15.111:0/2964385271 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a814aa1310 con 0x55a814a87ba0 2024-09-15T22:46:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.603+0000 7f8962000640 1 -- 172.21.15.111:0/2964385271 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f897000fdf0 con 0x55a814a87ba0 2024-09-15T22:46:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.603+0000 7f8962000640 1 -- 172.21.15.111:0/2964385271 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f89640a0150 con 0x55a814a87ba0 2024-09-15T22:46:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.603+0000 7f89820c9740 10 client.25103 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:46:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.603+0000 7f89820c9740 10 client.25103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:46:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.603+0000 7f89820c9740 20 client.25103 populate_metadata read hostname 'smithi111' 2024-09-15T22:46:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.603+0000 7f89820c9740 1 --2- 172.21.15.111:0/2964385271 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a814aa5750 0x55a814b12a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:46:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.603+0000 7f89820c9740 4 client.25103 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:46:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.603+0000 7f89820c9740 1 -- 172.21.15.111:0/2964385271 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a8149f3b80 con 0x55a814aa5750 2024-09-15T22:46:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.603+0000 7f8977e00640 1 --2- 172.21.15.111:0/2964385271 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a814aa5750 0x55a814b12a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:46:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.604+0000 7f8977e00640 1 --2- 172.21.15.111:0/2964385271 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a814aa5750 0x55a814b12a90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7abcf18e2a66168 server_cookie=f07ebfbee0c5f309 in_seq=0 out_seq=0 2024-09-15T22:46:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.604+0000 7f8962000640 10 client.25103 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:46:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.608+0000 7f8962000640 1 -- 172.21.15.111:0/2964385271 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a8149f3b80 con 0x55a814aa5750 2024-09-15T22:46:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.608+0000 7f8962000640 10 client.25103 handle_command_reply: tid=0 2024-09-15T22:46:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.608+0000 7f89820c9740 1 client.25103 shutdown 2024-09-15T22:46:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.608+0000 7f89820c9740 1 -- 172.21.15.111:0/2964385271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a814a87ba0 msgr2=0x55a814a9e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.608+0000 7f89820c9740 1 --2- 172.21.15.111:0/2964385271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a814a87ba0 0x55a814a9e9c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f89640097d0 tx=0x7f896409c140 comp rx=0 tx=0).stop 2024-09-15T22:46:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.608+0000 7f89820c9740 1 -- 172.21.15.111:0/2964385271 shutdown_connections 2024-09-15T22:46:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.608+0000 7f89820c9740 1 --2- 172.21.15.111:0/2964385271 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a814aa5750 0x55a814b12a90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.608+0000 7f89820c9740 1 --2- 172.21.15.111:0/2964385271 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a814a8cc30 0x55a814a9f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.608+0000 7f89820c9740 1 --2- 172.21.15.111:0/2964385271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a814a87ba0 0x55a814a9e9c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.608+0000 7f8962000640 0 client.25103 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:46:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.608+0000 7f89820c9740 1 --2- 172.21.15.111:0/2964385271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a814a87260 0x55a814a9e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.608+0000 7f89820c9740 1 -- 172.21.15.111:0/2964385271 >> 172.21.15.111:0/2964385271 conn(0x55a814a828d0 msgr2=0x55a814a811f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.608+0000 7f89820c9740 1 -- 172.21.15.111:0/2964385271 shutdown_connections 2024-09-15T22:46:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.608+0000 7f89820c9740 1 -- 172.21.15.111:0/2964385271 wait complete. 2024-09-15T22:46:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.608+0000 7f89820c9740 20 client.25103 trim_cache size 0 max 16384 2024-09-15T22:46:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:46:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:46:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:46:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:46:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.609+0000 7f897dc00640 1 -- 172.21.15.111:0/395870089 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f895c0a3350 msgr2=0x7f895c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.609+0000 7f897dc00640 1 --2- 172.21.15.111:0/395870089 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f895c0a3350 0x7f895c0a5810 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8970005fd0 tx=0x7f8970009500 comp rx=0 tx=0).stop 2024-09-15T22:46:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.609+0000 7f897dc00640 1 -- 172.21.15.111:0/395870089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8978104d60 msgr2=0x7f897819a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:46:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.609+0000 7f897dc00640 1 --2- 172.21.15.111:0/395870089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8978104d60 0x7f897819a3e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f896400e9f0 tx=0x7f896400edf0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.616 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.609+0000 7f897dc00640 1 -- 172.21.15.111:0/395870089 shutdown_connections 2024-09-15T22:46:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.609+0000 7f897dc00640 1 --2- 172.21.15.111:0/395870089 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f895c0a3350 0x7f895c0a5810 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.609+0000 7f897dc00640 1 --2- 172.21.15.111:0/395870089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8978105720 0x7f89781a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.609+0000 7f897dc00640 1 --2- 172.21.15.111:0/395870089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8978104d60 0x7f897819a3e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.609+0000 7f897dc00640 1 --2- 172.21.15.111:0/395870089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8978103b70 0x7f8978199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:46:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.609+0000 7f897dc00640 1 -- 172.21.15.111:0/395870089 >> 172.21.15.111:0/395870089 conn(0x7f89780ff2f0 msgr2=0x7f89781080e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:46:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.610+0000 7f897dc00640 1 -- 172.21.15.111:0/395870089 shutdown_connections 2024-09-15T22:46:54.617 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:46:54.610+0000 7f897dc00640 1 -- 172.21.15.111:0/395870089 wait complete. 2024-09-15T22:46:55.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:55 smithi121 ceph-mon[34815]: pgmap v380: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-09-15T22:46:55.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:55 smithi111 ceph-mon[30629]: pgmap v380: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-09-15T22:46:56.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:55 smithi192 ceph-mon[35427]: pgmap v380: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-09-15T22:46:57.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:57 smithi121 ceph-mon[34815]: pgmap v381: 97 pgs: 97 active+clean; 659 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-15T22:46:57.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:57 smithi111 ceph-mon[30629]: pgmap v381: 97 pgs: 97 active+clean; 659 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-15T22:46:58.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:57 smithi192 ceph-mon[35427]: pgmap v381: 97 pgs: 97 active+clean; 659 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-15T22:46:59.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:46:59 smithi121 ceph-mon[34815]: pgmap v382: 97 pgs: 97 active+clean; 659 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-15T22:46:59.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:46:59 smithi111 ceph-mon[30629]: pgmap v382: 97 pgs: 97 active+clean; 659 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-15T22:47:00.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:46:59 smithi192 ceph-mon[35427]: pgmap v382: 97 pgs: 97 active+clean; 659 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-15T22:47:01.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:01 smithi121 ceph-mon[34815]: pgmap v383: 97 pgs: 97 active+clean; 659 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-15T22:47:01.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:01 smithi111 ceph-mon[30629]: pgmap v383: 97 pgs: 97 active+clean; 659 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-15T22:47:02.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:01 smithi192 ceph-mon[35427]: pgmap v383: 97 pgs: 97 active+clean; 659 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-15T22:47:03.867 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:03 smithi192 ceph-mon[35427]: pgmap v384: 97 pgs: 97 active+clean; 659 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-15T22:47:03.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:03 smithi121 ceph-mon[34815]: pgmap v384: 97 pgs: 97 active+clean; 659 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-15T22:47:03.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:03 smithi111 ceph-mon[30629]: pgmap v384: 97 pgs: 97 active+clean; 659 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-15T22:47:05.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:05 smithi121 ceph-mon[34815]: pgmap v385: 97 pgs: 97 active+clean; 659 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-15T22:47:05.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:05 smithi111 ceph-mon[30629]: pgmap v385: 97 pgs: 97 active+clean; 659 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-15T22:47:06.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:05 smithi192 ceph-mon[35427]: pgmap v385: 97 pgs: 97 active+clean; 659 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-15T22:47:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.789+0000 7fa790c00640 1 -- 172.21.15.111:0/3395438740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa78c103b70 msgr2=0x7fa78c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.789+0000 7fa790c00640 1 --2- 172.21.15.111:0/3395438740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa78c103b70 0x7fa78c103f70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa77c009900 tx=0x7fa77c02a410 comp rx=0 tx=0).stop 2024-09-15T22:47:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.789+0000 7fa790c00640 1 -- 172.21.15.111:0/3395438740 shutdown_connections 2024-09-15T22:47:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.789+0000 7fa790c00640 1 --2- 172.21.15.111:0/3395438740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa78c105720 0x7fa78c10bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.789+0000 7fa790c00640 1 --2- 172.21.15.111:0/3395438740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa78c104d60 0x7fa78c1051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.789+0000 7fa790c00640 1 --2- 172.21.15.111:0/3395438740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa78c103b70 0x7fa78c103f70 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.789+0000 7fa790c00640 1 -- 172.21.15.111:0/3395438740 >> 172.21.15.111:0/3395438740 conn(0x7fa78c0ff2f0 msgr2=0x7fa78c101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.789+0000 7fa790c00640 1 -- 172.21.15.111:0/3395438740 shutdown_connections 2024-09-15T22:47:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.789+0000 7fa790c00640 1 -- 172.21.15.111:0/3395438740 wait complete. 2024-09-15T22:47:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.790+0000 7fa790c00640 1 Processor -- start 2024-09-15T22:47:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.791+0000 7fa790c00640 1 -- start start 2024-09-15T22:47:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.791+0000 7fa790c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa78c103b70 0x7fa78c079fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.791+0000 7fa790c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa78c104d60 0x7fa78c07a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.791+0000 7fa790c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa78c105720 0x7fa78c076b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.791+0000 7fa790c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c10f820 con 0x7fa78c103b70 2024-09-15T22:47:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.791+0000 7fa790c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c10f6a0 con 0x7fa78c105720 2024-09-15T22:47:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.791+0000 7fa790c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c10f9a0 con 0x7fa78c104d60 2024-09-15T22:47:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.791+0000 7fa78b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa78c103b70 0x7fa78c079fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.791+0000 7fa78be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa78c105720 0x7fa78c076b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.791+0000 7fa78b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa78c103b70 0x7fa78c079fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39980/0 (socket says 172.21.15.111:39980) 2024-09-15T22:47:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.791+0000 7fa78b400640 1 -- 172.21.15.111:0/3955579853 learned_addr learned my addr 172.21.15.111:0/3955579853 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:47:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.791+0000 7fa78aa00640 1 --2- 172.21.15.111:0/3955579853 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa78c104d60 0x7fa78c07a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.792+0000 7fa78aa00640 1 -- 172.21.15.111:0/3955579853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa78c105720 msgr2=0x7fa78c076b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.792+0000 7fa78aa00640 1 --2- 172.21.15.111:0/3955579853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa78c105720 0x7fa78c076b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.792+0000 7fa78aa00640 1 -- 172.21.15.111:0/3955579853 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa78c103b70 msgr2=0x7fa78c079fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.792+0000 7fa78aa00640 1 --2- 172.21.15.111:0/3955579853 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa78c103b70 0x7fa78c079fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.792+0000 7fa78aa00640 1 -- 172.21.15.111:0/3955579853 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa78c0773d0 con 0x7fa78c104d60 2024-09-15T22:47:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.792+0000 7fa78aa00640 1 --2- 172.21.15.111:0/3955579853 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa78c104d60 0x7fa78c07a4e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa77800e9f0 tx=0x7fa77800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.793+0000 7fa783e00640 1 -- 172.21.15.111:0/3955579853 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77800ccc0 con 0x7fa78c104d60 2024-09-15T22:47:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.793+0000 7fa790c00640 1 -- 172.21.15.111:0/3955579853 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa78c1a6300 con 0x7fa78c104d60 2024-09-15T22:47:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.793+0000 7fa783e00640 1 -- 172.21.15.111:0/3955579853 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa778010910 con 0x7fa78c104d60 2024-09-15T22:47:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.793+0000 7fa783e00640 1 -- 172.21.15.111:0/3955579853 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa778018790 con 0x7fa78c104d60 2024-09-15T22:47:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.793+0000 7fa790c00640 1 -- 172.21.15.111:0/3955579853 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa78c1a6760 con 0x7fa78c104d60 2024-09-15T22:47:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.797+0000 7fa7951c5740 1 Processor -- start 2024-09-15T22:47:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.797+0000 7fa7951c5740 1 -- start start 2024-09-15T22:47:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.798+0000 7fa7951c5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d65da7f260 0x55d65da7f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.798+0000 7fa7951c5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d65da7fba0 0x55d65da84660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.798+0000 7fa7951c5740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65da84c30 0x55d65da870f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.798+0000 7fa7951c5740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d65d9e4100 con 0x55d65da84c30 2024-09-15T22:47:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.798+0000 7fa7951c5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d65d98c1b0 con 0x55d65da7f260 2024-09-15T22:47:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.798+0000 7fa7951c5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d65d9e00c0 con 0x55d65da7fba0 2024-09-15T22:47:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.798+0000 7fa78b400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d65da7fba0 0x55d65da84660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.798+0000 7fa78be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d65da7f260 0x55d65da7f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.798+0000 7fa78b400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d65da7fba0 0x55d65da84660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:55152/0 (socket says 172.21.15.111:55152) 2024-09-15T22:47:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.798+0000 7fa78b400640 1 -- 172.21.15.111:0/3116446452 learned_addr learned my addr 172.21.15.111:0/3116446452 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:47:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.798+0000 7fa783e00640 1 -- 172.21.15.111:0/3955579853 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7fa778010ac0 con 0x7fa78c104d60 2024-09-15T22:47:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.799+0000 7fa78b400640 1 -- 172.21.15.111:0/3116446452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d65da7f260 msgr2=0x55d65da7f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.799+0000 7fa78b400640 1 --2- 172.21.15.111:0/3116446452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d65da7f260 0x55d65da7f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.799+0000 7fa78b400640 1 -- 172.21.15.111:0/3116446452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65da84c30 msgr2=0x55d65da870f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:47:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.799+0000 7fa78aa00640 1 --2- 172.21.15.111:0/3116446452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65da84c30 0x55d65da870f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.799+0000 7fa78b400640 1 --2- 172.21.15.111:0/3116446452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65da84c30 0x55d65da870f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.799+0000 7fa78b400640 1 -- 172.21.15.111:0/3116446452 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d65d6f7210 con 0x55d65da7fba0 2024-09-15T22:47:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.799+0000 7fa78aa00640 1 --2- 172.21.15.111:0/3116446452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65da84c30 0x55d65da870f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:47:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.800+0000 7fa78b400640 1 --2- 172.21.15.111:0/3116446452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d65da7fba0 0x55d65da84660 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa77c02a930 tx=0x7fa77c004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bb4aec2b7aa1481 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.800+0000 7fa777e00640 1 -- 172.21.15.111:0/3116446452 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c040070 con 0x55d65da7fba0 2024-09-15T22:47:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.800+0000 7fa777e00640 1 -- 172.21.15.111:0/3116446452 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa77c005500 con 0x55d65da7fba0 2024-09-15T22:47:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.800+0000 7fa777e00640 1 -- 172.21.15.111:0/3116446452 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c035040 con 0x55d65da7fba0 2024-09-15T22:47:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.801+0000 7fa783e00640 1 --2- 172.21.15.111:0/3955579853 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa7700a33d0 0x7fa7700a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.801+0000 7fa783e00640 1 -- 172.21.15.111:0/3955579853 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fa7780cfd70 con 0x7fa78c104d60 2024-09-15T22:47:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.801+0000 7fa7951c5740 1 -- 172.21.15.111:0/3116446452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d65da7fba0 msgr2=0x55d65da84660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.801+0000 7fa7951c5740 1 --2- 172.21.15.111:0/3116446452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d65da7fba0 0x55d65da84660 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa77c02a930 tx=0x7fa77c004040 comp rx=0 tx=0).stop 2024-09-15T22:47:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.802+0000 7fa7951c5740 1 -- 172.21.15.111:0/3116446452 shutdown_connections 2024-09-15T22:47:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.802+0000 7fa7951c5740 1 --2- 172.21.15.111:0/3116446452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65da84c30 0x55d65da870f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.802+0000 7fa7951c5740 1 --2- 172.21.15.111:0/3116446452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d65da7fba0 0x55d65da84660 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.802+0000 7fa7951c5740 1 --2- 172.21.15.111:0/3116446452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d65da7f260 0x55d65da7f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.802+0000 7fa7951c5740 1 -- 172.21.15.111:0/3116446452 >> 172.21.15.111:0/3116446452 conn(0x55d65da7a8d0 msgr2=0x55d65da7acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.802+0000 7fa78be00640 1 --2- 172.21.15.111:0/3955579853 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa7700a33d0 0x7fa7700a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.802+0000 7fa7951c5740 1 -- 172.21.15.111:0/3116446452 shutdown_connections 2024-09-15T22:47:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.802+0000 7fa7951c5740 1 -- 172.21.15.111:0/3116446452 wait complete. 2024-09-15T22:47:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.803+0000 7fa78be00640 1 --2- 172.21.15.111:0/3955579853 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa7700a33d0 0x7fa7700a5890 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa784005fd0 tx=0x7fa784009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.803+0000 7fa7951c5740 1 -- start start 2024-09-15T22:47:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.804+0000 7fa7951c5740 1 Processor -- start 2024-09-15T22:47:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.804+0000 7fa7951c5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d65da7f260 0x55d65da89440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.804+0000 7fa7951c5740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65da7fba0 0x55d65da89980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.804+0000 7fa7951c5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d65da84c30 0x55d65da909c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.804+0000 7fa7951c5740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d65da87ba0 con 0x55d65da7fba0 2024-09-15T22:47:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.804+0000 7fa7951c5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d65d96d090 con 0x55d65da7f260 2024-09-15T22:47:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.804+0000 7fa7951c5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d65da87d20 con 0x55d65da84c30 2024-09-15T22:47:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.804+0000 7fa7951c5740 10 client.0 _ll_register_callbacks cb 0x55d65d9c1f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.805+0000 7fa78be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d65da84c30 0x55d65da909c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.805+0000 7fa78aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65da7fba0 0x55d65da89980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.805+0000 7fa78be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d65da84c30 0x55d65da909c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:55164/0 (socket says 172.21.15.111:55164) 2024-09-15T22:47:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.805+0000 7fa78be00640 1 -- 172.21.15.111:0/1653232089 learned_addr learned my addr 172.21.15.111:0/1653232089 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:47:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.805+0000 7fa78b400640 1 --2- 172.21.15.111:0/1653232089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d65da7f260 0x55d65da89440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.805+0000 7fa78be00640 1 -- 172.21.15.111:0/1653232089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d65da7f260 msgr2=0x55d65da89440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:07.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.805+0000 7fa78be00640 1 --2- 172.21.15.111:0/1653232089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d65da7f260 0x55d65da89440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.805+0000 7fa78be00640 1 -- 172.21.15.111:0/1653232089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65da7fba0 msgr2=0x55d65da89980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.805+0000 7fa78be00640 1 --2- 172.21.15.111:0/1653232089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65da7fba0 0x55d65da89980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.805+0000 7fa78be00640 1 -- 172.21.15.111:0/1653232089 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d65d7c3620 con 0x55d65da84c30 2024-09-15T22:47:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.805+0000 7fa78aa00640 1 --2- 172.21.15.111:0/1653232089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65da7fba0 0x55d65da89980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.806+0000 7fa78be00640 1 --2- 172.21.15.111:0/1653232089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d65da84c30 0x55d65da909c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa784003fa0 tx=0x7fa784004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2369f68d75f4149 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.806+0000 7fa774c00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:47:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.806+0000 7fa774c00640 1 -- 172.21.15.111:0/1653232089 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78400f910 con 0x55d65da84c30 2024-09-15T22:47:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.806+0000 7fa774c00640 1 -- 172.21.15.111:0/1653232089 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fa78400fab0 con 0x55d65da84c30 2024-09-15T22:47:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.806+0000 7fa7951c5740 1 -- 172.21.15.111:0/1653232089 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa77c005500 con 0x55d65da84c30 2024-09-15T22:47:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.807+0000 7fa774c00640 1 -- 172.21.15.111:0/1653232089 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa78400fcb0 con 0x55d65da84c30 2024-09-15T22:47:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.807+0000 7fa7951c5740 10 client.25114 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.807+0000 7fa7951c5740 1 -- 172.21.15.111:0/1653232089 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d65da937f0 con 0x55d65da84c30 2024-09-15T22:47:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.807+0000 7fa774c00640 1 -- 172.21.15.111:0/1653232089 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa77c005500 con 0x55d65da84c30 2024-09-15T22:47:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.808+0000 7fa774c00640 1 -- 172.21.15.111:0/1653232089 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fa784027070 con 0x55d65da84c30 2024-09-15T22:47:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.808+0000 7fa7951c5740 10 client.25114 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.808+0000 7fa7951c5740 10 client.25114 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.808+0000 7fa7951c5740 20 client.25114 populate_metadata read hostname 'smithi111' 2024-09-15T22:47:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.808+0000 7fa7951c5740 1 --2- 172.21.15.111:0/1653232089 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d65da94ee0 0x55d65da972d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.808+0000 7fa7951c5740 4 client.25114 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:47:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.808+0000 7fa7951c5740 1 -- 172.21.15.111:0/1653232089 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d65d9f1830 con 0x55d65da94ee0 2024-09-15T22:47:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.808+0000 7fa78b400640 1 --2- 172.21.15.111:0/1653232089 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d65da94ee0 0x55d65da972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.809+0000 7fa78b400640 1 --2- 172.21.15.111:0/1653232089 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d65da94ee0 0x55d65da972d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fb3df6c1555382a server_cookie=7f783d59e0289503 in_seq=0 out_seq=0 2024-09-15T22:47:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.809+0000 7fa774c00640 10 client.25114 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:47:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.810+0000 7fa774c00640 1 -- 172.21.15.111:0/1653232089 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d65d9f1830 con 0x55d65da94ee0 2024-09-15T22:47:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.810+0000 7fa774c00640 10 client.25114 handle_command_reply: tid=0 2024-09-15T22:47:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.810+0000 7fa7951c5740 1 client.25114 shutdown 2024-09-15T22:47:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.810+0000 7fa7951c5740 1 -- 172.21.15.111:0/1653232089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d65da84c30 msgr2=0x55d65da909c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.810+0000 7fa7951c5740 1 --2- 172.21.15.111:0/1653232089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d65da84c30 0x55d65da909c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa784003fa0 tx=0x7fa784004ab0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.810+0000 7fa7951c5740 1 -- 172.21.15.111:0/1653232089 shutdown_connections 2024-09-15T22:47:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.810+0000 7fa7951c5740 1 --2- 172.21.15.111:0/1653232089 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d65da94ee0 0x55d65da972d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.810+0000 7fa7951c5740 1 --2- 172.21.15.111:0/1653232089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d65da84c30 0x55d65da909c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.810+0000 7fa7951c5740 1 --2- 172.21.15.111:0/1653232089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65da7fba0 0x55d65da89980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.810+0000 7fa7951c5740 1 --2- 172.21.15.111:0/1653232089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d65da7f260 0x55d65da89440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.810+0000 7fa7951c5740 1 -- 172.21.15.111:0/1653232089 >> 172.21.15.111:0/1653232089 conn(0x55d65da7a8d0 msgr2=0x55d65da7c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.810+0000 7fa774c00640 0 client.25114 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:47:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.811+0000 7fa7951c5740 1 -- 172.21.15.111:0/1653232089 shutdown_connections 2024-09-15T22:47:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.811+0000 7fa7951c5740 1 -- 172.21.15.111:0/1653232089 wait complete. 2024-09-15T22:47:07.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.811+0000 7fa7951c5740 20 client.25114 trim_cache size 0 max 16384 2024-09-15T22:47:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.840+0000 7fa7951c5740 1 Processor -- start 2024-09-15T22:47:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.840+0000 7fa7951c5740 1 -- start start 2024-09-15T22:47:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.841+0000 7fa7951c5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d65da7f260 0x55d65da89440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.841+0000 7fa7951c5740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65da7fba0 0x55d65da90980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.841+0000 7fa7951c5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d65da84c30 0x55d65da89980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.841+0000 7fa7951c5740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c1b1bf0 con 0x55d65da7fba0 2024-09-15T22:47:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.841+0000 7fa7951c5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c1af4c0 con 0x55d65da84c30 2024-09-15T22:47:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.841+0000 7fa7951c5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c1ad310 con 0x55d65da7f260 2024-09-15T22:47:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.841+0000 7fa78aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65da7fba0 0x55d65da90980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.841+0000 7fa78be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d65da84c30 0x55d65da89980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.841+0000 7fa78aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65da7fba0 0x55d65da90980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40006/0 (socket says 172.21.15.111:40006) 2024-09-15T22:47:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.841+0000 7fa78aa00640 1 -- 172.21.15.111:0/4257978163 learned_addr learned my addr 172.21.15.111:0/4257978163 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:47:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.841+0000 7fa78be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d65da84c30 0x55d65da89980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:54366/0 (socket says 172.21.15.111:54366) 2024-09-15T22:47:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.841+0000 7fa78b400640 1 --2- 172.21.15.111:0/4257978163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d65da7f260 0x55d65da89440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.842+0000 7fa78be00640 1 -- 172.21.15.111:0/4257978163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d65da7f260 msgr2=0x55d65da89440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.842+0000 7fa78be00640 1 --2- 172.21.15.111:0/4257978163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d65da7f260 0x55d65da89440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.842+0000 7fa78be00640 1 -- 172.21.15.111:0/4257978163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65da7fba0 msgr2=0x55d65da90980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.842+0000 7fa78be00640 1 --2- 172.21.15.111:0/4257978163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65da7fba0 0x55d65da90980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.842+0000 7fa78be00640 1 -- 172.21.15.111:0/4257978163 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d65da891f0 con 0x55d65da84c30 2024-09-15T22:47:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.842+0000 7fa78be00640 1 --2- 172.21.15.111:0/4257978163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d65da84c30 0x55d65da89980 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa78400a010 tx=0x7fa78400f500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9afa73a2f59440a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.842+0000 7fa777400640 1 -- 172.21.15.111:0/4257978163 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa784003d90 con 0x55d65da84c30 2024-09-15T22:47:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.842+0000 7fa777400640 1 -- 172.21.15.111:0/4257978163 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa784038020 con 0x55d65da84c30 2024-09-15T22:47:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.842+0000 7fa777400640 1 -- 172.21.15.111:0/4257978163 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa784029440 con 0x55d65da84c30 2024-09-15T22:47:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.843+0000 7fa7951c5740 1 -- 172.21.15.111:0/4257978163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d65da84c30 msgr2=0x55d65da89980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.843+0000 7fa7951c5740 1 --2- 172.21.15.111:0/4257978163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d65da84c30 0x55d65da89980 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa78400a010 tx=0x7fa78400f500 comp rx=0 tx=0).stop 2024-09-15T22:47:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.843+0000 7fa7951c5740 1 -- 172.21.15.111:0/4257978163 shutdown_connections 2024-09-15T22:47:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.843+0000 7fa7951c5740 1 --2- 172.21.15.111:0/4257978163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d65da84c30 0x55d65da89980 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.843+0000 7fa7951c5740 1 --2- 172.21.15.111:0/4257978163 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65da7fba0 0x55d65da90980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.843+0000 7fa7951c5740 1 --2- 172.21.15.111:0/4257978163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d65da7f260 0x55d65da89440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.843+0000 7fa7951c5740 1 -- 172.21.15.111:0/4257978163 >> 172.21.15.111:0/4257978163 conn(0x55d65da7a8d0 msgr2=0x55d65da80e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.844+0000 7fa7951c5740 1 -- 172.21.15.111:0/4257978163 shutdown_connections 2024-09-15T22:47:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.844+0000 7fa7951c5740 1 -- 172.21.15.111:0/4257978163 wait complete. 2024-09-15T22:47:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.844+0000 7fa7951c5740 1 -- start start 2024-09-15T22:47:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.844+0000 7fa7951c5740 1 Processor -- start 2024-09-15T22:47:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.845+0000 7fa7951c5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d65da7f260 0x55d65da8d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.845+0000 7fa7951c5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d65da7fba0 0x55d65da8dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.845+0000 7fa7951c5740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65da84c30 0x55d65dafdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.845+0000 7fa7951c5740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c1ad980 con 0x55d65da84c30 2024-09-15T22:47:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.845+0000 7fa7951c5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c1aedf0 con 0x55d65da7fba0 2024-09-15T22:47:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.845+0000 7fa7951c5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa78c1ae050 con 0x55d65da7f260 2024-09-15T22:47:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.845+0000 7fa7951c5740 10 client.0 _ll_register_callbacks cb 0x7fa784009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.845+0000 7fa78b400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d65da7f260 0x55d65da8d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.845+0000 7fa78aa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d65da7fba0 0x55d65da8dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.845+0000 7fa78be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65da84c30 0x55d65dafdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.845+0000 7fa78b400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d65da7f260 0x55d65da8d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:55188/0 (socket says 172.21.15.111:55188) 2024-09-15T22:47:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.845+0000 7fa78b400640 1 -- 172.21.15.111:0/1044138214 learned_addr learned my addr 172.21.15.111:0/1044138214 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:47:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.846+0000 7fa78b400640 1 -- 172.21.15.111:0/1044138214 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d65da7fba0 msgr2=0x55d65da8dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.846+0000 7fa78b400640 1 --2- 172.21.15.111:0/1044138214 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d65da7fba0 0x55d65da8dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.846+0000 7fa78b400640 1 -- 172.21.15.111:0/1044138214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65da84c30 msgr2=0x55d65dafdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.846+0000 7fa78b400640 1 --2- 172.21.15.111:0/1044138214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65da84c30 0x55d65dafdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.846+0000 7fa78b400640 1 -- 172.21.15.111:0/1044138214 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d65da8e2e0 con 0x55d65da7f260 2024-09-15T22:47:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.846+0000 7fa78aa00640 1 --2- 172.21.15.111:0/1044138214 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d65da7fba0 0x55d65da8dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:47:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.846+0000 7fa78b400640 1 --2- 172.21.15.111:0/1044138214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d65da7f260 0x55d65da8d760 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa77c02a930 tx=0x7fa77c004520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21a817f6bedab693 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.846+0000 7fa774c00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:47:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.847+0000 7fa774c00640 1 -- 172.21.15.111:0/1044138214 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c040040 con 0x55d65da7f260 2024-09-15T22:47:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.847+0000 7fa7951c5740 1 -- 172.21.15.111:0/1044138214 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa784038020 con 0x55d65da7f260 2024-09-15T22:47:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.847+0000 7fa774c00640 1 -- 172.21.15.111:0/1044138214 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fa77c004f50 con 0x55d65da7f260 2024-09-15T22:47:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.847+0000 7fa774c00640 1 -- 172.21.15.111:0/1044138214 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa77c060050 con 0x55d65da7f260 2024-09-15T22:47:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.847+0000 7fa7951c5740 10 client.25120 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.847+0000 7fa7951c5740 1 -- 172.21.15.111:0/1044138214 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d65daff220 con 0x55d65da7f260 2024-09-15T22:47:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.848+0000 7fa774c00640 1 -- 172.21.15.111:0/1044138214 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa784038020 con 0x55d65da7f260 2024-09-15T22:47:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.848+0000 7fa774c00640 1 -- 172.21.15.111:0/1044138214 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7fa77c031b90 con 0x55d65da7f260 2024-09-15T22:47:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.848+0000 7fa7951c5740 10 client.25120 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.848+0000 7fa7951c5740 10 client.25120 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.848+0000 7fa7951c5740 20 client.25120 populate_metadata read hostname 'smithi111' 2024-09-15T22:47:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.848+0000 7fa7951c5740 1 --2- 172.21.15.111:0/1044138214 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d65daff790 0x55d65db01b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.848+0000 7fa7951c5740 4 client.25120 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:47:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.848+0000 7fa7951c5740 1 -- 172.21.15.111:0/1044138214 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d65d9ebb80 con 0x55d65daff790 2024-09-15T22:47:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.848+0000 7fa78aa00640 1 --2- 172.21.15.111:0/1044138214 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d65daff790 0x55d65db01b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.849+0000 7fa78aa00640 1 --2- 172.21.15.111:0/1044138214 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d65daff790 0x55d65db01b80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8b5da57534e5c64 server_cookie=8396c173ae945868 in_seq=0 out_seq=0 2024-09-15T22:47:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.849+0000 7fa774c00640 10 client.25120 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:47:07.859 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.857+0000 7fa774c00640 1 -- 172.21.15.111:0/1044138214 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d65d9ebb80 con 0x55d65daff790 2024-09-15T22:47:07.859 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.857+0000 7fa774c00640 10 client.25120 handle_command_reply: tid=0 2024-09-15T22:47:07.859 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.857+0000 7fa7951c5740 1 client.25120 shutdown 2024-09-15T22:47:07.860 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.857+0000 7fa7951c5740 1 -- 172.21.15.111:0/1044138214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d65da7f260 msgr2=0x55d65da8d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:07.860 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.857+0000 7fa7951c5740 1 --2- 172.21.15.111:0/1044138214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d65da7f260 0x55d65da8d760 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa77c02a930 tx=0x7fa77c004520 comp rx=0 tx=0).stop 2024-09-15T22:47:07.860 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.857+0000 7fa7951c5740 1 -- 172.21.15.111:0/1044138214 shutdown_connections 2024-09-15T22:47:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.857+0000 7fa7951c5740 1 --2- 172.21.15.111:0/1044138214 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d65daff790 0x55d65db01b80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.857+0000 7fa7951c5740 1 --2- 172.21.15.111:0/1044138214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d65da84c30 0x55d65dafdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.857+0000 7fa774c00640 0 client.25120 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:47:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.857+0000 7fa7951c5740 1 --2- 172.21.15.111:0/1044138214 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d65da7fba0 0x55d65da8dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.857+0000 7fa7951c5740 1 --2- 172.21.15.111:0/1044138214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d65da7f260 0x55d65da8d760 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.857+0000 7fa7951c5740 1 -- 172.21.15.111:0/1044138214 >> 172.21.15.111:0/1044138214 conn(0x55d65da7a8d0 msgr2=0x55d65da89440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.858+0000 7fa7951c5740 1 -- 172.21.15.111:0/1044138214 shutdown_connections 2024-09-15T22:47:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.858+0000 7fa7951c5740 1 -- 172.21.15.111:0/1044138214 wait complete. 2024-09-15T22:47:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.858+0000 7fa7951c5740 20 client.25120 trim_cache size 0 max 16384 2024-09-15T22:47:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:47:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:47:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:47:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:47:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.859+0000 7fa790c00640 1 -- 172.21.15.111:0/3955579853 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa7700a33d0 msgr2=0x7fa7700a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.859+0000 7fa790c00640 1 --2- 172.21.15.111:0/3955579853 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa7700a33d0 0x7fa7700a5890 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa784005fd0 tx=0x7fa784009500 comp rx=0 tx=0).stop 2024-09-15T22:47:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.859+0000 7fa790c00640 1 -- 172.21.15.111:0/3955579853 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa78c104d60 msgr2=0x7fa78c07a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.859+0000 7fa790c00640 1 --2- 172.21.15.111:0/3955579853 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa78c104d60 0x7fa78c07a4e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa77800e9f0 tx=0x7fa77800edf0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.859+0000 7fa790c00640 1 -- 172.21.15.111:0/3955579853 shutdown_connections 2024-09-15T22:47:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.859+0000 7fa790c00640 1 --2- 172.21.15.111:0/3955579853 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa7700a33d0 0x7fa7700a5890 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.859+0000 7fa790c00640 1 --2- 172.21.15.111:0/3955579853 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa78c105720 0x7fa78c076b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.859+0000 7fa790c00640 1 --2- 172.21.15.111:0/3955579853 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa78c104d60 0x7fa78c07a4e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.859+0000 7fa790c00640 1 --2- 172.21.15.111:0/3955579853 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa78c103b70 0x7fa78c079fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.859+0000 7fa790c00640 1 -- 172.21.15.111:0/3955579853 >> 172.21.15.111:0/3955579853 conn(0x7fa78c0ff2f0 msgr2=0x7fa78c108120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.861+0000 7fa790c00640 1 -- 172.21.15.111:0/3955579853 shutdown_connections 2024-09-15T22:47:07.864 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:07.861+0000 7fa790c00640 1 -- 172.21.15.111:0/3955579853 wait complete. 2024-09-15T22:47:07.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:07 smithi121 ceph-mon[34815]: pgmap v386: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-15T22:47:07.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:07 smithi111 ceph-mon[30629]: pgmap v386: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-15T22:47:08.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:07 smithi192 ceph-mon[35427]: pgmap v386: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-15T22:47:09.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:09 smithi121 ceph-mon[34815]: pgmap v387: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-15T22:47:09.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:09 smithi111 ceph-mon[30629]: pgmap v387: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-15T22:47:10.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:09 smithi192 ceph-mon[35427]: pgmap v387: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-09-15T22:47:11.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:11 smithi121 ceph-mon[34815]: pgmap v388: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-15T22:47:11.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:11 smithi111 ceph-mon[30629]: pgmap v388: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-15T22:47:12.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:11 smithi192 ceph-mon[35427]: pgmap v388: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-15T22:47:13.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:13 smithi121 ceph-mon[34815]: pgmap v389: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-15T22:47:13.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:13 smithi111 ceph-mon[30629]: pgmap v389: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-15T22:47:14.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:13 smithi192 ceph-mon[35427]: pgmap v389: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-15T22:47:15.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:15 smithi121 ceph-mon[34815]: pgmap v390: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-15T22:47:15.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:15 smithi111 ceph-mon[30629]: pgmap v390: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-15T22:47:16.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:15 smithi192 ceph-mon[35427]: pgmap v390: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-15T22:47:17.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:17 smithi121 ceph-mon[34815]: pgmap v391: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-15T22:47:17.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:17 smithi111 ceph-mon[30629]: pgmap v391: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-15T22:47:18.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:17 smithi192 ceph-mon[35427]: pgmap v391: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-15T22:47:19.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:19 smithi121 ceph-mon[34815]: pgmap v392: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-15T22:47:19.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:19 smithi111 ceph-mon[30629]: pgmap v392: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-15T22:47:20.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:19 smithi192 ceph-mon[35427]: pgmap v392: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-15T22:47:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.036+0000 7f45b5800640 1 -- 172.21.15.111:0/751981982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45b0078790 msgr2=0x7f45b0079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.036+0000 7f45b5800640 1 --2- 172.21.15.111:0/751981982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45b0078790 0x7f45b0079030 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009ca0 tx=0x7f45a802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.036+0000 7f45b5800640 1 -- 172.21.15.111:0/751981982 shutdown_connections 2024-09-15T22:47:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.036+0000 7f45b5800640 1 --2- 172.21.15.111:0/751981982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45b0078790 0x7f45b0079030 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.036+0000 7f45b5800640 1 --2- 172.21.15.111:0/751981982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45b0077df0 0x7f45b0078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.036+0000 7f45b5800640 1 --2- 172.21.15.111:0/751981982 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45b0076bb0 0x7f45b0076fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.036+0000 7f45b5800640 1 -- 172.21.15.111:0/751981982 >> 172.21.15.111:0/751981982 conn(0x7f45b00ff2f0 msgr2=0x7f45b0101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.036+0000 7f45b5800640 1 -- 172.21.15.111:0/751981982 shutdown_connections 2024-09-15T22:47:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.036+0000 7f45b5800640 1 -- 172.21.15.111:0/751981982 wait complete. 2024-09-15T22:47:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.037+0000 7f45b5800640 1 Processor -- start 2024-09-15T22:47:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.037+0000 7f45b5800640 1 -- start start 2024-09-15T22:47:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.037+0000 7f45b5800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45b0076bb0 0x7f45b019e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.037+0000 7f45b5800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45b0077df0 0x7f45b019e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.037+0000 7f45b5800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45b0078790 0x7f45b01a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.037+0000 7f45b5800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f45b0113c50 con 0x7f45b0078790 2024-09-15T22:47:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.037+0000 7f45b5800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f45b0113ad0 con 0x7f45b0077df0 2024-09-15T22:47:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.037+0000 7f45b5800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f45b0113dd0 con 0x7f45b0076bb0 2024-09-15T22:47:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.038+0000 7f45afe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45b0076bb0 0x7f45b019e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.038+0000 7f45afe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45b0076bb0 0x7f45b019e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:60632/0 (socket says 172.21.15.111:60632) 2024-09-15T22:47:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.038+0000 7f45afe00640 1 -- 172.21.15.111:0/3297602562 learned_addr learned my addr 172.21.15.111:0/3297602562 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:47:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.038+0000 7f45af400640 1 --2- 172.21.15.111:0/3297602562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45b0077df0 0x7f45b019e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.038+0000 7f45b4e00640 1 --2- 172.21.15.111:0/3297602562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45b0078790 0x7f45b01a57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.038+0000 7f45afe00640 1 -- 172.21.15.111:0/3297602562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45b0077df0 msgr2=0x7f45b019e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.038+0000 7f45afe00640 1 --2- 172.21.15.111:0/3297602562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45b0077df0 0x7f45b019e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.038+0000 7f45afe00640 1 -- 172.21.15.111:0/3297602562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45b0078790 msgr2=0x7f45b01a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.038+0000 7f45afe00640 1 --2- 172.21.15.111:0/3297602562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45b0078790 0x7f45b01a57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.038+0000 7f45afe00640 1 -- 172.21.15.111:0/3297602562 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b01a5e50 con 0x7f45b0076bb0 2024-09-15T22:47:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.038+0000 7f45b4e00640 1 --2- 172.21.15.111:0/3297602562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45b0078790 0x7f45b01a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.038+0000 7f45afe00640 1 --2- 172.21.15.111:0/3297602562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45b0076bb0 0x7f45b019e230 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f45a000e9f0 tx=0x7f45a000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.039+0000 7f45acc00640 1 -- 172.21.15.111:0/3297602562 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a000cd80 con 0x7f45b0076bb0 2024-09-15T22:47:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.039+0000 7f45acc00640 1 -- 172.21.15.111:0/3297602562 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45a0010910 con 0x7f45b0076bb0 2024-09-15T22:47:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.039+0000 7f45b5800640 1 -- 172.21.15.111:0/3297602562 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b01a6140 con 0x7f45b0076bb0 2024-09-15T22:47:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.039+0000 7f45acc00640 1 -- 172.21.15.111:0/3297602562 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a00188f0 con 0x7f45b0076bb0 2024-09-15T22:47:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.039+0000 7f45b5800640 1 -- 172.21.15.111:0/3297602562 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b01a66e0 con 0x7f45b0076bb0 2024-09-15T22:47:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.042+0000 7f45acc00640 1 -- 172.21.15.111:0/3297602562 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f45a0010ac0 con 0x7f45b0076bb0 2024-09-15T22:47:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.042+0000 7f45b9dc4740 1 Processor -- start 2024-09-15T22:47:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.042+0000 7f45b9dc4740 1 -- start start 2024-09-15T22:47:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.043+0000 7f45b9dc4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d78093b260 0x55d78093b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.043+0000 7f45b9dc4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d78093bba0 0x55d780940660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.043+0000 7f45b9dc4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d780940c30 0x55d7809430f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.043+0000 7f45b9dc4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d7808a0100 con 0x55d78093b260 2024-09-15T22:47:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.043+0000 7f45b9dc4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d7808481b0 con 0x55d78093bba0 2024-09-15T22:47:21.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.043+0000 7f45b9dc4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d78089c0c0 con 0x55d780940c30 2024-09-15T22:47:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.043+0000 7f45b4e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d78093b260 0x55d78093b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.043+0000 7f45b4e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d78093b260 0x55d78093b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59288/0 (socket says 172.21.15.111:59288) 2024-09-15T22:47:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.043+0000 7f45b4e00640 1 -- 172.21.15.111:0/2326862476 learned_addr learned my addr 172.21.15.111:0/2326862476 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:47:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.043+0000 7f45af400640 1 --2- 172.21.15.111:0/2326862476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d780940c30 0x55d7809430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.043+0000 7f45afe00640 1 --2- 172.21.15.111:0/2326862476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d78093bba0 0x55d780940660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.044+0000 7f45b4e00640 1 -- 172.21.15.111:0/2326862476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d780940c30 msgr2=0x55d7809430f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.044+0000 7f45b4e00640 1 --2- 172.21.15.111:0/2326862476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d780940c30 0x55d7809430f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.044+0000 7f45b4e00640 1 -- 172.21.15.111:0/2326862476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d78093bba0 msgr2=0x55d780940660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.044+0000 7f45b4e00640 1 --2- 172.21.15.111:0/2326862476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d78093bba0 0x55d780940660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.044+0000 7f45b4e00640 1 -- 172.21.15.111:0/2326862476 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7805b3210 con 0x55d78093b260 2024-09-15T22:47:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.044+0000 7f45af400640 1 --2- 172.21.15.111:0/2326862476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d780940c30 0x55d7809430f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:47:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.044+0000 7f45acc00640 1 --2- 172.21.15.111:0/3297602562 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f45940a3350 0x7f45940a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.044+0000 7f45b4e00640 1 --2- 172.21.15.111:0/2326862476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d78093b260 0x55d78093b660 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f45a8005e30 tx=0x7f45a8005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37043e30130d459b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.044+0000 7f45afe00640 1 --2- 172.21.15.111:0/3297602562 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f45940a3350 0x7f45940a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.044+0000 7f45acc00640 1 -- 172.21.15.111:0/3297602562 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f45a00cfe90 con 0x7f45b0076bb0 2024-09-15T22:47:21.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.045+0000 7f45a5600640 1 -- 172.21.15.111:0/2326862476 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8031650 con 0x55d78093b260 2024-09-15T22:47:21.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.045+0000 7f45a5600640 1 -- 172.21.15.111:0/2326862476 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45a8036040 con 0x55d78093b260 2024-09-15T22:47:21.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.045+0000 7f45a5600640 1 -- 172.21.15.111:0/2326862476 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8004760 con 0x55d78093b260 2024-09-15T22:47:21.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.046+0000 7f45b9dc4740 1 -- 172.21.15.111:0/2326862476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d78093b260 msgr2=0x55d78093b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:21.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.046+0000 7f45afe00640 1 --2- 172.21.15.111:0/3297602562 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f45940a3350 0x7f45940a5810 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f45a009c050 tx=0x7f45a00d3940 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:21.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.046+0000 7f45b9dc4740 1 --2- 172.21.15.111:0/2326862476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d78093b260 0x55d78093b660 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f45a8005e30 tx=0x7f45a8005f70 comp rx=0 tx=0).stop 2024-09-15T22:47:21.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.046+0000 7f45b9dc4740 1 -- 172.21.15.111:0/2326862476 shutdown_connections 2024-09-15T22:47:21.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.046+0000 7f45b9dc4740 1 --2- 172.21.15.111:0/2326862476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d780940c30 0x55d7809430f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.046+0000 7f45b9dc4740 1 --2- 172.21.15.111:0/2326862476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d78093bba0 0x55d780940660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.046+0000 7f45b9dc4740 1 --2- 172.21.15.111:0/2326862476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d78093b260 0x55d78093b660 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.046+0000 7f45b9dc4740 1 -- 172.21.15.111:0/2326862476 >> 172.21.15.111:0/2326862476 conn(0x55d7809368d0 msgr2=0x55d780936cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:21.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.046+0000 7f45b9dc4740 1 -- 172.21.15.111:0/2326862476 shutdown_connections 2024-09-15T22:47:21.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.046+0000 7f45b9dc4740 1 -- 172.21.15.111:0/2326862476 wait complete. 2024-09-15T22:47:21.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.047+0000 7f45b9dc4740 1 -- start start 2024-09-15T22:47:21.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.047+0000 7f45b9dc4740 1 Processor -- start 2024-09-15T22:47:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.047+0000 7f45b9dc4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d78093b260 0x55d780945410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.047+0000 7f45b9dc4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d78093bba0 0x55d780945950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.048+0000 7f45b9dc4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d780940c30 0x55d78094c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:21.052 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.048+0000 7f45b9dc4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d780943ba0 con 0x55d780940c30 2024-09-15T22:47:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.048+0000 7f45b9dc4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d780829090 con 0x55d78093b260 2024-09-15T22:47:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.048+0000 7f45b9dc4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d780943d20 con 0x55d78093bba0 2024-09-15T22:47:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.048+0000 7f45b9dc4740 10 client.0 _ll_register_callbacks cb 0x55d78087df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.048+0000 7f45af400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d78093bba0 0x55d780945950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.048+0000 7f45af400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d78093bba0 0x55d780945950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:60650/0 (socket says 172.21.15.111:60650) 2024-09-15T22:47:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.048+0000 7f45b4e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d780940c30 0x55d78094c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.048+0000 7f45af400640 1 -- 172.21.15.111:0/2942701416 learned_addr learned my addr 172.21.15.111:0/2942701416 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:47:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.048+0000 7f45b4e00640 1 --2- 172.21.15.111:0/2942701416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d78093b260 0x55d780945410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:21.053 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.048+0000 7f45af400640 1 -- 172.21.15.111:0/2942701416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d78093b260 msgr2=0x55d780945410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.048+0000 7f45af400640 1 --2- 172.21.15.111:0/2942701416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d78093b260 0x55d780945410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.048+0000 7f45af400640 1 -- 172.21.15.111:0/2942701416 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d780940c30 msgr2=0x55d78094c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.048+0000 7f45af400640 1 --2- 172.21.15.111:0/2942701416 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d780940c30 0x55d78094c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.048+0000 7f45af400640 1 -- 172.21.15.111:0/2942701416 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d78067f620 con 0x55d78093bba0 2024-09-15T22:47:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.049+0000 7f45af400640 1 --2- 172.21.15.111:0/2942701416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d78093bba0 0x55d780945950 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f459c012720 tx=0x7f459c012be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c6288392032327c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.049+0000 7f459a000640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:47:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.049+0000 7f459a000640 1 -- 172.21.15.111:0/2942701416 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c004050 con 0x55d78093bba0 2024-09-15T22:47:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.049+0000 7f45b9dc4740 1 -- 172.21.15.111:0/2942701416 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45a8036040 con 0x55d78093bba0 2024-09-15T22:47:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.049+0000 7f459a000640 1 -- 172.21.15.111:0/2942701416 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f459c0041f0 con 0x55d78093bba0 2024-09-15T22:47:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.049+0000 7f459a000640 1 -- 172.21.15.111:0/2942701416 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f459c0043f0 con 0x55d78093bba0 2024-09-15T22:47:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.049+0000 7f45b9dc4740 10 client.25132 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:21.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.049+0000 7f45b9dc4740 1 -- 172.21.15.111:0/2942701416 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d78094f8a0 con 0x55d78093bba0 2024-09-15T22:47:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.050+0000 7f459a000640 1 -- 172.21.15.111:0/2942701416 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45a8036040 con 0x55d78093bba0 2024-09-15T22:47:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.050+0000 7f459a000640 1 -- 172.21.15.111:0/2942701416 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f459c003c10 con 0x55d78093bba0 2024-09-15T22:47:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.050+0000 7f45b9dc4740 10 client.25132 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.050+0000 7f45b9dc4740 10 client.25132 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.050+0000 7f45b9dc4740 20 client.25132 populate_metadata read hostname 'smithi111' 2024-09-15T22:47:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.050+0000 7f45b9dc4740 1 --2- 172.21.15.111:0/2942701416 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d780950fc0 0x55d7809533b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.050+0000 7f45b9dc4740 4 client.25132 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:47:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.050+0000 7f45b9dc4740 1 -- 172.21.15.111:0/2942701416 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7808ad830 con 0x55d780950fc0 2024-09-15T22:47:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.051+0000 7f45b4e00640 1 --2- 172.21.15.111:0/2942701416 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d780950fc0 0x55d7809533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:21.055 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.051+0000 7f459a000640 10 client.25132 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:47:21.056 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.051+0000 7f45b4e00640 1 --2- 172.21.15.111:0/2942701416 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d780950fc0 0x55d7809533b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3578301d755dbb2c server_cookie=2444d632358228f4 in_seq=0 out_seq=0 2024-09-15T22:47:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.052+0000 7f459a000640 1 -- 172.21.15.111:0/2942701416 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d7808ad830 con 0x55d780950fc0 2024-09-15T22:47:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.052+0000 7f459a000640 10 client.25132 handle_command_reply: tid=0 2024-09-15T22:47:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.052+0000 7f45b9dc4740 1 client.25132 shutdown 2024-09-15T22:47:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.052+0000 7f45b9dc4740 1 -- 172.21.15.111:0/2942701416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d78093bba0 msgr2=0x55d780945950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.052+0000 7f45b9dc4740 1 --2- 172.21.15.111:0/2942701416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d78093bba0 0x55d780945950 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f459c012720 tx=0x7f459c012be0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.057 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.052+0000 7f45b9dc4740 1 -- 172.21.15.111:0/2942701416 shutdown_connections 2024-09-15T22:47:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.052+0000 7f45b9dc4740 1 --2- 172.21.15.111:0/2942701416 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d780950fc0 0x55d7809533b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.052+0000 7f45b9dc4740 1 --2- 172.21.15.111:0/2942701416 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d780940c30 0x55d78094c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.052+0000 7f45b9dc4740 1 --2- 172.21.15.111:0/2942701416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d78093bba0 0x55d780945950 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.052+0000 7f45b9dc4740 1 --2- 172.21.15.111:0/2942701416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d78093b260 0x55d780945410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.052+0000 7f45b9dc4740 1 -- 172.21.15.111:0/2942701416 >> 172.21.15.111:0/2942701416 conn(0x55d7809368d0 msgr2=0x55d7809380e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.052+0000 7f459a000640 0 client.25132 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:47:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.052+0000 7f45b9dc4740 1 -- 172.21.15.111:0/2942701416 shutdown_connections 2024-09-15T22:47:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.052+0000 7f45b9dc4740 1 -- 172.21.15.111:0/2942701416 wait complete. 2024-09-15T22:47:21.058 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.053+0000 7f45b9dc4740 20 client.25132 trim_cache size 0 max 16384 2024-09-15T22:47:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.067+0000 7f45b9dc4740 1 Processor -- start 2024-09-15T22:47:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.067+0000 7f45b9dc4740 1 -- start start 2024-09-15T22:47:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.068+0000 7f45b9dc4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d78093b260 0x55d78093b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.068+0000 7f45b9dc4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d78093bba0 0x55d78094c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.068+0000 7f45b9dc4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d780940c30 0x55d780937750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.068+0000 7f45b9dc4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f45b01b1bc0 con 0x55d78093b260 2024-09-15T22:47:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.068+0000 7f45b9dc4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f45b01af490 con 0x55d780940c30 2024-09-15T22:47:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.068+0000 7f45b9dc4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f45b01ad2e0 con 0x55d78093bba0 2024-09-15T22:47:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.068+0000 7f45b4e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d78093b260 0x55d78093b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.068+0000 7f45b4e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d78093b260 0x55d78093b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59294/0 (socket says 172.21.15.111:59294) 2024-09-15T22:47:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.068+0000 7f45b4e00640 1 -- 172.21.15.111:0/9464579 learned_addr learned my addr 172.21.15.111:0/9464579 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:47:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.068+0000 7f45af400640 1 --2- 172.21.15.111:0/9464579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d78093bba0 0x55d78094c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.068+0000 7f45b4e00640 1 -- 172.21.15.111:0/9464579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d78093bba0 msgr2=0x55d78094c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.068+0000 7f45b4e00640 1 --2- 172.21.15.111:0/9464579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d78093bba0 0x55d78094c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.068+0000 7f45b4e00640 1 -- 172.21.15.111:0/9464579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d780940c30 msgr2=0x55d780937750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.068+0000 7f45b4e00640 1 --2- 172.21.15.111:0/9464579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d780940c30 0x55d780937750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.068+0000 7f45b4e00640 1 -- 172.21.15.111:0/9464579 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d780937560 con 0x55d78093b260 2024-09-15T22:47:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.069+0000 7f45b4e00640 1 --2- 172.21.15.111:0/9464579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d78093b260 0x55d78093b640 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f45a8005fe0 tx=0x7f45a802f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f585be42e9091659 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.069+0000 7f45a4c00640 1 -- 172.21.15.111:0/9464579 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8057020 con 0x55d78093b260 2024-09-15T22:47:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.069+0000 7f45a4c00640 1 -- 172.21.15.111:0/9464579 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45a8036070 con 0x55d78093b260 2024-09-15T22:47:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.069+0000 7f45a4c00640 1 -- 172.21.15.111:0/9464579 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8003bd0 con 0x55d78093b260 2024-09-15T22:47:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.070+0000 7f45b9dc4740 1 -- 172.21.15.111:0/9464579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d78093b260 msgr2=0x55d78093b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.070+0000 7f45b9dc4740 1 --2- 172.21.15.111:0/9464579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d78093b260 0x55d78093b640 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f45a8005fe0 tx=0x7f45a802f310 comp rx=0 tx=0).stop 2024-09-15T22:47:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.070+0000 7f45b9dc4740 1 -- 172.21.15.111:0/9464579 shutdown_connections 2024-09-15T22:47:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.070+0000 7f45b9dc4740 1 --2- 172.21.15.111:0/9464579 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d780940c30 0x55d780937750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.070+0000 7f45b9dc4740 1 --2- 172.21.15.111:0/9464579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d78093bba0 0x55d78094c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.070+0000 7f45b9dc4740 1 --2- 172.21.15.111:0/9464579 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d78093b260 0x55d78093b640 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.070+0000 7f45b9dc4740 1 -- 172.21.15.111:0/9464579 >> 172.21.15.111:0/9464579 conn(0x55d7809368d0 msgr2=0x55d78093cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.070+0000 7f45b9dc4740 1 -- 172.21.15.111:0/9464579 shutdown_connections 2024-09-15T22:47:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.070+0000 7f45b9dc4740 1 -- 172.21.15.111:0/9464579 wait complete. 2024-09-15T22:47:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.070+0000 7f45b9dc4740 1 -- start start 2024-09-15T22:47:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.070+0000 7f45b9dc4740 1 Processor -- start 2024-09-15T22:47:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.071+0000 7f45b9dc4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d78093b260 0x55d78094a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.071+0000 7f45b9dc4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d78093bba0 0x55d78094a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:21.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.071+0000 7f45b9dc4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d780940c30 0x55d78094b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.071+0000 7f45b9dc4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f45b01ad950 con 0x55d78093b260 2024-09-15T22:47:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.071+0000 7f45b9dc4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f45b01aedc0 con 0x55d78093bba0 2024-09-15T22:47:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.071+0000 7f45b9dc4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f45b01ae020 con 0x55d780940c30 2024-09-15T22:47:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.071+0000 7f45b9dc4740 10 client.0 _ll_register_callbacks cb 0x7f459c009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.071+0000 7f45b4e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d78093b260 0x55d78094a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.071+0000 7f45b4e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d78093b260 0x55d78094a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59304/0 (socket says 172.21.15.111:59304) 2024-09-15T22:47:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.071+0000 7f45b4e00640 1 -- 172.21.15.111:0/997049702 learned_addr learned my addr 172.21.15.111:0/997049702 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:47:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.071+0000 7f45af400640 1 --2- 172.21.15.111:0/997049702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d78093bba0 0x55d78094a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.071+0000 7f45b4e00640 1 --2- 172.21.15.111:0/997049702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d780940c30 0x55d78094b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.072+0000 7f45b4e00640 1 -- 172.21.15.111:0/997049702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d780940c30 msgr2=0x55d78094b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.072+0000 7f45b4e00640 1 --2- 172.21.15.111:0/997049702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d780940c30 0x55d78094b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.072+0000 7f45b4e00640 1 -- 172.21.15.111:0/997049702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d78093bba0 msgr2=0x55d78094a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.072+0000 7f45b4e00640 1 --2- 172.21.15.111:0/997049702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d78093bba0 0x55d78094a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.072+0000 7f45b4e00640 1 -- 172.21.15.111:0/997049702 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d78094bb80 con 0x55d78093b260 2024-09-15T22:47:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.072+0000 7f45b4e00640 1 --2- 172.21.15.111:0/997049702 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d78093b260 0x55d78094a020 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f45a8005e30 tx=0x7f45a80041e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46eb08b7c567bae2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.072+0000 7f459a000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:47:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.072+0000 7f459a000640 1 -- 172.21.15.111:0/997049702 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8057070 con 0x55d78093b260 2024-09-15T22:47:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.072+0000 7f459a000640 1 -- 172.21.15.111:0/997049702 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f45a80316d0 con 0x55d78093b260 2024-09-15T22:47:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.072+0000 7f45b9dc4740 1 -- 172.21.15.111:0/997049702 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45a8036070 con 0x55d78093b260 2024-09-15T22:47:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.073+0000 7f459a000640 1 -- 172.21.15.111:0/997049702 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f45a80318d0 con 0x55d78093b260 2024-09-15T22:47:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.073+0000 7f45b9dc4740 10 client.15108 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.073+0000 7f45b9dc4740 1 -- 172.21.15.111:0/997049702 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d7809bbc50 con 0x55d78093b260 2024-09-15T22:47:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.073+0000 7f459a000640 1 -- 172.21.15.111:0/997049702 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45a8036070 con 0x55d78093b260 2024-09-15T22:47:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.073+0000 7f459a000640 1 -- 172.21.15.111:0/997049702 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f45a802f7d0 con 0x55d78093b260 2024-09-15T22:47:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.073+0000 7f45b9dc4740 10 client.15108 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.073+0000 7f45b9dc4740 10 client.15108 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.073+0000 7f45b9dc4740 20 client.15108 populate_metadata read hostname 'smithi111' 2024-09-15T22:47:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.073+0000 7f45b9dc4740 1 --2- 172.21.15.111:0/997049702 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d7809bc020 0x55d7809be410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.073+0000 7f45b9dc4740 4 client.15108 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:47:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.073+0000 7f45b9dc4740 1 -- 172.21.15.111:0/997049702 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d7808a7b80 con 0x55d7809bc020 2024-09-15T22:47:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.074+0000 7f45af400640 1 --2- 172.21.15.111:0/997049702 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d7809bc020 0x55d7809be410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.074+0000 7f45af400640 1 --2- 172.21.15.111:0/997049702 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d7809bc020 0x55d7809be410 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc9c99c8e7f9c1e2 server_cookie=182081451b8c0758 in_seq=0 out_seq=0 2024-09-15T22:47:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.074+0000 7f459a000640 10 client.15108 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:47:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.078+0000 7f459a000640 1 -- 172.21.15.111:0/997049702 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d7808a7b80 con 0x55d7809bc020 2024-09-15T22:47:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.078+0000 7f459a000640 10 client.15108 handle_command_reply: tid=0 2024-09-15T22:47:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.078+0000 7f45b9dc4740 1 client.15108 shutdown 2024-09-15T22:47:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.078+0000 7f45b9dc4740 1 -- 172.21.15.111:0/997049702 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d78093b260 msgr2=0x55d78094a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:21.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.078+0000 7f45b9dc4740 1 --2- 172.21.15.111:0/997049702 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d78093b260 0x55d78094a020 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f45a8005e30 tx=0x7f45a80041e0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.078+0000 7f45b9dc4740 1 -- 172.21.15.111:0/997049702 shutdown_connections 2024-09-15T22:47:21.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.078+0000 7f45b9dc4740 1 --2- 172.21.15.111:0/997049702 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d7809bc020 0x55d7809be410 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.078+0000 7f45b9dc4740 1 --2- 172.21.15.111:0/997049702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d780940c30 0x55d78094b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.078+0000 7f45b9dc4740 1 --2- 172.21.15.111:0/997049702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d78093bba0 0x55d78094a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.078+0000 7f45b9dc4740 1 --2- 172.21.15.111:0/997049702 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d78093b260 0x55d78094a020 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.078+0000 7f45b9dc4740 1 -- 172.21.15.111:0/997049702 >> 172.21.15.111:0/997049702 conn(0x55d7809368d0 msgr2=0x55d7809351f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:21.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.078+0000 7f459a000640 0 client.15108 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:47:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.078+0000 7f45b9dc4740 1 -- 172.21.15.111:0/997049702 shutdown_connections 2024-09-15T22:47:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.078+0000 7f45b9dc4740 1 -- 172.21.15.111:0/997049702 wait complete. 2024-09-15T22:47:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.078+0000 7f45b9dc4740 20 client.15108 trim_cache size 0 max 16384 2024-09-15T22:47:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:47:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:47:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:47:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:47:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.079+0000 7f45b5800640 1 -- 172.21.15.111:0/3297602562 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f45940a3350 msgr2=0x7f45940a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.079+0000 7f45b5800640 1 --2- 172.21.15.111:0/3297602562 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f45940a3350 0x7f45940a5810 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f45a009c050 tx=0x7f45a00d3940 comp rx=0 tx=0).stop 2024-09-15T22:47:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.079+0000 7f45b5800640 1 -- 172.21.15.111:0/3297602562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45b0076bb0 msgr2=0x7f45b019e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.079+0000 7f45b5800640 1 --2- 172.21.15.111:0/3297602562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45b0076bb0 0x7f45b019e230 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f45a000e9f0 tx=0x7f45a000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.079+0000 7f45b5800640 1 -- 172.21.15.111:0/3297602562 shutdown_connections 2024-09-15T22:47:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.079+0000 7f45b5800640 1 --2- 172.21.15.111:0/3297602562 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45b0078790 0x7f45b01a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.079+0000 7f45b5800640 1 --2- 172.21.15.111:0/3297602562 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f45940a3350 0x7f45940a5810 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.079+0000 7f45b5800640 1 --2- 172.21.15.111:0/3297602562 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45b0077df0 0x7f45b019e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.079+0000 7f45b5800640 1 --2- 172.21.15.111:0/3297602562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45b0076bb0 0x7f45b019e230 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.079+0000 7f45b5800640 1 -- 172.21.15.111:0/3297602562 >> 172.21.15.111:0/3297602562 conn(0x7f45b00ff2f0 msgr2=0x7f45b0100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.080+0000 7f45b5800640 1 -- 172.21.15.111:0/3297602562 shutdown_connections 2024-09-15T22:47:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:21.080+0000 7f45b5800640 1 -- 172.21.15.111:0/3297602562 wait complete. 2024-09-15T22:47:21.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:21 smithi121 ceph-mon[34815]: pgmap v393: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-15T22:47:21.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:21 smithi111 ceph-mon[30629]: pgmap v393: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-15T22:47:22.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:21 smithi192 ceph-mon[35427]: pgmap v393: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-15T22:47:23.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:23 smithi121 ceph-mon[34815]: pgmap v394: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-15T22:47:23.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:23 smithi111 ceph-mon[30629]: pgmap v394: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-15T22:47:24.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:23 smithi192 ceph-mon[35427]: pgmap v394: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-15T22:47:25.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:25 smithi121 ceph-mon[34815]: pgmap v395: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:47:25.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:25 smithi111 ceph-mon[30629]: pgmap v395: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:47:26.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:25 smithi192 ceph-mon[35427]: pgmap v395: 97 pgs: 97 active+clean; 669 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:47:27.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:27 smithi121 ceph-mon[34815]: pgmap v396: 97 pgs: 97 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-15T22:47:27.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:27 smithi111 ceph-mon[30629]: pgmap v396: 97 pgs: 97 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-15T22:47:28.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:27 smithi192 ceph-mon[35427]: pgmap v396: 97 pgs: 97 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-15T22:47:29.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:29 smithi121 ceph-mon[34815]: pgmap v397: 97 pgs: 97 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-15T22:47:29.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:29 smithi111 ceph-mon[30629]: pgmap v397: 97 pgs: 97 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-15T22:47:30.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:29 smithi192 ceph-mon[35427]: pgmap v397: 97 pgs: 97 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-15T22:47:30.197 INFO:tasks.workunit.client.0.smithi111.stderr:Updating files: 65% (8436/12889) Updating files: 66% (8507/12889) Updating files: 67% (8636/12889) Updating files: 68% (8765/12889) Updating files: 69% (8894/12889) Updating files: 70% (9023/12889) Updating files: 71% (9152/12889) Updating files: 72% (9281/12889) Updating files: 73% (9409/12889) Updating files: 74% (9538/12889) Updating files: 75% (9667/12889) Updating files: 76% (9796/12889) Updating files: 77% (9925/12889) Updating files: 78% (10054/12889) Updating files: 79% (10183/12889) Updating files: 80% (10312/12889) Updating files: 81% (10441/12889) Updating files: 82% (10569/12889) Updating files: 83% (10698/12889) Updating files: 84% (10827/12889) Updating files: 85% (10956/12889) Updating files: 86% (11085/12889) Updating files: 87% (11214/12889) Updating files: 88% (11343/12889) Updating files: 89% (11472/12889) Updating files: 90% (11601/12889) Updating files: 91% (11729/12889) Updating files: 92% (11858/12889) Updating files: 93% (11987/12889) Updating files: 94% (12116/12889) Updating files: 95% (12245/12889) Updating files: 96% (12374/12889) Updating files: 97% (12503/12889) Updating files: 98% (12632/12889) Updating files: 99% (12761/12889) Updating files: 100% (12889/12889) Updating files: 100% (12889/12889), done. 2024-09-15T22:47:30.869 INFO:tasks.workunit.client.0.smithi111.stderr:Note: switching to 'ad35db06e8924912f01e32b35576297c0f1352b4'. 2024-09-15T22:47:30.869 INFO:tasks.workunit.client.0.smithi111.stderr: 2024-09-15T22:47:30.869 INFO:tasks.workunit.client.0.smithi111.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-15T22:47:30.869 INFO:tasks.workunit.client.0.smithi111.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-15T22:47:30.869 INFO:tasks.workunit.client.0.smithi111.stderr:state without impacting any branches by switching back to a branch. 2024-09-15T22:47:30.869 INFO:tasks.workunit.client.0.smithi111.stderr: 2024-09-15T22:47:30.869 INFO:tasks.workunit.client.0.smithi111.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-15T22:47:30.869 INFO:tasks.workunit.client.0.smithi111.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-15T22:47:30.870 INFO:tasks.workunit.client.0.smithi111.stderr: 2024-09-15T22:47:30.870 INFO:tasks.workunit.client.0.smithi111.stderr: git switch -c 2024-09-15T22:47:30.870 INFO:tasks.workunit.client.0.smithi111.stderr: 2024-09-15T22:47:30.870 INFO:tasks.workunit.client.0.smithi111.stderr:Or undo this operation with: 2024-09-15T22:47:30.870 INFO:tasks.workunit.client.0.smithi111.stderr: 2024-09-15T22:47:30.870 INFO:tasks.workunit.client.0.smithi111.stderr: git switch - 2024-09-15T22:47:30.870 INFO:tasks.workunit.client.0.smithi111.stderr: 2024-09-15T22:47:30.870 INFO:tasks.workunit.client.0.smithi111.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-15T22:47:30.870 INFO:tasks.workunit.client.0.smithi111.stderr: 2024-09-15T22:47:30.870 INFO:tasks.workunit.client.0.smithi111.stderr:HEAD is now at ad35db06e89 Merge pull request #59794 from zdover23/wip-doc-2024-09-15-backport-59613-to-squid 2024-09-15T22:47:30.882 DEBUG:teuthology.orchestra.run.smithi111:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-15T22:47:30.976 INFO:tasks.workunit.client.0.smithi111.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-15T22:47:30.981 INFO:tasks.workunit.client.0.smithi111.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-15T22:47:30.981 INFO:tasks.workunit.client.0.smithi111.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-15T22:47:31.747 INFO:tasks.workunit.client.0.smithi111.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-15T22:47:31.838 INFO:tasks.workunit.client.0.smithi111.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-15T22:47:31.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:31 smithi121 ceph-mon[34815]: pgmap v398: 97 pgs: 97 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-15T22:47:31.903 INFO:tasks.workunit.client.0.smithi111.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-15T22:47:31.907 INFO:tasks.workunit.client.0.smithi111.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-15T22:47:31.907 INFO:tasks.workunit.client.0.smithi111.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-15T22:47:31.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:31 smithi111 ceph-mon[30629]: pgmap v398: 97 pgs: 97 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-15T22:47:31.972 INFO:tasks.workunit.client.0.smithi111.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-15T22:47:31.979 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-15T22:47:31.979 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-15T22:47:32.014 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-09-15T22:47:32.015 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-09-15T22:47:32.016 DEBUG:teuthology.orchestra.run.smithi111:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ad35db06e8924912f01e32b35576297c0f1352b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-09-15T22:47:32.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:31 smithi192 ceph-mon[35427]: pgmap v398: 97 pgs: 97 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-15T22:47:32.103 INFO:tasks.workunit.client.0.smithi111.stderr:+ dbench 1 2024-09-15T22:47:32.121 INFO:tasks.workunit.client.0.smithi111.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-15T22:47:32.121 INFO:tasks.workunit.client.0.smithi111.stdout: 2024-09-15T22:47:32.121 INFO:tasks.workunit.client.0.smithi111.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-15T22:47:32.126 INFO:tasks.workunit.client.0.smithi111.stdout:0 of 1 processes prepared for launch 0 sec 2024-09-15T22:47:32.227 INFO:tasks.workunit.client.0.smithi111.stdout:1 of 1 processes prepared for launch 0 sec 2024-09-15T22:47:32.227 INFO:tasks.workunit.client.0.smithi111.stdout:releasing clients 2024-09-15T22:47:33.227 INFO:tasks.workunit.client.0.smithi111.stdout: 1 3628 38.59 MB/sec warmup 1 sec latency 6.796 ms 2024-09-15T22:47:33.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:33 smithi121 ceph-mon[34815]: pgmap v399: 97 pgs: 97 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-15T22:47:33.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:33 smithi111 ceph-mon[30629]: pgmap v399: 97 pgs: 97 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-15T22:47:34.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:33 smithi192 ceph-mon[35427]: pgmap v399: 97 pgs: 97 active+clean; 670 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-15T22:47:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.197+0000 7f3120000640 1 -- 172.21.15.111:0/4082001925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3118101b40 msgr2=0x7f3118101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.197+0000 7f3120000640 1 --2- 172.21.15.111:0/4082001925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3118101b40 0x7f3118101f40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f310c009900 tx=0x7f310c02a400 comp rx=0 tx=0).stop 2024-09-15T22:47:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.197+0000 7f3120000640 1 -- 172.21.15.111:0/4082001925 shutdown_connections 2024-09-15T22:47:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.197+0000 7f3120000640 1 --2- 172.21.15.111:0/4082001925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31181036f0 0x7f3118109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.197+0000 7f3120000640 1 --2- 172.21.15.111:0/4082001925 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3118102d30 0x7f31181031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.197+0000 7f3120000640 1 --2- 172.21.15.111:0/4082001925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3118101b40 0x7f3118101f40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.197+0000 7f3120000640 1 -- 172.21.15.111:0/4082001925 >> 172.21.15.111:0/4082001925 conn(0x7f31180fd2e0 msgr2=0x7f31180ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.197+0000 7f3120000640 1 -- 172.21.15.111:0/4082001925 shutdown_connections 2024-09-15T22:47:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.197+0000 7f3120000640 1 -- 172.21.15.111:0/4082001925 wait complete. 2024-09-15T22:47:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.198+0000 7f3120000640 1 Processor -- start 2024-09-15T22:47:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.198+0000 7f3120000640 1 -- start start 2024-09-15T22:47:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.198+0000 7f3120000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3118101b40 0x7f311819a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.198+0000 7f3120000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3118102d30 0x7f311819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.198+0000 7f3120000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31181036f0 0x7f31181a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.198+0000 7f3120000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f311810d810 con 0x7f31181036f0 2024-09-15T22:47:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.198+0000 7f3120000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f311810d690 con 0x7f3118101b40 2024-09-15T22:47:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.198+0000 7f3120000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f311810d990 con 0x7f3118102d30 2024-09-15T22:47:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.199+0000 7f311d600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3118101b40 0x7f311819a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.199+0000 7f311d600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3118101b40 0x7f311819a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:53220/0 (socket says 172.21.15.111:53220) 2024-09-15T22:47:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.199+0000 7f311cc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3118102d30 0x7f311819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.199+0000 7f311d600640 1 -- 172.21.15.111:0/1016951393 learned_addr learned my addr 172.21.15.111:0/1016951393 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:47:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.199+0000 7f311e000640 1 --2- 172.21.15.111:0/1016951393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31181036f0 0x7f31181a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.199+0000 7f311d600640 1 -- 172.21.15.111:0/1016951393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3118102d30 msgr2=0x7f311819a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.199+0000 7f311d600640 1 --2- 172.21.15.111:0/1016951393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3118102d30 0x7f311819a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.199+0000 7f311d600640 1 -- 172.21.15.111:0/1016951393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31181036f0 msgr2=0x7f31181a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.199+0000 7f311d600640 1 --2- 172.21.15.111:0/1016951393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31181036f0 0x7f31181a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.199+0000 7f311d600640 1 -- 172.21.15.111:0/1016951393 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31181a1c60 con 0x7f3118101b40 2024-09-15T22:47:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.199+0000 7f311cc00640 1 --2- 172.21.15.111:0/1016951393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3118102d30 0x7f311819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:47:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.199+0000 7f311d600640 1 --2- 172.21.15.111:0/1016951393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3118101b40 0x7f311819a040 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f310c031440 tx=0x7f310c032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.200+0000 7f3116000640 1 -- 172.21.15.111:0/1016951393 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c03f030 con 0x7f3118101b40 2024-09-15T22:47:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.200+0000 7f3116000640 1 -- 172.21.15.111:0/1016951393 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f310c003ce0 con 0x7f3118101b40 2024-09-15T22:47:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.200+0000 7f3120000640 1 -- 172.21.15.111:0/1016951393 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31181a1ef0 con 0x7f3118101b40 2024-09-15T22:47:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.200+0000 7f3116000640 1 -- 172.21.15.111:0/1016951393 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c035070 con 0x7f3118101b40 2024-09-15T22:47:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.200+0000 7f3120000640 1 -- 172.21.15.111:0/1016951393 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31181a2300 con 0x7f3118101b40 2024-09-15T22:47:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.203+0000 7f3122e8a740 1 Processor -- start 2024-09-15T22:47:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.203+0000 7f3116000640 1 -- 172.21.15.111:0/1016951393 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7f310c042030 con 0x7f3118101b40 2024-09-15T22:47:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.203+0000 7f3122e8a740 1 -- start start 2024-09-15T22:47:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.203+0000 7f3122e8a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f29b543090 0x55f29b543490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.203+0000 7f3122e8a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f29b5439d0 0x55f29b548490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.203+0000 7f3122e8a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f29b548a60 0x55f29b54af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.203+0000 7f3122e8a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f29b4a8100 con 0x55f29b5439d0 2024-09-15T22:47:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.203+0000 7f3122e8a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f29b4501b0 con 0x55f29b548a60 2024-09-15T22:47:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.203+0000 7f3122e8a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f29b4a40c0 con 0x55f29b543090 2024-09-15T22:47:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.203+0000 7f311e000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f29b543090 0x55f29b543490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.203+0000 7f311d600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f29b5439d0 0x55f29b548490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.203+0000 7f311cc00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f29b548a60 0x55f29b54af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.204+0000 7f311e000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f29b543090 0x55f29b543490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:52000/0 (socket says 172.21.15.111:52000) 2024-09-15T22:47:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.204+0000 7f311e000640 1 -- 172.21.15.111:0/1520643136 learned_addr learned my addr 172.21.15.111:0/1520643136 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:47:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.204+0000 7f311cc00640 1 -- 172.21.15.111:0/1520643136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f29b543090 msgr2=0x55f29b543490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.204+0000 7f311cc00640 1 --2- 172.21.15.111:0/1520643136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f29b543090 0x55f29b543490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.204+0000 7f311cc00640 1 -- 172.21.15.111:0/1520643136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f29b5439d0 msgr2=0x55f29b548490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.204+0000 7f311cc00640 1 --2- 172.21.15.111:0/1520643136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f29b5439d0 0x55f29b548490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.204+0000 7f311cc00640 1 -- 172.21.15.111:0/1520643136 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f29b1bb210 con 0x55f29b548a60 2024-09-15T22:47:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.204+0000 7f311e000640 1 --2- 172.21.15.111:0/1520643136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f29b543090 0x55f29b543490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:47:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.204+0000 7f311cc00640 1 --2- 172.21.15.111:0/1520643136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f29b548a60 0x55f29b54af20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3104009fa0 tx=0x7f31040118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7db94404255dc002 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.204+0000 7f310a600640 1 -- 172.21.15.111:0/1520643136 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3104004310 con 0x55f29b548a60 2024-09-15T22:47:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.204+0000 7f310a600640 1 -- 172.21.15.111:0/1520643136 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31040044b0 con 0x55f29b548a60 2024-09-15T22:47:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.204+0000 7f310a600640 1 -- 172.21.15.111:0/1520643136 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31040035f0 con 0x55f29b548a60 2024-09-15T22:47:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.205+0000 7f3116000640 1 --2- 172.21.15.111:0/1016951393 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f31000a3390 0x7f31000a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.205+0000 7f3116000640 1 -- 172.21.15.111:0/1016951393 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f310c0f0180 con 0x7f3118101b40 2024-09-15T22:47:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.205+0000 7f311e000640 1 --2- 172.21.15.111:0/1016951393 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f31000a3390 0x7f31000a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.205+0000 7f3122e8a740 1 -- 172.21.15.111:0/1520643136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f29b548a60 msgr2=0x55f29b54af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.205+0000 7f3122e8a740 1 --2- 172.21.15.111:0/1520643136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f29b548a60 0x55f29b54af20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3104009fa0 tx=0x7f31040118d0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.205+0000 7f3122e8a740 1 -- 172.21.15.111:0/1520643136 shutdown_connections 2024-09-15T22:47:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.205+0000 7f3122e8a740 1 --2- 172.21.15.111:0/1520643136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f29b548a60 0x55f29b54af20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.205+0000 7f3122e8a740 1 --2- 172.21.15.111:0/1520643136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f29b5439d0 0x55f29b548490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.205+0000 7f3122e8a740 1 --2- 172.21.15.111:0/1520643136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f29b543090 0x55f29b543490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.205+0000 7f3122e8a740 1 -- 172.21.15.111:0/1520643136 >> 172.21.15.111:0/1520643136 conn(0x55f29b53e700 msgr2=0x55f29b53eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.205+0000 7f3122e8a740 1 -- 172.21.15.111:0/1520643136 shutdown_connections 2024-09-15T22:47:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.206+0000 7f3122e8a740 1 -- 172.21.15.111:0/1520643136 wait complete. 2024-09-15T22:47:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.206+0000 7f311e000640 1 --2- 172.21.15.111:0/1016951393 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f31000a3390 0x7f31000a5850 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3110012840 tx=0x7f3110009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.206+0000 7f3122e8a740 1 -- start start 2024-09-15T22:47:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.206+0000 7f3122e8a740 1 Processor -- start 2024-09-15T22:47:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.206+0000 7f3122e8a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f29b543090 0x55f29b54d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.206+0000 7f3122e8a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f29b5439d0 0x55f29b54d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.207+0000 7f3122e8a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f29b548a60 0x55f29b554730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.207+0000 7f3122e8a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f29b54ba30 con 0x55f29b5439d0 2024-09-15T22:47:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.207+0000 7f3122e8a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f29b431090 con 0x55f29b548a60 2024-09-15T22:47:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.207+0000 7f3122e8a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f29b54bbb0 con 0x55f29b543090 2024-09-15T22:47:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.207+0000 7f3122e8a740 10 client.0 _ll_register_callbacks cb 0x55f29b485f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:34.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.207+0000 7f311d600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f29b543090 0x55f29b54d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.207+0000 7f311cc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f29b5439d0 0x55f29b54d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.207+0000 7f311e000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f29b548a60 0x55f29b554730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.207+0000 7f311d600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f29b543090 0x55f29b54d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:52016/0 (socket says 172.21.15.111:52016) 2024-09-15T22:47:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.207+0000 7f311d600640 1 -- 172.21.15.111:0/4076088349 learned_addr learned my addr 172.21.15.111:0/4076088349 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:47:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.207+0000 7f311d600640 1 -- 172.21.15.111:0/4076088349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f29b548a60 msgr2=0x55f29b554730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.207+0000 7f311d600640 1 --2- 172.21.15.111:0/4076088349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f29b548a60 0x55f29b554730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.207+0000 7f311d600640 1 -- 172.21.15.111:0/4076088349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f29b5439d0 msgr2=0x55f29b54d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.207+0000 7f311d600640 1 --2- 172.21.15.111:0/4076088349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f29b5439d0 0x55f29b54d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.207+0000 7f311d600640 1 -- 172.21.15.111:0/4076088349 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f29b287620 con 0x55f29b543090 2024-09-15T22:47:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.207+0000 7f311d600640 1 --2- 172.21.15.111:0/4076088349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f29b543090 0x55f29b54d1b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f310c032fd0 tx=0x7f310c0f3890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42286d914fc2295f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.207+0000 7f30fea00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:47:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.208+0000 7f30fea00640 1 -- 172.21.15.111:0/4076088349 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c049050 con 0x55f29b543090 2024-09-15T22:47:34.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.208+0000 7f30fea00640 1 -- 172.21.15.111:0/4076088349 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f310c03bc50 con 0x55f29b543090 2024-09-15T22:47:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.208+0000 7f3122e8a740 1 -- 172.21.15.111:0/4076088349 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31040044b0 con 0x55f29b543090 2024-09-15T22:47:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.208+0000 7f30fea00640 1 -- 172.21.15.111:0/4076088349 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f310c0b89e0 con 0x55f29b543090 2024-09-15T22:47:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.208+0000 7f3122e8a740 10 client.25150 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:34.213 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.208+0000 7f3122e8a740 1 -- 172.21.15.111:0/4076088349 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f29b54baa0 con 0x55f29b543090 2024-09-15T22:47:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.208+0000 7f30fea00640 1 -- 172.21.15.111:0/4076088349 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31040044b0 con 0x55f29b543090 2024-09-15T22:47:34.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.208+0000 7f30fea00640 1 -- 172.21.15.111:0/4076088349 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f310c0bc9a0 con 0x55f29b543090 2024-09-15T22:47:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.208+0000 7f3122e8a740 10 client.25150 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.208+0000 7f3122e8a740 10 client.25150 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.208+0000 7f3122e8a740 20 client.25150 populate_metadata read hostname 'smithi111' 2024-09-15T22:47:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.208+0000 7f3122e8a740 1 --2- 172.21.15.111:0/4076088349 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f29b558c70 0x55f29b55b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.208+0000 7f3122e8a740 4 client.25150 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:47:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.208+0000 7f3122e8a740 1 -- 172.21.15.111:0/4076088349 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f29b4b5830 con 0x55f29b558c70 2024-09-15T22:47:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.209+0000 7f311cc00640 1 --2- 172.21.15.111:0/4076088349 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f29b558c70 0x55f29b55b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.209+0000 7f311cc00640 1 --2- 172.21.15.111:0/4076088349 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f29b558c70 0x55f29b55b0a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d26c821e59ff171 server_cookie=fe127778e47d4aba in_seq=0 out_seq=0 2024-09-15T22:47:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.209+0000 7f30fea00640 10 client.25150 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:47:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.210+0000 7f30fea00640 1 -- 172.21.15.111:0/4076088349 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f29b4b5830 con 0x55f29b558c70 2024-09-15T22:47:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.210+0000 7f30fea00640 10 client.25150 handle_command_reply: tid=0 2024-09-15T22:47:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.210+0000 7f3122e8a740 1 client.25150 shutdown 2024-09-15T22:47:34.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.211+0000 7f3122e8a740 1 -- 172.21.15.111:0/4076088349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f29b543090 msgr2=0x55f29b54d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.211+0000 7f3122e8a740 1 --2- 172.21.15.111:0/4076088349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f29b543090 0x55f29b54d1b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f310c032fd0 tx=0x7f310c0f3890 comp rx=0 tx=0).stop 2024-09-15T22:47:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.211+0000 7f3122e8a740 1 -- 172.21.15.111:0/4076088349 shutdown_connections 2024-09-15T22:47:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.211+0000 7f3122e8a740 1 --2- 172.21.15.111:0/4076088349 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f29b558c70 0x55f29b55b0a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.211+0000 7f3122e8a740 1 --2- 172.21.15.111:0/4076088349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f29b548a60 0x55f29b554730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.211+0000 7f3122e8a740 1 --2- 172.21.15.111:0/4076088349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f29b5439d0 0x55f29b54d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.211+0000 7f3122e8a740 1 --2- 172.21.15.111:0/4076088349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f29b543090 0x55f29b54d1b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.211+0000 7f30fea00640 0 client.25150 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:47:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.211+0000 7f3122e8a740 1 -- 172.21.15.111:0/4076088349 >> 172.21.15.111:0/4076088349 conn(0x55f29b53e700 msgr2=0x55f29b53ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.211+0000 7f3122e8a740 1 -- 172.21.15.111:0/4076088349 shutdown_connections 2024-09-15T22:47:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.211+0000 7f3122e8a740 1 -- 172.21.15.111:0/4076088349 wait complete. 2024-09-15T22:47:34.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.211+0000 7f3122e8a740 20 client.25150 trim_cache size 0 max 16384 2024-09-15T22:47:34.227 INFO:tasks.workunit.client.0.smithi111.stdout: 1 6946 27.30 MB/sec warmup 2 sec latency 11.309 ms 2024-09-15T22:47:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.229+0000 7f3122e8a740 1 Processor -- start 2024-09-15T22:47:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.229+0000 7f3122e8a740 1 -- start start 2024-09-15T22:47:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.229+0000 7f3122e8a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f29b543090 0x55f29b54d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.229+0000 7f3122e8a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f29b5439d0 0x55f29b5546c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.229+0000 7f3122e8a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f29b548a60 0x55f29b53d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.229+0000 7f3122e8a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f31181ad950 con 0x55f29b548a60 2024-09-15T22:47:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.229+0000 7f3122e8a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f31181ab220 con 0x55f29b5439d0 2024-09-15T22:47:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.229+0000 7f3122e8a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f31181a9070 con 0x55f29b543090 2024-09-15T22:47:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.229+0000 7f311e000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f29b548a60 0x55f29b53d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.230+0000 7f311e000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f29b548a60 0x55f29b53d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55784/0 (socket says 172.21.15.111:55784) 2024-09-15T22:47:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.230+0000 7f311d600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f29b543090 0x55f29b54d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.230+0000 7f311e000640 1 -- 172.21.15.111:0/4096295445 learned_addr learned my addr 172.21.15.111:0/4096295445 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:47:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.230+0000 7f311cc00640 1 --2- 172.21.15.111:0/4096295445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f29b5439d0 0x55f29b5546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.230+0000 7f311d600640 1 -- 172.21.15.111:0/4096295445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f29b5439d0 msgr2=0x55f29b5546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.230+0000 7f311d600640 1 --2- 172.21.15.111:0/4096295445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f29b5439d0 0x55f29b5546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.230+0000 7f311d600640 1 -- 172.21.15.111:0/4096295445 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f29b548a60 msgr2=0x55f29b53d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.230+0000 7f311d600640 1 --2- 172.21.15.111:0/4096295445 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f29b548a60 0x55f29b53d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.230+0000 7f311d600640 1 -- 172.21.15.111:0/4096295445 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f29b53f390 con 0x55f29b543090 2024-09-15T22:47:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.230+0000 7f311d600640 1 --2- 172.21.15.111:0/4096295445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f29b543090 0x55f29b54d780 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f310c0b8ba0 tx=0x7f310c0b83e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0d32bc2ec72d60f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.230+0000 7f3109c00640 1 -- 172.21.15.111:0/4096295445 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c049070 con 0x55f29b543090 2024-09-15T22:47:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.230+0000 7f3109c00640 1 -- 172.21.15.111:0/4096295445 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f310c0bce40 con 0x55f29b543090 2024-09-15T22:47:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.230+0000 7f3109c00640 1 -- 172.21.15.111:0/4096295445 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c0bc0a0 con 0x55f29b543090 2024-09-15T22:47:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.231+0000 7f3122e8a740 1 -- 172.21.15.111:0/4096295445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f29b543090 msgr2=0x55f29b54d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.231+0000 7f3122e8a740 1 --2- 172.21.15.111:0/4096295445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f29b543090 0x55f29b54d780 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f310c0b8ba0 tx=0x7f310c0b83e0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.231+0000 7f3122e8a740 1 -- 172.21.15.111:0/4096295445 shutdown_connections 2024-09-15T22:47:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.231+0000 7f3122e8a740 1 --2- 172.21.15.111:0/4096295445 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f29b548a60 0x55f29b53d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.231+0000 7f3122e8a740 1 --2- 172.21.15.111:0/4096295445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f29b5439d0 0x55f29b5546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.231+0000 7f3122e8a740 1 --2- 172.21.15.111:0/4096295445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f29b543090 0x55f29b54d780 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.231+0000 7f3122e8a740 1 -- 172.21.15.111:0/4096295445 >> 172.21.15.111:0/4096295445 conn(0x55f29b53e700 msgr2=0x55f29b53d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.231+0000 7f3122e8a740 1 -- 172.21.15.111:0/4096295445 shutdown_connections 2024-09-15T22:47:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.231+0000 7f3122e8a740 1 -- 172.21.15.111:0/4096295445 wait complete. 2024-09-15T22:47:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.232+0000 7f3122e8a740 1 -- start start 2024-09-15T22:47:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.232+0000 7f3122e8a740 1 Processor -- start 2024-09-15T22:47:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.232+0000 7f3122e8a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f29b543090 0x55f29b551c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.232+0000 7f3122e8a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f29b5439d0 0x55f29b552160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.232+0000 7f3122e8a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f29b548a60 0x55f29b553170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.232+0000 7f3122e8a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f31181a96e0 con 0x55f29b548a60 2024-09-15T22:47:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.232+0000 7f3122e8a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f31181aab50 con 0x55f29b5439d0 2024-09-15T22:47:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.232+0000 7f3122e8a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f31181a9db0 con 0x55f29b543090 2024-09-15T22:47:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.232+0000 7f3122e8a740 10 client.0 _ll_register_callbacks cb 0x7f310c005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.233+0000 7f311d600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f29b543090 0x55f29b551c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.233+0000 7f311cc00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f29b5439d0 0x55f29b552160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.233+0000 7f311d600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f29b543090 0x55f29b551c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:52030/0 (socket says 172.21.15.111:52030) 2024-09-15T22:47:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.233+0000 7f311d600640 1 -- 172.21.15.111:0/1699020115 learned_addr learned my addr 172.21.15.111:0/1699020115 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:47:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.233+0000 7f311e000640 1 --2- 172.21.15.111:0/1699020115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f29b548a60 0x55f29b553170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.233+0000 7f311d600640 1 -- 172.21.15.111:0/1699020115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f29b5439d0 msgr2=0x55f29b552160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.233+0000 7f311d600640 1 --2- 172.21.15.111:0/1699020115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f29b5439d0 0x55f29b552160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.233+0000 7f311d600640 1 -- 172.21.15.111:0/1699020115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f29b548a60 msgr2=0x55f29b553170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.233+0000 7f311d600640 1 --2- 172.21.15.111:0/1699020115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f29b548a60 0x55f29b553170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.233+0000 7f311d600640 1 -- 172.21.15.111:0/1699020115 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f29b553780 con 0x55f29b543090 2024-09-15T22:47:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.233+0000 7f311e000640 1 --2- 172.21.15.111:0/1699020115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f29b548a60 0x55f29b553170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.233+0000 7f311d600640 1 --2- 172.21.15.111:0/1699020115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f29b543090 0x55f29b551c20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f310c004650 tx=0x7f310c0f3950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccabe4a19fb21d9d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.233+0000 7f30fea00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:47:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.233+0000 7f30fea00640 1 -- 172.21.15.111:0/1699020115 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c0d4070 con 0x55f29b543090 2024-09-15T22:47:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.233+0000 7f30fea00640 1 -- 172.21.15.111:0/1699020115 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f310c0c5120 con 0x55f29b543090 2024-09-15T22:47:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.233+0000 7f3122e8a740 1 -- 172.21.15.111:0/1699020115 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f310c0bce40 con 0x55f29b543090 2024-09-15T22:47:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.234+0000 7f30fea00640 1 -- 172.21.15.111:0/1699020115 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f310c049070 con 0x55f29b543090 2024-09-15T22:47:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.234+0000 7f3122e8a740 10 client.25156 fetch_fsmap learned FSMap version 26 2024-09-15T22:47:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.234+0000 7f3122e8a740 1 -- 172.21.15.111:0/1699020115 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f29b553780 con 0x55f29b543090 2024-09-15T22:47:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.234+0000 7f30fea00640 1 -- 172.21.15.111:0/1699020115 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f310c0bce40 con 0x55f29b543090 2024-09-15T22:47:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.234+0000 7f30fea00640 1 -- 172.21.15.111:0/1699020115 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 26) ==== 8346+0+0 (secure 0 0 0) 0x7f310c0bbd20 con 0x55f29b543090 2024-09-15T22:47:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.234+0000 7f3122e8a740 10 client.25156 fetch_fsmap finished waiting for FSMap version 26 2024-09-15T22:47:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.234+0000 7f3122e8a740 10 client.25156 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.234+0000 7f3122e8a740 20 client.25156 populate_metadata read hostname 'smithi111' 2024-09-15T22:47:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.234+0000 7f3122e8a740 1 --2- 172.21.15.111:0/1699020115 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f29b5c3cc0 0x55f29b5c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.234+0000 7f3122e8a740 4 client.25156 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:47:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.234+0000 7f3122e8a740 1 -- 172.21.15.111:0/1699020115 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f29b4afb80 con 0x55f29b5c3cc0 2024-09-15T22:47:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.234+0000 7f311cc00640 1 --2- 172.21.15.111:0/1699020115 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f29b5c3cc0 0x55f29b5c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.235+0000 7f311cc00640 1 --2- 172.21.15.111:0/1699020115 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f29b5c3cc0 0x55f29b5c60b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49c9a4ea03aebbed server_cookie=2158375847d2acf0 in_seq=0 out_seq=0 2024-09-15T22:47:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.235+0000 7f30fea00640 10 client.25156 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:47:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.303+0000 7f30fea00640 1 -- 172.21.15.111:0/1699020115 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f29b4afb80 con 0x55f29b5c3cc0 2024-09-15T22:47:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.303+0000 7f30fea00640 10 client.25156 handle_command_reply: tid=0 2024-09-15T22:47:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.303+0000 7f3122e8a740 1 client.25156 shutdown 2024-09-15T22:47:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.303+0000 7f3122e8a740 1 -- 172.21.15.111:0/1699020115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f29b543090 msgr2=0x55f29b551c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.303+0000 7f3122e8a740 1 --2- 172.21.15.111:0/1699020115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f29b543090 0x55f29b551c20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f310c004650 tx=0x7f310c0f3950 comp rx=0 tx=0).stop 2024-09-15T22:47:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.304+0000 7f3122e8a740 1 -- 172.21.15.111:0/1699020115 shutdown_connections 2024-09-15T22:47:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.304+0000 7f3122e8a740 1 --2- 172.21.15.111:0/1699020115 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f29b5c3cc0 0x55f29b5c60b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.304+0000 7f3122e8a740 1 --2- 172.21.15.111:0/1699020115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f29b548a60 0x55f29b553170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.304+0000 7f3122e8a740 1 --2- 172.21.15.111:0/1699020115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f29b5439d0 0x55f29b552160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.304+0000 7f3122e8a740 1 --2- 172.21.15.111:0/1699020115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f29b543090 0x55f29b551c20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.304+0000 7f30fea00640 0 client.25156 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:47:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.304+0000 7f3122e8a740 1 -- 172.21.15.111:0/1699020115 >> 172.21.15.111:0/1699020115 conn(0x55f29b53e700 msgr2=0x55f29b53f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.304+0000 7f3122e8a740 1 -- 172.21.15.111:0/1699020115 shutdown_connections 2024-09-15T22:47:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.304+0000 7f3122e8a740 1 -- 172.21.15.111:0/1699020115 wait complete. 2024-09-15T22:47:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.304+0000 7f3122e8a740 20 client.25156 trim_cache size 0 max 16384 2024-09-15T22:47:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:47:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:47:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:47:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:47:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.305+0000 7f3120000640 1 -- 172.21.15.111:0/1016951393 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f31000a3390 msgr2=0x7f31000a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.305+0000 7f3120000640 1 --2- 172.21.15.111:0/1016951393 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f31000a3390 0x7f31000a5850 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3110012840 tx=0x7f3110009500 comp rx=0 tx=0).stop 2024-09-15T22:47:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.305+0000 7f3120000640 1 -- 172.21.15.111:0/1016951393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3118101b40 msgr2=0x7f311819a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.305+0000 7f3120000640 1 --2- 172.21.15.111:0/1016951393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3118101b40 0x7f311819a040 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f310c031440 tx=0x7f310c032ce0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.305+0000 7f3120000640 1 -- 172.21.15.111:0/1016951393 shutdown_connections 2024-09-15T22:47:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.305+0000 7f3120000640 1 --2- 172.21.15.111:0/1016951393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f31181036f0 0x7f31181a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.305+0000 7f3120000640 1 --2- 172.21.15.111:0/1016951393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3118102d30 0x7f311819a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.305+0000 7f3120000640 1 --2- 172.21.15.111:0/1016951393 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f31000a3390 0x7f31000a5850 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.305+0000 7f3120000640 1 --2- 172.21.15.111:0/1016951393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3118101b40 0x7f311819a040 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.305+0000 7f3120000640 1 -- 172.21.15.111:0/1016951393 >> 172.21.15.111:0/1016951393 conn(0x7f31180fd2e0 msgr2=0x7f3118074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.305+0000 7f3120000640 1 -- 172.21.15.111:0/1016951393 shutdown_connections 2024-09-15T22:47:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:34.305+0000 7f3120000640 1 -- 172.21.15.111:0/1016951393 wait complete. 2024-09-15T22:47:35.227 INFO:tasks.workunit.client.0.smithi111.stdout: 1 9715 22.43 MB/sec warmup 3 sec latency 28.953 ms 2024-09-15T22:47:35.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:35 smithi121 ceph-mon[34815]: pgmap v400: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 41 KiB/s wr, 7 op/s 2024-09-15T22:47:35.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:35 smithi111 ceph-mon[30629]: pgmap v400: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 41 KiB/s wr, 7 op/s 2024-09-15T22:47:36.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:35 smithi192 ceph-mon[35427]: pgmap v400: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 41 KiB/s wr, 7 op/s 2024-09-15T22:47:36.227 INFO:tasks.workunit.client.0.smithi111.stdout: 1 11064 19.57 MB/sec warmup 4 sec latency 591.626 ms 2024-09-15T22:47:36.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:36 smithi111 ceph-mon[30629]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:47:37.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:36 smithi192 ceph-mon[35427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:47:37.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:36 smithi121 ceph-mon[34815]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:47:37.227 INFO:tasks.workunit.client.0.smithi111.stdout: 1 11064 15.66 MB/sec warmup 5 sec latency 1591.683 ms 2024-09-15T22:47:37.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:37 smithi111 ceph-mon[30629]: pgmap v401: 97 pgs: 97 active+clean; 4.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 431 KiB/s wr, 85 op/s 2024-09-15T22:47:37.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:37 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:47:37.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:37 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:47:37.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:37 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:47:37.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:37 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:47:37.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:37 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:47:38.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:37 smithi192 ceph-mon[35427]: pgmap v401: 97 pgs: 97 active+clean; 4.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 431 KiB/s wr, 85 op/s 2024-09-15T22:47:38.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:37 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:47:38.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:37 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:47:38.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:37 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:47:38.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:37 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:47:38.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:37 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:47:38.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:37 smithi121 ceph-mon[34815]: pgmap v401: 97 pgs: 97 active+clean; 4.9 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 431 KiB/s wr, 85 op/s 2024-09-15T22:47:38.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:47:38.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:47:38.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:47:38.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:47:38.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:37 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:47:38.227 INFO:tasks.workunit.client.0.smithi111.stdout: 1 11064 13.05 MB/sec warmup 6 sec latency 2591.729 ms 2024-09-15T22:47:38.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:38 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:47:38.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:38 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:47:38.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:38 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:47:38.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:38 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:47:38.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:38 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:47:39.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:38 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:47:39.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:38 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:47:39.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:38 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:47:39.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:38 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:47:39.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:38 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:47:39.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:38 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:47:39.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:38 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:47:39.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:38 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:47:39.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:38 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:47:39.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:38 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:47:39.227 INFO:tasks.workunit.client.0.smithi111.stdout: 1 11064 11.19 MB/sec warmup 7 sec latency 3591.814 ms 2024-09-15T22:47:40.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:39 smithi192 ceph-mon[35427]: pgmap v402: 97 pgs: 97 active+clean; 5.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 505 KiB/s wr, 109 op/s 2024-09-15T22:47:40.119 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:39 smithi111 ceph-mon[30629]: pgmap v402: 97 pgs: 97 active+clean; 5.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 505 KiB/s wr, 109 op/s 2024-09-15T22:47:40.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:39 smithi121 ceph-mon[34815]: pgmap v402: 97 pgs: 97 active+clean; 5.6 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 505 KiB/s wr, 109 op/s 2024-09-15T22:47:40.228 INFO:tasks.workunit.client.0.smithi111.stdout: 1 11563 9.88 MB/sec warmup 8 sec latency 4323.511 ms 2024-09-15T22:47:41.228 INFO:tasks.workunit.client.0.smithi111.stdout: 1 14860 10.84 MB/sec warmup 9 sec latency 9.765 ms 2024-09-15T22:47:42.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:41 smithi192 ceph-mon[35427]: pgmap v403: 97 pgs: 97 active+clean; 5.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 535 KiB/s wr, 124 op/s 2024-09-15T22:47:42.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:41 smithi121 ceph-mon[34815]: pgmap v403: 97 pgs: 97 active+clean; 5.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 535 KiB/s wr, 124 op/s 2024-09-15T22:47:42.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:41 smithi111 ceph-mon[30629]: pgmap v403: 97 pgs: 97 active+clean; 5.7 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 535 KiB/s wr, 124 op/s 2024-09-15T22:47:42.228 INFO:tasks.workunit.client.0.smithi111.stdout: 1 18323 11.66 MB/sec warmup 10 sec latency 12.811 ms 2024-09-15T22:47:43.228 INFO:tasks.workunit.client.0.smithi111.stdout: 1 21721 12.34 MB/sec warmup 11 sec latency 16.380 ms 2024-09-15T22:47:43.968 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:43 smithi111 ceph-mon[30629]: pgmap v404: 97 pgs: 97 active+clean; 6.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 610 KiB/s wr, 133 op/s 2024-09-15T22:47:44.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:43 smithi192 ceph-mon[35427]: pgmap v404: 97 pgs: 97 active+clean; 6.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 610 KiB/s wr, 133 op/s 2024-09-15T22:47:44.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:43 smithi121 ceph-mon[34815]: pgmap v404: 97 pgs: 97 active+clean; 6.2 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 610 KiB/s wr, 133 op/s 2024-09-15T22:47:44.228 INFO:tasks.workunit.client.0.smithi111.stdout: 1 25150 12.88 MB/sec warmup 12 sec latency 17.763 ms 2024-09-15T22:47:45.250 INFO:tasks.workunit.client.0.smithi111.stdout: 1 28381 13.25 MB/sec warmup 13 sec latency 14.717 ms 2024-09-15T22:47:46.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:45 smithi192 ceph-mon[35427]: pgmap v405: 97 pgs: 97 active+clean; 9.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 962 KiB/s wr, 186 op/s 2024-09-15T22:47:46.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:45 smithi121 ceph-mon[34815]: pgmap v405: 97 pgs: 97 active+clean; 9.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 962 KiB/s wr, 186 op/s 2024-09-15T22:47:46.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:45 smithi111 ceph-mon[30629]: pgmap v405: 97 pgs: 97 active+clean; 9.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 962 KiB/s wr, 186 op/s 2024-09-15T22:47:46.228 INFO:tasks.workunit.client.0.smithi111.stdout: 1 31415 13.32 MB/sec warmup 14 sec latency 10.980 ms 2024-09-15T22:47:47.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:46 smithi192 ceph-mon[35427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:47:47.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:46 smithi121 ceph-mon[34815]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:47:47.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:46 smithi111 ceph-mon[30629]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:47:47.228 INFO:tasks.workunit.client.0.smithi111.stdout: 1 34462 13.39 MB/sec warmup 15 sec latency 12.232 ms 2024-09-15T22:47:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.431+0000 7fedcfe00640 1 -- 172.21.15.111:0/3419430102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedd0101580 msgr2=0x7fedd0101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.431+0000 7fedcfe00640 1 --2- 172.21.15.111:0/3419430102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedd0101580 0x7fedd0101a00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009900 tx=0x7fedbc02a400 comp rx=0 tx=0).stop 2024-09-15T22:47:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.431+0000 7fedcfe00640 1 -- 172.21.15.111:0/3419430102 shutdown_connections 2024-09-15T22:47:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.431+0000 7fedcfe00640 1 --2- 172.21.15.111:0/3419430102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedd0101f40 0x7fedd0109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.431+0000 7fedcfe00640 1 --2- 172.21.15.111:0/3419430102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedd0101580 0x7fedd0101a00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.431+0000 7fedcfe00640 1 --2- 172.21.15.111:0/3419430102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fedd0103050 0x7fedd0103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.431+0000 7fedcfe00640 1 -- 172.21.15.111:0/3419430102 >> 172.21.15.111:0/3419430102 conn(0x7fedd00fd0d0 msgr2=0x7fedd00ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.431+0000 7fedcfe00640 1 -- 172.21.15.111:0/3419430102 shutdown_connections 2024-09-15T22:47:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.431+0000 7fedcfe00640 1 -- 172.21.15.111:0/3419430102 wait complete. 2024-09-15T22:47:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.432+0000 7fedcfe00640 1 Processor -- start 2024-09-15T22:47:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.432+0000 7fedcfe00640 1 -- start start 2024-09-15T22:47:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.432+0000 7fedcfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedd0101580 0x7fedd0197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.432+0000 7fedcfe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedd0101f40 0x7fedd0198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.433+0000 7fedcfe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fedd0103050 0x7fedd019f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.433+0000 7fedcfe00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fedd010d720 con 0x7fedd0101580 2024-09-15T22:47:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.433+0000 7fedcfe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fedd010d5a0 con 0x7fedd0101f40 2024-09-15T22:47:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.433+0000 7fedcfe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fedd010d8a0 con 0x7fedd0103050 2024-09-15T22:47:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.433+0000 7fedce000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedd0101f40 0x7fedd0198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.433+0000 7fedcea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedd0101580 0x7fedd0197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.433+0000 7fedce000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedd0101f40 0x7fedd0198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:55680/0 (socket says 172.21.15.111:55680) 2024-09-15T22:47:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.433+0000 7fedce000640 1 -- 172.21.15.111:0/835483045 learned_addr learned my addr 172.21.15.111:0/835483045 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:47:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.433+0000 7fedcf400640 1 --2- 172.21.15.111:0/835483045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fedd0103050 0x7fedd019f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.433+0000 7fedce000640 1 -- 172.21.15.111:0/835483045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fedd0103050 msgr2=0x7fedd019f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:47.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.433+0000 7fedce000640 1 --2- 172.21.15.111:0/835483045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fedd0103050 0x7fedd019f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.433+0000 7fedce000640 1 -- 172.21.15.111:0/835483045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedd0101580 msgr2=0x7fedd0197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.433+0000 7fedce000640 1 --2- 172.21.15.111:0/835483045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedd0101580 0x7fedd0197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.433+0000 7fedce000640 1 -- 172.21.15.111:0/835483045 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd019f960 con 0x7fedd0101f40 2024-09-15T22:47:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.433+0000 7fedcf400640 1 --2- 172.21.15.111:0/835483045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fedd0103050 0x7fedd019f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.433+0000 7fedcea00640 1 --2- 172.21.15.111:0/835483045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedd0101580 0x7fedd0197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:47:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.434+0000 7fedce000640 1 --2- 172.21.15.111:0/835483045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedd0101f40 0x7fedd0198280 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fedbc031440 tx=0x7fedbc032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.434+0000 7fedc7400640 1 -- 172.21.15.111:0/835483045 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedbc03f030 con 0x7fedd0101f40 2024-09-15T22:47:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.434+0000 7fedc7400640 1 -- 172.21.15.111:0/835483045 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedbc003ce0 con 0x7fedd0101f40 2024-09-15T22:47:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.434+0000 7fedcfe00640 1 -- 172.21.15.111:0/835483045 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd019fbf0 con 0x7fedd0101f40 2024-09-15T22:47:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.434+0000 7fedc7400640 1 -- 172.21.15.111:0/835483045 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedbc035070 con 0x7fedd0101f40 2024-09-15T22:47:47.437 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.434+0000 7fedcfe00640 1 -- 172.21.15.111:0/835483045 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd01a00e0 con 0x7fedd0101f40 2024-09-15T22:47:47.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.437+0000 7fedc7400640 1 -- 172.21.15.111:0/835483045 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 19) ==== 138407+0+0 (secure 0 0 0) 0x7fedbc042030 con 0x7fedd0101f40 2024-09-15T22:47:47.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.437+0000 7fedd8be9740 1 Processor -- start 2024-09-15T22:47:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.437+0000 7fedd8be9740 1 -- start start 2024-09-15T22:47:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.437+0000 7fedd8be9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665d210670 0x55665d210a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.437+0000 7fedd8be9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55665d210fb0 0x55665d215a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.437+0000 7fedd8be9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55665d216040 0x55665d218500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.437+0000 7fedd8be9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55665d171100 con 0x55665d216040 2024-09-15T22:47:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.437+0000 7fedd8be9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55665d1191b0 con 0x55665d210fb0 2024-09-15T22:47:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.437+0000 7fedd8be9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55665d16d0c0 con 0x55665d210670 2024-09-15T22:47:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.438+0000 7fedcf400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665d210670 0x55665d210a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.438+0000 7fedcf400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665d210670 0x55665d210a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:51110/0 (socket says 172.21.15.111:51110) 2024-09-15T22:47:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.438+0000 7fedcf400640 1 -- 172.21.15.111:0/1074670319 learned_addr learned my addr 172.21.15.111:0/1074670319 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:47:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.438+0000 7fedce000640 1 --2- 172.21.15.111:0/1074670319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55665d216040 0x55665d218500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.438+0000 7fedcea00640 1 --2- 172.21.15.111:0/1074670319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55665d210fb0 0x55665d215a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.438+0000 7fedcf400640 1 -- 172.21.15.111:0/1074670319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55665d210fb0 msgr2=0x55665d215a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.438+0000 7fedcf400640 1 --2- 172.21.15.111:0/1074670319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55665d210fb0 0x55665d215a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.438+0000 7fedcf400640 1 -- 172.21.15.111:0/1074670319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55665d216040 msgr2=0x55665d218500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.438+0000 7fedcf400640 1 --2- 172.21.15.111:0/1074670319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55665d216040 0x55665d218500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.438+0000 7fedcf400640 1 -- 172.21.15.111:0/1074670319 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55665ce84210 con 0x55665d210670 2024-09-15T22:47:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.438+0000 7fedcea00640 1 --2- 172.21.15.111:0/1074670319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55665d210fb0 0x55665d215a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.438+0000 7fedcf400640 1 --2- 172.21.15.111:0/1074670319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665d210670 0x55665d210a70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009cc0 tx=0x7fedc80117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24c1fd12e49447d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.438+0000 7fedc7400640 1 --2- 172.21.15.111:0/835483045 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fedb40a32d0 0x7fedb40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.438+0000 7fedbbe00640 1 -- 172.21.15.111:0/1074670319 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8012450 con 0x55665d210670 2024-09-15T22:47:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.439+0000 7fedbbe00640 1 -- 172.21.15.111:0/1074670319 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc8012ad0 con 0x55665d210670 2024-09-15T22:47:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.439+0000 7fedbbe00640 1 -- 172.21.15.111:0/1074670319 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc800fc30 con 0x55665d210670 2024-09-15T22:47:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.439+0000 7fedc7400640 1 -- 172.21.15.111:0/835483045 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fedbc0f0060 con 0x7fedd0101f40 2024-09-15T22:47:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.439+0000 7fedcea00640 1 --2- 172.21.15.111:0/835483045 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fedb40a32d0 0x7fedb40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.439+0000 7fedd8be9740 1 -- 172.21.15.111:0/1074670319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665d210670 msgr2=0x55665d210a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.439+0000 7fedd8be9740 1 --2- 172.21.15.111:0/1074670319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665d210670 0x55665d210a70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009cc0 tx=0x7fedc80117b0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.439+0000 7fedcea00640 1 --2- 172.21.15.111:0/835483045 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fedb40a32d0 0x7fedb40a5790 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55665d2119e0 tx=0x7fedc0007600 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.439+0000 7fedd8be9740 1 -- 172.21.15.111:0/1074670319 shutdown_connections 2024-09-15T22:47:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.439+0000 7fedd8be9740 1 --2- 172.21.15.111:0/1074670319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55665d216040 0x55665d218500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.439+0000 7fedd8be9740 1 --2- 172.21.15.111:0/1074670319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55665d210fb0 0x55665d215a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.439+0000 7fedd8be9740 1 --2- 172.21.15.111:0/1074670319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665d210670 0x55665d210a70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.439+0000 7fedd8be9740 1 -- 172.21.15.111:0/1074670319 >> 172.21.15.111:0/1074670319 conn(0x55665d20bce0 msgr2=0x55665d20c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.440+0000 7fedd8be9740 1 -- 172.21.15.111:0/1074670319 shutdown_connections 2024-09-15T22:47:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.440+0000 7fedd8be9740 1 -- 172.21.15.111:0/1074670319 wait complete. 2024-09-15T22:47:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.440+0000 7fedd8be9740 1 -- start start 2024-09-15T22:47:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.440+0000 7fedd8be9740 1 Processor -- start 2024-09-15T22:47:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.441+0000 7fedd8be9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665d210670 0x55665d21a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.441+0000 7fedd8be9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55665d210fb0 0x55665d21acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.441+0000 7fedd8be9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55665d216040 0x55665d221d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.441+0000 7fedd8be9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55665d218fb0 con 0x55665d210fb0 2024-09-15T22:47:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.441+0000 7fedd8be9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55665d0fa090 con 0x55665d216040 2024-09-15T22:47:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.441+0000 7fedd8be9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55665d219130 con 0x55665d210670 2024-09-15T22:47:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.441+0000 7fedd8be9740 10 client.0 _ll_register_callbacks cb 0x55665d14ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.441+0000 7fedcea00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665d210670 0x55665d21a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.441+0000 7fedcea00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665d210670 0x55665d21a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:51114/0 (socket says 172.21.15.111:51114) 2024-09-15T22:47:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.441+0000 7fedcea00640 1 -- 172.21.15.111:0/804486404 learned_addr learned my addr 172.21.15.111:0/804486404 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:47:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.441+0000 7fedcf400640 1 --2- 172.21.15.111:0/804486404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55665d216040 0x55665d221d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.441+0000 7fedce000640 1 --2- 172.21.15.111:0/804486404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55665d210fb0 0x55665d21acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.441+0000 7fedcea00640 1 -- 172.21.15.111:0/804486404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55665d216040 msgr2=0x55665d221d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.441+0000 7fedcea00640 1 --2- 172.21.15.111:0/804486404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55665d216040 0x55665d221d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.441+0000 7fedcea00640 1 -- 172.21.15.111:0/804486404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55665d210fb0 msgr2=0x55665d21acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.441+0000 7fedcea00640 1 --2- 172.21.15.111:0/804486404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55665d210fb0 0x55665d21acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.442+0000 7fedcea00640 1 -- 172.21.15.111:0/804486404 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55665cf50620 con 0x55665d210670 2024-09-15T22:47:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.442+0000 7fedcea00640 1 --2- 172.21.15.111:0/804486404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665d210670 0x55665d21a780 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fedc001c650 tx=0x7fedc0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=789824b7b6c68922 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.442+0000 7fedb8c00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:47:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.442+0000 7fedb8c00640 1 -- 172.21.15.111:0/804486404 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc0028680 con 0x55665d210670 2024-09-15T22:47:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.442+0000 7fedd8be9740 1 -- 172.21.15.111:0/804486404 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedc8012ad0 con 0x55665d210670 2024-09-15T22:47:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.442+0000 7fedb8c00640 1 -- 172.21.15.111:0/804486404 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fedc0028820 con 0x55665d210670 2024-09-15T22:47:47.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.442+0000 7fedb8c00640 1 -- 172.21.15.111:0/804486404 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fedc0028a20 con 0x55665d210670 2024-09-15T22:47:47.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.442+0000 7fedd8be9740 10 client.25165 fetch_fsmap learned FSMap version 28 2024-09-15T22:47:47.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.442+0000 7fedd8be9740 1 -- 172.21.15.111:0/804486404 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55665d224c10 con 0x55665d210670 2024-09-15T22:47:47.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.443+0000 7fedb8c00640 1 -- 172.21.15.111:0/804486404 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc8012ad0 con 0x55665d210670 2024-09-15T22:47:47.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.443+0000 7fedb8c00640 1 -- 172.21.15.111:0/804486404 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fedc001dbf0 con 0x55665d210670 2024-09-15T22:47:47.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.443+0000 7fedd8be9740 10 client.25165 fetch_fsmap finished waiting for FSMap version 28 2024-09-15T22:47:47.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.443+0000 7fedd8be9740 10 client.25165 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:47.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.443+0000 7fedd8be9740 20 client.25165 populate_metadata read hostname 'smithi111' 2024-09-15T22:47:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.443+0000 7fedd8be9740 1 --2- 172.21.15.111:0/804486404 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55665d226300 0x55665d228780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.443+0000 7fedd8be9740 4 client.25165 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:47:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.443+0000 7fedd8be9740 1 -- 172.21.15.111:0/804486404 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55665d17e830 con 0x55665d226300 2024-09-15T22:47:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.443+0000 7fedce000640 1 --2- 172.21.15.111:0/804486404 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55665d226300 0x55665d228780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.444+0000 7fedce000640 1 --2- 172.21.15.111:0/804486404 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55665d226300 0x55665d228780 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36f990571900f9d6 server_cookie=f261723c6da3c5bf in_seq=0 out_seq=0 2024-09-15T22:47:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.444+0000 7fedb8c00640 10 client.25165 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:47:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.444+0000 7fedb8c00640 1 -- 172.21.15.111:0/804486404 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55665d17e830 con 0x55665d226300 2024-09-15T22:47:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.444+0000 7fedb8c00640 10 client.25165 handle_command_reply: tid=0 2024-09-15T22:47:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.444+0000 7fedd8be9740 1 client.25165 shutdown 2024-09-15T22:47:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.444+0000 7fedd8be9740 1 -- 172.21.15.111:0/804486404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665d210670 msgr2=0x55665d21a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.444+0000 7fedd8be9740 1 --2- 172.21.15.111:0/804486404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665d210670 0x55665d21a780 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fedc001c650 tx=0x7fedc0004a40 comp rx=0 tx=0).stop 2024-09-15T22:47:47.450 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.445+0000 7fedd8be9740 1 -- 172.21.15.111:0/804486404 shutdown_connections 2024-09-15T22:47:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.445+0000 7fedd8be9740 1 --2- 172.21.15.111:0/804486404 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55665d226300 0x55665d228780 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.445+0000 7fedd8be9740 1 --2- 172.21.15.111:0/804486404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55665d216040 0x55665d221d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.445+0000 7fedd8be9740 1 --2- 172.21.15.111:0/804486404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55665d210fb0 0x55665d21acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.445+0000 7fedd8be9740 1 --2- 172.21.15.111:0/804486404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665d210670 0x55665d21a780 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.445+0000 7fedd8be9740 1 -- 172.21.15.111:0/804486404 >> 172.21.15.111:0/804486404 conn(0x55665d20bce0 msgr2=0x55665d20d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.445+0000 7fedb8c00640 0 client.25165 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:47:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.445+0000 7fedd8be9740 1 -- 172.21.15.111:0/804486404 shutdown_connections 2024-09-15T22:47:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.445+0000 7fedd8be9740 1 -- 172.21.15.111:0/804486404 wait complete. 2024-09-15T22:47:47.451 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.445+0000 7fedd8be9740 20 client.25165 trim_cache size 0 max 16384 2024-09-15T22:47:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.461+0000 7fedd8be9740 1 Processor -- start 2024-09-15T22:47:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.461+0000 7fedd8be9740 1 -- start start 2024-09-15T22:47:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.461+0000 7fedd8be9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665d20bce0 0x55665d21aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.461+0000 7fedd8be9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55665d210670 0x55665d221cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.461+0000 7fedd8be9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55665d210fb0 0x55665d20d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.461+0000 7fedd8be9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fedd01a9320 con 0x55665d210670 2024-09-15T22:47:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.461+0000 7fedd8be9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fedd01a6bf0 con 0x55665d210fb0 2024-09-15T22:47:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.462+0000 7fedd8be9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fedd01a4a40 con 0x55665d20bce0 2024-09-15T22:47:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.462+0000 7fedcea00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665d20bce0 0x55665d21aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.462+0000 7fedcea00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665d20bce0 0x55665d21aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:51128/0 (socket says 172.21.15.111:51128) 2024-09-15T22:47:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.462+0000 7fedcea00640 1 -- 172.21.15.111:0/2066548716 learned_addr learned my addr 172.21.15.111:0/2066548716 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:47:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.462+0000 7fedcf400640 1 --2- 172.21.15.111:0/2066548716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55665d210fb0 0x55665d20d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.462+0000 7fedce000640 1 --2- 172.21.15.111:0/2066548716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55665d210670 0x55665d221cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.462+0000 7fedcea00640 1 -- 172.21.15.111:0/2066548716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55665d210fb0 msgr2=0x55665d20d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.462+0000 7fedcea00640 1 --2- 172.21.15.111:0/2066548716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55665d210fb0 0x55665d20d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.462+0000 7fedcea00640 1 -- 172.21.15.111:0/2066548716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55665d210670 msgr2=0x55665d221cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.462+0000 7fedcea00640 1 --2- 172.21.15.111:0/2066548716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55665d210670 0x55665d221cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.462+0000 7fedcea00640 1 -- 172.21.15.111:0/2066548716 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55665d20bb00 con 0x55665d20bce0 2024-09-15T22:47:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.462+0000 7fedce000640 1 --2- 172.21.15.111:0/2066548716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55665d210670 0x55665d221cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.462+0000 7fedcea00640 1 --2- 172.21.15.111:0/2066548716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665d20bce0 0x55665d21aba0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fedc0010010 tx=0x7fedc0005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62ea3ffaf6759d43 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.463+0000 7fedbb400640 1 -- 172.21.15.111:0/2066548716 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc0041070 con 0x55665d20bce0 2024-09-15T22:47:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.463+0000 7fedbb400640 1 -- 172.21.15.111:0/2066548716 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc0037070 con 0x55665d20bce0 2024-09-15T22:47:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.463+0000 7fedbb400640 1 -- 172.21.15.111:0/2066548716 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc0033920 con 0x55665d20bce0 2024-09-15T22:47:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.463+0000 7fedd8be9740 1 -- 172.21.15.111:0/2066548716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665d20bce0 msgr2=0x55665d21aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.463+0000 7fedd8be9740 1 --2- 172.21.15.111:0/2066548716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665d20bce0 0x55665d21aba0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fedc0010010 tx=0x7fedc0005fd0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.463+0000 7fedd8be9740 1 -- 172.21.15.111:0/2066548716 shutdown_connections 2024-09-15T22:47:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.463+0000 7fedd8be9740 1 --2- 172.21.15.111:0/2066548716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55665d210fb0 0x55665d20d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.463+0000 7fedd8be9740 1 --2- 172.21.15.111:0/2066548716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55665d210670 0x55665d221cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.463+0000 7fedd8be9740 1 --2- 172.21.15.111:0/2066548716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665d20bce0 0x55665d21aba0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.464+0000 7fedd8be9740 1 -- 172.21.15.111:0/2066548716 >> 172.21.15.111:0/2066548716 conn(0x55665d215b90 msgr2=0x55665d206a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.464+0000 7fedd8be9740 1 -- 172.21.15.111:0/2066548716 shutdown_connections 2024-09-15T22:47:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.464+0000 7fedd8be9740 1 -- 172.21.15.111:0/2066548716 wait complete. 2024-09-15T22:47:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.464+0000 7fedd8be9740 1 -- start start 2024-09-15T22:47:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.464+0000 7fedd8be9740 1 Processor -- start 2024-09-15T22:47:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.464+0000 7fedd8be9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665d20bce0 0x55665d21f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.464+0000 7fedd8be9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55665d210670 0x55665d21fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.464+0000 7fedd8be9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55665d210fb0 0x55665d220a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.464+0000 7fedd8be9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fedd01a50b0 con 0x55665d210670 2024-09-15T22:47:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.464+0000 7fedd8be9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fedd01a6520 con 0x55665d210fb0 2024-09-15T22:47:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.464+0000 7fedd8be9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fedd01a5780 con 0x55665d20bce0 2024-09-15T22:47:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.465+0000 7fedd8be9740 10 client.0 _ll_register_callbacks cb 0x7fedc0007340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:47:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.465+0000 7fedcf400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55665d210fb0 0x55665d220a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.465+0000 7fedcf400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55665d210fb0 0x55665d220a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:55716/0 (socket says 172.21.15.111:55716) 2024-09-15T22:47:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.465+0000 7fedcf400640 1 -- 172.21.15.111:0/253609780 learned_addr learned my addr 172.21.15.111:0/253609780 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:47:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.465+0000 7fedce000640 1 --2- 172.21.15.111:0/253609780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55665d210670 0x55665d21fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.465+0000 7fedcea00640 1 --2- 172.21.15.111:0/253609780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665d20bce0 0x55665d21f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.465+0000 7fedcf400640 1 -- 172.21.15.111:0/253609780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665d20bce0 msgr2=0x55665d21f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.465+0000 7fedcf400640 1 --2- 172.21.15.111:0/253609780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665d20bce0 0x55665d21f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.465+0000 7fedcf400640 1 -- 172.21.15.111:0/253609780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55665d210670 msgr2=0x55665d21fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:47:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.465+0000 7fedcf400640 1 --2- 172.21.15.111:0/253609780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55665d210670 0x55665d21fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.465+0000 7fedcf400640 1 -- 172.21.15.111:0/253609780 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55665d221020 con 0x55665d210fb0 2024-09-15T22:47:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.465+0000 7fedce000640 1 --2- 172.21.15.111:0/253609780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55665d210670 0x55665d21fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:47:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.465+0000 7fedcf400640 1 --2- 172.21.15.111:0/253609780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55665d210fb0 0x55665d220a10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009cc0 tx=0x7fedc8005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b434d3751f81629 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:47:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.465+0000 7fedb8c00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:47:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.466+0000 7fedb8c00640 1 -- 172.21.15.111:0/253609780 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8004660 con 0x55665d210fb0 2024-09-15T22:47:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.466+0000 7fedd8be9740 1 -- 172.21.15.111:0/253609780 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedc0037070 con 0x55665d210fb0 2024-09-15T22:47:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.466+0000 7fedb8c00640 1 -- 172.21.15.111:0/253609780 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fedc8004800 con 0x55665d210fb0 2024-09-15T22:47:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.466+0000 7fedb8c00640 1 -- 172.21.15.111:0/253609780 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fedc8004a00 con 0x55665d210fb0 2024-09-15T22:47:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.466+0000 7fedd8be9740 10 client.25172 fetch_fsmap learned FSMap version 28 2024-09-15T22:47:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.466+0000 7fedd8be9740 1 -- 172.21.15.111:0/253609780 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55665d2911f0 con 0x55665d210fb0 2024-09-15T22:47:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.466+0000 7fedb8c00640 1 -- 172.21.15.111:0/253609780 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc0037070 con 0x55665d210fb0 2024-09-15T22:47:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.466+0000 7fedb8c00640 1 -- 172.21.15.111:0/253609780 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fedc802fc20 con 0x55665d210fb0 2024-09-15T22:47:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.467+0000 7fedd8be9740 10 client.25172 fetch_fsmap finished waiting for FSMap version 28 2024-09-15T22:47:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.467+0000 7fedd8be9740 10 client.25172 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:47:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.467+0000 7fedd8be9740 20 client.25172 populate_metadata read hostname 'smithi111' 2024-09-15T22:47:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.467+0000 7fedd8be9740 1 --2- 172.21.15.111:0/253609780 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55665d216040 0x55665d293960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:47:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.467+0000 7fedd8be9740 4 client.25172 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:47:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.467+0000 7fedd8be9740 1 -- 172.21.15.111:0/253609780 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55665d178b80 con 0x55665d216040 2024-09-15T22:47:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.467+0000 7fedcea00640 1 --2- 172.21.15.111:0/253609780 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55665d216040 0x55665d293960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:47:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.467+0000 7fedb8c00640 10 client.25172 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:47:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.467+0000 7fedcea00640 1 --2- 172.21.15.111:0/253609780 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55665d216040 0x55665d293960 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a9548ce736b10e3 server_cookie=11b116433b954dd0 in_seq=0 out_seq=0 2024-09-15T22:47:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.486+0000 7fedb8c00640 1 -- 172.21.15.111:0/253609780 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55665d178b80 con 0x55665d216040 2024-09-15T22:47:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.487+0000 7fedb8c00640 10 client.25172 handle_command_reply: tid=0 2024-09-15T22:47:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.487+0000 7fedd8be9740 1 client.25172 shutdown 2024-09-15T22:47:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.487+0000 7fedd8be9740 1 -- 172.21.15.111:0/253609780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55665d210fb0 msgr2=0x55665d220a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.487+0000 7fedd8be9740 1 --2- 172.21.15.111:0/253609780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55665d210fb0 0x55665d220a10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009cc0 tx=0x7fedc8005800 comp rx=0 tx=0).stop 2024-09-15T22:47:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.487+0000 7fedd8be9740 1 -- 172.21.15.111:0/253609780 shutdown_connections 2024-09-15T22:47:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.487+0000 7fedd8be9740 1 --2- 172.21.15.111:0/253609780 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55665d216040 0x55665d293960 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.487+0000 7fedd8be9740 1 --2- 172.21.15.111:0/253609780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55665d210fb0 0x55665d220a10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.487+0000 7fedd8be9740 1 --2- 172.21.15.111:0/253609780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55665d210670 0x55665d21fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.487+0000 7fedb8c00640 0 client.25172 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:47:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.487+0000 7fedd8be9740 1 --2- 172.21.15.111:0/253609780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55665d20bce0 0x55665d21f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.487+0000 7fedd8be9740 1 -- 172.21.15.111:0/253609780 >> 172.21.15.111:0/253609780 conn(0x55665d215b90 msgr2=0x55665d206a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.487+0000 7fedd8be9740 1 -- 172.21.15.111:0/253609780 shutdown_connections 2024-09-15T22:47:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.487+0000 7fedd8be9740 1 -- 172.21.15.111:0/253609780 wait complete. 2024-09-15T22:47:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.487+0000 7fedd8be9740 20 client.25172 trim_cache size 0 max 16384 2024-09-15T22:47:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:47:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:47:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:47:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:47:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.488+0000 7fedcfe00640 1 -- 172.21.15.111:0/835483045 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fedb40a32d0 msgr2=0x7fedb40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.488+0000 7fedcfe00640 1 --2- 172.21.15.111:0/835483045 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fedb40a32d0 0x7fedb40a5790 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55665d2119e0 tx=0x7fedc0007600 comp rx=0 tx=0).stop 2024-09-15T22:47:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.488+0000 7fedcfe00640 1 -- 172.21.15.111:0/835483045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedd0101f40 msgr2=0x7fedd0198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:47:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.488+0000 7fedcfe00640 1 --2- 172.21.15.111:0/835483045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedd0101f40 0x7fedd0198280 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fedbc031440 tx=0x7fedbc032ce0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.488+0000 7fedcfe00640 1 -- 172.21.15.111:0/835483045 shutdown_connections 2024-09-15T22:47:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.488+0000 7fedcfe00640 1 --2- 172.21.15.111:0/835483045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fedd0103050 0x7fedd019f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.488+0000 7fedcfe00640 1 --2- 172.21.15.111:0/835483045 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fedb40a32d0 0x7fedb40a5790 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.488+0000 7fedcfe00640 1 --2- 172.21.15.111:0/835483045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fedd0101f40 0x7fedd0198280 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.488+0000 7fedcfe00640 1 --2- 172.21.15.111:0/835483045 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fedd0101580 0x7fedd0197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:47:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.488+0000 7fedcfe00640 1 -- 172.21.15.111:0/835483045 >> 172.21.15.111:0/835483045 conn(0x7fedd00fd0d0 msgr2=0x7fedd0105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:47:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.488+0000 7fedcfe00640 1 -- 172.21.15.111:0/835483045 shutdown_connections 2024-09-15T22:47:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:47:47.488+0000 7fedcfe00640 1 -- 172.21.15.111:0/835483045 wait complete. 2024-09-15T22:47:48.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:47 smithi192 ceph-mon[35427]: pgmap v406: 97 pgs: 97 active+clean; 11 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 1.1 MiB/s wr, 196 op/s 2024-09-15T22:47:48.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:47 smithi121 ceph-mon[34815]: pgmap v406: 97 pgs: 97 active+clean; 11 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 1.1 MiB/s wr, 196 op/s 2024-09-15T22:47:48.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:47 smithi111 ceph-mon[30629]: pgmap v406: 97 pgs: 97 active+clean; 11 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 1.1 MiB/s wr, 196 op/s 2024-09-15T22:47:48.228 INFO:tasks.workunit.client.0.smithi111.stdout: 1 37221 13.59 MB/sec warmup 16 sec latency 15.752 ms 2024-09-15T22:47:49.228 INFO:tasks.workunit.client.0.smithi111.stdout: 1 40181 13.73 MB/sec warmup 17 sec latency 10.982 ms 2024-09-15T22:47:50.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:49 smithi192 ceph-mon[35427]: pgmap v407: 97 pgs: 97 active+clean; 15 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-09-15T22:47:50.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:49 smithi121 ceph-mon[34815]: pgmap v407: 97 pgs: 97 active+clean; 15 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-09-15T22:47:50.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:49 smithi111 ceph-mon[30629]: pgmap v407: 97 pgs: 97 active+clean; 15 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-09-15T22:47:50.228 INFO:tasks.workunit.client.0.smithi111.stdout: 1 43330 13.93 MB/sec warmup 18 sec latency 17.842 ms 2024-09-15T22:47:51.228 INFO:tasks.workunit.client.0.smithi111.stdout: 1 46678 14.19 MB/sec warmup 19 sec latency 7.911 ms 2024-09-15T22:47:52.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:51 smithi192 ceph-mon[35427]: pgmap v408: 97 pgs: 97 active+clean; 17 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.2 MiB/s wr, 172 op/s 2024-09-15T22:47:52.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:51 smithi192 ceph-mon[35427]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-09-15T22:47:52.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:51 smithi121 ceph-mon[34815]: pgmap v408: 97 pgs: 97 active+clean; 17 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.2 MiB/s wr, 172 op/s 2024-09-15T22:47:52.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:51 smithi121 ceph-mon[34815]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-09-15T22:47:52.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:51 smithi111 ceph-mon[30629]: pgmap v408: 97 pgs: 97 active+clean; 17 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.2 MiB/s wr, 172 op/s 2024-09-15T22:47:52.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:51 smithi111 ceph-mon[30629]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-09-15T22:47:52.228 INFO:tasks.workunit.client.0.smithi111.stdout: 1 50028 14.43 MB/sec warmup 20 sec latency 13.587 ms 2024-09-15T22:47:53.228 INFO:tasks.workunit.client.0.smithi111.stdout: 1 53313 14.59 MB/sec warmup 21 sec latency 10.743 ms 2024-09-15T22:47:54.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:53 smithi192 ceph-mon[35427]: pgmap v409: 97 pgs: 97 active+clean; 18 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-09-15T22:47:54.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:53 smithi121 ceph-mon[34815]: pgmap v409: 97 pgs: 97 active+clean; 18 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-09-15T22:47:54.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:53 smithi111 ceph-mon[30629]: pgmap v409: 97 pgs: 97 active+clean; 18 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-09-15T22:47:54.228 INFO:tasks.workunit.client.0.smithi111.stdout: 1 56235 14.56 MB/sec warmup 22 sec latency 15.190 ms 2024-09-15T22:47:55.228 INFO:tasks.workunit.client.0.smithi111.stdout: 1 59197 14.52 MB/sec warmup 23 sec latency 18.365 ms 2024-09-15T22:47:55.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:55 smithi121 ceph-mon[34815]: pgmap v410: 97 pgs: 97 active+clean; 21 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 1.5 MiB/s wr, 197 op/s 2024-09-15T22:47:55.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:55 smithi111 ceph-mon[30629]: pgmap v410: 97 pgs: 97 active+clean; 21 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 1.5 MiB/s wr, 197 op/s 2024-09-15T22:47:55.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:55 smithi192 ceph-mon[35427]: pgmap v410: 97 pgs: 97 active+clean; 21 MiB data, 477 MiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 1.5 MiB/s wr, 197 op/s 2024-09-15T22:47:56.228 INFO:tasks.workunit.client.0.smithi111.stdout: 1 62024 14.64 MB/sec warmup 24 sec latency 10.421 ms 2024-09-15T22:47:57.228 INFO:tasks.workunit.client.0.smithi111.stdout: 1 64670 14.63 MB/sec warmup 25 sec latency 18.794 ms 2024-09-15T22:47:57.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:57 smithi192 ceph-mon[35427]: pgmap v411: 97 pgs: 97 active+clean; 25 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 1.7 MiB/s wr, 215 op/s 2024-09-15T22:47:57.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:57 smithi121 ceph-mon[34815]: pgmap v411: 97 pgs: 97 active+clean; 25 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 1.7 MiB/s wr, 215 op/s 2024-09-15T22:47:57.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:57 smithi111 ceph-mon[30629]: pgmap v411: 97 pgs: 97 active+clean; 25 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 1.7 MiB/s wr, 215 op/s 2024-09-15T22:47:58.228 INFO:tasks.workunit.client.0.smithi111.stdout: 1 67827 14.78 MB/sec warmup 26 sec latency 9.316 ms 2024-09-15T22:47:59.228 INFO:tasks.workunit.client.0.smithi111.stdout: 1 71206 14.93 MB/sec warmup 27 sec latency 12.007 ms 2024-09-15T22:47:59.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:47:59 smithi121 ceph-mon[34815]: pgmap v412: 97 pgs: 97 active+clean; 26 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.6 MiB/s wr, 211 op/s 2024-09-15T22:47:59.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:47:59 smithi111 ceph-mon[30629]: pgmap v412: 97 pgs: 97 active+clean; 26 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.6 MiB/s wr, 211 op/s 2024-09-15T22:48:00.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:47:59 smithi192 ceph-mon[35427]: pgmap v412: 97 pgs: 97 active+clean; 26 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.6 MiB/s wr, 211 op/s 2024-09-15T22:48:00.228 INFO:tasks.workunit.client.0.smithi111.stdout: 1 74290 14.92 MB/sec warmup 28 sec latency 15.042 ms 2024-09-15T22:48:00.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.612+0000 7f071e200640 1 -- 172.21.15.111:0/2473989594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0718104f50 msgr2=0x7f07181053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:00.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.612+0000 7f071e200640 1 --2- 172.21.15.111:0/2473989594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0718104f50 0x7f07181053d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0704009900 tx=0x7f070402a410 comp rx=0 tx=0).stop 2024-09-15T22:48:00.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.612+0000 7f071e200640 1 -- 172.21.15.111:0/2473989594 shutdown_connections 2024-09-15T22:48:00.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.612+0000 7f071e200640 1 --2- 172.21.15.111:0/2473989594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0718105910 0x7f071810c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.612+0000 7f071e200640 1 --2- 172.21.15.111:0/2473989594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0718104f50 0x7f07181053d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.612+0000 7f071e200640 1 --2- 172.21.15.111:0/2473989594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0718103d60 0x7f0718104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.612+0000 7f071e200640 1 -- 172.21.15.111:0/2473989594 >> 172.21.15.111:0/2473989594 conn(0x7f07180ff4e0 msgr2=0x7f0718101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:00.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.612+0000 7f071e200640 1 -- 172.21.15.111:0/2473989594 shutdown_connections 2024-09-15T22:48:00.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.612+0000 7f071e200640 1 -- 172.21.15.111:0/2473989594 wait complete. 2024-09-15T22:48:00.617 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.613+0000 7f071e200640 1 Processor -- start 2024-09-15T22:48:00.617 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.613+0000 7f071e200640 1 -- start start 2024-09-15T22:48:00.617 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.614+0000 7f071e200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0718103d60 0x7f0718199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:00.617 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.614+0000 7f071e200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0718104f50 0x7f071819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:00.617 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.614+0000 7f071e200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0718105910 0x7f07181a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:00.617 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.614+0000 7f071e200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f071810f9c0 con 0x7f0718103d60 2024-09-15T22:48:00.617 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.614+0000 7f071e200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f071810f840 con 0x7f0718105910 2024-09-15T22:48:00.617 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.614+0000 7f071e200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f071810fb40 con 0x7f0718104f50 2024-09-15T22:48:00.617 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.614+0000 7f0717400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0718103d60 0x7f0718199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.614+0000 7f0717e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0718105910 0x7f07181a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.614+0000 7f0717400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0718103d60 0x7f0718199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39498/0 (socket says 172.21.15.111:39498) 2024-09-15T22:48:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.614+0000 7f0717400640 1 -- 172.21.15.111:0/2082781909 learned_addr learned my addr 172.21.15.111:0/2082781909 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:48:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.614+0000 7f0716a00640 1 --2- 172.21.15.111:0/2082781909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0718104f50 0x7f071819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.614+0000 7f0717400640 1 -- 172.21.15.111:0/2082781909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0718104f50 msgr2=0x7f071819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.614+0000 7f0717400640 1 --2- 172.21.15.111:0/2082781909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0718104f50 0x7f071819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.614+0000 7f0717400640 1 -- 172.21.15.111:0/2082781909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0718105910 msgr2=0x7f07181a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.614+0000 7f0717400640 1 --2- 172.21.15.111:0/2082781909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0718105910 0x7f07181a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.614+0000 7f0717400640 1 -- 172.21.15.111:0/2082781909 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07181a1ba0 con 0x7f0718103d60 2024-09-15T22:48:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.615+0000 7f0717400640 1 --2- 172.21.15.111:0/2082781909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0718103d60 0x7f0718199f80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f070800e9c0 tx=0x7f070800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.615+0000 7f070fe00640 1 -- 172.21.15.111:0/2082781909 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070800cde0 con 0x7f0718103d60 2024-09-15T22:48:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.615+0000 7f070fe00640 1 -- 172.21.15.111:0/2082781909 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0708010910 con 0x7f0718103d60 2024-09-15T22:48:00.618 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.615+0000 7f070fe00640 1 -- 172.21.15.111:0/2082781909 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0708018a00 con 0x7f0718103d60 2024-09-15T22:48:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.615+0000 7f071e200640 1 -- 172.21.15.111:0/2082781909 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07181a1e90 con 0x7f0718103d60 2024-09-15T22:48:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.615+0000 7f071e200640 1 -- 172.21.15.111:0/2082781909 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07181a22a0 con 0x7f0718103d60 2024-09-15T22:48:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.618+0000 7f070fe00640 1 -- 172.21.15.111:0/2082781909 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f070801f020 con 0x7f0718103d60 2024-09-15T22:48:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.618+0000 7f072102e740 1 Processor -- start 2024-09-15T22:48:00.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.618+0000 7f072102e740 1 -- start start 2024-09-15T22:48:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.619+0000 7f072102e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571739db090 0x5571739db490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:00.621 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.619+0000 7f072102e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571739db9d0 0x5571739e0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.619+0000 7f072102e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571739e0a60 0x5571739e2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.619+0000 7f072102e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557173940100 con 0x5571739e0a60 2024-09-15T22:48:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.619+0000 7f072102e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5571738e81b0 con 0x5571739db090 2024-09-15T22:48:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.619+0000 7f072102e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55717393c0c0 con 0x5571739db9d0 2024-09-15T22:48:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.619+0000 7f0717e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571739db090 0x5571739db490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.619+0000 7f0717e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571739db090 0x5571739db490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:35596/0 (socket says 172.21.15.111:35596) 2024-09-15T22:48:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.619+0000 7f0717e00640 1 -- 172.21.15.111:0/748760289 learned_addr learned my addr 172.21.15.111:0/748760289 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:48:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.619+0000 7f0716a00640 1 --2- 172.21.15.111:0/748760289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571739e0a60 0x5571739e2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.619+0000 7f0717400640 1 --2- 172.21.15.111:0/748760289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571739db9d0 0x5571739e0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.619+0000 7f0717e00640 1 -- 172.21.15.111:0/748760289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571739db9d0 msgr2=0x5571739e0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.619+0000 7f0717e00640 1 --2- 172.21.15.111:0/748760289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571739db9d0 0x5571739e0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.619+0000 7f0717e00640 1 -- 172.21.15.111:0/748760289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571739e0a60 msgr2=0x5571739e2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:00.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.619+0000 7f0717e00640 1 --2- 172.21.15.111:0/748760289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571739e0a60 0x5571739e2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.619+0000 7f0717e00640 1 -- 172.21.15.111:0/748760289 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557173653210 con 0x5571739db090 2024-09-15T22:48:00.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.619+0000 7f0717400640 1 --2- 172.21.15.111:0/748760289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571739db9d0 0x5571739e0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:48:00.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.619+0000 7f0716a00640 1 --2- 172.21.15.111:0/748760289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571739e0a60 0x5571739e2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:48:00.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.620+0000 7f0717e00640 1 --2- 172.21.15.111:0/748760289 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571739db090 0x5571739db490 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0710009fd0 tx=0x7f07100118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=536b7e0d9e94b491 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:00.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.620+0000 7f070fe00640 1 --2- 172.21.15.111:0/2082781909 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f06fc0a3370 0x7f06fc0a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:00.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.620+0000 7f0703e00640 1 -- 172.21.15.111:0/748760289 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0710004d20 con 0x5571739db090 2024-09-15T22:48:00.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.620+0000 7f0703e00640 1 -- 172.21.15.111:0/748760289 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0710004050 con 0x5571739db090 2024-09-15T22:48:00.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.620+0000 7f0703e00640 1 -- 172.21.15.111:0/748760289 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0710005260 con 0x5571739db090 2024-09-15T22:48:00.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.620+0000 7f070fe00640 1 -- 172.21.15.111:0/2082781909 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f07080cfa50 con 0x7f0718103d60 2024-09-15T22:48:00.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.620+0000 7f0717400640 1 --2- 172.21.15.111:0/2082781909 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f06fc0a3370 0x7f06fc0a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.621+0000 7f072102e740 1 -- 172.21.15.111:0/748760289 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571739db090 msgr2=0x5571739db490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.621+0000 7f072102e740 1 --2- 172.21.15.111:0/748760289 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571739db090 0x5571739db490 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0710009fd0 tx=0x7f07100118d0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.621+0000 7f0717400640 1 --2- 172.21.15.111:0/2082781909 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f06fc0a3370 0x7f06fc0a5830 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f07080a47b0 tx=0x7f070800efb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.621+0000 7f072102e740 1 -- 172.21.15.111:0/748760289 shutdown_connections 2024-09-15T22:48:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.621+0000 7f072102e740 1 --2- 172.21.15.111:0/748760289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571739e0a60 0x5571739e2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.621+0000 7f072102e740 1 --2- 172.21.15.111:0/748760289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571739db9d0 0x5571739e0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.621+0000 7f072102e740 1 --2- 172.21.15.111:0/748760289 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571739db090 0x5571739db490 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.621+0000 7f072102e740 1 -- 172.21.15.111:0/748760289 >> 172.21.15.111:0/748760289 conn(0x5571739d6700 msgr2=0x5571739d6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.621+0000 7f072102e740 1 -- 172.21.15.111:0/748760289 shutdown_connections 2024-09-15T22:48:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.621+0000 7f072102e740 1 -- 172.21.15.111:0/748760289 wait complete. 2024-09-15T22:48:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.621+0000 7f072102e740 1 -- start start 2024-09-15T22:48:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.621+0000 7f072102e740 1 Processor -- start 2024-09-15T22:48:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.622+0000 7f072102e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571739db090 0x5571739e51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.622+0000 7f072102e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571739db9d0 0x5571739e5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.622+0000 7f072102e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571739e0a60 0x5571739ec760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.622+0000 7f072102e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5571739e3a30 con 0x5571739e0a60 2024-09-15T22:48:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.622+0000 7f072102e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5571738c9090 con 0x5571739db090 2024-09-15T22:48:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.622+0000 7f072102e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5571739e3bb0 con 0x5571739db9d0 2024-09-15T22:48:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.622+0000 7f072102e740 10 client.0 _ll_register_callbacks cb 0x55717391df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:48:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.622+0000 7f0717e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571739db090 0x5571739e51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.622+0000 7f0717e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571739e0a60 0x5571739ec760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.622+0000 7f0716a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571739db9d0 0x5571739e5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.622+0000 7f0716a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571739db9d0 0x5571739e5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:59610/0 (socket says 172.21.15.111:59610) 2024-09-15T22:48:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.622+0000 7f0716a00640 1 -- 172.21.15.111:0/828087102 learned_addr learned my addr 172.21.15.111:0/828087102 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:48:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.622+0000 7f0717e00640 1 --2- 172.21.15.111:0/828087102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571739e0a60 0x5571739ec760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39506/0 (socket says 172.21.15.111:39506) 2024-09-15T22:48:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.623+0000 7f0716a00640 1 -- 172.21.15.111:0/828087102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571739db090 msgr2=0x5571739e51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.623+0000 7f0716a00640 1 --2- 172.21.15.111:0/828087102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571739db090 0x5571739e51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.623+0000 7f0716a00640 1 -- 172.21.15.111:0/828087102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571739e0a60 msgr2=0x5571739ec760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.623+0000 7f0716a00640 1 --2- 172.21.15.111:0/828087102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571739e0a60 0x5571739ec760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.623+0000 7f0716a00640 1 -- 172.21.15.111:0/828087102 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55717371f620 con 0x5571739db9d0 2024-09-15T22:48:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.623+0000 7f0716a00640 1 --2- 172.21.15.111:0/828087102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571739db9d0 0x5571739e5720 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0704031610 tx=0x7f0704032e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e4ece791672498d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.623+0000 7f0700c00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:48:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.623+0000 7f0700c00640 1 -- 172.21.15.111:0/828087102 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0704035040 con 0x5571739db9d0 2024-09-15T22:48:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.623+0000 7f0700c00640 1 -- 172.21.15.111:0/828087102 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f0704031970 con 0x5571739db9d0 2024-09-15T22:48:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.623+0000 7f072102e740 1 -- 172.21.15.111:0/828087102 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0710004050 con 0x5571739db9d0 2024-09-15T22:48:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.623+0000 7f0700c00640 1 -- 172.21.15.111:0/828087102 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0704004040 con 0x5571739db9d0 2024-09-15T22:48:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.624+0000 7f072102e740 10 client.25180 fetch_fsmap learned FSMap version 28 2024-09-15T22:48:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.624+0000 7f072102e740 1 -- 172.21.15.111:0/828087102 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5571739ef530 con 0x5571739db9d0 2024-09-15T22:48:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.624+0000 7f0700c00640 1 -- 172.21.15.111:0/828087102 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0710004050 con 0x5571739db9d0 2024-09-15T22:48:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.624+0000 7f0700c00640 1 -- 172.21.15.111:0/828087102 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0704004810 con 0x5571739db9d0 2024-09-15T22:48:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.624+0000 7f072102e740 10 client.25180 fetch_fsmap finished waiting for FSMap version 28 2024-09-15T22:48:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.624+0000 7f072102e740 10 client.25180 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:48:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.624+0000 7f072102e740 20 client.25180 populate_metadata read hostname 'smithi111' 2024-09-15T22:48:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.624+0000 7f072102e740 1 --2- 172.21.15.111:0/828087102 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5571739f0c80 0x5571739f30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.624+0000 7f072102e740 4 client.25180 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:48:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.624+0000 7f072102e740 1 -- 172.21.15.111:0/828087102 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55717394d830 con 0x5571739f0c80 2024-09-15T22:48:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.624+0000 7f0717e00640 1 --2- 172.21.15.111:0/828087102 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5571739f0c80 0x5571739f30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.625+0000 7f0717e00640 1 --2- 172.21.15.111:0/828087102 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5571739f0c80 0x5571739f30a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77eed621e2a0dab server_cookie=a140af32b681e2 in_seq=0 out_seq=0 2024-09-15T22:48:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.625+0000 7f0700c00640 10 client.25180 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:48:00.631 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.625+0000 7f0700c00640 1 -- 172.21.15.111:0/828087102 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55717394d830 con 0x5571739f0c80 2024-09-15T22:48:00.631 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.626+0000 7f0700c00640 10 client.25180 handle_command_reply: tid=0 2024-09-15T22:48:00.631 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.626+0000 7f072102e740 1 client.25180 shutdown 2024-09-15T22:48:00.631 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.626+0000 7f072102e740 1 -- 172.21.15.111:0/828087102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571739db9d0 msgr2=0x5571739e5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:00.632 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.626+0000 7f072102e740 1 --2- 172.21.15.111:0/828087102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571739db9d0 0x5571739e5720 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0704031610 tx=0x7f0704032e60 comp rx=0 tx=0).stop 2024-09-15T22:48:00.632 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.626+0000 7f072102e740 1 -- 172.21.15.111:0/828087102 shutdown_connections 2024-09-15T22:48:00.632 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.626+0000 7f072102e740 1 --2- 172.21.15.111:0/828087102 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5571739f0c80 0x5571739f30a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.632 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.626+0000 7f072102e740 1 --2- 172.21.15.111:0/828087102 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571739e0a60 0x5571739ec760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.632 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.626+0000 7f072102e740 1 --2- 172.21.15.111:0/828087102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571739db9d0 0x5571739e5720 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.632 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.626+0000 7f072102e740 1 --2- 172.21.15.111:0/828087102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571739db090 0x5571739e51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.632 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.626+0000 7f072102e740 1 -- 172.21.15.111:0/828087102 >> 172.21.15.111:0/828087102 conn(0x5571739d6700 msgr2=0x5571739d7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:00.632 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.626+0000 7f0700c00640 0 client.25180 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:48:00.632 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.626+0000 7f072102e740 1 -- 172.21.15.111:0/828087102 shutdown_connections 2024-09-15T22:48:00.632 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.626+0000 7f072102e740 1 -- 172.21.15.111:0/828087102 wait complete. 2024-09-15T22:48:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.626+0000 7f072102e740 20 client.25180 trim_cache size 0 max 16384 2024-09-15T22:48:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.646+0000 7f072102e740 1 Processor -- start 2024-09-15T22:48:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.646+0000 7f072102e740 1 -- start start 2024-09-15T22:48:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.646+0000 7f072102e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571739db090 0x5571739e51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.646+0000 7f072102e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571739e0a60 0x5571739ec720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.646+0000 7f072102e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571739dd280 0x5571739e5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.646+0000 7f072102e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f07181ad9b0 con 0x5571739dd280 2024-09-15T22:48:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.646+0000 7f072102e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f07181ab280 con 0x5571739e0a60 2024-09-15T22:48:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.646+0000 7f072102e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f07181a90d0 con 0x5571739db090 2024-09-15T22:48:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.646+0000 7f0716a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571739e0a60 0x5571739ec720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.646+0000 7f0717e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571739dd280 0x5571739e5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.646+0000 7f0716a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571739e0a60 0x5571739ec720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:35608/0 (socket says 172.21.15.111:35608) 2024-09-15T22:48:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.646+0000 7f0716a00640 1 -- 172.21.15.111:0/2053441320 learned_addr learned my addr 172.21.15.111:0/2053441320 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:48:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.646+0000 7f0717e00640 1 --2- 172.21.15.111:0/2053441320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571739db090 0x5571739e51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.647+0000 7f0716a00640 1 -- 172.21.15.111:0/2053441320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571739db090 msgr2=0x5571739e51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.647+0000 7f0716a00640 1 --2- 172.21.15.111:0/2053441320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571739db090 0x5571739e51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.647+0000 7f0716a00640 1 -- 172.21.15.111:0/2053441320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571739dd280 msgr2=0x5571739e5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.647+0000 7f0716a00640 1 --2- 172.21.15.111:0/2053441320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571739dd280 0x5571739e5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.647+0000 7f0716a00640 1 -- 172.21.15.111:0/2053441320 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571739f3720 con 0x5571739e0a60 2024-09-15T22:48:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.647+0000 7f0716a00640 1 --2- 172.21.15.111:0/2053441320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571739e0a60 0x5571739ec720 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f070402a930 tx=0x7f070402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2eff7b1951d85561 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.647+0000 7f0703400640 1 -- 172.21.15.111:0/2053441320 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0704035030 con 0x5571739e0a60 2024-09-15T22:48:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.647+0000 7f0703400640 1 -- 172.21.15.111:0/2053441320 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0704004980 con 0x5571739e0a60 2024-09-15T22:48:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.647+0000 7f0703400640 1 -- 172.21.15.111:0/2053441320 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07040044d0 con 0x5571739e0a60 2024-09-15T22:48:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.648+0000 7f072102e740 1 -- 172.21.15.111:0/2053441320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571739e0a60 msgr2=0x5571739ec720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.648+0000 7f072102e740 1 --2- 172.21.15.111:0/2053441320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571739e0a60 0x5571739ec720 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f070402a930 tx=0x7f070402a410 comp rx=0 tx=0).stop 2024-09-15T22:48:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.648+0000 7f072102e740 1 -- 172.21.15.111:0/2053441320 shutdown_connections 2024-09-15T22:48:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.648+0000 7f072102e740 1 --2- 172.21.15.111:0/2053441320 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571739dd280 0x5571739e5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.648+0000 7f072102e740 1 --2- 172.21.15.111:0/2053441320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571739e0a60 0x5571739ec720 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.648+0000 7f072102e740 1 --2- 172.21.15.111:0/2053441320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571739db090 0x5571739e51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.648+0000 7f072102e740 1 -- 172.21.15.111:0/2053441320 >> 172.21.15.111:0/2053441320 conn(0x5571739d6700 msgr2=0x5571739dcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.648+0000 7f072102e740 1 -- 172.21.15.111:0/2053441320 shutdown_connections 2024-09-15T22:48:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.648+0000 7f072102e740 1 -- 172.21.15.111:0/2053441320 wait complete. 2024-09-15T22:48:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.649+0000 7f072102e740 1 -- start start 2024-09-15T22:48:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.649+0000 7f072102e740 1 Processor -- start 2024-09-15T22:48:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.649+0000 7f072102e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571739db090 0x5571739e9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.649+0000 7f072102e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571739e0a60 0x5571739ea150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.649+0000 7f072102e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571739eb160 0x5571739eb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:00.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.649+0000 7f072102e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f07181a9740 con 0x5571739eb160 2024-09-15T22:48:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.649+0000 7f072102e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f07181aabb0 con 0x5571739db090 2024-09-15T22:48:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.649+0000 7f072102e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f07181a9e10 con 0x5571739e0a60 2024-09-15T22:48:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.649+0000 7f072102e740 10 client.0 _ll_register_callbacks cb 0x7f0708005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:48:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.649+0000 7f0717e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571739db090 0x5571739e9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.649+0000 7f0716a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571739e0a60 0x5571739ea150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.649+0000 7f0717e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571739db090 0x5571739e9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:35618/0 (socket says 172.21.15.111:35618) 2024-09-15T22:48:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.650+0000 7f0717e00640 1 -- 172.21.15.111:0/543076263 learned_addr learned my addr 172.21.15.111:0/543076263 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:48:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.650+0000 7f0716a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571739e0a60 0x5571739ea150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:59626/0 (socket says 172.21.15.111:59626) 2024-09-15T22:48:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.650+0000 7f0717e00640 1 --2- 172.21.15.111:0/543076263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571739eb160 0x5571739eb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.650+0000 7f0717e00640 1 -- 172.21.15.111:0/543076263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571739e0a60 msgr2=0x5571739ea150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.650+0000 7f0717e00640 1 --2- 172.21.15.111:0/543076263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571739e0a60 0x5571739ea150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.650+0000 7f0717e00640 1 -- 172.21.15.111:0/543076263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571739eb160 msgr2=0x5571739eb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.650+0000 7f0717e00640 1 --2- 172.21.15.111:0/543076263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571739eb160 0x5571739eb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.650+0000 7f0717e00640 1 -- 172.21.15.111:0/543076263 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571739ebb50 con 0x5571739db090 2024-09-15T22:48:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.650+0000 7f0716a00640 1 --2- 172.21.15.111:0/543076263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571739e0a60 0x5571739ea150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:48:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.650+0000 7f0717e00640 1 --2- 172.21.15.111:0/543076263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571739db090 0x5571739e9c10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0710009f70 tx=0x7f07100118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6162a0969eb11b24 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.650+0000 7f0700c00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:48:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.650+0000 7f0700c00640 1 -- 172.21.15.111:0/543076263 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0710004440 con 0x5571739db090 2024-09-15T22:48:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.650+0000 7f072102e740 1 -- 172.21.15.111:0/543076263 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0704004980 con 0x5571739db090 2024-09-15T22:48:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.650+0000 7f0700c00640 1 -- 172.21.15.111:0/543076263 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f071001f040 con 0x5571739db090 2024-09-15T22:48:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.651+0000 7f0700c00640 1 -- 172.21.15.111:0/543076263 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0710043050 con 0x5571739db090 2024-09-15T22:48:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.651+0000 7f072102e740 10 client.25190 fetch_fsmap learned FSMap version 28 2024-09-15T22:48:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.651+0000 7f072102e740 1 -- 172.21.15.111:0/543076263 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557173a5bb50 con 0x5571739db090 2024-09-15T22:48:00.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.651+0000 7f0700c00640 1 -- 172.21.15.111:0/543076263 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0704004980 con 0x5571739db090 2024-09-15T22:48:00.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.651+0000 7f0700c00640 1 -- 172.21.15.111:0/543076263 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f071003a070 con 0x5571739db090 2024-09-15T22:48:00.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.651+0000 7f072102e740 10 client.25190 fetch_fsmap finished waiting for FSMap version 28 2024-09-15T22:48:00.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.651+0000 7f072102e740 10 client.25190 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:48:00.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.651+0000 7f072102e740 20 client.25190 populate_metadata read hostname 'smithi111' 2024-09-15T22:48:00.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.651+0000 7f072102e740 1 --2- 172.21.15.111:0/543076263 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557173a5c3f0 0x557173a5e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:00.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.651+0000 7f072102e740 4 client.25190 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:48:00.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.651+0000 7f072102e740 1 -- 172.21.15.111:0/543076263 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557173947b80 con 0x557173a5c3f0 2024-09-15T22:48:00.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.652+0000 7f0716a00640 1 --2- 172.21.15.111:0/543076263 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557173a5c3f0 0x557173a5e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:00.658 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.652+0000 7f0716a00640 1 --2- 172.21.15.111:0/543076263 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557173a5c3f0 0x557173a5e810 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c26c52382165c41b server_cookie=ee1416a419683dc0 in_seq=0 out_seq=0 2024-09-15T22:48:00.658 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.652+0000 7f0700c00640 10 client.25190 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:48:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.662+0000 7f0700c00640 1 -- 172.21.15.111:0/543076263 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557173947b80 con 0x557173a5c3f0 2024-09-15T22:48:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.662+0000 7f0700c00640 10 client.25190 handle_command_reply: tid=0 2024-09-15T22:48:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.662+0000 7f072102e740 1 client.25190 shutdown 2024-09-15T22:48:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.662+0000 7f072102e740 1 -- 172.21.15.111:0/543076263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571739db090 msgr2=0x5571739e9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.662+0000 7f072102e740 1 --2- 172.21.15.111:0/543076263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571739db090 0x5571739e9c10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0710009f70 tx=0x7f07100118d0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.662+0000 7f072102e740 1 -- 172.21.15.111:0/543076263 shutdown_connections 2024-09-15T22:48:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.662+0000 7f072102e740 1 --2- 172.21.15.111:0/543076263 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557173a5c3f0 0x557173a5e810 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.662+0000 7f072102e740 1 --2- 172.21.15.111:0/543076263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571739eb160 0x5571739eb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.662+0000 7f072102e740 1 --2- 172.21.15.111:0/543076263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5571739e0a60 0x5571739ea150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.662+0000 7f0700c00640 0 client.25190 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:48:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.662+0000 7f072102e740 1 --2- 172.21.15.111:0/543076263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571739db090 0x5571739e9c10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.664 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.662+0000 7f072102e740 1 -- 172.21.15.111:0/543076263 >> 172.21.15.111:0/543076263 conn(0x5571739d6700 msgr2=0x5571739e5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.662+0000 7f072102e740 1 -- 172.21.15.111:0/543076263 shutdown_connections 2024-09-15T22:48:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.662+0000 7f072102e740 1 -- 172.21.15.111:0/543076263 wait complete. 2024-09-15T22:48:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.662+0000 7f072102e740 20 client.25190 trim_cache size 0 max 16384 2024-09-15T22:48:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:48:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:48:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:48:00.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:48:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.663+0000 7f071e200640 1 -- 172.21.15.111:0/2082781909 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f06fc0a3370 msgr2=0x7f06fc0a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.663+0000 7f071e200640 1 --2- 172.21.15.111:0/2082781909 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f06fc0a3370 0x7f06fc0a5830 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f07080a47b0 tx=0x7f070800efb0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.663+0000 7f071e200640 1 -- 172.21.15.111:0/2082781909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0718103d60 msgr2=0x7f0718199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.663+0000 7f071e200640 1 --2- 172.21.15.111:0/2082781909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0718103d60 0x7f0718199f80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f070800e9c0 tx=0x7f070800ee80 comp rx=0 tx=0).stop 2024-09-15T22:48:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.663+0000 7f071e200640 1 -- 172.21.15.111:0/2082781909 shutdown_connections 2024-09-15T22:48:00.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.663+0000 7f071e200640 1 --2- 172.21.15.111:0/2082781909 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f06fc0a3370 0x7f06fc0a5830 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.663+0000 7f071e200640 1 --2- 172.21.15.111:0/2082781909 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0718105910 0x7f07181a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.663+0000 7f071e200640 1 --2- 172.21.15.111:0/2082781909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0718104f50 0x7f071819a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.663+0000 7f071e200640 1 --2- 172.21.15.111:0/2082781909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0718103d60 0x7f0718199f80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.663+0000 7f071e200640 1 -- 172.21.15.111:0/2082781909 >> 172.21.15.111:0/2082781909 conn(0x7f07180ff4e0 msgr2=0x7f07181082d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.663+0000 7f071e200640 1 -- 172.21.15.111:0/2082781909 shutdown_connections 2024-09-15T22:48:00.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:00.663+0000 7f071e200640 1 -- 172.21.15.111:0/2082781909 wait complete. 2024-09-15T22:48:01.228 INFO:tasks.workunit.client.0.smithi111.stdout: 1 77322 14.95 MB/sec warmup 29 sec latency 14.640 ms 2024-09-15T22:48:01.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:01 smithi121 ceph-mon[34815]: pgmap v413: 97 pgs: 97 active+clean; 31 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 1.7 MiB/s wr, 229 op/s 2024-09-15T22:48:01.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:01 smithi111 ceph-mon[30629]: pgmap v413: 97 pgs: 97 active+clean; 31 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 1.7 MiB/s wr, 229 op/s 2024-09-15T22:48:02.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:01 smithi192 ceph-mon[35427]: pgmap v413: 97 pgs: 97 active+clean; 31 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 1.7 MiB/s wr, 229 op/s 2024-09-15T22:48:02.229 INFO:tasks.workunit.client.0.smithi111.stdout: 1 79913 14.90 MB/sec warmup 30 sec latency 10.923 ms 2024-09-15T22:48:03.229 INFO:tasks.workunit.client.0.smithi111.stdout: 1 82370 14.88 MB/sec warmup 31 sec latency 44.398 ms 2024-09-15T22:48:03.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:03 smithi111 ceph-mon[30629]: pgmap v414: 97 pgs: 97 active+clean; 34 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-09-15T22:48:03.866 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:03 smithi192 ceph-mon[35427]: pgmap v414: 97 pgs: 97 active+clean; 34 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-09-15T22:48:03.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:03 smithi121 ceph-mon[34815]: pgmap v414: 97 pgs: 97 active+clean; 34 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-09-15T22:48:04.229 INFO:tasks.workunit.client.0.smithi111.stdout: 1 84845 14.82 MB/sec warmup 32 sec latency 261.369 ms 2024-09-15T22:48:05.229 INFO:tasks.workunit.client.0.smithi111.stdout: 1 87578 14.76 MB/sec warmup 33 sec latency 18.708 ms 2024-09-15T22:48:05.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:05 smithi121 ceph-mon[34815]: pgmap v415: 97 pgs: 97 active+clean; 38 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 2.0 MiB/s wr, 232 op/s 2024-09-15T22:48:05.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:05 smithi111 ceph-mon[30629]: pgmap v415: 97 pgs: 97 active+clean; 38 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 2.0 MiB/s wr, 232 op/s 2024-09-15T22:48:06.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:05 smithi192 ceph-mon[35427]: pgmap v415: 97 pgs: 97 active+clean; 38 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 2.0 MiB/s wr, 232 op/s 2024-09-15T22:48:06.229 INFO:tasks.workunit.client.0.smithi111.stdout: 1 89905 14.74 MB/sec warmup 34 sec latency 10.810 ms 2024-09-15T22:48:07.229 INFO:tasks.workunit.client.0.smithi111.stdout: 1 92870 14.81 MB/sec warmup 35 sec latency 8.654 ms 2024-09-15T22:48:07.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:07 smithi121 ceph-mon[34815]: pgmap v416: 97 pgs: 97 active+clean; 51 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 261 op/s 2024-09-15T22:48:07.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:07 smithi111 ceph-mon[30629]: pgmap v416: 97 pgs: 97 active+clean; 51 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 261 op/s 2024-09-15T22:48:08.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:07 smithi192 ceph-mon[35427]: pgmap v416: 97 pgs: 97 active+clean; 51 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 261 op/s 2024-09-15T22:48:08.238 INFO:tasks.workunit.client.0.smithi111.stdout: 1 96031 14.91 MB/sec warmup 36 sec latency 78.965 ms 2024-09-15T22:48:09.238 INFO:tasks.workunit.client.0.smithi111.stdout: 1 96043 14.51 MB/sec warmup 37 sec latency 611.859 ms 2024-09-15T22:48:09.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:09 smithi121 ceph-mon[34815]: pgmap v417: 97 pgs: 97 active+clean; 60 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 3.4 MiB/s wr, 198 op/s 2024-09-15T22:48:09.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:09 smithi111 ceph-mon[30629]: pgmap v417: 97 pgs: 97 active+clean; 60 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 3.4 MiB/s wr, 198 op/s 2024-09-15T22:48:10.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:09 smithi192 ceph-mon[35427]: pgmap v417: 97 pgs: 97 active+clean; 60 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 3.4 MiB/s wr, 198 op/s 2024-09-15T22:48:10.238 INFO:tasks.workunit.client.0.smithi111.stdout: 1 98267 14.32 MB/sec warmup 38 sec latency 628.735 ms 2024-09-15T22:48:11.238 INFO:tasks.workunit.client.0.smithi111.stdout: 1 100786 14.39 MB/sec warmup 39 sec latency 11.146 ms 2024-09-15T22:48:11.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:11 smithi121 ceph-mon[34815]: pgmap v418: 97 pgs: 97 active+clean; 64 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 253 op/s 2024-09-15T22:48:11.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:11 smithi111 ceph-mon[30629]: pgmap v418: 97 pgs: 97 active+clean; 64 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 253 op/s 2024-09-15T22:48:12.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:11 smithi192 ceph-mon[35427]: pgmap v418: 97 pgs: 97 active+clean; 64 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 253 op/s 2024-09-15T22:48:12.238 INFO:tasks.workunit.client.0.smithi111.stdout: 1 103593 14.39 MB/sec warmup 40 sec latency 13.121 ms 2024-09-15T22:48:13.238 INFO:tasks.workunit.client.0.smithi111.stdout: 1 106722 14.49 MB/sec warmup 41 sec latency 14.984 ms 2024-09-15T22:48:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.785+0000 7feae0a00640 1 -- 172.21.15.111:0/33219062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feadc102b00 msgr2=0x7feadc104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.785+0000 7feae0a00640 1 --2- 172.21.15.111:0/33219062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feadc102b00 0x7feadc104f70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feac8009900 tx=0x7feac802a410 comp rx=0 tx=0).stop 2024-09-15T22:48:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.785+0000 7feae0a00640 1 -- 172.21.15.111:0/33219062 shutdown_connections 2024-09-15T22:48:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.785+0000 7feae0a00640 1 --2- 172.21.15.111:0/33219062 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feadc1054b0 0x7feadc105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.785+0000 7feae0a00640 1 --2- 172.21.15.111:0/33219062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feadc102b00 0x7feadc104f70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.785+0000 7feae0a00640 1 --2- 172.21.15.111:0/33219062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feadc1001b0 0x7feadc1025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.785+0000 7feae0a00640 1 -- 172.21.15.111:0/33219062 >> 172.21.15.111:0/33219062 conn(0x7feadc0fbfa0 msgr2=0x7feadc0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.785+0000 7feae0a00640 1 -- 172.21.15.111:0/33219062 shutdown_connections 2024-09-15T22:48:13.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.785+0000 7feae0a00640 1 -- 172.21.15.111:0/33219062 wait complete. 2024-09-15T22:48:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.786+0000 7feae0a00640 1 Processor -- start 2024-09-15T22:48:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.786+0000 7feae0a00640 1 -- start start 2024-09-15T22:48:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.786+0000 7feae0a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feadc1001b0 0x7feadc19e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.786+0000 7feae0a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feadc102b00 0x7feadc19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.786+0000 7feae0a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feadc1054b0 0x7feadc1a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.786+0000 7feae0a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7feadc113c80 con 0x7feadc1001b0 2024-09-15T22:48:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.786+0000 7feae0a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7feadc113b00 con 0x7feadc102b00 2024-09-15T22:48:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.786+0000 7feae0a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7feadc113e00 con 0x7feadc1054b0 2024-09-15T22:48:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.787+0000 7feadb200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feadc1054b0 0x7feadc1a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.787+0000 7feadb200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feadc1054b0 0x7feadc1a58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:43124/0 (socket says 172.21.15.111:43124) 2024-09-15T22:48:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.787+0000 7feadb200640 1 -- 172.21.15.111:0/574452255 learned_addr learned my addr 172.21.15.111:0/574452255 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:48:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.787+0000 7feada800640 1 --2- 172.21.15.111:0/574452255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feadc1001b0 0x7feadc19e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.787+0000 7fead9e00640 1 --2- 172.21.15.111:0/574452255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feadc102b00 0x7feadc19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.787+0000 7feadb200640 1 -- 172.21.15.111:0/574452255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feadc102b00 msgr2=0x7feadc19e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.787+0000 7feadb200640 1 --2- 172.21.15.111:0/574452255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feadc102b00 0x7feadc19e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.787+0000 7feadb200640 1 -- 172.21.15.111:0/574452255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feadc1001b0 msgr2=0x7feadc19e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.787+0000 7feadb200640 1 --2- 172.21.15.111:0/574452255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feadc1001b0 0x7feadc19e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.787+0000 7feadb200640 1 -- 172.21.15.111:0/574452255 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feadc1a5f20 con 0x7feadc1054b0 2024-09-15T22:48:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.787+0000 7feada800640 1 --2- 172.21.15.111:0/574452255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feadc1001b0 0x7feadc19e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:48:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.787+0000 7feadb200640 1 --2- 172.21.15.111:0/574452255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feadc1054b0 0x7feadc1a58b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fead400eb10 tx=0x7fead400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.788+0000 7fead3400640 1 -- 172.21.15.111:0/574452255 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead4004b20 con 0x7feadc1054b0 2024-09-15T22:48:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.788+0000 7feae0a00640 1 -- 172.21.15.111:0/574452255 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feadc1a6210 con 0x7feadc1054b0 2024-09-15T22:48:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.788+0000 7fead3400640 1 -- 172.21.15.111:0/574452255 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fead4004cc0 con 0x7feadc1054b0 2024-09-15T22:48:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.788+0000 7fead3400640 1 -- 172.21.15.111:0/574452255 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead4010760 con 0x7feadc1054b0 2024-09-15T22:48:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.788+0000 7feae0a00640 1 -- 172.21.15.111:0/574452255 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feadc1a67b0 con 0x7feadc1054b0 2024-09-15T22:48:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.791+0000 7feae43e8740 1 Processor -- start 2024-09-15T22:48:13.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.791+0000 7fead3400640 1 -- 172.21.15.111:0/574452255 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fead4005020 con 0x7feadc1054b0 2024-09-15T22:48:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.791+0000 7feae43e8740 1 -- start start 2024-09-15T22:48:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.791+0000 7feae43e8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646cbf59260 0x5646cbf59660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.792+0000 7feae43e8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646cbf59ba0 0x5646cbf5e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.792+0000 7feae43e8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646cbf5ec30 0x5646cbf610f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.792+0000 7feae43e8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5646cbebe100 con 0x5646cbf5ec30 2024-09-15T22:48:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.792+0000 7feae43e8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5646cbe661b0 con 0x5646cbf59ba0 2024-09-15T22:48:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.792+0000 7feae43e8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5646cbeba0c0 con 0x5646cbf59260 2024-09-15T22:48:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.792+0000 7fead9e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646cbf59260 0x5646cbf59660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.792+0000 7fead9e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646cbf59260 0x5646cbf59660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:43134/0 (socket says 172.21.15.111:43134) 2024-09-15T22:48:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.792+0000 7fead9e00640 1 -- 172.21.15.111:0/2398134302 learned_addr learned my addr 172.21.15.111:0/2398134302 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:48:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.792+0000 7fead9e00640 1 --2- 172.21.15.111:0/2398134302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646cbf5ec30 0x5646cbf610f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.792+0000 7fead9e00640 1 -- 172.21.15.111:0/2398134302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646cbf59ba0 msgr2=0x5646cbf5e660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:48:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.792+0000 7fead9e00640 1 --2- 172.21.15.111:0/2398134302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646cbf59ba0 0x5646cbf5e660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.792+0000 7fead9e00640 1 -- 172.21.15.111:0/2398134302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646cbf5ec30 msgr2=0x5646cbf610f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.792+0000 7fead9e00640 1 --2- 172.21.15.111:0/2398134302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646cbf5ec30 0x5646cbf610f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.792+0000 7fead9e00640 1 -- 172.21.15.111:0/2398134302 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646cbbd1210 con 0x5646cbf59260 2024-09-15T22:48:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.792+0000 7fead3400640 1 --2- 172.21.15.111:0/574452255 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7feac00a3240 0x7feac00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.793+0000 7fead9e00640 1 --2- 172.21.15.111:0/2398134302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646cbf59260 0x5646cbf59660 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feac80314e0 tx=0x7feac8005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7becef8a1112e4e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.793+0000 7fead3400640 1 -- 172.21.15.111:0/574452255 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fead40cff70 con 0x7feadc1054b0 2024-09-15T22:48:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.793+0000 7feada800640 1 --2- 172.21.15.111:0/574452255 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7feac00a3240 0x7feac00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.793+0000 7feac7400640 1 -- 172.21.15.111:0/2398134302 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac8035070 con 0x5646cbf59260 2024-09-15T22:48:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.793+0000 7feac7400640 1 -- 172.21.15.111:0/2398134302 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feac8032b90 con 0x5646cbf59260 2024-09-15T22:48:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.793+0000 7feac7400640 1 -- 172.21.15.111:0/2398134302 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac803d450 con 0x5646cbf59260 2024-09-15T22:48:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.794+0000 7feae43e8740 1 -- 172.21.15.111:0/2398134302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646cbf59260 msgr2=0x5646cbf59660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.794+0000 7feada800640 1 --2- 172.21.15.111:0/574452255 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7feac00a3240 0x7feac00a5700 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7feacc006fd0 tx=0x7feacc008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:13.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.794+0000 7feae43e8740 1 --2- 172.21.15.111:0/2398134302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646cbf59260 0x5646cbf59660 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feac80314e0 tx=0x7feac8005fd0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.794+0000 7feae43e8740 1 -- 172.21.15.111:0/2398134302 shutdown_connections 2024-09-15T22:48:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.794+0000 7feae43e8740 1 --2- 172.21.15.111:0/2398134302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646cbf5ec30 0x5646cbf610f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.794+0000 7feae43e8740 1 --2- 172.21.15.111:0/2398134302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646cbf59ba0 0x5646cbf5e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.794+0000 7feae43e8740 1 --2- 172.21.15.111:0/2398134302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646cbf59260 0x5646cbf59660 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.794+0000 7feae43e8740 1 -- 172.21.15.111:0/2398134302 >> 172.21.15.111:0/2398134302 conn(0x5646cbf548d0 msgr2=0x5646cbf54cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:13.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.794+0000 7feae43e8740 1 -- 172.21.15.111:0/2398134302 shutdown_connections 2024-09-15T22:48:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.794+0000 7feae43e8740 1 -- 172.21.15.111:0/2398134302 wait complete. 2024-09-15T22:48:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.794+0000 7feae43e8740 1 -- start start 2024-09-15T22:48:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.794+0000 7feae43e8740 1 Processor -- start 2024-09-15T22:48:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.795+0000 7feae43e8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646cbf59260 0x5646cbf63440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.795+0000 7feae43e8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646cbf59ba0 0x5646cbf63980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.795+0000 7feae43e8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646cbf5ec30 0x5646cbf6a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.795+0000 7feae43e8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5646cbf61ba0 con 0x5646cbf59ba0 2024-09-15T22:48:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.795+0000 7feae43e8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5646cbe47090 con 0x5646cbf59260 2024-09-15T22:48:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.795+0000 7feae43e8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5646cbf61d20 con 0x5646cbf5ec30 2024-09-15T22:48:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.795+0000 7feae43e8740 10 client.0 _ll_register_callbacks cb 0x5646cbe9bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:48:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.795+0000 7fead9e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646cbf59ba0 0x5646cbf63980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.795+0000 7fead9e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646cbf59ba0 0x5646cbf63980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47114/0 (socket says 172.21.15.111:47114) 2024-09-15T22:48:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.795+0000 7fead9e00640 1 -- 172.21.15.111:0/2997971323 learned_addr learned my addr 172.21.15.111:0/2997971323 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:48:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.795+0000 7feada800640 1 --2- 172.21.15.111:0/2997971323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646cbf59260 0x5646cbf63440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.795+0000 7feadb200640 1 --2- 172.21.15.111:0/2997971323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646cbf5ec30 0x5646cbf6a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.795+0000 7fead9e00640 1 -- 172.21.15.111:0/2997971323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646cbf5ec30 msgr2=0x5646cbf6a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.795+0000 7fead9e00640 1 --2- 172.21.15.111:0/2997971323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646cbf5ec30 0x5646cbf6a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.796+0000 7fead9e00640 1 -- 172.21.15.111:0/2997971323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646cbf59260 msgr2=0x5646cbf63440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.796+0000 7fead9e00640 1 --2- 172.21.15.111:0/2997971323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646cbf59260 0x5646cbf63440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.796+0000 7fead9e00640 1 -- 172.21.15.111:0/2997971323 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646cbc9d620 con 0x5646cbf59ba0 2024-09-15T22:48:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.796+0000 7feadb200640 1 --2- 172.21.15.111:0/2997971323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646cbf5ec30 0x5646cbf6a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:48:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.796+0000 7feada800640 1 --2- 172.21.15.111:0/2997971323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646cbf59260 0x5646cbf63440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:48:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.796+0000 7fead9e00640 1 --2- 172.21.15.111:0/2997971323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646cbf59ba0 0x5646cbf63980 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7feac80096c0 tx=0x7feac80315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107b094739505614 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.796+0000 7feabbe00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:48:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.796+0000 7feabbe00640 1 -- 172.21.15.111:0/2997971323 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac804f070 con 0x5646cbf59ba0 2024-09-15T22:48:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.796+0000 7feabbe00640 1 -- 172.21.15.111:0/2997971323 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7feac80035f0 con 0x5646cbf59ba0 2024-09-15T22:48:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.796+0000 7feae43e8740 1 -- 172.21.15.111:0/2997971323 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feac8032b90 con 0x5646cbf59ba0 2024-09-15T22:48:13.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.796+0000 7feabbe00640 1 -- 172.21.15.111:0/2997971323 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feac805b050 con 0x5646cbf59ba0 2024-09-15T22:48:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.796+0000 7feae43e8740 10 client.15174 fetch_fsmap learned FSMap version 28 2024-09-15T22:48:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.796+0000 7feae43e8740 1 -- 172.21.15.111:0/2997971323 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5646cbf6d7b0 con 0x5646cbf59ba0 2024-09-15T22:48:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.797+0000 7feabbe00640 1 -- 172.21.15.111:0/2997971323 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feac8032b90 con 0x5646cbf59ba0 2024-09-15T22:48:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.797+0000 7feabbe00640 1 -- 172.21.15.111:0/2997971323 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7feac8049070 con 0x5646cbf59ba0 2024-09-15T22:48:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.797+0000 7feae43e8740 10 client.15174 fetch_fsmap finished waiting for FSMap version 28 2024-09-15T22:48:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.797+0000 7feae43e8740 10 client.15174 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:48:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.797+0000 7feae43e8740 20 client.15174 populate_metadata read hostname 'smithi111' 2024-09-15T22:48:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.797+0000 7feae43e8740 1 --2- 172.21.15.111:0/2997971323 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5646cbf6eb00 0x5646cbf70ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.797+0000 7feae43e8740 4 client.15174 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:48:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.797+0000 7feae43e8740 1 -- 172.21.15.111:0/2997971323 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646cbecb830 con 0x5646cbf6eb00 2024-09-15T22:48:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.798+0000 7feada800640 1 --2- 172.21.15.111:0/2997971323 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5646cbf6eb00 0x5646cbf70ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.798+0000 7feabbe00640 10 client.15174 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:48:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.798+0000 7feada800640 1 --2- 172.21.15.111:0/2997971323 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5646cbf6eb00 0x5646cbf70ef0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9e4decf3a2f104b server_cookie=45504ae29f88daab in_seq=0 out_seq=0 2024-09-15T22:48:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.799+0000 7feabbe00640 1 -- 172.21.15.111:0/2997971323 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5646cbecb830 con 0x5646cbf6eb00 2024-09-15T22:48:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.799+0000 7feabbe00640 10 client.15174 handle_command_reply: tid=0 2024-09-15T22:48:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.799+0000 7feae43e8740 1 client.15174 shutdown 2024-09-15T22:48:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.799+0000 7feae43e8740 1 -- 172.21.15.111:0/2997971323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646cbf59ba0 msgr2=0x5646cbf63980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.799+0000 7feae43e8740 1 --2- 172.21.15.111:0/2997971323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646cbf59ba0 0x5646cbf63980 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7feac80096c0 tx=0x7feac80315b0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.799+0000 7feae43e8740 1 -- 172.21.15.111:0/2997971323 shutdown_connections 2024-09-15T22:48:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.799+0000 7feae43e8740 1 --2- 172.21.15.111:0/2997971323 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5646cbf6eb00 0x5646cbf70ef0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.799+0000 7feae43e8740 1 --2- 172.21.15.111:0/2997971323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646cbf5ec30 0x5646cbf6a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.799+0000 7feae43e8740 1 --2- 172.21.15.111:0/2997971323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646cbf59ba0 0x5646cbf63980 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.799+0000 7feae43e8740 1 --2- 172.21.15.111:0/2997971323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646cbf59260 0x5646cbf63440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.799+0000 7feabbe00640 0 client.15174 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:48:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.799+0000 7feae43e8740 1 -- 172.21.15.111:0/2997971323 >> 172.21.15.111:0/2997971323 conn(0x5646cbf548d0 msgr2=0x5646cbf560e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.799+0000 7feae43e8740 1 -- 172.21.15.111:0/2997971323 shutdown_connections 2024-09-15T22:48:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.799+0000 7feae43e8740 1 -- 172.21.15.111:0/2997971323 wait complete. 2024-09-15T22:48:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.799+0000 7feae43e8740 20 client.15174 trim_cache size 0 max 16384 2024-09-15T22:48:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.814+0000 7feae43e8740 1 Processor -- start 2024-09-15T22:48:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.814+0000 7feae43e8740 1 -- start start 2024-09-15T22:48:13.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.814+0000 7feae43e8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646cbf59260 0x5646cbf627d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.814+0000 7feae43e8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646cbf59ba0 0x5646cbf6a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.814+0000 7feae43e8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646cbf5ec30 0x5646cbf55780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.814+0000 7feae43e8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1b1bf0 con 0x5646cbf59ba0 2024-09-15T22:48:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.814+0000 7feae43e8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1af4c0 con 0x5646cbf59260 2024-09-15T22:48:13.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.814+0000 7feae43e8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1ad330 con 0x5646cbf5ec30 2024-09-15T22:48:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.815+0000 7fead9e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646cbf59ba0 0x5646cbf6a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.815+0000 7feada800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646cbf59260 0x5646cbf627d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.815+0000 7feada800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646cbf59260 0x5646cbf627d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:55188/0 (socket says 172.21.15.111:55188) 2024-09-15T22:48:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.815+0000 7feada800640 1 -- 172.21.15.111:0/4007731477 learned_addr learned my addr 172.21.15.111:0/4007731477 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:48:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.815+0000 7feadb200640 1 --2- 172.21.15.111:0/4007731477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646cbf5ec30 0x5646cbf55780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:13.825 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.815+0000 7feada800640 1 -- 172.21.15.111:0/4007731477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646cbf5ec30 msgr2=0x5646cbf55780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.815+0000 7feada800640 1 --2- 172.21.15.111:0/4007731477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646cbf5ec30 0x5646cbf55780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.815+0000 7feada800640 1 -- 172.21.15.111:0/4007731477 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646cbf59ba0 msgr2=0x5646cbf6a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.815+0000 7feada800640 1 --2- 172.21.15.111:0/4007731477 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646cbf59ba0 0x5646cbf6a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.815+0000 7feada800640 1 -- 172.21.15.111:0/4007731477 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646cbf61e60 con 0x5646cbf59260 2024-09-15T22:48:13.826 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.815+0000 7feada800640 1 --2- 172.21.15.111:0/4007731477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646cbf59260 0x5646cbf627d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x5646cbf55750 tx=0x7feacc010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2b2ecad76d89b44 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.816+0000 7feac6a00640 1 -- 172.21.15.111:0/4007731477 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc030070 con 0x5646cbf59260 2024-09-15T22:48:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.816+0000 7feac6a00640 1 -- 172.21.15.111:0/4007731477 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feacc02b450 con 0x5646cbf59260 2024-09-15T22:48:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.816+0000 7feac6a00640 1 -- 172.21.15.111:0/4007731477 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc014430 con 0x5646cbf59260 2024-09-15T22:48:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.816+0000 7feae43e8740 1 -- 172.21.15.111:0/4007731477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646cbf59260 msgr2=0x5646cbf627d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.816+0000 7feae43e8740 1 --2- 172.21.15.111:0/4007731477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646cbf59260 0x5646cbf627d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x5646cbf55750 tx=0x7feacc010000 comp rx=0 tx=0).stop 2024-09-15T22:48:13.827 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.816+0000 7feae43e8740 1 -- 172.21.15.111:0/4007731477 shutdown_connections 2024-09-15T22:48:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.816+0000 7feae43e8740 1 --2- 172.21.15.111:0/4007731477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646cbf5ec30 0x5646cbf55780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.816+0000 7feae43e8740 1 --2- 172.21.15.111:0/4007731477 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646cbf59ba0 0x5646cbf6a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.816+0000 7feae43e8740 1 --2- 172.21.15.111:0/4007731477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646cbf59260 0x5646cbf627d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.816+0000 7feae43e8740 1 -- 172.21.15.111:0/4007731477 >> 172.21.15.111:0/4007731477 conn(0x5646cbf548d0 msgr2=0x5646cbf620c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:13.828 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.816+0000 7feae43e8740 1 -- 172.21.15.111:0/4007731477 shutdown_connections 2024-09-15T22:48:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.816+0000 7feae43e8740 1 -- 172.21.15.111:0/4007731477 wait complete. 2024-09-15T22:48:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.817+0000 7feae43e8740 1 -- start start 2024-09-15T22:48:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.817+0000 7feae43e8740 1 Processor -- start 2024-09-15T22:48:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.817+0000 7feae43e8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646cbf59260 0x5646cbf676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.817+0000 7feae43e8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646cbf59ba0 0x5646cbf67c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:13.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.817+0000 7feae43e8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646cbf5ec30 0x5646cbfd7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.817+0000 7feae43e8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1ad980 con 0x5646cbf5ec30 2024-09-15T22:48:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.817+0000 7feae43e8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1aedf0 con 0x5646cbf59260 2024-09-15T22:48:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.817+0000 7feae43e8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1ae050 con 0x5646cbf59ba0 2024-09-15T22:48:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.817+0000 7feae43e8740 10 client.0 _ll_register_callbacks cb 0x7feac80042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:48:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.818+0000 7feadb200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646cbf5ec30 0x5646cbfd7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:13.830 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.818+0000 7feada800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646cbf59260 0x5646cbf676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.818+0000 7feadb200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646cbf5ec30 0x5646cbfd7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47144/0 (socket says 172.21.15.111:47144) 2024-09-15T22:48:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.818+0000 7feadb200640 1 -- 172.21.15.111:0/3469649741 learned_addr learned my addr 172.21.15.111:0/3469649741 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:48:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.818+0000 7feadb200640 1 -- 172.21.15.111:0/3469649741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646cbf59ba0 msgr2=0x5646cbf67c10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:48:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.818+0000 7feadb200640 1 --2- 172.21.15.111:0/3469649741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646cbf59ba0 0x5646cbf67c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.831 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.818+0000 7feadb200640 1 -- 172.21.15.111:0/3469649741 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646cbf59260 msgr2=0x5646cbf676d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:13.832 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.818+0000 7feadb200640 1 --2- 172.21.15.111:0/3469649741 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646cbf59260 0x5646cbf676d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.832 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.818+0000 7feadb200640 1 -- 172.21.15.111:0/3469649741 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646cbfd7c80 con 0x5646cbf5ec30 2024-09-15T22:48:13.832 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.818+0000 7feadb200640 1 --2- 172.21.15.111:0/3469649741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646cbf5ec30 0x5646cbfd7740 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fead40049d0 tx=0x7fead409bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc8b8df66a48d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:13.832 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.818+0000 7feabbe00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:48:13.832 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.818+0000 7feabbe00640 1 -- 172.21.15.111:0/3469649741 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead40ba060 con 0x5646cbf5ec30 2024-09-15T22:48:13.832 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.818+0000 7feabbe00640 1 -- 172.21.15.111:0/3469649741 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fead40a4d70 con 0x5646cbf5ec30 2024-09-15T22:48:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.818+0000 7feae43e8740 1 -- 172.21.15.111:0/3469649741 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feacc02b450 con 0x5646cbf5ec30 2024-09-15T22:48:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.819+0000 7feabbe00640 1 -- 172.21.15.111:0/3469649741 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fead40a0940 con 0x5646cbf5ec30 2024-09-15T22:48:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.819+0000 7feae43e8740 10 client.15180 fetch_fsmap learned FSMap version 28 2024-09-15T22:48:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.819+0000 7feae43e8740 1 -- 172.21.15.111:0/3469649741 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5646cbfd8ea0 con 0x5646cbf5ec30 2024-09-15T22:48:13.833 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.819+0000 7feabbe00640 1 -- 172.21.15.111:0/3469649741 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feacc02b450 con 0x5646cbf5ec30 2024-09-15T22:48:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.819+0000 7feabbe00640 1 -- 172.21.15.111:0/3469649741 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fead4067170 con 0x5646cbf5ec30 2024-09-15T22:48:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.819+0000 7feae43e8740 10 client.15180 fetch_fsmap finished waiting for FSMap version 28 2024-09-15T22:48:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.819+0000 7feae43e8740 10 client.15180 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:48:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.819+0000 7feae43e8740 20 client.15180 populate_metadata read hostname 'smithi111' 2024-09-15T22:48:13.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.819+0000 7feae43e8740 1 --2- 172.21.15.111:0/3469649741 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5646cbfd9860 0x5646cbfdbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.819+0000 7feae43e8740 4 client.15180 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:48:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.819+0000 7feae43e8740 1 -- 172.21.15.111:0/3469649741 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646cbec5b80 con 0x5646cbfd9860 2024-09-15T22:48:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.819+0000 7feada800640 1 --2- 172.21.15.111:0/3469649741 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5646cbfd9860 0x5646cbfdbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.820+0000 7feada800640 1 --2- 172.21.15.111:0/3469649741 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5646cbfd9860 0x5646cbfdbc80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79aa58bcd6a822ad server_cookie=a118303bf1af1621 in_seq=0 out_seq=0 2024-09-15T22:48:13.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.820+0000 7feabbe00640 10 client.15180 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:48:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.832+0000 7feabbe00640 1 -- 172.21.15.111:0/3469649741 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5646cbec5b80 con 0x5646cbfd9860 2024-09-15T22:48:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.832+0000 7feabbe00640 10 client.15180 handle_command_reply: tid=0 2024-09-15T22:48:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.832+0000 7feae43e8740 1 client.15180 shutdown 2024-09-15T22:48:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.833+0000 7feae43e8740 1 -- 172.21.15.111:0/3469649741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646cbf5ec30 msgr2=0x5646cbfd7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.833+0000 7feae43e8740 1 --2- 172.21.15.111:0/3469649741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646cbf5ec30 0x5646cbfd7740 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fead40049d0 tx=0x7fead409bfb0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.838 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.833+0000 7feae43e8740 1 -- 172.21.15.111:0/3469649741 shutdown_connections 2024-09-15T22:48:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.833+0000 7feae43e8740 1 --2- 172.21.15.111:0/3469649741 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5646cbfd9860 0x5646cbfdbc80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.833+0000 7feae43e8740 1 --2- 172.21.15.111:0/3469649741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646cbf5ec30 0x5646cbfd7740 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.833+0000 7feae43e8740 1 --2- 172.21.15.111:0/3469649741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646cbf59ba0 0x5646cbf67c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.833+0000 7feae43e8740 1 --2- 172.21.15.111:0/3469649741 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646cbf59260 0x5646cbf676d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.833+0000 7feae43e8740 1 -- 172.21.15.111:0/3469649741 >> 172.21.15.111:0/3469649741 conn(0x5646cbf548d0 msgr2=0x5646cbf53a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.833+0000 7feabbe00640 0 client.15180 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:48:13.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.833+0000 7feae43e8740 1 -- 172.21.15.111:0/3469649741 shutdown_connections 2024-09-15T22:48:13.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.833+0000 7feae43e8740 1 -- 172.21.15.111:0/3469649741 wait complete. 2024-09-15T22:48:13.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.833+0000 7feae43e8740 20 client.15180 trim_cache size 0 max 16384 2024-09-15T22:48:13.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:48:13.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:48:13.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:48:13.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:48:13.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.834+0000 7feae0a00640 1 -- 172.21.15.111:0/574452255 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7feac00a3240 msgr2=0x7feac00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:13.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.834+0000 7feae0a00640 1 --2- 172.21.15.111:0/574452255 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7feac00a3240 0x7feac00a5700 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7feacc006fd0 tx=0x7feacc008040 comp rx=0 tx=0).stop 2024-09-15T22:48:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.834+0000 7feae0a00640 1 -- 172.21.15.111:0/574452255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feadc1054b0 msgr2=0x7feadc1a58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.834+0000 7feae0a00640 1 --2- 172.21.15.111:0/574452255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feadc1054b0 0x7feadc1a58b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fead400eb10 tx=0x7fead400c2d0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.834+0000 7feae0a00640 1 -- 172.21.15.111:0/574452255 shutdown_connections 2024-09-15T22:48:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.834+0000 7feae0a00640 1 --2- 172.21.15.111:0/574452255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feadc1054b0 0x7feadc1a58b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.834+0000 7feae0a00640 1 --2- 172.21.15.111:0/574452255 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7feac00a3240 0x7feac00a5700 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.834+0000 7feae0a00640 1 --2- 172.21.15.111:0/574452255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feadc102b00 0x7feadc19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.834+0000 7feae0a00640 1 --2- 172.21.15.111:0/574452255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feadc1001b0 0x7feadc19e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.834+0000 7feae0a00640 1 -- 172.21.15.111:0/574452255 >> 172.21.15.111:0/574452255 conn(0x7feadc0fbfa0 msgr2=0x7feadc100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.834+0000 7feae0a00640 1 -- 172.21.15.111:0/574452255 shutdown_connections 2024-09-15T22:48:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:13.834+0000 7feae0a00640 1 -- 172.21.15.111:0/574452255 wait complete. 2024-09-15T22:48:13.855 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:13 smithi111 ceph-mon[30629]: pgmap v419: 97 pgs: 97 active+clean; 64 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 208 op/s 2024-09-15T22:48:13.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:13 smithi121 ceph-mon[34815]: pgmap v419: 97 pgs: 97 active+clean; 64 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 208 op/s 2024-09-15T22:48:14.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:13 smithi192 ceph-mon[35427]: pgmap v419: 97 pgs: 97 active+clean; 64 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 208 op/s 2024-09-15T22:48:14.238 INFO:tasks.workunit.client.0.smithi111.stdout: 1 106773 14.15 MB/sec warmup 42 sec latency 938.790 ms 2024-09-15T22:48:15.238 INFO:tasks.workunit.client.0.smithi111.stdout: 1 106900 13.83 MB/sec warmup 43 sec latency 1791.235 ms 2024-09-15T22:48:15.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:15 smithi121 ceph-mon[34815]: pgmap v420: 97 pgs: 97 active+clean; 68 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 233 op/s 2024-09-15T22:48:15.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:15 smithi111 ceph-mon[30629]: pgmap v420: 97 pgs: 97 active+clean; 68 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 233 op/s 2024-09-15T22:48:16.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:15 smithi192 ceph-mon[35427]: pgmap v420: 97 pgs: 97 active+clean; 68 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 233 op/s 2024-09-15T22:48:16.239 INFO:tasks.workunit.client.0.smithi111.stdout: 1 110106 13.92 MB/sec warmup 44 sec latency 11.156 ms 2024-09-15T22:48:17.238 INFO:tasks.workunit.client.0.smithi111.stdout: 1 112876 13.89 MB/sec warmup 45 sec latency 11.013 ms 2024-09-15T22:48:17.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:17 smithi121 ceph-mon[34815]: pgmap v421: 97 pgs: 97 active+clean; 71 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 237 op/s 2024-09-15T22:48:17.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:17 smithi111 ceph-mon[30629]: pgmap v421: 97 pgs: 97 active+clean; 71 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 237 op/s 2024-09-15T22:48:18.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:17 smithi192 ceph-mon[35427]: pgmap v421: 97 pgs: 97 active+clean; 71 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 237 op/s 2024-09-15T22:48:18.238 INFO:tasks.workunit.client.0.smithi111.stdout: 1 115101 13.87 MB/sec warmup 46 sec latency 49.427 ms 2024-09-15T22:48:19.239 INFO:tasks.workunit.client.0.smithi111.stdout: 1 115952 13.61 MB/sec warmup 47 sec latency 725.619 ms 2024-09-15T22:48:19.844 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:19 smithi111 ceph-mon[30629]: pgmap v422: 97 pgs: 97 active+clean; 71 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 196 op/s 2024-09-15T22:48:19.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:19 smithi121 ceph-mon[34815]: pgmap v422: 97 pgs: 97 active+clean; 71 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 196 op/s 2024-09-15T22:48:20.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:19 smithi192 ceph-mon[35427]: pgmap v422: 97 pgs: 97 active+clean; 71 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 196 op/s 2024-09-15T22:48:20.239 INFO:tasks.workunit.client.0.smithi111.stdout: 1 115952 13.33 MB/sec warmup 48 sec latency 1725.688 ms 2024-09-15T22:48:21.239 INFO:tasks.workunit.client.0.smithi111.stdout: 1 115952 13.06 MB/sec warmup 49 sec latency 2725.731 ms 2024-09-15T22:48:21.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:21 smithi121 ceph-mon[34815]: pgmap v423: 97 pgs: 97 active+clean; 73 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2024-09-15T22:48:21.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:21 smithi111 ceph-mon[30629]: pgmap v423: 97 pgs: 97 active+clean; 73 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2024-09-15T22:48:22.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:21 smithi192 ceph-mon[35427]: pgmap v423: 97 pgs: 97 active+clean; 73 MiB data, 667 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2024-09-15T22:48:22.239 INFO:tasks.workunit.client.0.smithi111.stdout: 1 117912 13.05 MB/sec warmup 50 sec latency 2728.194 ms 2024-09-15T22:48:23.239 INFO:tasks.workunit.client.0.smithi111.stdout: 1 121009 13.16 MB/sec warmup 51 sec latency 11.093 ms 2024-09-15T22:48:23.796 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:23 smithi111 ceph-mon[30629]: pgmap v424: 97 pgs: 97 active+clean; 76 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 210 op/s 2024-09-15T22:48:23.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:23 smithi121 ceph-mon[34815]: pgmap v424: 97 pgs: 97 active+clean; 76 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 210 op/s 2024-09-15T22:48:24.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:23 smithi192 ceph-mon[35427]: pgmap v424: 97 pgs: 97 active+clean; 76 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 210 op/s 2024-09-15T22:48:24.241 INFO:tasks.workunit.client.0.smithi111.stdout: 1 121453 12.91 MB/sec warmup 52 sec latency 414.282 ms 2024-09-15T22:48:25.239 INFO:tasks.workunit.client.0.smithi111.stdout: 1 121453 12.67 MB/sec warmup 53 sec latency 1305.115 ms 2024-09-15T22:48:25.805 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:25 smithi111 ceph-mon[30629]: pgmap v425: 97 pgs: 97 active+clean; 76 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 200 op/s 2024-09-15T22:48:25.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:25 smithi121 ceph-mon[34815]: pgmap v425: 97 pgs: 97 active+clean; 76 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 200 op/s 2024-09-15T22:48:26.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:25 smithi192 ceph-mon[35427]: pgmap v425: 97 pgs: 97 active+clean; 76 MiB data, 703 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 200 op/s 2024-09-15T22:48:26.239 INFO:tasks.workunit.client.0.smithi111.stdout: 1 121453 12.44 MB/sec warmup 54 sec latency 2305.204 ms 2024-09-15T22:48:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.954+0000 7f869d800640 1 -- 172.21.15.111:0/2193188234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8698103b70 msgr2=0x7f8698103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.954+0000 7f869d800640 1 --2- 172.21.15.111:0/2193188234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8698103b70 0x7f8698103f70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8688009900 tx=0x7f868802a3f0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.954+0000 7f869d800640 1 -- 172.21.15.111:0/2193188234 shutdown_connections 2024-09-15T22:48:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.954+0000 7f869d800640 1 --2- 172.21.15.111:0/2193188234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8698105720 0x7f869810bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.954+0000 7f869d800640 1 --2- 172.21.15.111:0/2193188234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8698104d60 0x7f86981051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.954+0000 7f869d800640 1 --2- 172.21.15.111:0/2193188234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8698103b70 0x7f8698103f70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.954+0000 7f869d800640 1 -- 172.21.15.111:0/2193188234 >> 172.21.15.111:0/2193188234 conn(0x7f86980ff2f0 msgr2=0x7f8698101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.955+0000 7f869d800640 1 -- 172.21.15.111:0/2193188234 shutdown_connections 2024-09-15T22:48:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.955+0000 7f869d800640 1 -- 172.21.15.111:0/2193188234 wait complete. 2024-09-15T22:48:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.955+0000 7f869d800640 1 Processor -- start 2024-09-15T22:48:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.956+0000 7f869d800640 1 -- start start 2024-09-15T22:48:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.956+0000 7f869d800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8698103b70 0x7f8698199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.956+0000 7f869d800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8698104d60 0x7f869819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.956+0000 7f869d800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8698105720 0x7f86981a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.956+0000 7f869d800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f869810f7f0 con 0x7f8698105720 2024-09-15T22:48:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.956+0000 7f869d800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f869810f670 con 0x7f8698104d60 2024-09-15T22:48:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.956+0000 7f869d800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f869810f970 con 0x7f8698103b70 2024-09-15T22:48:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.956+0000 7f8697400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8698104d60 0x7f869819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.956+0000 7f8697400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8698104d60 0x7f869819a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:54124/0 (socket says 172.21.15.111:54124) 2024-09-15T22:48:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.956+0000 7f8697400640 1 -- 172.21.15.111:0/1925624269 learned_addr learned my addr 172.21.15.111:0/1925624269 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:48:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.956+0000 7f8697e00640 1 --2- 172.21.15.111:0/1925624269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8698103b70 0x7f8698199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.956+0000 7f869ce00640 1 --2- 172.21.15.111:0/1925624269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8698105720 0x7f86981a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.956+0000 7f8697400640 1 -- 172.21.15.111:0/1925624269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8698103b70 msgr2=0x7f8698199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.956+0000 7f8697400640 1 --2- 172.21.15.111:0/1925624269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8698103b70 0x7f8698199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.956+0000 7f8697400640 1 -- 172.21.15.111:0/1925624269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8698105720 msgr2=0x7f86981a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.956+0000 7f8697400640 1 --2- 172.21.15.111:0/1925624269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8698105720 0x7f86981a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.956+0000 7f8697400640 1 -- 172.21.15.111:0/1925624269 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86981a1ac0 con 0x7f8698104d60 2024-09-15T22:48:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.957+0000 7f8697e00640 1 --2- 172.21.15.111:0/1925624269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8698103b70 0x7f8698199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:48:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.957+0000 7f869ce00640 1 --2- 172.21.15.111:0/1925624269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8698105720 0x7f86981a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:48:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.957+0000 7f8697400640 1 --2- 172.21.15.111:0/1925624269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8698104d60 0x7f869819a3e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f868400e9f0 tx=0x7f868400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.957+0000 7f8694c00640 1 -- 172.21.15.111:0/1925624269 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868400cd80 con 0x7f8698104d60 2024-09-15T22:48:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.957+0000 7f869d800640 1 -- 172.21.15.111:0/1925624269 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86981a1db0 con 0x7f8698104d60 2024-09-15T22:48:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.957+0000 7f8694c00640 1 -- 172.21.15.111:0/1925624269 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8684005500 con 0x7f8698104d60 2024-09-15T22:48:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.957+0000 7f8694c00640 1 -- 172.21.15.111:0/1925624269 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684010690 con 0x7f8698104d60 2024-09-15T22:48:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.957+0000 7f869d800640 1 -- 172.21.15.111:0/1925624269 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86981a2270 con 0x7f8698104d60 2024-09-15T22:48:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.960+0000 7f86a1bf4740 1 Processor -- start 2024-09-15T22:48:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.960+0000 7f86a1bf4740 1 -- start start 2024-09-15T22:48:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.960+0000 7f86a1bf4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7feacf260 0x55f7feacf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.960+0000 7f86a1bf4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7feacfba0 0x55f7fead4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.960+0000 7f86a1bf4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7fead4c30 0x55f7fead70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.960+0000 7f86a1bf4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f7fea34100 con 0x55f7feacf260 2024-09-15T22:48:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.960+0000 7f86a1bf4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f7fe9dc1b0 con 0x55f7fead4c30 2024-09-15T22:48:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.961+0000 7f86a1bf4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f7fea300c0 con 0x55f7feacfba0 2024-09-15T22:48:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.961+0000 7f8694c00640 1 -- 172.21.15.111:0/1925624269 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f86840046c0 con 0x7f8698104d60 2024-09-15T22:48:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.961+0000 7f8697e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7feacfba0 0x55f7fead4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.961+0000 7f869ce00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7feacf260 0x55f7feacf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.961+0000 7f8697e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7feacfba0 0x55f7fead4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:49650/0 (socket says 172.21.15.111:49650) 2024-09-15T22:48:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.961+0000 7f8697e00640 1 -- 172.21.15.111:0/2275834625 learned_addr learned my addr 172.21.15.111:0/2275834625 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:48:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.961+0000 7f8697400640 1 --2- 172.21.15.111:0/2275834625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7fead4c30 0x55f7fead70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.961+0000 7f8697e00640 1 -- 172.21.15.111:0/2275834625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7fead4c30 msgr2=0x55f7fead70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.961+0000 7f8697e00640 1 --2- 172.21.15.111:0/2275834625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7fead4c30 0x55f7fead70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.961+0000 7f8697e00640 1 -- 172.21.15.111:0/2275834625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7feacf260 msgr2=0x55f7feacf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.961+0000 7f8697e00640 1 --2- 172.21.15.111:0/2275834625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7feacf260 0x55f7feacf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.961+0000 7f8697e00640 1 -- 172.21.15.111:0/2275834625 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7fe747210 con 0x55f7feacfba0 2024-09-15T22:48:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.961+0000 7f869ce00640 1 --2- 172.21.15.111:0/2275834625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7feacf260 0x55f7feacf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:48:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.962+0000 7f8697e00640 1 --2- 172.21.15.111:0/2275834625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7feacfba0 0x55f7fead4660 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8698104bc0 tx=0x7f8688005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b53dae3d74009d60 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.962+0000 7f868d600640 1 -- 172.21.15.111:0/2275834625 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8688040070 con 0x55f7feacfba0 2024-09-15T22:48:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.962+0000 7f868d600640 1 -- 172.21.15.111:0/2275834625 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86880319b0 con 0x55f7feacfba0 2024-09-15T22:48:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.962+0000 7f868d600640 1 -- 172.21.15.111:0/2275834625 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8688035040 con 0x55f7feacfba0 2024-09-15T22:48:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.962+0000 7f8694c00640 1 --2- 172.21.15.111:0/1925624269 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f867c0a32c0 0x7f867c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.962+0000 7f8694c00640 1 -- 172.21.15.111:0/1925624269 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f86840cfc60 con 0x7f8698104d60 2024-09-15T22:48:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.962+0000 7f869ce00640 1 --2- 172.21.15.111:0/1925624269 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f867c0a32c0 0x7f867c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.962+0000 7f86a1bf4740 1 -- 172.21.15.111:0/2275834625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7feacfba0 msgr2=0x55f7fead4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.963+0000 7f86a1bf4740 1 --2- 172.21.15.111:0/2275834625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7feacfba0 0x55f7fead4660 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8698104bc0 tx=0x7f8688005de0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.963+0000 7f86a1bf4740 1 -- 172.21.15.111:0/2275834625 shutdown_connections 2024-09-15T22:48:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.963+0000 7f86a1bf4740 1 --2- 172.21.15.111:0/2275834625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7fead4c30 0x55f7fead70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.963+0000 7f86a1bf4740 1 --2- 172.21.15.111:0/2275834625 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7feacfba0 0x55f7fead4660 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.963+0000 7f86a1bf4740 1 --2- 172.21.15.111:0/2275834625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7feacf260 0x55f7feacf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.963+0000 7f86a1bf4740 1 -- 172.21.15.111:0/2275834625 >> 172.21.15.111:0/2275834625 conn(0x55f7feaca8d0 msgr2=0x55f7feacacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.963+0000 7f86a1bf4740 1 -- 172.21.15.111:0/2275834625 shutdown_connections 2024-09-15T22:48:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.963+0000 7f86a1bf4740 1 -- 172.21.15.111:0/2275834625 wait complete. 2024-09-15T22:48:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.963+0000 7f869ce00640 1 --2- 172.21.15.111:0/1925624269 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f867c0a32c0 0x7f867c0a5780 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8690009fd0 tx=0x7f8690009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.963+0000 7f86a1bf4740 1 -- start start 2024-09-15T22:48:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.964+0000 7f86a1bf4740 1 Processor -- start 2024-09-15T22:48:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.964+0000 7f86a1bf4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7feacf260 0x55f7fead9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.964+0000 7f86a1bf4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7feacfba0 0x55f7fead9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.964+0000 7f86a1bf4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7fead4c30 0x55f7feae0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.964+0000 7f86a1bf4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f7fead7ba0 con 0x55f7feacfba0 2024-09-15T22:48:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.964+0000 7f86a1bf4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f7fe9bd090 con 0x55f7fead4c30 2024-09-15T22:48:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.964+0000 7f86a1bf4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f7fead7d20 con 0x55f7feacf260 2024-09-15T22:48:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.964+0000 7f86a1bf4740 10 client.0 _ll_register_callbacks cb 0x55f7fea11f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:48:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.964+0000 7f8697e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7feacf260 0x55f7fead9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.964+0000 7f869ce00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7fead4c30 0x55f7feae0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.964+0000 7f8697e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7feacf260 0x55f7fead9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:49664/0 (socket says 172.21.15.111:49664) 2024-09-15T22:48:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.964+0000 7f8697e00640 1 -- 172.21.15.111:0/1137488601 learned_addr learned my addr 172.21.15.111:0/1137488601 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:48:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.964+0000 7f8697400640 1 --2- 172.21.15.111:0/1137488601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7feacfba0 0x55f7fead9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.965+0000 7f8697e00640 1 -- 172.21.15.111:0/1137488601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7fead4c30 msgr2=0x55f7feae0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.965+0000 7f8697e00640 1 --2- 172.21.15.111:0/1137488601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7fead4c30 0x55f7feae0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.965+0000 7f8697e00640 1 -- 172.21.15.111:0/1137488601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7feacfba0 msgr2=0x55f7fead9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.965+0000 7f8697e00640 1 --2- 172.21.15.111:0/1137488601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7feacfba0 0x55f7fead9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.965+0000 7f8697e00640 1 -- 172.21.15.111:0/1137488601 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7fe813620 con 0x55f7feacf260 2024-09-15T22:48:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.965+0000 7f869ce00640 1 --2- 172.21.15.111:0/1137488601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7fead4c30 0x55f7feae0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:48:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.965+0000 7f8697e00640 1 --2- 172.21.15.111:0/1137488601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7feacf260 0x55f7fead9410 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f868802a910 tx=0x7f868802aba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cc1cf69fd2ca5a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.965+0000 7f8682000640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:48:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.965+0000 7f8682000640 1 -- 172.21.15.111:0/1137488601 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868804b070 con 0x55f7feacf260 2024-09-15T22:48:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.965+0000 7f86a1bf4740 1 -- 172.21.15.111:0/1137488601 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86880319b0 con 0x55f7feacf260 2024-09-15T22:48:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.965+0000 7f8682000640 1 -- 172.21.15.111:0/1137488601 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f86880035f0 con 0x55f7feacf260 2024-09-15T22:48:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.965+0000 7f8682000640 1 -- 172.21.15.111:0/1137488601 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8688057050 con 0x55f7feacf260 2024-09-15T22:48:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.965+0000 7f86a1bf4740 10 client.25213 fetch_fsmap learned FSMap version 28 2024-09-15T22:48:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.965+0000 7f86a1bf4740 1 -- 172.21.15.111:0/1137488601 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f7fead7ba0 con 0x55f7feacf260 2024-09-15T22:48:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.966+0000 7f8682000640 1 -- 172.21.15.111:0/1137488601 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86880319b0 con 0x55f7feacf260 2024-09-15T22:48:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.966+0000 7f8682000640 1 -- 172.21.15.111:0/1137488601 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f868803ca10 con 0x55f7feacf260 2024-09-15T22:48:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.966+0000 7f86a1bf4740 10 client.25213 fetch_fsmap finished waiting for FSMap version 28 2024-09-15T22:48:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.966+0000 7f86a1bf4740 10 client.25213 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:48:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.966+0000 7f86a1bf4740 20 client.25213 populate_metadata read hostname 'smithi111' 2024-09-15T22:48:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.966+0000 7f86a1bf4740 1 --2- 172.21.15.111:0/1137488601 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f7feae49a0 0x55f7feae6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.966+0000 7f86a1bf4740 4 client.25213 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:48:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.966+0000 7f86a1bf4740 1 -- 172.21.15.111:0/1137488601 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7fea41830 con 0x55f7feae49a0 2024-09-15T22:48:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.966+0000 7f8697400640 1 --2- 172.21.15.111:0/1137488601 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f7feae49a0 0x55f7feae6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.967+0000 7f8697400640 1 --2- 172.21.15.111:0/1137488601 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f7feae49a0 0x55f7feae6e00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c78b43dcf1ff120e server_cookie=15095e21dfe0f30f in_seq=0 out_seq=0 2024-09-15T22:48:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.967+0000 7f8682000640 10 client.25213 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:48:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.967+0000 7f8682000640 1 -- 172.21.15.111:0/1137488601 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f7fea41830 con 0x55f7feae49a0 2024-09-15T22:48:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.967+0000 7f8682000640 10 client.25213 handle_command_reply: tid=0 2024-09-15T22:48:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.968+0000 7f86a1bf4740 1 client.25213 shutdown 2024-09-15T22:48:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.968+0000 7f86a1bf4740 1 -- 172.21.15.111:0/1137488601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7feacf260 msgr2=0x55f7fead9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.968+0000 7f86a1bf4740 1 --2- 172.21.15.111:0/1137488601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7feacf260 0x55f7fead9410 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f868802a910 tx=0x7f868802aba0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.968+0000 7f86a1bf4740 1 -- 172.21.15.111:0/1137488601 shutdown_connections 2024-09-15T22:48:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.968+0000 7f86a1bf4740 1 --2- 172.21.15.111:0/1137488601 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f7feae49a0 0x55f7feae6e00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.968+0000 7f86a1bf4740 1 --2- 172.21.15.111:0/1137488601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7fead4c30 0x55f7feae0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.968+0000 7f86a1bf4740 1 --2- 172.21.15.111:0/1137488601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7feacfba0 0x55f7fead9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.968+0000 7f8682000640 0 client.25213 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:48:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.968+0000 7f86a1bf4740 1 --2- 172.21.15.111:0/1137488601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7feacf260 0x55f7fead9410 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.968+0000 7f86a1bf4740 1 -- 172.21.15.111:0/1137488601 >> 172.21.15.111:0/1137488601 conn(0x55f7feaca8d0 msgr2=0x55f7feacc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.968+0000 7f86a1bf4740 1 -- 172.21.15.111:0/1137488601 shutdown_connections 2024-09-15T22:48:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.968+0000 7f86a1bf4740 1 -- 172.21.15.111:0/1137488601 wait complete. 2024-09-15T22:48:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.968+0000 7f86a1bf4740 20 client.25213 trim_cache size 0 max 16384 2024-09-15T22:48:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.986+0000 7f86a1bf4740 1 Processor -- start 2024-09-15T22:48:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.987+0000 7f86a1bf4740 1 -- start start 2024-09-15T22:48:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.987+0000 7f86a1bf4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7feacf260 0x55f7feac9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.987+0000 7f86a1bf4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7feacfba0 0x55f7feac9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.987+0000 7f86a1bf4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7fead4c30 0x55f7feae0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.987+0000 7f86a1bf4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f86981ad710 con 0x55f7fead4c30 2024-09-15T22:48:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.987+0000 7f86a1bf4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f86981aafe0 con 0x55f7feacfba0 2024-09-15T22:48:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.987+0000 7f86a1bf4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f86981a8e80 con 0x55f7feacf260 2024-09-15T22:48:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.987+0000 7f8697400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7feacfba0 0x55f7feac9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.987+0000 7f8697e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7feacf260 0x55f7feac9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.987+0000 7f8697400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7feacfba0 0x55f7feac9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:54158/0 (socket says 172.21.15.111:54158) 2024-09-15T22:48:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.987+0000 7f8697400640 1 -- 172.21.15.111:0/1111758230 learned_addr learned my addr 172.21.15.111:0/1111758230 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:48:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.987+0000 7f869ce00640 1 --2- 172.21.15.111:0/1111758230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7fead4c30 0x55f7feae0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.987+0000 7f8697400640 1 -- 172.21.15.111:0/1111758230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7feacf260 msgr2=0x55f7feac9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.987+0000 7f8697400640 1 --2- 172.21.15.111:0/1111758230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7feacf260 0x55f7feac9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.987+0000 7f8697400640 1 -- 172.21.15.111:0/1111758230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7fead4c30 msgr2=0x55f7feae0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.987+0000 7f8697400640 1 --2- 172.21.15.111:0/1111758230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7fead4c30 0x55f7feae0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.987+0000 7f8697400640 1 -- 172.21.15.111:0/1111758230 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7feaca6f0 con 0x55f7feacfba0 2024-09-15T22:48:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.987+0000 7f869ce00640 1 --2- 172.21.15.111:0/1111758230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7fead4c30 0x55f7feae0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:48:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.988+0000 7f8697400640 1 --2- 172.21.15.111:0/1111758230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7feacfba0 0x55f7feac9e40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f86840986e0 tx=0x7f86840d36a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7798a2fe2d891bfd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.988+0000 7f868cc00640 1 -- 172.21.15.111:0/1111758230 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868409c050 con 0x55f7feacfba0 2024-09-15T22:48:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.988+0000 7f868cc00640 1 -- 172.21.15.111:0/1111758230 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f868409c980 con 0x55f7feacfba0 2024-09-15T22:48:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.988+0000 7f868cc00640 1 -- 172.21.15.111:0/1111758230 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868409bbe0 con 0x55f7feacfba0 2024-09-15T22:48:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.988+0000 7f86a1bf4740 1 -- 172.21.15.111:0/1111758230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7feacfba0 msgr2=0x55f7feac9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.988+0000 7f86a1bf4740 1 --2- 172.21.15.111:0/1111758230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7feacfba0 0x55f7feac9e40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f86840986e0 tx=0x7f86840d36a0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.989+0000 7f86a1bf4740 1 -- 172.21.15.111:0/1111758230 shutdown_connections 2024-09-15T22:48:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.989+0000 7f86a1bf4740 1 --2- 172.21.15.111:0/1111758230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7fead4c30 0x55f7feae0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.989+0000 7f86a1bf4740 1 --2- 172.21.15.111:0/1111758230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7feacfba0 0x55f7feac9e40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.989+0000 7f86a1bf4740 1 --2- 172.21.15.111:0/1111758230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7feacf260 0x55f7feac9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.989+0000 7f86a1bf4740 1 -- 172.21.15.111:0/1111758230 >> 172.21.15.111:0/1111758230 conn(0x55f7feaca8d0 msgr2=0x55f7feac91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.989+0000 7f86a1bf4740 1 -- 172.21.15.111:0/1111758230 shutdown_connections 2024-09-15T22:48:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.989+0000 7f86a1bf4740 1 -- 172.21.15.111:0/1111758230 wait complete. 2024-09-15T22:48:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.989+0000 7f86a1bf4740 1 -- start start 2024-09-15T22:48:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.989+0000 7f86a1bf4740 1 Processor -- start 2024-09-15T22:48:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.990+0000 7f86a1bf4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7feacf260 0x55f7feadd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.990+0000 7f86a1bf4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7feacfba0 0x55f7feaddeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.990+0000 7f86a1bf4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7fead4c30 0x55f7feadeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.990+0000 7f86a1bf4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f86981a94a0 con 0x55f7feacf260 2024-09-15T22:48:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.990+0000 7f86a1bf4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f86981aa910 con 0x55f7fead4c30 2024-09-15T22:48:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.990+0000 7f86a1bf4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f86981a9b70 con 0x55f7feacfba0 2024-09-15T22:48:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.990+0000 7f86a1bf4740 10 client.0 _ll_register_callbacks cb 0x7f8690009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:48:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.990+0000 7f869ce00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7fead4c30 0x55f7feadeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.990+0000 7f8697400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7feacfba0 0x55f7feaddeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.990+0000 7f8697e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7feacf260 0x55f7feadd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.990+0000 7f8697400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7feacfba0 0x55f7feaddeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:49676/0 (socket says 172.21.15.111:49676) 2024-09-15T22:48:26.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.990+0000 7f869ce00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7fead4c30 0x55f7feadeec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:54162/0 (socket says 172.21.15.111:54162) 2024-09-15T22:48:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.990+0000 7f8697400640 1 -- 172.21.15.111:0/2680167017 learned_addr learned my addr 172.21.15.111:0/2680167017 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:48:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.990+0000 7f8697400640 1 -- 172.21.15.111:0/2680167017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7fead4c30 msgr2=0x55f7feadeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.990+0000 7f8697400640 1 --2- 172.21.15.111:0/2680167017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7fead4c30 0x55f7feadeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.990+0000 7f8697400640 1 -- 172.21.15.111:0/2680167017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7feacf260 msgr2=0x55f7feadd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.990+0000 7f8697400640 1 --2- 172.21.15.111:0/2680167017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7feacf260 0x55f7feadd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.990+0000 7f8697400640 1 -- 172.21.15.111:0/2680167017 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7feadf4d0 con 0x55f7feacfba0 2024-09-15T22:48:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.990+0000 7f8697400640 1 --2- 172.21.15.111:0/2680167017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7feacfba0 0x55f7feaddeb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f868409bf80 tx=0x7f86840a06b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7243fdff52a2e7cf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.990+0000 7f8682000640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:48:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.991+0000 7f8682000640 1 -- 172.21.15.111:0/2680167017 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86840b4020 con 0x55f7feacfba0 2024-09-15T22:48:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.991+0000 7f86a1bf4740 1 -- 172.21.15.111:0/2680167017 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f868409c980 con 0x55f7feacfba0 2024-09-15T22:48:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.991+0000 7f8682000640 1 -- 172.21.15.111:0/2680167017 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f86840988f0 con 0x55f7feacfba0 2024-09-15T22:48:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.991+0000 7f8682000640 1 -- 172.21.15.111:0/2680167017 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f868409bdc0 con 0x55f7feacfba0 2024-09-15T22:48:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.991+0000 7f86a1bf4740 10 client.25219 fetch_fsmap learned FSMap version 28 2024-09-15T22:48:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.991+0000 7f86a1bf4740 1 -- 172.21.15.111:0/2680167017 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f7feae0850 con 0x55f7feacfba0 2024-09-15T22:48:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.991+0000 7f8682000640 1 -- 172.21.15.111:0/2680167017 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f868409c980 con 0x55f7feacfba0 2024-09-15T22:48:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.991+0000 7f8682000640 1 -- 172.21.15.111:0/2680167017 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f86840a4a60 con 0x55f7feacfba0 2024-09-15T22:48:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.991+0000 7f86a1bf4740 10 client.25219 fetch_fsmap finished waiting for FSMap version 28 2024-09-15T22:48:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.991+0000 7f86a1bf4740 10 client.25219 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:48:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.991+0000 7f86a1bf4740 20 client.25219 populate_metadata read hostname 'smithi111' 2024-09-15T22:48:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.991+0000 7f86a1bf4740 1 --2- 172.21.15.111:0/2680167017 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f7feb4fb80 0x55f7feb51fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.991+0000 7f86a1bf4740 4 client.25219 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:48:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.991+0000 7f86a1bf4740 1 -- 172.21.15.111:0/2680167017 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f7fea3bb80 con 0x55f7feb4fb80 2024-09-15T22:48:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.991+0000 7f8697e00640 1 --2- 172.21.15.111:0/2680167017 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f7feb4fb80 0x55f7feb51fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.992+0000 7f8697e00640 1 --2- 172.21.15.111:0/2680167017 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f7feb4fb80 0x55f7feb51fa0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a2214ba3e7732ef server_cookie=2abeafa3ef00608d in_seq=0 out_seq=0 2024-09-15T22:48:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.992+0000 7f8682000640 10 client.25219 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:48:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.999+0000 7f8682000640 1 -- 172.21.15.111:0/2680167017 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f7fea3bb80 con 0x55f7feb4fb80 2024-09-15T22:48:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.999+0000 7f8682000640 10 client.25219 handle_command_reply: tid=0 2024-09-15T22:48:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.999+0000 7f86a1bf4740 1 client.25219 shutdown 2024-09-15T22:48:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.999+0000 7f86a1bf4740 1 -- 172.21.15.111:0/2680167017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7feacfba0 msgr2=0x55f7feaddeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.999+0000 7f86a1bf4740 1 --2- 172.21.15.111:0/2680167017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7feacfba0 0x55f7feaddeb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f868409bf80 tx=0x7f86840a06b0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.999+0000 7f86a1bf4740 1 -- 172.21.15.111:0/2680167017 shutdown_connections 2024-09-15T22:48:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.999+0000 7f86a1bf4740 1 --2- 172.21.15.111:0/2680167017 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f7feb4fb80 0x55f7feb51fa0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.999+0000 7f86a1bf4740 1 --2- 172.21.15.111:0/2680167017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f7fead4c30 0x55f7feadeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.999+0000 7f86a1bf4740 1 --2- 172.21.15.111:0/2680167017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f7feacfba0 0x55f7feaddeb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.999+0000 7f8682000640 0 client.25219 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:48:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.999+0000 7f86a1bf4740 1 --2- 172.21.15.111:0/2680167017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f7feacf260 0x55f7feadd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.999+0000 7f86a1bf4740 1 -- 172.21.15.111:0/2680167017 >> 172.21.15.111:0/2680167017 conn(0x55f7feaca8d0 msgr2=0x55f7feac9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.999+0000 7f86a1bf4740 1 -- 172.21.15.111:0/2680167017 shutdown_connections 2024-09-15T22:48:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:26.999+0000 7f86a1bf4740 1 -- 172.21.15.111:0/2680167017 wait complete. 2024-09-15T22:48:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:27.000+0000 7f86a1bf4740 20 client.25219 trim_cache size 0 max 16384 2024-09-15T22:48:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:48:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:48:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:48:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:48:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:27.000+0000 7f869d800640 1 -- 172.21.15.111:0/1925624269 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f867c0a32c0 msgr2=0x7f867c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:27.000+0000 7f869d800640 1 --2- 172.21.15.111:0/1925624269 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f867c0a32c0 0x7f867c0a5780 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8690009fd0 tx=0x7f8690009500 comp rx=0 tx=0).stop 2024-09-15T22:48:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:27.000+0000 7f869d800640 1 -- 172.21.15.111:0/1925624269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8698104d60 msgr2=0x7f869819a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:27.000+0000 7f869d800640 1 --2- 172.21.15.111:0/1925624269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8698104d60 0x7f869819a3e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f868400e9f0 tx=0x7f868400edf0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:27.000+0000 7f869d800640 1 -- 172.21.15.111:0/1925624269 shutdown_connections 2024-09-15T22:48:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:27.000+0000 7f869d800640 1 --2- 172.21.15.111:0/1925624269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8698105720 0x7f86981a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:27.000+0000 7f869d800640 1 --2- 172.21.15.111:0/1925624269 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f867c0a32c0 0x7f867c0a5780 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:27.000+0000 7f869d800640 1 --2- 172.21.15.111:0/1925624269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8698104d60 0x7f869819a3e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:27.000+0000 7f869d800640 1 --2- 172.21.15.111:0/1925624269 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8698103b70 0x7f8698199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:27.000+0000 7f869d800640 1 -- 172.21.15.111:0/1925624269 >> 172.21.15.111:0/1925624269 conn(0x7f86980ff2f0 msgr2=0x7f8698108180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:27.000+0000 7f869d800640 1 -- 172.21.15.111:0/1925624269 shutdown_connections 2024-09-15T22:48:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:27.001+0000 7f869d800640 1 -- 172.21.15.111:0/1925624269 wait complete. 2024-09-15T22:48:27.239 INFO:tasks.workunit.client.0.smithi111.stdout: 1 124449 12.54 MB/sec warmup 55 sec latency 2433.008 ms 2024-09-15T22:48:27.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:27 smithi121 ceph-mon[34815]: pgmap v426: 97 pgs: 97 active+clean; 75 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 193 op/s 2024-09-15T22:48:27.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:27 smithi111 ceph-mon[30629]: pgmap v426: 97 pgs: 97 active+clean; 75 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 193 op/s 2024-09-15T22:48:28.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:27 smithi192 ceph-mon[35427]: pgmap v426: 97 pgs: 97 active+clean; 75 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 193 op/s 2024-09-15T22:48:28.239 INFO:tasks.workunit.client.0.smithi111.stdout: 1 127681 12.63 MB/sec warmup 56 sec latency 13.237 ms 2024-09-15T22:48:29.239 INFO:tasks.workunit.client.0.smithi111.stdout: 1 128285 12.44 MB/sec warmup 57 sec latency 579.772 ms 2024-09-15T22:48:29.792 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:29 smithi111 ceph-mon[30629]: pgmap v427: 97 pgs: 97 active+clean; 73 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 194 op/s 2024-09-15T22:48:29.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:29 smithi121 ceph-mon[34815]: pgmap v427: 97 pgs: 97 active+clean; 73 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 194 op/s 2024-09-15T22:48:30.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:29 smithi192 ceph-mon[35427]: pgmap v427: 97 pgs: 97 active+clean; 73 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 194 op/s 2024-09-15T22:48:30.239 INFO:tasks.workunit.client.0.smithi111.stdout: 1 128285 12.23 MB/sec warmup 58 sec latency 1579.815 ms 2024-09-15T22:48:31.239 INFO:tasks.workunit.client.0.smithi111.stdout: 1 128304 12.02 MB/sec warmup 59 sec latency 2563.526 ms 2024-09-15T22:48:31.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:31 smithi121 ceph-mon[34815]: pgmap v428: 97 pgs: 97 active+clean; 72 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 221 op/s 2024-09-15T22:48:31.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:31 smithi111 ceph-mon[30629]: pgmap v428: 97 pgs: 97 active+clean; 72 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 221 op/s 2024-09-15T22:48:32.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:31 smithi192 ceph-mon[35427]: pgmap v428: 97 pgs: 97 active+clean; 72 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 221 op/s 2024-09-15T22:48:32.239 INFO:tasks.workunit.client.0.smithi111.stdout: 1 131438 12.12 MB/sec warmup 60 sec latency 44.651 ms 2024-09-15T22:48:33.239 INFO:tasks.workunit.client.0.smithi111.stdout: 1 134196 12.15 MB/sec warmup 61 sec latency 16.164 ms 2024-09-15T22:48:33.889 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:33 smithi111 ceph-mon[30629]: pgmap v429: 97 pgs: 97 active+clean; 77 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 205 op/s 2024-09-15T22:48:33.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:33 smithi121 ceph-mon[34815]: pgmap v429: 97 pgs: 97 active+clean; 77 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 205 op/s 2024-09-15T22:48:34.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:33 smithi192 ceph-mon[35427]: pgmap v429: 97 pgs: 97 active+clean; 77 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 205 op/s 2024-09-15T22:48:34.240 INFO:tasks.workunit.client.0.smithi111.stdout: 1 135290 12.06 MB/sec warmup 62 sec latency 475.138 ms 2024-09-15T22:48:35.240 INFO:tasks.workunit.client.0.smithi111.stdout: 1 135290 11.87 MB/sec warmup 63 sec latency 1475.193 ms 2024-09-15T22:48:35.852 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:35 smithi111 ceph-mon[30629]: pgmap v430: 97 pgs: 97 active+clean; 79 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 203 op/s 2024-09-15T22:48:35.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:35 smithi121 ceph-mon[34815]: pgmap v430: 97 pgs: 97 active+clean; 79 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 203 op/s 2024-09-15T22:48:36.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:35 smithi192 ceph-mon[35427]: pgmap v430: 97 pgs: 97 active+clean; 79 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 203 op/s 2024-09-15T22:48:36.239 INFO:tasks.workunit.client.0.smithi111.stdout: 1 135418 11.68 MB/sec warmup 64 sec latency 2349.233 ms 2024-09-15T22:48:37.240 INFO:tasks.workunit.client.0.smithi111.stdout: 1 138585 11.79 MB/sec warmup 65 sec latency 13.295 ms 2024-09-15T22:48:37.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:37 smithi121 ceph-mon[34815]: pgmap v431: 97 pgs: 97 active+clean; 78 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 212 op/s 2024-09-15T22:48:37.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:37 smithi111 ceph-mon[30629]: pgmap v431: 97 pgs: 97 active+clean; 78 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 212 op/s 2024-09-15T22:48:38.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:37 smithi192 ceph-mon[35427]: pgmap v431: 97 pgs: 97 active+clean; 78 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 212 op/s 2024-09-15T22:48:38.240 INFO:tasks.workunit.client.0.smithi111.stdout: 1 141092 11.78 MB/sec warmup 66 sec latency 10.378 ms 2024-09-15T22:48:38.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:38 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:48:38.893 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:38 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:48:39.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:38 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:48:39.240 INFO:tasks.workunit.client.0.smithi111.stdout: 1 142463 11.73 MB/sec warmup 67 sec latency 343.005 ms 2024-09-15T22:48:39.774 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:39 smithi111 ceph-mon[30629]: pgmap v432: 97 pgs: 97 active+clean; 77 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 171 op/s 2024-09-15T22:48:39.774 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:39 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:48:39.774 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:39 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:48:39.774 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:39 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:48:39.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:39 smithi121 ceph-mon[34815]: pgmap v432: 97 pgs: 97 active+clean; 77 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 171 op/s 2024-09-15T22:48:39.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:39 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:48:39.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:39 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:48:39.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:39 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:48:40.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:39 smithi192 ceph-mon[35427]: pgmap v432: 97 pgs: 97 active+clean; 77 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 171 op/s 2024-09-15T22:48:40.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:39 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:48:40.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:39 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:48:40.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:39 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:48:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.115+0000 7f0cd1a00640 1 -- 172.21.15.111:0/3746268663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ccc103d60 msgr2=0x7f0ccc104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.115+0000 7f0cd1a00640 1 --2- 172.21.15.111:0/3746268663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ccc103d60 0x7f0ccc104160 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc009900 tx=0x7f0cbc02a410 comp rx=0 tx=0).stop 2024-09-15T22:48:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.115+0000 7f0cd1a00640 1 -- 172.21.15.111:0/3746268663 shutdown_connections 2024-09-15T22:48:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.115+0000 7f0cd1a00640 1 --2- 172.21.15.111:0/3746268663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ccc105910 0x7f0ccc10c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.115+0000 7f0cd1a00640 1 --2- 172.21.15.111:0/3746268663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ccc104f50 0x7f0ccc1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.115+0000 7f0cd1a00640 1 --2- 172.21.15.111:0/3746268663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ccc103d60 0x7f0ccc104160 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.115+0000 7f0cd1a00640 1 -- 172.21.15.111:0/3746268663 >> 172.21.15.111:0/3746268663 conn(0x7f0ccc0ff4e0 msgr2=0x7f0ccc101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.116+0000 7f0cd1a00640 1 -- 172.21.15.111:0/3746268663 shutdown_connections 2024-09-15T22:48:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.116+0000 7f0cd1a00640 1 -- 172.21.15.111:0/3746268663 wait complete. 2024-09-15T22:48:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.116+0000 7f0cd1a00640 1 Processor -- start 2024-09-15T22:48:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.117+0000 7f0cd1a00640 1 -- start start 2024-09-15T22:48:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.117+0000 7f0cd1a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ccc103d60 0x7f0ccc199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.117+0000 7f0cd1a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ccc104f50 0x7f0ccc19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.117+0000 7f0cd1a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ccc105910 0x7f0ccc1a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.117+0000 7f0cd1a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc10fa30 con 0x7f0ccc103d60 2024-09-15T22:48:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.117+0000 7f0cd1a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc10f8b0 con 0x7f0ccc105910 2024-09-15T22:48:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.117+0000 7f0cd1a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc10fbb0 con 0x7f0ccc104f50 2024-09-15T22:48:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.117+0000 7f0ccaa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ccc103d60 0x7f0ccc199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.117+0000 7f0ccaa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ccc103d60 0x7f0ccc199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39528/0 (socket says 172.21.15.111:39528) 2024-09-15T22:48:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.117+0000 7f0ccaa00640 1 -- 172.21.15.111:0/3868029350 learned_addr learned my addr 172.21.15.111:0/3868029350 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:48:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.117+0000 7f0ccb400640 1 --2- 172.21.15.111:0/3868029350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ccc105910 0x7f0ccc1a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.117+0000 7f0cca000640 1 --2- 172.21.15.111:0/3868029350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ccc104f50 0x7f0ccc19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.118+0000 7f0ccaa00640 1 -- 172.21.15.111:0/3868029350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ccc104f50 msgr2=0x7f0ccc19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.118+0000 7f0ccaa00640 1 --2- 172.21.15.111:0/3868029350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ccc104f50 0x7f0ccc19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.118+0000 7f0ccaa00640 1 -- 172.21.15.111:0/3868029350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ccc105910 msgr2=0x7f0ccc1a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.118+0000 7f0ccaa00640 1 --2- 172.21.15.111:0/3868029350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ccc105910 0x7f0ccc1a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.118+0000 7f0ccaa00640 1 -- 172.21.15.111:0/3868029350 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ccc1a1be0 con 0x7f0ccc103d60 2024-09-15T22:48:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.118+0000 7f0ccaa00640 1 --2- 172.21.15.111:0/3868029350 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ccc103d60 0x7f0ccc199fc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc031440 tx=0x7f0cbc032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.118+0000 7f0cc3400640 1 -- 172.21.15.111:0/3868029350 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc03f030 con 0x7f0ccc103d60 2024-09-15T22:48:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.118+0000 7f0cc3400640 1 -- 172.21.15.111:0/3868029350 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cbc003ce0 con 0x7f0ccc103d60 2024-09-15T22:48:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.118+0000 7f0cc3400640 1 -- 172.21.15.111:0/3868029350 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc035070 con 0x7f0ccc103d60 2024-09-15T22:48:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.118+0000 7f0cd1a00640 1 -- 172.21.15.111:0/3868029350 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ccc1a1e70 con 0x7f0ccc103d60 2024-09-15T22:48:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.118+0000 7f0cd1a00640 1 -- 172.21.15.111:0/3868029350 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ccc1a2280 con 0x7f0ccc103d60 2024-09-15T22:48:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.121+0000 7f0cc3400640 1 -- 172.21.15.111:0/3868029350 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f0cbc042030 con 0x7f0ccc103d60 2024-09-15T22:48:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.121+0000 7f0cd47e7740 1 Processor -- start 2024-09-15T22:48:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.121+0000 7f0cd47e7740 1 -- start start 2024-09-15T22:48:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.122+0000 7f0cd47e7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf00a08090 0x55bf00a08490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.122+0000 7f0cd47e7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf00a089d0 0x55bf00a0d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.122+0000 7f0cd47e7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bf00a0da60 0x55bf00a0ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.122+0000 7f0cd47e7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55bf0096d100 con 0x55bf00a089d0 2024-09-15T22:48:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.122+0000 7f0cd47e7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55bf009151b0 con 0x55bf00a08090 2024-09-15T22:48:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.122+0000 7f0cd47e7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55bf009690c0 con 0x55bf00a0da60 2024-09-15T22:48:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.122+0000 7f0ccaa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf00a089d0 0x55bf00a0d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.122+0000 7f0ccb400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf00a08090 0x55bf00a08490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.122+0000 7f0ccaa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf00a089d0 0x55bf00a0d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39544/0 (socket says 172.21.15.111:39544) 2024-09-15T22:48:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.122+0000 7f0ccaa00640 1 -- 172.21.15.111:0/439279651 learned_addr learned my addr 172.21.15.111:0/439279651 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:48:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.122+0000 7f0cca000640 1 --2- 172.21.15.111:0/439279651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bf00a0da60 0x55bf00a0ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.122+0000 7f0ccb400640 1 -- 172.21.15.111:0/439279651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bf00a0da60 msgr2=0x55bf00a0ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.122+0000 7f0ccb400640 1 --2- 172.21.15.111:0/439279651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bf00a0da60 0x55bf00a0ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.122+0000 7f0ccb400640 1 -- 172.21.15.111:0/439279651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf00a089d0 msgr2=0x55bf00a0d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.122+0000 7f0ccb400640 1 --2- 172.21.15.111:0/439279651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf00a089d0 0x55bf00a0d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.122+0000 7f0ccb400640 1 -- 172.21.15.111:0/439279651 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf00680210 con 0x55bf00a08090 2024-09-15T22:48:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.123+0000 7f0ccb400640 1 --2- 172.21.15.111:0/439279651 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf00a08090 0x55bf00a08490 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4009f00 tx=0x7f0cc4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61394ddc6d6a869 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.123+0000 7f0cb7e00640 1 -- 172.21.15.111:0/439279651 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4004d20 con 0x55bf00a08090 2024-09-15T22:48:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.123+0000 7f0cc3400640 1 --2- 172.21.15.111:0/3868029350 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0cb00a3370 0x7f0cb00a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.123+0000 7f0cb7e00640 1 -- 172.21.15.111:0/439279651 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cc4004030 con 0x55bf00a08090 2024-09-15T22:48:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.123+0000 7f0cb7e00640 1 -- 172.21.15.111:0/439279651 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4005260 con 0x55bf00a08090 2024-09-15T22:48:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.123+0000 7f0cc3400640 1 -- 172.21.15.111:0/3868029350 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f0cbc0efd20 con 0x7f0ccc103d60 2024-09-15T22:48:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.123+0000 7f0ccaa00640 1 --2- 172.21.15.111:0/3868029350 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0cb00a3370 0x7f0cb00a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.124+0000 7f0cd47e7740 1 -- 172.21.15.111:0/439279651 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf00a08090 msgr2=0x55bf00a08490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.124+0000 7f0cd47e7740 1 --2- 172.21.15.111:0/439279651 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf00a08090 0x55bf00a08490 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4009f00 tx=0x7f0cc4011810 comp rx=0 tx=0).stop 2024-09-15T22:48:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.124+0000 7f0cd47e7740 1 -- 172.21.15.111:0/439279651 shutdown_connections 2024-09-15T22:48:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.124+0000 7f0cd47e7740 1 --2- 172.21.15.111:0/439279651 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bf00a0da60 0x55bf00a0ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.124+0000 7f0cd47e7740 1 --2- 172.21.15.111:0/439279651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf00a089d0 0x55bf00a0d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.124+0000 7f0cd47e7740 1 --2- 172.21.15.111:0/439279651 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf00a08090 0x55bf00a08490 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.124+0000 7f0cd47e7740 1 -- 172.21.15.111:0/439279651 >> 172.21.15.111:0/439279651 conn(0x55bf00a03700 msgr2=0x55bf00a03b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.124+0000 7f0ccaa00640 1 --2- 172.21.15.111:0/3868029350 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0cb00a3370 0x7f0cb00a5830 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc006010 tx=0x7f0cbc0b8040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.124+0000 7f0cd47e7740 1 -- 172.21.15.111:0/439279651 shutdown_connections 2024-09-15T22:48:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.124+0000 7f0cd47e7740 1 -- 172.21.15.111:0/439279651 wait complete. 2024-09-15T22:48:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.124+0000 7f0cd47e7740 1 -- start start 2024-09-15T22:48:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.125+0000 7f0cd47e7740 1 Processor -- start 2024-09-15T22:48:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.125+0000 7f0cd47e7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf00a08090 0x55bf00a12280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.125+0000 7f0cd47e7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf00a089d0 0x55bf00a127c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.125+0000 7f0cd47e7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bf00a0da60 0x55bf00a19790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.125+0000 7f0cd47e7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55bf00a10a30 con 0x55bf00a089d0 2024-09-15T22:48:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.125+0000 7f0cd47e7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55bf008f6090 con 0x55bf00a08090 2024-09-15T22:48:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.125+0000 7f0cd47e7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55bf00a10bb0 con 0x55bf00a0da60 2024-09-15T22:48:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.125+0000 7f0cd47e7740 10 client.0 _ll_register_callbacks cb 0x55bf0094af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:48:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.125+0000 7f0ccb400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf00a08090 0x55bf00a12280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.125+0000 7f0cca000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf00a089d0 0x55bf00a127c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.125+0000 7f0ccb400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf00a08090 0x55bf00a12280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:59536/0 (socket says 172.21.15.111:59536) 2024-09-15T22:48:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.125+0000 7f0ccb400640 1 -- 172.21.15.111:0/2063218074 learned_addr learned my addr 172.21.15.111:0/2063218074 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:48:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.126+0000 7f0ccb400640 1 --2- 172.21.15.111:0/2063218074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bf00a0da60 0x55bf00a19790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.126+0000 7f0ccb400640 1 -- 172.21.15.111:0/2063218074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bf00a0da60 msgr2=0x55bf00a19790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.126+0000 7f0ccb400640 1 --2- 172.21.15.111:0/2063218074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bf00a0da60 0x55bf00a19790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.126+0000 7f0ccb400640 1 -- 172.21.15.111:0/2063218074 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf00a089d0 msgr2=0x55bf00a127c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.126+0000 7f0ccb400640 1 --2- 172.21.15.111:0/2063218074 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf00a089d0 0x55bf00a127c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.126+0000 7f0ccb400640 1 -- 172.21.15.111:0/2063218074 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf0074c620 con 0x55bf00a08090 2024-09-15T22:48:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.126+0000 7f0ccb400640 1 --2- 172.21.15.111:0/2063218074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf00a08090 0x55bf00a12280 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4004000 tx=0x7f0cc4009be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=781d3c73ae6498fc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.126+0000 7f0cb4c00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:48:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.126+0000 7f0cb4c00640 1 -- 172.21.15.111:0/2063218074 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4005950 con 0x55bf00a08090 2024-09-15T22:48:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.126+0000 7f0cb4c00640 1 -- 172.21.15.111:0/2063218074 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f0cc4004a40 con 0x55bf00a08090 2024-09-15T22:48:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.126+0000 7f0cd47e7740 1 -- 172.21.15.111:0/2063218074 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cc4004030 con 0x55bf00a08090 2024-09-15T22:48:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.126+0000 7f0cb4c00640 1 -- 172.21.15.111:0/2063218074 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0cc4004c40 con 0x55bf00a08090 2024-09-15T22:48:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.126+0000 7f0cd47e7740 10 client.25232 fetch_fsmap learned FSMap version 28 2024-09-15T22:48:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.126+0000 7f0cd47e7740 1 -- 172.21.15.111:0/2063218074 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bf00a1c580 con 0x55bf00a08090 2024-09-15T22:48:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.127+0000 7f0cb4c00640 1 -- 172.21.15.111:0/2063218074 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cc4004030 con 0x55bf00a08090 2024-09-15T22:48:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.127+0000 7f0cb4c00640 1 -- 172.21.15.111:0/2063218074 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0cc4018070 con 0x55bf00a08090 2024-09-15T22:48:40.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.127+0000 7f0cd47e7740 10 client.25232 fetch_fsmap finished waiting for FSMap version 28 2024-09-15T22:48:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.127+0000 7f0cd47e7740 10 client.25232 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:48:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.127+0000 7f0cd47e7740 20 client.25232 populate_metadata read hostname 'smithi111' 2024-09-15T22:48:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.127+0000 7f0cd47e7740 1 --2- 172.21.15.111:0/2063218074 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55bf00a1db70 0x55bf00a1ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.127+0000 7f0cd47e7740 4 client.25232 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:48:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.127+0000 7f0cd47e7740 1 -- 172.21.15.111:0/2063218074 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf0097a830 con 0x55bf00a1db70 2024-09-15T22:48:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.127+0000 7f0cca000640 1 --2- 172.21.15.111:0/2063218074 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55bf00a1db70 0x55bf00a1ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.128+0000 7f0cca000640 1 --2- 172.21.15.111:0/2063218074 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55bf00a1db70 0x55bf00a1ff90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6cdd2152a02836b server_cookie=a8e518eb8dfc7102 in_seq=0 out_seq=0 2024-09-15T22:48:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.128+0000 7f0cb4c00640 10 client.25232 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:48:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.128+0000 7f0cb4c00640 1 -- 172.21.15.111:0/2063218074 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf0097a830 con 0x55bf00a1db70 2024-09-15T22:48:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.128+0000 7f0cb4c00640 10 client.25232 handle_command_reply: tid=0 2024-09-15T22:48:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.128+0000 7f0cd47e7740 1 client.25232 shutdown 2024-09-15T22:48:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.129+0000 7f0cd47e7740 1 -- 172.21.15.111:0/2063218074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf00a08090 msgr2=0x55bf00a12280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.129+0000 7f0cd47e7740 1 --2- 172.21.15.111:0/2063218074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf00a08090 0x55bf00a12280 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0cc4004000 tx=0x7f0cc4009be0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.129+0000 7f0cd47e7740 1 -- 172.21.15.111:0/2063218074 shutdown_connections 2024-09-15T22:48:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.129+0000 7f0cd47e7740 1 --2- 172.21.15.111:0/2063218074 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55bf00a1db70 0x55bf00a1ff90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.129+0000 7f0cd47e7740 1 --2- 172.21.15.111:0/2063218074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bf00a0da60 0x55bf00a19790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.129+0000 7f0cb4c00640 0 client.25232 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:48:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.129+0000 7f0cd47e7740 1 --2- 172.21.15.111:0/2063218074 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf00a089d0 0x55bf00a127c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.129+0000 7f0cd47e7740 1 --2- 172.21.15.111:0/2063218074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf00a08090 0x55bf00a12280 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.129+0000 7f0cd47e7740 1 -- 172.21.15.111:0/2063218074 >> 172.21.15.111:0/2063218074 conn(0x55bf00a03700 msgr2=0x55bf00a04f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.129+0000 7f0cd47e7740 1 -- 172.21.15.111:0/2063218074 shutdown_connections 2024-09-15T22:48:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.129+0000 7f0cd47e7740 1 -- 172.21.15.111:0/2063218074 wait complete. 2024-09-15T22:48:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.129+0000 7f0cd47e7740 20 client.25232 trim_cache size 0 max 16384 2024-09-15T22:48:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.144+0000 7f0cd47e7740 1 Processor -- start 2024-09-15T22:48:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.144+0000 7f0cd47e7740 1 -- start start 2024-09-15T22:48:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.144+0000 7f0cd47e7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf00a08090 0x55bf00a04cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.144+0000 7f0cd47e7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bf00a089d0 0x55bf00a05200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.144+0000 7f0cd47e7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf00a0da60 0x55bf00a196b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.144+0000 7f0cd47e7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1ad920 con 0x55bf00a08090 2024-09-15T22:48:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.144+0000 7f0cd47e7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1ab1f0 con 0x55bf00a0da60 2024-09-15T22:48:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.144+0000 7f0cd47e7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1a9040 con 0x55bf00a089d0 2024-09-15T22:48:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.144+0000 7f0cca000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bf00a089d0 0x55bf00a05200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.144+0000 7f0cca000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bf00a089d0 0x55bf00a05200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:54364/0 (socket says 172.21.15.111:54364) 2024-09-15T22:48:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.144+0000 7f0cca000640 1 -- 172.21.15.111:0/2784362756 learned_addr learned my addr 172.21.15.111:0/2784362756 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:48:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.144+0000 7f0ccb400640 1 --2- 172.21.15.111:0/2784362756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf00a08090 0x55bf00a04cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.144+0000 7f0cca000640 1 -- 172.21.15.111:0/2784362756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf00a0da60 msgr2=0x55bf00a196b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.145+0000 7f0cca000640 1 --2- 172.21.15.111:0/2784362756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf00a0da60 0x55bf00a196b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.145+0000 7f0cca000640 1 -- 172.21.15.111:0/2784362756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf00a08090 msgr2=0x55bf00a04cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.145+0000 7f0cca000640 1 --2- 172.21.15.111:0/2784362756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf00a08090 0x55bf00a04cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.145+0000 7f0cca000640 1 -- 172.21.15.111:0/2784362756 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf00a03520 con 0x55bf00a089d0 2024-09-15T22:48:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.145+0000 7f0cca000640 1 --2- 172.21.15.111:0/2784362756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bf00a089d0 0x55bf00a05200 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55bf00a04580 tx=0x7f0cb8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7455aaf8774254e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.145+0000 7f0cb7400640 1 -- 172.21.15.111:0/2784362756 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8003cc0 con 0x55bf00a089d0 2024-09-15T22:48:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.145+0000 7f0cb7400640 1 -- 172.21.15.111:0/2784362756 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cb8010910 con 0x55bf00a089d0 2024-09-15T22:48:40.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.145+0000 7f0cb7400640 1 -- 172.21.15.111:0/2784362756 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb801e980 con 0x55bf00a089d0 2024-09-15T22:48:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.146+0000 7f0cd47e7740 1 -- 172.21.15.111:0/2784362756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bf00a089d0 msgr2=0x55bf00a05200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.146+0000 7f0cd47e7740 1 --2- 172.21.15.111:0/2784362756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bf00a089d0 0x55bf00a05200 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55bf00a04580 tx=0x7f0cb8005f90 comp rx=0 tx=0).stop 2024-09-15T22:48:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.146+0000 7f0cd47e7740 1 -- 172.21.15.111:0/2784362756 shutdown_connections 2024-09-15T22:48:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.146+0000 7f0cd47e7740 1 --2- 172.21.15.111:0/2784362756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf00a0da60 0x55bf00a196b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.146+0000 7f0cd47e7740 1 --2- 172.21.15.111:0/2784362756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bf00a089d0 0x55bf00a05200 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.146+0000 7f0cd47e7740 1 --2- 172.21.15.111:0/2784362756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf00a08090 0x55bf00a04cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.146+0000 7f0cd47e7740 1 -- 172.21.15.111:0/2784362756 >> 172.21.15.111:0/2784362756 conn(0x55bf00a03700 msgr2=0x55bf00a045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.146+0000 7f0cd47e7740 1 -- 172.21.15.111:0/2784362756 shutdown_connections 2024-09-15T22:48:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.146+0000 7f0cd47e7740 1 -- 172.21.15.111:0/2784362756 wait complete. 2024-09-15T22:48:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.146+0000 7f0cd47e7740 1 -- start start 2024-09-15T22:48:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.146+0000 7f0cd47e7740 1 Processor -- start 2024-09-15T22:48:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.147+0000 7f0cd47e7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bf00a08090 0x55bf00a16a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:40.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.147+0000 7f0cd47e7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf00a089d0 0x55bf00a16f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.147+0000 7f0cd47e7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf00a0da60 0x55bf00a17f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.147+0000 7f0cd47e7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1a96b0 con 0x55bf00a089d0 2024-09-15T22:48:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.147+0000 7f0cd47e7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1aab20 con 0x55bf00a0da60 2024-09-15T22:48:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.147+0000 7f0cd47e7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1a9d80 con 0x55bf00a08090 2024-09-15T22:48:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.147+0000 7f0cd47e7740 10 client.0 _ll_register_callbacks cb 0x7f0cbc0f3630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:48:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.147+0000 7f0cca000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf00a089d0 0x55bf00a16f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.147+0000 7f0ccb400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bf00a08090 0x55bf00a16a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.147+0000 7f0cca000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf00a089d0 0x55bf00a16f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39572/0 (socket says 172.21.15.111:39572) 2024-09-15T22:48:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.147+0000 7f0ccb400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bf00a08090 0x55bf00a16a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:54368/0 (socket says 172.21.15.111:54368) 2024-09-15T22:48:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.147+0000 7f0cca000640 1 -- 172.21.15.111:0/93570468 learned_addr learned my addr 172.21.15.111:0/93570468 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:48:40.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.147+0000 7f0ccb400640 1 --2- 172.21.15.111:0/93570468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf00a0da60 0x55bf00a17f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.147+0000 7f0cca000640 1 -- 172.21.15.111:0/93570468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bf00a08090 msgr2=0x55bf00a16a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.147+0000 7f0cca000640 1 --2- 172.21.15.111:0/93570468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bf00a08090 0x55bf00a16a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.147+0000 7f0cca000640 1 -- 172.21.15.111:0/93570468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf00a0da60 msgr2=0x55bf00a17f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.147+0000 7f0cca000640 1 --2- 172.21.15.111:0/93570468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf00a0da60 0x55bf00a17f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.147+0000 7f0cca000640 1 -- 172.21.15.111:0/93570468 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf00a185a0 con 0x55bf00a089d0 2024-09-15T22:48:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.148+0000 7f0cca000640 1 --2- 172.21.15.111:0/93570468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf00a089d0 0x55bf00a16f80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0cb80104c0 tx=0x7f0cb80104f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c383313da6220d50 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.148+0000 7f0cb4c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:48:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.148+0000 7f0cb4c00640 1 -- 172.21.15.111:0/93570468 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb8036060 con 0x55bf00a089d0 2024-09-15T22:48:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.148+0000 7f0cb4c00640 1 -- 172.21.15.111:0/93570468 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f0cb8010690 con 0x55bf00a089d0 2024-09-15T22:48:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.148+0000 7f0cd47e7740 1 -- 172.21.15.111:0/93570468 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cb8010910 con 0x55bf00a089d0 2024-09-15T22:48:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.148+0000 7f0cb4c00640 1 -- 172.21.15.111:0/93570468 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0cb8010ac0 con 0x55bf00a089d0 2024-09-15T22:48:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.148+0000 7f0cd47e7740 10 client.15216 fetch_fsmap learned FSMap version 28 2024-09-15T22:48:40.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.148+0000 7f0cd47e7740 1 -- 172.21.15.111:0/93570468 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bf00a19920 con 0x55bf00a089d0 2024-09-15T22:48:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.148+0000 7f0cb4c00640 1 -- 172.21.15.111:0/93570468 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cb8010910 con 0x55bf00a089d0 2024-09-15T22:48:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.149+0000 7f0cb4c00640 1 -- 172.21.15.111:0/93570468 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0cb801eca0 con 0x55bf00a089d0 2024-09-15T22:48:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.149+0000 7f0cd47e7740 10 client.15216 fetch_fsmap finished waiting for FSMap version 28 2024-09-15T22:48:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.149+0000 7f0cd47e7740 10 client.15216 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:48:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.149+0000 7f0cd47e7740 20 client.15216 populate_metadata read hostname 'smithi111' 2024-09-15T22:48:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.149+0000 7f0cd47e7740 1 --2- 172.21.15.111:0/93570468 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55bf00a88eb0 0x55bf00a8b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.149+0000 7f0cd47e7740 4 client.15216 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:48:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.149+0000 7f0cd47e7740 1 -- 172.21.15.111:0/93570468 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bf00974b80 con 0x55bf00a88eb0 2024-09-15T22:48:40.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.149+0000 7f0ccb400640 1 --2- 172.21.15.111:0/93570468 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55bf00a88eb0 0x55bf00a8b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.150+0000 7f0ccb400640 1 --2- 172.21.15.111:0/93570468 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55bf00a88eb0 0x55bf00a8b2d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecadd0fa5c325cd2 server_cookie=79c5bb5062041bec in_seq=0 out_seq=0 2024-09-15T22:48:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.150+0000 7f0cb4c00640 10 client.15216 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:48:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.166+0000 7f0cb4c00640 1 -- 172.21.15.111:0/93570468 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bf00974b80 con 0x55bf00a88eb0 2024-09-15T22:48:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.166+0000 7f0cb4c00640 10 client.15216 handle_command_reply: tid=0 2024-09-15T22:48:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.166+0000 7f0cd47e7740 1 client.15216 shutdown 2024-09-15T22:48:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.166+0000 7f0cd47e7740 1 -- 172.21.15.111:0/93570468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf00a089d0 msgr2=0x55bf00a16f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.166+0000 7f0cd47e7740 1 --2- 172.21.15.111:0/93570468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf00a089d0 0x55bf00a16f80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0cb80104c0 tx=0x7f0cb80104f0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.166+0000 7f0cd47e7740 1 -- 172.21.15.111:0/93570468 shutdown_connections 2024-09-15T22:48:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.166+0000 7f0cd47e7740 1 --2- 172.21.15.111:0/93570468 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55bf00a88eb0 0x55bf00a8b2d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.166+0000 7f0cd47e7740 1 --2- 172.21.15.111:0/93570468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bf00a0da60 0x55bf00a17f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.166+0000 7f0cd47e7740 1 --2- 172.21.15.111:0/93570468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55bf00a089d0 0x55bf00a16f80 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.166+0000 7f0cb4c00640 0 client.15216 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:48:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.166+0000 7f0cd47e7740 1 --2- 172.21.15.111:0/93570468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55bf00a08090 0x55bf00a16a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.166+0000 7f0cd47e7740 1 -- 172.21.15.111:0/93570468 >> 172.21.15.111:0/93570468 conn(0x55bf00a03700 msgr2=0x55bf00a12550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.166+0000 7f0cd47e7740 1 -- 172.21.15.111:0/93570468 shutdown_connections 2024-09-15T22:48:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.166+0000 7f0cd47e7740 1 -- 172.21.15.111:0/93570468 wait complete. 2024-09-15T22:48:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.166+0000 7f0cd47e7740 20 client.15216 trim_cache size 0 max 16384 2024-09-15T22:48:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:48:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:48:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:48:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:48:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.167+0000 7f0cd1a00640 1 -- 172.21.15.111:0/3868029350 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0cb00a3370 msgr2=0x7f0cb00a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.167+0000 7f0cd1a00640 1 --2- 172.21.15.111:0/3868029350 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0cb00a3370 0x7f0cb00a5830 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc006010 tx=0x7f0cbc0b8040 comp rx=0 tx=0).stop 2024-09-15T22:48:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.167+0000 7f0cd1a00640 1 -- 172.21.15.111:0/3868029350 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ccc103d60 msgr2=0x7f0ccc199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.167+0000 7f0cd1a00640 1 --2- 172.21.15.111:0/3868029350 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ccc103d60 0x7f0ccc199fc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc031440 tx=0x7f0cbc032ce0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.167+0000 7f0cd1a00640 1 -- 172.21.15.111:0/3868029350 shutdown_connections 2024-09-15T22:48:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.167+0000 7f0cd1a00640 1 --2- 172.21.15.111:0/3868029350 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0cb00a3370 0x7f0cb00a5830 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.167+0000 7f0cd1a00640 1 --2- 172.21.15.111:0/3868029350 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ccc105910 0x7f0ccc1a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.167+0000 7f0cd1a00640 1 --2- 172.21.15.111:0/3868029350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ccc104f50 0x7f0ccc19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.167+0000 7f0cd1a00640 1 --2- 172.21.15.111:0/3868029350 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0ccc103d60 0x7f0ccc199fc0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.167+0000 7f0cd1a00640 1 -- 172.21.15.111:0/3868029350 >> 172.21.15.111:0/3868029350 conn(0x7f0ccc0ff4e0 msgr2=0x7f0ccc108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.167+0000 7f0cd1a00640 1 -- 172.21.15.111:0/3868029350 shutdown_connections 2024-09-15T22:48:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:40.167+0000 7f0cd1a00640 1 -- 172.21.15.111:0/3868029350 wait complete. 2024-09-15T22:48:40.240 INFO:tasks.workunit.client.0.smithi111.stdout: 1 142463 11.56 MB/sec warmup 68 sec latency 1343.080 ms 2024-09-15T22:48:41.240 INFO:tasks.workunit.client.0.smithi111.stdout: 1 142621 11.39 MB/sec warmup 69 sec latency 2211.755 ms 2024-09-15T22:48:41.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:41 smithi111 ceph-mon[30629]: pgmap v433: 97 pgs: 97 active+clean; 80 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 202 op/s 2024-09-15T22:48:42.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:41 smithi192 ceph-mon[35427]: pgmap v433: 97 pgs: 97 active+clean; 80 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 202 op/s 2024-09-15T22:48:42.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:41 smithi121 ceph-mon[34815]: pgmap v433: 97 pgs: 97 active+clean; 80 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 202 op/s 2024-09-15T22:48:42.240 INFO:tasks.workunit.client.0.smithi111.stdout: 1 145726 11.49 MB/sec warmup 70 sec latency 13.626 ms 2024-09-15T22:48:43.240 INFO:tasks.workunit.client.0.smithi111.stdout: 1 148815 11.54 MB/sec warmup 71 sec latency 10.880 ms 2024-09-15T22:48:43.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:43 smithi111 ceph-mon[30629]: pgmap v434: 97 pgs: 97 active+clean; 79 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 188 op/s 2024-09-15T22:48:44.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:43 smithi192 ceph-mon[35427]: pgmap v434: 97 pgs: 97 active+clean; 79 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 188 op/s 2024-09-15T22:48:44.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:43 smithi121 ceph-mon[34815]: pgmap v434: 97 pgs: 97 active+clean; 79 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 188 op/s 2024-09-15T22:48:44.240 INFO:tasks.workunit.client.0.smithi111.stdout: 1 150258 11.47 MB/sec warmup 72 sec latency 222.489 ms 2024-09-15T22:48:45.240 INFO:tasks.workunit.client.0.smithi111.stdout: 1 150258 11.32 MB/sec warmup 73 sec latency 1018.302 ms 2024-09-15T22:48:45.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:45 smithi111 ceph-mon[30629]: pgmap v435: 97 pgs: 97 active+clean; 79 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 192 op/s 2024-09-15T22:48:46.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:45 smithi192 ceph-mon[35427]: pgmap v435: 97 pgs: 97 active+clean; 79 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 192 op/s 2024-09-15T22:48:46.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:45 smithi121 ceph-mon[34815]: pgmap v435: 97 pgs: 97 active+clean; 79 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 192 op/s 2024-09-15T22:48:46.240 INFO:tasks.workunit.client.0.smithi111.stdout: 1 151992 11.31 MB/sec warmup 74 sec latency 1567.261 ms 2024-09-15T22:48:47.240 INFO:tasks.workunit.client.0.smithi111.stdout: 1 154069 11.31 MB/sec warmup 75 sec latency 40.533 ms 2024-09-15T22:48:47.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:47 smithi111 ceph-mon[30629]: pgmap v436: 97 pgs: 97 active+clean; 79 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 193 op/s 2024-09-15T22:48:48.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:47 smithi192 ceph-mon[35427]: pgmap v436: 97 pgs: 97 active+clean; 79 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 193 op/s 2024-09-15T22:48:48.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:47 smithi121 ceph-mon[34815]: pgmap v436: 97 pgs: 97 active+clean; 79 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 193 op/s 2024-09-15T22:48:48.240 INFO:tasks.workunit.client.0.smithi111.stdout: 1 156809 11.35 MB/sec warmup 76 sec latency 10.682 ms 2024-09-15T22:48:49.240 INFO:tasks.workunit.client.0.smithi111.stdout: 1 159841 11.44 MB/sec warmup 77 sec latency 103.897 ms 2024-09-15T22:48:49.678 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:49 smithi111 ceph-mon[30629]: pgmap v437: 97 pgs: 97 active+clean; 80 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 183 op/s 2024-09-15T22:48:50.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:49 smithi192 ceph-mon[35427]: pgmap v437: 97 pgs: 97 active+clean; 80 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 183 op/s 2024-09-15T22:48:50.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:49 smithi121 ceph-mon[34815]: pgmap v437: 97 pgs: 97 active+clean; 80 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 183 op/s 2024-09-15T22:48:50.240 INFO:tasks.workunit.client.0.smithi111.stdout: 1 159841 11.30 MB/sec warmup 78 sec latency 1103.940 ms 2024-09-15T22:48:51.240 INFO:tasks.workunit.client.0.smithi111.stdout: 1 160608 11.17 MB/sec warmup 79 sec latency 1548.089 ms 2024-09-15T22:48:51.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:51 smithi111 ceph-mon[30629]: pgmap v438: 97 pgs: 97 active+clean; 80 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 189 op/s 2024-09-15T22:48:52.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:51 smithi192 ceph-mon[35427]: pgmap v438: 97 pgs: 97 active+clean; 80 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 189 op/s 2024-09-15T22:48:52.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:51 smithi121 ceph-mon[34815]: pgmap v438: 97 pgs: 97 active+clean; 80 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 189 op/s 2024-09-15T22:48:52.241 INFO:tasks.workunit.client.0.smithi111.stdout: 1 163606 11.26 MB/sec warmup 80 sec latency 7.197 ms 2024-09-15T22:48:53.240 INFO:tasks.workunit.client.0.smithi111.stdout: 1 166799 11.34 MB/sec warmup 81 sec latency 13.540 ms 2024-09-15T22:48:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.289+0000 7f2a14e00640 1 -- 172.21.15.111:0/1547881389 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a10103b50 msgr2=0x7f2a10103f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.289+0000 7f2a14e00640 1 --2- 172.21.15.111:0/1547881389 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a10103b50 0x7f2a10103f50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2a00009900 tx=0x7f2a0002a410 comp rx=0 tx=0).stop 2024-09-15T22:48:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.289+0000 7f2a14e00640 1 -- 172.21.15.111:0/1547881389 shutdown_connections 2024-09-15T22:48:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.289+0000 7f2a14e00640 1 --2- 172.21.15.111:0/1547881389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a10105700 0x7f2a1010bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.289+0000 7f2a14e00640 1 --2- 172.21.15.111:0/1547881389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a10104d40 0x7f2a101051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.289+0000 7f2a14e00640 1 --2- 172.21.15.111:0/1547881389 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a10103b50 0x7f2a10103f50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.289+0000 7f2a14e00640 1 -- 172.21.15.111:0/1547881389 >> 172.21.15.111:0/1547881389 conn(0x7f2a100ff2b0 msgr2=0x7f2a101016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:53.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.289+0000 7f2a14e00640 1 -- 172.21.15.111:0/1547881389 shutdown_connections 2024-09-15T22:48:53.292 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.289+0000 7f2a14e00640 1 -- 172.21.15.111:0/1547881389 wait complete. 2024-09-15T22:48:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.290+0000 7f2a14e00640 1 Processor -- start 2024-09-15T22:48:53.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.290+0000 7f2a14e00640 1 -- start start 2024-09-15T22:48:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.290+0000 7f2a14e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a10103b50 0x7f2a10199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.290+0000 7f2a14e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a10104d40 0x7f2a1019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.290+0000 7f2a14e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a10105700 0x7f2a101a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.290+0000 7f2a14e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1010f800 con 0x7f2a10104d40 2024-09-15T22:48:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.290+0000 7f2a14e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1010f680 con 0x7f2a10105700 2024-09-15T22:48:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.290+0000 7f2a14e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1010f980 con 0x7f2a10103b50 2024-09-15T22:48:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.291+0000 7f2a0fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a10105700 0x7f2a101a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.291+0000 7f2a0ea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a10104d40 0x7f2a1019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.291+0000 7f2a0f400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a10103b50 0x7f2a10199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.291+0000 7f2a0fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a10105700 0x7f2a101a1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:57586/0 (socket says 172.21.15.111:57586) 2024-09-15T22:48:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.291+0000 7f2a0fe00640 1 -- 172.21.15.111:0/833705980 learned_addr learned my addr 172.21.15.111:0/833705980 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:48:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.291+0000 7f2a0fe00640 1 -- 172.21.15.111:0/833705980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a10103b50 msgr2=0x7f2a10199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.291+0000 7f2a0fe00640 1 --2- 172.21.15.111:0/833705980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a10103b50 0x7f2a10199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.291+0000 7f2a0fe00640 1 -- 172.21.15.111:0/833705980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a10104d40 msgr2=0x7f2a1019a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.291+0000 7f2a0fe00640 1 --2- 172.21.15.111:0/833705980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a10104d40 0x7f2a1019a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.291+0000 7f2a0fe00640 1 -- 172.21.15.111:0/833705980 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a101a1ad0 con 0x7f2a10105700 2024-09-15T22:48:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.291+0000 7f2a0fe00640 1 --2- 172.21.15.111:0/833705980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a10105700 0x7f2a101a1460 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800eb10 tx=0x7f2a0800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.291+0000 7f2a07e00640 1 -- 172.21.15.111:0/833705980 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08004b20 con 0x7f2a10105700 2024-09-15T22:48:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.291+0000 7f2a07e00640 1 -- 172.21.15.111:0/833705980 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a08004cc0 con 0x7f2a10105700 2024-09-15T22:48:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.291+0000 7f2a14e00640 1 -- 172.21.15.111:0/833705980 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a101a1dc0 con 0x7f2a10105700 2024-09-15T22:48:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.291+0000 7f2a07e00640 1 -- 172.21.15.111:0/833705980 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08010760 con 0x7f2a10105700 2024-09-15T22:48:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.292+0000 7f2a14e00640 1 -- 172.21.15.111:0/833705980 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a101a2230 con 0x7f2a10105700 2024-09-15T22:48:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.294+0000 7f2a193f0740 1 Processor -- start 2024-09-15T22:48:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.295+0000 7f2a07e00640 1 -- 172.21.15.111:0/833705980 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f2a08005020 con 0x7f2a10105700 2024-09-15T22:48:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.295+0000 7f2a193f0740 1 -- start start 2024-09-15T22:48:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.295+0000 7f2a193f0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a38bc6e260 0x55a38bc6e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:53.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.295+0000 7f2a193f0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc6eba0 0x55a38bc73660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.295+0000 7f2a193f0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a38bc73c30 0x55a38bc760f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.295+0000 7f2a193f0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a38bbd3100 con 0x55a38bc6e260 2024-09-15T22:48:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.295+0000 7f2a193f0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a38bb7b1b0 con 0x55a38bc6eba0 2024-09-15T22:48:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.295+0000 7f2a193f0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a38bbcf0c0 con 0x55a38bc73c30 2024-09-15T22:48:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.295+0000 7f2a0ea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a38bc6e260 0x55a38bc6e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.295+0000 7f2a0f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc6eba0 0x55a38bc73660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.295+0000 7f2a0ea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a38bc6e260 0x55a38bc6e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42412/0 (socket says 172.21.15.111:42412) 2024-09-15T22:48:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.295+0000 7f2a0f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc6eba0 0x55a38bc73660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:57590/0 (socket says 172.21.15.111:57590) 2024-09-15T22:48:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.295+0000 7f2a0ea00640 1 -- 172.21.15.111:0/4054816394 learned_addr learned my addr 172.21.15.111:0/4054816394 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:48:53.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.296+0000 7f2a0ea00640 1 --2- 172.21.15.111:0/4054816394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a38bc73c30 0x55a38bc760f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.296+0000 7f2a0f400640 1 -- 172.21.15.111:0/4054816394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a38bc73c30 msgr2=0x55a38bc760f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.296+0000 7f2a0f400640 1 --2- 172.21.15.111:0/4054816394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a38bc73c30 0x55a38bc760f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.296+0000 7f2a0f400640 1 -- 172.21.15.111:0/4054816394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a38bc6e260 msgr2=0x55a38bc6e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.296+0000 7f2a0f400640 1 --2- 172.21.15.111:0/4054816394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a38bc6e260 0x55a38bc6e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.296+0000 7f2a0f400640 1 -- 172.21.15.111:0/4054816394 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a38b8e6210 con 0x55a38bc6eba0 2024-09-15T22:48:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.296+0000 7f2a0ea00640 1 --2- 172.21.15.111:0/4054816394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a38bc73c30 0x55a38bc760f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:48:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.296+0000 7f2a07e00640 1 --2- 172.21.15.111:0/833705980 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f29f40a3240 0x7f29f40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.296+0000 7f2a07e00640 1 -- 172.21.15.111:0/833705980 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f2a080cfec0 con 0x7f2a10105700 2024-09-15T22:48:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.296+0000 7f2a0f400640 1 --2- 172.21.15.111:0/4054816394 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc6eba0 0x55a38bc73660 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2a00009230 tx=0x7f2a00004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45cd33902eb4e5d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.296+0000 7f2a0ea00640 1 --2- 172.21.15.111:0/833705980 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f29f40a3240 0x7f29f40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:53.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.296+0000 7f2a04c00640 1 -- 172.21.15.111:0/4054816394 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a00035040 con 0x55a38bc6eba0 2024-09-15T22:48:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.296+0000 7f2a04c00640 1 -- 172.21.15.111:0/4054816394 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a00005520 con 0x55a38bc6eba0 2024-09-15T22:48:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.296+0000 7f2a04c00640 1 -- 172.21.15.111:0/4054816394 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a000316b0 con 0x55a38bc6eba0 2024-09-15T22:48:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.297+0000 7f2a193f0740 1 -- 172.21.15.111:0/4054816394 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc6eba0 msgr2=0x55a38bc73660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.297+0000 7f2a0ea00640 1 --2- 172.21.15.111:0/833705980 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f29f40a3240 0x7f29f40a5700 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009fd0 tx=0x7f29fc0097a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.297+0000 7f2a193f0740 1 --2- 172.21.15.111:0/4054816394 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc6eba0 0x55a38bc73660 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2a00009230 tx=0x7f2a00004040 comp rx=0 tx=0).stop 2024-09-15T22:48:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.297+0000 7f2a193f0740 1 -- 172.21.15.111:0/4054816394 shutdown_connections 2024-09-15T22:48:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.297+0000 7f2a193f0740 1 --2- 172.21.15.111:0/4054816394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a38bc73c30 0x55a38bc760f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.297+0000 7f2a193f0740 1 --2- 172.21.15.111:0/4054816394 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc6eba0 0x55a38bc73660 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.297+0000 7f2a193f0740 1 --2- 172.21.15.111:0/4054816394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a38bc6e260 0x55a38bc6e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.297+0000 7f2a193f0740 1 -- 172.21.15.111:0/4054816394 >> 172.21.15.111:0/4054816394 conn(0x55a38bc698d0 msgr2=0x55a38bc69cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.297+0000 7f2a193f0740 1 -- 172.21.15.111:0/4054816394 shutdown_connections 2024-09-15T22:48:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.297+0000 7f2a193f0740 1 -- 172.21.15.111:0/4054816394 wait complete. 2024-09-15T22:48:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.298+0000 7f2a193f0740 1 -- start start 2024-09-15T22:48:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.298+0000 7f2a193f0740 1 Processor -- start 2024-09-15T22:48:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.298+0000 7f2a193f0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a38bc6e260 0x55a38bc783f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.298+0000 7f2a193f0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc6eba0 0x55a38bc78930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.298+0000 7f2a193f0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a38bc73c30 0x55a38bc7f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.298+0000 7f2a193f0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a38bc76ba0 con 0x55a38bc6e260 2024-09-15T22:48:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.298+0000 7f2a193f0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a38bb5c090 con 0x55a38bc6eba0 2024-09-15T22:48:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.298+0000 7f2a193f0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a38bc76d20 con 0x55a38bc73c30 2024-09-15T22:48:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.298+0000 7f2a193f0740 10 client.0 _ll_register_callbacks cb 0x55a38bbb0f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:48:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.299+0000 7f2a0ea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc6eba0 0x55a38bc78930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.299+0000 7f2a0ea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc6eba0 0x55a38bc78930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:57606/0 (socket says 172.21.15.111:57606) 2024-09-15T22:48:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.299+0000 7f2a0ea00640 1 -- 172.21.15.111:0/1818171719 learned_addr learned my addr 172.21.15.111:0/1818171719 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:48:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.299+0000 7f2a0fe00640 1 --2- 172.21.15.111:0/1818171719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a38bc73c30 0x55a38bc7f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.299+0000 7f2a0f400640 1 --2- 172.21.15.111:0/1818171719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a38bc6e260 0x55a38bc783f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.299+0000 7f2a0ea00640 1 -- 172.21.15.111:0/1818171719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a38bc73c30 msgr2=0x55a38bc7f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.299+0000 7f2a0ea00640 1 --2- 172.21.15.111:0/1818171719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a38bc73c30 0x55a38bc7f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.299+0000 7f2a0ea00640 1 -- 172.21.15.111:0/1818171719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a38bc6e260 msgr2=0x55a38bc783f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.299+0000 7f2a0ea00640 1 --2- 172.21.15.111:0/1818171719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a38bc6e260 0x55a38bc783f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.299+0000 7f2a0ea00640 1 -- 172.21.15.111:0/1818171719 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a38b9b2620 con 0x55a38bc6eba0 2024-09-15T22:48:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.299+0000 7f2a0ea00640 1 --2- 172.21.15.111:0/1818171719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc6eba0 0x55a38bc78930 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f29fc013fd0 tx=0x7f29fc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82d44853f261f460 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.299+0000 7f29f9600640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:48:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.299+0000 7f29f9600640 1 -- 172.21.15.111:0/1818171719 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc026070 con 0x55a38bc6eba0 2024-09-15T22:48:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.300+0000 7f2a193f0740 1 -- 172.21.15.111:0/1818171719 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a00005520 con 0x55a38bc6eba0 2024-09-15T22:48:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.300+0000 7f29f9600640 1 -- 172.21.15.111:0/1818171719 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f29fc003cf0 con 0x55a38bc6eba0 2024-09-15T22:48:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.300+0000 7f29f9600640 1 -- 172.21.15.111:0/1818171719 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f29fc02b460 con 0x55a38bc6eba0 2024-09-15T22:48:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.300+0000 7f2a193f0740 10 client.25247 fetch_fsmap learned FSMap version 28 2024-09-15T22:48:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.300+0000 7f2a193f0740 1 -- 172.21.15.111:0/1818171719 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a38bc82880 con 0x55a38bc6eba0 2024-09-15T22:48:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.300+0000 7f29f9600640 1 -- 172.21.15.111:0/1818171719 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a00005520 con 0x55a38bc6eba0 2024-09-15T22:48:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.300+0000 7f29f9600640 1 -- 172.21.15.111:0/1818171719 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f29fc022560 con 0x55a38bc6eba0 2024-09-15T22:48:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.300+0000 7f2a193f0740 10 client.25247 fetch_fsmap finished waiting for FSMap version 28 2024-09-15T22:48:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.300+0000 7f2a193f0740 10 client.25247 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:48:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.300+0000 7f2a193f0740 20 client.25247 populate_metadata read hostname 'smithi111' 2024-09-15T22:48:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.300+0000 7f2a193f0740 1 --2- 172.21.15.111:0/1818171719 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a38bc83f00 0x55a38bc86370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.300+0000 7f2a193f0740 4 client.25247 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:48:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.300+0000 7f2a193f0740 1 -- 172.21.15.111:0/1818171719 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a38bbe0830 con 0x55a38bc83f00 2024-09-15T22:48:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.301+0000 7f2a0f400640 1 --2- 172.21.15.111:0/1818171719 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a38bc83f00 0x55a38bc86370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.301+0000 7f2a0f400640 1 --2- 172.21.15.111:0/1818171719 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a38bc83f00 0x55a38bc86370 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c4d021996585563 server_cookie=36cf96fa92448f6e in_seq=0 out_seq=0 2024-09-15T22:48:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.301+0000 7f29f9600640 10 client.25247 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:48:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.302+0000 7f29f9600640 1 -- 172.21.15.111:0/1818171719 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a38bbe0830 con 0x55a38bc83f00 2024-09-15T22:48:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.302+0000 7f29f9600640 10 client.25247 handle_command_reply: tid=0 2024-09-15T22:48:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.302+0000 7f2a193f0740 1 client.25247 shutdown 2024-09-15T22:48:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.302+0000 7f2a193f0740 1 -- 172.21.15.111:0/1818171719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc6eba0 msgr2=0x55a38bc78930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.302+0000 7f2a193f0740 1 --2- 172.21.15.111:0/1818171719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc6eba0 0x55a38bc78930 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f29fc013fd0 tx=0x7f29fc005f90 comp rx=0 tx=0).stop 2024-09-15T22:48:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.302+0000 7f2a193f0740 1 -- 172.21.15.111:0/1818171719 shutdown_connections 2024-09-15T22:48:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.302+0000 7f2a193f0740 1 --2- 172.21.15.111:0/1818171719 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a38bc83f00 0x55a38bc86370 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.302+0000 7f2a193f0740 1 --2- 172.21.15.111:0/1818171719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a38bc73c30 0x55a38bc7f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.302+0000 7f2a193f0740 1 --2- 172.21.15.111:0/1818171719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc6eba0 0x55a38bc78930 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.302+0000 7f2a193f0740 1 --2- 172.21.15.111:0/1818171719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a38bc6e260 0x55a38bc783f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.302+0000 7f29f9600640 0 client.25247 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:48:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.302+0000 7f2a193f0740 1 -- 172.21.15.111:0/1818171719 >> 172.21.15.111:0/1818171719 conn(0x55a38bc698d0 msgr2=0x55a38bc6b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.302+0000 7f2a193f0740 1 -- 172.21.15.111:0/1818171719 shutdown_connections 2024-09-15T22:48:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.302+0000 7f2a193f0740 1 -- 172.21.15.111:0/1818171719 wait complete. 2024-09-15T22:48:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.302+0000 7f2a193f0740 20 client.25247 trim_cache size 0 max 16384 2024-09-15T22:48:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.318+0000 7f2a193f0740 1 Processor -- start 2024-09-15T22:48:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.318+0000 7f2a193f0740 1 -- start start 2024-09-15T22:48:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.319+0000 7f2a193f0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a38bc6e260 0x55a38bc6a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.319+0000 7f2a193f0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a38bc6eba0 0x55a38bc7f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.319+0000 7f2a193f0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc73c30 0x55a38bc6acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:53.320 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.319+0000 7f2a193f0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2a101ad7c0 con 0x55a38bc6e260 2024-09-15T22:48:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.319+0000 7f2a193f0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2a101ab090 con 0x55a38bc73c30 2024-09-15T22:48:53.321 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.319+0000 7f2a193f0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2a101a8ee0 con 0x55a38bc6eba0 2024-09-15T22:48:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.319+0000 7f2a0fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc73c30 0x55a38bc6acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.319+0000 7f2a0ea00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a38bc6eba0 0x55a38bc7f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.319+0000 7f2a0fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc73c30 0x55a38bc6acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:57616/0 (socket says 172.21.15.111:57616) 2024-09-15T22:48:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.319+0000 7f2a0ea00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a38bc6eba0 0x55a38bc7f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:55196/0 (socket says 172.21.15.111:55196) 2024-09-15T22:48:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.319+0000 7f2a0fe00640 1 -- 172.21.15.111:0/492276005 learned_addr learned my addr 172.21.15.111:0/492276005 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:48:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.319+0000 7f2a0f400640 1 --2- 172.21.15.111:0/492276005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a38bc6e260 0x55a38bc6a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:53.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.319+0000 7f2a0fe00640 1 -- 172.21.15.111:0/492276005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a38bc6eba0 msgr2=0x55a38bc7f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.319+0000 7f2a0fe00640 1 --2- 172.21.15.111:0/492276005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a38bc6eba0 0x55a38bc7f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.319+0000 7f2a0fe00640 1 -- 172.21.15.111:0/492276005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a38bc6e260 msgr2=0x55a38bc6a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.319+0000 7f2a0fe00640 1 --2- 172.21.15.111:0/492276005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a38bc6e260 0x55a38bc6a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.319+0000 7f2a0fe00640 1 -- 172.21.15.111:0/492276005 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a38bc696f0 con 0x55a38bc73c30 2024-09-15T22:48:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.319+0000 7f2a0ea00640 1 --2- 172.21.15.111:0/492276005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a38bc6eba0 0x55a38bc7f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:48:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.319+0000 7f2a0f400640 1 --2- 172.21.15.111:0/492276005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a38bc6e260 0x55a38bc6a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:48:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.320+0000 7f2a0fe00640 1 --2- 172.21.15.111:0/492276005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc73c30 0x55a38bc6acc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2a0809c1e0 tx=0x7f2a080058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8fab8b0c4ffc7b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.320+0000 7f29fbe00640 1 -- 172.21.15.111:0/492276005 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a080b4070 con 0x55a38bc73c30 2024-09-15T22:48:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.320+0000 7f29fbe00640 1 -- 172.21.15.111:0/492276005 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a0809cbe0 con 0x55a38bc73c30 2024-09-15T22:48:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.320+0000 7f29fbe00640 1 -- 172.21.15.111:0/492276005 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a080b5260 con 0x55a38bc73c30 2024-09-15T22:48:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.320+0000 7f2a193f0740 1 -- 172.21.15.111:0/492276005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc73c30 msgr2=0x55a38bc6acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.321+0000 7f2a193f0740 1 --2- 172.21.15.111:0/492276005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc73c30 0x55a38bc6acc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2a0809c1e0 tx=0x7f2a080058d0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.321+0000 7f2a193f0740 1 -- 172.21.15.111:0/492276005 shutdown_connections 2024-09-15T22:48:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.321+0000 7f2a193f0740 1 --2- 172.21.15.111:0/492276005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc73c30 0x55a38bc6acc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.321+0000 7f2a193f0740 1 --2- 172.21.15.111:0/492276005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a38bc6eba0 0x55a38bc7f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.321+0000 7f2a193f0740 1 --2- 172.21.15.111:0/492276005 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a38bc6e260 0x55a38bc6a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.321+0000 7f2a193f0740 1 -- 172.21.15.111:0/492276005 >> 172.21.15.111:0/492276005 conn(0x55a38bc698d0 msgr2=0x55a38bc646a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.321+0000 7f2a193f0740 1 -- 172.21.15.111:0/492276005 shutdown_connections 2024-09-15T22:48:53.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.321+0000 7f2a193f0740 1 -- 172.21.15.111:0/492276005 wait complete. 2024-09-15T22:48:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.321+0000 7f2a193f0740 1 -- start start 2024-09-15T22:48:53.326 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.321+0000 7f2a193f0740 1 Processor -- start 2024-09-15T22:48:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.322+0000 7f2a193f0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc6e260 0x55a38bc7d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.322+0000 7f2a193f0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a38bc6eba0 0x55a38bc7db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.322+0000 7f2a193f0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a38bc73c30 0x55a38bc7eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.322+0000 7f2a193f0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2a101a9550 con 0x55a38bc6eba0 2024-09-15T22:48:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.322+0000 7f2a193f0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2a101aa9c0 con 0x55a38bc6e260 2024-09-15T22:48:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.322+0000 7f2a193f0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2a101a9c20 con 0x55a38bc73c30 2024-09-15T22:48:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.322+0000 7f2a193f0740 10 client.0 _ll_register_callbacks cb 0x7f29fc009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:48:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.322+0000 7f2a0f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc6e260 0x55a38bc7d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.322+0000 7f2a0f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc6e260 0x55a38bc7d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:57622/0 (socket says 172.21.15.111:57622) 2024-09-15T22:48:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.322+0000 7f2a0f400640 1 -- 172.21.15.111:0/3799774495 learned_addr learned my addr 172.21.15.111:0/3799774495 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:48:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.322+0000 7f2a0ea00640 1 --2- 172.21.15.111:0/3799774495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a38bc6eba0 0x55a38bc7db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:53.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.322+0000 7f2a0fe00640 1 --2- 172.21.15.111:0/3799774495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a38bc73c30 0x55a38bc7eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.322+0000 7f2a0f400640 1 -- 172.21.15.111:0/3799774495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a38bc73c30 msgr2=0x55a38bc7eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.322+0000 7f2a0f400640 1 --2- 172.21.15.111:0/3799774495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a38bc73c30 0x55a38bc7eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.322+0000 7f2a0f400640 1 -- 172.21.15.111:0/3799774495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a38bc6eba0 msgr2=0x55a38bc7db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:48:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.322+0000 7f2a0f400640 1 --2- 172.21.15.111:0/3799774495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a38bc6eba0 0x55a38bc7db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.322+0000 7f2a0f400640 1 -- 172.21.15.111:0/3799774495 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a38bc7f100 con 0x55a38bc6e260 2024-09-15T22:48:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.322+0000 7f2a0f400640 1 --2- 172.21.15.111:0/3799774495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc6e260 0x55a38bc7d610 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2a0002aa00 tx=0x7f2a00005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80fa0e26bd881a78 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:48:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.322+0000 7f29f9600640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:48:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.323+0000 7f29f9600640 1 -- 172.21.15.111:0/3799774495 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a00035040 con 0x55a38bc6e260 2024-09-15T22:48:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.323+0000 7f29f9600640 1 -- 172.21.15.111:0/3799774495 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f2a00004970 con 0x55a38bc6e260 2024-09-15T22:48:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.323+0000 7f2a193f0740 1 -- 172.21.15.111:0/3799774495 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a0809cbe0 con 0x55a38bc6e260 2024-09-15T22:48:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.323+0000 7f29f9600640 1 -- 172.21.15.111:0/3799774495 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2a00004df0 con 0x55a38bc6e260 2024-09-15T22:48:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.323+0000 7f2a193f0740 10 client.25253 fetch_fsmap learned FSMap version 28 2024-09-15T22:48:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.323+0000 7f2a193f0740 1 -- 172.21.15.111:0/3799774495 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a38bcef230 con 0x55a38bc6e260 2024-09-15T22:48:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.323+0000 7f29f9600640 1 -- 172.21.15.111:0/3799774495 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a0809cbe0 con 0x55a38bc6e260 2024-09-15T22:48:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.323+0000 7f29f9600640 1 -- 172.21.15.111:0/3799774495 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2a0004f070 con 0x55a38bc6e260 2024-09-15T22:48:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.323+0000 7f2a193f0740 10 client.25253 fetch_fsmap finished waiting for FSMap version 28 2024-09-15T22:48:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.323+0000 7f2a193f0740 10 client.25253 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:48:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.323+0000 7f2a193f0740 20 client.25253 populate_metadata read hostname 'smithi111' 2024-09-15T22:48:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.323+0000 7f2a193f0740 1 --2- 172.21.15.111:0/3799774495 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a38bcef7b0 0x55a38bcf1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:48:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.323+0000 7f2a193f0740 4 client.25253 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:48:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.323+0000 7f2a193f0740 1 -- 172.21.15.111:0/3799774495 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a38bbdab80 con 0x55a38bcef7b0 2024-09-15T22:48:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.324+0000 7f2a0ea00640 1 --2- 172.21.15.111:0/3799774495 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a38bcef7b0 0x55a38bcf1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:48:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.324+0000 7f2a0ea00640 1 --2- 172.21.15.111:0/3799774495 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a38bcef7b0 0x55a38bcf1bd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9a4f7c7347dd3d9 server_cookie=569acd373fc417fa in_seq=0 out_seq=0 2024-09-15T22:48:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.324+0000 7f29f9600640 10 client.25253 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:48:53.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.331+0000 7f29f9600640 1 -- 172.21.15.111:0/3799774495 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a38bbdab80 con 0x55a38bcef7b0 2024-09-15T22:48:53.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.331+0000 7f29f9600640 10 client.25253 handle_command_reply: tid=0 2024-09-15T22:48:53.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.331+0000 7f2a193f0740 1 client.25253 shutdown 2024-09-15T22:48:53.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.331+0000 7f2a193f0740 1 -- 172.21.15.111:0/3799774495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc6e260 msgr2=0x55a38bc7d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.331+0000 7f2a193f0740 1 --2- 172.21.15.111:0/3799774495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc6e260 0x55a38bc7d610 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2a0002aa00 tx=0x7f2a00005d50 comp rx=0 tx=0).stop 2024-09-15T22:48:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.331+0000 7f2a193f0740 1 -- 172.21.15.111:0/3799774495 shutdown_connections 2024-09-15T22:48:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.331+0000 7f2a193f0740 1 --2- 172.21.15.111:0/3799774495 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a38bcef7b0 0x55a38bcf1bd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.331+0000 7f2a193f0740 1 --2- 172.21.15.111:0/3799774495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a38bc73c30 0x55a38bc7eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.331+0000 7f2a193f0740 1 --2- 172.21.15.111:0/3799774495 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a38bc6eba0 0x55a38bc7db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.331+0000 7f29f9600640 0 client.25253 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:48:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.331+0000 7f2a193f0740 1 --2- 172.21.15.111:0/3799774495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a38bc6e260 0x55a38bc7d610 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.331+0000 7f2a193f0740 1 -- 172.21.15.111:0/3799774495 >> 172.21.15.111:0/3799774495 conn(0x55a38bc698d0 msgr2=0x55a38bc73030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.332+0000 7f2a193f0740 1 -- 172.21.15.111:0/3799774495 shutdown_connections 2024-09-15T22:48:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.332+0000 7f2a193f0740 1 -- 172.21.15.111:0/3799774495 wait complete. 2024-09-15T22:48:53.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.332+0000 7f2a193f0740 20 client.25253 trim_cache size 0 max 16384 2024-09-15T22:48:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:48:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:48:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:48:53.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:48:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.332+0000 7f2a14e00640 1 -- 172.21.15.111:0/833705980 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f29f40a3240 msgr2=0x7f29f40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.332+0000 7f2a14e00640 1 --2- 172.21.15.111:0/833705980 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f29f40a3240 0x7f29f40a5700 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009fd0 tx=0x7f29fc0097a0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.332+0000 7f2a14e00640 1 -- 172.21.15.111:0/833705980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a10105700 msgr2=0x7f2a101a1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:48:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.332+0000 7f2a14e00640 1 --2- 172.21.15.111:0/833705980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a10105700 0x7f2a101a1460 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800eb10 tx=0x7f2a0800c2d0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.332+0000 7f2a14e00640 1 -- 172.21.15.111:0/833705980 shutdown_connections 2024-09-15T22:48:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.332+0000 7f2a14e00640 1 --2- 172.21.15.111:0/833705980 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f29f40a3240 0x7f29f40a5700 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.332+0000 7f2a14e00640 1 --2- 172.21.15.111:0/833705980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a10105700 0x7f2a101a1460 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.332+0000 7f2a14e00640 1 --2- 172.21.15.111:0/833705980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a10104d40 0x7f2a1019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.332+0000 7f2a14e00640 1 --2- 172.21.15.111:0/833705980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a10103b50 0x7f2a10199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:48:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.332+0000 7f2a14e00640 1 -- 172.21.15.111:0/833705980 >> 172.21.15.111:0/833705980 conn(0x7f2a100ff2b0 msgr2=0x7f2a10108110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:48:53.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.332+0000 7f2a14e00640 1 -- 172.21.15.111:0/833705980 shutdown_connections 2024-09-15T22:48:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:48:53.333+0000 7f2a14e00640 1 -- 172.21.15.111:0/833705980 wait complete. 2024-09-15T22:48:53.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:53 smithi111 ceph-mon[30629]: pgmap v439: 97 pgs: 97 active+clean; 80 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 195 op/s 2024-09-15T22:48:54.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:53 smithi192 ceph-mon[35427]: pgmap v439: 97 pgs: 97 active+clean; 80 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 195 op/s 2024-09-15T22:48:54.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:53 smithi121 ceph-mon[34815]: pgmap v439: 97 pgs: 97 active+clean; 80 MiB data, 738 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 195 op/s 2024-09-15T22:48:54.241 INFO:tasks.workunit.client.0.smithi111.stdout: 1 169174 11.30 MB/sec warmup 82 sec latency 12.471 ms 2024-09-15T22:48:55.241 INFO:tasks.workunit.client.0.smithi111.stdout: 1 169657 11.23 MB/sec warmup 83 sec latency 770.360 ms 2024-09-15T22:48:55.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:55 smithi111 ceph-mon[30629]: pgmap v440: 97 pgs: 97 active+clean; 83 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 221 op/s 2024-09-15T22:48:56.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:55 smithi192 ceph-mon[35427]: pgmap v440: 97 pgs: 97 active+clean; 83 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 221 op/s 2024-09-15T22:48:56.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:55 smithi121 ceph-mon[34815]: pgmap v440: 97 pgs: 97 active+clean; 83 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 221 op/s 2024-09-15T22:48:56.241 INFO:tasks.workunit.client.0.smithi111.stdout: 1 172134 11.24 MB/sec warmup 84 sec latency 13.099 ms 2024-09-15T22:48:57.241 INFO:tasks.workunit.client.0.smithi111.stdout: 1 174690 11.28 MB/sec warmup 85 sec latency 13.174 ms 2024-09-15T22:48:57.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:57 smithi111 ceph-mon[30629]: pgmap v441: 97 pgs: 97 active+clean; 81 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 213 op/s 2024-09-15T22:48:58.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:57 smithi192 ceph-mon[35427]: pgmap v441: 97 pgs: 97 active+clean; 81 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 213 op/s 2024-09-15T22:48:58.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:57 smithi121 ceph-mon[34815]: pgmap v441: 97 pgs: 97 active+clean; 81 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 213 op/s 2024-09-15T22:48:58.241 INFO:tasks.workunit.client.0.smithi111.stdout: 1 177691 11.36 MB/sec warmup 86 sec latency 36.981 ms 2024-09-15T22:48:59.241 INFO:tasks.workunit.client.0.smithi111.stdout: 1 180551 11.38 MB/sec warmup 87 sec latency 12.380 ms 2024-09-15T22:48:59.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:48:59 smithi111 ceph-mon[30629]: pgmap v442: 97 pgs: 97 active+clean; 81 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 177 op/s 2024-09-15T22:49:00.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:48:59 smithi192 ceph-mon[35427]: pgmap v442: 97 pgs: 97 active+clean; 81 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 177 op/s 2024-09-15T22:49:00.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:48:59 smithi121 ceph-mon[34815]: pgmap v442: 97 pgs: 97 active+clean; 81 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 177 op/s 2024-09-15T22:49:00.241 INFO:tasks.workunit.client.0.smithi111.stdout: 1 180916 11.30 MB/sec warmup 88 sec latency 538.826 ms 2024-09-15T22:49:01.241 INFO:tasks.workunit.client.0.smithi111.stdout: 1 181981 11.21 MB/sec warmup 89 sec latency 830.763 ms 2024-09-15T22:49:02.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:01 smithi192 ceph-mon[35427]: pgmap v443: 97 pgs: 97 active+clean; 85 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 226 op/s 2024-09-15T22:49:02.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:01 smithi121 ceph-mon[34815]: pgmap v443: 97 pgs: 97 active+clean; 85 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 226 op/s 2024-09-15T22:49:02.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:01 smithi111 ceph-mon[30629]: pgmap v443: 97 pgs: 97 active+clean; 85 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 226 op/s 2024-09-15T22:49:02.241 INFO:tasks.workunit.client.0.smithi111.stdout: 1 184996 11.28 MB/sec warmup 90 sec latency 7.345 ms 2024-09-15T22:49:03.241 INFO:tasks.workunit.client.0.smithi111.stdout: 1 188224 11.36 MB/sec warmup 91 sec latency 13.298 ms 2024-09-15T22:49:03.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:03 smithi111 ceph-mon[30629]: pgmap v444: 97 pgs: 97 active+clean; 87 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 224 op/s 2024-09-15T22:49:04.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:03 smithi192 ceph-mon[35427]: pgmap v444: 97 pgs: 97 active+clean; 87 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 224 op/s 2024-09-15T22:49:04.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:03 smithi121 ceph-mon[34815]: pgmap v444: 97 pgs: 97 active+clean; 87 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 224 op/s 2024-09-15T22:49:04.241 INFO:tasks.workunit.client.0.smithi111.stdout: 1 190976 11.38 MB/sec warmup 92 sec latency 13.084 ms 2024-09-15T22:49:05.241 INFO:tasks.workunit.client.0.smithi111.stdout: 1 191670 11.31 MB/sec warmup 93 sec latency 738.673 ms 2024-09-15T22:49:05.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:05 smithi121 ceph-mon[34815]: pgmap v445: 97 pgs: 97 active+clean; 89 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 195 op/s 2024-09-15T22:49:06.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:05 smithi192 ceph-mon[35427]: pgmap v445: 97 pgs: 97 active+clean; 89 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 195 op/s 2024-09-15T22:49:06.092 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:05 smithi111 ceph-mon[30629]: pgmap v445: 97 pgs: 97 active+clean; 89 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 195 op/s 2024-09-15T22:49:06.241 INFO:tasks.workunit.client.0.smithi111.stdout: 1 192626 11.23 MB/sec warmup 94 sec latency 1166.218 ms 2024-09-15T22:49:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.456+0000 7f0045a00640 1 -- 172.21.15.111:0/252721023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0040076230 msgr2=0x7f0040074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.456+0000 7f0045a00640 1 --2- 172.21.15.111:0/252721023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0040076230 0x7f0040074680 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0030009900 tx=0x7f003002a410 comp rx=0 tx=0).stop 2024-09-15T22:49:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.456+0000 7f0045a00640 1 -- 172.21.15.111:0/252721023 shutdown_connections 2024-09-15T22:49:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.456+0000 7f0045a00640 1 --2- 172.21.15.111:0/252721023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f004010e130 0x7f0040110520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.456+0000 7f0045a00640 1 --2- 172.21.15.111:0/252721023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0040074c50 0x7f00400750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.456+0000 7f0045a00640 1 --2- 172.21.15.111:0/252721023 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0040076230 0x7f0040074680 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.456+0000 7f0045a00640 1 -- 172.21.15.111:0/252721023 >> 172.21.15.111:0/252721023 conn(0x7f00400fd2e0 msgr2=0x7f00400ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.456+0000 7f0045a00640 1 -- 172.21.15.111:0/252721023 shutdown_connections 2024-09-15T22:49:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.456+0000 7f0045a00640 1 -- 172.21.15.111:0/252721023 wait complete. 2024-09-15T22:49:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.457+0000 7f0045a00640 1 Processor -- start 2024-09-15T22:49:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.457+0000 7f0045a00640 1 -- start start 2024-09-15T22:49:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.457+0000 7f0045a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0040074c50 0x7f004019e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.457+0000 7f0045a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0040076230 0x7f004019e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.457+0000 7f0045a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f004010e130 0x7f00401a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.457+0000 7f0045a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0040113dc0 con 0x7f0040074c50 2024-09-15T22:49:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.457+0000 7f0045a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0040113c40 con 0x7f004010e130 2024-09-15T22:49:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.457+0000 7f0045a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0040113f40 con 0x7f0040076230 2024-09-15T22:49:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.457+0000 7f003ea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0040074c50 0x7f004019e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.457+0000 7f003f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f004010e130 0x7f00401a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.457+0000 7f003ea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0040074c50 0x7f004019e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40806/0 (socket says 172.21.15.111:40806) 2024-09-15T22:49:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.457+0000 7f003ea00640 1 -- 172.21.15.111:0/3217280031 learned_addr learned my addr 172.21.15.111:0/3217280031 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:49:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.457+0000 7f0036000640 1 --2- 172.21.15.111:0/3217280031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0040076230 0x7f004019e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.458+0000 7f003f400640 1 -- 172.21.15.111:0/3217280031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0040076230 msgr2=0x7f004019e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.458+0000 7f003f400640 1 --2- 172.21.15.111:0/3217280031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0040076230 0x7f004019e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.458+0000 7f003f400640 1 -- 172.21.15.111:0/3217280031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0040074c50 msgr2=0x7f004019e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.458+0000 7f003f400640 1 --2- 172.21.15.111:0/3217280031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0040074c50 0x7f004019e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.458+0000 7f003f400640 1 -- 172.21.15.111:0/3217280031 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00401a5fa0 con 0x7f004010e130 2024-09-15T22:49:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.458+0000 7f003f400640 1 --2- 172.21.15.111:0/3217280031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f004010e130 0x7f00401a5930 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f003800e940 tx=0x7f003800ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.458+0000 7f0037e00640 1 -- 172.21.15.111:0/3217280031 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0038004d20 con 0x7f004010e130 2024-09-15T22:49:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.458+0000 7f0037e00640 1 -- 172.21.15.111:0/3217280031 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0038005020 con 0x7f004010e130 2024-09-15T22:49:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.458+0000 7f0045a00640 1 -- 172.21.15.111:0/3217280031 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00401a6290 con 0x7f004010e130 2024-09-15T22:49:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.458+0000 7f0037e00640 1 -- 172.21.15.111:0/3217280031 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00380116b0 con 0x7f004010e130 2024-09-15T22:49:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.458+0000 7f0045a00640 1 -- 172.21.15.111:0/3217280031 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00401a6790 con 0x7f004010e130 2024-09-15T22:49:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.461+0000 7f00487cb740 1 Processor -- start 2024-09-15T22:49:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.462+0000 7f0037e00640 1 -- 172.21.15.111:0/3217280031 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f00380054d0 con 0x7f004010e130 2024-09-15T22:49:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.462+0000 7f00487cb740 1 -- start start 2024-09-15T22:49:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.462+0000 7f00487cb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a8f7f7090 0x557a8f7f7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.462+0000 7f00487cb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557a8f7f79d0 0x557a8f7fc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.462+0000 7f00487cb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a8f7fca60 0x557a8f7fef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.462+0000 7f00487cb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557a8f75c100 con 0x557a8f7f7090 2024-09-15T22:49:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.462+0000 7f00487cb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x557a8f7041b0 con 0x557a8f7fca60 2024-09-15T22:49:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.462+0000 7f00487cb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557a8f7580c0 con 0x557a8f7f79d0 2024-09-15T22:49:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.463+0000 7f0036000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a8f7f7090 0x557a8f7f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.463+0000 7f0036000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a8f7f7090 0x557a8f7f7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40808/0 (socket says 172.21.15.111:40808) 2024-09-15T22:49:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.463+0000 7f0036000640 1 -- 172.21.15.111:0/3043211966 learned_addr learned my addr 172.21.15.111:0/3043211966 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:49:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.463+0000 7f003ea00640 1 --2- 172.21.15.111:0/3043211966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557a8f7f79d0 0x557a8f7fc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.465 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.463+0000 7f0036000640 1 --2- 172.21.15.111:0/3043211966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a8f7fca60 0x557a8f7fef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.466 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.463+0000 7f0036000640 1 -- 172.21.15.111:0/3043211966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557a8f7f79d0 msgr2=0x557a8f7fc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.463+0000 7f0036000640 1 --2- 172.21.15.111:0/3043211966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557a8f7f79d0 0x557a8f7fc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.463+0000 7f0036000640 1 -- 172.21.15.111:0/3043211966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a8f7fca60 msgr2=0x557a8f7fef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.463+0000 7f0036000640 1 --2- 172.21.15.111:0/3043211966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a8f7fca60 0x557a8f7fef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.463+0000 7f0036000640 1 -- 172.21.15.111:0/3043211966 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a8f46f210 con 0x557a8f7f7090 2024-09-15T22:49:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.463+0000 7f0037e00640 1 --2- 172.21.15.111:0/3217280031 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f00240a32f0 0x7f00240a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.464+0000 7f0036000640 1 --2- 172.21.15.111:0/3043211966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a8f7f7090 0x557a8f7f7490 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0028013750 tx=0x7f00280096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5295c50a03206e33 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.464+0000 7f0037e00640 1 -- 172.21.15.111:0/3217280031 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f00380cfa20 con 0x7f004010e130 2024-09-15T22:49:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.464+0000 7f003ea00640 1 --2- 172.21.15.111:0/3217280031 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f00240a32f0 0x7f00240a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.464+0000 7f002fe00640 1 -- 172.21.15.111:0/3043211966 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028013cd0 con 0x557a8f7f7090 2024-09-15T22:49:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.464+0000 7f002fe00640 1 -- 172.21.15.111:0/3043211966 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00280130d0 con 0x557a8f7f7090 2024-09-15T22:49:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.464+0000 7f002fe00640 1 -- 172.21.15.111:0/3043211966 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028011890 con 0x557a8f7f7090 2024-09-15T22:49:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.465+0000 7f00487cb740 1 -- 172.21.15.111:0/3043211966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a8f7f7090 msgr2=0x557a8f7f7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:06.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.465+0000 7f00487cb740 1 --2- 172.21.15.111:0/3043211966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a8f7f7090 0x557a8f7f7490 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0028013750 tx=0x7f00280096c0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.465+0000 7f003ea00640 1 --2- 172.21.15.111:0/3217280031 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f00240a32f0 0x7f00240a57b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0030031440 tx=0x7f00300315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.465+0000 7f00487cb740 1 -- 172.21.15.111:0/3043211966 shutdown_connections 2024-09-15T22:49:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.465+0000 7f00487cb740 1 --2- 172.21.15.111:0/3043211966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a8f7fca60 0x557a8f7fef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.465+0000 7f00487cb740 1 --2- 172.21.15.111:0/3043211966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557a8f7f79d0 0x557a8f7fc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.465+0000 7f00487cb740 1 --2- 172.21.15.111:0/3043211966 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a8f7f7090 0x557a8f7f7490 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.465+0000 7f00487cb740 1 -- 172.21.15.111:0/3043211966 >> 172.21.15.111:0/3043211966 conn(0x557a8f7f2700 msgr2=0x557a8f7f2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.465+0000 7f00487cb740 1 -- 172.21.15.111:0/3043211966 shutdown_connections 2024-09-15T22:49:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.465+0000 7f00487cb740 1 -- 172.21.15.111:0/3043211966 wait complete. 2024-09-15T22:49:06.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.465+0000 7f00487cb740 1 -- start start 2024-09-15T22:49:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.466+0000 7f00487cb740 1 Processor -- start 2024-09-15T22:49:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.466+0000 7f00487cb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a8f7f7090 0x557a8f8012d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.466+0000 7f00487cb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557a8f7f79d0 0x557a8f801810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.466+0000 7f00487cb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a8f7fca60 0x557a8f808850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.466+0000 7f00487cb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557a8f7ff9d0 con 0x557a8f7fca60 2024-09-15T22:49:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.466+0000 7f00487cb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x557a8f6e5090 con 0x557a8f7f7090 2024-09-15T22:49:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.466+0000 7f00487cb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557a8f7ffb50 con 0x557a8f7f79d0 2024-09-15T22:49:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.466+0000 7f00487cb740 10 client.0 _ll_register_callbacks cb 0x557a8f739f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:49:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.466+0000 7f003ea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a8f7f7090 0x557a8f8012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.466+0000 7f003ea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a8f7f7090 0x557a8f8012d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:33328/0 (socket says 172.21.15.111:33328) 2024-09-15T22:49:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.466+0000 7f003ea00640 1 -- 172.21.15.111:0/1271888847 learned_addr learned my addr 172.21.15.111:0/1271888847 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:49:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.466+0000 7f003f400640 1 --2- 172.21.15.111:0/1271888847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a8f7fca60 0x557a8f808850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.466+0000 7f0036000640 1 --2- 172.21.15.111:0/1271888847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557a8f7f79d0 0x557a8f801810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.467+0000 7f003ea00640 1 -- 172.21.15.111:0/1271888847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557a8f7f79d0 msgr2=0x557a8f801810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.467+0000 7f003ea00640 1 --2- 172.21.15.111:0/1271888847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557a8f7f79d0 0x557a8f801810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.467+0000 7f003ea00640 1 -- 172.21.15.111:0/1271888847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a8f7fca60 msgr2=0x557a8f808850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.467+0000 7f003ea00640 1 --2- 172.21.15.111:0/1271888847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a8f7fca60 0x557a8f808850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.467+0000 7f003ea00640 1 -- 172.21.15.111:0/1271888847 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a8f53b620 con 0x557a8f7f7090 2024-09-15T22:49:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.467+0000 7f0036000640 1 --2- 172.21.15.111:0/1271888847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557a8f7f79d0 0x557a8f801810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:49:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.467+0000 7f003f400640 1 --2- 172.21.15.111:0/1271888847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a8f7fca60 0x557a8f808850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:49:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.467+0000 7f003ea00640 1 --2- 172.21.15.111:0/1271888847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a8f7f7090 0x557a8f8012d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0030009230 tx=0x7f0030009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5bf6d4d7e470885 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.467+0000 7f002cc00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:49:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.467+0000 7f002cc00640 1 -- 172.21.15.111:0/1271888847 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003004e070 con 0x557a8f7f7090 2024-09-15T22:49:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.467+0000 7f00487cb740 1 -- 172.21.15.111:0/1271888847 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00280130d0 con 0x557a8f7f7090 2024-09-15T22:49:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.467+0000 7f002cc00640 1 -- 172.21.15.111:0/1271888847 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f0030003ac0 con 0x557a8f7f7090 2024-09-15T22:49:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.468+0000 7f002cc00640 1 -- 172.21.15.111:0/1271888847 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f003005a050 con 0x557a8f7f7090 2024-09-15T22:49:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.468+0000 7f00487cb740 10 client.25265 fetch_fsmap learned FSMap version 28 2024-09-15T22:49:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.468+0000 7f00487cb740 1 -- 172.21.15.111:0/1271888847 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557a8f80b700 con 0x557a8f7f7090 2024-09-15T22:49:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.468+0000 7f002cc00640 1 -- 172.21.15.111:0/1271888847 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00280130d0 con 0x557a8f7f7090 2024-09-15T22:49:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.469+0000 7f002cc00640 1 -- 172.21.15.111:0/1271888847 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f003004a450 con 0x557a8f7f7090 2024-09-15T22:49:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.469+0000 7f00487cb740 10 client.25265 fetch_fsmap finished waiting for FSMap version 28 2024-09-15T22:49:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.469+0000 7f00487cb740 10 client.25265 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:49:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.469+0000 7f00487cb740 20 client.25265 populate_metadata read hostname 'smithi111' 2024-09-15T22:49:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.469+0000 7f00487cb740 1 --2- 172.21.15.111:0/1271888847 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557a8f80cdd0 0x557a8f80f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.469+0000 7f00487cb740 4 client.25265 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:49:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.469+0000 7f00487cb740 1 -- 172.21.15.111:0/1271888847 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a8f769830 con 0x557a8f80cdd0 2024-09-15T22:49:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.469+0000 7f0036000640 1 --2- 172.21.15.111:0/1271888847 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557a8f80cdd0 0x557a8f80f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.469+0000 7f0036000640 1 --2- 172.21.15.111:0/1271888847 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557a8f80cdd0 0x557a8f80f220 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f2b58a590e5a825 server_cookie=30a577ec3c33236a in_seq=0 out_seq=0 2024-09-15T22:49:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.469+0000 7f002cc00640 10 client.25265 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:49:06.475 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.470+0000 7f002cc00640 1 -- 172.21.15.111:0/1271888847 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a8f769830 con 0x557a8f80cdd0 2024-09-15T22:49:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.470+0000 7f002cc00640 10 client.25265 handle_command_reply: tid=0 2024-09-15T22:49:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.470+0000 7f00487cb740 1 client.25265 shutdown 2024-09-15T22:49:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.470+0000 7f00487cb740 1 -- 172.21.15.111:0/1271888847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a8f7f7090 msgr2=0x557a8f8012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.470+0000 7f00487cb740 1 --2- 172.21.15.111:0/1271888847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a8f7f7090 0x557a8f8012d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0030009230 tx=0x7f0030009260 comp rx=0 tx=0).stop 2024-09-15T22:49:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.470+0000 7f00487cb740 1 -- 172.21.15.111:0/1271888847 shutdown_connections 2024-09-15T22:49:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.470+0000 7f00487cb740 1 --2- 172.21.15.111:0/1271888847 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557a8f80cdd0 0x557a8f80f220 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.470+0000 7f00487cb740 1 --2- 172.21.15.111:0/1271888847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a8f7fca60 0x557a8f808850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.470+0000 7f00487cb740 1 --2- 172.21.15.111:0/1271888847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557a8f7f79d0 0x557a8f801810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.470+0000 7f00487cb740 1 --2- 172.21.15.111:0/1271888847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a8f7f7090 0x557a8f8012d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.470+0000 7f002cc00640 0 client.25265 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:49:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.470+0000 7f00487cb740 1 -- 172.21.15.111:0/1271888847 >> 172.21.15.111:0/1271888847 conn(0x557a8f7f2700 msgr2=0x557a8f7f3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.471+0000 7f00487cb740 1 -- 172.21.15.111:0/1271888847 shutdown_connections 2024-09-15T22:49:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.471+0000 7f00487cb740 1 -- 172.21.15.111:0/1271888847 wait complete. 2024-09-15T22:49:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.471+0000 7f00487cb740 20 client.25265 trim_cache size 0 max 16384 2024-09-15T22:49:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.486+0000 7f00487cb740 1 Processor -- start 2024-09-15T22:49:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.486+0000 7f00487cb740 1 -- start start 2024-09-15T22:49:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.486+0000 7f00487cb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557a8f7f79d0 0x557a8f7f2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.486+0000 7f00487cb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a8f7fca60 0x557a8f808810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.486+0000 7f00487cb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a8f801790 0x557a8f7f3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.486+0000 7f00487cb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f00401b1c90 con 0x557a8f801790 2024-09-15T22:49:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.486+0000 7f00487cb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f00401af560 con 0x557a8f7fca60 2024-09-15T22:49:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.486+0000 7f00487cb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f00401ad3b0 con 0x557a8f7f79d0 2024-09-15T22:49:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.487+0000 7f003ea00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557a8f7f79d0 0x557a8f7f2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.487+0000 7f003ea00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557a8f7f79d0 0x557a8f7f2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:42268/0 (socket says 172.21.15.111:42268) 2024-09-15T22:49:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.487+0000 7f003ea00640 1 -- 172.21.15.111:0/3269530283 learned_addr learned my addr 172.21.15.111:0/3269530283 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:49:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.487+0000 7f003f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a8f801790 0x557a8f7f3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.487+0000 7f0036000640 1 --2- 172.21.15.111:0/3269530283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a8f7fca60 0x557a8f808810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.487+0000 7f0036000640 1 -- 172.21.15.111:0/3269530283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557a8f7f79d0 msgr2=0x557a8f7f2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.487+0000 7f0036000640 1 --2- 172.21.15.111:0/3269530283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557a8f7f79d0 0x557a8f7f2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.487+0000 7f0036000640 1 -- 172.21.15.111:0/3269530283 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a8f801790 msgr2=0x557a8f7f3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.487+0000 7f0036000640 1 --2- 172.21.15.111:0/3269530283 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a8f801790 0x557a8f7f3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.487+0000 7f0036000640 1 -- 172.21.15.111:0/3269530283 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a8f80f8a0 con 0x557a8f7fca60 2024-09-15T22:49:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.487+0000 7f0036000640 1 --2- 172.21.15.111:0/3269530283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a8f7fca60 0x557a8f808810 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x557a8f8027b0 tx=0x7f0028009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bb422fbaff96eea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.487+0000 7f002f400640 1 -- 172.21.15.111:0/3269530283 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00280130d0 con 0x557a8f7fca60 2024-09-15T22:49:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.487+0000 7f002f400640 1 -- 172.21.15.111:0/3269530283 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0028005520 con 0x557a8f7fca60 2024-09-15T22:49:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.487+0000 7f002f400640 1 -- 172.21.15.111:0/3269530283 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00280048e0 con 0x557a8f7fca60 2024-09-15T22:49:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.488+0000 7f00487cb740 1 -- 172.21.15.111:0/3269530283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a8f7fca60 msgr2=0x557a8f808810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.488+0000 7f00487cb740 1 --2- 172.21.15.111:0/3269530283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a8f7fca60 0x557a8f808810 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x557a8f8027b0 tx=0x7f0028009370 comp rx=0 tx=0).stop 2024-09-15T22:49:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.488+0000 7f00487cb740 1 -- 172.21.15.111:0/3269530283 shutdown_connections 2024-09-15T22:49:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.488+0000 7f00487cb740 1 --2- 172.21.15.111:0/3269530283 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a8f801790 0x557a8f7f3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.488+0000 7f00487cb740 1 --2- 172.21.15.111:0/3269530283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a8f7fca60 0x557a8f808810 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.488+0000 7f00487cb740 1 --2- 172.21.15.111:0/3269530283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557a8f7f79d0 0x557a8f7f2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.488+0000 7f00487cb740 1 -- 172.21.15.111:0/3269530283 >> 172.21.15.111:0/3269530283 conn(0x557a8f7f2700 msgr2=0x557a8f801080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.488+0000 7f00487cb740 1 -- 172.21.15.111:0/3269530283 shutdown_connections 2024-09-15T22:49:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.488+0000 7f00487cb740 1 -- 172.21.15.111:0/3269530283 wait complete. 2024-09-15T22:49:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.489+0000 7f00487cb740 1 -- start start 2024-09-15T22:49:06.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.489+0000 7f00487cb740 1 Processor -- start 2024-09-15T22:49:06.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.489+0000 7f00487cb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557a8f7f79d0 0x557a8f8058d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.489+0000 7f00487cb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a8f7fca60 0x557a8f805e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.489+0000 7f00487cb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a8f801790 0x557a8f806e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.489+0000 7f00487cb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f00401ada20 con 0x557a8f801790 2024-09-15T22:49:06.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.489+0000 7f00487cb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f00401aee90 con 0x557a8f7fca60 2024-09-15T22:49:06.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.489+0000 7f00487cb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f00401ae0f0 con 0x557a8f7f79d0 2024-09-15T22:49:06.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.489+0000 7f00487cb740 10 client.0 _ll_register_callbacks cb 0x7f0030005e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:49:06.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.489+0000 7f0036000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a8f7fca60 0x557a8f805e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.489+0000 7f003ea00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557a8f7f79d0 0x557a8f8058d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.489+0000 7f0036000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a8f7fca60 0x557a8f805e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:33354/0 (socket says 172.21.15.111:33354) 2024-09-15T22:49:06.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.489+0000 7f0036000640 1 -- 172.21.15.111:0/1290583180 learned_addr learned my addr 172.21.15.111:0/1290583180 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:49:06.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.489+0000 7f003f400640 1 --2- 172.21.15.111:0/1290583180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a8f801790 0x557a8f806e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.490+0000 7f0036000640 1 -- 172.21.15.111:0/1290583180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557a8f7f79d0 msgr2=0x557a8f8058d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.490+0000 7f0036000640 1 --2- 172.21.15.111:0/1290583180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557a8f7f79d0 0x557a8f8058d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.490+0000 7f0036000640 1 -- 172.21.15.111:0/1290583180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a8f801790 msgr2=0x557a8f806e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.490+0000 7f0036000640 1 --2- 172.21.15.111:0/1290583180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a8f801790 0x557a8f806e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.495 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.490+0000 7f0036000640 1 -- 172.21.15.111:0/1290583180 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a8f807430 con 0x557a8f7fca60 2024-09-15T22:49:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.490+0000 7f003ea00640 1 --2- 172.21.15.111:0/1290583180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557a8f7f79d0 0x557a8f8058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:49:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.490+0000 7f0036000640 1 --2- 172.21.15.111:0/1290583180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a8f7fca60 0x557a8f805e10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f002800f730 tx=0x7f00280096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0823db0cb3a7c79 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.490+0000 7f002cc00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:49:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.490+0000 7f002cc00640 1 -- 172.21.15.111:0/1290583180 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00280130d0 con 0x557a8f7fca60 2024-09-15T22:49:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.490+0000 7f002cc00640 1 -- 172.21.15.111:0/1290583180 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f0028029540 con 0x557a8f7fca60 2024-09-15T22:49:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.490+0000 7f00487cb740 1 -- 172.21.15.111:0/1290583180 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0028005520 con 0x557a8f7fca60 2024-09-15T22:49:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.491+0000 7f002cc00640 1 -- 172.21.15.111:0/1290583180 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f00280297c0 con 0x557a8f7fca60 2024-09-15T22:49:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.491+0000 7f00487cb740 10 client.25271 fetch_fsmap learned FSMap version 28 2024-09-15T22:49:06.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.491+0000 7f00487cb740 1 -- 172.21.15.111:0/1290583180 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557a8f808740 con 0x557a8f7fca60 2024-09-15T22:49:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.491+0000 7f002cc00640 1 -- 172.21.15.111:0/1290583180 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0028005520 con 0x557a8f7fca60 2024-09-15T22:49:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.491+0000 7f002cc00640 1 -- 172.21.15.111:0/1290583180 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0028014ce0 con 0x557a8f7fca60 2024-09-15T22:49:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.491+0000 7f00487cb740 10 client.25271 fetch_fsmap finished waiting for FSMap version 28 2024-09-15T22:49:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.491+0000 7f00487cb740 10 client.25271 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:49:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.491+0000 7f00487cb740 20 client.25271 populate_metadata read hostname 'smithi111' 2024-09-15T22:49:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.491+0000 7f00487cb740 1 --2- 172.21.15.111:0/1290583180 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557a8f877c40 0x557a8f87a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.491+0000 7f00487cb740 4 client.25271 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:49:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.491+0000 7f00487cb740 1 -- 172.21.15.111:0/1290583180 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a8f763b80 con 0x557a8f877c40 2024-09-15T22:49:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.491+0000 7f003ea00640 1 --2- 172.21.15.111:0/1290583180 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557a8f877c40 0x557a8f87a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:06.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.492+0000 7f003ea00640 1 --2- 172.21.15.111:0/1290583180 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557a8f877c40 0x557a8f87a060 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2701b184a454c91c server_cookie=a802cc13ff3ff34f in_seq=0 out_seq=0 2024-09-15T22:49:06.498 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.492+0000 7f002cc00640 10 client.25271 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:49:06.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.499+0000 7f002cc00640 1 -- 172.21.15.111:0/1290583180 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557a8f763b80 con 0x557a8f877c40 2024-09-15T22:49:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.499+0000 7f002cc00640 10 client.25271 handle_command_reply: tid=0 2024-09-15T22:49:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.499+0000 7f00487cb740 1 client.25271 shutdown 2024-09-15T22:49:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.499+0000 7f00487cb740 1 -- 172.21.15.111:0/1290583180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a8f7fca60 msgr2=0x557a8f805e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.499+0000 7f00487cb740 1 --2- 172.21.15.111:0/1290583180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a8f7fca60 0x557a8f805e10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f002800f730 tx=0x7f00280096c0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.499+0000 7f00487cb740 1 -- 172.21.15.111:0/1290583180 shutdown_connections 2024-09-15T22:49:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.499+0000 7f00487cb740 1 --2- 172.21.15.111:0/1290583180 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557a8f877c40 0x557a8f87a060 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.499+0000 7f00487cb740 1 --2- 172.21.15.111:0/1290583180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557a8f801790 0x557a8f806e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.499+0000 7f00487cb740 1 --2- 172.21.15.111:0/1290583180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a8f7fca60 0x557a8f805e10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.499+0000 7f00487cb740 1 --2- 172.21.15.111:0/1290583180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557a8f7f79d0 0x557a8f8058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.499+0000 7f002cc00640 0 client.25271 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:49:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.499+0000 7f00487cb740 1 -- 172.21.15.111:0/1290583180 >> 172.21.15.111:0/1290583180 conn(0x557a8f7f2700 msgr2=0x557a8f7f1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.499+0000 7f00487cb740 1 -- 172.21.15.111:0/1290583180 shutdown_connections 2024-09-15T22:49:06.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.499+0000 7f00487cb740 1 -- 172.21.15.111:0/1290583180 wait complete. 2024-09-15T22:49:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.499+0000 7f00487cb740 20 client.25271 trim_cache size 0 max 16384 2024-09-15T22:49:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:49:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:49:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:49:06.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:49:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.500+0000 7f0045a00640 1 -- 172.21.15.111:0/3217280031 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f00240a32f0 msgr2=0x7f00240a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.500+0000 7f0045a00640 1 --2- 172.21.15.111:0/3217280031 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f00240a32f0 0x7f00240a57b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0030031440 tx=0x7f00300315b0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.500+0000 7f0045a00640 1 -- 172.21.15.111:0/3217280031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f004010e130 msgr2=0x7f00401a5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.500+0000 7f0045a00640 1 --2- 172.21.15.111:0/3217280031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f004010e130 0x7f00401a5930 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f003800e940 tx=0x7f003800ee00 comp rx=0 tx=0).stop 2024-09-15T22:49:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.500+0000 7f0045a00640 1 -- 172.21.15.111:0/3217280031 shutdown_connections 2024-09-15T22:49:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.500+0000 7f0045a00640 1 --2- 172.21.15.111:0/3217280031 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f00240a32f0 0x7f00240a57b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.500+0000 7f0045a00640 1 --2- 172.21.15.111:0/3217280031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f004010e130 0x7f00401a5930 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.500+0000 7f0045a00640 1 --2- 172.21.15.111:0/3217280031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0040076230 0x7f004019e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.500+0000 7f0045a00640 1 --2- 172.21.15.111:0/3217280031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0040074c50 0x7f004019e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:06.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.500+0000 7f0045a00640 1 -- 172.21.15.111:0/3217280031 >> 172.21.15.111:0/3217280031 conn(0x7f00400fd2e0 msgr2=0x7f00401013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:06.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.500+0000 7f0045a00640 1 -- 172.21.15.111:0/3217280031 shutdown_connections 2024-09-15T22:49:06.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:06.500+0000 7f0045a00640 1 -- 172.21.15.111:0/3217280031 wait complete. 2024-09-15T22:49:07.244 INFO:tasks.workunit.client.0.smithi111.stdout: 1 195571 11.29 MB/sec warmup 95 sec latency 7.433 ms 2024-09-15T22:49:08.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:07 smithi192 ceph-mon[35427]: pgmap v446: 97 pgs: 97 active+clean; 93 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 203 op/s 2024-09-15T22:49:08.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:07 smithi121 ceph-mon[34815]: pgmap v446: 97 pgs: 97 active+clean; 93 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 203 op/s 2024-09-15T22:49:08.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:07 smithi111 ceph-mon[30629]: pgmap v446: 97 pgs: 97 active+clean; 93 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 203 op/s 2024-09-15T22:49:08.241 INFO:tasks.workunit.client.0.smithi111.stdout: 1 198861 11.37 MB/sec warmup 96 sec latency 13.137 ms 2024-09-15T22:49:09.242 INFO:tasks.workunit.client.0.smithi111.stdout: 1 200975 11.33 MB/sec warmup 97 sec latency 207.566 ms 2024-09-15T22:49:09.888 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:09 smithi111 ceph-mon[30629]: pgmap v447: 97 pgs: 97 active+clean; 93 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 195 op/s 2024-09-15T22:49:10.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:09 smithi192 ceph-mon[35427]: pgmap v447: 97 pgs: 97 active+clean; 93 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 195 op/s 2024-09-15T22:49:10.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:09 smithi121 ceph-mon[34815]: pgmap v447: 97 pgs: 97 active+clean; 93 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 195 op/s 2024-09-15T22:49:10.241 INFO:tasks.workunit.client.0.smithi111.stdout: 1 202196 11.32 MB/sec warmup 98 sec latency 570.904 ms 2024-09-15T22:49:11.242 INFO:tasks.workunit.client.0.smithi111.stdout: 1 202753 11.22 MB/sec warmup 99 sec latency 606.020 ms 2024-09-15T22:49:12.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:11 smithi192 ceph-mon[35427]: pgmap v448: 97 pgs: 97 active+clean; 92 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 194 op/s 2024-09-15T22:49:12.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:11 smithi121 ceph-mon[34815]: pgmap v448: 97 pgs: 97 active+clean; 92 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 194 op/s 2024-09-15T22:49:12.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:11 smithi111 ceph-mon[30629]: pgmap v448: 97 pgs: 97 active+clean; 92 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 194 op/s 2024-09-15T22:49:12.242 INFO:tasks.workunit.client.0.smithi111.stdout: 1 205954 11.30 MB/sec warmup 100 sec latency 12.243 ms 2024-09-15T22:49:13.242 INFO:tasks.workunit.client.0.smithi111.stdout: 1 208725 11.31 MB/sec warmup 101 sec latency 11.068 ms 2024-09-15T22:49:13.967 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:13 smithi111 ceph-mon[30629]: pgmap v449: 97 pgs: 97 active+clean; 96 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 217 op/s 2024-09-15T22:49:14.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:13 smithi192 ceph-mon[35427]: pgmap v449: 97 pgs: 97 active+clean; 96 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 217 op/s 2024-09-15T22:49:14.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:13 smithi121 ceph-mon[34815]: pgmap v449: 97 pgs: 97 active+clean; 96 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 217 op/s 2024-09-15T22:49:14.242 INFO:tasks.workunit.client.0.smithi111.stdout: 1 211104 11.32 MB/sec warmup 102 sec latency 13.946 ms 2024-09-15T22:49:15.242 INFO:tasks.workunit.client.0.smithi111.stdout: 1 212918 11.33 MB/sec warmup 103 sec latency 474.674 ms 2024-09-15T22:49:15.878 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:15 smithi111 ceph-mon[30629]: pgmap v450: 97 pgs: 97 active+clean; 97 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 214 op/s 2024-09-15T22:49:16.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:15 smithi192 ceph-mon[35427]: pgmap v450: 97 pgs: 97 active+clean; 97 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 214 op/s 2024-09-15T22:49:16.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:15 smithi121 ceph-mon[34815]: pgmap v450: 97 pgs: 97 active+clean; 97 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 214 op/s 2024-09-15T22:49:16.242 INFO:tasks.workunit.client.0.smithi111.stdout: 1 213134 11.23 MB/sec warmup 104 sec latency 1304.294 ms 2024-09-15T22:49:17.242 INFO:tasks.workunit.client.0.smithi111.stdout: 1 216048 11.25 MB/sec warmup 105 sec latency 13.272 ms 2024-09-15T22:49:18.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:17 smithi192 ceph-mon[35427]: pgmap v451: 97 pgs: 97 active+clean; 97 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 219 op/s 2024-09-15T22:49:18.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:17 smithi121 ceph-mon[34815]: pgmap v451: 97 pgs: 97 active+clean; 97 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 219 op/s 2024-09-15T22:49:18.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:17 smithi111 ceph-mon[30629]: pgmap v451: 97 pgs: 97 active+clean; 97 MiB data, 870 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 219 op/s 2024-09-15T22:49:18.242 INFO:tasks.workunit.client.0.smithi111.stdout: 1 218378 11.25 MB/sec warmup 106 sec latency 16.104 ms 2024-09-15T22:49:19.242 INFO:tasks.workunit.client.0.smithi111.stdout: 1 220688 11.28 MB/sec warmup 107 sec latency 19.828 ms 2024-09-15T22:49:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.622+0000 7f3083200640 1 -- 172.21.15.111:0/3563444081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30841001b0 msgr2=0x7f30841025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.622+0000 7f3083200640 1 --2- 172.21.15.111:0/3563444081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30841001b0 0x7f30841025c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3074009900 tx=0x7f307402a400 comp rx=0 tx=0).stop 2024-09-15T22:49:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.622+0000 7f3083200640 1 -- 172.21.15.111:0/3563444081 shutdown_connections 2024-09-15T22:49:19.624 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.622+0000 7f3083200640 1 --2- 172.21.15.111:0/3563444081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30841054b0 0x7f3084105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.622+0000 7f3083200640 1 --2- 172.21.15.111:0/3563444081 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3084102b00 0x7f3084104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.622+0000 7f3083200640 1 --2- 172.21.15.111:0/3563444081 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30841001b0 0x7f30841025c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.622+0000 7f3083200640 1 -- 172.21.15.111:0/3563444081 >> 172.21.15.111:0/3563444081 conn(0x7f30840fbfa0 msgr2=0x7f30840fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.622+0000 7f3083200640 1 -- 172.21.15.111:0/3563444081 shutdown_connections 2024-09-15T22:49:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.622+0000 7f3083200640 1 -- 172.21.15.111:0/3563444081 wait complete. 2024-09-15T22:49:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.623+0000 7f3083200640 1 Processor -- start 2024-09-15T22:49:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.623+0000 7f3083200640 1 -- start start 2024-09-15T22:49:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.623+0000 7f3083200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30841001b0 0x7f308419e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.623+0000 7f3083200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3084102b00 0x7f308419e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.623+0000 7f3083200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30841054b0 0x7f30841a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.623+0000 7f3083200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3084113bc0 con 0x7f30841001b0 2024-09-15T22:49:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.623+0000 7f3083200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3084113a40 con 0x7f30841054b0 2024-09-15T22:49:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.623+0000 7f3083200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3084113d40 con 0x7f3084102b00 2024-09-15T22:49:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.624+0000 7f3081e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30841001b0 0x7f308419e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.624+0000 7f3081e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30841001b0 0x7f308419e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52648/0 (socket says 172.21.15.111:52648) 2024-09-15T22:49:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.624+0000 7f3081e00640 1 -- 172.21.15.111:0/4086975998 learned_addr learned my addr 172.21.15.111:0/4086975998 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:49:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.624+0000 7f3082800640 1 --2- 172.21.15.111:0/4086975998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30841054b0 0x7f30841a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.624+0000 7f3081e00640 1 -- 172.21.15.111:0/4086975998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3084102b00 msgr2=0x7f308419e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.624+0000 7f3081e00640 1 --2- 172.21.15.111:0/4086975998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3084102b00 0x7f308419e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.624+0000 7f3081e00640 1 -- 172.21.15.111:0/4086975998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30841054b0 msgr2=0x7f30841a5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.624+0000 7f3081e00640 1 --2- 172.21.15.111:0/4086975998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30841054b0 0x7f30841a5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.624+0000 7f3081e00640 1 -- 172.21.15.111:0/4086975998 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30841a5e80 con 0x7f30841001b0 2024-09-15T22:49:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.624+0000 7f3082800640 1 --2- 172.21.15.111:0/4086975998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30841054b0 0x7f30841a5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:49:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.624+0000 7f3081e00640 1 --2- 172.21.15.111:0/4086975998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30841001b0 0x7f308419e260 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3074009230 tx=0x7f30740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.625+0000 7f307aa00640 1 -- 172.21.15.111:0/4086975998 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307403f030 con 0x7f30841001b0 2024-09-15T22:49:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.625+0000 7f307aa00640 1 -- 172.21.15.111:0/4086975998 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3074003ce0 con 0x7f30841001b0 2024-09-15T22:49:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.625+0000 7f3083200640 1 -- 172.21.15.111:0/4086975998 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30841a6110 con 0x7f30841001b0 2024-09-15T22:49:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.625+0000 7f307aa00640 1 -- 172.21.15.111:0/4086975998 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3074035070 con 0x7f30841001b0 2024-09-15T22:49:19.630 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.625+0000 7f3083200640 1 -- 172.21.15.111:0/4086975998 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30841a6650 con 0x7f30841001b0 2024-09-15T22:49:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.628+0000 7f308c01a740 1 Processor -- start 2024-09-15T22:49:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.628+0000 7f308c01a740 1 -- start start 2024-09-15T22:49:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.628+0000 7f307aa00640 1 -- 172.21.15.111:0/4086975998 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f3074042030 con 0x7f30841001b0 2024-09-15T22:49:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.628+0000 7f308c01a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be2e43260 0x560be2e43660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.628+0000 7f308c01a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be2e43ba0 0x560be2e48660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.628+0000 7f308c01a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be2e48c30 0x560be2e4b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.628+0000 7f308c01a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560be2da8100 con 0x560be2e48c30 2024-09-15T22:49:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.628+0000 7f308c01a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560be2d501b0 con 0x560be2e43ba0 2024-09-15T22:49:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.628+0000 7f308c01a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x560be2da40c0 con 0x560be2e43260 2024-09-15T22:49:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.628+0000 7f3082800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be2e43260 0x560be2e43660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.629+0000 7f3082800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be2e43260 0x560be2e43660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:51528/0 (socket says 172.21.15.111:51528) 2024-09-15T22:49:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.629+0000 7f3082800640 1 -- 172.21.15.111:0/2728252242 learned_addr learned my addr 172.21.15.111:0/2728252242 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:49:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.629+0000 7f3081e00640 1 --2- 172.21.15.111:0/2728252242 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be2e43ba0 0x560be2e48660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.629+0000 7f3081400640 1 --2- 172.21.15.111:0/2728252242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be2e48c30 0x560be2e4b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.629+0000 7f3082800640 1 -- 172.21.15.111:0/2728252242 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be2e43ba0 msgr2=0x560be2e48660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.629+0000 7f3082800640 1 --2- 172.21.15.111:0/2728252242 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be2e43ba0 0x560be2e48660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.629+0000 7f3082800640 1 -- 172.21.15.111:0/2728252242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be2e48c30 msgr2=0x560be2e4b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.629+0000 7f3082800640 1 --2- 172.21.15.111:0/2728252242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be2e48c30 0x560be2e4b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.629+0000 7f3082800640 1 -- 172.21.15.111:0/2728252242 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560be2abb210 con 0x560be2e43260 2024-09-15T22:49:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.629+0000 7f3082800640 1 --2- 172.21.15.111:0/2728252242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be2e43260 0x560be2e43660 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f307c0117e0 tx=0x7f307c011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5343e85e518ae2d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.629+0000 7f306f400640 1 -- 172.21.15.111:0/2728252242 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c011310 con 0x560be2e43260 2024-09-15T22:49:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.630+0000 7f306f400640 1 -- 172.21.15.111:0/2728252242 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f307c003e50 con 0x560be2e43260 2024-09-15T22:49:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.630+0000 7f306f400640 1 -- 172.21.15.111:0/2728252242 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c004620 con 0x560be2e43260 2024-09-15T22:49:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.630+0000 7f307aa00640 1 --2- 172.21.15.111:0/4086975998 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f30680a32c0 0x7f30680a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.630+0000 7f307aa00640 1 -- 172.21.15.111:0/4086975998 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f30740efd90 con 0x7f30841001b0 2024-09-15T22:49:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.630+0000 7f308c01a740 1 -- 172.21.15.111:0/2728252242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be2e43260 msgr2=0x560be2e43660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.630+0000 7f3081e00640 1 --2- 172.21.15.111:0/4086975998 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f30680a32c0 0x7f30680a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.630+0000 7f308c01a740 1 --2- 172.21.15.111:0/2728252242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be2e43260 0x560be2e43660 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f307c0117e0 tx=0x7f307c011ca0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.630+0000 7f308c01a740 1 -- 172.21.15.111:0/2728252242 shutdown_connections 2024-09-15T22:49:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.630+0000 7f308c01a740 1 --2- 172.21.15.111:0/2728252242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be2e48c30 0x560be2e4b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.630+0000 7f308c01a740 1 --2- 172.21.15.111:0/2728252242 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be2e43ba0 0x560be2e48660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.630+0000 7f308c01a740 1 --2- 172.21.15.111:0/2728252242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be2e43260 0x560be2e43660 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.630+0000 7f308c01a740 1 -- 172.21.15.111:0/2728252242 >> 172.21.15.111:0/2728252242 conn(0x560be2e3e8d0 msgr2=0x560be2e3ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.631+0000 7f308c01a740 1 -- 172.21.15.111:0/2728252242 shutdown_connections 2024-09-15T22:49:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.631+0000 7f308c01a740 1 -- 172.21.15.111:0/2728252242 wait complete. 2024-09-15T22:49:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.631+0000 7f3081e00640 1 --2- 172.21.15.111:0/4086975998 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f30680a32c0 0x7f30680a5780 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f30740b8810 tx=0x7f30740f37d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.631+0000 7f308c01a740 1 -- start start 2024-09-15T22:49:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.631+0000 7f308c01a740 1 Processor -- start 2024-09-15T22:49:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.632+0000 7f308c01a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be2e43260 0x560be2e4d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.632+0000 7f308c01a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be2e43ba0 0x560be2e4da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.632+0000 7f308c01a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be2e48c30 0x560be2e549f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.632+0000 7f308c01a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560be2e4bc00 con 0x560be2e43260 2024-09-15T22:49:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.632+0000 7f308c01a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560be2d31090 con 0x560be2e43ba0 2024-09-15T22:49:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.632+0000 7f308c01a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x560be2e4bd80 con 0x560be2e48c30 2024-09-15T22:49:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.632+0000 7f308c01a740 10 client.0 _ll_register_callbacks cb 0x560be2d85f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:49:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.632+0000 7f3082800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be2e43260 0x560be2e4d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.632+0000 7f3081400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be2e43ba0 0x560be2e4da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.632+0000 7f3082800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be2e43260 0x560be2e4d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52668/0 (socket says 172.21.15.111:52668) 2024-09-15T22:49:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.632+0000 7f3082800640 1 -- 172.21.15.111:0/720685304 learned_addr learned my addr 172.21.15.111:0/720685304 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:49:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.632+0000 7f3082800640 1 --2- 172.21.15.111:0/720685304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be2e48c30 0x560be2e549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.632+0000 7f3081400640 1 -- 172.21.15.111:0/720685304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be2e48c30 msgr2=0x560be2e549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.632+0000 7f3081400640 1 --2- 172.21.15.111:0/720685304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be2e48c30 0x560be2e549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.632+0000 7f3081400640 1 -- 172.21.15.111:0/720685304 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be2e43260 msgr2=0x560be2e4d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.632+0000 7f3081400640 1 --2- 172.21.15.111:0/720685304 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be2e43260 0x560be2e4d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.632+0000 7f3081400640 1 -- 172.21.15.111:0/720685304 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560be2b87620 con 0x560be2e43ba0 2024-09-15T22:49:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.633+0000 7f3081400640 1 --2- 172.21.15.111:0/720685304 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be2e43ba0 0x560be2e4da20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f30700095f0 tx=0x7f3070011b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8cf9560257560c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.633+0000 7f3063e00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:49:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.633+0000 7f3063e00640 1 -- 172.21.15.111:0/720685304 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307000fd00 con 0x560be2e43ba0 2024-09-15T22:49:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.633+0000 7f308c01a740 1 -- 172.21.15.111:0/720685304 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f307c003e50 con 0x560be2e43ba0 2024-09-15T22:49:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.633+0000 7f3063e00640 1 -- 172.21.15.111:0/720685304 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f3070004030 con 0x560be2e43ba0 2024-09-15T22:49:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.633+0000 7f3063e00640 1 -- 172.21.15.111:0/720685304 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f307000fea0 con 0x560be2e43ba0 2024-09-15T22:49:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.633+0000 7f308c01a740 10 client.25283 fetch_fsmap learned FSMap version 28 2024-09-15T22:49:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.633+0000 7f308c01a740 1 -- 172.21.15.111:0/720685304 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560be2e57900 con 0x560be2e43ba0 2024-09-15T22:49:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.633+0000 7f3063e00640 1 -- 172.21.15.111:0/720685304 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f307c003e50 con 0x560be2e43ba0 2024-09-15T22:49:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.634+0000 7f3063e00640 1 -- 172.21.15.111:0/720685304 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f30700194c0 con 0x560be2e43ba0 2024-09-15T22:49:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.634+0000 7f308c01a740 10 client.25283 fetch_fsmap finished waiting for FSMap version 28 2024-09-15T22:49:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.634+0000 7f308c01a740 10 client.25283 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:49:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.634+0000 7f308c01a740 20 client.25283 populate_metadata read hostname 'smithi111' 2024-09-15T22:49:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.634+0000 7f308c01a740 1 --2- 172.21.15.111:0/720685304 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560be2e58fb0 0x560be2e5b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.634+0000 7f308c01a740 4 client.25283 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:49:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.634+0000 7f308c01a740 1 -- 172.21.15.111:0/720685304 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560be2db5830 con 0x560be2e58fb0 2024-09-15T22:49:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.634+0000 7f3082800640 1 --2- 172.21.15.111:0/720685304 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560be2e58fb0 0x560be2e5b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.634+0000 7f3082800640 1 --2- 172.21.15.111:0/720685304 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560be2e58fb0 0x560be2e5b430 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3998da8ade050b8f server_cookie=e3f0f164af2ff07f in_seq=0 out_seq=0 2024-09-15T22:49:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.634+0000 7f3063e00640 10 client.25283 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:49:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.635+0000 7f3063e00640 1 -- 172.21.15.111:0/720685304 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560be2db5830 con 0x560be2e58fb0 2024-09-15T22:49:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.635+0000 7f3063e00640 10 client.25283 handle_command_reply: tid=0 2024-09-15T22:49:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.635+0000 7f308c01a740 1 client.25283 shutdown 2024-09-15T22:49:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.636+0000 7f308c01a740 1 -- 172.21.15.111:0/720685304 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be2e43ba0 msgr2=0x560be2e4da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.636+0000 7f308c01a740 1 --2- 172.21.15.111:0/720685304 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be2e43ba0 0x560be2e4da20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f30700095f0 tx=0x7f3070011b50 comp rx=0 tx=0).stop 2024-09-15T22:49:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.636+0000 7f308c01a740 1 -- 172.21.15.111:0/720685304 shutdown_connections 2024-09-15T22:49:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.636+0000 7f308c01a740 1 --2- 172.21.15.111:0/720685304 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560be2e58fb0 0x560be2e5b430 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.636+0000 7f308c01a740 1 --2- 172.21.15.111:0/720685304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be2e48c30 0x560be2e549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.636+0000 7f308c01a740 1 --2- 172.21.15.111:0/720685304 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be2e43ba0 0x560be2e4da20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.636+0000 7f308c01a740 1 --2- 172.21.15.111:0/720685304 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be2e43260 0x560be2e4d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.636+0000 7f308c01a740 1 -- 172.21.15.111:0/720685304 >> 172.21.15.111:0/720685304 conn(0x560be2e3e8d0 msgr2=0x560be2e400e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.636+0000 7f3063e00640 0 client.25283 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:49:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.636+0000 7f308c01a740 1 -- 172.21.15.111:0/720685304 shutdown_connections 2024-09-15T22:49:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.636+0000 7f308c01a740 1 -- 172.21.15.111:0/720685304 wait complete. 2024-09-15T22:49:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.636+0000 7f308c01a740 20 client.25283 trim_cache size 0 max 16384 2024-09-15T22:49:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.651+0000 7f308c01a740 1 Processor -- start 2024-09-15T22:49:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.651+0000 7f308c01a740 1 -- start start 2024-09-15T22:49:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.651+0000 7f308c01a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be2e43260 0x560be2e3ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.651+0000 7f308c01a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be2e43ba0 0x560be2e549b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.651+0000 7f308c01a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be2e48c30 0x560be2e3f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.651+0000 7f308c01a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f30841b1b30 con 0x560be2e48c30 2024-09-15T22:49:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.651+0000 7f308c01a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f30841af400 con 0x560be2e43260 2024-09-15T22:49:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.651+0000 7f308c01a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f30841ad250 con 0x560be2e43ba0 2024-09-15T22:49:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.651+0000 7f3082800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be2e43260 0x560be2e3ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.652+0000 7f3081400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be2e43ba0 0x560be2e549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.652+0000 7f3082800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be2e43260 0x560be2e3ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:58656/0 (socket says 172.21.15.111:58656) 2024-09-15T22:49:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.652+0000 7f3082800640 1 -- 172.21.15.111:0/3809834104 learned_addr learned my addr 172.21.15.111:0/3809834104 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:49:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.652+0000 7f3082800640 1 --2- 172.21.15.111:0/3809834104 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be2e48c30 0x560be2e3f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.652+0000 7f3081400640 1 -- 172.21.15.111:0/3809834104 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be2e43260 msgr2=0x560be2e3ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.652+0000 7f3081400640 1 --2- 172.21.15.111:0/3809834104 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be2e43260 0x560be2e3ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.652+0000 7f3081400640 1 -- 172.21.15.111:0/3809834104 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be2e48c30 msgr2=0x560be2e3f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.652+0000 7f3081400640 1 --2- 172.21.15.111:0/3809834104 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be2e48c30 0x560be2e3f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.652+0000 7f3081400640 1 -- 172.21.15.111:0/3809834104 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560be2e3e6f0 con 0x560be2e43ba0 2024-09-15T22:49:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.652+0000 7f3082800640 1 --2- 172.21.15.111:0/3809834104 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be2e48c30 0x560be2e3f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:49:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.652+0000 7f3081400640 1 --2- 172.21.15.111:0/3809834104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be2e43ba0 0x560be2e549b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f30700198b0 tx=0x7f307000fd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6444f59c45d183b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.652+0000 7f306ea00640 1 -- 172.21.15.111:0/3809834104 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3070019d20 con 0x560be2e43ba0 2024-09-15T22:49:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.652+0000 7f306ea00640 1 -- 172.21.15.111:0/3809834104 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3070004550 con 0x560be2e43ba0 2024-09-15T22:49:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.652+0000 7f306ea00640 1 -- 172.21.15.111:0/3809834104 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3070003b70 con 0x560be2e43ba0 2024-09-15T22:49:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.653+0000 7f308c01a740 1 -- 172.21.15.111:0/3809834104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be2e43ba0 msgr2=0x560be2e549b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.653+0000 7f308c01a740 1 --2- 172.21.15.111:0/3809834104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be2e43ba0 0x560be2e549b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f30700198b0 tx=0x7f307000fd10 comp rx=0 tx=0).stop 2024-09-15T22:49:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.653+0000 7f308c01a740 1 -- 172.21.15.111:0/3809834104 shutdown_connections 2024-09-15T22:49:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.653+0000 7f308c01a740 1 --2- 172.21.15.111:0/3809834104 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be2e48c30 0x560be2e3f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.653+0000 7f308c01a740 1 --2- 172.21.15.111:0/3809834104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be2e43ba0 0x560be2e549b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.653+0000 7f308c01a740 1 --2- 172.21.15.111:0/3809834104 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be2e43260 0x560be2e3ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.653+0000 7f308c01a740 1 -- 172.21.15.111:0/3809834104 >> 172.21.15.111:0/3809834104 conn(0x560be2e3e8d0 msgr2=0x560be2e44e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.653+0000 7f308c01a740 1 -- 172.21.15.111:0/3809834104 shutdown_connections 2024-09-15T22:49:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.653+0000 7f308c01a740 1 -- 172.21.15.111:0/3809834104 wait complete. 2024-09-15T22:49:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.654+0000 7f308c01a740 1 -- start start 2024-09-15T22:49:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.654+0000 7f308c01a740 1 Processor -- start 2024-09-15T22:49:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.654+0000 7f308c01a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be2e43260 0x560be2e51740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.654+0000 7f308c01a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be2e43ba0 0x560be2e51c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.654+0000 7f308c01a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be2e48c30 0x560be2ec1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.654+0000 7f308c01a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f30841ad8c0 con 0x560be2e48c30 2024-09-15T22:49:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.654+0000 7f308c01a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f30841aed30 con 0x560be2e43ba0 2024-09-15T22:49:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.654+0000 7f308c01a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f30841adf90 con 0x560be2e43260 2024-09-15T22:49:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.654+0000 7f308c01a740 10 client.0 _ll_register_callbacks cb 0x7f30700097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:49:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.654+0000 7f3081400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be2e43ba0 0x560be2e51c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.654+0000 7f3081400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be2e43ba0 0x560be2e51c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:58660/0 (socket says 172.21.15.111:58660) 2024-09-15T22:49:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.654+0000 7f3081400640 1 -- 172.21.15.111:0/2281639547 learned_addr learned my addr 172.21.15.111:0/2281639547 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:49:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.654+0000 7f3082800640 1 --2- 172.21.15.111:0/2281639547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be2e48c30 0x560be2ec1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.655+0000 7f3082800640 1 --2- 172.21.15.111:0/2281639547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be2e43260 0x560be2e51740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.655+0000 7f3081400640 1 -- 172.21.15.111:0/2281639547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be2e43260 msgr2=0x560be2e51740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.655+0000 7f3081400640 1 --2- 172.21.15.111:0/2281639547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be2e43260 0x560be2e51740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.655+0000 7f3081400640 1 -- 172.21.15.111:0/2281639547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be2e48c30 msgr2=0x560be2ec1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.655+0000 7f3081400640 1 --2- 172.21.15.111:0/2281639547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be2e48c30 0x560be2ec1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.655+0000 7f3081400640 1 -- 172.21.15.111:0/2281639547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560be2e54c10 con 0x560be2e43ba0 2024-09-15T22:49:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.655+0000 7f3081400640 1 --2- 172.21.15.111:0/2281639547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be2e43ba0 0x560be2e51c80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f30700094e0 tx=0x7f3070003660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3324e965ac1b1f99 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.655+0000 7f3063e00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:49:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.655+0000 7f3063e00640 1 -- 172.21.15.111:0/2281639547 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3070019640 con 0x560be2e43ba0 2024-09-15T22:49:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.655+0000 7f308c01a740 1 -- 172.21.15.111:0/2281639547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3070004550 con 0x560be2e43ba0 2024-09-15T22:49:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.655+0000 7f3063e00640 1 -- 172.21.15.111:0/2281639547 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f3070005020 con 0x560be2e43ba0 2024-09-15T22:49:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.656+0000 7f3063e00640 1 -- 172.21.15.111:0/2281639547 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3070005240 con 0x560be2e43ba0 2024-09-15T22:49:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.656+0000 7f308c01a740 10 client.25289 fetch_fsmap learned FSMap version 28 2024-09-15T22:49:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.656+0000 7f308c01a740 1 -- 172.21.15.111:0/2281639547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560be2ec31e0 con 0x560be2e43ba0 2024-09-15T22:49:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.656+0000 7f3063e00640 1 -- 172.21.15.111:0/2281639547 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3070004550 con 0x560be2e43ba0 2024-09-15T22:49:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.656+0000 7f3063e00640 1 -- 172.21.15.111:0/2281639547 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f307003e050 con 0x560be2e43ba0 2024-09-15T22:49:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.656+0000 7f308c01a740 10 client.25289 fetch_fsmap finished waiting for FSMap version 28 2024-09-15T22:49:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.656+0000 7f308c01a740 10 client.25289 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:49:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.656+0000 7f308c01a740 20 client.25289 populate_metadata read hostname 'smithi111' 2024-09-15T22:49:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.656+0000 7f308c01a740 1 --2- 172.21.15.111:0/2281639547 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560be2ec3a80 0x560be2ec5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.656+0000 7f308c01a740 4 client.25289 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:49:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.656+0000 7f308c01a740 1 -- 172.21.15.111:0/2281639547 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560be2dafb80 con 0x560be2ec3a80 2024-09-15T22:49:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.656+0000 7f3082800640 1 --2- 172.21.15.111:0/2281639547 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560be2ec3a80 0x560be2ec5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.657+0000 7f3082800640 1 --2- 172.21.15.111:0/2281639547 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560be2ec3a80 0x560be2ec5ea0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ea391b80ed9e9ea server_cookie=53a7e71919cbf4a2 in_seq=0 out_seq=0 2024-09-15T22:49:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.657+0000 7f3063e00640 10 client.25289 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:49:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.664+0000 7f3063e00640 1 -- 172.21.15.111:0/2281639547 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560be2dafb80 con 0x560be2ec3a80 2024-09-15T22:49:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.664+0000 7f3063e00640 10 client.25289 handle_command_reply: tid=0 2024-09-15T22:49:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.664+0000 7f308c01a740 1 client.25289 shutdown 2024-09-15T22:49:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.664+0000 7f308c01a740 1 -- 172.21.15.111:0/2281639547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be2e43ba0 msgr2=0x560be2e51c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.664+0000 7f308c01a740 1 --2- 172.21.15.111:0/2281639547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be2e43ba0 0x560be2e51c80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f30700094e0 tx=0x7f3070003660 comp rx=0 tx=0).stop 2024-09-15T22:49:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.664+0000 7f308c01a740 1 -- 172.21.15.111:0/2281639547 shutdown_connections 2024-09-15T22:49:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.664+0000 7f308c01a740 1 --2- 172.21.15.111:0/2281639547 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560be2ec3a80 0x560be2ec5ea0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.664+0000 7f308c01a740 1 --2- 172.21.15.111:0/2281639547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be2e48c30 0x560be2ec1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.664+0000 7f3063e00640 0 client.25289 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:49:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.664+0000 7f308c01a740 1 --2- 172.21.15.111:0/2281639547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be2e43ba0 0x560be2e51c80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.664+0000 7f308c01a740 1 --2- 172.21.15.111:0/2281639547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be2e43260 0x560be2e51740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.664+0000 7f308c01a740 1 -- 172.21.15.111:0/2281639547 >> 172.21.15.111:0/2281639547 conn(0x560be2e3e8d0 msgr2=0x560be2e3d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.664+0000 7f308c01a740 1 -- 172.21.15.111:0/2281639547 shutdown_connections 2024-09-15T22:49:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.664+0000 7f308c01a740 1 -- 172.21.15.111:0/2281639547 wait complete. 2024-09-15T22:49:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.665+0000 7f308c01a740 20 client.25289 trim_cache size 0 max 16384 2024-09-15T22:49:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:49:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:49:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:49:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:49:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.665+0000 7f3083200640 1 -- 172.21.15.111:0/4086975998 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f30680a32c0 msgr2=0x7f30680a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.665+0000 7f3083200640 1 --2- 172.21.15.111:0/4086975998 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f30680a32c0 0x7f30680a5780 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f30740b8810 tx=0x7f30740f37d0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.665+0000 7f3083200640 1 -- 172.21.15.111:0/4086975998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30841001b0 msgr2=0x7f308419e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.665+0000 7f3083200640 1 --2- 172.21.15.111:0/4086975998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30841001b0 0x7f308419e260 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3074009230 tx=0x7f30740042f0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.665+0000 7f3083200640 1 -- 172.21.15.111:0/4086975998 shutdown_connections 2024-09-15T22:49:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.665+0000 7f3083200640 1 --2- 172.21.15.111:0/4086975998 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f30680a32c0 0x7f30680a5780 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.665+0000 7f3083200640 1 --2- 172.21.15.111:0/4086975998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f30841054b0 0x7f30841a5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.665+0000 7f3083200640 1 --2- 172.21.15.111:0/4086975998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3084102b00 0x7f308419e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.665+0000 7f3083200640 1 --2- 172.21.15.111:0/4086975998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30841001b0 0x7f308419e260 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.665+0000 7f3083200640 1 -- 172.21.15.111:0/4086975998 >> 172.21.15.111:0/4086975998 conn(0x7f30840fbfa0 msgr2=0x7f3084103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.665+0000 7f3083200640 1 -- 172.21.15.111:0/4086975998 shutdown_connections 2024-09-15T22:49:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:19.665+0000 7f3083200640 1 -- 172.21.15.111:0/4086975998 wait complete. 2024-09-15T22:49:19.954 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:19 smithi111 ceph-mon[30629]: pgmap v452: 97 pgs: 97 active+clean; 99 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 186 op/s 2024-09-15T22:49:20.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:19 smithi192 ceph-mon[35427]: pgmap v452: 97 pgs: 97 active+clean; 99 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 186 op/s 2024-09-15T22:49:20.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:19 smithi121 ceph-mon[34815]: pgmap v452: 97 pgs: 97 active+clean; 99 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 186 op/s 2024-09-15T22:49:20.249 INFO:tasks.workunit.client.0.smithi111.stdout: 1 222884 11.29 MB/sec warmup 108 sec latency 417.267 ms 2024-09-15T22:49:21.250 INFO:tasks.workunit.client.0.smithi111.stdout: 1 222887 11.18 MB/sec warmup 109 sec latency 937.973 ms 2024-09-15T22:49:21.953 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:21 smithi111 ceph-mon[30629]: pgmap v453: 97 pgs: 97 active+clean; 99 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 186 op/s 2024-09-15T22:49:22.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:21 smithi192 ceph-mon[35427]: pgmap v453: 97 pgs: 97 active+clean; 99 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 186 op/s 2024-09-15T22:49:22.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:21 smithi121 ceph-mon[34815]: pgmap v453: 97 pgs: 97 active+clean; 99 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 186 op/s 2024-09-15T22:49:22.250 INFO:tasks.workunit.client.0.smithi111.stdout: 1 222887 11.08 MB/sec warmup 110 sec latency 1479.348 ms 2024-09-15T22:49:23.250 INFO:tasks.workunit.client.0.smithi111.stdout: 1 223272 11.00 MB/sec warmup 111 sec latency 2370.410 ms 2024-09-15T22:49:24.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:23 smithi192 ceph-mon[35427]: pgmap v454: 97 pgs: 97 active+clean; 107 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 240 op/s 2024-09-15T22:49:24.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:23 smithi121 ceph-mon[34815]: pgmap v454: 97 pgs: 97 active+clean; 107 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 240 op/s 2024-09-15T22:49:24.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:23 smithi111 ceph-mon[30629]: pgmap v454: 97 pgs: 97 active+clean; 107 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 240 op/s 2024-09-15T22:49:24.250 INFO:tasks.workunit.client.0.smithi111.stdout: 1 225931 11.01 MB/sec warmup 112 sec latency 17.078 ms 2024-09-15T22:49:25.250 INFO:tasks.workunit.client.0.smithi111.stdout: 1 227635 11.01 MB/sec warmup 113 sec latency 271.283 ms 2024-09-15T22:49:25.813 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:25 smithi111 ceph-mon[30629]: pgmap v455: 97 pgs: 97 active+clean; 109 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 176 op/s 2024-09-15T22:49:26.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:25 smithi192 ceph-mon[35427]: pgmap v455: 97 pgs: 97 active+clean; 109 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 176 op/s 2024-09-15T22:49:26.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:25 smithi121 ceph-mon[34815]: pgmap v455: 97 pgs: 97 active+clean; 109 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 176 op/s 2024-09-15T22:49:26.250 INFO:tasks.workunit.client.0.smithi111.stdout: 1 227637 10.92 MB/sec warmup 114 sec latency 863.955 ms 2024-09-15T22:49:27.250 INFO:tasks.workunit.client.0.smithi111.stdout: 1 229824 10.92 MB/sec warmup 115 sec latency 1162.388 ms 2024-09-15T22:49:28.013 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:27 smithi111 ceph-mon[30629]: pgmap v456: 97 pgs: 97 active+clean; 106 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 172 op/s 2024-09-15T22:49:28.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:27 smithi192 ceph-mon[35427]: pgmap v456: 97 pgs: 97 active+clean; 106 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 172 op/s 2024-09-15T22:49:28.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:27 smithi121 ceph-mon[34815]: pgmap v456: 97 pgs: 97 active+clean; 106 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 172 op/s 2024-09-15T22:49:28.250 INFO:tasks.workunit.client.0.smithi111.stdout: 1 232057 10.94 MB/sec warmup 116 sec latency 85.265 ms 2024-09-15T22:49:29.252 INFO:tasks.workunit.client.0.smithi111.stdout: 1 234864 10.97 MB/sec warmup 117 sec latency 14.662 ms 2024-09-15T22:49:30.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:29 smithi192 ceph-mon[35427]: pgmap v457: 97 pgs: 97 active+clean; 103 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 158 op/s 2024-09-15T22:49:30.099 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:29 smithi111 ceph-mon[30629]: pgmap v457: 97 pgs: 97 active+clean; 103 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 158 op/s 2024-09-15T22:49:30.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:29 smithi121 ceph-mon[34815]: pgmap v457: 97 pgs: 97 active+clean; 103 MiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 158 op/s 2024-09-15T22:49:30.252 INFO:tasks.workunit.client.0.smithi111.stdout: 1 237781 11.02 MB/sec warmup 118 sec latency 144.002 ms 2024-09-15T22:49:31.253 INFO:tasks.workunit.client.0.smithi111.stdout: 1 237781 10.93 MB/sec warmup 119 sec latency 1144.073 ms 2024-09-15T22:49:31.810 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:31 smithi111 ceph-mon[30629]: pgmap v458: 97 pgs: 97 active+clean; 106 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 186 op/s 2024-09-15T22:49:31.810 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:31 smithi111 ceph-mon[30629]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:49:32.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:31 smithi192 ceph-mon[35427]: pgmap v458: 97 pgs: 97 active+clean; 106 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 186 op/s 2024-09-15T22:49:32.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:31 smithi192 ceph-mon[35427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:49:32.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:31 smithi121 ceph-mon[34815]: pgmap v458: 97 pgs: 97 active+clean; 106 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 186 op/s 2024-09-15T22:49:32.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:31 smithi121 ceph-mon[34815]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:49:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.800+0000 7fc7e6000640 1 -- 172.21.15.111:0/3628799025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7e0100b50 msgr2=0x7fc7e0100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.800+0000 7fc7e6000640 1 --2- 172.21.15.111:0/3628799025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7e0100b50 0x7fc7e0100fb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc009900 tx=0x7fc7cc02a3f0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.800+0000 7fc7e6000640 1 -- 172.21.15.111:0/3628799025 shutdown_connections 2024-09-15T22:49:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.800+0000 7fc7e6000640 1 --2- 172.21.15.111:0/3628799025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7e01014f0 0x7fc7e01019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.800+0000 7fc7e6000640 1 --2- 172.21.15.111:0/3628799025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7e0100b50 0x7fc7e0100fb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.800+0000 7fc7e6000640 1 --2- 172.21.15.111:0/3628799025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7e0100210 0x7fc7e0100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.800+0000 7fc7e6000640 1 -- 172.21.15.111:0/3628799025 >> 172.21.15.111:0/3628799025 conn(0x7fc7e00fbfa0 msgr2=0x7fc7e00fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.801+0000 7fc7e6000640 1 -- 172.21.15.111:0/3628799025 shutdown_connections 2024-09-15T22:49:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.801+0000 7fc7e6000640 1 -- 172.21.15.111:0/3628799025 wait complete. 2024-09-15T22:49:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.802+0000 7fc7e6000640 1 Processor -- start 2024-09-15T22:49:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.802+0000 7fc7e6000640 1 -- start start 2024-09-15T22:49:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.802+0000 7fc7e6000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7e0100210 0x7fc7e0199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.802+0000 7fc7e6000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7e0100b50 0x7fc7e019a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.802+0000 7fc7e6000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7e01014f0 0x7fc7e01a1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.802+0000 7fc7e6000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e010f730 con 0x7fc7e0100210 2024-09-15T22:49:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.802+0000 7fc7e6000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e010f5b0 con 0x7fc7e01014f0 2024-09-15T22:49:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.802+0000 7fc7e6000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e010f8b0 con 0x7fc7e0100b50 2024-09-15T22:49:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.802+0000 7fc7dfe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7e0100b50 0x7fc7e019a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.802+0000 7fc7e5600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7e01014f0 0x7fc7e01a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.802+0000 7fc7dfe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7e0100b50 0x7fc7e019a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:48190/0 (socket says 172.21.15.111:48190) 2024-09-15T22:49:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.802+0000 7fc7dfe00640 1 -- 172.21.15.111:0/3611454670 learned_addr learned my addr 172.21.15.111:0/3611454670 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:49:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.802+0000 7fc7e4c00640 1 --2- 172.21.15.111:0/3611454670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7e0100210 0x7fc7e0199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.803+0000 7fc7e5600640 1 -- 172.21.15.111:0/3611454670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7e0100b50 msgr2=0x7fc7e019a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.803+0000 7fc7e5600640 1 --2- 172.21.15.111:0/3611454670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7e0100b50 0x7fc7e019a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.803+0000 7fc7e5600640 1 -- 172.21.15.111:0/3611454670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7e0100210 msgr2=0x7fc7e0199e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.803+0000 7fc7e5600640 1 --2- 172.21.15.111:0/3611454670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7e0100210 0x7fc7e0199e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.803+0000 7fc7e5600640 1 -- 172.21.15.111:0/3611454670 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e01a1aa0 con 0x7fc7e01014f0 2024-09-15T22:49:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.803+0000 7fc7dfe00640 1 --2- 172.21.15.111:0/3611454670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7e0100b50 0x7fc7e019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:49:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.803+0000 7fc7e5600640 1 --2- 172.21.15.111:0/3611454670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7e01014f0 0x7fc7e01a1430 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8009fd0 tx=0x7fc7d800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.803+0000 7fc7dd600640 1 -- 172.21.15.111:0/3611454670 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d800ce20 con 0x7fc7e01014f0 2024-09-15T22:49:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.803+0000 7fc7dd600640 1 -- 172.21.15.111:0/3611454670 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7d8005020 con 0x7fc7e01014f0 2024-09-15T22:49:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.803+0000 7fc7e6000640 1 -- 172.21.15.111:0/3611454670 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e01a1d90 con 0x7fc7e01014f0 2024-09-15T22:49:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.803+0000 7fc7dd600640 1 -- 172.21.15.111:0/3611454670 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8010720 con 0x7fc7e01014f0 2024-09-15T22:49:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.803+0000 7fc7e6000640 1 -- 172.21.15.111:0/3611454670 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e01a2250 con 0x7fc7e01014f0 2024-09-15T22:49:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.807+0000 7fc7ea631740 1 Processor -- start 2024-09-15T22:49:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.807+0000 7fc7dd600640 1 -- 172.21.15.111:0/3611454670 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fc7d8005540 con 0x7fc7e01014f0 2024-09-15T22:49:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.807+0000 7fc7ea631740 1 -- start start 2024-09-15T22:49:32.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.807+0000 7fc7ea631740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f4cb52260 0x556f4cb52660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.807+0000 7fc7ea631740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f4cb52ba0 0x556f4cb57660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.807+0000 7fc7ea631740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4cb57c30 0x556f4cb5a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.807+0000 7fc7ea631740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556f4cab7100 con 0x556f4cb52ba0 2024-09-15T22:49:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.807+0000 7fc7ea631740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556f4ca5f1b0 con 0x556f4cb52260 2024-09-15T22:49:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.807+0000 7fc7ea631740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556f4cab30c0 con 0x556f4cb57c30 2024-09-15T22:49:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.808+0000 7fc7dfe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f4cb52260 0x556f4cb52660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.808+0000 7fc7dfe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f4cb52260 0x556f4cb52660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:57028/0 (socket says 172.21.15.111:57028) 2024-09-15T22:49:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.808+0000 7fc7e4c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f4cb52ba0 0x556f4cb57660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.808+0000 7fc7dfe00640 1 -- 172.21.15.111:0/2528781402 learned_addr learned my addr 172.21.15.111:0/2528781402 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:49:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.808+0000 7fc7dfe00640 1 --2- 172.21.15.111:0/2528781402 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4cb57c30 0x556f4cb5a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.808+0000 7fc7dfe00640 1 -- 172.21.15.111:0/2528781402 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4cb57c30 msgr2=0x556f4cb5a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.808+0000 7fc7dfe00640 1 --2- 172.21.15.111:0/2528781402 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4cb57c30 0x556f4cb5a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.808+0000 7fc7dfe00640 1 -- 172.21.15.111:0/2528781402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f4cb52ba0 msgr2=0x556f4cb57660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.808+0000 7fc7dfe00640 1 --2- 172.21.15.111:0/2528781402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f4cb52ba0 0x556f4cb57660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.808+0000 7fc7dfe00640 1 -- 172.21.15.111:0/2528781402 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f4c7ca210 con 0x556f4cb52260 2024-09-15T22:49:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.808+0000 7fc7dfe00640 1 --2- 172.21.15.111:0/2528781402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f4cb52260 0x556f4cb52660 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc005ec0 tx=0x7fc7cc005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9618042f2d17e9bc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.808+0000 7fc7dd600640 1 --2- 172.21.15.111:0/3611454670 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc7c80a32c0 0x7fc7c80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.808+0000 7fc7d1c00640 1 -- 172.21.15.111:0/2528781402 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc035070 con 0x556f4cb52260 2024-09-15T22:49:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.809+0000 7fc7d1c00640 1 -- 172.21.15.111:0/2528781402 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7cc0316b0 con 0x556f4cb52260 2024-09-15T22:49:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.809+0000 7fc7d1c00640 1 -- 172.21.15.111:0/2528781402 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc005520 con 0x556f4cb52260 2024-09-15T22:49:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.809+0000 7fc7dd600640 1 -- 172.21.15.111:0/3611454670 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fc7d80cfed0 con 0x7fc7e01014f0 2024-09-15T22:49:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.809+0000 7fc7e4c00640 1 --2- 172.21.15.111:0/3611454670 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc7c80a32c0 0x7fc7c80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.809+0000 7fc7e4c00640 1 --2- 172.21.15.111:0/3611454670 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc7c80a32c0 0x7fc7c80a5780 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x556f4cb535d0 tx=0x7fc7d4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.809+0000 7fc7ea631740 1 -- 172.21.15.111:0/2528781402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f4cb52260 msgr2=0x556f4cb52660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.809+0000 7fc7ea631740 1 --2- 172.21.15.111:0/2528781402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f4cb52260 0x556f4cb52660 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc005ec0 tx=0x7fc7cc005b50 comp rx=0 tx=0).stop 2024-09-15T22:49:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.810+0000 7fc7ea631740 1 -- 172.21.15.111:0/2528781402 shutdown_connections 2024-09-15T22:49:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.810+0000 7fc7ea631740 1 --2- 172.21.15.111:0/2528781402 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4cb57c30 0x556f4cb5a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.810+0000 7fc7ea631740 1 --2- 172.21.15.111:0/2528781402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f4cb52ba0 0x556f4cb57660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.810+0000 7fc7ea631740 1 --2- 172.21.15.111:0/2528781402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f4cb52260 0x556f4cb52660 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.810+0000 7fc7ea631740 1 -- 172.21.15.111:0/2528781402 >> 172.21.15.111:0/2528781402 conn(0x556f4cb4d8d0 msgr2=0x556f4cb4dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.810+0000 7fc7ea631740 1 -- 172.21.15.111:0/2528781402 shutdown_connections 2024-09-15T22:49:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.810+0000 7fc7ea631740 1 -- 172.21.15.111:0/2528781402 wait complete. 2024-09-15T22:49:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.810+0000 7fc7ea631740 1 -- start start 2024-09-15T22:49:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.810+0000 7fc7ea631740 1 Processor -- start 2024-09-15T22:49:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.811+0000 7fc7ea631740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4cb52260 0x556f4cb5c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.811+0000 7fc7ea631740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f4cb52ba0 0x556f4cb5c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.811+0000 7fc7ea631740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f4cb57c30 0x556f4cb639a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.811+0000 7fc7ea631740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556f4cb5ac00 con 0x556f4cb57c30 2024-09-15T22:49:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.811+0000 7fc7ea631740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556f4ca40090 con 0x556f4cb52ba0 2024-09-15T22:49:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.811+0000 7fc7ea631740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556f4cb5ad80 con 0x556f4cb52260 2024-09-15T22:49:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.811+0000 7fc7ea631740 10 client.0 _ll_register_callbacks cb 0x556f4ca94f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:49:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.811+0000 7fc7e4c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4cb52260 0x556f4cb5c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.811+0000 7fc7e4c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4cb52260 0x556f4cb5c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:48202/0 (socket says 172.21.15.111:48202) 2024-09-15T22:49:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.811+0000 7fc7e4c00640 1 -- 172.21.15.111:0/1567191880 learned_addr learned my addr 172.21.15.111:0/1567191880 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:49:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.811+0000 7fc7dfe00640 1 --2- 172.21.15.111:0/1567191880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f4cb52ba0 0x556f4cb5c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.811+0000 7fc7e5600640 1 --2- 172.21.15.111:0/1567191880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f4cb57c30 0x556f4cb639a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.811+0000 7fc7e4c00640 1 -- 172.21.15.111:0/1567191880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f4cb52ba0 msgr2=0x556f4cb5c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.811+0000 7fc7e4c00640 1 --2- 172.21.15.111:0/1567191880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f4cb52ba0 0x556f4cb5c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.812+0000 7fc7e4c00640 1 -- 172.21.15.111:0/1567191880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f4cb57c30 msgr2=0x556f4cb639a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.812+0000 7fc7e4c00640 1 --2- 172.21.15.111:0/1567191880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f4cb57c30 0x556f4cb639a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.812+0000 7fc7e4c00640 1 -- 172.21.15.111:0/1567191880 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f4c896620 con 0x556f4cb52260 2024-09-15T22:49:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.812+0000 7fc7e5600640 1 --2- 172.21.15.111:0/1567191880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f4cb57c30 0x556f4cb639a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:49:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.812+0000 7fc7e4c00640 1 --2- 172.21.15.111:0/1567191880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4cb52260 0x556f4cb5c420 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc7d40045f0 tx=0x7fc7d4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0faa7086bf064af server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.812+0000 7fc7c6000640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:49:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.812+0000 7fc7c6000640 1 -- 172.21.15.111:0/1567191880 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d401f640 con 0x556f4cb52260 2024-09-15T22:49:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.812+0000 7fc7ea631740 1 -- 172.21.15.111:0/1567191880 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7cc0316b0 con 0x556f4cb52260 2024-09-15T22:49:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.812+0000 7fc7c6000640 1 -- 172.21.15.111:0/1567191880 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fc7d401f7e0 con 0x556f4cb52260 2024-09-15T22:49:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.813+0000 7fc7c6000640 1 -- 172.21.15.111:0/1567191880 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc7d401f9e0 con 0x556f4cb52260 2024-09-15T22:49:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.813+0000 7fc7ea631740 10 client.25300 fetch_fsmap learned FSMap version 29 2024-09-15T22:49:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.813+0000 7fc7ea631740 1 -- 172.21.15.111:0/1567191880 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556f4cb668b0 con 0x556f4cb52260 2024-09-15T22:49:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.813+0000 7fc7c6000640 1 -- 172.21.15.111:0/1567191880 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7cc0316b0 con 0x556f4cb52260 2024-09-15T22:49:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.813+0000 7fc7c6000640 1 -- 172.21.15.111:0/1567191880 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7fc7d4027070 con 0x556f4cb52260 2024-09-15T22:49:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.813+0000 7fc7ea631740 10 client.25300 fetch_fsmap finished waiting for FSMap version 29 2024-09-15T22:49:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.813+0000 7fc7ea631740 10 client.25300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:49:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.813+0000 7fc7ea631740 20 client.25300 populate_metadata read hostname 'smithi111' 2024-09-15T22:49:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.813+0000 7fc7ea631740 1 --2- 172.21.15.111:0/1567191880 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556f4cb67f30 0x556f4cb6a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.813+0000 7fc7ea631740 4 client.25300 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:49:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.813+0000 7fc7ea631740 1 -- 172.21.15.111:0/1567191880 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f4cac4830 con 0x556f4cb67f30 2024-09-15T22:49:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.814+0000 7fc7dfe00640 1 --2- 172.21.15.111:0/1567191880 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556f4cb67f30 0x556f4cb6a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.814+0000 7fc7dfe00640 1 --2- 172.21.15.111:0/1567191880 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556f4cb67f30 0x556f4cb6a3a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4466006c77b4f19f server_cookie=163459b135081a3d in_seq=0 out_seq=0 2024-09-15T22:49:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.814+0000 7fc7c6000640 10 client.25300 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:49:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.815+0000 7fc7c6000640 1 -- 172.21.15.111:0/1567191880 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556f4cac4830 con 0x556f4cb67f30 2024-09-15T22:49:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.815+0000 7fc7c6000640 10 client.25300 handle_command_reply: tid=0 2024-09-15T22:49:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.815+0000 7fc7ea631740 1 client.25300 shutdown 2024-09-15T22:49:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.815+0000 7fc7ea631740 1 -- 172.21.15.111:0/1567191880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4cb52260 msgr2=0x556f4cb5c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.815+0000 7fc7ea631740 1 --2- 172.21.15.111:0/1567191880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4cb52260 0x556f4cb5c420 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc7d40045f0 tx=0x7fc7d4004730 comp rx=0 tx=0).stop 2024-09-15T22:49:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.815+0000 7fc7ea631740 1 -- 172.21.15.111:0/1567191880 shutdown_connections 2024-09-15T22:49:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.815+0000 7fc7ea631740 1 --2- 172.21.15.111:0/1567191880 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556f4cb67f30 0x556f4cb6a3a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.815+0000 7fc7ea631740 1 --2- 172.21.15.111:0/1567191880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f4cb57c30 0x556f4cb639a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.815+0000 7fc7ea631740 1 --2- 172.21.15.111:0/1567191880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f4cb52ba0 0x556f4cb5c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.815+0000 7fc7ea631740 1 --2- 172.21.15.111:0/1567191880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4cb52260 0x556f4cb5c420 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.815+0000 7fc7ea631740 1 -- 172.21.15.111:0/1567191880 >> 172.21.15.111:0/1567191880 conn(0x556f4cb4d8d0 msgr2=0x556f4cb4f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.815+0000 7fc7c6000640 0 client.25300 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:49:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.815+0000 7fc7ea631740 1 -- 172.21.15.111:0/1567191880 shutdown_connections 2024-09-15T22:49:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.815+0000 7fc7ea631740 1 -- 172.21.15.111:0/1567191880 wait complete. 2024-09-15T22:49:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.816+0000 7fc7ea631740 20 client.25300 trim_cache size 0 max 16384 2024-09-15T22:49:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.832+0000 7fc7ea631740 1 Processor -- start 2024-09-15T22:49:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.833+0000 7fc7ea631740 1 -- start start 2024-09-15T22:49:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.833+0000 7fc7ea631740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f4cb52260 0x556f4cb5c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.833+0000 7fc7ea631740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4cb52ba0 0x556f4cb488d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.833+0000 7fc7ea631740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f4cb57c30 0x556f4cb63960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.833+0000 7fc7ea631740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e01ad710 con 0x556f4cb52260 2024-09-15T22:49:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.833+0000 7fc7ea631740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e01aafe0 con 0x556f4cb57c30 2024-09-15T22:49:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.833+0000 7fc7ea631740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e01a8e30 con 0x556f4cb52ba0 2024-09-15T22:49:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.833+0000 7fc7e4c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f4cb52260 0x556f4cb5c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.833+0000 7fc7dfe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4cb52ba0 0x556f4cb488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.833+0000 7fc7e5600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f4cb57c30 0x556f4cb63960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.833+0000 7fc7dfe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4cb52ba0 0x556f4cb488d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:48216/0 (socket says 172.21.15.111:48216) 2024-09-15T22:49:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.833+0000 7fc7dfe00640 1 -- 172.21.15.111:0/1397610794 learned_addr learned my addr 172.21.15.111:0/1397610794 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:49:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.833+0000 7fc7e5600640 1 --2- 172.21.15.111:0/1397610794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f4cb57c30 0x556f4cb63960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:57046/0 (socket says 172.21.15.111:57046) 2024-09-15T22:49:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.834+0000 7fc7e5600640 1 -- 172.21.15.111:0/1397610794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4cb52ba0 msgr2=0x556f4cb488d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.834+0000 7fc7e5600640 1 --2- 172.21.15.111:0/1397610794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4cb52ba0 0x556f4cb488d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.834+0000 7fc7e5600640 1 -- 172.21.15.111:0/1397610794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f4cb52260 msgr2=0x556f4cb5c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.834+0000 7fc7e5600640 1 --2- 172.21.15.111:0/1397610794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f4cb52260 0x556f4cb5c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.834+0000 7fc7e5600640 1 -- 172.21.15.111:0/1397610794 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f4cb53080 con 0x556f4cb57c30 2024-09-15T22:49:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.834+0000 7fc7e5600640 1 --2- 172.21.15.111:0/1397610794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f4cb57c30 0x556f4cb63960 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc7d80cdac0 tx=0x7fc7d809cc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=244cbc59c290e6e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.834+0000 7fc7d1200640 1 -- 172.21.15.111:0/1397610794 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d80b4070 con 0x556f4cb57c30 2024-09-15T22:49:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.834+0000 7fc7d1200640 1 -- 172.21.15.111:0/1397610794 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7d80a5200 con 0x556f4cb57c30 2024-09-15T22:49:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.834+0000 7fc7d1200640 1 -- 172.21.15.111:0/1397610794 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d80a4d70 con 0x556f4cb57c30 2024-09-15T22:49:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.835+0000 7fc7ea631740 1 -- 172.21.15.111:0/1397610794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f4cb57c30 msgr2=0x556f4cb63960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.835+0000 7fc7ea631740 1 --2- 172.21.15.111:0/1397610794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f4cb57c30 0x556f4cb63960 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc7d80cdac0 tx=0x7fc7d809cc90 comp rx=0 tx=0).stop 2024-09-15T22:49:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.835+0000 7fc7ea631740 1 -- 172.21.15.111:0/1397610794 shutdown_connections 2024-09-15T22:49:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.835+0000 7fc7ea631740 1 --2- 172.21.15.111:0/1397610794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f4cb57c30 0x556f4cb63960 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.835+0000 7fc7ea631740 1 --2- 172.21.15.111:0/1397610794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4cb52ba0 0x556f4cb488d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.835+0000 7fc7ea631740 1 --2- 172.21.15.111:0/1397610794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f4cb52260 0x556f4cb5c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.835+0000 7fc7ea631740 1 -- 172.21.15.111:0/1397610794 >> 172.21.15.111:0/1397610794 conn(0x556f4cb4d8d0 msgr2=0x556f4cb4c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.835+0000 7fc7ea631740 1 -- 172.21.15.111:0/1397610794 shutdown_connections 2024-09-15T22:49:32.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.835+0000 7fc7ea631740 1 -- 172.21.15.111:0/1397610794 wait complete. 2024-09-15T22:49:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.835+0000 7fc7ea631740 1 -- start start 2024-09-15T22:49:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.836+0000 7fc7ea631740 1 Processor -- start 2024-09-15T22:49:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.836+0000 7fc7ea631740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f4cb52260 0x556f4cb61290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.836+0000 7fc7ea631740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f4cb52ba0 0x556f4cb617d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.836+0000 7fc7ea631740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4cb57c30 0x556f4cb627e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.836+0000 7fc7ea631740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e01a94a0 con 0x556f4cb52260 2024-09-15T22:49:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.836+0000 7fc7ea631740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e01aa910 con 0x556f4cb52ba0 2024-09-15T22:49:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.836+0000 7fc7ea631740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e01a9b70 con 0x556f4cb57c30 2024-09-15T22:49:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.836+0000 7fc7ea631740 10 client.0 _ll_register_callbacks cb 0x7fc7d4009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:49:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.836+0000 7fc7e4c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f4cb52260 0x556f4cb61290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.836+0000 7fc7e5600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4cb57c30 0x556f4cb627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.836+0000 7fc7dfe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f4cb52ba0 0x556f4cb617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.836+0000 7fc7e4c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f4cb52260 0x556f4cb61290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43600/0 (socket says 172.21.15.111:43600) 2024-09-15T22:49:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.836+0000 7fc7e4c00640 1 -- 172.21.15.111:0/3801033770 learned_addr learned my addr 172.21.15.111:0/3801033770 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:49:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.836+0000 7fc7e5600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4cb57c30 0x556f4cb627e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:48218/0 (socket says 172.21.15.111:48218) 2024-09-15T22:49:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.837+0000 7fc7dfe00640 1 -- 172.21.15.111:0/3801033770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4cb57c30 msgr2=0x556f4cb627e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.837+0000 7fc7dfe00640 1 --2- 172.21.15.111:0/3801033770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4cb57c30 0x556f4cb627e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.837+0000 7fc7dfe00640 1 -- 172.21.15.111:0/3801033770 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f4cb52260 msgr2=0x556f4cb61290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.837+0000 7fc7dfe00640 1 --2- 172.21.15.111:0/3801033770 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f4cb52260 0x556f4cb61290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.837+0000 7fc7dfe00640 1 -- 172.21.15.111:0/3801033770 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f4cb62df0 con 0x556f4cb52ba0 2024-09-15T22:49:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.837+0000 7fc7e5600640 1 --2- 172.21.15.111:0/3801033770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4cb57c30 0x556f4cb627e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:49:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.837+0000 7fc7e4c00640 1 --2- 172.21.15.111:0/3801033770 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f4cb52260 0x556f4cb61290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:49:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.837+0000 7fc7dfe00640 1 --2- 172.21.15.111:0/3801033770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f4cb52ba0 0x556f4cb617d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc009850 tx=0x7fc7cc032b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=770bfcd58bda1892 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.837+0000 7fc7c6000640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:49:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.837+0000 7fc7c6000640 1 -- 172.21.15.111:0/3801033770 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc035040 con 0x556f4cb52ba0 2024-09-15T22:49:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.837+0000 7fc7ea631740 1 -- 172.21.15.111:0/3801033770 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7d80a5200 con 0x556f4cb52ba0 2024-09-15T22:49:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.837+0000 7fc7c6000640 1 -- 172.21.15.111:0/3801033770 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fc7cc005460 con 0x556f4cb52ba0 2024-09-15T22:49:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.837+0000 7fc7c6000640 1 -- 172.21.15.111:0/3801033770 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc7cc06c050 con 0x556f4cb52ba0 2024-09-15T22:49:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.837+0000 7fc7ea631740 10 client.25307 fetch_fsmap learned FSMap version 29 2024-09-15T22:49:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.837+0000 7fc7ea631740 1 -- 172.21.15.111:0/3801033770 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556f4cbd2e30 con 0x556f4cb52ba0 2024-09-15T22:49:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.838+0000 7fc7c6000640 1 -- 172.21.15.111:0/3801033770 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7d80a5200 con 0x556f4cb52ba0 2024-09-15T22:49:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.838+0000 7fc7c6000640 1 -- 172.21.15.111:0/3801033770 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7fc7cc05c770 con 0x556f4cb52ba0 2024-09-15T22:49:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.838+0000 7fc7ea631740 10 client.25307 fetch_fsmap finished waiting for FSMap version 29 2024-09-15T22:49:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.838+0000 7fc7ea631740 10 client.25307 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:49:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.838+0000 7fc7ea631740 20 client.25307 populate_metadata read hostname 'smithi111' 2024-09-15T22:49:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.838+0000 7fc7ea631740 1 --2- 172.21.15.111:0/3801033770 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556f4cbd32e0 0x556f4cbd56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.838+0000 7fc7ea631740 4 client.25307 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:49:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.838+0000 7fc7ea631740 1 -- 172.21.15.111:0/3801033770 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556f4cabeb80 con 0x556f4cbd32e0 2024-09-15T22:49:32.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.838+0000 7fc7e4c00640 1 --2- 172.21.15.111:0/3801033770 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556f4cbd32e0 0x556f4cbd56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.839+0000 7fc7e4c00640 1 --2- 172.21.15.111:0/3801033770 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556f4cbd32e0 0x556f4cbd56d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab74cd6a54e39d59 server_cookie=ada8bff68d056013 in_seq=0 out_seq=0 2024-09-15T22:49:32.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.839+0000 7fc7c6000640 10 client.25307 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:49:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.846+0000 7fc7c6000640 1 -- 172.21.15.111:0/3801033770 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556f4cabeb80 con 0x556f4cbd32e0 2024-09-15T22:49:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.846+0000 7fc7c6000640 10 client.25307 handle_command_reply: tid=0 2024-09-15T22:49:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.846+0000 7fc7ea631740 1 client.25307 shutdown 2024-09-15T22:49:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.846+0000 7fc7ea631740 1 -- 172.21.15.111:0/3801033770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f4cb52ba0 msgr2=0x556f4cb617d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.846+0000 7fc7ea631740 1 --2- 172.21.15.111:0/3801033770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f4cb52ba0 0x556f4cb617d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc009850 tx=0x7fc7cc032b20 comp rx=0 tx=0).stop 2024-09-15T22:49:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.846+0000 7fc7ea631740 1 -- 172.21.15.111:0/3801033770 shutdown_connections 2024-09-15T22:49:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.846+0000 7fc7ea631740 1 --2- 172.21.15.111:0/3801033770 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556f4cbd32e0 0x556f4cbd56d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.846+0000 7fc7ea631740 1 --2- 172.21.15.111:0/3801033770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f4cb57c30 0x556f4cb627e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.846+0000 7fc7ea631740 1 --2- 172.21.15.111:0/3801033770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f4cb52ba0 0x556f4cb617d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.846+0000 7fc7c6000640 0 client.25307 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:49:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.846+0000 7fc7ea631740 1 --2- 172.21.15.111:0/3801033770 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f4cb52260 0x556f4cb61290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.846+0000 7fc7ea631740 1 -- 172.21.15.111:0/3801033770 >> 172.21.15.111:0/3801033770 conn(0x556f4cb4d8d0 msgr2=0x556f4cb4ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.846+0000 7fc7ea631740 1 -- 172.21.15.111:0/3801033770 shutdown_connections 2024-09-15T22:49:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.846+0000 7fc7ea631740 1 -- 172.21.15.111:0/3801033770 wait complete. 2024-09-15T22:49:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.846+0000 7fc7ea631740 20 client.25307 trim_cache size 0 max 16384 2024-09-15T22:49:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:49:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:49:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:49:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:49:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.847+0000 7fc7e6000640 1 -- 172.21.15.111:0/3611454670 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc7c80a32c0 msgr2=0x7fc7c80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.847+0000 7fc7e6000640 1 --2- 172.21.15.111:0/3611454670 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc7c80a32c0 0x7fc7c80a5780 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x556f4cb535d0 tx=0x7fc7d4005f90 comp rx=0 tx=0).stop 2024-09-15T22:49:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.847+0000 7fc7e6000640 1 -- 172.21.15.111:0/3611454670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7e01014f0 msgr2=0x7fc7e01a1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.847+0000 7fc7e6000640 1 --2- 172.21.15.111:0/3611454670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7e01014f0 0x7fc7e01a1430 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8009fd0 tx=0x7fc7d800ef10 comp rx=0 tx=0).stop 2024-09-15T22:49:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.847+0000 7fc7e6000640 1 -- 172.21.15.111:0/3611454670 shutdown_connections 2024-09-15T22:49:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.847+0000 7fc7e6000640 1 --2- 172.21.15.111:0/3611454670 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc7c80a32c0 0x7fc7c80a5780 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.847+0000 7fc7e6000640 1 --2- 172.21.15.111:0/3611454670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7e01014f0 0x7fc7e01a1430 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.847+0000 7fc7e6000640 1 --2- 172.21.15.111:0/3611454670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7e0100b50 0x7fc7e019a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.847+0000 7fc7e6000640 1 --2- 172.21.15.111:0/3611454670 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc7e0100210 0x7fc7e0199e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.847+0000 7fc7e6000640 1 -- 172.21.15.111:0/3611454670 >> 172.21.15.111:0/3611454670 conn(0x7fc7e00fbfa0 msgr2=0x7fc7e00fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.848+0000 7fc7e6000640 1 -- 172.21.15.111:0/3611454670 shutdown_connections 2024-09-15T22:49:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:32.848+0000 7fc7e6000640 1 -- 172.21.15.111:0/3611454670 wait complete. 2024-09-15T22:49:33.252 INFO:tasks.workunit.client.0.smithi111.stdout: 1 240186 9.20 MB/sec execute 1 sec latency 2155.760 ms 2024-09-15T22:49:34.013 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:33 smithi111 ceph-mon[30629]: pgmap v459: 97 pgs: 97 active+clean; 109 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 216 op/s 2024-09-15T22:49:34.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:33 smithi192 ceph-mon[35427]: pgmap v459: 97 pgs: 97 active+clean; 109 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 216 op/s 2024-09-15T22:49:34.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:33 smithi121 ceph-mon[34815]: pgmap v459: 97 pgs: 97 active+clean; 109 MiB data, 993 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 216 op/s 2024-09-15T22:49:34.253 INFO:tasks.workunit.client.0.smithi111.stdout: 1 242706 12.82 MB/sec execute 2 sec latency 13.738 ms 2024-09-15T22:49:35.253 INFO:tasks.workunit.client.0.smithi111.stdout: 1 245344 13.23 MB/sec execute 3 sec latency 57.135 ms 2024-09-15T22:49:35.748 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:35 smithi111 ceph-mon[30629]: pgmap v460: 97 pgs: 97 active+clean; 110 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 170 op/s 2024-09-15T22:49:36.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:35 smithi192 ceph-mon[35427]: pgmap v460: 97 pgs: 97 active+clean; 110 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 170 op/s 2024-09-15T22:49:36.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:35 smithi121 ceph-mon[34815]: pgmap v460: 97 pgs: 97 active+clean; 110 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 170 op/s 2024-09-15T22:49:36.253 INFO:tasks.workunit.client.0.smithi111.stdout: 1 245347 9.92 MB/sec execute 4 sec latency 862.540 ms 2024-09-15T22:49:37.253 INFO:tasks.workunit.client.0.smithi111.stdout: 1 245854 8.32 MB/sec execute 5 sec latency 960.379 ms 2024-09-15T22:49:38.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:37 smithi192 ceph-mon[35427]: pgmap v461: 97 pgs: 97 active+clean; 113 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 198 op/s 2024-09-15T22:49:38.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:37 smithi121 ceph-mon[34815]: pgmap v461: 97 pgs: 97 active+clean; 113 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 198 op/s 2024-09-15T22:49:38.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:37 smithi111 ceph-mon[30629]: pgmap v461: 97 pgs: 97 active+clean; 113 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 198 op/s 2024-09-15T22:49:38.253 INFO:tasks.workunit.client.0.smithi111.stdout: 1 248807 9.80 MB/sec execute 6 sec latency 8.863 ms 2024-09-15T22:49:39.253 INFO:tasks.workunit.client.0.smithi111.stdout: 1 251953 10.92 MB/sec execute 7 sec latency 13.393 ms 2024-09-15T22:49:40.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:39 smithi192 ceph-mon[35427]: pgmap v462: 97 pgs: 97 active+clean; 110 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 205 op/s 2024-09-15T22:49:40.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:39 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:49:40.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:39 smithi121 ceph-mon[34815]: pgmap v462: 97 pgs: 97 active+clean; 110 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 205 op/s 2024-09-15T22:49:40.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:39 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:49:40.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:39 smithi111 ceph-mon[30629]: pgmap v462: 97 pgs: 97 active+clean; 110 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 205 op/s 2024-09-15T22:49:40.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:39 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:49:40.253 INFO:tasks.workunit.client.0.smithi111.stdout: 1 254645 11.12 MB/sec execute 8 sec latency 12.179 ms 2024-09-15T22:49:40.878 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:40 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:49:40.878 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:40 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:49:40.878 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:40 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:49:41.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:40 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:49:41.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:40 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:49:41.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:40 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:49:41.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:40 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:49:41.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:40 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:49:41.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:40 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:49:41.253 INFO:tasks.workunit.client.0.smithi111.stdout: 1 254860 10.05 MB/sec execute 9 sec latency 614.070 ms 2024-09-15T22:49:42.026 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:41 smithi111 ceph-mon[30629]: pgmap v463: 97 pgs: 97 active+clean; 112 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 235 op/s 2024-09-15T22:49:42.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:41 smithi192 ceph-mon[35427]: pgmap v463: 97 pgs: 97 active+clean; 112 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 235 op/s 2024-09-15T22:49:42.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:41 smithi121 ceph-mon[34815]: pgmap v463: 97 pgs: 97 active+clean; 112 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 235 op/s 2024-09-15T22:49:42.253 INFO:tasks.workunit.client.0.smithi111.stdout: 1 254860 9.05 MB/sec execute 10 sec latency 1316.247 ms 2024-09-15T22:49:43.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:42 smithi192 ceph-mon[35427]: pgmap v464: 97 pgs: 97 active+clean; 111 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 202 op/s 2024-09-15T22:49:43.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:42 smithi121 ceph-mon[34815]: pgmap v464: 97 pgs: 97 active+clean; 111 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 202 op/s 2024-09-15T22:49:43.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:42 smithi111 ceph-mon[30629]: pgmap v464: 97 pgs: 97 active+clean; 111 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 202 op/s 2024-09-15T22:49:43.253 INFO:tasks.workunit.client.0.smithi111.stdout: 1 256423 8.91 MB/sec execute 11 sec latency 1541.593 ms 2024-09-15T22:49:44.253 INFO:tasks.workunit.client.0.smithi111.stdout: 1 259395 9.68 MB/sec execute 12 sec latency 13.023 ms 2024-09-15T22:49:45.254 INFO:tasks.workunit.client.0.smithi111.stdout: 1 262723 10.38 MB/sec execute 13 sec latency 13.613 ms 2024-09-15T22:49:45.706 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:45 smithi111 ceph-mon[30629]: pgmap v465: 97 pgs: 97 active+clean; 113 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 204 op/s 2024-09-15T22:49:45.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:45 smithi121 ceph-mon[34815]: pgmap v465: 97 pgs: 97 active+clean; 113 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 204 op/s 2024-09-15T22:49:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.982+0000 7f4d1b200640 1 -- 172.21.15.111:0/3925278727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d10095680 msgr2=0x7f4d10095a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.982+0000 7f4d1b200640 1 --2- 172.21.15.111:0/3925278727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d10095680 0x7f4d10095a80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009900 tx=0x7f4d0802a410 comp rx=0 tx=0).stop 2024-09-15T22:49:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.982+0000 7f4d1b200640 1 -- 172.21.15.111:0/3925278727 shutdown_connections 2024-09-15T22:49:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.982+0000 7f4d1b200640 1 --2- 172.21.15.111:0/3925278727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d10097230 0x7f4d1009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.982+0000 7f4d1b200640 1 --2- 172.21.15.111:0/3925278727 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d10096870 0x7f4d10096cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.982+0000 7f4d1b200640 1 --2- 172.21.15.111:0/3925278727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d10095680 0x7f4d10095a80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.982+0000 7f4d1b200640 1 -- 172.21.15.111:0/3925278727 >> 172.21.15.111:0/3925278727 conn(0x7f4d10090dc0 msgr2=0x7f4d10093220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.982+0000 7f4d1b200640 1 -- 172.21.15.111:0/3925278727 shutdown_connections 2024-09-15T22:49:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.982+0000 7f4d1b200640 1 -- 172.21.15.111:0/3925278727 wait complete. 2024-09-15T22:49:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.983+0000 7f4d1b200640 1 Processor -- start 2024-09-15T22:49:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.983+0000 7f4d1b200640 1 -- start start 2024-09-15T22:49:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.984+0000 7f4d1b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d10095680 0x7f4d1012dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.984+0000 7f4d1b200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d10096870 0x7f4d1012e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.984+0000 7f4d1b200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d10097230 0x7f4d10135160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.984+0000 7f4d1b200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4d100a1350 con 0x7f4d10096870 2024-09-15T22:49:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.984+0000 7f4d1b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4d100a11d0 con 0x7f4d10095680 2024-09-15T22:49:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.984+0000 7f4d1b200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4d100a14d0 con 0x7f4d10097230 2024-09-15T22:49:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.984+0000 7f4d19400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d10096870 0x7f4d1012e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.984+0000 7f4d19e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d10095680 0x7f4d1012dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:45.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.984+0000 7f4d1a800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d10097230 0x7f4d10135160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.984+0000 7f4d19e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d10095680 0x7f4d1012dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:34106/0 (socket says 172.21.15.111:34106) 2024-09-15T22:49:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.984+0000 7f4d19e00640 1 -- 172.21.15.111:0/2846441092 learned_addr learned my addr 172.21.15.111:0/2846441092 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:49:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.984+0000 7f4d1a800640 1 -- 172.21.15.111:0/2846441092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d10095680 msgr2=0x7f4d1012dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.984+0000 7f4d1a800640 1 --2- 172.21.15.111:0/2846441092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d10095680 0x7f4d1012dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.984+0000 7f4d1a800640 1 -- 172.21.15.111:0/2846441092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d10096870 msgr2=0x7f4d1012e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.984+0000 7f4d1a800640 1 --2- 172.21.15.111:0/2846441092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d10096870 0x7f4d1012e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.984+0000 7f4d1a800640 1 -- 172.21.15.111:0/2846441092 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d101357d0 con 0x7f4d10097230 2024-09-15T22:49:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.984+0000 7f4d1a800640 1 --2- 172.21.15.111:0/2846441092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d10097230 0x7f4d10135160 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4d14050280 tx=0x7f4d14068ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.984+0000 7f4d0ea00640 1 -- 172.21.15.111:0/2846441092 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d14082070 con 0x7f4d10097230 2024-09-15T22:49:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.984+0000 7f4d1b200640 1 -- 172.21.15.111:0/2846441092 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d10135ac0 con 0x7f4d10097230 2024-09-15T22:49:45.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.984+0000 7f4d0ea00640 1 -- 172.21.15.111:0/2846441092 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d1407e900 con 0x7f4d10097230 2024-09-15T22:49:45.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.984+0000 7f4d0ea00640 1 -- 172.21.15.111:0/2846441092 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1407d830 con 0x7f4d10097230 2024-09-15T22:49:45.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.984+0000 7f4d1b200640 1 -- 172.21.15.111:0/2846441092 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d10135f90 con 0x7f4d10097230 2024-09-15T22:49:45.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.984+0000 7f4d1f66b740 1 Processor -- start 2024-09-15T22:49:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.984+0000 7f4d0ea00640 1 -- 172.21.15.111:0/2846441092 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f4d1407b040 con 0x7f4d10097230 2024-09-15T22:49:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.984+0000 7f4d1f66b740 1 -- start start 2024-09-15T22:49:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1f66b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636c4a26260 0x5636c4a26660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1f66b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5636c4a26ba0 0x5636c4a2b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1f66b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636c4a2bc30 0x5636c4a2e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1f66b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5636c498b100 con 0x5636c4a26ba0 2024-09-15T22:49:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1f66b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5636c49331b0 con 0x5636c4a2bc30 2024-09-15T22:49:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1f66b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5636c49870c0 con 0x5636c4a26260 2024-09-15T22:49:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d19400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636c4a26260 0x5636c4a26660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:45.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d19e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5636c4a26ba0 0x5636c4a2b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d19e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5636c4a26ba0 0x5636c4a2b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60826/0 (socket says 172.21.15.111:60826) 2024-09-15T22:49:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d19e00640 1 -- 172.21.15.111:0/2135799085 learned_addr learned my addr 172.21.15.111:0/2135799085 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:49:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d19400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636c4a26260 0x5636c4a26660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:34810/0 (socket says 172.21.15.111:34810) 2024-09-15T22:49:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d19400640 1 --2- 172.21.15.111:0/2135799085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636c4a2bc30 0x5636c4a2e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d19e00640 1 -- 172.21.15.111:0/2135799085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636c4a26260 msgr2=0x5636c4a26660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d19e00640 1 --2- 172.21.15.111:0/2135799085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636c4a26260 0x5636c4a26660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d19e00640 1 -- 172.21.15.111:0/2135799085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636c4a2bc30 msgr2=0x5636c4a2e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d19e00640 1 --2- 172.21.15.111:0/2135799085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636c4a2bc30 0x5636c4a2e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d19e00640 1 -- 172.21.15.111:0/2135799085 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636c469e210 con 0x5636c4a26ba0 2024-09-15T22:49:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d19e00640 1 --2- 172.21.15.111:0/2135799085 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5636c4a26ba0 0x5636c4a2b660 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4d0802a930 tx=0x7f4d080040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea80fd59caf1cb0c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d0ea00640 1 --2- 172.21.15.111:0/2846441092 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4cfc0a3250 0x7f4cfc0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:45.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d0ea00640 1 -- 172.21.15.111:0/2846441092 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f4d1413c360 con 0x7f4d10097230 2024-09-15T22:49:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d03000640 1 -- 172.21.15.111:0/2135799085 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d08040070 con 0x5636c4a26ba0 2024-09-15T22:49:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d03000640 1 -- 172.21.15.111:0/2135799085 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d08005500 con 0x5636c4a26ba0 2024-09-15T22:49:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d03000640 1 -- 172.21.15.111:0/2135799085 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d08035040 con 0x5636c4a26ba0 2024-09-15T22:49:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d19400640 1 --2- 172.21.15.111:0/2846441092 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4cfc0a3250 0x7f4cfc0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d19400640 1 --2- 172.21.15.111:0/2846441092 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4cfc0a3250 0x7f4cfc0a5710 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4d04009fd0 tx=0x7f4d040046c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1f66b740 1 -- 172.21.15.111:0/2135799085 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5636c4a26ba0 msgr2=0x5636c4a2b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1f66b740 1 --2- 172.21.15.111:0/2135799085 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5636c4a26ba0 0x5636c4a2b660 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4d0802a930 tx=0x7f4d080040c0 comp rx=0 tx=0).stop 2024-09-15T22:49:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1f66b740 1 -- 172.21.15.111:0/2135799085 shutdown_connections 2024-09-15T22:49:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1f66b740 1 --2- 172.21.15.111:0/2135799085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636c4a2bc30 0x5636c4a2e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1f66b740 1 --2- 172.21.15.111:0/2135799085 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5636c4a26ba0 0x5636c4a2b660 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1f66b740 1 --2- 172.21.15.111:0/2135799085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636c4a26260 0x5636c4a26660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1f66b740 1 -- 172.21.15.111:0/2135799085 >> 172.21.15.111:0/2135799085 conn(0x5636c4a218d0 msgr2=0x5636c4a21cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1f66b740 1 -- 172.21.15.111:0/2135799085 shutdown_connections 2024-09-15T22:49:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1f66b740 1 -- 172.21.15.111:0/2135799085 wait complete. 2024-09-15T22:49:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1f66b740 1 -- start start 2024-09-15T22:49:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1f66b740 1 Processor -- start 2024-09-15T22:49:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1f66b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636c4a26260 0x5636c4a30410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1f66b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5636c4a26ba0 0x5636c4a30950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1f66b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636c4a2bc30 0x5636c4a37920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1f66b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5636c4a2eba0 con 0x5636c4a26ba0 2024-09-15T22:49:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1f66b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5636c4914090 con 0x5636c4a2bc30 2024-09-15T22:49:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1f66b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5636c4a2ed20 con 0x5636c4a26260 2024-09-15T22:49:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1f66b740 10 client.0 _ll_register_callbacks cb 0x5636c4968f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:49:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1a800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636c4a2bc30 0x5636c4a37920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d19400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5636c4a26ba0 0x5636c4a30950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1a800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636c4a2bc30 0x5636c4a37920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:34130/0 (socket says 172.21.15.111:34130) 2024-09-15T22:49:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1a800640 1 -- 172.21.15.111:0/1070448002 learned_addr learned my addr 172.21.15.111:0/1070448002 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:49:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d19e00640 1 --2- 172.21.15.111:0/1070448002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636c4a26260 0x5636c4a30410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1a800640 1 -- 172.21.15.111:0/1070448002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636c4a26260 msgr2=0x5636c4a30410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1a800640 1 --2- 172.21.15.111:0/1070448002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636c4a26260 0x5636c4a30410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1a800640 1 -- 172.21.15.111:0/1070448002 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5636c4a26ba0 msgr2=0x5636c4a30950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1a800640 1 --2- 172.21.15.111:0/1070448002 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5636c4a26ba0 0x5636c4a30950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1a800640 1 -- 172.21.15.111:0/1070448002 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636c476a620 con 0x5636c4a2bc30 2024-09-15T22:49:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4d1a800640 1 --2- 172.21.15.111:0/1070448002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636c4a2bc30 0x5636c4a37920 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4d14051790 tx=0x7f4d14108930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcfcad48edba47bd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4cf7400640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:49:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.989+0000 7f4cf7400640 1 -- 172.21.15.111:0/1070448002 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1408d070 con 0x5636c4a2bc30 2024-09-15T22:49:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4d1f66b740 1 -- 172.21.15.111:0/1070448002 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d08005500 con 0x5636c4a2bc30 2024-09-15T22:49:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4cf7400640 1 -- 172.21.15.111:0/1070448002 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f4d14111160 con 0x5636c4a2bc30 2024-09-15T22:49:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4cf7400640 1 -- 172.21.15.111:0/1070448002 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4d141113a0 con 0x5636c4a2bc30 2024-09-15T22:49:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4d1f66b740 10 client.25319 fetch_fsmap learned FSMap version 29 2024-09-15T22:49:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4d1f66b740 1 -- 172.21.15.111:0/1070448002 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5636c4a3a830 con 0x5636c4a2bc30 2024-09-15T22:49:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4cf7400640 1 -- 172.21.15.111:0/1070448002 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d08005500 con 0x5636c4a2bc30 2024-09-15T22:49:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4cf7400640 1 -- 172.21.15.111:0/1070448002 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f4d1407eab0 con 0x5636c4a2bc30 2024-09-15T22:49:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4d1f66b740 10 client.25319 fetch_fsmap finished waiting for FSMap version 29 2024-09-15T22:49:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4d1f66b740 10 client.25319 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:49:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4d1f66b740 20 client.25319 populate_metadata read hostname 'smithi111' 2024-09-15T22:49:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4d1f66b740 1 --2- 172.21.15.111:0/1070448002 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5636c4a3be60 0x5636c4a3e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4d1f66b740 4 client.25319 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:49:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4d1f66b740 1 -- 172.21.15.111:0/1070448002 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636c4998830 con 0x5636c4a3be60 2024-09-15T22:49:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4d19e00640 1 --2- 172.21.15.111:0/1070448002 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5636c4a3be60 0x5636c4a3e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4d19e00640 1 --2- 172.21.15.111:0/1070448002 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5636c4a3be60 0x5636c4a3e290 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=141b9e827218ef80 server_cookie=c4504b10d5a27e20 in_seq=0 out_seq=0 2024-09-15T22:49:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4cf7400640 10 client.25319 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:49:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4cf7400640 1 -- 172.21.15.111:0/1070448002 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5636c4998830 con 0x5636c4a3be60 2024-09-15T22:49:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4cf7400640 10 client.25319 handle_command_reply: tid=0 2024-09-15T22:49:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4d1f66b740 1 client.25319 shutdown 2024-09-15T22:49:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4d1f66b740 1 -- 172.21.15.111:0/1070448002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636c4a2bc30 msgr2=0x5636c4a37920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4d1f66b740 1 --2- 172.21.15.111:0/1070448002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636c4a2bc30 0x5636c4a37920 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4d14051790 tx=0x7f4d14108930 comp rx=0 tx=0).stop 2024-09-15T22:49:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4d1f66b740 1 -- 172.21.15.111:0/1070448002 shutdown_connections 2024-09-15T22:49:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4d1f66b740 1 --2- 172.21.15.111:0/1070448002 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5636c4a3be60 0x5636c4a3e290 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4d1f66b740 1 --2- 172.21.15.111:0/1070448002 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636c4a2bc30 0x5636c4a37920 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4d1f66b740 1 --2- 172.21.15.111:0/1070448002 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5636c4a26ba0 0x5636c4a30950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4cf7400640 0 client.25319 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:49:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4d1f66b740 1 --2- 172.21.15.111:0/1070448002 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636c4a26260 0x5636c4a30410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4d1f66b740 1 -- 172.21.15.111:0/1070448002 >> 172.21.15.111:0/1070448002 conn(0x5636c4a218d0 msgr2=0x5636c4a230e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4d1f66b740 1 -- 172.21.15.111:0/1070448002 shutdown_connections 2024-09-15T22:49:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4d1f66b740 1 -- 172.21.15.111:0/1070448002 wait complete. 2024-09-15T22:49:46.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:45.994+0000 7f4d1f66b740 20 client.25319 trim_cache size 0 max 16384 2024-09-15T22:49:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1f66b740 1 Processor -- start 2024-09-15T22:49:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1f66b740 1 -- start start 2024-09-15T22:49:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1f66b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5636c4a218d0 0x5636c4a22780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1f66b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636c4a26260 0x5636c4a378e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1f66b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636c4a26ba0 0x5636c4a22cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1f66b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4d10141400 con 0x5636c4a218d0 2024-09-15T22:49:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1f66b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1013ecd0 con 0x5636c4a26260 2024-09-15T22:49:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1f66b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1013cb20 con 0x5636c4a26ba0 2024-09-15T22:49:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d19e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5636c4a218d0 0x5636c4a22780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d19e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5636c4a218d0 0x5636c4a22780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60844/0 (socket says 172.21.15.111:60844) 2024-09-15T22:49:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d19400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636c4a26260 0x5636c4a378e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1a800640 1 --2- 172.21.15.111:0/1371185074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636c4a26ba0 0x5636c4a22cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d19e00640 1 -- 172.21.15.111:0/1371185074 learned_addr learned my addr 172.21.15.111:0/1371185074 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:49:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d19e00640 1 -- 172.21.15.111:0/1371185074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636c4a26ba0 msgr2=0x5636c4a22cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d19e00640 1 --2- 172.21.15.111:0/1371185074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636c4a26ba0 0x5636c4a22cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d19e00640 1 -- 172.21.15.111:0/1371185074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636c4a26260 msgr2=0x5636c4a378e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d19e00640 1 --2- 172.21.15.111:0/1371185074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636c4a26260 0x5636c4a378e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d19e00640 1 -- 172.21.15.111:0/1371185074 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636c4a216f0 con 0x5636c4a218d0 2024-09-15T22:49:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d19400640 1 --2- 172.21.15.111:0/1371185074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636c4a26260 0x5636c4a378e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:49:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1a800640 1 --2- 172.21.15.111:0/1371185074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636c4a26ba0 0x5636c4a22cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:49:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d19e00640 1 --2- 172.21.15.111:0/1371185074 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5636c4a218d0 0x5636c4a22780 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4d08043000 tx=0x7f4d08031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0882aa41551c45 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d02600640 1 -- 172.21.15.111:0/1371185074 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d08040040 con 0x5636c4a218d0 2024-09-15T22:49:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d02600640 1 -- 172.21.15.111:0/1371185074 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d08004430 con 0x5636c4a218d0 2024-09-15T22:49:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d02600640 1 -- 172.21.15.111:0/1371185074 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d08035040 con 0x5636c4a218d0 2024-09-15T22:49:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1f66b740 1 -- 172.21.15.111:0/1371185074 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5636c4a218d0 msgr2=0x5636c4a22780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1f66b740 1 --2- 172.21.15.111:0/1371185074 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5636c4a218d0 0x5636c4a22780 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4d08043000 tx=0x7f4d08031400 comp rx=0 tx=0).stop 2024-09-15T22:49:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1f66b740 1 -- 172.21.15.111:0/1371185074 shutdown_connections 2024-09-15T22:49:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1f66b740 1 --2- 172.21.15.111:0/1371185074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636c4a26ba0 0x5636c4a22cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1f66b740 1 --2- 172.21.15.111:0/1371185074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636c4a26260 0x5636c4a378e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1f66b740 1 --2- 172.21.15.111:0/1371185074 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5636c4a218d0 0x5636c4a22780 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1f66b740 1 -- 172.21.15.111:0/1371185074 >> 172.21.15.111:0/1371185074 conn(0x5636c4a2b7c0 msgr2=0x5636c4a30410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1f66b740 1 -- 172.21.15.111:0/1371185074 shutdown_connections 2024-09-15T22:49:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1f66b740 1 -- 172.21.15.111:0/1371185074 wait complete. 2024-09-15T22:49:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1f66b740 1 -- start start 2024-09-15T22:49:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1f66b740 1 Processor -- start 2024-09-15T22:49:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1f66b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5636c4a218d0 0x5636c4a34aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1f66b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636c4a26260 0x5636c4a34fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1f66b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636c4a26ba0 0x5636c4a35fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1f66b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1013d190 con 0x5636c4a218d0 2024-09-15T22:49:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1f66b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1013e600 con 0x5636c4a26ba0 2024-09-15T22:49:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1f66b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1013d860 con 0x5636c4a26260 2024-09-15T22:49:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1f66b740 10 client.0 _ll_register_callbacks cb 0x7f4d14074e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:49:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d19e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5636c4a218d0 0x5636c4a34aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d19400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636c4a26260 0x5636c4a34fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d19e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5636c4a218d0 0x5636c4a34aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60846/0 (socket says 172.21.15.111:60846) 2024-09-15T22:49:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d19e00640 1 -- 172.21.15.111:0/2255536366 learned_addr learned my addr 172.21.15.111:0/2255536366 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:49:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1a800640 1 --2- 172.21.15.111:0/2255536366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636c4a26ba0 0x5636c4a35fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d19e00640 1 -- 172.21.15.111:0/2255536366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636c4a26260 msgr2=0x5636c4a34fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d19e00640 1 --2- 172.21.15.111:0/2255536366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636c4a26260 0x5636c4a34fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d19e00640 1 -- 172.21.15.111:0/2255536366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636c4a26ba0 msgr2=0x5636c4a35fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d19e00640 1 --2- 172.21.15.111:0/2255536366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636c4a26ba0 0x5636c4a35fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d19e00640 1 -- 172.21.15.111:0/2255536366 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636c4a365b0 con 0x5636c4a218d0 2024-09-15T22:49:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d19e00640 1 --2- 172.21.15.111:0/2255536366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5636c4a218d0 0x5636c4a34aa0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4d0802aa00 tx=0x7f4d08005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e381772b03869f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4cf7400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:49:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4cf7400640 1 -- 172.21.15.111:0/2255536366 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0805d060 con 0x5636c4a218d0 2024-09-15T22:49:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4cf7400640 1 -- 172.21.15.111:0/2255536366 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f4d08004970 con 0x5636c4a218d0 2024-09-15T22:49:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.011+0000 7f4d1f66b740 1 -- 172.21.15.111:0/2255536366 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d08004430 con 0x5636c4a218d0 2024-09-15T22:49:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.016+0000 7f4cf7400640 1 -- 172.21.15.111:0/2255536366 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4d08040050 con 0x5636c4a218d0 2024-09-15T22:49:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.016+0000 7f4d1f66b740 10 client.15303 fetch_fsmap learned FSMap version 29 2024-09-15T22:49:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.016+0000 7f4d1f66b740 1 -- 172.21.15.111:0/2255536366 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5636c4a378f0 con 0x5636c4a218d0 2024-09-15T22:49:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.016+0000 7f4cf7400640 1 -- 172.21.15.111:0/2255536366 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d08004430 con 0x5636c4a218d0 2024-09-15T22:49:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.016+0000 7f4cf7400640 1 -- 172.21.15.111:0/2255536366 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f4d08004cc0 con 0x5636c4a218d0 2024-09-15T22:49:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.016+0000 7f4d1f66b740 10 client.15303 fetch_fsmap finished waiting for FSMap version 29 2024-09-15T22:49:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.016+0000 7f4d1f66b740 10 client.15303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:49:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.016+0000 7f4d1f66b740 20 client.15303 populate_metadata read hostname 'smithi111' 2024-09-15T22:49:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.016+0000 7f4d1f66b740 1 --2- 172.21.15.111:0/2255536366 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5636c4a2bc30 0x5636c4aa8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.016+0000 7f4d1f66b740 4 client.15303 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:49:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.016+0000 7f4d1f66b740 1 -- 172.21.15.111:0/2255536366 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5636c4992b80 con 0x5636c4a2bc30 2024-09-15T22:49:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.016+0000 7f4d19400640 1 --2- 172.21.15.111:0/2255536366 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5636c4a2bc30 0x5636c4aa8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.017+0000 7f4d19400640 1 --2- 172.21.15.111:0/2255536366 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5636c4a2bc30 0x5636c4aa8bd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e0aa3daa0eccec6 server_cookie=1426e77d3bb4791a in_seq=0 out_seq=0 2024-09-15T22:49:46.024 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.017+0000 7f4cf7400640 10 client.15303 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:49:46.026 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.023+0000 7f4cf7400640 1 -- 172.21.15.111:0/2255536366 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5636c4992b80 con 0x5636c4a2bc30 2024-09-15T22:49:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.023+0000 7f4cf7400640 10 client.15303 handle_command_reply: tid=0 2024-09-15T22:49:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.023+0000 7f4d1f66b740 1 client.15303 shutdown 2024-09-15T22:49:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.023+0000 7f4d1f66b740 1 -- 172.21.15.111:0/2255536366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5636c4a218d0 msgr2=0x5636c4a34aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.023+0000 7f4d1f66b740 1 --2- 172.21.15.111:0/2255536366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5636c4a218d0 0x5636c4a34aa0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4d0802aa00 tx=0x7f4d08005e30 comp rx=0 tx=0).stop 2024-09-15T22:49:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.023+0000 7f4d1f66b740 1 -- 172.21.15.111:0/2255536366 shutdown_connections 2024-09-15T22:49:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.023+0000 7f4d1f66b740 1 --2- 172.21.15.111:0/2255536366 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5636c4a2bc30 0x5636c4aa8bd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.023+0000 7f4d1f66b740 1 --2- 172.21.15.111:0/2255536366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5636c4a26ba0 0x5636c4a35fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.023+0000 7f4d1f66b740 1 --2- 172.21.15.111:0/2255536366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5636c4a26260 0x5636c4a34fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.023+0000 7f4d1f66b740 1 --2- 172.21.15.111:0/2255536366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5636c4a218d0 0x5636c4a34aa0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.023+0000 7f4d1f66b740 1 -- 172.21.15.111:0/2255536366 >> 172.21.15.111:0/2255536366 conn(0x5636c4a2b7c0 msgr2=0x5636c4a201f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.023+0000 7f4cf7400640 0 client.15303 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:49:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.023+0000 7f4d1f66b740 1 -- 172.21.15.111:0/2255536366 shutdown_connections 2024-09-15T22:49:46.027 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.023+0000 7f4d1f66b740 1 -- 172.21.15.111:0/2255536366 wait complete. 2024-09-15T22:49:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.023+0000 7f4d1f66b740 20 client.15303 trim_cache size 0 max 16384 2024-09-15T22:49:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:49:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:49:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:49:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:49:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.023+0000 7f4d1b200640 1 -- 172.21.15.111:0/2846441092 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4cfc0a3250 msgr2=0x7f4cfc0a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.023+0000 7f4d1b200640 1 --2- 172.21.15.111:0/2846441092 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4cfc0a3250 0x7f4cfc0a5710 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4d04009fd0 tx=0x7f4d040046c0 comp rx=0 tx=0).stop 2024-09-15T22:49:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.023+0000 7f4d1b200640 1 -- 172.21.15.111:0/2846441092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d10097230 msgr2=0x7f4d10135160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.023+0000 7f4d1b200640 1 --2- 172.21.15.111:0/2846441092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d10097230 0x7f4d10135160 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4d14050280 tx=0x7f4d14068ae0 comp rx=0 tx=0).stop 2024-09-15T22:49:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.023+0000 7f4d1b200640 1 -- 172.21.15.111:0/2846441092 shutdown_connections 2024-09-15T22:49:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.023+0000 7f4d1b200640 1 --2- 172.21.15.111:0/2846441092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d10097230 0x7f4d10135160 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.023+0000 7f4d1b200640 1 --2- 172.21.15.111:0/2846441092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d10096870 0x7f4d1012e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.023+0000 7f4d1b200640 1 --2- 172.21.15.111:0/2846441092 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4cfc0a3250 0x7f4cfc0a5710 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.023+0000 7f4d1b200640 1 --2- 172.21.15.111:0/2846441092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d10095680 0x7f4d1012dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.023+0000 7f4d1b200640 1 -- 172.21.15.111:0/2846441092 >> 172.21.15.111:0/2846441092 conn(0x7f4d10090dc0 msgr2=0x7f4d100069e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.023+0000 7f4d1b200640 1 -- 172.21.15.111:0/2846441092 shutdown_connections 2024-09-15T22:49:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:46.023+0000 7f4d1b200640 1 -- 172.21.15.111:0/2846441092 wait complete. 2024-09-15T22:49:46.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:45 smithi192 ceph-mon[35427]: pgmap v465: 97 pgs: 97 active+clean; 113 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 204 op/s 2024-09-15T22:49:46.253 INFO:tasks.workunit.client.0.smithi111.stdout: 1 262926 9.66 MB/sec execute 14 sec latency 448.363 ms 2024-09-15T22:49:47.254 INFO:tasks.workunit.client.0.smithi111.stdout: 1 262930 9.02 MB/sec execute 15 sec latency 1443.943 ms 2024-09-15T22:49:47.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:47 smithi121 ceph-mon[34815]: pgmap v466: 97 pgs: 97 active+clean; 115 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 222 op/s 2024-09-15T22:49:47.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:47 smithi111 ceph-mon[30629]: pgmap v466: 97 pgs: 97 active+clean; 115 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 222 op/s 2024-09-15T22:49:48.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:47 smithi192 ceph-mon[35427]: pgmap v466: 97 pgs: 97 active+clean; 115 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 222 op/s 2024-09-15T22:49:48.254 INFO:tasks.workunit.client.0.smithi111.stdout: 1 266085 9.55 MB/sec execute 16 sec latency 15.863 ms 2024-09-15T22:49:49.254 INFO:tasks.workunit.client.0.smithi111.stdout: 1 268820 9.73 MB/sec execute 17 sec latency 11.823 ms 2024-09-15T22:49:49.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:49 smithi121 ceph-mon[34815]: pgmap v467: 97 pgs: 97 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 182 op/s 2024-09-15T22:49:49.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:49 smithi111 ceph-mon[30629]: pgmap v467: 97 pgs: 97 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 182 op/s 2024-09-15T22:49:50.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:49 smithi192 ceph-mon[35427]: pgmap v467: 97 pgs: 97 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 182 op/s 2024-09-15T22:49:50.254 INFO:tasks.workunit.client.0.smithi111.stdout: 1 271261 9.93 MB/sec execute 18 sec latency 13.283 ms 2024-09-15T22:49:51.254 INFO:tasks.workunit.client.0.smithi111.stdout: 1 272582 9.79 MB/sec execute 19 sec latency 696.730 ms 2024-09-15T22:49:51.695 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:51 smithi111 ceph-mon[30629]: pgmap v468: 97 pgs: 97 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 179 op/s 2024-09-15T22:49:51.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:51 smithi121 ceph-mon[34815]: pgmap v468: 97 pgs: 97 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 179 op/s 2024-09-15T22:49:52.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:51 smithi192 ceph-mon[35427]: pgmap v468: 97 pgs: 97 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 179 op/s 2024-09-15T22:49:52.254 INFO:tasks.workunit.client.0.smithi111.stdout: 1 272582 9.30 MB/sec execute 20 sec latency 1696.769 ms 2024-09-15T22:49:53.254 INFO:tasks.workunit.client.0.smithi111.stdout: 1 274846 9.42 MB/sec execute 21 sec latency 1718.869 ms 2024-09-15T22:49:53.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:53 smithi121 ceph-mon[34815]: pgmap v469: 97 pgs: 97 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 189 op/s 2024-09-15T22:49:53.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:53 smithi111 ceph-mon[30629]: pgmap v469: 97 pgs: 97 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 189 op/s 2024-09-15T22:49:54.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:53 smithi192 ceph-mon[35427]: pgmap v469: 97 pgs: 97 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 189 op/s 2024-09-15T22:49:54.254 INFO:tasks.workunit.client.0.smithi111.stdout: 1 277452 9.64 MB/sec execute 22 sec latency 17.917 ms 2024-09-15T22:49:55.254 INFO:tasks.workunit.client.0.smithi111.stdout: 1 280565 10.02 MB/sec execute 23 sec latency 7.189 ms 2024-09-15T22:49:55.780 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:55 smithi111 ceph-mon[30629]: pgmap v470: 97 pgs: 97 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 198 op/s 2024-09-15T22:49:55.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:55 smithi121 ceph-mon[34815]: pgmap v470: 97 pgs: 97 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 198 op/s 2024-09-15T22:49:56.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:55 smithi192 ceph-mon[35427]: pgmap v470: 97 pgs: 97 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 198 op/s 2024-09-15T22:49:56.254 INFO:tasks.workunit.client.0.smithi111.stdout: 1 281335 9.70 MB/sec execute 24 sec latency 450.715 ms 2024-09-15T22:49:57.254 INFO:tasks.workunit.client.0.smithi111.stdout: 1 283944 9.95 MB/sec execute 25 sec latency 687.674 ms 2024-09-15T22:49:57.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:57 smithi121 ceph-mon[34815]: pgmap v471: 97 pgs: 97 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 168 op/s 2024-09-15T22:49:57.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:57 smithi111 ceph-mon[30629]: pgmap v471: 97 pgs: 97 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 168 op/s 2024-09-15T22:49:58.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:57 smithi192 ceph-mon[35427]: pgmap v471: 97 pgs: 97 active+clean; 117 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 168 op/s 2024-09-15T22:49:58.254 INFO:tasks.workunit.client.0.smithi111.stdout: 1 286658 10.05 MB/sec execute 26 sec latency 18.232 ms 2024-09-15T22:49:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.144+0000 7f833ac00640 1 -- 172.21.15.111:0/1573822178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8334103d60 msgr2=0x7f8334104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.144+0000 7f833ac00640 1 --2- 172.21.15.111:0/1573822178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8334103d60 0x7f8334104160 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8324009900 tx=0x7f832402a3f0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.144+0000 7f833ac00640 1 -- 172.21.15.111:0/1573822178 shutdown_connections 2024-09-15T22:49:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.144+0000 7f833ac00640 1 --2- 172.21.15.111:0/1573822178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8334105910 0x7f833410c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.144+0000 7f833ac00640 1 --2- 172.21.15.111:0/1573822178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8334104f50 0x7f83341053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.144+0000 7f833ac00640 1 --2- 172.21.15.111:0/1573822178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8334103d60 0x7f8334104160 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.144+0000 7f833ac00640 1 -- 172.21.15.111:0/1573822178 >> 172.21.15.111:0/1573822178 conn(0x7f83340ff4e0 msgr2=0x7f8334101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.144+0000 7f833ac00640 1 -- 172.21.15.111:0/1573822178 shutdown_connections 2024-09-15T22:49:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.144+0000 7f833ac00640 1 -- 172.21.15.111:0/1573822178 wait complete. 2024-09-15T22:49:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.145+0000 7f833ac00640 1 Processor -- start 2024-09-15T22:49:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.145+0000 7f833ac00640 1 -- start start 2024-09-15T22:49:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.145+0000 7f833ac00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8334103d60 0x7f8334199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.145+0000 7f833ac00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8334104f50 0x7f833419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:59.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.145+0000 7f833ac00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8334105910 0x7f83341a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.145+0000 7f833ac00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f833410fa10 con 0x7f8334103d60 2024-09-15T22:49:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.145+0000 7f833ac00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f833410f890 con 0x7f8334104f50 2024-09-15T22:49:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.145+0000 7f833ac00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f833410fb90 con 0x7f8334105910 2024-09-15T22:49:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.145+0000 7f8333e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8334103d60 0x7f8334199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.145+0000 7f8333400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8334104f50 0x7f833419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.145+0000 7f8338c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8334105910 0x7f83341a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.146+0000 7f8333400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8334104f50 0x7f833419a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:50136/0 (socket says 172.21.15.111:50136) 2024-09-15T22:49:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.146+0000 7f8338c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8334105910 0x7f83341a1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:52486/0 (socket says 172.21.15.111:52486) 2024-09-15T22:49:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.146+0000 7f8333400640 1 -- 172.21.15.111:0/1183905819 learned_addr learned my addr 172.21.15.111:0/1183905819 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:49:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.146+0000 7f8333400640 1 -- 172.21.15.111:0/1183905819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8334105910 msgr2=0x7f83341a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.146+0000 7f8333400640 1 --2- 172.21.15.111:0/1183905819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8334105910 0x7f83341a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.146+0000 7f8333400640 1 -- 172.21.15.111:0/1183905819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8334103d60 msgr2=0x7f8334199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.146+0000 7f8333400640 1 --2- 172.21.15.111:0/1183905819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8334103d60 0x7f8334199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.146+0000 7f8333400640 1 -- 172.21.15.111:0/1183905819 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83341a1bf0 con 0x7f8334104f50 2024-09-15T22:49:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.146+0000 7f8333400640 1 --2- 172.21.15.111:0/1183905819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8334104f50 0x7f833419a510 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f832000e9f0 tx=0x7f832000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.146+0000 7f8330c00640 1 -- 172.21.15.111:0/1183905819 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832000cd80 con 0x7f8334104f50 2024-09-15T22:49:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.146+0000 7f8330c00640 1 -- 172.21.15.111:0/1183905819 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8320010910 con 0x7f8334104f50 2024-09-15T22:49:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.146+0000 7f8330c00640 1 -- 172.21.15.111:0/1183905819 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320018990 con 0x7f8334104f50 2024-09-15T22:49:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.146+0000 7f833ac00640 1 -- 172.21.15.111:0/1183905819 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83341a1ee0 con 0x7f8334104f50 2024-09-15T22:49:59.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.147+0000 7f833ac00640 1 -- 172.21.15.111:0/1183905819 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83341a2350 con 0x7f8334104f50 2024-09-15T22:49:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.149+0000 7f833dc4b740 1 Processor -- start 2024-09-15T22:49:59.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.150+0000 7f833dc4b740 1 -- start start 2024-09-15T22:49:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.150+0000 7f833dc4b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5fcaee090 0x55b5fcaee490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.150+0000 7f833dc4b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5fcaee9d0 0x55b5fcaf3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.150+0000 7f833dc4b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5fcaf3a60 0x55b5fcaf5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:59.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.150+0000 7f833dc4b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b5fca53100 con 0x55b5fcaf3a60 2024-09-15T22:49:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.150+0000 7f833dc4b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b5fc9fb1b0 con 0x55b5fcaee090 2024-09-15T22:49:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.150+0000 7f833dc4b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b5fca4f0c0 con 0x55b5fcaee9d0 2024-09-15T22:49:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.150+0000 7f8330c00640 1 -- 172.21.15.111:0/1183905819 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f8320010ac0 con 0x7f8334104f50 2024-09-15T22:49:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.150+0000 7f8338c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5fcaee090 0x55b5fcaee490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.150+0000 7f8338c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5fcaee090 0x55b5fcaee490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:50152/0 (socket says 172.21.15.111:50152) 2024-09-15T22:49:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.150+0000 7f8338c00640 1 -- 172.21.15.111:0/1287353448 learned_addr learned my addr 172.21.15.111:0/1287353448 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:49:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.150+0000 7f8333e00640 1 --2- 172.21.15.111:0/1287353448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5fcaee9d0 0x55b5fcaf3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.150+0000 7f8333400640 1 --2- 172.21.15.111:0/1287353448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5fcaf3a60 0x55b5fcaf5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.150+0000 7f8338c00640 1 -- 172.21.15.111:0/1287353448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5fcaee9d0 msgr2=0x55b5fcaf3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.150+0000 7f8338c00640 1 --2- 172.21.15.111:0/1287353448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5fcaee9d0 0x55b5fcaf3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.150+0000 7f8338c00640 1 -- 172.21.15.111:0/1287353448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5fcaf3a60 msgr2=0x55b5fcaf5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:59.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.151+0000 7f8338c00640 1 --2- 172.21.15.111:0/1287353448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5fcaf3a60 0x55b5fcaf5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.151+0000 7f8338c00640 1 -- 172.21.15.111:0/1287353448 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5fc766210 con 0x55b5fcaee090 2024-09-15T22:49:59.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.151+0000 7f8338c00640 1 --2- 172.21.15.111:0/1287353448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5fcaee090 0x55b5fcaee490 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f832c009fd0 tx=0x7f832c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a0d2e134665b5b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.151+0000 7f8329200640 1 -- 172.21.15.111:0/1287353448 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c004d20 con 0x55b5fcaee090 2024-09-15T22:49:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.151+0000 7f8329200640 1 -- 172.21.15.111:0/1287353448 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f832c004050 con 0x55b5fcaee090 2024-09-15T22:49:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.151+0000 7f8329200640 1 -- 172.21.15.111:0/1287353448 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c005260 con 0x55b5fcaee090 2024-09-15T22:49:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.152+0000 7f8330c00640 1 --2- 172.21.15.111:0/1183905819 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f83180a3370 0x7f83180a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.152+0000 7f833dc4b740 1 -- 172.21.15.111:0/1287353448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5fcaee090 msgr2=0x55b5fcaee490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:59.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.152+0000 7f833dc4b740 1 --2- 172.21.15.111:0/1287353448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5fcaee090 0x55b5fcaee490 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f832c009fd0 tx=0x7f832c0118d0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.152+0000 7f8330c00640 1 -- 172.21.15.111:0/1183905819 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f83200cfb90 con 0x7f8334104f50 2024-09-15T22:49:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.152+0000 7f833dc4b740 1 -- 172.21.15.111:0/1287353448 shutdown_connections 2024-09-15T22:49:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.152+0000 7f833dc4b740 1 --2- 172.21.15.111:0/1287353448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5fcaf3a60 0x55b5fcaf5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.152+0000 7f833dc4b740 1 --2- 172.21.15.111:0/1287353448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5fcaee9d0 0x55b5fcaf3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.152+0000 7f833dc4b740 1 --2- 172.21.15.111:0/1287353448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5fcaee090 0x55b5fcaee490 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.152+0000 7f833dc4b740 1 -- 172.21.15.111:0/1287353448 >> 172.21.15.111:0/1287353448 conn(0x55b5fcae9700 msgr2=0x55b5fcae9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.152+0000 7f8333e00640 1 --2- 172.21.15.111:0/1183905819 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f83180a3370 0x7f83180a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.152+0000 7f833dc4b740 1 -- 172.21.15.111:0/1287353448 shutdown_connections 2024-09-15T22:49:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.152+0000 7f833dc4b740 1 -- 172.21.15.111:0/1287353448 wait complete. 2024-09-15T22:49:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.152+0000 7f833dc4b740 1 -- start start 2024-09-15T22:49:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.153+0000 7f8333e00640 1 --2- 172.21.15.111:0/1183905819 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f83180a3370 0x7f83180a5830 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f832402a9e0 tx=0x7f83240315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.153+0000 7f833dc4b740 1 Processor -- start 2024-09-15T22:49:59.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.153+0000 7f833dc4b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5fcaee090 0x55b5fcaf8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.153+0000 7f833dc4b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5fcaee9d0 0x55b5fcaf8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.153+0000 7f833dc4b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5fcaf3a60 0x55b5fcaff730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.153+0000 7f833dc4b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b5fcaf6a30 con 0x55b5fcaee9d0 2024-09-15T22:49:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.153+0000 7f833dc4b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b5fc9dc090 con 0x55b5fcaf3a60 2024-09-15T22:49:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.153+0000 7f833dc4b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b5fcaf6bb0 con 0x55b5fcaee090 2024-09-15T22:49:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.153+0000 7f833dc4b740 10 client.0 _ll_register_callbacks cb 0x55b5fca30f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:49:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.153+0000 7f8333e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5fcaee090 0x55b5fcaf8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.153+0000 7f8338c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5fcaf3a60 0x55b5fcaff730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.154+0000 7f8333400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5fcaee9d0 0x55b5fcaf8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.154+0000 7f8333e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5fcaee090 0x55b5fcaf8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:52508/0 (socket says 172.21.15.111:52508) 2024-09-15T22:49:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.154+0000 7f8333e00640 1 -- 172.21.15.111:0/3314143676 learned_addr learned my addr 172.21.15.111:0/3314143676 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:49:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.154+0000 7f8338c00640 1 -- 172.21.15.111:0/3314143676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5fcaee090 msgr2=0x55b5fcaf8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:59.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.154+0000 7f8338c00640 1 --2- 172.21.15.111:0/3314143676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5fcaee090 0x55b5fcaf8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.154+0000 7f8338c00640 1 -- 172.21.15.111:0/3314143676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5fcaee9d0 msgr2=0x55b5fcaf8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.154+0000 7f8338c00640 1 --2- 172.21.15.111:0/3314143676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5fcaee9d0 0x55b5fcaf8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.154+0000 7f8338c00640 1 -- 172.21.15.111:0/3314143676 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5fc832620 con 0x55b5fcaf3a60 2024-09-15T22:49:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.154+0000 7f8338c00640 1 --2- 172.21.15.111:0/3314143676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5fcaf3a60 0x55b5fcaff730 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f832c011dd0 tx=0x7f832c004cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdd0b9f3cd810023 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.154+0000 7f831d600640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:49:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.154+0000 7f831d600640 1 -- 172.21.15.111:0/3314143676 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c028020 con 0x55b5fcaf3a60 2024-09-15T22:49:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.155+0000 7f833dc4b740 1 -- 172.21.15.111:0/3314143676 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f832c004050 con 0x55b5fcaf3a60 2024-09-15T22:49:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.155+0000 7f831d600640 1 -- 172.21.15.111:0/3314143676 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f832c004410 con 0x55b5fcaf3a60 2024-09-15T22:49:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.155+0000 7f831d600640 1 -- 172.21.15.111:0/3314143676 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f832c004610 con 0x55b5fcaf3a60 2024-09-15T22:49:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.155+0000 7f833dc4b740 10 client.25337 fetch_fsmap learned FSMap version 29 2024-09-15T22:49:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.155+0000 7f833dc4b740 1 -- 172.21.15.111:0/3314143676 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b5fcb02520 con 0x55b5fcaf3a60 2024-09-15T22:49:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.155+0000 7f831d600640 1 -- 172.21.15.111:0/3314143676 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f832c004050 con 0x55b5fcaf3a60 2024-09-15T22:49:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.155+0000 7f831d600640 1 -- 172.21.15.111:0/3314143676 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f832c00f6a0 con 0x55b5fcaf3a60 2024-09-15T22:49:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.155+0000 7f833dc4b740 10 client.25337 fetch_fsmap finished waiting for FSMap version 29 2024-09-15T22:49:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.155+0000 7f833dc4b740 10 client.25337 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:49:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.155+0000 7f833dc4b740 20 client.25337 populate_metadata read hostname 'smithi111' 2024-09-15T22:49:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.155+0000 7f833dc4b740 1 --2- 172.21.15.111:0/3314143676 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55b5fcb03a50 0x55b5fcb05eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.155+0000 7f833dc4b740 4 client.25337 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:49:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.155+0000 7f833dc4b740 1 -- 172.21.15.111:0/3314143676 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5fca60830 con 0x55b5fcb03a50 2024-09-15T22:49:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.156+0000 7f8333e00640 1 --2- 172.21.15.111:0/3314143676 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55b5fcb03a50 0x55b5fcb05eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.156+0000 7f8333e00640 1 --2- 172.21.15.111:0/3314143676 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55b5fcb03a50 0x55b5fcb05eb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=536caaafb8c2c30c server_cookie=c2cefa3bb9e42003 in_seq=0 out_seq=0 2024-09-15T22:49:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.156+0000 7f831d600640 10 client.25337 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:49:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.157+0000 7f831d600640 1 -- 172.21.15.111:0/3314143676 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b5fca60830 con 0x55b5fcb03a50 2024-09-15T22:49:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.157+0000 7f831d600640 10 client.25337 handle_command_reply: tid=0 2024-09-15T22:49:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.157+0000 7f833dc4b740 1 client.25337 shutdown 2024-09-15T22:49:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.157+0000 7f833dc4b740 1 -- 172.21.15.111:0/3314143676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5fcaf3a60 msgr2=0x55b5fcaff730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.157+0000 7f833dc4b740 1 --2- 172.21.15.111:0/3314143676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5fcaf3a60 0x55b5fcaff730 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f832c011dd0 tx=0x7f832c004cf0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.157+0000 7f833dc4b740 1 -- 172.21.15.111:0/3314143676 shutdown_connections 2024-09-15T22:49:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.157+0000 7f833dc4b740 1 --2- 172.21.15.111:0/3314143676 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55b5fcb03a50 0x55b5fcb05eb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.157+0000 7f833dc4b740 1 --2- 172.21.15.111:0/3314143676 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5fcaf3a60 0x55b5fcaff730 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.157+0000 7f833dc4b740 1 --2- 172.21.15.111:0/3314143676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5fcaee9d0 0x55b5fcaf8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.157+0000 7f833dc4b740 1 --2- 172.21.15.111:0/3314143676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5fcaee090 0x55b5fcaf8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.157+0000 7f833dc4b740 1 -- 172.21.15.111:0/3314143676 >> 172.21.15.111:0/3314143676 conn(0x55b5fcae9700 msgr2=0x55b5fcae8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.157+0000 7f831d600640 0 client.25337 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:49:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.157+0000 7f833dc4b740 1 -- 172.21.15.111:0/3314143676 shutdown_connections 2024-09-15T22:49:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.157+0000 7f833dc4b740 1 -- 172.21.15.111:0/3314143676 wait complete. 2024-09-15T22:49:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.157+0000 7f833dc4b740 20 client.25337 trim_cache size 0 max 16384 2024-09-15T22:49:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.173+0000 7f833dc4b740 1 Processor -- start 2024-09-15T22:49:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.173+0000 7f833dc4b740 1 -- start start 2024-09-15T22:49:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.173+0000 7f833dc4b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5fcaee090 0x55b5fcae8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.174+0000 7f833dc4b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5fcaee9d0 0x55b5fcaff6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:59.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.174+0000 7f833dc4b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5fcaf3a60 0x55b5fcaea610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.174+0000 7f833dc4b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f83341ad930 con 0x55b5fcaf3a60 2024-09-15T22:49:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.174+0000 7f833dc4b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f83341ab200 con 0x55b5fcaee9d0 2024-09-15T22:49:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.174+0000 7f833dc4b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f83341a9070 con 0x55b5fcaee090 2024-09-15T22:49:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.174+0000 7f8333400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5fcaee9d0 0x55b5fcaff6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.174+0000 7f8333e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5fcaee090 0x55b5fcae8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.174+0000 7f8333400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5fcaee9d0 0x55b5fcaff6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:50170/0 (socket says 172.21.15.111:50170) 2024-09-15T22:49:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.174+0000 7f8333e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5fcaee090 0x55b5fcae8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:52516/0 (socket says 172.21.15.111:52516) 2024-09-15T22:49:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.174+0000 7f8333400640 1 -- 172.21.15.111:0/176502803 learned_addr learned my addr 172.21.15.111:0/176502803 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:49:59.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.174+0000 7f8338c00640 1 --2- 172.21.15.111:0/176502803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5fcaf3a60 0x55b5fcaea610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.174+0000 7f8333e00640 1 -- 172.21.15.111:0/176502803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5fcaee9d0 msgr2=0x55b5fcaff6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.174+0000 7f8333e00640 1 --2- 172.21.15.111:0/176502803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5fcaee9d0 0x55b5fcaff6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.174+0000 7f8333e00640 1 -- 172.21.15.111:0/176502803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5fcaf3a60 msgr2=0x55b5fcaea610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.174+0000 7f8333e00640 1 --2- 172.21.15.111:0/176502803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5fcaf3a60 0x55b5fcaea610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.174+0000 7f8333e00640 1 -- 172.21.15.111:0/176502803 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5fcaf6910 con 0x55b5fcaee090 2024-09-15T22:49:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.174+0000 7f8333e00640 1 --2- 172.21.15.111:0/176502803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5fcaee090 0x55b5fcae8770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55b5fcb06720 tx=0x7f832403f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0b12c105cc642c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.175+0000 7f831fe00640 1 -- 172.21.15.111:0/176502803 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324035040 con 0x55b5fcaee090 2024-09-15T22:49:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.175+0000 7f831fe00640 1 -- 172.21.15.111:0/176502803 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8324004d10 con 0x55b5fcaee090 2024-09-15T22:49:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.175+0000 7f831fe00640 1 -- 172.21.15.111:0/176502803 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324003850 con 0x55b5fcaee090 2024-09-15T22:49:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.175+0000 7f833dc4b740 1 -- 172.21.15.111:0/176502803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5fcaee090 msgr2=0x55b5fcae8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.175+0000 7f833dc4b740 1 --2- 172.21.15.111:0/176502803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5fcaee090 0x55b5fcae8770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55b5fcb06720 tx=0x7f832403f000 comp rx=0 tx=0).stop 2024-09-15T22:49:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.175+0000 7f833dc4b740 1 -- 172.21.15.111:0/176502803 shutdown_connections 2024-09-15T22:49:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.175+0000 7f833dc4b740 1 --2- 172.21.15.111:0/176502803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5fcaf3a60 0x55b5fcaea610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.175+0000 7f833dc4b740 1 --2- 172.21.15.111:0/176502803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5fcaee9d0 0x55b5fcaff6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.175+0000 7f833dc4b740 1 --2- 172.21.15.111:0/176502803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5fcaee090 0x55b5fcae8770 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.175+0000 7f833dc4b740 1 -- 172.21.15.111:0/176502803 >> 172.21.15.111:0/176502803 conn(0x55b5fcae9700 msgr2=0x55b5fcae8060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.176+0000 7f833dc4b740 1 -- 172.21.15.111:0/176502803 shutdown_connections 2024-09-15T22:49:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.176+0000 7f833dc4b740 1 -- 172.21.15.111:0/176502803 wait complete. 2024-09-15T22:49:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.176+0000 7f833dc4b740 1 -- start start 2024-09-15T22:49:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.176+0000 7f833dc4b740 1 Processor -- start 2024-09-15T22:49:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.176+0000 7f833dc4b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5fcaee090 0x55b5fcafcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.176+0000 7f833dc4b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5fcaee9d0 0x55b5fcafd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.176+0000 7f833dc4b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5fcaf3a60 0x55b5fcafe120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.176+0000 7f833dc4b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f83341a96c0 con 0x55b5fcaf3a60 2024-09-15T22:49:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.176+0000 7f833dc4b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f83341aab30 con 0x55b5fcaee090 2024-09-15T22:49:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.176+0000 7f833dc4b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f83341a9d90 con 0x55b5fcaee9d0 2024-09-15T22:49:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.177+0000 7f833dc4b740 10 client.0 _ll_register_callbacks cb 0x7f832c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:49:59.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.177+0000 7f8333e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5fcaee090 0x55b5fcafcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.177+0000 7f8333e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5fcaee090 0x55b5fcafcc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:50184/0 (socket says 172.21.15.111:50184) 2024-09-15T22:49:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.177+0000 7f8333400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5fcaee9d0 0x55b5fcafd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.177+0000 7f8333e00640 1 -- 172.21.15.111:0/3855827398 learned_addr learned my addr 172.21.15.111:0/3855827398 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:49:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.177+0000 7f8338c00640 1 --2- 172.21.15.111:0/3855827398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5fcaf3a60 0x55b5fcafe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.177+0000 7f8333400640 1 -- 172.21.15.111:0/3855827398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5fcaee090 msgr2=0x55b5fcafcc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.177+0000 7f8333400640 1 --2- 172.21.15.111:0/3855827398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5fcaee090 0x55b5fcafcc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.177+0000 7f8333400640 1 -- 172.21.15.111:0/3855827398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5fcaf3a60 msgr2=0x55b5fcafe120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:49:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.177+0000 7f8333400640 1 --2- 172.21.15.111:0/3855827398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5fcaf3a60 0x55b5fcafe120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.177+0000 7f8333400640 1 -- 172.21.15.111:0/3855827398 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5fcafe730 con 0x55b5fcaee9d0 2024-09-15T22:49:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.177+0000 7f8333e00640 1 --2- 172.21.15.111:0/3855827398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5fcaee090 0x55b5fcafcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:49:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.177+0000 7f8333400640 1 --2- 172.21.15.111:0/3855827398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5fcaee9d0 0x55b5fcafd160 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8320098c30 tx=0x7f832009bbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1939a710cb285971 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:49:59.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.177+0000 7f831d600640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:49:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.178+0000 7f831d600640 1 -- 172.21.15.111:0/3855827398 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83200d5070 con 0x55b5fcaee9d0 2024-09-15T22:49:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.178+0000 7f833dc4b740 1 -- 172.21.15.111:0/3855827398 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8324004d10 con 0x55b5fcaee9d0 2024-09-15T22:49:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.178+0000 7f831d600640 1 -- 172.21.15.111:0/3855827398 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f83200bc0b0 con 0x55b5fcaee9d0 2024-09-15T22:49:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.178+0000 7f831d600640 1 -- 172.21.15.111:0/3855827398 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f83200bc2b0 con 0x55b5fcaee9d0 2024-09-15T22:49:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.178+0000 7f833dc4b740 10 client.25342 fetch_fsmap learned FSMap version 29 2024-09-15T22:49:59.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.178+0000 7f833dc4b740 1 -- 172.21.15.111:0/3855827398 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b5fcb6e750 con 0x55b5fcaee9d0 2024-09-15T22:49:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.178+0000 7f831d600640 1 -- 172.21.15.111:0/3855827398 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8324004d10 con 0x55b5fcaee9d0 2024-09-15T22:49:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.178+0000 7f831d600640 1 -- 172.21.15.111:0/3855827398 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 29) ==== 8350+0+0 (secure 0 0 0) 0x7f8320018b30 con 0x55b5fcaee9d0 2024-09-15T22:49:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.178+0000 7f833dc4b740 10 client.25342 fetch_fsmap finished waiting for FSMap version 29 2024-09-15T22:49:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.178+0000 7f833dc4b740 10 client.25342 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:49:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.178+0000 7f833dc4b740 20 client.25342 populate_metadata read hostname 'smithi111' 2024-09-15T22:49:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.178+0000 7f833dc4b740 1 --2- 172.21.15.111:0/3855827398 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55b5fcb6f010 0x55b5fcb71430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:49:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.178+0000 7f833dc4b740 4 client.25342 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:49:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.178+0000 7f833dc4b740 1 -- 172.21.15.111:0/3855827398 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b5fca5ab80 con 0x55b5fcb6f010 2024-09-15T22:49:59.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.179+0000 7f8333e00640 1 --2- 172.21.15.111:0/3855827398 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55b5fcb6f010 0x55b5fcb71430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:49:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.179+0000 7f8333e00640 1 --2- 172.21.15.111:0/3855827398 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55b5fcb6f010 0x55b5fcb71430 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8a365aedc4770b4 server_cookie=98e3bd533bb8dd02 in_seq=0 out_seq=0 2024-09-15T22:49:59.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.179+0000 7f831d600640 10 client.25342 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:49:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.186+0000 7f831d600640 1 -- 172.21.15.111:0/3855827398 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b5fca5ab80 con 0x55b5fcb6f010 2024-09-15T22:49:59.188 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.186+0000 7f831d600640 10 client.25342 handle_command_reply: tid=0 2024-09-15T22:49:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.186+0000 7f833dc4b740 1 client.25342 shutdown 2024-09-15T22:49:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.187+0000 7f833dc4b740 1 -- 172.21.15.111:0/3855827398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5fcaee9d0 msgr2=0x55b5fcafd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.187+0000 7f833dc4b740 1 --2- 172.21.15.111:0/3855827398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5fcaee9d0 0x55b5fcafd160 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8320098c30 tx=0x7f832009bbc0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.187+0000 7f833dc4b740 1 -- 172.21.15.111:0/3855827398 shutdown_connections 2024-09-15T22:49:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.187+0000 7f833dc4b740 1 --2- 172.21.15.111:0/3855827398 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55b5fcb6f010 0x55b5fcb71430 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.187+0000 7f833dc4b740 1 --2- 172.21.15.111:0/3855827398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5fcaf3a60 0x55b5fcafe120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.187+0000 7f833dc4b740 1 --2- 172.21.15.111:0/3855827398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b5fcaee9d0 0x55b5fcafd160 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.187+0000 7f831d600640 0 client.25342 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:49:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.187+0000 7f833dc4b740 1 --2- 172.21.15.111:0/3855827398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5fcaee090 0x55b5fcafcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.187+0000 7f833dc4b740 1 -- 172.21.15.111:0/3855827398 >> 172.21.15.111:0/3855827398 conn(0x55b5fcae9700 msgr2=0x55b5fcae8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.187+0000 7f833dc4b740 1 -- 172.21.15.111:0/3855827398 shutdown_connections 2024-09-15T22:49:59.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.187+0000 7f833dc4b740 1 -- 172.21.15.111:0/3855827398 wait complete. 2024-09-15T22:49:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.187+0000 7f833dc4b740 20 client.25342 trim_cache size 0 max 16384 2024-09-15T22:49:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:49:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:49:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:49:59.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:49:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.188+0000 7f833ac00640 1 -- 172.21.15.111:0/1183905819 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f83180a3370 msgr2=0x7f83180a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.188+0000 7f833ac00640 1 --2- 172.21.15.111:0/1183905819 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f83180a3370 0x7f83180a5830 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f832402a9e0 tx=0x7f83240315b0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.188+0000 7f833ac00640 1 -- 172.21.15.111:0/1183905819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8334104f50 msgr2=0x7f833419a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:49:59.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.188+0000 7f833ac00640 1 --2- 172.21.15.111:0/1183905819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8334104f50 0x7f833419a510 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f832000e9f0 tx=0x7f832000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.188+0000 7f833ac00640 1 -- 172.21.15.111:0/1183905819 shutdown_connections 2024-09-15T22:49:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.188+0000 7f833ac00640 1 --2- 172.21.15.111:0/1183905819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8334105910 0x7f83341a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.188+0000 7f833ac00640 1 --2- 172.21.15.111:0/1183905819 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f83180a3370 0x7f83180a5830 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.188+0000 7f833ac00640 1 --2- 172.21.15.111:0/1183905819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8334104f50 0x7f833419a510 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.188+0000 7f833ac00640 1 --2- 172.21.15.111:0/1183905819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8334103d60 0x7f8334199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:49:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.188+0000 7f833ac00640 1 -- 172.21.15.111:0/1183905819 >> 172.21.15.111:0/1183905819 conn(0x7f83340ff4e0 msgr2=0x7f8334108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:49:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.188+0000 7f833ac00640 1 -- 172.21.15.111:0/1183905819 shutdown_connections 2024-09-15T22:49:59.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:49:59.188+0000 7f833ac00640 1 -- 172.21.15.111:0/1183905819 wait complete. 2024-09-15T22:49:59.254 INFO:tasks.workunit.client.0.smithi111.stdout: 1 289015 10.16 MB/sec execute 27 sec latency 13.855 ms 2024-09-15T22:49:59.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:49:59 smithi121 ceph-mon[34815]: pgmap v472: 97 pgs: 97 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-09-15T22:49:59.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:49:59 smithi111 ceph-mon[30629]: pgmap v472: 97 pgs: 97 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-09-15T22:50:00.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:49:59 smithi192 ceph-mon[35427]: pgmap v472: 97 pgs: 97 active+clean; 119 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-09-15T22:50:00.255 INFO:tasks.workunit.client.0.smithi111.stdout: 1 291608 10.31 MB/sec execute 28 sec latency 12.111 ms 2024-09-15T22:50:00.889 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:00 smithi111 ceph-mon[30629]: overall HEALTH_OK 2024-09-15T22:50:00.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:00 smithi121 ceph-mon[34815]: overall HEALTH_OK 2024-09-15T22:50:01.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:00 smithi192 ceph-mon[35427]: overall HEALTH_OK 2024-09-15T22:50:01.255 INFO:tasks.workunit.client.0.smithi111.stdout: 1 293643 10.32 MB/sec execute 29 sec latency 429.419 ms 2024-09-15T22:50:01.708 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:01 smithi111 ceph-mon[30629]: pgmap v473: 97 pgs: 97 active+clean; 120 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 187 op/s 2024-09-15T22:50:01.895 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:01 smithi121 ceph-mon[34815]: pgmap v473: 97 pgs: 97 active+clean; 120 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 187 op/s 2024-09-15T22:50:02.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:01 smithi192 ceph-mon[35427]: pgmap v473: 97 pgs: 97 active+clean; 120 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 187 op/s 2024-09-15T22:50:02.255 INFO:tasks.workunit.client.0.smithi111.stdout: 1 293643 9.97 MB/sec execute 30 sec latency 1429.483 ms 2024-09-15T22:50:03.255 INFO:tasks.workunit.client.0.smithi111.stdout: 1 295084 9.92 MB/sec execute 31 sec latency 1745.589 ms 2024-09-15T22:50:03.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:03 smithi121 ceph-mon[34815]: pgmap v474: 97 pgs: 97 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 243 op/s 2024-09-15T22:50:03.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:03 smithi111 ceph-mon[30629]: pgmap v474: 97 pgs: 97 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 243 op/s 2024-09-15T22:50:04.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:03 smithi192 ceph-mon[35427]: pgmap v474: 97 pgs: 97 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 243 op/s 2024-09-15T22:50:04.255 INFO:tasks.workunit.client.0.smithi111.stdout: 1 298256 10.19 MB/sec execute 32 sec latency 13.344 ms 2024-09-15T22:50:05.255 INFO:tasks.workunit.client.0.smithi111.stdout: 1 301162 10.30 MB/sec execute 33 sec latency 13.150 ms 2024-09-15T22:50:05.887 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:05 smithi111 ceph-mon[30629]: pgmap v475: 97 pgs: 97 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 202 op/s 2024-09-15T22:50:05.889 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:05 smithi121 ceph-mon[34815]: pgmap v475: 97 pgs: 97 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 202 op/s 2024-09-15T22:50:06.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:05 smithi192 ceph-mon[35427]: pgmap v475: 97 pgs: 97 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 202 op/s 2024-09-15T22:50:06.255 INFO:tasks.workunit.client.0.smithi111.stdout: 1 302277 10.18 MB/sec execute 34 sec latency 246.125 ms 2024-09-15T22:50:06.754 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:06 smithi111 ceph-mon[30629]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:50:07.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:06 smithi192 ceph-mon[35427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:50:07.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:06 smithi121 ceph-mon[34815]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:50:07.255 INFO:tasks.workunit.client.0.smithi111.stdout: 1 302277 9.88 MB/sec execute 35 sec latency 1080.115 ms 2024-09-15T22:50:07.892 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:07 smithi111 ceph-mon[30629]: pgmap v476: 97 pgs: 97 active+clean; 125 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 214 op/s 2024-09-15T22:50:08.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:07 smithi192 ceph-mon[35427]: pgmap v476: 97 pgs: 97 active+clean; 125 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 214 op/s 2024-09-15T22:50:08.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:07 smithi121 ceph-mon[34815]: pgmap v476: 97 pgs: 97 active+clean; 125 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 214 op/s 2024-09-15T22:50:08.255 INFO:tasks.workunit.client.0.smithi111.stdout: 1 303041 9.74 MB/sec execute 36 sec latency 1811.096 ms 2024-09-15T22:50:09.255 INFO:tasks.workunit.client.0.smithi111.stdout: 1 305851 9.87 MB/sec execute 37 sec latency 20.641 ms 2024-09-15T22:50:09.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:09 smithi111 ceph-mon[30629]: pgmap v477: 97 pgs: 97 active+clean; 127 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 220 op/s 2024-09-15T22:50:10.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:09 smithi192 ceph-mon[35427]: pgmap v477: 97 pgs: 97 active+clean; 127 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 220 op/s 2024-09-15T22:50:10.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:09 smithi121 ceph-mon[34815]: pgmap v477: 97 pgs: 97 active+clean; 127 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.4 MiB/s wr, 220 op/s 2024-09-15T22:50:10.255 INFO:tasks.workunit.client.0.smithi111.stdout: 1 308992 10.10 MB/sec execute 38 sec latency 7.802 ms 2024-09-15T22:50:11.255 INFO:tasks.workunit.client.0.smithi111.stdout: 1 311376 10.12 MB/sec execute 39 sec latency 109.779 ms 2024-09-15T22:50:11.894 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:11 smithi111 ceph-mon[30629]: pgmap v478: 97 pgs: 97 active+clean; 127 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 172 op/s 2024-09-15T22:50:12.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:11 smithi192 ceph-mon[35427]: pgmap v478: 97 pgs: 97 active+clean; 127 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 172 op/s 2024-09-15T22:50:12.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:11 smithi121 ceph-mon[34815]: pgmap v478: 97 pgs: 97 active+clean; 127 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 172 op/s 2024-09-15T22:50:12.255 INFO:tasks.workunit.client.0.smithi111.stdout: 1 311376 9.87 MB/sec execute 40 sec latency 1109.828 ms 2024-09-15T22:50:12.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.303+0000 7f9206800640 1 -- 172.21.15.111:0/4264118425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9200074680 msgr2=0x7f9200074a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:12.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.303+0000 7f9206800640 1 --2- 172.21.15.111:0/4264118425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9200074680 0x7f9200074a80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009860 tx=0x7f91f402a330 comp rx=0 tx=0).stop 2024-09-15T22:50:12.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.303+0000 7f9206800640 1 -- 172.21.15.111:0/4264118425 shutdown_connections 2024-09-15T22:50:12.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.303+0000 7f9206800640 1 --2- 172.21.15.111:0/4264118425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9200109c90 0x7f920010c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.303+0000 7f9206800640 1 --2- 172.21.15.111:0/4264118425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9200074fc0 0x7f9200109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.303+0000 7f9206800640 1 --2- 172.21.15.111:0/4264118425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9200074680 0x7f9200074a80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.303+0000 7f9206800640 1 -- 172.21.15.111:0/4264118425 >> 172.21.15.111:0/4264118425 conn(0x7f92000fd0d0 msgr2=0x7f92000ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:50:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.303+0000 7f9206800640 1 -- 172.21.15.111:0/4264118425 shutdown_connections 2024-09-15T22:50:12.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.303+0000 7f9206800640 1 -- 172.21.15.111:0/4264118425 wait complete. 2024-09-15T22:50:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.304+0000 7f9206800640 1 Processor -- start 2024-09-15T22:50:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.304+0000 7f9206800640 1 -- start start 2024-09-15T22:50:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.305+0000 7f9206800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9200074680 0x7f920019a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.305+0000 7f9206800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9200074fc0 0x7f920019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.305+0000 7f9206800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9200109c90 0x7f92001a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.305+0000 7f9206800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f920010f8f0 con 0x7f9200109c90 2024-09-15T22:50:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.305+0000 7f9206800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f920010f770 con 0x7f9200074fc0 2024-09-15T22:50:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.305+0000 7f9206800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f920010fa70 con 0x7f9200074680 2024-09-15T22:50:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.305+0000 7f9204a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9200074fc0 0x7f920019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.305+0000 7f9204a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9200074fc0 0x7f920019a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:44234/0 (socket says 172.21.15.111:44234) 2024-09-15T22:50:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.305+0000 7f9204a00640 1 -- 172.21.15.111:0/237652078 learned_addr learned my addr 172.21.15.111:0/237652078 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:50:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.305+0000 7f9205400640 1 --2- 172.21.15.111:0/237652078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9200074680 0x7f920019a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.305+0000 7f9205e00640 1 --2- 172.21.15.111:0/237652078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9200109c90 0x7f92001a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.305+0000 7f9204a00640 1 -- 172.21.15.111:0/237652078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9200074680 msgr2=0x7f920019a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.305+0000 7f9204a00640 1 --2- 172.21.15.111:0/237652078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9200074680 0x7f920019a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.305+0000 7f9204a00640 1 -- 172.21.15.111:0/237652078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9200109c90 msgr2=0x7f92001a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.305+0000 7f9204a00640 1 --2- 172.21.15.111:0/237652078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9200109c90 0x7f92001a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.305+0000 7f9204a00640 1 -- 172.21.15.111:0/237652078 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92001a1c20 con 0x7f9200074fc0 2024-09-15T22:50:12.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.305+0000 7f9204a00640 1 --2- 172.21.15.111:0/237652078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9200074fc0 0x7f920019a540 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f91ec00e9f0 tx=0x7f91ec00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:50:12.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.306+0000 7f91fe000640 1 -- 172.21.15.111:0/237652078 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ec00cd80 con 0x7f9200074fc0 2024-09-15T22:50:12.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.306+0000 7f91fe000640 1 -- 172.21.15.111:0/237652078 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91ec005500 con 0x7f9200074fc0 2024-09-15T22:50:12.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.306+0000 7f91fe000640 1 -- 172.21.15.111:0/237652078 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ec010690 con 0x7f9200074fc0 2024-09-15T22:50:12.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.306+0000 7f9206800640 1 -- 172.21.15.111:0/237652078 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92001a1f10 con 0x7f9200074fc0 2024-09-15T22:50:12.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.306+0000 7f9206800640 1 -- 172.21.15.111:0/237652078 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92001a2380 con 0x7f9200074fc0 2024-09-15T22:50:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.309+0000 7f91fe000640 1 -- 172.21.15.111:0/237652078 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f91ec0046c0 con 0x7f9200074fc0 2024-09-15T22:50:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.309+0000 7f920a922740 1 Processor -- start 2024-09-15T22:50:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.309+0000 7f920a922740 1 -- start start 2024-09-15T22:50:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.309+0000 7f920a922740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b451260 0x556a5b451660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.309+0000 7f920a922740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a5b451ba0 0x556a5b456660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.309+0000 7f920a922740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5b456c30 0x556a5b4590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.309+0000 7f920a922740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556a5b3b6100 con 0x556a5b451ba0 2024-09-15T22:50:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.309+0000 7f920a922740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556a5b35e1b0 con 0x556a5b451260 2024-09-15T22:50:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.309+0000 7f920a922740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556a5b3b20c0 con 0x556a5b456c30 2024-09-15T22:50:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.310+0000 7f9205e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b451260 0x556a5b451660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.310+0000 7f9205400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a5b451ba0 0x556a5b456660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.310+0000 7f9205e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b451260 0x556a5b451660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:44244/0 (socket says 172.21.15.111:44244) 2024-09-15T22:50:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.310+0000 7f9205e00640 1 -- 172.21.15.111:0/2111557426 learned_addr learned my addr 172.21.15.111:0/2111557426 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:50:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.310+0000 7f9204a00640 1 --2- 172.21.15.111:0/2111557426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5b456c30 0x556a5b4590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.310+0000 7f9205e00640 1 -- 172.21.15.111:0/2111557426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5b456c30 msgr2=0x556a5b4590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.310+0000 7f9205e00640 1 --2- 172.21.15.111:0/2111557426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5b456c30 0x556a5b4590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.310+0000 7f9205e00640 1 -- 172.21.15.111:0/2111557426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a5b451ba0 msgr2=0x556a5b456660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.310+0000 7f9205e00640 1 --2- 172.21.15.111:0/2111557426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a5b451ba0 0x556a5b456660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.310+0000 7f9205e00640 1 -- 172.21.15.111:0/2111557426 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a5b0c9210 con 0x556a5b451260 2024-09-15T22:50:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.310+0000 7f9205e00640 1 --2- 172.21.15.111:0/2111557426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b451260 0x556a5b451660 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f91f800f480 tx=0x7f91f800f940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd267027f884467c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:50:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.310+0000 7f91fe000640 1 --2- 172.21.15.111:0/237652078 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f91e80a32c0 0x7f91e80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.311+0000 7f91f2600640 1 -- 172.21.15.111:0/2111557426 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f801d070 con 0x556a5b451260 2024-09-15T22:50:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.311+0000 7f91f2600640 1 -- 172.21.15.111:0/2111557426 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91f800cb80 con 0x556a5b451260 2024-09-15T22:50:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.311+0000 7f91f2600640 1 -- 172.21.15.111:0/2111557426 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8018d50 con 0x556a5b451260 2024-09-15T22:50:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.311+0000 7f91fe000640 1 -- 172.21.15.111:0/237652078 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f91ec0cfd10 con 0x7f9200074fc0 2024-09-15T22:50:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.311+0000 7f9205400640 1 --2- 172.21.15.111:0/237652078 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f91e80a32c0 0x7f91e80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.311+0000 7f920a922740 1 -- 172.21.15.111:0/2111557426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b451260 msgr2=0x556a5b451660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.311+0000 7f920a922740 1 --2- 172.21.15.111:0/2111557426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b451260 0x556a5b451660 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f91f800f480 tx=0x7f91f800f940 comp rx=0 tx=0).stop 2024-09-15T22:50:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.311+0000 7f9205400640 1 --2- 172.21.15.111:0/237652078 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f91e80a32c0 0x7f91e80a5780 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009830 tx=0x7f91f40305a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:50:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.312+0000 7f920a922740 1 -- 172.21.15.111:0/2111557426 shutdown_connections 2024-09-15T22:50:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.312+0000 7f920a922740 1 --2- 172.21.15.111:0/2111557426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5b456c30 0x556a5b4590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.312+0000 7f920a922740 1 --2- 172.21.15.111:0/2111557426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a5b451ba0 0x556a5b456660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.312+0000 7f920a922740 1 --2- 172.21.15.111:0/2111557426 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b451260 0x556a5b451660 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.312+0000 7f920a922740 1 -- 172.21.15.111:0/2111557426 >> 172.21.15.111:0/2111557426 conn(0x556a5b44c8d0 msgr2=0x556a5b44ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:50:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.312+0000 7f920a922740 1 -- 172.21.15.111:0/2111557426 shutdown_connections 2024-09-15T22:50:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.312+0000 7f920a922740 1 -- 172.21.15.111:0/2111557426 wait complete. 2024-09-15T22:50:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.312+0000 7f920a922740 1 -- start start 2024-09-15T22:50:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.312+0000 7f920a922740 1 Processor -- start 2024-09-15T22:50:12.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.313+0000 7f920a922740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b451260 0x556a5b45b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.313+0000 7f920a922740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a5b451ba0 0x556a5b45b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.313+0000 7f920a922740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5b456c30 0x556a5b4629f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.313+0000 7f920a922740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556a5b459c00 con 0x556a5b451ba0 2024-09-15T22:50:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.313+0000 7f920a922740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556a5b33f090 con 0x556a5b451260 2024-09-15T22:50:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.313+0000 7f920a922740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556a5b459d80 con 0x556a5b456c30 2024-09-15T22:50:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.313+0000 7f920a922740 10 client.0 _ll_register_callbacks cb 0x556a5b393f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:50:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.313+0000 7f9204a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a5b451ba0 0x556a5b45b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.313+0000 7f9205400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b451260 0x556a5b45b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.313+0000 7f9204a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a5b451ba0 0x556a5b45b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34314/0 (socket says 172.21.15.111:34314) 2024-09-15T22:50:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.313+0000 7f9204a00640 1 -- 172.21.15.111:0/4164462719 learned_addr learned my addr 172.21.15.111:0/4164462719 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:50:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.313+0000 7f9205e00640 1 --2- 172.21.15.111:0/4164462719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5b456c30 0x556a5b4629f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.313+0000 7f9204a00640 1 -- 172.21.15.111:0/4164462719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5b456c30 msgr2=0x556a5b4629f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.313+0000 7f9204a00640 1 --2- 172.21.15.111:0/4164462719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5b456c30 0x556a5b4629f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.313+0000 7f9204a00640 1 -- 172.21.15.111:0/4164462719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b451260 msgr2=0x556a5b45b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.313+0000 7f9204a00640 1 --2- 172.21.15.111:0/4164462719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b451260 0x556a5b45b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.313+0000 7f9204a00640 1 -- 172.21.15.111:0/4164462719 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a5b195620 con 0x556a5b451ba0 2024-09-15T22:50:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.314+0000 7f9204a00640 1 --2- 172.21.15.111:0/4164462719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a5b451ba0 0x556a5b45b9b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f91ec098790 tx=0x7f91ec009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=590b5d2a8df70fb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:50:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.314+0000 7f91e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:50:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.314+0000 7f91e6a00640 1 -- 172.21.15.111:0/4164462719 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ec09be70 con 0x556a5b451ba0 2024-09-15T22:50:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.314+0000 7f91e6a00640 1 -- 172.21.15.111:0/4164462719 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f91ec0a4b10 con 0x556a5b451ba0 2024-09-15T22:50:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.314+0000 7f920a922740 1 -- 172.21.15.111:0/4164462719 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91f800cb80 con 0x556a5b451ba0 2024-09-15T22:50:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.314+0000 7f91e6a00640 1 -- 172.21.15.111:0/4164462719 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f91ec09bc90 con 0x556a5b451ba0 2024-09-15T22:50:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.314+0000 7f920a922740 10 client.15333 fetch_fsmap learned FSMap version 30 2024-09-15T22:50:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.314+0000 7f920a922740 1 -- 172.21.15.111:0/4164462719 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556a5b4658b0 con 0x556a5b451ba0 2024-09-15T22:50:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.314+0000 7f91e6a00640 1 -- 172.21.15.111:0/4164462719 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91f800cb80 con 0x556a5b451ba0 2024-09-15T22:50:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.314+0000 7f91e6a00640 1 -- 172.21.15.111:0/4164462719 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f91ec0d5070 con 0x556a5b451ba0 2024-09-15T22:50:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.315+0000 7f920a922740 10 client.15333 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:50:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.315+0000 7f920a922740 10 client.15333 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:50:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.315+0000 7f920a922740 20 client.15333 populate_metadata read hostname 'smithi111' 2024-09-15T22:50:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.315+0000 7f920a922740 1 --2- 172.21.15.111:0/4164462719 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556a5b466f30 0x556a5b469320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.315+0000 7f920a922740 4 client.15333 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:50:12.320 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.315+0000 7f920a922740 1 -- 172.21.15.111:0/4164462719 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a5b3c3830 con 0x556a5b466f30 2024-09-15T22:50:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.315+0000 7f9205400640 1 --2- 172.21.15.111:0/4164462719 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556a5b466f30 0x556a5b469320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.315+0000 7f9205400640 1 --2- 172.21.15.111:0/4164462719 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556a5b466f30 0x556a5b469320 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=702f3d769a1d2312 server_cookie=48f924cc95da9a10 in_seq=0 out_seq=0 2024-09-15T22:50:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.315+0000 7f91e6a00640 10 client.15333 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:50:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.316+0000 7f91e6a00640 1 -- 172.21.15.111:0/4164462719 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556a5b3c3830 con 0x556a5b466f30 2024-09-15T22:50:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.316+0000 7f91e6a00640 10 client.15333 handle_command_reply: tid=0 2024-09-15T22:50:12.321 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.316+0000 7f920a922740 1 client.15333 shutdown 2024-09-15T22:50:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.316+0000 7f920a922740 1 -- 172.21.15.111:0/4164462719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a5b451ba0 msgr2=0x556a5b45b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.316+0000 7f920a922740 1 --2- 172.21.15.111:0/4164462719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a5b451ba0 0x556a5b45b9b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f91ec098790 tx=0x7f91ec009310 comp rx=0 tx=0).stop 2024-09-15T22:50:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.316+0000 7f920a922740 1 -- 172.21.15.111:0/4164462719 shutdown_connections 2024-09-15T22:50:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.316+0000 7f920a922740 1 --2- 172.21.15.111:0/4164462719 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556a5b466f30 0x556a5b469320 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.316+0000 7f920a922740 1 --2- 172.21.15.111:0/4164462719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5b456c30 0x556a5b4629f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.316+0000 7f920a922740 1 --2- 172.21.15.111:0/4164462719 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a5b451ba0 0x556a5b45b9b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.316+0000 7f920a922740 1 --2- 172.21.15.111:0/4164462719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b451260 0x556a5b45b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.316+0000 7f920a922740 1 -- 172.21.15.111:0/4164462719 >> 172.21.15.111:0/4164462719 conn(0x556a5b44c8d0 msgr2=0x556a5b44e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:50:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.316+0000 7f91e6a00640 0 client.15333 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:50:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.317+0000 7f920a922740 1 -- 172.21.15.111:0/4164462719 shutdown_connections 2024-09-15T22:50:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.317+0000 7f920a922740 1 -- 172.21.15.111:0/4164462719 wait complete. 2024-09-15T22:50:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.317+0000 7f920a922740 20 client.15333 trim_cache size 0 max 16384 2024-09-15T22:50:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.331+0000 7f920a922740 1 Processor -- start 2024-09-15T22:50:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.331+0000 7f920a922740 1 -- start start 2024-09-15T22:50:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.331+0000 7f920a922740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5b451260 0x556a5b44b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.331+0000 7f920a922740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b451ba0 0x556a5b44be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.331+0000 7f920a922740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a5b456c30 0x556a5b4629b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.331+0000 7f920a922740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f92001ad850 con 0x556a5b456c30 2024-09-15T22:50:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.331+0000 7f920a922740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f92001ab120 con 0x556a5b451ba0 2024-09-15T22:50:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.331+0000 7f920a922740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f92001a8f70 con 0x556a5b451260 2024-09-15T22:50:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.332+0000 7f9205e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a5b456c30 0x556a5b4629b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.332+0000 7f9205e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a5b456c30 0x556a5b4629b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34324/0 (socket says 172.21.15.111:34324) 2024-09-15T22:50:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.332+0000 7f9205e00640 1 -- 172.21.15.111:0/249306095 learned_addr learned my addr 172.21.15.111:0/249306095 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:50:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.332+0000 7f9204a00640 1 --2- 172.21.15.111:0/249306095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b451ba0 0x556a5b44be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.332+0000 7f9205400640 1 --2- 172.21.15.111:0/249306095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5b451260 0x556a5b44b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.332+0000 7f9205e00640 1 -- 172.21.15.111:0/249306095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5b451260 msgr2=0x556a5b44b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.332+0000 7f9205e00640 1 --2- 172.21.15.111:0/249306095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5b451260 0x556a5b44b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.332+0000 7f9205e00640 1 -- 172.21.15.111:0/249306095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b451ba0 msgr2=0x556a5b44be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.332+0000 7f9205e00640 1 --2- 172.21.15.111:0/249306095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b451ba0 0x556a5b44be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.332+0000 7f9205e00640 1 -- 172.21.15.111:0/249306095 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a5b452080 con 0x556a5b456c30 2024-09-15T22:50:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.332+0000 7f9204a00640 1 --2- 172.21.15.111:0/249306095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b451ba0 0x556a5b44be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:50:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.332+0000 7f9205e00640 1 --2- 172.21.15.111:0/249306095 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a5b456c30 0x556a5b4629b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f91f800f480 tx=0x7f91f80185a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ba34d79ce14885 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:50:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.333+0000 7f91f1c00640 1 -- 172.21.15.111:0/249306095 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8026020 con 0x556a5b456c30 2024-09-15T22:50:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.333+0000 7f91f1c00640 1 -- 172.21.15.111:0/249306095 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91f8009ae0 con 0x556a5b456c30 2024-09-15T22:50:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.333+0000 7f91f1c00640 1 -- 172.21.15.111:0/249306095 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f801d040 con 0x556a5b456c30 2024-09-15T22:50:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.333+0000 7f920a922740 1 -- 172.21.15.111:0/249306095 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a5b456c30 msgr2=0x556a5b4629b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.333+0000 7f920a922740 1 --2- 172.21.15.111:0/249306095 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a5b456c30 0x556a5b4629b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f91f800f480 tx=0x7f91f80185a0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.333+0000 7f920a922740 1 -- 172.21.15.111:0/249306095 shutdown_connections 2024-09-15T22:50:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.333+0000 7f920a922740 1 --2- 172.21.15.111:0/249306095 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a5b456c30 0x556a5b4629b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.333+0000 7f920a922740 1 --2- 172.21.15.111:0/249306095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b451ba0 0x556a5b44be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.333+0000 7f920a922740 1 --2- 172.21.15.111:0/249306095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5b451260 0x556a5b44b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.333+0000 7f920a922740 1 -- 172.21.15.111:0/249306095 >> 172.21.15.111:0/249306095 conn(0x556a5b44c8d0 msgr2=0x556a5b44b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:50:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.333+0000 7f920a922740 1 -- 172.21.15.111:0/249306095 shutdown_connections 2024-09-15T22:50:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.333+0000 7f920a922740 1 -- 172.21.15.111:0/249306095 wait complete. 2024-09-15T22:50:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.334+0000 7f920a922740 1 -- start start 2024-09-15T22:50:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.334+0000 7f920a922740 1 Processor -- start 2024-09-15T22:50:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.334+0000 7f920a922740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a5b451260 0x556a5b4604c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.334+0000 7f920a922740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5b451ba0 0x556a5b460a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.334+0000 7f920a922740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b456c30 0x556a5b461a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.334+0000 7f920a922740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f92001a95e0 con 0x556a5b451260 2024-09-15T22:50:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.334+0000 7f920a922740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f92001aaa50 con 0x556a5b456c30 2024-09-15T22:50:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.334+0000 7f920a922740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f92001a9cb0 con 0x556a5b451ba0 2024-09-15T22:50:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.334+0000 7f920a922740 10 client.0 _ll_register_callbacks cb 0x7f91ec0d3440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:50:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.334+0000 7f9205400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a5b451260 0x556a5b4604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.334+0000 7f9205400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a5b451260 0x556a5b4604c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34326/0 (socket says 172.21.15.111:34326) 2024-09-15T22:50:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.334+0000 7f9205400640 1 -- 172.21.15.111:0/987056397 learned_addr learned my addr 172.21.15.111:0/987056397 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:50:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.335+0000 7f9205e00640 1 --2- 172.21.15.111:0/987056397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b456c30 0x556a5b461a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:12.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.335+0000 7f9204a00640 1 --2- 172.21.15.111:0/987056397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5b451ba0 0x556a5b460a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.335+0000 7f9205400640 1 -- 172.21.15.111:0/987056397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5b451ba0 msgr2=0x556a5b460a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.335+0000 7f9205400640 1 --2- 172.21.15.111:0/987056397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5b451ba0 0x556a5b460a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.335+0000 7f9205400640 1 -- 172.21.15.111:0/987056397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b456c30 msgr2=0x556a5b461a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.335+0000 7f9205400640 1 --2- 172.21.15.111:0/987056397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b456c30 0x556a5b461a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.335+0000 7f9205400640 1 -- 172.21.15.111:0/987056397 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a5b462020 con 0x556a5b451260 2024-09-15T22:50:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.335+0000 7f9204a00640 1 --2- 172.21.15.111:0/987056397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5b451ba0 0x556a5b460a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:50:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.335+0000 7f9205400640 1 --2- 172.21.15.111:0/987056397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a5b451260 0x556a5b4604c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f91f402a920 tx=0x7f91f4030a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b62e0cb63d6170 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:50:12.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.335+0000 7f91e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:50:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.335+0000 7f91e6a00640 1 -- 172.21.15.111:0/987056397 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f4057060 con 0x556a5b451260 2024-09-15T22:50:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.335+0000 7f91e6a00640 1 -- 172.21.15.111:0/987056397 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f91f4005240 con 0x556a5b451260 2024-09-15T22:50:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.336+0000 7f920a922740 1 -- 172.21.15.111:0/987056397 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91f8009ae0 con 0x556a5b451260 2024-09-15T22:50:12.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.336+0000 7f91e6a00640 1 -- 172.21.15.111:0/987056397 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f91f4037040 con 0x556a5b451260 2024-09-15T22:50:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.336+0000 7f920a922740 10 client.15339 fetch_fsmap learned FSMap version 30 2024-09-15T22:50:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.336+0000 7f920a922740 1 -- 172.21.15.111:0/987056397 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556a5b4d20b0 con 0x556a5b451260 2024-09-15T22:50:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.336+0000 7f91e6a00640 1 -- 172.21.15.111:0/987056397 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91f8009ae0 con 0x556a5b451260 2024-09-15T22:50:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.336+0000 7f91e6a00640 1 -- 172.21.15.111:0/987056397 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f91f405c070 con 0x556a5b451260 2024-09-15T22:50:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.336+0000 7f920a922740 10 client.15339 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:50:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.336+0000 7f920a922740 10 client.15339 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:50:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.336+0000 7f920a922740 20 client.15339 populate_metadata read hostname 'smithi111' 2024-09-15T22:50:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.336+0000 7f920a922740 1 --2- 172.21.15.111:0/987056397 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556a5b4d2a90 0x556a5b4d4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.336+0000 7f920a922740 4 client.15339 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:50:12.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.336+0000 7f920a922740 1 -- 172.21.15.111:0/987056397 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556a5b447c80 con 0x556a5b4d2a90 2024-09-15T22:50:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.337+0000 7f9204a00640 1 --2- 172.21.15.111:0/987056397 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556a5b4d2a90 0x556a5b4d4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.337+0000 7f9204a00640 1 --2- 172.21.15.111:0/987056397 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556a5b4d2a90 0x556a5b4d4eb0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dec2f0d9a59ca201 server_cookie=b6ac035d53749a18 in_seq=0 out_seq=0 2024-09-15T22:50:12.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.337+0000 7f91e6a00640 10 client.15339 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:50:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.345+0000 7f91e6a00640 1 -- 172.21.15.111:0/987056397 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556a5b447c80 con 0x556a5b4d2a90 2024-09-15T22:50:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.345+0000 7f91e6a00640 10 client.15339 handle_command_reply: tid=0 2024-09-15T22:50:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.345+0000 7f920a922740 1 client.15339 shutdown 2024-09-15T22:50:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.345+0000 7f920a922740 1 -- 172.21.15.111:0/987056397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a5b451260 msgr2=0x556a5b4604c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.345+0000 7f920a922740 1 --2- 172.21.15.111:0/987056397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a5b451260 0x556a5b4604c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f91f402a920 tx=0x7f91f4030a50 comp rx=0 tx=0).stop 2024-09-15T22:50:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.345+0000 7f920a922740 1 -- 172.21.15.111:0/987056397 shutdown_connections 2024-09-15T22:50:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.345+0000 7f920a922740 1 --2- 172.21.15.111:0/987056397 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556a5b4d2a90 0x556a5b4d4eb0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.345+0000 7f920a922740 1 --2- 172.21.15.111:0/987056397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a5b456c30 0x556a5b461a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.345+0000 7f920a922740 1 --2- 172.21.15.111:0/987056397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556a5b451ba0 0x556a5b460a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.345+0000 7f920a922740 1 --2- 172.21.15.111:0/987056397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556a5b451260 0x556a5b4604c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.345+0000 7f920a922740 1 -- 172.21.15.111:0/987056397 >> 172.21.15.111:0/987056397 conn(0x556a5b44c8d0 msgr2=0x556a5b44ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:50:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.345+0000 7f91e6a00640 0 client.15339 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:50:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.346+0000 7f920a922740 1 -- 172.21.15.111:0/987056397 shutdown_connections 2024-09-15T22:50:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.346+0000 7f920a922740 1 -- 172.21.15.111:0/987056397 wait complete. 2024-09-15T22:50:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.346+0000 7f920a922740 20 client.15339 trim_cache size 0 max 16384 2024-09-15T22:50:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:50:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:50:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:50:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:50:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.346+0000 7f9206800640 1 -- 172.21.15.111:0/237652078 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f91e80a32c0 msgr2=0x7f91e80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.346+0000 7f9206800640 1 --2- 172.21.15.111:0/237652078 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f91e80a32c0 0x7f91e80a5780 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009830 tx=0x7f91f40305a0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.346+0000 7f9206800640 1 -- 172.21.15.111:0/237652078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9200074fc0 msgr2=0x7f920019a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.346+0000 7f9206800640 1 --2- 172.21.15.111:0/237652078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9200074fc0 0x7f920019a540 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f91ec00e9f0 tx=0x7f91ec00edf0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.346+0000 7f9206800640 1 -- 172.21.15.111:0/237652078 shutdown_connections 2024-09-15T22:50:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.346+0000 7f9206800640 1 --2- 172.21.15.111:0/237652078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9200109c90 0x7f92001a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.346+0000 7f9206800640 1 --2- 172.21.15.111:0/237652078 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f91e80a32c0 0x7f91e80a5780 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.346+0000 7f9206800640 1 --2- 172.21.15.111:0/237652078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9200074fc0 0x7f920019a540 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.346+0000 7f9206800640 1 --2- 172.21.15.111:0/237652078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9200074680 0x7f920019a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.346+0000 7f9206800640 1 -- 172.21.15.111:0/237652078 >> 172.21.15.111:0/237652078 conn(0x7f92000fd0d0 msgr2=0x7f9200105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:50:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.347+0000 7f9206800640 1 -- 172.21.15.111:0/237652078 shutdown_connections 2024-09-15T22:50:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:12.347+0000 7f9206800640 1 -- 172.21.15.111:0/237652078 wait complete. 2024-09-15T22:50:13.255 INFO:tasks.workunit.client.0.smithi111.stdout: 1 312530 9.83 MB/sec execute 41 sec latency 1635.517 ms 2024-09-15T22:50:13.917 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:13 smithi111 ceph-mon[30629]: pgmap v479: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2024-09-15T22:50:14.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:13 smithi192 ceph-mon[35427]: pgmap v479: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2024-09-15T22:50:14.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:13 smithi121 ceph-mon[34815]: pgmap v479: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2024-09-15T22:50:14.255 INFO:tasks.workunit.client.0.smithi111.stdout: 1 315646 10.01 MB/sec execute 42 sec latency 11.163 ms 2024-09-15T22:50:15.255 INFO:tasks.workunit.client.0.smithi111.stdout: 1 318082 9.98 MB/sec execute 43 sec latency 29.520 ms 2024-09-15T22:50:15.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:15 smithi111 ceph-mon[30629]: pgmap v480: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 171 op/s 2024-09-15T22:50:16.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:15 smithi192 ceph-mon[35427]: pgmap v480: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 171 op/s 2024-09-15T22:50:16.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:15 smithi121 ceph-mon[34815]: pgmap v480: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 171 op/s 2024-09-15T22:50:16.255 INFO:tasks.workunit.client.0.smithi111.stdout: 1 319998 10.04 MB/sec execute 44 sec latency 77.665 ms 2024-09-15T22:50:17.255 INFO:tasks.workunit.client.0.smithi111.stdout: 1 320001 9.81 MB/sec execute 45 sec latency 768.073 ms 2024-09-15T22:50:17.833 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:17 smithi111 ceph-mon[30629]: pgmap v481: 97 pgs: 97 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 209 op/s 2024-09-15T22:50:18.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:17 smithi192 ceph-mon[35427]: pgmap v481: 97 pgs: 97 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 209 op/s 2024-09-15T22:50:18.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:17 smithi121 ceph-mon[34815]: pgmap v481: 97 pgs: 97 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 209 op/s 2024-09-15T22:50:18.255 INFO:tasks.workunit.client.0.smithi111.stdout: 1 320001 9.60 MB/sec execute 46 sec latency 1309.696 ms 2024-09-15T22:50:19.255 INFO:tasks.workunit.client.0.smithi111.stdout: 1 323023 9.79 MB/sec execute 47 sec latency 1434.798 ms 2024-09-15T22:50:19.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:19 smithi111 ceph-mon[30629]: pgmap v482: 97 pgs: 97 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 173 op/s 2024-09-15T22:50:20.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:19 smithi192 ceph-mon[35427]: pgmap v482: 97 pgs: 97 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 173 op/s 2024-09-15T22:50:20.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:19 smithi121 ceph-mon[34815]: pgmap v482: 97 pgs: 97 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 173 op/s 2024-09-15T22:50:20.256 INFO:tasks.workunit.client.0.smithi111.stdout: 1 325859 9.86 MB/sec execute 48 sec latency 15.250 ms 2024-09-15T22:50:21.256 INFO:tasks.workunit.client.0.smithi111.stdout: 1 328429 9.90 MB/sec execute 49 sec latency 13.370 ms 2024-09-15T22:50:21.867 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:21 smithi111 ceph-mon[30629]: pgmap v483: 97 pgs: 97 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 168 op/s 2024-09-15T22:50:22.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:21 smithi192 ceph-mon[35427]: pgmap v483: 97 pgs: 97 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 168 op/s 2024-09-15T22:50:22.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:21 smithi121 ceph-mon[34815]: pgmap v483: 97 pgs: 97 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 168 op/s 2024-09-15T22:50:22.256 INFO:tasks.workunit.client.0.smithi111.stdout: 1 328634 9.72 MB/sec execute 50 sec latency 875.301 ms 2024-09-15T22:50:23.256 INFO:tasks.workunit.client.0.smithi111.stdout: 1 329640 9.68 MB/sec execute 51 sec latency 724.505 ms 2024-09-15T22:50:23.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:23 smithi111 ceph-mon[30629]: pgmap v484: 97 pgs: 97 active+clean; 134 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 189 op/s 2024-09-15T22:50:24.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:23 smithi192 ceph-mon[35427]: pgmap v484: 97 pgs: 97 active+clean; 134 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 189 op/s 2024-09-15T22:50:24.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:23 smithi121 ceph-mon[34815]: pgmap v484: 97 pgs: 97 active+clean; 134 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 189 op/s 2024-09-15T22:50:24.256 INFO:tasks.workunit.client.0.smithi111.stdout: 1 332303 9.72 MB/sec execute 52 sec latency 11.549 ms 2024-09-15T22:50:25.256 INFO:tasks.workunit.client.0.smithi111.stdout: 1 334549 9.81 MB/sec execute 53 sec latency 23.497 ms 2024-09-15T22:50:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.464+0000 7f10e9000640 1 -- 172.21.15.111:0/2650717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10e4103050 msgr2=0x7f10e4103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.464+0000 7f10e9000640 1 --2- 172.21.15.111:0/2650717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10e4103050 0x7f10e4103450 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009900 tx=0x7f10d402a3f0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.465+0000 7f10e9000640 1 -- 172.21.15.111:0/2650717 shutdown_connections 2024-09-15T22:50:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.465+0000 7f10e9000640 1 --2- 172.21.15.111:0/2650717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10e4101f40 0x7f10e4109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.465+0000 7f10e9000640 1 --2- 172.21.15.111:0/2650717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10e4101580 0x7f10e4101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.465+0000 7f10e9000640 1 --2- 172.21.15.111:0/2650717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10e4103050 0x7f10e4103450 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.465+0000 7f10e9000640 1 -- 172.21.15.111:0/2650717 >> 172.21.15.111:0/2650717 conn(0x7f10e40fd0d0 msgr2=0x7f10e40ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:50:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.465+0000 7f10e9000640 1 -- 172.21.15.111:0/2650717 shutdown_connections 2024-09-15T22:50:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.465+0000 7f10e9000640 1 -- 172.21.15.111:0/2650717 wait complete. 2024-09-15T22:50:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.466+0000 7f10e9000640 1 Processor -- start 2024-09-15T22:50:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.466+0000 7f10e9000640 1 -- start start 2024-09-15T22:50:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.466+0000 7f10e9000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10e4101580 0x7f10e4199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.466+0000 7f10e9000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10e4101f40 0x7f10e419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.466+0000 7f10e9000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10e4103050 0x7f10e41a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.466+0000 7f10e9000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f10e410d720 con 0x7f10e4103050 2024-09-15T22:50:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.466+0000 7f10e9000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f10e410d5a0 con 0x7f10e4101580 2024-09-15T22:50:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.466+0000 7f10e9000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f10e410d8a0 con 0x7f10e4101f40 2024-09-15T22:50:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.466+0000 7f10e3400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10e4101580 0x7f10e4199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.466+0000 7f10e3e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10e4103050 0x7f10e41a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.466+0000 7f10e3400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10e4101580 0x7f10e4199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:34542/0 (socket says 172.21.15.111:34542) 2024-09-15T22:50:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.466+0000 7f10e3400640 1 -- 172.21.15.111:0/2779613382 learned_addr learned my addr 172.21.15.111:0/2779613382 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:50:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.466+0000 7f10daa00640 1 --2- 172.21.15.111:0/2779613382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10e4101f40 0x7f10e419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.467+0000 7f10e3400640 1 -- 172.21.15.111:0/2779613382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10e4101f40 msgr2=0x7f10e419a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.467+0000 7f10e3400640 1 --2- 172.21.15.111:0/2779613382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10e4101f40 0x7f10e419a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.467+0000 7f10e3400640 1 -- 172.21.15.111:0/2779613382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10e4103050 msgr2=0x7f10e41a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.467+0000 7f10e3400640 1 --2- 172.21.15.111:0/2779613382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10e4103050 0x7f10e41a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.467+0000 7f10e3400640 1 -- 172.21.15.111:0/2779613382 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e41a1bf0 con 0x7f10e4101580 2024-09-15T22:50:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.467+0000 7f10daa00640 1 --2- 172.21.15.111:0/2779613382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10e4101f40 0x7f10e419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:50:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.467+0000 7f10e3e00640 1 --2- 172.21.15.111:0/2779613382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10e4103050 0x7f10e41a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:50:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.467+0000 7f10e3400640 1 --2- 172.21.15.111:0/2779613382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10e4101580 0x7f10e4199fd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f10d4031440 tx=0x7f10d4032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:50:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.467+0000 7f10e0c00640 1 -- 172.21.15.111:0/2779613382 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d403f030 con 0x7f10e4101580 2024-09-15T22:50:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.467+0000 7f10e0c00640 1 -- 172.21.15.111:0/2779613382 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10d4003ce0 con 0x7f10e4101580 2024-09-15T22:50:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.467+0000 7f10e9000640 1 -- 172.21.15.111:0/2779613382 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e41a1e80 con 0x7f10e4101580 2024-09-15T22:50:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.467+0000 7f10e0c00640 1 -- 172.21.15.111:0/2779613382 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4035070 con 0x7f10e4101580 2024-09-15T22:50:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.468+0000 7f10e9000640 1 -- 172.21.15.111:0/2779613382 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e41a22e0 con 0x7f10e4101580 2024-09-15T22:50:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.470+0000 7f10ed63a740 1 Processor -- start 2024-09-15T22:50:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.470+0000 7f10ed63a740 1 -- start start 2024-09-15T22:50:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.471+0000 7f10ed63a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c004919260 0x55c004919660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.471+0000 7f10ed63a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c004919ba0 0x55c00491e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.471+0000 7f10ed63a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c00491ec30 0x55c0049210f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.471+0000 7f10ed63a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c00487e100 con 0x55c00491ec30 2024-09-15T22:50:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.471+0000 7f10ed63a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c0048261b0 con 0x55c004919260 2024-09-15T22:50:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.471+0000 7f10ed63a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c00487a0c0 con 0x55c004919ba0 2024-09-15T22:50:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.471+0000 7f10e0c00640 1 -- 172.21.15.111:0/2779613382 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f10d4042030 con 0x7f10e4101580 2024-09-15T22:50:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.471+0000 7f10e3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c004919260 0x55c004919660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.471+0000 7f10daa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c00491ec30 0x55c0049210f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.471+0000 7f10e3400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c004919ba0 0x55c00491e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.471+0000 7f10e3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c004919260 0x55c004919660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:34546/0 (socket says 172.21.15.111:34546) 2024-09-15T22:50:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.471+0000 7f10e3e00640 1 -- 172.21.15.111:0/3170990358 learned_addr learned my addr 172.21.15.111:0/3170990358 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:50:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.471+0000 7f10daa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c00491ec30 0x55c0049210f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54564/0 (socket says 172.21.15.111:54564) 2024-09-15T22:50:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.472+0000 7f10daa00640 1 -- 172.21.15.111:0/3170990358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c004919ba0 msgr2=0x55c00491e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.472+0000 7f10daa00640 1 --2- 172.21.15.111:0/3170990358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c004919ba0 0x55c00491e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.472+0000 7f10daa00640 1 -- 172.21.15.111:0/3170990358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c004919260 msgr2=0x55c004919660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.472+0000 7f10daa00640 1 --2- 172.21.15.111:0/3170990358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c004919260 0x55c004919660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.472+0000 7f10daa00640 1 -- 172.21.15.111:0/3170990358 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c004591210 con 0x55c00491ec30 2024-09-15T22:50:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.472+0000 7f10e3e00640 1 --2- 172.21.15.111:0/3170990358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c004919260 0x55c004919660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:50:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.472+0000 7f10daa00640 1 --2- 172.21.15.111:0/3170990358 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c00491ec30 0x55c0049210f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f10cc009fa0 tx=0x7f10cc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6bf02cc71a93370 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:50:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.472+0000 7f10d8c00640 1 -- 172.21.15.111:0/3170990358 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc004030 con 0x55c00491ec30 2024-09-15T22:50:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.472+0000 7f10d8c00640 1 -- 172.21.15.111:0/3170990358 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10cc005500 con 0x55c00491ec30 2024-09-15T22:50:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.472+0000 7f10d8c00640 1 -- 172.21.15.111:0/3170990358 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc004db0 con 0x55c00491ec30 2024-09-15T22:50:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.472+0000 7f10e0c00640 1 --2- 172.21.15.111:0/2779613382 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f10c80a32c0 0x7f10c80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.472+0000 7f10e0c00640 1 -- 172.21.15.111:0/2779613382 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f10d40efc60 con 0x7f10e4101580 2024-09-15T22:50:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.473+0000 7f10e3e00640 1 --2- 172.21.15.111:0/2779613382 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f10c80a32c0 0x7f10c80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.473+0000 7f10ed63a740 1 -- 172.21.15.111:0/3170990358 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c00491ec30 msgr2=0x55c0049210f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.473+0000 7f10ed63a740 1 --2- 172.21.15.111:0/3170990358 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c00491ec30 0x55c0049210f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f10cc009fa0 tx=0x7f10cc0118d0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.473+0000 7f10ed63a740 1 -- 172.21.15.111:0/3170990358 shutdown_connections 2024-09-15T22:50:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.473+0000 7f10ed63a740 1 --2- 172.21.15.111:0/3170990358 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c00491ec30 0x55c0049210f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.473+0000 7f10ed63a740 1 --2- 172.21.15.111:0/3170990358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c004919ba0 0x55c00491e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.473+0000 7f10ed63a740 1 --2- 172.21.15.111:0/3170990358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c004919260 0x55c004919660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.473+0000 7f10ed63a740 1 -- 172.21.15.111:0/3170990358 >> 172.21.15.111:0/3170990358 conn(0x55c0049148d0 msgr2=0x55c004914cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:50:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.473+0000 7f10e3e00640 1 --2- 172.21.15.111:0/2779613382 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f10c80a32c0 0x7f10c80a5780 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f10dc009fd0 tx=0x7f10dc009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:50:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.473+0000 7f10ed63a740 1 -- 172.21.15.111:0/3170990358 shutdown_connections 2024-09-15T22:50:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.473+0000 7f10ed63a740 1 -- 172.21.15.111:0/3170990358 wait complete. 2024-09-15T22:50:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.474+0000 7f10ed63a740 1 -- start start 2024-09-15T22:50:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.474+0000 7f10ed63a740 1 Processor -- start 2024-09-15T22:50:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.474+0000 7f10ed63a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c004919260 0x55c0049233e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.474+0000 7f10ed63a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c004919ba0 0x55c004923920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.474+0000 7f10ed63a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c00491ec30 0x55c00492a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.474+0000 7f10ed63a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c004921c00 con 0x55c00491ec30 2024-09-15T22:50:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.474+0000 7f10ed63a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c004807090 con 0x55c004919260 2024-09-15T22:50:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.474+0000 7f10ed63a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c004921d80 con 0x55c004919ba0 2024-09-15T22:50:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.474+0000 7f10ed63a740 10 client.0 _ll_register_callbacks cb 0x55c00485bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:50:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.475+0000 7f10e3400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c004919260 0x55c0049233e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.475+0000 7f10daa00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c004919ba0 0x55c004923920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.475+0000 7f10e3400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c004919260 0x55c0049233e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:34560/0 (socket says 172.21.15.111:34560) 2024-09-15T22:50:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.475+0000 7f10e3400640 1 -- 172.21.15.111:0/1813971934 learned_addr learned my addr 172.21.15.111:0/1813971934 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:50:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.475+0000 7f10e3e00640 1 --2- 172.21.15.111:0/1813971934 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c00491ec30 0x55c00492a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.475+0000 7f10daa00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c004919ba0 0x55c004923920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:50016/0 (socket says 172.21.15.111:50016) 2024-09-15T22:50:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.475+0000 7f10e3400640 1 -- 172.21.15.111:0/1813971934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c004919ba0 msgr2=0x55c004923920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.475+0000 7f10e3400640 1 --2- 172.21.15.111:0/1813971934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c004919ba0 0x55c004923920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.475+0000 7f10e3400640 1 -- 172.21.15.111:0/1813971934 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c00491ec30 msgr2=0x55c00492a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.475+0000 7f10e3400640 1 --2- 172.21.15.111:0/1813971934 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c00491ec30 0x55c00492a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.475+0000 7f10e3400640 1 -- 172.21.15.111:0/1813971934 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c00465d620 con 0x55c004919260 2024-09-15T22:50:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.475+0000 7f10e3e00640 1 --2- 172.21.15.111:0/1813971934 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c00491ec30 0x55c00492a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:50:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.475+0000 7f10e3400640 1 --2- 172.21.15.111:0/1813971934 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c004919260 0x55c0049233e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f10d4005ec0 tx=0x7f10d40f32b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55d01efa4e064e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:50:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.475+0000 7f10d1600640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:50:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.475+0000 7f10d1600640 1 -- 172.21.15.111:0/1813971934 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4049030 con 0x55c004919260 2024-09-15T22:50:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.475+0000 7f10d1600640 1 -- 172.21.15.111:0/1813971934 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f10d40b8a40 con 0x55c004919260 2024-09-15T22:50:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.475+0000 7f10ed63a740 1 -- 172.21.15.111:0/1813971934 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10cc005500 con 0x55c004919260 2024-09-15T22:50:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.476+0000 7f10d1600640 1 -- 172.21.15.111:0/1813971934 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f10d40b8670 con 0x55c004919260 2024-09-15T22:50:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.476+0000 7f10ed63a740 10 client.25373 fetch_fsmap learned FSMap version 30 2024-09-15T22:50:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.476+0000 7f10ed63a740 1 -- 172.21.15.111:0/1813971934 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c00492d750 con 0x55c004919260 2024-09-15T22:50:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.476+0000 7f10d1600640 1 -- 172.21.15.111:0/1813971934 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10cc005500 con 0x55c004919260 2024-09-15T22:50:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.476+0000 7f10d1600640 1 -- 172.21.15.111:0/1813971934 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f10d403bca0 con 0x55c004919260 2024-09-15T22:50:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.476+0000 7f10ed63a740 10 client.25373 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:50:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.476+0000 7f10ed63a740 10 client.25373 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:50:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.476+0000 7f10ed63a740 20 client.25373 populate_metadata read hostname 'smithi111' 2024-09-15T22:50:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.476+0000 7f10ed63a740 1 --2- 172.21.15.111:0/1813971934 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c00492ee80 0x55c0049312b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.476+0000 7f10ed63a740 4 client.25373 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:50:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.476+0000 7f10ed63a740 1 -- 172.21.15.111:0/1813971934 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c00488b830 con 0x55c00492ee80 2024-09-15T22:50:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.476+0000 7f10daa00640 1 --2- 172.21.15.111:0/1813971934 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c00492ee80 0x55c0049312b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.477+0000 7f10daa00640 1 --2- 172.21.15.111:0/1813971934 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c00492ee80 0x55c0049312b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29132c072f12fc74 server_cookie=66c03f8914172a3e in_seq=0 out_seq=0 2024-09-15T22:50:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.477+0000 7f10d1600640 10 client.25373 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:50:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.478+0000 7f10d1600640 1 -- 172.21.15.111:0/1813971934 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c00488b830 con 0x55c00492ee80 2024-09-15T22:50:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.478+0000 7f10d1600640 10 client.25373 handle_command_reply: tid=0 2024-09-15T22:50:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.478+0000 7f10ed63a740 1 client.25373 shutdown 2024-09-15T22:50:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.478+0000 7f10ed63a740 1 -- 172.21.15.111:0/1813971934 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c004919260 msgr2=0x55c0049233e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.478+0000 7f10ed63a740 1 --2- 172.21.15.111:0/1813971934 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c004919260 0x55c0049233e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f10d4005ec0 tx=0x7f10d40f32b0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.478+0000 7f10ed63a740 1 -- 172.21.15.111:0/1813971934 shutdown_connections 2024-09-15T22:50:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.478+0000 7f10ed63a740 1 --2- 172.21.15.111:0/1813971934 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c00492ee80 0x55c0049312b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.478+0000 7f10ed63a740 1 --2- 172.21.15.111:0/1813971934 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c00491ec30 0x55c00492a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.478+0000 7f10ed63a740 1 --2- 172.21.15.111:0/1813971934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c004919ba0 0x55c004923920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.478+0000 7f10ed63a740 1 --2- 172.21.15.111:0/1813971934 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c004919260 0x55c0049233e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.478+0000 7f10d1600640 0 client.25373 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:50:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.478+0000 7f10ed63a740 1 -- 172.21.15.111:0/1813971934 >> 172.21.15.111:0/1813971934 conn(0x55c0049148d0 msgr2=0x55c0049160e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:50:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.478+0000 7f10ed63a740 1 -- 172.21.15.111:0/1813971934 shutdown_connections 2024-09-15T22:50:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.478+0000 7f10ed63a740 1 -- 172.21.15.111:0/1813971934 wait complete. 2024-09-15T22:50:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.478+0000 7f10ed63a740 20 client.25373 trim_cache size 0 max 16384 2024-09-15T22:50:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.493+0000 7f10ed63a740 1 Processor -- start 2024-09-15T22:50:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.493+0000 7f10ed63a740 1 -- start start 2024-09-15T22:50:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.494+0000 7f10ed63a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c004919260 0x55c004915780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.494+0000 7f10ed63a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c004919ba0 0x55c00492a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.494+0000 7f10ed63a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c00491ec30 0x55c004915cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.494+0000 7f10ed63a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f10e41a9370 con 0x55c004919ba0 2024-09-15T22:50:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.494+0000 7f10ed63a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f10e41a6c40 con 0x55c00491ec30 2024-09-15T22:50:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.494+0000 7f10ed63a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f10e41a4af0 con 0x55c004919260 2024-09-15T22:50:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.494+0000 7f10e3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c00491ec30 0x55c004915cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.494+0000 7f10e3400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c004919260 0x55c004915780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.494+0000 7f10e3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c00491ec30 0x55c004915cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:34566/0 (socket says 172.21.15.111:34566) 2024-09-15T22:50:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.494+0000 7f10e3e00640 1 -- 172.21.15.111:0/2381957977 learned_addr learned my addr 172.21.15.111:0/2381957977 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:50:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.494+0000 7f10daa00640 1 --2- 172.21.15.111:0/2381957977 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c004919ba0 0x55c00492a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.494+0000 7f10e3e00640 1 -- 172.21.15.111:0/2381957977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c004919260 msgr2=0x55c004915780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.494+0000 7f10e3e00640 1 --2- 172.21.15.111:0/2381957977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c004919260 0x55c004915780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.494+0000 7f10e3e00640 1 -- 172.21.15.111:0/2381957977 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c004919ba0 msgr2=0x55c00492a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.494+0000 7f10e3e00640 1 --2- 172.21.15.111:0/2381957977 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c004919ba0 0x55c00492a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.494+0000 7f10e3e00640 1 -- 172.21.15.111:0/2381957977 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0049146f0 con 0x55c00491ec30 2024-09-15T22:50:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.494+0000 7f10daa00640 1 --2- 172.21.15.111:0/2381957977 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c004919ba0 0x55c00492a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:50:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.494+0000 7f10e3400640 1 --2- 172.21.15.111:0/2381957977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c004919260 0x55c004915780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:50:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.495+0000 7f10e3e00640 1 --2- 172.21.15.111:0/2381957977 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c00491ec30 0x55c004915cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f10dc00fa60 tx=0x7f10dc01e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f3bf978d7ffbbba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:50:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.495+0000 7f10d3e00640 1 -- 172.21.15.111:0/2381957977 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc004b20 con 0x55c00491ec30 2024-09-15T22:50:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.495+0000 7f10d3e00640 1 -- 172.21.15.111:0/2381957977 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10dc004cc0 con 0x55c00491ec30 2024-09-15T22:50:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.495+0000 7f10d3e00640 1 -- 172.21.15.111:0/2381957977 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc005360 con 0x55c00491ec30 2024-09-15T22:50:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.495+0000 7f10ed63a740 1 -- 172.21.15.111:0/2381957977 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c00491ec30 msgr2=0x55c004915cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.495+0000 7f10ed63a740 1 --2- 172.21.15.111:0/2381957977 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c00491ec30 0x55c004915cc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f10dc00fa60 tx=0x7f10dc01e000 comp rx=0 tx=0).stop 2024-09-15T22:50:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.495+0000 7f10ed63a740 1 -- 172.21.15.111:0/2381957977 shutdown_connections 2024-09-15T22:50:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.496+0000 7f10ed63a740 1 --2- 172.21.15.111:0/2381957977 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c00491ec30 0x55c004915cc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.496+0000 7f10ed63a740 1 --2- 172.21.15.111:0/2381957977 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c004919ba0 0x55c00492a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.496+0000 7f10ed63a740 1 --2- 172.21.15.111:0/2381957977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c004919260 0x55c004915780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.496+0000 7f10ed63a740 1 -- 172.21.15.111:0/2381957977 >> 172.21.15.111:0/2381957977 conn(0x55c0049148d0 msgr2=0x55c004923190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:50:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.496+0000 7f10ed63a740 1 -- 172.21.15.111:0/2381957977 shutdown_connections 2024-09-15T22:50:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.496+0000 7f10ed63a740 1 -- 172.21.15.111:0/2381957977 wait complete. 2024-09-15T22:50:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.496+0000 7f10ed63a740 1 -- start start 2024-09-15T22:50:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.496+0000 7f10ed63a740 1 Processor -- start 2024-09-15T22:50:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.497+0000 7f10ed63a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c004919260 0x55c004927bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.497+0000 7f10ed63a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c004919ba0 0x55c004928130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.497+0000 7f10ed63a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c00491ec30 0x55c0049290f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.497+0000 7f10ed63a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f10e41a5140 con 0x55c00491ec30 2024-09-15T22:50:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.497+0000 7f10ed63a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f10e41a6570 con 0x55c004919ba0 2024-09-15T22:50:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.497+0000 7f10ed63a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f10e41a57d0 con 0x55c004919260 2024-09-15T22:50:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.497+0000 7f10ed63a740 10 client.0 _ll_register_callbacks cb 0x7f10d40f3470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:50:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.497+0000 7f10daa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c004919ba0 0x55c004928130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.497+0000 7f10e3e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c00491ec30 0x55c0049290f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.497+0000 7f10daa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c004919ba0 0x55c004928130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:34572/0 (socket says 172.21.15.111:34572) 2024-09-15T22:50:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.497+0000 7f10daa00640 1 -- 172.21.15.111:0/2956821478 learned_addr learned my addr 172.21.15.111:0/2956821478 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:50:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.497+0000 7f10e3400640 1 --2- 172.21.15.111:0/2956821478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c004919260 0x55c004927bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.497+0000 7f10daa00640 1 -- 172.21.15.111:0/2956821478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c004919260 msgr2=0x55c004927bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.497+0000 7f10daa00640 1 --2- 172.21.15.111:0/2956821478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c004919260 0x55c004927bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.497+0000 7f10daa00640 1 -- 172.21.15.111:0/2956821478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c00491ec30 msgr2=0x55c0049290f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.497+0000 7f10daa00640 1 --2- 172.21.15.111:0/2956821478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c00491ec30 0x55c0049290f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.497+0000 7f10daa00640 1 -- 172.21.15.111:0/2956821478 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c004929700 con 0x55c004919ba0 2024-09-15T22:50:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.497+0000 7f10e3400640 1 --2- 172.21.15.111:0/2956821478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c004919260 0x55c004927bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:50:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.498+0000 7f10daa00640 1 --2- 172.21.15.111:0/2956821478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c004919ba0 0x55c004928130 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55c004915750 tx=0x7f10cc009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c9c590446c06e65 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:50:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.498+0000 7f10d1600640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:50:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.498+0000 7f10d1600640 1 -- 172.21.15.111:0/2956821478 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10cc033030 con 0x55c004919ba0 2024-09-15T22:50:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.498+0000 7f10ed63a740 1 -- 172.21.15.111:0/2956821478 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10dc004cc0 con 0x55c004919ba0 2024-09-15T22:50:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.498+0000 7f10d1600640 1 -- 172.21.15.111:0/2956821478 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f10cc016040 con 0x55c004919ba0 2024-09-15T22:50:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.498+0000 7f10d1600640 1 -- 172.21.15.111:0/2956821478 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f10cc01d040 con 0x55c004919ba0 2024-09-15T22:50:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.498+0000 7f10ed63a740 10 client.25379 fetch_fsmap learned FSMap version 30 2024-09-15T22:50:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.498+0000 7f10ed63a740 1 -- 172.21.15.111:0/2956821478 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c00492aa70 con 0x55c004919ba0 2024-09-15T22:50:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.498+0000 7f10d1600640 1 -- 172.21.15.111:0/2956821478 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10dc004cc0 con 0x55c004919ba0 2024-09-15T22:50:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.498+0000 7f10d1600640 1 -- 172.21.15.111:0/2956821478 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f10cc02e030 con 0x55c004919ba0 2024-09-15T22:50:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.498+0000 7f10ed63a740 10 client.25379 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:50:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.498+0000 7f10ed63a740 10 client.25379 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:50:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.498+0000 7f10ed63a740 20 client.25379 populate_metadata read hostname 'smithi111' 2024-09-15T22:50:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.498+0000 7f10ed63a740 1 --2- 172.21.15.111:0/2956821478 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c00499a110 0x55c00499c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.498+0000 7f10ed63a740 4 client.25379 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:50:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.499+0000 7f10ed63a740 1 -- 172.21.15.111:0/2956821478 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c004885b80 con 0x55c00499a110 2024-09-15T22:50:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.499+0000 7f10e3400640 1 --2- 172.21.15.111:0/2956821478 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c00499a110 0x55c00499c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.499+0000 7f10d1600640 10 client.25379 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:50:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.499+0000 7f10e3400640 1 --2- 172.21.15.111:0/2956821478 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c00499a110 0x55c00499c530 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=867c82059688bb95 server_cookie=ce03b87d747ef096 in_seq=0 out_seq=0 2024-09-15T22:50:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.507+0000 7f10d1600640 1 -- 172.21.15.111:0/2956821478 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c004885b80 con 0x55c00499a110 2024-09-15T22:50:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.507+0000 7f10d1600640 10 client.25379 handle_command_reply: tid=0 2024-09-15T22:50:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.507+0000 7f10ed63a740 1 client.25379 shutdown 2024-09-15T22:50:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.507+0000 7f10ed63a740 1 -- 172.21.15.111:0/2956821478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c004919ba0 msgr2=0x55c004928130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.507+0000 7f10ed63a740 1 --2- 172.21.15.111:0/2956821478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c004919ba0 0x55c004928130 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55c004915750 tx=0x7f10cc009f30 comp rx=0 tx=0).stop 2024-09-15T22:50:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.507+0000 7f10ed63a740 1 -- 172.21.15.111:0/2956821478 shutdown_connections 2024-09-15T22:50:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.507+0000 7f10ed63a740 1 --2- 172.21.15.111:0/2956821478 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c00499a110 0x55c00499c530 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.507+0000 7f10ed63a740 1 --2- 172.21.15.111:0/2956821478 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c00491ec30 0x55c0049290f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.507+0000 7f10d1600640 0 client.25379 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:50:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.507+0000 7f10ed63a740 1 --2- 172.21.15.111:0/2956821478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c004919ba0 0x55c004928130 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.507+0000 7f10ed63a740 1 --2- 172.21.15.111:0/2956821478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c004919260 0x55c004927bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.507+0000 7f10ed63a740 1 -- 172.21.15.111:0/2956821478 >> 172.21.15.111:0/2956821478 conn(0x55c0049148d0 msgr2=0x55c0049131f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:50:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.507+0000 7f10ed63a740 1 -- 172.21.15.111:0/2956821478 shutdown_connections 2024-09-15T22:50:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.507+0000 7f10ed63a740 1 -- 172.21.15.111:0/2956821478 wait complete. 2024-09-15T22:50:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.507+0000 7f10ed63a740 20 client.25379 trim_cache size 0 max 16384 2024-09-15T22:50:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:50:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:50:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:50:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:50:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.508+0000 7f10e9000640 1 -- 172.21.15.111:0/2779613382 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f10c80a32c0 msgr2=0x7f10c80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.508+0000 7f10e9000640 1 --2- 172.21.15.111:0/2779613382 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f10c80a32c0 0x7f10c80a5780 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f10dc009fd0 tx=0x7f10dc009500 comp rx=0 tx=0).stop 2024-09-15T22:50:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.508+0000 7f10e9000640 1 -- 172.21.15.111:0/2779613382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10e4101580 msgr2=0x7f10e4199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:25.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.508+0000 7f10e9000640 1 --2- 172.21.15.111:0/2779613382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10e4101580 0x7f10e4199fd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f10d4031440 tx=0x7f10d4032ce0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.508+0000 7f10e9000640 1 -- 172.21.15.111:0/2779613382 shutdown_connections 2024-09-15T22:50:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.508+0000 7f10e9000640 1 --2- 172.21.15.111:0/2779613382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10e4103050 0x7f10e41a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.508+0000 7f10e9000640 1 --2- 172.21.15.111:0/2779613382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10e4101f40 0x7f10e419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.508+0000 7f10e9000640 1 --2- 172.21.15.111:0/2779613382 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f10c80a32c0 0x7f10c80a5780 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.508+0000 7f10e9000640 1 --2- 172.21.15.111:0/2779613382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10e4101580 0x7f10e4199fd0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.508+0000 7f10e9000640 1 -- 172.21.15.111:0/2779613382 >> 172.21.15.111:0/2779613382 conn(0x7f10e40fd0d0 msgr2=0x7f10e4108330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:50:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.508+0000 7f10e9000640 1 -- 172.21.15.111:0/2779613382 shutdown_connections 2024-09-15T22:50:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:25.508+0000 7f10e9000640 1 -- 172.21.15.111:0/2779613382 wait complete. 2024-09-15T22:50:25.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:25 smithi111 ceph-mon[30629]: pgmap v485: 97 pgs: 97 active+clean; 135 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 189 op/s 2024-09-15T22:50:26.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:25 smithi192 ceph-mon[35427]: pgmap v485: 97 pgs: 97 active+clean; 135 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 189 op/s 2024-09-15T22:50:26.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:25 smithi121 ceph-mon[34815]: pgmap v485: 97 pgs: 97 active+clean; 135 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 189 op/s 2024-09-15T22:50:26.256 INFO:tasks.workunit.client.0.smithi111.stdout: 1 337516 9.94 MB/sec execute 54 sec latency 10.203 ms 2024-09-15T22:50:27.256 INFO:tasks.workunit.client.0.smithi111.stdout: 1 337767 9.77 MB/sec execute 55 sec latency 461.734 ms 2024-09-15T22:50:27.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:27 smithi111 ceph-mon[30629]: pgmap v486: 97 pgs: 97 active+clean; 135 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 185 op/s 2024-09-15T22:50:28.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:27 smithi192 ceph-mon[35427]: pgmap v486: 97 pgs: 97 active+clean; 135 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 185 op/s 2024-09-15T22:50:28.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:27 smithi121 ceph-mon[34815]: pgmap v486: 97 pgs: 97 active+clean; 135 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 185 op/s 2024-09-15T22:50:28.256 INFO:tasks.workunit.client.0.smithi111.stdout: 1 339445 9.72 MB/sec execute 56 sec latency 949.887 ms 2024-09-15T22:50:29.256 INFO:tasks.workunit.client.0.smithi111.stdout: 1 341986 9.84 MB/sec execute 57 sec latency 15.740 ms 2024-09-15T22:50:30.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:29 smithi192 ceph-mon[35427]: pgmap v487: 97 pgs: 97 active+clean; 136 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 141 op/s 2024-09-15T22:50:30.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:29 smithi121 ceph-mon[34815]: pgmap v487: 97 pgs: 97 active+clean; 136 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 141 op/s 2024-09-15T22:50:30.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:29 smithi111 ceph-mon[30629]: pgmap v487: 97 pgs: 97 active+clean; 136 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 141 op/s 2024-09-15T22:50:30.257 INFO:tasks.workunit.client.0.smithi111.stdout: 1 344778 9.92 MB/sec execute 58 sec latency 11.871 ms 2024-09-15T22:50:31.258 INFO:tasks.workunit.client.0.smithi111.stdout: 1 347976 10.07 MB/sec execute 59 sec latency 7.237 ms 2024-09-15T22:50:31.764 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:31 smithi111 ceph-mon[30629]: pgmap v488: 97 pgs: 97 active+clean; 140 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 192 op/s 2024-09-15T22:50:32.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:31 smithi192 ceph-mon[35427]: pgmap v488: 97 pgs: 97 active+clean; 140 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 192 op/s 2024-09-15T22:50:32.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:31 smithi121 ceph-mon[34815]: pgmap v488: 97 pgs: 97 active+clean; 140 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 192 op/s 2024-09-15T22:50:32.257 INFO:tasks.workunit.client.0.smithi111.stdout: 1 348343 9.91 MB/sec execute 60 sec latency 745.071 ms 2024-09-15T22:50:33.257 INFO:tasks.workunit.client.0.smithi111.stdout: 1 348365 9.75 MB/sec execute 61 sec latency 883.050 ms 2024-09-15T22:50:33.744 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:33 smithi111 ceph-mon[30629]: pgmap v489: 97 pgs: 97 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 202 op/s 2024-09-15T22:50:34.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:33 smithi192 ceph-mon[35427]: pgmap v489: 97 pgs: 97 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 202 op/s 2024-09-15T22:50:34.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:33 smithi121 ceph-mon[34815]: pgmap v489: 97 pgs: 97 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 202 op/s 2024-09-15T22:50:34.257 INFO:tasks.workunit.client.0.smithi111.stdout: 1 348365 9.59 MB/sec execute 62 sec latency 1883.116 ms 2024-09-15T22:50:35.257 INFO:tasks.workunit.client.0.smithi111.stdout: 1 351164 9.71 MB/sec execute 63 sec latency 2137.325 ms 2024-09-15T22:50:36.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:35 smithi192 ceph-mon[35427]: pgmap v490: 97 pgs: 97 active+clean; 145 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2024-09-15T22:50:36.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:35 smithi121 ceph-mon[34815]: pgmap v490: 97 pgs: 97 active+clean; 145 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2024-09-15T22:50:36.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:35 smithi111 ceph-mon[30629]: pgmap v490: 97 pgs: 97 active+clean; 145 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2024-09-15T22:50:36.257 INFO:tasks.workunit.client.0.smithi111.stdout: 1 353613 9.69 MB/sec execute 64 sec latency 13.403 ms 2024-09-15T22:50:37.257 INFO:tasks.workunit.client.0.smithi111.stdout: 1 354861 9.71 MB/sec execute 65 sec latency 311.553 ms 2024-09-15T22:50:37.761 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:37 smithi111 ceph-mon[30629]: pgmap v491: 97 pgs: 97 active+clean; 145 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 176 op/s 2024-09-15T22:50:38.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:37 smithi192 ceph-mon[35427]: pgmap v491: 97 pgs: 97 active+clean; 145 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 176 op/s 2024-09-15T22:50:38.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:37 smithi121 ceph-mon[34815]: pgmap v491: 97 pgs: 97 active+clean; 145 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 176 op/s 2024-09-15T22:50:38.257 INFO:tasks.workunit.client.0.smithi111.stdout: 1 355134 9.58 MB/sec execute 66 sec latency 1069.690 ms 2024-09-15T22:50:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.633+0000 7fa885000640 1 -- 172.21.15.111:0/1134279835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa880103d60 msgr2=0x7fa880104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.633+0000 7fa885000640 1 --2- 172.21.15.111:0/1134279835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa880103d60 0x7fa880104160 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa870009900 tx=0x7fa87002a3f0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.633+0000 7fa885000640 1 -- 172.21.15.111:0/1134279835 shutdown_connections 2024-09-15T22:50:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.633+0000 7fa885000640 1 --2- 172.21.15.111:0/1134279835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa880105910 0x7fa88010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.633+0000 7fa885000640 1 --2- 172.21.15.111:0/1134279835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa880104f50 0x7fa8801053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.633+0000 7fa885000640 1 --2- 172.21.15.111:0/1134279835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa880103d60 0x7fa880104160 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.633+0000 7fa885000640 1 -- 172.21.15.111:0/1134279835 >> 172.21.15.111:0/1134279835 conn(0x7fa8800ff4e0 msgr2=0x7fa880101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:50:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.633+0000 7fa885000640 1 -- 172.21.15.111:0/1134279835 shutdown_connections 2024-09-15T22:50:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.633+0000 7fa885000640 1 -- 172.21.15.111:0/1134279835 wait complete. 2024-09-15T22:50:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.634+0000 7fa885000640 1 Processor -- start 2024-09-15T22:50:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.634+0000 7fa885000640 1 -- start start 2024-09-15T22:50:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.634+0000 7fa885000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa880103d60 0x7fa880199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.634+0000 7fa885000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa880104f50 0x7fa88019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.634+0000 7fa885000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa880105910 0x7fa8801a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.634+0000 7fa885000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa88010f9c0 con 0x7fa880104f50 2024-09-15T22:50:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.634+0000 7fa885000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa88010f840 con 0x7fa880103d60 2024-09-15T22:50:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.634+0000 7fa885000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa88010fb40 con 0x7fa880105910 2024-09-15T22:50:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.634+0000 7fa87de00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa880103d60 0x7fa880199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.634+0000 7fa87de00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa880103d60 0x7fa880199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:50610/0 (socket says 172.21.15.111:50610) 2024-09-15T22:50:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.634+0000 7fa87de00640 1 -- 172.21.15.111:0/4184753510 learned_addr learned my addr 172.21.15.111:0/4184753510 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:50:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.634+0000 7fa87e800640 1 --2- 172.21.15.111:0/4184753510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa880105910 0x7fa8801a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.634+0000 7fa875400640 1 --2- 172.21.15.111:0/4184753510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa880104f50 0x7fa88019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.635+0000 7fa87de00640 1 -- 172.21.15.111:0/4184753510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa880105910 msgr2=0x7fa8801a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.635+0000 7fa87de00640 1 --2- 172.21.15.111:0/4184753510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa880105910 0x7fa8801a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.635+0000 7fa87de00640 1 -- 172.21.15.111:0/4184753510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa880104f50 msgr2=0x7fa88019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.635+0000 7fa87de00640 1 --2- 172.21.15.111:0/4184753510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa880104f50 0x7fa88019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.635+0000 7fa87de00640 1 -- 172.21.15.111:0/4184753510 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8801a1bd0 con 0x7fa880103d60 2024-09-15T22:50:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.635+0000 7fa87de00640 1 --2- 172.21.15.111:0/4184753510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa880103d60 0x7fa880199fb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa870031610 tx=0x7fa870004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:50:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.635+0000 7fa877400640 1 -- 172.21.15.111:0/4184753510 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87003f030 con 0x7fa880103d60 2024-09-15T22:50:38.640 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.635+0000 7fa877400640 1 -- 172.21.15.111:0/4184753510 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa870003ad0 con 0x7fa880103d60 2024-09-15T22:50:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.635+0000 7fa877400640 1 -- 172.21.15.111:0/4184753510 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870035070 con 0x7fa880103d60 2024-09-15T22:50:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.636+0000 7fa885000640 1 -- 172.21.15.111:0/4184753510 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8801a1e60 con 0x7fa880103d60 2024-09-15T22:50:38.641 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.636+0000 7fa885000640 1 -- 172.21.15.111:0/4184753510 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8801a2270 con 0x7fa880103d60 2024-09-15T22:50:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.639+0000 7fa887ddb740 1 Processor -- start 2024-09-15T22:50:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.639+0000 7fa887ddb740 1 -- start start 2024-09-15T22:50:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.639+0000 7fa877400640 1 -- 172.21.15.111:0/4184753510 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa870042030 con 0x7fa880103d60 2024-09-15T22:50:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.639+0000 7fa887ddb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563824561090 0x563824561490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:38.644 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.639+0000 7fa887ddb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5638245619d0 0x563824566490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.639+0000 7fa887ddb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563824566a60 0x563824568f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.639+0000 7fa887ddb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5638244c6100 con 0x5638245619d0 2024-09-15T22:50:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.639+0000 7fa887ddb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56382446e1b0 con 0x563824561090 2024-09-15T22:50:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.639+0000 7fa887ddb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5638244c20c0 con 0x563824566a60 2024-09-15T22:50:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.639+0000 7fa87e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563824561090 0x563824561490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.640+0000 7fa87de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5638245619d0 0x563824566490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.640+0000 7fa875400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563824566a60 0x563824568f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:38.645 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.640+0000 7fa87e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563824561090 0x563824561490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:50620/0 (socket says 172.21.15.111:50620) 2024-09-15T22:50:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.640+0000 7fa87e800640 1 -- 172.21.15.111:0/2814288092 learned_addr learned my addr 172.21.15.111:0/2814288092 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:50:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.640+0000 7fa87e800640 1 -- 172.21.15.111:0/2814288092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563824566a60 msgr2=0x563824568f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.640+0000 7fa87e800640 1 --2- 172.21.15.111:0/2814288092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563824566a60 0x563824568f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.640+0000 7fa87e800640 1 -- 172.21.15.111:0/2814288092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5638245619d0 msgr2=0x563824566490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.640+0000 7fa87e800640 1 --2- 172.21.15.111:0/2814288092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5638245619d0 0x563824566490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.640+0000 7fa87e800640 1 -- 172.21.15.111:0/2814288092 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638241d9210 con 0x563824561090 2024-09-15T22:50:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.640+0000 7fa875400640 1 --2- 172.21.15.111:0/2814288092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563824566a60 0x563824568f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:50:38.646 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.640+0000 7fa87e800640 1 --2- 172.21.15.111:0/2814288092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563824561090 0x563824561490 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa878009fd0 tx=0x7fa8780118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69d3ef0f0a997c22 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:50:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.640+0000 7fa86f400640 1 -- 172.21.15.111:0/2814288092 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878004d20 con 0x563824561090 2024-09-15T22:50:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.640+0000 7fa86f400640 1 -- 172.21.15.111:0/2814288092 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa878004050 con 0x563824561090 2024-09-15T22:50:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.640+0000 7fa86f400640 1 -- 172.21.15.111:0/2814288092 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878005260 con 0x563824561090 2024-09-15T22:50:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.641+0000 7fa877400640 1 --2- 172.21.15.111:0/4184753510 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa8640a32f0 0x7fa8640a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.641+0000 7fa877400640 1 -- 172.21.15.111:0/4184753510 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fa8700efc60 con 0x7fa880103d60 2024-09-15T22:50:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.641+0000 7fa87de00640 1 --2- 172.21.15.111:0/4184753510 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa8640a32f0 0x7fa8640a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.641+0000 7fa887ddb740 1 -- 172.21.15.111:0/2814288092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563824561090 msgr2=0x563824561490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.641+0000 7fa887ddb740 1 --2- 172.21.15.111:0/2814288092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563824561090 0x563824561490 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa878009fd0 tx=0x7fa8780118d0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.641+0000 7fa887ddb740 1 -- 172.21.15.111:0/2814288092 shutdown_connections 2024-09-15T22:50:38.647 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.641+0000 7fa887ddb740 1 --2- 172.21.15.111:0/2814288092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563824566a60 0x563824568f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.641+0000 7fa887ddb740 1 --2- 172.21.15.111:0/2814288092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5638245619d0 0x563824566490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.641+0000 7fa887ddb740 1 --2- 172.21.15.111:0/2814288092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563824561090 0x563824561490 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.641+0000 7fa887ddb740 1 -- 172.21.15.111:0/2814288092 >> 172.21.15.111:0/2814288092 conn(0x56382455c700 msgr2=0x56382455cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:50:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.642+0000 7fa887ddb740 1 -- 172.21.15.111:0/2814288092 shutdown_connections 2024-09-15T22:50:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.642+0000 7fa887ddb740 1 -- 172.21.15.111:0/2814288092 wait complete. 2024-09-15T22:50:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.642+0000 7fa87de00640 1 --2- 172.21.15.111:0/4184753510 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa8640a32f0 0x7fa8640a57b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa8700b8d00 tx=0x7fa8700f38b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:50:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.642+0000 7fa887ddb740 1 -- start start 2024-09-15T22:50:38.648 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.643+0000 7fa887ddb740 1 Processor -- start 2024-09-15T22:50:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.643+0000 7fa887ddb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563824561090 0x56382456b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:38.652 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.643+0000 7fa887ddb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5638245619d0 0x56382456b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.643+0000 7fa887ddb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563824566a60 0x563824572770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.643+0000 7fa887ddb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563824569a30 con 0x563824561090 2024-09-15T22:50:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.643+0000 7fa887ddb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56382444f090 con 0x563824566a60 2024-09-15T22:50:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.643+0000 7fa887ddb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563824569bb0 con 0x5638245619d0 2024-09-15T22:50:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.643+0000 7fa887ddb740 10 client.0 _ll_register_callbacks cb 0x5638244a3f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:50:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.643+0000 7fa875400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5638245619d0 0x56382456b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.643+0000 7fa875400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5638245619d0 0x56382456b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:59788/0 (socket says 172.21.15.111:59788) 2024-09-15T22:50:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.643+0000 7fa875400640 1 -- 172.21.15.111:0/2038503352 learned_addr learned my addr 172.21.15.111:0/2038503352 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:50:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.643+0000 7fa87e800640 1 --2- 172.21.15.111:0/2038503352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563824561090 0x56382456b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.643+0000 7fa87e800640 1 --2- 172.21.15.111:0/2038503352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563824566a60 0x563824572770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:38.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.644+0000 7fa875400640 1 -- 172.21.15.111:0/2038503352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563824566a60 msgr2=0x563824572770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.644+0000 7fa875400640 1 --2- 172.21.15.111:0/2038503352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563824566a60 0x563824572770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.644+0000 7fa875400640 1 -- 172.21.15.111:0/2038503352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563824561090 msgr2=0x56382456b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.644+0000 7fa875400640 1 --2- 172.21.15.111:0/2038503352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563824561090 0x56382456b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.644+0000 7fa875400640 1 -- 172.21.15.111:0/2038503352 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638242a5620 con 0x5638245619d0 2024-09-15T22:50:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.644+0000 7fa87e800640 1 --2- 172.21.15.111:0/2038503352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563824561090 0x56382456b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:50:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.644+0000 7fa875400640 1 --2- 172.21.15.111:0/2038503352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5638245619d0 0x56382456b730 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa868012700 tx=0x7fa868012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6be5b5cfbca6ca5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:50:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.644+0000 7fa863e00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:50:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.644+0000 7fa863e00640 1 -- 172.21.15.111:0/2038503352 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa868004030 con 0x5638245619d0 2024-09-15T22:50:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.644+0000 7fa863e00640 1 -- 172.21.15.111:0/2038503352 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fa8680041d0 con 0x5638245619d0 2024-09-15T22:50:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.644+0000 7fa887ddb740 1 -- 172.21.15.111:0/2038503352 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa878004050 con 0x5638245619d0 2024-09-15T22:50:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.644+0000 7fa863e00640 1 -- 172.21.15.111:0/2038503352 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa868004410 con 0x5638245619d0 2024-09-15T22:50:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.644+0000 7fa887ddb740 10 client.25390 fetch_fsmap learned FSMap version 30 2024-09-15T22:50:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.644+0000 7fa887ddb740 1 -- 172.21.15.111:0/2038503352 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563824575680 con 0x5638245619d0 2024-09-15T22:50:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.645+0000 7fa863e00640 1 -- 172.21.15.111:0/2038503352 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa878004050 con 0x5638245619d0 2024-09-15T22:50:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.645+0000 7fa863e00640 1 -- 172.21.15.111:0/2038503352 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fa868003bf0 con 0x5638245619d0 2024-09-15T22:50:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.645+0000 7fa887ddb740 10 client.25390 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:50:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.645+0000 7fa887ddb740 10 client.25390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:50:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.645+0000 7fa887ddb740 20 client.25390 populate_metadata read hostname 'smithi111' 2024-09-15T22:50:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.645+0000 7fa887ddb740 1 --2- 172.21.15.111:0/2038503352 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563824576d50 0x563824579140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.645+0000 7fa887ddb740 4 client.25390 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:50:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.645+0000 7fa887ddb740 1 -- 172.21.15.111:0/2038503352 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638244d3830 con 0x563824576d50 2024-09-15T22:50:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.645+0000 7fa87e800640 1 --2- 172.21.15.111:0/2038503352 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563824576d50 0x563824579140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.646+0000 7fa87e800640 1 --2- 172.21.15.111:0/2038503352 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563824576d50 0x563824579140 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ebeb7067acba9c1 server_cookie=1212fe84f1bbf5a3 in_seq=0 out_seq=0 2024-09-15T22:50:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.646+0000 7fa863e00640 10 client.25390 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:50:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.646+0000 7fa863e00640 1 -- 172.21.15.111:0/2038503352 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5638244d3830 con 0x563824576d50 2024-09-15T22:50:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.646+0000 7fa863e00640 10 client.25390 handle_command_reply: tid=0 2024-09-15T22:50:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.647+0000 7fa887ddb740 1 client.25390 shutdown 2024-09-15T22:50:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.647+0000 7fa887ddb740 1 -- 172.21.15.111:0/2038503352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5638245619d0 msgr2=0x56382456b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.647+0000 7fa887ddb740 1 --2- 172.21.15.111:0/2038503352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5638245619d0 0x56382456b730 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa868012700 tx=0x7fa868012bc0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.647+0000 7fa887ddb740 1 -- 172.21.15.111:0/2038503352 shutdown_connections 2024-09-15T22:50:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.647+0000 7fa887ddb740 1 --2- 172.21.15.111:0/2038503352 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563824576d50 0x563824579140 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.647+0000 7fa887ddb740 1 --2- 172.21.15.111:0/2038503352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563824566a60 0x563824572770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.647+0000 7fa887ddb740 1 --2- 172.21.15.111:0/2038503352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5638245619d0 0x56382456b730 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.647+0000 7fa887ddb740 1 --2- 172.21.15.111:0/2038503352 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563824561090 0x56382456b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.647+0000 7fa887ddb740 1 -- 172.21.15.111:0/2038503352 >> 172.21.15.111:0/2038503352 conn(0x56382455c700 msgr2=0x56382455df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:50:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.647+0000 7fa863e00640 0 client.25390 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:50:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.647+0000 7fa887ddb740 1 -- 172.21.15.111:0/2038503352 shutdown_connections 2024-09-15T22:50:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.647+0000 7fa887ddb740 1 -- 172.21.15.111:0/2038503352 wait complete. 2024-09-15T22:50:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.647+0000 7fa887ddb740 20 client.25390 trim_cache size 0 max 16384 2024-09-15T22:50:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.663+0000 7fa887ddb740 1 Processor -- start 2024-09-15T22:50:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.664+0000 7fa887ddb740 1 -- start start 2024-09-15T22:50:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.664+0000 7fa887ddb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563824561090 0x56382455cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.664+0000 7fa887ddb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5638245619d0 0x563824572730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.664+0000 7fa887ddb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563824566a60 0x56382455d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.664+0000 7fa887ddb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801ad8b0 con 0x563824561090 2024-09-15T22:50:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.664+0000 7fa887ddb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801ab180 con 0x563824566a60 2024-09-15T22:50:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.664+0000 7fa887ddb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801a8fd0 con 0x5638245619d0 2024-09-15T22:50:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.664+0000 7fa875400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5638245619d0 0x563824572730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.664+0000 7fa87e800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563824561090 0x56382455cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.664+0000 7fa875400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5638245619d0 0x563824572730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:59790/0 (socket says 172.21.15.111:59790) 2024-09-15T22:50:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.664+0000 7fa875400640 1 -- 172.21.15.111:0/1132930793 learned_addr learned my addr 172.21.15.111:0/1132930793 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:50:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.664+0000 7fa87e800640 1 --2- 172.21.15.111:0/1132930793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563824566a60 0x56382455d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.664+0000 7fa875400640 1 -- 172.21.15.111:0/1132930793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563824566a60 msgr2=0x56382455d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.664+0000 7fa875400640 1 --2- 172.21.15.111:0/1132930793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563824566a60 0x56382455d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.664+0000 7fa875400640 1 -- 172.21.15.111:0/1132930793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563824561090 msgr2=0x56382455cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.664+0000 7fa875400640 1 --2- 172.21.15.111:0/1132930793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563824561090 0x56382455cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.664+0000 7fa875400640 1 -- 172.21.15.111:0/1132930793 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638245754a0 con 0x5638245619d0 2024-09-15T22:50:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.665+0000 7fa875400640 1 --2- 172.21.15.111:0/1132930793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5638245619d0 0x563824572730 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa8680126d0 tx=0x7fa8680106f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69d9fc82a53b7a0b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:50:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.665+0000 7fa86ea00640 1 -- 172.21.15.111:0/1132930793 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa868010ac0 con 0x5638245619d0 2024-09-15T22:50:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.665+0000 7fa86ea00640 1 -- 172.21.15.111:0/1132930793 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa868010c60 con 0x5638245619d0 2024-09-15T22:50:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.665+0000 7fa86ea00640 1 -- 172.21.15.111:0/1132930793 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86801c7e0 con 0x5638245619d0 2024-09-15T22:50:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.666+0000 7fa887ddb740 1 -- 172.21.15.111:0/1132930793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5638245619d0 msgr2=0x563824572730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.666+0000 7fa887ddb740 1 --2- 172.21.15.111:0/1132930793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5638245619d0 0x563824572730 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa8680126d0 tx=0x7fa8680106f0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.666+0000 7fa887ddb740 1 -- 172.21.15.111:0/1132930793 shutdown_connections 2024-09-15T22:50:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.666+0000 7fa887ddb740 1 --2- 172.21.15.111:0/1132930793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563824566a60 0x56382455d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.666+0000 7fa887ddb740 1 --2- 172.21.15.111:0/1132930793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5638245619d0 0x563824572730 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.666+0000 7fa887ddb740 1 --2- 172.21.15.111:0/1132930793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563824561090 0x56382455cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.666+0000 7fa887ddb740 1 -- 172.21.15.111:0/1132930793 >> 172.21.15.111:0/1132930793 conn(0x56382455c700 msgr2=0x56382456b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:50:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.666+0000 7fa887ddb740 1 -- 172.21.15.111:0/1132930793 shutdown_connections 2024-09-15T22:50:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.666+0000 7fa887ddb740 1 -- 172.21.15.111:0/1132930793 wait complete. 2024-09-15T22:50:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.666+0000 7fa887ddb740 1 -- start start 2024-09-15T22:50:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.667+0000 7fa887ddb740 1 Processor -- start 2024-09-15T22:50:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.667+0000 7fa887ddb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563824561090 0x56382456fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.667+0000 7fa887ddb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5638245619d0 0x56382456ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.667+0000 7fa887ddb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563824566a60 0x563824570eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.667+0000 7fa887ddb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801a9640 con 0x5638245619d0 2024-09-15T22:50:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.667+0000 7fa887ddb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801aaab0 con 0x563824566a60 2024-09-15T22:50:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.667+0000 7fa887ddb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa8801a9d10 con 0x563824561090 2024-09-15T22:50:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.667+0000 7fa887ddb740 10 client.0 _ll_register_callbacks cb 0x7fa868009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:50:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.667+0000 7fa875400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5638245619d0 0x56382456ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.667+0000 7fa875400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5638245619d0 0x56382456ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57474/0 (socket says 172.21.15.111:57474) 2024-09-15T22:50:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.667+0000 7fa87e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563824566a60 0x563824570eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.667+0000 7fa875400640 1 -- 172.21.15.111:0/196307686 learned_addr learned my addr 172.21.15.111:0/196307686 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:50:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.667+0000 7fa87e800640 1 --2- 172.21.15.111:0/196307686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563824561090 0x56382456fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.668+0000 7fa87e800640 1 -- 172.21.15.111:0/196307686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563824561090 msgr2=0x56382456fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.668+0000 7fa87e800640 1 --2- 172.21.15.111:0/196307686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563824561090 0x56382456fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.668+0000 7fa87e800640 1 -- 172.21.15.111:0/196307686 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5638245619d0 msgr2=0x56382456ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.668+0000 7fa87e800640 1 --2- 172.21.15.111:0/196307686 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5638245619d0 0x56382456ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.668+0000 7fa87e800640 1 -- 172.21.15.111:0/196307686 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638245714c0 con 0x563824566a60 2024-09-15T22:50:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.668+0000 7fa87e800640 1 --2- 172.21.15.111:0/196307686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563824566a60 0x563824570eb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa878009de0 tx=0x7fa8780048e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=928d7ec2c4c6cf3d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:50:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.668+0000 7fa863e00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:50:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.668+0000 7fa863e00640 1 -- 172.21.15.111:0/196307686 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878004520 con 0x563824566a60 2024-09-15T22:50:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.668+0000 7fa887ddb740 1 -- 172.21.15.111:0/196307686 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa868010c60 con 0x563824566a60 2024-09-15T22:50:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.668+0000 7fa863e00640 1 -- 172.21.15.111:0/196307686 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fa87800f4e0 con 0x563824566a60 2024-09-15T22:50:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.668+0000 7fa863e00640 1 -- 172.21.15.111:0/196307686 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa8780046c0 con 0x563824566a60 2024-09-15T22:50:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.668+0000 7fa887ddb740 10 client.25397 fetch_fsmap learned FSMap version 30 2024-09-15T22:50:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.668+0000 7fa887ddb740 1 -- 172.21.15.111:0/196307686 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5638245727d0 con 0x563824566a60 2024-09-15T22:50:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.668+0000 7fa863e00640 1 -- 172.21.15.111:0/196307686 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa868010c60 con 0x563824566a60 2024-09-15T22:50:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.669+0000 7fa863e00640 1 -- 172.21.15.111:0/196307686 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fa87802f040 con 0x563824566a60 2024-09-15T22:50:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.669+0000 7fa887ddb740 10 client.25397 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:50:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.669+0000 7fa887ddb740 10 client.25397 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:50:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.669+0000 7fa887ddb740 20 client.25397 populate_metadata read hostname 'smithi111' 2024-09-15T22:50:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.669+0000 7fa887ddb740 1 --2- 172.21.15.111:0/196307686 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5638245e1ca0 0x5638245e4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.669+0000 7fa887ddb740 4 client.25397 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:50:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.669+0000 7fa887ddb740 1 -- 172.21.15.111:0/196307686 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5638244cdb80 con 0x5638245e1ca0 2024-09-15T22:50:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.669+0000 7fa875400640 1 --2- 172.21.15.111:0/196307686 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5638245e1ca0 0x5638245e4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.669+0000 7fa875400640 1 --2- 172.21.15.111:0/196307686 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5638245e1ca0 0x5638245e4090 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fe16e09f27c945 server_cookie=e4321f969caae136 in_seq=0 out_seq=0 2024-09-15T22:50:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.669+0000 7fa863e00640 10 client.25397 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:50:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.677+0000 7fa863e00640 1 -- 172.21.15.111:0/196307686 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5638244cdb80 con 0x5638245e1ca0 2024-09-15T22:50:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.677+0000 7fa863e00640 10 client.25397 handle_command_reply: tid=0 2024-09-15T22:50:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.677+0000 7fa887ddb740 1 client.25397 shutdown 2024-09-15T22:50:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.677+0000 7fa887ddb740 1 -- 172.21.15.111:0/196307686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563824566a60 msgr2=0x563824570eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.677+0000 7fa887ddb740 1 --2- 172.21.15.111:0/196307686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563824566a60 0x563824570eb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa878009de0 tx=0x7fa8780048e0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.677+0000 7fa887ddb740 1 -- 172.21.15.111:0/196307686 shutdown_connections 2024-09-15T22:50:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.677+0000 7fa887ddb740 1 --2- 172.21.15.111:0/196307686 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5638245e1ca0 0x5638245e4090 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.677+0000 7fa887ddb740 1 --2- 172.21.15.111:0/196307686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563824566a60 0x563824570eb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.677+0000 7fa887ddb740 1 --2- 172.21.15.111:0/196307686 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5638245619d0 0x56382456ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.677+0000 7fa863e00640 0 client.25397 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:50:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.677+0000 7fa887ddb740 1 --2- 172.21.15.111:0/196307686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563824561090 0x56382456fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.677+0000 7fa887ddb740 1 -- 172.21.15.111:0/196307686 >> 172.21.15.111:0/196307686 conn(0x56382455c700 msgr2=0x56382455b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:50:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.678+0000 7fa887ddb740 1 -- 172.21.15.111:0/196307686 shutdown_connections 2024-09-15T22:50:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.678+0000 7fa887ddb740 1 -- 172.21.15.111:0/196307686 wait complete. 2024-09-15T22:50:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.678+0000 7fa887ddb740 20 client.25397 trim_cache size 0 max 16384 2024-09-15T22:50:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:50:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:50:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:50:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:50:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.678+0000 7fa885000640 1 -- 172.21.15.111:0/4184753510 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa8640a32f0 msgr2=0x7fa8640a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.678+0000 7fa885000640 1 --2- 172.21.15.111:0/4184753510 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa8640a32f0 0x7fa8640a57b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa8700b8d00 tx=0x7fa8700f38b0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.678+0000 7fa885000640 1 -- 172.21.15.111:0/4184753510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa880103d60 msgr2=0x7fa880199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.678+0000 7fa885000640 1 --2- 172.21.15.111:0/4184753510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa880103d60 0x7fa880199fb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa870031610 tx=0x7fa870004270 comp rx=0 tx=0).stop 2024-09-15T22:50:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.678+0000 7fa885000640 1 -- 172.21.15.111:0/4184753510 shutdown_connections 2024-09-15T22:50:38.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.678+0000 7fa885000640 1 --2- 172.21.15.111:0/4184753510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa880105910 0x7fa8801a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.678+0000 7fa885000640 1 --2- 172.21.15.111:0/4184753510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa880104f50 0x7fa88019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.678+0000 7fa885000640 1 --2- 172.21.15.111:0/4184753510 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa8640a32f0 0x7fa8640a57b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.678+0000 7fa885000640 1 --2- 172.21.15.111:0/4184753510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa880103d60 0x7fa880199fb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.678+0000 7fa885000640 1 -- 172.21.15.111:0/4184753510 >> 172.21.15.111:0/4184753510 conn(0x7fa8800ff4e0 msgr2=0x7fa8801082a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:50:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.679+0000 7fa885000640 1 -- 172.21.15.111:0/4184753510 shutdown_connections 2024-09-15T22:50:38.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:38.679+0000 7fa885000640 1 -- 172.21.15.111:0/4184753510 wait complete. 2024-09-15T22:50:39.257 INFO:tasks.workunit.client.0.smithi111.stdout: 1 358210 9.70 MB/sec execute 67 sec latency 13.227 ms 2024-09-15T22:50:40.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:39 smithi192 ceph-mon[35427]: pgmap v492: 97 pgs: 97 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 205 op/s 2024-09-15T22:50:40.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:39 smithi121 ceph-mon[34815]: pgmap v492: 97 pgs: 97 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 205 op/s 2024-09-15T22:50:40.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:39 smithi111 ceph-mon[30629]: pgmap v492: 97 pgs: 97 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 205 op/s 2024-09-15T22:50:40.257 INFO:tasks.workunit.client.0.smithi111.stdout: 1 360747 9.70 MB/sec execute 68 sec latency 11.881 ms 2024-09-15T22:50:41.000 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:40 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:50:41.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:40 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:50:41.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:40 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:50:41.257 INFO:tasks.workunit.client.0.smithi111.stdout: 1 363297 9.79 MB/sec execute 69 sec latency 13.136 ms 2024-09-15T22:50:41.976 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:41 smithi111 ceph-mon[30629]: pgmap v493: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 207 op/s 2024-09-15T22:50:41.976 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:41 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:50:41.977 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:41 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:50:41.977 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:41 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:50:42.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:41 smithi192 ceph-mon[35427]: pgmap v493: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 207 op/s 2024-09-15T22:50:42.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:41 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:50:42.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:41 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:50:42.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:41 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:50:42.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:41 smithi121 ceph-mon[34815]: pgmap v493: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 207 op/s 2024-09-15T22:50:42.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:41 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:50:42.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:41 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:50:42.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:41 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:50:42.257 INFO:tasks.workunit.client.0.smithi111.stdout: 1 365179 9.79 MB/sec execute 70 sec latency 318.596 ms 2024-09-15T22:50:43.257 INFO:tasks.workunit.client.0.smithi111.stdout: 1 365179 9.65 MB/sec execute 71 sec latency 1318.664 ms 2024-09-15T22:50:44.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:43 smithi192 ceph-mon[35427]: pgmap v494: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 170 op/s 2024-09-15T22:50:44.133 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:43 smithi111 ceph-mon[30629]: pgmap v494: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 170 op/s 2024-09-15T22:50:44.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:43 smithi121 ceph-mon[34815]: pgmap v494: 97 pgs: 97 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 170 op/s 2024-09-15T22:50:44.257 INFO:tasks.workunit.client.0.smithi111.stdout: 1 366283 9.59 MB/sec execute 72 sec latency 1621.149 ms 2024-09-15T22:50:45.257 INFO:tasks.workunit.client.0.smithi111.stdout: 1 369332 9.71 MB/sec execute 73 sec latency 6.875 ms 2024-09-15T22:50:46.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:45 smithi192 ceph-mon[35427]: pgmap v495: 97 pgs: 97 active+clean; 148 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 212 op/s 2024-09-15T22:50:46.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:45 smithi121 ceph-mon[34815]: pgmap v495: 97 pgs: 97 active+clean; 148 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 212 op/s 2024-09-15T22:50:46.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:45 smithi111 ceph-mon[30629]: pgmap v495: 97 pgs: 97 active+clean; 148 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 212 op/s 2024-09-15T22:50:46.257 INFO:tasks.workunit.client.0.smithi111.stdout: 1 372567 9.82 MB/sec execute 74 sec latency 11.454 ms 2024-09-15T22:50:47.258 INFO:tasks.workunit.client.0.smithi111.stdout: 1 373831 9.77 MB/sec execute 75 sec latency 311.424 ms 2024-09-15T22:50:47.763 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:47 smithi111 ceph-mon[30629]: pgmap v496: 97 pgs: 97 active+clean; 148 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-09-15T22:50:48.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:47 smithi192 ceph-mon[35427]: pgmap v496: 97 pgs: 97 active+clean; 148 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-09-15T22:50:48.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:47 smithi121 ceph-mon[34815]: pgmap v496: 97 pgs: 97 active+clean; 148 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-09-15T22:50:48.258 INFO:tasks.workunit.client.0.smithi111.stdout: 1 373831 9.65 MB/sec execute 76 sec latency 1311.512 ms 2024-09-15T22:50:49.258 INFO:tasks.workunit.client.0.smithi111.stdout: 1 376094 9.68 MB/sec execute 77 sec latency 1596.539 ms 2024-09-15T22:50:50.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:49 smithi192 ceph-mon[35427]: pgmap v497: 97 pgs: 97 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 186 op/s 2024-09-15T22:50:50.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:49 smithi121 ceph-mon[34815]: pgmap v497: 97 pgs: 97 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 186 op/s 2024-09-15T22:50:50.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:49 smithi111 ceph-mon[30629]: pgmap v497: 97 pgs: 97 active+clean; 149 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 186 op/s 2024-09-15T22:50:50.258 INFO:tasks.workunit.client.0.smithi111.stdout: 1 378787 9.72 MB/sec execute 78 sec latency 13.351 ms 2024-09-15T22:50:51.258 INFO:tasks.workunit.client.0.smithi111.stdout: 1 381172 9.77 MB/sec execute 79 sec latency 15.336 ms 2024-09-15T22:50:51.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.803+0000 7f9b51800640 1 -- 172.21.15.111:0/1708120179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b4c101580 msgr2=0x7f9b4c101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.803+0000 7f9b51800640 1 --2- 172.21.15.111:0/1708120179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b4c101580 0x7f9b4c101a00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9b38009900 tx=0x7f9b3802a3f0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.803+0000 7f9b51800640 1 -- 172.21.15.111:0/1708120179 shutdown_connections 2024-09-15T22:50:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.803+0000 7f9b51800640 1 --2- 172.21.15.111:0/1708120179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b4c101f40 0x7f9b4c109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.803+0000 7f9b51800640 1 --2- 172.21.15.111:0/1708120179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b4c101580 0x7f9b4c101a00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.803+0000 7f9b51800640 1 --2- 172.21.15.111:0/1708120179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b4c103050 0x7f9b4c103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.803+0000 7f9b51800640 1 -- 172.21.15.111:0/1708120179 >> 172.21.15.111:0/1708120179 conn(0x7f9b4c0fd0d0 msgr2=0x7f9b4c0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:50:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.803+0000 7f9b51800640 1 -- 172.21.15.111:0/1708120179 shutdown_connections 2024-09-15T22:50:51.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.803+0000 7f9b51800640 1 -- 172.21.15.111:0/1708120179 wait complete. 2024-09-15T22:50:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.804+0000 7f9b51800640 1 Processor -- start 2024-09-15T22:50:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.804+0000 7f9b51800640 1 -- start start 2024-09-15T22:50:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.804+0000 7f9b51800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b4c101580 0x7f9b4c199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.804+0000 7f9b51800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b4c101f40 0x7f9b4c19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:51.807 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.804+0000 7f9b51800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b4c103050 0x7f9b4c1a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.804+0000 7f9b51800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c10d6f0 con 0x7f9b4c101f40 2024-09-15T22:50:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.804+0000 7f9b51800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c10d570 con 0x7f9b4c101580 2024-09-15T22:50:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.804+0000 7f9b51800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c10d870 con 0x7f9b4c103050 2024-09-15T22:50:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.804+0000 7f9b50e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b4c103050 0x7f9b4c1a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.804+0000 7f9b4b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b4c101f40 0x7f9b4c19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.804+0000 7f9b50e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b4c103050 0x7f9b4c1a15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:39304/0 (socket says 172.21.15.111:39304) 2024-09-15T22:50:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.804+0000 7f9b50e00640 1 -- 172.21.15.111:0/4009465788 learned_addr learned my addr 172.21.15.111:0/4009465788 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:50:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.805+0000 7f9b4be00640 1 --2- 172.21.15.111:0/4009465788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b4c101580 0x7f9b4c199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.805+0000 7f9b4b400640 1 -- 172.21.15.111:0/4009465788 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b4c103050 msgr2=0x7f9b4c1a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.805+0000 7f9b4b400640 1 --2- 172.21.15.111:0/4009465788 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b4c103050 0x7f9b4c1a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.805+0000 7f9b4b400640 1 -- 172.21.15.111:0/4009465788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b4c101580 msgr2=0x7f9b4c199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.805+0000 7f9b4b400640 1 --2- 172.21.15.111:0/4009465788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b4c101580 0x7f9b4c199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.805+0000 7f9b4b400640 1 -- 172.21.15.111:0/4009465788 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4c1a1c10 con 0x7f9b4c101f40 2024-09-15T22:50:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.805+0000 7f9b4b400640 1 --2- 172.21.15.111:0/4009465788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b4c101f40 0x7f9b4c19a530 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9b38031440 tx=0x7f9b38004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:50:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.805+0000 7f9b48c00640 1 -- 172.21.15.111:0/4009465788 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3803f070 con 0x7f9b4c101f40 2024-09-15T22:50:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.805+0000 7f9b48c00640 1 -- 172.21.15.111:0/4009465788 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b38003cd0 con 0x7f9b4c101f40 2024-09-15T22:50:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.805+0000 7f9b51800640 1 -- 172.21.15.111:0/4009465788 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b4c1a1ea0 con 0x7f9b4c101f40 2024-09-15T22:50:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.806+0000 7f9b48c00640 1 -- 172.21.15.111:0/4009465788 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b38035070 con 0x7f9b4c101f40 2024-09-15T22:50:51.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.806+0000 7f9b51800640 1 -- 172.21.15.111:0/4009465788 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b4c1a2300 con 0x7f9b4c101f40 2024-09-15T22:50:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.809+0000 7f9b55c28740 1 Processor -- start 2024-09-15T22:50:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.809+0000 7f9b48c00640 1 -- 172.21.15.111:0/4009465788 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f9b3803b4f0 con 0x7f9b4c101f40 2024-09-15T22:50:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.809+0000 7f9b55c28740 1 -- start start 2024-09-15T22:50:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.809+0000 7f9b55c28740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594effc6260 0x5594effc6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.809+0000 7f9b55c28740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5594effc6ba0 0x5594effcb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.809+0000 7f9b55c28740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5594effcbc30 0x5594effce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.809+0000 7f9b55c28740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5594eff2b100 con 0x5594effc6260 2024-09-15T22:50:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.809+0000 7f9b55c28740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5594efed31b0 con 0x5594effc6ba0 2024-09-15T22:50:51.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.809+0000 7f9b55c28740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5594eff270c0 con 0x5594effcbc30 2024-09-15T22:50:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.810+0000 7f9b4be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5594effc6ba0 0x5594effcb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.810+0000 7f9b50e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594effc6260 0x5594effc6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.810+0000 7f9b4be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5594effc6ba0 0x5594effcb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:57128/0 (socket says 172.21.15.111:57128) 2024-09-15T22:50:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.810+0000 7f9b4be00640 1 -- 172.21.15.111:0/434394961 learned_addr learned my addr 172.21.15.111:0/434394961 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:50:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.810+0000 7f9b50e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594effc6260 0x5594effc6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36892/0 (socket says 172.21.15.111:36892) 2024-09-15T22:50:51.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.810+0000 7f9b4b400640 1 --2- 172.21.15.111:0/434394961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5594effcbc30 0x5594effce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.810+0000 7f9b4be00640 1 -- 172.21.15.111:0/434394961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5594effcbc30 msgr2=0x5594effce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.810+0000 7f9b4be00640 1 --2- 172.21.15.111:0/434394961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5594effcbc30 0x5594effce0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.810+0000 7f9b4be00640 1 -- 172.21.15.111:0/434394961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594effc6260 msgr2=0x5594effc6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.810+0000 7f9b4be00640 1 --2- 172.21.15.111:0/434394961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594effc6260 0x5594effc6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.810+0000 7f9b4be00640 1 -- 172.21.15.111:0/434394961 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594efc3e210 con 0x5594effc6ba0 2024-09-15T22:50:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.810+0000 7f9b50e00640 1 --2- 172.21.15.111:0/434394961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594effc6260 0x5594effc6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:50:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.810+0000 7f9b4be00640 1 --2- 172.21.15.111:0/434394961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5594effc6ba0 0x5594effcb660 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c009ca0 tx=0x7f9b3c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e55cdca651c47763 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:50:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.810+0000 7f9b41600640 1 -- 172.21.15.111:0/434394961 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c004220 con 0x5594effc6ba0 2024-09-15T22:50:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.810+0000 7f9b41600640 1 -- 172.21.15.111:0/434394961 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b3c009e50 con 0x5594effc6ba0 2024-09-15T22:50:51.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.811+0000 7f9b41600640 1 -- 172.21.15.111:0/434394961 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c0197a0 con 0x5594effc6ba0 2024-09-15T22:50:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.811+0000 7f9b48c00640 1 --2- 172.21.15.111:0/4009465788 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9b300a3240 0x7f9b300a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.811+0000 7f9b48c00640 1 -- 172.21.15.111:0/4009465788 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f9b380efea0 con 0x7f9b4c101f40 2024-09-15T22:50:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.811+0000 7f9b50e00640 1 --2- 172.21.15.111:0/4009465788 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9b300a3240 0x7f9b300a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.811+0000 7f9b55c28740 1 -- 172.21.15.111:0/434394961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5594effc6ba0 msgr2=0x5594effcb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.811+0000 7f9b55c28740 1 --2- 172.21.15.111:0/434394961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5594effc6ba0 0x5594effcb660 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c009ca0 tx=0x7f9b3c011810 comp rx=0 tx=0).stop 2024-09-15T22:50:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.812+0000 7f9b55c28740 1 -- 172.21.15.111:0/434394961 shutdown_connections 2024-09-15T22:50:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.812+0000 7f9b55c28740 1 --2- 172.21.15.111:0/434394961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5594effcbc30 0x5594effce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.812+0000 7f9b55c28740 1 --2- 172.21.15.111:0/434394961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5594effc6ba0 0x5594effcb660 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.812+0000 7f9b55c28740 1 --2- 172.21.15.111:0/434394961 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594effc6260 0x5594effc6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.812+0000 7f9b55c28740 1 -- 172.21.15.111:0/434394961 >> 172.21.15.111:0/434394961 conn(0x5594effc18d0 msgr2=0x5594effc1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:50:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.812+0000 7f9b50e00640 1 --2- 172.21.15.111:0/4009465788 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9b300a3240 0x7f9b300a5700 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9b440049d0 tx=0x7f9b44009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:50:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.812+0000 7f9b55c28740 1 -- 172.21.15.111:0/434394961 shutdown_connections 2024-09-15T22:50:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.812+0000 7f9b55c28740 1 -- 172.21.15.111:0/434394961 wait complete. 2024-09-15T22:50:51.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.812+0000 7f9b55c28740 1 -- start start 2024-09-15T22:50:51.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.812+0000 7f9b55c28740 1 Processor -- start 2024-09-15T22:50:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.813+0000 7f9b55c28740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594effc6260 0x5594effd0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.813+0000 7f9b55c28740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5594effc6ba0 0x5594effd09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.813+0000 7f9b55c28740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5594effcbc30 0x5594effd7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.813+0000 7f9b55c28740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5594effcec00 con 0x5594effc6260 2024-09-15T22:50:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.813+0000 7f9b55c28740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5594efeb4090 con 0x5594effcbc30 2024-09-15T22:50:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.813+0000 7f9b55c28740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5594effced80 con 0x5594effc6ba0 2024-09-15T22:50:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.813+0000 7f9b55c28740 10 client.0 _ll_register_callbacks cb 0x5594eff08f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:50:51.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.813+0000 7f9b4b400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5594effc6ba0 0x5594effd09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.813+0000 7f9b50e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5594effcbc30 0x5594effd7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.813+0000 7f9b4be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594effc6260 0x5594effd0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.813+0000 7f9b4b400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5594effc6ba0 0x5594effd09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:39324/0 (socket says 172.21.15.111:39324) 2024-09-15T22:50:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.813+0000 7f9b4b400640 1 -- 172.21.15.111:0/3529249164 learned_addr learned my addr 172.21.15.111:0/3529249164 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:50:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.813+0000 7f9b4b400640 1 -- 172.21.15.111:0/3529249164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5594effcbc30 msgr2=0x5594effd7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.814+0000 7f9b4b400640 1 --2- 172.21.15.111:0/3529249164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5594effcbc30 0x5594effd7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.814+0000 7f9b4b400640 1 -- 172.21.15.111:0/3529249164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594effc6260 msgr2=0x5594effd0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.814+0000 7f9b4b400640 1 --2- 172.21.15.111:0/3529249164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594effc6260 0x5594effd0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.814+0000 7f9b4b400640 1 -- 172.21.15.111:0/3529249164 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594efd0a620 con 0x5594effc6ba0 2024-09-15T22:50:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.814+0000 7f9b50e00640 1 --2- 172.21.15.111:0/3529249164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5594effcbc30 0x5594effd7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:50:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.814+0000 7f9b4b400640 1 --2- 172.21.15.111:0/3529249164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5594effc6ba0 0x5594effd09b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9b380bc270 tx=0x7f9b3803b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b4740ce58c4d203 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:50:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.814+0000 7f9b36000640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:50:51.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.814+0000 7f9b36000640 1 -- 172.21.15.111:0/3529249164 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b38048070 con 0x5594effc6ba0 2024-09-15T22:50:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.814+0000 7f9b36000640 1 -- 172.21.15.111:0/3529249164 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f9b38031bd0 con 0x5594effc6ba0 2024-09-15T22:50:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.814+0000 7f9b55c28740 1 -- 172.21.15.111:0/3529249164 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b3c009e50 con 0x5594effc6ba0 2024-09-15T22:50:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.814+0000 7f9b36000640 1 -- 172.21.15.111:0/3529249164 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9b380b8810 con 0x5594effc6ba0 2024-09-15T22:50:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.814+0000 7f9b55c28740 10 client.25408 fetch_fsmap learned FSMap version 30 2024-09-15T22:50:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.814+0000 7f9b55c28740 1 -- 172.21.15.111:0/3529249164 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5594effcec00 con 0x5594effc6ba0 2024-09-15T22:50:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.815+0000 7f9b36000640 1 -- 172.21.15.111:0/3529249164 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b3c009e50 con 0x5594effc6ba0 2024-09-15T22:50:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.815+0000 7f9b36000640 1 -- 172.21.15.111:0/3529249164 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f9b3804b060 con 0x5594effc6ba0 2024-09-15T22:50:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.815+0000 7f9b55c28740 10 client.25408 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:50:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.815+0000 7f9b55c28740 10 client.25408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:50:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.815+0000 7f9b55c28740 20 client.25408 populate_metadata read hostname 'smithi111' 2024-09-15T22:50:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.815+0000 7f9b55c28740 1 --2- 172.21.15.111:0/3529249164 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5594effdbf50 0x5594effde360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.815+0000 7f9b55c28740 4 client.25408 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:50:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.815+0000 7f9b55c28740 1 -- 172.21.15.111:0/3529249164 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594eff38830 con 0x5594effdbf50 2024-09-15T22:50:51.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.815+0000 7f9b4be00640 1 --2- 172.21.15.111:0/3529249164 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5594effdbf50 0x5594effde360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.816+0000 7f9b4be00640 1 --2- 172.21.15.111:0/3529249164 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5594effdbf50 0x5594effde360 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4b4099fbcfa1cbd server_cookie=ac7bd456397620f9 in_seq=0 out_seq=0 2024-09-15T22:50:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.816+0000 7f9b36000640 10 client.25408 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:50:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.817+0000 7f9b36000640 1 -- 172.21.15.111:0/3529249164 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5594eff38830 con 0x5594effdbf50 2024-09-15T22:50:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.817+0000 7f9b36000640 10 client.25408 handle_command_reply: tid=0 2024-09-15T22:50:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.817+0000 7f9b55c28740 1 client.25408 shutdown 2024-09-15T22:50:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.817+0000 7f9b55c28740 1 -- 172.21.15.111:0/3529249164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5594effc6ba0 msgr2=0x5594effd09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.817+0000 7f9b55c28740 1 --2- 172.21.15.111:0/3529249164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5594effc6ba0 0x5594effd09b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9b380bc270 tx=0x7f9b3803b460 comp rx=0 tx=0).stop 2024-09-15T22:50:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.817+0000 7f9b55c28740 1 -- 172.21.15.111:0/3529249164 shutdown_connections 2024-09-15T22:50:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.817+0000 7f9b55c28740 1 --2- 172.21.15.111:0/3529249164 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5594effdbf50 0x5594effde360 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.817+0000 7f9b55c28740 1 --2- 172.21.15.111:0/3529249164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5594effcbc30 0x5594effd7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.817+0000 7f9b36000640 0 client.25408 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:50:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.817+0000 7f9b55c28740 1 --2- 172.21.15.111:0/3529249164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5594effc6ba0 0x5594effd09b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.817+0000 7f9b55c28740 1 --2- 172.21.15.111:0/3529249164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594effc6260 0x5594effd0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.817+0000 7f9b55c28740 1 -- 172.21.15.111:0/3529249164 >> 172.21.15.111:0/3529249164 conn(0x5594effc18d0 msgr2=0x5594effc30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:50:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.817+0000 7f9b55c28740 1 -- 172.21.15.111:0/3529249164 shutdown_connections 2024-09-15T22:50:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.817+0000 7f9b55c28740 1 -- 172.21.15.111:0/3529249164 wait complete. 2024-09-15T22:50:51.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.817+0000 7f9b55c28740 20 client.25408 trim_cache size 0 max 16384 2024-09-15T22:50:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.832+0000 7f9b55c28740 1 Processor -- start 2024-09-15T22:50:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.832+0000 7f9b55c28740 1 -- start start 2024-09-15T22:50:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.833+0000 7f9b55c28740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5594effc6260 0x5594effbc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.833+0000 7f9b55c28740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5594effc6ba0 0x5594effd7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.833+0000 7f9b55c28740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594effcbc30 0x5594effc2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:51.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.833+0000 7f9b55c28740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c1ad740 con 0x5594effcbc30 2024-09-15T22:50:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.833+0000 7f9b55c28740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c1ab010 con 0x5594effc6ba0 2024-09-15T22:50:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.833+0000 7f9b55c28740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c1a8e60 con 0x5594effc6260 2024-09-15T22:50:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.833+0000 7f9b4b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5594effc6ba0 0x5594effd7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.833+0000 7f9b4b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5594effc6ba0 0x5594effd7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:57152/0 (socket says 172.21.15.111:57152) 2024-09-15T22:50:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.833+0000 7f9b4be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5594effc6260 0x5594effbc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.833+0000 7f9b4b400640 1 -- 172.21.15.111:0/771189550 learned_addr learned my addr 172.21.15.111:0/771189550 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:50:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.833+0000 7f9b50e00640 1 --2- 172.21.15.111:0/771189550 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594effcbc30 0x5594effc2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.833+0000 7f9b4b400640 1 -- 172.21.15.111:0/771189550 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5594effc6260 msgr2=0x5594effbc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:51.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.833+0000 7f9b4b400640 1 --2- 172.21.15.111:0/771189550 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5594effc6260 0x5594effbc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.833+0000 7f9b4b400640 1 -- 172.21.15.111:0/771189550 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594effcbc30 msgr2=0x5594effc2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.833+0000 7f9b4b400640 1 --2- 172.21.15.111:0/771189550 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594effcbc30 0x5594effc2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.833+0000 7f9b4b400640 1 -- 172.21.15.111:0/771189550 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594effc16f0 con 0x5594effc6ba0 2024-09-15T22:50:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.833+0000 7f9b50e00640 1 --2- 172.21.15.111:0/771189550 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594effcbc30 0x5594effc2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:50:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.834+0000 7f9b4b400640 1 --2- 172.21.15.111:0/771189550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5594effc6ba0 0x5594effd7940 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9b380bc500 tx=0x7f9b38009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33d6b82ddc52032 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:50:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.834+0000 7f9b40c00640 1 -- 172.21.15.111:0/771189550 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b380f7020 con 0x5594effc6ba0 2024-09-15T22:50:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.834+0000 7f9b40c00640 1 -- 172.21.15.111:0/771189550 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b380c5130 con 0x5594effc6ba0 2024-09-15T22:50:51.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.834+0000 7f9b40c00640 1 -- 172.21.15.111:0/771189550 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b38048070 con 0x5594effc6ba0 2024-09-15T22:50:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.834+0000 7f9b55c28740 1 -- 172.21.15.111:0/771189550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5594effc6ba0 msgr2=0x5594effd7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:51.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.834+0000 7f9b55c28740 1 --2- 172.21.15.111:0/771189550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5594effc6ba0 0x5594effd7940 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9b380bc500 tx=0x7f9b38009620 comp rx=0 tx=0).stop 2024-09-15T22:50:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.835+0000 7f9b55c28740 1 -- 172.21.15.111:0/771189550 shutdown_connections 2024-09-15T22:50:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.835+0000 7f9b55c28740 1 --2- 172.21.15.111:0/771189550 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594effcbc30 0x5594effc2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.835+0000 7f9b55c28740 1 --2- 172.21.15.111:0/771189550 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5594effc6ba0 0x5594effd7940 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.835+0000 7f9b55c28740 1 --2- 172.21.15.111:0/771189550 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5594effc6260 0x5594effbc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.835+0000 7f9b55c28740 1 -- 172.21.15.111:0/771189550 >> 172.21.15.111:0/771189550 conn(0x5594effc18d0 msgr2=0x5594effd0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:50:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.835+0000 7f9b55c28740 1 -- 172.21.15.111:0/771189550 shutdown_connections 2024-09-15T22:50:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.835+0000 7f9b55c28740 1 -- 172.21.15.111:0/771189550 wait complete. 2024-09-15T22:50:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.835+0000 7f9b55c28740 1 -- start start 2024-09-15T22:50:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.835+0000 7f9b55c28740 1 Processor -- start 2024-09-15T22:50:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.835+0000 7f9b55c28740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5594effc6260 0x5594effd4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:51.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.835+0000 7f9b55c28740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5594effc6ba0 0x5594effd51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.835+0000 7f9b55c28740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594effcbc30 0x5594effd61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.835+0000 7f9b55c28740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c1a94d0 con 0x5594effcbc30 2024-09-15T22:50:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.836+0000 7f9b55c28740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c1aa940 con 0x5594effc6ba0 2024-09-15T22:50:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.836+0000 7f9b55c28740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c1a9ba0 con 0x5594effc6260 2024-09-15T22:50:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.836+0000 7f9b55c28740 10 client.0 _ll_register_callbacks cb 0x7f9b380f35d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:50:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.836+0000 7f9b4b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5594effc6ba0 0x5594effd51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.836+0000 7f9b4be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5594effc6260 0x5594effd4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.836+0000 7f9b4b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5594effc6ba0 0x5594effd51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:57164/0 (socket says 172.21.15.111:57164) 2024-09-15T22:50:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.836+0000 7f9b4b400640 1 -- 172.21.15.111:0/4058705532 learned_addr learned my addr 172.21.15.111:0/4058705532 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:50:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.836+0000 7f9b50e00640 1 --2- 172.21.15.111:0/4058705532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594effcbc30 0x5594effd61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:51.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.836+0000 7f9b4b400640 1 -- 172.21.15.111:0/4058705532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5594effc6260 msgr2=0x5594effd4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.836+0000 7f9b4b400640 1 --2- 172.21.15.111:0/4058705532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5594effc6260 0x5594effd4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.836+0000 7f9b4b400640 1 -- 172.21.15.111:0/4058705532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594effcbc30 msgr2=0x5594effd61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:50:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.836+0000 7f9b4b400640 1 --2- 172.21.15.111:0/4058705532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594effcbc30 0x5594effd61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.836+0000 7f9b4b400640 1 -- 172.21.15.111:0/4058705532 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594effd67d0 con 0x5594effc6ba0 2024-09-15T22:50:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.836+0000 7f9b4be00640 1 --2- 172.21.15.111:0/4058705532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5594effc6260 0x5594effd4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:50:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.836+0000 7f9b4b400640 1 --2- 172.21.15.111:0/4058705532 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5594effc6ba0 0x5594effd51b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9b380f3680 tx=0x7f9b380f3860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef51284035122a1d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:50:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.836+0000 7f9b36000640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:50:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.836+0000 7f9b36000640 1 -- 172.21.15.111:0/4058705532 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3809f020 con 0x5594effc6ba0 2024-09-15T22:50:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.837+0000 7f9b36000640 1 -- 172.21.15.111:0/4058705532 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f9b380bbbc0 con 0x5594effc6ba0 2024-09-15T22:50:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.837+0000 7f9b55c28740 1 -- 172.21.15.111:0/4058705532 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b380c5130 con 0x5594effc6ba0 2024-09-15T22:50:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.837+0000 7f9b36000640 1 -- 172.21.15.111:0/4058705532 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9b380f7030 con 0x5594effc6ba0 2024-09-15T22:50:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.837+0000 7f9b55c28740 10 client.25415 fetch_fsmap learned FSMap version 30 2024-09-15T22:50:51.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.837+0000 7f9b55c28740 1 -- 172.21.15.111:0/4058705532 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5594effd67d0 con 0x5594effc6ba0 2024-09-15T22:50:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.837+0000 7f9b36000640 1 -- 172.21.15.111:0/4058705532 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b380c5130 con 0x5594effc6ba0 2024-09-15T22:50:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.837+0000 7f9b36000640 1 -- 172.21.15.111:0/4058705532 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f9b380a4070 con 0x5594effc6ba0 2024-09-15T22:50:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.837+0000 7f9b55c28740 10 client.25415 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:50:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.837+0000 7f9b55c28740 10 client.25415 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:50:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.837+0000 7f9b55c28740 20 client.25415 populate_metadata read hostname 'smithi111' 2024-09-15T22:50:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.837+0000 7f9b55c28740 1 --2- 172.21.15.111:0/4058705532 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5594f0046e40 0x5594f0049230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:50:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.838+0000 7f9b55c28740 4 client.25415 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:50:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.838+0000 7f9b55c28740 1 -- 172.21.15.111:0/4058705532 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5594eff32b80 con 0x5594f0046e40 2024-09-15T22:50:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.838+0000 7f9b4be00640 1 --2- 172.21.15.111:0/4058705532 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5594f0046e40 0x5594f0049230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:50:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.838+0000 7f9b4be00640 1 --2- 172.21.15.111:0/4058705532 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5594f0046e40 0x5594f0049230 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f55cc0b56393c65 server_cookie=f64dd97b88cedefb in_seq=0 out_seq=0 2024-09-15T22:50:51.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.838+0000 7f9b36000640 10 client.25415 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:50:51.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.846+0000 7f9b36000640 1 -- 172.21.15.111:0/4058705532 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5594eff32b80 con 0x5594f0046e40 2024-09-15T22:50:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.846+0000 7f9b36000640 10 client.25415 handle_command_reply: tid=0 2024-09-15T22:50:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.846+0000 7f9b55c28740 1 client.25415 shutdown 2024-09-15T22:50:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.847+0000 7f9b55c28740 1 -- 172.21.15.111:0/4058705532 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5594effc6ba0 msgr2=0x5594effd51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.847+0000 7f9b55c28740 1 --2- 172.21.15.111:0/4058705532 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5594effc6ba0 0x5594effd51b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9b380f3680 tx=0x7f9b380f3860 comp rx=0 tx=0).stop 2024-09-15T22:50:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.847+0000 7f9b55c28740 1 -- 172.21.15.111:0/4058705532 shutdown_connections 2024-09-15T22:50:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.847+0000 7f9b55c28740 1 --2- 172.21.15.111:0/4058705532 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5594f0046e40 0x5594f0049230 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.847+0000 7f9b55c28740 1 --2- 172.21.15.111:0/4058705532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5594effcbc30 0x5594effd61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.847+0000 7f9b36000640 0 client.25415 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:50:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.847+0000 7f9b55c28740 1 --2- 172.21.15.111:0/4058705532 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5594effc6ba0 0x5594effd51b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.847+0000 7f9b55c28740 1 --2- 172.21.15.111:0/4058705532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5594effc6260 0x5594effd4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.847+0000 7f9b55c28740 1 -- 172.21.15.111:0/4058705532 >> 172.21.15.111:0/4058705532 conn(0x5594effc18d0 msgr2=0x5594effc01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:50:51.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.847+0000 7f9b55c28740 1 -- 172.21.15.111:0/4058705532 shutdown_connections 2024-09-15T22:50:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.847+0000 7f9b55c28740 1 -- 172.21.15.111:0/4058705532 wait complete. 2024-09-15T22:50:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.847+0000 7f9b55c28740 20 client.25415 trim_cache size 0 max 16384 2024-09-15T22:50:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:50:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:50:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:50:51.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:50:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.848+0000 7f9b51800640 1 -- 172.21.15.111:0/4009465788 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9b300a3240 msgr2=0x7f9b300a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.848+0000 7f9b51800640 1 --2- 172.21.15.111:0/4009465788 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9b300a3240 0x7f9b300a5700 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9b440049d0 tx=0x7f9b44009500 comp rx=0 tx=0).stop 2024-09-15T22:50:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.848+0000 7f9b51800640 1 -- 172.21.15.111:0/4009465788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b4c101f40 msgr2=0x7f9b4c19a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:50:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.848+0000 7f9b51800640 1 --2- 172.21.15.111:0/4009465788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b4c101f40 0x7f9b4c19a530 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9b38031440 tx=0x7f9b38004270 comp rx=0 tx=0).stop 2024-09-15T22:50:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.848+0000 7f9b51800640 1 -- 172.21.15.111:0/4009465788 shutdown_connections 2024-09-15T22:50:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.848+0000 7f9b51800640 1 --2- 172.21.15.111:0/4009465788 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b4c103050 0x7f9b4c1a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.848+0000 7f9b51800640 1 --2- 172.21.15.111:0/4009465788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9b4c101f40 0x7f9b4c19a530 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.848+0000 7f9b51800640 1 --2- 172.21.15.111:0/4009465788 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9b300a3240 0x7f9b300a5700 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.848+0000 7f9b51800640 1 --2- 172.21.15.111:0/4009465788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b4c101580 0x7f9b4c199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:50:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.848+0000 7f9b51800640 1 -- 172.21.15.111:0/4009465788 >> 172.21.15.111:0/4009465788 conn(0x7f9b4c0fd0d0 msgr2=0x7f9b4c1083c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:50:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.848+0000 7f9b51800640 1 -- 172.21.15.111:0/4009465788 shutdown_connections 2024-09-15T22:50:51.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:50:51.848+0000 7f9b51800640 1 -- 172.21.15.111:0/4009465788 wait complete. 2024-09-15T22:50:52.029 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:51 smithi111 ceph-mon[30629]: pgmap v498: 97 pgs: 97 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 183 op/s 2024-09-15T22:50:52.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:51 smithi192 ceph-mon[35427]: pgmap v498: 97 pgs: 97 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 183 op/s 2024-09-15T22:50:52.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:51 smithi121 ceph-mon[34815]: pgmap v498: 97 pgs: 97 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 183 op/s 2024-09-15T22:50:52.258 INFO:tasks.workunit.client.0.smithi111.stdout: 1 383471 9.82 MB/sec execute 80 sec latency 229.972 ms 2024-09-15T22:50:53.258 INFO:tasks.workunit.client.0.smithi111.stdout: 1 383474 9.70 MB/sec execute 81 sec latency 774.440 ms 2024-09-15T22:50:54.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:53 smithi192 ceph-mon[35427]: pgmap v499: 97 pgs: 97 active+clean; 153 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 233 op/s 2024-09-15T22:50:54.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:53 smithi121 ceph-mon[34815]: pgmap v499: 97 pgs: 97 active+clean; 153 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 233 op/s 2024-09-15T22:50:54.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:53 smithi111 ceph-mon[30629]: pgmap v499: 97 pgs: 97 active+clean; 153 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 233 op/s 2024-09-15T22:50:54.258 INFO:tasks.workunit.client.0.smithi111.stdout: 1 384918 9.64 MB/sec execute 82 sec latency 699.143 ms 2024-09-15T22:50:55.258 INFO:tasks.workunit.client.0.smithi111.stdout: 1 387591 9.70 MB/sec execute 83 sec latency 18.322 ms 2024-09-15T22:50:56.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:55 smithi192 ceph-mon[35427]: pgmap v500: 97 pgs: 97 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 237 op/s 2024-09-15T22:50:56.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:55 smithi121 ceph-mon[34815]: pgmap v500: 97 pgs: 97 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 237 op/s 2024-09-15T22:50:56.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:55 smithi111 ceph-mon[30629]: pgmap v500: 97 pgs: 97 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 237 op/s 2024-09-15T22:50:56.258 INFO:tasks.workunit.client.0.smithi111.stdout: 1 390680 9.81 MB/sec execute 84 sec latency 7.192 ms 2024-09-15T22:50:57.258 INFO:tasks.workunit.client.0.smithi111.stdout: 1 393580 9.86 MB/sec execute 85 sec latency 125.764 ms 2024-09-15T22:50:57.965 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:57 smithi111 ceph-mon[30629]: pgmap v501: 97 pgs: 97 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 188 op/s 2024-09-15T22:50:58.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:57 smithi192 ceph-mon[35427]: pgmap v501: 97 pgs: 97 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 188 op/s 2024-09-15T22:50:58.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:57 smithi121 ceph-mon[34815]: pgmap v501: 97 pgs: 97 active+clean; 154 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 188 op/s 2024-09-15T22:50:58.258 INFO:tasks.workunit.client.0.smithi111.stdout: 1 393581 9.75 MB/sec execute 86 sec latency 804.209 ms 2024-09-15T22:50:59.258 INFO:tasks.workunit.client.0.smithi111.stdout: 1 394756 9.70 MB/sec execute 87 sec latency 1149.984 ms 2024-09-15T22:51:00.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:50:59 smithi192 ceph-mon[35427]: pgmap v502: 97 pgs: 97 active+clean; 155 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 189 op/s 2024-09-15T22:51:00.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:50:59 smithi121 ceph-mon[34815]: pgmap v502: 97 pgs: 97 active+clean; 155 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 189 op/s 2024-09-15T22:51:00.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:50:59 smithi111 ceph-mon[30629]: pgmap v502: 97 pgs: 97 active+clean; 155 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 189 op/s 2024-09-15T22:51:00.258 INFO:tasks.workunit.client.0.smithi111.stdout: 1 397841 9.79 MB/sec execute 88 sec latency 15.994 ms 2024-09-15T22:51:01.258 INFO:tasks.workunit.client.0.smithi111.stdout: 1 400995 9.88 MB/sec execute 89 sec latency 11.778 ms 2024-09-15T22:51:02.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:01 smithi192 ceph-mon[35427]: pgmap v503: 97 pgs: 97 active+clean; 157 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 221 op/s 2024-09-15T22:51:02.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:01 smithi121 ceph-mon[34815]: pgmap v503: 97 pgs: 97 active+clean; 157 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 221 op/s 2024-09-15T22:51:02.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:01 smithi111 ceph-mon[30629]: pgmap v503: 97 pgs: 97 active+clean; 157 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 221 op/s 2024-09-15T22:51:02.264 INFO:tasks.workunit.client.0.smithi111.stdout: 1 403466 9.88 MB/sec execute 90 sec latency 19.086 ms 2024-09-15T22:51:03.259 INFO:tasks.workunit.client.0.smithi111.stdout: 1 403576 9.80 MB/sec execute 91 sec latency 572.430 ms 2024-09-15T22:51:03.937 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:03 smithi111 ceph-mon[30629]: pgmap v504: 97 pgs: 97 active+clean; 157 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 203 op/s 2024-09-15T22:51:04.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:03 smithi192 ceph-mon[35427]: pgmap v504: 97 pgs: 97 active+clean; 157 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 203 op/s 2024-09-15T22:51:04.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:03 smithi121 ceph-mon[34815]: pgmap v504: 97 pgs: 97 active+clean; 157 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 203 op/s 2024-09-15T22:51:04.259 INFO:tasks.workunit.client.0.smithi111.stdout: 1 404194 9.73 MB/sec execute 92 sec latency 1194.786 ms 2024-09-15T22:51:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.965+0000 7f7df7200640 1 -- 172.21.15.111:0/1504763039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7df8105720 msgr2=0x7f7df810bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.965+0000 7f7df7200640 1 --2- 172.21.15.111:0/1504763039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7df8105720 0x7f7df810bfb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009ca0 tx=0x7f7df002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:51:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.965+0000 7f7df7200640 1 -- 172.21.15.111:0/1504763039 shutdown_connections 2024-09-15T22:51:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.965+0000 7f7df7200640 1 --2- 172.21.15.111:0/1504763039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7df8105720 0x7f7df810bfb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.965+0000 7f7df7200640 1 --2- 172.21.15.111:0/1504763039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7df8104d60 0x7f7df81051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.965+0000 7f7df7200640 1 --2- 172.21.15.111:0/1504763039 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7df8103b70 0x7f7df8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.965+0000 7f7df7200640 1 -- 172.21.15.111:0/1504763039 >> 172.21.15.111:0/1504763039 conn(0x7f7df80ff2f0 msgr2=0x7f7df8101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.965+0000 7f7df7200640 1 -- 172.21.15.111:0/1504763039 shutdown_connections 2024-09-15T22:51:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.965+0000 7f7df7200640 1 -- 172.21.15.111:0/1504763039 wait complete. 2024-09-15T22:51:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.966+0000 7f7df7200640 1 Processor -- start 2024-09-15T22:51:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.967+0000 7f7df7200640 1 -- start start 2024-09-15T22:51:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.967+0000 7f7df7200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7df8103b70 0x7f7df8199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.967+0000 7f7df7200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7df8104d60 0x7f7df819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.967+0000 7f7df7200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7df8105720 0x7f7df81a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.967+0000 7f7df7200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7df810f840 con 0x7f7df8105720 2024-09-15T22:51:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.967+0000 7f7df7200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7df810f6c0 con 0x7f7df8103b70 2024-09-15T22:51:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.967+0000 7f7df7200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7df810f9c0 con 0x7f7df8104d60 2024-09-15T22:51:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.967+0000 7f7df6800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7df8105720 0x7f7df81a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.967+0000 7f7df5e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7df8103b70 0x7f7df8199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.967+0000 7f7df5e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7df8103b70 0x7f7df8199e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:35754/0 (socket says 172.21.15.111:35754) 2024-09-15T22:51:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.967+0000 7f7df6800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7df8105720 0x7f7df81a13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44272/0 (socket says 172.21.15.111:44272) 2024-09-15T22:51:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.967+0000 7f7df5e00640 1 -- 172.21.15.111:0/333218664 learned_addr learned my addr 172.21.15.111:0/333218664 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:51:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.967+0000 7f7df5400640 1 --2- 172.21.15.111:0/333218664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7df8104d60 0x7f7df819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.967+0000 7f7df5e00640 1 -- 172.21.15.111:0/333218664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7df8104d60 msgr2=0x7f7df819a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.967+0000 7f7df5e00640 1 --2- 172.21.15.111:0/333218664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7df8104d60 0x7f7df819a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.967+0000 7f7df5e00640 1 -- 172.21.15.111:0/333218664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7df8105720 msgr2=0x7f7df81a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.967+0000 7f7df5e00640 1 --2- 172.21.15.111:0/333218664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7df8105720 0x7f7df81a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.967+0000 7f7df5e00640 1 -- 172.21.15.111:0/333218664 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df81a1a20 con 0x7f7df8103b70 2024-09-15T22:51:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.968+0000 7f7df5e00640 1 --2- 172.21.15.111:0/333218664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7df8103b70 0x7f7df8199e00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7de800e9c0 tx=0x7f7de800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.968+0000 7f7deea00640 1 -- 172.21.15.111:0/333218664 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de800cd80 con 0x7f7df8103b70 2024-09-15T22:51:04.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.968+0000 7f7df7200640 1 -- 172.21.15.111:0/333218664 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df81a1d10 con 0x7f7df8103b70 2024-09-15T22:51:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.968+0000 7f7deea00640 1 -- 172.21.15.111:0/333218664 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de80035f0 con 0x7f7df8103b70 2024-09-15T22:51:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.968+0000 7f7deea00640 1 -- 172.21.15.111:0/333218664 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de8010690 con 0x7f7df8103b70 2024-09-15T22:51:04.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.968+0000 7f7df7200640 1 -- 172.21.15.111:0/333218664 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df81a21b0 con 0x7f7df8103b70 2024-09-15T22:51:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.972+0000 7f7dff890740 1 Processor -- start 2024-09-15T22:51:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.972+0000 7f7dff890740 1 -- start start 2024-09-15T22:51:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.972+0000 7f7dff890740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f682cd260 0x559f682cd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.972+0000 7f7dff890740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f682cdba0 0x559f682d2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.972+0000 7f7dff890740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f682d2c30 0x559f682d50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.972+0000 7f7dff890740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559f68232100 con 0x559f682cd260 2024-09-15T22:51:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.972+0000 7f7dff890740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559f681da1b0 con 0x559f682d2c30 2024-09-15T22:51:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.972+0000 7f7dff890740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x559f6822e0c0 con 0x559f682cdba0 2024-09-15T22:51:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.973+0000 7f7deea00640 1 -- 172.21.15.111:0/333218664 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f7de80037a0 con 0x7f7df8103b70 2024-09-15T22:51:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.973+0000 7f7df6800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f682cd260 0x559f682cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.973+0000 7f7df6800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f682cd260 0x559f682cd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44276/0 (socket says 172.21.15.111:44276) 2024-09-15T22:51:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.973+0000 7f7df6800640 1 -- 172.21.15.111:0/947065696 learned_addr learned my addr 172.21.15.111:0/947065696 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:51:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.973+0000 7f7df5400640 1 --2- 172.21.15.111:0/947065696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f682d2c30 0x559f682d50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.973+0000 7f7df5e00640 1 --2- 172.21.15.111:0/947065696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f682cdba0 0x559f682d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.973+0000 7f7df6800640 1 -- 172.21.15.111:0/947065696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f682cdba0 msgr2=0x559f682d2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.973+0000 7f7df6800640 1 --2- 172.21.15.111:0/947065696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f682cdba0 0x559f682d2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.973+0000 7f7df6800640 1 -- 172.21.15.111:0/947065696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f682d2c30 msgr2=0x559f682d50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.973+0000 7f7df6800640 1 --2- 172.21.15.111:0/947065696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f682d2c30 0x559f682d50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.973+0000 7f7df6800640 1 -- 172.21.15.111:0/947065696 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f67f45210 con 0x559f682cd260 2024-09-15T22:51:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.973+0000 7f7df5400640 1 --2- 172.21.15.111:0/947065696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f682d2c30 0x559f682d50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:51:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.973+0000 7f7df6800640 1 --2- 172.21.15.111:0/947065696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f682cd260 0x559f682cd660 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009ca0 tx=0x7f7df00093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35772f50f1aada8e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.974+0000 7f7de3000640 1 -- 172.21.15.111:0/947065696 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df0031650 con 0x559f682cd260 2024-09-15T22:51:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.974+0000 7f7de3000640 1 -- 172.21.15.111:0/947065696 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7df0036030 con 0x559f682cd260 2024-09-15T22:51:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.974+0000 7f7de3000640 1 -- 172.21.15.111:0/947065696 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df0004720 con 0x559f682cd260 2024-09-15T22:51:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.974+0000 7f7deea00640 1 --2- 172.21.15.111:0/333218664 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7ddc0a32c0 0x7f7ddc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.974+0000 7f7dff890740 1 -- 172.21.15.111:0/947065696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f682cd260 msgr2=0x559f682cd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.974+0000 7f7dff890740 1 --2- 172.21.15.111:0/947065696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f682cd260 0x559f682cd660 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009ca0 tx=0x7f7df00093c0 comp rx=0 tx=0).stop 2024-09-15T22:51:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.974+0000 7f7deea00640 1 -- 172.21.15.111:0/333218664 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f7de80cfc10 con 0x7f7df8103b70 2024-09-15T22:51:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.974+0000 7f7df5e00640 1 --2- 172.21.15.111:0/333218664 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7ddc0a32c0 0x7f7ddc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.975+0000 7f7dff890740 1 -- 172.21.15.111:0/947065696 shutdown_connections 2024-09-15T22:51:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.975+0000 7f7dff890740 1 --2- 172.21.15.111:0/947065696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f682d2c30 0x559f682d50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.975+0000 7f7dff890740 1 --2- 172.21.15.111:0/947065696 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f682cdba0 0x559f682d2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.975+0000 7f7dff890740 1 --2- 172.21.15.111:0/947065696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f682cd260 0x559f682cd660 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.975+0000 7f7dff890740 1 -- 172.21.15.111:0/947065696 >> 172.21.15.111:0/947065696 conn(0x559f682c88d0 msgr2=0x559f682c8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.975+0000 7f7dff890740 1 -- 172.21.15.111:0/947065696 shutdown_connections 2024-09-15T22:51:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.975+0000 7f7dff890740 1 -- 172.21.15.111:0/947065696 wait complete. 2024-09-15T22:51:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.975+0000 7f7df5e00640 1 --2- 172.21.15.111:0/333218664 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7ddc0a32c0 0x7f7ddc0a5780 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7de80cd710 tx=0x7f7de80045f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.975+0000 7f7dff890740 1 -- start start 2024-09-15T22:51:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.975+0000 7f7dff890740 1 Processor -- start 2024-09-15T22:51:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.976+0000 7f7dff890740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f682cd260 0x559f682d7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.976+0000 7f7dff890740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f682cdba0 0x559f682d7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.976+0000 7f7dff890740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f682d2c30 0x559f682de790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.976+0000 7f7dff890740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559f682d5a20 con 0x559f682d2c30 2024-09-15T22:51:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.976+0000 7f7dff890740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559f681bb090 con 0x559f682cdba0 2024-09-15T22:51:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.976+0000 7f7dff890740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x559f682d5ba0 con 0x559f682cd260 2024-09-15T22:51:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.976+0000 7f7dff890740 10 client.0 _ll_register_callbacks cb 0x559f6820ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:51:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.976+0000 7f7df5400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f682cdba0 0x559f682d7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.976+0000 7f7df5400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f682cdba0 0x559f682d7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:35772/0 (socket says 172.21.15.111:35772) 2024-09-15T22:51:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.976+0000 7f7df5400640 1 -- 172.21.15.111:0/3934858900 learned_addr learned my addr 172.21.15.111:0/3934858900 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:51:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.976+0000 7f7df6800640 1 --2- 172.21.15.111:0/3934858900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f682cd260 0x559f682d7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.976+0000 7f7df6800640 1 --2- 172.21.15.111:0/3934858900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f682d2c30 0x559f682de790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.976+0000 7f7df5400640 1 -- 172.21.15.111:0/3934858900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f682cd260 msgr2=0x559f682d7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.976+0000 7f7df5400640 1 --2- 172.21.15.111:0/3934858900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f682cd260 0x559f682d7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.976+0000 7f7df5400640 1 -- 172.21.15.111:0/3934858900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f682d2c30 msgr2=0x559f682de790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.976+0000 7f7df5400640 1 --2- 172.21.15.111:0/3934858900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f682d2c30 0x559f682de790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.976+0000 7f7df5400640 1 -- 172.21.15.111:0/3934858900 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f68011620 con 0x559f682cdba0 2024-09-15T22:51:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.977+0000 7f7df5400640 1 --2- 172.21.15.111:0/3934858900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f682cdba0 0x559f682d7750 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7de4012720 tx=0x7f7de4012be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77cde9c68027ea03 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.977+0000 7f7dd7400640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:51:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.977+0000 7f7dd7400640 1 -- 172.21.15.111:0/3934858900 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4004050 con 0x559f682cdba0 2024-09-15T22:51:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.977+0000 7f7dd7400640 1 -- 172.21.15.111:0/3934858900 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f7de40041f0 con 0x559f682cdba0 2024-09-15T22:51:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.977+0000 7f7dff890740 1 -- 172.21.15.111:0/3934858900 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7df0036030 con 0x559f682cdba0 2024-09-15T22:51:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.977+0000 7f7dd7400640 1 -- 172.21.15.111:0/3934858900 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7de40043f0 con 0x559f682cdba0 2024-09-15T22:51:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.977+0000 7f7dff890740 10 client.25427 fetch_fsmap learned FSMap version 30 2024-09-15T22:51:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.977+0000 7f7dff890740 1 -- 172.21.15.111:0/3934858900 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559f682e16a0 con 0x559f682cdba0 2024-09-15T22:51:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.978+0000 7f7dd7400640 1 -- 172.21.15.111:0/3934858900 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7df0036030 con 0x559f682cdba0 2024-09-15T22:51:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.978+0000 7f7dd7400640 1 -- 172.21.15.111:0/3934858900 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f7de4003ce0 con 0x559f682cdba0 2024-09-15T22:51:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.978+0000 7f7dff890740 10 client.25427 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:51:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.978+0000 7f7dff890740 10 client.25427 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:51:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.978+0000 7f7dff890740 20 client.25427 populate_metadata read hostname 'smithi111' 2024-09-15T22:51:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.978+0000 7f7dff890740 1 --2- 172.21.15.111:0/3934858900 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559f682e2d70 0x559f682e5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.978+0000 7f7dff890740 4 client.25427 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:51:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.978+0000 7f7dff890740 1 -- 172.21.15.111:0/3934858900 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f6823f830 con 0x559f682e2d70 2024-09-15T22:51:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.978+0000 7f7df6800640 1 --2- 172.21.15.111:0/3934858900 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559f682e2d70 0x559f682e5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.979+0000 7f7df6800640 1 --2- 172.21.15.111:0/3934858900 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559f682e2d70 0x559f682e5160 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21d65097733136a7 server_cookie=2b72dd60d67d50e8 in_seq=0 out_seq=0 2024-09-15T22:51:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.979+0000 7f7dd7400640 10 client.25427 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:51:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.980+0000 7f7dd7400640 1 -- 172.21.15.111:0/3934858900 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559f6823f830 con 0x559f682e2d70 2024-09-15T22:51:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.980+0000 7f7dd7400640 10 client.25427 handle_command_reply: tid=0 2024-09-15T22:51:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.980+0000 7f7dff890740 1 client.25427 shutdown 2024-09-15T22:51:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.980+0000 7f7dff890740 1 -- 172.21.15.111:0/3934858900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f682cdba0 msgr2=0x559f682d7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.980+0000 7f7dff890740 1 --2- 172.21.15.111:0/3934858900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f682cdba0 0x559f682d7750 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7de4012720 tx=0x7f7de4012be0 comp rx=0 tx=0).stop 2024-09-15T22:51:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.980+0000 7f7dff890740 1 -- 172.21.15.111:0/3934858900 shutdown_connections 2024-09-15T22:51:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.980+0000 7f7dff890740 1 --2- 172.21.15.111:0/3934858900 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559f682e2d70 0x559f682e5160 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.980+0000 7f7dff890740 1 --2- 172.21.15.111:0/3934858900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f682d2c30 0x559f682de790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.980+0000 7f7dd7400640 0 client.25427 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:51:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.980+0000 7f7dff890740 1 --2- 172.21.15.111:0/3934858900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f682cdba0 0x559f682d7750 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.980+0000 7f7dff890740 1 --2- 172.21.15.111:0/3934858900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f682cd260 0x559f682d7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.980+0000 7f7dff890740 1 -- 172.21.15.111:0/3934858900 >> 172.21.15.111:0/3934858900 conn(0x559f682c88d0 msgr2=0x559f682ca0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.980+0000 7f7dff890740 1 -- 172.21.15.111:0/3934858900 shutdown_connections 2024-09-15T22:51:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.980+0000 7f7dff890740 1 -- 172.21.15.111:0/3934858900 wait complete. 2024-09-15T22:51:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.980+0000 7f7dff890740 20 client.25427 trim_cache size 0 max 16384 2024-09-15T22:51:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.996+0000 7f7dff890740 1 Processor -- start 2024-09-15T22:51:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.997+0000 7f7dff890740 1 -- start start 2024-09-15T22:51:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.997+0000 7f7dff890740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f682cd260 0x559f682c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.997+0000 7f7dff890740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f682cdba0 0x559f682de750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.997+0000 7f7dff890740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f682d2c30 0x559f682c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.997+0000 7f7dff890740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7df81ad640 con 0x559f682cdba0 2024-09-15T22:51:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.997+0000 7f7dff890740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7df81aaf10 con 0x559f682d2c30 2024-09-15T22:51:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.997+0000 7f7dff890740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7df81a8d60 con 0x559f682cd260 2024-09-15T22:51:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.997+0000 7f7df6800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f682cd260 0x559f682c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:04.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.997+0000 7f7df6800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f682cd260 0x559f682c8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:48684/0 (socket says 172.21.15.111:48684) 2024-09-15T22:51:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.997+0000 7f7df6800640 1 -- 172.21.15.111:0/1893332086 learned_addr learned my addr 172.21.15.111:0/1893332086 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:51:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.997+0000 7f7df6800640 1 --2- 172.21.15.111:0/1893332086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f682d2c30 0x559f682c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.997+0000 7f7df5400640 1 --2- 172.21.15.111:0/1893332086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f682cdba0 0x559f682de750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.997+0000 7f7df6800640 1 -- 172.21.15.111:0/1893332086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f682d2c30 msgr2=0x559f682c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.997+0000 7f7df6800640 1 --2- 172.21.15.111:0/1893332086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f682d2c30 0x559f682c91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.997+0000 7f7df6800640 1 -- 172.21.15.111:0/1893332086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f682cdba0 msgr2=0x559f682de750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.997+0000 7f7df6800640 1 --2- 172.21.15.111:0/1893332086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f682cdba0 0x559f682de750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.997+0000 7f7df6800640 1 -- 172.21.15.111:0/1893332086 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f682ce080 con 0x559f682cd260 2024-09-15T22:51:05.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.998+0000 7f7df6800640 1 --2- 172.21.15.111:0/1893332086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f682cd260 0x559f682c8cb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009430 tx=0x7f7df0031bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b274744f2d553a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:05.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.998+0000 7f7de2600640 1 -- 172.21.15.111:0/1893332086 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df00047f0 con 0x559f682cd260 2024-09-15T22:51:05.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.998+0000 7f7de2600640 1 -- 172.21.15.111:0/1893332086 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7df0036040 con 0x559f682cd260 2024-09-15T22:51:05.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.998+0000 7f7de2600640 1 -- 172.21.15.111:0/1893332086 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df0003d90 con 0x559f682cd260 2024-09-15T22:51:05.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.999+0000 7f7dff890740 1 -- 172.21.15.111:0/1893332086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f682cd260 msgr2=0x559f682c8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:05.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.999+0000 7f7dff890740 1 --2- 172.21.15.111:0/1893332086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f682cd260 0x559f682c8cb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009430 tx=0x7f7df0031bb0 comp rx=0 tx=0).stop 2024-09-15T22:51:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.999+0000 7f7dff890740 1 -- 172.21.15.111:0/1893332086 shutdown_connections 2024-09-15T22:51:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.999+0000 7f7dff890740 1 --2- 172.21.15.111:0/1893332086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f682d2c30 0x559f682c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.999+0000 7f7dff890740 1 --2- 172.21.15.111:0/1893332086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f682cdba0 0x559f682de750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.999+0000 7f7dff890740 1 --2- 172.21.15.111:0/1893332086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f682cd260 0x559f682c8cb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.999+0000 7f7dff890740 1 -- 172.21.15.111:0/1893332086 >> 172.21.15.111:0/1893332086 conn(0x559f682c88d0 msgr2=0x559f682c36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.999+0000 7f7dff890740 1 -- 172.21.15.111:0/1893332086 shutdown_connections 2024-09-15T22:51:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.999+0000 7f7dff890740 1 -- 172.21.15.111:0/1893332086 wait complete. 2024-09-15T22:51:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:04.999+0000 7f7dff890740 1 -- start start 2024-09-15T22:51:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.000+0000 7f7dff890740 1 Processor -- start 2024-09-15T22:51:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.000+0000 7f7dff890740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f682cd260 0x559f682dbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.000+0000 7f7dff890740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f682cdba0 0x559f682dc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:05.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.000+0000 7f7dff890740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f682d2c30 0x559f682dd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.000+0000 7f7dff890740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7df81a93d0 con 0x559f682d2c30 2024-09-15T22:51:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.000+0000 7f7dff890740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7df81aa840 con 0x559f682cd260 2024-09-15T22:51:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.000+0000 7f7dff890740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7df81a9aa0 con 0x559f682cdba0 2024-09-15T22:51:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.000+0000 7f7dff890740 10 client.0 _ll_register_callbacks cb 0x7f7de4009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:51:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.000+0000 7f7df6800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f682cd260 0x559f682dbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.000+0000 7f7df6800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f682cd260 0x559f682dbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:35778/0 (socket says 172.21.15.111:35778) 2024-09-15T22:51:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.000+0000 7f7df6800640 1 -- 172.21.15.111:0/2809276130 learned_addr learned my addr 172.21.15.111:0/2809276130 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:51:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.000+0000 7f7df6800640 1 --2- 172.21.15.111:0/2809276130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f682d2c30 0x559f682dd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.001+0000 7f7df5400640 1 --2- 172.21.15.111:0/2809276130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f682cdba0 0x559f682dc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.001+0000 7f7df6800640 1 -- 172.21.15.111:0/2809276130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f682cdba0 msgr2=0x559f682dc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.001+0000 7f7df6800640 1 --2- 172.21.15.111:0/2809276130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f682cdba0 0x559f682dc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.001+0000 7f7df6800640 1 -- 172.21.15.111:0/2809276130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f682d2c30 msgr2=0x559f682dd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.001+0000 7f7df6800640 1 --2- 172.21.15.111:0/2809276130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f682d2c30 0x559f682dd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:05.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.001+0000 7f7df6800640 1 -- 172.21.15.111:0/2809276130 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f682dd8a0 con 0x559f682cd260 2024-09-15T22:51:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.001+0000 7f7df5400640 1 --2- 172.21.15.111:0/2809276130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f682cdba0 0x559f682dc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:51:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.001+0000 7f7df6800640 1 --2- 172.21.15.111:0/2809276130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f682cd260 0x559f682dbd40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7df0006010 tx=0x7f7df0033040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf94a1346f774861 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.001+0000 7f7dd7400640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:51:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.001+0000 7f7dd7400640 1 -- 172.21.15.111:0/2809276130 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df004a040 con 0x559f682cd260 2024-09-15T22:51:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.001+0000 7f7dff890740 1 -- 172.21.15.111:0/2809276130 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7df0036040 con 0x559f682cd260 2024-09-15T22:51:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.001+0000 7f7dd7400640 1 -- 172.21.15.111:0/2809276130 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f7df0003b10 con 0x559f682cd260 2024-09-15T22:51:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.001+0000 7f7dd7400640 1 -- 172.21.15.111:0/2809276130 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7df0003d10 con 0x559f682cd260 2024-09-15T22:51:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.001+0000 7f7dff890740 10 client.25433 fetch_fsmap learned FSMap version 30 2024-09-15T22:51:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.001+0000 7f7dff890740 1 -- 172.21.15.111:0/2809276130 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559f682debb0 con 0x559f682cd260 2024-09-15T22:51:05.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.002+0000 7f7dd7400640 1 -- 172.21.15.111:0/2809276130 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7df0036040 con 0x559f682cd260 2024-09-15T22:51:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.002+0000 7f7dd7400640 1 -- 172.21.15.111:0/2809276130 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f7df0043070 con 0x559f682cd260 2024-09-15T22:51:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.002+0000 7f7dff890740 10 client.25433 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:51:05.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.002+0000 7f7dff890740 10 client.25433 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:51:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.002+0000 7f7dff890740 20 client.25433 populate_metadata read hostname 'smithi111' 2024-09-15T22:51:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.002+0000 7f7dff890740 1 --2- 172.21.15.111:0/2809276130 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559f6834df40 0x559f68350330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.002+0000 7f7dff890740 4 client.25433 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:51:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.002+0000 7f7dff890740 1 -- 172.21.15.111:0/2809276130 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559f68239b80 con 0x559f6834df40 2024-09-15T22:51:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.002+0000 7f7df5400640 1 --2- 172.21.15.111:0/2809276130 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559f6834df40 0x559f68350330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.003+0000 7f7df5400640 1 --2- 172.21.15.111:0/2809276130 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559f6834df40 0x559f68350330 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93a7bf3b855b1cb3 server_cookie=a2fdfefb16e8d9ef in_seq=0 out_seq=0 2024-09-15T22:51:05.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.003+0000 7f7dd7400640 10 client.25433 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:51:05.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.010+0000 7f7dd7400640 1 -- 172.21.15.111:0/2809276130 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559f68239b80 con 0x559f6834df40 2024-09-15T22:51:05.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.010+0000 7f7dd7400640 10 client.25433 handle_command_reply: tid=0 2024-09-15T22:51:05.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.010+0000 7f7dff890740 1 client.25433 shutdown 2024-09-15T22:51:05.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.010+0000 7f7dff890740 1 -- 172.21.15.111:0/2809276130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f682cd260 msgr2=0x559f682dbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:05.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.010+0000 7f7dff890740 1 --2- 172.21.15.111:0/2809276130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f682cd260 0x559f682dbd40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7df0006010 tx=0x7f7df0033040 comp rx=0 tx=0).stop 2024-09-15T22:51:05.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.010+0000 7f7dff890740 1 -- 172.21.15.111:0/2809276130 shutdown_connections 2024-09-15T22:51:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.010+0000 7f7dff890740 1 --2- 172.21.15.111:0/2809276130 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559f6834df40 0x559f68350330 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.011+0000 7f7dff890740 1 --2- 172.21.15.111:0/2809276130 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559f682d2c30 0x559f682dd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.011+0000 7f7dff890740 1 --2- 172.21.15.111:0/2809276130 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559f682cdba0 0x559f682dc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.011+0000 7f7dff890740 1 --2- 172.21.15.111:0/2809276130 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559f682cd260 0x559f682dbd40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.011+0000 7f7dd7400640 0 client.25433 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:51:05.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.011+0000 7f7dff890740 1 -- 172.21.15.111:0/2809276130 >> 172.21.15.111:0/2809276130 conn(0x559f682c88d0 msgr2=0x559f682c71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:05.015 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.011+0000 7f7dff890740 1 -- 172.21.15.111:0/2809276130 shutdown_connections 2024-09-15T22:51:05.015 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.011+0000 7f7dff890740 1 -- 172.21.15.111:0/2809276130 wait complete. 2024-09-15T22:51:05.015 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.011+0000 7f7dff890740 20 client.25433 trim_cache size 0 max 16384 2024-09-15T22:51:05.015 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:51:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:51:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:51:05.016 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:51:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.011+0000 7f7df7200640 1 -- 172.21.15.111:0/333218664 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7ddc0a32c0 msgr2=0x7f7ddc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:05.017 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.011+0000 7f7df7200640 1 --2- 172.21.15.111:0/333218664 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7ddc0a32c0 0x7f7ddc0a5780 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7de80cd710 tx=0x7f7de80045f0 comp rx=0 tx=0).stop 2024-09-15T22:51:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.011+0000 7f7df7200640 1 -- 172.21.15.111:0/333218664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7df8103b70 msgr2=0x7f7df8199e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.011+0000 7f7df7200640 1 --2- 172.21.15.111:0/333218664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7df8103b70 0x7f7df8199e00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7de800e9c0 tx=0x7f7de800ee80 comp rx=0 tx=0).stop 2024-09-15T22:51:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.011+0000 7f7df7200640 1 -- 172.21.15.111:0/333218664 shutdown_connections 2024-09-15T22:51:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.011+0000 7f7df7200640 1 --2- 172.21.15.111:0/333218664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7df8105720 0x7f7df81a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.011+0000 7f7df7200640 1 --2- 172.21.15.111:0/333218664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7df8104d60 0x7f7df819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:05.018 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.011+0000 7f7df7200640 1 --2- 172.21.15.111:0/333218664 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7ddc0a32c0 0x7f7ddc0a5780 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:05.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.011+0000 7f7df7200640 1 --2- 172.21.15.111:0/333218664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7df8103b70 0x7f7df8199e00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:05.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.011+0000 7f7df7200640 1 -- 172.21.15.111:0/333218664 >> 172.21.15.111:0/333218664 conn(0x7f7df80ff2f0 msgr2=0x7f7df8108060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:05.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.012+0000 7f7df7200640 1 -- 172.21.15.111:0/333218664 shutdown_connections 2024-09-15T22:51:05.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:05.012+0000 7f7df7200640 1 -- 172.21.15.111:0/333218664 wait complete. 2024-09-15T22:51:05.259 INFO:tasks.workunit.client.0.smithi111.stdout: 1 406211 9.74 MB/sec execute 93 sec latency 29.829 ms 2024-09-15T22:51:06.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:05 smithi192 ceph-mon[35427]: pgmap v505: 97 pgs: 97 active+clean; 162 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 192 op/s 2024-09-15T22:51:06.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:05 smithi121 ceph-mon[34815]: pgmap v505: 97 pgs: 97 active+clean; 162 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 192 op/s 2024-09-15T22:51:06.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:05 smithi111 ceph-mon[30629]: pgmap v505: 97 pgs: 97 active+clean; 162 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 192 op/s 2024-09-15T22:51:06.259 INFO:tasks.workunit.client.0.smithi111.stdout: 1 408739 9.78 MB/sec execute 94 sec latency 16.493 ms 2024-09-15T22:51:07.259 INFO:tasks.workunit.client.0.smithi111.stdout: 1 411886 9.88 MB/sec execute 95 sec latency 11.868 ms 2024-09-15T22:51:07.908 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:07 smithi111 ceph-mon[30629]: pgmap v506: 97 pgs: 97 active+clean; 162 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 181 op/s 2024-09-15T22:51:08.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:07 smithi192 ceph-mon[35427]: pgmap v506: 97 pgs: 97 active+clean; 162 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 181 op/s 2024-09-15T22:51:08.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:07 smithi121 ceph-mon[34815]: pgmap v506: 97 pgs: 97 active+clean; 162 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 181 op/s 2024-09-15T22:51:08.259 INFO:tasks.workunit.client.0.smithi111.stdout: 1 412001 9.77 MB/sec execute 96 sec latency 848.841 ms 2024-09-15T22:51:09.259 INFO:tasks.workunit.client.0.smithi111.stdout: 1 414416 9.80 MB/sec execute 97 sec latency 1099.753 ms 2024-09-15T22:51:10.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:09 smithi192 ceph-mon[35427]: pgmap v507: 97 pgs: 97 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 175 op/s 2024-09-15T22:51:10.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:09 smithi121 ceph-mon[34815]: pgmap v507: 97 pgs: 97 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 175 op/s 2024-09-15T22:51:10.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:09 smithi111 ceph-mon[30629]: pgmap v507: 97 pgs: 97 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 175 op/s 2024-09-15T22:51:10.259 INFO:tasks.workunit.client.0.smithi111.stdout: 1 416816 9.82 MB/sec execute 98 sec latency 10.981 ms 2024-09-15T22:51:11.259 INFO:tasks.workunit.client.0.smithi111.stdout: 1 419498 9.87 MB/sec execute 99 sec latency 11.038 ms 2024-09-15T22:51:12.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:11 smithi192 ceph-mon[35427]: pgmap v508: 97 pgs: 97 active+clean; 161 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 197 op/s 2024-09-15T22:51:12.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:11 smithi121 ceph-mon[34815]: pgmap v508: 97 pgs: 97 active+clean; 161 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 197 op/s 2024-09-15T22:51:12.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:11 smithi111 ceph-mon[30629]: pgmap v508: 97 pgs: 97 active+clean; 161 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 197 op/s 2024-09-15T22:51:12.259 INFO:tasks.workunit.client.0.smithi111.stdout: 1 422528 9.96 MB/sec execute 100 sec latency 12.095 ms 2024-09-15T22:51:13.259 INFO:tasks.workunit.client.0.smithi111.stdout: 1 422883 9.86 MB/sec execute 101 sec latency 686.163 ms 2024-09-15T22:51:13.897 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:13 smithi111 ceph-mon[30629]: pgmap v509: 97 pgs: 97 active+clean; 164 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 205 op/s 2024-09-15T22:51:14.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:13 smithi192 ceph-mon[35427]: pgmap v509: 97 pgs: 97 active+clean; 164 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 205 op/s 2024-09-15T22:51:14.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:13 smithi121 ceph-mon[34815]: pgmap v509: 97 pgs: 97 active+clean; 164 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 205 op/s 2024-09-15T22:51:14.260 INFO:tasks.workunit.client.0.smithi111.stdout: 1 422883 9.77 MB/sec execute 102 sec latency 1686.202 ms 2024-09-15T22:51:14.996 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:14 smithi111 ceph-mon[30629]: pgmap v510: 97 pgs: 97 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 201 op/s 2024-09-15T22:51:15.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:14 smithi121 ceph-mon[34815]: pgmap v510: 97 pgs: 97 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 201 op/s 2024-09-15T22:51:15.259 INFO:tasks.workunit.client.0.smithi111.stdout: 1 422883 9.67 MB/sec execute 103 sec latency 2686.241 ms 2024-09-15T22:51:15.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:14 smithi192 ceph-mon[35427]: pgmap v510: 97 pgs: 97 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 201 op/s 2024-09-15T22:51:16.259 INFO:tasks.workunit.client.0.smithi111.stdout: 1 424193 9.63 MB/sec execute 104 sec latency 3292.347 ms 2024-09-15T22:51:17.259 INFO:tasks.workunit.client.0.smithi111.stdout: 1 426653 9.67 MB/sec execute 105 sec latency 12.695 ms 2024-09-15T22:51:17.875 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:17 smithi111 ceph-mon[30629]: pgmap v511: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 178 op/s 2024-09-15T22:51:17.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:17 smithi121 ceph-mon[34815]: pgmap v511: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 178 op/s 2024-09-15T22:51:18.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:17 smithi192 ceph-mon[35427]: pgmap v511: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 178 op/s 2024-09-15T22:51:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.126+0000 7f8732c00640 1 -- 172.21.15.111:0/2062066835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f872c104f50 msgr2=0x7f872c1053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:18.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.126+0000 7f8732c00640 1 --2- 172.21.15.111:0/2062066835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f872c104f50 0x7f872c1053d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8718009900 tx=0x7f871802a410 comp rx=0 tx=0).stop 2024-09-15T22:51:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.126+0000 7f8732c00640 1 -- 172.21.15.111:0/2062066835 shutdown_connections 2024-09-15T22:51:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.126+0000 7f8732c00640 1 --2- 172.21.15.111:0/2062066835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f872c105910 0x7f872c10c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.126+0000 7f8732c00640 1 --2- 172.21.15.111:0/2062066835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f872c104f50 0x7f872c1053d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.126+0000 7f8732c00640 1 --2- 172.21.15.111:0/2062066835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f872c103d60 0x7f872c104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.126+0000 7f8732c00640 1 -- 172.21.15.111:0/2062066835 >> 172.21.15.111:0/2062066835 conn(0x7f872c0ff4e0 msgr2=0x7f872c101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.126+0000 7f8732c00640 1 -- 172.21.15.111:0/2062066835 shutdown_connections 2024-09-15T22:51:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.126+0000 7f8732c00640 1 -- 172.21.15.111:0/2062066835 wait complete. 2024-09-15T22:51:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.127+0000 7f8732c00640 1 Processor -- start 2024-09-15T22:51:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.127+0000 7f8732c00640 1 -- start start 2024-09-15T22:51:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.128+0000 7f8732c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f872c103d60 0x7f872c19a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.128+0000 7f8732c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f872c104f50 0x7f872c19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:18.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.128+0000 7f8732c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f872c105910 0x7f872c1a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.128+0000 7f8732c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f872c10fad0 con 0x7f872c103d60 2024-09-15T22:51:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.128+0000 7f8732c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f872c10f950 con 0x7f872c104f50 2024-09-15T22:51:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.128+0000 7f8732c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f872c10fc50 con 0x7f872c105910 2024-09-15T22:51:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.128+0000 7f872be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f872c103d60 0x7f872c19a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.128+0000 7f872be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f872c103d60 0x7f872c19a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38358/0 (socket says 172.21.15.111:38358) 2024-09-15T22:51:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.128+0000 7f872be00640 1 -- 172.21.15.111:0/3204608462 learned_addr learned my addr 172.21.15.111:0/3204608462 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:51:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.128+0000 7f8730c00640 1 --2- 172.21.15.111:0/3204608462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f872c105910 0x7f872c1a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.128+0000 7f872be00640 1 -- 172.21.15.111:0/3204608462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f872c105910 msgr2=0x7f872c1a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.128+0000 7f872be00640 1 --2- 172.21.15.111:0/3204608462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f872c105910 0x7f872c1a15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.128+0000 7f872be00640 1 -- 172.21.15.111:0/3204608462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f872c104f50 msgr2=0x7f872c19a580 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:51:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.128+0000 7f872be00640 1 --2- 172.21.15.111:0/3204608462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f872c104f50 0x7f872c19a580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.128+0000 7f872be00640 1 -- 172.21.15.111:0/3204608462 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f872c1a1c60 con 0x7f872c103d60 2024-09-15T22:51:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.129+0000 7f872be00640 1 --2- 172.21.15.111:0/3204608462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f872c103d60 0x7f872c19a040 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f871c00b470 tx=0x7f871c00b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.129+0000 7f8728c00640 1 -- 172.21.15.111:0/3204608462 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c015020 con 0x7f872c103d60 2024-09-15T22:51:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.129+0000 7f8728c00640 1 -- 172.21.15.111:0/3204608462 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f871c011d70 con 0x7f872c103d60 2024-09-15T22:51:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.129+0000 7f8728c00640 1 -- 172.21.15.111:0/3204608462 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871c00f340 con 0x7f872c103d60 2024-09-15T22:51:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.129+0000 7f8732c00640 1 -- 172.21.15.111:0/3204608462 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f872c1a1f50 con 0x7f872c103d60 2024-09-15T22:51:18.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.129+0000 7f8732c00640 1 -- 172.21.15.111:0/3204608462 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f872c1a23e0 con 0x7f872c103d60 2024-09-15T22:51:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.132+0000 7f8735bc7740 1 Processor -- start 2024-09-15T22:51:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.132+0000 7f8735bc7740 1 -- start start 2024-09-15T22:51:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.132+0000 7f8728c00640 1 -- 172.21.15.111:0/3204608462 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f871c018020 con 0x7f872c103d60 2024-09-15T22:51:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.132+0000 7f8735bc7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b02c766090 0x55b02c766490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.132+0000 7f8735bc7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b02c7669d0 0x55b02c76b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.132+0000 7f8735bc7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b02c76ba60 0x55b02c76df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.133+0000 7f8735bc7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b02c6cb100 con 0x55b02c766090 2024-09-15T22:51:18.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.133+0000 7f8735bc7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b02c6731b0 con 0x55b02c7669d0 2024-09-15T22:51:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.133+0000 7f8735bc7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b02c6c70c0 con 0x55b02c76ba60 2024-09-15T22:51:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.133+0000 7f8730c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b02c766090 0x55b02c766490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.133+0000 7f8730c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b02c766090 0x55b02c766490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38360/0 (socket says 172.21.15.111:38360) 2024-09-15T22:51:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.133+0000 7f8730c00640 1 -- 172.21.15.111:0/668612992 learned_addr learned my addr 172.21.15.111:0/668612992 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:51:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.133+0000 7f872b400640 1 --2- 172.21.15.111:0/668612992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b02c76ba60 0x55b02c76df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.133+0000 7f8730c00640 1 -- 172.21.15.111:0/668612992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b02c76ba60 msgr2=0x55b02c76df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.133+0000 7f8730c00640 1 --2- 172.21.15.111:0/668612992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b02c76ba60 0x55b02c76df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.133+0000 7f8730c00640 1 -- 172.21.15.111:0/668612992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b02c7669d0 msgr2=0x55b02c76b490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:51:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.133+0000 7f8730c00640 1 --2- 172.21.15.111:0/668612992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b02c7669d0 0x55b02c76b490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.133+0000 7f8730c00640 1 -- 172.21.15.111:0/668612992 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b02c3de210 con 0x55b02c766090 2024-09-15T22:51:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.133+0000 7f8730c00640 1 --2- 172.21.15.111:0/668612992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b02c766090 0x55b02c766490 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8724010370 tx=0x7f8724010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b6420d407d5de5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.134+0000 7f8721200640 1 -- 172.21.15.111:0/668612992 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8724016070 con 0x55b02c766090 2024-09-15T22:51:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.134+0000 7f8721200640 1 -- 172.21.15.111:0/668612992 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8724004d20 con 0x55b02c766090 2024-09-15T22:51:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.134+0000 7f8721200640 1 -- 172.21.15.111:0/668612992 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8724005480 con 0x55b02c766090 2024-09-15T22:51:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.134+0000 7f8728c00640 1 --2- 172.21.15.111:0/3204608462 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f87100a3370 0x7f87100a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.134+0000 7f8728c00640 1 -- 172.21.15.111:0/3204608462 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f871c0cf330 con 0x7f872c103d60 2024-09-15T22:51:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.134+0000 7f872be00640 1 --2- 172.21.15.111:0/3204608462 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f87100a3370 0x7f87100a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.135+0000 7f8735bc7740 1 -- 172.21.15.111:0/668612992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b02c766090 msgr2=0x55b02c766490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.135+0000 7f8735bc7740 1 --2- 172.21.15.111:0/668612992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b02c766090 0x55b02c766490 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8724010370 tx=0x7f8724010830 comp rx=0 tx=0).stop 2024-09-15T22:51:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.135+0000 7f8735bc7740 1 -- 172.21.15.111:0/668612992 shutdown_connections 2024-09-15T22:51:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.135+0000 7f8735bc7740 1 --2- 172.21.15.111:0/668612992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b02c76ba60 0x55b02c76df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.135+0000 7f8735bc7740 1 --2- 172.21.15.111:0/668612992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b02c7669d0 0x55b02c76b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.135+0000 7f8735bc7740 1 --2- 172.21.15.111:0/668612992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b02c766090 0x55b02c766490 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.135+0000 7f8735bc7740 1 -- 172.21.15.111:0/668612992 >> 172.21.15.111:0/668612992 conn(0x55b02c761700 msgr2=0x55b02c761b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.135+0000 7f8735bc7740 1 -- 172.21.15.111:0/668612992 shutdown_connections 2024-09-15T22:51:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.135+0000 7f872be00640 1 --2- 172.21.15.111:0/3204608462 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f87100a3370 0x7f87100a5830 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f871c0a4090 tx=0x7f871c0107f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.135+0000 7f8735bc7740 1 -- 172.21.15.111:0/668612992 wait complete. 2024-09-15T22:51:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.135+0000 7f8735bc7740 1 -- start start 2024-09-15T22:51:18.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.136+0000 7f8735bc7740 1 Processor -- start 2024-09-15T22:51:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.136+0000 7f8735bc7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b02c766090 0x55b02c7702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.136+0000 7f8735bc7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b02c7669d0 0x55b02c7707e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:18.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.136+0000 7f8735bc7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b02c76ba60 0x55b02c777820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.136+0000 7f8735bc7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55b02c76ea30 con 0x55b02c7669d0 2024-09-15T22:51:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.136+0000 7f8735bc7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b02c654090 con 0x55b02c76ba60 2024-09-15T22:51:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.136+0000 7f8735bc7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b02c76ebb0 con 0x55b02c766090 2024-09-15T22:51:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.136+0000 7f8735bc7740 10 client.0 _ll_register_callbacks cb 0x55b02c6a8f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:51:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.136+0000 7f8730c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b02c766090 0x55b02c7702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.136+0000 7f8730c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b02c766090 0x55b02c7702a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:50152/0 (socket says 172.21.15.111:50152) 2024-09-15T22:51:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.136+0000 7f8730c00640 1 -- 172.21.15.111:0/4245852647 learned_addr learned my addr 172.21.15.111:0/4245852647 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:51:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.136+0000 7f872b400640 1 --2- 172.21.15.111:0/4245852647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b02c7669d0 0x55b02c7707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.136+0000 7f8730c00640 1 --2- 172.21.15.111:0/4245852647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b02c76ba60 0x55b02c777820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.137+0000 7f8730c00640 1 -- 172.21.15.111:0/4245852647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b02c76ba60 msgr2=0x55b02c777820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.137+0000 7f8730c00640 1 --2- 172.21.15.111:0/4245852647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b02c76ba60 0x55b02c777820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.137+0000 7f8730c00640 1 -- 172.21.15.111:0/4245852647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b02c7669d0 msgr2=0x55b02c7707e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.137+0000 7f8730c00640 1 --2- 172.21.15.111:0/4245852647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b02c7669d0 0x55b02c7707e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.137+0000 7f8730c00640 1 -- 172.21.15.111:0/4245852647 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b02c4aa620 con 0x55b02c766090 2024-09-15T22:51:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.137+0000 7f8730c00640 1 --2- 172.21.15.111:0/4245852647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b02c766090 0x55b02c7702a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8724009db0 tx=0x7f8724009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29761a706babe3d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.137+0000 7f8715600640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:51:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.137+0000 7f8715600640 1 -- 172.21.15.111:0/4245852647 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8724016070 con 0x55b02c766090 2024-09-15T22:51:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.137+0000 7f8735bc7740 1 -- 172.21.15.111:0/4245852647 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8724004d20 con 0x55b02c766090 2024-09-15T22:51:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.137+0000 7f8715600640 1 -- 172.21.15.111:0/4245852647 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f8724003970 con 0x55b02c766090 2024-09-15T22:51:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.137+0000 7f8715600640 1 -- 172.21.15.111:0/4245852647 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8724030050 con 0x55b02c766090 2024-09-15T22:51:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.137+0000 7f8735bc7740 10 client.25444 fetch_fsmap learned FSMap version 30 2024-09-15T22:51:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.137+0000 7f8735bc7740 1 -- 172.21.15.111:0/4245852647 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b02c77a610 con 0x55b02c766090 2024-09-15T22:51:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.138+0000 7f8715600640 1 -- 172.21.15.111:0/4245852647 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8724004d20 con 0x55b02c766090 2024-09-15T22:51:18.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.138+0000 7f8715600640 1 -- 172.21.15.111:0/4245852647 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f8724028020 con 0x55b02c766090 2024-09-15T22:51:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.138+0000 7f8735bc7740 10 client.25444 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:51:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.138+0000 7f8735bc7740 10 client.25444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:51:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.138+0000 7f8735bc7740 20 client.25444 populate_metadata read hostname 'smithi111' 2024-09-15T22:51:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.138+0000 7f8735bc7740 1 --2- 172.21.15.111:0/4245852647 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55b02c77ba20 0x55b02c77de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.138+0000 7f8735bc7740 4 client.25444 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:51:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.138+0000 7f8735bc7740 1 -- 172.21.15.111:0/4245852647 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b02c6d8830 con 0x55b02c77ba20 2024-09-15T22:51:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.138+0000 7f872b400640 1 --2- 172.21.15.111:0/4245852647 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55b02c77ba20 0x55b02c77de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.139+0000 7f8715600640 10 client.25444 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:51:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.139+0000 7f872b400640 1 --2- 172.21.15.111:0/4245852647 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55b02c77ba20 0x55b02c77de10 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42b519247437fb6d server_cookie=f411dc9bb1116596 in_seq=0 out_seq=0 2024-09-15T22:51:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.139+0000 7f8715600640 1 -- 172.21.15.111:0/4245852647 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b02c6d8830 con 0x55b02c77ba20 2024-09-15T22:51:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.139+0000 7f8715600640 10 client.25444 handle_command_reply: tid=0 2024-09-15T22:51:18.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.140+0000 7f8735bc7740 1 client.25444 shutdown 2024-09-15T22:51:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.140+0000 7f8735bc7740 1 -- 172.21.15.111:0/4245852647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b02c766090 msgr2=0x55b02c7702a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.140+0000 7f8735bc7740 1 --2- 172.21.15.111:0/4245852647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b02c766090 0x55b02c7702a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8724009db0 tx=0x7f8724009bb0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.140+0000 7f8735bc7740 1 -- 172.21.15.111:0/4245852647 shutdown_connections 2024-09-15T22:51:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.140+0000 7f8735bc7740 1 --2- 172.21.15.111:0/4245852647 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55b02c77ba20 0x55b02c77de10 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.140+0000 7f8735bc7740 1 --2- 172.21.15.111:0/4245852647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b02c76ba60 0x55b02c777820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.140+0000 7f8735bc7740 1 --2- 172.21.15.111:0/4245852647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b02c7669d0 0x55b02c7707e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.140+0000 7f8715600640 0 client.25444 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:51:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.140+0000 7f8735bc7740 1 --2- 172.21.15.111:0/4245852647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b02c766090 0x55b02c7702a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.140+0000 7f8735bc7740 1 -- 172.21.15.111:0/4245852647 >> 172.21.15.111:0/4245852647 conn(0x55b02c761700 msgr2=0x55b02c762f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.140+0000 7f8735bc7740 1 -- 172.21.15.111:0/4245852647 shutdown_connections 2024-09-15T22:51:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.140+0000 7f8735bc7740 1 -- 172.21.15.111:0/4245852647 wait complete. 2024-09-15T22:51:18.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.140+0000 7f8735bc7740 20 client.25444 trim_cache size 0 max 16384 2024-09-15T22:51:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.168+0000 7f8735bc7740 1 Processor -- start 2024-09-15T22:51:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.168+0000 7f8735bc7740 1 -- start start 2024-09-15T22:51:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.168+0000 7f8735bc7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b02c766090 0x55b02c770870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.168+0000 7f8735bc7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b02c7669d0 0x55b02c75c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.168+0000 7f8735bc7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b02c76ba60 0x55b02c777740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:18.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.168+0000 7f8735bc7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f872c1ada70 con 0x55b02c76ba60 2024-09-15T22:51:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.168+0000 7f8735bc7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f872c1ab340 con 0x55b02c7669d0 2024-09-15T22:51:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.168+0000 7f8735bc7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f872c1a9190 con 0x55b02c766090 2024-09-15T22:51:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.168+0000 7f8730c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b02c76ba60 0x55b02c777740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.168+0000 7f8730c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b02c76ba60 0x55b02c777740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38382/0 (socket says 172.21.15.111:38382) 2024-09-15T22:51:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.168+0000 7f8730c00640 1 -- 172.21.15.111:0/85539490 learned_addr learned my addr 172.21.15.111:0/85539490 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:51:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.168+0000 7f8730c00640 1 --2- 172.21.15.111:0/85539490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b02c766090 0x55b02c770870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.169+0000 7f8730c00640 1 -- 172.21.15.111:0/85539490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b02c766090 msgr2=0x55b02c770870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.169+0000 7f8730c00640 1 --2- 172.21.15.111:0/85539490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b02c766090 0x55b02c770870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.169+0000 7f8730c00640 1 -- 172.21.15.111:0/85539490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b02c7669d0 msgr2=0x55b02c75c8d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:51:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.169+0000 7f8730c00640 1 --2- 172.21.15.111:0/85539490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b02c7669d0 0x55b02c75c8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.169+0000 7f8730c00640 1 -- 172.21.15.111:0/85539490 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b02c761520 con 0x55b02c76ba60 2024-09-15T22:51:18.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.169+0000 7f8730c00640 1 --2- 172.21.15.111:0/85539490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b02c76ba60 0x55b02c777740 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8724009ac0 tx=0x7f8724010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfaa61775723c8f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:18.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.169+0000 7f8717e00640 1 -- 172.21.15.111:0/85539490 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8724016030 con 0x55b02c76ba60 2024-09-15T22:51:18.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.169+0000 7f8717e00640 1 -- 172.21.15.111:0/85539490 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f872401f020 con 0x55b02c76ba60 2024-09-15T22:51:18.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.169+0000 7f8717e00640 1 -- 172.21.15.111:0/85539490 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872401a3e0 con 0x55b02c76ba60 2024-09-15T22:51:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.170+0000 7f8735bc7740 1 -- 172.21.15.111:0/85539490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b02c76ba60 msgr2=0x55b02c777740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.170+0000 7f8735bc7740 1 --2- 172.21.15.111:0/85539490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b02c76ba60 0x55b02c777740 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8724009ac0 tx=0x7f8724010830 comp rx=0 tx=0).stop 2024-09-15T22:51:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.170+0000 7f8735bc7740 1 -- 172.21.15.111:0/85539490 shutdown_connections 2024-09-15T22:51:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.170+0000 7f8735bc7740 1 --2- 172.21.15.111:0/85539490 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b02c76ba60 0x55b02c777740 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.170+0000 7f8735bc7740 1 --2- 172.21.15.111:0/85539490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b02c7669d0 0x55b02c75c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.170+0000 7f8735bc7740 1 --2- 172.21.15.111:0/85539490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b02c766090 0x55b02c770870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.170+0000 7f8735bc7740 1 -- 172.21.15.111:0/85539490 >> 172.21.15.111:0/85539490 conn(0x55b02c761700 msgr2=0x55b02c760060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.170+0000 7f8735bc7740 1 -- 172.21.15.111:0/85539490 shutdown_connections 2024-09-15T22:51:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.170+0000 7f8735bc7740 1 -- 172.21.15.111:0/85539490 wait complete. 2024-09-15T22:51:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.171+0000 7f8735bc7740 1 -- start start 2024-09-15T22:51:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.171+0000 7f8735bc7740 1 Processor -- start 2024-09-15T22:51:18.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.171+0000 7f8735bc7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b02c766090 0x55b02c774900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.171+0000 7f8735bc7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b02c7669d0 0x55b02c774e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.171+0000 7f8735bc7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b02c76ba60 0x55b02c775e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.171+0000 7f8735bc7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f872c1a9800 con 0x55b02c76ba60 2024-09-15T22:51:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.171+0000 7f8735bc7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f872c1aac70 con 0x55b02c766090 2024-09-15T22:51:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.171+0000 7f8735bc7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f872c1a9ed0 con 0x55b02c7669d0 2024-09-15T22:51:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.171+0000 7f8735bc7740 10 client.0 _ll_register_callbacks cb 0x7f8718009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:51:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.172+0000 7f8730c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b02c76ba60 0x55b02c775e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.172+0000 7f8730c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b02c76ba60 0x55b02c775e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38394/0 (socket says 172.21.15.111:38394) 2024-09-15T22:51:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.172+0000 7f8730c00640 1 -- 172.21.15.111:0/694441057 learned_addr learned my addr 172.21.15.111:0/694441057 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:51:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.172+0000 7f8730c00640 1 --2- 172.21.15.111:0/694441057 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b02c766090 0x55b02c774900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.172+0000 7f8730c00640 1 -- 172.21.15.111:0/694441057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b02c7669d0 msgr2=0x55b02c774e40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:51:18.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.172+0000 7f8730c00640 1 --2- 172.21.15.111:0/694441057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b02c7669d0 0x55b02c774e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.172+0000 7f8730c00640 1 -- 172.21.15.111:0/694441057 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b02c766090 msgr2=0x55b02c774900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.172+0000 7f8730c00640 1 --2- 172.21.15.111:0/694441057 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b02c766090 0x55b02c774900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.172+0000 7f8730c00640 1 -- 172.21.15.111:0/694441057 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b02c776410 con 0x55b02c76ba60 2024-09-15T22:51:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.172+0000 7f8730c00640 1 --2- 172.21.15.111:0/694441057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b02c76ba60 0x55b02c775e00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f872402d040 tx=0x7f8724009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a48a091c3674b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.172+0000 7f8715600640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:51:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.172+0000 7f8715600640 1 -- 172.21.15.111:0/694441057 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872401a4b0 con 0x55b02c76ba60 2024-09-15T22:51:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.172+0000 7f8715600640 1 -- 172.21.15.111:0/694441057 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f87240054a0 con 0x55b02c76ba60 2024-09-15T22:51:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.172+0000 7f8735bc7740 1 -- 172.21.15.111:0/694441057 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f872401f020 con 0x55b02c76ba60 2024-09-15T22:51:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.173+0000 7f8715600640 1 -- 172.21.15.111:0/694441057 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8724012510 con 0x55b02c76ba60 2024-09-15T22:51:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.173+0000 7f8735bc7740 10 client.15429 fetch_fsmap learned FSMap version 30 2024-09-15T22:51:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.173+0000 7f8735bc7740 1 -- 172.21.15.111:0/694441057 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b02c777720 con 0x55b02c76ba60 2024-09-15T22:51:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.173+0000 7f8715600640 1 -- 172.21.15.111:0/694441057 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f872401f020 con 0x55b02c76ba60 2024-09-15T22:51:18.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.173+0000 7f8715600640 1 -- 172.21.15.111:0/694441057 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f8724028040 con 0x55b02c76ba60 2024-09-15T22:51:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.173+0000 7f8735bc7740 10 client.15429 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:51:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.173+0000 7f8735bc7740 10 client.15429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:51:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.173+0000 7f8735bc7740 20 client.15429 populate_metadata read hostname 'smithi111' 2024-09-15T22:51:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.173+0000 7f8735bc7740 1 --2- 172.21.15.111:0/694441057 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55b02c7e6a70 0x55b02c7e8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.173+0000 7f8735bc7740 4 client.15429 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:51:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.173+0000 7f8735bc7740 1 -- 172.21.15.111:0/694441057 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b02c6d2b80 con 0x55b02c7e6a70 2024-09-15T22:51:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.174+0000 7f872b400640 1 --2- 172.21.15.111:0/694441057 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55b02c7e6a70 0x55b02c7e8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.174+0000 7f872b400640 1 --2- 172.21.15.111:0/694441057 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55b02c7e6a70 0x55b02c7e8e90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24f965104ecf0fd9 server_cookie=d8bb4672fb32a50b in_seq=0 out_seq=0 2024-09-15T22:51:18.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.174+0000 7f8715600640 10 client.15429 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:51:18.188 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.186+0000 7f8715600640 1 -- 172.21.15.111:0/694441057 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b02c6d2b80 con 0x55b02c7e6a70 2024-09-15T22:51:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.186+0000 7f8715600640 10 client.15429 handle_command_reply: tid=0 2024-09-15T22:51:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.186+0000 7f8735bc7740 1 client.15429 shutdown 2024-09-15T22:51:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.187+0000 7f8735bc7740 1 -- 172.21.15.111:0/694441057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b02c76ba60 msgr2=0x55b02c775e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.187+0000 7f8735bc7740 1 --2- 172.21.15.111:0/694441057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b02c76ba60 0x55b02c775e00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f872402d040 tx=0x7f8724009a00 comp rx=0 tx=0).stop 2024-09-15T22:51:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.187+0000 7f8735bc7740 1 -- 172.21.15.111:0/694441057 shutdown_connections 2024-09-15T22:51:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.187+0000 7f8735bc7740 1 --2- 172.21.15.111:0/694441057 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55b02c7e6a70 0x55b02c7e8e90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.187+0000 7f8735bc7740 1 --2- 172.21.15.111:0/694441057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b02c76ba60 0x55b02c775e00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.187+0000 7f8735bc7740 1 --2- 172.21.15.111:0/694441057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b02c7669d0 0x55b02c774e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.187+0000 7f8735bc7740 1 --2- 172.21.15.111:0/694441057 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b02c766090 0x55b02c774900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.187+0000 7f8735bc7740 1 -- 172.21.15.111:0/694441057 >> 172.21.15.111:0/694441057 conn(0x55b02c761700 msgr2=0x55b02c760880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.187+0000 7f8715600640 0 client.15429 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:51:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.187+0000 7f8735bc7740 1 -- 172.21.15.111:0/694441057 shutdown_connections 2024-09-15T22:51:18.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.187+0000 7f8735bc7740 1 -- 172.21.15.111:0/694441057 wait complete. 2024-09-15T22:51:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.187+0000 7f8735bc7740 20 client.15429 trim_cache size 0 max 16384 2024-09-15T22:51:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:51:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:51:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:51:18.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:51:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.188+0000 7f8732c00640 1 -- 172.21.15.111:0/3204608462 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f87100a3370 msgr2=0x7f87100a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.188+0000 7f8732c00640 1 --2- 172.21.15.111:0/3204608462 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f87100a3370 0x7f87100a5830 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f871c0a4090 tx=0x7f871c0107f0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.188+0000 7f8732c00640 1 -- 172.21.15.111:0/3204608462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f872c103d60 msgr2=0x7f872c19a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.188+0000 7f8732c00640 1 --2- 172.21.15.111:0/3204608462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f872c103d60 0x7f872c19a040 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f871c00b470 tx=0x7f871c00b930 comp rx=0 tx=0).stop 2024-09-15T22:51:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.188+0000 7f8732c00640 1 -- 172.21.15.111:0/3204608462 shutdown_connections 2024-09-15T22:51:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.188+0000 7f8732c00640 1 --2- 172.21.15.111:0/3204608462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f872c105910 0x7f872c1a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.188+0000 7f8732c00640 1 --2- 172.21.15.111:0/3204608462 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f87100a3370 0x7f87100a5830 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.188+0000 7f8732c00640 1 --2- 172.21.15.111:0/3204608462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f872c104f50 0x7f872c19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.188+0000 7f8732c00640 1 --2- 172.21.15.111:0/3204608462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f872c103d60 0x7f872c19a040 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.188+0000 7f8732c00640 1 -- 172.21.15.111:0/3204608462 >> 172.21.15.111:0/3204608462 conn(0x7f872c0ff4e0 msgr2=0x7f872c108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.188+0000 7f8732c00640 1 -- 172.21.15.111:0/3204608462 shutdown_connections 2024-09-15T22:51:18.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:18.188+0000 7f8732c00640 1 -- 172.21.15.111:0/3204608462 wait complete. 2024-09-15T22:51:18.260 INFO:tasks.workunit.client.0.smithi111.stdout: 1 428115 9.64 MB/sec execute 106 sec latency 454.230 ms 2024-09-15T22:51:19.260 INFO:tasks.workunit.client.0.smithi111.stdout: 1 428977 9.62 MB/sec execute 107 sec latency 899.994 ms 2024-09-15T22:51:19.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:19 smithi121 ceph-mon[34815]: pgmap v512: 97 pgs: 97 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-09-15T22:51:19.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:19 smithi111 ceph-mon[30629]: pgmap v512: 97 pgs: 97 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-09-15T22:51:20.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:19 smithi192 ceph-mon[35427]: pgmap v512: 97 pgs: 97 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-09-15T22:51:20.260 INFO:tasks.workunit.client.0.smithi111.stdout: 1 431619 9.64 MB/sec execute 108 sec latency 11.801 ms 2024-09-15T22:51:21.260 INFO:tasks.workunit.client.0.smithi111.stdout: 1 433833 9.68 MB/sec execute 109 sec latency 16.715 ms 2024-09-15T22:51:21.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:21 smithi121 ceph-mon[34815]: pgmap v513: 97 pgs: 97 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 195 op/s 2024-09-15T22:51:21.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:21 smithi111 ceph-mon[30629]: pgmap v513: 97 pgs: 97 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 195 op/s 2024-09-15T22:51:22.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:21 smithi192 ceph-mon[35427]: pgmap v513: 97 pgs: 97 active+clean; 169 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 195 op/s 2024-09-15T22:51:22.260 INFO:tasks.workunit.client.0.smithi111.stdout: 1 436775 9.75 MB/sec execute 110 sec latency 7.805 ms 2024-09-15T22:51:23.260 INFO:tasks.workunit.client.0.smithi111.stdout: 1 437962 9.71 MB/sec execute 111 sec latency 351.314 ms 2024-09-15T22:51:23.904 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:23 smithi111 ceph-mon[30629]: pgmap v514: 97 pgs: 97 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 187 op/s 2024-09-15T22:51:24.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:23 smithi192 ceph-mon[35427]: pgmap v514: 97 pgs: 97 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 187 op/s 2024-09-15T22:51:24.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:23 smithi121 ceph-mon[34815]: pgmap v514: 97 pgs: 97 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 187 op/s 2024-09-15T22:51:24.260 INFO:tasks.workunit.client.0.smithi111.stdout: 1 437962 9.62 MB/sec execute 112 sec latency 1351.386 ms 2024-09-15T22:51:25.260 INFO:tasks.workunit.client.0.smithi111.stdout: 1 438362 9.54 MB/sec execute 113 sec latency 2227.714 ms 2024-09-15T22:51:25.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:25 smithi111 ceph-mon[30629]: pgmap v515: 97 pgs: 97 active+clean; 168 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 163 op/s 2024-09-15T22:51:26.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:25 smithi192 ceph-mon[35427]: pgmap v515: 97 pgs: 97 active+clean; 168 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 163 op/s 2024-09-15T22:51:26.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:25 smithi121 ceph-mon[34815]: pgmap v515: 97 pgs: 97 active+clean; 168 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 163 op/s 2024-09-15T22:51:26.260 INFO:tasks.workunit.client.0.smithi111.stdout: 1 440646 9.58 MB/sec execute 114 sec latency 13.603 ms 2024-09-15T22:51:27.260 INFO:tasks.workunit.client.0.smithi111.stdout: 1 443722 9.65 MB/sec execute 115 sec latency 7.099 ms 2024-09-15T22:51:27.861 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:27 smithi111 ceph-mon[30629]: pgmap v516: 97 pgs: 97 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 185 op/s 2024-09-15T22:51:28.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:27 smithi192 ceph-mon[35427]: pgmap v516: 97 pgs: 97 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 185 op/s 2024-09-15T22:51:28.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:27 smithi121 ceph-mon[34815]: pgmap v516: 97 pgs: 97 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 185 op/s 2024-09-15T22:51:28.260 INFO:tasks.workunit.client.0.smithi111.stdout: 1 444933 9.62 MB/sec execute 116 sec latency 392.973 ms 2024-09-15T22:51:29.260 INFO:tasks.workunit.client.0.smithi111.stdout: 1 444933 9.53 MB/sec execute 117 sec latency 1009.573 ms 2024-09-15T22:51:29.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:29 smithi111 ceph-mon[30629]: pgmap v517: 97 pgs: 97 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 173 op/s 2024-09-15T22:51:30.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:29 smithi192 ceph-mon[35427]: pgmap v517: 97 pgs: 97 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 173 op/s 2024-09-15T22:51:30.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:29 smithi121 ceph-mon[34815]: pgmap v517: 97 pgs: 97 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 173 op/s 2024-09-15T22:51:30.260 INFO:tasks.workunit.client.0.smithi111.stdout: 1 447149 9.57 MB/sec execute 118 sec latency 1290.904 ms 2024-09-15T22:51:31.261 INFO:tasks.workunit.client.0.smithi111.stdout: 1 449937 9.60 MB/sec execute 119 sec latency 13.802 ms 2024-09-15T22:51:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.325+0000 7efe18e00640 1 -- 172.21.15.111:0/1594932456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe14104cf0 msgr2=0x7efe141070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.325+0000 7efe18e00640 1 --2- 172.21.15.111:0/1594932456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe14104cf0 0x7efe141070e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efe00009900 tx=0x7efe0002a3f0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.325+0000 7efe18e00640 1 -- 172.21.15.111:0/1594932456 shutdown_connections 2024-09-15T22:51:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.325+0000 7efe18e00640 1 --2- 172.21.15.111:0/1594932456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe14107620 0x7efe14109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.325+0000 7efe18e00640 1 --2- 172.21.15.111:0/1594932456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe14104cf0 0x7efe141070e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.325+0000 7efe18e00640 1 --2- 172.21.15.111:0/1594932456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efe141023c0 0x7efe141047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.325+0000 7efe18e00640 1 -- 172.21.15.111:0/1594932456 >> 172.21.15.111:0/1594932456 conn(0x7efe140fbfa0 msgr2=0x7efe140fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:31.327 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.325+0000 7efe18e00640 1 -- 172.21.15.111:0/1594932456 shutdown_connections 2024-09-15T22:51:31.328 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.325+0000 7efe18e00640 1 -- 172.21.15.111:0/1594932456 wait complete. 2024-09-15T22:51:31.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.326+0000 7efe18e00640 1 Processor -- start 2024-09-15T22:51:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.326+0000 7efe18e00640 1 -- start start 2024-09-15T22:51:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.326+0000 7efe18e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe141023c0 0x7efe14199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.326+0000 7efe18e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efe14104cf0 0x7efe1419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.326+0000 7efe18e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe14107620 0x7efe141a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.326+0000 7efe18e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efe1410d650 con 0x7efe14104cf0 2024-09-15T22:51:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.327+0000 7efe18e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efe1410d4d0 con 0x7efe14107620 2024-09-15T22:51:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.327+0000 7efe18e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efe1410d7d0 con 0x7efe141023c0 2024-09-15T22:51:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.327+0000 7efe13400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe141023c0 0x7efe14199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.327+0000 7efe13e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe14107620 0x7efe141a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.327+0000 7efe13400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe141023c0 0x7efe14199ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:37054/0 (socket says 172.21.15.111:37054) 2024-09-15T22:51:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.327+0000 7efe13400640 1 -- 172.21.15.111:0/4259286461 learned_addr learned my addr 172.21.15.111:0/4259286461 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:51:31.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.327+0000 7efe13e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe14107620 0x7efe141a14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:47768/0 (socket says 172.21.15.111:47768) 2024-09-15T22:51:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.327+0000 7efe12a00640 1 --2- 172.21.15.111:0/4259286461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efe14104cf0 0x7efe1419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.327+0000 7efe13400640 1 -- 172.21.15.111:0/4259286461 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe14107620 msgr2=0x7efe141a14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.327+0000 7efe13400640 1 --2- 172.21.15.111:0/4259286461 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe14107620 0x7efe141a14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.327+0000 7efe13400640 1 -- 172.21.15.111:0/4259286461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efe14104cf0 msgr2=0x7efe1419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.327+0000 7efe13400640 1 --2- 172.21.15.111:0/4259286461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efe14104cf0 0x7efe1419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.327+0000 7efe13400640 1 -- 172.21.15.111:0/4259286461 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe141a1b10 con 0x7efe141023c0 2024-09-15T22:51:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.327+0000 7efe12a00640 1 --2- 172.21.15.111:0/4259286461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efe14104cf0 0x7efe1419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:51:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.328+0000 7efe13400640 1 --2- 172.21.15.111:0/4259286461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe141023c0 0x7efe14199ef0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efe0400d950 tx=0x7efe0400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.328+0000 7efe0be00640 1 -- 172.21.15.111:0/4259286461 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04017030 con 0x7efe141023c0 2024-09-15T22:51:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.328+0000 7efe18e00640 1 -- 172.21.15.111:0/4259286461 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe141a1e00 con 0x7efe141023c0 2024-09-15T22:51:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.328+0000 7efe0be00640 1 -- 172.21.15.111:0/4259286461 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe04004020 con 0x7efe141023c0 2024-09-15T22:51:31.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.328+0000 7efe0be00640 1 -- 172.21.15.111:0/4259286461 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe040047a0 con 0x7efe141023c0 2024-09-15T22:51:31.332 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.328+0000 7efe18e00640 1 -- 172.21.15.111:0/4259286461 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe141a2270 con 0x7efe141023c0 2024-09-15T22:51:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.332+0000 7efe1d421740 1 Processor -- start 2024-09-15T22:51:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.332+0000 7efe1d421740 1 -- start start 2024-09-15T22:51:31.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.332+0000 7efe0be00640 1 -- 172.21.15.111:0/4259286461 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7efe040041d0 con 0x7efe141023c0 2024-09-15T22:51:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.332+0000 7efe1d421740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557251fa5260 0x557251fa5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.332+0000 7efe1d421740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557251fa5ba0 0x557251faa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:31.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.332+0000 7efe1d421740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557251faac30 0x557251fad0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.332+0000 7efe1d421740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557251f0a100 con 0x557251faac30 2024-09-15T22:51:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.332+0000 7efe1d421740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x557251eb21b0 con 0x557251fa5ba0 2024-09-15T22:51:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.332+0000 7efe1d421740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557251f060c0 con 0x557251fa5260 2024-09-15T22:51:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.332+0000 7efe13400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557251fa5ba0 0x557251faa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.332+0000 7efe13e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557251fa5260 0x557251fa5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.332+0000 7efe12a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557251faac30 0x557251fad0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.332+0000 7efe13e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557251fa5260 0x557251fa5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:37058/0 (socket says 172.21.15.111:37058) 2024-09-15T22:51:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.332+0000 7efe13e00640 1 -- 172.21.15.111:0/3149421758 learned_addr learned my addr 172.21.15.111:0/3149421758 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:51:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.333+0000 7efe13400640 1 -- 172.21.15.111:0/3149421758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557251fa5260 msgr2=0x557251fa5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.333+0000 7efe13400640 1 --2- 172.21.15.111:0/3149421758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557251fa5260 0x557251fa5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.333+0000 7efe13400640 1 -- 172.21.15.111:0/3149421758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557251faac30 msgr2=0x557251fad0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.333+0000 7efe13400640 1 --2- 172.21.15.111:0/3149421758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557251faac30 0x557251fad0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.333+0000 7efe13400640 1 -- 172.21.15.111:0/3149421758 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557251c1d210 con 0x557251fa5ba0 2024-09-15T22:51:31.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.333+0000 7efe13400640 1 --2- 172.21.15.111:0/3149421758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557251fa5ba0 0x557251faa660 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efe040cce40 tx=0x7efe040d7800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91e2d3193d9afd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:31.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.333+0000 7efe08c00640 1 -- 172.21.15.111:0/3149421758 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe040d1520 con 0x557251fa5ba0 2024-09-15T22:51:31.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.333+0000 7efe08c00640 1 -- 172.21.15.111:0/3149421758 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe040cfc70 con 0x557251fa5ba0 2024-09-15T22:51:31.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.333+0000 7efe08c00640 1 -- 172.21.15.111:0/3149421758 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe040dedc0 con 0x557251fa5ba0 2024-09-15T22:51:31.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.334+0000 7efe0be00640 1 --2- 172.21.15.111:0/4259286461 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7efdf80a3240 0x7efdf80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:31.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.334+0000 7efe0be00640 1 -- 172.21.15.111:0/4259286461 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7efe040cf3a0 con 0x7efe141023c0 2024-09-15T22:51:31.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.334+0000 7efe13e00640 1 --2- 172.21.15.111:0/4259286461 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7efdf80a3240 0x7efdf80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:31.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.334+0000 7efe1d421740 1 -- 172.21.15.111:0/3149421758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557251fa5ba0 msgr2=0x557251faa660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:31.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.334+0000 7efe1d421740 1 --2- 172.21.15.111:0/3149421758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557251fa5ba0 0x557251faa660 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efe040cce40 tx=0x7efe040d7800 comp rx=0 tx=0).stop 2024-09-15T22:51:31.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.334+0000 7efe1d421740 1 -- 172.21.15.111:0/3149421758 shutdown_connections 2024-09-15T22:51:31.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.334+0000 7efe1d421740 1 --2- 172.21.15.111:0/3149421758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557251faac30 0x557251fad0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.334+0000 7efe1d421740 1 --2- 172.21.15.111:0/3149421758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557251fa5ba0 0x557251faa660 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.334+0000 7efe1d421740 1 --2- 172.21.15.111:0/3149421758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557251fa5260 0x557251fa5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.334+0000 7efe1d421740 1 -- 172.21.15.111:0/3149421758 >> 172.21.15.111:0/3149421758 conn(0x557251fa08d0 msgr2=0x557251fa0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:31.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.335+0000 7efe1d421740 1 -- 172.21.15.111:0/3149421758 shutdown_connections 2024-09-15T22:51:31.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.335+0000 7efe13e00640 1 --2- 172.21.15.111:0/4259286461 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7efdf80a3240 0x7efdf80a5700 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efe0c013520 tx=0x7efe0c009600 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:31.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.335+0000 7efe1d421740 1 -- 172.21.15.111:0/3149421758 wait complete. 2024-09-15T22:51:31.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.336+0000 7efe1d421740 1 -- start start 2024-09-15T22:51:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.336+0000 7efe1d421740 1 Processor -- start 2024-09-15T22:51:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.336+0000 7efe1d421740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557251fa5260 0x557251faf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.336+0000 7efe1d421740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557251fa5ba0 0x557251faf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.336+0000 7efe1d421740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557251faac30 0x557251fb6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.336+0000 7efe1d421740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557251fadc00 con 0x557251fa5ba0 2024-09-15T22:51:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.336+0000 7efe1d421740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x557251e93090 con 0x557251fa5260 2024-09-15T22:51:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.336+0000 7efe1d421740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557251fadd80 con 0x557251faac30 2024-09-15T22:51:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.337+0000 7efe1d421740 10 client.0 _ll_register_callbacks cb 0x557251ee7f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:51:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.337+0000 7efe13400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557251fa5260 0x557251faf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.337+0000 7efe13400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557251fa5260 0x557251faf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:47788/0 (socket says 172.21.15.111:47788) 2024-09-15T22:51:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.337+0000 7efe13400640 1 -- 172.21.15.111:0/3587215361 learned_addr learned my addr 172.21.15.111:0/3587215361 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:51:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.337+0000 7efe12a00640 1 --2- 172.21.15.111:0/3587215361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557251fa5ba0 0x557251faf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.337+0000 7efe13e00640 1 --2- 172.21.15.111:0/3587215361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557251faac30 0x557251fb6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.337+0000 7efe13400640 1 -- 172.21.15.111:0/3587215361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557251faac30 msgr2=0x557251fb6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.337+0000 7efe13400640 1 --2- 172.21.15.111:0/3587215361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557251faac30 0x557251fb6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.337+0000 7efe13400640 1 -- 172.21.15.111:0/3587215361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557251fa5ba0 msgr2=0x557251faf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.337+0000 7efe13400640 1 --2- 172.21.15.111:0/3587215361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557251fa5ba0 0x557251faf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.337+0000 7efe13400640 1 -- 172.21.15.111:0/3587215361 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557251ce9620 con 0x557251fa5260 2024-09-15T22:51:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.337+0000 7efe13e00640 1 --2- 172.21.15.111:0/3587215361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557251faac30 0x557251fb6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:51:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.337+0000 7efe13400640 1 --2- 172.21.15.111:0/3587215361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557251fa5260 0x557251faf410 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efe04098390 tx=0x7efe040046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db3af9ec0cf37395 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.337+0000 7efdfd600640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:51:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.338+0000 7efdfd600640 1 -- 172.21.15.111:0/3587215361 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04097c60 con 0x557251fa5260 2024-09-15T22:51:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.338+0000 7efdfd600640 1 -- 172.21.15.111:0/3587215361 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7efe040860b0 con 0x557251fa5260 2024-09-15T22:51:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.338+0000 7efe1d421740 1 -- 172.21.15.111:0/3587215361 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe040cfc70 con 0x557251fa5260 2024-09-15T22:51:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.338+0000 7efdfd600640 1 -- 172.21.15.111:0/3587215361 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efe0409b320 con 0x557251fa5260 2024-09-15T22:51:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.338+0000 7efe1d421740 10 client.25451 fetch_fsmap learned FSMap version 30 2024-09-15T22:51:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.338+0000 7efe1d421740 1 -- 172.21.15.111:0/3587215361 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557251fadc00 con 0x557251fa5260 2024-09-15T22:51:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.338+0000 7efdfd600640 1 -- 172.21.15.111:0/3587215361 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe040cfc70 con 0x557251fa5260 2024-09-15T22:51:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.339+0000 7efdfd600640 1 -- 172.21.15.111:0/3587215361 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7efe04098030 con 0x557251fa5260 2024-09-15T22:51:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.339+0000 7efe1d421740 10 client.25451 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:51:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.339+0000 7efe1d421740 10 client.25451 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:51:31.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.339+0000 7efe1d421740 20 client.25451 populate_metadata read hostname 'smithi111' 2024-09-15T22:51:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.339+0000 7efe1d421740 1 --2- 172.21.15.111:0/3587215361 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557251fba900 0x557251fbccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.339+0000 7efe1d421740 4 client.25451 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:51:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.339+0000 7efe1d421740 1 -- 172.21.15.111:0/3587215361 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557251f17830 con 0x557251fba900 2024-09-15T22:51:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.339+0000 7efe12a00640 1 --2- 172.21.15.111:0/3587215361 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557251fba900 0x557251fbccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.339+0000 7efe12a00640 1 --2- 172.21.15.111:0/3587215361 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557251fba900 0x557251fbccf0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=111fe672462587d1 server_cookie=39a435ba30f6b5d6 in_seq=0 out_seq=0 2024-09-15T22:51:31.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.339+0000 7efdfd600640 10 client.25451 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:51:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.340+0000 7efdfd600640 1 -- 172.21.15.111:0/3587215361 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557251f17830 con 0x557251fba900 2024-09-15T22:51:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.340+0000 7efdfd600640 10 client.25451 handle_command_reply: tid=0 2024-09-15T22:51:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.340+0000 7efe1d421740 1 client.25451 shutdown 2024-09-15T22:51:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.340+0000 7efe1d421740 1 -- 172.21.15.111:0/3587215361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557251fa5260 msgr2=0x557251faf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.340+0000 7efe1d421740 1 --2- 172.21.15.111:0/3587215361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557251fa5260 0x557251faf410 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efe04098390 tx=0x7efe040046c0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.341+0000 7efe1d421740 1 -- 172.21.15.111:0/3587215361 shutdown_connections 2024-09-15T22:51:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.341+0000 7efe1d421740 1 --2- 172.21.15.111:0/3587215361 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557251fba900 0x557251fbccf0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.341+0000 7efe1d421740 1 --2- 172.21.15.111:0/3587215361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557251faac30 0x557251fb6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.341+0000 7efe1d421740 1 --2- 172.21.15.111:0/3587215361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557251fa5ba0 0x557251faf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.341+0000 7efe1d421740 1 --2- 172.21.15.111:0/3587215361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557251fa5260 0x557251faf410 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.341+0000 7efdfd600640 0 client.25451 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:51:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.341+0000 7efe1d421740 1 -- 172.21.15.111:0/3587215361 >> 172.21.15.111:0/3587215361 conn(0x557251fa08d0 msgr2=0x557251fa20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.341+0000 7efe1d421740 1 -- 172.21.15.111:0/3587215361 shutdown_connections 2024-09-15T22:51:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.341+0000 7efe1d421740 1 -- 172.21.15.111:0/3587215361 wait complete. 2024-09-15T22:51:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.341+0000 7efe1d421740 20 client.25451 trim_cache size 0 max 16384 2024-09-15T22:51:31.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.357+0000 7efe1d421740 1 Processor -- start 2024-09-15T22:51:31.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.357+0000 7efe1d421740 1 -- start start 2024-09-15T22:51:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.357+0000 7efe1d421740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557251fa5260 0x557251faf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.357+0000 7efe1d421740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557251fa5ba0 0x557251f9b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.357+0000 7efe1d421740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557251faac30 0x557251fb6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.357+0000 7efe1d421740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efe141ad860 con 0x557251fa5ba0 2024-09-15T22:51:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.357+0000 7efe1d421740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efe141ab130 con 0x557251fa5260 2024-09-15T22:51:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.357+0000 7efe1d421740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efe141a8f80 con 0x557251faac30 2024-09-15T22:51:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.358+0000 7efe13400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557251fa5260 0x557251faf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.358+0000 7efe13400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557251fa5260 0x557251faf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:47800/0 (socket says 172.21.15.111:47800) 2024-09-15T22:51:31.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.358+0000 7efe13400640 1 -- 172.21.15.111:0/2357390519 learned_addr learned my addr 172.21.15.111:0/2357390519 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:51:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.358+0000 7efe12a00640 1 --2- 172.21.15.111:0/2357390519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557251fa5ba0 0x557251f9b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.358+0000 7efe13e00640 1 --2- 172.21.15.111:0/2357390519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557251faac30 0x557251fb6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.358+0000 7efe13400640 1 -- 172.21.15.111:0/2357390519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557251faac30 msgr2=0x557251fb6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.358+0000 7efe13400640 1 --2- 172.21.15.111:0/2357390519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557251faac30 0x557251fb6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.358+0000 7efe13400640 1 -- 172.21.15.111:0/2357390519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557251fa5ba0 msgr2=0x557251f9b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.358+0000 7efe13400640 1 --2- 172.21.15.111:0/2357390519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557251fa5ba0 0x557251f9b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.358+0000 7efe13400640 1 -- 172.21.15.111:0/2357390519 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557251fae480 con 0x557251fa5260 2024-09-15T22:51:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.358+0000 7efe12a00640 1 --2- 172.21.15.111:0/2357390519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557251fa5ba0 0x557251f9b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:51:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.358+0000 7efe13400640 1 --2- 172.21.15.111:0/2357390519 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557251fa5260 0x557251faf9e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efe040cf690 tx=0x7efe04099ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2691226e6a932a4d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.358+0000 7efdffe00640 1 -- 172.21.15.111:0/2357390519 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0409b500 con 0x557251fa5260 2024-09-15T22:51:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.358+0000 7efdffe00640 1 -- 172.21.15.111:0/2357390519 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe0409c0c0 con 0x557251fa5260 2024-09-15T22:51:31.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.358+0000 7efdffe00640 1 -- 172.21.15.111:0/2357390519 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0409b790 con 0x557251fa5260 2024-09-15T22:51:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.359+0000 7efe1d421740 1 -- 172.21.15.111:0/2357390519 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557251fa5260 msgr2=0x557251faf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.359+0000 7efe1d421740 1 --2- 172.21.15.111:0/2357390519 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557251fa5260 0x557251faf9e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efe040cf690 tx=0x7efe04099ed0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.359+0000 7efe1d421740 1 -- 172.21.15.111:0/2357390519 shutdown_connections 2024-09-15T22:51:31.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.359+0000 7efe1d421740 1 --2- 172.21.15.111:0/2357390519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557251faac30 0x557251fb6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.359+0000 7efe1d421740 1 --2- 172.21.15.111:0/2357390519 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557251fa5ba0 0x557251f9b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.359+0000 7efe1d421740 1 --2- 172.21.15.111:0/2357390519 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557251fa5260 0x557251faf9e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.359+0000 7efe1d421740 1 -- 172.21.15.111:0/2357390519 >> 172.21.15.111:0/2357390519 conn(0x557251fa08d0 msgr2=0x557251f9f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.359+0000 7efe1d421740 1 -- 172.21.15.111:0/2357390519 shutdown_connections 2024-09-15T22:51:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.359+0000 7efe1d421740 1 -- 172.21.15.111:0/2357390519 wait complete. 2024-09-15T22:51:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.360+0000 7efe1d421740 1 -- start start 2024-09-15T22:51:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.360+0000 7efe1d421740 1 Processor -- start 2024-09-15T22:51:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.360+0000 7efe1d421740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557251fa5260 0x557251fb3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.360+0000 7efe1d421740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557251fa5ba0 0x557251fb41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.360+0000 7efe1d421740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557251faac30 0x557251fb5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.360+0000 7efe1d421740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efe141a95f0 con 0x557251faac30 2024-09-15T22:51:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.360+0000 7efe1d421740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efe141aaa60 con 0x557251fa5260 2024-09-15T22:51:31.365 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.360+0000 7efe1d421740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efe141a9cc0 con 0x557251fa5ba0 2024-09-15T22:51:31.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.360+0000 7efe1d421740 10 client.0 _ll_register_callbacks cb 0x7efe00009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:51:31.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.360+0000 7efe12a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557251fa5ba0 0x557251fb41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:31.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.360+0000 7efe13400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557251fa5260 0x557251fb3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:31.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.361+0000 7efe12a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557251fa5ba0 0x557251fb41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:37096/0 (socket says 172.21.15.111:37096) 2024-09-15T22:51:31.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.361+0000 7efe12a00640 1 -- 172.21.15.111:0/2822720733 learned_addr learned my addr 172.21.15.111:0/2822720733 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:51:31.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.361+0000 7efe13e00640 1 --2- 172.21.15.111:0/2822720733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557251faac30 0x557251fb5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:31.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.361+0000 7efe13400640 1 -- 172.21.15.111:0/2822720733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557251fa5ba0 msgr2=0x557251fb41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:31.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.361+0000 7efe13400640 1 --2- 172.21.15.111:0/2822720733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557251fa5ba0 0x557251fb41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.361+0000 7efe13400640 1 -- 172.21.15.111:0/2822720733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557251faac30 msgr2=0x557251fb5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:31.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.361+0000 7efe13400640 1 --2- 172.21.15.111:0/2822720733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557251faac30 0x557251fb5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.361+0000 7efe13400640 1 -- 172.21.15.111:0/2822720733 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557251fb5770 con 0x557251fa5260 2024-09-15T22:51:31.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.361+0000 7efe12a00640 1 --2- 172.21.15.111:0/2822720733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557251fa5ba0 0x557251fb41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:51:31.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.361+0000 7efe13e00640 1 --2- 172.21.15.111:0/2822720733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557251faac30 0x557251fb5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:51:31.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.361+0000 7efe13400640 1 --2- 172.21.15.111:0/2822720733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557251fa5260 0x557251fb3c60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x557251fb01c0 tx=0x7efe04097d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b221beb0c27197a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:31.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.361+0000 7efdfd600640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:51:31.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.361+0000 7efdfd600640 1 -- 172.21.15.111:0/2822720733 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0409b790 con 0x557251fa5260 2024-09-15T22:51:31.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.361+0000 7efdfd600640 1 -- 172.21.15.111:0/2822720733 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7efe040a41a0 con 0x557251fa5260 2024-09-15T22:51:31.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.361+0000 7efe1d421740 1 -- 172.21.15.111:0/2822720733 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe0409c0c0 con 0x557251fa5260 2024-09-15T22:51:31.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.361+0000 7efdfd600640 1 -- 172.21.15.111:0/2822720733 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efe0409b500 con 0x557251fa5260 2024-09-15T22:51:31.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.361+0000 7efe1d421740 10 client.25457 fetch_fsmap learned FSMap version 30 2024-09-15T22:51:31.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.361+0000 7efe1d421740 1 -- 172.21.15.111:0/2822720733 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557251fb5770 con 0x557251fa5260 2024-09-15T22:51:31.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.362+0000 7efdfd600640 1 -- 172.21.15.111:0/2822720733 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe0409c0c0 con 0x557251fa5260 2024-09-15T22:51:31.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.362+0000 7efdfd600640 1 -- 172.21.15.111:0/2822720733 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7efe0409ae10 con 0x557251fa5260 2024-09-15T22:51:31.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.362+0000 7efe1d421740 10 client.25457 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:51:31.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.362+0000 7efe1d421740 10 client.25457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:51:31.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.362+0000 7efe1d421740 20 client.25457 populate_metadata read hostname 'smithi111' 2024-09-15T22:51:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.362+0000 7efe1d421740 1 --2- 172.21.15.111:0/2822720733 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557252025720 0x557252027b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.362+0000 7efe1d421740 4 client.25457 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:51:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.362+0000 7efe1d421740 1 -- 172.21.15.111:0/2822720733 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557251f11b80 con 0x557252025720 2024-09-15T22:51:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.362+0000 7efe12a00640 1 --2- 172.21.15.111:0/2822720733 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557252025720 0x557252027b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.363+0000 7efe12a00640 1 --2- 172.21.15.111:0/2822720733 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557252025720 0x557252027b10 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c99bdcfe89277c5b server_cookie=94e1e4d321668240 in_seq=0 out_seq=0 2024-09-15T22:51:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.363+0000 7efdfd600640 10 client.25457 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:51:31.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.371+0000 7efdfd600640 1 -- 172.21.15.111:0/2822720733 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557251f11b80 con 0x557252025720 2024-09-15T22:51:31.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.371+0000 7efdfd600640 10 client.25457 handle_command_reply: tid=0 2024-09-15T22:51:31.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.371+0000 7efe1d421740 1 client.25457 shutdown 2024-09-15T22:51:31.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.371+0000 7efe1d421740 1 -- 172.21.15.111:0/2822720733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557251fa5260 msgr2=0x557251fb3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:31.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.371+0000 7efe1d421740 1 --2- 172.21.15.111:0/2822720733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557251fa5260 0x557251fb3c60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x557251fb01c0 tx=0x7efe04097d10 comp rx=0 tx=0).stop 2024-09-15T22:51:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.371+0000 7efe1d421740 1 -- 172.21.15.111:0/2822720733 shutdown_connections 2024-09-15T22:51:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.371+0000 7efe1d421740 1 --2- 172.21.15.111:0/2822720733 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557252025720 0x557252027b10 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.371+0000 7efe1d421740 1 --2- 172.21.15.111:0/2822720733 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557251faac30 0x557251fb5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.371+0000 7efdfd600640 0 client.25457 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:51:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.371+0000 7efe1d421740 1 --2- 172.21.15.111:0/2822720733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557251fa5ba0 0x557251fb41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.371+0000 7efe1d421740 1 --2- 172.21.15.111:0/2822720733 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557251fa5260 0x557251fb3c60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.371+0000 7efe1d421740 1 -- 172.21.15.111:0/2822720733 >> 172.21.15.111:0/2822720733 conn(0x557251fa08d0 msgr2=0x557251f9f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.372+0000 7efe1d421740 1 -- 172.21.15.111:0/2822720733 shutdown_connections 2024-09-15T22:51:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.372+0000 7efe1d421740 1 -- 172.21.15.111:0/2822720733 wait complete. 2024-09-15T22:51:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.372+0000 7efe1d421740 20 client.25457 trim_cache size 0 max 16384 2024-09-15T22:51:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:51:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:51:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:51:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:51:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.372+0000 7efe18e00640 1 -- 172.21.15.111:0/4259286461 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7efdf80a3240 msgr2=0x7efdf80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.372+0000 7efe18e00640 1 --2- 172.21.15.111:0/4259286461 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7efdf80a3240 0x7efdf80a5700 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efe0c013520 tx=0x7efe0c009600 comp rx=0 tx=0).stop 2024-09-15T22:51:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.372+0000 7efe18e00640 1 -- 172.21.15.111:0/4259286461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe141023c0 msgr2=0x7efe14199ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.372+0000 7efe18e00640 1 --2- 172.21.15.111:0/4259286461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe141023c0 0x7efe14199ef0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efe0400d950 tx=0x7efe0400de10 comp rx=0 tx=0).stop 2024-09-15T22:51:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.373+0000 7efe18e00640 1 -- 172.21.15.111:0/4259286461 shutdown_connections 2024-09-15T22:51:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.373+0000 7efe18e00640 1 --2- 172.21.15.111:0/4259286461 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7efdf80a3240 0x7efdf80a5700 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.373+0000 7efe18e00640 1 --2- 172.21.15.111:0/4259286461 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe14107620 0x7efe141a14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.373+0000 7efe18e00640 1 --2- 172.21.15.111:0/4259286461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efe14104cf0 0x7efe1419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.373+0000 7efe18e00640 1 --2- 172.21.15.111:0/4259286461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe141023c0 0x7efe14199ef0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.373+0000 7efe18e00640 1 -- 172.21.15.111:0/4259286461 >> 172.21.15.111:0/4259286461 conn(0x7efe140fbfa0 msgr2=0x7efe14101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.373+0000 7efe18e00640 1 -- 172.21.15.111:0/4259286461 shutdown_connections 2024-09-15T22:51:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:31.373+0000 7efe18e00640 1 -- 172.21.15.111:0/4259286461 wait complete. 2024-09-15T22:51:31.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:31 smithi111 ceph-mon[30629]: pgmap v518: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 170 op/s 2024-09-15T22:51:32.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:31 smithi192 ceph-mon[35427]: pgmap v518: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 170 op/s 2024-09-15T22:51:32.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:31 smithi121 ceph-mon[34815]: pgmap v518: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 170 op/s 2024-09-15T22:51:32.260 INFO:tasks.workunit.client.0.smithi111.stdout: 1 452356 9.62 MB/sec execute 120 sec latency 15.517 ms 2024-09-15T22:51:33.260 INFO:tasks.workunit.client.0.smithi111.stdout: 1 454448 9.65 MB/sec execute 121 sec latency 286.339 ms 2024-09-15T22:51:33.798 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:33 smithi111 ceph-mon[30629]: pgmap v519: 97 pgs: 97 active+clean; 174 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 198 op/s 2024-09-15T22:51:34.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:33 smithi192 ceph-mon[35427]: pgmap v519: 97 pgs: 97 active+clean; 174 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 198 op/s 2024-09-15T22:51:34.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:33 smithi121 ceph-mon[34815]: pgmap v519: 97 pgs: 97 active+clean; 174 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 198 op/s 2024-09-15T22:51:34.261 INFO:tasks.workunit.client.0.smithi111.stdout: 1 454451 9.57 MB/sec execute 122 sec latency 941.882 ms 2024-09-15T22:51:35.261 INFO:tasks.workunit.client.0.smithi111.stdout: 1 455901 9.54 MB/sec execute 123 sec latency 760.275 ms 2024-09-15T22:51:35.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:35 smithi121 ceph-mon[34815]: pgmap v520: 97 pgs: 97 active+clean; 174 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 186 op/s 2024-09-15T22:51:36.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:35 smithi192 ceph-mon[35427]: pgmap v520: 97 pgs: 97 active+clean; 174 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 186 op/s 2024-09-15T22:51:36.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:35 smithi111 ceph-mon[30629]: pgmap v520: 97 pgs: 97 active+clean; 174 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 186 op/s 2024-09-15T22:51:36.261 INFO:tasks.workunit.client.0.smithi111.stdout: 1 458344 9.57 MB/sec execute 124 sec latency 41.276 ms 2024-09-15T22:51:37.261 INFO:tasks.workunit.client.0.smithi111.stdout: 1 461112 9.75 MB/sec execute 125 sec latency 45.873 ms 2024-09-15T22:51:38.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:37 smithi192 ceph-mon[35427]: pgmap v521: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 203 op/s 2024-09-15T22:51:38.094 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:37 smithi111 ceph-mon[30629]: pgmap v521: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 203 op/s 2024-09-15T22:51:38.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:37 smithi121 ceph-mon[34815]: pgmap v521: 97 pgs: 97 active+clean; 177 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 203 op/s 2024-09-15T22:51:38.261 INFO:tasks.workunit.client.0.smithi111.stdout: 1 463073 9.76 MB/sec execute 126 sec latency 123.692 ms 2024-09-15T22:51:39.261 INFO:tasks.workunit.client.0.smithi111.stdout: 1 463073 9.68 MB/sec execute 127 sec latency 1123.742 ms 2024-09-15T22:51:40.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:39 smithi192 ceph-mon[35427]: pgmap v522: 97 pgs: 97 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 192 op/s 2024-09-15T22:51:40.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:39 smithi121 ceph-mon[34815]: pgmap v522: 97 pgs: 97 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 192 op/s 2024-09-15T22:51:40.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:39 smithi111 ceph-mon[30629]: pgmap v522: 97 pgs: 97 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.8 MiB/s wr, 192 op/s 2024-09-15T22:51:40.261 INFO:tasks.workunit.client.0.smithi111.stdout: 1 465740 9.73 MB/sec execute 128 sec latency 1291.053 ms 2024-09-15T22:51:41.261 INFO:tasks.workunit.client.0.smithi111.stdout: 1 468740 9.77 MB/sec execute 129 sec latency 12.330 ms 2024-09-15T22:51:42.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:41 smithi192 ceph-mon[35427]: pgmap v523: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 182 op/s 2024-09-15T22:51:42.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:41 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:51:42.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:41 smithi121 ceph-mon[34815]: pgmap v523: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 182 op/s 2024-09-15T22:51:42.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:41 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:51:42.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:41 smithi111 ceph-mon[30629]: pgmap v523: 97 pgs: 97 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 182 op/s 2024-09-15T22:51:42.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:41 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:51:42.261 INFO:tasks.workunit.client.0.smithi111.stdout: 1 471256 9.78 MB/sec execute 130 sec latency 11.332 ms 2024-09-15T22:51:43.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:42 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:51:43.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:42 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:51:43.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:42 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:51:43.067 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:42 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:51:43.067 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:42 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:51:43.067 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:42 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:51:43.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:42 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:51:43.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:42 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:51:43.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:42 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:51:43.261 INFO:tasks.workunit.client.0.smithi111.stdout: 1 473476 9.81 MB/sec execute 131 sec latency 36.110 ms 2024-09-15T22:51:43.898 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:43 smithi111 ceph-mon[30629]: pgmap v524: 97 pgs: 97 active+clean; 166 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 250 op/s 2024-09-15T22:51:44.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:43 smithi192 ceph-mon[35427]: pgmap v524: 97 pgs: 97 active+clean; 166 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 250 op/s 2024-09-15T22:51:44.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:43 smithi121 ceph-mon[34815]: pgmap v524: 97 pgs: 97 active+clean; 166 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 250 op/s 2024-09-15T22:51:44.261 INFO:tasks.workunit.client.0.smithi111.stdout: 1 473476 9.74 MB/sec execute 132 sec latency 1036.171 ms 2024-09-15T22:51:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.488+0000 7f5f32000640 1 -- 172.21.15.111:0/687724554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f2c103b70 msgr2=0x7f5f2c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.488+0000 7f5f32000640 1 --2- 172.21.15.111:0/687724554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f2c103b70 0x7f5f2c103f70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009900 tx=0x7f5f2002a410 comp rx=0 tx=0).stop 2024-09-15T22:51:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.488+0000 7f5f32000640 1 -- 172.21.15.111:0/687724554 shutdown_connections 2024-09-15T22:51:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.488+0000 7f5f32000640 1 --2- 172.21.15.111:0/687724554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f2c105720 0x7f5f2c10bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.488+0000 7f5f32000640 1 --2- 172.21.15.111:0/687724554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f2c104d60 0x7f5f2c1051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.488+0000 7f5f32000640 1 --2- 172.21.15.111:0/687724554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f2c103b70 0x7f5f2c103f70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.488+0000 7f5f32000640 1 -- 172.21.15.111:0/687724554 >> 172.21.15.111:0/687724554 conn(0x7f5f2c0ff2f0 msgr2=0x7f5f2c101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.488+0000 7f5f32000640 1 -- 172.21.15.111:0/687724554 shutdown_connections 2024-09-15T22:51:44.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.488+0000 7f5f32000640 1 -- 172.21.15.111:0/687724554 wait complete. 2024-09-15T22:51:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.489+0000 7f5f32000640 1 Processor -- start 2024-09-15T22:51:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.489+0000 7f5f32000640 1 -- start start 2024-09-15T22:51:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.489+0000 7f5f32000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f2c103b70 0x7f5f2c199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.489+0000 7f5f32000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f2c104d60 0x7f5f2c19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.489+0000 7f5f32000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f2c105720 0x7f5f2c1a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.489+0000 7f5f32000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c10f7f0 con 0x7f5f2c105720 2024-09-15T22:51:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.489+0000 7f5f32000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c10f670 con 0x7f5f2c103b70 2024-09-15T22:51:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.489+0000 7f5f32000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c10f970 con 0x7f5f2c104d60 2024-09-15T22:51:44.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.490+0000 7f5f2be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f2c104d60 0x7f5f2c19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.490+0000 7f5f31600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f2c105720 0x7f5f2c1a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.490+0000 7f5f2be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f2c104d60 0x7f5f2c19a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:41378/0 (socket says 172.21.15.111:41378) 2024-09-15T22:51:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.490+0000 7f5f2be00640 1 -- 172.21.15.111:0/3115999218 learned_addr learned my addr 172.21.15.111:0/3115999218 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:51:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.490+0000 7f5f2be00640 1 -- 172.21.15.111:0/3115999218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f2c103b70 msgr2=0x7f5f2c199ea0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:51:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.490+0000 7f5f2be00640 1 --2- 172.21.15.111:0/3115999218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f2c103b70 0x7f5f2c199ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.490+0000 7f5f2be00640 1 -- 172.21.15.111:0/3115999218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f2c105720 msgr2=0x7f5f2c1a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.490+0000 7f5f2be00640 1 --2- 172.21.15.111:0/3115999218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f2c105720 0x7f5f2c1a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.490+0000 7f5f2be00640 1 -- 172.21.15.111:0/3115999218 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2c1a1ac0 con 0x7f5f2c104d60 2024-09-15T22:51:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.490+0000 7f5f31600640 1 --2- 172.21.15.111:0/3115999218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f2c105720 0x7f5f2c1a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:51:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.490+0000 7f5f2be00640 1 --2- 172.21.15.111:0/3115999218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f2c104d60 0x7f5f2c19a3e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800e9f0 tx=0x7f5f1800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.490+0000 7f5f29600640 1 -- 172.21.15.111:0/3115999218 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1800cd80 con 0x7f5f2c104d60 2024-09-15T22:51:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.491+0000 7f5f29600640 1 -- 172.21.15.111:0/3115999218 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f18005500 con 0x7f5f2c104d60 2024-09-15T22:51:44.494 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.491+0000 7f5f32000640 1 -- 172.21.15.111:0/3115999218 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f2c1a1db0 con 0x7f5f2c104d60 2024-09-15T22:51:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.491+0000 7f5f29600640 1 -- 172.21.15.111:0/3115999218 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f18010690 con 0x7f5f2c104d60 2024-09-15T22:51:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.491+0000 7f5f32000640 1 -- 172.21.15.111:0/3115999218 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f2c1a2270 con 0x7f5f2c104d60 2024-09-15T22:51:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.494+0000 7f5f363f7740 1 Processor -- start 2024-09-15T22:51:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.494+0000 7f5f363f7740 1 -- start start 2024-09-15T22:51:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.494+0000 7f5f29600640 1 -- 172.21.15.111:0/3115999218 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5f180046c0 con 0x7f5f2c104d60 2024-09-15T22:51:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.494+0000 7f5f363f7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d94cb1260 0x555d94cb1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.494+0000 7f5f363f7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d94cb1ba0 0x555d94cb6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.494+0000 7f5f363f7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d94cb6c30 0x555d94cb90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.494+0000 7f5f363f7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555d94c16100 con 0x555d94cb6c30 2024-09-15T22:51:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.494+0000 7f5f363f7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x555d94bbe1b0 con 0x555d94cb1ba0 2024-09-15T22:51:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.494+0000 7f5f363f7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555d94c120c0 con 0x555d94cb1260 2024-09-15T22:51:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.494+0000 7f5f31600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d94cb1260 0x555d94cb1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.494+0000 7f5f31600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d94cb1260 0x555d94cb1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:41388/0 (socket says 172.21.15.111:41388) 2024-09-15T22:51:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.494+0000 7f5f31600640 1 -- 172.21.15.111:0/950871292 learned_addr learned my addr 172.21.15.111:0/950871292 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:51:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.494+0000 7f5f2be00640 1 --2- 172.21.15.111:0/950871292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d94cb6c30 0x555d94cb90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.495+0000 7f5f31600640 1 -- 172.21.15.111:0/950871292 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d94cb1ba0 msgr2=0x555d94cb6660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:51:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.495+0000 7f5f31600640 1 --2- 172.21.15.111:0/950871292 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d94cb1ba0 0x555d94cb6660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.495+0000 7f5f31600640 1 -- 172.21.15.111:0/950871292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d94cb6c30 msgr2=0x555d94cb90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.495+0000 7f5f31600640 1 --2- 172.21.15.111:0/950871292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d94cb6c30 0x555d94cb90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.495+0000 7f5f31600640 1 -- 172.21.15.111:0/950871292 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d94929210 con 0x555d94cb1260 2024-09-15T22:51:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.495+0000 7f5f31600640 1 --2- 172.21.15.111:0/950871292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d94cb1260 0x555d94cb1660 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5f240117e0 tx=0x7f5f24011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d9392c210c253b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.495+0000 7f5f1dc00640 1 -- 172.21.15.111:0/950871292 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24011310 con 0x555d94cb1260 2024-09-15T22:51:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.495+0000 7f5f1dc00640 1 -- 172.21.15.111:0/950871292 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f24003e50 con 0x555d94cb1260 2024-09-15T22:51:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.495+0000 7f5f1dc00640 1 -- 172.21.15.111:0/950871292 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24004560 con 0x555d94cb1260 2024-09-15T22:51:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.496+0000 7f5f29600640 1 --2- 172.21.15.111:0/3115999218 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5f140a32c0 0x7f5f140a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.496+0000 7f5f363f7740 1 -- 172.21.15.111:0/950871292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d94cb1260 msgr2=0x555d94cb1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.496+0000 7f5f363f7740 1 --2- 172.21.15.111:0/950871292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d94cb1260 0x555d94cb1660 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5f240117e0 tx=0x7f5f24011ca0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.496+0000 7f5f29600640 1 -- 172.21.15.111:0/3115999218 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f5f180cfd10 con 0x7f5f2c104d60 2024-09-15T22:51:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.496+0000 7f5f363f7740 1 -- 172.21.15.111:0/950871292 shutdown_connections 2024-09-15T22:51:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.496+0000 7f5f363f7740 1 --2- 172.21.15.111:0/950871292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d94cb6c30 0x555d94cb90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.496+0000 7f5f363f7740 1 --2- 172.21.15.111:0/950871292 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d94cb1ba0 0x555d94cb6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.496+0000 7f5f30c00640 1 --2- 172.21.15.111:0/3115999218 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5f140a32c0 0x7f5f140a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.496+0000 7f5f363f7740 1 --2- 172.21.15.111:0/950871292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d94cb1260 0x555d94cb1660 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.496+0000 7f5f363f7740 1 -- 172.21.15.111:0/950871292 >> 172.21.15.111:0/950871292 conn(0x555d94cac8d0 msgr2=0x555d94caccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.496+0000 7f5f363f7740 1 -- 172.21.15.111:0/950871292 shutdown_connections 2024-09-15T22:51:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.496+0000 7f5f363f7740 1 -- 172.21.15.111:0/950871292 wait complete. 2024-09-15T22:51:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.497+0000 7f5f363f7740 1 -- start start 2024-09-15T22:51:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.497+0000 7f5f30c00640 1 --2- 172.21.15.111:0/3115999218 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5f140a32c0 0x7f5f140a5780 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x555d94cad750 tx=0x7f5f20000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.497+0000 7f5f363f7740 1 Processor -- start 2024-09-15T22:51:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.497+0000 7f5f363f7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d94cb1260 0x555d94cbb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.498+0000 7f5f363f7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d94cb1ba0 0x555d94cbb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.498+0000 7f5f363f7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d94cb6c30 0x555d94cc2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.498+0000 7f5f363f7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555d94cb9c00 con 0x555d94cb1260 2024-09-15T22:51:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.498+0000 7f5f363f7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x555d94b9f090 con 0x555d94cb1ba0 2024-09-15T22:51:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.498+0000 7f5f363f7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555d94cb9d80 con 0x555d94cb6c30 2024-09-15T22:51:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.498+0000 7f5f363f7740 10 client.0 _ll_register_callbacks cb 0x555d94bf3f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:51:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.498+0000 7f5f30c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d94cb1260 0x555d94cbb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.498+0000 7f5f30c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d94cb1260 0x555d94cbb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43884/0 (socket says 172.21.15.111:43884) 2024-09-15T22:51:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.498+0000 7f5f30c00640 1 -- 172.21.15.111:0/1339144368 learned_addr learned my addr 172.21.15.111:0/1339144368 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:51:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.498+0000 7f5f31600640 1 --2- 172.21.15.111:0/1339144368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d94cb6c30 0x555d94cc2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.498+0000 7f5f30c00640 1 -- 172.21.15.111:0/1339144368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d94cb6c30 msgr2=0x555d94cc2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.498+0000 7f5f30c00640 1 --2- 172.21.15.111:0/1339144368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d94cb6c30 0x555d94cc2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.498+0000 7f5f30c00640 1 -- 172.21.15.111:0/1339144368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d94cb1ba0 msgr2=0x555d94cbb950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:51:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.498+0000 7f5f30c00640 1 --2- 172.21.15.111:0/1339144368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d94cb1ba0 0x555d94cbb950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.498+0000 7f5f30c00640 1 -- 172.21.15.111:0/1339144368 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d949f5620 con 0x555d94cb1260 2024-09-15T22:51:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.498+0000 7f5f31600640 1 --2- 172.21.15.111:0/1339144368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d94cb6c30 0x555d94cc2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:51:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.499+0000 7f5f30c00640 1 --2- 172.21.15.111:0/1339144368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d94cb1260 0x555d94cbb410 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5f2002a930 tx=0x7f5f2002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b3263b705d4087 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.499+0000 7f5f12000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:51:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.499+0000 7f5f12000640 1 -- 172.21.15.111:0/1339144368 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20040070 con 0x555d94cb1260 2024-09-15T22:51:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.499+0000 7f5f12000640 1 -- 172.21.15.111:0/1339144368 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f5f200035f0 con 0x555d94cb1260 2024-09-15T22:51:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.499+0000 7f5f363f7740 1 -- 172.21.15.111:0/1339144368 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f24003e50 con 0x555d94cb1260 2024-09-15T22:51:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.499+0000 7f5f12000640 1 -- 172.21.15.111:0/1339144368 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5f20035040 con 0x555d94cb1260 2024-09-15T22:51:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.499+0000 7f5f363f7740 10 client.15459 fetch_fsmap learned FSMap version 30 2024-09-15T22:51:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.499+0000 7f5f363f7740 1 -- 172.21.15.111:0/1339144368 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555d94cc57f0 con 0x555d94cb1260 2024-09-15T22:51:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.499+0000 7f5f12000640 1 -- 172.21.15.111:0/1339144368 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f24003e50 con 0x555d94cb1260 2024-09-15T22:51:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.500+0000 7f5f12000640 1 -- 172.21.15.111:0/1339144368 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f5f20047a00 con 0x555d94cb1260 2024-09-15T22:51:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.500+0000 7f5f363f7740 10 client.15459 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:51:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.500+0000 7f5f363f7740 10 client.15459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:51:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.500+0000 7f5f363f7740 20 client.15459 populate_metadata read hostname 'smithi111' 2024-09-15T22:51:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.500+0000 7f5f363f7740 1 --2- 172.21.15.111:0/1339144368 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555d94cc6f30 0x555d94cc93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.500+0000 7f5f363f7740 4 client.15459 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:51:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.500+0000 7f5f363f7740 1 -- 172.21.15.111:0/1339144368 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d94c23830 con 0x555d94cc6f30 2024-09-15T22:51:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.500+0000 7f5f2be00640 1 --2- 172.21.15.111:0/1339144368 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555d94cc6f30 0x555d94cc93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.501+0000 7f5f12000640 10 client.15459 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:51:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.501+0000 7f5f2be00640 1 --2- 172.21.15.111:0/1339144368 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555d94cc6f30 0x555d94cc93c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc2fe3948750285c server_cookie=af0fa7701d377722 in_seq=0 out_seq=0 2024-09-15T22:51:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.501+0000 7f5f12000640 1 -- 172.21.15.111:0/1339144368 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555d94c23830 con 0x555d94cc6f30 2024-09-15T22:51:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.501+0000 7f5f12000640 10 client.15459 handle_command_reply: tid=0 2024-09-15T22:51:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.502+0000 7f5f363f7740 1 client.15459 shutdown 2024-09-15T22:51:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.502+0000 7f5f363f7740 1 -- 172.21.15.111:0/1339144368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d94cb1260 msgr2=0x555d94cbb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.502+0000 7f5f363f7740 1 --2- 172.21.15.111:0/1339144368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d94cb1260 0x555d94cbb410 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5f2002a930 tx=0x7f5f2002a410 comp rx=0 tx=0).stop 2024-09-15T22:51:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.502+0000 7f5f363f7740 1 -- 172.21.15.111:0/1339144368 shutdown_connections 2024-09-15T22:51:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.502+0000 7f5f363f7740 1 --2- 172.21.15.111:0/1339144368 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555d94cc6f30 0x555d94cc93c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.502+0000 7f5f363f7740 1 --2- 172.21.15.111:0/1339144368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d94cb6c30 0x555d94cc2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.502+0000 7f5f363f7740 1 --2- 172.21.15.111:0/1339144368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d94cb1ba0 0x555d94cbb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.502+0000 7f5f12000640 0 client.15459 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:51:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.502+0000 7f5f363f7740 1 --2- 172.21.15.111:0/1339144368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d94cb1260 0x555d94cbb410 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.502+0000 7f5f363f7740 1 -- 172.21.15.111:0/1339144368 >> 172.21.15.111:0/1339144368 conn(0x555d94cac8d0 msgr2=0x555d94cae0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.502+0000 7f5f363f7740 1 -- 172.21.15.111:0/1339144368 shutdown_connections 2024-09-15T22:51:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.502+0000 7f5f363f7740 1 -- 172.21.15.111:0/1339144368 wait complete. 2024-09-15T22:51:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.502+0000 7f5f363f7740 20 client.15459 trim_cache size 0 max 16384 2024-09-15T22:51:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.517+0000 7f5f363f7740 1 Processor -- start 2024-09-15T22:51:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.517+0000 7f5f363f7740 1 -- start start 2024-09-15T22:51:44.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.517+0000 7f5f363f7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d94cac8d0 0x555d94cad780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.517+0000 7f5f363f7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d94cb1260 0x555d94cc2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.517+0000 7f5f363f7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d94cb1ba0 0x555d94cadcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.517+0000 7f5f363f7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c1ad830 con 0x555d94cb1ba0 2024-09-15T22:51:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.517+0000 7f5f363f7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c1ab100 con 0x555d94cac8d0 2024-09-15T22:51:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.517+0000 7f5f363f7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c1a8f50 con 0x555d94cb1260 2024-09-15T22:51:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.517+0000 7f5f31600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d94cb1ba0 0x555d94cadcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.517+0000 7f5f30c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d94cac8d0 0x555d94cad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.517+0000 7f5f30c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d94cac8d0 0x555d94cad780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:38764/0 (socket says 172.21.15.111:38764) 2024-09-15T22:51:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.517+0000 7f5f30c00640 1 -- 172.21.15.111:0/2393740347 learned_addr learned my addr 172.21.15.111:0/2393740347 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:51:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.518+0000 7f5f30c00640 1 -- 172.21.15.111:0/2393740347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d94cb1260 msgr2=0x555d94cc2950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:51:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.518+0000 7f5f30c00640 1 --2- 172.21.15.111:0/2393740347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d94cb1260 0x555d94cc2950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.518+0000 7f5f30c00640 1 -- 172.21.15.111:0/2393740347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d94cb1ba0 msgr2=0x555d94cadcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.518+0000 7f5f30c00640 1 --2- 172.21.15.111:0/2393740347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d94cb1ba0 0x555d94cadcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.518+0000 7f5f30c00640 1 -- 172.21.15.111:0/2393740347 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d94cb2080 con 0x555d94cac8d0 2024-09-15T22:51:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.518+0000 7f5f30c00640 1 --2- 172.21.15.111:0/2393740347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d94cac8d0 0x555d94cad780 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009560 tx=0x7f5f20004450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f02e34651241d86 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.518+0000 7f5f1d200640 1 -- 172.21.15.111:0/2393740347 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2005f070 con 0x555d94cac8d0 2024-09-15T22:51:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.518+0000 7f5f1d200640 1 -- 172.21.15.111:0/2393740347 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f2004c070 con 0x555d94cac8d0 2024-09-15T22:51:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.518+0000 7f5f1d200640 1 -- 172.21.15.111:0/2393740347 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20040070 con 0x555d94cac8d0 2024-09-15T22:51:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.519+0000 7f5f363f7740 1 -- 172.21.15.111:0/2393740347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d94cac8d0 msgr2=0x555d94cad780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.519+0000 7f5f363f7740 1 --2- 172.21.15.111:0/2393740347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d94cac8d0 0x555d94cad780 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009560 tx=0x7f5f20004450 comp rx=0 tx=0).stop 2024-09-15T22:51:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.519+0000 7f5f363f7740 1 -- 172.21.15.111:0/2393740347 shutdown_connections 2024-09-15T22:51:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.519+0000 7f5f363f7740 1 --2- 172.21.15.111:0/2393740347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d94cb1ba0 0x555d94cadcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.519+0000 7f5f363f7740 1 --2- 172.21.15.111:0/2393740347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d94cb1260 0x555d94cc2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.519+0000 7f5f363f7740 1 --2- 172.21.15.111:0/2393740347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d94cac8d0 0x555d94cad780 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.519+0000 7f5f363f7740 1 -- 172.21.15.111:0/2393740347 >> 172.21.15.111:0/2393740347 conn(0x555d94cb6750 msgr2=0x555d94cbb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.519+0000 7f5f363f7740 1 -- 172.21.15.111:0/2393740347 shutdown_connections 2024-09-15T22:51:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.519+0000 7f5f363f7740 1 -- 172.21.15.111:0/2393740347 wait complete. 2024-09-15T22:51:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.519+0000 7f5f363f7740 1 -- start start 2024-09-15T22:51:44.523 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.519+0000 7f5f363f7740 1 Processor -- start 2024-09-15T22:51:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.520+0000 7f5f363f7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d94cac8d0 0x555d94cc00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.520+0000 7f5f363f7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d94cb1260 0x555d94cc05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.520+0000 7f5f363f7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d94cb1ba0 0x555d94cc15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.520+0000 7f5f363f7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c1a95c0 con 0x555d94cb1ba0 2024-09-15T22:51:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.520+0000 7f5f363f7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c1aaa30 con 0x555d94cac8d0 2024-09-15T22:51:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.520+0000 7f5f363f7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c1a9c90 con 0x555d94cb1260 2024-09-15T22:51:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.520+0000 7f5f363f7740 10 client.0 _ll_register_callbacks cb 0x7f5f240093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:51:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.520+0000 7f5f31600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d94cb1ba0 0x555d94cc15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.520+0000 7f5f31600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d94cb1ba0 0x555d94cc15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43906/0 (socket says 172.21.15.111:43906) 2024-09-15T22:51:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.520+0000 7f5f31600640 1 -- 172.21.15.111:0/4035915649 learned_addr learned my addr 172.21.15.111:0/4035915649 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:51:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.520+0000 7f5f2be00640 1 --2- 172.21.15.111:0/4035915649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d94cb1260 0x555d94cc05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.521+0000 7f5f31600640 1 -- 172.21.15.111:0/4035915649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d94cb1260 msgr2=0x555d94cc05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.521+0000 7f5f31600640 1 --2- 172.21.15.111:0/4035915649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d94cb1260 0x555d94cc05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.521+0000 7f5f31600640 1 -- 172.21.15.111:0/4035915649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d94cac8d0 msgr2=0x555d94cc00a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:51:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.521+0000 7f5f31600640 1 --2- 172.21.15.111:0/4035915649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d94cac8d0 0x555d94cc00a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.521+0000 7f5f31600640 1 -- 172.21.15.111:0/4035915649 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d94cc1c00 con 0x555d94cb1ba0 2024-09-15T22:51:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.521+0000 7f5f31600640 1 --2- 172.21.15.111:0/4035915649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d94cb1ba0 0x555d94cc15f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400f790 tx=0x7f5f24003d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673e8b81d9794a14 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.521+0000 7f5f12000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:51:44.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.521+0000 7f5f12000640 1 -- 172.21.15.111:0/4035915649 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24003e20 con 0x555d94cb1ba0 2024-09-15T22:51:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.521+0000 7f5f363f7740 1 -- 172.21.15.111:0/4035915649 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f2004c070 con 0x555d94cb1ba0 2024-09-15T22:51:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.521+0000 7f5f12000640 1 -- 172.21.15.111:0/4035915649 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f5f2400fbb0 con 0x555d94cb1ba0 2024-09-15T22:51:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.521+0000 7f5f12000640 1 -- 172.21.15.111:0/4035915649 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5f2400fdb0 con 0x555d94cb1ba0 2024-09-15T22:51:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.521+0000 7f5f363f7740 10 client.15465 fetch_fsmap learned FSMap version 30 2024-09-15T22:51:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.521+0000 7f5f363f7740 1 -- 172.21.15.111:0/4035915649 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555d94d31dc0 con 0x555d94cb1ba0 2024-09-15T22:51:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.522+0000 7f5f12000640 1 -- 172.21.15.111:0/4035915649 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f2004c070 con 0x555d94cb1ba0 2024-09-15T22:51:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.522+0000 7f5f12000640 1 -- 172.21.15.111:0/4035915649 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f5f24018020 con 0x555d94cb1ba0 2024-09-15T22:51:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.522+0000 7f5f363f7740 10 client.15465 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:51:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.522+0000 7f5f363f7740 10 client.15465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:51:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.522+0000 7f5f363f7740 20 client.15465 populate_metadata read hostname 'smithi111' 2024-09-15T22:51:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.522+0000 7f5f363f7740 1 --2- 172.21.15.111:0/4035915649 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555d94cb6c30 0x555d94d344e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.522+0000 7f5f363f7740 4 client.15465 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:51:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.522+0000 7f5f363f7740 1 -- 172.21.15.111:0/4035915649 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555d94c1db80 con 0x555d94cb6c30 2024-09-15T22:51:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.522+0000 7f5f30c00640 1 --2- 172.21.15.111:0/4035915649 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555d94cb6c30 0x555d94d344e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.523+0000 7f5f30c00640 1 --2- 172.21.15.111:0/4035915649 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555d94cb6c30 0x555d94d344e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8eae69b97e86da05 server_cookie=e0f9d3ac5f8bec9f in_seq=0 out_seq=0 2024-09-15T22:51:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.523+0000 7f5f12000640 10 client.15465 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:51:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.533+0000 7f5f12000640 1 -- 172.21.15.111:0/4035915649 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555d94c1db80 con 0x555d94cb6c30 2024-09-15T22:51:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.533+0000 7f5f12000640 10 client.15465 handle_command_reply: tid=0 2024-09-15T22:51:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.533+0000 7f5f363f7740 1 client.15465 shutdown 2024-09-15T22:51:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.533+0000 7f5f363f7740 1 -- 172.21.15.111:0/4035915649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d94cb1ba0 msgr2=0x555d94cc15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.533+0000 7f5f363f7740 1 --2- 172.21.15.111:0/4035915649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d94cb1ba0 0x555d94cc15f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5f2400f790 tx=0x7f5f24003d30 comp rx=0 tx=0).stop 2024-09-15T22:51:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.533+0000 7f5f363f7740 1 -- 172.21.15.111:0/4035915649 shutdown_connections 2024-09-15T22:51:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.533+0000 7f5f363f7740 1 --2- 172.21.15.111:0/4035915649 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555d94cb6c30 0x555d94d344e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.533+0000 7f5f363f7740 1 --2- 172.21.15.111:0/4035915649 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555d94cb1ba0 0x555d94cc15f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.533+0000 7f5f12000640 0 client.15465 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:51:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.533+0000 7f5f363f7740 1 --2- 172.21.15.111:0/4035915649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555d94cb1260 0x555d94cc05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.533+0000 7f5f363f7740 1 --2- 172.21.15.111:0/4035915649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d94cac8d0 0x555d94cc00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.533+0000 7f5f363f7740 1 -- 172.21.15.111:0/4035915649 >> 172.21.15.111:0/4035915649 conn(0x555d94cb6750 msgr2=0x555d94cbb190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.533+0000 7f5f363f7740 1 -- 172.21.15.111:0/4035915649 shutdown_connections 2024-09-15T22:51:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.533+0000 7f5f363f7740 1 -- 172.21.15.111:0/4035915649 wait complete. 2024-09-15T22:51:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.533+0000 7f5f363f7740 20 client.15465 trim_cache size 0 max 16384 2024-09-15T22:51:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:51:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:51:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:51:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:51:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.534+0000 7f5f32000640 1 -- 172.21.15.111:0/3115999218 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5f140a32c0 msgr2=0x7f5f140a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.534+0000 7f5f32000640 1 --2- 172.21.15.111:0/3115999218 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5f140a32c0 0x7f5f140a5780 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x555d94cad750 tx=0x7f5f20000bc0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.534+0000 7f5f32000640 1 -- 172.21.15.111:0/3115999218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f2c104d60 msgr2=0x7f5f2c19a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.534+0000 7f5f32000640 1 --2- 172.21.15.111:0/3115999218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f2c104d60 0x7f5f2c19a3e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800e9f0 tx=0x7f5f1800edf0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.534+0000 7f5f32000640 1 -- 172.21.15.111:0/3115999218 shutdown_connections 2024-09-15T22:51:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.534+0000 7f5f32000640 1 --2- 172.21.15.111:0/3115999218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f2c105720 0x7f5f2c1a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.534+0000 7f5f32000640 1 --2- 172.21.15.111:0/3115999218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f2c104d60 0x7f5f2c19a3e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.534+0000 7f5f32000640 1 --2- 172.21.15.111:0/3115999218 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5f140a32c0 0x7f5f140a5780 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.534+0000 7f5f32000640 1 --2- 172.21.15.111:0/3115999218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f2c103b70 0x7f5f2c199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.534+0000 7f5f32000640 1 -- 172.21.15.111:0/3115999218 >> 172.21.15.111:0/3115999218 conn(0x7f5f2c0ff2f0 msgr2=0x7f5f2c1081d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.534+0000 7f5f32000640 1 -- 172.21.15.111:0/3115999218 shutdown_connections 2024-09-15T22:51:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:44.534+0000 7f5f32000640 1 -- 172.21.15.111:0/3115999218 wait complete. 2024-09-15T22:51:45.261 INFO:tasks.workunit.client.0.smithi111.stdout: 1 473483 9.66 MB/sec execute 133 sec latency 1054.765 ms 2024-09-15T22:51:46.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:45 smithi192 ceph-mon[35427]: pgmap v525: 97 pgs: 97 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 215 op/s 2024-09-15T22:51:46.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:45 smithi121 ceph-mon[34815]: pgmap v525: 97 pgs: 97 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 215 op/s 2024-09-15T22:51:46.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:45 smithi111 ceph-mon[30629]: pgmap v525: 97 pgs: 97 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 215 op/s 2024-09-15T22:51:46.261 INFO:tasks.workunit.client.0.smithi111.stdout: 1 476387 9.73 MB/sec execute 134 sec latency 1107.008 ms 2024-09-15T22:51:47.261 INFO:tasks.workunit.client.0.smithi111.stdout: 1 479420 9.76 MB/sec execute 135 sec latency 36.841 ms 2024-09-15T22:51:48.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:47 smithi192 ceph-mon[35427]: pgmap v526: 97 pgs: 97 active+clean; 169 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 211 op/s 2024-09-15T22:51:48.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:47 smithi121 ceph-mon[34815]: pgmap v526: 97 pgs: 97 active+clean; 169 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 211 op/s 2024-09-15T22:51:48.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:47 smithi111 ceph-mon[30629]: pgmap v526: 97 pgs: 97 active+clean; 169 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 211 op/s 2024-09-15T22:51:48.261 INFO:tasks.workunit.client.0.smithi111.stdout: 1 482110 9.78 MB/sec execute 136 sec latency 11.418 ms 2024-09-15T22:51:49.261 INFO:tasks.workunit.client.0.smithi111.stdout: 1 482250 9.71 MB/sec execute 137 sec latency 734.148 ms 2024-09-15T22:51:50.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:49 smithi192 ceph-mon[35427]: pgmap v527: 97 pgs: 97 active+clean; 168 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 2.5 MiB/s wr, 177 op/s 2024-09-15T22:51:50.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:49 smithi121 ceph-mon[34815]: pgmap v527: 97 pgs: 97 active+clean; 168 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 2.5 MiB/s wr, 177 op/s 2024-09-15T22:51:50.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:49 smithi111 ceph-mon[30629]: pgmap v527: 97 pgs: 97 active+clean; 168 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 2.5 MiB/s wr, 177 op/s 2024-09-15T22:51:50.261 INFO:tasks.workunit.client.0.smithi111.stdout: 1 484671 9.76 MB/sec execute 138 sec latency 741.217 ms 2024-09-15T22:51:51.261 INFO:tasks.workunit.client.0.smithi111.stdout: 1 487493 9.79 MB/sec execute 139 sec latency 10.777 ms 2024-09-15T22:51:52.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:51 smithi192 ceph-mon[35427]: pgmap v528: 97 pgs: 97 active+clean; 169 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 2.4 MiB/s wr, 190 op/s 2024-09-15T22:51:52.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:51 smithi121 ceph-mon[34815]: pgmap v528: 97 pgs: 97 active+clean; 169 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 2.4 MiB/s wr, 190 op/s 2024-09-15T22:51:52.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:51 smithi111 ceph-mon[30629]: pgmap v528: 97 pgs: 97 active+clean; 169 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 2.4 MiB/s wr, 190 op/s 2024-09-15T22:51:52.261 INFO:tasks.workunit.client.0.smithi111.stdout: 1 490702 9.86 MB/sec execute 140 sec latency 10.147 ms 2024-09-15T22:51:53.261 INFO:tasks.workunit.client.0.smithi111.stdout: 1 493941 9.91 MB/sec execute 141 sec latency 10.956 ms 2024-09-15T22:51:53.721 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:53 smithi111 ceph-mon[30629]: pgmap v529: 97 pgs: 97 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 2.7 MiB/s wr, 205 op/s 2024-09-15T22:51:54.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:53 smithi192 ceph-mon[35427]: pgmap v529: 97 pgs: 97 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 2.7 MiB/s wr, 205 op/s 2024-09-15T22:51:54.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:53 smithi121 ceph-mon[34815]: pgmap v529: 97 pgs: 97 active+clean; 167 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 2.7 MiB/s wr, 205 op/s 2024-09-15T22:51:54.262 INFO:tasks.workunit.client.0.smithi111.stdout: 1 494218 9.85 MB/sec execute 142 sec latency 780.715 ms 2024-09-15T22:51:55.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:55 smithi111 ceph-mon[30629]: pgmap v530: 97 pgs: 97 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 2.4 MiB/s wr, 195 op/s 2024-09-15T22:51:55.263 INFO:tasks.workunit.client.0.smithi111.stdout: 1 494395 9.79 MB/sec execute 143 sec latency 1599.944 ms 2024-09-15T22:51:55.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:55 smithi121 ceph-mon[34815]: pgmap v530: 97 pgs: 97 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 2.4 MiB/s wr, 195 op/s 2024-09-15T22:51:55.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:55 smithi192 ceph-mon[35427]: pgmap v530: 97 pgs: 97 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 2.4 MiB/s wr, 195 op/s 2024-09-15T22:51:56.262 INFO:tasks.workunit.client.0.smithi111.stdout: 1 497658 9.85 MB/sec execute 144 sec latency 19.323 ms 2024-09-15T22:51:57.262 INFO:tasks.workunit.client.0.smithi111.stdout: 1 500380 9.87 MB/sec execute 145 sec latency 18.204 ms 2024-09-15T22:51:57.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.652+0000 7f580b200640 1 -- 172.21.15.111:0/3361554138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f580c103b70 msgr2=0x7f580c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.652+0000 7f580b200640 1 --2- 172.21.15.111:0/3361554138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f580c103b70 0x7f580c103f70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f57fc009900 tx=0x7f57fc02a3f0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.652+0000 7f580b200640 1 -- 172.21.15.111:0/3361554138 shutdown_connections 2024-09-15T22:51:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.652+0000 7f580b200640 1 --2- 172.21.15.111:0/3361554138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f580c105720 0x7f580c10bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.652+0000 7f580b200640 1 --2- 172.21.15.111:0/3361554138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f580c104d60 0x7f580c1051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.652+0000 7f580b200640 1 --2- 172.21.15.111:0/3361554138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f580c103b70 0x7f580c103f70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.652+0000 7f580b200640 1 -- 172.21.15.111:0/3361554138 >> 172.21.15.111:0/3361554138 conn(0x7f580c0ff310 msgr2=0x7f580c101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.652+0000 7f580b200640 1 -- 172.21.15.111:0/3361554138 shutdown_connections 2024-09-15T22:51:57.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.652+0000 7f580b200640 1 -- 172.21.15.111:0/3361554138 wait complete. 2024-09-15T22:51:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.653+0000 7f580b200640 1 Processor -- start 2024-09-15T22:51:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.653+0000 7f580b200640 1 -- start start 2024-09-15T22:51:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.653+0000 7f580b200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f580c103b70 0x7f580c199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.653+0000 7f580b200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f580c104d60 0x7f580c19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.653+0000 7f580b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f580c105720 0x7f580c1a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.653+0000 7f580b200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f580c10f820 con 0x7f580c104d60 2024-09-15T22:51:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.653+0000 7f580b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f580c10f6a0 con 0x7f580c105720 2024-09-15T22:51:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.653+0000 7f580b200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f580c10f9a0 con 0x7f580c103b70 2024-09-15T22:51:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.653+0000 7f5809e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f580c103b70 0x7f580c199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.653+0000 7f5801400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f580c104d60 0x7f580c19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.653+0000 7f5809e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f580c103b70 0x7f580c199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:53662/0 (socket says 172.21.15.111:53662) 2024-09-15T22:51:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.653+0000 7f5809e00640 1 -- 172.21.15.111:0/2642048988 learned_addr learned my addr 172.21.15.111:0/2642048988 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:51:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.653+0000 7f580a800640 1 --2- 172.21.15.111:0/2642048988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f580c105720 0x7f580c1a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.654+0000 7f5809e00640 1 -- 172.21.15.111:0/2642048988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f580c105720 msgr2=0x7f580c1a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.654+0000 7f5809e00640 1 --2- 172.21.15.111:0/2642048988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f580c105720 0x7f580c1a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.654+0000 7f5809e00640 1 -- 172.21.15.111:0/2642048988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f580c104d60 msgr2=0x7f580c19a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.654+0000 7f5809e00640 1 --2- 172.21.15.111:0/2642048988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f580c104d60 0x7f580c19a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.654+0000 7f5809e00640 1 -- 172.21.15.111:0/2642048988 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f580c1a1ac0 con 0x7f580c103b70 2024-09-15T22:51:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.654+0000 7f580a800640 1 --2- 172.21.15.111:0/2642048988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f580c105720 0x7f580c1a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:51:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.654+0000 7f5801400640 1 --2- 172.21.15.111:0/2642048988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f580c104d60 0x7f580c19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:51:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.654+0000 7f5809e00640 1 --2- 172.21.15.111:0/2642048988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f580c103b70 0x7f580c199ea0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f57fc031440 tx=0x7f57fc032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.654+0000 7f5803400640 1 -- 172.21.15.111:0/2642048988 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc03f030 con 0x7f580c103b70 2024-09-15T22:51:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.654+0000 7f5803400640 1 -- 172.21.15.111:0/2642048988 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57fc003ce0 con 0x7f580c103b70 2024-09-15T22:51:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.654+0000 7f580b200640 1 -- 172.21.15.111:0/2642048988 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f580c1a1d50 con 0x7f580c103b70 2024-09-15T22:51:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.654+0000 7f5803400640 1 -- 172.21.15.111:0/2642048988 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc035070 con 0x7f580c103b70 2024-09-15T22:51:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.654+0000 7f580b200640 1 -- 172.21.15.111:0/2642048988 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f580c1a2190 con 0x7f580c103b70 2024-09-15T22:51:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.657+0000 7f5813886740 1 Processor -- start 2024-09-15T22:51:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.657+0000 7f5813886740 1 -- start start 2024-09-15T22:51:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.658+0000 7f5803400640 1 -- 172.21.15.111:0/2642048988 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f57fc042030 con 0x7f580c103b70 2024-09-15T22:51:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.658+0000 7f5813886740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba97c30260 0x55ba97c30660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.658+0000 7f5813886740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba97c30ba0 0x55ba97c35660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.658+0000 7f5813886740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba97c35c30 0x55ba97c380f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.658+0000 7f5813886740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ba97b95100 con 0x55ba97c35c30 2024-09-15T22:51:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.658+0000 7f5813886740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ba97b3d1b0 con 0x55ba97c30ba0 2024-09-15T22:51:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.658+0000 7f5813886740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ba97b910c0 con 0x55ba97c30260 2024-09-15T22:51:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.658+0000 7f5809e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba97c30ba0 0x55ba97c35660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.658+0000 7f580a800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba97c30260 0x55ba97c30660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.658+0000 7f5801400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba97c35c30 0x55ba97c380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.658+0000 7f580a800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba97c30260 0x55ba97c30660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:53664/0 (socket says 172.21.15.111:53664) 2024-09-15T22:51:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.658+0000 7f580a800640 1 -- 172.21.15.111:0/3048555389 learned_addr learned my addr 172.21.15.111:0/3048555389 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:51:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.658+0000 7f580a800640 1 -- 172.21.15.111:0/3048555389 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba97c30ba0 msgr2=0x55ba97c35660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:57.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.658+0000 7f580a800640 1 --2- 172.21.15.111:0/3048555389 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba97c30ba0 0x55ba97c35660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.658+0000 7f580a800640 1 -- 172.21.15.111:0/3048555389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba97c35c30 msgr2=0x55ba97c380f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.658+0000 7f580a800640 1 --2- 172.21.15.111:0/3048555389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba97c35c30 0x55ba97c380f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.661 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.658+0000 7f580a800640 1 -- 172.21.15.111:0/3048555389 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba978a8210 con 0x55ba97c30260 2024-09-15T22:51:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.659+0000 7f580a800640 1 --2- 172.21.15.111:0/3048555389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba97c30260 0x55ba97c30660 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f58040117e0 tx=0x7f5804011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d6777fd724fc797 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.659+0000 7f57fb000640 1 -- 172.21.15.111:0/3048555389 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804011310 con 0x55ba97c30260 2024-09-15T22:51:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.659+0000 7f57fb000640 1 -- 172.21.15.111:0/3048555389 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5804003e50 con 0x55ba97c30260 2024-09-15T22:51:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.659+0000 7f57fb000640 1 -- 172.21.15.111:0/3048555389 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58040045e0 con 0x55ba97c30260 2024-09-15T22:51:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.659+0000 7f5803400640 1 --2- 172.21.15.111:0/2642048988 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f57f00a3240 0x7f57f00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.659+0000 7f5803400640 1 -- 172.21.15.111:0/2642048988 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f57fc0f0140 con 0x7f580c103b70 2024-09-15T22:51:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.659+0000 7f5809e00640 1 --2- 172.21.15.111:0/2642048988 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f57f00a3240 0x7f57f00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.659+0000 7f5813886740 1 -- 172.21.15.111:0/3048555389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba97c30260 msgr2=0x55ba97c30660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.659+0000 7f5813886740 1 --2- 172.21.15.111:0/3048555389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba97c30260 0x55ba97c30660 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f58040117e0 tx=0x7f5804011ca0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.660+0000 7f5813886740 1 -- 172.21.15.111:0/3048555389 shutdown_connections 2024-09-15T22:51:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.660+0000 7f5813886740 1 --2- 172.21.15.111:0/3048555389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba97c35c30 0x55ba97c380f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.660+0000 7f5813886740 1 --2- 172.21.15.111:0/3048555389 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba97c30ba0 0x55ba97c35660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.660+0000 7f5813886740 1 --2- 172.21.15.111:0/3048555389 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba97c30260 0x55ba97c30660 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.660+0000 7f5813886740 1 -- 172.21.15.111:0/3048555389 >> 172.21.15.111:0/3048555389 conn(0x55ba97c2b8d0 msgr2=0x55ba97c2bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.660+0000 7f5813886740 1 -- 172.21.15.111:0/3048555389 shutdown_connections 2024-09-15T22:51:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.660+0000 7f5809e00640 1 --2- 172.21.15.111:0/2642048988 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f57f00a3240 0x7f57f00a5700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f57fc02a910 tx=0x7f57fc0f3b80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.660+0000 7f5813886740 1 -- 172.21.15.111:0/3048555389 wait complete. 2024-09-15T22:51:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.660+0000 7f5813886740 1 -- start start 2024-09-15T22:51:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.660+0000 7f5813886740 1 Processor -- start 2024-09-15T22:51:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.661+0000 7f5813886740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba97c30260 0x55ba97c3a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.661+0000 7f5813886740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba97c30ba0 0x55ba97c3a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.661+0000 7f5813886740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba97c35c30 0x55ba97c41980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:57.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.661+0000 7f5813886740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ba97c38c00 con 0x55ba97c30260 2024-09-15T22:51:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.661+0000 7f5813886740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ba97b1e090 con 0x55ba97c30ba0 2024-09-15T22:51:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.661+0000 7f5813886740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ba97c38d80 con 0x55ba97c35c30 2024-09-15T22:51:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.661+0000 7f5813886740 10 client.0 _ll_register_callbacks cb 0x55ba97b72f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:51:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.661+0000 7f580a800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba97c30260 0x55ba97c3a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.661+0000 7f5801400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba97c30ba0 0x55ba97c3a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.661+0000 7f580a800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba97c30260 0x55ba97c3a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46744/0 (socket says 172.21.15.111:46744) 2024-09-15T22:51:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.661+0000 7f580a800640 1 -- 172.21.15.111:0/1071395632 learned_addr learned my addr 172.21.15.111:0/1071395632 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:51:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.661+0000 7f580a800640 1 --2- 172.21.15.111:0/1071395632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba97c35c30 0x55ba97c41980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.661+0000 7f5801400640 1 -- 172.21.15.111:0/1071395632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba97c35c30 msgr2=0x55ba97c41980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.661+0000 7f5801400640 1 --2- 172.21.15.111:0/1071395632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba97c35c30 0x55ba97c41980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.661+0000 7f5801400640 1 -- 172.21.15.111:0/1071395632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba97c30260 msgr2=0x55ba97c3a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.661+0000 7f5801400640 1 --2- 172.21.15.111:0/1071395632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba97c30260 0x55ba97c3a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.661+0000 7f5801400640 1 -- 172.21.15.111:0/1071395632 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba97974620 con 0x55ba97c30ba0 2024-09-15T22:51:57.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.662+0000 7f5801400640 1 --2- 172.21.15.111:0/1071395632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba97c30ba0 0x55ba97c3a9b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f580c19b3b0 tx=0x7f57f400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6b20e5b4cd36f7b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.662+0000 7f57ef400640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:51:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.662+0000 7f57ef400640 1 -- 172.21.15.111:0/1071395632 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4012110 con 0x55ba97c30ba0 2024-09-15T22:51:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.662+0000 7f57ef400640 1 -- 172.21.15.111:0/1071395632 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f57f4004050 con 0x55ba97c30ba0 2024-09-15T22:51:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.662+0000 7f5813886740 1 -- 172.21.15.111:0/1071395632 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5804003e50 con 0x55ba97c30ba0 2024-09-15T22:51:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.662+0000 7f57ef400640 1 -- 172.21.15.111:0/1071395632 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f57f40122b0 con 0x55ba97c30ba0 2024-09-15T22:51:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.662+0000 7f5813886740 10 client.25475 fetch_fsmap learned FSMap version 30 2024-09-15T22:51:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.662+0000 7f5813886740 1 -- 172.21.15.111:0/1071395632 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ba97c44840 con 0x55ba97c30ba0 2024-09-15T22:51:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.662+0000 7f57ef400640 1 -- 172.21.15.111:0/1071395632 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5804003e50 con 0x55ba97c30ba0 2024-09-15T22:51:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.663+0000 7f57ef400640 1 -- 172.21.15.111:0/1071395632 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f57f40104c0 con 0x55ba97c30ba0 2024-09-15T22:51:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.663+0000 7f5813886740 10 client.25475 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:51:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.663+0000 7f5813886740 10 client.25475 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:51:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.663+0000 7f5813886740 20 client.25475 populate_metadata read hostname 'smithi111' 2024-09-15T22:51:57.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.663+0000 7f5813886740 1 --2- 172.21.15.111:0/1071395632 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ba97c45ea0 0x55ba97c482b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.663+0000 7f5813886740 4 client.25475 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:51:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.663+0000 7f5813886740 1 -- 172.21.15.111:0/1071395632 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba97ba2830 con 0x55ba97c45ea0 2024-09-15T22:51:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.663+0000 7f580a800640 1 --2- 172.21.15.111:0/1071395632 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ba97c45ea0 0x55ba97c482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.663+0000 7f580a800640 1 --2- 172.21.15.111:0/1071395632 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ba97c45ea0 0x55ba97c482b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f2f6b5e07a82f86 server_cookie=5be06732e3ae5295 in_seq=0 out_seq=0 2024-09-15T22:51:57.669 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.663+0000 7f57ef400640 10 client.25475 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:51:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.664+0000 7f57ef400640 1 -- 172.21.15.111:0/1071395632 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ba97ba2830 con 0x55ba97c45ea0 2024-09-15T22:51:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.664+0000 7f57ef400640 10 client.25475 handle_command_reply: tid=0 2024-09-15T22:51:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.664+0000 7f5813886740 1 client.25475 shutdown 2024-09-15T22:51:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.664+0000 7f5813886740 1 -- 172.21.15.111:0/1071395632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba97c30ba0 msgr2=0x55ba97c3a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.664+0000 7f5813886740 1 --2- 172.21.15.111:0/1071395632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba97c30ba0 0x55ba97c3a9b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f580c19b3b0 tx=0x7f57f400f790 comp rx=0 tx=0).stop 2024-09-15T22:51:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.664+0000 7f5813886740 1 -- 172.21.15.111:0/1071395632 shutdown_connections 2024-09-15T22:51:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.664+0000 7f5813886740 1 --2- 172.21.15.111:0/1071395632 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ba97c45ea0 0x55ba97c482b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.664+0000 7f5813886740 1 --2- 172.21.15.111:0/1071395632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba97c35c30 0x55ba97c41980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.670 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.664+0000 7f5813886740 1 --2- 172.21.15.111:0/1071395632 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba97c30ba0 0x55ba97c3a9b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.664+0000 7f5813886740 1 --2- 172.21.15.111:0/1071395632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba97c30260 0x55ba97c3a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.664+0000 7f57ef400640 0 client.25475 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:51:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.664+0000 7f5813886740 1 -- 172.21.15.111:0/1071395632 >> 172.21.15.111:0/1071395632 conn(0x55ba97c2b8d0 msgr2=0x55ba97c2d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.665+0000 7f5813886740 1 -- 172.21.15.111:0/1071395632 shutdown_connections 2024-09-15T22:51:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.665+0000 7f5813886740 1 -- 172.21.15.111:0/1071395632 wait complete. 2024-09-15T22:51:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.665+0000 7f5813886740 20 client.25475 trim_cache size 0 max 16384 2024-09-15T22:51:57.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.681+0000 7f5813886740 1 Processor -- start 2024-09-15T22:51:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.681+0000 7f5813886740 1 -- start start 2024-09-15T22:51:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.681+0000 7f5813886740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba97c2b8d0 0x55ba97c39150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.681+0000 7f5813886740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba97c30260 0x55ba97c41940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.681+0000 7f5813886740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba97c30ba0 0x55ba97c2a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.681+0000 7f5813886740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1ad7d0 con 0x55ba97c2b8d0 2024-09-15T22:51:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.681+0000 7f5813886740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1ab0a0 con 0x55ba97c30260 2024-09-15T22:51:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.681+0000 7f5813886740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1a8ef0 con 0x55ba97c30ba0 2024-09-15T22:51:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.681+0000 7f5801400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba97c30260 0x55ba97c41940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.681+0000 7f580a800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba97c2b8d0 0x55ba97c39150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.681+0000 7f5801400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba97c30260 0x55ba97c41940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:48008/0 (socket says 172.21.15.111:48008) 2024-09-15T22:51:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.681+0000 7f5801400640 1 -- 172.21.15.111:0/2013662434 learned_addr learned my addr 172.21.15.111:0/2013662434 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:51:57.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.681+0000 7f580a800640 1 --2- 172.21.15.111:0/2013662434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba97c30ba0 0x55ba97c2a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.682+0000 7f5801400640 1 -- 172.21.15.111:0/2013662434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba97c30ba0 msgr2=0x55ba97c2a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.682+0000 7f5801400640 1 --2- 172.21.15.111:0/2013662434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba97c30ba0 0x55ba97c2a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.682+0000 7f5801400640 1 -- 172.21.15.111:0/2013662434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba97c2b8d0 msgr2=0x55ba97c39150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.682+0000 7f5801400640 1 --2- 172.21.15.111:0/2013662434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba97c2b8d0 0x55ba97c39150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.682+0000 7f5801400640 1 -- 172.21.15.111:0/2013662434 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba97c2b6f0 con 0x55ba97c30260 2024-09-15T22:51:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.682+0000 7f5801400640 1 --2- 172.21.15.111:0/2013662434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba97c30260 0x55ba97c41940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f57f40127f0 tx=0x7f57f4012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a12009fff86cd1d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.682+0000 7f57fa600640 1 -- 172.21.15.111:0/2013662434 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4010b50 con 0x55ba97c30260 2024-09-15T22:51:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.682+0000 7f57fa600640 1 -- 172.21.15.111:0/2013662434 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57f4010cf0 con 0x55ba97c30260 2024-09-15T22:51:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.682+0000 7f57fa600640 1 -- 172.21.15.111:0/2013662434 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4003920 con 0x55ba97c30260 2024-09-15T22:51:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.683+0000 7f5813886740 1 -- 172.21.15.111:0/2013662434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba97c30260 msgr2=0x55ba97c41940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.683+0000 7f5813886740 1 --2- 172.21.15.111:0/2013662434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba97c30260 0x55ba97c41940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f57f40127f0 tx=0x7f57f4012820 comp rx=0 tx=0).stop 2024-09-15T22:51:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.683+0000 7f5813886740 1 -- 172.21.15.111:0/2013662434 shutdown_connections 2024-09-15T22:51:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.683+0000 7f5813886740 1 --2- 172.21.15.111:0/2013662434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba97c30ba0 0x55ba97c2a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.683+0000 7f5813886740 1 --2- 172.21.15.111:0/2013662434 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba97c30260 0x55ba97c41940 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.683+0000 7f5813886740 1 --2- 172.21.15.111:0/2013662434 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba97c2b8d0 0x55ba97c39150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.683+0000 7f5813886740 1 -- 172.21.15.111:0/2013662434 >> 172.21.15.111:0/2013662434 conn(0x55ba97c357c0 msgr2=0x55ba97c2a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.683+0000 7f5813886740 1 -- 172.21.15.111:0/2013662434 shutdown_connections 2024-09-15T22:51:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.683+0000 7f5813886740 1 -- 172.21.15.111:0/2013662434 wait complete. 2024-09-15T22:51:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.683+0000 7f5813886740 1 -- start start 2024-09-15T22:51:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.683+0000 7f5813886740 1 Processor -- start 2024-09-15T22:51:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.684+0000 7f5813886740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba97c2b8d0 0x55ba97c3ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.684+0000 7f5813886740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba97c30260 0x55ba97c3f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.684+0000 7f5813886740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba97c30ba0 0x55ba97c403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.684+0000 7f5813886740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1a9560 con 0x55ba97c30260 2024-09-15T22:51:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.684+0000 7f5813886740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1aa9d0 con 0x55ba97c2b8d0 2024-09-15T22:51:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.684+0000 7f5813886740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f580c1a9c30 con 0x55ba97c30ba0 2024-09-15T22:51:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.684+0000 7f5813886740 10 client.0 _ll_register_callbacks cb 0x7f57f4009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:51:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.684+0000 7f580a800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba97c2b8d0 0x55ba97c3ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.684+0000 7f580a800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba97c2b8d0 0x55ba97c3ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:48022/0 (socket says 172.21.15.111:48022) 2024-09-15T22:51:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.684+0000 7f580a800640 1 -- 172.21.15.111:0/4246241025 learned_addr learned my addr 172.21.15.111:0/4246241025 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:51:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.684+0000 7f580a800640 1 --2- 172.21.15.111:0/4246241025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba97c30ba0 0x55ba97c403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.684+0000 7f5801400640 1 --2- 172.21.15.111:0/4246241025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba97c30260 0x55ba97c3f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.684+0000 7f580a800640 1 -- 172.21.15.111:0/4246241025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba97c30ba0 msgr2=0x55ba97c403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.684+0000 7f580a800640 1 --2- 172.21.15.111:0/4246241025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba97c30ba0 0x55ba97c403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.684+0000 7f580a800640 1 -- 172.21.15.111:0/4246241025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba97c30260 msgr2=0x55ba97c3f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:51:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.684+0000 7f580a800640 1 --2- 172.21.15.111:0/4246241025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba97c30260 0x55ba97c3f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.684+0000 7f580a800640 1 -- 172.21.15.111:0/4246241025 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba97c409d0 con 0x55ba97c2b8d0 2024-09-15T22:51:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.684+0000 7f5801400640 1 --2- 172.21.15.111:0/4246241025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba97c30260 0x55ba97c3f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:51:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.685+0000 7f580a800640 1 --2- 172.21.15.111:0/4246241025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba97c2b8d0 0x55ba97c3ee70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5804011ce0 tx=0x7f5804009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee5816d2f928589c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:51:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.685+0000 7f57ef400640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:51:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.685+0000 7f57ef400640 1 -- 172.21.15.111:0/4246241025 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804023040 con 0x55ba97c2b8d0 2024-09-15T22:51:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.685+0000 7f5813886740 1 -- 172.21.15.111:0/4246241025 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57f4010cf0 con 0x55ba97c2b8d0 2024-09-15T22:51:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.685+0000 7f57ef400640 1 -- 172.21.15.111:0/4246241025 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f580400fbc0 con 0x55ba97c2b8d0 2024-09-15T22:51:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.685+0000 7f57ef400640 1 -- 172.21.15.111:0/4246241025 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f580400fdc0 con 0x55ba97c2b8d0 2024-09-15T22:51:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.685+0000 7f5813886740 10 client.25481 fetch_fsmap learned FSMap version 30 2024-09-15T22:51:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.685+0000 7f5813886740 1 -- 172.21.15.111:0/4246241025 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ba97c41d40 con 0x55ba97c2b8d0 2024-09-15T22:51:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.685+0000 7f57ef400640 1 -- 172.21.15.111:0/4246241025 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57f4010cf0 con 0x55ba97c2b8d0 2024-09-15T22:51:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.685+0000 7f57ef400640 1 -- 172.21.15.111:0/4246241025 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f5804018020 con 0x55ba97c2b8d0 2024-09-15T22:51:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.686+0000 7f5813886740 10 client.25481 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:51:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.686+0000 7f5813886740 10 client.25481 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:51:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.686+0000 7f5813886740 20 client.25481 populate_metadata read hostname 'smithi111' 2024-09-15T22:51:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.686+0000 7f5813886740 1 --2- 172.21.15.111:0/4246241025 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ba97c35c30 0x55ba97cb3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:51:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.686+0000 7f5813886740 4 client.25481 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:51:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.686+0000 7f5813886740 1 -- 172.21.15.111:0/4246241025 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ba97b9cb80 con 0x55ba97c35c30 2024-09-15T22:51:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.686+0000 7f5801400640 1 --2- 172.21.15.111:0/4246241025 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ba97c35c30 0x55ba97cb3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:51:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.686+0000 7f5801400640 1 --2- 172.21.15.111:0/4246241025 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ba97c35c30 0x55ba97cb3170 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90a68915084bbefd server_cookie=c4f1fcd44256ebc9 in_seq=0 out_seq=0 2024-09-15T22:51:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.686+0000 7f57ef400640 10 client.25481 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:51:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.694+0000 7f57ef400640 1 -- 172.21.15.111:0/4246241025 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ba97b9cb80 con 0x55ba97c35c30 2024-09-15T22:51:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.694+0000 7f57ef400640 10 client.25481 handle_command_reply: tid=0 2024-09-15T22:51:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.694+0000 7f5813886740 1 client.25481 shutdown 2024-09-15T22:51:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.694+0000 7f5813886740 1 -- 172.21.15.111:0/4246241025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba97c2b8d0 msgr2=0x55ba97c3ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.694+0000 7f5813886740 1 --2- 172.21.15.111:0/4246241025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba97c2b8d0 0x55ba97c3ee70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5804011ce0 tx=0x7f5804009500 comp rx=0 tx=0).stop 2024-09-15T22:51:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.695+0000 7f5813886740 1 -- 172.21.15.111:0/4246241025 shutdown_connections 2024-09-15T22:51:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.695+0000 7f5813886740 1 --2- 172.21.15.111:0/4246241025 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ba97c35c30 0x55ba97cb3170 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.695+0000 7f5813886740 1 --2- 172.21.15.111:0/4246241025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba97c30ba0 0x55ba97c403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.695+0000 7f5813886740 1 --2- 172.21.15.111:0/4246241025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba97c30260 0x55ba97c3f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.695+0000 7f5813886740 1 --2- 172.21.15.111:0/4246241025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba97c2b8d0 0x55ba97c3ee70 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.695+0000 7f57ef400640 0 client.25481 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:51:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.695+0000 7f5813886740 1 -- 172.21.15.111:0/4246241025 >> 172.21.15.111:0/4246241025 conn(0x55ba97c357c0 msgr2=0x55ba97c2aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.695+0000 7f5813886740 1 -- 172.21.15.111:0/4246241025 shutdown_connections 2024-09-15T22:51:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.695+0000 7f5813886740 1 -- 172.21.15.111:0/4246241025 wait complete. 2024-09-15T22:51:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.695+0000 7f5813886740 20 client.25481 trim_cache size 0 max 16384 2024-09-15T22:51:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:51:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:51:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:51:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:51:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.695+0000 7f580b200640 1 -- 172.21.15.111:0/2642048988 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f57f00a3240 msgr2=0x7f57f00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.695+0000 7f580b200640 1 --2- 172.21.15.111:0/2642048988 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f57f00a3240 0x7f57f00a5700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f57fc02a910 tx=0x7f57fc0f3b80 comp rx=0 tx=0).stop 2024-09-15T22:51:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.696+0000 7f580b200640 1 -- 172.21.15.111:0/2642048988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f580c103b70 msgr2=0x7f580c199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:51:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.696+0000 7f580b200640 1 --2- 172.21.15.111:0/2642048988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f580c103b70 0x7f580c199ea0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f57fc031440 tx=0x7f57fc032ce0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.696+0000 7f580b200640 1 -- 172.21.15.111:0/2642048988 shutdown_connections 2024-09-15T22:51:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.696+0000 7f580b200640 1 --2- 172.21.15.111:0/2642048988 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f57f00a3240 0x7f57f00a5700 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.696+0000 7f580b200640 1 --2- 172.21.15.111:0/2642048988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f580c105720 0x7f580c1a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.696+0000 7f580b200640 1 --2- 172.21.15.111:0/2642048988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f580c104d60 0x7f580c19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.696+0000 7f580b200640 1 --2- 172.21.15.111:0/2642048988 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f580c103b70 0x7f580c199ea0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:51:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.696+0000 7f580b200640 1 -- 172.21.15.111:0/2642048988 >> 172.21.15.111:0/2642048988 conn(0x7f580c0ff310 msgr2=0x7f580c1081a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:51:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.696+0000 7f580b200640 1 -- 172.21.15.111:0/2642048988 shutdown_connections 2024-09-15T22:51:57.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:51:57.696+0000 7f580b200640 1 -- 172.21.15.111:0/2642048988 wait complete. 2024-09-15T22:51:57.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:57 smithi121 ceph-mon[34815]: pgmap v531: 97 pgs: 97 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 2.2 MiB/s wr, 183 op/s 2024-09-15T22:51:57.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:57 smithi111 ceph-mon[30629]: pgmap v531: 97 pgs: 97 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 2.2 MiB/s wr, 183 op/s 2024-09-15T22:51:58.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:57 smithi192 ceph-mon[35427]: pgmap v531: 97 pgs: 97 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 2.2 MiB/s wr, 183 op/s 2024-09-15T22:51:58.262 INFO:tasks.workunit.client.0.smithi111.stdout: 1 502831 9.88 MB/sec execute 146 sec latency 18.661 ms 2024-09-15T22:51:59.262 INFO:tasks.workunit.client.0.smithi111.stdout: 1 503941 9.87 MB/sec execute 147 sec latency 443.759 ms 2024-09-15T22:51:59.706 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:51:59 smithi111 ceph-mon[30629]: pgmap v532: 97 pgs: 97 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 2.2 MiB/s wr, 189 op/s 2024-09-15T22:52:00.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:51:59 smithi192 ceph-mon[35427]: pgmap v532: 97 pgs: 97 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 2.2 MiB/s wr, 189 op/s 2024-09-15T22:52:00.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:51:59 smithi121 ceph-mon[34815]: pgmap v532: 97 pgs: 97 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 2.2 MiB/s wr, 189 op/s 2024-09-15T22:52:00.262 INFO:tasks.workunit.client.0.smithi111.stdout: 1 505048 9.84 MB/sec execute 148 sec latency 924.088 ms 2024-09-15T22:52:01.262 INFO:tasks.workunit.client.0.smithi111.stdout: 1 508191 9.90 MB/sec execute 149 sec latency 23.560 ms 2024-09-15T22:52:01.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:01 smithi111 ceph-mon[30629]: pgmap v533: 97 pgs: 97 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 2.3 MiB/s wr, 215 op/s 2024-09-15T22:52:02.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:01 smithi192 ceph-mon[35427]: pgmap v533: 97 pgs: 97 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 2.3 MiB/s wr, 215 op/s 2024-09-15T22:52:02.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:01 smithi121 ceph-mon[34815]: pgmap v533: 97 pgs: 97 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 2.3 MiB/s wr, 215 op/s 2024-09-15T22:52:02.262 INFO:tasks.workunit.client.0.smithi111.stdout: 1 510793 9.91 MB/sec execute 150 sec latency 16.167 ms 2024-09-15T22:52:03.262 INFO:tasks.workunit.client.0.smithi111.stdout: 1 513391 9.94 MB/sec execute 151 sec latency 11.923 ms 2024-09-15T22:52:03.903 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:03 smithi111 ceph-mon[30629]: pgmap v534: 97 pgs: 97 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 2.6 MiB/s wr, 196 op/s 2024-09-15T22:52:04.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:03 smithi192 ceph-mon[35427]: pgmap v534: 97 pgs: 97 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 2.6 MiB/s wr, 196 op/s 2024-09-15T22:52:04.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:03 smithi121 ceph-mon[34815]: pgmap v534: 97 pgs: 97 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 2.6 MiB/s wr, 196 op/s 2024-09-15T22:52:04.262 INFO:tasks.workunit.client.0.smithi111.stdout: 1 515198 9.95 MB/sec execute 152 sec latency 432.178 ms 2024-09-15T22:52:05.262 INFO:tasks.workunit.client.0.smithi111.stdout: 1 515198 9.89 MB/sec execute 153 sec latency 1432.237 ms 2024-09-15T22:52:05.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:05 smithi111 ceph-mon[30629]: pgmap v535: 97 pgs: 97 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 2.5 MiB/s wr, 220 op/s 2024-09-15T22:52:05.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:05 smithi121 ceph-mon[34815]: pgmap v535: 97 pgs: 97 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 2.5 MiB/s wr, 220 op/s 2024-09-15T22:52:06.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:05 smithi192 ceph-mon[35427]: pgmap v535: 97 pgs: 97 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 2.5 MiB/s wr, 220 op/s 2024-09-15T22:52:06.262 INFO:tasks.workunit.client.0.smithi111.stdout: 1 517725 9.88 MB/sec execute 154 sec latency 1481.113 ms 2024-09-15T22:52:07.262 INFO:tasks.workunit.client.0.smithi111.stdout: 1 520368 9.93 MB/sec execute 155 sec latency 29.775 ms 2024-09-15T22:52:07.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:07 smithi111 ceph-mon[30629]: pgmap v536: 97 pgs: 97 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 2.4 MiB/s wr, 184 op/s 2024-09-15T22:52:08.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:07 smithi192 ceph-mon[35427]: pgmap v536: 97 pgs: 97 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 2.4 MiB/s wr, 184 op/s 2024-09-15T22:52:08.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:07 smithi121 ceph-mon[34815]: pgmap v536: 97 pgs: 97 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 2.4 MiB/s wr, 184 op/s 2024-09-15T22:52:08.262 INFO:tasks.workunit.client.0.smithi111.stdout: 1 523140 9.96 MB/sec execute 156 sec latency 19.266 ms 2024-09-15T22:52:09.262 INFO:tasks.workunit.client.0.smithi111.stdout: 1 525219 9.97 MB/sec execute 157 sec latency 258.120 ms 2024-09-15T22:52:09.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:09 smithi111 ceph-mon[30629]: pgmap v537: 97 pgs: 97 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 2.6 MiB/s wr, 190 op/s 2024-09-15T22:52:10.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:09 smithi192 ceph-mon[35427]: pgmap v537: 97 pgs: 97 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 2.6 MiB/s wr, 190 op/s 2024-09-15T22:52:10.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:09 smithi121 ceph-mon[34815]: pgmap v537: 97 pgs: 97 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 2.6 MiB/s wr, 190 op/s 2024-09-15T22:52:10.262 INFO:tasks.workunit.client.0.smithi111.stdout: 1 525603 9.92 MB/sec execute 158 sec latency 1055.335 ms 2024-09-15T22:52:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.815+0000 7f7ed1600640 1 -- 172.21.15.111:0/1633189117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ecc100b50 msgr2=0x7f7ecc100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.815+0000 7f7ed1600640 1 --2- 172.21.15.111:0/1633189117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ecc100b50 0x7f7ecc100fb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009900 tx=0x7f7eb802a3f0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.815+0000 7f7ed1600640 1 -- 172.21.15.111:0/1633189117 shutdown_connections 2024-09-15T22:52:10.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.815+0000 7f7ed1600640 1 --2- 172.21.15.111:0/1633189117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ecc1014f0 0x7f7ecc1019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.815+0000 7f7ed1600640 1 --2- 172.21.15.111:0/1633189117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ecc100b50 0x7f7ecc100fb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.815+0000 7f7ed1600640 1 --2- 172.21.15.111:0/1633189117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ecc100210 0x7f7ecc100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.815+0000 7f7ed1600640 1 -- 172.21.15.111:0/1633189117 >> 172.21.15.111:0/1633189117 conn(0x7f7ecc0fbfa0 msgr2=0x7f7ecc0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:52:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.815+0000 7f7ed1600640 1 -- 172.21.15.111:0/1633189117 shutdown_connections 2024-09-15T22:52:10.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.815+0000 7f7ed1600640 1 -- 172.21.15.111:0/1633189117 wait complete. 2024-09-15T22:52:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.816+0000 7f7ed1600640 1 Processor -- start 2024-09-15T22:52:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.816+0000 7f7ed1600640 1 -- start start 2024-09-15T22:52:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.817+0000 7f7ed1600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ecc100210 0x7f7ecc199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.817+0000 7f7ed1600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ecc100b50 0x7f7ecc19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:10.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.817+0000 7f7ed1600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ecc1014f0 0x7f7ecc1a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:10.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.817+0000 7f7ed1600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc10f7a0 con 0x7f7ecc100b50 2024-09-15T22:52:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.817+0000 7f7ed1600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc10f620 con 0x7f7ecc100210 2024-09-15T22:52:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.817+0000 7f7ed1600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc10f920 con 0x7f7ecc1014f0 2024-09-15T22:52:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.817+0000 7f7ecbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ecc100210 0x7f7ecc199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.817+0000 7f7ecbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ecc100210 0x7f7ecc199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:46488/0 (socket says 172.21.15.111:46488) 2024-09-15T22:52:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.817+0000 7f7ecbe00640 1 -- 172.21.15.111:0/3054956313 learned_addr learned my addr 172.21.15.111:0/3054956313 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:52:10.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.817+0000 7f7ecb400640 1 --2- 172.21.15.111:0/3054956313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ecc100b50 0x7f7ecc19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.817+0000 7f7ed0c00640 1 --2- 172.21.15.111:0/3054956313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ecc1014f0 0x7f7ecc1a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.817+0000 7f7ecbe00640 1 -- 172.21.15.111:0/3054956313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ecc1014f0 msgr2=0x7f7ecc1a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.817+0000 7f7ecbe00640 1 --2- 172.21.15.111:0/3054956313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ecc1014f0 0x7f7ecc1a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.817+0000 7f7ecbe00640 1 -- 172.21.15.111:0/3054956313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ecc100b50 msgr2=0x7f7ecc19a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.817+0000 7f7ecbe00640 1 --2- 172.21.15.111:0/3054956313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ecc100b50 0x7f7ecc19a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.817+0000 7f7ecbe00640 1 -- 172.21.15.111:0/3054956313 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ecc1a1a50 con 0x7f7ecc100210 2024-09-15T22:52:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.817+0000 7f7ecb400640 1 --2- 172.21.15.111:0/3054956313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ecc100b50 0x7f7ecc19a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:52:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.818+0000 7f7ecbe00640 1 --2- 172.21.15.111:0/3054956313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ecc100210 0x7f7ecc199e30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00e9f0 tx=0x7f7ebc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:52:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.818+0000 7f7ec8c00640 1 -- 172.21.15.111:0/3054956313 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc00cd80 con 0x7f7ecc100210 2024-09-15T22:52:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.818+0000 7f7ec8c00640 1 -- 172.21.15.111:0/3054956313 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ebc010910 con 0x7f7ecc100210 2024-09-15T22:52:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.818+0000 7f7ec8c00640 1 -- 172.21.15.111:0/3054956313 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc018990 con 0x7f7ecc100210 2024-09-15T22:52:10.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.818+0000 7f7ed1600640 1 -- 172.21.15.111:0/3054956313 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ecc1a1d40 con 0x7f7ecc100210 2024-09-15T22:52:10.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.818+0000 7f7ed1600640 1 -- 172.21.15.111:0/3054956313 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ecc1a21f0 con 0x7f7ecc100210 2024-09-15T22:52:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.821+0000 7f7ed5a8e740 1 Processor -- start 2024-09-15T22:52:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.821+0000 7f7ed5a8e740 1 -- start start 2024-09-15T22:52:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.822+0000 7f7ed5a8e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28eb71260 0x55a28eb71660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:10.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.822+0000 7f7ed5a8e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28eb71ba0 0x55a28eb76660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.822+0000 7f7ed5a8e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28eb76c30 0x55a28eb790f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.822+0000 7f7ed5a8e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a28ead6100 con 0x55a28eb71260 2024-09-15T22:52:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.822+0000 7f7ed5a8e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a28ea7e1b0 con 0x55a28eb76c30 2024-09-15T22:52:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.822+0000 7f7ed5a8e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a28ead20c0 con 0x55a28eb71ba0 2024-09-15T22:52:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.822+0000 7f7ed0c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28eb71260 0x55a28eb71660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.822+0000 7f7ed0c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28eb71260 0x55a28eb71660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46972/0 (socket says 172.21.15.111:46972) 2024-09-15T22:52:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.822+0000 7f7ed0c00640 1 -- 172.21.15.111:0/2541752263 learned_addr learned my addr 172.21.15.111:0/2541752263 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:52:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.822+0000 7f7ec8c00640 1 -- 172.21.15.111:0/3054956313 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f7ebc010ac0 con 0x7f7ecc100210 2024-09-15T22:52:10.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.822+0000 7f7ecb400640 1 --2- 172.21.15.111:0/2541752263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28eb76c30 0x55a28eb790f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.822+0000 7f7ed0c00640 1 -- 172.21.15.111:0/2541752263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28eb71ba0 msgr2=0x55a28eb76660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.822+0000 7f7ecbe00640 1 --2- 172.21.15.111:0/2541752263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28eb71ba0 0x55a28eb76660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.822+0000 7f7ed0c00640 1 --2- 172.21.15.111:0/2541752263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28eb71ba0 0x55a28eb76660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.822+0000 7f7ed0c00640 1 -- 172.21.15.111:0/2541752263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28eb76c30 msgr2=0x55a28eb790f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.822+0000 7f7ed0c00640 1 --2- 172.21.15.111:0/2541752263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28eb76c30 0x55a28eb790f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.822+0000 7f7ed0c00640 1 -- 172.21.15.111:0/2541752263 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a28e7e9210 con 0x55a28eb71260 2024-09-15T22:52:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.822+0000 7f7ecbe00640 1 --2- 172.21.15.111:0/2541752263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28eb71ba0 0x55a28eb76660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:52:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.823+0000 7f7ed0c00640 1 --2- 172.21.15.111:0/2541752263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28eb71260 0x55a28eb71660 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4009fd0 tx=0x7f7ec40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3212154fbafcc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:52:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.823+0000 7f7ec1200640 1 -- 172.21.15.111:0/2541752263 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4004d20 con 0x55a28eb71260 2024-09-15T22:52:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.823+0000 7f7ec1200640 1 -- 172.21.15.111:0/2541752263 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ec4004050 con 0x55a28eb71260 2024-09-15T22:52:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.823+0000 7f7ec1200640 1 -- 172.21.15.111:0/2541752263 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4005260 con 0x55a28eb71260 2024-09-15T22:52:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.824+0000 7f7ec8c00640 1 --2- 172.21.15.111:0/3054956313 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7eb00a32c0 0x7f7eb00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.824+0000 7f7ed5a8e740 1 -- 172.21.15.111:0/2541752263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28eb71260 msgr2=0x55a28eb71660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.824+0000 7f7ed5a8e740 1 --2- 172.21.15.111:0/2541752263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28eb71260 0x55a28eb71660 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4009fd0 tx=0x7f7ec40118d0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.824+0000 7f7ed5a8e740 1 -- 172.21.15.111:0/2541752263 shutdown_connections 2024-09-15T22:52:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.824+0000 7f7ed5a8e740 1 --2- 172.21.15.111:0/2541752263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28eb76c30 0x55a28eb790f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.824+0000 7f7ed5a8e740 1 --2- 172.21.15.111:0/2541752263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28eb71ba0 0x55a28eb76660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.824+0000 7f7ed5a8e740 1 --2- 172.21.15.111:0/2541752263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28eb71260 0x55a28eb71660 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.824+0000 7f7ed5a8e740 1 -- 172.21.15.111:0/2541752263 >> 172.21.15.111:0/2541752263 conn(0x55a28eb6c8d0 msgr2=0x55a28eb6ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:52:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.824+0000 7f7ec8c00640 1 -- 172.21.15.111:0/3054956313 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f7ebc0cfb90 con 0x7f7ecc100210 2024-09-15T22:52:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.824+0000 7f7ecbe00640 1 --2- 172.21.15.111:0/3054956313 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7eb00a32c0 0x7f7eb00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.824+0000 7f7ed5a8e740 1 -- 172.21.15.111:0/2541752263 shutdown_connections 2024-09-15T22:52:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.824+0000 7f7ed5a8e740 1 -- 172.21.15.111:0/2541752263 wait complete. 2024-09-15T22:52:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.825+0000 7f7ecbe00640 1 --2- 172.21.15.111:0/3054956313 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7eb00a32c0 0x7f7eb00a5780 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc098c20 tx=0x7f7ebc000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:52:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.825+0000 7f7ed5a8e740 1 -- start start 2024-09-15T22:52:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.825+0000 7f7ed5a8e740 1 Processor -- start 2024-09-15T22:52:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.826+0000 7f7ed5a8e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28eb71260 0x55a28eb7b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.826+0000 7f7ed5a8e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28eb71ba0 0x55a28eb7b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.826+0000 7f7ed5a8e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28eb76c30 0x55a28eb828e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.826+0000 7f7ed5a8e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a28eb79c00 con 0x55a28eb71ba0 2024-09-15T22:52:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.826+0000 7f7ed5a8e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a28ea5f090 con 0x55a28eb71260 2024-09-15T22:52:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.826+0000 7f7ed5a8e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a28eb79d80 con 0x55a28eb76c30 2024-09-15T22:52:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.826+0000 7f7ed5a8e740 10 client.0 _ll_register_callbacks cb 0x55a28eab3f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:52:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.826+0000 7f7ed0c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28eb71260 0x55a28eb7b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.826+0000 7f7ecb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28eb71ba0 0x55a28eb7b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.826+0000 7f7ed0c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28eb71260 0x55a28eb7b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:46508/0 (socket says 172.21.15.111:46508) 2024-09-15T22:52:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.826+0000 7f7ed0c00640 1 -- 172.21.15.111:0/2324654235 learned_addr learned my addr 172.21.15.111:0/2324654235 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:52:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.826+0000 7f7ed0c00640 1 --2- 172.21.15.111:0/2324654235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28eb76c30 0x55a28eb828e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.826+0000 7f7ed0c00640 1 -- 172.21.15.111:0/2324654235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28eb76c30 msgr2=0x55a28eb828e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.826+0000 7f7ed0c00640 1 --2- 172.21.15.111:0/2324654235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28eb76c30 0x55a28eb828e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.826+0000 7f7ed0c00640 1 -- 172.21.15.111:0/2324654235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28eb71ba0 msgr2=0x55a28eb7b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.826+0000 7f7ed0c00640 1 --2- 172.21.15.111:0/2324654235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28eb71ba0 0x55a28eb7b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.826+0000 7f7ed0c00640 1 -- 172.21.15.111:0/2324654235 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a28e8b5620 con 0x55a28eb71260 2024-09-15T22:52:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.827+0000 7f7ed0c00640 1 --2- 172.21.15.111:0/2324654235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28eb71260 0x55a28eb7b360 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4004480 tx=0x7f7ec4009c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=995a452d05806eda server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:52:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.827+0000 7f7eb5600640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:52:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.827+0000 7f7eb5600640 1 -- 172.21.15.111:0/2324654235 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec400f6e0 con 0x55a28eb71260 2024-09-15T22:52:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.827+0000 7f7eb5600640 1 -- 172.21.15.111:0/2324654235 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f7ec400f880 con 0x55a28eb71260 2024-09-15T22:52:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.827+0000 7f7ed5a8e740 1 -- 172.21.15.111:0/2324654235 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ec4004050 con 0x55a28eb71260 2024-09-15T22:52:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.827+0000 7f7eb5600640 1 -- 172.21.15.111:0/2324654235 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7ec400fa80 con 0x55a28eb71260 2024-09-15T22:52:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.827+0000 7f7ed5a8e740 10 client.25493 fetch_fsmap learned FSMap version 30 2024-09-15T22:52:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.827+0000 7f7ed5a8e740 1 -- 172.21.15.111:0/2324654235 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a28eb85640 con 0x55a28eb71260 2024-09-15T22:52:10.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.827+0000 7f7eb5600640 1 -- 172.21.15.111:0/2324654235 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ec4004050 con 0x55a28eb71260 2024-09-15T22:52:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.828+0000 7f7eb5600640 1 -- 172.21.15.111:0/2324654235 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f7ec4025040 con 0x55a28eb71260 2024-09-15T22:52:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.828+0000 7f7ed5a8e740 10 client.25493 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:52:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.828+0000 7f7ed5a8e740 10 client.25493 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:52:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.828+0000 7f7ed5a8e740 20 client.25493 populate_metadata read hostname 'smithi111' 2024-09-15T22:52:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.828+0000 7f7ed5a8e740 1 --2- 172.21.15.111:0/2324654235 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a28eb86b90 0x55a28eb88f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.828+0000 7f7ed5a8e740 4 client.25493 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:52:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.828+0000 7f7ed5a8e740 1 -- 172.21.15.111:0/2324654235 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a28eae3830 con 0x55a28eb86b90 2024-09-15T22:52:10.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.828+0000 7f7ecb400640 1 --2- 172.21.15.111:0/2324654235 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a28eb86b90 0x55a28eb88f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.828+0000 7f7ecb400640 1 --2- 172.21.15.111:0/2324654235 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a28eb86b90 0x55a28eb88f80 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c920a1b5d655a2f server_cookie=3af0397df6dcd82d in_seq=0 out_seq=0 2024-09-15T22:52:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.828+0000 7f7eb5600640 10 client.25493 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:52:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.829+0000 7f7eb5600640 1 -- 172.21.15.111:0/2324654235 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a28eae3830 con 0x55a28eb86b90 2024-09-15T22:52:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.829+0000 7f7eb5600640 10 client.25493 handle_command_reply: tid=0 2024-09-15T22:52:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.829+0000 7f7ed5a8e740 1 client.25493 shutdown 2024-09-15T22:52:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.829+0000 7f7ed5a8e740 1 -- 172.21.15.111:0/2324654235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28eb71260 msgr2=0x55a28eb7b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.829+0000 7f7ed5a8e740 1 --2- 172.21.15.111:0/2324654235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28eb71260 0x55a28eb7b360 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4004480 tx=0x7f7ec4009c80 comp rx=0 tx=0).stop 2024-09-15T22:52:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.829+0000 7f7ed5a8e740 1 -- 172.21.15.111:0/2324654235 shutdown_connections 2024-09-15T22:52:10.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.829+0000 7f7ed5a8e740 1 --2- 172.21.15.111:0/2324654235 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a28eb86b90 0x55a28eb88f80 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.829+0000 7f7ed5a8e740 1 --2- 172.21.15.111:0/2324654235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28eb76c30 0x55a28eb828e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.829+0000 7f7ed5a8e740 1 --2- 172.21.15.111:0/2324654235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28eb71ba0 0x55a28eb7b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.829+0000 7f7ed5a8e740 1 --2- 172.21.15.111:0/2324654235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28eb71260 0x55a28eb7b360 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.829+0000 7f7eb5600640 0 client.25493 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:52:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.829+0000 7f7ed5a8e740 1 -- 172.21.15.111:0/2324654235 >> 172.21.15.111:0/2324654235 conn(0x55a28eb6c8d0 msgr2=0x55a28eb6e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:52:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.830+0000 7f7ed5a8e740 1 -- 172.21.15.111:0/2324654235 shutdown_connections 2024-09-15T22:52:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.830+0000 7f7ed5a8e740 1 -- 172.21.15.111:0/2324654235 wait complete. 2024-09-15T22:52:10.838 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.830+0000 7f7ed5a8e740 20 client.25493 trim_cache size 0 max 16384 2024-09-15T22:52:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.845+0000 7f7ed5a8e740 1 Processor -- start 2024-09-15T22:52:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.845+0000 7f7ed5a8e740 1 -- start start 2024-09-15T22:52:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.845+0000 7f7ed5a8e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28eb71260 0x55a28eb71640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.845+0000 7f7ed5a8e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28eb71ba0 0x55a28eb82800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:10.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.845+0000 7f7ed5a8e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28eb76c30 0x55a28eb6b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.845+0000 7f7ed5a8e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1ad7d0 con 0x55a28eb71ba0 2024-09-15T22:52:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.845+0000 7f7ed5a8e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1ab0a0 con 0x55a28eb76c30 2024-09-15T22:52:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.845+0000 7f7ed5a8e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1a8ef0 con 0x55a28eb71260 2024-09-15T22:52:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.846+0000 7f7ed0c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28eb71260 0x55a28eb71640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.846+0000 7f7ecb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28eb71ba0 0x55a28eb82800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.846+0000 7f7ed0c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28eb71260 0x55a28eb71640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:59198/0 (socket says 172.21.15.111:59198) 2024-09-15T22:52:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.846+0000 7f7ed0c00640 1 -- 172.21.15.111:0/2018605065 learned_addr learned my addr 172.21.15.111:0/2018605065 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:52:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.846+0000 7f7ed0c00640 1 --2- 172.21.15.111:0/2018605065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28eb76c30 0x55a28eb6b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.846+0000 7f7ecb400640 1 -- 172.21.15.111:0/2018605065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28eb71260 msgr2=0x55a28eb71640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.846+0000 7f7ecb400640 1 --2- 172.21.15.111:0/2018605065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28eb71260 0x55a28eb71640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.846+0000 7f7ecb400640 1 -- 172.21.15.111:0/2018605065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28eb76c30 msgr2=0x55a28eb6b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.846+0000 7f7ecb400640 1 --2- 172.21.15.111:0/2018605065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28eb76c30 0x55a28eb6b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.846+0000 7f7ecb400640 1 -- 172.21.15.111:0/2018605065 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a28eb6c6f0 con 0x55a28eb71ba0 2024-09-15T22:52:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.846+0000 7f7ed0c00640 1 --2- 172.21.15.111:0/2018605065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28eb76c30 0x55a28eb6b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:52:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.846+0000 7f7ecb400640 1 --2- 172.21.15.111:0/2018605065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28eb71ba0 0x55a28eb82800 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7eb802a910 tx=0x7f7eb8005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7269c8cbb455eaed server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:52:10.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.847+0000 7f7eb7e00640 1 -- 172.21.15.111:0/2018605065 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8035040 con 0x55a28eb71ba0 2024-09-15T22:52:10.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.847+0000 7f7eb7e00640 1 -- 172.21.15.111:0/2018605065 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eb804a070 con 0x55a28eb71ba0 2024-09-15T22:52:10.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.847+0000 7f7eb7e00640 1 -- 172.21.15.111:0/2018605065 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb80316b0 con 0x55a28eb71ba0 2024-09-15T22:52:10.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.848+0000 7f7ed5a8e740 1 -- 172.21.15.111:0/2018605065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28eb71ba0 msgr2=0x55a28eb82800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:10.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.848+0000 7f7ed5a8e740 1 --2- 172.21.15.111:0/2018605065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28eb71ba0 0x55a28eb82800 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7eb802a910 tx=0x7f7eb8005a20 comp rx=0 tx=0).stop 2024-09-15T22:52:10.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.848+0000 7f7ed5a8e740 1 -- 172.21.15.111:0/2018605065 shutdown_connections 2024-09-15T22:52:10.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.848+0000 7f7ed5a8e740 1 --2- 172.21.15.111:0/2018605065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28eb76c30 0x55a28eb6b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.848+0000 7f7ed5a8e740 1 --2- 172.21.15.111:0/2018605065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28eb71ba0 0x55a28eb82800 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.848+0000 7f7ed5a8e740 1 --2- 172.21.15.111:0/2018605065 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28eb71260 0x55a28eb71640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.848+0000 7f7ed5a8e740 1 -- 172.21.15.111:0/2018605065 >> 172.21.15.111:0/2018605065 conn(0x55a28eb6c8d0 msgr2=0x55a28eb6b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:52:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.848+0000 7f7ed5a8e740 1 -- 172.21.15.111:0/2018605065 shutdown_connections 2024-09-15T22:52:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.848+0000 7f7ed5a8e740 1 -- 172.21.15.111:0/2018605065 wait complete. 2024-09-15T22:52:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.848+0000 7f7ed5a8e740 1 -- start start 2024-09-15T22:52:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.848+0000 7f7ed5a8e740 1 Processor -- start 2024-09-15T22:52:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.849+0000 7f7ed5a8e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28eb71260 0x55a28eb7f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.849+0000 7f7ed5a8e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28eb71ba0 0x55a28eb7fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.849+0000 7f7ed5a8e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28eb76c30 0x55a28ebef830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:10.853 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.849+0000 7f7ed5a8e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1a9560 con 0x55a28eb71ba0 2024-09-15T22:52:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.849+0000 7f7ed5a8e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1aa9d0 con 0x55a28eb76c30 2024-09-15T22:52:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.849+0000 7f7ed5a8e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1a9c30 con 0x55a28eb71260 2024-09-15T22:52:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.849+0000 7f7ed5a8e740 10 client.0 _ll_register_callbacks cb 0x7f7ebc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:52:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.849+0000 7f7ed0c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28eb71260 0x55a28eb7f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.849+0000 7f7ed0c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28eb71260 0x55a28eb7f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:59202/0 (socket says 172.21.15.111:59202) 2024-09-15T22:52:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.849+0000 7f7ed0c00640 1 -- 172.21.15.111:0/2699462171 learned_addr learned my addr 172.21.15.111:0/2699462171 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:52:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.849+0000 7f7ed0c00640 1 --2- 172.21.15.111:0/2699462171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28eb76c30 0x55a28ebef830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.849+0000 7f7ecb400640 1 --2- 172.21.15.111:0/2699462171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28eb71ba0 0x55a28eb7fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.849+0000 7f7ed0c00640 1 -- 172.21.15.111:0/2699462171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28eb76c30 msgr2=0x55a28ebef830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.849+0000 7f7ed0c00640 1 --2- 172.21.15.111:0/2699462171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28eb76c30 0x55a28ebef830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.854 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.849+0000 7f7ed0c00640 1 -- 172.21.15.111:0/2699462171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28eb71ba0 msgr2=0x55a28eb7fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.849+0000 7f7ed0c00640 1 --2- 172.21.15.111:0/2699462171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28eb71ba0 0x55a28eb7fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.849+0000 7f7ed0c00640 1 -- 172.21.15.111:0/2699462171 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a28ebefdc0 con 0x55a28eb71260 2024-09-15T22:52:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.849+0000 7f7ecb400640 1 --2- 172.21.15.111:0/2699462171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28eb71ba0 0x55a28eb7fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:52:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.850+0000 7f7ed0c00640 1 --2- 172.21.15.111:0/2699462171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28eb71260 0x55a28eb7f8a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4003ee0 tx=0x7f7ec4009c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4258da99a58d0786 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:52:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.850+0000 7f7eb5600640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:52:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.850+0000 7f7eb5600640 1 -- 172.21.15.111:0/2699462171 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec400fe50 con 0x55a28eb71260 2024-09-15T22:52:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.850+0000 7f7eb5600640 1 -- 172.21.15.111:0/2699462171 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f7ec400f4e0 con 0x55a28eb71260 2024-09-15T22:52:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.850+0000 7f7ed5a8e740 1 -- 172.21.15.111:0/2699462171 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7eb804a070 con 0x55a28eb71260 2024-09-15T22:52:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.850+0000 7f7eb5600640 1 -- 172.21.15.111:0/2699462171 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7ec400f6e0 con 0x55a28eb71260 2024-09-15T22:52:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.850+0000 7f7ed5a8e740 10 client.25513 fetch_fsmap learned FSMap version 30 2024-09-15T22:52:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.850+0000 7f7ed5a8e740 1 -- 172.21.15.111:0/2699462171 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a28ebf10d0 con 0x55a28eb71260 2024-09-15T22:52:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.851+0000 7f7eb5600640 1 -- 172.21.15.111:0/2699462171 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eb804a070 con 0x55a28eb71260 2024-09-15T22:52:10.857 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.851+0000 7f7eb5600640 1 -- 172.21.15.111:0/2699462171 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f7ec4019770 con 0x55a28eb71260 2024-09-15T22:52:10.857 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.851+0000 7f7ed5a8e740 10 client.25513 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:52:10.857 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.851+0000 7f7ed5a8e740 10 client.25513 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:52:10.857 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.851+0000 7f7ed5a8e740 20 client.25513 populate_metadata read hostname 'smithi111' 2024-09-15T22:52:10.857 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.851+0000 7f7ed5a8e740 1 --2- 172.21.15.111:0/2699462171 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a28ebf1930 0x55a28ebf3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:10.857 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.851+0000 7f7ed5a8e740 4 client.25513 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:52:10.857 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.851+0000 7f7ed5a8e740 1 -- 172.21.15.111:0/2699462171 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a28eb769f0 con 0x55a28ebf1930 2024-09-15T22:52:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.851+0000 7f7ecb400640 1 --2- 172.21.15.111:0/2699462171 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a28ebf1930 0x55a28ebf3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.852+0000 7f7ecb400640 1 --2- 172.21.15.111:0/2699462171 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a28ebf1930 0x55a28ebf3d50 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ca5ddcd2873ab89 server_cookie=c293cec9618df92d in_seq=0 out_seq=0 2024-09-15T22:52:10.858 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.852+0000 7f7eb5600640 10 client.25513 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:52:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.861+0000 7f7eb5600640 1 -- 172.21.15.111:0/2699462171 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a28eb769f0 con 0x55a28ebf1930 2024-09-15T22:52:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.861+0000 7f7eb5600640 10 client.25513 handle_command_reply: tid=0 2024-09-15T22:52:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.861+0000 7f7ed5a8e740 1 client.25513 shutdown 2024-09-15T22:52:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.861+0000 7f7ed5a8e740 1 -- 172.21.15.111:0/2699462171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28eb71260 msgr2=0x55a28eb7f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.861+0000 7f7ed5a8e740 1 --2- 172.21.15.111:0/2699462171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28eb71260 0x55a28eb7f8a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4003ee0 tx=0x7f7ec4009c80 comp rx=0 tx=0).stop 2024-09-15T22:52:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.861+0000 7f7ed5a8e740 1 -- 172.21.15.111:0/2699462171 shutdown_connections 2024-09-15T22:52:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.861+0000 7f7ed5a8e740 1 --2- 172.21.15.111:0/2699462171 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a28ebf1930 0x55a28ebf3d50 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.861+0000 7f7ed5a8e740 1 --2- 172.21.15.111:0/2699462171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a28eb76c30 0x55a28ebef830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.861+0000 7f7ed5a8e740 1 --2- 172.21.15.111:0/2699462171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a28eb71ba0 0x55a28eb7fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.861+0000 7f7ed5a8e740 1 --2- 172.21.15.111:0/2699462171 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a28eb71260 0x55a28eb7f8a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.861+0000 7f7ed5a8e740 1 -- 172.21.15.111:0/2699462171 >> 172.21.15.111:0/2699462171 conn(0x55a28eb6c8d0 msgr2=0x55a28eb7b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:52:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.861+0000 7f7eb5600640 0 client.25513 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:52:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.861+0000 7f7ed5a8e740 1 -- 172.21.15.111:0/2699462171 shutdown_connections 2024-09-15T22:52:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.861+0000 7f7ed5a8e740 1 -- 172.21.15.111:0/2699462171 wait complete. 2024-09-15T22:52:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.861+0000 7f7ed5a8e740 20 client.25513 trim_cache size 0 max 16384 2024-09-15T22:52:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:52:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:52:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:52:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:52:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.862+0000 7f7ed1600640 1 -- 172.21.15.111:0/3054956313 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7eb00a32c0 msgr2=0x7f7eb00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.862+0000 7f7ed1600640 1 --2- 172.21.15.111:0/3054956313 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7eb00a32c0 0x7f7eb00a5780 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc098c20 tx=0x7f7ebc000bc0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.862+0000 7f7ed1600640 1 -- 172.21.15.111:0/3054956313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ecc100210 msgr2=0x7f7ecc199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.862+0000 7f7ed1600640 1 --2- 172.21.15.111:0/3054956313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ecc100210 0x7f7ecc199e30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00e9f0 tx=0x7f7ebc00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.862+0000 7f7ed1600640 1 -- 172.21.15.111:0/3054956313 shutdown_connections 2024-09-15T22:52:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.862+0000 7f7ed1600640 1 --2- 172.21.15.111:0/3054956313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ecc1014f0 0x7f7ecc1a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.862+0000 7f7ed1600640 1 --2- 172.21.15.111:0/3054956313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ecc100b50 0x7f7ecc19a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.862+0000 7f7ed1600640 1 --2- 172.21.15.111:0/3054956313 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7eb00a32c0 0x7f7eb00a5780 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.862+0000 7f7ed1600640 1 --2- 172.21.15.111:0/3054956313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ecc100210 0x7f7ecc199e30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.862+0000 7f7ed1600640 1 -- 172.21.15.111:0/3054956313 >> 172.21.15.111:0/3054956313 conn(0x7f7ecc0fbfa0 msgr2=0x7f7ecc0fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:52:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.862+0000 7f7ed1600640 1 -- 172.21.15.111:0/3054956313 shutdown_connections 2024-09-15T22:52:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:10.862+0000 7f7ed1600640 1 -- 172.21.15.111:0/3054956313 wait complete. 2024-09-15T22:52:11.263 INFO:tasks.workunit.client.0.smithi111.stdout: 1 528153 9.93 MB/sec execute 159 sec latency 54.387 ms 2024-09-15T22:52:11.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:11 smithi111 ceph-mon[30629]: pgmap v538: 97 pgs: 97 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 2.4 MiB/s wr, 177 op/s 2024-09-15T22:52:12.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:11 smithi192 ceph-mon[35427]: pgmap v538: 97 pgs: 97 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 2.4 MiB/s wr, 177 op/s 2024-09-15T22:52:12.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:11 smithi121 ceph-mon[34815]: pgmap v538: 97 pgs: 97 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 2.4 MiB/s wr, 177 op/s 2024-09-15T22:52:12.263 INFO:tasks.workunit.client.0.smithi111.stdout: 1 530538 9.95 MB/sec execute 160 sec latency 16.179 ms 2024-09-15T22:52:13.264 INFO:tasks.workunit.client.0.smithi111.stdout: 1 533549 10.00 MB/sec execute 161 sec latency 15.327 ms 2024-09-15T22:52:13.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:13 smithi111 ceph-mon[30629]: pgmap v539: 97 pgs: 97 active+clean; 196 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 4.3 MiB/s wr, 204 op/s 2024-09-15T22:52:14.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:13 smithi192 ceph-mon[35427]: pgmap v539: 97 pgs: 97 active+clean; 196 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 4.3 MiB/s wr, 204 op/s 2024-09-15T22:52:14.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:13 smithi121 ceph-mon[34815]: pgmap v539: 97 pgs: 97 active+clean; 196 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 4.3 MiB/s wr, 204 op/s 2024-09-15T22:52:14.263 INFO:tasks.workunit.client.0.smithi111.stdout: 1 535811 10.01 MB/sec execute 162 sec latency 226.390 ms 2024-09-15T22:52:15.263 INFO:tasks.workunit.client.0.smithi111.stdout: 1 535811 9.95 MB/sec execute 163 sec latency 1110.842 ms 2024-09-15T22:52:15.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:15 smithi111 ceph-mon[30629]: pgmap v540: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 4.1 MiB/s wr, 192 op/s 2024-09-15T22:52:16.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:15 smithi192 ceph-mon[35427]: pgmap v540: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 4.1 MiB/s wr, 192 op/s 2024-09-15T22:52:16.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:15 smithi121 ceph-mon[34815]: pgmap v540: 97 pgs: 97 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 4.1 MiB/s wr, 192 op/s 2024-09-15T22:52:16.263 INFO:tasks.workunit.client.0.smithi111.stdout: 1 537006 9.93 MB/sec execute 164 sec latency 1525.262 ms 2024-09-15T22:52:17.263 INFO:tasks.workunit.client.0.smithi111.stdout: 1 540177 9.99 MB/sec execute 165 sec latency 24.583 ms 2024-09-15T22:52:18.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:17 smithi192 ceph-mon[35427]: pgmap v541: 97 pgs: 97 active+clean; 204 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 201 op/s 2024-09-15T22:52:18.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:17 smithi121 ceph-mon[34815]: pgmap v541: 97 pgs: 97 active+clean; 204 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 201 op/s 2024-09-15T22:52:18.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:17 smithi111 ceph-mon[30629]: pgmap v541: 97 pgs: 97 active+clean; 204 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 201 op/s 2024-09-15T22:52:18.263 INFO:tasks.workunit.client.0.smithi111.stdout: 1 542991 10.00 MB/sec execute 166 sec latency 11.462 ms 2024-09-15T22:52:19.263 INFO:tasks.workunit.client.0.smithi111.stdout: 1 544588 9.99 MB/sec execute 167 sec latency 163.962 ms 2024-09-15T22:52:19.930 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:19 smithi111 ceph-mon[30629]: pgmap v542: 97 pgs: 97 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 209 op/s 2024-09-15T22:52:20.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:19 smithi192 ceph-mon[35427]: pgmap v542: 97 pgs: 97 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 209 op/s 2024-09-15T22:52:20.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:19 smithi121 ceph-mon[34815]: pgmap v542: 97 pgs: 97 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 209 op/s 2024-09-15T22:52:20.263 INFO:tasks.workunit.client.0.smithi111.stdout: 1 544588 9.93 MB/sec execute 168 sec latency 1164.028 ms 2024-09-15T22:52:21.263 INFO:tasks.workunit.client.0.smithi111.stdout: 1 547072 9.97 MB/sec execute 169 sec latency 1493.817 ms 2024-09-15T22:52:22.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:21 smithi192 ceph-mon[35427]: pgmap v543: 97 pgs: 97 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.2 MiB/s wr, 212 op/s 2024-09-15T22:52:22.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:21 smithi121 ceph-mon[34815]: pgmap v543: 97 pgs: 97 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.2 MiB/s wr, 212 op/s 2024-09-15T22:52:22.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:21 smithi111 ceph-mon[30629]: pgmap v543: 97 pgs: 97 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.2 MiB/s wr, 212 op/s 2024-09-15T22:52:22.265 INFO:tasks.workunit.client.0.smithi111.stdout: 1 549448 9.96 MB/sec execute 170 sec latency 16.218 ms 2024-09-15T22:52:23.263 INFO:tasks.workunit.client.0.smithi111.stdout: 1 551582 9.98 MB/sec execute 171 sec latency 13.422 ms 2024-09-15T22:52:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.992+0000 7f605d000640 1 -- 172.21.15.111:0/688042593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6058076bb0 msgr2=0x7f6058076fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.992+0000 7f605d000640 1 --2- 172.21.15.111:0/688042593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6058076bb0 0x7f6058076fb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6048009900 tx=0x7f604802a410 comp rx=0 tx=0).stop 2024-09-15T22:52:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.992+0000 7f605d000640 1 -- 172.21.15.111:0/688042593 shutdown_connections 2024-09-15T22:52:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.992+0000 7f605d000640 1 --2- 172.21.15.111:0/688042593 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6058078790 0x7f6058079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:23.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.992+0000 7f605d000640 1 --2- 172.21.15.111:0/688042593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6058077df0 0x7f6058078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.992+0000 7f605d000640 1 --2- 172.21.15.111:0/688042593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6058076bb0 0x7f6058076fb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.992+0000 7f605d000640 1 -- 172.21.15.111:0/688042593 >> 172.21.15.111:0/688042593 conn(0x7f60580ff4e0 msgr2=0x7f6058101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:52:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.992+0000 7f605d000640 1 -- 172.21.15.111:0/688042593 shutdown_connections 2024-09-15T22:52:23.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.992+0000 7f605d000640 1 -- 172.21.15.111:0/688042593 wait complete. 2024-09-15T22:52:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.993+0000 7f605d000640 1 Processor -- start 2024-09-15T22:52:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.993+0000 7f605d000640 1 -- start start 2024-09-15T22:52:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.993+0000 7f605d000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6058076bb0 0x7f605819e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.993+0000 7f605d000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6058077df0 0x7f605819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.993+0000 7f605d000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6058078790 0x7f60581a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.994+0000 7f605d000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6058113e40 con 0x7f6058077df0 2024-09-15T22:52:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.994+0000 7f605d000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6058113cc0 con 0x7f6058078790 2024-09-15T22:52:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.994+0000 7f605d000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6058113fc0 con 0x7f6058076bb0 2024-09-15T22:52:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.994+0000 7f6055e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6058076bb0 0x7f605819e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.994+0000 7f6056800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6058078790 0x7f60581a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:23.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.994+0000 7f6055e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6058076bb0 0x7f605819e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:38580/0 (socket says 172.21.15.111:38580) 2024-09-15T22:52:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.994+0000 7f6055e00640 1 -- 172.21.15.111:0/3937607293 learned_addr learned my addr 172.21.15.111:0/3937607293 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:52:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.994+0000 7f604d400640 1 --2- 172.21.15.111:0/3937607293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6058077df0 0x7f605819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.994+0000 7f6056800640 1 -- 172.21.15.111:0/3937607293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6058076bb0 msgr2=0x7f605819e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.994+0000 7f6056800640 1 --2- 172.21.15.111:0/3937607293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6058076bb0 0x7f605819e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.994+0000 7f6056800640 1 -- 172.21.15.111:0/3937607293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6058077df0 msgr2=0x7f605819e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.994+0000 7f6056800640 1 --2- 172.21.15.111:0/3937607293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6058077df0 0x7f605819e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.994+0000 7f6056800640 1 -- 172.21.15.111:0/3937607293 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60581a5f90 con 0x7f6058078790 2024-09-15T22:52:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.994+0000 7f6056800640 1 --2- 172.21.15.111:0/3937607293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6058078790 0x7f60581a5920 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6050009fd0 tx=0x7f605000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:52:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.995+0000 7f604f400640 1 -- 172.21.15.111:0/3937607293 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605000ce20 con 0x7f6058078790 2024-09-15T22:52:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.995+0000 7f604f400640 1 -- 172.21.15.111:0/3937607293 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6050005020 con 0x7f6058078790 2024-09-15T22:52:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.995+0000 7f604f400640 1 -- 172.21.15.111:0/3937607293 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6050010720 con 0x7f6058078790 2024-09-15T22:52:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.995+0000 7f605d000640 1 -- 172.21.15.111:0/3937607293 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60581a6280 con 0x7f6058078790 2024-09-15T22:52:23.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.995+0000 7f605d000640 1 -- 172.21.15.111:0/3937607293 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60581a67d0 con 0x7f6058078790 2024-09-15T22:52:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.998+0000 7f604f400640 1 -- 172.21.15.111:0/3937607293 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f6050005540 con 0x7f6058078790 2024-09-15T22:52:23.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.998+0000 7f606004e740 1 Processor -- start 2024-09-15T22:52:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.998+0000 7f606004e740 1 -- start start 2024-09-15T22:52:24.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.998+0000 7f606004e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d90c3cb090 0x55d90c3cb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.998+0000 7f606004e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d90c3cb9d0 0x55d90c3d0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.999+0000 7f606004e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d90c3d0a60 0x55d90c3d2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.999+0000 7f606004e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d90c330100 con 0x55d90c3cb090 2024-09-15T22:52:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.999+0000 7f606004e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d90c2d81b0 con 0x55d90c3d0a60 2024-09-15T22:52:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.999+0000 7f606004e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d90c32c0c0 con 0x55d90c3cb9d0 2024-09-15T22:52:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.999+0000 7f6055e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d90c3cb9d0 0x55d90c3d0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.999+0000 7f6055e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d90c3cb9d0 0x55d90c3d0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:38582/0 (socket says 172.21.15.111:38582) 2024-09-15T22:52:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.999+0000 7f6055e00640 1 -- 172.21.15.111:0/2689199743 learned_addr learned my addr 172.21.15.111:0/2689199743 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:52:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.999+0000 7f604d400640 1 --2- 172.21.15.111:0/2689199743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d90c3cb090 0x55d90c3cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.999+0000 7f6055e00640 1 -- 172.21.15.111:0/2689199743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d90c3d0a60 msgr2=0x55d90c3d2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.999+0000 7f6055e00640 1 --2- 172.21.15.111:0/2689199743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d90c3d0a60 0x55d90c3d2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.999+0000 7f6055e00640 1 -- 172.21.15.111:0/2689199743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d90c3cb090 msgr2=0x55d90c3cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:24.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.999+0000 7f6055e00640 1 --2- 172.21.15.111:0/2689199743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d90c3cb090 0x55d90c3cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:24.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:23.999+0000 7f6055e00640 1 -- 172.21.15.111:0/2689199743 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d90c043210 con 0x55d90c3cb9d0 2024-09-15T22:52:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.000+0000 7f604f400640 1 --2- 172.21.15.111:0/3937607293 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f603c0a32f0 0x7f603c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.000+0000 7f6055e00640 1 --2- 172.21.15.111:0/2689199743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d90c3cb9d0 0x55d90c3d0490 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f604802a930 tx=0x7f6048004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97aa74fd9125f0e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:52:24.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.000+0000 7f604f400640 1 -- 172.21.15.111:0/3937607293 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f60500cff10 con 0x7f6058078790 2024-09-15T22:52:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.000+0000 7f6047400640 1 -- 172.21.15.111:0/2689199743 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6048040070 con 0x55d90c3cb9d0 2024-09-15T22:52:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.000+0000 7f6047400640 1 -- 172.21.15.111:0/2689199743 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6048005500 con 0x55d90c3cb9d0 2024-09-15T22:52:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.000+0000 7f604d400640 1 --2- 172.21.15.111:0/3937607293 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f603c0a32f0 0x7f603c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.000+0000 7f6047400640 1 -- 172.21.15.111:0/2689199743 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6048035040 con 0x55d90c3cb9d0 2024-09-15T22:52:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.000+0000 7f606004e740 1 -- 172.21.15.111:0/2689199743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d90c3cb9d0 msgr2=0x55d90c3d0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.000+0000 7f606004e740 1 --2- 172.21.15.111:0/2689199743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d90c3cb9d0 0x55d90c3d0490 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f604802a930 tx=0x7f6048004040 comp rx=0 tx=0).stop 2024-09-15T22:52:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.001+0000 7f604d400640 1 --2- 172.21.15.111:0/3937607293 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f603c0a32f0 0x7f603c0a57b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f60400095f0 tx=0x7f6040013040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:52:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.001+0000 7f606004e740 1 -- 172.21.15.111:0/2689199743 shutdown_connections 2024-09-15T22:52:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.001+0000 7f606004e740 1 --2- 172.21.15.111:0/2689199743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d90c3d0a60 0x55d90c3d2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.001+0000 7f606004e740 1 --2- 172.21.15.111:0/2689199743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d90c3cb9d0 0x55d90c3d0490 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.001+0000 7f606004e740 1 --2- 172.21.15.111:0/2689199743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d90c3cb090 0x55d90c3cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.001+0000 7f606004e740 1 -- 172.21.15.111:0/2689199743 >> 172.21.15.111:0/2689199743 conn(0x55d90c3c6700 msgr2=0x55d90c3c6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:52:24.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.001+0000 7f606004e740 1 -- 172.21.15.111:0/2689199743 shutdown_connections 2024-09-15T22:52:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.001+0000 7f606004e740 1 -- 172.21.15.111:0/2689199743 wait complete. 2024-09-15T22:52:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.001+0000 7f606004e740 1 -- start start 2024-09-15T22:52:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.001+0000 7f606004e740 1 Processor -- start 2024-09-15T22:52:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.002+0000 7f606004e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d90c3cb090 0x55d90c3d5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.002+0000 7f606004e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d90c3cb9d0 0x55d90c3d5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.002+0000 7f606004e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d90c3d0a60 0x55d90c3dc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.002+0000 7f606004e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d90c3d39d0 con 0x55d90c3cb9d0 2024-09-15T22:52:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.002+0000 7f606004e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d90c2b9090 con 0x55d90c3cb090 2024-09-15T22:52:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.002+0000 7f606004e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d90c3d3b50 con 0x55d90c3d0a60 2024-09-15T22:52:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.002+0000 7f606004e740 10 client.0 _ll_register_callbacks cb 0x55d90c30df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:52:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.002+0000 7f6055e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d90c3cb090 0x55d90c3d5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.002+0000 7f604d400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d90c3cb9d0 0x55d90c3d5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.002+0000 7f6055e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d90c3cb090 0x55d90c3d5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:34480/0 (socket says 172.21.15.111:34480) 2024-09-15T22:52:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.002+0000 7f6055e00640 1 -- 172.21.15.111:0/3699434908 learned_addr learned my addr 172.21.15.111:0/3699434908 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:52:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.002+0000 7f6056800640 1 --2- 172.21.15.111:0/3699434908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d90c3d0a60 0x55d90c3dc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.003+0000 7f6055e00640 1 -- 172.21.15.111:0/3699434908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d90c3d0a60 msgr2=0x55d90c3dc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.003+0000 7f6055e00640 1 --2- 172.21.15.111:0/3699434908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d90c3d0a60 0x55d90c3dc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.003+0000 7f6055e00640 1 -- 172.21.15.111:0/3699434908 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d90c3cb9d0 msgr2=0x55d90c3d5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.003+0000 7f6055e00640 1 --2- 172.21.15.111:0/3699434908 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d90c3cb9d0 0x55d90c3d5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.003+0000 7f6055e00640 1 -- 172.21.15.111:0/3699434908 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d90c10f620 con 0x55d90c3cb090 2024-09-15T22:52:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.003+0000 7f6056800640 1 --2- 172.21.15.111:0/3699434908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d90c3d0a60 0x55d90c3dc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:52:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.003+0000 7f6055e00640 1 --2- 172.21.15.111:0/3699434908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d90c3cb090 0x55d90c3d5210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6048031440 tx=0x7f6048004550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6009ab96359b061d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:52:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.003+0000 7f603be00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:52:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.003+0000 7f603be00640 1 -- 172.21.15.111:0/3699434908 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6048040070 con 0x55d90c3cb090 2024-09-15T22:52:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.003+0000 7f606004e740 1 -- 172.21.15.111:0/3699434908 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6048005500 con 0x55d90c3cb090 2024-09-15T22:52:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.003+0000 7f603be00640 1 -- 172.21.15.111:0/3699434908 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f6048005220 con 0x55d90c3cb090 2024-09-15T22:52:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.004+0000 7f603be00640 1 -- 172.21.15.111:0/3699434908 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6048035040 con 0x55d90c3cb090 2024-09-15T22:52:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.004+0000 7f606004e740 10 client.25508 fetch_fsmap learned FSMap version 30 2024-09-15T22:52:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.004+0000 7f606004e740 1 -- 172.21.15.111:0/3699434908 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d90c3df580 con 0x55d90c3cb090 2024-09-15T22:52:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.004+0000 7f603be00640 1 -- 172.21.15.111:0/3699434908 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6048005500 con 0x55d90c3cb090 2024-09-15T22:52:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.004+0000 7f603be00640 1 -- 172.21.15.111:0/3699434908 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f60480035f0 con 0x55d90c3cb090 2024-09-15T22:52:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.004+0000 7f606004e740 10 client.25508 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:52:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.004+0000 7f606004e740 10 client.25508 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:52:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.004+0000 7f606004e740 20 client.25508 populate_metadata read hostname 'smithi111' 2024-09-15T22:52:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.004+0000 7f606004e740 1 --2- 172.21.15.111:0/3699434908 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d90c3e08d0 0x55d90c3e2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.004+0000 7f606004e740 4 client.25508 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:52:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.004+0000 7f606004e740 1 -- 172.21.15.111:0/3699434908 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d90c33d830 con 0x55d90c3e08d0 2024-09-15T22:52:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.005+0000 7f604d400640 1 --2- 172.21.15.111:0/3699434908 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d90c3e08d0 0x55d90c3e2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.005+0000 7f604d400640 1 --2- 172.21.15.111:0/3699434908 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d90c3e08d0 0x55d90c3e2cc0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8665b2ca63e561ca server_cookie=2684c5ca939876d8 in_seq=0 out_seq=0 2024-09-15T22:52:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.005+0000 7f603be00640 10 client.25508 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:52:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.006+0000 7f603be00640 1 -- 172.21.15.111:0/3699434908 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d90c33d830 con 0x55d90c3e08d0 2024-09-15T22:52:24.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.006+0000 7f603be00640 10 client.25508 handle_command_reply: tid=0 2024-09-15T22:52:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.006+0000 7f606004e740 1 client.25508 shutdown 2024-09-15T22:52:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.006+0000 7f606004e740 1 -- 172.21.15.111:0/3699434908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d90c3cb090 msgr2=0x55d90c3d5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.006+0000 7f606004e740 1 --2- 172.21.15.111:0/3699434908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d90c3cb090 0x55d90c3d5210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6048031440 tx=0x7f6048004550 comp rx=0 tx=0).stop 2024-09-15T22:52:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.006+0000 7f606004e740 1 -- 172.21.15.111:0/3699434908 shutdown_connections 2024-09-15T22:52:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.006+0000 7f606004e740 1 --2- 172.21.15.111:0/3699434908 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d90c3e08d0 0x55d90c3e2cc0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.006+0000 7f606004e740 1 --2- 172.21.15.111:0/3699434908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d90c3d0a60 0x55d90c3dc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.006+0000 7f606004e740 1 --2- 172.21.15.111:0/3699434908 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d90c3cb9d0 0x55d90c3d5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.006+0000 7f606004e740 1 --2- 172.21.15.111:0/3699434908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d90c3cb090 0x55d90c3d5210 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.006+0000 7f603be00640 0 client.25508 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:52:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.006+0000 7f606004e740 1 -- 172.21.15.111:0/3699434908 >> 172.21.15.111:0/3699434908 conn(0x55d90c3c6700 msgr2=0x55d90c3c7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:52:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.007+0000 7f606004e740 1 -- 172.21.15.111:0/3699434908 shutdown_connections 2024-09-15T22:52:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.007+0000 7f606004e740 1 -- 172.21.15.111:0/3699434908 wait complete. 2024-09-15T22:52:24.011 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.007+0000 7f606004e740 20 client.25508 trim_cache size 0 max 16384 2024-09-15T22:52:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.023+0000 7f606004e740 1 Processor -- start 2024-09-15T22:52:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.023+0000 7f606004e740 1 -- start start 2024-09-15T22:52:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.023+0000 7f606004e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d90c3cb090 0x55d90c3d56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.023+0000 7f606004e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d90c3cb9d0 0x55d90c3dc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.023+0000 7f606004e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d90c3d0a60 0x55d90c3c5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.023+0000 7f606004e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f60581b1c10 con 0x55d90c3d0a60 2024-09-15T22:52:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.023+0000 7f606004e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f60581af4e0 con 0x55d90c3cb090 2024-09-15T22:52:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.023+0000 7f606004e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f60581ad350 con 0x55d90c3cb9d0 2024-09-15T22:52:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.023+0000 7f6055e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d90c3cb090 0x55d90c3d56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.023+0000 7f604d400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d90c3cb9d0 0x55d90c3dc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.023+0000 7f6055e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d90c3cb090 0x55d90c3d56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:34496/0 (socket says 172.21.15.111:34496) 2024-09-15T22:52:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.023+0000 7f604d400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d90c3cb9d0 0x55d90c3dc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:38600/0 (socket says 172.21.15.111:38600) 2024-09-15T22:52:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.023+0000 7f6055e00640 1 -- 172.21.15.111:0/616831527 learned_addr learned my addr 172.21.15.111:0/616831527 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:52:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.023+0000 7f6056800640 1 --2- 172.21.15.111:0/616831527 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d90c3d0a60 0x55d90c3c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.023+0000 7f6055e00640 1 -- 172.21.15.111:0/616831527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d90c3cb9d0 msgr2=0x55d90c3dc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.024+0000 7f6055e00640 1 --2- 172.21.15.111:0/616831527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d90c3cb9d0 0x55d90c3dc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.024+0000 7f6055e00640 1 -- 172.21.15.111:0/616831527 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d90c3d0a60 msgr2=0x55d90c3c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.024+0000 7f6055e00640 1 --2- 172.21.15.111:0/616831527 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d90c3d0a60 0x55d90c3c5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.024+0000 7f6055e00640 1 -- 172.21.15.111:0/616831527 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d90c3c6520 con 0x55d90c3cb090 2024-09-15T22:52:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.024+0000 7f6055e00640 1 --2- 172.21.15.111:0/616831527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d90c3cb090 0x55d90c3d56d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f60480163e0 tx=0x7f60480058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c3d75e31d18e37a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:52:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.024+0000 7f6046a00640 1 -- 172.21.15.111:0/616831527 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6048040070 con 0x55d90c3cb090 2024-09-15T22:52:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.024+0000 7f6046a00640 1 -- 172.21.15.111:0/616831527 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6048003bd0 con 0x55d90c3cb090 2024-09-15T22:52:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.024+0000 7f6046a00640 1 -- 172.21.15.111:0/616831527 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604803c730 con 0x55d90c3cb090 2024-09-15T22:52:24.030 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.025+0000 7f606004e740 1 -- 172.21.15.111:0/616831527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d90c3cb090 msgr2=0x55d90c3d56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:24.030 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.025+0000 7f606004e740 1 --2- 172.21.15.111:0/616831527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d90c3cb090 0x55d90c3d56d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f60480163e0 tx=0x7f60480058d0 comp rx=0 tx=0).stop 2024-09-15T22:52:24.030 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.025+0000 7f606004e740 1 -- 172.21.15.111:0/616831527 shutdown_connections 2024-09-15T22:52:24.030 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.025+0000 7f606004e740 1 --2- 172.21.15.111:0/616831527 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d90c3d0a60 0x55d90c3c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:24.030 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.025+0000 7f606004e740 1 --2- 172.21.15.111:0/616831527 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d90c3cb9d0 0x55d90c3dc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:24.030 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.025+0000 7f606004e740 1 --2- 172.21.15.111:0/616831527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d90c3cb090 0x55d90c3d56d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:24.030 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.025+0000 7f606004e740 1 -- 172.21.15.111:0/616831527 >> 172.21.15.111:0/616831527 conn(0x55d90c3c6700 msgr2=0x55d90c3d4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:52:24.030 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.025+0000 7f606004e740 1 -- 172.21.15.111:0/616831527 shutdown_connections 2024-09-15T22:52:24.030 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.025+0000 7f606004e740 1 -- 172.21.15.111:0/616831527 wait complete. 2024-09-15T22:52:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.025+0000 7f606004e740 1 -- start start 2024-09-15T22:52:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.025+0000 7f606004e740 1 Processor -- start 2024-09-15T22:52:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.026+0000 7f606004e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d90c3cb090 0x55d90c3d9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.026+0000 7f606004e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d90c3cb9d0 0x55d90c3da050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.026+0000 7f606004e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d90c3d0a60 0x55d90c3db060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.026+0000 7f606004e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f60581ad9a0 con 0x55d90c3cb090 2024-09-15T22:52:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.026+0000 7f606004e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f60581aee10 con 0x55d90c3d0a60 2024-09-15T22:52:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.026+0000 7f606004e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f60581ae070 con 0x55d90c3cb9d0 2024-09-15T22:52:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.026+0000 7f606004e740 10 client.0 _ll_register_callbacks cb 0x7f6048009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:52:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.026+0000 7f6055e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d90c3cb090 0x55d90c3d9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.026+0000 7f6056800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d90c3d0a60 0x55d90c3db060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.026+0000 7f6056800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d90c3d0a60 0x55d90c3db060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:34512/0 (socket says 172.21.15.111:34512) 2024-09-15T22:52:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.026+0000 7f6056800640 1 -- 172.21.15.111:0/1985743092 learned_addr learned my addr 172.21.15.111:0/1985743092 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:52:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.026+0000 7f604d400640 1 --2- 172.21.15.111:0/1985743092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d90c3cb9d0 0x55d90c3da050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.026+0000 7f6056800640 1 -- 172.21.15.111:0/1985743092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d90c3cb9d0 msgr2=0x55d90c3da050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.026+0000 7f6056800640 1 --2- 172.21.15.111:0/1985743092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d90c3cb9d0 0x55d90c3da050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.026+0000 7f6056800640 1 -- 172.21.15.111:0/1985743092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d90c3cb090 msgr2=0x55d90c3d9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.026+0000 7f6056800640 1 --2- 172.21.15.111:0/1985743092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d90c3cb090 0x55d90c3d9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.027+0000 7f6056800640 1 -- 172.21.15.111:0/1985743092 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d90c3db670 con 0x55d90c3d0a60 2024-09-15T22:52:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.027+0000 7f604d400640 1 --2- 172.21.15.111:0/1985743092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d90c3cb9d0 0x55d90c3da050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:52:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.027+0000 7f6056800640 1 --2- 172.21.15.111:0/1985743092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d90c3d0a60 0x55d90c3db060 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6050098f00 tx=0x7f605009cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3c161d08af7510e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:52:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.027+0000 7f603be00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:52:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.027+0000 7f603be00640 1 -- 172.21.15.111:0/1985743092 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60500ba060 con 0x55d90c3d0a60 2024-09-15T22:52:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.027+0000 7f603be00640 1 -- 172.21.15.111:0/1985743092 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f60500a4d10 con 0x55d90c3d0a60 2024-09-15T22:52:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.027+0000 7f606004e740 1 -- 172.21.15.111:0/1985743092 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6048003bd0 con 0x55d90c3d0a60 2024-09-15T22:52:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.027+0000 7f603be00640 1 -- 172.21.15.111:0/1985743092 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f60500bc040 con 0x55d90c3d0a60 2024-09-15T22:52:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.027+0000 7f606004e740 10 client.25514 fetch_fsmap learned FSMap version 30 2024-09-15T22:52:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.027+0000 7f606004e740 1 -- 172.21.15.111:0/1985743092 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d90c3dca50 con 0x55d90c3d0a60 2024-09-15T22:52:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.027+0000 7f603be00640 1 -- 172.21.15.111:0/1985743092 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6048003bd0 con 0x55d90c3d0a60 2024-09-15T22:52:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.027+0000 7f603be00640 1 -- 172.21.15.111:0/1985743092 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f6050081020 con 0x55d90c3d0a60 2024-09-15T22:52:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.028+0000 7f606004e740 10 client.25514 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:52:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.028+0000 7f606004e740 10 client.25514 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:52:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.028+0000 7f606004e740 20 client.25514 populate_metadata read hostname 'smithi111' 2024-09-15T22:52:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.028+0000 7f606004e740 1 --2- 172.21.15.111:0/1985743092 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d90c44bac0 0x55d90c44dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.028+0000 7f606004e740 4 client.25514 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:52:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.028+0000 7f606004e740 1 -- 172.21.15.111:0/1985743092 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d90c3c1d80 con 0x55d90c44bac0 2024-09-15T22:52:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.028+0000 7f6055e00640 1 --2- 172.21.15.111:0/1985743092 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d90c44bac0 0x55d90c44dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.028+0000 7f6055e00640 1 --2- 172.21.15.111:0/1985743092 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d90c44bac0 0x55d90c44dee0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc09e38e6a5080c3 server_cookie=ed03bda76d254e2c in_seq=0 out_seq=0 2024-09-15T22:52:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.028+0000 7f603be00640 10 client.25514 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:52:24.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.037+0000 7f603be00640 1 -- 172.21.15.111:0/1985743092 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d90c3c1d80 con 0x55d90c44bac0 2024-09-15T22:52:24.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.037+0000 7f603be00640 10 client.25514 handle_command_reply: tid=0 2024-09-15T22:52:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.037+0000 7f606004e740 1 client.25514 shutdown 2024-09-15T22:52:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.037+0000 7f606004e740 1 -- 172.21.15.111:0/1985743092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d90c3d0a60 msgr2=0x55d90c3db060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.037+0000 7f606004e740 1 --2- 172.21.15.111:0/1985743092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d90c3d0a60 0x55d90c3db060 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6050098f00 tx=0x7f605009cd70 comp rx=0 tx=0).stop 2024-09-15T22:52:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.037+0000 7f606004e740 1 -- 172.21.15.111:0/1985743092 shutdown_connections 2024-09-15T22:52:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.037+0000 7f606004e740 1 --2- 172.21.15.111:0/1985743092 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d90c44bac0 0x55d90c44dee0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.037+0000 7f606004e740 1 --2- 172.21.15.111:0/1985743092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d90c3d0a60 0x55d90c3db060 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.037+0000 7f606004e740 1 --2- 172.21.15.111:0/1985743092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d90c3cb9d0 0x55d90c3da050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.037+0000 7f603be00640 0 client.25514 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:52:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.037+0000 7f606004e740 1 --2- 172.21.15.111:0/1985743092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d90c3cb090 0x55d90c3d9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.037+0000 7f606004e740 1 -- 172.21.15.111:0/1985743092 >> 172.21.15.111:0/1985743092 conn(0x55d90c3c6700 msgr2=0x55d90c3c7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:52:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.037+0000 7f606004e740 1 -- 172.21.15.111:0/1985743092 shutdown_connections 2024-09-15T22:52:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.037+0000 7f606004e740 1 -- 172.21.15.111:0/1985743092 wait complete. 2024-09-15T22:52:24.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.037+0000 7f606004e740 20 client.25514 trim_cache size 0 max 16384 2024-09-15T22:52:24.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:52:24.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:52:24.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:52:24.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:52:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.038+0000 7f605d000640 1 -- 172.21.15.111:0/3937607293 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f603c0a32f0 msgr2=0x7f603c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.038+0000 7f605d000640 1 --2- 172.21.15.111:0/3937607293 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f603c0a32f0 0x7f603c0a57b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f60400095f0 tx=0x7f6040013040 comp rx=0 tx=0).stop 2024-09-15T22:52:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.038+0000 7f605d000640 1 -- 172.21.15.111:0/3937607293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6058078790 msgr2=0x7f60581a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.038+0000 7f605d000640 1 --2- 172.21.15.111:0/3937607293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6058078790 0x7f60581a5920 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6050009fd0 tx=0x7f605000ef10 comp rx=0 tx=0).stop 2024-09-15T22:52:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.038+0000 7f605d000640 1 -- 172.21.15.111:0/3937607293 shutdown_connections 2024-09-15T22:52:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.038+0000 7f605d000640 1 --2- 172.21.15.111:0/3937607293 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f603c0a32f0 0x7f603c0a57b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.038+0000 7f605d000640 1 --2- 172.21.15.111:0/3937607293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6058078790 0x7f60581a5920 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.038+0000 7f605d000640 1 --2- 172.21.15.111:0/3937607293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6058077df0 0x7f605819e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.038+0000 7f605d000640 1 --2- 172.21.15.111:0/3937607293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6058076bb0 0x7f605819e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.038+0000 7f605d000640 1 -- 172.21.15.111:0/3937607293 >> 172.21.15.111:0/3937607293 conn(0x7f60580ff4e0 msgr2=0x7f6058100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:52:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.038+0000 7f605d000640 1 -- 172.21.15.111:0/3937607293 shutdown_connections 2024-09-15T22:52:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:24.038+0000 7f605d000640 1 -- 172.21.15.111:0/3937607293 wait complete. 2024-09-15T22:52:24.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:23 smithi192 ceph-mon[35427]: pgmap v544: 97 pgs: 97 active+clean; 207 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 266 op/s 2024-09-15T22:52:24.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:23 smithi121 ceph-mon[34815]: pgmap v544: 97 pgs: 97 active+clean; 207 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 266 op/s 2024-09-15T22:52:24.171 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:23 smithi111 ceph-mon[30629]: pgmap v544: 97 pgs: 97 active+clean; 207 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 266 op/s 2024-09-15T22:52:24.263 INFO:tasks.workunit.client.0.smithi111.stdout: 1 554476 10.02 MB/sec execute 172 sec latency 76.391 ms 2024-09-15T22:52:25.263 INFO:tasks.workunit.client.0.smithi111.stdout: 1 554480 9.97 MB/sec execute 173 sec latency 867.461 ms 2024-09-15T22:52:25.868 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:25 smithi111 ceph-mon[30629]: pgmap v545: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 201 op/s 2024-09-15T22:52:26.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:25 smithi192 ceph-mon[35427]: pgmap v545: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 201 op/s 2024-09-15T22:52:26.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:25 smithi121 ceph-mon[34815]: pgmap v545: 97 pgs: 97 active+clean; 208 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 201 op/s 2024-09-15T22:52:26.264 INFO:tasks.workunit.client.0.smithi111.stdout: 1 554570 9.91 MB/sec execute 174 sec latency 1756.255 ms 2024-09-15T22:52:27.264 INFO:tasks.workunit.client.0.smithi111.stdout: 1 557510 9.94 MB/sec execute 175 sec latency 19.870 ms 2024-09-15T22:52:28.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:27 smithi192 ceph-mon[35427]: pgmap v546: 97 pgs: 97 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 229 op/s 2024-09-15T22:52:28.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:27 smithi121 ceph-mon[34815]: pgmap v546: 97 pgs: 97 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 229 op/s 2024-09-15T22:52:28.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:27 smithi111 ceph-mon[30629]: pgmap v546: 97 pgs: 97 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 229 op/s 2024-09-15T22:52:28.264 INFO:tasks.workunit.client.0.smithi111.stdout: 1 560107 9.95 MB/sec execute 176 sec latency 13.439 ms 2024-09-15T22:52:29.264 INFO:tasks.workunit.client.0.smithi111.stdout: 1 562198 9.96 MB/sec execute 177 sec latency 20.220 ms 2024-09-15T22:52:29.952 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:29 smithi111 ceph-mon[30629]: pgmap v547: 97 pgs: 97 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 194 op/s 2024-09-15T22:52:30.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:29 smithi192 ceph-mon[35427]: pgmap v547: 97 pgs: 97 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 194 op/s 2024-09-15T22:52:30.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:29 smithi121 ceph-mon[34815]: pgmap v547: 97 pgs: 97 active+clean; 213 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 194 op/s 2024-09-15T22:52:30.264 INFO:tasks.workunit.client.0.smithi111.stdout: 1 562233 9.91 MB/sec execute 178 sec latency 953.326 ms 2024-09-15T22:52:31.264 INFO:tasks.workunit.client.0.smithi111.stdout: 1 563601 9.89 MB/sec execute 179 sec latency 1361.239 ms 2024-09-15T22:52:32.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:31 smithi192 ceph-mon[35427]: pgmap v548: 97 pgs: 97 active+clean; 210 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 174 op/s 2024-09-15T22:52:32.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:31 smithi121 ceph-mon[34815]: pgmap v548: 97 pgs: 97 active+clean; 210 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 174 op/s 2024-09-15T22:52:32.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:31 smithi111 ceph-mon[30629]: pgmap v548: 97 pgs: 97 active+clean; 210 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 174 op/s 2024-09-15T22:52:32.264 INFO:tasks.workunit.client.0.smithi111.stdout: 1 565902 9.91 MB/sec execute 180 sec latency 12.099 ms 2024-09-15T22:52:33.264 INFO:tasks.workunit.client.0.smithi111.stdout: 1 568848 9.95 MB/sec execute 181 sec latency 8.129 ms 2024-09-15T22:52:34.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:33 smithi192 ceph-mon[35427]: pgmap v549: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 207 op/s 2024-09-15T22:52:34.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:33 smithi121 ceph-mon[34815]: pgmap v549: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 207 op/s 2024-09-15T22:52:34.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:33 smithi111 ceph-mon[30629]: pgmap v549: 97 pgs: 97 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 207 op/s 2024-09-15T22:52:34.264 INFO:tasks.workunit.client.0.smithi111.stdout: 1 572036 9.99 MB/sec execute 182 sec latency 20.718 ms 2024-09-15T22:52:35.264 INFO:tasks.workunit.client.0.smithi111.stdout: 1 572195 9.94 MB/sec execute 183 sec latency 723.591 ms 2024-09-15T22:52:35.866 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:35 smithi111 ceph-mon[30629]: pgmap v550: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 199 op/s 2024-09-15T22:52:36.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:35 smithi192 ceph-mon[35427]: pgmap v550: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 199 op/s 2024-09-15T22:52:36.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:35 smithi121 ceph-mon[34815]: pgmap v550: 97 pgs: 97 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 199 op/s 2024-09-15T22:52:36.264 INFO:tasks.workunit.client.0.smithi111.stdout: 1 572195 9.89 MB/sec execute 184 sec latency 1723.641 ms 2024-09-15T22:52:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.166+0000 7fd8d2200640 1 -- 172.21.15.111:0/2742949839 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8cc103050 msgr2=0x7fd8cc103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.166+0000 7fd8d2200640 1 --2- 172.21.15.111:0/2742949839 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8cc103050 0x7fd8cc103450 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009900 tx=0x7fd8c002a3f0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.166+0000 7fd8d2200640 1 -- 172.21.15.111:0/2742949839 shutdown_connections 2024-09-15T22:52:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.166+0000 7fd8d2200640 1 --2- 172.21.15.111:0/2742949839 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8cc101f40 0x7fd8cc109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.166+0000 7fd8d2200640 1 --2- 172.21.15.111:0/2742949839 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8cc101580 0x7fd8cc101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.166+0000 7fd8d2200640 1 --2- 172.21.15.111:0/2742949839 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8cc103050 0x7fd8cc103450 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.166+0000 7fd8d2200640 1 -- 172.21.15.111:0/2742949839 >> 172.21.15.111:0/2742949839 conn(0x7fd8cc0fd0d0 msgr2=0x7fd8cc0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:52:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.166+0000 7fd8d2200640 1 -- 172.21.15.111:0/2742949839 shutdown_connections 2024-09-15T22:52:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.166+0000 7fd8d2200640 1 -- 172.21.15.111:0/2742949839 wait complete. 2024-09-15T22:52:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.167+0000 7fd8d2200640 1 Processor -- start 2024-09-15T22:52:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.168+0000 7fd8d2200640 1 -- start start 2024-09-15T22:52:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.168+0000 7fd8d2200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8cc101580 0x7fd8cc197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.168+0000 7fd8d2200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8cc101f40 0x7fd8cc198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.168+0000 7fd8d2200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8cc103050 0x7fd8cc19f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.168+0000 7fd8d2200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc10d740 con 0x7fd8cc101f40 2024-09-15T22:52:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.168+0000 7fd8d2200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc10d5c0 con 0x7fd8cc103050 2024-09-15T22:52:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.168+0000 7fd8d2200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc10d8c0 con 0x7fd8cc101580 2024-09-15T22:52:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.168+0000 7fd8cbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8cc101f40 0x7fd8cc198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.168+0000 7fd8d1800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8cc103050 0x7fd8cc19f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.168+0000 7fd8d1800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8cc103050 0x7fd8cc19f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:48484/0 (socket says 172.21.15.111:48484) 2024-09-15T22:52:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.168+0000 7fd8cbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8cc101f40 0x7fd8cc198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53846/0 (socket says 172.21.15.111:53846) 2024-09-15T22:52:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.168+0000 7fd8d1800640 1 -- 172.21.15.111:0/1134579358 learned_addr learned my addr 172.21.15.111:0/1134579358 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:52:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.168+0000 7fd8d0e00640 1 --2- 172.21.15.111:0/1134579358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8cc101580 0x7fd8cc197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.169+0000 7fd8d1800640 1 -- 172.21.15.111:0/1134579358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8cc101580 msgr2=0x7fd8cc197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.169+0000 7fd8d1800640 1 --2- 172.21.15.111:0/1134579358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8cc101580 0x7fd8cc197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.169+0000 7fd8d1800640 1 -- 172.21.15.111:0/1134579358 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8cc101f40 msgr2=0x7fd8cc198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.169+0000 7fd8d1800640 1 --2- 172.21.15.111:0/1134579358 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8cc101f40 0x7fd8cc198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.169+0000 7fd8d1800640 1 -- 172.21.15.111:0/1134579358 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8cc19fa40 con 0x7fd8cc103050 2024-09-15T22:52:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.169+0000 7fd8d1800640 1 --2- 172.21.15.111:0/1134579358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8cc103050 0x7fd8cc19f3d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd8c400efd0 tx=0x7fd8c400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:52:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.169+0000 7fd8c9600640 1 -- 172.21.15.111:0/1134579358 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c4003b70 con 0x7fd8cc103050 2024-09-15T22:52:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.169+0000 7fd8c9600640 1 -- 172.21.15.111:0/1134579358 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8c4003d10 con 0x7fd8cc103050 2024-09-15T22:52:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.169+0000 7fd8d2200640 1 -- 172.21.15.111:0/1134579358 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8cc19fd30 con 0x7fd8cc103050 2024-09-15T22:52:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.169+0000 7fd8c9600640 1 -- 172.21.15.111:0/1134579358 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c40109f0 con 0x7fd8cc103050 2024-09-15T22:52:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.169+0000 7fd8d2200640 1 -- 172.21.15.111:0/1134579358 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8cc1a0220 con 0x7fd8cc103050 2024-09-15T22:52:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.173+0000 7fd8d67d3740 1 Processor -- start 2024-09-15T22:52:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.173+0000 7fd8c9600640 1 -- 172.21.15.111:0/1134579358 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd8c4004020 con 0x7fd8cc103050 2024-09-15T22:52:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.173+0000 7fd8d67d3740 1 -- start start 2024-09-15T22:52:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.173+0000 7fd8d67d3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9d457b260 0x55d9d457b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.173+0000 7fd8d67d3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9d457bba0 0x55d9d4580660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.174+0000 7fd8d67d3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9d4580c30 0x55d9d45830f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.174+0000 7fd8d67d3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d9d44e0100 con 0x55d9d457b260 2024-09-15T22:52:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.174+0000 7fd8d67d3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d9d44881b0 con 0x55d9d457bba0 2024-09-15T22:52:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.174+0000 7fd8d67d3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d9d44dc0c0 con 0x55d9d4580c30 2024-09-15T22:52:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.174+0000 7fd8cbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9d457b260 0x55d9d457b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.174+0000 7fd8d0e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9d457bba0 0x55d9d4580660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.174+0000 7fd8d0e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9d457bba0 0x55d9d4580660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:48492/0 (socket says 172.21.15.111:48492) 2024-09-15T22:52:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.174+0000 7fd8d0e00640 1 -- 172.21.15.111:0/2582744263 learned_addr learned my addr 172.21.15.111:0/2582744263 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:52:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.174+0000 7fd8cbe00640 1 --2- 172.21.15.111:0/2582744263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9d4580c30 0x55d9d45830f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.174+0000 7fd8d0e00640 1 -- 172.21.15.111:0/2582744263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9d4580c30 msgr2=0x55d9d45830f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.174+0000 7fd8d0e00640 1 --2- 172.21.15.111:0/2582744263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9d4580c30 0x55d9d45830f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.174+0000 7fd8d0e00640 1 -- 172.21.15.111:0/2582744263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9d457b260 msgr2=0x55d9d457b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.174+0000 7fd8d0e00640 1 --2- 172.21.15.111:0/2582744263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9d457b260 0x55d9d457b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.174+0000 7fd8d0e00640 1 -- 172.21.15.111:0/2582744263 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9d41f3210 con 0x55d9d457bba0 2024-09-15T22:52:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.174+0000 7fd8cbe00640 1 --2- 172.21.15.111:0/2582744263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9d4580c30 0x55d9d45830f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:52:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.174+0000 7fd8d0e00640 1 --2- 172.21.15.111:0/2582744263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9d457bba0 0x55d9d4580660 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0006010 tx=0x7fd8c0004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf4ca80386256ea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:52:37.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.175+0000 7fd8c9600640 1 --2- 172.21.15.111:0/1134579358 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd8b40a32c0 0x7fd8b40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.175+0000 7fd8be000640 1 -- 172.21.15.111:0/2582744263 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0035040 con 0x55d9d457bba0 2024-09-15T22:52:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.175+0000 7fd8be000640 1 -- 172.21.15.111:0/2582744263 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8c0004430 con 0x55d9d457bba0 2024-09-15T22:52:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.175+0000 7fd8be000640 1 -- 172.21.15.111:0/2582744263 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c00316b0 con 0x55d9d457bba0 2024-09-15T22:52:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.175+0000 7fd8c9600640 1 -- 172.21.15.111:0/1134579358 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fd8c40cfcb0 con 0x7fd8cc103050 2024-09-15T22:52:37.178 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.175+0000 7fd8cbe00640 1 --2- 172.21.15.111:0/1134579358 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd8b40a32c0 0x7fd8b40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.176+0000 7fd8d67d3740 1 -- 172.21.15.111:0/2582744263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9d457bba0 msgr2=0x55d9d4580660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.176+0000 7fd8d67d3740 1 --2- 172.21.15.111:0/2582744263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9d457bba0 0x55d9d4580660 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0006010 tx=0x7fd8c0004040 comp rx=0 tx=0).stop 2024-09-15T22:52:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.176+0000 7fd8cbe00640 1 --2- 172.21.15.111:0/1134579358 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd8b40a32c0 0x7fd8b40a5780 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009710 tx=0x7fd8b8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:52:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.176+0000 7fd8d67d3740 1 -- 172.21.15.111:0/2582744263 shutdown_connections 2024-09-15T22:52:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.176+0000 7fd8d67d3740 1 --2- 172.21.15.111:0/2582744263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9d4580c30 0x55d9d45830f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.176+0000 7fd8d67d3740 1 --2- 172.21.15.111:0/2582744263 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9d457bba0 0x55d9d4580660 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.176+0000 7fd8d67d3740 1 --2- 172.21.15.111:0/2582744263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9d457b260 0x55d9d457b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.176+0000 7fd8d67d3740 1 -- 172.21.15.111:0/2582744263 >> 172.21.15.111:0/2582744263 conn(0x55d9d45768d0 msgr2=0x55d9d4576cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:52:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.176+0000 7fd8d67d3740 1 -- 172.21.15.111:0/2582744263 shutdown_connections 2024-09-15T22:52:37.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.176+0000 7fd8d67d3740 1 -- 172.21.15.111:0/2582744263 wait complete. 2024-09-15T22:52:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.177+0000 7fd8d67d3740 1 -- start start 2024-09-15T22:52:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.177+0000 7fd8d67d3740 1 Processor -- start 2024-09-15T22:52:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.177+0000 7fd8d67d3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9d457b260 0x55d9d45853e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.177+0000 7fd8d67d3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9d457bba0 0x55d9d4585920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.177+0000 7fd8d67d3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9d4580c30 0x55d9d458c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.177+0000 7fd8d67d3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d9d4583ba0 con 0x55d9d457bba0 2024-09-15T22:52:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.177+0000 7fd8d67d3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d9d4469090 con 0x55d9d4580c30 2024-09-15T22:52:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.177+0000 7fd8d67d3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d9d4583d20 con 0x55d9d457b260 2024-09-15T22:52:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.177+0000 7fd8d67d3740 10 client.0 _ll_register_callbacks cb 0x55d9d44bdf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:52:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.178+0000 7fd8d1800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9d4580c30 0x55d9d458c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.178+0000 7fd8cbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9d457bba0 0x55d9d4585920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.178+0000 7fd8d1800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9d4580c30 0x55d9d458c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:48500/0 (socket says 172.21.15.111:48500) 2024-09-15T22:52:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.178+0000 7fd8d1800640 1 -- 172.21.15.111:0/248055933 learned_addr learned my addr 172.21.15.111:0/248055933 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:52:37.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.178+0000 7fd8d0e00640 1 --2- 172.21.15.111:0/248055933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9d457b260 0x55d9d45853e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:37.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.178+0000 7fd8d1800640 1 -- 172.21.15.111:0/248055933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9d457b260 msgr2=0x55d9d45853e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:37.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.178+0000 7fd8d1800640 1 --2- 172.21.15.111:0/248055933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9d457b260 0x55d9d45853e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.178+0000 7fd8d1800640 1 -- 172.21.15.111:0/248055933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9d457bba0 msgr2=0x55d9d4585920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:37.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.178+0000 7fd8d1800640 1 --2- 172.21.15.111:0/248055933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9d457bba0 0x55d9d4585920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.178+0000 7fd8d1800640 1 -- 172.21.15.111:0/248055933 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9d42bf620 con 0x55d9d4580c30 2024-09-15T22:52:37.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.178+0000 7fd8d1800640 1 --2- 172.21.15.111:0/248055933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9d4580c30 0x55d9d458c960 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4098ca0 tx=0x7fd8c409bcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=809304f066088ccd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:52:37.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.178+0000 7fd8b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:52:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.179+0000 7fd8b2a00640 1 -- 172.21.15.111:0/248055933 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c40b5060 con 0x55d9d4580c30 2024-09-15T22:52:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.179+0000 7fd8b2a00640 1 -- 172.21.15.111:0/248055933 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fd8c4004620 con 0x55d9d4580c30 2024-09-15T22:52:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.179+0000 7fd8d67d3740 1 -- 172.21.15.111:0/248055933 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8c0004430 con 0x55d9d4580c30 2024-09-15T22:52:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.179+0000 7fd8b2a00640 1 -- 172.21.15.111:0/248055933 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd8c409c2d0 con 0x55d9d4580c30 2024-09-15T22:52:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.179+0000 7fd8d67d3740 10 client.25526 fetch_fsmap learned FSMap version 30 2024-09-15T22:52:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.179+0000 7fd8d67d3740 1 -- 172.21.15.111:0/248055933 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d9d458f7e0 con 0x55d9d4580c30 2024-09-15T22:52:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.179+0000 7fd8b2a00640 1 -- 172.21.15.111:0/248055933 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8c0004430 con 0x55d9d4580c30 2024-09-15T22:52:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.179+0000 7fd8b2a00640 1 -- 172.21.15.111:0/248055933 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fd8c401f020 con 0x55d9d4580c30 2024-09-15T22:52:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.179+0000 7fd8d67d3740 10 client.25526 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:52:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.179+0000 7fd8d67d3740 10 client.25526 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:52:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.179+0000 7fd8d67d3740 20 client.25526 populate_metadata read hostname 'smithi111' 2024-09-15T22:52:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.179+0000 7fd8d67d3740 1 --2- 172.21.15.111:0/248055933 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d9d4590e60 0x55d9d4593250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.179+0000 7fd8d67d3740 4 client.25526 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:52:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.179+0000 7fd8d67d3740 1 -- 172.21.15.111:0/248055933 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9d44ed830 con 0x55d9d4590e60 2024-09-15T22:52:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.180+0000 7fd8d0e00640 1 --2- 172.21.15.111:0/248055933 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d9d4590e60 0x55d9d4593250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.180+0000 7fd8d0e00640 1 --2- 172.21.15.111:0/248055933 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d9d4590e60 0x55d9d4593250 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=500fccec3a6f58f0 server_cookie=8e008d93549e47b in_seq=0 out_seq=0 2024-09-15T22:52:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.180+0000 7fd8b2a00640 10 client.25526 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:52:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.181+0000 7fd8b2a00640 1 -- 172.21.15.111:0/248055933 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d9d44ed830 con 0x55d9d4590e60 2024-09-15T22:52:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.181+0000 7fd8b2a00640 10 client.25526 handle_command_reply: tid=0 2024-09-15T22:52:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.181+0000 7fd8d67d3740 1 client.25526 shutdown 2024-09-15T22:52:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.181+0000 7fd8d67d3740 1 -- 172.21.15.111:0/248055933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9d4580c30 msgr2=0x55d9d458c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.181+0000 7fd8d67d3740 1 --2- 172.21.15.111:0/248055933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9d4580c30 0x55d9d458c960 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4098ca0 tx=0x7fd8c409bcf0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.181+0000 7fd8d67d3740 1 -- 172.21.15.111:0/248055933 shutdown_connections 2024-09-15T22:52:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.181+0000 7fd8d67d3740 1 --2- 172.21.15.111:0/248055933 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d9d4590e60 0x55d9d4593250 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.181+0000 7fd8d67d3740 1 --2- 172.21.15.111:0/248055933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9d4580c30 0x55d9d458c960 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.181+0000 7fd8d67d3740 1 --2- 172.21.15.111:0/248055933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9d457bba0 0x55d9d4585920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.181+0000 7fd8d67d3740 1 --2- 172.21.15.111:0/248055933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9d457b260 0x55d9d45853e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.181+0000 7fd8b2a00640 0 client.25526 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:52:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.181+0000 7fd8d67d3740 1 -- 172.21.15.111:0/248055933 >> 172.21.15.111:0/248055933 conn(0x55d9d45768d0 msgr2=0x55d9d45780e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:52:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.181+0000 7fd8d67d3740 1 -- 172.21.15.111:0/248055933 shutdown_connections 2024-09-15T22:52:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.182+0000 7fd8d67d3740 1 -- 172.21.15.111:0/248055933 wait complete. 2024-09-15T22:52:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.182+0000 7fd8d67d3740 20 client.25526 trim_cache size 0 max 16384 2024-09-15T22:52:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.198+0000 7fd8d67d3740 1 Processor -- start 2024-09-15T22:52:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.198+0000 7fd8d67d3740 1 -- start start 2024-09-15T22:52:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.198+0000 7fd8d67d3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9d457b260 0x55d9d4577ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.198+0000 7fd8d67d3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9d457bba0 0x55d9d458c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.198+0000 7fd8d67d3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9d4580c30 0x55d9d4597b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:37.200 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.198+0000 7fd8d67d3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc1ab540 con 0x55d9d457bba0 2024-09-15T22:52:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.198+0000 7fd8d67d3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc1a8e10 con 0x55d9d457b260 2024-09-15T22:52:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.198+0000 7fd8d67d3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc1a6c80 con 0x55d9d4580c30 2024-09-15T22:52:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.199+0000 7fd8d0e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9d457b260 0x55d9d4577ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.199+0000 7fd8d0e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9d457b260 0x55d9d4577ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:48510/0 (socket says 172.21.15.111:48510) 2024-09-15T22:52:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.199+0000 7fd8d0e00640 1 -- 172.21.15.111:0/1598392998 learned_addr learned my addr 172.21.15.111:0/1598392998 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:52:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.199+0000 7fd8d1800640 1 --2- 172.21.15.111:0/1598392998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9d4580c30 0x55d9d4597b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.199+0000 7fd8cbe00640 1 --2- 172.21.15.111:0/1598392998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9d457bba0 0x55d9d458c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:37.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.199+0000 7fd8d0e00640 1 -- 172.21.15.111:0/1598392998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9d4580c30 msgr2=0x55d9d4597b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:37.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.199+0000 7fd8d0e00640 1 --2- 172.21.15.111:0/1598392998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9d4580c30 0x55d9d4597b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.199+0000 7fd8d0e00640 1 -- 172.21.15.111:0/1598392998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9d457bba0 msgr2=0x55d9d458c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.199+0000 7fd8d0e00640 1 --2- 172.21.15.111:0/1598392998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9d457bba0 0x55d9d458c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.199+0000 7fd8d0e00640 1 -- 172.21.15.111:0/1598392998 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9d45766f0 con 0x55d9d457b260 2024-09-15T22:52:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.199+0000 7fd8cbe00640 1 --2- 172.21.15.111:0/1598392998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9d457bba0 0x55d9d458c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:52:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.199+0000 7fd8d0e00640 1 --2- 172.21.15.111:0/1598392998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9d457b260 0x55d9d4577ed0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0004650 tx=0x7fd8c0031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67f68456b2fc3763 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:52:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.200+0000 7fd8bd600640 1 -- 172.21.15.111:0/1598392998 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0035040 con 0x55d9d457b260 2024-09-15T22:52:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.200+0000 7fd8bd600640 1 -- 172.21.15.111:0/1598392998 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8c00054e0 con 0x55d9d457b260 2024-09-15T22:52:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.200+0000 7fd8bd600640 1 -- 172.21.15.111:0/1598392998 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0031c40 con 0x55d9d457b260 2024-09-15T22:52:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.200+0000 7fd8d67d3740 1 -- 172.21.15.111:0/1598392998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9d457b260 msgr2=0x55d9d4577ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.200+0000 7fd8d67d3740 1 --2- 172.21.15.111:0/1598392998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9d457b260 0x55d9d4577ed0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0004650 tx=0x7fd8c0031400 comp rx=0 tx=0).stop 2024-09-15T22:52:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.201+0000 7fd8d67d3740 1 -- 172.21.15.111:0/1598392998 shutdown_connections 2024-09-15T22:52:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.201+0000 7fd8d67d3740 1 --2- 172.21.15.111:0/1598392998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9d4580c30 0x55d9d4597b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.201+0000 7fd8d67d3740 1 --2- 172.21.15.111:0/1598392998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9d457bba0 0x55d9d458c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.201+0000 7fd8d67d3740 1 --2- 172.21.15.111:0/1598392998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9d457b260 0x55d9d4577ed0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.201+0000 7fd8d67d3740 1 -- 172.21.15.111:0/1598392998 >> 172.21.15.111:0/1598392998 conn(0x55d9d45768d0 msgr2=0x55d9d45777c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:52:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.201+0000 7fd8d67d3740 1 -- 172.21.15.111:0/1598392998 shutdown_connections 2024-09-15T22:52:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.201+0000 7fd8d67d3740 1 -- 172.21.15.111:0/1598392998 wait complete. 2024-09-15T22:52:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.201+0000 7fd8d67d3740 1 -- start start 2024-09-15T22:52:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.201+0000 7fd8d67d3740 1 Processor -- start 2024-09-15T22:52:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.202+0000 7fd8d67d3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9d457b260 0x55d9d458a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.202+0000 7fd8d67d3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9d457bba0 0x55d9d458a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.202+0000 7fd8d67d3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9d4580c30 0x55d9d458b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.202+0000 7fd8d67d3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc1a72d0 con 0x55d9d457bba0 2024-09-15T22:52:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.202+0000 7fd8d67d3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc1a8740 con 0x55d9d4580c30 2024-09-15T22:52:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.202+0000 7fd8d67d3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd8cc1a79a0 con 0x55d9d457b260 2024-09-15T22:52:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.202+0000 7fd8d67d3740 10 client.0 _ll_register_callbacks cb 0x7fd8c4097e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:52:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.202+0000 7fd8d0e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9d457b260 0x55d9d458a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.202+0000 7fd8cbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9d457bba0 0x55d9d458a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.202+0000 7fd8d1800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9d4580c30 0x55d9d458b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.202+0000 7fd8d0e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9d457b260 0x55d9d458a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:55544/0 (socket says 172.21.15.111:55544) 2024-09-15T22:52:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.202+0000 7fd8d1800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9d4580c30 0x55d9d458b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:48520/0 (socket says 172.21.15.111:48520) 2024-09-15T22:52:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.202+0000 7fd8d0e00640 1 -- 172.21.15.111:0/4206572255 learned_addr learned my addr 172.21.15.111:0/4206572255 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:52:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.202+0000 7fd8d0e00640 1 -- 172.21.15.111:0/4206572255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9d4580c30 msgr2=0x55d9d458b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.202+0000 7fd8d0e00640 1 --2- 172.21.15.111:0/4206572255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9d4580c30 0x55d9d458b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.202+0000 7fd8d0e00640 1 -- 172.21.15.111:0/4206572255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9d457bba0 msgr2=0x55d9d458a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.202+0000 7fd8d0e00640 1 --2- 172.21.15.111:0/4206572255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9d457bba0 0x55d9d458a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.202+0000 7fd8d0e00640 1 -- 172.21.15.111:0/4206572255 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9d458bbb0 con 0x55d9d457b260 2024-09-15T22:52:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.202+0000 7fd8d0e00640 1 --2- 172.21.15.111:0/4206572255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9d457b260 0x55d9d458a0a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd8c003f040 tx=0x7fd8c0004530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a16f195b8419d9bb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:52:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.202+0000 7fd8b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:52:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.203+0000 7fd8b2a00640 1 -- 172.21.15.111:0/4206572255 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0005200 con 0x55d9d457b260 2024-09-15T22:52:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.203+0000 7fd8d67d3740 1 -- 172.21.15.111:0/4206572255 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8c00054e0 con 0x55d9d457b260 2024-09-15T22:52:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.203+0000 7fd8b2a00640 1 -- 172.21.15.111:0/4206572255 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fd8c0004770 con 0x55d9d457b260 2024-09-15T22:52:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.203+0000 7fd8b2a00640 1 -- 172.21.15.111:0/4206572255 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd8c0004970 con 0x55d9d457b260 2024-09-15T22:52:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.203+0000 7fd8d67d3740 10 client.25546 fetch_fsmap learned FSMap version 30 2024-09-15T22:52:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.203+0000 7fd8d67d3740 1 -- 172.21.15.111:0/4206572255 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d9d45fbb40 con 0x55d9d457b260 2024-09-15T22:52:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.203+0000 7fd8b2a00640 1 -- 172.21.15.111:0/4206572255 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8c00054e0 con 0x55d9d457b260 2024-09-15T22:52:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.203+0000 7fd8b2a00640 1 -- 172.21.15.111:0/4206572255 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fd8c00039e0 con 0x55d9d457b260 2024-09-15T22:52:37.208 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.204+0000 7fd8d67d3740 10 client.25546 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:52:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.204+0000 7fd8d67d3740 10 client.25546 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:52:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.204+0000 7fd8d67d3740 20 client.25546 populate_metadata read hostname 'smithi111' 2024-09-15T22:52:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.204+0000 7fd8d67d3740 1 --2- 172.21.15.111:0/4206572255 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d9d45fc220 0x55d9d45fe610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.204+0000 7fd8d67d3740 4 client.25546 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:52:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.204+0000 7fd8d67d3740 1 -- 172.21.15.111:0/4206572255 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d9d44e7b80 con 0x55d9d45fc220 2024-09-15T22:52:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.204+0000 7fd8cbe00640 1 --2- 172.21.15.111:0/4206572255 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d9d45fc220 0x55d9d45fe610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.204+0000 7fd8cbe00640 1 --2- 172.21.15.111:0/4206572255 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d9d45fc220 0x55d9d45fe610 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9de9e9cf58453201 server_cookie=b608e543abe6d992 in_seq=0 out_seq=0 2024-09-15T22:52:37.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.204+0000 7fd8b2a00640 10 client.25546 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:52:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.212+0000 7fd8b2a00640 1 -- 172.21.15.111:0/4206572255 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d9d44e7b80 con 0x55d9d45fc220 2024-09-15T22:52:37.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.212+0000 7fd8b2a00640 10 client.25546 handle_command_reply: tid=0 2024-09-15T22:52:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.212+0000 7fd8d67d3740 1 client.25546 shutdown 2024-09-15T22:52:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.213+0000 7fd8d67d3740 1 -- 172.21.15.111:0/4206572255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9d457b260 msgr2=0x55d9d458a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.213+0000 7fd8d67d3740 1 --2- 172.21.15.111:0/4206572255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9d457b260 0x55d9d458a0a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd8c003f040 tx=0x7fd8c0004530 comp rx=0 tx=0).stop 2024-09-15T22:52:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.213+0000 7fd8d67d3740 1 -- 172.21.15.111:0/4206572255 shutdown_connections 2024-09-15T22:52:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.213+0000 7fd8d67d3740 1 --2- 172.21.15.111:0/4206572255 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d9d45fc220 0x55d9d45fe610 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.213+0000 7fd8d67d3740 1 --2- 172.21.15.111:0/4206572255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9d4580c30 0x55d9d458b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.213+0000 7fd8d67d3740 1 --2- 172.21.15.111:0/4206572255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9d457bba0 0x55d9d458a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.213+0000 7fd8d67d3740 1 --2- 172.21.15.111:0/4206572255 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9d457b260 0x55d9d458a0a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.213+0000 7fd8b2a00640 0 client.25546 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:52:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.213+0000 7fd8d67d3740 1 -- 172.21.15.111:0/4206572255 >> 172.21.15.111:0/4206572255 conn(0x55d9d45768d0 msgr2=0x55d9d4575a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:52:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.213+0000 7fd8d67d3740 1 -- 172.21.15.111:0/4206572255 shutdown_connections 2024-09-15T22:52:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.213+0000 7fd8d67d3740 1 -- 172.21.15.111:0/4206572255 wait complete. 2024-09-15T22:52:37.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.213+0000 7fd8d67d3740 20 client.25546 trim_cache size 0 max 16384 2024-09-15T22:52:37.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:52:37.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:52:37.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:52:37.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:52:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.214+0000 7fd8d2200640 1 -- 172.21.15.111:0/1134579358 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd8b40a32c0 msgr2=0x7fd8b40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.214+0000 7fd8d2200640 1 --2- 172.21.15.111:0/1134579358 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd8b40a32c0 0x7fd8b40a5780 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009710 tx=0x7fd8b8005f90 comp rx=0 tx=0).stop 2024-09-15T22:52:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.214+0000 7fd8d2200640 1 -- 172.21.15.111:0/1134579358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8cc103050 msgr2=0x7fd8cc19f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.214+0000 7fd8d2200640 1 --2- 172.21.15.111:0/1134579358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8cc103050 0x7fd8cc19f3d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd8c400efd0 tx=0x7fd8c400c6a0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.214+0000 7fd8d2200640 1 -- 172.21.15.111:0/1134579358 shutdown_connections 2024-09-15T22:52:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.214+0000 7fd8d2200640 1 --2- 172.21.15.111:0/1134579358 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd8b40a32c0 0x7fd8b40a5780 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.214+0000 7fd8d2200640 1 --2- 172.21.15.111:0/1134579358 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8cc103050 0x7fd8cc19f3d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.214+0000 7fd8d2200640 1 --2- 172.21.15.111:0/1134579358 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8cc101f40 0x7fd8cc198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.214+0000 7fd8d2200640 1 --2- 172.21.15.111:0/1134579358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8cc101580 0x7fd8cc197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.214+0000 7fd8d2200640 1 -- 172.21.15.111:0/1134579358 >> 172.21.15.111:0/1134579358 conn(0x7fd8cc0fd0d0 msgr2=0x7fd8cc105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:52:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.214+0000 7fd8d2200640 1 -- 172.21.15.111:0/1134579358 shutdown_connections 2024-09-15T22:52:37.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:37.214+0000 7fd8d2200640 1 -- 172.21.15.111:0/1134579358 wait complete. 2024-09-15T22:52:37.264 INFO:tasks.workunit.client.0.smithi111.stdout: 1 574852 9.91 MB/sec execute 185 sec latency 1738.815 ms 2024-09-15T22:52:38.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:37 smithi192 ceph-mon[35427]: pgmap v551: 97 pgs: 97 active+clean; 215 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 199 op/s 2024-09-15T22:52:38.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:37 smithi121 ceph-mon[34815]: pgmap v551: 97 pgs: 97 active+clean; 215 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 199 op/s 2024-09-15T22:52:38.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:37 smithi111 ceph-mon[30629]: pgmap v551: 97 pgs: 97 active+clean; 215 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 199 op/s 2024-09-15T22:52:38.264 INFO:tasks.workunit.client.0.smithi111.stdout: 1 577209 9.92 MB/sec execute 186 sec latency 13.159 ms 2024-09-15T22:52:39.264 INFO:tasks.workunit.client.0.smithi111.stdout: 1 579688 9.94 MB/sec execute 187 sec latency 11.809 ms 2024-09-15T22:52:39.849 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:39 smithi111 ceph-mon[30629]: pgmap v552: 97 pgs: 97 active+clean; 215 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 182 op/s 2024-09-15T22:52:40.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:39 smithi192 ceph-mon[35427]: pgmap v552: 97 pgs: 97 active+clean; 215 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 182 op/s 2024-09-15T22:52:40.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:39 smithi121 ceph-mon[34815]: pgmap v552: 97 pgs: 97 active+clean; 215 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 182 op/s 2024-09-15T22:52:40.264 INFO:tasks.workunit.client.0.smithi111.stdout: 1 580261 9.90 MB/sec execute 188 sec latency 703.203 ms 2024-09-15T22:52:41.264 INFO:tasks.workunit.client.0.smithi111.stdout: 1 580387 9.86 MB/sec execute 189 sec latency 1397.613 ms 2024-09-15T22:52:42.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:41 smithi192 ceph-mon[35427]: pgmap v553: 97 pgs: 97 active+clean; 216 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 226 op/s 2024-09-15T22:52:42.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:41 smithi121 ceph-mon[34815]: pgmap v553: 97 pgs: 97 active+clean; 216 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 226 op/s 2024-09-15T22:52:42.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:41 smithi111 ceph-mon[30629]: pgmap v553: 97 pgs: 97 active+clean; 216 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 226 op/s 2024-09-15T22:52:42.265 INFO:tasks.workunit.client.0.smithi111.stdout: 1 583058 9.88 MB/sec execute 190 sec latency 32.649 ms 2024-09-15T22:52:43.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:42 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:52:43.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:42 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:52:43.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:42 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:52:43.264 INFO:tasks.workunit.client.0.smithi111.stdout: 1 586056 9.92 MB/sec execute 191 sec latency 15.680 ms 2024-09-15T22:52:44.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:43 smithi192 ceph-mon[35427]: pgmap v554: 97 pgs: 97 active+clean; 215 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 229 op/s 2024-09-15T22:52:44.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:43 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:52:44.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:43 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:52:44.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:43 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:52:44.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:43 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:52:44.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:43 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:52:44.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:43 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:52:44.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:43 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:52:44.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:43 smithi121 ceph-mon[34815]: pgmap v554: 97 pgs: 97 active+clean; 215 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 229 op/s 2024-09-15T22:52:44.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:43 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:52:44.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:43 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:52:44.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:43 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:52:44.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:43 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:52:44.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:43 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:52:44.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:43 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:52:44.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:43 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:52:44.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:43 smithi111 ceph-mon[30629]: pgmap v554: 97 pgs: 97 active+clean; 215 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 229 op/s 2024-09-15T22:52:44.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:43 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:52:44.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:43 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:52:44.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:43 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:52:44.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:43 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:52:44.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:43 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:52:44.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:43 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:52:44.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:43 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:52:44.264 INFO:tasks.workunit.client.0.smithi111.stdout: 1 588569 9.92 MB/sec execute 192 sec latency 13.335 ms 2024-09-15T22:52:44.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:44 smithi111 ceph-mon[30629]: Adjusting osd_memory_target on smithi111 to 176.4M 2024-09-15T22:52:44.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:44 smithi111 ceph-mon[30629]: Unable to set osd_memory_target on smithi111 to 185037619: error parsing value: Value '185037619' is below minimum 939524096 2024-09-15T22:52:45.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:44 smithi192 ceph-mon[35427]: Adjusting osd_memory_target on smithi111 to 176.4M 2024-09-15T22:52:45.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:44 smithi192 ceph-mon[35427]: Unable to set osd_memory_target on smithi111 to 185037619: error parsing value: Value '185037619' is below minimum 939524096 2024-09-15T22:52:45.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:44 smithi121 ceph-mon[34815]: Adjusting osd_memory_target on smithi111 to 176.4M 2024-09-15T22:52:45.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:44 smithi121 ceph-mon[34815]: Unable to set osd_memory_target on smithi111 to 185037619: error parsing value: Value '185037619' is below minimum 939524096 2024-09-15T22:52:45.265 INFO:tasks.workunit.client.0.smithi111.stdout: 1 589821 9.92 MB/sec execute 193 sec latency 596.486 ms 2024-09-15T22:52:45.773 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:45 smithi111 ceph-mon[30629]: pgmap v555: 97 pgs: 97 active+clean; 215 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 185 op/s 2024-09-15T22:52:46.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:45 smithi192 ceph-mon[35427]: pgmap v555: 97 pgs: 97 active+clean; 215 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 185 op/s 2024-09-15T22:52:46.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:45 smithi121 ceph-mon[34815]: pgmap v555: 97 pgs: 97 active+clean; 215 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 185 op/s 2024-09-15T22:52:46.265 INFO:tasks.workunit.client.0.smithi111.stdout: 1 589821 9.87 MB/sec execute 194 sec latency 1596.571 ms 2024-09-15T22:52:47.265 INFO:tasks.workunit.client.0.smithi111.stdout: 1 591766 9.85 MB/sec execute 195 sec latency 1769.960 ms 2024-09-15T22:52:48.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:47 smithi192 ceph-mon[35427]: pgmap v556: 97 pgs: 97 active+clean; 218 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 174 op/s 2024-09-15T22:52:48.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:47 smithi121 ceph-mon[34815]: pgmap v556: 97 pgs: 97 active+clean; 218 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 174 op/s 2024-09-15T22:52:48.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:47 smithi111 ceph-mon[30629]: pgmap v556: 97 pgs: 97 active+clean; 218 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 174 op/s 2024-09-15T22:52:48.265 INFO:tasks.workunit.client.0.smithi111.stdout: 1 594048 9.88 MB/sec execute 196 sec latency 13.904 ms 2024-09-15T22:52:49.265 INFO:tasks.workunit.client.0.smithi111.stdout: 1 597137 9.92 MB/sec execute 197 sec latency 13.573 ms 2024-09-15T22:52:50.004 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:49 smithi111 ceph-mon[30629]: pgmap v557: 97 pgs: 97 active+clean; 222 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 208 op/s 2024-09-15T22:52:50.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:49 smithi192 ceph-mon[35427]: pgmap v557: 97 pgs: 97 active+clean; 222 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 208 op/s 2024-09-15T22:52:50.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:49 smithi121 ceph-mon[34815]: pgmap v557: 97 pgs: 97 active+clean; 222 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 208 op/s 2024-09-15T22:52:50.265 INFO:tasks.workunit.client.0.smithi111.stdout: 1 598403 9.89 MB/sec execute 198 sec latency 286.930 ms 2024-09-15T22:52:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.329+0000 7faa74e00640 1 -- 172.21.15.111:0/3157955843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa70104d00 msgr2=0x7faa701070f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.329+0000 7faa74e00640 1 --2- 172.21.15.111:0/3157955843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa70104d00 0x7faa701070f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009900 tx=0x7faa5c02a420 comp rx=0 tx=0).stop 2024-09-15T22:52:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.329+0000 7faa74e00640 1 -- 172.21.15.111:0/3157955843 shutdown_connections 2024-09-15T22:52:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.329+0000 7faa74e00640 1 --2- 172.21.15.111:0/3157955843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa70107630 0x7faa70109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.329+0000 7faa74e00640 1 --2- 172.21.15.111:0/3157955843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa70104d00 0x7faa701070f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.329+0000 7faa74e00640 1 --2- 172.21.15.111:0/3157955843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa701023d0 0x7faa701047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.329+0000 7faa74e00640 1 -- 172.21.15.111:0/3157955843 >> 172.21.15.111:0/3157955843 conn(0x7faa700fbfb0 msgr2=0x7faa700fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:52:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.329+0000 7faa74e00640 1 -- 172.21.15.111:0/3157955843 shutdown_connections 2024-09-15T22:52:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.329+0000 7faa74e00640 1 -- 172.21.15.111:0/3157955843 wait complete. 2024-09-15T22:52:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.330+0000 7faa74e00640 1 Processor -- start 2024-09-15T22:52:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.330+0000 7faa74e00640 1 -- start start 2024-09-15T22:52:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.330+0000 7faa74e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa701023d0 0x7faa70199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.330+0000 7faa74e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa70104d00 0x7faa7019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.330+0000 7faa74e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa70107630 0x7faa701a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.330+0000 7faa74e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faa7010d6d0 con 0x7faa70107630 2024-09-15T22:52:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.330+0000 7faa74e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faa7010d550 con 0x7faa70104d00 2024-09-15T22:52:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.330+0000 7faa74e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faa7010d850 con 0x7faa701023d0 2024-09-15T22:52:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.330+0000 7faa6fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa70107630 0x7faa701a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.330+0000 7faa6fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa70107630 0x7faa701a1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55158/0 (socket says 172.21.15.111:55158) 2024-09-15T22:52:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.330+0000 7faa6fe00640 1 -- 172.21.15.111:0/36626669 learned_addr learned my addr 172.21.15.111:0/36626669 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:52:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.330+0000 7faa6ea00640 1 --2- 172.21.15.111:0/36626669 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa70104d00 0x7faa7019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.330+0000 7faa6fe00640 1 -- 172.21.15.111:0/36626669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa701023d0 msgr2=0x7faa70199fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:52:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.330+0000 7faa6fe00640 1 --2- 172.21.15.111:0/36626669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa701023d0 0x7faa70199fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.330+0000 7faa6fe00640 1 -- 172.21.15.111:0/36626669 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa70104d00 msgr2=0x7faa7019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:50.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.331+0000 7faa6fe00640 1 --2- 172.21.15.111:0/36626669 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa70104d00 0x7faa7019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.331+0000 7faa6fe00640 1 -- 172.21.15.111:0/36626669 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa701a1bd0 con 0x7faa70107630 2024-09-15T22:52:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.331+0000 7faa6fe00640 1 --2- 172.21.15.111:0/36626669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa70107630 0x7faa701a1560 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7faa6800eb40 tx=0x7faa6800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:52:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.331+0000 7faa67e00640 1 -- 172.21.15.111:0/36626669 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa68004b20 con 0x7faa70107630 2024-09-15T22:52:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.331+0000 7faa67e00640 1 -- 172.21.15.111:0/36626669 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa68004cc0 con 0x7faa70107630 2024-09-15T22:52:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.331+0000 7faa74e00640 1 -- 172.21.15.111:0/36626669 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa701a1ec0 con 0x7faa70107630 2024-09-15T22:52:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.331+0000 7faa67e00640 1 -- 172.21.15.111:0/36626669 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa68010850 con 0x7faa70107630 2024-09-15T22:52:50.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.331+0000 7faa74e00640 1 -- 172.21.15.111:0/36626669 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa701a2400 con 0x7faa70107630 2024-09-15T22:52:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.334+0000 7faa67e00640 1 -- 172.21.15.111:0/36626669 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7faa68005020 con 0x7faa70107630 2024-09-15T22:52:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa7928c740 1 Processor -- start 2024-09-15T22:52:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa7928c740 1 -- start start 2024-09-15T22:52:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa7928c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313e260 0x55596313e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:50.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa7928c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55596313eba0 0x555963143660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa7928c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555963143c30 0x5559631460f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa7928c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5559630a3100 con 0x55596313e260 2024-09-15T22:52:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa7928c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55596304b1b0 con 0x55596313eba0 2024-09-15T22:52:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa7928c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55596309f0c0 con 0x555963143c30 2024-09-15T22:52:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa67e00640 1 --2- 172.21.15.111:0/36626669 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faa540a32c0 0x7faa540a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa6f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55596313eba0 0x555963143660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa6ea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313e260 0x55596313e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa6ea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313e260 0x55596313e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55174/0 (socket says 172.21.15.111:55174) 2024-09-15T22:52:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa6ea00640 1 -- 172.21.15.111:0/1182483409 learned_addr learned my addr 172.21.15.111:0/1182483409 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:52:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa6f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55596313eba0 0x555963143660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:34906/0 (socket says 172.21.15.111:34906) 2024-09-15T22:52:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa6ea00640 1 --2- 172.21.15.111:0/1182483409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555963143c30 0x5559631460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:50.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa67e00640 1 -- 172.21.15.111:0/36626669 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7faa680d0140 con 0x7faa70107630 2024-09-15T22:52:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa6fe00640 1 --2- 172.21.15.111:0/36626669 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faa540a32c0 0x7faa540a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa6ea00640 1 -- 172.21.15.111:0/1182483409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555963143c30 msgr2=0x5559631460f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa6ea00640 1 --2- 172.21.15.111:0/1182483409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555963143c30 0x5559631460f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa6ea00640 1 -- 172.21.15.111:0/1182483409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55596313eba0 msgr2=0x555963143660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa6ea00640 1 --2- 172.21.15.111:0/1182483409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55596313eba0 0x555963143660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa6ea00640 1 -- 172.21.15.111:0/1182483409 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555962db6210 con 0x55596313e260 2024-09-15T22:52:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa6ea00640 1 --2- 172.21.15.111:0/1182483409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313e260 0x55596313e660 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7faa5c006010 tx=0x7faa5c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500cbb072763fdbb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:52:50.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa6fe00640 1 --2- 172.21.15.111:0/36626669 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faa540a32c0 0x7faa540a5780 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7faa68004af0 tx=0x7faa68004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:52:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa64c00640 1 -- 172.21.15.111:0/1182483409 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c035070 con 0x55596313e260 2024-09-15T22:52:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa64c00640 1 -- 172.21.15.111:0/1182483409 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa5c032b90 con 0x55596313e260 2024-09-15T22:52:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa64c00640 1 -- 172.21.15.111:0/1182483409 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c03d450 con 0x55596313e260 2024-09-15T22:52:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa7928c740 1 -- 172.21.15.111:0/1182483409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313e260 msgr2=0x55596313e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.336+0000 7faa7928c740 1 --2- 172.21.15.111:0/1182483409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313e260 0x55596313e660 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7faa5c006010 tx=0x7faa5c000bc0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.339+0000 7faa7928c740 1 -- 172.21.15.111:0/1182483409 shutdown_connections 2024-09-15T22:52:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.339+0000 7faa7928c740 1 --2- 172.21.15.111:0/1182483409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555963143c30 0x5559631460f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.339+0000 7faa7928c740 1 --2- 172.21.15.111:0/1182483409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55596313eba0 0x555963143660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.339+0000 7faa7928c740 1 --2- 172.21.15.111:0/1182483409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313e260 0x55596313e660 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.339+0000 7faa7928c740 1 -- 172.21.15.111:0/1182483409 >> 172.21.15.111:0/1182483409 conn(0x5559631398d0 msgr2=0x555963139cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:52:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.339+0000 7faa7928c740 1 -- 172.21.15.111:0/1182483409 shutdown_connections 2024-09-15T22:52:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.339+0000 7faa7928c740 1 -- 172.21.15.111:0/1182483409 wait complete. 2024-09-15T22:52:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.339+0000 7faa7928c740 1 -- start start 2024-09-15T22:52:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.339+0000 7faa7928c740 1 Processor -- start 2024-09-15T22:52:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.339+0000 7faa7928c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313e260 0x5559631483c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.339+0000 7faa7928c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55596313eba0 0x555963148900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa7928c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555963143c30 0x55596314f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa7928c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555963146ba0 con 0x55596313e260 2024-09-15T22:52:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa7928c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55596302c090 con 0x55596313eba0 2024-09-15T22:52:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa7928c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555963146d20 con 0x555963143c30 2024-09-15T22:52:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa7928c740 10 client.0 _ll_register_callbacks cb 0x555963080f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:52:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa6ea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313e260 0x5559631483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa6ea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313e260 0x5559631483c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55190/0 (socket says 172.21.15.111:55190) 2024-09-15T22:52:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa6ea00640 1 -- 172.21.15.111:0/4253972459 learned_addr learned my addr 172.21.15.111:0/4253972459 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:52:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa6f400640 1 --2- 172.21.15.111:0/4253972459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55596313eba0 0x555963148900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:50.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa6ea00640 1 --2- 172.21.15.111:0/4253972459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555963143c30 0x55596314f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:50.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa6ea00640 1 -- 172.21.15.111:0/4253972459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555963143c30 msgr2=0x55596314f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:50.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa6ea00640 1 --2- 172.21.15.111:0/4253972459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555963143c30 0x55596314f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa6ea00640 1 -- 172.21.15.111:0/4253972459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55596313eba0 msgr2=0x555963148900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:50.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa6ea00640 1 --2- 172.21.15.111:0/4253972459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55596313eba0 0x555963148900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa6ea00640 1 -- 172.21.15.111:0/4253972459 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555962e82620 con 0x55596313e260 2024-09-15T22:52:50.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa6ea00640 1 --2- 172.21.15.111:0/4253972459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313e260 0x5559631483c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7faa5c005d80 tx=0x7faa5c005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f4db59330852f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:52:50.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa59600640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:52:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa59600640 1 -- 172.21.15.111:0/4253972459 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c056060 con 0x55596313e260 2024-09-15T22:52:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa59600640 1 -- 172.21.15.111:0/4253972459 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7faa5c0037f0 con 0x55596313e260 2024-09-15T22:52:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa7928c740 1 -- 172.21.15.111:0/4253972459 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa5c032b90 con 0x55596313e260 2024-09-15T22:52:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa59600640 1 -- 172.21.15.111:0/4253972459 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faa5c035040 con 0x55596313e260 2024-09-15T22:52:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa7928c740 10 client.15546 fetch_fsmap learned FSMap version 30 2024-09-15T22:52:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa7928c740 1 -- 172.21.15.111:0/4253972459 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555963152730 con 0x55596313e260 2024-09-15T22:52:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa59600640 1 -- 172.21.15.111:0/4253972459 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa5c032b90 con 0x55596313e260 2024-09-15T22:52:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa59600640 1 -- 172.21.15.111:0/4253972459 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7faa5c049070 con 0x55596313e260 2024-09-15T22:52:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa7928c740 10 client.15546 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:52:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa7928c740 10 client.15546 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:52:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa7928c740 20 client.15546 populate_metadata read hostname 'smithi111' 2024-09-15T22:52:50.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa7928c740 1 --2- 172.21.15.111:0/4253972459 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555963153a80 0x555963155e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa7928c740 4 client.15546 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:52:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa7928c740 1 -- 172.21.15.111:0/4253972459 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559630b0830 con 0x555963153a80 2024-09-15T22:52:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa6f400640 1 --2- 172.21.15.111:0/4253972459 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555963153a80 0x555963155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa6f400640 1 --2- 172.21.15.111:0/4253972459 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555963153a80 0x555963155e70 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b4b410386074bd0 server_cookie=b79a9c81b0ca8d1a in_seq=0 out_seq=0 2024-09-15T22:52:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa59600640 10 client.15546 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:52:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa59600640 1 -- 172.21.15.111:0/4253972459 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5559630b0830 con 0x555963153a80 2024-09-15T22:52:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa59600640 10 client.15546 handle_command_reply: tid=0 2024-09-15T22:52:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa7928c740 1 client.15546 shutdown 2024-09-15T22:52:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa7928c740 1 -- 172.21.15.111:0/4253972459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313e260 msgr2=0x5559631483c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa7928c740 1 --2- 172.21.15.111:0/4253972459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313e260 0x5559631483c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7faa5c005d80 tx=0x7faa5c005e50 comp rx=0 tx=0).stop 2024-09-15T22:52:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa7928c740 1 -- 172.21.15.111:0/4253972459 shutdown_connections 2024-09-15T22:52:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa7928c740 1 --2- 172.21.15.111:0/4253972459 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555963153a80 0x555963155e70 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa7928c740 1 --2- 172.21.15.111:0/4253972459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555963143c30 0x55596314f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa7928c740 1 --2- 172.21.15.111:0/4253972459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55596313eba0 0x555963148900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa59600640 0 client.15546 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:52:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa7928c740 1 --2- 172.21.15.111:0/4253972459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313e260 0x5559631483c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa7928c740 1 -- 172.21.15.111:0/4253972459 >> 172.21.15.111:0/4253972459 conn(0x5559631398d0 msgr2=0x55596313b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:52:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa7928c740 1 -- 172.21.15.111:0/4253972459 shutdown_connections 2024-09-15T22:52:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa7928c740 1 -- 172.21.15.111:0/4253972459 wait complete. 2024-09-15T22:52:50.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.340+0000 7faa7928c740 20 client.15546 trim_cache size 0 max 16384 2024-09-15T22:52:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa7928c740 1 Processor -- start 2024-09-15T22:52:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa7928c740 1 -- start start 2024-09-15T22:52:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa7928c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55596313e260 0x555963148170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa7928c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313eba0 0x55596314f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa7928c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555963143c30 0x5559631486b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa7928c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faa701ad890 con 0x55596313eba0 2024-09-15T22:52:50.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa7928c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faa701ab160 con 0x555963143c30 2024-09-15T22:52:50.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa7928c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faa701a8fd0 con 0x55596313e260 2024-09-15T22:52:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa6f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313eba0 0x55596314f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa6f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313eba0 0x55596314f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55204/0 (socket says 172.21.15.111:55204) 2024-09-15T22:52:50.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa6f400640 1 -- 172.21.15.111:0/3469636603 learned_addr learned my addr 172.21.15.111:0/3469636603 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:52:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa6ea00640 1 --2- 172.21.15.111:0/3469636603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55596313e260 0x555963148170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa6ea00640 1 --2- 172.21.15.111:0/3469636603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555963143c30 0x5559631486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa6f400640 1 -- 172.21.15.111:0/3469636603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55596313e260 msgr2=0x555963148170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa6f400640 1 --2- 172.21.15.111:0/3469636603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55596313e260 0x555963148170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa6f400640 1 -- 172.21.15.111:0/3469636603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555963143c30 msgr2=0x5559631486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa6f400640 1 --2- 172.21.15.111:0/3469636603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555963143c30 0x5559631486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa6f400640 1 -- 172.21.15.111:0/3469636603 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555963146e60 con 0x55596313eba0 2024-09-15T22:52:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa6ea00640 1 --2- 172.21.15.111:0/3469636603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555963143c30 0x5559631486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:52:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa6f400640 1 --2- 172.21.15.111:0/3469636603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313eba0 0x55596314f900 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x555963156670 tx=0x7faa6000b370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dcc1dd330444a44 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:52:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa5be00640 1 -- 172.21.15.111:0/3469636603 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa600155a0 con 0x55596313eba0 2024-09-15T22:52:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa5be00640 1 -- 172.21.15.111:0/3469636603 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa60015c20 con 0x55596313eba0 2024-09-15T22:52:50.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa5be00640 1 -- 172.21.15.111:0/3469636603 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa60023c10 con 0x55596313eba0 2024-09-15T22:52:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa7928c740 1 -- 172.21.15.111:0/3469636603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313eba0 msgr2=0x55596314f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa7928c740 1 --2- 172.21.15.111:0/3469636603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313eba0 0x55596314f900 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x555963156670 tx=0x7faa6000b370 comp rx=0 tx=0).stop 2024-09-15T22:52:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa7928c740 1 -- 172.21.15.111:0/3469636603 shutdown_connections 2024-09-15T22:52:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa7928c740 1 --2- 172.21.15.111:0/3469636603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555963143c30 0x5559631486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa7928c740 1 --2- 172.21.15.111:0/3469636603 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313eba0 0x55596314f900 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa7928c740 1 --2- 172.21.15.111:0/3469636603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55596313e260 0x555963148170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa7928c740 1 -- 172.21.15.111:0/3469636603 >> 172.21.15.111:0/3469636603 conn(0x5559631398d0 msgr2=0x555963143400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:52:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa7928c740 1 -- 172.21.15.111:0/3469636603 shutdown_connections 2024-09-15T22:52:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa7928c740 1 -- 172.21.15.111:0/3469636603 wait complete. 2024-09-15T22:52:50.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa7928c740 1 -- start start 2024-09-15T22:52:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa7928c740 1 Processor -- start 2024-09-15T22:52:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa7928c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313e260 0x55596314cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa7928c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55596313eba0 0x55596314d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa7928c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555963143c30 0x55596314e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa7928c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faa701a9620 con 0x55596313e260 2024-09-15T22:52:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa7928c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faa701aaa90 con 0x55596313eba0 2024-09-15T22:52:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa7928c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faa701a9cf0 con 0x555963143c30 2024-09-15T22:52:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa7928c740 10 client.0 _ll_register_callbacks cb 0x7faa5c0042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:52:50.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa6ea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313e260 0x55596314cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa6ea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313e260 0x55596314cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55208/0 (socket says 172.21.15.111:55208) 2024-09-15T22:52:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa6ea00640 1 -- 172.21.15.111:0/1714303873 learned_addr learned my addr 172.21.15.111:0/1714303873 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:52:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa6f400640 1 --2- 172.21.15.111:0/1714303873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55596313eba0 0x55596314d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa6ea00640 1 -- 172.21.15.111:0/1714303873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555963143c30 msgr2=0x55596314e1f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:52:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa6ea00640 1 --2- 172.21.15.111:0/1714303873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555963143c30 0x55596314e1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa6ea00640 1 -- 172.21.15.111:0/1714303873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55596313eba0 msgr2=0x55596314d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:52:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa6ea00640 1 --2- 172.21.15.111:0/1714303873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55596313eba0 0x55596314d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.357+0000 7faa6ea00640 1 -- 172.21.15.111:0/1714303873 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55596314e800 con 0x55596313e260 2024-09-15T22:52:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.363+0000 7faa6ea00640 1 --2- 172.21.15.111:0/1714303873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313e260 0x55596314cca0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009900 tx=0x7faa5c0045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2400787d25cb735c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:52:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.363+0000 7faa59600640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:52:50.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.363+0000 7faa59600640 1 -- 172.21.15.111:0/1714303873 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5c060070 con 0x55596313e260 2024-09-15T22:52:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.363+0000 7faa59600640 1 -- 172.21.15.111:0/1714303873 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7faa5c03dd30 con 0x55596313e260 2024-09-15T22:52:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.363+0000 7faa7928c740 1 -- 172.21.15.111:0/1714303873 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa60015c20 con 0x55596313e260 2024-09-15T22:52:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.363+0000 7faa59600640 1 -- 172.21.15.111:0/1714303873 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faa5c0035f0 con 0x55596313e260 2024-09-15T22:52:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.363+0000 7faa7928c740 10 client.15552 fetch_fsmap learned FSMap version 30 2024-09-15T22:52:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.363+0000 7faa7928c740 1 -- 172.21.15.111:0/1714303873 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55596314fa80 con 0x55596313e260 2024-09-15T22:52:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.363+0000 7faa59600640 1 -- 172.21.15.111:0/1714303873 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa60015c20 con 0x55596313e260 2024-09-15T22:52:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.363+0000 7faa59600640 1 -- 172.21.15.111:0/1714303873 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7faa5c049040 con 0x55596313e260 2024-09-15T22:52:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.363+0000 7faa7928c740 10 client.15552 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:52:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.363+0000 7faa7928c740 10 client.15552 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:52:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.363+0000 7faa7928c740 20 client.15552 populate_metadata read hostname 'smithi111' 2024-09-15T22:52:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.363+0000 7faa7928c740 1 --2- 172.21.15.111:0/1714303873 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5559631be9e0 0x5559631c0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:52:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.363+0000 7faa7928c740 4 client.15552 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:52:50.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.363+0000 7faa7928c740 1 -- 172.21.15.111:0/1714303873 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5559630aab80 con 0x5559631be9e0 2024-09-15T22:52:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.363+0000 7faa6f400640 1 --2- 172.21.15.111:0/1714303873 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5559631be9e0 0x5559631c0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:52:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.363+0000 7faa59600640 10 client.15552 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:52:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.363+0000 7faa6f400640 1 --2- 172.21.15.111:0/1714303873 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5559631be9e0 0x5559631c0dd0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dceffec0ace2bb server_cookie=f1be21ef2c56e630 in_seq=0 out_seq=0 2024-09-15T22:52:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.386+0000 7faa59600640 1 -- 172.21.15.111:0/1714303873 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5559630aab80 con 0x5559631be9e0 2024-09-15T22:52:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.386+0000 7faa59600640 10 client.15552 handle_command_reply: tid=0 2024-09-15T22:52:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.386+0000 7faa7928c740 1 client.15552 shutdown 2024-09-15T22:52:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.386+0000 7faa7928c740 1 -- 172.21.15.111:0/1714303873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313e260 msgr2=0x55596314cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.386+0000 7faa7928c740 1 --2- 172.21.15.111:0/1714303873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313e260 0x55596314cca0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009900 tx=0x7faa5c0045f0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.386+0000 7faa7928c740 1 -- 172.21.15.111:0/1714303873 shutdown_connections 2024-09-15T22:52:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.386+0000 7faa7928c740 1 --2- 172.21.15.111:0/1714303873 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5559631be9e0 0x5559631c0dd0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.386+0000 7faa7928c740 1 --2- 172.21.15.111:0/1714303873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555963143c30 0x55596314e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.386+0000 7faa7928c740 1 --2- 172.21.15.111:0/1714303873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55596313eba0 0x55596314d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.386+0000 7faa59600640 0 client.15552 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:52:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.386+0000 7faa7928c740 1 --2- 172.21.15.111:0/1714303873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55596313e260 0x55596314cca0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.386+0000 7faa7928c740 1 -- 172.21.15.111:0/1714303873 >> 172.21.15.111:0/1714303873 conn(0x5559631398d0 msgr2=0x555963147fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:52:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.386+0000 7faa7928c740 1 -- 172.21.15.111:0/1714303873 shutdown_connections 2024-09-15T22:52:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.386+0000 7faa7928c740 1 -- 172.21.15.111:0/1714303873 wait complete. 2024-09-15T22:52:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.386+0000 7faa7928c740 20 client.15552 trim_cache size 0 max 16384 2024-09-15T22:52:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:52:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:52:50.392 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:52:50.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:52:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.386+0000 7faa74e00640 1 -- 172.21.15.111:0/36626669 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faa540a32c0 msgr2=0x7faa540a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.386+0000 7faa74e00640 1 --2- 172.21.15.111:0/36626669 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faa540a32c0 0x7faa540a5780 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7faa68004af0 tx=0x7faa68004a40 comp rx=0 tx=0).stop 2024-09-15T22:52:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.386+0000 7faa74e00640 1 -- 172.21.15.111:0/36626669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa70107630 msgr2=0x7faa701a1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:52:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.386+0000 7faa74e00640 1 --2- 172.21.15.111:0/36626669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa70107630 0x7faa701a1560 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7faa6800eb40 tx=0x7faa6800c2d0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.386+0000 7faa74e00640 1 -- 172.21.15.111:0/36626669 shutdown_connections 2024-09-15T22:52:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.386+0000 7faa74e00640 1 --2- 172.21.15.111:0/36626669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa70107630 0x7faa701a1560 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.386+0000 7faa74e00640 1 --2- 172.21.15.111:0/36626669 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faa540a32c0 0x7faa540a5780 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.386+0000 7faa74e00640 1 --2- 172.21.15.111:0/36626669 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa70104d00 0x7faa7019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.386+0000 7faa74e00640 1 --2- 172.21.15.111:0/36626669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa701023d0 0x7faa70199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:52:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.386+0000 7faa74e00640 1 -- 172.21.15.111:0/36626669 >> 172.21.15.111:0/36626669 conn(0x7faa700fbfb0 msgr2=0x7faa701018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:52:50.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.386+0000 7faa74e00640 1 -- 172.21.15.111:0/36626669 shutdown_connections 2024-09-15T22:52:50.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:52:50.386+0000 7faa74e00640 1 -- 172.21.15.111:0/36626669 wait complete. 2024-09-15T22:52:51.265 INFO:tasks.workunit.client.0.smithi111.stdout: 1 598403 9.85 MB/sec execute 199 sec latency 1286.974 ms 2024-09-15T22:52:52.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:51 smithi192 ceph-mon[35427]: pgmap v558: 97 pgs: 97 active+clean; 222 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 203 op/s 2024-09-15T22:52:52.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:51 smithi121 ceph-mon[34815]: pgmap v558: 97 pgs: 97 active+clean; 222 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 203 op/s 2024-09-15T22:52:52.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:51 smithi111 ceph-mon[30629]: pgmap v558: 97 pgs: 97 active+clean; 222 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 203 op/s 2024-09-15T22:52:52.267 INFO:tasks.workunit.client.0.smithi111.stdout: 1 600921 9.87 MB/sec execute 200 sec latency 1297.883 ms 2024-09-15T22:52:53.265 INFO:tasks.workunit.client.0.smithi111.stdout: 1 604062 9.91 MB/sec execute 201 sec latency 13.381 ms 2024-09-15T22:52:53.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:53 smithi111 ceph-mon[30629]: pgmap v559: 97 pgs: 97 active+clean; 223 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 197 op/s 2024-09-15T22:52:54.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:53 smithi192 ceph-mon[35427]: pgmap v559: 97 pgs: 97 active+clean; 223 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 197 op/s 2024-09-15T22:52:54.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:53 smithi121 ceph-mon[34815]: pgmap v559: 97 pgs: 97 active+clean; 223 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 197 op/s 2024-09-15T22:52:54.265 INFO:tasks.workunit.client.0.smithi111.stdout: 1 606977 9.93 MB/sec execute 202 sec latency 15.389 ms 2024-09-15T22:52:55.265 INFO:tasks.workunit.client.0.smithi111.stdout: 1 608066 9.91 MB/sec execute 203 sec latency 395.619 ms 2024-09-15T22:52:55.992 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:55 smithi111 ceph-mon[30629]: pgmap v560: 97 pgs: 97 active+clean; 223 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 188 op/s 2024-09-15T22:52:56.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:55 smithi192 ceph-mon[35427]: pgmap v560: 97 pgs: 97 active+clean; 223 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 188 op/s 2024-09-15T22:52:56.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:55 smithi121 ceph-mon[34815]: pgmap v560: 97 pgs: 97 active+clean; 223 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 188 op/s 2024-09-15T22:52:56.265 INFO:tasks.workunit.client.0.smithi111.stdout: 1 608066 9.86 MB/sec execute 204 sec latency 1395.680 ms 2024-09-15T22:52:57.265 INFO:tasks.workunit.client.0.smithi111.stdout: 1 610318 9.87 MB/sec execute 205 sec latency 1734.409 ms 2024-09-15T22:52:58.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:57 smithi192 ceph-mon[35427]: pgmap v561: 97 pgs: 97 active+clean; 222 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 191 op/s 2024-09-15T22:52:58.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:57 smithi121 ceph-mon[34815]: pgmap v561: 97 pgs: 97 active+clean; 222 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 191 op/s 2024-09-15T22:52:58.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:57 smithi111 ceph-mon[30629]: pgmap v561: 97 pgs: 97 active+clean; 222 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 191 op/s 2024-09-15T22:52:58.265 INFO:tasks.workunit.client.0.smithi111.stdout: 1 612586 9.88 MB/sec execute 206 sec latency 13.029 ms 2024-09-15T22:52:59.265 INFO:tasks.workunit.client.0.smithi111.stdout: 1 615056 9.90 MB/sec execute 207 sec latency 13.260 ms 2024-09-15T22:53:00.017 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:52:59 smithi111 ceph-mon[30629]: pgmap v562: 97 pgs: 97 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 210 op/s 2024-09-15T22:53:00.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:52:59 smithi192 ceph-mon[35427]: pgmap v562: 97 pgs: 97 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 210 op/s 2024-09-15T22:53:00.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:52:59 smithi121 ceph-mon[34815]: pgmap v562: 97 pgs: 97 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 210 op/s 2024-09-15T22:53:00.265 INFO:tasks.workunit.client.0.smithi111.stdout: 1 617434 9.91 MB/sec execute 208 sec latency 277.242 ms 2024-09-15T22:53:01.266 INFO:tasks.workunit.client.0.smithi111.stdout: 1 617436 9.87 MB/sec execute 209 sec latency 906.332 ms 2024-09-15T22:53:02.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:01 smithi192 ceph-mon[35427]: pgmap v563: 97 pgs: 97 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 177 op/s 2024-09-15T22:53:02.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:01 smithi121 ceph-mon[34815]: pgmap v563: 97 pgs: 97 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 177 op/s 2024-09-15T22:53:02.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:01 smithi111 ceph-mon[30629]: pgmap v563: 97 pgs: 97 active+clean; 228 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 177 op/s 2024-09-15T22:53:02.266 INFO:tasks.workunit.client.0.smithi111.stdout: 1 618896 9.85 MB/sec execute 210 sec latency 1137.241 ms 2024-09-15T22:53:03.266 INFO:tasks.workunit.client.0.smithi111.stdout: 1 621929 9.89 MB/sec execute 211 sec latency 13.178 ms 2024-09-15T22:53:03.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.520+0000 7fd871c00640 1 -- 172.21.15.111:0/780565235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd86c105910 msgr2=0x7fd86c10c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:03.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.520+0000 7fd871c00640 1 --2- 172.21.15.111:0/780565235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd86c105910 0x7fd86c10c1a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd864009ca0 tx=0x7fd86402a8b0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.523 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.520+0000 7fd871c00640 1 -- 172.21.15.111:0/780565235 shutdown_connections 2024-09-15T22:53:03.523 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.520+0000 7fd871c00640 1 --2- 172.21.15.111:0/780565235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd86c105910 0x7fd86c10c1a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.523 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.520+0000 7fd871c00640 1 --2- 172.21.15.111:0/780565235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd86c104f50 0x7fd86c1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.523 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.520+0000 7fd871c00640 1 --2- 172.21.15.111:0/780565235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd86c103d60 0x7fd86c104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.523 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.520+0000 7fd871c00640 1 -- 172.21.15.111:0/780565235 >> 172.21.15.111:0/780565235 conn(0x7fd86c0ff4e0 msgr2=0x7fd86c101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:03.523 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.521+0000 7fd871c00640 1 -- 172.21.15.111:0/780565235 shutdown_connections 2024-09-15T22:53:03.523 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.521+0000 7fd871c00640 1 -- 172.21.15.111:0/780565235 wait complete. 2024-09-15T22:53:03.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.521+0000 7fd871c00640 1 Processor -- start 2024-09-15T22:53:03.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.522+0000 7fd871c00640 1 -- start start 2024-09-15T22:53:03.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.522+0000 7fd871c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd86c103d60 0x7fd86c199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:03.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.522+0000 7fd871c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd86c104f50 0x7fd86c19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:03.525 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.522+0000 7fd871c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd86c105910 0x7fd86c1a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.522+0000 7fd871c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd86c10fa10 con 0x7fd86c104f50 2024-09-15T22:53:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.522+0000 7fd871c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd86c10f890 con 0x7fd86c103d60 2024-09-15T22:53:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.522+0000 7fd871c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd86c10fb90 con 0x7fd86c105910 2024-09-15T22:53:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.522+0000 7fd86aa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd86c103d60 0x7fd86c199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.522+0000 7fd86aa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd86c103d60 0x7fd86c199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:59682/0 (socket says 172.21.15.111:59682) 2024-09-15T22:53:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.522+0000 7fd86aa00640 1 -- 172.21.15.111:0/1330582721 learned_addr learned my addr 172.21.15.111:0/1330582721 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:53:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.522+0000 7fd86b400640 1 --2- 172.21.15.111:0/1330582721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd86c105910 0x7fd86c1a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.522+0000 7fd86a000640 1 --2- 172.21.15.111:0/1330582721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd86c104f50 0x7fd86c19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.522+0000 7fd86aa00640 1 -- 172.21.15.111:0/1330582721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd86c105910 msgr2=0x7fd86c1a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.522+0000 7fd86aa00640 1 --2- 172.21.15.111:0/1330582721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd86c105910 0x7fd86c1a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.522+0000 7fd86aa00640 1 -- 172.21.15.111:0/1330582721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd86c104f50 msgr2=0x7fd86c19a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.522+0000 7fd86aa00640 1 --2- 172.21.15.111:0/1330582721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd86c104f50 0x7fd86c19a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.526 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.522+0000 7fd86aa00640 1 -- 172.21.15.111:0/1330582721 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86c1a1bf0 con 0x7fd86c103d60 2024-09-15T22:53:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.522+0000 7fd86a000640 1 --2- 172.21.15.111:0/1330582721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd86c104f50 0x7fd86c19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:53:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.523+0000 7fd86aa00640 1 --2- 172.21.15.111:0/1330582721 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd86c103d60 0x7fd86c199fd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00e9c0 tx=0x7fd85c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.523+0000 7fd863400640 1 -- 172.21.15.111:0/1330582721 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c00cde0 con 0x7fd86c103d60 2024-09-15T22:53:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.523+0000 7fd863400640 1 -- 172.21.15.111:0/1330582721 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd85c010910 con 0x7fd86c103d60 2024-09-15T22:53:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.523+0000 7fd871c00640 1 -- 172.21.15.111:0/1330582721 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd86c1a1ee0 con 0x7fd86c103d60 2024-09-15T22:53:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.523+0000 7fd863400640 1 -- 172.21.15.111:0/1330582721 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c018a00 con 0x7fd86c103d60 2024-09-15T22:53:03.527 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.523+0000 7fd871c00640 1 -- 172.21.15.111:0/1330582721 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd86c1a2340 con 0x7fd86c103d60 2024-09-15T22:53:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.526+0000 7fd874a72740 1 Processor -- start 2024-09-15T22:53:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.526+0000 7fd863400640 1 -- 172.21.15.111:0/1330582721 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd85c01f020 con 0x7fd86c103d60 2024-09-15T22:53:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.526+0000 7fd874a72740 1 -- start start 2024-09-15T22:53:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.526+0000 7fd874a72740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556505d9d090 0x556505d9d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.526+0000 7fd874a72740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556505d9d9d0 0x556505da2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:03.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.526+0000 7fd874a72740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556505da2a60 0x556505da4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.527+0000 7fd874a72740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556505d02100 con 0x556505da2a60 2024-09-15T22:53:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.527+0000 7fd874a72740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556505caa1b0 con 0x556505d9d9d0 2024-09-15T22:53:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.527+0000 7fd874a72740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556505cfe0c0 con 0x556505d9d090 2024-09-15T22:53:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.527+0000 7fd86b400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556505d9d090 0x556505d9d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.527+0000 7fd86b400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556505d9d090 0x556505d9d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:57368/0 (socket says 172.21.15.111:57368) 2024-09-15T22:53:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.527+0000 7fd86b400640 1 -- 172.21.15.111:0/3613248768 learned_addr learned my addr 172.21.15.111:0/3613248768 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:53:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.527+0000 7fd86aa00640 1 --2- 172.21.15.111:0/3613248768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556505d9d9d0 0x556505da2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.527+0000 7fd86a000640 1 --2- 172.21.15.111:0/3613248768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556505da2a60 0x556505da4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.527+0000 7fd86b400640 1 -- 172.21.15.111:0/3613248768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556505d9d9d0 msgr2=0x556505da2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.527+0000 7fd86b400640 1 --2- 172.21.15.111:0/3613248768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556505d9d9d0 0x556505da2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.527+0000 7fd86b400640 1 -- 172.21.15.111:0/3613248768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556505da2a60 msgr2=0x556505da4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.527+0000 7fd86b400640 1 --2- 172.21.15.111:0/3613248768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556505da2a60 0x556505da4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.527+0000 7fd86b400640 1 -- 172.21.15.111:0/3613248768 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556505a15210 con 0x556505d9d090 2024-09-15T22:53:03.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.527+0000 7fd86a000640 1 --2- 172.21.15.111:0/3613248768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556505da2a60 0x556505da4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:53:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.527+0000 7fd86b400640 1 --2- 172.21.15.111:0/3613248768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556505d9d090 0x556505d9d490 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd864009ca0 tx=0x7fd864009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a1c2de0d7a0b8a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.528+0000 7fd857e00640 1 -- 172.21.15.111:0/3613248768 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864030a30 con 0x556505d9d090 2024-09-15T22:53:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.528+0000 7fd857e00640 1 -- 172.21.15.111:0/3613248768 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd864036030 con 0x556505d9d090 2024-09-15T22:53:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.528+0000 7fd857e00640 1 -- 172.21.15.111:0/3613248768 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8640045e0 con 0x556505d9d090 2024-09-15T22:53:03.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.528+0000 7fd863400640 1 --2- 172.21.15.111:0/1330582721 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd8500a32f0 0x7fd8500a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.528+0000 7fd863400640 1 -- 172.21.15.111:0/1330582721 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fd85c0cfbb0 con 0x7fd86c103d60 2024-09-15T22:53:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.528+0000 7fd86aa00640 1 --2- 172.21.15.111:0/1330582721 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd8500a32f0 0x7fd8500a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.529+0000 7fd874a72740 1 -- 172.21.15.111:0/3613248768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556505d9d090 msgr2=0x556505d9d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:03.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.529+0000 7fd874a72740 1 --2- 172.21.15.111:0/3613248768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556505d9d090 0x556505d9d490 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd864009ca0 tx=0x7fd864009a20 comp rx=0 tx=0).stop 2024-09-15T22:53:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.529+0000 7fd874a72740 1 -- 172.21.15.111:0/3613248768 shutdown_connections 2024-09-15T22:53:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.529+0000 7fd874a72740 1 --2- 172.21.15.111:0/3613248768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556505da2a60 0x556505da4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.529+0000 7fd874a72740 1 --2- 172.21.15.111:0/3613248768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556505d9d9d0 0x556505da2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.529+0000 7fd874a72740 1 --2- 172.21.15.111:0/3613248768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556505d9d090 0x556505d9d490 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.529+0000 7fd874a72740 1 -- 172.21.15.111:0/3613248768 >> 172.21.15.111:0/3613248768 conn(0x556505d98700 msgr2=0x556505d98b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.529+0000 7fd874a72740 1 -- 172.21.15.111:0/3613248768 shutdown_connections 2024-09-15T22:53:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.529+0000 7fd874a72740 1 -- 172.21.15.111:0/3613248768 wait complete. 2024-09-15T22:53:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.529+0000 7fd86aa00640 1 --2- 172.21.15.111:0/1330582721 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd8500a32f0 0x7fd8500a57b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd85c098ba0 tx=0x7fd85c0d35f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.529+0000 7fd874a72740 1 -- start start 2024-09-15T22:53:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.529+0000 7fd874a72740 1 Processor -- start 2024-09-15T22:53:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.530+0000 7fd874a72740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556505d9d090 0x556505da71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:03.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.530+0000 7fd874a72740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556505d9d9d0 0x556505da7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.530+0000 7fd874a72740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556505da2a60 0x556505dae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.530+0000 7fd874a72740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556505da59d0 con 0x556505d9d9d0 2024-09-15T22:53:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.530+0000 7fd874a72740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556505c8b090 con 0x556505d9d090 2024-09-15T22:53:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.530+0000 7fd874a72740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556505da5b50 con 0x556505da2a60 2024-09-15T22:53:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.530+0000 7fd874a72740 10 client.0 _ll_register_callbacks cb 0x556505cdff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:53:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.530+0000 7fd86b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556505d9d090 0x556505da71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.530+0000 7fd86a000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556505d9d9d0 0x556505da7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.530+0000 7fd86b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556505d9d090 0x556505da71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:59694/0 (socket says 172.21.15.111:59694) 2024-09-15T22:53:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.530+0000 7fd86a000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556505d9d9d0 0x556505da7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34552/0 (socket says 172.21.15.111:34552) 2024-09-15T22:53:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.530+0000 7fd86b400640 1 -- 172.21.15.111:0/389482746 learned_addr learned my addr 172.21.15.111:0/389482746 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:53:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.530+0000 7fd86b400640 1 --2- 172.21.15.111:0/389482746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556505da2a60 0x556505dae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:03.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.531+0000 7fd86a000640 1 -- 172.21.15.111:0/389482746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556505da2a60 msgr2=0x556505dae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.531+0000 7fd86a000640 1 --2- 172.21.15.111:0/389482746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556505da2a60 0x556505dae760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.531+0000 7fd86a000640 1 -- 172.21.15.111:0/389482746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556505d9d090 msgr2=0x556505da71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.531+0000 7fd86a000640 1 --2- 172.21.15.111:0/389482746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556505d9d090 0x556505da71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.531+0000 7fd86a000640 1 -- 172.21.15.111:0/389482746 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556505ae1620 con 0x556505d9d9d0 2024-09-15T22:53:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.531+0000 7fd86b400640 1 --2- 172.21.15.111:0/389482746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556505da2a60 0x556505dae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:53:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.531+0000 7fd86a000640 1 --2- 172.21.15.111:0/389482746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556505d9d9d0 0x556505da7720 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd86c19b4e0 tx=0x7fd858011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af61afcbaf3a171 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.531+0000 7fd854c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:53:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.531+0000 7fd854c00640 1 -- 172.21.15.111:0/389482746 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd858004230 con 0x556505d9d9d0 2024-09-15T22:53:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.531+0000 7fd874a72740 1 -- 172.21.15.111:0/389482746 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd864036030 con 0x556505d9d9d0 2024-09-15T22:53:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.531+0000 7fd854c00640 1 -- 172.21.15.111:0/389482746 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fd8580043d0 con 0x556505d9d9d0 2024-09-15T22:53:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.531+0000 7fd854c00640 1 -- 172.21.15.111:0/389482746 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd8580035f0 con 0x556505d9d9d0 2024-09-15T22:53:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.531+0000 7fd874a72740 10 client.15564 fetch_fsmap learned FSMap version 30 2024-09-15T22:53:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.531+0000 7fd874a72740 1 -- 172.21.15.111:0/389482746 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556505db1670 con 0x556505d9d9d0 2024-09-15T22:53:03.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.532+0000 7fd854c00640 1 -- 172.21.15.111:0/389482746 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd864036030 con 0x556505d9d9d0 2024-09-15T22:53:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.532+0000 7fd854c00640 1 -- 172.21.15.111:0/389482746 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fd85801b9d0 con 0x556505d9d9d0 2024-09-15T22:53:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.532+0000 7fd874a72740 10 client.15564 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:53:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.532+0000 7fd874a72740 10 client.15564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:53:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.532+0000 7fd874a72740 20 client.15564 populate_metadata read hostname 'smithi111' 2024-09-15T22:53:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.532+0000 7fd874a72740 1 --2- 172.21.15.111:0/389482746 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556505db2c80 0x556505db50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.532+0000 7fd874a72740 4 client.15564 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:53:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.532+0000 7fd874a72740 1 -- 172.21.15.111:0/389482746 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556505d0f830 con 0x556505db2c80 2024-09-15T22:53:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.532+0000 7fd86b400640 1 --2- 172.21.15.111:0/389482746 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556505db2c80 0x556505db50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.533+0000 7fd86b400640 1 --2- 172.21.15.111:0/389482746 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556505db2c80 0x556505db50b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ae28285ce45c0f8 server_cookie=d56c0bf87b5f0abd in_seq=0 out_seq=0 2024-09-15T22:53:03.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.533+0000 7fd854c00640 10 client.15564 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:53:03.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.533+0000 7fd854c00640 1 -- 172.21.15.111:0/389482746 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556505d0f830 con 0x556505db2c80 2024-09-15T22:53:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.533+0000 7fd854c00640 10 client.15564 handle_command_reply: tid=0 2024-09-15T22:53:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.533+0000 7fd874a72740 1 client.15564 shutdown 2024-09-15T22:53:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.534+0000 7fd874a72740 1 -- 172.21.15.111:0/389482746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556505d9d9d0 msgr2=0x556505da7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.534+0000 7fd874a72740 1 --2- 172.21.15.111:0/389482746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556505d9d9d0 0x556505da7720 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd86c19b4e0 tx=0x7fd858011810 comp rx=0 tx=0).stop 2024-09-15T22:53:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.534+0000 7fd874a72740 1 -- 172.21.15.111:0/389482746 shutdown_connections 2024-09-15T22:53:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.534+0000 7fd874a72740 1 --2- 172.21.15.111:0/389482746 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556505db2c80 0x556505db50b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.534+0000 7fd874a72740 1 --2- 172.21.15.111:0/389482746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556505da2a60 0x556505dae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.534+0000 7fd874a72740 1 --2- 172.21.15.111:0/389482746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556505d9d9d0 0x556505da7720 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.534+0000 7fd874a72740 1 --2- 172.21.15.111:0/389482746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556505d9d090 0x556505da71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.534+0000 7fd874a72740 1 -- 172.21.15.111:0/389482746 >> 172.21.15.111:0/389482746 conn(0x556505d98700 msgr2=0x556505d99f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.534+0000 7fd854c00640 0 client.15564 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:53:03.540 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.534+0000 7fd874a72740 1 -- 172.21.15.111:0/389482746 shutdown_connections 2024-09-15T22:53:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.534+0000 7fd874a72740 1 -- 172.21.15.111:0/389482746 wait complete. 2024-09-15T22:53:03.541 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.534+0000 7fd874a72740 20 client.15564 trim_cache size 0 max 16384 2024-09-15T22:53:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.551+0000 7fd874a72740 1 Processor -- start 2024-09-15T22:53:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.551+0000 7fd874a72740 1 -- start start 2024-09-15T22:53:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.551+0000 7fd874a72740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556505d9d090 0x556505d97770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.551+0000 7fd874a72740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556505d9d9d0 0x556505dae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.551+0000 7fd874a72740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556505da2a60 0x556505d99af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.551+0000 7fd874a72740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd86c1ad980 con 0x556505d9d090 2024-09-15T22:53:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.551+0000 7fd874a72740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd86c1ab250 con 0x556505d9d9d0 2024-09-15T22:53:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.551+0000 7fd874a72740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd86c1a90a0 con 0x556505da2a60 2024-09-15T22:53:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.552+0000 7fd86a000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556505d9d9d0 0x556505dae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.552+0000 7fd86a000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556505d9d9d0 0x556505dae720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:59704/0 (socket says 172.21.15.111:59704) 2024-09-15T22:53:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.552+0000 7fd86a000640 1 -- 172.21.15.111:0/2057003936 learned_addr learned my addr 172.21.15.111:0/2057003936 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:53:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.552+0000 7fd86b400640 1 --2- 172.21.15.111:0/2057003936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556505d9d090 0x556505d97770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.552+0000 7fd86b400640 1 --2- 172.21.15.111:0/2057003936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556505da2a60 0x556505d99af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.553+0000 7fd86b400640 1 -- 172.21.15.111:0/2057003936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556505da2a60 msgr2=0x556505d99af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.553+0000 7fd86b400640 1 --2- 172.21.15.111:0/2057003936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556505da2a60 0x556505d99af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.553+0000 7fd86b400640 1 -- 172.21.15.111:0/2057003936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556505d9d9d0 msgr2=0x556505dae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.553+0000 7fd86b400640 1 --2- 172.21.15.111:0/2057003936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556505d9d9d0 0x556505dae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.553+0000 7fd86b400640 1 -- 172.21.15.111:0/2057003936 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556505d93d60 con 0x556505d9d090 2024-09-15T22:53:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.553+0000 7fd86b400640 1 --2- 172.21.15.111:0/2057003936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556505d9d090 0x556505d97770 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd8640096c0 tx=0x7fd86402a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcabd50bb7b42c50 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.553+0000 7fd857400640 1 -- 172.21.15.111:0/2057003936 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864030620 con 0x556505d9d090 2024-09-15T22:53:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.553+0000 7fd857400640 1 -- 172.21.15.111:0/2057003936 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd864036040 con 0x556505d9d090 2024-09-15T22:53:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.553+0000 7fd857400640 1 -- 172.21.15.111:0/2057003936 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8640045e0 con 0x556505d9d090 2024-09-15T22:53:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.554+0000 7fd874a72740 1 -- 172.21.15.111:0/2057003936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556505d9d090 msgr2=0x556505d97770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.554+0000 7fd874a72740 1 --2- 172.21.15.111:0/2057003936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556505d9d090 0x556505d97770 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd8640096c0 tx=0x7fd86402a8b0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.554+0000 7fd874a72740 1 -- 172.21.15.111:0/2057003936 shutdown_connections 2024-09-15T22:53:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.554+0000 7fd874a72740 1 --2- 172.21.15.111:0/2057003936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556505da2a60 0x556505d99af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.554+0000 7fd874a72740 1 --2- 172.21.15.111:0/2057003936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556505d9d9d0 0x556505dae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.554+0000 7fd874a72740 1 --2- 172.21.15.111:0/2057003936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556505d9d090 0x556505d97770 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.554+0000 7fd874a72740 1 -- 172.21.15.111:0/2057003936 >> 172.21.15.111:0/2057003936 conn(0x556505d98700 msgr2=0x556505d97060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.554+0000 7fd874a72740 1 -- 172.21.15.111:0/2057003936 shutdown_connections 2024-09-15T22:53:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.554+0000 7fd874a72740 1 -- 172.21.15.111:0/2057003936 wait complete. 2024-09-15T22:53:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.555+0000 7fd874a72740 1 -- start start 2024-09-15T22:53:03.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.555+0000 7fd874a72740 1 Processor -- start 2024-09-15T22:53:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.555+0000 7fd874a72740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556505d9d090 0x556505daa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.555+0000 7fd874a72740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556505d9d9d0 0x556505daa560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.555+0000 7fd874a72740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556505da2a60 0x556505dad530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.555+0000 7fd874a72740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd86c1a9710 con 0x556505d9d9d0 2024-09-15T22:53:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.555+0000 7fd874a72740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd86c1aab80 con 0x556505da2a60 2024-09-15T22:53:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.555+0000 7fd874a72740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd86c1a9de0 con 0x556505d9d090 2024-09-15T22:53:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.555+0000 7fd874a72740 10 client.0 _ll_register_callbacks cb 0x7fd85800da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:53:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.556+0000 7fd86b400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556505d9d090 0x556505daa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:03.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.556+0000 7fd86a000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556505d9d9d0 0x556505daa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.556+0000 7fd86b400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556505d9d090 0x556505daa020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:57412/0 (socket says 172.21.15.111:57412) 2024-09-15T22:53:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.556+0000 7fd86b400640 1 -- 172.21.15.111:0/2805732195 learned_addr learned my addr 172.21.15.111:0/2805732195 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:53:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.556+0000 7fd86b400640 1 --2- 172.21.15.111:0/2805732195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556505da2a60 0x556505dad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.556+0000 7fd86a000640 1 -- 172.21.15.111:0/2805732195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556505d9d090 msgr2=0x556505daa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.556+0000 7fd86a000640 1 --2- 172.21.15.111:0/2805732195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556505d9d090 0x556505daa020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.556+0000 7fd86a000640 1 -- 172.21.15.111:0/2805732195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556505da2a60 msgr2=0x556505dad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.556+0000 7fd86a000640 1 --2- 172.21.15.111:0/2805732195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556505da2a60 0x556505dad530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.556+0000 7fd86a000640 1 -- 172.21.15.111:0/2805732195 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556505dadb40 con 0x556505d9d9d0 2024-09-15T22:53:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.556+0000 7fd86b400640 1 --2- 172.21.15.111:0/2805732195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556505d9d090 0x556505daa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:53:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.556+0000 7fd86a000640 1 --2- 172.21.15.111:0/2805732195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556505d9d9d0 0x556505daa560 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x556505d8cef0 tx=0x7fd858025630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=174eae25ffdc228b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.556+0000 7fd854c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:53:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.557+0000 7fd854c00640 1 -- 172.21.15.111:0/2805732195 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8580055a0 con 0x556505d9d9d0 2024-09-15T22:53:03.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.557+0000 7fd854c00640 1 -- 172.21.15.111:0/2805732195 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fd858025870 con 0x556505d9d9d0 2024-09-15T22:53:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.557+0000 7fd874a72740 1 -- 172.21.15.111:0/2805732195 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd864036040 con 0x556505d9d9d0 2024-09-15T22:53:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.557+0000 7fd854c00640 1 -- 172.21.15.111:0/2805732195 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd858025a70 con 0x556505d9d9d0 2024-09-15T22:53:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.557+0000 7fd874a72740 10 client.15570 fetch_fsmap learned FSMap version 30 2024-09-15T22:53:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.557+0000 7fd874a72740 1 -- 172.21.15.111:0/2805732195 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556505e263e0 con 0x556505d9d9d0 2024-09-15T22:53:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.557+0000 7fd854c00640 1 -- 172.21.15.111:0/2805732195 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd864036040 con 0x556505d9d9d0 2024-09-15T22:53:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.557+0000 7fd854c00640 1 -- 172.21.15.111:0/2805732195 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fd858037050 con 0x556505d9d9d0 2024-09-15T22:53:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.558+0000 7fd874a72740 10 client.15570 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:53:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.558+0000 7fd874a72740 10 client.15570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:53:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.558+0000 7fd874a72740 20 client.15570 populate_metadata read hostname 'smithi111' 2024-09-15T22:53:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.558+0000 7fd874a72740 1 --2- 172.21.15.111:0/2805732195 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556505e26ce0 0x556505e29100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.558+0000 7fd874a72740 4 client.15570 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:53:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.558+0000 7fd874a72740 1 -- 172.21.15.111:0/2805732195 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556505d09b80 con 0x556505e26ce0 2024-09-15T22:53:03.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.558+0000 7fd86b400640 1 --2- 172.21.15.111:0/2805732195 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556505e26ce0 0x556505e29100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.558+0000 7fd86b400640 1 --2- 172.21.15.111:0/2805732195 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556505e26ce0 0x556505e29100 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=319f56426133acec server_cookie=841e4588ae574605 in_seq=0 out_seq=0 2024-09-15T22:53:03.563 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.558+0000 7fd854c00640 10 client.15570 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:53:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.567+0000 7fd854c00640 1 -- 172.21.15.111:0/2805732195 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556505d09b80 con 0x556505e26ce0 2024-09-15T22:53:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.567+0000 7fd854c00640 10 client.15570 handle_command_reply: tid=0 2024-09-15T22:53:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.567+0000 7fd874a72740 1 client.15570 shutdown 2024-09-15T22:53:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.567+0000 7fd874a72740 1 -- 172.21.15.111:0/2805732195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556505d9d9d0 msgr2=0x556505daa560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.567+0000 7fd874a72740 1 --2- 172.21.15.111:0/2805732195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556505d9d9d0 0x556505daa560 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x556505d8cef0 tx=0x7fd858025630 comp rx=0 tx=0).stop 2024-09-15T22:53:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.567+0000 7fd874a72740 1 -- 172.21.15.111:0/2805732195 shutdown_connections 2024-09-15T22:53:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.567+0000 7fd874a72740 1 --2- 172.21.15.111:0/2805732195 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556505e26ce0 0x556505e29100 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.567+0000 7fd874a72740 1 --2- 172.21.15.111:0/2805732195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556505da2a60 0x556505dad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.567+0000 7fd874a72740 1 --2- 172.21.15.111:0/2805732195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556505d9d9d0 0x556505daa560 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.567+0000 7fd854c00640 0 client.15570 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:53:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.567+0000 7fd874a72740 1 --2- 172.21.15.111:0/2805732195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556505d9d090 0x556505daa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.567+0000 7fd874a72740 1 -- 172.21.15.111:0/2805732195 >> 172.21.15.111:0/2805732195 conn(0x556505d98700 msgr2=0x556505d97880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.567+0000 7fd874a72740 1 -- 172.21.15.111:0/2805732195 shutdown_connections 2024-09-15T22:53:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.567+0000 7fd874a72740 1 -- 172.21.15.111:0/2805732195 wait complete. 2024-09-15T22:53:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.567+0000 7fd874a72740 20 client.15570 trim_cache size 0 max 16384 2024-09-15T22:53:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:53:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:53:03.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:53:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:53:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.568+0000 7fd871c00640 1 -- 172.21.15.111:0/1330582721 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd8500a32f0 msgr2=0x7fd8500a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.568+0000 7fd871c00640 1 --2- 172.21.15.111:0/1330582721 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd8500a32f0 0x7fd8500a57b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd85c098ba0 tx=0x7fd85c0d35f0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.568+0000 7fd871c00640 1 -- 172.21.15.111:0/1330582721 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd86c103d60 msgr2=0x7fd86c199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.568+0000 7fd871c00640 1 --2- 172.21.15.111:0/1330582721 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd86c103d60 0x7fd86c199fd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00e9c0 tx=0x7fd85c00ee80 comp rx=0 tx=0).stop 2024-09-15T22:53:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.568+0000 7fd871c00640 1 -- 172.21.15.111:0/1330582721 shutdown_connections 2024-09-15T22:53:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.568+0000 7fd871c00640 1 --2- 172.21.15.111:0/1330582721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd86c105910 0x7fd86c1a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.568+0000 7fd871c00640 1 --2- 172.21.15.111:0/1330582721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd86c104f50 0x7fd86c19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.568+0000 7fd871c00640 1 --2- 172.21.15.111:0/1330582721 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd8500a32f0 0x7fd8500a57b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.568+0000 7fd871c00640 1 --2- 172.21.15.111:0/1330582721 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd86c103d60 0x7fd86c199fd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.568+0000 7fd871c00640 1 -- 172.21.15.111:0/1330582721 >> 172.21.15.111:0/1330582721 conn(0x7fd86c0ff4e0 msgr2=0x7fd86c108180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.568+0000 7fd871c00640 1 -- 172.21.15.111:0/1330582721 shutdown_connections 2024-09-15T22:53:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:03.568+0000 7fd871c00640 1 -- 172.21.15.111:0/1330582721 wait complete. 2024-09-15T22:53:04.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:03 smithi192 ceph-mon[35427]: pgmap v564: 97 pgs: 97 active+clean; 226 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 223 op/s 2024-09-15T22:53:04.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:03 smithi121 ceph-mon[34815]: pgmap v564: 97 pgs: 97 active+clean; 226 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 223 op/s 2024-09-15T22:53:04.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:03 smithi111 ceph-mon[30629]: pgmap v564: 97 pgs: 97 active+clean; 226 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 223 op/s 2024-09-15T22:53:04.266 INFO:tasks.workunit.client.0.smithi111.stdout: 1 624897 9.92 MB/sec execute 212 sec latency 13.334 ms 2024-09-15T22:53:05.269 INFO:tasks.workunit.client.0.smithi111.stdout: 1 626783 9.92 MB/sec execute 213 sec latency 130.908 ms 2024-09-15T22:53:05.972 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:05 smithi111 ceph-mon[30629]: pgmap v565: 97 pgs: 97 active+clean; 226 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 181 op/s 2024-09-15T22:53:06.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:05 smithi192 ceph-mon[35427]: pgmap v565: 97 pgs: 97 active+clean; 226 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 181 op/s 2024-09-15T22:53:06.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:05 smithi121 ceph-mon[34815]: pgmap v565: 97 pgs: 97 active+clean; 226 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 181 op/s 2024-09-15T22:53:06.269 INFO:tasks.workunit.client.0.smithi111.stdout: 1 626841 9.87 MB/sec execute 214 sec latency 746.034 ms 2024-09-15T22:53:07.270 INFO:tasks.workunit.client.0.smithi111.stdout: 1 628413 9.87 MB/sec execute 215 sec latency 1337.897 ms 2024-09-15T22:53:08.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:07 smithi192 ceph-mon[35427]: pgmap v566: 97 pgs: 97 active+clean; 229 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 181 op/s 2024-09-15T22:53:08.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:07 smithi121 ceph-mon[34815]: pgmap v566: 97 pgs: 97 active+clean; 229 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 181 op/s 2024-09-15T22:53:08.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:07 smithi111 ceph-mon[30629]: pgmap v566: 97 pgs: 97 active+clean; 229 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 181 op/s 2024-09-15T22:53:08.270 INFO:tasks.workunit.client.0.smithi111.stdout: 1 630956 9.87 MB/sec execute 216 sec latency 18.334 ms 2024-09-15T22:53:09.270 INFO:tasks.workunit.client.0.smithi111.stdout: 1 633353 9.89 MB/sec execute 217 sec latency 13.639 ms 2024-09-15T22:53:10.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:09 smithi192 ceph-mon[35427]: pgmap v567: 97 pgs: 97 active+clean; 232 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 211 op/s 2024-09-15T22:53:10.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:09 smithi121 ceph-mon[34815]: pgmap v567: 97 pgs: 97 active+clean; 232 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 211 op/s 2024-09-15T22:53:10.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:09 smithi111 ceph-mon[30629]: pgmap v567: 97 pgs: 97 active+clean; 232 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 211 op/s 2024-09-15T22:53:10.270 INFO:tasks.workunit.client.0.smithi111.stdout: 1 636266 9.93 MB/sec execute 218 sec latency 9.946 ms 2024-09-15T22:53:11.270 INFO:tasks.workunit.client.0.smithi111.stdout: 1 636266 9.88 MB/sec execute 219 sec latency 1009.994 ms 2024-09-15T22:53:12.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:11 smithi192 ceph-mon[35427]: pgmap v568: 97 pgs: 97 active+clean; 235 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 206 op/s 2024-09-15T22:53:12.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:11 smithi121 ceph-mon[34815]: pgmap v568: 97 pgs: 97 active+clean; 235 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 206 op/s 2024-09-15T22:53:12.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:11 smithi111 ceph-mon[30629]: pgmap v568: 97 pgs: 97 active+clean; 235 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 206 op/s 2024-09-15T22:53:12.270 INFO:tasks.workunit.client.0.smithi111.stdout: 1 638317 9.87 MB/sec execute 220 sec latency 1318.441 ms 2024-09-15T22:53:13.270 INFO:tasks.workunit.client.0.smithi111.stdout: 1 640666 9.90 MB/sec execute 221 sec latency 17.208 ms 2024-09-15T22:53:14.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:13 smithi192 ceph-mon[35427]: pgmap v569: 97 pgs: 97 active+clean; 233 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 216 op/s 2024-09-15T22:53:14.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:13 smithi121 ceph-mon[34815]: pgmap v569: 97 pgs: 97 active+clean; 233 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 216 op/s 2024-09-15T22:53:14.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:13 smithi111 ceph-mon[30629]: pgmap v569: 97 pgs: 97 active+clean; 233 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 216 op/s 2024-09-15T22:53:14.270 INFO:tasks.workunit.client.0.smithi111.stdout: 1 643405 9.92 MB/sec execute 222 sec latency 16.370 ms 2024-09-15T22:53:15.270 INFO:tasks.workunit.client.0.smithi111.stdout: 1 646581 9.96 MB/sec execute 223 sec latency 13.157 ms 2024-09-15T22:53:15.937 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:15 smithi111 ceph-mon[30629]: pgmap v570: 97 pgs: 97 active+clean; 236 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 200 op/s 2024-09-15T22:53:16.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:15 smithi192 ceph-mon[35427]: pgmap v570: 97 pgs: 97 active+clean; 236 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 200 op/s 2024-09-15T22:53:16.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:15 smithi121 ceph-mon[34815]: pgmap v570: 97 pgs: 97 active+clean; 236 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 200 op/s 2024-09-15T22:53:16.270 INFO:tasks.workunit.client.0.smithi111.stdout: 1 646691 9.92 MB/sec execute 224 sec latency 796.562 ms 2024-09-15T22:53:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.684+0000 7ffa5e400640 1 -- 172.21.15.111:0/571796013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa58074fc0 msgr2=0x7ffa58109750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.684+0000 7ffa5e400640 1 --2- 172.21.15.111:0/571796013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa58074fc0 0x7ffa58109750 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009900 tx=0x7ffa4402a400 comp rx=0 tx=0).stop 2024-09-15T22:53:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.685+0000 7ffa5e400640 1 -- 172.21.15.111:0/571796013 shutdown_connections 2024-09-15T22:53:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.685+0000 7ffa5e400640 1 --2- 172.21.15.111:0/571796013 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa58109c90 0x7ffa5810c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.685+0000 7ffa5e400640 1 --2- 172.21.15.111:0/571796013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa58074fc0 0x7ffa58109750 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.685+0000 7ffa5e400640 1 --2- 172.21.15.111:0/571796013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa58074680 0x7ffa58074a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.685+0000 7ffa5e400640 1 -- 172.21.15.111:0/571796013 >> 172.21.15.111:0/571796013 conn(0x7ffa580fd0d0 msgr2=0x7ffa580ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.685+0000 7ffa5e400640 1 -- 172.21.15.111:0/571796013 shutdown_connections 2024-09-15T22:53:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.685+0000 7ffa5e400640 1 -- 172.21.15.111:0/571796013 wait complete. 2024-09-15T22:53:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.685+0000 7ffa5e400640 1 Processor -- start 2024-09-15T22:53:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.686+0000 7ffa5e400640 1 -- start start 2024-09-15T22:53:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.686+0000 7ffa5e400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa58074680 0x7ffa58199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.686+0000 7ffa5e400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa58074fc0 0x7ffa5819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.686+0000 7ffa5e400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa58109c90 0x7ffa581a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.686+0000 7ffa5e400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5810f940 con 0x7ffa58074fc0 2024-09-15T22:53:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.686+0000 7ffa5e400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5810f7c0 con 0x7ffa58074680 2024-09-15T22:53:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.686+0000 7ffa5e400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5810fac0 con 0x7ffa58109c90 2024-09-15T22:53:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.686+0000 7ffa57e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa58074fc0 0x7ffa5819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.686+0000 7ffa57e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa58074fc0 0x7ffa5819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45124/0 (socket says 172.21.15.111:45124) 2024-09-15T22:53:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.686+0000 7ffa5d000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa58074680 0x7ffa58199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.686+0000 7ffa57e00640 1 -- 172.21.15.111:0/2174755346 learned_addr learned my addr 172.21.15.111:0/2174755346 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:53:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.686+0000 7ffa5da00640 1 --2- 172.21.15.111:0/2174755346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa58109c90 0x7ffa581a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.687+0000 7ffa5d000640 1 -- 172.21.15.111:0/2174755346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa58109c90 msgr2=0x7ffa581a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.687+0000 7ffa5d000640 1 --2- 172.21.15.111:0/2174755346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa58109c90 0x7ffa581a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.687+0000 7ffa5d000640 1 -- 172.21.15.111:0/2174755346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa58074fc0 msgr2=0x7ffa5819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.687+0000 7ffa5d000640 1 --2- 172.21.15.111:0/2174755346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa58074fc0 0x7ffa5819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.687+0000 7ffa5d000640 1 -- 172.21.15.111:0/2174755346 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa581a1bc0 con 0x7ffa58074680 2024-09-15T22:53:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.687+0000 7ffa57e00640 1 --2- 172.21.15.111:0/2174755346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa58074fc0 0x7ffa5819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:53:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.687+0000 7ffa5d000640 1 --2- 172.21.15.111:0/2174755346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa58074680 0x7ffa58199fa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c005fd0 tx=0x7ffa4c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.687+0000 7ffa55600640 1 -- 172.21.15.111:0/2174755346 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c009c30 con 0x7ffa58074680 2024-09-15T22:53:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.687+0000 7ffa55600640 1 -- 172.21.15.111:0/2174755346 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa4c009dd0 con 0x7ffa58074680 2024-09-15T22:53:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.687+0000 7ffa5e400640 1 -- 172.21.15.111:0/2174755346 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa581a1eb0 con 0x7ffa58074680 2024-09-15T22:53:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.687+0000 7ffa55600640 1 -- 172.21.15.111:0/2174755346 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c010460 con 0x7ffa58074680 2024-09-15T22:53:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.687+0000 7ffa5e400640 1 -- 172.21.15.111:0/2174755346 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa581a2310 con 0x7ffa58074680 2024-09-15T22:53:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.690+0000 7ffa55600640 1 -- 172.21.15.111:0/2174755346 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7ffa4c003ef0 con 0x7ffa58074680 2024-09-15T22:53:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.690+0000 7ffa62861740 1 Processor -- start 2024-09-15T22:53:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.691+0000 7ffa62861740 1 -- start start 2024-09-15T22:53:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.691+0000 7ffa62861740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9134a260 0x555b9134a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.691+0000 7ffa62861740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b9134aba0 0x555b9134f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.691+0000 7ffa62861740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b9134fc30 0x555b913520f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.691+0000 7ffa62861740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555b912af100 con 0x555b9134aba0 2024-09-15T22:53:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.691+0000 7ffa62861740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x555b912571b0 con 0x555b9134fc30 2024-09-15T22:53:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.691+0000 7ffa62861740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555b912ab0c0 con 0x555b9134a260 2024-09-15T22:53:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.691+0000 7ffa5d000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b9134aba0 0x555b9134f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.691+0000 7ffa57e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b9134fc30 0x555b913520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.691+0000 7ffa5d000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b9134aba0 0x555b9134f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45136/0 (socket says 172.21.15.111:45136) 2024-09-15T22:53:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.691+0000 7ffa5da00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9134a260 0x555b9134a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.691+0000 7ffa5d000640 1 -- 172.21.15.111:0/4169078305 learned_addr learned my addr 172.21.15.111:0/4169078305 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:53:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.692+0000 7ffa57e00640 1 -- 172.21.15.111:0/4169078305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9134a260 msgr2=0x555b9134a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.692+0000 7ffa57e00640 1 --2- 172.21.15.111:0/4169078305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9134a260 0x555b9134a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.692+0000 7ffa57e00640 1 -- 172.21.15.111:0/4169078305 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b9134aba0 msgr2=0x555b9134f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.692+0000 7ffa57e00640 1 --2- 172.21.15.111:0/4169078305 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b9134aba0 0x555b9134f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.692+0000 7ffa57e00640 1 -- 172.21.15.111:0/4169078305 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b90fc2210 con 0x555b9134fc30 2024-09-15T22:53:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.692+0000 7ffa55600640 1 --2- 172.21.15.111:0/2174755346 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ffa400a3240 0x7ffa400a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.692+0000 7ffa57e00640 1 --2- 172.21.15.111:0/4169078305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b9134fc30 0x555b913520f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffa44006010 tx=0x7ffa44004520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=347a4760bddffe2b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.692+0000 7ffa55600640 1 -- 172.21.15.111:0/2174755346 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7ffa4c0cf460 con 0x7ffa58074680 2024-09-15T22:53:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.692+0000 7ffa5da00640 1 --2- 172.21.15.111:0/2174755346 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ffa400a3240 0x7ffa400a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.692+0000 7ffa4a000640 1 -- 172.21.15.111:0/4169078305 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44040070 con 0x555b9134fc30 2024-09-15T22:53:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.692+0000 7ffa4a000640 1 -- 172.21.15.111:0/4169078305 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa440047a0 con 0x555b9134fc30 2024-09-15T22:53:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.692+0000 7ffa4a000640 1 -- 172.21.15.111:0/4169078305 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44035040 con 0x555b9134fc30 2024-09-15T22:53:16.707 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.693+0000 7ffa62861740 1 -- 172.21.15.111:0/4169078305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b9134fc30 msgr2=0x555b913520f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.693+0000 7ffa5da00640 1 --2- 172.21.15.111:0/2174755346 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ffa400a3240 0x7ffa400a5700 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ffa50005fd0 tx=0x7ffa50009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.693+0000 7ffa62861740 1 --2- 172.21.15.111:0/4169078305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b9134fc30 0x555b913520f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffa44006010 tx=0x7ffa44004520 comp rx=0 tx=0).stop 2024-09-15T22:53:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.693+0000 7ffa62861740 1 -- 172.21.15.111:0/4169078305 shutdown_connections 2024-09-15T22:53:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.693+0000 7ffa62861740 1 --2- 172.21.15.111:0/4169078305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b9134fc30 0x555b913520f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.693+0000 7ffa62861740 1 --2- 172.21.15.111:0/4169078305 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b9134aba0 0x555b9134f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.693+0000 7ffa62861740 1 --2- 172.21.15.111:0/4169078305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9134a260 0x555b9134a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.693+0000 7ffa62861740 1 -- 172.21.15.111:0/4169078305 >> 172.21.15.111:0/4169078305 conn(0x555b913458d0 msgr2=0x555b91345cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.693+0000 7ffa62861740 1 -- 172.21.15.111:0/4169078305 shutdown_connections 2024-09-15T22:53:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.693+0000 7ffa62861740 1 -- 172.21.15.111:0/4169078305 wait complete. 2024-09-15T22:53:16.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.694+0000 7ffa62861740 1 -- start start 2024-09-15T22:53:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.694+0000 7ffa62861740 1 Processor -- start 2024-09-15T22:53:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.695+0000 7ffa62861740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b9134a260 0x555b913543b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.695+0000 7ffa62861740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b9134aba0 0x555b913548f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.695+0000 7ffa62861740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9134fc30 0x555b9135b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.695+0000 7ffa62861740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555b91352ba0 con 0x555b9134aba0 2024-09-15T22:53:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.695+0000 7ffa62861740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x555b91238090 con 0x555b9134a260 2024-09-15T22:53:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.695+0000 7ffa62861740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555b91352d20 con 0x555b9134fc30 2024-09-15T22:53:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.695+0000 7ffa62861740 10 client.0 _ll_register_callbacks cb 0x555b9128cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:53:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.695+0000 7ffa57e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b9134aba0 0x555b913548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.695+0000 7ffa5d000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b9134a260 0x555b913543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.695+0000 7ffa5d000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b9134a260 0x555b913543b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:59040/0 (socket says 172.21.15.111:59040) 2024-09-15T22:53:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.695+0000 7ffa5d000640 1 -- 172.21.15.111:0/1800845729 learned_addr learned my addr 172.21.15.111:0/1800845729 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:53:16.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.695+0000 7ffa5da00640 1 --2- 172.21.15.111:0/1800845729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9134fc30 0x555b9135b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.695+0000 7ffa5d000640 1 -- 172.21.15.111:0/1800845729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9134fc30 msgr2=0x555b9135b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.695+0000 7ffa5d000640 1 --2- 172.21.15.111:0/1800845729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9134fc30 0x555b9135b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.695+0000 7ffa5d000640 1 -- 172.21.15.111:0/1800845729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b9134aba0 msgr2=0x555b913548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.695+0000 7ffa5d000640 1 --2- 172.21.15.111:0/1800845729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b9134aba0 0x555b913548f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.695+0000 7ffa5d000640 1 -- 172.21.15.111:0/1800845729 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b9108e620 con 0x555b9134a260 2024-09-15T22:53:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.695+0000 7ffa5da00640 1 --2- 172.21.15.111:0/1800845729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9134fc30 0x555b9135b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:53:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.695+0000 7ffa5d000640 1 --2- 172.21.15.111:0/1800845729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b9134a260 0x555b913543b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0a41c0 tx=0x7ffa4c097e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=292d25470120c90f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:16.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.695+0000 7ffa3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:53:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.696+0000 7ffa3ea00640 1 -- 172.21.15.111:0/1800845729 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c020030 con 0x555b9134a260 2024-09-15T22:53:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.696+0000 7ffa3ea00640 1 -- 172.21.15.111:0/1800845729 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7ffa4c0981a0 con 0x555b9134a260 2024-09-15T22:53:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.696+0000 7ffa62861740 1 -- 172.21.15.111:0/1800845729 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa440047a0 con 0x555b9134a260 2024-09-15T22:53:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.696+0000 7ffa3ea00640 1 -- 172.21.15.111:0/1800845729 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffa4c09bb30 con 0x555b9134a260 2024-09-15T22:53:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.696+0000 7ffa62861740 10 client.25574 fetch_fsmap learned FSMap version 30 2024-09-15T22:53:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.696+0000 7ffa62861740 1 -- 172.21.15.111:0/1800845729 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555b9135e7e0 con 0x555b9134a260 2024-09-15T22:53:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.696+0000 7ffa3ea00640 1 -- 172.21.15.111:0/1800845729 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa440047a0 con 0x555b9134a260 2024-09-15T22:53:16.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.696+0000 7ffa3ea00640 1 -- 172.21.15.111:0/1800845729 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7ffa4c0a4260 con 0x555b9134a260 2024-09-15T22:53:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.696+0000 7ffa62861740 10 client.25574 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:53:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.696+0000 7ffa62861740 10 client.25574 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:53:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.696+0000 7ffa62861740 20 client.25574 populate_metadata read hostname 'smithi111' 2024-09-15T22:53:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.696+0000 7ffa62861740 1 --2- 172.21.15.111:0/1800845729 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555b9135fe10 0x555b91362200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.696+0000 7ffa62861740 4 client.25574 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:53:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.696+0000 7ffa62861740 1 -- 172.21.15.111:0/1800845729 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b912bc830 con 0x555b9135fe10 2024-09-15T22:53:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.697+0000 7ffa57e00640 1 --2- 172.21.15.111:0/1800845729 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555b9135fe10 0x555b91362200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:16.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.697+0000 7ffa57e00640 1 --2- 172.21.15.111:0/1800845729 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555b9135fe10 0x555b91362200 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=901ac7965be2145d server_cookie=ef4b4d7eb46fe9a8 in_seq=0 out_seq=0 2024-09-15T22:53:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.697+0000 7ffa3ea00640 10 client.25574 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:53:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.698+0000 7ffa3ea00640 1 -- 172.21.15.111:0/1800845729 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b912bc830 con 0x555b9135fe10 2024-09-15T22:53:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.698+0000 7ffa3ea00640 10 client.25574 handle_command_reply: tid=0 2024-09-15T22:53:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.698+0000 7ffa62861740 1 client.25574 shutdown 2024-09-15T22:53:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.698+0000 7ffa62861740 1 -- 172.21.15.111:0/1800845729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b9134a260 msgr2=0x555b913543b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.698+0000 7ffa62861740 1 --2- 172.21.15.111:0/1800845729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b9134a260 0x555b913543b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0a41c0 tx=0x7ffa4c097e90 comp rx=0 tx=0).stop 2024-09-15T22:53:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.698+0000 7ffa62861740 1 -- 172.21.15.111:0/1800845729 shutdown_connections 2024-09-15T22:53:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.698+0000 7ffa62861740 1 --2- 172.21.15.111:0/1800845729 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555b9135fe10 0x555b91362200 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.698+0000 7ffa62861740 1 --2- 172.21.15.111:0/1800845729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9134fc30 0x555b9135b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.698+0000 7ffa62861740 1 --2- 172.21.15.111:0/1800845729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b9134aba0 0x555b913548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.698+0000 7ffa62861740 1 --2- 172.21.15.111:0/1800845729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b9134a260 0x555b913543b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.698+0000 7ffa3ea00640 0 client.25574 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:53:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.698+0000 7ffa62861740 1 -- 172.21.15.111:0/1800845729 >> 172.21.15.111:0/1800845729 conn(0x555b913458d0 msgr2=0x555b913470e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:16.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.698+0000 7ffa62861740 1 -- 172.21.15.111:0/1800845729 shutdown_connections 2024-09-15T22:53:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.698+0000 7ffa62861740 1 -- 172.21.15.111:0/1800845729 wait complete. 2024-09-15T22:53:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.698+0000 7ffa62861740 20 client.25574 trim_cache size 0 max 16384 2024-09-15T22:53:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.713+0000 7ffa62861740 1 Processor -- start 2024-09-15T22:53:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.713+0000 7ffa62861740 1 -- start start 2024-09-15T22:53:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.713+0000 7ffa62861740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b913458d0 0x555b91354870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.713+0000 7ffa62861740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b9134a260 0x555b9135b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.713+0000 7ffa62861740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b9134aba0 0x555b91346780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.713+0000 7ffa62861740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffa581ad760 con 0x555b9134a260 2024-09-15T22:53:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.713+0000 7ffa62861740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffa581ab030 con 0x555b9134aba0 2024-09-15T22:53:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.713+0000 7ffa62861740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ffa581a8e80 con 0x555b913458d0 2024-09-15T22:53:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.713+0000 7ffa57e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b9134a260 0x555b9135b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.713+0000 7ffa57e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b9134a260 0x555b9135b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45156/0 (socket says 172.21.15.111:45156) 2024-09-15T22:53:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.713+0000 7ffa57e00640 1 -- 172.21.15.111:0/1660829180 learned_addr learned my addr 172.21.15.111:0/1660829180 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:53:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.713+0000 7ffa5d000640 1 --2- 172.21.15.111:0/1660829180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b913458d0 0x555b91354870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.714+0000 7ffa5da00640 1 --2- 172.21.15.111:0/1660829180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b9134aba0 0x555b91346780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.714+0000 7ffa57e00640 1 -- 172.21.15.111:0/1660829180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b913458d0 msgr2=0x555b91354870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.714+0000 7ffa57e00640 1 --2- 172.21.15.111:0/1660829180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b913458d0 0x555b91354870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.714+0000 7ffa57e00640 1 -- 172.21.15.111:0/1660829180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b9134aba0 msgr2=0x555b91346780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.714+0000 7ffa57e00640 1 --2- 172.21.15.111:0/1660829180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b9134aba0 0x555b91346780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.714+0000 7ffa57e00640 1 -- 172.21.15.111:0/1660829180 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b9134b080 con 0x555b9134a260 2024-09-15T22:53:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.714+0000 7ffa57e00640 1 --2- 172.21.15.111:0/1660829180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b9134a260 0x555b9135b8f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ffa44006010 tx=0x7ffa44047040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1e0ae92a45b80d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.714+0000 7ffa49600640 1 -- 172.21.15.111:0/1660829180 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44040040 con 0x555b9134a260 2024-09-15T22:53:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.714+0000 7ffa49600640 1 -- 172.21.15.111:0/1660829180 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa44009520 con 0x555b9134a260 2024-09-15T22:53:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.714+0000 7ffa49600640 1 -- 172.21.15.111:0/1660829180 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44035040 con 0x555b9134a260 2024-09-15T22:53:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.715+0000 7ffa62861740 1 -- 172.21.15.111:0/1660829180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b9134a260 msgr2=0x555b9135b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.715+0000 7ffa62861740 1 --2- 172.21.15.111:0/1660829180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b9134a260 0x555b9135b8f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ffa44006010 tx=0x7ffa44047040 comp rx=0 tx=0).stop 2024-09-15T22:53:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.715+0000 7ffa62861740 1 -- 172.21.15.111:0/1660829180 shutdown_connections 2024-09-15T22:53:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.715+0000 7ffa62861740 1 --2- 172.21.15.111:0/1660829180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b9134aba0 0x555b91346780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.715+0000 7ffa62861740 1 --2- 172.21.15.111:0/1660829180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b9134a260 0x555b9135b8f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.715+0000 7ffa62861740 1 --2- 172.21.15.111:0/1660829180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b913458d0 0x555b91354870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.715+0000 7ffa62861740 1 -- 172.21.15.111:0/1660829180 >> 172.21.15.111:0/1660829180 conn(0x555b9134b660 msgr2=0x555b91354160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.715+0000 7ffa62861740 1 -- 172.21.15.111:0/1660829180 shutdown_connections 2024-09-15T22:53:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.715+0000 7ffa62861740 1 -- 172.21.15.111:0/1660829180 wait complete. 2024-09-15T22:53:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.715+0000 7ffa62861740 1 -- start start 2024-09-15T22:53:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.715+0000 7ffa62861740 1 Processor -- start 2024-09-15T22:53:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.716+0000 7ffa62861740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b913458d0 0x555b91358f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.716+0000 7ffa62861740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b9134a260 0x555b91359440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.716+0000 7ffa62861740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9134aba0 0x555b9135a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.716+0000 7ffa62861740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffa581a94f0 con 0x555b913458d0 2024-09-15T22:53:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.716+0000 7ffa62861740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffa581aa960 con 0x555b9134a260 2024-09-15T22:53:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.716+0000 7ffa62861740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ffa581a9bc0 con 0x555b9134aba0 2024-09-15T22:53:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.716+0000 7ffa62861740 10 client.0 _ll_register_callbacks cb 0x7ffa4c005d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:53:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.716+0000 7ffa5d000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b913458d0 0x555b91358f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.716+0000 7ffa5d000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b913458d0 0x555b91358f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45170/0 (socket says 172.21.15.111:45170) 2024-09-15T22:53:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.716+0000 7ffa5d000640 1 -- 172.21.15.111:0/2332893790 learned_addr learned my addr 172.21.15.111:0/2332893790 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:53:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.716+0000 7ffa5da00640 1 --2- 172.21.15.111:0/2332893790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9134aba0 0x555b9135a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.716+0000 7ffa5d000640 1 -- 172.21.15.111:0/2332893790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9134aba0 msgr2=0x555b9135a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.716+0000 7ffa5d000640 1 --2- 172.21.15.111:0/2332893790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9134aba0 0x555b9135a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.716+0000 7ffa5d000640 1 -- 172.21.15.111:0/2332893790 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b9134a260 msgr2=0x555b91359440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:53:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.716+0000 7ffa5d000640 1 --2- 172.21.15.111:0/2332893790 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b9134a260 0x555b91359440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.716+0000 7ffa5d000640 1 -- 172.21.15.111:0/2332893790 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b9135aa60 con 0x555b913458d0 2024-09-15T22:53:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.717+0000 7ffa5d000640 1 --2- 172.21.15.111:0/2332893790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b913458d0 0x555b91358f00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c09b830 tx=0x7ffa4c097f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff989c9afa5fde8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.717+0000 7ffa3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:53:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.717+0000 7ffa3ea00640 1 -- 172.21.15.111:0/2332893790 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c020040 con 0x555b913458d0 2024-09-15T22:53:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.717+0000 7ffa3ea00640 1 -- 172.21.15.111:0/2332893790 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7ffa4c097880 con 0x555b913458d0 2024-09-15T22:53:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.717+0000 7ffa62861740 1 -- 172.21.15.111:0/2332893790 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa44009520 con 0x555b913458d0 2024-09-15T22:53:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.717+0000 7ffa3ea00640 1 -- 172.21.15.111:0/2332893790 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffa4c0a4460 con 0x555b913458d0 2024-09-15T22:53:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.717+0000 7ffa62861740 10 client.15588 fetch_fsmap learned FSMap version 30 2024-09-15T22:53:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.717+0000 7ffa62861740 1 -- 172.21.15.111:0/2332893790 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555b913cab00 con 0x555b913458d0 2024-09-15T22:53:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.717+0000 7ffa3ea00640 1 -- 172.21.15.111:0/2332893790 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa44009520 con 0x555b913458d0 2024-09-15T22:53:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.717+0000 7ffa3ea00640 1 -- 172.21.15.111:0/2332893790 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7ffa4c0ba1a0 con 0x555b913458d0 2024-09-15T22:53:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.718+0000 7ffa62861740 10 client.15588 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:53:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.718+0000 7ffa62861740 10 client.15588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:53:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.718+0000 7ffa62861740 20 client.15588 populate_metadata read hostname 'smithi111' 2024-09-15T22:53:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.718+0000 7ffa62861740 1 --2- 172.21.15.111:0/2332893790 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555b9134fc30 0x555b913cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.718+0000 7ffa62861740 4 client.15588 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:53:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.718+0000 7ffa62861740 1 -- 172.21.15.111:0/2332893790 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555b912b6b80 con 0x555b9134fc30 2024-09-15T22:53:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.718+0000 7ffa57e00640 1 --2- 172.21.15.111:0/2332893790 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555b9134fc30 0x555b913cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.718+0000 7ffa57e00640 1 --2- 172.21.15.111:0/2332893790 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555b9134fc30 0x555b913cd340 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=360f5678f9afa4ba server_cookie=d153fd3a7cdfcb5 in_seq=0 out_seq=0 2024-09-15T22:53:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.718+0000 7ffa3ea00640 10 client.15588 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:53:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.741+0000 7ffa3ea00640 1 -- 172.21.15.111:0/2332893790 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555b912b6b80 con 0x555b9134fc30 2024-09-15T22:53:16.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.741+0000 7ffa3ea00640 10 client.15588 handle_command_reply: tid=0 2024-09-15T22:53:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.741+0000 7ffa62861740 1 client.15588 shutdown 2024-09-15T22:53:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.741+0000 7ffa62861740 1 -- 172.21.15.111:0/2332893790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b913458d0 msgr2=0x555b91358f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.741+0000 7ffa62861740 1 --2- 172.21.15.111:0/2332893790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b913458d0 0x555b91358f00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c09b830 tx=0x7ffa4c097f00 comp rx=0 tx=0).stop 2024-09-15T22:53:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.741+0000 7ffa62861740 1 -- 172.21.15.111:0/2332893790 shutdown_connections 2024-09-15T22:53:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.741+0000 7ffa62861740 1 --2- 172.21.15.111:0/2332893790 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555b9134fc30 0x555b913cd340 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.742+0000 7ffa62861740 1 --2- 172.21.15.111:0/2332893790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b9134aba0 0x555b9135a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.742+0000 7ffa3ea00640 0 client.15588 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:53:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.742+0000 7ffa62861740 1 --2- 172.21.15.111:0/2332893790 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b9134a260 0x555b91359440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.742+0000 7ffa62861740 1 --2- 172.21.15.111:0/2332893790 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b913458d0 0x555b91358f00 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.742+0000 7ffa62861740 1 -- 172.21.15.111:0/2332893790 >> 172.21.15.111:0/2332893790 conn(0x555b9134b660 msgr2=0x555b913441f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.742+0000 7ffa62861740 1 -- 172.21.15.111:0/2332893790 shutdown_connections 2024-09-15T22:53:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.742+0000 7ffa62861740 1 -- 172.21.15.111:0/2332893790 wait complete. 2024-09-15T22:53:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.742+0000 7ffa62861740 20 client.15588 trim_cache size 0 max 16384 2024-09-15T22:53:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:53:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:53:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:53:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:53:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.742+0000 7ffa5e400640 1 -- 172.21.15.111:0/2174755346 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ffa400a3240 msgr2=0x7ffa400a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.742+0000 7ffa5e400640 1 --2- 172.21.15.111:0/2174755346 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ffa400a3240 0x7ffa400a5700 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ffa50005fd0 tx=0x7ffa50009500 comp rx=0 tx=0).stop 2024-09-15T22:53:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.743+0000 7ffa5e400640 1 -- 172.21.15.111:0/2174755346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa58074680 msgr2=0x7ffa58199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.743+0000 7ffa5e400640 1 --2- 172.21.15.111:0/2174755346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa58074680 0x7ffa58199fa0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c005fd0 tx=0x7ffa4c00dde0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.743+0000 7ffa5e400640 1 -- 172.21.15.111:0/2174755346 shutdown_connections 2024-09-15T22:53:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.743+0000 7ffa5e400640 1 --2- 172.21.15.111:0/2174755346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa58109c90 0x7ffa581a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.743+0000 7ffa5e400640 1 --2- 172.21.15.111:0/2174755346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa58074fc0 0x7ffa5819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.743+0000 7ffa5e400640 1 --2- 172.21.15.111:0/2174755346 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ffa400a3240 0x7ffa400a5700 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.743+0000 7ffa5e400640 1 --2- 172.21.15.111:0/2174755346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa58074680 0x7ffa58199fa0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.743+0000 7ffa5e400640 1 -- 172.21.15.111:0/2174755346 >> 172.21.15.111:0/2174755346 conn(0x7ffa580fd0d0 msgr2=0x7ffa58105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.743+0000 7ffa5e400640 1 -- 172.21.15.111:0/2174755346 shutdown_connections 2024-09-15T22:53:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:16.743+0000 7ffa5e400640 1 -- 172.21.15.111:0/2174755346 wait complete. 2024-09-15T22:53:17.270 INFO:tasks.workunit.client.0.smithi111.stdout: 1 646969 9.87 MB/sec execute 225 sec latency 1552.192 ms 2024-09-15T22:53:18.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:17 smithi192 ceph-mon[35427]: pgmap v571: 97 pgs: 97 active+clean; 237 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 193 op/s 2024-09-15T22:53:18.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:17 smithi121 ceph-mon[34815]: pgmap v571: 97 pgs: 97 active+clean; 237 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 193 op/s 2024-09-15T22:53:18.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:17 smithi111 ceph-mon[30629]: pgmap v571: 97 pgs: 97 active+clean; 237 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 193 op/s 2024-09-15T22:53:18.270 INFO:tasks.workunit.client.0.smithi111.stdout: 1 649981 9.91 MB/sec execute 226 sec latency 73.316 ms 2024-09-15T22:53:19.270 INFO:tasks.workunit.client.0.smithi111.stdout: 1 652769 9.92 MB/sec execute 227 sec latency 13.132 ms 2024-09-15T22:53:20.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:19 smithi192 ceph-mon[35427]: pgmap v572: 97 pgs: 97 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 231 op/s 2024-09-15T22:53:20.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:19 smithi121 ceph-mon[34815]: pgmap v572: 97 pgs: 97 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 231 op/s 2024-09-15T22:53:20.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:19 smithi111 ceph-mon[30629]: pgmap v572: 97 pgs: 97 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 231 op/s 2024-09-15T22:53:20.270 INFO:tasks.workunit.client.0.smithi111.stdout: 1 655144 9.93 MB/sec execute 228 sec latency 11.587 ms 2024-09-15T22:53:21.270 INFO:tasks.workunit.client.0.smithi111.stdout: 1 656048 9.90 MB/sec execute 229 sec latency 761.369 ms 2024-09-15T22:53:21.883 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:21 smithi111 ceph-mon[30629]: pgmap v573: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 234 op/s 2024-09-15T22:53:22.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:21 smithi192 ceph-mon[35427]: pgmap v573: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 234 op/s 2024-09-15T22:53:22.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:21 smithi121 ceph-mon[34815]: pgmap v573: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 234 op/s 2024-09-15T22:53:22.270 INFO:tasks.workunit.client.0.smithi111.stdout: 1 657141 9.90 MB/sec execute 230 sec latency 1326.737 ms 2024-09-15T22:53:23.271 INFO:tasks.workunit.client.0.smithi111.stdout: 1 659911 9.92 MB/sec execute 231 sec latency 15.876 ms 2024-09-15T22:53:24.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:23 smithi192 ceph-mon[35427]: pgmap v574: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 203 op/s 2024-09-15T22:53:24.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:23 smithi121 ceph-mon[34815]: pgmap v574: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 203 op/s 2024-09-15T22:53:24.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:23 smithi111 ceph-mon[30629]: pgmap v574: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 203 op/s 2024-09-15T22:53:24.270 INFO:tasks.workunit.client.0.smithi111.stdout: 1 662306 9.93 MB/sec execute 232 sec latency 15.309 ms 2024-09-15T22:53:25.271 INFO:tasks.workunit.client.0.smithi111.stdout: 1 664811 9.95 MB/sec execute 233 sec latency 24.566 ms 2024-09-15T22:53:25.880 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:25 smithi111 ceph-mon[30629]: pgmap v575: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 192 op/s 2024-09-15T22:53:26.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:25 smithi192 ceph-mon[35427]: pgmap v575: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 192 op/s 2024-09-15T22:53:26.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:25 smithi121 ceph-mon[34815]: pgmap v575: 97 pgs: 97 active+clean; 242 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 192 op/s 2024-09-15T22:53:26.271 INFO:tasks.workunit.client.0.smithi111.stdout: 1 665796 9.93 MB/sec execute 234 sec latency 633.612 ms 2024-09-15T22:53:27.271 INFO:tasks.workunit.client.0.smithi111.stdout: 1 666419 9.89 MB/sec execute 235 sec latency 1520.443 ms 2024-09-15T22:53:28.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:27 smithi192 ceph-mon[35427]: pgmap v576: 97 pgs: 97 active+clean; 244 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 204 op/s 2024-09-15T22:53:28.127 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:27 smithi111 ceph-mon[30629]: pgmap v576: 97 pgs: 97 active+clean; 244 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 204 op/s 2024-09-15T22:53:28.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:27 smithi121 ceph-mon[34815]: pgmap v576: 97 pgs: 97 active+clean; 244 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 204 op/s 2024-09-15T22:53:28.271 INFO:tasks.workunit.client.0.smithi111.stdout: 1 668681 9.91 MB/sec execute 236 sec latency 15.515 ms 2024-09-15T22:53:29.271 INFO:tasks.workunit.client.0.smithi111.stdout: 1 671678 9.94 MB/sec execute 237 sec latency 11.553 ms 2024-09-15T22:53:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.873+0000 7f16a3200640 1 -- 172.21.15.111:0/4281056507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169c101580 msgr2=0x7f169c101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.873+0000 7f16a3200640 1 --2- 172.21.15.111:0/4281056507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169c101580 0x7f169c101a00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f168c009900 tx=0x7f168c02a410 comp rx=0 tx=0).stop 2024-09-15T22:53:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.873+0000 7f16a3200640 1 -- 172.21.15.111:0/4281056507 shutdown_connections 2024-09-15T22:53:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.873+0000 7f16a3200640 1 --2- 172.21.15.111:0/4281056507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f169c101f40 0x7f169c109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.873+0000 7f16a3200640 1 --2- 172.21.15.111:0/4281056507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169c101580 0x7f169c101a00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.873+0000 7f16a3200640 1 --2- 172.21.15.111:0/4281056507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f169c103050 0x7f169c103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.873+0000 7f16a3200640 1 -- 172.21.15.111:0/4281056507 >> 172.21.15.111:0/4281056507 conn(0x7f169c0fd0d0 msgr2=0x7f169c0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.874+0000 7f16a3200640 1 -- 172.21.15.111:0/4281056507 shutdown_connections 2024-09-15T22:53:29.876 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.874+0000 7f16a3200640 1 -- 172.21.15.111:0/4281056507 wait complete. 2024-09-15T22:53:29.878 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.874+0000 7f16a3200640 1 Processor -- start 2024-09-15T22:53:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.875+0000 7f16a3200640 1 -- start start 2024-09-15T22:53:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.875+0000 7f16a3200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f169c101580 0x7f169c075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.875+0000 7f16a3200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f169c101f40 0x7f169c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.875+0000 7f16a3200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169c103050 0x7f169c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.875+0000 7f16a3200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f169c10d740 con 0x7f169c101580 2024-09-15T22:53:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.875+0000 7f16a3200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f169c10d5c0 con 0x7f169c103050 2024-09-15T22:53:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.875+0000 7f16a3200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f169c10d8c0 con 0x7f169c101f40 2024-09-15T22:53:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.875+0000 7f16a2800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169c103050 0x7f169c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.875+0000 7f16a1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f169c101580 0x7f169c075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.875+0000 7f16a2800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169c103050 0x7f169c074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:35736/0 (socket says 172.21.15.111:35736) 2024-09-15T22:53:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.875+0000 7f16a2800640 1 -- 172.21.15.111:0/2662052204 learned_addr learned my addr 172.21.15.111:0/2662052204 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:53:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.875+0000 7f16a1400640 1 --2- 172.21.15.111:0/2662052204 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f169c101f40 0x7f169c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.876+0000 7f16a2800640 1 -- 172.21.15.111:0/2662052204 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f169c101f40 msgr2=0x7f169c0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:29.879 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.876+0000 7f16a2800640 1 --2- 172.21.15.111:0/2662052204 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f169c101f40 0x7f169c0745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.876+0000 7f16a2800640 1 -- 172.21.15.111:0/2662052204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f169c101580 msgr2=0x7f169c075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.876+0000 7f16a2800640 1 --2- 172.21.15.111:0/2662052204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f169c101580 0x7f169c075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.876+0000 7f16a2800640 1 -- 172.21.15.111:0/2662052204 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169c0753b0 con 0x7f169c103050 2024-09-15T22:53:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.876+0000 7f16a1e00640 1 --2- 172.21.15.111:0/2662052204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f169c101580 0x7f169c075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:53:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.876+0000 7f16a2800640 1 --2- 172.21.15.111:0/2662052204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169c103050 0x7f169c074b20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1698009fd0 tx=0x7f169800b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.876+0000 7f1696a00640 1 -- 172.21.15.111:0/2662052204 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1698017070 con 0x7f169c103050 2024-09-15T22:53:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.876+0000 7f1696a00640 1 -- 172.21.15.111:0/2662052204 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1698003e50 con 0x7f169c103050 2024-09-15T22:53:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.876+0000 7f16a3200640 1 -- 172.21.15.111:0/2662052204 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169c1a1f40 con 0x7f169c103050 2024-09-15T22:53:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.876+0000 7f1696a00640 1 -- 172.21.15.111:0/2662052204 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16980106c0 con 0x7f169c103050 2024-09-15T22:53:29.880 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.876+0000 7f16a3200640 1 -- 172.21.15.111:0/2662052204 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f169c1a2350 con 0x7f169c103050 2024-09-15T22:53:29.881 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.880+0000 7f16a77e7740 1 Processor -- start 2024-09-15T22:53:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.880+0000 7f1696a00640 1 -- 172.21.15.111:0/2662052204 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f1698003970 con 0x7f169c103050 2024-09-15T22:53:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.880+0000 7f16a77e7740 1 -- start start 2024-09-15T22:53:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.881+0000 7f16a77e7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5577befd6260 0x5577befd6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.881+0000 7f16a77e7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5577befd6ba0 0x5577befdb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.881+0000 7f16a77e7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577befdbc30 0x5577befde0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.881+0000 7f16a77e7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5577bef3b100 con 0x5577befd6260 2024-09-15T22:53:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.881+0000 7f16a77e7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5577beee31b0 con 0x5577befdbc30 2024-09-15T22:53:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.881+0000 7f16a77e7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5577bef370c0 con 0x5577befd6ba0 2024-09-15T22:53:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.881+0000 7f16a1400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5577befd6260 0x5577befd6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.881+0000 7f16a1e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5577befd6ba0 0x5577befdb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.881+0000 7f16a1400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5577befd6260 0x5577befd6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52622/0 (socket says 172.21.15.111:52622) 2024-09-15T22:53:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.881+0000 7f16a1400640 1 -- 172.21.15.111:0/4147917015 learned_addr learned my addr 172.21.15.111:0/4147917015 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:53:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.881+0000 7f16a1400640 1 --2- 172.21.15.111:0/4147917015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577befdbc30 0x5577befde0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.882+0000 7f16a1e00640 1 -- 172.21.15.111:0/4147917015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577befdbc30 msgr2=0x5577befde0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.882+0000 7f16a1e00640 1 --2- 172.21.15.111:0/4147917015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577befdbc30 0x5577befde0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.882+0000 7f16a1e00640 1 -- 172.21.15.111:0/4147917015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5577befd6260 msgr2=0x5577befd6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.882+0000 7f16a1e00640 1 --2- 172.21.15.111:0/4147917015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5577befd6260 0x5577befd6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.882+0000 7f16a1e00640 1 -- 172.21.15.111:0/4147917015 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577bec4e210 con 0x5577befd6ba0 2024-09-15T22:53:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.882+0000 7f16a1400640 1 --2- 172.21.15.111:0/4147917015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577befdbc30 0x5577befde0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:53:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.882+0000 7f16a1e00640 1 --2- 172.21.15.111:0/4147917015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5577befd6ba0 0x5577befdb660 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1690009f80 tx=0x7f16900118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9294f8a71a4fb72c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.882+0000 7f1696a00640 1 --2- 172.21.15.111:0/2662052204 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f16840a3240 0x7f16840a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.882+0000 7f1696a00640 1 -- 172.21.15.111:0/2662052204 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f16980cf540 con 0x7f169c103050 2024-09-15T22:53:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.882+0000 7f168b000640 1 -- 172.21.15.111:0/4147917015 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169001e070 con 0x5577befd6ba0 2024-09-15T22:53:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.882+0000 7f168b000640 1 -- 172.21.15.111:0/4147917015 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16900035f0 con 0x5577befd6ba0 2024-09-15T22:53:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.882+0000 7f168b000640 1 -- 172.21.15.111:0/4147917015 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169000f740 con 0x5577befd6ba0 2024-09-15T22:53:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.882+0000 7f16a1400640 1 --2- 172.21.15.111:0/2662052204 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f16840a3240 0x7f16840a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.883+0000 7f16a77e7740 1 -- 172.21.15.111:0/4147917015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5577befd6ba0 msgr2=0x5577befdb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.883+0000 7f16a1400640 1 --2- 172.21.15.111:0/2662052204 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f16840a3240 0x7f16840a5700 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f168c000c00 tx=0x7f168c005e50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.883+0000 7f16a77e7740 1 --2- 172.21.15.111:0/4147917015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5577befd6ba0 0x5577befdb660 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1690009f80 tx=0x7f16900118d0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.883+0000 7f16a77e7740 1 -- 172.21.15.111:0/4147917015 shutdown_connections 2024-09-15T22:53:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.883+0000 7f16a77e7740 1 --2- 172.21.15.111:0/4147917015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577befdbc30 0x5577befde0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.883+0000 7f16a77e7740 1 --2- 172.21.15.111:0/4147917015 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5577befd6ba0 0x5577befdb660 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.883+0000 7f16a77e7740 1 --2- 172.21.15.111:0/4147917015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5577befd6260 0x5577befd6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.883+0000 7f16a77e7740 1 -- 172.21.15.111:0/4147917015 >> 172.21.15.111:0/4147917015 conn(0x5577befd18d0 msgr2=0x5577befd1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.883+0000 7f16a77e7740 1 -- 172.21.15.111:0/4147917015 shutdown_connections 2024-09-15T22:53:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.883+0000 7f16a77e7740 1 -- 172.21.15.111:0/4147917015 wait complete. 2024-09-15T22:53:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.884+0000 7f16a77e7740 1 -- start start 2024-09-15T22:53:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.884+0000 7f16a77e7740 1 Processor -- start 2024-09-15T22:53:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.884+0000 7f16a77e7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5577befd6260 0x5577befe0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.884+0000 7f16a77e7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5577befd6ba0 0x5577befe0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.884+0000 7f16a77e7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577befdbc30 0x5577befe7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.884+0000 7f16a77e7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5577befdec00 con 0x5577befd6260 2024-09-15T22:53:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.884+0000 7f16a77e7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5577beec4090 con 0x5577befdbc30 2024-09-15T22:53:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.884+0000 7f16a77e7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5577befded80 con 0x5577befd6ba0 2024-09-15T22:53:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.885+0000 7f16a1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5577befd6260 0x5577befe0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.885+0000 7f16a1400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5577befd6ba0 0x5577befe0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.885+0000 7f16a2800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577befdbc30 0x5577befe7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.885+0000 7f16a77e7740 10 client.0 _ll_register_callbacks cb 0x5577bef18f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:53:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.885+0000 7f16a1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5577befd6260 0x5577befe0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52634/0 (socket says 172.21.15.111:52634) 2024-09-15T22:53:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.885+0000 7f16a1e00640 1 -- 172.21.15.111:0/479437836 learned_addr learned my addr 172.21.15.111:0/479437836 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:53:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.885+0000 7f16a1400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5577befd6ba0 0x5577befe0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:35274/0 (socket says 172.21.15.111:35274) 2024-09-15T22:53:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.885+0000 7f16a2800640 1 -- 172.21.15.111:0/479437836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5577befd6ba0 msgr2=0x5577befe0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.885+0000 7f16a2800640 1 --2- 172.21.15.111:0/479437836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5577befd6ba0 0x5577befe0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.885+0000 7f16a2800640 1 -- 172.21.15.111:0/479437836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5577befd6260 msgr2=0x5577befe0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.885+0000 7f16a2800640 1 --2- 172.21.15.111:0/479437836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5577befd6260 0x5577befe0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.885+0000 7f16a2800640 1 -- 172.21.15.111:0/479437836 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577bed1a620 con 0x5577befdbc30 2024-09-15T22:53:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.885+0000 7f16a1e00640 1 --2- 172.21.15.111:0/479437836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5577befd6260 0x5577befe0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:53:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.885+0000 7f16a2800640 1 --2- 172.21.15.111:0/479437836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577befdbc30 0x5577befe7990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f16980cd040 tx=0x7f169809b630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb91906634fafeb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.885+0000 7f167f400640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:53:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.885+0000 7f167f400640 1 -- 172.21.15.111:0/479437836 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1698020070 con 0x5577befdbc30 2024-09-15T22:53:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.885+0000 7f16a77e7740 1 -- 172.21.15.111:0/479437836 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16900035f0 con 0x5577befdbc30 2024-09-15T22:53:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.885+0000 7f167f400640 1 -- 172.21.15.111:0/479437836 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f1698098280 con 0x5577befdbc30 2024-09-15T22:53:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.886+0000 7f167f400640 1 -- 172.21.15.111:0/479437836 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f16980a47d0 con 0x5577befdbc30 2024-09-15T22:53:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.886+0000 7f16a77e7740 10 client.25589 fetch_fsmap learned FSMap version 30 2024-09-15T22:53:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.886+0000 7f16a77e7740 1 -- 172.21.15.111:0/479437836 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5577befea850 con 0x5577befdbc30 2024-09-15T22:53:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.886+0000 7f167f400640 1 -- 172.21.15.111:0/479437836 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16900035f0 con 0x5577befdbc30 2024-09-15T22:53:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.886+0000 7f167f400640 1 -- 172.21.15.111:0/479437836 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f169809b080 con 0x5577befdbc30 2024-09-15T22:53:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.886+0000 7f16a77e7740 10 client.25589 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:53:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.886+0000 7f16a77e7740 10 client.25589 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:53:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.886+0000 7f16a77e7740 20 client.25589 populate_metadata read hostname 'smithi111' 2024-09-15T22:53:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.886+0000 7f16a77e7740 1 --2- 172.21.15.111:0/479437836 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5577befebfb0 0x5577befee3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.886+0000 7f16a77e7740 4 client.25589 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:53:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.886+0000 7f16a77e7740 1 -- 172.21.15.111:0/479437836 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577bef48830 con 0x5577befebfb0 2024-09-15T22:53:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.887+0000 7f16a1e00640 1 --2- 172.21.15.111:0/479437836 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5577befebfb0 0x5577befee3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.887+0000 7f16a1e00640 1 --2- 172.21.15.111:0/479437836 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5577befebfb0 0x5577befee3a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba3ae61a78c8df60 server_cookie=63f7547175b62c18 in_seq=0 out_seq=0 2024-09-15T22:53:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.887+0000 7f167f400640 10 client.25589 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:53:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.888+0000 7f167f400640 1 -- 172.21.15.111:0/479437836 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5577bef48830 con 0x5577befebfb0 2024-09-15T22:53:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.888+0000 7f167f400640 10 client.25589 handle_command_reply: tid=0 2024-09-15T22:53:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.888+0000 7f16a77e7740 1 client.25589 shutdown 2024-09-15T22:53:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.888+0000 7f16a77e7740 1 -- 172.21.15.111:0/479437836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577befdbc30 msgr2=0x5577befe7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.888+0000 7f16a77e7740 1 --2- 172.21.15.111:0/479437836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577befdbc30 0x5577befe7990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f16980cd040 tx=0x7f169809b630 comp rx=0 tx=0).stop 2024-09-15T22:53:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.888+0000 7f16a77e7740 1 -- 172.21.15.111:0/479437836 shutdown_connections 2024-09-15T22:53:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.888+0000 7f16a77e7740 1 --2- 172.21.15.111:0/479437836 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5577befebfb0 0x5577befee3a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.888+0000 7f16a77e7740 1 --2- 172.21.15.111:0/479437836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577befdbc30 0x5577befe7990 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.888+0000 7f16a77e7740 1 --2- 172.21.15.111:0/479437836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5577befd6ba0 0x5577befe0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.888+0000 7f16a77e7740 1 --2- 172.21.15.111:0/479437836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5577befd6260 0x5577befe0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.888+0000 7f16a77e7740 1 -- 172.21.15.111:0/479437836 >> 172.21.15.111:0/479437836 conn(0x5577befd18d0 msgr2=0x5577befd30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.888+0000 7f167f400640 0 client.25589 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:53:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.889+0000 7f16a77e7740 1 -- 172.21.15.111:0/479437836 shutdown_connections 2024-09-15T22:53:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.889+0000 7f16a77e7740 1 -- 172.21.15.111:0/479437836 wait complete. 2024-09-15T22:53:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.889+0000 7f16a77e7740 20 client.25589 trim_cache size 0 max 16384 2024-09-15T22:53:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.906+0000 7f16a77e7740 1 Processor -- start 2024-09-15T22:53:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.906+0000 7f16a77e7740 1 -- start start 2024-09-15T22:53:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.906+0000 7f16a77e7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577befd6260 0x5577befd0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.906+0000 7f16a77e7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5577befd6ba0 0x5577befe7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.906+0000 7f16a77e7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5577befdbc30 0x5577befd2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.906+0000 7f16a77e7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f169c1ad6b0 con 0x5577befdbc30 2024-09-15T22:53:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.906+0000 7f16a77e7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f169c1aaf80 con 0x5577befd6260 2024-09-15T22:53:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.906+0000 7f16a77e7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f169c1a8dd0 con 0x5577befd6ba0 2024-09-15T22:53:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.906+0000 7f16a1e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577befd6260 0x5577befd0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.906+0000 7f16a1400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5577befd6ba0 0x5577befe7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.906+0000 7f16a2800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5577befdbc30 0x5577befd2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.906+0000 7f16a1e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577befd6260 0x5577befd0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:35760/0 (socket says 172.21.15.111:35760) 2024-09-15T22:53:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.906+0000 7f16a1e00640 1 -- 172.21.15.111:0/2741163104 learned_addr learned my addr 172.21.15.111:0/2741163104 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:53:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.907+0000 7f16a1e00640 1 -- 172.21.15.111:0/2741163104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5577befd6ba0 msgr2=0x5577befe7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.907+0000 7f16a1e00640 1 --2- 172.21.15.111:0/2741163104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5577befd6ba0 0x5577befe7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.907+0000 7f16a1e00640 1 -- 172.21.15.111:0/2741163104 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5577befdbc30 msgr2=0x5577befd2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.907+0000 7f16a1e00640 1 --2- 172.21.15.111:0/2741163104 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5577befdbc30 0x5577befd2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.907+0000 7f16a1e00640 1 -- 172.21.15.111:0/2741163104 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577befd16f0 con 0x5577befd6260 2024-09-15T22:53:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.907+0000 7f16a1e00640 1 --2- 172.21.15.111:0/2741163104 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577befd6260 0x5577befd0900 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1690004630 tx=0x7f1690009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f3cb348b74aa035 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.907+0000 7f168a600640 1 -- 172.21.15.111:0/2741163104 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169001e040 con 0x5577befd6260 2024-09-15T22:53:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.907+0000 7f168a600640 1 -- 172.21.15.111:0/2741163104 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1690004180 con 0x5577befd6260 2024-09-15T22:53:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.907+0000 7f168a600640 1 -- 172.21.15.111:0/2741163104 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690004660 con 0x5577befd6260 2024-09-15T22:53:29.913 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.908+0000 7f16a77e7740 1 -- 172.21.15.111:0/2741163104 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577befd6260 msgr2=0x5577befd0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:29.913 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.908+0000 7f16a77e7740 1 --2- 172.21.15.111:0/2741163104 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577befd6260 0x5577befd0900 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1690004630 tx=0x7f1690009b30 comp rx=0 tx=0).stop 2024-09-15T22:53:29.913 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.908+0000 7f16a77e7740 1 -- 172.21.15.111:0/2741163104 shutdown_connections 2024-09-15T22:53:29.913 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.908+0000 7f16a77e7740 1 --2- 172.21.15.111:0/2741163104 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5577befdbc30 0x5577befd2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.913 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.908+0000 7f16a77e7740 1 --2- 172.21.15.111:0/2741163104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5577befd6ba0 0x5577befe7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.913 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.908+0000 7f16a77e7740 1 --2- 172.21.15.111:0/2741163104 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577befd6260 0x5577befd0900 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.913 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.908+0000 7f16a77e7740 1 -- 172.21.15.111:0/2741163104 >> 172.21.15.111:0/2741163104 conn(0x5577befd18d0 msgr2=0x5577befd01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.908+0000 7f16a77e7740 1 -- 172.21.15.111:0/2741163104 shutdown_connections 2024-09-15T22:53:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.908+0000 7f16a77e7740 1 -- 172.21.15.111:0/2741163104 wait complete. 2024-09-15T22:53:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.909+0000 7f16a77e7740 1 -- start start 2024-09-15T22:53:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.909+0000 7f16a77e7740 1 Processor -- start 2024-09-15T22:53:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.909+0000 7f16a77e7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577befd6260 0x5577befe76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.909+0000 7f16a77e7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5577befd6ba0 0x5577befe4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.909+0000 7f16a77e7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5577befdbc30 0x5577befe46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.909+0000 7f16a77e7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f169c1a9440 con 0x5577befdbc30 2024-09-15T22:53:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.909+0000 7f16a77e7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f169c1aa8b0 con 0x5577befd6260 2024-09-15T22:53:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.909+0000 7f16a77e7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f169c1a9b10 con 0x5577befd6ba0 2024-09-15T22:53:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.909+0000 7f16a77e7740 10 client.0 _ll_register_callbacks cb 0x7f169800bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:53:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.909+0000 7f16a1400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5577befd6ba0 0x5577befe4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.910+0000 7f16a1400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5577befd6ba0 0x5577befe4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:35284/0 (socket says 172.21.15.111:35284) 2024-09-15T22:53:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.910+0000 7f16a1400640 1 -- 172.21.15.111:0/1736418483 learned_addr learned my addr 172.21.15.111:0/1736418483 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:53:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.910+0000 7f16a1e00640 1 --2- 172.21.15.111:0/1736418483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577befd6260 0x5577befe76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.910+0000 7f16a2800640 1 --2- 172.21.15.111:0/1736418483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5577befdbc30 0x5577befe46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.910+0000 7f16a1400640 1 -- 172.21.15.111:0/1736418483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577befd6260 msgr2=0x5577befe76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:29.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.910+0000 7f16a1400640 1 --2- 172.21.15.111:0/1736418483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577befd6260 0x5577befe76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.910+0000 7f16a1400640 1 -- 172.21.15.111:0/1736418483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5577befdbc30 msgr2=0x5577befe46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.910+0000 7f16a1400640 1 --2- 172.21.15.111:0/1736418483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5577befdbc30 0x5577befe46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.910+0000 7f16a1400640 1 -- 172.21.15.111:0/1736418483 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577befe4c70 con 0x5577befd6ba0 2024-09-15T22:53:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.910+0000 7f16a1e00640 1 --2- 172.21.15.111:0/1736418483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577befd6260 0x5577befe76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:53:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.910+0000 7f16a1400640 1 --2- 172.21.15.111:0/1736418483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5577befd6ba0 0x5577befe4170 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f168c0314e0 tx=0x7f168c0315b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a791cd838c526b96 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.910+0000 7f167f400640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:53:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.910+0000 7f167f400640 1 -- 172.21.15.111:0/1736418483 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c035040 con 0x5577befd6ba0 2024-09-15T22:53:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.910+0000 7f167f400640 1 -- 172.21.15.111:0/1736418483 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f168c004c40 con 0x5577befd6ba0 2024-09-15T22:53:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.911+0000 7f16a77e7740 1 -- 172.21.15.111:0/1736418483 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1690004180 con 0x5577befd6ba0 2024-09-15T22:53:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.911+0000 7f167f400640 1 -- 172.21.15.111:0/1736418483 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f168c004e40 con 0x5577befd6ba0 2024-09-15T22:53:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.911+0000 7f16a77e7740 10 client.25612 fetch_fsmap learned FSMap version 30 2024-09-15T22:53:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.911+0000 7f16a77e7740 1 -- 172.21.15.111:0/1736418483 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5577bf055cc0 con 0x5577befd6ba0 2024-09-15T22:53:29.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.911+0000 7f167f400640 1 -- 172.21.15.111:0/1736418483 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1690004180 con 0x5577befd6ba0 2024-09-15T22:53:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.911+0000 7f167f400640 1 -- 172.21.15.111:0/1736418483 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f168c054020 con 0x5577befd6ba0 2024-09-15T22:53:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.911+0000 7f16a77e7740 10 client.25612 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:53:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.911+0000 7f16a77e7740 10 client.25612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:53:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.911+0000 7f16a77e7740 20 client.25612 populate_metadata read hostname 'smithi111' 2024-09-15T22:53:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.911+0000 7f16a77e7740 1 --2- 172.21.15.111:0/1736418483 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5577bf0566a0 0x5577bf058ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.911+0000 7f16a77e7740 4 client.25612 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:53:29.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.911+0000 7f16a77e7740 1 -- 172.21.15.111:0/1736418483 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5577bef42b80 con 0x5577bf0566a0 2024-09-15T22:53:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.912+0000 7f16a1e00640 1 --2- 172.21.15.111:0/1736418483 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5577bf0566a0 0x5577bf058ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.912+0000 7f16a1e00640 1 --2- 172.21.15.111:0/1736418483 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5577bf0566a0 0x5577bf058ac0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32584729b9efc715 server_cookie=13d978438567a406 in_seq=0 out_seq=0 2024-09-15T22:53:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.912+0000 7f167f400640 10 client.25612 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:53:29.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.921+0000 7f167f400640 1 -- 172.21.15.111:0/1736418483 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5577bef42b80 con 0x5577bf0566a0 2024-09-15T22:53:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.921+0000 7f167f400640 10 client.25612 handle_command_reply: tid=0 2024-09-15T22:53:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.921+0000 7f16a77e7740 1 client.25612 shutdown 2024-09-15T22:53:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.921+0000 7f16a77e7740 1 -- 172.21.15.111:0/1736418483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5577befd6ba0 msgr2=0x5577befe4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.921+0000 7f16a77e7740 1 --2- 172.21.15.111:0/1736418483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5577befd6ba0 0x5577befe4170 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f168c0314e0 tx=0x7f168c0315b0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.921+0000 7f16a77e7740 1 -- 172.21.15.111:0/1736418483 shutdown_connections 2024-09-15T22:53:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.921+0000 7f16a77e7740 1 --2- 172.21.15.111:0/1736418483 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5577bf0566a0 0x5577bf058ac0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.921+0000 7f16a77e7740 1 --2- 172.21.15.111:0/1736418483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5577befdbc30 0x5577befe46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.921+0000 7f167f400640 0 client.25612 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:53:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.921+0000 7f16a77e7740 1 --2- 172.21.15.111:0/1736418483 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5577befd6ba0 0x5577befe4170 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.921+0000 7f16a77e7740 1 --2- 172.21.15.111:0/1736418483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5577befd6260 0x5577befe76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.921+0000 7f16a77e7740 1 -- 172.21.15.111:0/1736418483 >> 172.21.15.111:0/1736418483 conn(0x5577befd18d0 msgr2=0x5577befd0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.922+0000 7f16a77e7740 1 -- 172.21.15.111:0/1736418483 shutdown_connections 2024-09-15T22:53:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.922+0000 7f16a77e7740 1 -- 172.21.15.111:0/1736418483 wait complete. 2024-09-15T22:53:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.922+0000 7f16a77e7740 20 client.25612 trim_cache size 0 max 16384 2024-09-15T22:53:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:53:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:53:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:53:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:53:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.922+0000 7f16a3200640 1 -- 172.21.15.111:0/2662052204 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f16840a3240 msgr2=0x7f16840a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.922+0000 7f16a3200640 1 --2- 172.21.15.111:0/2662052204 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f16840a3240 0x7f16840a5700 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f168c000c00 tx=0x7f168c005e50 comp rx=0 tx=0).stop 2024-09-15T22:53:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.922+0000 7f16a3200640 1 -- 172.21.15.111:0/2662052204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169c103050 msgr2=0x7f169c074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.922+0000 7f16a3200640 1 --2- 172.21.15.111:0/2662052204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169c103050 0x7f169c074b20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1698009fd0 tx=0x7f169800b260 comp rx=0 tx=0).stop 2024-09-15T22:53:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.923+0000 7f16a3200640 1 -- 172.21.15.111:0/2662052204 shutdown_connections 2024-09-15T22:53:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.923+0000 7f16a3200640 1 --2- 172.21.15.111:0/2662052204 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f16840a3240 0x7f16840a5700 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.923+0000 7f16a3200640 1 --2- 172.21.15.111:0/2662052204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f169c103050 0x7f169c074b20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.923+0000 7f16a3200640 1 --2- 172.21.15.111:0/2662052204 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f169c101f40 0x7f169c0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.923+0000 7f16a3200640 1 --2- 172.21.15.111:0/2662052204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f169c101580 0x7f169c075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.923+0000 7f16a3200640 1 -- 172.21.15.111:0/2662052204 >> 172.21.15.111:0/2662052204 conn(0x7f169c0fd0d0 msgr2=0x7f169c105fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.923+0000 7f16a3200640 1 -- 172.21.15.111:0/2662052204 shutdown_connections 2024-09-15T22:53:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:29.923+0000 7f16a3200640 1 -- 172.21.15.111:0/2662052204 wait complete. 2024-09-15T22:53:30.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:29 smithi192 ceph-mon[35427]: pgmap v577: 97 pgs: 97 active+clean; 247 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 212 op/s 2024-09-15T22:53:30.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:29 smithi121 ceph-mon[34815]: pgmap v577: 97 pgs: 97 active+clean; 247 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 212 op/s 2024-09-15T22:53:30.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:29 smithi111 ceph-mon[30629]: pgmap v577: 97 pgs: 97 active+clean; 247 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 212 op/s 2024-09-15T22:53:30.271 INFO:tasks.workunit.client.0.smithi111.stdout: 1 674893 9.97 MB/sec execute 238 sec latency 19.702 ms 2024-09-15T22:53:31.271 INFO:tasks.workunit.client.0.smithi111.stdout: 1 675567 9.94 MB/sec execute 239 sec latency 543.185 ms 2024-09-15T22:53:31.859 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:31 smithi111 ceph-mon[30629]: pgmap v578: 97 pgs: 97 active+clean; 249 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 218 op/s 2024-09-15T22:53:32.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:31 smithi192 ceph-mon[35427]: pgmap v578: 97 pgs: 97 active+clean; 249 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 218 op/s 2024-09-15T22:53:32.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:31 smithi121 ceph-mon[34815]: pgmap v578: 97 pgs: 97 active+clean; 249 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 218 op/s 2024-09-15T22:53:32.271 INFO:tasks.workunit.client.0.smithi111.stdout: 1 675589 9.90 MB/sec execute 240 sec latency 1474.370 ms 2024-09-15T22:53:33.271 INFO:tasks.workunit.client.0.smithi111.stdout: 1 678127 9.91 MB/sec execute 241 sec latency 288.573 ms 2024-09-15T22:53:34.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:33 smithi192 ceph-mon[35427]: pgmap v579: 97 pgs: 97 active+clean; 246 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 198 op/s 2024-09-15T22:53:34.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:33 smithi121 ceph-mon[34815]: pgmap v579: 97 pgs: 97 active+clean; 246 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 198 op/s 2024-09-15T22:53:34.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:33 smithi111 ceph-mon[30629]: pgmap v579: 97 pgs: 97 active+clean; 246 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 198 op/s 2024-09-15T22:53:34.271 INFO:tasks.workunit.client.0.smithi111.stdout: 1 680678 9.92 MB/sec execute 242 sec latency 10.943 ms 2024-09-15T22:53:35.271 INFO:tasks.workunit.client.0.smithi111.stdout: 1 682971 9.94 MB/sec execute 243 sec latency 15.636 ms 2024-09-15T22:53:36.060 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:35 smithi111 ceph-mon[30629]: pgmap v580: 97 pgs: 97 active+clean; 245 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 188 op/s 2024-09-15T22:53:36.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:35 smithi192 ceph-mon[35427]: pgmap v580: 97 pgs: 97 active+clean; 245 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 188 op/s 2024-09-15T22:53:36.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:35 smithi121 ceph-mon[34815]: pgmap v580: 97 pgs: 97 active+clean; 245 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 188 op/s 2024-09-15T22:53:36.271 INFO:tasks.workunit.client.0.smithi111.stdout: 1 685195 9.95 MB/sec execute 244 sec latency 377.149 ms 2024-09-15T22:53:37.271 INFO:tasks.workunit.client.0.smithi111.stdout: 1 685195 9.91 MB/sec execute 245 sec latency 1377.187 ms 2024-09-15T22:53:38.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:37 smithi192 ceph-mon[35427]: pgmap v581: 97 pgs: 97 active+clean; 245 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 188 op/s 2024-09-15T22:53:38.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:37 smithi121 ceph-mon[34815]: pgmap v581: 97 pgs: 97 active+clean; 245 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 188 op/s 2024-09-15T22:53:38.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:37 smithi111 ceph-mon[30629]: pgmap v581: 97 pgs: 97 active+clean; 245 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 188 op/s 2024-09-15T22:53:38.271 INFO:tasks.workunit.client.0.smithi111.stdout: 1 686888 9.91 MB/sec execute 246 sec latency 1556.890 ms 2024-09-15T22:53:39.271 INFO:tasks.workunit.client.0.smithi111.stdout: 1 689625 9.92 MB/sec execute 247 sec latency 15.800 ms 2024-09-15T22:53:40.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:39 smithi192 ceph-mon[35427]: pgmap v582: 97 pgs: 97 active+clean; 248 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 192 op/s 2024-09-15T22:53:40.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:39 smithi121 ceph-mon[34815]: pgmap v582: 97 pgs: 97 active+clean; 248 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 192 op/s 2024-09-15T22:53:40.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:39 smithi111 ceph-mon[30629]: pgmap v582: 97 pgs: 97 active+clean; 248 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 192 op/s 2024-09-15T22:53:40.272 INFO:tasks.workunit.client.0.smithi111.stdout: 1 692775 9.96 MB/sec execute 248 sec latency 13.197 ms 2024-09-15T22:53:41.272 INFO:tasks.workunit.client.0.smithi111.stdout: 1 694485 9.94 MB/sec execute 249 sec latency 283.587 ms 2024-09-15T22:53:42.081 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:41 smithi111 ceph-mon[30629]: pgmap v583: 97 pgs: 97 active+clean; 251 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 222 op/s 2024-09-15T22:53:42.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:41 smithi121 ceph-mon[34815]: pgmap v583: 97 pgs: 97 active+clean; 251 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 222 op/s 2024-09-15T22:53:42.272 INFO:tasks.workunit.client.0.smithi111.stdout: 1 694526 9.90 MB/sec execute 250 sec latency 1033.969 ms 2024-09-15T22:53:42.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:41 smithi192 ceph-mon[35427]: pgmap v583: 97 pgs: 97 active+clean; 251 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 222 op/s 2024-09-15T22:53:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.043+0000 7fb038c00640 1 -- 172.21.15.111:0/132619881 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb034104f50 msgr2=0x7fb0341053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.043+0000 7fb038c00640 1 --2- 172.21.15.111:0/132619881 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb034104f50 0x7fb0341053d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb020009900 tx=0x7fb02002a3f0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.043+0000 7fb038c00640 1 -- 172.21.15.111:0/132619881 shutdown_connections 2024-09-15T22:53:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.043+0000 7fb038c00640 1 --2- 172.21.15.111:0/132619881 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb034105910 0x7fb03410c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.043+0000 7fb038c00640 1 --2- 172.21.15.111:0/132619881 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb034104f50 0x7fb0341053d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.045 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.043+0000 7fb038c00640 1 --2- 172.21.15.111:0/132619881 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb034103d60 0x7fb034104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.043+0000 7fb038c00640 1 -- 172.21.15.111:0/132619881 >> 172.21.15.111:0/132619881 conn(0x7fb0340ff4e0 msgr2=0x7fb034101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.043+0000 7fb038c00640 1 -- 172.21.15.111:0/132619881 shutdown_connections 2024-09-15T22:53:43.046 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.043+0000 7fb038c00640 1 -- 172.21.15.111:0/132619881 wait complete. 2024-09-15T22:53:43.047 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.044+0000 7fb038c00640 1 Processor -- start 2024-09-15T22:53:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.044+0000 7fb038c00640 1 -- start start 2024-09-15T22:53:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.044+0000 7fb038c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb034103d60 0x7fb03419a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.045+0000 7fb038c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb034104f50 0x7fb03419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.045+0000 7fb038c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb034105910 0x7fb0341a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.045+0000 7fb038c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb03410fa10 con 0x7fb034104f50 2024-09-15T22:53:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.045+0000 7fb038c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb03410f890 con 0x7fb034105910 2024-09-15T22:53:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.045+0000 7fb038c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb03410fb90 con 0x7fb034103d60 2024-09-15T22:53:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.045+0000 7fb031e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb034103d60 0x7fb03419a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.045+0000 7fb032800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb034105910 0x7fb0341a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.045+0000 7fb031e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb034103d60 0x7fb03419a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:35726/0 (socket says 172.21.15.111:35726) 2024-09-15T22:53:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.045+0000 7fb031e00640 1 -- 172.21.15.111:0/717254086 learned_addr learned my addr 172.21.15.111:0/717254086 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:53:43.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.045+0000 7fb031400640 1 --2- 172.21.15.111:0/717254086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb034104f50 0x7fb03419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.045+0000 7fb031e00640 1 -- 172.21.15.111:0/717254086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb034105910 msgr2=0x7fb0341a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.045+0000 7fb031e00640 1 --2- 172.21.15.111:0/717254086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb034105910 0x7fb0341a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.045+0000 7fb031e00640 1 -- 172.21.15.111:0/717254086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb034104f50 msgr2=0x7fb03419a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.045+0000 7fb031e00640 1 --2- 172.21.15.111:0/717254086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb034104f50 0x7fb03419a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.045+0000 7fb031e00640 1 -- 172.21.15.111:0/717254086 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0341a1c20 con 0x7fb034103d60 2024-09-15T22:53:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.045+0000 7fb032800640 1 --2- 172.21.15.111:0/717254086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb034105910 0x7fb0341a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:53:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.045+0000 7fb031e00640 1 --2- 172.21.15.111:0/717254086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb034103d60 0x7fb03419a000 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb02400e9c0 tx=0x7fb02400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.046+0000 7fb02aa00640 1 -- 172.21.15.111:0/717254086 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02400cd80 con 0x7fb034103d60 2024-09-15T22:53:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.046+0000 7fb02aa00640 1 -- 172.21.15.111:0/717254086 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0240035f0 con 0x7fb034103d60 2024-09-15T22:53:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.046+0000 7fb038c00640 1 -- 172.21.15.111:0/717254086 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0341a1f10 con 0x7fb034103d60 2024-09-15T22:53:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.046+0000 7fb02aa00640 1 -- 172.21.15.111:0/717254086 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024010690 con 0x7fb034103d60 2024-09-15T22:53:43.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.046+0000 7fb038c00640 1 -- 172.21.15.111:0/717254086 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0341a2370 con 0x7fb034103d60 2024-09-15T22:53:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.049+0000 7fb02aa00640 1 -- 172.21.15.111:0/717254086 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fb0240037a0 con 0x7fb034103d60 2024-09-15T22:53:43.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.049+0000 7fb03ba69740 1 Processor -- start 2024-09-15T22:53:43.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.049+0000 7fb03ba69740 1 -- start start 2024-09-15T22:53:43.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.050+0000 7fb03ba69740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4d70b9090 0x55e4d70b9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:43.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.050+0000 7fb03ba69740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e4d70b99d0 0x55e4d70be490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.050+0000 7fb03ba69740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4d70bea60 0x55e4d70c0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.050+0000 7fb03ba69740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e4d701e100 con 0x55e4d70b9090 2024-09-15T22:53:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.050+0000 7fb03ba69740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e4d6fc61b0 con 0x55e4d70bea60 2024-09-15T22:53:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.050+0000 7fb03ba69740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55e4d701a0c0 con 0x55e4d70b99d0 2024-09-15T22:53:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.050+0000 7fb032800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4d70b9090 0x55e4d70b9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.050+0000 7fb031e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e4d70b99d0 0x55e4d70be490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.050+0000 7fb032800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4d70b9090 0x55e4d70b9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42158/0 (socket says 172.21.15.111:42158) 2024-09-15T22:53:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.050+0000 7fb032800640 1 -- 172.21.15.111:0/3368262900 learned_addr learned my addr 172.21.15.111:0/3368262900 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:53:43.052 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.050+0000 7fb031400640 1 --2- 172.21.15.111:0/3368262900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4d70bea60 0x55e4d70c0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.050+0000 7fb031400640 1 -- 172.21.15.111:0/3368262900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e4d70b99d0 msgr2=0x55e4d70be490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.050+0000 7fb031400640 1 --2- 172.21.15.111:0/3368262900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e4d70b99d0 0x55e4d70be490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.050+0000 7fb031400640 1 -- 172.21.15.111:0/3368262900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4d70b9090 msgr2=0x55e4d70b9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.050+0000 7fb031400640 1 --2- 172.21.15.111:0/3368262900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4d70b9090 0x55e4d70b9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.050+0000 7fb031400640 1 -- 172.21.15.111:0/3368262900 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4d6d31210 con 0x55e4d70bea60 2024-09-15T22:53:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.050+0000 7fb031e00640 1 --2- 172.21.15.111:0/3368262900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e4d70b99d0 0x55e4d70be490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:53:43.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.051+0000 7fb02aa00640 1 --2- 172.21.15.111:0/717254086 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb0180a32f0 0x7fb0180a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.051+0000 7fb031400640 1 --2- 172.21.15.111:0/3368262900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4d70bea60 0x55e4d70c0f20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb02002a9e0 tx=0x7fb020005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4bc3b30b0b647cf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.051+0000 7fb02aa00640 1 -- 172.21.15.111:0/717254086 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fb0240cfce0 con 0x7fb034103d60 2024-09-15T22:53:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.051+0000 7fb01f000640 1 -- 172.21.15.111:0/3368262900 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb020035040 con 0x55e4d70bea60 2024-09-15T22:53:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.051+0000 7fb01f000640 1 -- 172.21.15.111:0/3368262900 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb020004400 con 0x55e4d70bea60 2024-09-15T22:53:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.051+0000 7fb01f000640 1 -- 172.21.15.111:0/3368262900 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb020004e60 con 0x55e4d70bea60 2024-09-15T22:53:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.051+0000 7fb032800640 1 --2- 172.21.15.111:0/717254086 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb0180a32f0 0x7fb0180a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.052+0000 7fb03ba69740 1 -- 172.21.15.111:0/3368262900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4d70bea60 msgr2=0x55e4d70c0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.052+0000 7fb032800640 1 --2- 172.21.15.111:0/717254086 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb0180a32f0 0x7fb0180a57b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009fd0 tx=0x7fb02c009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.052+0000 7fb03ba69740 1 --2- 172.21.15.111:0/3368262900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4d70bea60 0x55e4d70c0f20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb02002a9e0 tx=0x7fb020005dd0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.052+0000 7fb03ba69740 1 -- 172.21.15.111:0/3368262900 shutdown_connections 2024-09-15T22:53:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.052+0000 7fb03ba69740 1 --2- 172.21.15.111:0/3368262900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4d70bea60 0x55e4d70c0f20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.052+0000 7fb03ba69740 1 --2- 172.21.15.111:0/3368262900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e4d70b99d0 0x55e4d70be490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.055 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.052+0000 7fb03ba69740 1 --2- 172.21.15.111:0/3368262900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4d70b9090 0x55e4d70b9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.052+0000 7fb03ba69740 1 -- 172.21.15.111:0/3368262900 >> 172.21.15.111:0/3368262900 conn(0x55e4d70b4700 msgr2=0x55e4d70b4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.052+0000 7fb03ba69740 1 -- 172.21.15.111:0/3368262900 shutdown_connections 2024-09-15T22:53:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.052+0000 7fb03ba69740 1 -- 172.21.15.111:0/3368262900 wait complete. 2024-09-15T22:53:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.053+0000 7fb03ba69740 1 -- start start 2024-09-15T22:53:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.053+0000 7fb03ba69740 1 Processor -- start 2024-09-15T22:53:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.053+0000 7fb03ba69740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4d70b9090 0x55e4d70c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.053+0000 7fb03ba69740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e4d70b99d0 0x55e4d70c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.053+0000 7fb03ba69740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4d70bea60 0x55e4d70ca820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.053+0000 7fb03ba69740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55e4d70c19d0 con 0x55e4d70b9090 2024-09-15T22:53:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.053+0000 7fb03ba69740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e4d6fa7090 con 0x55e4d70bea60 2024-09-15T22:53:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.053+0000 7fb03ba69740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55e4d70c1b50 con 0x55e4d70b99d0 2024-09-15T22:53:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.053+0000 7fb03ba69740 10 client.0 _ll_register_callbacks cb 0x55e4d6ffbf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:53:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.053+0000 7fb032800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4d70bea60 0x55e4d70ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.053+0000 7fb031e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4d70b9090 0x55e4d70c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.053+0000 7fb032800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4d70bea60 0x55e4d70ca820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:44220/0 (socket says 172.21.15.111:44220) 2024-09-15T22:53:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.053+0000 7fb032800640 1 -- 172.21.15.111:0/1604673668 learned_addr learned my addr 172.21.15.111:0/1604673668 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:53:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.053+0000 7fb031400640 1 --2- 172.21.15.111:0/1604673668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e4d70b99d0 0x55e4d70c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.054+0000 7fb032800640 1 -- 172.21.15.111:0/1604673668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e4d70b99d0 msgr2=0x55e4d70c37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.054+0000 7fb032800640 1 --2- 172.21.15.111:0/1604673668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e4d70b99d0 0x55e4d70c37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.054+0000 7fb032800640 1 -- 172.21.15.111:0/1604673668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4d70b9090 msgr2=0x55e4d70c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.054+0000 7fb032800640 1 --2- 172.21.15.111:0/1604673668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4d70b9090 0x55e4d70c32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.054+0000 7fb032800640 1 -- 172.21.15.111:0/1604673668 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4d6dfd620 con 0x55e4d70bea60 2024-09-15T22:53:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.054+0000 7fb032800640 1 --2- 172.21.15.111:0/1604673668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4d70bea60 0x55e4d70ca820 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb02c011b80 tx=0x7fb02c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8660386c14e887c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.054+0000 7fb013400640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:53:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.054+0000 7fb013400640 1 -- 172.21.15.111:0/1604673668 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c02d060 con 0x55e4d70bea60 2024-09-15T22:53:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.054+0000 7fb013400640 1 -- 172.21.15.111:0/1604673668 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fb02c017030 con 0x55e4d70bea60 2024-09-15T22:53:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.054+0000 7fb03ba69740 1 -- 172.21.15.111:0/1604673668 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb020004400 con 0x55e4d70bea60 2024-09-15T22:53:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.055+0000 7fb013400640 1 -- 172.21.15.111:0/1604673668 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb02c01d560 con 0x55e4d70bea60 2024-09-15T22:53:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.055+0000 7fb03ba69740 10 client.25607 fetch_fsmap learned FSMap version 30 2024-09-15T22:53:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.055+0000 7fb03ba69740 1 -- 172.21.15.111:0/1604673668 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e4d70cd730 con 0x55e4d70bea60 2024-09-15T22:53:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.055+0000 7fb013400640 1 -- 172.21.15.111:0/1604673668 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb020004400 con 0x55e4d70bea60 2024-09-15T22:53:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.055+0000 7fb013400640 1 -- 172.21.15.111:0/1604673668 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fb02c032070 con 0x55e4d70bea60 2024-09-15T22:53:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.055+0000 7fb03ba69740 10 client.25607 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:53:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.055+0000 7fb03ba69740 10 client.25607 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:53:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.055+0000 7fb03ba69740 20 client.25607 populate_metadata read hostname 'smithi111' 2024-09-15T22:53:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.055+0000 7fb03ba69740 1 --2- 172.21.15.111:0/1604673668 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55e4d70cee00 0x55e4d70d1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.055+0000 7fb03ba69740 4 client.25607 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:53:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.055+0000 7fb03ba69740 1 -- 172.21.15.111:0/1604673668 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4d702b830 con 0x55e4d70cee00 2024-09-15T22:53:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.056+0000 7fb031e00640 1 --2- 172.21.15.111:0/1604673668 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55e4d70cee00 0x55e4d70d1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.056+0000 7fb031e00640 1 --2- 172.21.15.111:0/1604673668 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55e4d70cee00 0x55e4d70d1230 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae7529d2e247d022 server_cookie=37c0378d79fe33e3 in_seq=0 out_seq=0 2024-09-15T22:53:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.056+0000 7fb013400640 10 client.25607 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:53:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.057+0000 7fb013400640 1 -- 172.21.15.111:0/1604673668 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e4d702b830 con 0x55e4d70cee00 2024-09-15T22:53:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.057+0000 7fb013400640 10 client.25607 handle_command_reply: tid=0 2024-09-15T22:53:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.057+0000 7fb03ba69740 1 client.25607 shutdown 2024-09-15T22:53:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.057+0000 7fb03ba69740 1 -- 172.21.15.111:0/1604673668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4d70bea60 msgr2=0x55e4d70ca820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.057+0000 7fb03ba69740 1 --2- 172.21.15.111:0/1604673668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4d70bea60 0x55e4d70ca820 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb02c011b80 tx=0x7fb02c004ab0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.057+0000 7fb03ba69740 1 -- 172.21.15.111:0/1604673668 shutdown_connections 2024-09-15T22:53:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.057+0000 7fb03ba69740 1 --2- 172.21.15.111:0/1604673668 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55e4d70cee00 0x55e4d70d1230 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.057+0000 7fb03ba69740 1 --2- 172.21.15.111:0/1604673668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4d70bea60 0x55e4d70ca820 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.057+0000 7fb03ba69740 1 --2- 172.21.15.111:0/1604673668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e4d70b99d0 0x55e4d70c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.057+0000 7fb03ba69740 1 --2- 172.21.15.111:0/1604673668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4d70b9090 0x55e4d70c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.057+0000 7fb013400640 0 client.25607 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:53:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.057+0000 7fb03ba69740 1 -- 172.21.15.111:0/1604673668 >> 172.21.15.111:0/1604673668 conn(0x55e4d70b4700 msgr2=0x55e4d70b5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.057+0000 7fb03ba69740 1 -- 172.21.15.111:0/1604673668 shutdown_connections 2024-09-15T22:53:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.057+0000 7fb03ba69740 1 -- 172.21.15.111:0/1604673668 wait complete. 2024-09-15T22:53:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.057+0000 7fb03ba69740 20 client.25607 trim_cache size 0 max 16384 2024-09-15T22:53:43.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.074+0000 7fb03ba69740 1 Processor -- start 2024-09-15T22:53:43.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.074+0000 7fb03ba69740 1 -- start start 2024-09-15T22:53:43.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.074+0000 7fb03ba69740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4d70b9090 0x55e4d70b3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.074+0000 7fb03ba69740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e4d70b99d0 0x55e4d70ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.074+0000 7fb03ba69740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4d70bea60 0x55e4d70d5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.074+0000 7fb03ba69740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb0341ad8d0 con 0x55e4d70b9090 2024-09-15T22:53:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.074+0000 7fb03ba69740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb0341ab1a0 con 0x55e4d70bea60 2024-09-15T22:53:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.074+0000 7fb03ba69740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb0341a8ff0 con 0x55e4d70b99d0 2024-09-15T22:53:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.074+0000 7fb031e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4d70b9090 0x55e4d70b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.074+0000 7fb031e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4d70b9090 0x55e4d70b3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42166/0 (socket says 172.21.15.111:42166) 2024-09-15T22:53:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.074+0000 7fb031e00640 1 -- 172.21.15.111:0/368122636 learned_addr learned my addr 172.21.15.111:0/368122636 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:53:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.074+0000 7fb031400640 1 --2- 172.21.15.111:0/368122636 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e4d70b99d0 0x55e4d70ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.075+0000 7fb032800640 1 --2- 172.21.15.111:0/368122636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4d70bea60 0x55e4d70d5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.075+0000 7fb031e00640 1 -- 172.21.15.111:0/368122636 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e4d70b99d0 msgr2=0x55e4d70ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:43.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.075+0000 7fb031e00640 1 --2- 172.21.15.111:0/368122636 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e4d70b99d0 0x55e4d70ca7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.078 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.075+0000 7fb031e00640 1 -- 172.21.15.111:0/368122636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4d70bea60 msgr2=0x55e4d70d5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:43.078 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.075+0000 7fb031e00640 1 --2- 172.21.15.111:0/368122636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4d70bea60 0x55e4d70d5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.078 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.075+0000 7fb031e00640 1 -- 172.21.15.111:0/368122636 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4d70c3070 con 0x55e4d70b9090 2024-09-15T22:53:43.078 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.075+0000 7fb031400640 1 --2- 172.21.15.111:0/368122636 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e4d70b99d0 0x55e4d70ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:53:43.078 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.075+0000 7fb031e00640 1 --2- 172.21.15.111:0/368122636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4d70b9090 0x55e4d70b3770 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb0240058c0 tx=0x7fb0240cd6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7666bec9cd564616 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.075+0000 7fb01e600640 1 -- 172.21.15.111:0/368122636 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02409be40 con 0x55e4d70b9090 2024-09-15T22:53:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.075+0000 7fb01e600640 1 -- 172.21.15.111:0/368122636 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb02409ca00 con 0x55e4d70b9090 2024-09-15T22:53:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.075+0000 7fb01e600640 1 -- 172.21.15.111:0/368122636 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02409bc60 con 0x55e4d70b9090 2024-09-15T22:53:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.076+0000 7fb03ba69740 1 -- 172.21.15.111:0/368122636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4d70b9090 msgr2=0x55e4d70b3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.076+0000 7fb03ba69740 1 --2- 172.21.15.111:0/368122636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4d70b9090 0x55e4d70b3770 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb0240058c0 tx=0x7fb0240cd6b0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.076+0000 7fb03ba69740 1 -- 172.21.15.111:0/368122636 shutdown_connections 2024-09-15T22:53:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.076+0000 7fb03ba69740 1 --2- 172.21.15.111:0/368122636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4d70bea60 0x55e4d70d5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.076+0000 7fb03ba69740 1 --2- 172.21.15.111:0/368122636 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e4d70b99d0 0x55e4d70ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.076+0000 7fb03ba69740 1 --2- 172.21.15.111:0/368122636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4d70b9090 0x55e4d70b3770 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.076+0000 7fb03ba69740 1 -- 172.21.15.111:0/368122636 >> 172.21.15.111:0/368122636 conn(0x55e4d70b4700 msgr2=0x55e4d70b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.076+0000 7fb03ba69740 1 -- 172.21.15.111:0/368122636 shutdown_connections 2024-09-15T22:53:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.076+0000 7fb03ba69740 1 -- 172.21.15.111:0/368122636 wait complete. 2024-09-15T22:53:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.077+0000 7fb03ba69740 1 -- start start 2024-09-15T22:53:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.077+0000 7fb03ba69740 1 Processor -- start 2024-09-15T22:53:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.077+0000 7fb03ba69740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4d70b9090 0x55e4d70c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.077+0000 7fb03ba69740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e4d70b99d0 0x55e4d70c8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.077+0000 7fb03ba69740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4d70bea60 0x55e4d70c9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.077+0000 7fb03ba69740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb0341a9660 con 0x55e4d70bea60 2024-09-15T22:53:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.077+0000 7fb03ba69740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb0341aaad0 con 0x55e4d70b9090 2024-09-15T22:53:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.077+0000 7fb03ba69740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb0341a9d30 con 0x55e4d70b99d0 2024-09-15T22:53:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.077+0000 7fb03ba69740 10 client.0 _ll_register_callbacks cb 0x7fb02c003bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:53:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.078+0000 7fb031e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4d70b9090 0x55e4d70c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.078+0000 7fb031400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e4d70b99d0 0x55e4d70c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.078+0000 7fb031e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4d70b9090 0x55e4d70c7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:44238/0 (socket says 172.21.15.111:44238) 2024-09-15T22:53:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.078+0000 7fb031e00640 1 -- 172.21.15.111:0/963002559 learned_addr learned my addr 172.21.15.111:0/963002559 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:53:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.078+0000 7fb032800640 1 --2- 172.21.15.111:0/963002559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4d70bea60 0x55e4d70c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.078+0000 7fb031e00640 1 -- 172.21.15.111:0/963002559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e4d70b99d0 msgr2=0x55e4d70c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.078+0000 7fb031e00640 1 --2- 172.21.15.111:0/963002559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e4d70b99d0 0x55e4d70c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.078+0000 7fb031e00640 1 -- 172.21.15.111:0/963002559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4d70bea60 msgr2=0x55e4d70c9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.078+0000 7fb031e00640 1 --2- 172.21.15.111:0/963002559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4d70bea60 0x55e4d70c9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.078+0000 7fb031e00640 1 -- 172.21.15.111:0/963002559 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4d70c9a80 con 0x55e4d70b9090 2024-09-15T22:53:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.078+0000 7fb032800640 1 --2- 172.21.15.111:0/963002559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4d70bea60 0x55e4d70c9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:53:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.078+0000 7fb031e00640 1 --2- 172.21.15.111:0/963002559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4d70b9090 0x55e4d70c7f20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb024098cd0 tx=0x7fb024005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef7705aaa051aa8d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.078+0000 7fb013400640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:53:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.078+0000 7fb013400640 1 -- 172.21.15.111:0/963002559 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02409be40 con 0x55e4d70b9090 2024-09-15T22:53:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.078+0000 7fb013400640 1 -- 172.21.15.111:0/963002559 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fb0240b5170 con 0x55e4d70b9090 2024-09-15T22:53:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.078+0000 7fb03ba69740 1 -- 172.21.15.111:0/963002559 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb02409ca00 con 0x55e4d70b9090 2024-09-15T22:53:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.079+0000 7fb013400640 1 -- 172.21.15.111:0/963002559 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb02409c300 con 0x55e4d70b9090 2024-09-15T22:53:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.079+0000 7fb03ba69740 10 client.25613 fetch_fsmap learned FSMap version 30 2024-09-15T22:53:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.079+0000 7fb03ba69740 1 -- 172.21.15.111:0/963002559 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e4d7139b10 con 0x55e4d70b9090 2024-09-15T22:53:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.079+0000 7fb013400640 1 -- 172.21.15.111:0/963002559 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb02409ca00 con 0x55e4d70b9090 2024-09-15T22:53:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.079+0000 7fb013400640 1 -- 172.21.15.111:0/963002559 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fb0240a02b0 con 0x55e4d70b9090 2024-09-15T22:53:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.079+0000 7fb03ba69740 10 client.25613 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:53:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.079+0000 7fb03ba69740 10 client.25613 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:53:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.079+0000 7fb03ba69740 20 client.25613 populate_metadata read hostname 'smithi111' 2024-09-15T22:53:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.079+0000 7fb03ba69740 1 --2- 172.21.15.111:0/963002559 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55e4d713a360 0x55e4d713c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.079+0000 7fb03ba69740 4 client.25613 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:53:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.079+0000 7fb03ba69740 1 -- 172.21.15.111:0/963002559 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e4d7025b80 con 0x55e4d713a360 2024-09-15T22:53:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.079+0000 7fb031400640 1 --2- 172.21.15.111:0/963002559 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55e4d713a360 0x55e4d713c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.080+0000 7fb031400640 1 --2- 172.21.15.111:0/963002559 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55e4d713a360 0x55e4d713c780 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf7a3459822053ae server_cookie=f8407f8643ea8129 in_seq=0 out_seq=0 2024-09-15T22:53:43.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.080+0000 7fb013400640 10 client.25613 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:53:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.089+0000 7fb013400640 1 -- 172.21.15.111:0/963002559 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e4d7025b80 con 0x55e4d713a360 2024-09-15T22:53:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.089+0000 7fb013400640 10 client.25613 handle_command_reply: tid=0 2024-09-15T22:53:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.089+0000 7fb03ba69740 1 client.25613 shutdown 2024-09-15T22:53:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.089+0000 7fb03ba69740 1 -- 172.21.15.111:0/963002559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4d70b9090 msgr2=0x55e4d70c7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.089+0000 7fb03ba69740 1 --2- 172.21.15.111:0/963002559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4d70b9090 0x55e4d70c7f20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb024098cd0 tx=0x7fb024005bf0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.089+0000 7fb03ba69740 1 -- 172.21.15.111:0/963002559 shutdown_connections 2024-09-15T22:53:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.089+0000 7fb03ba69740 1 --2- 172.21.15.111:0/963002559 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55e4d713a360 0x55e4d713c780 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.089+0000 7fb03ba69740 1 --2- 172.21.15.111:0/963002559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55e4d70bea60 0x55e4d70c9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.089+0000 7fb03ba69740 1 --2- 172.21.15.111:0/963002559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e4d70b99d0 0x55e4d70c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.089+0000 7fb03ba69740 1 --2- 172.21.15.111:0/963002559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e4d70b9090 0x55e4d70c7f20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.089+0000 7fb013400640 0 client.25613 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:53:43.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.089+0000 7fb03ba69740 1 -- 172.21.15.111:0/963002559 >> 172.21.15.111:0/963002559 conn(0x55e4d70b4700 msgr2=0x55e4d70b5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:43.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.089+0000 7fb03ba69740 1 -- 172.21.15.111:0/963002559 shutdown_connections 2024-09-15T22:53:43.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.089+0000 7fb03ba69740 1 -- 172.21.15.111:0/963002559 wait complete. 2024-09-15T22:53:43.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.089+0000 7fb03ba69740 20 client.25613 trim_cache size 0 max 16384 2024-09-15T22:53:43.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:53:43.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:53:43.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:53:43.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:53:43.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.090+0000 7fb038c00640 1 -- 172.21.15.111:0/717254086 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb0180a32f0 msgr2=0x7fb0180a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:43.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.090+0000 7fb038c00640 1 --2- 172.21.15.111:0/717254086 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb0180a32f0 0x7fb0180a57b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009fd0 tx=0x7fb02c009500 comp rx=0 tx=0).stop 2024-09-15T22:53:43.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.090+0000 7fb038c00640 1 -- 172.21.15.111:0/717254086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb034103d60 msgr2=0x7fb03419a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:43.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.090+0000 7fb038c00640 1 --2- 172.21.15.111:0/717254086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb034103d60 0x7fb03419a000 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb02400e9c0 tx=0x7fb02400ee80 comp rx=0 tx=0).stop 2024-09-15T22:53:43.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.090+0000 7fb038c00640 1 -- 172.21.15.111:0/717254086 shutdown_connections 2024-09-15T22:53:43.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.090+0000 7fb038c00640 1 --2- 172.21.15.111:0/717254086 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb0180a32f0 0x7fb0180a57b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.090+0000 7fb038c00640 1 --2- 172.21.15.111:0/717254086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb034105910 0x7fb0341a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.090+0000 7fb038c00640 1 --2- 172.21.15.111:0/717254086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb034104f50 0x7fb03419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.090+0000 7fb038c00640 1 --2- 172.21.15.111:0/717254086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb034103d60 0x7fb03419a000 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:43.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.090+0000 7fb038c00640 1 -- 172.21.15.111:0/717254086 >> 172.21.15.111:0/717254086 conn(0x7fb0340ff4e0 msgr2=0x7fb0341081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:43.094 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.090+0000 7fb038c00640 1 -- 172.21.15.111:0/717254086 shutdown_connections 2024-09-15T22:53:43.094 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:43.090+0000 7fb038c00640 1 -- 172.21.15.111:0/717254086 wait complete. 2024-09-15T22:53:43.272 INFO:tasks.workunit.client.0.smithi111.stdout: 1 696512 9.92 MB/sec execute 251 sec latency 473.323 ms 2024-09-15T22:53:44.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:43 smithi121 ceph-mon[34815]: pgmap v584: 97 pgs: 97 active+clean; 255 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 186 op/s 2024-09-15T22:53:44.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:43 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:53:44.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:43 smithi111 ceph-mon[30629]: pgmap v584: 97 pgs: 97 active+clean; 255 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 186 op/s 2024-09-15T22:53:44.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:43 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:53:44.272 INFO:tasks.workunit.client.0.smithi111.stdout: 1 699654 9.95 MB/sec execute 252 sec latency 12.425 ms 2024-09-15T22:53:44.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:43 smithi192 ceph-mon[35427]: pgmap v584: 97 pgs: 97 active+clean; 255 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 186 op/s 2024-09-15T22:53:44.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:43 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:53:45.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:44 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:53:45.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:44 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:53:45.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:44 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:53:45.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:44 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:53:45.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:44 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:53:45.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:44 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:53:45.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:44 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:53:45.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:44 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:53:45.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:44 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:53:45.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:44 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:53:45.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:44 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:53:45.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:44 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:53:45.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:44 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:53:45.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:44 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:53:45.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:44 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:53:45.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:44 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:53:45.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:44 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:53:45.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:44 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:53:45.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:44 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:53:45.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:44 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:53:45.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:44 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:53:45.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:44 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:53:45.272 INFO:tasks.workunit.client.0.smithi111.stdout: 1 702038 9.94 MB/sec execute 253 sec latency 15.006 ms 2024-09-15T22:53:45.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:44 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:53:45.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:44 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:53:45.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:44 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:53:45.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:44 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:53:45.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:44 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:53:45.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:44 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:53:45.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:44 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:53:45.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:44 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:53:45.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:44 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:53:45.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:44 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:53:45.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:44 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:53:45.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:45 smithi111 ceph-mon[30629]: Adjusting osd_memory_target on smithi121 to 175.9M 2024-09-15T22:53:45.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:45 smithi111 ceph-mon[30629]: Adjusting osd_memory_target on smithi192 to 176.0M 2024-09-15T22:53:45.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:45 smithi111 ceph-mon[30629]: Unable to set osd_memory_target on smithi121 to 184532275: error parsing value: Value '184532275' is below minimum 939524096 2024-09-15T22:53:45.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:45 smithi111 ceph-mon[30629]: Unable to set osd_memory_target on smithi192 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-15T22:53:45.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:45 smithi111 ceph-mon[30629]: pgmap v585: 97 pgs: 97 active+clean; 255 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-09-15T22:53:46.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:45 smithi121 ceph-mon[34815]: Adjusting osd_memory_target on smithi121 to 175.9M 2024-09-15T22:53:46.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:45 smithi121 ceph-mon[34815]: Adjusting osd_memory_target on smithi192 to 176.0M 2024-09-15T22:53:46.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:45 smithi121 ceph-mon[34815]: Unable to set osd_memory_target on smithi121 to 184532275: error parsing value: Value '184532275' is below minimum 939524096 2024-09-15T22:53:46.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:45 smithi121 ceph-mon[34815]: Unable to set osd_memory_target on smithi192 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-15T22:53:46.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:45 smithi121 ceph-mon[34815]: pgmap v585: 97 pgs: 97 active+clean; 255 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-09-15T22:53:46.272 INFO:tasks.workunit.client.0.smithi111.stdout: 1 703863 9.95 MB/sec execute 254 sec latency 190.275 ms 2024-09-15T22:53:46.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:45 smithi192 ceph-mon[35427]: Adjusting osd_memory_target on smithi121 to 175.9M 2024-09-15T22:53:46.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:45 smithi192 ceph-mon[35427]: Adjusting osd_memory_target on smithi192 to 176.0M 2024-09-15T22:53:46.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:45 smithi192 ceph-mon[35427]: Unable to set osd_memory_target on smithi121 to 184532275: error parsing value: Value '184532275' is below minimum 939524096 2024-09-15T22:53:46.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:45 smithi192 ceph-mon[35427]: Unable to set osd_memory_target on smithi192 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-15T22:53:46.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:45 smithi192 ceph-mon[35427]: pgmap v585: 97 pgs: 97 active+clean; 255 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-09-15T22:53:47.272 INFO:tasks.workunit.client.0.smithi111.stdout: 1 703863 9.91 MB/sec execute 255 sec latency 1190.317 ms 2024-09-15T22:53:48.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:47 smithi121 ceph-mon[34815]: pgmap v586: 97 pgs: 97 active+clean; 257 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 217 op/s 2024-09-15T22:53:48.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:47 smithi111 ceph-mon[30629]: pgmap v586: 97 pgs: 97 active+clean; 257 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 217 op/s 2024-09-15T22:53:48.272 INFO:tasks.workunit.client.0.smithi111.stdout: 1 705293 9.90 MB/sec execute 256 sec latency 1783.970 ms 2024-09-15T22:53:48.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:47 smithi192 ceph-mon[35427]: pgmap v586: 97 pgs: 97 active+clean; 257 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 217 op/s 2024-09-15T22:53:49.272 INFO:tasks.workunit.client.0.smithi111.stdout: 1 707732 9.92 MB/sec execute 257 sec latency 11.757 ms 2024-09-15T22:53:50.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:49 smithi121 ceph-mon[34815]: pgmap v587: 97 pgs: 97 active+clean; 255 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 227 op/s 2024-09-15T22:53:50.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:49 smithi111 ceph-mon[30629]: pgmap v587: 97 pgs: 97 active+clean; 255 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 227 op/s 2024-09-15T22:53:50.272 INFO:tasks.workunit.client.0.smithi111.stdout: 1 710756 9.94 MB/sec execute 258 sec latency 11.969 ms 2024-09-15T22:53:50.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:49 smithi192 ceph-mon[35427]: pgmap v587: 97 pgs: 97 active+clean; 255 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 227 op/s 2024-09-15T22:53:51.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:50 smithi121 ceph-mon[34815]: pgmap v588: 97 pgs: 97 active+clean; 259 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 223 op/s 2024-09-15T22:53:51.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:50 smithi111 ceph-mon[30629]: pgmap v588: 97 pgs: 97 active+clean; 259 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 223 op/s 2024-09-15T22:53:51.272 INFO:tasks.workunit.client.0.smithi111.stdout: 1 713775 9.97 MB/sec execute 259 sec latency 76.800 ms 2024-09-15T22:53:51.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:50 smithi192 ceph-mon[35427]: pgmap v588: 97 pgs: 97 active+clean; 259 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 223 op/s 2024-09-15T22:53:52.272 INFO:tasks.workunit.client.0.smithi111.stdout: 1 713776 9.94 MB/sec execute 260 sec latency 852.459 ms 2024-09-15T22:53:53.272 INFO:tasks.workunit.client.0.smithi111.stdout: 1 714450 9.90 MB/sec execute 261 sec latency 1400.090 ms 2024-09-15T22:53:54.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:53 smithi192 ceph-mon[35427]: pgmap v589: 97 pgs: 97 active+clean; 261 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 190 op/s 2024-09-15T22:53:54.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:53 smithi121 ceph-mon[34815]: pgmap v589: 97 pgs: 97 active+clean; 261 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 190 op/s 2024-09-15T22:53:54.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:53 smithi111 ceph-mon[30629]: pgmap v589: 97 pgs: 97 active+clean; 261 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 190 op/s 2024-09-15T22:53:54.272 INFO:tasks.workunit.client.0.smithi111.stdout: 1 717555 9.94 MB/sec execute 262 sec latency 13.201 ms 2024-09-15T22:53:55.273 INFO:tasks.workunit.client.0.smithi111.stdout: 1 720289 9.95 MB/sec execute 263 sec latency 18.537 ms 2024-09-15T22:53:55.936 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:55 smithi111 ceph-mon[30629]: pgmap v590: 97 pgs: 97 active+clean; 261 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 183 op/s 2024-09-15T22:53:56.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:55 smithi192 ceph-mon[35427]: pgmap v590: 97 pgs: 97 active+clean; 261 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 183 op/s 2024-09-15T22:53:56.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:55 smithi121 ceph-mon[34815]: pgmap v590: 97 pgs: 97 active+clean; 261 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 183 op/s 2024-09-15T22:53:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.218+0000 7f8b7ca00640 1 -- 172.21.15.111:0/2742172765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b78102680 msgr2=0x7f8b78104a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.218+0000 7f8b7ca00640 1 --2- 172.21.15.111:0/2742172765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b78102680 0x7f8b78104a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009900 tx=0x7f8b6802a3f0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.218+0000 7f8b7ca00640 1 -- 172.21.15.111:0/2742172765 shutdown_connections 2024-09-15T22:53:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.218+0000 7f8b7ca00640 1 --2- 172.21.15.111:0/2742172765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b781078e0 0x7f8b78109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.218+0000 7f8b7ca00640 1 --2- 172.21.15.111:0/2742172765 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b78104fb0 0x7f8b781073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.218+0000 7f8b7ca00640 1 --2- 172.21.15.111:0/2742172765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b78102680 0x7f8b78104a70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.218+0000 7f8b7ca00640 1 -- 172.21.15.111:0/2742172765 >> 172.21.15.111:0/2742172765 conn(0x7f8b780fc1e0 msgr2=0x7f8b780fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.218+0000 7f8b7ca00640 1 -- 172.21.15.111:0/2742172765 shutdown_connections 2024-09-15T22:53:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.218+0000 7f8b7ca00640 1 -- 172.21.15.111:0/2742172765 wait complete. 2024-09-15T22:53:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.219+0000 7f8b7ca00640 1 Processor -- start 2024-09-15T22:53:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.219+0000 7f8b7ca00640 1 -- start start 2024-09-15T22:53:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.219+0000 7f8b7ca00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b78102680 0x7f8b7819a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.219+0000 7f8b7ca00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b78104fb0 0x7f8b7819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.219+0000 7f8b7ca00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b781078e0 0x7f8b781a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.219+0000 7f8b7ca00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7810d850 con 0x7f8b781078e0 2024-09-15T22:53:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.219+0000 7f8b7ca00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7810d6d0 con 0x7f8b78104fb0 2024-09-15T22:53:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.219+0000 7f8b7ca00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7810d9d0 con 0x7f8b78102680 2024-09-15T22:53:56.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.220+0000 7f8b75400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b78104fb0 0x7f8b7819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.220+0000 7f8b75400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b78104fb0 0x7f8b7819a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:38580/0 (socket says 172.21.15.111:38580) 2024-09-15T22:53:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.220+0000 7f8b75400640 1 -- 172.21.15.111:0/1407675333 learned_addr learned my addr 172.21.15.111:0/1407675333 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:53:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.220+0000 7f8b75e00640 1 --2- 172.21.15.111:0/1407675333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b78102680 0x7f8b7819a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.220+0000 7f8b76800640 1 --2- 172.21.15.111:0/1407675333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b781078e0 0x7f8b781a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.220+0000 7f8b75400640 1 -- 172.21.15.111:0/1407675333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b78102680 msgr2=0x7f8b7819a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.220+0000 7f8b75400640 1 --2- 172.21.15.111:0/1407675333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b78102680 0x7f8b7819a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.220+0000 7f8b75400640 1 -- 172.21.15.111:0/1407675333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b781078e0 msgr2=0x7f8b781a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:56.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.220+0000 7f8b75400640 1 --2- 172.21.15.111:0/1407675333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b781078e0 0x7f8b781a1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.220+0000 7f8b75400640 1 -- 172.21.15.111:0/1407675333 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b781a1c70 con 0x7f8b78104fb0 2024-09-15T22:53:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.220+0000 7f8b75e00640 1 --2- 172.21.15.111:0/1407675333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b78102680 0x7f8b7819a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:53:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.221+0000 7f8b75400640 1 --2- 172.21.15.111:0/1407675333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b78104fb0 0x7f8b7819a590 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400d770 tx=0x7f8b6400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.221+0000 7f8b6ea00640 1 -- 172.21.15.111:0/1407675333 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b64017070 con 0x7f8b78104fb0 2024-09-15T22:53:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.221+0000 7f8b7ca00640 1 -- 172.21.15.111:0/1407675333 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b781a1f60 con 0x7f8b78104fb0 2024-09-15T22:53:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.221+0000 7f8b6ea00640 1 -- 172.21.15.111:0/1407675333 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b64009c30 con 0x7f8b78104fb0 2024-09-15T22:53:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.221+0000 7f8b6ea00640 1 -- 172.21.15.111:0/1407675333 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6400f620 con 0x7f8b78104fb0 2024-09-15T22:53:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.221+0000 7f8b7ca00640 1 -- 172.21.15.111:0/1407675333 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b781a2470 con 0x7f8b78104fb0 2024-09-15T22:53:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.224+0000 7f8b7f8c3740 1 Processor -- start 2024-09-15T22:53:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.224+0000 7f8b6ea00640 1 -- 172.21.15.111:0/1407675333 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f8b6401a070 con 0x7f8b78104fb0 2024-09-15T22:53:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.224+0000 7f8b7f8c3740 1 -- start start 2024-09-15T22:53:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.225+0000 7f8b7f8c3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564de5fe4090 0x564de5fe4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.225+0000 7f8b7f8c3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de5fe49d0 0x564de5fe9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.225+0000 7f8b7f8c3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564de5fe9a60 0x564de5febf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.225+0000 7f8b7f8c3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564de5f49100 con 0x564de5fe4090 2024-09-15T22:53:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.225+0000 7f8b7f8c3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564de5ef11b0 con 0x564de5fe9a60 2024-09-15T22:53:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.225+0000 7f8b7f8c3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564de5f450c0 con 0x564de5fe49d0 2024-09-15T22:53:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.225+0000 7f8b76800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564de5fe4090 0x564de5fe4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.225+0000 7f8b75e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de5fe49d0 0x564de5fe9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.225+0000 7f8b75400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564de5fe9a60 0x564de5febf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.225+0000 7f8b76800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564de5fe4090 0x564de5fe4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60964/0 (socket says 172.21.15.111:60964) 2024-09-15T22:53:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.225+0000 7f8b76800640 1 -- 172.21.15.111:0/2643822692 learned_addr learned my addr 172.21.15.111:0/2643822692 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:53:56.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.225+0000 7f8b75e00640 1 -- 172.21.15.111:0/2643822692 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564de5fe9a60 msgr2=0x564de5febf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.225+0000 7f8b75e00640 1 --2- 172.21.15.111:0/2643822692 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564de5fe9a60 0x564de5febf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.225+0000 7f8b75e00640 1 -- 172.21.15.111:0/2643822692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564de5fe4090 msgr2=0x564de5fe4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.225+0000 7f8b75e00640 1 --2- 172.21.15.111:0/2643822692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564de5fe4090 0x564de5fe4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.225+0000 7f8b75e00640 1 -- 172.21.15.111:0/2643822692 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564de5c5c210 con 0x564de5fe49d0 2024-09-15T22:53:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.226+0000 7f8b75e00640 1 --2- 172.21.15.111:0/2643822692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de5fe49d0 0x564de5fe9490 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8b6802a910 tx=0x7f8b68004520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c45973cc88353e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.226+0000 7f8b63000640 1 -- 172.21.15.111:0/2643822692 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68035040 con 0x564de5fe49d0 2024-09-15T22:53:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.226+0000 7f8b63000640 1 -- 172.21.15.111:0/2643822692 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b68004770 con 0x564de5fe49d0 2024-09-15T22:53:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.226+0000 7f8b6ea00640 1 --2- 172.21.15.111:0/1407675333 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8b5c0a3370 0x7f8b5c0a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.226+0000 7f8b63000640 1 -- 172.21.15.111:0/2643822692 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68031a40 con 0x564de5fe49d0 2024-09-15T22:53:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.226+0000 7f8b6ea00640 1 -- 172.21.15.111:0/1407675333 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f8b640cf2f0 con 0x7f8b78104fb0 2024-09-15T22:53:56.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.226+0000 7f8b76800640 1 --2- 172.21.15.111:0/1407675333 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8b5c0a3370 0x7f8b5c0a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.227+0000 7f8b7f8c3740 1 -- 172.21.15.111:0/2643822692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de5fe49d0 msgr2=0x564de5fe9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.227+0000 7f8b7f8c3740 1 --2- 172.21.15.111:0/2643822692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de5fe49d0 0x564de5fe9490 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8b6802a910 tx=0x7f8b68004520 comp rx=0 tx=0).stop 2024-09-15T22:53:56.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.227+0000 7f8b7f8c3740 1 -- 172.21.15.111:0/2643822692 shutdown_connections 2024-09-15T22:53:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.227+0000 7f8b76800640 1 --2- 172.21.15.111:0/1407675333 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8b5c0a3370 0x7f8b5c0a5830 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8b70011670 tx=0x7f8b70009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.227+0000 7f8b7f8c3740 1 --2- 172.21.15.111:0/2643822692 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564de5fe9a60 0x564de5febf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.227+0000 7f8b7f8c3740 1 --2- 172.21.15.111:0/2643822692 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de5fe49d0 0x564de5fe9490 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.227+0000 7f8b7f8c3740 1 --2- 172.21.15.111:0/2643822692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564de5fe4090 0x564de5fe4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.227+0000 7f8b7f8c3740 1 -- 172.21.15.111:0/2643822692 >> 172.21.15.111:0/2643822692 conn(0x564de5fdf700 msgr2=0x564de5fdfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.227+0000 7f8b7f8c3740 1 -- 172.21.15.111:0/2643822692 shutdown_connections 2024-09-15T22:53:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.227+0000 7f8b7f8c3740 1 -- 172.21.15.111:0/2643822692 wait complete. 2024-09-15T22:53:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.227+0000 7f8b7f8c3740 1 -- start start 2024-09-15T22:53:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.228+0000 7f8b7f8c3740 1 Processor -- start 2024-09-15T22:53:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.228+0000 7f8b7f8c3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564de5fe4090 0x564de5fee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:56.233 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.228+0000 7f8b7f8c3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de5fe49d0 0x564de5fee720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.228+0000 7f8b7f8c3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564de5fe9a60 0x564de5ff5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.228+0000 7f8b7f8c3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564de5fec9d0 con 0x564de5fe4090 2024-09-15T22:53:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.228+0000 7f8b7f8c3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564de5ed2090 con 0x564de5fe9a60 2024-09-15T22:53:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.228+0000 7f8b7f8c3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564de5fecb50 con 0x564de5fe49d0 2024-09-15T22:53:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.228+0000 7f8b7f8c3740 10 client.0 _ll_register_callbacks cb 0x564de5f26f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:53:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.229+0000 7f8b75400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de5fe49d0 0x564de5fee720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.229+0000 7f8b75400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de5fe49d0 0x564de5fee720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:41854/0 (socket says 172.21.15.111:41854) 2024-09-15T22:53:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.229+0000 7f8b76800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564de5fe9a60 0x564de5ff5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.229+0000 7f8b75400640 1 -- 172.21.15.111:0/1207543902 learned_addr learned my addr 172.21.15.111:0/1207543902 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:53:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.229+0000 7f8b75e00640 1 --2- 172.21.15.111:0/1207543902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564de5fe4090 0x564de5fee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.229+0000 7f8b75400640 1 -- 172.21.15.111:0/1207543902 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564de5fe9a60 msgr2=0x564de5ff5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.229+0000 7f8b75400640 1 --2- 172.21.15.111:0/1207543902 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564de5fe9a60 0x564de5ff5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.229+0000 7f8b75400640 1 -- 172.21.15.111:0/1207543902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564de5fe4090 msgr2=0x564de5fee1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:56.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.229+0000 7f8b75400640 1 --2- 172.21.15.111:0/1207543902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564de5fe4090 0x564de5fee1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.229+0000 7f8b75400640 1 -- 172.21.15.111:0/1207543902 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564de5d28620 con 0x564de5fe49d0 2024-09-15T22:53:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.229+0000 7f8b75400640 1 --2- 172.21.15.111:0/1207543902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de5fe49d0 0x564de5fee720 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8b64097e20 tx=0x7f8b640d2f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fb44498c9377795 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.229+0000 7f8b57400640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:53:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.229+0000 7f8b57400640 1 -- 172.21.15.111:0/1207543902 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b64021030 con 0x564de5fe49d0 2024-09-15T22:53:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.230+0000 7f8b7f8c3740 1 -- 172.21.15.111:0/1207543902 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b68004770 con 0x564de5fe49d0 2024-09-15T22:53:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.230+0000 7f8b57400640 1 -- 172.21.15.111:0/1207543902 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f8b64098030 con 0x564de5fe49d0 2024-09-15T22:53:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.230+0000 7f8b57400640 1 -- 172.21.15.111:0/1207543902 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8b6409b320 con 0x564de5fe49d0 2024-09-15T22:53:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.230+0000 7f8b7f8c3740 10 client.25642 fetch_fsmap learned FSMap version 30 2024-09-15T22:53:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.230+0000 7f8b7f8c3740 1 -- 172.21.15.111:0/1207543902 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564de5ff8610 con 0x564de5fe49d0 2024-09-15T22:53:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.230+0000 7f8b57400640 1 -- 172.21.15.111:0/1207543902 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b68004770 con 0x564de5fe49d0 2024-09-15T22:53:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.230+0000 7f8b57400640 1 -- 172.21.15.111:0/1207543902 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f8b6409ae10 con 0x564de5fe49d0 2024-09-15T22:53:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.230+0000 7f8b7f8c3740 10 client.25642 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:53:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.230+0000 7f8b7f8c3740 10 client.25642 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:53:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.230+0000 7f8b7f8c3740 20 client.25642 populate_metadata read hostname 'smithi111' 2024-09-15T22:53:56.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.230+0000 7f8b7f8c3740 1 --2- 172.21.15.111:0/1207543902 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564de5ff9c90 0x564de5ffc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.230+0000 7f8b7f8c3740 4 client.25642 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:53:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.230+0000 7f8b7f8c3740 1 -- 172.21.15.111:0/1207543902 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564de5f56830 con 0x564de5ff9c90 2024-09-15T22:53:56.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.231+0000 7f8b75e00640 1 --2- 172.21.15.111:0/1207543902 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564de5ff9c90 0x564de5ffc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.231+0000 7f8b75e00640 1 --2- 172.21.15.111:0/1207543902 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564de5ff9c90 0x564de5ffc0a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f35c24cdff7bd94f server_cookie=6d06c8c4fcb49006 in_seq=0 out_seq=0 2024-09-15T22:53:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.231+0000 7f8b57400640 10 client.25642 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:53:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.232+0000 7f8b57400640 1 -- 172.21.15.111:0/1207543902 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564de5f56830 con 0x564de5ff9c90 2024-09-15T22:53:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.232+0000 7f8b57400640 10 client.25642 handle_command_reply: tid=0 2024-09-15T22:53:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.232+0000 7f8b7f8c3740 1 client.25642 shutdown 2024-09-15T22:53:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.232+0000 7f8b7f8c3740 1 -- 172.21.15.111:0/1207543902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de5fe49d0 msgr2=0x564de5fee720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:56.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.232+0000 7f8b7f8c3740 1 --2- 172.21.15.111:0/1207543902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de5fe49d0 0x564de5fee720 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8b64097e20 tx=0x7f8b640d2f60 comp rx=0 tx=0).stop 2024-09-15T22:53:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.232+0000 7f8b7f8c3740 1 -- 172.21.15.111:0/1207543902 shutdown_connections 2024-09-15T22:53:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.232+0000 7f8b7f8c3740 1 --2- 172.21.15.111:0/1207543902 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564de5ff9c90 0x564de5ffc0a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.232+0000 7f8b7f8c3740 1 --2- 172.21.15.111:0/1207543902 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564de5fe9a60 0x564de5ff5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.232+0000 7f8b7f8c3740 1 --2- 172.21.15.111:0/1207543902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de5fe49d0 0x564de5fee720 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.232+0000 7f8b57400640 0 client.25642 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:53:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.232+0000 7f8b7f8c3740 1 --2- 172.21.15.111:0/1207543902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564de5fe4090 0x564de5fee1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.232+0000 7f8b7f8c3740 1 -- 172.21.15.111:0/1207543902 >> 172.21.15.111:0/1207543902 conn(0x564de5fdf700 msgr2=0x564de5fe0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.233+0000 7f8b7f8c3740 1 -- 172.21.15.111:0/1207543902 shutdown_connections 2024-09-15T22:53:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.233+0000 7f8b7f8c3740 1 -- 172.21.15.111:0/1207543902 wait complete. 2024-09-15T22:53:56.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.233+0000 7f8b7f8c3740 20 client.25642 trim_cache size 0 max 16384 2024-09-15T22:53:56.253 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.250+0000 7f8b7f8c3740 1 Processor -- start 2024-09-15T22:53:56.253 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.250+0000 7f8b7f8c3740 1 -- start start 2024-09-15T22:53:56.253 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.251+0000 7f8b7f8c3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564de5fe4090 0x564de5fee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:56.253 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.251+0000 7f8b7f8c3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564de5fe49d0 0x564de5fee720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:56.253 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.251+0000 7f8b7f8c3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de5fe9a60 0x564de5ff5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:56.253 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.251+0000 7f8b7f8c3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8b781ad9b0 con 0x564de5fe4090 2024-09-15T22:53:56.253 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.251+0000 7f8b7f8c3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8b781ab280 con 0x564de5fe49d0 2024-09-15T22:53:56.253 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.251+0000 7f8b7f8c3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8b781a90f0 con 0x564de5fe9a60 2024-09-15T22:53:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.251+0000 7f8b75e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564de5fe4090 0x564de5fee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.251+0000 7f8b75400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564de5fe49d0 0x564de5fee720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.251+0000 7f8b76800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de5fe9a60 0x564de5ff5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.251+0000 7f8b75400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564de5fe49d0 0x564de5fee720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:38602/0 (socket says 172.21.15.111:38602) 2024-09-15T22:53:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.251+0000 7f8b75400640 1 -- 172.21.15.111:0/1768880316 learned_addr learned my addr 172.21.15.111:0/1768880316 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:53:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.251+0000 7f8b75400640 1 -- 172.21.15.111:0/1768880316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de5fe9a60 msgr2=0x564de5ff5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.251+0000 7f8b75400640 1 --2- 172.21.15.111:0/1768880316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de5fe9a60 0x564de5ff5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.251+0000 7f8b75400640 1 -- 172.21.15.111:0/1768880316 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564de5fe4090 msgr2=0x564de5fee1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.251+0000 7f8b75400640 1 --2- 172.21.15.111:0/1768880316 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564de5fe4090 0x564de5fee1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.251+0000 7f8b75400640 1 -- 172.21.15.111:0/1768880316 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564de5fdf520 con 0x564de5fe49d0 2024-09-15T22:53:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.251+0000 7f8b75e00640 1 --2- 172.21.15.111:0/1768880316 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564de5fe4090 0x564de5fee1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:53:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.252+0000 7f8b75400640 1 --2- 172.21.15.111:0/1768880316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564de5fe49d0 0x564de5fee720 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8b64010fd0 tx=0x7f8b64099ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9516c80840f4906e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.252+0000 7f8b62600640 1 -- 172.21.15.111:0/1768880316 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b64014020 con 0x564de5fe49d0 2024-09-15T22:53:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.252+0000 7f8b62600640 1 -- 172.21.15.111:0/1768880316 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b6409ac00 con 0x564de5fe49d0 2024-09-15T22:53:56.255 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.252+0000 7f8b62600640 1 -- 172.21.15.111:0/1768880316 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b64021070 con 0x564de5fe49d0 2024-09-15T22:53:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.252+0000 7f8b7f8c3740 1 -- 172.21.15.111:0/1768880316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564de5fe49d0 msgr2=0x564de5fee720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.252+0000 7f8b7f8c3740 1 --2- 172.21.15.111:0/1768880316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564de5fe49d0 0x564de5fee720 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8b64010fd0 tx=0x7f8b64099ed0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.253+0000 7f8b7f8c3740 1 -- 172.21.15.111:0/1768880316 shutdown_connections 2024-09-15T22:53:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.253+0000 7f8b7f8c3740 1 --2- 172.21.15.111:0/1768880316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de5fe9a60 0x564de5ff5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.253+0000 7f8b7f8c3740 1 --2- 172.21.15.111:0/1768880316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564de5fe49d0 0x564de5fee720 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.253+0000 7f8b7f8c3740 1 --2- 172.21.15.111:0/1768880316 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564de5fe4090 0x564de5fee1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.253+0000 7f8b7f8c3740 1 -- 172.21.15.111:0/1768880316 >> 172.21.15.111:0/1768880316 conn(0x564de5fdf700 msgr2=0x564de5fe5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.253+0000 7f8b7f8c3740 1 -- 172.21.15.111:0/1768880316 shutdown_connections 2024-09-15T22:53:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.253+0000 7f8b7f8c3740 1 -- 172.21.15.111:0/1768880316 wait complete. 2024-09-15T22:53:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.253+0000 7f8b7f8c3740 1 -- start start 2024-09-15T22:53:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.253+0000 7f8b7f8c3740 1 Processor -- start 2024-09-15T22:53:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.253+0000 7f8b7f8c3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564de5fe4090 0x564de5ff2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.254+0000 7f8b7f8c3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de5fe49d0 0x564de5ff2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.254+0000 7f8b7f8c3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564de5fe9a60 0x564de5ff3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.254+0000 7f8b7f8c3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8b781a9740 con 0x564de5fe9a60 2024-09-15T22:53:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.254+0000 7f8b7f8c3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8b781aabb0 con 0x564de5fe4090 2024-09-15T22:53:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.254+0000 7f8b7f8c3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8b781a9e10 con 0x564de5fe49d0 2024-09-15T22:53:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.254+0000 7f8b7f8c3740 10 client.0 _ll_register_callbacks cb 0x7f8b640d2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:53:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.254+0000 7f8b75e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564de5fe4090 0x564de5ff2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.254+0000 7f8b75e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564de5fe4090 0x564de5ff2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:38614/0 (socket says 172.21.15.111:38614) 2024-09-15T22:53:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.254+0000 7f8b75e00640 1 -- 172.21.15.111:0/1779869297 learned_addr learned my addr 172.21.15.111:0/1779869297 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:53:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.254+0000 7f8b75400640 1 --2- 172.21.15.111:0/1779869297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de5fe49d0 0x564de5ff2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.254+0000 7f8b76800640 1 --2- 172.21.15.111:0/1779869297 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564de5fe9a60 0x564de5ff3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.254+0000 7f8b75e00640 1 -- 172.21.15.111:0/1779869297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de5fe49d0 msgr2=0x564de5ff2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.254+0000 7f8b75e00640 1 --2- 172.21.15.111:0/1779869297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de5fe49d0 0x564de5ff2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.254+0000 7f8b75e00640 1 -- 172.21.15.111:0/1779869297 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564de5fe9a60 msgr2=0x564de5ff3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:53:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.254+0000 7f8b75e00640 1 --2- 172.21.15.111:0/1779869297 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564de5fe9a60 0x564de5ff3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.254+0000 7f8b75e00640 1 -- 172.21.15.111:0/1779869297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564de5ff42b0 con 0x564de5fe4090 2024-09-15T22:53:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.254+0000 7f8b75400640 1 --2- 172.21.15.111:0/1779869297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de5fe49d0 0x564de5ff2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:53:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.254+0000 7f8b75e00640 1 --2- 172.21.15.111:0/1779869297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564de5fe4090 0x564de5ff2750 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009900 tx=0x7f8b68004590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42f33b60d1b2d6aa server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:53:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.254+0000 7f8b57400640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:53:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.255+0000 7f8b57400640 1 -- 172.21.15.111:0/1779869297 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b68035040 con 0x564de5fe4090 2024-09-15T22:53:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.255+0000 7f8b7f8c3740 1 -- 172.21.15.111:0/1779869297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b6409ac00 con 0x564de5fe4090 2024-09-15T22:53:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.255+0000 7f8b57400640 1 -- 172.21.15.111:0/1779869297 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f8b68005240 con 0x564de5fe4090 2024-09-15T22:53:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.255+0000 7f8b57400640 1 -- 172.21.15.111:0/1779869297 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8b68060050 con 0x564de5fe4090 2024-09-15T22:53:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.255+0000 7f8b7f8c3740 10 client.25631 fetch_fsmap learned FSMap version 30 2024-09-15T22:53:56.260 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.255+0000 7f8b7f8c3740 1 -- 172.21.15.111:0/1779869297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564de5ff5630 con 0x564de5fe4090 2024-09-15T22:53:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.255+0000 7f8b57400640 1 -- 172.21.15.111:0/1779869297 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b6409ac00 con 0x564de5fe4090 2024-09-15T22:53:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.255+0000 7f8b57400640 1 -- 172.21.15.111:0/1779869297 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f8b680035f0 con 0x564de5fe4090 2024-09-15T22:53:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.255+0000 7f8b7f8c3740 10 client.25631 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:53:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.255+0000 7f8b7f8c3740 10 client.25631 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:53:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.255+0000 7f8b7f8c3740 20 client.25631 populate_metadata read hostname 'smithi111' 2024-09-15T22:53:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.256+0000 7f8b7f8c3740 1 --2- 172.21.15.111:0/1779869297 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564de6064950 0x564de6066d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:53:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.256+0000 7f8b7f8c3740 4 client.25631 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:53:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.256+0000 7f8b7f8c3740 1 -- 172.21.15.111:0/1779869297 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564de5f50b80 con 0x564de6064950 2024-09-15T22:53:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.256+0000 7f8b75400640 1 --2- 172.21.15.111:0/1779869297 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564de6064950 0x564de6066d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:53:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.256+0000 7f8b75400640 1 --2- 172.21.15.111:0/1779869297 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564de6064950 0x564de6066d40 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ff2a818682a8999 server_cookie=22a4449d929914dc in_seq=0 out_seq=0 2024-09-15T22:53:56.261 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.256+0000 7f8b57400640 10 client.25631 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:53:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.266+0000 7f8b57400640 1 -- 172.21.15.111:0/1779869297 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564de5f50b80 con 0x564de6064950 2024-09-15T22:53:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.266+0000 7f8b57400640 10 client.25631 handle_command_reply: tid=0 2024-09-15T22:53:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.266+0000 7f8b7f8c3740 1 client.25631 shutdown 2024-09-15T22:53:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.266+0000 7f8b7f8c3740 1 -- 172.21.15.111:0/1779869297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564de5fe4090 msgr2=0x564de5ff2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.266+0000 7f8b7f8c3740 1 --2- 172.21.15.111:0/1779869297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564de5fe4090 0x564de5ff2750 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009900 tx=0x7f8b68004590 comp rx=0 tx=0).stop 2024-09-15T22:53:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.266+0000 7f8b7f8c3740 1 -- 172.21.15.111:0/1779869297 shutdown_connections 2024-09-15T22:53:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.266+0000 7f8b7f8c3740 1 --2- 172.21.15.111:0/1779869297 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564de6064950 0x564de6066d40 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.266+0000 7f8b7f8c3740 1 --2- 172.21.15.111:0/1779869297 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564de5fe9a60 0x564de5ff3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.266+0000 7f8b7f8c3740 1 --2- 172.21.15.111:0/1779869297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564de5fe49d0 0x564de5ff2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.266+0000 7f8b7f8c3740 1 --2- 172.21.15.111:0/1779869297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564de5fe4090 0x564de5ff2750 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.266+0000 7f8b57400640 0 client.25631 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:53:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.266+0000 7f8b7f8c3740 1 -- 172.21.15.111:0/1779869297 >> 172.21.15.111:0/1779869297 conn(0x564de5fdf700 msgr2=0x564de5fee1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.266+0000 7f8b7f8c3740 1 -- 172.21.15.111:0/1779869297 shutdown_connections 2024-09-15T22:53:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.266+0000 7f8b7f8c3740 1 -- 172.21.15.111:0/1779869297 wait complete. 2024-09-15T22:53:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.266+0000 7f8b7f8c3740 20 client.25631 trim_cache size 0 max 16384 2024-09-15T22:53:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:53:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:53:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:53:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:53:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.267+0000 7f8b7ca00640 1 -- 172.21.15.111:0/1407675333 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8b5c0a3370 msgr2=0x7f8b5c0a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.267+0000 7f8b7ca00640 1 --2- 172.21.15.111:0/1407675333 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8b5c0a3370 0x7f8b5c0a5830 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8b70011670 tx=0x7f8b70009500 comp rx=0 tx=0).stop 2024-09-15T22:53:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.267+0000 7f8b7ca00640 1 -- 172.21.15.111:0/1407675333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b78104fb0 msgr2=0x7f8b7819a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:53:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.267+0000 7f8b7ca00640 1 --2- 172.21.15.111:0/1407675333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b78104fb0 0x7f8b7819a590 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400d770 tx=0x7f8b6400dc30 comp rx=0 tx=0).stop 2024-09-15T22:53:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.267+0000 7f8b7ca00640 1 -- 172.21.15.111:0/1407675333 shutdown_connections 2024-09-15T22:53:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.267+0000 7f8b7ca00640 1 --2- 172.21.15.111:0/1407675333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8b781078e0 0x7f8b781a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.267+0000 7f8b7ca00640 1 --2- 172.21.15.111:0/1407675333 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8b5c0a3370 0x7f8b5c0a5830 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.267+0000 7f8b7ca00640 1 --2- 172.21.15.111:0/1407675333 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b78104fb0 0x7f8b7819a590 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.267+0000 7f8b7ca00640 1 --2- 172.21.15.111:0/1407675333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b78102680 0x7f8b7819a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:53:56.270 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.267+0000 7f8b7ca00640 1 -- 172.21.15.111:0/1407675333 >> 172.21.15.111:0/1407675333 conn(0x7f8b780fc1e0 msgr2=0x7f8b78101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:53:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.267+0000 7f8b7ca00640 1 -- 172.21.15.111:0/1407675333 shutdown_connections 2024-09-15T22:53:56.271 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:53:56.267+0000 7f8b7ca00640 1 -- 172.21.15.111:0/1407675333 wait complete. 2024-09-15T22:53:56.272 INFO:tasks.workunit.client.0.smithi111.stdout: 1 722548 9.96 MB/sec execute 264 sec latency 13.188 ms 2024-09-15T22:53:57.273 INFO:tasks.workunit.client.0.smithi111.stdout: 1 722751 9.92 MB/sec execute 265 sec latency 903.618 ms 2024-09-15T22:53:57.729 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:57 smithi111 ceph-mon[30629]: pgmap v591: 97 pgs: 97 active+clean; 264 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 228 op/s 2024-09-15T22:53:58.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:57 smithi192 ceph-mon[35427]: pgmap v591: 97 pgs: 97 active+clean; 264 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 228 op/s 2024-09-15T22:53:58.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:57 smithi121 ceph-mon[34815]: pgmap v591: 97 pgs: 97 active+clean; 264 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 228 op/s 2024-09-15T22:53:58.273 INFO:tasks.workunit.client.0.smithi111.stdout: 1 722772 9.88 MB/sec execute 266 sec latency 1431.045 ms 2024-09-15T22:53:59.273 INFO:tasks.workunit.client.0.smithi111.stdout: 1 725064 9.90 MB/sec execute 267 sec latency 593.277 ms 2024-09-15T22:54:00.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:53:59 smithi192 ceph-mon[35427]: pgmap v592: 97 pgs: 97 active+clean; 262 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 194 op/s 2024-09-15T22:54:00.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:53:59 smithi121 ceph-mon[34815]: pgmap v592: 97 pgs: 97 active+clean; 262 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 194 op/s 2024-09-15T22:54:00.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:53:59 smithi111 ceph-mon[30629]: pgmap v592: 97 pgs: 97 active+clean; 262 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 194 op/s 2024-09-15T22:54:00.273 INFO:tasks.workunit.client.0.smithi111.stdout: 1 728251 9.93 MB/sec execute 268 sec latency 13.042 ms 2024-09-15T22:54:01.273 INFO:tasks.workunit.client.0.smithi111.stdout: 1 731141 9.94 MB/sec execute 269 sec latency 11.478 ms 2024-09-15T22:54:01.703 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:01 smithi111 ceph-mon[30629]: pgmap v593: 97 pgs: 97 active+clean; 262 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 187 op/s 2024-09-15T22:54:02.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:01 smithi192 ceph-mon[35427]: pgmap v593: 97 pgs: 97 active+clean; 262 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 187 op/s 2024-09-15T22:54:02.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:01 smithi121 ceph-mon[34815]: pgmap v593: 97 pgs: 97 active+clean; 262 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 187 op/s 2024-09-15T22:54:02.273 INFO:tasks.workunit.client.0.smithi111.stdout: 1 731611 9.92 MB/sec execute 270 sec latency 593.280 ms 2024-09-15T22:54:03.273 INFO:tasks.workunit.client.0.smithi111.stdout: 1 731711 9.89 MB/sec execute 271 sec latency 1530.693 ms 2024-09-15T22:54:04.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:03 smithi192 ceph-mon[35427]: pgmap v594: 97 pgs: 97 active+clean; 265 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 155 op/s 2024-09-15T22:54:04.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:03 smithi121 ceph-mon[34815]: pgmap v594: 97 pgs: 97 active+clean; 265 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 155 op/s 2024-09-15T22:54:04.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:03 smithi111 ceph-mon[30629]: pgmap v594: 97 pgs: 97 active+clean; 265 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 155 op/s 2024-09-15T22:54:04.273 INFO:tasks.workunit.client.0.smithi111.stdout: 1 734276 9.90 MB/sec execute 272 sec latency 19.241 ms 2024-09-15T22:54:05.273 INFO:tasks.workunit.client.0.smithi111.stdout: 1 736628 9.91 MB/sec execute 273 sec latency 18.105 ms 2024-09-15T22:54:05.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:05 smithi111 ceph-mon[30629]: pgmap v595: 97 pgs: 97 active+clean; 268 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 192 op/s 2024-09-15T22:54:05.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:05 smithi192 ceph-mon[35427]: pgmap v595: 97 pgs: 97 active+clean; 268 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 192 op/s 2024-09-15T22:54:05.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:05 smithi121 ceph-mon[34815]: pgmap v595: 97 pgs: 97 active+clean; 268 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 192 op/s 2024-09-15T22:54:06.273 INFO:tasks.workunit.client.0.smithi111.stdout: 1 739319 9.92 MB/sec execute 274 sec latency 13.037 ms 2024-09-15T22:54:07.273 INFO:tasks.workunit.client.0.smithi111.stdout: 1 740235 9.91 MB/sec execute 275 sec latency 618.090 ms 2024-09-15T22:54:07.737 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:07 smithi111 ceph-mon[30629]: pgmap v596: 97 pgs: 97 active+clean; 268 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 192 op/s 2024-09-15T22:54:08.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:07 smithi192 ceph-mon[35427]: pgmap v596: 97 pgs: 97 active+clean; 268 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 192 op/s 2024-09-15T22:54:08.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:07 smithi121 ceph-mon[34815]: pgmap v596: 97 pgs: 97 active+clean; 268 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 192 op/s 2024-09-15T22:54:08.273 INFO:tasks.workunit.client.0.smithi111.stdout: 1 740281 9.87 MB/sec execute 276 sec latency 1584.643 ms 2024-09-15T22:54:09.273 INFO:tasks.workunit.client.0.smithi111.stdout: 1 742845 9.88 MB/sec execute 277 sec latency 10.909 ms 2024-09-15T22:54:09.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.394+0000 7f3265a00640 1 -- 172.21.15.111:0/400388354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3260103b70 msgr2=0x7f3260103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.394+0000 7f3265a00640 1 --2- 172.21.15.111:0/400388354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3260103b70 0x7f3260103f70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3250009900 tx=0x7f325002a400 comp rx=0 tx=0).stop 2024-09-15T22:54:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.394+0000 7f3265a00640 1 -- 172.21.15.111:0/400388354 shutdown_connections 2024-09-15T22:54:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.394+0000 7f3265a00640 1 --2- 172.21.15.111:0/400388354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3260105720 0x7f326010bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.394+0000 7f3265a00640 1 --2- 172.21.15.111:0/400388354 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3260104d60 0x7f32601051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.394+0000 7f3265a00640 1 --2- 172.21.15.111:0/400388354 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3260103b70 0x7f3260103f70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.394+0000 7f3265a00640 1 -- 172.21.15.111:0/400388354 >> 172.21.15.111:0/400388354 conn(0x7f32600ff2f0 msgr2=0x7f3260101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:54:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.394+0000 7f3265a00640 1 -- 172.21.15.111:0/400388354 shutdown_connections 2024-09-15T22:54:09.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.394+0000 7f3265a00640 1 -- 172.21.15.111:0/400388354 wait complete. 2024-09-15T22:54:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.395+0000 7f3265a00640 1 Processor -- start 2024-09-15T22:54:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.395+0000 7f3265a00640 1 -- start start 2024-09-15T22:54:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.395+0000 7f3265a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3260103b70 0x7f3260199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.395+0000 7f3265a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3260104d60 0x7f326019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.395+0000 7f3265a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3260105720 0x7f32601a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.395+0000 7f3265a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f326010f820 con 0x7f3260105720 2024-09-15T22:54:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.395+0000 7f3265a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f326010f6a0 con 0x7f3260104d60 2024-09-15T22:54:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.395+0000 7f3265a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f326010f9a0 con 0x7f3260103b70 2024-09-15T22:54:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.395+0000 7f325fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3260103b70 0x7f3260199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.395+0000 7f325f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3260104d60 0x7f326019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.395+0000 7f325fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3260103b70 0x7f3260199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:56200/0 (socket says 172.21.15.111:56200) 2024-09-15T22:54:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.395+0000 7f325fe00640 1 -- 172.21.15.111:0/3064160932 learned_addr learned my addr 172.21.15.111:0/3064160932 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:54:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.396+0000 7f3265000640 1 --2- 172.21.15.111:0/3064160932 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3260105720 0x7f32601a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.396+0000 7f325f400640 1 -- 172.21.15.111:0/3064160932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3260103b70 msgr2=0x7f3260199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.396+0000 7f325f400640 1 --2- 172.21.15.111:0/3064160932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3260103b70 0x7f3260199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.396+0000 7f325f400640 1 -- 172.21.15.111:0/3064160932 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3260105720 msgr2=0x7f32601a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.396+0000 7f325f400640 1 --2- 172.21.15.111:0/3064160932 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3260105720 0x7f32601a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.396+0000 7f325f400640 1 -- 172.21.15.111:0/3064160932 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32601a1ac0 con 0x7f3260104d60 2024-09-15T22:54:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.396+0000 7f325f400640 1 --2- 172.21.15.111:0/3064160932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3260104d60 0x7f326019a3e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f324c00e9f0 tx=0x7f324c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:54:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.396+0000 7f325cc00640 1 -- 172.21.15.111:0/3064160932 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c00ccc0 con 0x7f3260104d60 2024-09-15T22:54:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.396+0000 7f325cc00640 1 -- 172.21.15.111:0/3064160932 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f324c010910 con 0x7f3260104d60 2024-09-15T22:54:09.398 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.396+0000 7f3265a00640 1 -- 172.21.15.111:0/3064160932 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32601a1db0 con 0x7f3260104d60 2024-09-15T22:54:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.396+0000 7f325cc00640 1 -- 172.21.15.111:0/3064160932 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c018960 con 0x7f3260104d60 2024-09-15T22:54:09.399 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.397+0000 7f3265a00640 1 -- 172.21.15.111:0/3064160932 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32601a2280 con 0x7f3260104d60 2024-09-15T22:54:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.399+0000 7f3269ed3740 1 Processor -- start 2024-09-15T22:54:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.400+0000 7f3269ed3740 1 -- start start 2024-09-15T22:54:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.400+0000 7f3269ed3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604822db260 0x5604822db660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.400+0000 7f3269ed3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604822dbba0 0x5604822e0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.400+0000 7f3269ed3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604822e0c30 0x5604822e30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.400+0000 7f3269ed3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560482240100 con 0x5604822e0c30 2024-09-15T22:54:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.400+0000 7f3269ed3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5604821e81b0 con 0x5604822dbba0 2024-09-15T22:54:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.400+0000 7f3269ed3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56048223c0c0 con 0x5604822db260 2024-09-15T22:54:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.400+0000 7f3265000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604822db260 0x5604822db660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.400+0000 7f325fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604822dbba0 0x5604822e0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.400+0000 7f3265000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604822db260 0x5604822db660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:56216/0 (socket says 172.21.15.111:56216) 2024-09-15T22:54:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.400+0000 7f325cc00640 1 -- 172.21.15.111:0/3064160932 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f324c010ac0 con 0x7f3260104d60 2024-09-15T22:54:09.401 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.400+0000 7f3265000640 1 -- 172.21.15.111:0/3842356418 learned_addr learned my addr 172.21.15.111:0/3842356418 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:54:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.401+0000 7f325fe00640 1 -- 172.21.15.111:0/3842356418 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604822db260 msgr2=0x5604822db660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.401+0000 7f325fe00640 1 --2- 172.21.15.111:0/3842356418 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604822db260 0x5604822db660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.401+0000 7f325fe00640 1 -- 172.21.15.111:0/3842356418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604822e0c30 msgr2=0x5604822e30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.401+0000 7f325fe00640 1 --2- 172.21.15.111:0/3842356418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604822e0c30 0x5604822e30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.401+0000 7f325fe00640 1 -- 172.21.15.111:0/3842356418 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560481f53210 con 0x5604822dbba0 2024-09-15T22:54:09.403 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.401+0000 7f325fe00640 1 --2- 172.21.15.111:0/3842356418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604822dbba0 0x5604822e0660 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3250004770 tx=0x7f3250004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaf155dc68b66a5e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:54:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.401+0000 7f3255600640 1 -- 172.21.15.111:0/3842356418 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3250040070 con 0x5604822dbba0 2024-09-15T22:54:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.401+0000 7f3255600640 1 -- 172.21.15.111:0/3842356418 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3250005500 con 0x5604822dbba0 2024-09-15T22:54:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.401+0000 7f3255600640 1 -- 172.21.15.111:0/3842356418 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3250035040 con 0x5604822dbba0 2024-09-15T22:54:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.402+0000 7f3269ed3740 1 -- 172.21.15.111:0/3842356418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604822dbba0 msgr2=0x5604822e0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.402+0000 7f325cc00640 1 --2- 172.21.15.111:0/3064160932 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f32440a32c0 0x7f32440a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.402+0000 7f3269ed3740 1 --2- 172.21.15.111:0/3842356418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604822dbba0 0x5604822e0660 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3250004770 tx=0x7f3250004040 comp rx=0 tx=0).stop 2024-09-15T22:54:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.402+0000 7f325cc00640 1 -- 172.21.15.111:0/3064160932 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f324c0cfc20 con 0x7f3260104d60 2024-09-15T22:54:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.402+0000 7f3269ed3740 1 -- 172.21.15.111:0/3842356418 shutdown_connections 2024-09-15T22:54:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.402+0000 7f3269ed3740 1 --2- 172.21.15.111:0/3842356418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604822e0c30 0x5604822e30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.402+0000 7f3269ed3740 1 --2- 172.21.15.111:0/3842356418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604822dbba0 0x5604822e0660 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.402+0000 7f3269ed3740 1 --2- 172.21.15.111:0/3842356418 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604822db260 0x5604822db660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.402+0000 7f3269ed3740 1 -- 172.21.15.111:0/3842356418 >> 172.21.15.111:0/3842356418 conn(0x5604822d68d0 msgr2=0x5604822d6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:54:09.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.402+0000 7f3265000640 1 --2- 172.21.15.111:0/3064160932 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f32440a32c0 0x7f32440a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.402+0000 7f3269ed3740 1 -- 172.21.15.111:0/3842356418 shutdown_connections 2024-09-15T22:54:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.402+0000 7f3269ed3740 1 -- 172.21.15.111:0/3842356418 wait complete. 2024-09-15T22:54:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.403+0000 7f3269ed3740 1 -- start start 2024-09-15T22:54:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.403+0000 7f3265000640 1 --2- 172.21.15.111:0/3064160932 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f32440a32c0 0x7f32440a5780 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3258005fd0 tx=0x7f3258009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:54:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.403+0000 7f3269ed3740 1 Processor -- start 2024-09-15T22:54:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.403+0000 7f3269ed3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604822db260 0x5604822e5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.403+0000 7f3269ed3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604822dbba0 0x5604822e5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.403+0000 7f3269ed3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604822e0c30 0x5604822ec990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.403+0000 7f3269ed3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5604822e3ba0 con 0x5604822e0c30 2024-09-15T22:54:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.403+0000 7f3269ed3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5604821c9090 con 0x5604822dbba0 2024-09-15T22:54:09.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.403+0000 7f3269ed3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5604822e3d20 con 0x5604822db260 2024-09-15T22:54:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.404+0000 7f3269ed3740 10 client.0 _ll_register_callbacks cb 0x56048221df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:54:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.404+0000 7f325f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604822dbba0 0x5604822e5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.404+0000 7f325fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604822db260 0x5604822e5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.404+0000 7f325f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604822dbba0 0x5604822e5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:34282/0 (socket says 172.21.15.111:34282) 2024-09-15T22:54:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.404+0000 7f3265000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604822e0c30 0x5604822ec990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.404+0000 7f325f400640 1 -- 172.21.15.111:0/356914353 learned_addr learned my addr 172.21.15.111:0/356914353 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:54:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.404+0000 7f325f400640 1 -- 172.21.15.111:0/356914353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604822db260 msgr2=0x5604822e5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.404+0000 7f325f400640 1 --2- 172.21.15.111:0/356914353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604822db260 0x5604822e5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.404+0000 7f325f400640 1 -- 172.21.15.111:0/356914353 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604822e0c30 msgr2=0x5604822ec990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.404+0000 7f325f400640 1 --2- 172.21.15.111:0/356914353 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604822e0c30 0x5604822ec990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.404+0000 7f325f400640 1 -- 172.21.15.111:0/356914353 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56048201f620 con 0x5604822dbba0 2024-09-15T22:54:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.404+0000 7f325fe00640 1 --2- 172.21.15.111:0/356914353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604822db260 0x5604822e5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:54:09.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.404+0000 7f325f400640 1 --2- 172.21.15.111:0/356914353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604822dbba0 0x5604822e5950 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f324c0a4980 tx=0x7f324c09c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a529aa681059cef server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:54:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.404+0000 7f324a000640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:54:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.404+0000 7f324a000640 1 -- 172.21.15.111:0/356914353 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c096e60 con 0x5604822dbba0 2024-09-15T22:54:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.404+0000 7f324a000640 1 -- 172.21.15.111:0/356914353 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f324c0a0120 con 0x5604822dbba0 2024-09-15T22:54:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.404+0000 7f3269ed3740 1 -- 172.21.15.111:0/356914353 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3250005500 con 0x5604822dbba0 2024-09-15T22:54:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.405+0000 7f324a000640 1 -- 172.21.15.111:0/356914353 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f324c0cd8f0 con 0x5604822dbba0 2024-09-15T22:54:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.405+0000 7f3269ed3740 10 client.25643 fetch_fsmap learned FSMap version 30 2024-09-15T22:54:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.405+0000 7f3269ed3740 1 -- 172.21.15.111:0/356914353 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5604822ef7f0 con 0x5604822dbba0 2024-09-15T22:54:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.405+0000 7f324a000640 1 -- 172.21.15.111:0/356914353 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3250005500 con 0x5604822dbba0 2024-09-15T22:54:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.405+0000 7f324a000640 1 -- 172.21.15.111:0/356914353 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f324c09b740 con 0x5604822dbba0 2024-09-15T22:54:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.405+0000 7f3269ed3740 10 client.25643 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:54:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.405+0000 7f3269ed3740 10 client.25643 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:54:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.405+0000 7f3269ed3740 20 client.25643 populate_metadata read hostname 'smithi111' 2024-09-15T22:54:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.405+0000 7f3269ed3740 1 --2- 172.21.15.111:0/356914353 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5604822f0f10 0x5604822f3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.405+0000 7f3269ed3740 4 client.25643 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:54:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.405+0000 7f3269ed3740 1 -- 172.21.15.111:0/356914353 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56048224d830 con 0x5604822f0f10 2024-09-15T22:54:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.405+0000 7f325fe00640 1 --2- 172.21.15.111:0/356914353 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5604822f0f10 0x5604822f3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.406+0000 7f325fe00640 1 --2- 172.21.15.111:0/356914353 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5604822f0f10 0x5604822f3300 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e6aec3a41cafbbc server_cookie=92d3116e72f0f797 in_seq=0 out_seq=0 2024-09-15T22:54:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.406+0000 7f324a000640 10 client.25643 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:54:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.406+0000 7f324a000640 1 -- 172.21.15.111:0/356914353 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56048224d830 con 0x5604822f0f10 2024-09-15T22:54:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.406+0000 7f324a000640 10 client.25643 handle_command_reply: tid=0 2024-09-15T22:54:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.407+0000 7f3269ed3740 1 client.25643 shutdown 2024-09-15T22:54:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.407+0000 7f3269ed3740 1 -- 172.21.15.111:0/356914353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604822dbba0 msgr2=0x5604822e5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.407+0000 7f3269ed3740 1 --2- 172.21.15.111:0/356914353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604822dbba0 0x5604822e5950 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f324c0a4980 tx=0x7f324c09c430 comp rx=0 tx=0).stop 2024-09-15T22:54:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.407+0000 7f3269ed3740 1 -- 172.21.15.111:0/356914353 shutdown_connections 2024-09-15T22:54:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.407+0000 7f3269ed3740 1 --2- 172.21.15.111:0/356914353 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5604822f0f10 0x5604822f3300 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.407+0000 7f3269ed3740 1 --2- 172.21.15.111:0/356914353 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604822e0c30 0x5604822ec990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.407+0000 7f324a000640 0 client.25643 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:54:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.407+0000 7f3269ed3740 1 --2- 172.21.15.111:0/356914353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604822dbba0 0x5604822e5950 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.407+0000 7f3269ed3740 1 --2- 172.21.15.111:0/356914353 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604822db260 0x5604822e5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.407+0000 7f3269ed3740 1 -- 172.21.15.111:0/356914353 >> 172.21.15.111:0/356914353 conn(0x5604822d68d0 msgr2=0x5604822d80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:54:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.407+0000 7f3269ed3740 1 -- 172.21.15.111:0/356914353 shutdown_connections 2024-09-15T22:54:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.407+0000 7f3269ed3740 1 -- 172.21.15.111:0/356914353 wait complete. 2024-09-15T22:54:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.407+0000 7f3269ed3740 20 client.25643 trim_cache size 0 max 16384 2024-09-15T22:54:09.423 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.422+0000 7f3269ed3740 1 Processor -- start 2024-09-15T22:54:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.422+0000 7f3269ed3740 1 -- start start 2024-09-15T22:54:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.423+0000 7f3269ed3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604822db260 0x5604822d51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.423+0000 7f3269ed3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604822dbba0 0x5604822ec950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.423+0000 7f3269ed3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604822e0c30 0x5604822d5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.423+0000 7f3269ed3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f32601ad790 con 0x5604822db260 2024-09-15T22:54:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.423+0000 7f3269ed3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f32601ab060 con 0x5604822dbba0 2024-09-15T22:54:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.423+0000 7f3269ed3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f32601a8eb0 con 0x5604822e0c30 2024-09-15T22:54:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.423+0000 7f325f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604822dbba0 0x5604822ec950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.423+0000 7f325f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604822dbba0 0x5604822ec950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:34294/0 (socket says 172.21.15.111:34294) 2024-09-15T22:54:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.423+0000 7f325f400640 1 -- 172.21.15.111:0/2396854419 learned_addr learned my addr 172.21.15.111:0/2396854419 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:54:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.423+0000 7f325fe00640 1 --2- 172.21.15.111:0/2396854419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604822db260 0x5604822d51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:09.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.423+0000 7f3265000640 1 --2- 172.21.15.111:0/2396854419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604822e0c30 0x5604822d5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.423+0000 7f325f400640 1 -- 172.21.15.111:0/2396854419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604822e0c30 msgr2=0x5604822d5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.423+0000 7f325f400640 1 --2- 172.21.15.111:0/2396854419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604822e0c30 0x5604822d5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.423+0000 7f325f400640 1 -- 172.21.15.111:0/2396854419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604822db260 msgr2=0x5604822d51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.423+0000 7f325f400640 1 --2- 172.21.15.111:0/2396854419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604822db260 0x5604822d51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.423+0000 7f325f400640 1 -- 172.21.15.111:0/2396854419 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604822d66f0 con 0x5604822dbba0 2024-09-15T22:54:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.424+0000 7f325f400640 1 --2- 172.21.15.111:0/2396854419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604822dbba0 0x5604822ec950 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f324c098cc0 tx=0x7f324c009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc024ad024f6fbd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:54:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.424+0000 7f3254c00640 1 -- 172.21.15.111:0/2396854419 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c096e60 con 0x5604822dbba0 2024-09-15T22:54:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.424+0000 7f3254c00640 1 -- 172.21.15.111:0/2396854419 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f324c0bc280 con 0x5604822dbba0 2024-09-15T22:54:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.424+0000 7f3254c00640 1 -- 172.21.15.111:0/2396854419 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c0cdd80 con 0x5604822dbba0 2024-09-15T22:54:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.425+0000 7f3269ed3740 1 -- 172.21.15.111:0/2396854419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604822dbba0 msgr2=0x5604822ec950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.425+0000 7f3269ed3740 1 --2- 172.21.15.111:0/2396854419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604822dbba0 0x5604822ec950 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f324c098cc0 tx=0x7f324c009700 comp rx=0 tx=0).stop 2024-09-15T22:54:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.425+0000 7f3269ed3740 1 -- 172.21.15.111:0/2396854419 shutdown_connections 2024-09-15T22:54:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.425+0000 7f3269ed3740 1 --2- 172.21.15.111:0/2396854419 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604822e0c30 0x5604822d5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.425+0000 7f3269ed3740 1 --2- 172.21.15.111:0/2396854419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604822dbba0 0x5604822ec950 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.425+0000 7f3269ed3740 1 --2- 172.21.15.111:0/2396854419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604822db260 0x5604822d51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.425+0000 7f3269ed3740 1 -- 172.21.15.111:0/2396854419 >> 172.21.15.111:0/2396854419 conn(0x5604822d68d0 msgr2=0x5604822dcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:54:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.425+0000 7f3269ed3740 1 -- 172.21.15.111:0/2396854419 shutdown_connections 2024-09-15T22:54:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.425+0000 7f3269ed3740 1 -- 172.21.15.111:0/2396854419 wait complete. 2024-09-15T22:54:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.425+0000 7f3269ed3740 1 -- start start 2024-09-15T22:54:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.425+0000 7f3269ed3740 1 Processor -- start 2024-09-15T22:54:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.426+0000 7f3269ed3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604822db260 0x5604822e96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.426+0000 7f3269ed3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604822dbba0 0x5604822e9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.426+0000 7f3269ed3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604822e0c30 0x560482359b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.426+0000 7f3269ed3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f32601a9520 con 0x5604822e0c30 2024-09-15T22:54:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.426+0000 7f3269ed3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f32601aa990 con 0x5604822db260 2024-09-15T22:54:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.426+0000 7f3269ed3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f32601a9bf0 con 0x5604822dbba0 2024-09-15T22:54:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.426+0000 7f3269ed3740 10 client.0 _ll_register_callbacks cb 0x7f324c010e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:54:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.426+0000 7f325fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604822db260 0x5604822e96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.426+0000 7f3265000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604822e0c30 0x560482359b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.426+0000 7f325fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604822db260 0x5604822e96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:34306/0 (socket says 172.21.15.111:34306) 2024-09-15T22:54:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.426+0000 7f325fe00640 1 -- 172.21.15.111:0/263531197 learned_addr learned my addr 172.21.15.111:0/263531197 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:54:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.426+0000 7f325f400640 1 --2- 172.21.15.111:0/263531197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604822dbba0 0x5604822e9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.426+0000 7f325fe00640 1 -- 172.21.15.111:0/263531197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604822dbba0 msgr2=0x5604822e9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.426+0000 7f325fe00640 1 --2- 172.21.15.111:0/263531197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604822dbba0 0x5604822e9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.426+0000 7f325fe00640 1 -- 172.21.15.111:0/263531197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604822e0c30 msgr2=0x560482359b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.426+0000 7f325fe00640 1 --2- 172.21.15.111:0/263531197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604822e0c30 0x560482359b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.426+0000 7f325fe00640 1 -- 172.21.15.111:0/263531197 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56048235a0d0 con 0x5604822db260 2024-09-15T22:54:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.426+0000 7f325fe00640 1 --2- 172.21.15.111:0/263531197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604822db260 0x5604822e96d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3250009230 tx=0x7f32500315b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf90cde019388826 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:54:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.426+0000 7f324a000640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:54:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.427+0000 7f324a000640 1 -- 172.21.15.111:0/263531197 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3250040040 con 0x5604822db260 2024-09-15T22:54:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.427+0000 7f324a000640 1 -- 172.21.15.111:0/263531197 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f32500049a0 con 0x5604822db260 2024-09-15T22:54:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.427+0000 7f3269ed3740 1 -- 172.21.15.111:0/263531197 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f324c0bc280 con 0x5604822db260 2024-09-15T22:54:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.427+0000 7f324a000640 1 -- 172.21.15.111:0/263531197 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3250035040 con 0x5604822db260 2024-09-15T22:54:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.427+0000 7f3269ed3740 10 client.25649 fetch_fsmap learned FSMap version 30 2024-09-15T22:54:09.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.427+0000 7f3269ed3740 1 -- 172.21.15.111:0/263531197 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56048235b250 con 0x5604822db260 2024-09-15T22:54:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.427+0000 7f324a000640 1 -- 172.21.15.111:0/263531197 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f324c0bc280 con 0x5604822db260 2024-09-15T22:54:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.427+0000 7f324a000640 1 -- 172.21.15.111:0/263531197 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f3250031a90 con 0x5604822db260 2024-09-15T22:54:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.427+0000 7f3269ed3740 10 client.25649 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:54:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.427+0000 7f3269ed3740 10 client.25649 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:54:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.428+0000 7f3269ed3740 20 client.25649 populate_metadata read hostname 'smithi111' 2024-09-15T22:54:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.428+0000 7f3269ed3740 1 --2- 172.21.15.111:0/263531197 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56048235b780 0x56048235dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.428+0000 7f3269ed3740 4 client.25649 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:54:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.428+0000 7f3269ed3740 1 -- 172.21.15.111:0/263531197 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560482247b80 con 0x56048235b780 2024-09-15T22:54:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.428+0000 7f325f400640 1 --2- 172.21.15.111:0/263531197 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56048235b780 0x56048235dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.428+0000 7f325f400640 1 --2- 172.21.15.111:0/263531197 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56048235b780 0x56048235dba0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88c9e705db96eff1 server_cookie=6c115ebca0818f15 in_seq=0 out_seq=0 2024-09-15T22:54:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.428+0000 7f324a000640 10 client.25649 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:54:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.438+0000 7f324a000640 1 -- 172.21.15.111:0/263531197 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560482247b80 con 0x56048235b780 2024-09-15T22:54:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.438+0000 7f324a000640 10 client.25649 handle_command_reply: tid=0 2024-09-15T22:54:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.438+0000 7f3269ed3740 1 client.25649 shutdown 2024-09-15T22:54:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.438+0000 7f3269ed3740 1 -- 172.21.15.111:0/263531197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604822db260 msgr2=0x5604822e96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.438+0000 7f3269ed3740 1 --2- 172.21.15.111:0/263531197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604822db260 0x5604822e96d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3250009230 tx=0x7f32500315b0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.438+0000 7f3269ed3740 1 -- 172.21.15.111:0/263531197 shutdown_connections 2024-09-15T22:54:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.438+0000 7f3269ed3740 1 --2- 172.21.15.111:0/263531197 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56048235b780 0x56048235dba0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.438+0000 7f3269ed3740 1 --2- 172.21.15.111:0/263531197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604822e0c30 0x560482359b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.438+0000 7f3269ed3740 1 --2- 172.21.15.111:0/263531197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604822dbba0 0x5604822e9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.438+0000 7f3269ed3740 1 --2- 172.21.15.111:0/263531197 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604822db260 0x5604822e96d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.438+0000 7f324a000640 0 client.25649 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:54:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.438+0000 7f3269ed3740 1 -- 172.21.15.111:0/263531197 >> 172.21.15.111:0/263531197 conn(0x5604822d68d0 msgr2=0x5604822d7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:54:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.438+0000 7f3269ed3740 1 -- 172.21.15.111:0/263531197 shutdown_connections 2024-09-15T22:54:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.438+0000 7f3269ed3740 1 -- 172.21.15.111:0/263531197 wait complete. 2024-09-15T22:54:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.438+0000 7f3269ed3740 20 client.25649 trim_cache size 0 max 16384 2024-09-15T22:54:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:54:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:54:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:54:09.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:54:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.439+0000 7f3265a00640 1 -- 172.21.15.111:0/3064160932 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f32440a32c0 msgr2=0x7f32440a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.439+0000 7f3265a00640 1 --2- 172.21.15.111:0/3064160932 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f32440a32c0 0x7f32440a5780 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3258005fd0 tx=0x7f3258009500 comp rx=0 tx=0).stop 2024-09-15T22:54:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.439+0000 7f3265a00640 1 -- 172.21.15.111:0/3064160932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3260104d60 msgr2=0x7f326019a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.439+0000 7f3265a00640 1 --2- 172.21.15.111:0/3064160932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3260104d60 0x7f326019a3e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f324c00e9f0 tx=0x7f324c00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.439+0000 7f3265a00640 1 -- 172.21.15.111:0/3064160932 shutdown_connections 2024-09-15T22:54:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.439+0000 7f3265a00640 1 --2- 172.21.15.111:0/3064160932 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3260105720 0x7f32601a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.439+0000 7f3265a00640 1 --2- 172.21.15.111:0/3064160932 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f32440a32c0 0x7f32440a5780 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.439+0000 7f3265a00640 1 --2- 172.21.15.111:0/3064160932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3260104d60 0x7f326019a3e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.439+0000 7f3265a00640 1 --2- 172.21.15.111:0/3064160932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3260103b70 0x7f3260199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.439+0000 7f3265a00640 1 -- 172.21.15.111:0/3064160932 >> 172.21.15.111:0/3064160932 conn(0x7f32600ff2f0 msgr2=0x7f3260108100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:54:09.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.439+0000 7f3265a00640 1 -- 172.21.15.111:0/3064160932 shutdown_connections 2024-09-15T22:54:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:09.439+0000 7f3265a00640 1 -- 172.21.15.111:0/3064160932 wait complete. 2024-09-15T22:54:10.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:09 smithi192 ceph-mon[35427]: pgmap v597: 97 pgs: 97 active+clean; 267 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 170 op/s 2024-09-15T22:54:10.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:09 smithi121 ceph-mon[34815]: pgmap v597: 97 pgs: 97 active+clean; 267 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 170 op/s 2024-09-15T22:54:10.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:09 smithi111 ceph-mon[30629]: pgmap v597: 97 pgs: 97 active+clean; 267 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 170 op/s 2024-09-15T22:54:10.273 INFO:tasks.workunit.client.0.smithi111.stdout: 1 745918 9.92 MB/sec execute 278 sec latency 13.332 ms 2024-09-15T22:54:11.273 INFO:tasks.workunit.client.0.smithi111.stdout: 1 748605 9.93 MB/sec execute 279 sec latency 13.939 ms 2024-09-15T22:54:11.943 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:11 smithi111 ceph-mon[30629]: pgmap v598: 97 pgs: 97 active+clean; 268 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 183 op/s 2024-09-15T22:54:12.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:11 smithi192 ceph-mon[35427]: pgmap v598: 97 pgs: 97 active+clean; 268 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 183 op/s 2024-09-15T22:54:12.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:11 smithi121 ceph-mon[34815]: pgmap v598: 97 pgs: 97 active+clean; 268 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 183 op/s 2024-09-15T22:54:12.273 INFO:tasks.workunit.client.0.smithi111.stdout: 1 749548 9.91 MB/sec execute 280 sec latency 546.904 ms 2024-09-15T22:54:13.273 INFO:tasks.workunit.client.0.smithi111.stdout: 1 749548 9.88 MB/sec execute 281 sec latency 1546.969 ms 2024-09-15T22:54:14.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:13 smithi192 ceph-mon[35427]: pgmap v599: 97 pgs: 97 active+clean; 270 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 227 op/s 2024-09-15T22:54:14.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:13 smithi121 ceph-mon[34815]: pgmap v599: 97 pgs: 97 active+clean; 270 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 227 op/s 2024-09-15T22:54:14.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:13 smithi111 ceph-mon[30629]: pgmap v599: 97 pgs: 97 active+clean; 270 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 227 op/s 2024-09-15T22:54:14.273 INFO:tasks.workunit.client.0.smithi111.stdout: 1 751800 9.87 MB/sec execute 282 sec latency 1755.661 ms 2024-09-15T22:54:15.274 INFO:tasks.workunit.client.0.smithi111.stdout: 1 754203 9.89 MB/sec execute 283 sec latency 19.639 ms 2024-09-15T22:54:16.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:15 smithi192 ceph-mon[35427]: pgmap v600: 97 pgs: 97 active+clean; 270 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 214 op/s 2024-09-15T22:54:16.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:15 smithi121 ceph-mon[34815]: pgmap v600: 97 pgs: 97 active+clean; 270 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 214 op/s 2024-09-15T22:54:16.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:15 smithi111 ceph-mon[30629]: pgmap v600: 97 pgs: 97 active+clean; 270 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 214 op/s 2024-09-15T22:54:16.274 INFO:tasks.workunit.client.0.smithi111.stdout: 1 756984 9.91 MB/sec execute 284 sec latency 11.057 ms 2024-09-15T22:54:17.274 INFO:tasks.workunit.client.0.smithi111.stdout: 1 758754 9.90 MB/sec execute 285 sec latency 425.896 ms 2024-09-15T22:54:17.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:17 smithi111 ceph-mon[30629]: pgmap v601: 97 pgs: 97 active+clean; 272 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 201 op/s 2024-09-15T22:54:18.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:17 smithi192 ceph-mon[35427]: pgmap v601: 97 pgs: 97 active+clean; 272 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 201 op/s 2024-09-15T22:54:18.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:17 smithi121 ceph-mon[34815]: pgmap v601: 97 pgs: 97 active+clean; 272 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 201 op/s 2024-09-15T22:54:18.274 INFO:tasks.workunit.client.0.smithi111.stdout: 1 758754 9.87 MB/sec execute 286 sec latency 1425.965 ms 2024-09-15T22:54:19.274 INFO:tasks.workunit.client.0.smithi111.stdout: 1 760118 9.87 MB/sec execute 287 sec latency 1934.626 ms 2024-09-15T22:54:20.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:19 smithi192 ceph-mon[35427]: pgmap v602: 97 pgs: 97 active+clean; 274 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 207 op/s 2024-09-15T22:54:20.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:19 smithi121 ceph-mon[34815]: pgmap v602: 97 pgs: 97 active+clean; 274 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 207 op/s 2024-09-15T22:54:20.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:19 smithi111 ceph-mon[30629]: pgmap v602: 97 pgs: 97 active+clean; 274 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 207 op/s 2024-09-15T22:54:20.274 INFO:tasks.workunit.client.0.smithi111.stdout: 1 762838 9.88 MB/sec execute 288 sec latency 17.810 ms 2024-09-15T22:54:21.274 INFO:tasks.workunit.client.0.smithi111.stdout: 1 765551 9.89 MB/sec execute 289 sec latency 13.242 ms 2024-09-15T22:54:21.961 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:21 smithi111 ceph-mon[30629]: pgmap v603: 97 pgs: 97 active+clean; 276 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 212 op/s 2024-09-15T22:54:22.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:21 smithi192 ceph-mon[35427]: pgmap v603: 97 pgs: 97 active+clean; 276 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 212 op/s 2024-09-15T22:54:22.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:21 smithi121 ceph-mon[34815]: pgmap v603: 97 pgs: 97 active+clean; 276 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 212 op/s 2024-09-15T22:54:22.283 INFO:tasks.workunit.client.0.smithi111.stdout: 1 767316 9.90 MB/sec execute 290 sec latency 364.709 ms 2024-09-15T22:54:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.553+0000 7f6f73800640 1 -- 172.21.15.111:0/4239120318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f6c103d60 msgr2=0x7f6f6c104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.553+0000 7f6f73800640 1 --2- 172.21.15.111:0/4239120318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f6c103d60 0x7f6f6c104160 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6f60009900 tx=0x7f6f6002a410 comp rx=0 tx=0).stop 2024-09-15T22:54:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.553+0000 7f6f73800640 1 -- 172.21.15.111:0/4239120318 shutdown_connections 2024-09-15T22:54:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.553+0000 7f6f73800640 1 --2- 172.21.15.111:0/4239120318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f6c105910 0x7f6f6c10c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.553+0000 7f6f73800640 1 --2- 172.21.15.111:0/4239120318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f6c104f50 0x7f6f6c1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.553+0000 7f6f73800640 1 --2- 172.21.15.111:0/4239120318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f6c103d60 0x7f6f6c104160 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.553+0000 7f6f73800640 1 -- 172.21.15.111:0/4239120318 >> 172.21.15.111:0/4239120318 conn(0x7f6f6c0ff4e0 msgr2=0x7f6f6c101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:54:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.553+0000 7f6f73800640 1 -- 172.21.15.111:0/4239120318 shutdown_connections 2024-09-15T22:54:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.553+0000 7f6f73800640 1 -- 172.21.15.111:0/4239120318 wait complete. 2024-09-15T22:54:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.554+0000 7f6f73800640 1 Processor -- start 2024-09-15T22:54:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.554+0000 7f6f73800640 1 -- start start 2024-09-15T22:54:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.554+0000 7f6f73800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f6c103d60 0x7f6f6c19a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.554+0000 7f6f73800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f6c104f50 0x7f6f6c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.554+0000 7f6f73800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f6c105910 0x7f6f6c1a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.554+0000 7f6f73800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c10fad0 con 0x7f6f6c105910 2024-09-15T22:54:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.554+0000 7f6f73800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c10f950 con 0x7f6f6c104f50 2024-09-15T22:54:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.554+0000 7f6f73800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c10fc50 con 0x7f6f6c103d60 2024-09-15T22:54:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.555+0000 7f6f71800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f6c105910 0x7f6f6c1a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.555+0000 7f6f70e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f6c103d60 0x7f6f6c19a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.555+0000 7f6f70e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f6c103d60 0x7f6f6c19a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:40066/0 (socket says 172.21.15.111:40066) 2024-09-15T22:54:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.555+0000 7f6f70e00640 1 -- 172.21.15.111:0/3752915224 learned_addr learned my addr 172.21.15.111:0/3752915224 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:54:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.555+0000 7f6f6be00640 1 --2- 172.21.15.111:0/3752915224 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f6c104f50 0x7f6f6c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.555+0000 7f6f70e00640 1 -- 172.21.15.111:0/3752915224 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f6c104f50 msgr2=0x7f6f6c19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.555+0000 7f6f70e00640 1 --2- 172.21.15.111:0/3752915224 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f6c104f50 0x7f6f6c19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.555+0000 7f6f70e00640 1 -- 172.21.15.111:0/3752915224 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f6c105910 msgr2=0x7f6f6c1a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.555+0000 7f6f70e00640 1 --2- 172.21.15.111:0/3752915224 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f6c105910 0x7f6f6c1a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.555+0000 7f6f70e00640 1 -- 172.21.15.111:0/3752915224 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6c1a1c80 con 0x7f6f6c103d60 2024-09-15T22:54:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.555+0000 7f6f70e00640 1 --2- 172.21.15.111:0/3752915224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f6c103d60 0x7f6f6c19a060 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6f60031440 tx=0x7f6f60032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:54:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.556+0000 7f6f69600640 1 -- 172.21.15.111:0/3752915224 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6003f030 con 0x7f6f6c103d60 2024-09-15T22:54:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.556+0000 7f6f69600640 1 -- 172.21.15.111:0/3752915224 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f60005d10 con 0x7f6f6c103d60 2024-09-15T22:54:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.556+0000 7f6f69600640 1 -- 172.21.15.111:0/3752915224 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60035070 con 0x7f6f6c103d60 2024-09-15T22:54:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.556+0000 7f6f73800640 1 -- 172.21.15.111:0/3752915224 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f6c1a1f10 con 0x7f6f6c103d60 2024-09-15T22:54:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.556+0000 7f6f73800640 1 -- 172.21.15.111:0/3752915224 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f6c1a2340 con 0x7f6f6c103d60 2024-09-15T22:54:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.559+0000 7f6f69600640 1 -- 172.21.15.111:0/3752915224 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f6f60042030 con 0x7f6f6c103d60 2024-09-15T22:54:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.559+0000 7f6f767da740 1 Processor -- start 2024-09-15T22:54:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.559+0000 7f6f767da740 1 -- start start 2024-09-15T22:54:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.560+0000 7f6f767da740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5559f09d2090 0x5559f09d2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.560+0000 7f6f767da740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559f09d29d0 0x5559f09d7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.560+0000 7f6f767da740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d7a60 0x5559f09d9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.560+0000 7f6f767da740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5559f0937100 con 0x5559f09d7a60 2024-09-15T22:54:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.560+0000 7f6f767da740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5559f08df1b0 con 0x5559f09d29d0 2024-09-15T22:54:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.560+0000 7f6f767da740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5559f09330c0 con 0x5559f09d2090 2024-09-15T22:54:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.560+0000 7f6f6be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d7a60 0x5559f09d9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.560+0000 7f6f71800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5559f09d2090 0x5559f09d2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.560+0000 7f6f6be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d7a60 0x5559f09d9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34278/0 (socket says 172.21.15.111:34278) 2024-09-15T22:54:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.560+0000 7f6f6be00640 1 -- 172.21.15.111:0/1017003543 learned_addr learned my addr 172.21.15.111:0/1017003543 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:54:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.560+0000 7f6f70e00640 1 --2- 172.21.15.111:0/1017003543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559f09d29d0 0x5559f09d7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.560+0000 7f6f69600640 1 --2- 172.21.15.111:0/3752915224 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6f540a32f0 0x7f6f540a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.561+0000 7f6f70e00640 1 -- 172.21.15.111:0/1017003543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5559f09d2090 msgr2=0x5559f09d2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.561+0000 7f6f70e00640 1 --2- 172.21.15.111:0/1017003543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5559f09d2090 0x5559f09d2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.561+0000 7f6f70e00640 1 -- 172.21.15.111:0/1017003543 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d7a60 msgr2=0x5559f09d9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.561+0000 7f6f70e00640 1 --2- 172.21.15.111:0/1017003543 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d7a60 0x5559f09d9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.561+0000 7f6f70e00640 1 -- 172.21.15.111:0/1017003543 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559f064a210 con 0x5559f09d29d0 2024-09-15T22:54:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.561+0000 7f6f6be00640 1 --2- 172.21.15.111:0/1017003543 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d7a60 0x5559f09d9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:54:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.561+0000 7f6f71800640 1 --2- 172.21.15.111:0/3752915224 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6f540a32f0 0x7f6f540a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.561+0000 7f6f69600640 1 -- 172.21.15.111:0/3752915224 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f6f600efde0 con 0x7f6f6c103d60 2024-09-15T22:54:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.561+0000 7f6f70e00640 1 --2- 172.21.15.111:0/1017003543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559f09d29d0 0x5559f09d7490 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6f600ed880 tx=0x7f6f600b8880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1331a75ef50053cd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:54:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.561+0000 7f6f5e000640 1 -- 172.21.15.111:0/1017003543 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60049050 con 0x5559f09d29d0 2024-09-15T22:54:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.561+0000 7f6f5e000640 1 -- 172.21.15.111:0/1017003543 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f600bc4b0 con 0x5559f09d29d0 2024-09-15T22:54:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.561+0000 7f6f5e000640 1 -- 172.21.15.111:0/1017003543 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f600c07b0 con 0x5559f09d29d0 2024-09-15T22:54:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.562+0000 7f6f767da740 1 -- 172.21.15.111:0/1017003543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559f09d29d0 msgr2=0x5559f09d7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.562+0000 7f6f71800640 1 --2- 172.21.15.111:0/3752915224 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6f540a32f0 0x7f6f540a57b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6f6400a500 tx=0x7f6f64004ad0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:54:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.562+0000 7f6f767da740 1 --2- 172.21.15.111:0/1017003543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559f09d29d0 0x5559f09d7490 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6f600ed880 tx=0x7f6f600b8880 comp rx=0 tx=0).stop 2024-09-15T22:54:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.562+0000 7f6f767da740 1 -- 172.21.15.111:0/1017003543 shutdown_connections 2024-09-15T22:54:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.562+0000 7f6f767da740 1 --2- 172.21.15.111:0/1017003543 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d7a60 0x5559f09d9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.562+0000 7f6f767da740 1 --2- 172.21.15.111:0/1017003543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559f09d29d0 0x5559f09d7490 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.562+0000 7f6f767da740 1 --2- 172.21.15.111:0/1017003543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5559f09d2090 0x5559f09d2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.562+0000 7f6f767da740 1 -- 172.21.15.111:0/1017003543 >> 172.21.15.111:0/1017003543 conn(0x5559f09cd700 msgr2=0x5559f09cdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:54:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.562+0000 7f6f767da740 1 -- 172.21.15.111:0/1017003543 shutdown_connections 2024-09-15T22:54:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.562+0000 7f6f767da740 1 -- 172.21.15.111:0/1017003543 wait complete. 2024-09-15T22:54:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.562+0000 7f6f767da740 1 -- start start 2024-09-15T22:54:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.563+0000 7f6f767da740 1 Processor -- start 2024-09-15T22:54:22.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.563+0000 7f6f767da740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d2090 0x5559f09dbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:22.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.563+0000 7f6f767da740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5559f09d29d0 0x5559f09dc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:22.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.563+0000 7f6f767da740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559f09d7a60 0x5559f09e3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:22.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.563+0000 7f6f767da740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5559f08df1b0 con 0x5559f09d2090 2024-09-15T22:54:22.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.563+0000 7f6f767da740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5559f08c0090 con 0x5559f09d7a60 2024-09-15T22:54:22.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.563+0000 7f6f767da740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5559f09da7f0 con 0x5559f09d29d0 2024-09-15T22:54:22.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.563+0000 7f6f767da740 10 client.0 _ll_register_callbacks cb 0x5559f0914f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:54:22.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.563+0000 7f6f70e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d2090 0x5559f09dbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:22.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.563+0000 7f6f70e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d2090 0x5559f09dbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34294/0 (socket says 172.21.15.111:34294) 2024-09-15T22:54:22.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.563+0000 7f6f70e00640 1 -- 172.21.15.111:0/485228457 learned_addr learned my addr 172.21.15.111:0/485228457 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:54:22.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.563+0000 7f6f6be00640 1 --2- 172.21.15.111:0/485228457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5559f09d29d0 0x5559f09dc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:22.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.564+0000 7f6f70e00640 1 -- 172.21.15.111:0/485228457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5559f09d29d0 msgr2=0x5559f09dc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:22.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.564+0000 7f6f70e00640 1 --2- 172.21.15.111:0/485228457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5559f09d29d0 0x5559f09dc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.564+0000 7f6f70e00640 1 -- 172.21.15.111:0/485228457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559f09d7a60 msgr2=0x5559f09e3410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:54:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.564+0000 7f6f70e00640 1 --2- 172.21.15.111:0/485228457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559f09d7a60 0x5559f09e3410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.564+0000 7f6f70e00640 1 -- 172.21.15.111:0/485228457 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559f0716620 con 0x5559f09d2090 2024-09-15T22:54:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.564+0000 7f6f6be00640 1 --2- 172.21.15.111:0/485228457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5559f09d29d0 0x5559f09dc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:54:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.564+0000 7f6f70e00640 1 --2- 172.21.15.111:0/485228457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d2090 0x5559f09dbe90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6f600bbf80 tx=0x7f6f600b8cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2b9792332c960e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:54:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.564+0000 7f6f52a00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:54:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.564+0000 7f6f52a00640 1 -- 172.21.15.111:0/485228457 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60049040 con 0x5559f09d2090 2024-09-15T22:54:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.564+0000 7f6f52a00640 1 -- 172.21.15.111:0/485228457 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f6f600ba7b0 con 0x5559f09d2090 2024-09-15T22:54:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.564+0000 7f6f767da740 1 -- 172.21.15.111:0/485228457 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f600bc4b0 con 0x5559f09d2090 2024-09-15T22:54:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.564+0000 7f6f52a00640 1 -- 172.21.15.111:0/485228457 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6f600bc280 con 0x5559f09d2090 2024-09-15T22:54:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.564+0000 7f6f767da740 10 client.15669 fetch_fsmap learned FSMap version 30 2024-09-15T22:54:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.564+0000 7f6f767da740 1 -- 172.21.15.111:0/485228457 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5559f09e6200 con 0x5559f09d2090 2024-09-15T22:54:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.565+0000 7f6f52a00640 1 -- 172.21.15.111:0/485228457 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f600bc4b0 con 0x5559f09d2090 2024-09-15T22:54:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.565+0000 7f6f52a00640 1 -- 172.21.15.111:0/485228457 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f6f6003b460 con 0x5559f09d2090 2024-09-15T22:54:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.565+0000 7f6f767da740 10 client.15669 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:54:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.565+0000 7f6f767da740 10 client.15669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:54:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.565+0000 7f6f767da740 20 client.15669 populate_metadata read hostname 'smithi111' 2024-09-15T22:54:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.565+0000 7f6f767da740 1 --2- 172.21.15.111:0/485228457 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5559f09e77d0 0x5559f09e9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.565+0000 7f6f767da740 4 client.15669 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:54:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.565+0000 7f6f767da740 1 -- 172.21.15.111:0/485228457 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559f0944830 con 0x5559f09e77d0 2024-09-15T22:54:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.565+0000 7f6f6be00640 1 --2- 172.21.15.111:0/485228457 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5559f09e77d0 0x5559f09e9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.566+0000 7f6f6be00640 1 --2- 172.21.15.111:0/485228457 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5559f09e77d0 0x5559f09e9bc0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20505c31f6689f40 server_cookie=b292bdb5b5817b53 in_seq=0 out_seq=0 2024-09-15T22:54:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.566+0000 7f6f52a00640 10 client.15669 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:54:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.567+0000 7f6f52a00640 1 -- 172.21.15.111:0/485228457 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5559f0944830 con 0x5559f09e77d0 2024-09-15T22:54:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.567+0000 7f6f52a00640 10 client.15669 handle_command_reply: tid=0 2024-09-15T22:54:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.567+0000 7f6f767da740 1 client.15669 shutdown 2024-09-15T22:54:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.567+0000 7f6f767da740 1 -- 172.21.15.111:0/485228457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d2090 msgr2=0x5559f09dbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.567+0000 7f6f767da740 1 --2- 172.21.15.111:0/485228457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d2090 0x5559f09dbe90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6f600bbf80 tx=0x7f6f600b8cc0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.567+0000 7f6f767da740 1 -- 172.21.15.111:0/485228457 shutdown_connections 2024-09-15T22:54:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.567+0000 7f6f767da740 1 --2- 172.21.15.111:0/485228457 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5559f09e77d0 0x5559f09e9bc0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.567+0000 7f6f767da740 1 --2- 172.21.15.111:0/485228457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559f09d7a60 0x5559f09e3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.567+0000 7f6f767da740 1 --2- 172.21.15.111:0/485228457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5559f09d29d0 0x5559f09dc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.567+0000 7f6f52a00640 0 client.15669 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:54:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.567+0000 7f6f767da740 1 --2- 172.21.15.111:0/485228457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d2090 0x5559f09dbe90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.567+0000 7f6f767da740 1 -- 172.21.15.111:0/485228457 >> 172.21.15.111:0/485228457 conn(0x5559f09cd700 msgr2=0x5559f09cef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:54:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.567+0000 7f6f767da740 1 -- 172.21.15.111:0/485228457 shutdown_connections 2024-09-15T22:54:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.567+0000 7f6f767da740 1 -- 172.21.15.111:0/485228457 wait complete. 2024-09-15T22:54:22.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.567+0000 7f6f767da740 20 client.15669 trim_cache size 0 max 16384 2024-09-15T22:54:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.582+0000 7f6f767da740 1 Processor -- start 2024-09-15T22:54:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.582+0000 7f6f767da740 1 -- start start 2024-09-15T22:54:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.582+0000 7f6f767da740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559f09d2090 0x5559f09dc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.582+0000 7f6f767da740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5559f09d29d0 0x5559f09e33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.582+0000 7f6f767da740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d7a60 0x5559f09cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.582+0000 7f6f767da740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c1ad8f0 con 0x5559f09d7a60 2024-09-15T22:54:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.582+0000 7f6f767da740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c1ab1c0 con 0x5559f09d2090 2024-09-15T22:54:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.582+0000 7f6f767da740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c1a9010 con 0x5559f09d29d0 2024-09-15T22:54:22.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.582+0000 7f6f71800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d7a60 0x5559f09cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.582+0000 7f6f71800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d7a60 0x5559f09cc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34302/0 (socket says 172.21.15.111:34302) 2024-09-15T22:54:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.582+0000 7f6f71800640 1 -- 172.21.15.111:0/948807791 learned_addr learned my addr 172.21.15.111:0/948807791 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:54:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.582+0000 7f6f6be00640 1 --2- 172.21.15.111:0/948807791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5559f09d29d0 0x5559f09e33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.582+0000 7f6f70e00640 1 --2- 172.21.15.111:0/948807791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559f09d2090 0x5559f09dc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.583+0000 7f6f71800640 1 -- 172.21.15.111:0/948807791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5559f09d29d0 msgr2=0x5559f09e33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.583+0000 7f6f71800640 1 --2- 172.21.15.111:0/948807791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5559f09d29d0 0x5559f09e33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.583+0000 7f6f71800640 1 -- 172.21.15.111:0/948807791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559f09d2090 msgr2=0x5559f09dc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.583+0000 7f6f71800640 1 --2- 172.21.15.111:0/948807791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559f09d2090 0x5559f09dc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.583+0000 7f6f71800640 1 -- 172.21.15.111:0/948807791 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559f09da630 con 0x5559f09d7a60 2024-09-15T22:54:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.583+0000 7f6f70e00640 1 --2- 172.21.15.111:0/948807791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559f09d2090 0x5559f09dc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:54:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.583+0000 7f6f71800640 1 --2- 172.21.15.111:0/948807791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d7a60 0x5559f09cc770 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6f64016690 tx=0x7f6f64014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa98733b5337db1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:54:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.583+0000 7f6f5d600640 1 -- 172.21.15.111:0/948807791 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f64027070 con 0x5559f09d7a60 2024-09-15T22:54:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.583+0000 7f6f5d600640 1 -- 172.21.15.111:0/948807791 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f6401d050 con 0x5559f09d7a60 2024-09-15T22:54:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.583+0000 7f6f5d600640 1 -- 172.21.15.111:0/948807791 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f64022da0 con 0x5559f09d7a60 2024-09-15T22:54:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.584+0000 7f6f767da740 1 -- 172.21.15.111:0/948807791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d7a60 msgr2=0x5559f09cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.584+0000 7f6f767da740 1 --2- 172.21.15.111:0/948807791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d7a60 0x5559f09cc770 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6f64016690 tx=0x7f6f64014000 comp rx=0 tx=0).stop 2024-09-15T22:54:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.584+0000 7f6f767da740 1 -- 172.21.15.111:0/948807791 shutdown_connections 2024-09-15T22:54:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.584+0000 7f6f767da740 1 --2- 172.21.15.111:0/948807791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d7a60 0x5559f09cc770 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.584+0000 7f6f767da740 1 --2- 172.21.15.111:0/948807791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5559f09d29d0 0x5559f09e33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.584+0000 7f6f767da740 1 --2- 172.21.15.111:0/948807791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559f09d2090 0x5559f09dc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.584+0000 7f6f767da740 1 -- 172.21.15.111:0/948807791 >> 172.21.15.111:0/948807791 conn(0x5559f09cd700 msgr2=0x5559f09cc060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:54:22.588 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.584+0000 7f6f767da740 1 -- 172.21.15.111:0/948807791 shutdown_connections 2024-09-15T22:54:22.588 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.584+0000 7f6f767da740 1 -- 172.21.15.111:0/948807791 wait complete. 2024-09-15T22:54:22.588 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.584+0000 7f6f767da740 1 -- start start 2024-09-15T22:54:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.585+0000 7f6f767da740 1 Processor -- start 2024-09-15T22:54:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.585+0000 7f6f767da740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5559f09d2090 0x5559f09e1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.585+0000 7f6f767da740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d29d0 0x5559f09e1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.585+0000 7f6f767da740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559f09d7a60 0x5559f09e2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.585+0000 7f6f767da740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c1a9680 con 0x5559f09d29d0 2024-09-15T22:54:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.585+0000 7f6f767da740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c1aaaf0 con 0x5559f09d7a60 2024-09-15T22:54:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.585+0000 7f6f767da740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6f6c1a9d50 con 0x5559f09d2090 2024-09-15T22:54:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.585+0000 7f6f767da740 10 client.0 _ll_register_callbacks cb 0x7f6f58009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:54:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.585+0000 7f6f6be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d29d0 0x5559f09e1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:22.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.585+0000 7f6f70e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5559f09d2090 0x5559f09e1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.585+0000 7f6f6be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d29d0 0x5559f09e1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34316/0 (socket says 172.21.15.111:34316) 2024-09-15T22:54:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.585+0000 7f6f6be00640 1 -- 172.21.15.111:0/1711208523 learned_addr learned my addr 172.21.15.111:0/1711208523 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:54:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.585+0000 7f6f71800640 1 --2- 172.21.15.111:0/1711208523 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559f09d7a60 0x5559f09e2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.586+0000 7f6f70e00640 1 -- 172.21.15.111:0/1711208523 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559f09d7a60 msgr2=0x5559f09e2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.586+0000 7f6f70e00640 1 --2- 172.21.15.111:0/1711208523 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559f09d7a60 0x5559f09e2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.586+0000 7f6f70e00640 1 -- 172.21.15.111:0/1711208523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d29d0 msgr2=0x5559f09e1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.586+0000 7f6f70e00640 1 --2- 172.21.15.111:0/1711208523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d29d0 0x5559f09e1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.586+0000 7f6f70e00640 1 -- 172.21.15.111:0/1711208523 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559f09e2b90 con 0x5559f09d2090 2024-09-15T22:54:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.586+0000 7f6f6be00640 1 --2- 172.21.15.111:0/1711208523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d29d0 0x5559f09e1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:54:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.586+0000 7f6f70e00640 1 --2- 172.21.15.111:0/1711208523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5559f09d2090 0x5559f09e1030 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6f600314e0 tx=0x7f6f600b87d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c946975a4dde2cf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:54:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.586+0000 7f6f52a00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:54:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.586+0000 7f6f52a00640 1 -- 172.21.15.111:0/1711208523 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f60049040 con 0x5559f09d2090 2024-09-15T22:54:22.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.586+0000 7f6f767da740 1 -- 172.21.15.111:0/1711208523 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f6401d050 con 0x5559f09d2090 2024-09-15T22:54:22.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.586+0000 7f6f52a00640 1 -- 172.21.15.111:0/1711208523 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f6f60039350 con 0x5559f09d2090 2024-09-15T22:54:22.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.586+0000 7f6f52a00640 1 -- 172.21.15.111:0/1711208523 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6f600bbff0 con 0x5559f09d2090 2024-09-15T22:54:22.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.586+0000 7f6f767da740 10 client.25684 fetch_fsmap learned FSMap version 30 2024-09-15T22:54:22.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.586+0000 7f6f767da740 1 -- 172.21.15.111:0/1711208523 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5559f0a52870 con 0x5559f09d2090 2024-09-15T22:54:22.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.586+0000 7f6f52a00640 1 -- 172.21.15.111:0/1711208523 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f6401d050 con 0x5559f09d2090 2024-09-15T22:54:22.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.587+0000 7f6f52a00640 1 -- 172.21.15.111:0/1711208523 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f6f600c4be0 con 0x5559f09d2090 2024-09-15T22:54:22.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.587+0000 7f6f767da740 10 client.25684 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:54:22.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.587+0000 7f6f767da740 10 client.25684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:54:22.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.587+0000 7f6f767da740 20 client.25684 populate_metadata read hostname 'smithi111' 2024-09-15T22:54:22.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.587+0000 7f6f767da740 1 --2- 172.21.15.111:0/1711208523 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5559f0a52df0 0x5559f0a55210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:22.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.587+0000 7f6f767da740 4 client.25684 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:54:22.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.587+0000 7f6f767da740 1 -- 172.21.15.111:0/1711208523 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5559f093eb80 con 0x5559f0a52df0 2024-09-15T22:54:22.601 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.600+0000 7f6f6be00640 1 --2- 172.21.15.111:0/1711208523 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5559f0a52df0 0x5559f0a55210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:22.649 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.647+0000 7f6f52a00640 10 client.25684 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:54:22.649 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.647+0000 7f6f6be00640 1 --2- 172.21.15.111:0/1711208523 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5559f0a52df0 0x5559f0a55210 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89b031987651b85c server_cookie=ac644173a27bc3e8 in_seq=0 out_seq=0 2024-09-15T22:54:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.683+0000 7f6f52a00640 1 -- 172.21.15.111:0/1711208523 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5559f093eb80 con 0x5559f0a52df0 2024-09-15T22:54:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.683+0000 7f6f52a00640 10 client.25684 handle_command_reply: tid=0 2024-09-15T22:54:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.683+0000 7f6f767da740 1 client.25684 shutdown 2024-09-15T22:54:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.683+0000 7f6f767da740 1 -- 172.21.15.111:0/1711208523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5559f09d2090 msgr2=0x5559f09e1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:22.685 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.683+0000 7f6f767da740 1 --2- 172.21.15.111:0/1711208523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5559f09d2090 0x5559f09e1030 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6f600314e0 tx=0x7f6f600b87d0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.684+0000 7f6f767da740 1 -- 172.21.15.111:0/1711208523 shutdown_connections 2024-09-15T22:54:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.684+0000 7f6f767da740 1 --2- 172.21.15.111:0/1711208523 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5559f0a52df0 0x5559f0a55210 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.684+0000 7f6f767da740 1 --2- 172.21.15.111:0/1711208523 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5559f09d7a60 0x5559f09e2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.684+0000 7f6f767da740 1 --2- 172.21.15.111:0/1711208523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5559f09d29d0 0x5559f09e1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.684+0000 7f6f52a00640 0 client.25684 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:54:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.684+0000 7f6f767da740 1 --2- 172.21.15.111:0/1711208523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5559f09d2090 0x5559f09e1030 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.684+0000 7f6f767da740 1 -- 172.21.15.111:0/1711208523 >> 172.21.15.111:0/1711208523 conn(0x5559f09cd700 msgr2=0x5559f09cc880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:54:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.684+0000 7f6f767da740 1 -- 172.21.15.111:0/1711208523 shutdown_connections 2024-09-15T22:54:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.684+0000 7f6f767da740 1 -- 172.21.15.111:0/1711208523 wait complete. 2024-09-15T22:54:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.684+0000 7f6f767da740 20 client.25684 trim_cache size 0 max 16384 2024-09-15T22:54:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:54:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:54:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:54:22.687 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:54:22.688 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.684+0000 7f6f73800640 1 -- 172.21.15.111:0/3752915224 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6f540a32f0 msgr2=0x7f6f540a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.684+0000 7f6f73800640 1 --2- 172.21.15.111:0/3752915224 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6f540a32f0 0x7f6f540a57b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6f6400a500 tx=0x7f6f64004ad0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.684+0000 7f6f73800640 1 -- 172.21.15.111:0/3752915224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f6c103d60 msgr2=0x7f6f6c19a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.684+0000 7f6f73800640 1 --2- 172.21.15.111:0/3752915224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f6c103d60 0x7f6f6c19a060 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6f60031440 tx=0x7f6f60032ce0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.685+0000 7f6f73800640 1 -- 172.21.15.111:0/3752915224 shutdown_connections 2024-09-15T22:54:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.685+0000 7f6f73800640 1 --2- 172.21.15.111:0/3752915224 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f6c105910 0x7f6f6c1a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.685+0000 7f6f73800640 1 --2- 172.21.15.111:0/3752915224 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6f540a32f0 0x7f6f540a57b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.685+0000 7f6f73800640 1 --2- 172.21.15.111:0/3752915224 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f6c104f50 0x7f6f6c19a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.685+0000 7f6f73800640 1 --2- 172.21.15.111:0/3752915224 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f6c103d60 0x7f6f6c19a060 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.685+0000 7f6f73800640 1 -- 172.21.15.111:0/3752915224 >> 172.21.15.111:0/3752915224 conn(0x7f6f6c0ff4e0 msgr2=0x7f6f6c108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:54:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.685+0000 7f6f73800640 1 -- 172.21.15.111:0/3752915224 shutdown_connections 2024-09-15T22:54:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:22.685+0000 7f6f73800640 1 -- 172.21.15.111:0/3752915224 wait complete. 2024-09-15T22:54:23.283 INFO:tasks.workunit.client.0.smithi111.stdout: 1 767316 9.87 MB/sec execute 291 sec latency 1364.776 ms 2024-09-15T22:54:24.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:23 smithi192 ceph-mon[35427]: pgmap v604: 97 pgs: 97 active+clean; 274 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 194 op/s 2024-09-15T22:54:24.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:23 smithi121 ceph-mon[34815]: pgmap v604: 97 pgs: 97 active+clean; 274 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 194 op/s 2024-09-15T22:54:24.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:23 smithi111 ceph-mon[30629]: pgmap v604: 97 pgs: 97 active+clean; 274 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 194 op/s 2024-09-15T22:54:24.283 INFO:tasks.workunit.client.0.smithi111.stdout: 1 768733 9.85 MB/sec execute 292 sec latency 1747.641 ms 2024-09-15T22:54:25.283 INFO:tasks.workunit.client.0.smithi111.stdout: 1 771413 9.87 MB/sec execute 293 sec latency 13.537 ms 2024-09-15T22:54:26.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:25 smithi192 ceph-mon[35427]: pgmap v605: 97 pgs: 97 active+clean; 279 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 172 op/s 2024-09-15T22:54:26.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:25 smithi121 ceph-mon[34815]: pgmap v605: 97 pgs: 97 active+clean; 279 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 172 op/s 2024-09-15T22:54:26.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:25 smithi111 ceph-mon[30629]: pgmap v605: 97 pgs: 97 active+clean; 279 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 172 op/s 2024-09-15T22:54:26.283 INFO:tasks.workunit.client.0.smithi111.stdout: 1 774487 9.90 MB/sec execute 294 sec latency 9.553 ms 2024-09-15T22:54:27.288 INFO:tasks.workunit.client.0.smithi111.stdout: 1 776764 9.90 MB/sec execute 295 sec latency 209.590 ms 2024-09-15T22:54:27.903 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:27 smithi111 ceph-mon[30629]: pgmap v606: 97 pgs: 97 active+clean; 279 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 174 op/s 2024-09-15T22:54:28.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:27 smithi192 ceph-mon[35427]: pgmap v606: 97 pgs: 97 active+clean; 279 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 174 op/s 2024-09-15T22:54:28.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:27 smithi121 ceph-mon[34815]: pgmap v606: 97 pgs: 97 active+clean; 279 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 174 op/s 2024-09-15T22:54:28.288 INFO:tasks.workunit.client.0.smithi111.stdout: 1 776766 9.87 MB/sec execute 296 sec latency 622.276 ms 2024-09-15T22:54:29.289 INFO:tasks.workunit.client.0.smithi111.stdout: 1 778125 9.86 MB/sec execute 297 sec latency 913.401 ms 2024-09-15T22:54:30.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:29 smithi192 ceph-mon[35427]: pgmap v607: 97 pgs: 97 active+clean; 281 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 186 op/s 2024-09-15T22:54:30.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:29 smithi121 ceph-mon[34815]: pgmap v607: 97 pgs: 97 active+clean; 281 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 186 op/s 2024-09-15T22:54:30.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:29 smithi111 ceph-mon[30629]: pgmap v607: 97 pgs: 97 active+clean; 281 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 186 op/s 2024-09-15T22:54:30.289 INFO:tasks.workunit.client.0.smithi111.stdout: 1 781233 9.89 MB/sec execute 298 sec latency 14.819 ms 2024-09-15T22:54:31.289 INFO:tasks.workunit.client.0.smithi111.stdout: 1 783528 9.88 MB/sec execute 299 sec latency 84.891 ms 2024-09-15T22:54:32.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:31 smithi192 ceph-mon[35427]: pgmap v608: 97 pgs: 97 active+clean; 279 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 186 op/s 2024-09-15T22:54:32.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:31 smithi121 ceph-mon[34815]: pgmap v608: 97 pgs: 97 active+clean; 279 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 186 op/s 2024-09-15T22:54:32.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:31 smithi111 ceph-mon[30629]: pgmap v608: 97 pgs: 97 active+clean; 279 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 186 op/s 2024-09-15T22:54:32.289 INFO:tasks.workunit.client.0.smithi111.stdout: 1 785404 9.89 MB/sec execute 300 sec latency 140.890 ms 2024-09-15T22:54:33.289 INFO:tasks.workunit.client.0.smithi111.stdout: 1 785405 9.86 MB/sec execute 301 sec latency 619.508 ms 2024-09-15T22:54:33.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:33 smithi111 ceph-mon[30629]: pgmap v609: 97 pgs: 97 active+clean; 284 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 198 op/s 2024-09-15T22:54:34.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:33 smithi192 ceph-mon[35427]: pgmap v609: 97 pgs: 97 active+clean; 284 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 198 op/s 2024-09-15T22:54:34.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:33 smithi121 ceph-mon[34815]: pgmap v609: 97 pgs: 97 active+clean; 284 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 198 op/s 2024-09-15T22:54:34.289 INFO:tasks.workunit.client.0.smithi111.stdout: 1 787653 9.87 MB/sec execute 302 sec latency 973.938 ms 2024-09-15T22:54:35.289 INFO:tasks.workunit.client.0.smithi111.stdout: 1 790268 9.87 MB/sec execute 303 sec latency 11.054 ms 2024-09-15T22:54:35.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.803+0000 7f1915200640 1 -- 172.21.15.111:0/1960499073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1910104f50 msgr2=0x7f19101053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:35.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.803+0000 7f1915200640 1 --2- 172.21.15.111:0/1960499073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1910104f50 0x7f19101053d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009900 tx=0x7f18fc02a400 comp rx=0 tx=0).stop 2024-09-15T22:54:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.803+0000 7f1915200640 1 -- 172.21.15.111:0/1960499073 shutdown_connections 2024-09-15T22:54:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.803+0000 7f1915200640 1 --2- 172.21.15.111:0/1960499073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1910105910 0x7f191010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.803+0000 7f1915200640 1 --2- 172.21.15.111:0/1960499073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1910104f50 0x7f19101053d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.803+0000 7f1915200640 1 --2- 172.21.15.111:0/1960499073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1910103d60 0x7f1910104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.803+0000 7f1915200640 1 -- 172.21.15.111:0/1960499073 >> 172.21.15.111:0/1960499073 conn(0x7f19100ff4e0 msgr2=0x7f1910101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:54:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.803+0000 7f1915200640 1 -- 172.21.15.111:0/1960499073 shutdown_connections 2024-09-15T22:54:35.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.803+0000 7f1915200640 1 -- 172.21.15.111:0/1960499073 wait complete. 2024-09-15T22:54:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.804+0000 7f1915200640 1 Processor -- start 2024-09-15T22:54:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.804+0000 7f1915200640 1 -- start start 2024-09-15T22:54:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.805+0000 7f1915200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1910103d60 0x7f191019e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:35.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.805+0000 7f1915200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1910104f50 0x7f191019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.805+0000 7f1915200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1910105910 0x7f19101a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.805+0000 7f1915200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f191010fa10 con 0x7f1910104f50 2024-09-15T22:54:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.805+0000 7f1915200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f191010f890 con 0x7f1910105910 2024-09-15T22:54:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.805+0000 7f1915200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f191010fb90 con 0x7f1910103d60 2024-09-15T22:54:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.805+0000 7f190e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1910105910 0x7f19101a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.805+0000 7f190d400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1910104f50 0x7f191019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.805+0000 7f190e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1910105910 0x7f19101a5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:38994/0 (socket says 172.21.15.111:38994) 2024-09-15T22:54:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.805+0000 7f190e800640 1 -- 172.21.15.111:0/3066588856 learned_addr learned my addr 172.21.15.111:0/3066588856 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:54:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.805+0000 7f190de00640 1 --2- 172.21.15.111:0/3066588856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1910103d60 0x7f191019e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.805+0000 7f190e800640 1 -- 172.21.15.111:0/3066588856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1910103d60 msgr2=0x7f191019e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.805+0000 7f190e800640 1 --2- 172.21.15.111:0/3066588856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1910103d60 0x7f191019e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.805+0000 7f190e800640 1 -- 172.21.15.111:0/3066588856 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1910104f50 msgr2=0x7f191019e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:35.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.805+0000 7f190e800640 1 --2- 172.21.15.111:0/3066588856 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1910104f50 0x7f191019e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.805+0000 7f190e800640 1 -- 172.21.15.111:0/3066588856 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19101a5fd0 con 0x7f1910105910 2024-09-15T22:54:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.805+0000 7f190d400640 1 --2- 172.21.15.111:0/3066588856 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1910104f50 0x7f191019e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:54:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.806+0000 7f190e800640 1 --2- 172.21.15.111:0/3066588856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1910105910 0x7f19101a5960 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f190800eaa0 tx=0x7f190800ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:54:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.806+0000 7f1906a00640 1 -- 172.21.15.111:0/3066588856 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190800cdb0 con 0x7f1910105910 2024-09-15T22:54:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.806+0000 7f1915200640 1 -- 172.21.15.111:0/3066588856 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19101a62c0 con 0x7f1910105910 2024-09-15T22:54:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.806+0000 7f1906a00640 1 -- 172.21.15.111:0/3066588856 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1908005020 con 0x7f1910105910 2024-09-15T22:54:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.806+0000 7f1906a00640 1 -- 172.21.15.111:0/3066588856 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19080116b0 con 0x7f1910105910 2024-09-15T22:54:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.806+0000 7f1915200640 1 -- 172.21.15.111:0/3066588856 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19101a6790 con 0x7f1910105910 2024-09-15T22:54:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.809+0000 7f191823c740 1 Processor -- start 2024-09-15T22:54:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.809+0000 7f1906a00640 1 -- 172.21.15.111:0/3066588856 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f1908005560 con 0x7f1910105910 2024-09-15T22:54:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.809+0000 7f191823c740 1 -- start start 2024-09-15T22:54:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.809+0000 7f191823c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564830df4090 0x564830df4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.810+0000 7f191823c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564830df49d0 0x564830df9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.810+0000 7f191823c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564830df9a60 0x564830dfbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.810+0000 7f191823c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564830d59100 con 0x564830df4090 2024-09-15T22:54:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.810+0000 7f191823c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564830d011b0 con 0x564830df49d0 2024-09-15T22:54:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.810+0000 7f191823c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564830d550c0 con 0x564830df9a60 2024-09-15T22:54:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.810+0000 7f190de00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564830df49d0 0x564830df9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.810+0000 7f190d400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564830df4090 0x564830df4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.810+0000 7f190de00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564830df49d0 0x564830df9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:39002/0 (socket says 172.21.15.111:39002) 2024-09-15T22:54:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.810+0000 7f190de00640 1 -- 172.21.15.111:0/289894721 learned_addr learned my addr 172.21.15.111:0/289894721 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:54:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.810+0000 7f190d400640 1 --2- 172.21.15.111:0/289894721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564830df9a60 0x564830dfbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.810+0000 7f190de00640 1 -- 172.21.15.111:0/289894721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564830df9a60 msgr2=0x564830dfbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.810+0000 7f190de00640 1 --2- 172.21.15.111:0/289894721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564830df9a60 0x564830dfbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.810+0000 7f190de00640 1 -- 172.21.15.111:0/289894721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564830df4090 msgr2=0x564830df4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.810+0000 7f190de00640 1 --2- 172.21.15.111:0/289894721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564830df4090 0x564830df4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.810+0000 7f190de00640 1 -- 172.21.15.111:0/289894721 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564830a6c210 con 0x564830df49d0 2024-09-15T22:54:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.810+0000 7f190de00640 1 --2- 172.21.15.111:0/289894721 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564830df49d0 0x564830df9490 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1900009ca0 tx=0x7f1900011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a360e285cdf7fce server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:54:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.811+0000 7f18fb400640 1 -- 172.21.15.111:0/289894721 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1900004220 con 0x564830df49d0 2024-09-15T22:54:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.811+0000 7f18fb400640 1 -- 172.21.15.111:0/289894721 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19000043c0 con 0x564830df49d0 2024-09-15T22:54:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.811+0000 7f18fb400640 1 -- 172.21.15.111:0/289894721 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1900019890 con 0x564830df49d0 2024-09-15T22:54:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.811+0000 7f1906a00640 1 --2- 172.21.15.111:0/3066588856 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f18f40a32f0 0x7f18f40a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.811+0000 7f1906a00640 1 -- 172.21.15.111:0/3066588856 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f19080cfae0 con 0x7f1910105910 2024-09-15T22:54:35.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.811+0000 7f190d400640 1 --2- 172.21.15.111:0/3066588856 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f18f40a32f0 0x7f18f40a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.811+0000 7f191823c740 1 -- 172.21.15.111:0/289894721 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564830df49d0 msgr2=0x564830df9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.811+0000 7f191823c740 1 --2- 172.21.15.111:0/289894721 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564830df49d0 0x564830df9490 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1900009ca0 tx=0x7f1900011810 comp rx=0 tx=0).stop 2024-09-15T22:54:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.811+0000 7f191823c740 1 -- 172.21.15.111:0/289894721 shutdown_connections 2024-09-15T22:54:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.811+0000 7f191823c740 1 --2- 172.21.15.111:0/289894721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564830df9a60 0x564830dfbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.811+0000 7f191823c740 1 --2- 172.21.15.111:0/289894721 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564830df49d0 0x564830df9490 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.811+0000 7f191823c740 1 --2- 172.21.15.111:0/289894721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564830df4090 0x564830df4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.811+0000 7f191823c740 1 -- 172.21.15.111:0/289894721 >> 172.21.15.111:0/289894721 conn(0x564830def700 msgr2=0x564830defb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:54:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.811+0000 7f191823c740 1 -- 172.21.15.111:0/289894721 shutdown_connections 2024-09-15T22:54:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.812+0000 7f191823c740 1 -- 172.21.15.111:0/289894721 wait complete. 2024-09-15T22:54:35.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.812+0000 7f190d400640 1 --2- 172.21.15.111:0/3066588856 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f18f40a32f0 0x7f18f40a57b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f18fc031440 tx=0x7f18fc041040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:54:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.812+0000 7f191823c740 1 -- start start 2024-09-15T22:54:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.812+0000 7f191823c740 1 Processor -- start 2024-09-15T22:54:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.813+0000 7f191823c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564830df4090 0x564830dfe210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.813+0000 7f191823c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564830df49d0 0x564830dfe750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.813+0000 7f191823c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564830df9a60 0x564830e05720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.813+0000 7f191823c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564830dfca30 con 0x564830df4090 2024-09-15T22:54:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.813+0000 7f191823c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564830ce2090 con 0x564830df9a60 2024-09-15T22:54:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.813+0000 7f191823c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564830dfcbb0 con 0x564830df49d0 2024-09-15T22:54:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.813+0000 7f191823c740 10 client.0 _ll_register_callbacks cb 0x564830d36f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:54:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.813+0000 7f190de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564830df4090 0x564830dfe210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.813+0000 7f190e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564830df9a60 0x564830e05720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.813+0000 7f190e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564830df9a60 0x564830e05720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:39012/0 (socket says 172.21.15.111:39012) 2024-09-15T22:54:35.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.813+0000 7f190e800640 1 -- 172.21.15.111:0/2761272397 learned_addr learned my addr 172.21.15.111:0/2761272397 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:54:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.813+0000 7f190d400640 1 --2- 172.21.15.111:0/2761272397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564830df49d0 0x564830dfe750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.813+0000 7f190e800640 1 -- 172.21.15.111:0/2761272397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564830df49d0 msgr2=0x564830dfe750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.813+0000 7f190e800640 1 --2- 172.21.15.111:0/2761272397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564830df49d0 0x564830dfe750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.813+0000 7f190e800640 1 -- 172.21.15.111:0/2761272397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564830df4090 msgr2=0x564830dfe210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.813+0000 7f190e800640 1 --2- 172.21.15.111:0/2761272397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564830df4090 0x564830dfe210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.813+0000 7f190e800640 1 -- 172.21.15.111:0/2761272397 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564830b38620 con 0x564830df9a60 2024-09-15T22:54:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.813+0000 7f190e800640 1 --2- 172.21.15.111:0/2761272397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564830df9a60 0x564830e05720 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1908098610 tx=0x7f19080a0570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=780ecf424858b41e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:54:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.814+0000 7f18efe00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:54:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.814+0000 7f18efe00640 1 -- 172.21.15.111:0/2761272397 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190801f030 con 0x564830df9a60 2024-09-15T22:54:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.814+0000 7f191823c740 1 -- 172.21.15.111:0/2761272397 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19000043c0 con 0x564830df9a60 2024-09-15T22:54:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.814+0000 7f18efe00640 1 -- 172.21.15.111:0/2761272397 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f19080a0120 con 0x564830df9a60 2024-09-15T22:54:35.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.814+0000 7f18efe00640 1 -- 172.21.15.111:0/2761272397 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f190809a570 con 0x564830df9a60 2024-09-15T22:54:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.814+0000 7f191823c740 10 client.25673 fetch_fsmap learned FSMap version 30 2024-09-15T22:54:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.814+0000 7f191823c740 1 -- 172.21.15.111:0/2761272397 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564830e08630 con 0x564830df9a60 2024-09-15T22:54:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.814+0000 7f18efe00640 1 -- 172.21.15.111:0/2761272397 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19000043c0 con 0x564830df9a60 2024-09-15T22:54:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.814+0000 7f18efe00640 1 -- 172.21.15.111:0/2761272397 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f190800f040 con 0x564830df9a60 2024-09-15T22:54:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.814+0000 7f191823c740 10 client.25673 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:54:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.814+0000 7f191823c740 10 client.25673 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:54:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.815+0000 7f191823c740 20 client.25673 populate_metadata read hostname 'smithi111' 2024-09-15T22:54:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.815+0000 7f191823c740 1 --2- 172.21.15.111:0/2761272397 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564830e09cd0 0x564830e0c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.815+0000 7f191823c740 4 client.25673 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:54:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.815+0000 7f191823c740 1 -- 172.21.15.111:0/2761272397 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564830d66830 con 0x564830e09cd0 2024-09-15T22:54:35.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.815+0000 7f190de00640 1 --2- 172.21.15.111:0/2761272397 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564830e09cd0 0x564830e0c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.815+0000 7f190de00640 1 --2- 172.21.15.111:0/2761272397 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564830e09cd0 0x564830e0c0c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa817d2e8c7fb707 server_cookie=1ba6d985df8ad0c0 in_seq=0 out_seq=0 2024-09-15T22:54:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.815+0000 7f18efe00640 10 client.25673 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:54:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.816+0000 7f18efe00640 1 -- 172.21.15.111:0/2761272397 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564830d66830 con 0x564830e09cd0 2024-09-15T22:54:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.816+0000 7f18efe00640 10 client.25673 handle_command_reply: tid=0 2024-09-15T22:54:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.816+0000 7f191823c740 1 client.25673 shutdown 2024-09-15T22:54:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.816+0000 7f191823c740 1 -- 172.21.15.111:0/2761272397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564830df9a60 msgr2=0x564830e05720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.816+0000 7f191823c740 1 --2- 172.21.15.111:0/2761272397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564830df9a60 0x564830e05720 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1908098610 tx=0x7f19080a0570 comp rx=0 tx=0).stop 2024-09-15T22:54:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.817+0000 7f191823c740 1 -- 172.21.15.111:0/2761272397 shutdown_connections 2024-09-15T22:54:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.817+0000 7f191823c740 1 --2- 172.21.15.111:0/2761272397 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564830e09cd0 0x564830e0c0c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.817+0000 7f191823c740 1 --2- 172.21.15.111:0/2761272397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564830df9a60 0x564830e05720 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.817+0000 7f191823c740 1 --2- 172.21.15.111:0/2761272397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564830df49d0 0x564830dfe750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.817+0000 7f18efe00640 0 client.25673 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:54:35.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.817+0000 7f191823c740 1 --2- 172.21.15.111:0/2761272397 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564830df4090 0x564830dfe210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.817+0000 7f191823c740 1 -- 172.21.15.111:0/2761272397 >> 172.21.15.111:0/2761272397 conn(0x564830def700 msgr2=0x564830df0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:54:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.817+0000 7f191823c740 1 -- 172.21.15.111:0/2761272397 shutdown_connections 2024-09-15T22:54:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.817+0000 7f191823c740 1 -- 172.21.15.111:0/2761272397 wait complete. 2024-09-15T22:54:35.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.817+0000 7f191823c740 20 client.25673 trim_cache size 0 max 16384 2024-09-15T22:54:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.831+0000 7f191823c740 1 Processor -- start 2024-09-15T22:54:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.832+0000 7f191823c740 1 -- start start 2024-09-15T22:54:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.832+0000 7f191823c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564830df4090 0x564830dfe210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.832+0000 7f191823c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564830df49d0 0x564830e056e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.832+0000 7f191823c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564830df9a60 0x564830dfe750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.832+0000 7f191823c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f19101b1cf0 con 0x564830df4090 2024-09-15T22:54:35.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.832+0000 7f191823c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f19101af5c0 con 0x564830df49d0 2024-09-15T22:54:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.832+0000 7f191823c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f19101ad410 con 0x564830df9a60 2024-09-15T22:54:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.832+0000 7f190d400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564830df49d0 0x564830e056e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.832+0000 7f190e800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564830df9a60 0x564830dfe750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.832+0000 7f190de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564830df4090 0x564830dfe210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.832+0000 7f190d400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564830df49d0 0x564830e056e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:39020/0 (socket says 172.21.15.111:39020) 2024-09-15T22:54:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.832+0000 7f190e800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564830df9a60 0x564830dfe750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:34440/0 (socket says 172.21.15.111:34440) 2024-09-15T22:54:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.832+0000 7f190d400640 1 -- 172.21.15.111:0/2789499093 learned_addr learned my addr 172.21.15.111:0/2789499093 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:54:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.832+0000 7f190e800640 1 -- 172.21.15.111:0/2789499093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564830df49d0 msgr2=0x564830e056e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.832+0000 7f190e800640 1 --2- 172.21.15.111:0/2789499093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564830df49d0 0x564830e056e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.832+0000 7f190e800640 1 -- 172.21.15.111:0/2789499093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564830df4090 msgr2=0x564830dfe210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:35.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.832+0000 7f190e800640 1 --2- 172.21.15.111:0/2789499093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564830df4090 0x564830dfe210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.832+0000 7f190e800640 1 -- 172.21.15.111:0/2789499093 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564830def520 con 0x564830df9a60 2024-09-15T22:54:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.832+0000 7f190d400640 1 --2- 172.21.15.111:0/2789499093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564830df49d0 0x564830e056e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:54:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.833+0000 7f190de00640 1 --2- 172.21.15.111:0/2789499093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564830df4090 0x564830dfe210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:54:35.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.833+0000 7f190e800640 1 --2- 172.21.15.111:0/2789499093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564830df9a60 0x564830dfe750 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f19080a4840 tx=0x7f1908009710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69fa5b22bf02b46c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:54:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.833+0000 7f18faa00640 1 -- 172.21.15.111:0/2789499093 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190801f030 con 0x564830df9a60 2024-09-15T22:54:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.833+0000 7f18faa00640 1 -- 172.21.15.111:0/2789499093 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19080a4d70 con 0x564830df9a60 2024-09-15T22:54:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.833+0000 7f18faa00640 1 -- 172.21.15.111:0/2789499093 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19080a02b0 con 0x564830df9a60 2024-09-15T22:54:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.833+0000 7f191823c740 1 -- 172.21.15.111:0/2789499093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564830df9a60 msgr2=0x564830dfe750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:35.839 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.833+0000 7f191823c740 1 --2- 172.21.15.111:0/2789499093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564830df9a60 0x564830dfe750 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f19080a4840 tx=0x7f1908009710 comp rx=0 tx=0).stop 2024-09-15T22:54:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.834+0000 7f191823c740 1 -- 172.21.15.111:0/2789499093 shutdown_connections 2024-09-15T22:54:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.834+0000 7f191823c740 1 --2- 172.21.15.111:0/2789499093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564830df9a60 0x564830dfe750 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.834+0000 7f191823c740 1 --2- 172.21.15.111:0/2789499093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564830df49d0 0x564830e056e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.834+0000 7f191823c740 1 --2- 172.21.15.111:0/2789499093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564830df4090 0x564830dfe210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.834+0000 7f191823c740 1 -- 172.21.15.111:0/2789499093 >> 172.21.15.111:0/2789499093 conn(0x564830def700 msgr2=0x564830df5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:54:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.834+0000 7f191823c740 1 -- 172.21.15.111:0/2789499093 shutdown_connections 2024-09-15T22:54:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.834+0000 7f191823c740 1 -- 172.21.15.111:0/2789499093 wait complete. 2024-09-15T22:54:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.834+0000 7f191823c740 1 -- start start 2024-09-15T22:54:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.834+0000 7f191823c740 1 Processor -- start 2024-09-15T22:54:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.835+0000 7f191823c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564830df4090 0x564830e029f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:35.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.835+0000 7f191823c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564830df49d0 0x564830e02f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.835+0000 7f191823c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564830df9a60 0x564830e03f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.835+0000 7f191823c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f19101ada80 con 0x564830df4090 2024-09-15T22:54:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.835+0000 7f191823c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f19101aeef0 con 0x564830df9a60 2024-09-15T22:54:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.835+0000 7f191823c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f19101ae150 con 0x564830df49d0 2024-09-15T22:54:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.835+0000 7f191823c740 10 client.0 _ll_register_callbacks cb 0x7f1908098510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:54:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.835+0000 7f190de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564830df4090 0x564830e029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.835+0000 7f190d400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564830df49d0 0x564830e02f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.835+0000 7f190d400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564830df49d0 0x564830e02f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:34442/0 (socket says 172.21.15.111:34442) 2024-09-15T22:54:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.835+0000 7f190d400640 1 -- 172.21.15.111:0/36691725 learned_addr learned my addr 172.21.15.111:0/36691725 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:54:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.835+0000 7f190e800640 1 --2- 172.21.15.111:0/36691725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564830df9a60 0x564830e03f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.835+0000 7f190d400640 1 -- 172.21.15.111:0/36691725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564830df9a60 msgr2=0x564830e03f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.835+0000 7f190d400640 1 --2- 172.21.15.111:0/36691725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564830df9a60 0x564830e03f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.835+0000 7f190d400640 1 -- 172.21.15.111:0/36691725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564830df4090 msgr2=0x564830e029f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.835+0000 7f190d400640 1 --2- 172.21.15.111:0/36691725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564830df4090 0x564830e029f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.835+0000 7f190d400640 1 -- 172.21.15.111:0/36691725 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564830e04550 con 0x564830df49d0 2024-09-15T22:54:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.836+0000 7f190d400640 1 --2- 172.21.15.111:0/36691725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564830df49d0 0x564830e02f30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009230 tx=0x7f18fc0315b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24ad1e283079d87b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:54:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.836+0000 7f18efe00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:54:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.836+0000 7f18efe00640 1 -- 172.21.15.111:0/36691725 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc035040 con 0x564830df49d0 2024-09-15T22:54:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.836+0000 7f191823c740 1 -- 172.21.15.111:0/36691725 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19080a4d70 con 0x564830df49d0 2024-09-15T22:54:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.836+0000 7f18efe00640 1 -- 172.21.15.111:0/36691725 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f18fc032c50 con 0x564830df49d0 2024-09-15T22:54:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.836+0000 7f18efe00640 1 -- 172.21.15.111:0/36691725 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f18fc032e50 con 0x564830df49d0 2024-09-15T22:54:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.836+0000 7f191823c740 10 client.25702 fetch_fsmap learned FSMap version 30 2024-09-15T22:54:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.836+0000 7f191823c740 1 -- 172.21.15.111:0/36691725 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564830e05920 con 0x564830df49d0 2024-09-15T22:54:35.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.836+0000 7f18efe00640 1 -- 172.21.15.111:0/36691725 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19080a4d70 con 0x564830df49d0 2024-09-15T22:54:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.837+0000 7f18efe00640 1 -- 172.21.15.111:0/36691725 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f18fc053020 con 0x564830df49d0 2024-09-15T22:54:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.837+0000 7f191823c740 10 client.25702 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:54:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.837+0000 7f191823c740 10 client.25702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:54:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.837+0000 7f191823c740 20 client.25702 populate_metadata read hostname 'smithi111' 2024-09-15T22:54:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.837+0000 7f191823c740 1 --2- 172.21.15.111:0/36691725 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564830e74f70 0x564830e77390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.837+0000 7f191823c740 4 client.25702 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:54:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.837+0000 7f191823c740 1 -- 172.21.15.111:0/36691725 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564830d60b80 con 0x564830e74f70 2024-09-15T22:54:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.837+0000 7f190de00640 1 --2- 172.21.15.111:0/36691725 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564830e74f70 0x564830e77390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.837+0000 7f190de00640 1 --2- 172.21.15.111:0/36691725 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564830e74f70 0x564830e77390 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7a27a9999b6b603 server_cookie=4e9f120720362244 in_seq=0 out_seq=0 2024-09-15T22:54:35.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.837+0000 7f18efe00640 10 client.25702 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:54:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.847+0000 7f18efe00640 1 -- 172.21.15.111:0/36691725 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564830d60b80 con 0x564830e74f70 2024-09-15T22:54:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.847+0000 7f18efe00640 10 client.25702 handle_command_reply: tid=0 2024-09-15T22:54:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.847+0000 7f191823c740 1 client.25702 shutdown 2024-09-15T22:54:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.847+0000 7f191823c740 1 -- 172.21.15.111:0/36691725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564830df49d0 msgr2=0x564830e02f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.847+0000 7f191823c740 1 --2- 172.21.15.111:0/36691725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564830df49d0 0x564830e02f30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009230 tx=0x7f18fc0315b0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.847+0000 7f191823c740 1 -- 172.21.15.111:0/36691725 shutdown_connections 2024-09-15T22:54:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.847+0000 7f191823c740 1 --2- 172.21.15.111:0/36691725 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564830e74f70 0x564830e77390 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.847+0000 7f191823c740 1 --2- 172.21.15.111:0/36691725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564830df9a60 0x564830e03f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.847+0000 7f191823c740 1 --2- 172.21.15.111:0/36691725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564830df49d0 0x564830e02f30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.847+0000 7f191823c740 1 --2- 172.21.15.111:0/36691725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564830df4090 0x564830e029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.847+0000 7f18efe00640 0 client.25702 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:54:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.847+0000 7f191823c740 1 -- 172.21.15.111:0/36691725 >> 172.21.15.111:0/36691725 conn(0x564830def700 msgr2=0x564830dfe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:54:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.848+0000 7f191823c740 1 -- 172.21.15.111:0/36691725 shutdown_connections 2024-09-15T22:54:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.848+0000 7f191823c740 1 -- 172.21.15.111:0/36691725 wait complete. 2024-09-15T22:54:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.848+0000 7f191823c740 20 client.25702 trim_cache size 0 max 16384 2024-09-15T22:54:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:54:35.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:54:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:54:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:54:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.848+0000 7f1915200640 1 -- 172.21.15.111:0/3066588856 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f18f40a32f0 msgr2=0x7f18f40a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.848+0000 7f1915200640 1 --2- 172.21.15.111:0/3066588856 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f18f40a32f0 0x7f18f40a57b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f18fc031440 tx=0x7f18fc041040 comp rx=0 tx=0).stop 2024-09-15T22:54:35.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.848+0000 7f1915200640 1 -- 172.21.15.111:0/3066588856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1910105910 msgr2=0x7f19101a5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.848+0000 7f1915200640 1 --2- 172.21.15.111:0/3066588856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1910105910 0x7f19101a5960 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f190800eaa0 tx=0x7f190800ef60 comp rx=0 tx=0).stop 2024-09-15T22:54:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.848+0000 7f1915200640 1 -- 172.21.15.111:0/3066588856 shutdown_connections 2024-09-15T22:54:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.848+0000 7f1915200640 1 --2- 172.21.15.111:0/3066588856 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f18f40a32f0 0x7f18f40a57b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.848+0000 7f1915200640 1 --2- 172.21.15.111:0/3066588856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1910105910 0x7f19101a5960 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.848+0000 7f1915200640 1 --2- 172.21.15.111:0/3066588856 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1910104f50 0x7f191019e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.849+0000 7f1915200640 1 --2- 172.21.15.111:0/3066588856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1910103d60 0x7f191019e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.849+0000 7f1915200640 1 -- 172.21.15.111:0/3066588856 >> 172.21.15.111:0/3066588856 conn(0x7f19100ff4e0 msgr2=0x7f1910078640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:54:35.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.849+0000 7f1915200640 1 -- 172.21.15.111:0/3066588856 shutdown_connections 2024-09-15T22:54:35.853 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:35.849+0000 7f1915200640 1 -- 172.21.15.111:0/3066588856 wait complete. 2024-09-15T22:54:36.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:35 smithi192 ceph-mon[35427]: pgmap v610: 97 pgs: 97 active+clean; 283 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 191 op/s 2024-09-15T22:54:36.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:35 smithi121 ceph-mon[34815]: pgmap v610: 97 pgs: 97 active+clean; 283 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 191 op/s 2024-09-15T22:54:36.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:35 smithi111 ceph-mon[30629]: pgmap v610: 97 pgs: 97 active+clean; 283 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 191 op/s 2024-09-15T22:54:36.289 INFO:tasks.workunit.client.0.smithi111.stdout: 1 792840 9.89 MB/sec execute 304 sec latency 11.334 ms 2024-09-15T22:54:37.289 INFO:tasks.workunit.client.0.smithi111.stdout: 1 795857 9.92 MB/sec execute 305 sec latency 14.657 ms 2024-09-15T22:54:37.830 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:37 smithi111 ceph-mon[30629]: pgmap v611: 97 pgs: 97 active+clean; 285 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 185 op/s 2024-09-15T22:54:38.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:37 smithi192 ceph-mon[35427]: pgmap v611: 97 pgs: 97 active+clean; 285 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 185 op/s 2024-09-15T22:54:38.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:37 smithi121 ceph-mon[34815]: pgmap v611: 97 pgs: 97 active+clean; 285 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 185 op/s 2024-09-15T22:54:38.289 INFO:tasks.workunit.client.0.smithi111.stdout: 1 795863 9.88 MB/sec execute 306 sec latency 770.423 ms 2024-09-15T22:54:39.289 INFO:tasks.workunit.client.0.smithi111.stdout: 1 797650 9.87 MB/sec execute 307 sec latency 1212.238 ms 2024-09-15T22:54:40.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:39 smithi192 ceph-mon[35427]: pgmap v612: 97 pgs: 97 active+clean; 286 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 235 op/s 2024-09-15T22:54:40.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:39 smithi121 ceph-mon[34815]: pgmap v612: 97 pgs: 97 active+clean; 286 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 235 op/s 2024-09-15T22:54:40.150 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:39 smithi111 ceph-mon[30629]: pgmap v612: 97 pgs: 97 active+clean; 286 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 235 op/s 2024-09-15T22:54:40.289 INFO:tasks.workunit.client.0.smithi111.stdout: 1 799843 9.88 MB/sec execute 308 sec latency 18.167 ms 2024-09-15T22:54:41.289 INFO:tasks.workunit.client.0.smithi111.stdout: 1 802804 9.91 MB/sec execute 309 sec latency 9.907 ms 2024-09-15T22:54:41.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:41 smithi111 ceph-mon[30629]: pgmap v613: 97 pgs: 97 active+clean; 287 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 198 op/s 2024-09-15T22:54:42.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:41 smithi192 ceph-mon[35427]: pgmap v613: 97 pgs: 97 active+clean; 287 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 198 op/s 2024-09-15T22:54:42.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:41 smithi121 ceph-mon[34815]: pgmap v613: 97 pgs: 97 active+clean; 287 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 198 op/s 2024-09-15T22:54:42.289 INFO:tasks.workunit.client.0.smithi111.stdout: 1 805617 9.92 MB/sec execute 310 sec latency 13.220 ms 2024-09-15T22:54:43.289 INFO:tasks.workunit.client.0.smithi111.stdout: 1 805982 9.90 MB/sec execute 311 sec latency 458.487 ms 2024-09-15T22:54:43.812 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:43 smithi111 ceph-mon[30629]: pgmap v614: 97 pgs: 97 active+clean; 281 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 211 op/s 2024-09-15T22:54:44.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:43 smithi192 ceph-mon[35427]: pgmap v614: 97 pgs: 97 active+clean; 281 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 211 op/s 2024-09-15T22:54:44.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:43 smithi121 ceph-mon[34815]: pgmap v614: 97 pgs: 97 active+clean; 281 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 211 op/s 2024-09-15T22:54:44.290 INFO:tasks.workunit.client.0.smithi111.stdout: 1 806356 9.88 MB/sec execute 312 sec latency 1147.741 ms 2024-09-15T22:54:44.916 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:44 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:54:45.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:44 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:54:45.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:44 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:54:45.290 INFO:tasks.workunit.client.0.smithi111.stdout: 1 808933 9.89 MB/sec execute 313 sec latency 18.286 ms 2024-09-15T22:54:46.020 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:45 smithi111 ceph-mon[30629]: pgmap v615: 97 pgs: 97 active+clean; 266 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 186 op/s 2024-09-15T22:54:46.020 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:45 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:54:46.020 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:45 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:54:46.021 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:45 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:54:46.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:45 smithi192 ceph-mon[35427]: pgmap v615: 97 pgs: 97 active+clean; 266 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 186 op/s 2024-09-15T22:54:46.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:45 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:54:46.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:45 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:54:46.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:45 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:54:46.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:45 smithi121 ceph-mon[34815]: pgmap v615: 97 pgs: 97 active+clean; 266 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 186 op/s 2024-09-15T22:54:46.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:45 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:54:46.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:45 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:54:46.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:45 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:54:46.289 INFO:tasks.workunit.client.0.smithi111.stdout: 1 810995 9.89 MB/sec execute 314 sec latency 15.763 ms 2024-09-15T22:54:47.290 INFO:tasks.workunit.client.0.smithi111.stdout: 1 813711 9.91 MB/sec execute 315 sec latency 11.125 ms 2024-09-15T22:54:47.875 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:47 smithi111 ceph-mon[30629]: pgmap v616: 97 pgs: 97 active+clean; 269 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 224 op/s 2024-09-15T22:54:48.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:47 smithi192 ceph-mon[35427]: pgmap v616: 97 pgs: 97 active+clean; 269 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 224 op/s 2024-09-15T22:54:48.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:47 smithi121 ceph-mon[34815]: pgmap v616: 97 pgs: 97 active+clean; 269 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 224 op/s 2024-09-15T22:54:48.290 INFO:tasks.workunit.client.0.smithi111.stdout: 1 814233 9.88 MB/sec execute 316 sec latency 601.113 ms 2024-09-15T22:54:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.967+0000 7fd4f8800640 1 -- 172.21.15.111:0/693155740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4f0102d50 msgr2=0x7fd4f01031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.967+0000 7fd4f8800640 1 --2- 172.21.15.111:0/693155740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4f0102d50 0x7fd4f01031d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009900 tx=0x7fd4e002a3f0 comp rx=0 tx=0).stop 2024-09-15T22:54:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.967+0000 7fd4f8800640 1 -- 172.21.15.111:0/693155740 shutdown_connections 2024-09-15T22:54:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.967+0000 7fd4f8800640 1 --2- 172.21.15.111:0/693155740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4f0103710 0x7fd4f0109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.967+0000 7fd4f8800640 1 --2- 172.21.15.111:0/693155740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4f0102d50 0x7fd4f01031d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.967+0000 7fd4f8800640 1 --2- 172.21.15.111:0/693155740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f0101b60 0x7fd4f0101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.967+0000 7fd4f8800640 1 -- 172.21.15.111:0/693155740 >> 172.21.15.111:0/693155740 conn(0x7fd4f00fd2e0 msgr2=0x7fd4f00ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:54:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.967+0000 7fd4f8800640 1 -- 172.21.15.111:0/693155740 shutdown_connections 2024-09-15T22:54:48.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.967+0000 7fd4f8800640 1 -- 172.21.15.111:0/693155740 wait complete. 2024-09-15T22:54:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.968+0000 7fd4f8800640 1 Processor -- start 2024-09-15T22:54:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.968+0000 7fd4f8800640 1 -- start start 2024-09-15T22:54:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.968+0000 7fd4f8800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4f0101b60 0x7fd4f0197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.968+0000 7fd4f8800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4f0102d50 0x7fd4f0198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.968+0000 7fd4f8800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f0103710 0x7fd4f019f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:48.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.968+0000 7fd4f8800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f010d830 con 0x7fd4f0101b60 2024-09-15T22:54:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.968+0000 7fd4f8800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f010d6b0 con 0x7fd4f0103710 2024-09-15T22:54:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.968+0000 7fd4f8800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f010d9b0 con 0x7fd4f0102d50 2024-09-15T22:54:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.969+0000 7fd4f5e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4f0101b60 0x7fd4f0197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.969+0000 7fd4f6800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f0103710 0x7fd4f019f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.969+0000 7fd4f5400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4f0102d50 0x7fd4f0198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.969+0000 7fd4f5e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4f0101b60 0x7fd4f0197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46564/0 (socket says 172.21.15.111:46564) 2024-09-15T22:54:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.969+0000 7fd4f5e00640 1 -- 172.21.15.111:0/1072849376 learned_addr learned my addr 172.21.15.111:0/1072849376 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:54:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.969+0000 7fd4f6800640 1 -- 172.21.15.111:0/1072849376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4f0102d50 msgr2=0x7fd4f0198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.969+0000 7fd4f6800640 1 --2- 172.21.15.111:0/1072849376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4f0102d50 0x7fd4f0198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.969+0000 7fd4f6800640 1 -- 172.21.15.111:0/1072849376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4f0101b60 msgr2=0x7fd4f0197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.969+0000 7fd4f6800640 1 --2- 172.21.15.111:0/1072849376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4f0101b60 0x7fd4f0197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:48.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.969+0000 7fd4f6800640 1 -- 172.21.15.111:0/1072849376 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f019f9e0 con 0x7fd4f0103710 2024-09-15T22:54:48.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.969+0000 7fd4f6800640 1 --2- 172.21.15.111:0/1072849376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f0103710 0x7fd4f019f370 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec009fd0 tx=0x7fd4ec00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:54:48.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.969+0000 7fd4eaa00640 1 -- 172.21.15.111:0/1072849376 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec00ce20 con 0x7fd4f0103710 2024-09-15T22:54:48.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.970+0000 7fd4eaa00640 1 -- 172.21.15.111:0/1072849376 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4ec005020 con 0x7fd4f0103710 2024-09-15T22:54:48.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.970+0000 7fd4f8800640 1 -- 172.21.15.111:0/1072849376 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f019fcd0 con 0x7fd4f0103710 2024-09-15T22:54:48.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.970+0000 7fd4eaa00640 1 -- 172.21.15.111:0/1072849376 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec010720 con 0x7fd4f0103710 2024-09-15T22:54:48.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.970+0000 7fd4f8800640 1 -- 172.21.15.111:0/1072849376 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f01a01e0 con 0x7fd4f0103710 2024-09-15T22:54:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.972+0000 7fd4fb6db740 1 Processor -- start 2024-09-15T22:54:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.973+0000 7fd4eaa00640 1 -- 172.21.15.111:0/1072849376 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd4ec005540 con 0x7fd4f0103710 2024-09-15T22:54:48.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.973+0000 7fd4fb6db740 1 -- start start 2024-09-15T22:54:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.973+0000 7fd4fb6db740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561406063090 0x561406063490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.973+0000 7fd4fb6db740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614060639d0 0x561406068490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.973+0000 7fd4fb6db740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561406068a60 0x56140606af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.973+0000 7fd4fb6db740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561405fc8100 con 0x5614060639d0 2024-09-15T22:54:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.973+0000 7fd4fb6db740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561405f701b0 con 0x561406068a60 2024-09-15T22:54:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.973+0000 7fd4fb6db740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561405fc40c0 con 0x561406063090 2024-09-15T22:54:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.973+0000 7fd4f5e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614060639d0 0x561406068490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.973+0000 7fd4f5e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614060639d0 0x561406068490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46578/0 (socket says 172.21.15.111:46578) 2024-09-15T22:54:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.973+0000 7fd4f5e00640 1 -- 172.21.15.111:0/1599284164 learned_addr learned my addr 172.21.15.111:0/1599284164 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:54:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.973+0000 7fd4f5400640 1 --2- 172.21.15.111:0/1599284164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561406063090 0x561406063490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:48.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.974+0000 7fd4f5e00640 1 -- 172.21.15.111:0/1599284164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561406063090 msgr2=0x561406063490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.974+0000 7fd4f5e00640 1 --2- 172.21.15.111:0/1599284164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561406063090 0x561406063490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.974+0000 7fd4f5e00640 1 -- 172.21.15.111:0/1599284164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561406068a60 msgr2=0x56140606af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.974+0000 7fd4f5e00640 1 --2- 172.21.15.111:0/1599284164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561406068a60 0x56140606af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.974+0000 7fd4f5e00640 1 -- 172.21.15.111:0/1599284164 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561405cdb210 con 0x5614060639d0 2024-09-15T22:54:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.974+0000 7fd4eaa00640 1 --2- 172.21.15.111:0/1072849376 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd4d80a3370 0x7fd4d80a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.974+0000 7fd4f5e00640 1 --2- 172.21.15.111:0/1599284164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614060639d0 0x561406068490 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009d80 tx=0x7fd4e4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932496e041cae0df server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:54:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.974+0000 7fd4eaa00640 1 -- 172.21.15.111:0/1072849376 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fd4ec0cfea0 con 0x7fd4f0103710 2024-09-15T22:54:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.974+0000 7fd4f5400640 1 --2- 172.21.15.111:0/1072849376 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd4d80a3370 0x7fd4d80a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.974+0000 7fd4df000640 1 -- 172.21.15.111:0/1599284164 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e401e070 con 0x5614060639d0 2024-09-15T22:54:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.974+0000 7fd4df000640 1 -- 172.21.15.111:0/1599284164 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4e4019b10 con 0x5614060639d0 2024-09-15T22:54:48.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.974+0000 7fd4df000640 1 -- 172.21.15.111:0/1599284164 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e400eb80 con 0x5614060639d0 2024-09-15T22:54:48.981 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.975+0000 7fd4fb6db740 1 -- 172.21.15.111:0/1599284164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614060639d0 msgr2=0x561406068490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.975+0000 7fd4f5400640 1 --2- 172.21.15.111:0/1072849376 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd4d80a3370 0x7fd4d80a5830 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0031440 tx=0x7fd4e0005780 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:54:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.975+0000 7fd4fb6db740 1 --2- 172.21.15.111:0/1599284164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614060639d0 0x561406068490 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009d80 tx=0x7fd4e4011810 comp rx=0 tx=0).stop 2024-09-15T22:54:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.975+0000 7fd4fb6db740 1 -- 172.21.15.111:0/1599284164 shutdown_connections 2024-09-15T22:54:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.975+0000 7fd4fb6db740 1 --2- 172.21.15.111:0/1599284164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561406068a60 0x56140606af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.975+0000 7fd4fb6db740 1 --2- 172.21.15.111:0/1599284164 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614060639d0 0x561406068490 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.975+0000 7fd4fb6db740 1 --2- 172.21.15.111:0/1599284164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561406063090 0x561406063490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.975+0000 7fd4fb6db740 1 -- 172.21.15.111:0/1599284164 >> 172.21.15.111:0/1599284164 conn(0x56140605e700 msgr2=0x56140605eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:54:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.975+0000 7fd4fb6db740 1 -- 172.21.15.111:0/1599284164 shutdown_connections 2024-09-15T22:54:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.975+0000 7fd4fb6db740 1 -- 172.21.15.111:0/1599284164 wait complete. 2024-09-15T22:54:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.976+0000 7fd4fb6db740 1 -- start start 2024-09-15T22:54:48.982 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.976+0000 7fd4fb6db740 1 Processor -- start 2024-09-15T22:54:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.976+0000 7fd4fb6db740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561406063090 0x561406071710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.976+0000 7fd4fb6db740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614060639d0 0x561406071c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.976+0000 7fd4fb6db740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561406068a60 0x561406078c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.976+0000 7fd4fb6db740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56140606ba30 con 0x5614060639d0 2024-09-15T22:54:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.976+0000 7fd4fb6db740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561405f51090 con 0x561406068a60 2024-09-15T22:54:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.976+0000 7fd4fb6db740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56140606bbb0 con 0x561406063090 2024-09-15T22:54:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.976+0000 7fd4fb6db740 10 client.0 _ll_register_callbacks cb 0x561405fa5f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:54:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.977+0000 7fd4f5400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614060639d0 0x561406071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.977+0000 7fd4f5e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561406063090 0x561406071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.977+0000 7fd4f6800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561406068a60 0x561406078c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.977+0000 7fd4f5e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561406063090 0x561406071710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:50422/0 (socket says 172.21.15.111:50422) 2024-09-15T22:54:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.977+0000 7fd4f5e00640 1 -- 172.21.15.111:0/66294900 learned_addr learned my addr 172.21.15.111:0/66294900 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:54:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.977+0000 7fd4f6800640 1 -- 172.21.15.111:0/66294900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561406063090 msgr2=0x561406071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:48.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.977+0000 7fd4f6800640 1 --2- 172.21.15.111:0/66294900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561406063090 0x561406071710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.977+0000 7fd4f6800640 1 -- 172.21.15.111:0/66294900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614060639d0 msgr2=0x561406071c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.977+0000 7fd4f6800640 1 --2- 172.21.15.111:0/66294900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614060639d0 0x561406071c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.977+0000 7fd4f6800640 1 -- 172.21.15.111:0/66294900 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561405da7620 con 0x561406068a60 2024-09-15T22:54:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.977+0000 7fd4f5400640 1 --2- 172.21.15.111:0/66294900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614060639d0 0x561406071c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:54:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.977+0000 7fd4f6800640 1 --2- 172.21.15.111:0/66294900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561406068a60 0x561406078c90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec00c760 tx=0x7fd4ec09cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64dd6982750180e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:54:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.977+0000 7fd4d3400640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:54:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.977+0000 7fd4d3400640 1 -- 172.21.15.111:0/66294900 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec0a5130 con 0x561406068a60 2024-09-15T22:54:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.977+0000 7fd4fb6db740 1 -- 172.21.15.111:0/66294900 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4e4019b10 con 0x561406068a60 2024-09-15T22:54:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.977+0000 7fd4d3400640 1 -- 172.21.15.111:0/66294900 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fd4ec0051d0 con 0x561406068a60 2024-09-15T22:54:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.978+0000 7fd4d3400640 1 -- 172.21.15.111:0/66294900 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd4ec0a4ca0 con 0x561406068a60 2024-09-15T22:54:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.978+0000 7fd4fb6db740 10 client.25688 fetch_fsmap learned FSMap version 30 2024-09-15T22:54:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.978+0000 7fd4fb6db740 1 -- 172.21.15.111:0/66294900 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56140607bba0 con 0x561406068a60 2024-09-15T22:54:48.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.978+0000 7fd4d3400640 1 -- 172.21.15.111:0/66294900 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4e4019b10 con 0x561406068a60 2024-09-15T22:54:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.978+0000 7fd4d3400640 1 -- 172.21.15.111:0/66294900 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fd4ec09b9a0 con 0x561406068a60 2024-09-15T22:54:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.978+0000 7fd4fb6db740 10 client.25688 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:54:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.978+0000 7fd4fb6db740 10 client.25688 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:54:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.978+0000 7fd4fb6db740 20 client.25688 populate_metadata read hostname 'smithi111' 2024-09-15T22:54:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.978+0000 7fd4fb6db740 1 --2- 172.21.15.111:0/66294900 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56140607d220 0x56140607f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.978+0000 7fd4fb6db740 4 client.25688 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:54:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.978+0000 7fd4fb6db740 1 -- 172.21.15.111:0/66294900 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561405fd5830 con 0x56140607d220 2024-09-15T22:54:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.978+0000 7fd4f5e00640 1 --2- 172.21.15.111:0/66294900 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56140607d220 0x56140607f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.979+0000 7fd4f5e00640 1 --2- 172.21.15.111:0/66294900 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56140607d220 0x56140607f610 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6a3ec53ad5414d1 server_cookie=2447a2c224fefa97 in_seq=0 out_seq=0 2024-09-15T22:54:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.979+0000 7fd4d3400640 10 client.25688 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:54:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.979+0000 7fd4d3400640 1 -- 172.21.15.111:0/66294900 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561405fd5830 con 0x56140607d220 2024-09-15T22:54:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.979+0000 7fd4d3400640 10 client.25688 handle_command_reply: tid=0 2024-09-15T22:54:48.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.980+0000 7fd4fb6db740 1 client.25688 shutdown 2024-09-15T22:54:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.980+0000 7fd4fb6db740 1 -- 172.21.15.111:0/66294900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561406068a60 msgr2=0x561406078c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.980+0000 7fd4fb6db740 1 --2- 172.21.15.111:0/66294900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561406068a60 0x561406078c90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec00c760 tx=0x7fd4ec09cc80 comp rx=0 tx=0).stop 2024-09-15T22:54:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.980+0000 7fd4fb6db740 1 -- 172.21.15.111:0/66294900 shutdown_connections 2024-09-15T22:54:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.980+0000 7fd4fb6db740 1 --2- 172.21.15.111:0/66294900 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56140607d220 0x56140607f610 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.980+0000 7fd4fb6db740 1 --2- 172.21.15.111:0/66294900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561406068a60 0x561406078c90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.980+0000 7fd4d3400640 0 client.25688 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:54:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.980+0000 7fd4fb6db740 1 --2- 172.21.15.111:0/66294900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614060639d0 0x561406071c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.980+0000 7fd4fb6db740 1 --2- 172.21.15.111:0/66294900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561406063090 0x561406071710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.980+0000 7fd4fb6db740 1 -- 172.21.15.111:0/66294900 >> 172.21.15.111:0/66294900 conn(0x56140605e700 msgr2=0x56140605ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:54:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.980+0000 7fd4fb6db740 1 -- 172.21.15.111:0/66294900 shutdown_connections 2024-09-15T22:54:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.980+0000 7fd4fb6db740 1 -- 172.21.15.111:0/66294900 wait complete. 2024-09-15T22:54:48.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.980+0000 7fd4fb6db740 20 client.25688 trim_cache size 0 max 16384 2024-09-15T22:54:48.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.995+0000 7fd4fb6db740 1 Processor -- start 2024-09-15T22:54:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.995+0000 7fd4fb6db740 1 -- start start 2024-09-15T22:54:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.996+0000 7fd4fb6db740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561406063090 0x56140605eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.996+0000 7fd4fb6db740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614060639d0 0x561406078c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.996+0000 7fd4fb6db740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561406068a60 0x56140605f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.996+0000 7fd4fb6db740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01a94f0 con 0x5614060639d0 2024-09-15T22:54:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.996+0000 7fd4fb6db740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01a6dc0 con 0x561406068a60 2024-09-15T22:54:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.996+0000 7fd4fb6db740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01a4c10 con 0x561406063090 2024-09-15T22:54:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.996+0000 7fd4f5400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614060639d0 0x561406078c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.996+0000 7fd4f5400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614060639d0 0x561406078c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46598/0 (socket says 172.21.15.111:46598) 2024-09-15T22:54:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.996+0000 7fd4f5400640 1 -- 172.21.15.111:0/1542611140 learned_addr learned my addr 172.21.15.111:0/1542611140 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:54:48.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.996+0000 7fd4f6800640 1 --2- 172.21.15.111:0/1542611140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561406068a60 0x56140605f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.996+0000 7fd4f5e00640 1 --2- 172.21.15.111:0/1542611140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561406063090 0x56140605eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.996+0000 7fd4f5400640 1 -- 172.21.15.111:0/1542611140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561406063090 msgr2=0x56140605eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.996+0000 7fd4f5400640 1 --2- 172.21.15.111:0/1542611140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561406063090 0x56140605eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.996+0000 7fd4f5400640 1 -- 172.21.15.111:0/1542611140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561406068a60 msgr2=0x56140605f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.996+0000 7fd4f5400640 1 --2- 172.21.15.111:0/1542611140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561406068a60 0x56140605f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:48.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.996+0000 7fd4f5400640 1 -- 172.21.15.111:0/1542611140 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56140606bb20 con 0x5614060639d0 2024-09-15T22:54:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.997+0000 7fd4f5400640 1 --2- 172.21.15.111:0/1542611140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614060639d0 0x561406078c50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd4e00314e0 tx=0x7fd4e003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ada457954b0120 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:54:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.997+0000 7fd4de600640 1 -- 172.21.15.111:0/1542611140 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0035040 con 0x5614060639d0 2024-09-15T22:54:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.997+0000 7fd4de600640 1 -- 172.21.15.111:0/1542611140 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4e0031e40 con 0x5614060639d0 2024-09-15T22:54:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.997+0000 7fd4de600640 1 -- 172.21.15.111:0/1542611140 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e003dc40 con 0x5614060639d0 2024-09-15T22:54:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.998+0000 7fd4fb6db740 1 -- 172.21.15.111:0/1542611140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614060639d0 msgr2=0x561406078c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:49.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.998+0000 7fd4fb6db740 1 --2- 172.21.15.111:0/1542611140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614060639d0 0x561406078c50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd4e00314e0 tx=0x7fd4e003f000 comp rx=0 tx=0).stop 2024-09-15T22:54:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.998+0000 7fd4fb6db740 1 -- 172.21.15.111:0/1542611140 shutdown_connections 2024-09-15T22:54:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.998+0000 7fd4fb6db740 1 --2- 172.21.15.111:0/1542611140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561406068a60 0x56140605f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.998+0000 7fd4fb6db740 1 --2- 172.21.15.111:0/1542611140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614060639d0 0x561406078c50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.998+0000 7fd4fb6db740 1 --2- 172.21.15.111:0/1542611140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561406063090 0x56140605eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.998+0000 7fd4fb6db740 1 -- 172.21.15.111:0/1542611140 >> 172.21.15.111:0/1542611140 conn(0x56140605e700 msgr2=0x561406071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:54:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.998+0000 7fd4fb6db740 1 -- 172.21.15.111:0/1542611140 shutdown_connections 2024-09-15T22:54:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.998+0000 7fd4fb6db740 1 -- 172.21.15.111:0/1542611140 wait complete. 2024-09-15T22:54:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.998+0000 7fd4fb6db740 1 -- start start 2024-09-15T22:54:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.998+0000 7fd4fb6db740 1 Processor -- start 2024-09-15T22:54:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.999+0000 7fd4fb6db740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561406063090 0x561406075ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.999+0000 7fd4fb6db740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614060639d0 0x561406076400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:49.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.999+0000 7fd4fb6db740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561406068a60 0x561406077410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.999+0000 7fd4fb6db740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01a5280 con 0x561406068a60 2024-09-15T22:54:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.999+0000 7fd4fb6db740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01a66f0 con 0x561406063090 2024-09-15T22:54:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.999+0000 7fd4fb6db740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01a5950 con 0x5614060639d0 2024-09-15T22:54:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.999+0000 7fd4fb6db740 10 client.0 _ll_register_callbacks cb 0x7fd4ec0980a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:54:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.999+0000 7fd4f6800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561406068a60 0x561406077410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.999+0000 7fd4f6800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561406068a60 0x561406077410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46612/0 (socket says 172.21.15.111:46612) 2024-09-15T22:54:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.999+0000 7fd4f6800640 1 -- 172.21.15.111:0/369603965 learned_addr learned my addr 172.21.15.111:0/369603965 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:54:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.999+0000 7fd4f5400640 1 --2- 172.21.15.111:0/369603965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614060639d0 0x561406076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.999+0000 7fd4f6800640 1 -- 172.21.15.111:0/369603965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614060639d0 msgr2=0x561406076400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:54:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.999+0000 7fd4f6800640 1 --2- 172.21.15.111:0/369603965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614060639d0 0x561406076400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.999+0000 7fd4f6800640 1 -- 172.21.15.111:0/369603965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561406063090 msgr2=0x561406075ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:54:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.999+0000 7fd4f6800640 1 --2- 172.21.15.111:0/369603965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561406063090 0x561406075ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:48.999+0000 7fd4f6800640 1 -- 172.21.15.111:0/369603965 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561406077a20 con 0x561406068a60 2024-09-15T22:54:49.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.000+0000 7fd4f6800640 1 --2- 172.21.15.111:0/369603965 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561406068a60 0x561406077410 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec09c1c0 tx=0x7fd4ec0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4b9f8f7684e868 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:54:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.000+0000 7fd4d3400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:54:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.000+0000 7fd4d3400640 1 -- 172.21.15.111:0/369603965 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec098760 con 0x561406068a60 2024-09-15T22:54:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.000+0000 7fd4d3400640 1 -- 172.21.15.111:0/369603965 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fd4ec0b5200 con 0x561406068a60 2024-09-15T22:54:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.000+0000 7fd4fb6db740 1 -- 172.21.15.111:0/369603965 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561405fcfb80 con 0x561406068a60 2024-09-15T22:54:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.000+0000 7fd4d3400640 1 -- 172.21.15.111:0/369603965 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd4ec09bb10 con 0x561406068a60 2024-09-15T22:54:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.000+0000 7fd4fb6db740 10 client.15711 fetch_fsmap learned FSMap version 30 2024-09-15T22:54:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.000+0000 7fd4fb6db740 1 -- 172.21.15.111:0/369603965 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561406078d30 con 0x561406068a60 2024-09-15T22:54:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.000+0000 7fd4d3400640 1 -- 172.21.15.111:0/369603965 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561405fcfb80 con 0x561406068a60 2024-09-15T22:54:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.000+0000 7fd4d3400640 1 -- 172.21.15.111:0/369603965 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fd4ec0bc180 con 0x561406068a60 2024-09-15T22:54:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.000+0000 7fd4fb6db740 10 client.15711 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:54:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.000+0000 7fd4fb6db740 10 client.15711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:54:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.001+0000 7fd4fb6db740 20 client.15711 populate_metadata read hostname 'smithi111' 2024-09-15T22:54:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.001+0000 7fd4fb6db740 1 --2- 172.21.15.111:0/369603965 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5614060e8370 0x5614060ea790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:54:49.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.001+0000 7fd4fb6db740 4 client.15711 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:54:49.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.001+0000 7fd4fb6db740 1 -- 172.21.15.111:0/369603965 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561405f4c7c0 con 0x5614060e8370 2024-09-15T22:54:49.023 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.022+0000 7fd4f5e00640 1 --2- 172.21.15.111:0/369603965 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5614060e8370 0x5614060ea790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:54:49.070 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.069+0000 7fd4f5e00640 1 --2- 172.21.15.111:0/369603965 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5614060e8370 0x5614060ea790 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b56f2d68566a542 server_cookie=b6e2c4c0683e29ee in_seq=0 out_seq=0 2024-09-15T22:54:49.071 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.069+0000 7fd4d3400640 10 client.15711 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:54:49.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.081+0000 7fd4d3400640 1 -- 172.21.15.111:0/369603965 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561405f4c7c0 con 0x5614060e8370 2024-09-15T22:54:49.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.081+0000 7fd4d3400640 10 client.15711 handle_command_reply: tid=0 2024-09-15T22:54:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.081+0000 7fd4fb6db740 1 client.15711 shutdown 2024-09-15T22:54:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.081+0000 7fd4fb6db740 1 -- 172.21.15.111:0/369603965 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561406068a60 msgr2=0x561406077410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.081+0000 7fd4fb6db740 1 --2- 172.21.15.111:0/369603965 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561406068a60 0x561406077410 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec09c1c0 tx=0x7fd4ec0058b0 comp rx=0 tx=0).stop 2024-09-15T22:54:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.081+0000 7fd4fb6db740 1 -- 172.21.15.111:0/369603965 shutdown_connections 2024-09-15T22:54:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.081+0000 7fd4fb6db740 1 --2- 172.21.15.111:0/369603965 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5614060e8370 0x5614060ea790 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.081+0000 7fd4fb6db740 1 --2- 172.21.15.111:0/369603965 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561406068a60 0x561406077410 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.081+0000 7fd4fb6db740 1 --2- 172.21.15.111:0/369603965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614060639d0 0x561406076400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.081+0000 7fd4fb6db740 1 --2- 172.21.15.111:0/369603965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561406063090 0x561406075ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.081+0000 7fd4d3400640 0 client.15711 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:54:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.081+0000 7fd4fb6db740 1 -- 172.21.15.111:0/369603965 >> 172.21.15.111:0/369603965 conn(0x56140605e700 msgr2=0x56140605d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:54:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.082+0000 7fd4fb6db740 1 -- 172.21.15.111:0/369603965 shutdown_connections 2024-09-15T22:54:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.082+0000 7fd4fb6db740 1 -- 172.21.15.111:0/369603965 wait complete. 2024-09-15T22:54:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.082+0000 7fd4fb6db740 20 client.15711 trim_cache size 0 max 16384 2024-09-15T22:54:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:54:49.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:54:49.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:54:49.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:54:49.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.082+0000 7fd4f8800640 1 -- 172.21.15.111:0/1072849376 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd4d80a3370 msgr2=0x7fd4d80a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.082+0000 7fd4f8800640 1 --2- 172.21.15.111:0/1072849376 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd4d80a3370 0x7fd4d80a5830 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0031440 tx=0x7fd4e0005780 comp rx=0 tx=0).stop 2024-09-15T22:54:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.082+0000 7fd4f8800640 1 -- 172.21.15.111:0/1072849376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f0103710 msgr2=0x7fd4f019f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:54:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.082+0000 7fd4f8800640 1 --2- 172.21.15.111:0/1072849376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f0103710 0x7fd4f019f370 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec009fd0 tx=0x7fd4ec00ef10 comp rx=0 tx=0).stop 2024-09-15T22:54:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.083+0000 7fd4f8800640 1 -- 172.21.15.111:0/1072849376 shutdown_connections 2024-09-15T22:54:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.083+0000 7fd4f8800640 1 --2- 172.21.15.111:0/1072849376 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd4d80a3370 0x7fd4d80a5830 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.083+0000 7fd4f8800640 1 --2- 172.21.15.111:0/1072849376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f0103710 0x7fd4f019f370 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.083+0000 7fd4f8800640 1 --2- 172.21.15.111:0/1072849376 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4f0102d50 0x7fd4f0198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.083+0000 7fd4f8800640 1 --2- 172.21.15.111:0/1072849376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4f0101b60 0x7fd4f0197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:54:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.083+0000 7fd4f8800640 1 -- 172.21.15.111:0/1072849376 >> 172.21.15.111:0/1072849376 conn(0x7fd4f00fd2e0 msgr2=0x7fd4f0106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:54:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.083+0000 7fd4f8800640 1 -- 172.21.15.111:0/1072849376 shutdown_connections 2024-09-15T22:54:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:54:49.083+0000 7fd4f8800640 1 -- 172.21.15.111:0/1072849376 wait complete. 2024-09-15T22:54:49.290 INFO:tasks.workunit.client.0.smithi111.stdout: 1 814905 9.86 MB/sec execute 317 sec latency 974.006 ms 2024-09-15T22:54:50.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:49 smithi192 ceph-mon[35427]: pgmap v617: 97 pgs: 97 active+clean; 269 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 219 op/s 2024-09-15T22:54:50.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:49 smithi121 ceph-mon[34815]: pgmap v617: 97 pgs: 97 active+clean; 269 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 219 op/s 2024-09-15T22:54:50.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:49 smithi111 ceph-mon[30629]: pgmap v617: 97 pgs: 97 active+clean; 269 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 219 op/s 2024-09-15T22:54:50.290 INFO:tasks.workunit.client.0.smithi111.stdout: 1 817384 9.87 MB/sec execute 318 sec latency 13.173 ms 2024-09-15T22:54:51.290 INFO:tasks.workunit.client.0.smithi111.stdout: 1 820463 9.90 MB/sec execute 319 sec latency 14.850 ms 2024-09-15T22:54:51.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:51 smithi111 ceph-mon[30629]: pgmap v618: 97 pgs: 97 active+clean; 268 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 171 op/s 2024-09-15T22:54:52.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:51 smithi192 ceph-mon[35427]: pgmap v618: 97 pgs: 97 active+clean; 268 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 171 op/s 2024-09-15T22:54:52.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:51 smithi121 ceph-mon[34815]: pgmap v618: 97 pgs: 97 active+clean; 268 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 171 op/s 2024-09-15T22:54:52.290 INFO:tasks.workunit.client.0.smithi111.stdout: 1 823330 9.91 MB/sec execute 320 sec latency 13.262 ms 2024-09-15T22:54:53.290 INFO:tasks.workunit.client.0.smithi111.stdout: 1 824052 9.90 MB/sec execute 321 sec latency 605.391 ms 2024-09-15T22:54:53.787 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:53 smithi111 ceph-mon[30629]: pgmap v619: 97 pgs: 97 active+clean; 271 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 227 op/s 2024-09-15T22:54:54.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:53 smithi192 ceph-mon[35427]: pgmap v619: 97 pgs: 97 active+clean; 271 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 227 op/s 2024-09-15T22:54:54.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:53 smithi121 ceph-mon[34815]: pgmap v619: 97 pgs: 97 active+clean; 271 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 227 op/s 2024-09-15T22:54:54.290 INFO:tasks.workunit.client.0.smithi111.stdout: 1 824350 9.87 MB/sec execute 322 sec latency 1116.385 ms 2024-09-15T22:54:55.290 INFO:tasks.workunit.client.0.smithi111.stdout: 1 827482 9.90 MB/sec execute 323 sec latency 19.972 ms 2024-09-15T22:54:56.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:55 smithi192 ceph-mon[35427]: pgmap v620: 97 pgs: 97 active+clean; 271 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 209 op/s 2024-09-15T22:54:56.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:55 smithi121 ceph-mon[34815]: pgmap v620: 97 pgs: 97 active+clean; 271 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 209 op/s 2024-09-15T22:54:56.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:55 smithi111 ceph-mon[30629]: pgmap v620: 97 pgs: 97 active+clean; 271 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 209 op/s 2024-09-15T22:54:56.290 INFO:tasks.workunit.client.0.smithi111.stdout: 1 830225 9.90 MB/sec execute 324 sec latency 15.374 ms 2024-09-15T22:54:57.290 INFO:tasks.workunit.client.0.smithi111.stdout: 1 832479 9.91 MB/sec execute 325 sec latency 13.146 ms 2024-09-15T22:54:58.030 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:57 smithi111 ceph-mon[30629]: pgmap v621: 97 pgs: 97 active+clean; 272 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 195 op/s 2024-09-15T22:54:58.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:57 smithi192 ceph-mon[35427]: pgmap v621: 97 pgs: 97 active+clean; 272 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 195 op/s 2024-09-15T22:54:58.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:57 smithi121 ceph-mon[34815]: pgmap v621: 97 pgs: 97 active+clean; 272 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 195 op/s 2024-09-15T22:54:58.301 INFO:tasks.workunit.client.0.smithi111.stdout: 1 834277 9.92 MB/sec execute 326 sec latency 511.105 ms 2024-09-15T22:54:59.301 INFO:tasks.workunit.client.0.smithi111.stdout: 1 834280 9.89 MB/sec execute 327 sec latency 810.550 ms 2024-09-15T22:55:00.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:54:59 smithi192 ceph-mon[35427]: pgmap v622: 97 pgs: 97 active+clean; 274 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 201 op/s 2024-09-15T22:55:00.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:54:59 smithi121 ceph-mon[34815]: pgmap v622: 97 pgs: 97 active+clean; 274 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 201 op/s 2024-09-15T22:55:00.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:54:59 smithi111 ceph-mon[30629]: pgmap v622: 97 pgs: 97 active+clean; 274 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 201 op/s 2024-09-15T22:55:00.301 INFO:tasks.workunit.client.0.smithi111.stdout: 1 835958 9.88 MB/sec execute 328 sec latency 926.788 ms 2024-09-15T22:55:01.302 INFO:tasks.workunit.client.0.smithi111.stdout: 1 838623 9.89 MB/sec execute 329 sec latency 12.065 ms 2024-09-15T22:55:02.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:01 smithi192 ceph-mon[35427]: pgmap v623: 97 pgs: 97 active+clean; 274 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 195 op/s 2024-09-15T22:55:02.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:01 smithi121 ceph-mon[34815]: pgmap v623: 97 pgs: 97 active+clean; 274 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 195 op/s 2024-09-15T22:55:02.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:01 smithi111 ceph-mon[30629]: pgmap v623: 97 pgs: 97 active+clean; 274 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 195 op/s 2024-09-15T22:55:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.204+0000 7f8cd0e00640 1 -- 172.21.15.111:0/3458958124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ccc076bb0 msgr2=0x7f8ccc076fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.204+0000 7f8cd0e00640 1 --2- 172.21.15.111:0/3458958124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ccc076bb0 0x7f8ccc076fb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009900 tx=0x7f8cbc02a3f0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.205+0000 7f8cd0e00640 1 -- 172.21.15.111:0/3458958124 shutdown_connections 2024-09-15T22:55:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.205+0000 7f8cd0e00640 1 --2- 172.21.15.111:0/3458958124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ccc078790 0x7f8ccc079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.205+0000 7f8cd0e00640 1 --2- 172.21.15.111:0/3458958124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ccc077df0 0x7f8ccc078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.205+0000 7f8cd0e00640 1 --2- 172.21.15.111:0/3458958124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ccc076bb0 0x7f8ccc076fb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.205+0000 7f8cd0e00640 1 -- 172.21.15.111:0/3458958124 >> 172.21.15.111:0/3458958124 conn(0x7f8ccc0ff2f0 msgr2=0x7f8ccc101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.205+0000 7f8cd0e00640 1 -- 172.21.15.111:0/3458958124 shutdown_connections 2024-09-15T22:55:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.205+0000 7f8cd0e00640 1 -- 172.21.15.111:0/3458958124 wait complete. 2024-09-15T22:55:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.205+0000 7f8cd0e00640 1 Processor -- start 2024-09-15T22:55:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.206+0000 7f8cd0e00640 1 -- start start 2024-09-15T22:55:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.206+0000 7f8cd0e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ccc076bb0 0x7f8ccc19e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.206+0000 7f8cd0e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ccc077df0 0x7f8ccc19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.206+0000 7f8cd0e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ccc078790 0x7f8ccc1a5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.206+0000 7f8cd0e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc113c20 con 0x7f8ccc076bb0 2024-09-15T22:55:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.206+0000 7f8cd0e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc113aa0 con 0x7f8ccc078790 2024-09-15T22:55:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.206+0000 7f8cd0e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc113da0 con 0x7f8ccc077df0 2024-09-15T22:55:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.206+0000 7f8ccb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ccc076bb0 0x7f8ccc19e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.206+0000 7f8ccbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ccc078790 0x7f8ccc1a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.206+0000 7f8ccbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ccc078790 0x7f8ccc1a5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:43218/0 (socket says 172.21.15.111:43218) 2024-09-15T22:55:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.206+0000 7f8ccb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ccc076bb0 0x7f8ccc19e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54516/0 (socket says 172.21.15.111:54516) 2024-09-15T22:55:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.206+0000 7f8ccbe00640 1 -- 172.21.15.111:0/3697379303 learned_addr learned my addr 172.21.15.111:0/3697379303 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:55:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.206+0000 7f8ccaa00640 1 --2- 172.21.15.111:0/3697379303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ccc077df0 0x7f8ccc19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.207+0000 7f8ccaa00640 1 -- 172.21.15.111:0/3697379303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ccc078790 msgr2=0x7f8ccc1a5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.207+0000 7f8ccaa00640 1 --2- 172.21.15.111:0/3697379303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ccc078790 0x7f8ccc1a5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.207+0000 7f8ccaa00640 1 -- 172.21.15.111:0/3697379303 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ccc076bb0 msgr2=0x7f8ccc19e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.207+0000 7f8ccaa00640 1 --2- 172.21.15.111:0/3697379303 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ccc076bb0 0x7f8ccc19e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.207+0000 7f8ccaa00640 1 -- 172.21.15.111:0/3697379303 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ccc1a5e90 con 0x7f8ccc077df0 2024-09-15T22:55:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.207+0000 7f8ccb400640 1 --2- 172.21.15.111:0/3697379303 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ccc076bb0 0x7f8ccc19e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:55:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.207+0000 7f8ccaa00640 1 --2- 172.21.15.111:0/3697379303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ccc077df0 0x7f8ccc19e7b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800e9f0 tx=0x7f8cb800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.207+0000 7f8cc3e00640 1 -- 172.21.15.111:0/3697379303 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb800cd80 con 0x7f8ccc077df0 2024-09-15T22:55:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.207+0000 7f8cc3e00640 1 -- 172.21.15.111:0/3697379303 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cb80035f0 con 0x7f8ccc077df0 2024-09-15T22:55:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.207+0000 7f8cd0e00640 1 -- 172.21.15.111:0/3697379303 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ccc1a6180 con 0x7f8ccc077df0 2024-09-15T22:55:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.207+0000 7f8cc3e00640 1 -- 172.21.15.111:0/3697379303 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8010690 con 0x7f8ccc077df0 2024-09-15T22:55:02.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.207+0000 7f8cd0e00640 1 -- 172.21.15.111:0/3697379303 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ccc1a66d0 con 0x7f8ccc077df0 2024-09-15T22:55:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.212+0000 7f8cd53d9740 1 Processor -- start 2024-09-15T22:55:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.212+0000 7f8cd53d9740 1 -- start start 2024-09-15T22:55:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.212+0000 7f8cc3e00640 1 -- 172.21.15.111:0/3697379303 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f8cb8003b30 con 0x7f8ccc077df0 2024-09-15T22:55:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.212+0000 7f8cd53d9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617a4c8c260 0x5617a4c8c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.212+0000 7f8cd53d9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a4c8cba0 0x5617a4c91660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.212+0000 7f8cd53d9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617a4c91c30 0x5617a4c940f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.212+0000 7f8cd53d9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5617a4bf1100 con 0x5617a4c8cba0 2024-09-15T22:55:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.212+0000 7f8cd53d9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5617a4b991b0 con 0x5617a4c8c260 2024-09-15T22:55:02.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.212+0000 7f8cd53d9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5617a4bed0c0 con 0x5617a4c91c30 2024-09-15T22:55:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.212+0000 7f8ccbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617a4c8c260 0x5617a4c8c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.212+0000 7f8ccbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617a4c8c260 0x5617a4c8c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:43226/0 (socket says 172.21.15.111:43226) 2024-09-15T22:55:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.212+0000 7f8ccbe00640 1 -- 172.21.15.111:0/3572767275 learned_addr learned my addr 172.21.15.111:0/3572767275 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:55:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.212+0000 7f8ccaa00640 1 --2- 172.21.15.111:0/3572767275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617a4c91c30 0x5617a4c940f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:02.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.212+0000 7f8ccb400640 1 --2- 172.21.15.111:0/3572767275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a4c8cba0 0x5617a4c91660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.213+0000 7f8ccbe00640 1 -- 172.21.15.111:0/3572767275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617a4c91c30 msgr2=0x5617a4c940f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:02.219 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.213+0000 7f8ccbe00640 1 --2- 172.21.15.111:0/3572767275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617a4c91c30 0x5617a4c940f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.213+0000 7f8ccbe00640 1 -- 172.21.15.111:0/3572767275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a4c8cba0 msgr2=0x5617a4c91660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.213+0000 7f8ccbe00640 1 --2- 172.21.15.111:0/3572767275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a4c8cba0 0x5617a4c91660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.213+0000 7f8ccbe00640 1 -- 172.21.15.111:0/3572767275 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617a4904210 con 0x5617a4c8c260 2024-09-15T22:55:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.213+0000 7f8ccbe00640 1 --2- 172.21.15.111:0/3572767275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617a4c8c260 0x5617a4c8c660 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40104d0 tx=0x7f8cc4010990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a063342b20676b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.213+0000 7f8cc0c00640 1 -- 172.21.15.111:0/3572767275 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4011760 con 0x5617a4c8c260 2024-09-15T22:55:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.213+0000 7f8cc0c00640 1 -- 172.21.15.111:0/3572767275 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc4011de0 con 0x5617a4c8c260 2024-09-15T22:55:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.213+0000 7f8cc0c00640 1 -- 172.21.15.111:0/3572767275 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc40193d0 con 0x5617a4c8c260 2024-09-15T22:55:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.213+0000 7f8cc3e00640 1 --2- 172.21.15.111:0/3697379303 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8cb00a3240 0x7f8cb00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.214+0000 7f8cc3e00640 1 -- 172.21.15.111:0/3697379303 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f8cb80cfbf0 con 0x7f8ccc077df0 2024-09-15T22:55:02.220 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.214+0000 7f8ccb400640 1 --2- 172.21.15.111:0/3697379303 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8cb00a3240 0x7f8cb00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.214+0000 7f8cd53d9740 1 -- 172.21.15.111:0/3572767275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617a4c8c260 msgr2=0x5617a4c8c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.214+0000 7f8cd53d9740 1 --2- 172.21.15.111:0/3572767275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617a4c8c260 0x5617a4c8c660 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40104d0 tx=0x7f8cc4010990 comp rx=0 tx=0).stop 2024-09-15T22:55:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.214+0000 7f8cd53d9740 1 -- 172.21.15.111:0/3572767275 shutdown_connections 2024-09-15T22:55:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.214+0000 7f8cd53d9740 1 --2- 172.21.15.111:0/3572767275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617a4c91c30 0x5617a4c940f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.214+0000 7f8cd53d9740 1 --2- 172.21.15.111:0/3572767275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a4c8cba0 0x5617a4c91660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.214+0000 7f8cd53d9740 1 --2- 172.21.15.111:0/3572767275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617a4c8c260 0x5617a4c8c660 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.214+0000 7f8cd53d9740 1 -- 172.21.15.111:0/3572767275 >> 172.21.15.111:0/3572767275 conn(0x5617a4c878d0 msgr2=0x5617a4c87cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.214+0000 7f8cd53d9740 1 -- 172.21.15.111:0/3572767275 shutdown_connections 2024-09-15T22:55:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.214+0000 7f8cd53d9740 1 -- 172.21.15.111:0/3572767275 wait complete. 2024-09-15T22:55:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.214+0000 7f8ccb400640 1 --2- 172.21.15.111:0/3697379303 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8cb00a3240 0x7f8cb00a5700 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc031440 tx=0x7f8cbc0315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.215+0000 7f8cd53d9740 1 -- start start 2024-09-15T22:55:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.215+0000 7f8cd53d9740 1 Processor -- start 2024-09-15T22:55:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.215+0000 7f8cd53d9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617a4c8c260 0x5617a4c96440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:02.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.215+0000 7f8cd53d9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a4c8cba0 0x5617a4c96980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.215+0000 7f8cd53d9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617a4c91c30 0x5617a4c9d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.215+0000 7f8cd53d9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5617a4c94c00 con 0x5617a4c8cba0 2024-09-15T22:55:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.215+0000 7f8cd53d9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5617a4b7a090 con 0x5617a4c8c260 2024-09-15T22:55:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.215+0000 7f8cd53d9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5617a4c94d80 con 0x5617a4c91c30 2024-09-15T22:55:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.215+0000 7f8cd53d9740 10 client.0 _ll_register_callbacks cb 0x5617a4bcef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:55:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.215+0000 7f8ccb400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617a4c8c260 0x5617a4c96440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.215+0000 7f8ccaa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a4c8cba0 0x5617a4c96980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.215+0000 7f8ccbe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617a4c91c30 0x5617a4c9d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.215+0000 7f8ccbe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617a4c91c30 0x5617a4c9d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:43998/0 (socket says 172.21.15.111:43998) 2024-09-15T22:55:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.215+0000 7f8ccb400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617a4c8c260 0x5617a4c96440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:43242/0 (socket says 172.21.15.111:43242) 2024-09-15T22:55:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.216+0000 7f8ccbe00640 1 -- 172.21.15.111:0/3575578799 learned_addr learned my addr 172.21.15.111:0/3575578799 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:55:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.216+0000 7f8ccbe00640 1 -- 172.21.15.111:0/3575578799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617a4c8c260 msgr2=0x5617a4c96440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:02.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.216+0000 7f8ccbe00640 1 --2- 172.21.15.111:0/3575578799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617a4c8c260 0x5617a4c96440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.216+0000 7f8ccbe00640 1 -- 172.21.15.111:0/3575578799 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a4c8cba0 msgr2=0x5617a4c96980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.216+0000 7f8ccbe00640 1 --2- 172.21.15.111:0/3575578799 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a4c8cba0 0x5617a4c96980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.216+0000 7f8ccbe00640 1 -- 172.21.15.111:0/3575578799 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617a49d0620 con 0x5617a4c91c30 2024-09-15T22:55:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.216+0000 7f8ccbe00640 1 --2- 172.21.15.111:0/3575578799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617a4c91c30 0x5617a4c9d9c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40104a0 tx=0x7f8cc400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcbb95c28bdea081 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.216+0000 7f8cb5600640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:55:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.216+0000 7f8cb5600640 1 -- 172.21.15.111:0/3575578799 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc400e430 con 0x5617a4c91c30 2024-09-15T22:55:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.216+0000 7f8cd53d9740 1 -- 172.21.15.111:0/3575578799 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cc4011de0 con 0x5617a4c91c30 2024-09-15T22:55:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.217+0000 7f8cb5600640 1 -- 172.21.15.111:0/3575578799 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f8cc400e5d0 con 0x5617a4c91c30 2024-09-15T22:55:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.217+0000 7f8cb5600640 1 -- 172.21.15.111:0/3575578799 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8cc4004a40 con 0x5617a4c91c30 2024-09-15T22:55:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.217+0000 7f8cd53d9740 10 client.25732 fetch_fsmap learned FSMap version 30 2024-09-15T22:55:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.217+0000 7f8cd53d9740 1 -- 172.21.15.111:0/3575578799 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5617a4ca0800 con 0x5617a4c91c30 2024-09-15T22:55:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.217+0000 7f8cb5600640 1 -- 172.21.15.111:0/3575578799 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc4011de0 con 0x5617a4c91c30 2024-09-15T22:55:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.217+0000 7f8cb5600640 1 -- 172.21.15.111:0/3575578799 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f8cc40050e0 con 0x5617a4c91c30 2024-09-15T22:55:02.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.217+0000 7f8cd53d9740 10 client.25732 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:55:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.217+0000 7f8cd53d9740 10 client.25732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:55:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.217+0000 7f8cd53d9740 20 client.25732 populate_metadata read hostname 'smithi111' 2024-09-15T22:55:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.217+0000 7f8cd53d9740 1 --2- 172.21.15.111:0/3575578799 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5617a4ca1c10 0x5617a4ca4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.217+0000 7f8cd53d9740 4 client.25732 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:55:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.217+0000 7f8cd53d9740 1 -- 172.21.15.111:0/3575578799 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617a4bfe830 con 0x5617a4ca1c10 2024-09-15T22:55:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.218+0000 7f8ccb400640 1 --2- 172.21.15.111:0/3575578799 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5617a4ca1c10 0x5617a4ca4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.218+0000 7f8cb5600640 10 client.25732 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:55:02.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.218+0000 7f8ccb400640 1 --2- 172.21.15.111:0/3575578799 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5617a4ca1c10 0x5617a4ca4000 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bc55cd905924d0d server_cookie=c514d34aa0278dae in_seq=0 out_seq=0 2024-09-15T22:55:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.219+0000 7f8cb5600640 1 -- 172.21.15.111:0/3575578799 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5617a4bfe830 con 0x5617a4ca1c10 2024-09-15T22:55:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.219+0000 7f8cb5600640 10 client.25732 handle_command_reply: tid=0 2024-09-15T22:55:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.219+0000 7f8cd53d9740 1 client.25732 shutdown 2024-09-15T22:55:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.219+0000 7f8cd53d9740 1 -- 172.21.15.111:0/3575578799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617a4c91c30 msgr2=0x5617a4c9d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:02.225 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.219+0000 7f8cd53d9740 1 --2- 172.21.15.111:0/3575578799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617a4c91c30 0x5617a4c9d9c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40104a0 tx=0x7f8cc400ed60 comp rx=0 tx=0).stop 2024-09-15T22:55:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.219+0000 7f8cd53d9740 1 -- 172.21.15.111:0/3575578799 shutdown_connections 2024-09-15T22:55:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.219+0000 7f8cd53d9740 1 --2- 172.21.15.111:0/3575578799 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5617a4ca1c10 0x5617a4ca4000 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.219+0000 7f8cd53d9740 1 --2- 172.21.15.111:0/3575578799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617a4c91c30 0x5617a4c9d9c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.219+0000 7f8cd53d9740 1 --2- 172.21.15.111:0/3575578799 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a4c8cba0 0x5617a4c96980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.219+0000 7f8cd53d9740 1 --2- 172.21.15.111:0/3575578799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617a4c8c260 0x5617a4c96440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.219+0000 7f8cb5600640 0 client.25732 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:55:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.219+0000 7f8cd53d9740 1 -- 172.21.15.111:0/3575578799 >> 172.21.15.111:0/3575578799 conn(0x5617a4c878d0 msgr2=0x5617a4c890e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.219+0000 7f8cd53d9740 1 -- 172.21.15.111:0/3575578799 shutdown_connections 2024-09-15T22:55:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.219+0000 7f8cd53d9740 1 -- 172.21.15.111:0/3575578799 wait complete. 2024-09-15T22:55:02.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.219+0000 7f8cd53d9740 20 client.25732 trim_cache size 0 max 16384 2024-09-15T22:55:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.235+0000 7f8cd53d9740 1 Processor -- start 2024-09-15T22:55:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.235+0000 7f8cd53d9740 1 -- start start 2024-09-15T22:55:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.235+0000 7f8cd53d9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617a4c8c260 0x5617a4c86900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.235+0000 7f8cd53d9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a4c8cba0 0x5617a4c86e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.235+0000 7f8cd53d9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617a4c91c30 0x5617a4c9d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.235+0000 7f8cd53d9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1b1bc0 con 0x5617a4c8cba0 2024-09-15T22:55:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.235+0000 7f8cd53d9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1af490 con 0x5617a4c91c30 2024-09-15T22:55:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.235+0000 7f8cd53d9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1ad2e0 con 0x5617a4c8c260 2024-09-15T22:55:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.235+0000 7f8ccb400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617a4c8c260 0x5617a4c86900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:02.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.235+0000 7f8ccaa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a4c8cba0 0x5617a4c86e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:02.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.235+0000 7f8ccb400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617a4c8c260 0x5617a4c86900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:44008/0 (socket says 172.21.15.111:44008) 2024-09-15T22:55:02.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.235+0000 7f8ccb400640 1 -- 172.21.15.111:0/2020598092 learned_addr learned my addr 172.21.15.111:0/2020598092 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:55:02.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.235+0000 7f8ccbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617a4c91c30 0x5617a4c9d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.236+0000 7f8ccb400640 1 -- 172.21.15.111:0/2020598092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617a4c91c30 msgr2=0x5617a4c9d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:02.241 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.236+0000 7f8ccb400640 1 --2- 172.21.15.111:0/2020598092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617a4c91c30 0x5617a4c9d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.236+0000 7f8ccb400640 1 -- 172.21.15.111:0/2020598092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a4c8cba0 msgr2=0x5617a4c86e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.236+0000 7f8ccb400640 1 --2- 172.21.15.111:0/2020598092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a4c8cba0 0x5617a4c86e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.236+0000 7f8ccb400640 1 -- 172.21.15.111:0/2020598092 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617a4c876f0 con 0x5617a4c8c260 2024-09-15T22:55:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.236+0000 7f8ccbe00640 1 --2- 172.21.15.111:0/2020598092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617a4c91c30 0x5617a4c9d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:55:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.236+0000 7f8ccb400640 1 --2- 172.21.15.111:0/2020598092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617a4c8c260 0x5617a4c86900 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc0314e0 tx=0x7f8cbc040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b08ad50f132b059d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.236+0000 7f8cb7e00640 1 -- 172.21.15.111:0/2020598092 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc035040 con 0x5617a4c8c260 2024-09-15T22:55:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.236+0000 7f8cb7e00640 1 -- 172.21.15.111:0/2020598092 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cbc058070 con 0x5617a4c8c260 2024-09-15T22:55:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.236+0000 7f8cb7e00640 1 -- 172.21.15.111:0/2020598092 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc0039e0 con 0x5617a4c8c260 2024-09-15T22:55:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.237+0000 7f8cd53d9740 1 -- 172.21.15.111:0/2020598092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617a4c8c260 msgr2=0x5617a4c86900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.237+0000 7f8cd53d9740 1 --2- 172.21.15.111:0/2020598092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617a4c8c260 0x5617a4c86900 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc0314e0 tx=0x7f8cbc040000 comp rx=0 tx=0).stop 2024-09-15T22:55:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.237+0000 7f8cd53d9740 1 -- 172.21.15.111:0/2020598092 shutdown_connections 2024-09-15T22:55:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.237+0000 7f8cd53d9740 1 --2- 172.21.15.111:0/2020598092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617a4c91c30 0x5617a4c9d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.242 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.237+0000 7f8cd53d9740 1 --2- 172.21.15.111:0/2020598092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a4c8cba0 0x5617a4c86e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.237+0000 7f8cd53d9740 1 --2- 172.21.15.111:0/2020598092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617a4c8c260 0x5617a4c86900 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.237+0000 7f8cd53d9740 1 -- 172.21.15.111:0/2020598092 >> 172.21.15.111:0/2020598092 conn(0x5617a4c878d0 msgr2=0x5617a4c861f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.237+0000 7f8cd53d9740 1 -- 172.21.15.111:0/2020598092 shutdown_connections 2024-09-15T22:55:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.237+0000 7f8cd53d9740 1 -- 172.21.15.111:0/2020598092 wait complete. 2024-09-15T22:55:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.237+0000 7f8cd53d9740 1 -- start start 2024-09-15T22:55:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.237+0000 7f8cd53d9740 1 Processor -- start 2024-09-15T22:55:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.238+0000 7f8cd53d9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a4c8c260 0x5617a4c9b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.238+0000 7f8cd53d9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617a4c8cba0 0x5617a4c9b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.238+0000 7f8cd53d9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617a4c91c30 0x5617a4c9c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.238+0000 7f8cd53d9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1ad950 con 0x5617a4c8c260 2024-09-15T22:55:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.238+0000 7f8cd53d9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1aedc0 con 0x5617a4c8cba0 2024-09-15T22:55:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.238+0000 7f8cd53d9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc1ae020 con 0x5617a4c91c30 2024-09-15T22:55:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.238+0000 7f8cd53d9740 10 client.0 _ll_register_callbacks cb 0x7f8cbc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:55:02.243 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.238+0000 7f8ccaa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617a4c8cba0 0x5617a4c9b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.238+0000 7f8ccbe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617a4c91c30 0x5617a4c9c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.238+0000 7f8ccaa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617a4c8cba0 0x5617a4c9b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:43260/0 (socket says 172.21.15.111:43260) 2024-09-15T22:55:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.238+0000 7f8ccaa00640 1 -- 172.21.15.111:0/3076379024 learned_addr learned my addr 172.21.15.111:0/3076379024 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:55:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.238+0000 7f8ccb400640 1 --2- 172.21.15.111:0/3076379024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a4c8c260 0x5617a4c9b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.238+0000 7f8ccaa00640 1 -- 172.21.15.111:0/3076379024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617a4c91c30 msgr2=0x5617a4c9c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.238+0000 7f8ccaa00640 1 --2- 172.21.15.111:0/3076379024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617a4c91c30 0x5617a4c9c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.238+0000 7f8ccaa00640 1 -- 172.21.15.111:0/3076379024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a4c8c260 msgr2=0x5617a4c9b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.238+0000 7f8ccaa00640 1 --2- 172.21.15.111:0/3076379024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a4c8c260 0x5617a4c9b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.238+0000 7f8ccaa00640 1 -- 172.21.15.111:0/3076379024 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617a4c9cc40 con 0x5617a4c8cba0 2024-09-15T22:55:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.238+0000 7f8ccbe00640 1 --2- 172.21.15.111:0/3076379024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617a4c91c30 0x5617a4c9c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:55:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.239+0000 7f8ccaa00640 1 --2- 172.21.15.111:0/3076379024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617a4c8cba0 0x5617a4c9b620 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8cb809bf10 tx=0x7f8cb809bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=438385e8d18bfa8e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.239+0000 7f8cb5600640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:55:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.239+0000 7f8cb5600640 1 -- 172.21.15.111:0/3076379024 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb809c210 con 0x5617a4c8cba0 2024-09-15T22:55:02.244 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.239+0000 7f8cd53d9740 1 -- 172.21.15.111:0/3076379024 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cbc058070 con 0x5617a4c8cba0 2024-09-15T22:55:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.239+0000 7f8cb5600640 1 -- 172.21.15.111:0/3076379024 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f8cb80037a0 con 0x5617a4c8cba0 2024-09-15T22:55:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.239+0000 7f8cb5600640 1 -- 172.21.15.111:0/3076379024 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8cb80a05c0 con 0x5617a4c8cba0 2024-09-15T22:55:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.239+0000 7f8cd53d9740 10 client.25709 fetch_fsmap learned FSMap version 30 2024-09-15T22:55:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.239+0000 7f8cd53d9740 1 -- 172.21.15.111:0/3076379024 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5617a4d0c8c0 con 0x5617a4c8cba0 2024-09-15T22:55:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.239+0000 7f8cb5600640 1 -- 172.21.15.111:0/3076379024 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cbc058070 con 0x5617a4c8cba0 2024-09-15T22:55:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.239+0000 7f8cb5600640 1 -- 172.21.15.111:0/3076379024 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f8cb80a0150 con 0x5617a4c8cba0 2024-09-15T22:55:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.239+0000 7f8cd53d9740 10 client.25709 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:55:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.239+0000 7f8cd53d9740 10 client.25709 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:55:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.239+0000 7f8cd53d9740 20 client.25709 populate_metadata read hostname 'smithi111' 2024-09-15T22:55:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.239+0000 7f8cd53d9740 1 --2- 172.21.15.111:0/3076379024 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5617a4d0d1f0 0x5617a4d0f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.239+0000 7f8cd53d9740 4 client.25709 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:55:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.239+0000 7f8cd53d9740 1 -- 172.21.15.111:0/3076379024 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5617a4bf8b80 con 0x5617a4d0d1f0 2024-09-15T22:55:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.240+0000 7f8ccb400640 1 --2- 172.21.15.111:0/3076379024 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5617a4d0d1f0 0x5617a4d0f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:02.245 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.240+0000 7f8cb5600640 10 client.25709 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:55:02.246 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.240+0000 7f8ccb400640 1 --2- 172.21.15.111:0/3076379024 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5617a4d0d1f0 0x5617a4d0f610 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f0277b3b9f91e16 server_cookie=70d83fa34714cbc2 in_seq=0 out_seq=0 2024-09-15T22:55:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.251+0000 7f8cb5600640 1 -- 172.21.15.111:0/3076379024 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5617a4bf8b80 con 0x5617a4d0d1f0 2024-09-15T22:55:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.251+0000 7f8cb5600640 10 client.25709 handle_command_reply: tid=0 2024-09-15T22:55:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.251+0000 7f8cd53d9740 1 client.25709 shutdown 2024-09-15T22:55:02.253 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.251+0000 7f8cd53d9740 1 -- 172.21.15.111:0/3076379024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617a4c8cba0 msgr2=0x5617a4c9b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.251+0000 7f8cd53d9740 1 --2- 172.21.15.111:0/3076379024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617a4c8cba0 0x5617a4c9b620 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8cb809bf10 tx=0x7f8cb809bd50 comp rx=0 tx=0).stop 2024-09-15T22:55:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.251+0000 7f8cd53d9740 1 -- 172.21.15.111:0/3076379024 shutdown_connections 2024-09-15T22:55:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.251+0000 7f8cd53d9740 1 --2- 172.21.15.111:0/3076379024 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5617a4d0d1f0 0x5617a4d0f610 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.251+0000 7f8cd53d9740 1 --2- 172.21.15.111:0/3076379024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617a4c91c30 0x5617a4c9c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.251+0000 7f8cd53d9740 1 --2- 172.21.15.111:0/3076379024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617a4c8cba0 0x5617a4c9b620 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.251+0000 7f8cd53d9740 1 --2- 172.21.15.111:0/3076379024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617a4c8c260 0x5617a4c9b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.251+0000 7f8cb5600640 0 client.25709 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:55:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.251+0000 7f8cd53d9740 1 -- 172.21.15.111:0/3076379024 >> 172.21.15.111:0/3076379024 conn(0x5617a4c878d0 msgr2=0x5617a4c88fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.251+0000 7f8cd53d9740 1 -- 172.21.15.111:0/3076379024 shutdown_connections 2024-09-15T22:55:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.251+0000 7f8cd53d9740 1 -- 172.21.15.111:0/3076379024 wait complete. 2024-09-15T22:55:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.252+0000 7f8cd53d9740 20 client.25709 trim_cache size 0 max 16384 2024-09-15T22:55:02.254 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:55:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:55:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:55:02.255 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:55:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.252+0000 7f8cd0e00640 1 -- 172.21.15.111:0/3697379303 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8cb00a3240 msgr2=0x7f8cb00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.252+0000 7f8cd0e00640 1 --2- 172.21.15.111:0/3697379303 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8cb00a3240 0x7f8cb00a5700 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc031440 tx=0x7f8cbc0315b0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.252+0000 7f8cd0e00640 1 -- 172.21.15.111:0/3697379303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ccc077df0 msgr2=0x7f8ccc19e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.252+0000 7f8cd0e00640 1 --2- 172.21.15.111:0/3697379303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ccc077df0 0x7f8ccc19e7b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800e9f0 tx=0x7f8cb800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.252+0000 7f8cd0e00640 1 -- 172.21.15.111:0/3697379303 shutdown_connections 2024-09-15T22:55:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.252+0000 7f8cd0e00640 1 --2- 172.21.15.111:0/3697379303 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8cb00a3240 0x7f8cb00a5700 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.252+0000 7f8cd0e00640 1 --2- 172.21.15.111:0/3697379303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ccc078790 0x7f8ccc1a5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.252+0000 7f8cd0e00640 1 --2- 172.21.15.111:0/3697379303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ccc077df0 0x7f8ccc19e7b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.252+0000 7f8cd0e00640 1 --2- 172.21.15.111:0/3697379303 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ccc076bb0 0x7f8ccc19e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.252+0000 7f8cd0e00640 1 -- 172.21.15.111:0/3697379303 >> 172.21.15.111:0/3697379303 conn(0x7f8ccc0ff2f0 msgr2=0x7f8ccc100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.252+0000 7f8cd0e00640 1 -- 172.21.15.111:0/3697379303 shutdown_connections 2024-09-15T22:55:02.256 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:02.253+0000 7f8cd0e00640 1 -- 172.21.15.111:0/3697379303 wait complete. 2024-09-15T22:55:02.302 INFO:tasks.workunit.client.0.smithi111.stdout: 1 841743 9.92 MB/sec execute 330 sec latency 16.122 ms 2024-09-15T22:55:03.302 INFO:tasks.workunit.client.0.smithi111.stdout: 1 842967 9.90 MB/sec execute 331 sec latency 318.041 ms 2024-09-15T22:55:04.063 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:03 smithi111 ceph-mon[30629]: pgmap v624: 97 pgs: 97 active+clean; 275 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 216 op/s 2024-09-15T22:55:04.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:03 smithi192 ceph-mon[35427]: pgmap v624: 97 pgs: 97 active+clean; 275 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 216 op/s 2024-09-15T22:55:04.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:03 smithi121 ceph-mon[34815]: pgmap v624: 97 pgs: 97 active+clean; 275 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 216 op/s 2024-09-15T22:55:04.302 INFO:tasks.workunit.client.0.smithi111.stdout: 1 843070 9.87 MB/sec execute 332 sec latency 1277.627 ms 2024-09-15T22:55:05.302 INFO:tasks.workunit.client.0.smithi111.stdout: 1 845612 9.89 MB/sec execute 333 sec latency 8.958 ms 2024-09-15T22:55:06.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:05 smithi192 ceph-mon[35427]: pgmap v625: 97 pgs: 97 active+clean; 276 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 158 op/s 2024-09-15T22:55:06.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:05 smithi121 ceph-mon[34815]: pgmap v625: 97 pgs: 97 active+clean; 276 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 158 op/s 2024-09-15T22:55:06.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:05 smithi111 ceph-mon[30629]: pgmap v625: 97 pgs: 97 active+clean; 276 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 158 op/s 2024-09-15T22:55:06.302 INFO:tasks.workunit.client.0.smithi111.stdout: 1 848723 9.91 MB/sec execute 334 sec latency 12.303 ms 2024-09-15T22:55:07.302 INFO:tasks.workunit.client.0.smithi111.stdout: 1 851154 9.91 MB/sec execute 335 sec latency 15.798 ms 2024-09-15T22:55:08.058 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:07 smithi111 ceph-mon[30629]: pgmap v626: 97 pgs: 97 active+clean; 279 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 199 op/s 2024-09-15T22:55:08.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:07 smithi192 ceph-mon[35427]: pgmap v626: 97 pgs: 97 active+clean; 279 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 199 op/s 2024-09-15T22:55:08.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:07 smithi121 ceph-mon[34815]: pgmap v626: 97 pgs: 97 active+clean; 279 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 199 op/s 2024-09-15T22:55:08.302 INFO:tasks.workunit.client.0.smithi111.stdout: 1 852730 9.91 MB/sec execute 336 sec latency 299.110 ms 2024-09-15T22:55:09.302 INFO:tasks.workunit.client.0.smithi111.stdout: 1 852730 9.88 MB/sec execute 337 sec latency 1299.157 ms 2024-09-15T22:55:10.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:09 smithi192 ceph-mon[35427]: pgmap v627: 97 pgs: 97 active+clean; 279 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 202 op/s 2024-09-15T22:55:10.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:09 smithi121 ceph-mon[34815]: pgmap v627: 97 pgs: 97 active+clean; 279 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 202 op/s 2024-09-15T22:55:10.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:09 smithi111 ceph-mon[30629]: pgmap v627: 97 pgs: 97 active+clean; 279 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 202 op/s 2024-09-15T22:55:10.302 INFO:tasks.workunit.client.0.smithi111.stdout: 1 854026 9.87 MB/sec execute 338 sec latency 1778.170 ms 2024-09-15T22:55:11.302 INFO:tasks.workunit.client.0.smithi111.stdout: 1 856408 9.88 MB/sec execute 339 sec latency 13.181 ms 2024-09-15T22:55:12.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:11 smithi192 ceph-mon[35427]: pgmap v628: 97 pgs: 97 active+clean; 280 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 212 op/s 2024-09-15T22:55:12.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:11 smithi121 ceph-mon[34815]: pgmap v628: 97 pgs: 97 active+clean; 280 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 212 op/s 2024-09-15T22:55:12.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:11 smithi111 ceph-mon[30629]: pgmap v628: 97 pgs: 97 active+clean; 280 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 212 op/s 2024-09-15T22:55:12.302 INFO:tasks.workunit.client.0.smithi111.stdout: 1 859526 9.91 MB/sec execute 340 sec latency 11.352 ms 2024-09-15T22:55:13.302 INFO:tasks.workunit.client.0.smithi111.stdout: 1 861681 9.90 MB/sec execute 341 sec latency 191.942 ms 2024-09-15T22:55:13.927 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:13 smithi111 ceph-mon[30629]: pgmap v629: 97 pgs: 97 active+clean; 280 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 207 op/s 2024-09-15T22:55:14.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:13 smithi121 ceph-mon[34815]: pgmap v629: 97 pgs: 97 active+clean; 280 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 207 op/s 2024-09-15T22:55:14.303 INFO:tasks.workunit.client.0.smithi111.stdout: 1 861681 9.87 MB/sec execute 342 sec latency 1192.030 ms 2024-09-15T22:55:14.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:13 smithi192 ceph-mon[35427]: pgmap v629: 97 pgs: 97 active+clean; 280 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 207 op/s 2024-09-15T22:55:15.303 INFO:tasks.workunit.client.0.smithi111.stdout: 1 862942 9.88 MB/sec execute 343 sec latency 1559.502 ms 2024-09-15T22:55:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.368+0000 7f9e64e00640 1 -- 172.21.15.111:0/182241356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e60103d60 msgr2=0x7f9e60104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.368+0000 7f9e64e00640 1 --2- 172.21.15.111:0/182241356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e60103d60 0x7f9e60104160 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009900 tx=0x7f9e5002a410 comp rx=0 tx=0).stop 2024-09-15T22:55:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.368+0000 7f9e64e00640 1 -- 172.21.15.111:0/182241356 shutdown_connections 2024-09-15T22:55:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.368+0000 7f9e64e00640 1 --2- 172.21.15.111:0/182241356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e60105910 0x7f9e6010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.368+0000 7f9e64e00640 1 --2- 172.21.15.111:0/182241356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e60104f50 0x7f9e601053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.368+0000 7f9e64e00640 1 --2- 172.21.15.111:0/182241356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e60103d60 0x7f9e60104160 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.368+0000 7f9e64e00640 1 -- 172.21.15.111:0/182241356 >> 172.21.15.111:0/182241356 conn(0x7f9e600ff4e0 msgr2=0x7f9e60101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.368+0000 7f9e64e00640 1 -- 172.21.15.111:0/182241356 shutdown_connections 2024-09-15T22:55:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.368+0000 7f9e64e00640 1 -- 172.21.15.111:0/182241356 wait complete. 2024-09-15T22:55:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.369+0000 7f9e64e00640 1 Processor -- start 2024-09-15T22:55:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.369+0000 7f9e64e00640 1 -- start start 2024-09-15T22:55:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.369+0000 7f9e64e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e60103d60 0x7f9e60199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.369+0000 7f9e64e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e60104f50 0x7f9e6019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.369+0000 7f9e64e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e60105910 0x7f9e601a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.369+0000 7f9e64e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6010fa30 con 0x7f9e60105910 2024-09-15T22:55:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.369+0000 7f9e64e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6010f8b0 con 0x7f9e60103d60 2024-09-15T22:55:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.369+0000 7f9e64e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6010fbb0 con 0x7f9e60104f50 2024-09-15T22:55:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.370+0000 7f9e5de00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e60103d60 0x7f9e60199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.370+0000 7f9e5de00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e60103d60 0x7f9e60199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:50446/0 (socket says 172.21.15.111:50446) 2024-09-15T22:55:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.370+0000 7f9e5de00640 1 -- 172.21.15.111:0/1588307135 learned_addr learned my addr 172.21.15.111:0/1588307135 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:55:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.370+0000 7f9e5e800640 1 --2- 172.21.15.111:0/1588307135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e60105910 0x7f9e601a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.370+0000 7f9e5d400640 1 --2- 172.21.15.111:0/1588307135 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e60104f50 0x7f9e6019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.370+0000 7f9e5de00640 1 -- 172.21.15.111:0/1588307135 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e60104f50 msgr2=0x7f9e6019a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.370+0000 7f9e5de00640 1 --2- 172.21.15.111:0/1588307135 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e60104f50 0x7f9e6019a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.370+0000 7f9e5de00640 1 -- 172.21.15.111:0/1588307135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e60105910 msgr2=0x7f9e601a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.370+0000 7f9e5de00640 1 --2- 172.21.15.111:0/1588307135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e60105910 0x7f9e601a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.370+0000 7f9e5de00640 1 -- 172.21.15.111:0/1588307135 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e601a1bb0 con 0x7f9e60103d60 2024-09-15T22:55:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.370+0000 7f9e5de00640 1 --2- 172.21.15.111:0/1588307135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e60103d60 0x7f9e60199f90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9e50031440 tx=0x7f9e50032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.371+0000 7f9e56a00640 1 -- 172.21.15.111:0/1588307135 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5003f030 con 0x7f9e60103d60 2024-09-15T22:55:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.371+0000 7f9e56a00640 1 -- 172.21.15.111:0/1588307135 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e50003ce0 con 0x7f9e60103d60 2024-09-15T22:55:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.371+0000 7f9e56a00640 1 -- 172.21.15.111:0/1588307135 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50035070 con 0x7f9e60103d60 2024-09-15T22:55:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.371+0000 7f9e64e00640 1 -- 172.21.15.111:0/1588307135 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e601a1e40 con 0x7f9e60103d60 2024-09-15T22:55:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.371+0000 7f9e64e00640 1 -- 172.21.15.111:0/1588307135 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e601a2250 con 0x7f9e60103d60 2024-09-15T22:55:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.374+0000 7f9e67c8b740 1 Processor -- start 2024-09-15T22:55:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.375+0000 7f9e67c8b740 1 -- start start 2024-09-15T22:55:15.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.375+0000 7f9e67c8b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b5090 0x55612d3b5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:15.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.375+0000 7f9e67c8b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55612d3b59d0 0x55612d3ba490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.375+0000 7f9e67c8b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55612d3baa60 0x55612d3bcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.375+0000 7f9e56a00640 1 -- 172.21.15.111:0/1588307135 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f9e50042030 con 0x7f9e60103d60 2024-09-15T22:55:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.375+0000 7f9e67c8b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55612d31a100 con 0x55612d3b59d0 2024-09-15T22:55:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.375+0000 7f9e67c8b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55612d2c21b0 con 0x55612d3baa60 2024-09-15T22:55:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.375+0000 7f9e67c8b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55612d3160c0 con 0x55612d3b5090 2024-09-15T22:55:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.375+0000 7f9e5e800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b5090 0x55612d3b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.375+0000 7f9e5e800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b5090 0x55612d3b5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:33030/0 (socket says 172.21.15.111:33030) 2024-09-15T22:55:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.375+0000 7f9e5e800640 1 -- 172.21.15.111:0/2959939484 learned_addr learned my addr 172.21.15.111:0/2959939484 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:55:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.375+0000 7f9e5de00640 1 --2- 172.21.15.111:0/2959939484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55612d3b59d0 0x55612d3ba490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.375+0000 7f9e5d400640 1 --2- 172.21.15.111:0/2959939484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55612d3baa60 0x55612d3bcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.376+0000 7f9e5e800640 1 -- 172.21.15.111:0/2959939484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55612d3baa60 msgr2=0x55612d3bcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.376+0000 7f9e5e800640 1 --2- 172.21.15.111:0/2959939484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55612d3baa60 0x55612d3bcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.376+0000 7f9e5e800640 1 -- 172.21.15.111:0/2959939484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55612d3b59d0 msgr2=0x55612d3ba490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.376+0000 7f9e5e800640 1 --2- 172.21.15.111:0/2959939484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55612d3b59d0 0x55612d3ba490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.376+0000 7f9e5e800640 1 -- 172.21.15.111:0/2959939484 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55612d02d210 con 0x55612d3b5090 2024-09-15T22:55:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.376+0000 7f9e5de00640 1 --2- 172.21.15.111:0/2959939484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55612d3b59d0 0x55612d3ba490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:55:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.376+0000 7f9e5e800640 1 --2- 172.21.15.111:0/2959939484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b5090 0x55612d3b5490 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009fd0 tx=0x7f9e580118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5716b5cacb98cd9c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.376+0000 7f9e4b400640 1 -- 172.21.15.111:0/2959939484 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e58004d20 con 0x55612d3b5090 2024-09-15T22:55:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.376+0000 7f9e4b400640 1 -- 172.21.15.111:0/2959939484 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e58004050 con 0x55612d3b5090 2024-09-15T22:55:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.376+0000 7f9e4b400640 1 -- 172.21.15.111:0/2959939484 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e58005260 con 0x55612d3b5090 2024-09-15T22:55:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.377+0000 7f9e56a00640 1 --2- 172.21.15.111:0/1588307135 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9e440a3370 0x7f9e440a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.377+0000 7f9e67c8b740 1 -- 172.21.15.111:0/2959939484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b5090 msgr2=0x55612d3b5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.377+0000 7f9e56a00640 1 -- 172.21.15.111:0/1588307135 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f9e500efc40 con 0x7f9e60103d60 2024-09-15T22:55:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.377+0000 7f9e67c8b740 1 --2- 172.21.15.111:0/2959939484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b5090 0x55612d3b5490 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009fd0 tx=0x7f9e580118d0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.377+0000 7f9e67c8b740 1 -- 172.21.15.111:0/2959939484 shutdown_connections 2024-09-15T22:55:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.377+0000 7f9e67c8b740 1 --2- 172.21.15.111:0/2959939484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55612d3baa60 0x55612d3bcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.377+0000 7f9e67c8b740 1 --2- 172.21.15.111:0/2959939484 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55612d3b59d0 0x55612d3ba490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.377+0000 7f9e67c8b740 1 --2- 172.21.15.111:0/2959939484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b5090 0x55612d3b5490 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.377+0000 7f9e67c8b740 1 -- 172.21.15.111:0/2959939484 >> 172.21.15.111:0/2959939484 conn(0x55612d3b0700 msgr2=0x55612d3b0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.377+0000 7f9e5de00640 1 --2- 172.21.15.111:0/1588307135 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9e440a3370 0x7f9e440a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.377+0000 7f9e67c8b740 1 -- 172.21.15.111:0/2959939484 shutdown_connections 2024-09-15T22:55:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.377+0000 7f9e67c8b740 1 -- 172.21.15.111:0/2959939484 wait complete. 2024-09-15T22:55:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.378+0000 7f9e67c8b740 1 -- start start 2024-09-15T22:55:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.378+0000 7f9e5de00640 1 --2- 172.21.15.111:0/1588307135 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9e440a3370 0x7f9e440a5830 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9e50005ec0 tx=0x7f9e5002a410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.378+0000 7f9e67c8b740 1 Processor -- start 2024-09-15T22:55:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.378+0000 7f9e67c8b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b5090 0x55612d3bf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.378+0000 7f9e67c8b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55612d3b59d0 0x55612d3bf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.378+0000 7f9e67c8b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55612d3baa60 0x55612d3c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.378+0000 7f9e67c8b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55612d3bda30 con 0x55612d3b59d0 2024-09-15T22:55:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.378+0000 7f9e67c8b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55612d2a3090 con 0x55612d3baa60 2024-09-15T22:55:15.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.378+0000 7f9e67c8b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55612d3bdbb0 con 0x55612d3b5090 2024-09-15T22:55:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.378+0000 7f9e67c8b740 10 client.0 _ll_register_callbacks cb 0x55612d2f7f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:55:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.378+0000 7f9e5e800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b5090 0x55612d3bf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.379+0000 7f9e5e800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b5090 0x55612d3bf2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:33042/0 (socket says 172.21.15.111:33042) 2024-09-15T22:55:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.379+0000 7f9e5e800640 1 -- 172.21.15.111:0/2092529022 learned_addr learned my addr 172.21.15.111:0/2092529022 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:55:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.379+0000 7f9e5e800640 1 --2- 172.21.15.111:0/2092529022 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55612d3baa60 0x55612d3c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.379+0000 7f9e5d400640 1 --2- 172.21.15.111:0/2092529022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55612d3b59d0 0x55612d3bf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.379+0000 7f9e5e800640 1 -- 172.21.15.111:0/2092529022 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55612d3baa60 msgr2=0x55612d3c67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.379+0000 7f9e5e800640 1 --2- 172.21.15.111:0/2092529022 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55612d3baa60 0x55612d3c67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.379+0000 7f9e5e800640 1 -- 172.21.15.111:0/2092529022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55612d3b59d0 msgr2=0x55612d3bf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.379+0000 7f9e5e800640 1 --2- 172.21.15.111:0/2092529022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55612d3b59d0 0x55612d3bf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.379+0000 7f9e5e800640 1 -- 172.21.15.111:0/2092529022 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55612d0f9620 con 0x55612d3b5090 2024-09-15T22:55:15.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.379+0000 7f9e5e800640 1 --2- 172.21.15.111:0/2092529022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b5090 0x55612d3bf2a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9e5801b040 tx=0x7f9e58009c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f0af7c5b5f769bc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.379+0000 7f9e3fe00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:55:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.379+0000 7f9e3fe00640 1 -- 172.21.15.111:0/2092529022 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5800f4e0 con 0x55612d3b5090 2024-09-15T22:55:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.379+0000 7f9e67c8b740 1 -- 172.21.15.111:0/2092529022 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e58004050 con 0x55612d3b5090 2024-09-15T22:55:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.379+0000 7f9e3fe00640 1 -- 172.21.15.111:0/2092529022 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f9e5800f680 con 0x55612d3b5090 2024-09-15T22:55:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.380+0000 7f9e3fe00640 1 -- 172.21.15.111:0/2092529022 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9e5800f880 con 0x55612d3b5090 2024-09-15T22:55:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.380+0000 7f9e67c8b740 10 client.25750 fetch_fsmap learned FSMap version 30 2024-09-15T22:55:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.380+0000 7f9e67c8b740 1 -- 172.21.15.111:0/2092529022 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55612d3c95f0 con 0x55612d3b5090 2024-09-15T22:55:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.380+0000 7f9e3fe00640 1 -- 172.21.15.111:0/2092529022 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e58004050 con 0x55612d3b5090 2024-09-15T22:55:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.380+0000 7f9e3fe00640 1 -- 172.21.15.111:0/2092529022 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f9e58025040 con 0x55612d3b5090 2024-09-15T22:55:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.380+0000 7f9e67c8b740 10 client.25750 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:55:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.380+0000 7f9e67c8b740 10 client.25750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:55:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.380+0000 7f9e67c8b740 20 client.25750 populate_metadata read hostname 'smithi111' 2024-09-15T22:55:15.387 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.380+0000 7f9e67c8b740 1 --2- 172.21.15.111:0/2092529022 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55612d3caac0 0x55612d3ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.380+0000 7f9e67c8b740 4 client.25750 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:55:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.380+0000 7f9e67c8b740 1 -- 172.21.15.111:0/2092529022 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55612d327830 con 0x55612d3caac0 2024-09-15T22:55:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.380+0000 7f9e5d400640 1 --2- 172.21.15.111:0/2092529022 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55612d3caac0 0x55612d3ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.381+0000 7f9e5d400640 1 --2- 172.21.15.111:0/2092529022 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55612d3caac0 0x55612d3ccee0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30eba50639907b36 server_cookie=86ff98fbd12d4137 in_seq=0 out_seq=0 2024-09-15T22:55:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.381+0000 7f9e3fe00640 10 client.25750 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:55:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.381+0000 7f9e3fe00640 1 -- 172.21.15.111:0/2092529022 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55612d327830 con 0x55612d3caac0 2024-09-15T22:55:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.381+0000 7f9e3fe00640 10 client.25750 handle_command_reply: tid=0 2024-09-15T22:55:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.382+0000 7f9e67c8b740 1 client.25750 shutdown 2024-09-15T22:55:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.382+0000 7f9e67c8b740 1 -- 172.21.15.111:0/2092529022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b5090 msgr2=0x55612d3bf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.382+0000 7f9e67c8b740 1 --2- 172.21.15.111:0/2092529022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b5090 0x55612d3bf2a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9e5801b040 tx=0x7f9e58009c80 comp rx=0 tx=0).stop 2024-09-15T22:55:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.382+0000 7f9e67c8b740 1 -- 172.21.15.111:0/2092529022 shutdown_connections 2024-09-15T22:55:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.382+0000 7f9e67c8b740 1 --2- 172.21.15.111:0/2092529022 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55612d3caac0 0x55612d3ccee0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.382+0000 7f9e67c8b740 1 --2- 172.21.15.111:0/2092529022 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55612d3baa60 0x55612d3c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.382+0000 7f9e67c8b740 1 --2- 172.21.15.111:0/2092529022 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55612d3b59d0 0x55612d3bf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.382+0000 7f9e67c8b740 1 --2- 172.21.15.111:0/2092529022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b5090 0x55612d3bf2a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.382+0000 7f9e67c8b740 1 -- 172.21.15.111:0/2092529022 >> 172.21.15.111:0/2092529022 conn(0x55612d3b0700 msgr2=0x55612d3b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.382+0000 7f9e3fe00640 0 client.25750 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:55:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.382+0000 7f9e67c8b740 1 -- 172.21.15.111:0/2092529022 shutdown_connections 2024-09-15T22:55:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.382+0000 7f9e67c8b740 1 -- 172.21.15.111:0/2092529022 wait complete. 2024-09-15T22:55:15.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.382+0000 7f9e67c8b740 20 client.25750 trim_cache size 0 max 16384 2024-09-15T22:55:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.397+0000 7f9e67c8b740 1 Processor -- start 2024-09-15T22:55:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.397+0000 7f9e67c8b740 1 -- start start 2024-09-15T22:55:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.397+0000 7f9e67c8b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55612d3b5090 0x55612d3bf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.397+0000 7f9e67c8b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b59d0 0x55612d3c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.397+0000 7f9e67c8b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55612d3baa60 0x55612d3ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.397+0000 7f9e67c8b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9e601ad8a0 con 0x55612d3baa60 2024-09-15T22:55:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.397+0000 7f9e67c8b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9e601ab170 con 0x55612d3b5090 2024-09-15T22:55:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.397+0000 7f9e67c8b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9e601a8fc0 con 0x55612d3b59d0 2024-09-15T22:55:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.398+0000 7f9e5d400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b59d0 0x55612d3c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.398+0000 7f9e5e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55612d3b5090 0x55612d3bf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.398+0000 7f9e5d400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b59d0 0x55612d3c66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:33056/0 (socket says 172.21.15.111:33056) 2024-09-15T22:55:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.398+0000 7f9e5d400640 1 -- 172.21.15.111:0/3348161578 learned_addr learned my addr 172.21.15.111:0/3348161578 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:55:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.398+0000 7f9e5e800640 1 --2- 172.21.15.111:0/3348161578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55612d3baa60 0x55612d3ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.398+0000 7f9e5d400640 1 -- 172.21.15.111:0/3348161578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55612d3b5090 msgr2=0x55612d3bf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.398+0000 7f9e5d400640 1 --2- 172.21.15.111:0/3348161578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55612d3b5090 0x55612d3bf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.398+0000 7f9e5d400640 1 -- 172.21.15.111:0/3348161578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55612d3baa60 msgr2=0x55612d3ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.398+0000 7f9e5d400640 1 --2- 172.21.15.111:0/3348161578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55612d3baa60 0x55612d3ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.398+0000 7f9e5d400640 1 -- 172.21.15.111:0/3348161578 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55612d3bdd50 con 0x55612d3b59d0 2024-09-15T22:55:15.403 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.398+0000 7f9e5e800640 1 --2- 172.21.15.111:0/3348161578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55612d3baa60 0x55612d3ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:55:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.398+0000 7f9e5d400640 1 --2- 172.21.15.111:0/3348161578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b59d0 0x55612d3c66d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009dd0 tx=0x7f9e4c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f810e3885965dfb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.399+0000 7f9e4aa00640 1 -- 172.21.15.111:0/3348161578 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c022070 con 0x55612d3b59d0 2024-09-15T22:55:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.399+0000 7f9e4aa00640 1 -- 172.21.15.111:0/3348161578 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e4c0047a0 con 0x55612d3b59d0 2024-09-15T22:55:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.399+0000 7f9e4aa00640 1 -- 172.21.15.111:0/3348161578 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c01e8e0 con 0x55612d3b59d0 2024-09-15T22:55:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.399+0000 7f9e67c8b740 1 -- 172.21.15.111:0/3348161578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b59d0 msgr2=0x55612d3c66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.399+0000 7f9e67c8b740 1 --2- 172.21.15.111:0/3348161578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b59d0 0x55612d3c66d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009dd0 tx=0x7f9e4c005f90 comp rx=0 tx=0).stop 2024-09-15T22:55:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.399+0000 7f9e67c8b740 1 -- 172.21.15.111:0/3348161578 shutdown_connections 2024-09-15T22:55:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.399+0000 7f9e67c8b740 1 --2- 172.21.15.111:0/3348161578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55612d3baa60 0x55612d3ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.399+0000 7f9e67c8b740 1 --2- 172.21.15.111:0/3348161578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b59d0 0x55612d3c66d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.399+0000 7f9e67c8b740 1 --2- 172.21.15.111:0/3348161578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55612d3b5090 0x55612d3bf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.399+0000 7f9e67c8b740 1 -- 172.21.15.111:0/3348161578 >> 172.21.15.111:0/3348161578 conn(0x55612d3b0700 msgr2=0x55612d3bf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:15.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.399+0000 7f9e67c8b740 1 -- 172.21.15.111:0/3348161578 shutdown_connections 2024-09-15T22:55:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.399+0000 7f9e67c8b740 1 -- 172.21.15.111:0/3348161578 wait complete. 2024-09-15T22:55:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.400+0000 7f9e67c8b740 1 -- start start 2024-09-15T22:55:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.400+0000 7f9e67c8b740 1 Processor -- start 2024-09-15T22:55:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.400+0000 7f9e67c8b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55612d3b5090 0x55612d3c3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.400+0000 7f9e67c8b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b59d0 0x55612d3c39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.400+0000 7f9e67c8b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55612d3baa60 0x55612d433770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.400+0000 7f9e67c8b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9e601a9630 con 0x55612d3baa60 2024-09-15T22:55:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.400+0000 7f9e67c8b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9e601aaaa0 con 0x55612d3b5090 2024-09-15T22:55:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.400+0000 7f9e67c8b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9e601a9d00 con 0x55612d3b59d0 2024-09-15T22:55:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.400+0000 7f9e67c8b740 10 client.0 _ll_register_callbacks cb 0x7f9e4c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:55:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.401+0000 7f9e5d400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b59d0 0x55612d3c39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.401+0000 7f9e5d400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b59d0 0x55612d3c39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:33066/0 (socket says 172.21.15.111:33066) 2024-09-15T22:55:15.405 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.401+0000 7f9e5d400640 1 -- 172.21.15.111:0/3471372140 learned_addr learned my addr 172.21.15.111:0/3471372140 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:55:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.401+0000 7f9e5e800640 1 --2- 172.21.15.111:0/3471372140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55612d3baa60 0x55612d433770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.401+0000 7f9e5e800640 1 --2- 172.21.15.111:0/3471372140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55612d3b5090 0x55612d3c3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.401+0000 7f9e5d400640 1 -- 172.21.15.111:0/3471372140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55612d3b5090 msgr2=0x55612d3c3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.401+0000 7f9e5d400640 1 --2- 172.21.15.111:0/3471372140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55612d3b5090 0x55612d3c3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.401+0000 7f9e5d400640 1 -- 172.21.15.111:0/3471372140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55612d3baa60 msgr2=0x55612d433770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.401+0000 7f9e5d400640 1 --2- 172.21.15.111:0/3471372140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55612d3baa60 0x55612d433770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.401+0000 7f9e5d400640 1 -- 172.21.15.111:0/3471372140 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55612d3c4040 con 0x55612d3b59d0 2024-09-15T22:55:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.401+0000 7f9e5e800640 1 --2- 172.21.15.111:0/3471372140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55612d3baa60 0x55612d433770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:55:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.401+0000 7f9e5d400640 1 --2- 172.21.15.111:0/3471372140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b59d0 0x55612d3c39a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c023000 tx=0x7f9e4c009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fb86bbeea733a95 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.401+0000 7f9e3fe00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:55:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.401+0000 7f9e3fe00640 1 -- 172.21.15.111:0/3471372140 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c022070 con 0x55612d3b59d0 2024-09-15T22:55:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.402+0000 7f9e67c8b740 1 -- 172.21.15.111:0/3471372140 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e4c0047a0 con 0x55612d3b59d0 2024-09-15T22:55:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.402+0000 7f9e3fe00640 1 -- 172.21.15.111:0/3471372140 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f9e4c005000 con 0x55612d3b59d0 2024-09-15T22:55:15.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.402+0000 7f9e3fe00640 1 -- 172.21.15.111:0/3471372140 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9e4c005200 con 0x55612d3b59d0 2024-09-15T22:55:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.402+0000 7f9e67c8b740 10 client.25756 fetch_fsmap learned FSMap version 30 2024-09-15T22:55:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.402+0000 7f9e67c8b740 1 -- 172.21.15.111:0/3471372140 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55612d434e90 con 0x55612d3b59d0 2024-09-15T22:55:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.402+0000 7f9e3fe00640 1 -- 172.21.15.111:0/3471372140 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e4c0047a0 con 0x55612d3b59d0 2024-09-15T22:55:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.402+0000 7f9e3fe00640 1 -- 172.21.15.111:0/3471372140 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f9e4c027530 con 0x55612d3b59d0 2024-09-15T22:55:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.402+0000 7f9e67c8b740 10 client.25756 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:55:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.402+0000 7f9e67c8b740 10 client.25756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:55:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.402+0000 7f9e67c8b740 20 client.25756 populate_metadata read hostname 'smithi111' 2024-09-15T22:55:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.402+0000 7f9e67c8b740 1 --2- 172.21.15.111:0/3471372140 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55612d435730 0x55612d437b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.402+0000 7f9e67c8b740 4 client.25756 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:55:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.402+0000 7f9e67c8b740 1 -- 172.21.15.111:0/3471372140 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55612d3abd80 con 0x55612d435730 2024-09-15T22:55:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.403+0000 7f9e5e800640 1 --2- 172.21.15.111:0/3471372140 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55612d435730 0x55612d437b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.403+0000 7f9e3fe00640 10 client.25756 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:55:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.403+0000 7f9e5e800640 1 --2- 172.21.15.111:0/3471372140 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55612d435730 0x55612d437b50 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef67abd740ea78da server_cookie=8dbe6e9b6ba7e863 in_seq=0 out_seq=0 2024-09-15T22:55:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.416+0000 7f9e3fe00640 1 -- 172.21.15.111:0/3471372140 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55612d3abd80 con 0x55612d435730 2024-09-15T22:55:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.416+0000 7f9e3fe00640 10 client.25756 handle_command_reply: tid=0 2024-09-15T22:55:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.416+0000 7f9e67c8b740 1 client.25756 shutdown 2024-09-15T22:55:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.416+0000 7f9e67c8b740 1 -- 172.21.15.111:0/3471372140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b59d0 msgr2=0x55612d3c39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.416+0000 7f9e67c8b740 1 --2- 172.21.15.111:0/3471372140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b59d0 0x55612d3c39a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c023000 tx=0x7f9e4c009bb0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.416+0000 7f9e67c8b740 1 -- 172.21.15.111:0/3471372140 shutdown_connections 2024-09-15T22:55:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.416+0000 7f9e67c8b740 1 --2- 172.21.15.111:0/3471372140 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55612d435730 0x55612d437b50 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.416+0000 7f9e67c8b740 1 --2- 172.21.15.111:0/3471372140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55612d3baa60 0x55612d433770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.416+0000 7f9e67c8b740 1 --2- 172.21.15.111:0/3471372140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55612d3b59d0 0x55612d3c39a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.416+0000 7f9e3fe00640 0 client.25756 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:55:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.416+0000 7f9e67c8b740 1 --2- 172.21.15.111:0/3471372140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55612d3b5090 0x55612d3c3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.416+0000 7f9e67c8b740 1 -- 172.21.15.111:0/3471372140 >> 172.21.15.111:0/3471372140 conn(0x55612d3b0700 msgr2=0x55612d3af060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.416+0000 7f9e67c8b740 1 -- 172.21.15.111:0/3471372140 shutdown_connections 2024-09-15T22:55:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.416+0000 7f9e67c8b740 1 -- 172.21.15.111:0/3471372140 wait complete. 2024-09-15T22:55:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.416+0000 7f9e67c8b740 20 client.25756 trim_cache size 0 max 16384 2024-09-15T22:55:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:55:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:55:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:55:15.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:55:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.417+0000 7f9e64e00640 1 -- 172.21.15.111:0/1588307135 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9e440a3370 msgr2=0x7f9e440a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.417+0000 7f9e64e00640 1 --2- 172.21.15.111:0/1588307135 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9e440a3370 0x7f9e440a5830 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9e50005ec0 tx=0x7f9e5002a410 comp rx=0 tx=0).stop 2024-09-15T22:55:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.417+0000 7f9e64e00640 1 -- 172.21.15.111:0/1588307135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e60103d60 msgr2=0x7f9e60199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.417+0000 7f9e64e00640 1 --2- 172.21.15.111:0/1588307135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e60103d60 0x7f9e60199f90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9e50031440 tx=0x7f9e50032ce0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.417+0000 7f9e64e00640 1 -- 172.21.15.111:0/1588307135 shutdown_connections 2024-09-15T22:55:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.417+0000 7f9e64e00640 1 --2- 172.21.15.111:0/1588307135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e60105910 0x7f9e601a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.420 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.417+0000 7f9e64e00640 1 --2- 172.21.15.111:0/1588307135 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e60104f50 0x7f9e6019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.417+0000 7f9e64e00640 1 --2- 172.21.15.111:0/1588307135 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9e440a3370 0x7f9e440a5830 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.417+0000 7f9e64e00640 1 --2- 172.21.15.111:0/1588307135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e60103d60 0x7f9e60199f90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.417+0000 7f9e64e00640 1 -- 172.21.15.111:0/1588307135 >> 172.21.15.111:0/1588307135 conn(0x7f9e600ff4e0 msgr2=0x7f9e60108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.417+0000 7f9e64e00640 1 -- 172.21.15.111:0/1588307135 shutdown_connections 2024-09-15T22:55:15.421 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:15.417+0000 7f9e64e00640 1 -- 172.21.15.111:0/1588307135 wait complete. 2024-09-15T22:55:16.067 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:15 smithi111 ceph-mon[30629]: pgmap v630: 97 pgs: 97 active+clean; 281 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 181 op/s 2024-09-15T22:55:16.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:15 smithi121 ceph-mon[34815]: pgmap v630: 97 pgs: 97 active+clean; 281 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 181 op/s 2024-09-15T22:55:16.303 INFO:tasks.workunit.client.0.smithi111.stdout: 1 865742 9.88 MB/sec execute 344 sec latency 12.220 ms 2024-09-15T22:55:16.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:15 smithi192 ceph-mon[35427]: pgmap v630: 97 pgs: 97 active+clean; 281 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 181 op/s 2024-09-15T22:55:17.303 INFO:tasks.workunit.client.0.smithi111.stdout: 1 868356 9.89 MB/sec execute 345 sec latency 12.089 ms 2024-09-15T22:55:18.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:17 smithi121 ceph-mon[34815]: pgmap v631: 97 pgs: 97 active+clean; 283 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 214 op/s 2024-09-15T22:55:18.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:17 smithi111 ceph-mon[30629]: pgmap v631: 97 pgs: 97 active+clean; 283 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 214 op/s 2024-09-15T22:55:18.303 INFO:tasks.workunit.client.0.smithi111.stdout: 1 870533 9.90 MB/sec execute 346 sec latency 27.437 ms 2024-09-15T22:55:18.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:17 smithi192 ceph-mon[35427]: pgmap v631: 97 pgs: 97 active+clean; 283 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 214 op/s 2024-09-15T22:55:19.303 INFO:tasks.workunit.client.0.smithi111.stdout: 1 870533 9.87 MB/sec execute 347 sec latency 1023.492 ms 2024-09-15T22:55:20.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:19 smithi121 ceph-mon[34815]: pgmap v632: 97 pgs: 97 active+clean; 286 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 219 op/s 2024-09-15T22:55:20.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:19 smithi111 ceph-mon[30629]: pgmap v632: 97 pgs: 97 active+clean; 286 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 219 op/s 2024-09-15T22:55:20.303 INFO:tasks.workunit.client.0.smithi111.stdout: 1 872713 9.88 MB/sec execute 348 sec latency 1239.050 ms 2024-09-15T22:55:20.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:19 smithi192 ceph-mon[35427]: pgmap v632: 97 pgs: 97 active+clean; 286 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 219 op/s 2024-09-15T22:55:21.303 INFO:tasks.workunit.client.0.smithi111.stdout: 1 875043 9.89 MB/sec execute 349 sec latency 19.679 ms 2024-09-15T22:55:22.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:21 smithi121 ceph-mon[34815]: pgmap v633: 97 pgs: 97 active+clean; 285 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 221 op/s 2024-09-15T22:55:22.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:21 smithi111 ceph-mon[30629]: pgmap v633: 97 pgs: 97 active+clean; 285 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 221 op/s 2024-09-15T22:55:22.303 INFO:tasks.workunit.client.0.smithi111.stdout: 1 877712 9.90 MB/sec execute 350 sec latency 10.851 ms 2024-09-15T22:55:22.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:21 smithi192 ceph-mon[35427]: pgmap v633: 97 pgs: 97 active+clean; 285 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 221 op/s 2024-09-15T22:55:23.303 INFO:tasks.workunit.client.0.smithi111.stdout: 1 880799 9.92 MB/sec execute 351 sec latency 10.327 ms 2024-09-15T22:55:23.896 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:23 smithi111 ceph-mon[30629]: pgmap v634: 97 pgs: 97 active+clean; 285 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 168 op/s 2024-09-15T22:55:24.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:23 smithi121 ceph-mon[34815]: pgmap v634: 97 pgs: 97 active+clean; 285 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 168 op/s 2024-09-15T22:55:24.309 INFO:tasks.workunit.client.0.smithi111.stdout: 1 880866 9.90 MB/sec execute 352 sec latency 673.451 ms 2024-09-15T22:55:24.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:23 smithi192 ceph-mon[35427]: pgmap v634: 97 pgs: 97 active+clean; 285 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 168 op/s 2024-09-15T22:55:25.309 INFO:tasks.workunit.client.0.smithi111.stdout: 1 881208 9.87 MB/sec execute 353 sec latency 1419.374 ms 2024-09-15T22:55:26.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:25 smithi121 ceph-mon[34815]: pgmap v635: 97 pgs: 97 active+clean; 287 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 183 op/s 2024-09-15T22:55:26.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:25 smithi111 ceph-mon[30629]: pgmap v635: 97 pgs: 97 active+clean; 287 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 183 op/s 2024-09-15T22:55:26.309 INFO:tasks.workunit.client.0.smithi111.stdout: 1 884282 9.89 MB/sec execute 354 sec latency 13.407 ms 2024-09-15T22:55:26.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:25 smithi192 ceph-mon[35427]: pgmap v635: 97 pgs: 97 active+clean; 287 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 183 op/s 2024-09-15T22:55:27.309 INFO:tasks.workunit.client.0.smithi111.stdout: 1 887040 9.90 MB/sec execute 355 sec latency 13.543 ms 2024-09-15T22:55:28.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:27 smithi121 ceph-mon[34815]: pgmap v636: 97 pgs: 97 active+clean; 294 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 220 op/s 2024-09-15T22:55:28.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:27 smithi111 ceph-mon[30629]: pgmap v636: 97 pgs: 97 active+clean; 294 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 220 op/s 2024-09-15T22:55:28.309 INFO:tasks.workunit.client.0.smithi111.stdout: 1 889427 9.91 MB/sec execute 356 sec latency 13.251 ms 2024-09-15T22:55:28.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:27 smithi192 ceph-mon[35427]: pgmap v636: 97 pgs: 97 active+clean; 294 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 220 op/s 2024-09-15T22:55:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.567+0000 7f8abda00640 1 -- 172.21.15.111:0/153461928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab8103b70 msgr2=0x7f8ab8103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.567+0000 7f8abda00640 1 --2- 172.21.15.111:0/153461928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab8103b70 0x7f8ab8103f70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009900 tx=0x7f8aa802a410 comp rx=0 tx=0).stop 2024-09-15T22:55:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.567+0000 7f8abda00640 1 -- 172.21.15.111:0/153461928 shutdown_connections 2024-09-15T22:55:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.567+0000 7f8abda00640 1 --2- 172.21.15.111:0/153461928 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ab8105720 0x7f8ab810bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.567+0000 7f8abda00640 1 --2- 172.21.15.111:0/153461928 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ab8104d60 0x7f8ab81051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.567+0000 7f8abda00640 1 --2- 172.21.15.111:0/153461928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab8103b70 0x7f8ab8103f70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.567+0000 7f8abda00640 1 -- 172.21.15.111:0/153461928 >> 172.21.15.111:0/153461928 conn(0x7f8ab80ff2f0 msgr2=0x7f8ab8101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.567+0000 7f8abda00640 1 -- 172.21.15.111:0/153461928 shutdown_connections 2024-09-15T22:55:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.567+0000 7f8abda00640 1 -- 172.21.15.111:0/153461928 wait complete. 2024-09-15T22:55:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.568+0000 7f8abda00640 1 Processor -- start 2024-09-15T22:55:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.568+0000 7f8abda00640 1 -- start start 2024-09-15T22:55:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.569+0000 7f8abda00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab8103b70 0x7f8ab8199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.569+0000 7f8abda00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ab8104d60 0x7f8ab819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.569+0000 7f8abda00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ab8105720 0x7f8ab81a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:28.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.569+0000 7f8abda00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab810f820 con 0x7f8ab8104d60 2024-09-15T22:55:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.569+0000 7f8abda00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab810f6a0 con 0x7f8ab8105720 2024-09-15T22:55:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.569+0000 7f8abda00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab810f9a0 con 0x7f8ab8103b70 2024-09-15T22:55:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.569+0000 7f8abd000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ab8105720 0x7f8ab81a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.569+0000 7f8ab7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ab8104d60 0x7f8ab819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.569+0000 7f8abd000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ab8105720 0x7f8ab81a1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:60480/0 (socket says 172.21.15.111:60480) 2024-09-15T22:55:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.569+0000 7f8abd000640 1 -- 172.21.15.111:0/2082552114 learned_addr learned my addr 172.21.15.111:0/2082552114 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:55:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.569+0000 7f8ab7e00640 1 --2- 172.21.15.111:0/2082552114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab8103b70 0x7f8ab8199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.569+0000 7f8ab7400640 1 -- 172.21.15.111:0/2082552114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab8103b70 msgr2=0x7f8ab8199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.569+0000 7f8ab7400640 1 --2- 172.21.15.111:0/2082552114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab8103b70 0x7f8ab8199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.570+0000 7f8ab7400640 1 -- 172.21.15.111:0/2082552114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ab8105720 msgr2=0x7f8ab81a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.570+0000 7f8ab7400640 1 --2- 172.21.15.111:0/2082552114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ab8105720 0x7f8ab81a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.570+0000 7f8ab7400640 1 -- 172.21.15.111:0/2082552114 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab81a1ad0 con 0x7f8ab8104d60 2024-09-15T22:55:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.570+0000 7f8ab7400640 1 --2- 172.21.15.111:0/2082552114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ab8104d60 0x7f8ab819a3f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400e9f0 tx=0x7f8aa400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.570+0000 7f8ab4c00640 1 -- 172.21.15.111:0/2082552114 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa400ccc0 con 0x7f8ab8104d60 2024-09-15T22:55:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.570+0000 7f8abda00640 1 -- 172.21.15.111:0/2082552114 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab81a1dc0 con 0x7f8ab8104d60 2024-09-15T22:55:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.570+0000 7f8ab4c00640 1 -- 172.21.15.111:0/2082552114 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aa4010910 con 0x7f8ab8104d60 2024-09-15T22:55:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.570+0000 7f8ab4c00640 1 -- 172.21.15.111:0/2082552114 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4018960 con 0x7f8ab8104d60 2024-09-15T22:55:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.570+0000 7f8abda00640 1 -- 172.21.15.111:0/2082552114 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab81a2290 con 0x7f8ab8104d60 2024-09-15T22:55:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.573+0000 7f8ac1dc3740 1 Processor -- start 2024-09-15T22:55:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.573+0000 7f8ac1dc3740 1 -- start start 2024-09-15T22:55:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.574+0000 7f8ac1dc3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5652e3b80260 0x5652e3b80660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.574+0000 7f8ac1dc3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5652e3b80ba0 0x5652e3b85660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.574+0000 7f8ac1dc3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652e3b85c30 0x5652e3b880f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.574+0000 7f8ac1dc3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5652e3ae5100 con 0x5652e3b80ba0 2024-09-15T22:55:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.574+0000 7f8ac1dc3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5652e3a8d1b0 con 0x5652e3b80260 2024-09-15T22:55:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.574+0000 7f8ac1dc3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5652e3ae10c0 con 0x5652e3b85c30 2024-09-15T22:55:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.574+0000 7f8ab7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5652e3b80ba0 0x5652e3b85660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.574+0000 7f8ab4c00640 1 -- 172.21.15.111:0/2082552114 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f8aa4010ac0 con 0x7f8ab8104d60 2024-09-15T22:55:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.574+0000 7f8ab7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5652e3b80ba0 0x5652e3b85660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46254/0 (socket says 172.21.15.111:46254) 2024-09-15T22:55:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.574+0000 7f8ab7e00640 1 -- 172.21.15.111:0/3493223215 learned_addr learned my addr 172.21.15.111:0/3493223215 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:55:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.574+0000 7f8abd000640 1 --2- 172.21.15.111:0/3493223215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5652e3b80260 0x5652e3b80660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.574+0000 7f8ab7400640 1 --2- 172.21.15.111:0/3493223215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652e3b85c30 0x5652e3b880f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.574+0000 7f8ab7e00640 1 -- 172.21.15.111:0/3493223215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652e3b85c30 msgr2=0x5652e3b880f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.574+0000 7f8ab7e00640 1 --2- 172.21.15.111:0/3493223215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652e3b85c30 0x5652e3b880f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.574+0000 7f8ab7e00640 1 -- 172.21.15.111:0/3493223215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5652e3b80260 msgr2=0x5652e3b80660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.574+0000 7f8ab7e00640 1 --2- 172.21.15.111:0/3493223215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5652e3b80260 0x5652e3b80660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.574+0000 7f8ab7e00640 1 -- 172.21.15.111:0/3493223215 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652e37f8210 con 0x5652e3b80ba0 2024-09-15T22:55:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.575+0000 7f8ab7e00640 1 --2- 172.21.15.111:0/3493223215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5652e3b80ba0 0x5652e3b85660 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8006010 tx=0x7f8aa8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672fdacb50ce1a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.575+0000 7f8aad600640 1 -- 172.21.15.111:0/3493223215 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8035040 con 0x5652e3b80ba0 2024-09-15T22:55:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.575+0000 7f8aad600640 1 -- 172.21.15.111:0/3493223215 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aa8004430 con 0x5652e3b80ba0 2024-09-15T22:55:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.575+0000 7f8aad600640 1 -- 172.21.15.111:0/3493223215 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa80316b0 con 0x5652e3b80ba0 2024-09-15T22:55:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.575+0000 7f8ab4c00640 1 --2- 172.21.15.111:0/2082552114 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8a9c0a3240 0x7f8a9c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.576+0000 7f8ab4c00640 1 -- 172.21.15.111:0/2082552114 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f8aa40cfc90 con 0x7f8ab8104d60 2024-09-15T22:55:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.576+0000 7f8ac1dc3740 1 -- 172.21.15.111:0/3493223215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5652e3b80ba0 msgr2=0x5652e3b85660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.576+0000 7f8ac1dc3740 1 --2- 172.21.15.111:0/3493223215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5652e3b80ba0 0x5652e3b85660 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8006010 tx=0x7f8aa8004040 comp rx=0 tx=0).stop 2024-09-15T22:55:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.576+0000 7f8abd000640 1 --2- 172.21.15.111:0/2082552114 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8a9c0a3240 0x7f8a9c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.576+0000 7f8ac1dc3740 1 -- 172.21.15.111:0/3493223215 shutdown_connections 2024-09-15T22:55:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.576+0000 7f8ac1dc3740 1 --2- 172.21.15.111:0/3493223215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652e3b85c30 0x5652e3b880f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.576+0000 7f8ac1dc3740 1 --2- 172.21.15.111:0/3493223215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5652e3b80ba0 0x5652e3b85660 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.576+0000 7f8ac1dc3740 1 --2- 172.21.15.111:0/3493223215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5652e3b80260 0x5652e3b80660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.576+0000 7f8ac1dc3740 1 -- 172.21.15.111:0/3493223215 >> 172.21.15.111:0/3493223215 conn(0x5652e3b7b8d0 msgr2=0x5652e3b7bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.576+0000 7f8ac1dc3740 1 -- 172.21.15.111:0/3493223215 shutdown_connections 2024-09-15T22:55:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.576+0000 7f8ac1dc3740 1 -- 172.21.15.111:0/3493223215 wait complete. 2024-09-15T22:55:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.577+0000 7f8abd000640 1 --2- 172.21.15.111:0/2082552114 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8a9c0a3240 0x7f8a9c0a5700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8ab00049d0 tx=0x7f8ab0009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.577+0000 7f8ac1dc3740 1 -- start start 2024-09-15T22:55:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.577+0000 7f8ac1dc3740 1 Processor -- start 2024-09-15T22:55:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.577+0000 7f8ac1dc3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652e3b80260 0x5652e3b8a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.577+0000 7f8ac1dc3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5652e3b80ba0 0x5652e3b8a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.577+0000 7f8ac1dc3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5652e3b85c30 0x5652e3b91940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.577+0000 7f8ac1dc3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5652e3b88ba0 con 0x5652e3b85c30 2024-09-15T22:55:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.577+0000 7f8ac1dc3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5652e3a6e090 con 0x5652e3b80ba0 2024-09-15T22:55:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.577+0000 7f8ac1dc3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5652e3b88d20 con 0x5652e3b80260 2024-09-15T22:55:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.577+0000 7f8ac1dc3740 10 client.0 _ll_register_callbacks cb 0x5652e3ac2f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:55:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.578+0000 7f8ab7400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5652e3b80ba0 0x5652e3b8a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.578+0000 7f8abd000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5652e3b85c30 0x5652e3b91940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.578+0000 7f8ab7e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652e3b80260 0x5652e3b8a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.578+0000 7f8ab7400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5652e3b80ba0 0x5652e3b8a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:60492/0 (socket says 172.21.15.111:60492) 2024-09-15T22:55:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.578+0000 7f8ab7400640 1 -- 172.21.15.111:0/1770390356 learned_addr learned my addr 172.21.15.111:0/1770390356 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:55:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.578+0000 7f8ab7400640 1 -- 172.21.15.111:0/1770390356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652e3b80260 msgr2=0x5652e3b8a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.578+0000 7f8ab7400640 1 --2- 172.21.15.111:0/1770390356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652e3b80260 0x5652e3b8a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.578+0000 7f8ab7400640 1 -- 172.21.15.111:0/1770390356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5652e3b85c30 msgr2=0x5652e3b91940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.578+0000 7f8ab7400640 1 --2- 172.21.15.111:0/1770390356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5652e3b85c30 0x5652e3b91940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.578+0000 7f8ab7400640 1 -- 172.21.15.111:0/1770390356 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652e38c4620 con 0x5652e3b80ba0 2024-09-15T22:55:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.578+0000 7f8ab7400640 1 --2- 172.21.15.111:0/1770390356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5652e3b80ba0 0x5652e3b8a900 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8aa40a4a90 tx=0x7f8aa409be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed4dbd436055825 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.578+0000 7f8aa2000640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:55:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.578+0000 7f8aa2000640 1 -- 172.21.15.111:0/1770390356 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa409c350 con 0x5652e3b80ba0 2024-09-15T22:55:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.578+0000 7f8aa2000640 1 -- 172.21.15.111:0/1770390356 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f8aa409b7a0 con 0x5652e3b80ba0 2024-09-15T22:55:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.578+0000 7f8ac1dc3740 1 -- 172.21.15.111:0/1770390356 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8aa8004430 con 0x5652e3b80ba0 2024-09-15T22:55:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.579+0000 7f8aa2000640 1 -- 172.21.15.111:0/1770390356 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8aa40d3280 con 0x5652e3b80ba0 2024-09-15T22:55:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.579+0000 7f8ac1dc3740 10 client.25739 fetch_fsmap learned FSMap version 30 2024-09-15T22:55:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.579+0000 7f8ac1dc3740 1 -- 172.21.15.111:0/1770390356 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5652e3b947a0 con 0x5652e3b80ba0 2024-09-15T22:55:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.579+0000 7f8aa2000640 1 -- 172.21.15.111:0/1770390356 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aa8004430 con 0x5652e3b80ba0 2024-09-15T22:55:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.579+0000 7f8aa2000640 1 -- 172.21.15.111:0/1770390356 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f8aa4018b00 con 0x5652e3b80ba0 2024-09-15T22:55:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.579+0000 7f8ac1dc3740 10 client.25739 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:55:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.579+0000 7f8ac1dc3740 10 client.25739 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:55:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.579+0000 7f8ac1dc3740 20 client.25739 populate_metadata read hostname 'smithi111' 2024-09-15T22:55:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.579+0000 7f8ac1dc3740 1 --2- 172.21.15.111:0/1770390356 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5652e3b95e90 0x5652e3b98280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.579+0000 7f8ac1dc3740 4 client.25739 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:55:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.579+0000 7f8ac1dc3740 1 -- 172.21.15.111:0/1770390356 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652e3af2830 con 0x5652e3b95e90 2024-09-15T22:55:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.579+0000 7f8ab7e00640 1 --2- 172.21.15.111:0/1770390356 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5652e3b95e90 0x5652e3b98280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.580+0000 7f8aa2000640 10 client.25739 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:55:28.585 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.580+0000 7f8ab7e00640 1 --2- 172.21.15.111:0/1770390356 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5652e3b95e90 0x5652e3b98280 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14c853ad84ba4f9f server_cookie=7781d165f9af7c50 in_seq=0 out_seq=0 2024-09-15T22:55:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.581+0000 7f8aa2000640 1 -- 172.21.15.111:0/1770390356 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5652e3af2830 con 0x5652e3b95e90 2024-09-15T22:55:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.581+0000 7f8aa2000640 10 client.25739 handle_command_reply: tid=0 2024-09-15T22:55:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.581+0000 7f8ac1dc3740 1 client.25739 shutdown 2024-09-15T22:55:28.586 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.581+0000 7f8ac1dc3740 1 -- 172.21.15.111:0/1770390356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5652e3b80ba0 msgr2=0x5652e3b8a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.581+0000 7f8ac1dc3740 1 --2- 172.21.15.111:0/1770390356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5652e3b80ba0 0x5652e3b8a900 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8aa40a4a90 tx=0x7f8aa409be90 comp rx=0 tx=0).stop 2024-09-15T22:55:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.581+0000 7f8ac1dc3740 1 -- 172.21.15.111:0/1770390356 shutdown_connections 2024-09-15T22:55:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.581+0000 7f8ac1dc3740 1 --2- 172.21.15.111:0/1770390356 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5652e3b95e90 0x5652e3b98280 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.581+0000 7f8ac1dc3740 1 --2- 172.21.15.111:0/1770390356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5652e3b85c30 0x5652e3b91940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.581+0000 7f8ac1dc3740 1 --2- 172.21.15.111:0/1770390356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5652e3b80ba0 0x5652e3b8a900 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.581+0000 7f8ac1dc3740 1 --2- 172.21.15.111:0/1770390356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652e3b80260 0x5652e3b8a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.581+0000 7f8aa2000640 0 client.25739 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:55:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.581+0000 7f8ac1dc3740 1 -- 172.21.15.111:0/1770390356 >> 172.21.15.111:0/1770390356 conn(0x5652e3b7b8d0 msgr2=0x5652e3b7d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.581+0000 7f8ac1dc3740 1 -- 172.21.15.111:0/1770390356 shutdown_connections 2024-09-15T22:55:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.581+0000 7f8ac1dc3740 1 -- 172.21.15.111:0/1770390356 wait complete. 2024-09-15T22:55:28.587 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.581+0000 7f8ac1dc3740 20 client.25739 trim_cache size 0 max 16384 2024-09-15T22:55:28.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.606+0000 7f8ac1dc3740 1 Processor -- start 2024-09-15T22:55:28.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.606+0000 7f8ac1dc3740 1 -- start start 2024-09-15T22:55:28.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.606+0000 7f8ac1dc3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5652e3b80260 0x5652e3b8a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.606+0000 7f8ac1dc3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5652e3b80ba0 0x5652e3b8a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.606+0000 7f8ac1dc3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652e3b85c30 0x5652e3b91900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.606+0000 7f8ac1dc3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab81ad7d0 con 0x5652e3b80ba0 2024-09-15T22:55:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.606+0000 7f8ac1dc3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab81ab0a0 con 0x5652e3b80260 2024-09-15T22:55:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.606+0000 7f8ac1dc3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab81a8ef0 con 0x5652e3b85c30 2024-09-15T22:55:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.606+0000 7f8ab7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5652e3b80ba0 0x5652e3b8a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.606+0000 7f8ab7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5652e3b80260 0x5652e3b8a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.606+0000 7f8ab7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5652e3b80260 0x5652e3b8a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:60504/0 (socket says 172.21.15.111:60504) 2024-09-15T22:55:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.606+0000 7f8ab7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5652e3b80ba0 0x5652e3b8a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46274/0 (socket says 172.21.15.111:46274) 2024-09-15T22:55:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.606+0000 7f8ab7e00640 1 -- 172.21.15.111:0/3261029416 learned_addr learned my addr 172.21.15.111:0/3261029416 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:55:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.607+0000 7f8ab7400640 1 -- 172.21.15.111:0/3261029416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652e3b85c30 msgr2=0x5652e3b91900 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:55:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.607+0000 7f8ab7400640 1 --2- 172.21.15.111:0/3261029416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652e3b85c30 0x5652e3b91900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.607+0000 7f8ab7400640 1 -- 172.21.15.111:0/3261029416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5652e3b80260 msgr2=0x5652e3b8a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.607+0000 7f8ab7400640 1 --2- 172.21.15.111:0/3261029416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5652e3b80260 0x5652e3b8a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.607+0000 7f8ab7400640 1 -- 172.21.15.111:0/3261029416 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652e3b89450 con 0x5652e3b80ba0 2024-09-15T22:55:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.607+0000 7f8ab7e00640 1 --2- 172.21.15.111:0/3261029416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5652e3b80260 0x5652e3b8a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:55:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.607+0000 7f8ab7400640 1 --2- 172.21.15.111:0/3261029416 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5652e3b80ba0 0x5652e3b8a900 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8aa409a600 tx=0x7f8aa409a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8514a2ff0761eec3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.607+0000 7f8aacc00640 1 -- 172.21.15.111:0/3261029416 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa409c350 con 0x5652e3b80ba0 2024-09-15T22:55:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.607+0000 7f8aacc00640 1 -- 172.21.15.111:0/3261029416 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aa40a0700 con 0x5652e3b80ba0 2024-09-15T22:55:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.607+0000 7f8aacc00640 1 -- 172.21.15.111:0/3261029416 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa409ba00 con 0x5652e3b80ba0 2024-09-15T22:55:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.608+0000 7f8ac1dc3740 1 -- 172.21.15.111:0/3261029416 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5652e3b80ba0 msgr2=0x5652e3b8a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.608+0000 7f8ac1dc3740 1 --2- 172.21.15.111:0/3261029416 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5652e3b80ba0 0x5652e3b8a900 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8aa409a600 tx=0x7f8aa409a860 comp rx=0 tx=0).stop 2024-09-15T22:55:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.608+0000 7f8ac1dc3740 1 -- 172.21.15.111:0/3261029416 shutdown_connections 2024-09-15T22:55:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.608+0000 7f8ac1dc3740 1 --2- 172.21.15.111:0/3261029416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652e3b85c30 0x5652e3b91900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.608+0000 7f8ac1dc3740 1 --2- 172.21.15.111:0/3261029416 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5652e3b80ba0 0x5652e3b8a900 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.608+0000 7f8ac1dc3740 1 --2- 172.21.15.111:0/3261029416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5652e3b80260 0x5652e3b8a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.608+0000 7f8ac1dc3740 1 -- 172.21.15.111:0/3261029416 >> 172.21.15.111:0/3261029416 conn(0x5652e3b7b8d0 msgr2=0x5652e3b766a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.608+0000 7f8ac1dc3740 1 -- 172.21.15.111:0/3261029416 shutdown_connections 2024-09-15T22:55:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.608+0000 7f8ac1dc3740 1 -- 172.21.15.111:0/3261029416 wait complete. 2024-09-15T22:55:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.608+0000 7f8ac1dc3740 1 -- start start 2024-09-15T22:55:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.608+0000 7f8ac1dc3740 1 Processor -- start 2024-09-15T22:55:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.609+0000 7f8ac1dc3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5652e3b80260 0x5652e3b8eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.609+0000 7f8ac1dc3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5652e3b80ba0 0x5652e3b8f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.609+0000 7f8ac1dc3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652e3b85c30 0x5652e3b90440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.609+0000 7f8ac1dc3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab81a9560 con 0x5652e3b80ba0 2024-09-15T22:55:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.609+0000 7f8ac1dc3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab81aa9d0 con 0x5652e3b80260 2024-09-15T22:55:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.609+0000 7f8ac1dc3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab81a9c30 con 0x5652e3b85c30 2024-09-15T22:55:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.609+0000 7f8ac1dc3740 10 client.0 _ll_register_callbacks cb 0x7f8aa400ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:55:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.609+0000 7f8ab7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5652e3b80260 0x5652e3b8eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.609+0000 7f8ab7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5652e3b80260 0x5652e3b8eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:60508/0 (socket says 172.21.15.111:60508) 2024-09-15T22:55:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.609+0000 7f8ab7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5652e3b80ba0 0x5652e3b8f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.609+0000 7f8ab7e00640 1 -- 172.21.15.111:0/2689728636 learned_addr learned my addr 172.21.15.111:0/2689728636 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:55:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.609+0000 7f8abd000640 1 --2- 172.21.15.111:0/2689728636 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652e3b85c30 0x5652e3b90440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.609+0000 7f8ab7e00640 1 -- 172.21.15.111:0/2689728636 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652e3b85c30 msgr2=0x5652e3b90440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.609+0000 7f8ab7e00640 1 --2- 172.21.15.111:0/2689728636 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652e3b85c30 0x5652e3b90440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.609+0000 7f8ab7e00640 1 -- 172.21.15.111:0/2689728636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5652e3b80ba0 msgr2=0x5652e3b8f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.609+0000 7f8ab7e00640 1 --2- 172.21.15.111:0/2689728636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5652e3b80ba0 0x5652e3b8f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.609+0000 7f8ab7e00640 1 -- 172.21.15.111:0/2689728636 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652e3b90a50 con 0x5652e3b80260 2024-09-15T22:55:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.610+0000 7f8ab7e00640 1 --2- 172.21.15.111:0/2689728636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5652e3b80260 0x5652e3b8eef0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80314e0 tx=0x7f8aa80045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2812ecbd449dd209 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.610+0000 7f8aa2000640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:55:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.610+0000 7f8aa2000640 1 -- 172.21.15.111:0/2689728636 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8035040 con 0x5652e3b80260 2024-09-15T22:55:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.610+0000 7f8ac1dc3740 1 -- 172.21.15.111:0/2689728636 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8aa40a0700 con 0x5652e3b80260 2024-09-15T22:55:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.610+0000 7f8aa2000640 1 -- 172.21.15.111:0/2689728636 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f8aa8031b80 con 0x5652e3b80260 2024-09-15T22:55:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.610+0000 7f8aa2000640 1 -- 172.21.15.111:0/2689728636 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8aa8031d80 con 0x5652e3b80260 2024-09-15T22:55:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.610+0000 7f8ac1dc3740 10 client.25745 fetch_fsmap learned FSMap version 30 2024-09-15T22:55:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.610+0000 7f8ac1dc3740 1 -- 172.21.15.111:0/2689728636 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5652e3c00ad0 con 0x5652e3b80260 2024-09-15T22:55:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.610+0000 7f8aa2000640 1 -- 172.21.15.111:0/2689728636 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aa40a0700 con 0x5652e3b80260 2024-09-15T22:55:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.610+0000 7f8aa2000640 1 -- 172.21.15.111:0/2689728636 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f8aa803c590 con 0x5652e3b80260 2024-09-15T22:55:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.611+0000 7f8ac1dc3740 10 client.25745 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:55:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.611+0000 7f8ac1dc3740 10 client.25745 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:55:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.611+0000 7f8ac1dc3740 20 client.25745 populate_metadata read hostname 'smithi111' 2024-09-15T22:55:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.611+0000 7f8ac1dc3740 1 --2- 172.21.15.111:0/2689728636 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5652e3c01040 0x5652e3c03430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.611+0000 7f8ac1dc3740 4 client.25745 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:55:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.611+0000 7f8ac1dc3740 1 -- 172.21.15.111:0/2689728636 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5652e3aecb80 con 0x5652e3c01040 2024-09-15T22:55:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.611+0000 7f8ab7400640 1 --2- 172.21.15.111:0/2689728636 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5652e3c01040 0x5652e3c03430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.611+0000 7f8ab7400640 1 --2- 172.21.15.111:0/2689728636 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5652e3c01040 0x5652e3c03430 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c951cffb0f5ebd server_cookie=41bd119bc0aaf37a in_seq=0 out_seq=0 2024-09-15T22:55:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.611+0000 7f8aa2000640 10 client.25745 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:55:28.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.621+0000 7f8aa2000640 1 -- 172.21.15.111:0/2689728636 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5652e3aecb80 con 0x5652e3c01040 2024-09-15T22:55:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.621+0000 7f8aa2000640 10 client.25745 handle_command_reply: tid=0 2024-09-15T22:55:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.621+0000 7f8ac1dc3740 1 client.25745 shutdown 2024-09-15T22:55:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.621+0000 7f8ac1dc3740 1 -- 172.21.15.111:0/2689728636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5652e3b80260 msgr2=0x5652e3b8eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.621+0000 7f8ac1dc3740 1 --2- 172.21.15.111:0/2689728636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5652e3b80260 0x5652e3b8eef0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80314e0 tx=0x7f8aa80045e0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.621+0000 7f8ac1dc3740 1 -- 172.21.15.111:0/2689728636 shutdown_connections 2024-09-15T22:55:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.621+0000 7f8ac1dc3740 1 --2- 172.21.15.111:0/2689728636 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5652e3c01040 0x5652e3c03430 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.621+0000 7f8ac1dc3740 1 --2- 172.21.15.111:0/2689728636 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5652e3b85c30 0x5652e3b90440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.621+0000 7f8ac1dc3740 1 --2- 172.21.15.111:0/2689728636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5652e3b80ba0 0x5652e3b8f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.621+0000 7f8aa2000640 0 client.25745 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:55:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.621+0000 7f8ac1dc3740 1 --2- 172.21.15.111:0/2689728636 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5652e3b80260 0x5652e3b8eef0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.621+0000 7f8ac1dc3740 1 -- 172.21.15.111:0/2689728636 >> 172.21.15.111:0/2689728636 conn(0x5652e3b7b8d0 msgr2=0x5652e3b8a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.621+0000 7f8ac1dc3740 1 -- 172.21.15.111:0/2689728636 shutdown_connections 2024-09-15T22:55:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.621+0000 7f8ac1dc3740 1 -- 172.21.15.111:0/2689728636 wait complete. 2024-09-15T22:55:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.621+0000 7f8ac1dc3740 20 client.25745 trim_cache size 0 max 16384 2024-09-15T22:55:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:55:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:55:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:55:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:55:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.622+0000 7f8abda00640 1 -- 172.21.15.111:0/2082552114 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8a9c0a3240 msgr2=0x7f8a9c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.622+0000 7f8abda00640 1 --2- 172.21.15.111:0/2082552114 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8a9c0a3240 0x7f8a9c0a5700 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8ab00049d0 tx=0x7f8ab0009500 comp rx=0 tx=0).stop 2024-09-15T22:55:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.622+0000 7f8abda00640 1 -- 172.21.15.111:0/2082552114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ab8104d60 msgr2=0x7f8ab819a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.622+0000 7f8abda00640 1 --2- 172.21.15.111:0/2082552114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ab8104d60 0x7f8ab819a3f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400e9f0 tx=0x7f8aa400eeb0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.622+0000 7f8abda00640 1 -- 172.21.15.111:0/2082552114 shutdown_connections 2024-09-15T22:55:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.622+0000 7f8abda00640 1 --2- 172.21.15.111:0/2082552114 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8a9c0a3240 0x7f8a9c0a5700 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.622+0000 7f8abda00640 1 --2- 172.21.15.111:0/2082552114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ab8105720 0x7f8ab81a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.622+0000 7f8abda00640 1 --2- 172.21.15.111:0/2082552114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ab8104d60 0x7f8ab819a3f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.622+0000 7f8abda00640 1 --2- 172.21.15.111:0/2082552114 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ab8103b70 0x7f8ab8199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.622+0000 7f8abda00640 1 -- 172.21.15.111:0/2082552114 >> 172.21.15.111:0/2082552114 conn(0x7f8ab80ff2f0 msgr2=0x7f8ab81080d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.622+0000 7f8abda00640 1 -- 172.21.15.111:0/2082552114 shutdown_connections 2024-09-15T22:55:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:28.622+0000 7f8abda00640 1 -- 172.21.15.111:0/2082552114 wait complete. 2024-09-15T22:55:29.309 INFO:tasks.workunit.client.0.smithi111.stdout: 1 890124 9.89 MB/sec execute 357 sec latency 753.983 ms 2024-09-15T22:55:29.887 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:29 smithi111 ceph-mon[30629]: pgmap v637: 97 pgs: 97 active+clean; 283 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 189 op/s 2024-09-15T22:55:30.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:29 smithi121 ceph-mon[34815]: pgmap v637: 97 pgs: 97 active+clean; 283 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 189 op/s 2024-09-15T22:55:30.309 INFO:tasks.workunit.client.0.smithi111.stdout: 1 890461 9.87 MB/sec execute 358 sec latency 1659.304 ms 2024-09-15T22:55:30.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:29 smithi192 ceph-mon[35427]: pgmap v637: 97 pgs: 97 active+clean; 283 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 189 op/s 2024-09-15T22:55:31.309 INFO:tasks.workunit.client.0.smithi111.stdout: 1 892616 9.88 MB/sec execute 359 sec latency 19.465 ms 2024-09-15T22:55:32.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:31 smithi121 ceph-mon[34815]: pgmap v638: 97 pgs: 97 active+clean; 271 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 178 op/s 2024-09-15T22:55:32.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:31 smithi111 ceph-mon[30629]: pgmap v638: 97 pgs: 97 active+clean; 271 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 178 op/s 2024-09-15T22:55:32.309 INFO:tasks.workunit.client.0.smithi111.stdout: 1 895257 9.89 MB/sec execute 360 sec latency 9.032 ms 2024-09-15T22:55:32.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:31 smithi192 ceph-mon[35427]: pgmap v638: 97 pgs: 97 active+clean; 271 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 178 op/s 2024-09-15T22:55:33.309 INFO:tasks.workunit.client.0.smithi111.stdout: 1 898419 9.91 MB/sec execute 361 sec latency 13.844 ms 2024-09-15T22:55:33.887 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:33 smithi111 ceph-mon[30629]: pgmap v639: 97 pgs: 97 active+clean; 273 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 203 op/s 2024-09-15T22:55:34.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:33 smithi192 ceph-mon[35427]: pgmap v639: 97 pgs: 97 active+clean; 273 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 203 op/s 2024-09-15T22:55:34.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:33 smithi121 ceph-mon[34815]: pgmap v639: 97 pgs: 97 active+clean; 273 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 203 op/s 2024-09-15T22:55:34.313 INFO:tasks.workunit.client.0.smithi111.stdout: 1 898757 9.89 MB/sec execute 362 sec latency 720.889 ms 2024-09-15T22:55:35.313 INFO:tasks.workunit.client.0.smithi111.stdout: 1 899178 9.86 MB/sec execute 363 sec latency 1457.206 ms 2024-09-15T22:55:36.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:35 smithi121 ceph-mon[34815]: pgmap v640: 97 pgs: 97 active+clean; 271 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 202 op/s 2024-09-15T22:55:36.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:35 smithi111 ceph-mon[30629]: pgmap v640: 97 pgs: 97 active+clean; 271 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 202 op/s 2024-09-15T22:55:36.313 INFO:tasks.workunit.client.0.smithi111.stdout: 1 902143 9.88 MB/sec execute 364 sec latency 16.802 ms 2024-09-15T22:55:36.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:35 smithi192 ceph-mon[35427]: pgmap v640: 97 pgs: 97 active+clean; 271 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 202 op/s 2024-09-15T22:55:37.313 INFO:tasks.workunit.client.0.smithi111.stdout: 1 905354 9.91 MB/sec execute 365 sec latency 11.372 ms 2024-09-15T22:55:38.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:37 smithi121 ceph-mon[34815]: pgmap v641: 97 pgs: 97 active+clean; 272 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 191 op/s 2024-09-15T22:55:38.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:37 smithi111 ceph-mon[30629]: pgmap v641: 97 pgs: 97 active+clean; 272 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 191 op/s 2024-09-15T22:55:38.313 INFO:tasks.workunit.client.0.smithi111.stdout: 1 907618 9.90 MB/sec execute 366 sec latency 18.134 ms 2024-09-15T22:55:38.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:37 smithi192 ceph-mon[35427]: pgmap v641: 97 pgs: 97 active+clean; 272 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 191 op/s 2024-09-15T22:55:39.313 INFO:tasks.workunit.client.0.smithi111.stdout: 1 908868 9.90 MB/sec execute 367 sec latency 553.855 ms 2024-09-15T22:55:39.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:39 smithi111 ceph-mon[30629]: pgmap v642: 97 pgs: 97 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 217 op/s 2024-09-15T22:55:40.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:39 smithi121 ceph-mon[34815]: pgmap v642: 97 pgs: 97 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 217 op/s 2024-09-15T22:55:40.313 INFO:tasks.workunit.client.0.smithi111.stdout: 1 908868 9.88 MB/sec execute 368 sec latency 1553.905 ms 2024-09-15T22:55:40.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:39 smithi192 ceph-mon[35427]: pgmap v642: 97 pgs: 97 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 217 op/s 2024-09-15T22:55:41.313 INFO:tasks.workunit.client.0.smithi111.stdout: 1 909524 9.85 MB/sec execute 369 sec latency 1819.934 ms 2024-09-15T22:55:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.740+0000 7f09a1c00640 1 -- 172.21.15.111:0/4111286749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f099c103d60 msgr2=0x7f099c104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.740+0000 7f09a1c00640 1 --2- 172.21.15.111:0/4111286749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f099c103d60 0x7f099c104160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f098c009900 tx=0x7f098c02a410 comp rx=0 tx=0).stop 2024-09-15T22:55:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.740+0000 7f09a1c00640 1 -- 172.21.15.111:0/4111286749 shutdown_connections 2024-09-15T22:55:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.740+0000 7f09a1c00640 1 --2- 172.21.15.111:0/4111286749 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f099c105910 0x7f099c10c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.740+0000 7f09a1c00640 1 --2- 172.21.15.111:0/4111286749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f099c104f50 0x7f099c1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.740+0000 7f09a1c00640 1 --2- 172.21.15.111:0/4111286749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f099c103d60 0x7f099c104160 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.740+0000 7f09a1c00640 1 -- 172.21.15.111:0/4111286749 >> 172.21.15.111:0/4111286749 conn(0x7f099c0ff4e0 msgr2=0x7f099c101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.740+0000 7f09a1c00640 1 -- 172.21.15.111:0/4111286749 shutdown_connections 2024-09-15T22:55:41.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.740+0000 7f09a1c00640 1 -- 172.21.15.111:0/4111286749 wait complete. 2024-09-15T22:55:41.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.741+0000 7f09a1c00640 1 Processor -- start 2024-09-15T22:55:41.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.741+0000 7f09a1c00640 1 -- start start 2024-09-15T22:55:41.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.741+0000 7f09a1c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f099c103d60 0x7f099c199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:41.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.741+0000 7f09a1c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f099c104f50 0x7f099c19a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.741+0000 7f09a1c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f099c105910 0x7f099c1a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.741+0000 7f09a1c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f099c10f9e0 con 0x7f099c105910 2024-09-15T22:55:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.741+0000 7f09a1c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f099c10f860 con 0x7f099c103d60 2024-09-15T22:55:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.741+0000 7f09a1c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f099c10fb60 con 0x7f099c104f50 2024-09-15T22:55:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.742+0000 7f099aa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f099c103d60 0x7f099c199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.742+0000 7f099aa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f099c103d60 0x7f099c199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:51666/0 (socket says 172.21.15.111:51666) 2024-09-15T22:55:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.742+0000 7f099aa00640 1 -- 172.21.15.111:0/746250007 learned_addr learned my addr 172.21.15.111:0/746250007 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:55:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.742+0000 7f099b400640 1 --2- 172.21.15.111:0/746250007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f099c105910 0x7f099c1a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.742+0000 7f099a000640 1 --2- 172.21.15.111:0/746250007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f099c104f50 0x7f099c19a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:41.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.742+0000 7f099aa00640 1 -- 172.21.15.111:0/746250007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f099c104f50 msgr2=0x7f099c19a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.742+0000 7f099aa00640 1 --2- 172.21.15.111:0/746250007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f099c104f50 0x7f099c19a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.742+0000 7f099aa00640 1 -- 172.21.15.111:0/746250007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f099c105910 msgr2=0x7f099c1a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.742+0000 7f099aa00640 1 --2- 172.21.15.111:0/746250007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f099c105910 0x7f099c1a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.742+0000 7f099aa00640 1 -- 172.21.15.111:0/746250007 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f099c1a1c00 con 0x7f099c103d60 2024-09-15T22:55:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.742+0000 7f099a000640 1 --2- 172.21.15.111:0/746250007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f099c104f50 0x7f099c19a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:55:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.742+0000 7f099b400640 1 --2- 172.21.15.111:0/746250007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f099c105910 0x7f099c1a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:55:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.742+0000 7f099aa00640 1 --2- 172.21.15.111:0/746250007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f099c103d60 0x7f099c199fe0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f098c031610 tx=0x7f098c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.742+0000 7f0993400640 1 -- 172.21.15.111:0/746250007 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c03f030 con 0x7f099c103d60 2024-09-15T22:55:41.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.742+0000 7f0993400640 1 -- 172.21.15.111:0/746250007 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f098c003ad0 con 0x7f099c103d60 2024-09-15T22:55:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.743+0000 7f09a1c00640 1 -- 172.21.15.111:0/746250007 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f099c1a1e90 con 0x7f099c103d60 2024-09-15T22:55:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.743+0000 7f0993400640 1 -- 172.21.15.111:0/746250007 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c035070 con 0x7f099c103d60 2024-09-15T22:55:41.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.743+0000 7f09a1c00640 1 -- 172.21.15.111:0/746250007 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f099c1a22a0 con 0x7f099c103d60 2024-09-15T22:55:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.745+0000 7f09a4bf6740 1 Processor -- start 2024-09-15T22:55:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.746+0000 7f09a4bf6740 1 -- start start 2024-09-15T22:55:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.746+0000 7f0993400640 1 -- 172.21.15.111:0/746250007 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f098c042030 con 0x7f099c103d60 2024-09-15T22:55:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.746+0000 7f09a4bf6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a090 0x55d9a950a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.746+0000 7f09a4bf6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9a950a9d0 0x55d9a950f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.746+0000 7f09a4bf6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9a950fa60 0x55d9a9511f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.746+0000 7f09a4bf6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d9a946f100 con 0x55d9a950fa60 2024-09-15T22:55:41.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.746+0000 7f09a4bf6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d9a94171b0 con 0x55d9a950a090 2024-09-15T22:55:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.746+0000 7f09a4bf6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d9a946b0c0 con 0x55d9a950a9d0 2024-09-15T22:55:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.746+0000 7f099b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a090 0x55d9a950a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.746+0000 7f099b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a090 0x55d9a950a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:51676/0 (socket says 172.21.15.111:51676) 2024-09-15T22:55:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.746+0000 7f099b400640 1 -- 172.21.15.111:0/2662339906 learned_addr learned my addr 172.21.15.111:0/2662339906 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:55:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.746+0000 7f099a000640 1 --2- 172.21.15.111:0/2662339906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9a950fa60 0x55d9a9511f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.746+0000 7f099aa00640 1 --2- 172.21.15.111:0/2662339906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9a950a9d0 0x55d9a950f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.746+0000 7f099b400640 1 -- 172.21.15.111:0/2662339906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9a950a9d0 msgr2=0x55d9a950f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.746+0000 7f099b400640 1 --2- 172.21.15.111:0/2662339906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9a950a9d0 0x55d9a950f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.746+0000 7f099b400640 1 -- 172.21.15.111:0/2662339906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9a950fa60 msgr2=0x55d9a9511f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.746+0000 7f099b400640 1 --2- 172.21.15.111:0/2662339906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9a950fa60 0x55d9a9511f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.746+0000 7f099b400640 1 -- 172.21.15.111:0/2662339906 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9a9182210 con 0x55d9a950a090 2024-09-15T22:55:41.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.747+0000 7f099b400640 1 --2- 172.21.15.111:0/2662339906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a090 0x55d9a950a490 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f09940117e0 tx=0x7f0994011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=924c6280e07edde6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.747+0000 7f0987e00640 1 -- 172.21.15.111:0/2662339906 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0994011310 con 0x55d9a950a090 2024-09-15T22:55:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.747+0000 7f0987e00640 1 -- 172.21.15.111:0/2662339906 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0994003e50 con 0x55d9a950a090 2024-09-15T22:55:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.747+0000 7f0987e00640 1 -- 172.21.15.111:0/2662339906 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0994004560 con 0x55d9a950a090 2024-09-15T22:55:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.747+0000 7f0993400640 1 --2- 172.21.15.111:0/746250007 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f09800a32f0 0x7f09800a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.747+0000 7f09a4bf6740 1 -- 172.21.15.111:0/2662339906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a090 msgr2=0x55d9a950a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.748+0000 7f09a4bf6740 1 --2- 172.21.15.111:0/2662339906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a090 0x55d9a950a490 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f09940117e0 tx=0x7f0994011ca0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.747+0000 7f0993400640 1 -- 172.21.15.111:0/746250007 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f098c0efed0 con 0x7f099c103d60 2024-09-15T22:55:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.748+0000 7f099aa00640 1 --2- 172.21.15.111:0/746250007 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f09800a32f0 0x7f09800a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.748+0000 7f09a4bf6740 1 -- 172.21.15.111:0/2662339906 shutdown_connections 2024-09-15T22:55:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.748+0000 7f09a4bf6740 1 --2- 172.21.15.111:0/2662339906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9a950fa60 0x55d9a9511f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.748+0000 7f09a4bf6740 1 --2- 172.21.15.111:0/2662339906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9a950a9d0 0x55d9a950f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.748+0000 7f09a4bf6740 1 --2- 172.21.15.111:0/2662339906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a090 0x55d9a950a490 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.748+0000 7f09a4bf6740 1 -- 172.21.15.111:0/2662339906 >> 172.21.15.111:0/2662339906 conn(0x55d9a9505700 msgr2=0x55d9a9505b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.748+0000 7f09a4bf6740 1 -- 172.21.15.111:0/2662339906 shutdown_connections 2024-09-15T22:55:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.748+0000 7f09a4bf6740 1 -- 172.21.15.111:0/2662339906 wait complete. 2024-09-15T22:55:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.748+0000 7f099aa00640 1 --2- 172.21.15.111:0/746250007 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f09800a32f0 0x7f09800a57b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f098c0eddc0 tx=0x7f098c02a410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.748+0000 7f09a4bf6740 1 -- start start 2024-09-15T22:55:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.749+0000 7f09a4bf6740 1 Processor -- start 2024-09-15T22:55:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.749+0000 7f09a4bf6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9a950a090 0x55d9a9514240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.749+0000 7f09a4bf6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a9d0 0x55d9a9514780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.749+0000 7f09a4bf6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9a950fa60 0x55d9a951b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.749+0000 7f09a4bf6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d9a9512a30 con 0x55d9a950fa60 2024-09-15T22:55:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.749+0000 7f09a4bf6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d9a93f8090 con 0x55d9a950a9d0 2024-09-15T22:55:41.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.749+0000 7f09a4bf6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d9a9512bb0 con 0x55d9a950a090 2024-09-15T22:55:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.749+0000 7f09a4bf6740 10 client.0 _ll_register_callbacks cb 0x55d9a944cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:55:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.749+0000 7f099a000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a9d0 0x55d9a9514780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.749+0000 7f099a000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a9d0 0x55d9a9514780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:51690/0 (socket says 172.21.15.111:51690) 2024-09-15T22:55:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.749+0000 7f099a000640 1 -- 172.21.15.111:0/2955236698 learned_addr learned my addr 172.21.15.111:0/2955236698 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:55:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.749+0000 7f099b400640 1 --2- 172.21.15.111:0/2955236698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9a950fa60 0x55d9a951b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.750+0000 7f099b400640 1 --2- 172.21.15.111:0/2955236698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9a950a090 0x55d9a9514240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.750+0000 7f099a000640 1 -- 172.21.15.111:0/2955236698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9a950a090 msgr2=0x55d9a9514240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.750+0000 7f099a000640 1 --2- 172.21.15.111:0/2955236698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9a950a090 0x55d9a9514240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.750+0000 7f099a000640 1 -- 172.21.15.111:0/2955236698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9a950fa60 msgr2=0x55d9a951b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.750+0000 7f099a000640 1 --2- 172.21.15.111:0/2955236698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9a950fa60 0x55d9a951b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.750+0000 7f099a000640 1 -- 172.21.15.111:0/2955236698 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9a924e620 con 0x55d9a950a9d0 2024-09-15T22:55:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.750+0000 7f099b400640 1 --2- 172.21.15.111:0/2955236698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9a950a090 0x55d9a9514240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:55:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.750+0000 7f099a000640 1 --2- 172.21.15.111:0/2955236698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a9d0 0x55d9a9514780 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f09880126b0 tx=0x7f0988012b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31f89631a6b7755d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.750+0000 7f0984c00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:55:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.750+0000 7f0984c00640 1 -- 172.21.15.111:0/2955236698 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0988011c70 con 0x55d9a950a9d0 2024-09-15T22:55:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.750+0000 7f09a4bf6740 1 -- 172.21.15.111:0/2955236698 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0994003e50 con 0x55d9a950a9d0 2024-09-15T22:55:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.750+0000 7f0984c00640 1 -- 172.21.15.111:0/2955236698 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f0988011e10 con 0x55d9a950a9d0 2024-09-15T22:55:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.750+0000 7f0984c00640 1 -- 172.21.15.111:0/2955236698 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f098800f460 con 0x55d9a950a9d0 2024-09-15T22:55:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.750+0000 7f09a4bf6740 10 client.25757 fetch_fsmap learned FSMap version 30 2024-09-15T22:55:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.750+0000 7f09a4bf6740 1 -- 172.21.15.111:0/2955236698 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d9a951e6d0 con 0x55d9a950a9d0 2024-09-15T22:55:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.751+0000 7f0984c00640 1 -- 172.21.15.111:0/2955236698 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0994003e50 con 0x55d9a950a9d0 2024-09-15T22:55:41.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.751+0000 7f0984c00640 1 -- 172.21.15.111:0/2955236698 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f098801c740 con 0x55d9a950a9d0 2024-09-15T22:55:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.751+0000 7f09a4bf6740 10 client.25757 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:55:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.751+0000 7f09a4bf6740 10 client.25757 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:55:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.751+0000 7f09a4bf6740 20 client.25757 populate_metadata read hostname 'smithi111' 2024-09-15T22:55:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.751+0000 7f09a4bf6740 1 --2- 172.21.15.111:0/2955236698 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d9a951fdd0 0x55d9a9522230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.751+0000 7f09a4bf6740 4 client.25757 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:55:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.751+0000 7f09a4bf6740 1 -- 172.21.15.111:0/2955236698 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9a947c830 con 0x55d9a951fdd0 2024-09-15T22:55:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.751+0000 7f099b400640 1 --2- 172.21.15.111:0/2955236698 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d9a951fdd0 0x55d9a9522230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.752+0000 7f099b400640 1 --2- 172.21.15.111:0/2955236698 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d9a951fdd0 0x55d9a9522230 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84a61b075816fd server_cookie=5a5c62c9e5434fad in_seq=0 out_seq=0 2024-09-15T22:55:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.752+0000 7f0984c00640 10 client.25757 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:55:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.753+0000 7f0984c00640 1 -- 172.21.15.111:0/2955236698 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d9a947c830 con 0x55d9a951fdd0 2024-09-15T22:55:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.753+0000 7f0984c00640 10 client.25757 handle_command_reply: tid=0 2024-09-15T22:55:41.761 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.753+0000 7f09a4bf6740 1 client.25757 shutdown 2024-09-15T22:55:41.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.753+0000 7f09a4bf6740 1 -- 172.21.15.111:0/2955236698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a9d0 msgr2=0x55d9a9514780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:41.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.753+0000 7f09a4bf6740 1 --2- 172.21.15.111:0/2955236698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a9d0 0x55d9a9514780 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f09880126b0 tx=0x7f0988012b70 comp rx=0 tx=0).stop 2024-09-15T22:55:41.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.753+0000 7f09a4bf6740 1 -- 172.21.15.111:0/2955236698 shutdown_connections 2024-09-15T22:55:41.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.753+0000 7f09a4bf6740 1 --2- 172.21.15.111:0/2955236698 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d9a951fdd0 0x55d9a9522230 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.753+0000 7f09a4bf6740 1 --2- 172.21.15.111:0/2955236698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9a950fa60 0x55d9a951b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.753+0000 7f09a4bf6740 1 --2- 172.21.15.111:0/2955236698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a9d0 0x55d9a9514780 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.753+0000 7f0984c00640 0 client.25757 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:55:41.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.753+0000 7f09a4bf6740 1 --2- 172.21.15.111:0/2955236698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9a950a090 0x55d9a9514240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.753+0000 7f09a4bf6740 1 -- 172.21.15.111:0/2955236698 >> 172.21.15.111:0/2955236698 conn(0x55d9a9505700 msgr2=0x55d9a9506f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:41.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.753+0000 7f09a4bf6740 1 -- 172.21.15.111:0/2955236698 shutdown_connections 2024-09-15T22:55:41.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.753+0000 7f09a4bf6740 1 -- 172.21.15.111:0/2955236698 wait complete. 2024-09-15T22:55:41.763 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.753+0000 7f09a4bf6740 20 client.25757 trim_cache size 0 max 16384 2024-09-15T22:55:41.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.768+0000 7f09a4bf6740 1 Processor -- start 2024-09-15T22:55:41.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.768+0000 7f09a4bf6740 1 -- start start 2024-09-15T22:55:41.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.768+0000 7f09a4bf6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9a950a090 0x55d9a9506af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:41.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.768+0000 7f09a4bf6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a9d0 0x55d9a951b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:41.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.768+0000 7f09a4bf6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9a950fa60 0x55d9a9507030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:41.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.768+0000 7f09a4bf6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f099c1ad960 con 0x55d9a950a090 2024-09-15T22:55:41.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.768+0000 7f09a4bf6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f099c1ab230 con 0x55d9a950a9d0 2024-09-15T22:55:41.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.768+0000 7f09a4bf6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f099c1a9080 con 0x55d9a950fa60 2024-09-15T22:55:41.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.768+0000 7f099b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9a950a090 0x55d9a9506af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.768+0000 7f099a000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a9d0 0x55d9a951b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.768+0000 7f099a000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a9d0 0x55d9a951b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:51700/0 (socket says 172.21.15.111:51700) 2024-09-15T22:55:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.768+0000 7f099a000640 1 -- 172.21.15.111:0/2240292657 learned_addr learned my addr 172.21.15.111:0/2240292657 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:55:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.769+0000 7f099b400640 1 --2- 172.21.15.111:0/2240292657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9a950fa60 0x55d9a9507030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.769+0000 7f099a000640 1 -- 172.21.15.111:0/2240292657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9a950fa60 msgr2=0x55d9a9507030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.769+0000 7f099a000640 1 --2- 172.21.15.111:0/2240292657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9a950fa60 0x55d9a9507030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.769+0000 7f099a000640 1 -- 172.21.15.111:0/2240292657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9a950a090 msgr2=0x55d9a9506af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.769+0000 7f099a000640 1 --2- 172.21.15.111:0/2240292657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9a950a090 0x55d9a9506af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.769+0000 7f099a000640 1 -- 172.21.15.111:0/2240292657 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9a95132b0 con 0x55d9a950a9d0 2024-09-15T22:55:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.769+0000 7f099a000640 1 --2- 172.21.15.111:0/2240292657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a9d0 0x55d9a951b780 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0988001eb0 tx=0x7f09880118b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=503c665d19d886c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.769+0000 7f0987400640 1 -- 172.21.15.111:0/2240292657 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098802c070 con 0x55d9a950a9d0 2024-09-15T22:55:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.769+0000 7f0987400640 1 -- 172.21.15.111:0/2240292657 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0988009680 con 0x55d9a950a9d0 2024-09-15T22:55:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.769+0000 7f0987400640 1 -- 172.21.15.111:0/2240292657 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09880055d0 con 0x55d9a950a9d0 2024-09-15T22:55:41.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.770+0000 7f09a4bf6740 1 -- 172.21.15.111:0/2240292657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a9d0 msgr2=0x55d9a951b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.770+0000 7f09a4bf6740 1 --2- 172.21.15.111:0/2240292657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a9d0 0x55d9a951b780 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0988001eb0 tx=0x7f09880118b0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.770+0000 7f09a4bf6740 1 -- 172.21.15.111:0/2240292657 shutdown_connections 2024-09-15T22:55:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.770+0000 7f09a4bf6740 1 --2- 172.21.15.111:0/2240292657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9a950fa60 0x55d9a9507030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.770+0000 7f09a4bf6740 1 --2- 172.21.15.111:0/2240292657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a9d0 0x55d9a951b780 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.770+0000 7f09a4bf6740 1 --2- 172.21.15.111:0/2240292657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9a950a090 0x55d9a9506af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.770+0000 7f09a4bf6740 1 -- 172.21.15.111:0/2240292657 >> 172.21.15.111:0/2240292657 conn(0x55d9a9505700 msgr2=0x55d9a9504880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.770+0000 7f09a4bf6740 1 -- 172.21.15.111:0/2240292657 shutdown_connections 2024-09-15T22:55:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.770+0000 7f09a4bf6740 1 -- 172.21.15.111:0/2240292657 wait complete. 2024-09-15T22:55:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.770+0000 7f09a4bf6740 1 -- start start 2024-09-15T22:55:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.770+0000 7f09a4bf6740 1 Processor -- start 2024-09-15T22:55:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.771+0000 7f09a4bf6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a090 0x55d9a9518f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.771+0000 7f09a4bf6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9a950a9d0 0x55d9a95194a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:41.777 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.771+0000 7f09a4bf6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9a950fa60 0x55d9a951a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.771+0000 7f09a4bf6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f099c1a96f0 con 0x55d9a950a9d0 2024-09-15T22:55:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.771+0000 7f09a4bf6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f099c1aab60 con 0x55d9a950a090 2024-09-15T22:55:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.771+0000 7f09a4bf6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f099c1a9dc0 con 0x55d9a950fa60 2024-09-15T22:55:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.771+0000 7f09a4bf6740 10 client.0 _ll_register_callbacks cb 0x7f0988009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:55:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.771+0000 7f099b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a090 0x55d9a9518f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.771+0000 7f099b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a090 0x55d9a9518f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:51706/0 (socket says 172.21.15.111:51706) 2024-09-15T22:55:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.771+0000 7f099b400640 1 -- 172.21.15.111:0/842761534 learned_addr learned my addr 172.21.15.111:0/842761534 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:55:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.771+0000 7f099a000640 1 --2- 172.21.15.111:0/842761534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9a950a9d0 0x55d9a95194a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.771+0000 7f099b400640 1 --2- 172.21.15.111:0/842761534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9a950fa60 0x55d9a951a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.771+0000 7f099b400640 1 -- 172.21.15.111:0/842761534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9a950fa60 msgr2=0x55d9a951a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.771+0000 7f099b400640 1 --2- 172.21.15.111:0/842761534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9a950fa60 0x55d9a951a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.771+0000 7f099b400640 1 -- 172.21.15.111:0/842761534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9a950a9d0 msgr2=0x55d9a95194a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:41.778 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.771+0000 7f099b400640 1 --2- 172.21.15.111:0/842761534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9a950a9d0 0x55d9a95194a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.771+0000 7f099b400640 1 -- 172.21.15.111:0/842761534 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9a9504f90 con 0x55d9a950a090 2024-09-15T22:55:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.772+0000 7f099b400640 1 --2- 172.21.15.111:0/842761534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a090 0x55d9a9518f60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0994010010 tx=0x7f0994005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26ab21503f843c16 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.772+0000 7f0984c00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:55:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.772+0000 7f0984c00640 1 -- 172.21.15.111:0/842761534 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099402e030 con 0x55d9a950a090 2024-09-15T22:55:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.772+0000 7f0984c00640 1 -- 172.21.15.111:0/842761534 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f099400fdb0 con 0x55d9a950a090 2024-09-15T22:55:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.772+0000 7f09a4bf6740 1 -- 172.21.15.111:0/842761534 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0988009680 con 0x55d9a950a090 2024-09-15T22:55:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.772+0000 7f0984c00640 1 -- 172.21.15.111:0/842761534 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0994025040 con 0x55d9a950a090 2024-09-15T22:55:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.772+0000 7f09a4bf6740 10 client.25763 fetch_fsmap learned FSMap version 30 2024-09-15T22:55:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.772+0000 7f09a4bf6740 1 -- 172.21.15.111:0/842761534 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d9a958aad0 con 0x55d9a950a090 2024-09-15T22:55:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.773+0000 7f0984c00640 1 -- 172.21.15.111:0/842761534 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0988009680 con 0x55d9a950a090 2024-09-15T22:55:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.773+0000 7f0984c00640 1 -- 172.21.15.111:0/842761534 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f0994018020 con 0x55d9a950a090 2024-09-15T22:55:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.773+0000 7f09a4bf6740 10 client.25763 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:55:41.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.773+0000 7f09a4bf6740 10 client.25763 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:55:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.773+0000 7f09a4bf6740 20 client.25763 populate_metadata read hostname 'smithi111' 2024-09-15T22:55:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.773+0000 7f09a4bf6740 1 --2- 172.21.15.111:0/842761534 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d9a958b290 0x55d9a958d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.773+0000 7f09a4bf6740 4 client.25763 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:55:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.773+0000 7f09a4bf6740 1 -- 172.21.15.111:0/842761534 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d9a9476b80 con 0x55d9a958b290 2024-09-15T22:55:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.773+0000 7f099a000640 1 --2- 172.21.15.111:0/842761534 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d9a958b290 0x55d9a958d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.774+0000 7f099a000640 1 --2- 172.21.15.111:0/842761534 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d9a958b290 0x55d9a958d6b0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa28628e7a1f3b6d server_cookie=8ecbc31b8bbfd677 in_seq=0 out_seq=0 2024-09-15T22:55:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.774+0000 7f0984c00640 10 client.25763 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:55:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.784+0000 7f0984c00640 1 -- 172.21.15.111:0/842761534 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d9a9476b80 con 0x55d9a958b290 2024-09-15T22:55:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.784+0000 7f0984c00640 10 client.25763 handle_command_reply: tid=0 2024-09-15T22:55:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.784+0000 7f09a4bf6740 1 client.25763 shutdown 2024-09-15T22:55:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.784+0000 7f09a4bf6740 1 -- 172.21.15.111:0/842761534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a090 msgr2=0x55d9a9518f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.784+0000 7f09a4bf6740 1 --2- 172.21.15.111:0/842761534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a090 0x55d9a9518f60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0994010010 tx=0x7f0994005bd0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.784+0000 7f09a4bf6740 1 -- 172.21.15.111:0/842761534 shutdown_connections 2024-09-15T22:55:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.784+0000 7f09a4bf6740 1 --2- 172.21.15.111:0/842761534 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d9a958b290 0x55d9a958d6b0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.784+0000 7f09a4bf6740 1 --2- 172.21.15.111:0/842761534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d9a950fa60 0x55d9a951a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.784+0000 7f09a4bf6740 1 --2- 172.21.15.111:0/842761534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d9a950a9d0 0x55d9a95194a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.784+0000 7f09a4bf6740 1 --2- 172.21.15.111:0/842761534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9a950a090 0x55d9a9518f60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.784+0000 7f0984c00640 0 client.25763 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:55:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.784+0000 7f09a4bf6740 1 -- 172.21.15.111:0/842761534 >> 172.21.15.111:0/842761534 conn(0x55d9a9505700 msgr2=0x55d9a9505ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.785+0000 7f09a4bf6740 1 -- 172.21.15.111:0/842761534 shutdown_connections 2024-09-15T22:55:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.785+0000 7f09a4bf6740 1 -- 172.21.15.111:0/842761534 wait complete. 2024-09-15T22:55:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.785+0000 7f09a4bf6740 20 client.25763 trim_cache size 0 max 16384 2024-09-15T22:55:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:55:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:55:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:55:41.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:55:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.785+0000 7f09a1c00640 1 -- 172.21.15.111:0/746250007 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f09800a32f0 msgr2=0x7f09800a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.785+0000 7f09a1c00640 1 --2- 172.21.15.111:0/746250007 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f09800a32f0 0x7f09800a57b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f098c0eddc0 tx=0x7f098c02a410 comp rx=0 tx=0).stop 2024-09-15T22:55:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.785+0000 7f09a1c00640 1 -- 172.21.15.111:0/746250007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f099c103d60 msgr2=0x7f099c199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:41.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.785+0000 7f09a1c00640 1 --2- 172.21.15.111:0/746250007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f099c103d60 0x7f099c199fe0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f098c031610 tx=0x7f098c004270 comp rx=0 tx=0).stop 2024-09-15T22:55:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.785+0000 7f09a1c00640 1 -- 172.21.15.111:0/746250007 shutdown_connections 2024-09-15T22:55:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.785+0000 7f09a1c00640 1 --2- 172.21.15.111:0/746250007 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f099c105910 0x7f099c1a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.785+0000 7f09a1c00640 1 --2- 172.21.15.111:0/746250007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f099c104f50 0x7f099c19a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.785+0000 7f09a1c00640 1 --2- 172.21.15.111:0/746250007 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f09800a32f0 0x7f09800a57b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.785+0000 7f09a1c00640 1 --2- 172.21.15.111:0/746250007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f099c103d60 0x7f099c199fe0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.785+0000 7f09a1c00640 1 -- 172.21.15.111:0/746250007 >> 172.21.15.111:0/746250007 conn(0x7f099c0ff4e0 msgr2=0x7f099c108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.786+0000 7f09a1c00640 1 -- 172.21.15.111:0/746250007 shutdown_connections 2024-09-15T22:55:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:41.786+0000 7f09a1c00640 1 -- 172.21.15.111:0/746250007 wait complete. 2024-09-15T22:55:42.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:41 smithi121 ceph-mon[34815]: pgmap v643: 97 pgs: 97 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 222 op/s 2024-09-15T22:55:42.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:41 smithi111 ceph-mon[30629]: pgmap v643: 97 pgs: 97 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 222 op/s 2024-09-15T22:55:42.313 INFO:tasks.workunit.client.0.smithi111.stdout: 1 912621 9.88 MB/sec execute 370 sec latency 13.369 ms 2024-09-15T22:55:42.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:41 smithi192 ceph-mon[35427]: pgmap v643: 97 pgs: 97 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 222 op/s 2024-09-15T22:55:43.313 INFO:tasks.workunit.client.0.smithi111.stdout: 1 915398 9.89 MB/sec execute 371 sec latency 12.840 ms 2024-09-15T22:55:44.111 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:43 smithi111 ceph-mon[30629]: pgmap v644: 97 pgs: 97 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 187 op/s 2024-09-15T22:55:44.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:43 smithi121 ceph-mon[34815]: pgmap v644: 97 pgs: 97 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 187 op/s 2024-09-15T22:55:44.313 INFO:tasks.workunit.client.0.smithi111.stdout: 1 916385 9.88 MB/sec execute 372 sec latency 382.568 ms 2024-09-15T22:55:44.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:43 smithi192 ceph-mon[35427]: pgmap v644: 97 pgs: 97 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 187 op/s 2024-09-15T22:55:44.985 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:44 smithi111 ceph-mon[30629]: pgmap v645: 97 pgs: 97 active+clean; 278 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 191 op/s 2024-09-15T22:55:45.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:44 smithi121 ceph-mon[34815]: pgmap v645: 97 pgs: 97 active+clean; 278 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 191 op/s 2024-09-15T22:55:45.313 INFO:tasks.workunit.client.0.smithi111.stdout: 1 916385 9.85 MB/sec execute 373 sec latency 1382.613 ms 2024-09-15T22:55:45.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:44 smithi192 ceph-mon[35427]: pgmap v645: 97 pgs: 97 active+clean; 278 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 191 op/s 2024-09-15T22:55:46.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:45 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:55:46.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:45 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:55:46.314 INFO:tasks.workunit.client.0.smithi111.stdout: 1 918084 9.89 MB/sec execute 374 sec latency 1504.358 ms 2024-09-15T22:55:46.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:45 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:55:47.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:55:47.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:55:47.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:55:47.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:46 smithi121 ceph-mon[34815]: pgmap v646: 97 pgs: 97 active+clean; 276 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 197 op/s 2024-09-15T22:55:47.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:55:47.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:55:47.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:55:47.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:46 smithi111 ceph-mon[30629]: pgmap v646: 97 pgs: 97 active+clean; 276 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 197 op/s 2024-09-15T22:55:47.313 INFO:tasks.workunit.client.0.smithi111.stdout: 1 920980 9.91 MB/sec execute 375 sec latency 14.685 ms 2024-09-15T22:55:47.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:55:47.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:55:47.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:55:47.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:46 smithi192 ceph-mon[35427]: pgmap v646: 97 pgs: 97 active+clean; 276 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 197 op/s 2024-09-15T22:55:48.314 INFO:tasks.workunit.client.0.smithi111.stdout: 1 924191 9.93 MB/sec execute 376 sec latency 13.276 ms 2024-09-15T22:55:49.314 INFO:tasks.workunit.client.0.smithi111.stdout: 1 926199 9.92 MB/sec execute 377 sec latency 308.384 ms 2024-09-15T22:55:49.812 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:49 smithi111 ceph-mon[30629]: pgmap v647: 97 pgs: 97 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 237 op/s 2024-09-15T22:55:50.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:49 smithi192 ceph-mon[35427]: pgmap v647: 97 pgs: 97 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 237 op/s 2024-09-15T22:55:50.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:49 smithi121 ceph-mon[34815]: pgmap v647: 97 pgs: 97 active+clean; 277 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 237 op/s 2024-09-15T22:55:50.314 INFO:tasks.workunit.client.0.smithi111.stdout: 1 926208 9.90 MB/sec execute 378 sec latency 890.581 ms 2024-09-15T22:55:51.314 INFO:tasks.workunit.client.0.smithi111.stdout: 1 928223 9.90 MB/sec execute 379 sec latency 499.168 ms 2024-09-15T22:55:52.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:51 smithi192 ceph-mon[35427]: pgmap v648: 97 pgs: 97 active+clean; 273 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 185 op/s 2024-09-15T22:55:52.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:51 smithi121 ceph-mon[34815]: pgmap v648: 97 pgs: 97 active+clean; 273 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 185 op/s 2024-09-15T22:55:52.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:51 smithi111 ceph-mon[30629]: pgmap v648: 97 pgs: 97 active+clean; 273 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 185 op/s 2024-09-15T22:55:52.314 INFO:tasks.workunit.client.0.smithi111.stdout: 1 931459 9.93 MB/sec execute 380 sec latency 6.425 ms 2024-09-15T22:55:53.314 INFO:tasks.workunit.client.0.smithi111.stdout: 1 934703 9.95 MB/sec execute 381 sec latency 14.176 ms 2024-09-15T22:55:54.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:53 smithi192 ceph-mon[35427]: pgmap v649: 97 pgs: 97 active+clean; 264 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 182 op/s 2024-09-15T22:55:54.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:53 smithi121 ceph-mon[34815]: pgmap v649: 97 pgs: 97 active+clean; 264 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 182 op/s 2024-09-15T22:55:54.156 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:53 smithi111 ceph-mon[30629]: pgmap v649: 97 pgs: 97 active+clean; 264 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 182 op/s 2024-09-15T22:55:54.314 INFO:tasks.workunit.client.0.smithi111.stdout: 1 936909 9.94 MB/sec execute 382 sec latency 183.897 ms 2024-09-15T22:55:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.914+0000 7f1bfea00640 1 -- 172.21.15.111:0/1084265028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bf8102d50 msgr2=0x7f1bf81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.914+0000 7f1bfea00640 1 --2- 172.21.15.111:0/1084265028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bf8102d50 0x7f1bf81031d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1be4009900 tx=0x7f1be402a410 comp rx=0 tx=0).stop 2024-09-15T22:55:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.914+0000 7f1bfea00640 1 -- 172.21.15.111:0/1084265028 shutdown_connections 2024-09-15T22:55:54.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.914+0000 7f1bfea00640 1 --2- 172.21.15.111:0/1084265028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bf8103710 0x7f1bf8109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.914+0000 7f1bfea00640 1 --2- 172.21.15.111:0/1084265028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bf8102d50 0x7f1bf81031d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.914+0000 7f1bfea00640 1 --2- 172.21.15.111:0/1084265028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1bf8101b60 0x7f1bf8101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.914+0000 7f1bfea00640 1 -- 172.21.15.111:0/1084265028 >> 172.21.15.111:0/1084265028 conn(0x7f1bf80fd2e0 msgr2=0x7f1bf80ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.919+0000 7f1bfea00640 1 -- 172.21.15.111:0/1084265028 shutdown_connections 2024-09-15T22:55:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.919+0000 7f1bfea00640 1 -- 172.21.15.111:0/1084265028 wait complete. 2024-09-15T22:55:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.919+0000 7f1bfea00640 1 Processor -- start 2024-09-15T22:55:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.919+0000 7f1bfea00640 1 -- start start 2024-09-15T22:55:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.919+0000 7f1bfea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bf8101b60 0x7f1bf8197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.919+0000 7f1bfea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1bf8102d50 0x7f1bf8198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.919+0000 7f1bfea00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bf8103710 0x7f1bf819f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.919+0000 7f1bfea00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf810d810 con 0x7f1bf8102d50 2024-09-15T22:55:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.919+0000 7f1bfea00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf810d690 con 0x7f1bf8101b60 2024-09-15T22:55:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.919+0000 7f1bfea00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf810d990 con 0x7f1bf8103710 2024-09-15T22:55:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1bfca00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bf8103710 0x7f1bf819f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:54.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1bf7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1bf8102d50 0x7f1bf8198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:54.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1bfca00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bf8103710 0x7f1bf819f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:44222/0 (socket says 172.21.15.111:44222) 2024-09-15T22:55:54.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1bfca00640 1 -- 172.21.15.111:0/2992700757 learned_addr learned my addr 172.21.15.111:0/2992700757 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:55:54.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1bf7e00640 1 --2- 172.21.15.111:0/2992700757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bf8101b60 0x7f1bf8197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:54.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1bfca00640 1 -- 172.21.15.111:0/2992700757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bf8101b60 msgr2=0x7f1bf8197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:54.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1bfca00640 1 --2- 172.21.15.111:0/2992700757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bf8101b60 0x7f1bf8197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1bfca00640 1 -- 172.21.15.111:0/2992700757 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1bf8102d50 msgr2=0x7f1bf8198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:54.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1bfca00640 1 --2- 172.21.15.111:0/2992700757 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1bf8102d50 0x7f1bf8198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1bfca00640 1 -- 172.21.15.111:0/2992700757 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bf819fa00 con 0x7f1bf8103710 2024-09-15T22:55:54.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1bfca00640 1 --2- 172.21.15.111:0/2992700757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bf8103710 0x7f1bf819f390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000eb10 tx=0x7f1bf000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:54.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1bf4c00640 1 -- 172.21.15.111:0/2992700757 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0004b20 con 0x7f1bf8103710 2024-09-15T22:55:54.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1bf4c00640 1 -- 172.21.15.111:0/2992700757 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bf0004cc0 con 0x7f1bf8103710 2024-09-15T22:55:54.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1bf4c00640 1 -- 172.21.15.111:0/2992700757 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0010760 con 0x7f1bf8103710 2024-09-15T22:55:54.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1bfea00640 1 -- 172.21.15.111:0/2992700757 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bf819fcf0 con 0x7f1bf8103710 2024-09-15T22:55:54.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1bfea00640 1 -- 172.21.15.111:0/2992700757 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bf81a01c0 con 0x7f1bf8103710 2024-09-15T22:55:54.925 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1c0186e740 1 Processor -- start 2024-09-15T22:55:54.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1bf4c00640 1 -- 172.21.15.111:0/2992700757 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f1bf0005020 con 0x7f1bf8103710 2024-09-15T22:55:54.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1c0186e740 1 -- start start 2024-09-15T22:55:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1c0186e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619fa087090 0x5619fa087490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1c0186e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619fa0879d0 0x5619fa08c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1c0186e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5619fa08ca60 0x5619fa08ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1c0186e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5619f9fec100 con 0x5619fa08ca60 2024-09-15T22:55:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1c0186e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5619f9f941b0 con 0x5619fa0879d0 2024-09-15T22:55:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1c0186e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5619f9fe80c0 con 0x5619fa087090 2024-09-15T22:55:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1bf7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619fa0879d0 0x5619fa08c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1bf7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619fa0879d0 0x5619fa08c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:38506/0 (socket says 172.21.15.111:38506) 2024-09-15T22:55:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1bf7e00640 1 -- 172.21.15.111:0/4175773184 learned_addr learned my addr 172.21.15.111:0/4175773184 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:55:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1bf7400640 1 --2- 172.21.15.111:0/4175773184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619fa087090 0x5619fa087490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:54.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.921+0000 7f1bf7400640 1 --2- 172.21.15.111:0/4175773184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5619fa08ca60 0x5619fa08ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1bf7e00640 1 -- 172.21.15.111:0/4175773184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619fa087090 msgr2=0x5619fa087490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1bf7e00640 1 --2- 172.21.15.111:0/4175773184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619fa087090 0x5619fa087490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1bf7e00640 1 -- 172.21.15.111:0/4175773184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5619fa08ca60 msgr2=0x5619fa08ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1bf7e00640 1 --2- 172.21.15.111:0/4175773184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5619fa08ca60 0x5619fa08ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1bf7e00640 1 -- 172.21.15.111:0/4175773184 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619f9cff210 con 0x5619fa0879d0 2024-09-15T22:55:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1bf7400640 1 --2- 172.21.15.111:0/4175773184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5619fa08ca60 0x5619fa08ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:55:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1bf7e00640 1 --2- 172.21.15.111:0/4175773184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619fa0879d0 0x5619fa08c490 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1be8009ca0 tx=0x7f1be8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97b78fe749b57696 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1bf4c00640 1 --2- 172.21.15.111:0/2992700757 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1bdc0a32f0 0x7f1bdc0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1bed200640 1 -- 172.21.15.111:0/4175773184 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be8004220 con 0x5619fa0879d0 2024-09-15T22:55:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1bed200640 1 -- 172.21.15.111:0/4175773184 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1be80043c0 con 0x5619fa0879d0 2024-09-15T22:55:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1bed200640 1 -- 172.21.15.111:0/4175773184 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be8019a10 con 0x5619fa0879d0 2024-09-15T22:55:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1bf4c00640 1 -- 172.21.15.111:0/2992700757 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f1bf00cfec0 con 0x7f1bf8103710 2024-09-15T22:55:54.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1bf7400640 1 --2- 172.21.15.111:0/2992700757 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1bdc0a32f0 0x7f1bdc0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1c0186e740 1 -- 172.21.15.111:0/4175773184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619fa0879d0 msgr2=0x5619fa08c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1c0186e740 1 --2- 172.21.15.111:0/4175773184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619fa0879d0 0x5619fa08c490 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1be8009ca0 tx=0x7f1be8011810 comp rx=0 tx=0).stop 2024-09-15T22:55:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1c0186e740 1 -- 172.21.15.111:0/4175773184 shutdown_connections 2024-09-15T22:55:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1c0186e740 1 --2- 172.21.15.111:0/4175773184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5619fa08ca60 0x5619fa08ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1c0186e740 1 --2- 172.21.15.111:0/4175773184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619fa0879d0 0x5619fa08c490 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1c0186e740 1 --2- 172.21.15.111:0/4175773184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619fa087090 0x5619fa087490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1c0186e740 1 -- 172.21.15.111:0/4175773184 >> 172.21.15.111:0/4175773184 conn(0x5619fa082700 msgr2=0x5619fa082b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1c0186e740 1 -- 172.21.15.111:0/4175773184 shutdown_connections 2024-09-15T22:55:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1bf7400640 1 --2- 172.21.15.111:0/2992700757 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1bdc0a32f0 0x7f1bdc0a57b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1be4031440 tx=0x7f1be40315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:54.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1c0186e740 1 -- 172.21.15.111:0/4175773184 wait complete. 2024-09-15T22:55:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1c0186e740 1 -- start start 2024-09-15T22:55:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1c0186e740 1 Processor -- start 2024-09-15T22:55:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1c0186e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5619fa087090 0x5619fa0955f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1c0186e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619fa0879d0 0x5619fa095b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1c0186e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619fa08ca60 0x5619fa09cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1c0186e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5619fa08fa30 con 0x5619fa087090 2024-09-15T22:55:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1c0186e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5619f9f75090 con 0x5619fa0879d0 2024-09-15T22:55:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1c0186e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5619fa08fbb0 con 0x5619fa08ca60 2024-09-15T22:55:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1c0186e740 10 client.0 _ll_register_callbacks cb 0x5619f9fc9f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:55:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1bf7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5619fa087090 0x5619fa0955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1bfca00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619fa08ca60 0x5619fa09cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1bf7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5619fa087090 0x5619fa0955f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:35760/0 (socket says 172.21.15.111:35760) 2024-09-15T22:55:54.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1bf7e00640 1 -- 172.21.15.111:0/3861264127 learned_addr learned my addr 172.21.15.111:0/3861264127 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:55:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1bf7400640 1 --2- 172.21.15.111:0/3861264127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619fa0879d0 0x5619fa095b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1bfca00640 1 -- 172.21.15.111:0/3861264127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619fa0879d0 msgr2=0x5619fa095b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1bfca00640 1 --2- 172.21.15.111:0/3861264127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619fa0879d0 0x5619fa095b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1bfca00640 1 -- 172.21.15.111:0/3861264127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5619fa087090 msgr2=0x5619fa0955f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1bfca00640 1 --2- 172.21.15.111:0/3861264127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5619fa087090 0x5619fa0955f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1bfca00640 1 -- 172.21.15.111:0/3861264127 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619f9dcb620 con 0x5619fa08ca60 2024-09-15T22:55:54.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1bfca00640 1 --2- 172.21.15.111:0/3861264127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619fa08ca60 0x5619fa09cb70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1bf00cd9c0 tx=0x7f1bf009c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e066711949978a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1be1600640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:55:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1be1600640 1 -- 172.21.15.111:0/3861264127 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf00a4ec0 con 0x5619fa08ca60 2024-09-15T22:55:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1be1600640 1 -- 172.21.15.111:0/3861264127 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f1bf0005310 con 0x5619fa08ca60 2024-09-15T22:55:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1c0186e740 1 -- 172.21.15.111:0/3861264127 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1be80043c0 con 0x5619fa08ca60 2024-09-15T22:55:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1be1600640 1 -- 172.21.15.111:0/3861264127 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1bf009c4e0 con 0x5619fa08ca60 2024-09-15T22:55:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1c0186e740 10 client.25798 fetch_fsmap learned FSMap version 30 2024-09-15T22:55:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1c0186e740 1 -- 172.21.15.111:0/3861264127 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5619fa09fa20 con 0x5619fa08ca60 2024-09-15T22:55:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1be1600640 1 -- 172.21.15.111:0/3861264127 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1be80043c0 con 0x5619fa08ca60 2024-09-15T22:55:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1be1600640 1 -- 172.21.15.111:0/3861264127 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f1bf00b5320 con 0x5619fa08ca60 2024-09-15T22:55:54.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1c0186e740 10 client.25798 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:55:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1c0186e740 10 client.25798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:55:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1c0186e740 20 client.25798 populate_metadata read hostname 'smithi111' 2024-09-15T22:55:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1c0186e740 1 --2- 172.21.15.111:0/3861264127 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5619fa0a10f0 0x5619fa0a3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1c0186e740 4 client.25798 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:55:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1c0186e740 1 -- 172.21.15.111:0/3861264127 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619f9ff9830 con 0x5619fa0a10f0 2024-09-15T22:55:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.926+0000 7f1bf7e00640 1 --2- 172.21.15.111:0/3861264127 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5619fa0a10f0 0x5619fa0a3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.931+0000 7f1bf7e00640 1 --2- 172.21.15.111:0/3861264127 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5619fa0a10f0 0x5619fa0a3520 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d265dd1571975105 server_cookie=d27a9497917be2ae in_seq=0 out_seq=0 2024-09-15T22:55:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.931+0000 7f1be1600640 10 client.25798 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:55:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.931+0000 7f1be1600640 1 -- 172.21.15.111:0/3861264127 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5619f9ff9830 con 0x5619fa0a10f0 2024-09-15T22:55:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.931+0000 7f1be1600640 10 client.25798 handle_command_reply: tid=0 2024-09-15T22:55:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.931+0000 7f1c0186e740 1 client.25798 shutdown 2024-09-15T22:55:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.931+0000 7f1c0186e740 1 -- 172.21.15.111:0/3861264127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619fa08ca60 msgr2=0x5619fa09cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:54.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.931+0000 7f1c0186e740 1 --2- 172.21.15.111:0/3861264127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619fa08ca60 0x5619fa09cb70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1bf00cd9c0 tx=0x7f1bf009c370 comp rx=0 tx=0).stop 2024-09-15T22:55:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.933+0000 7f1c0186e740 1 -- 172.21.15.111:0/3861264127 shutdown_connections 2024-09-15T22:55:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.933+0000 7f1c0186e740 1 --2- 172.21.15.111:0/3861264127 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5619fa0a10f0 0x5619fa0a3520 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.933+0000 7f1c0186e740 1 --2- 172.21.15.111:0/3861264127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619fa08ca60 0x5619fa09cb70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.933+0000 7f1c0186e740 1 --2- 172.21.15.111:0/3861264127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619fa0879d0 0x5619fa095b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.933+0000 7f1c0186e740 1 --2- 172.21.15.111:0/3861264127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5619fa087090 0x5619fa0955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.933+0000 7f1c0186e740 1 -- 172.21.15.111:0/3861264127 >> 172.21.15.111:0/3861264127 conn(0x5619fa082700 msgr2=0x5619fa083f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.933+0000 7f1be1600640 0 client.25798 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:55:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.933+0000 7f1c0186e740 1 -- 172.21.15.111:0/3861264127 shutdown_connections 2024-09-15T22:55:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.933+0000 7f1c0186e740 1 -- 172.21.15.111:0/3861264127 wait complete. 2024-09-15T22:55:54.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.933+0000 7f1c0186e740 20 client.25798 trim_cache size 0 max 16384 2024-09-15T22:55:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1c0186e740 1 Processor -- start 2024-09-15T22:55:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1c0186e740 1 -- start start 2024-09-15T22:55:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1c0186e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619fa087090 0x5619fa082ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1c0186e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5619fa0879d0 0x5619fa09cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1c0186e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619fa08ca60 0x5619fa083020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1c0186e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf81a93c0 con 0x5619fa0879d0 2024-09-15T22:55:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1c0186e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf81a6c90 con 0x5619fa08ca60 2024-09-15T22:55:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1c0186e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf81a4ae0 con 0x5619fa087090 2024-09-15T22:55:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1bfca00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619fa08ca60 0x5619fa083020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1bf7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5619fa0879d0 0x5619fa09cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1bfca00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619fa08ca60 0x5619fa083020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:38526/0 (socket says 172.21.15.111:38526) 2024-09-15T22:55:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1bfca00640 1 -- 172.21.15.111:0/2685960997 learned_addr learned my addr 172.21.15.111:0/2685960997 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:55:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1bf7e00640 1 --2- 172.21.15.111:0/2685960997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619fa087090 0x5619fa082ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1bfca00640 1 -- 172.21.15.111:0/2685960997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619fa087090 msgr2=0x5619fa082ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1bfca00640 1 --2- 172.21.15.111:0/2685960997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619fa087090 0x5619fa082ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1bfca00640 1 -- 172.21.15.111:0/2685960997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5619fa0879d0 msgr2=0x5619fa09cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1bfca00640 1 --2- 172.21.15.111:0/2685960997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5619fa0879d0 0x5619fa09cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1bfca00640 1 -- 172.21.15.111:0/2685960997 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619fa0a0e90 con 0x5619fa08ca60 2024-09-15T22:55:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1bf7400640 1 --2- 172.21.15.111:0/2685960997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5619fa0879d0 0x5619fa09cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:55:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1bfca00640 1 --2- 172.21.15.111:0/2685960997 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619fa08ca60 0x5619fa083020 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1bf00049d0 tx=0x7f1bf009a9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b8f272038695c01 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1be3e00640 1 -- 172.21.15.111:0/2685960997 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf007d070 con 0x5619fa08ca60 2024-09-15T22:55:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1be3e00640 1 -- 172.21.15.111:0/2685960997 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bf00a0890 con 0x5619fa08ca60 2024-09-15T22:55:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1be3e00640 1 -- 172.21.15.111:0/2685960997 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf00b5200 con 0x5619fa08ca60 2024-09-15T22:55:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1c0186e740 1 -- 172.21.15.111:0/2685960997 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619fa08ca60 msgr2=0x5619fa083020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1c0186e740 1 --2- 172.21.15.111:0/2685960997 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619fa08ca60 0x5619fa083020 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1bf00049d0 tx=0x7f1bf009a9f0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1c0186e740 1 -- 172.21.15.111:0/2685960997 shutdown_connections 2024-09-15T22:55:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1c0186e740 1 --2- 172.21.15.111:0/2685960997 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619fa08ca60 0x5619fa083020 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1c0186e740 1 --2- 172.21.15.111:0/2685960997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5619fa0879d0 0x5619fa09cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1c0186e740 1 --2- 172.21.15.111:0/2685960997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619fa087090 0x5619fa082ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1c0186e740 1 -- 172.21.15.111:0/2685960997 >> 172.21.15.111:0/2685960997 conn(0x5619fa082700 msgr2=0x5619fa08c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1c0186e740 1 -- 172.21.15.111:0/2685960997 shutdown_connections 2024-09-15T22:55:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1c0186e740 1 -- 172.21.15.111:0/2685960997 wait complete. 2024-09-15T22:55:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1c0186e740 1 -- start start 2024-09-15T22:55:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.944+0000 7f1c0186e740 1 Processor -- start 2024-09-15T22:55:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1c0186e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5619fa087090 0x5619fa099fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1c0186e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619fa0879d0 0x5619fa09a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1c0186e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619fa08ca60 0x5619fa09b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1c0186e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf81a5150 con 0x5619fa087090 2024-09-15T22:55:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1c0186e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf81a65c0 con 0x5619fa08ca60 2024-09-15T22:55:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1c0186e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1bf81a5820 con 0x5619fa0879d0 2024-09-15T22:55:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1c0186e740 10 client.0 _ll_register_callbacks cb 0x7f1bf0098780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:55:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1bf7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5619fa087090 0x5619fa099fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1bf7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5619fa087090 0x5619fa099fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:35790/0 (socket says 172.21.15.111:35790) 2024-09-15T22:55:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1bf7e00640 1 -- 172.21.15.111:0/2164332261 learned_addr learned my addr 172.21.15.111:0/2164332261 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:55:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1bf7400640 1 --2- 172.21.15.111:0/2164332261 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619fa0879d0 0x5619fa09a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1bfca00640 1 --2- 172.21.15.111:0/2164332261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619fa08ca60 0x5619fa09b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1bf7e00640 1 -- 172.21.15.111:0/2164332261 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619fa0879d0 msgr2=0x5619fa09a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1bf7e00640 1 --2- 172.21.15.111:0/2164332261 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619fa0879d0 0x5619fa09a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1bf7e00640 1 -- 172.21.15.111:0/2164332261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619fa08ca60 msgr2=0x5619fa09b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:55:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1bf7e00640 1 --2- 172.21.15.111:0/2164332261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619fa08ca60 0x5619fa09b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1bf7e00640 1 -- 172.21.15.111:0/2164332261 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619fa09bb20 con 0x5619fa087090 2024-09-15T22:55:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1bf7e00640 1 --2- 172.21.15.111:0/2164332261 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5619fa087090 0x5619fa099fc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1be8005fd0 tx=0x7f1be8019430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3a26b1009e1c85 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:55:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1be1600640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:55:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1be1600640 1 -- 172.21.15.111:0/2164332261 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1be802f020 con 0x5619fa087090 2024-09-15T22:55:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1be1600640 1 -- 172.21.15.111:0/2164332261 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f1be80052b0 con 0x5619fa087090 2024-09-15T22:55:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1c0186e740 1 -- 172.21.15.111:0/2164332261 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bf00a0890 con 0x5619fa087090 2024-09-15T22:55:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1be1600640 1 -- 172.21.15.111:0/2164332261 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1be80054b0 con 0x5619fa087090 2024-09-15T22:55:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1c0186e740 10 client.15801 fetch_fsmap learned FSMap version 30 2024-09-15T22:55:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1c0186e740 1 -- 172.21.15.111:0/2164332261 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5619fa09cea0 con 0x5619fa087090 2024-09-15T22:55:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1be1600640 1 -- 172.21.15.111:0/2164332261 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bf00a0890 con 0x5619fa087090 2024-09-15T22:55:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1be1600640 1 -- 172.21.15.111:0/2164332261 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f1be8019810 con 0x5619fa087090 2024-09-15T22:55:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1c0186e740 10 client.15801 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:55:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1c0186e740 10 client.15801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:55:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1c0186e740 20 client.15801 populate_metadata read hostname 'smithi111' 2024-09-15T22:55:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1c0186e740 1 --2- 172.21.15.111:0/2164332261 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5619fa10c370 0x5619fa10e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:55:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1c0186e740 4 client.15801 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:55:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1c0186e740 1 -- 172.21.15.111:0/2164332261 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5619f9ff3b80 con 0x5619fa10c370 2024-09-15T22:55:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1bf7400640 1 --2- 172.21.15.111:0/2164332261 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5619fa10c370 0x5619fa10e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:55:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1be1600640 10 client.15801 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:55:54.958 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.950+0000 7f1bf7400640 1 --2- 172.21.15.111:0/2164332261 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5619fa10c370 0x5619fa10e790 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a439ac2afbc87b13 server_cookie=10aa95989bdc9a54 in_seq=0 out_seq=0 2024-09-15T22:55:54.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.960+0000 7f1be1600640 1 -- 172.21.15.111:0/2164332261 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5619f9ff3b80 con 0x5619fa10c370 2024-09-15T22:55:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.960+0000 7f1be1600640 10 client.15801 handle_command_reply: tid=0 2024-09-15T22:55:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.960+0000 7f1c0186e740 1 client.15801 shutdown 2024-09-15T22:55:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.960+0000 7f1c0186e740 1 -- 172.21.15.111:0/2164332261 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5619fa087090 msgr2=0x5619fa099fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.960+0000 7f1c0186e740 1 --2- 172.21.15.111:0/2164332261 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5619fa087090 0x5619fa099fc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1be8005fd0 tx=0x7f1be8019430 comp rx=0 tx=0).stop 2024-09-15T22:55:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.960+0000 7f1c0186e740 1 -- 172.21.15.111:0/2164332261 shutdown_connections 2024-09-15T22:55:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.960+0000 7f1c0186e740 1 --2- 172.21.15.111:0/2164332261 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5619fa10c370 0x5619fa10e790 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.960+0000 7f1c0186e740 1 --2- 172.21.15.111:0/2164332261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5619fa08ca60 0x5619fa09b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.960+0000 7f1be1600640 0 client.15801 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:55:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.960+0000 7f1c0186e740 1 --2- 172.21.15.111:0/2164332261 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5619fa0879d0 0x5619fa09a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.960+0000 7f1c0186e740 1 --2- 172.21.15.111:0/2164332261 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5619fa087090 0x5619fa099fc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.960+0000 7f1c0186e740 1 -- 172.21.15.111:0/2164332261 >> 172.21.15.111:0/2164332261 conn(0x5619fa082700 msgr2=0x5619fa081060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.960+0000 7f1c0186e740 1 -- 172.21.15.111:0/2164332261 shutdown_connections 2024-09-15T22:55:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.960+0000 7f1c0186e740 1 -- 172.21.15.111:0/2164332261 wait complete. 2024-09-15T22:55:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.960+0000 7f1c0186e740 20 client.15801 trim_cache size 0 max 16384 2024-09-15T22:55:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:55:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:55:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:55:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:55:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.960+0000 7f1bfea00640 1 -- 172.21.15.111:0/2992700757 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1bdc0a32f0 msgr2=0x7f1bdc0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.960+0000 7f1bfea00640 1 --2- 172.21.15.111:0/2992700757 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1bdc0a32f0 0x7f1bdc0a57b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1be4031440 tx=0x7f1be40315b0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.960+0000 7f1bfea00640 1 -- 172.21.15.111:0/2992700757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bf8103710 msgr2=0x7f1bf819f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:55:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.960+0000 7f1bfea00640 1 --2- 172.21.15.111:0/2992700757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bf8103710 0x7f1bf819f390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000eb10 tx=0x7f1bf000c2d0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.960+0000 7f1bfea00640 1 -- 172.21.15.111:0/2992700757 shutdown_connections 2024-09-15T22:55:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.960+0000 7f1bfea00640 1 --2- 172.21.15.111:0/2992700757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bf8103710 0x7f1bf819f390 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.960+0000 7f1bfea00640 1 --2- 172.21.15.111:0/2992700757 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1bf8102d50 0x7f1bf8198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.960+0000 7f1bfea00640 1 --2- 172.21.15.111:0/2992700757 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1bdc0a32f0 0x7f1bdc0a57b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.960+0000 7f1bfea00640 1 --2- 172.21.15.111:0/2992700757 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1bf8101b60 0x7f1bf8197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:55:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.960+0000 7f1bfea00640 1 -- 172.21.15.111:0/2992700757 >> 172.21.15.111:0/2992700757 conn(0x7f1bf80fd2e0 msgr2=0x7f1bf8106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:55:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.960+0000 7f1bfea00640 1 -- 172.21.15.111:0/2992700757 shutdown_connections 2024-09-15T22:55:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:55:54.960+0000 7f1bfea00640 1 -- 172.21.15.111:0/2992700757 wait complete. 2024-09-15T22:55:55.252 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:55 smithi111 ceph-mon[30629]: pgmap v650: 97 pgs: 97 active+clean; 265 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 2.4 MiB/s wr, 181 op/s 2024-09-15T22:55:55.314 INFO:tasks.workunit.client.0.smithi111.stdout: 1 936912 9.92 MB/sec execute 383 sec latency 636.109 ms 2024-09-15T22:55:55.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:55 smithi192 ceph-mon[35427]: pgmap v650: 97 pgs: 97 active+clean; 265 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 2.4 MiB/s wr, 181 op/s 2024-09-15T22:55:55.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:55 smithi121 ceph-mon[34815]: pgmap v650: 97 pgs: 97 active+clean; 265 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 2.4 MiB/s wr, 181 op/s 2024-09-15T22:55:56.314 INFO:tasks.workunit.client.0.smithi111.stdout: 1 938178 9.92 MB/sec execute 384 sec latency 1149.758 ms 2024-09-15T22:55:57.314 INFO:tasks.workunit.client.0.smithi111.stdout: 1 940942 9.93 MB/sec execute 385 sec latency 12.017 ms 2024-09-15T22:55:58.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:57 smithi192 ceph-mon[35427]: pgmap v651: 97 pgs: 97 active+clean; 271 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 2.7 MiB/s wr, 193 op/s 2024-09-15T22:55:58.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:57 smithi121 ceph-mon[34815]: pgmap v651: 97 pgs: 97 active+clean; 271 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 2.7 MiB/s wr, 193 op/s 2024-09-15T22:55:58.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:57 smithi111 ceph-mon[30629]: pgmap v651: 97 pgs: 97 active+clean; 271 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 2.7 MiB/s wr, 193 op/s 2024-09-15T22:55:58.314 INFO:tasks.workunit.client.0.smithi111.stdout: 1 943541 9.94 MB/sec execute 386 sec latency 21.808 ms 2024-09-15T22:55:59.314 INFO:tasks.workunit.client.0.smithi111.stdout: 1 945895 9.95 MB/sec execute 387 sec latency 119.074 ms 2024-09-15T22:55:59.747 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:55:59 smithi111 ceph-mon[30629]: pgmap v652: 97 pgs: 97 active+clean; 272 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 2.9 MiB/s wr, 210 op/s 2024-09-15T22:56:00.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:55:59 smithi192 ceph-mon[35427]: pgmap v652: 97 pgs: 97 active+clean; 272 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 2.9 MiB/s wr, 210 op/s 2024-09-15T22:56:00.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:55:59 smithi121 ceph-mon[34815]: pgmap v652: 97 pgs: 97 active+clean; 272 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 697 KiB/s rd, 2.9 MiB/s wr, 210 op/s 2024-09-15T22:56:00.314 INFO:tasks.workunit.client.0.smithi111.stdout: 1 945896 9.92 MB/sec execute 388 sec latency 577.436 ms 2024-09-15T22:56:01.314 INFO:tasks.workunit.client.0.smithi111.stdout: 1 947596 9.91 MB/sec execute 389 sec latency 1017.797 ms 2024-09-15T22:56:02.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:01 smithi192 ceph-mon[35427]: pgmap v653: 97 pgs: 97 active+clean; 271 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 2.4 MiB/s wr, 171 op/s 2024-09-15T22:56:02.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:01 smithi121 ceph-mon[34815]: pgmap v653: 97 pgs: 97 active+clean; 271 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 2.4 MiB/s wr, 171 op/s 2024-09-15T22:56:02.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:01 smithi111 ceph-mon[30629]: pgmap v653: 97 pgs: 97 active+clean; 271 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 2.4 MiB/s wr, 171 op/s 2024-09-15T22:56:02.314 INFO:tasks.workunit.client.0.smithi111.stdout: 1 950105 9.93 MB/sec execute 390 sec latency 11.312 ms 2024-09-15T22:56:03.314 INFO:tasks.workunit.client.0.smithi111.stdout: 1 952815 9.94 MB/sec execute 391 sec latency 10.552 ms 2024-09-15T22:56:04.065 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:03 smithi192 ceph-mon[35427]: pgmap v654: 97 pgs: 97 active+clean; 272 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 2.8 MiB/s wr, 204 op/s 2024-09-15T22:56:04.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:03 smithi121 ceph-mon[34815]: pgmap v654: 97 pgs: 97 active+clean; 272 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 2.8 MiB/s wr, 204 op/s 2024-09-15T22:56:04.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:03 smithi111 ceph-mon[30629]: pgmap v654: 97 pgs: 97 active+clean; 272 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 2.8 MiB/s wr, 204 op/s 2024-09-15T22:56:04.315 INFO:tasks.workunit.client.0.smithi111.stdout: 1 955980 9.96 MB/sec execute 392 sec latency 20.360 ms 2024-09-15T22:56:05.315 INFO:tasks.workunit.client.0.smithi111.stdout: 1 955983 9.94 MB/sec execute 393 sec latency 795.028 ms 2024-09-15T22:56:06.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:05 smithi192 ceph-mon[35427]: pgmap v655: 97 pgs: 97 active+clean; 272 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 2.5 MiB/s wr, 200 op/s 2024-09-15T22:56:06.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:05 smithi121 ceph-mon[34815]: pgmap v655: 97 pgs: 97 active+clean; 272 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 2.5 MiB/s wr, 200 op/s 2024-09-15T22:56:06.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:05 smithi111 ceph-mon[30629]: pgmap v655: 97 pgs: 97 active+clean; 272 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 2.5 MiB/s wr, 200 op/s 2024-09-15T22:56:06.315 INFO:tasks.workunit.client.0.smithi111.stdout: 1 956964 9.92 MB/sec execute 394 sec latency 1183.143 ms 2024-09-15T22:56:07.315 INFO:tasks.workunit.client.0.smithi111.stdout: 1 959758 9.94 MB/sec execute 395 sec latency 8.449 ms 2024-09-15T22:56:08.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:07 smithi192 ceph-mon[35427]: pgmap v656: 97 pgs: 97 active+clean; 273 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 2.8 MiB/s wr, 225 op/s 2024-09-15T22:56:08.095 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.094+0000 7f8112a00640 1 -- 172.21.15.111:0/4018923155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f810c100210 msgr2=0x7f810c100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:08.095 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.094+0000 7f8112a00640 1 --2- 172.21.15.111:0/4018923155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f810c100210 0x7f810c100610 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8100009900 tx=0x7f810002a410 comp rx=0 tx=0).stop 2024-09-15T22:56:08.095 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.095+0000 7f8112a00640 1 -- 172.21.15.111:0/4018923155 shutdown_connections 2024-09-15T22:56:08.095 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.095+0000 7f8112a00640 1 --2- 172.21.15.111:0/4018923155 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f810c1014f0 0x7f810c1019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.095 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.095+0000 7f8112a00640 1 --2- 172.21.15.111:0/4018923155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f810c100b50 0x7f810c100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.096 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.095+0000 7f8112a00640 1 --2- 172.21.15.111:0/4018923155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f810c100210 0x7f810c100610 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.096 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.095+0000 7f8112a00640 1 -- 172.21.15.111:0/4018923155 >> 172.21.15.111:0/4018923155 conn(0x7f810c0fbfa0 msgr2=0x7f810c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:56:08.096 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.095+0000 7f8112a00640 1 -- 172.21.15.111:0/4018923155 shutdown_connections 2024-09-15T22:56:08.096 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.095+0000 7f8112a00640 1 -- 172.21.15.111:0/4018923155 wait complete. 2024-09-15T22:56:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.096+0000 7f8112a00640 1 Processor -- start 2024-09-15T22:56:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.096+0000 7f8112a00640 1 -- start start 2024-09-15T22:56:08.098 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.096+0000 7f8112a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f810c100210 0x7f810c199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.096+0000 7f8112a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f810c100b50 0x7f810c19a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.096+0000 7f8112a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f810c1014f0 0x7f810c1a1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.096+0000 7f8112a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f810c10f7a0 con 0x7f810c100210 2024-09-15T22:56:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.096+0000 7f8112a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f810c10f620 con 0x7f810c100b50 2024-09-15T22:56:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.096+0000 7f8112a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f810c10f920 con 0x7f810c1014f0 2024-09-15T22:56:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.096+0000 7f8110c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f810c100b50 0x7f810c19a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.096+0000 7f8110c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f810c100b50 0x7f810c19a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:45116/0 (socket says 172.21.15.111:45116) 2024-09-15T22:56:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.096+0000 7f8111600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f810c100210 0x7f810c199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.096+0000 7f8110c00640 1 -- 172.21.15.111:0/2837189486 learned_addr learned my addr 172.21.15.111:0/2837189486 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:56:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.097+0000 7f8112000640 1 --2- 172.21.15.111:0/2837189486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f810c1014f0 0x7f810c1a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.097+0000 7f8110c00640 1 -- 172.21.15.111:0/2837189486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f810c1014f0 msgr2=0x7f810c1a1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.097+0000 7f8110c00640 1 --2- 172.21.15.111:0/2837189486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f810c1014f0 0x7f810c1a1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.097+0000 7f8110c00640 1 -- 172.21.15.111:0/2837189486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f810c100210 msgr2=0x7f810c199e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.097+0000 7f8110c00640 1 --2- 172.21.15.111:0/2837189486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f810c100210 0x7f810c199e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.097+0000 7f8110c00640 1 -- 172.21.15.111:0/2837189486 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810c1a1a70 con 0x7f810c100b50 2024-09-15T22:56:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.097+0000 7f8112000640 1 --2- 172.21.15.111:0/2837189486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f810c1014f0 0x7f810c1a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:56:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.097+0000 7f8110c00640 1 --2- 172.21.15.111:0/2837189486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f810c100b50 0x7f810c19a390 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f80f800e9f0 tx=0x7f80f800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:56:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.097+0000 7f810a000640 1 -- 172.21.15.111:0/2837189486 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f800cd80 con 0x7f810c100b50 2024-09-15T22:56:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.097+0000 7f810a000640 1 -- 172.21.15.111:0/2837189486 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80f8005500 con 0x7f810c100b50 2024-09-15T22:56:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.097+0000 7f8112a00640 1 -- 172.21.15.111:0/2837189486 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f810c1a1d60 con 0x7f810c100b50 2024-09-15T22:56:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.097+0000 7f810a000640 1 -- 172.21.15.111:0/2837189486 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f8010690 con 0x7f810c100b50 2024-09-15T22:56:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.097+0000 7f8112a00640 1 -- 172.21.15.111:0/2837189486 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f810c1a2250 con 0x7f810c100b50 2024-09-15T22:56:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.101+0000 7f8116e82740 1 Processor -- start 2024-09-15T22:56:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.101+0000 7f8116e82740 1 -- start start 2024-09-15T22:56:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.101+0000 7f8116e82740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560fa1d97260 0x560fa1d97660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.101+0000 7f8116e82740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa1d97ba0 0x560fa1d9c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.101+0000 7f8116e82740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fa1d9cc30 0x560fa1d9f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.101+0000 7f8116e82740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560fa1cfc100 con 0x560fa1d97ba0 2024-09-15T22:56:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.101+0000 7f8116e82740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560fa1ca41b0 con 0x560fa1d9cc30 2024-09-15T22:56:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.101+0000 7f8116e82740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x560fa1cf80c0 con 0x560fa1d97260 2024-09-15T22:56:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.102+0000 7f8111600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa1d97ba0 0x560fa1d9c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.102+0000 7f8112000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560fa1d97260 0x560fa1d97660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.102+0000 7f8111600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa1d97ba0 0x560fa1d9c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33440/0 (socket says 172.21.15.111:33440) 2024-09-15T22:56:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.102+0000 7f8111600640 1 -- 172.21.15.111:0/2649640498 learned_addr learned my addr 172.21.15.111:0/2649640498 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:56:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.102+0000 7f810a000640 1 -- 172.21.15.111:0/2837189486 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f80f80046c0 con 0x7f810c100b50 2024-09-15T22:56:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.102+0000 7f8110c00640 1 --2- 172.21.15.111:0/2649640498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fa1d9cc30 0x560fa1d9f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.102+0000 7f8112000640 1 -- 172.21.15.111:0/2649640498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fa1d9cc30 msgr2=0x560fa1d9f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.102+0000 7f8112000640 1 --2- 172.21.15.111:0/2649640498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fa1d9cc30 0x560fa1d9f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.102+0000 7f8112000640 1 -- 172.21.15.111:0/2649640498 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa1d97ba0 msgr2=0x560fa1d9c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.102+0000 7f8112000640 1 --2- 172.21.15.111:0/2649640498 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa1d97ba0 0x560fa1d9c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.102+0000 7f8112000640 1 -- 172.21.15.111:0/2649640498 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fa1a0f210 con 0x560fa1d97260 2024-09-15T22:56:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.102+0000 7f8111600640 1 --2- 172.21.15.111:0/2649640498 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa1d97ba0 0x560fa1d9c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:56:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.102+0000 7f8110c00640 1 --2- 172.21.15.111:0/2649640498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fa1d9cc30 0x560fa1d9f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:56:08.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.102+0000 7f8112000640 1 --2- 172.21.15.111:0/2649640498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560fa1d97260 0x560fa1d97660 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8104009f90 tx=0x7f8104010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21f594f17c354871 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:56:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.103+0000 7f80fe600640 1 -- 172.21.15.111:0/2649640498 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8104004d20 con 0x560fa1d97260 2024-09-15T22:56:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.103+0000 7f80fe600640 1 -- 172.21.15.111:0/2649640498 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81040119e0 con 0x560fa1d97260 2024-09-15T22:56:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.103+0000 7f80fe600640 1 -- 172.21.15.111:0/2649640498 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81040055f0 con 0x560fa1d97260 2024-09-15T22:56:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.103+0000 7f8116e82740 1 -- 172.21.15.111:0/2649640498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560fa1d97260 msgr2=0x560fa1d97660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.104+0000 7f8116e82740 1 --2- 172.21.15.111:0/2649640498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560fa1d97260 0x560fa1d97660 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8104009f90 tx=0x7f8104010860 comp rx=0 tx=0).stop 2024-09-15T22:56:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.104+0000 7f810a000640 1 --2- 172.21.15.111:0/2837189486 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f80f40a3340 0x7f80f40a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.104+0000 7f8116e82740 1 -- 172.21.15.111:0/2649640498 shutdown_connections 2024-09-15T22:56:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.104+0000 7f8116e82740 1 --2- 172.21.15.111:0/2649640498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fa1d9cc30 0x560fa1d9f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.104+0000 7f8116e82740 1 --2- 172.21.15.111:0/2649640498 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa1d97ba0 0x560fa1d9c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.104+0000 7f8116e82740 1 --2- 172.21.15.111:0/2649640498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560fa1d97260 0x560fa1d97660 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.104+0000 7f810a000640 1 -- 172.21.15.111:0/2837189486 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f80f80cfd40 con 0x7f810c100b50 2024-09-15T22:56:08.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.104+0000 7f8116e82740 1 -- 172.21.15.111:0/2649640498 >> 172.21.15.111:0/2649640498 conn(0x560fa1d928d0 msgr2=0x560fa1d92cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:56:08.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.104+0000 7f8112000640 1 --2- 172.21.15.111:0/2837189486 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f80f40a3340 0x7f80f40a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:08.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.104+0000 7f8116e82740 1 -- 172.21.15.111:0/2649640498 shutdown_connections 2024-09-15T22:56:08.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.104+0000 7f8116e82740 1 -- 172.21.15.111:0/2649640498 wait complete. 2024-09-15T22:56:08.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.104+0000 7f8116e82740 1 -- start start 2024-09-15T22:56:08.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.105+0000 7f8112000640 1 --2- 172.21.15.111:0/2837189486 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f80f40a3340 0x7f80f40a5800 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8104011010 tx=0x7f810401d040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:56:08.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.105+0000 7f8116e82740 1 Processor -- start 2024-09-15T22:56:08.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.105+0000 7f8116e82740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fa1d97260 0x560fa1da1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:08.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.105+0000 7f8116e82740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560fa1d97ba0 0x560fa1da1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:08.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.105+0000 7f8116e82740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa1d9cc30 0x560fa1da89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:08.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.105+0000 7f8116e82740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560fa1d9fc00 con 0x560fa1d9cc30 2024-09-15T22:56:08.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.105+0000 7f8116e82740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560fa1c85090 con 0x560fa1d97260 2024-09-15T22:56:08.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.105+0000 7f8116e82740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x560fa1d9fd80 con 0x560fa1d97ba0 2024-09-15T22:56:08.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.105+0000 7f8116e82740 10 client.0 _ll_register_callbacks cb 0x560fa1cd9f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:56:08.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.105+0000 7f8111600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fa1d97260 0x560fa1da1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:08.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.105+0000 7f8111600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fa1d97260 0x560fa1da1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:45132/0 (socket says 172.21.15.111:45132) 2024-09-15T22:56:08.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.105+0000 7f8111600640 1 -- 172.21.15.111:0/4134941819 learned_addr learned my addr 172.21.15.111:0/4134941819 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:56:08.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.105+0000 7f8112000640 1 --2- 172.21.15.111:0/4134941819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa1d9cc30 0x560fa1da89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:08.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.105+0000 7f8110c00640 1 --2- 172.21.15.111:0/4134941819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560fa1d97ba0 0x560fa1da1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:08.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.106+0000 7f8111600640 1 -- 172.21.15.111:0/4134941819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560fa1d97ba0 msgr2=0x560fa1da1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.106+0000 7f8111600640 1 --2- 172.21.15.111:0/4134941819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560fa1d97ba0 0x560fa1da1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.106+0000 7f8111600640 1 -- 172.21.15.111:0/4134941819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa1d9cc30 msgr2=0x560fa1da89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.106+0000 7f8111600640 1 --2- 172.21.15.111:0/4134941819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa1d9cc30 0x560fa1da89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.106+0000 7f8111600640 1 -- 172.21.15.111:0/4134941819 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fa1adb620 con 0x560fa1d97260 2024-09-15T22:56:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.106+0000 7f8111600640 1 --2- 172.21.15.111:0/4134941819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fa1d97260 0x560fa1da1420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f81000314e0 tx=0x7f81000043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9f9528d3753c5e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:56:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.106+0000 7f80f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:56:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.106+0000 7f80f2a00640 1 -- 172.21.15.111:0/4134941819 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100035040 con 0x560fa1d97260 2024-09-15T22:56:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.106+0000 7f80f2a00640 1 -- 172.21.15.111:0/4134941819 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f8100005450 con 0x560fa1d97260 2024-09-15T22:56:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.106+0000 7f8116e82740 1 -- 172.21.15.111:0/4134941819 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81040119e0 con 0x560fa1d97260 2024-09-15T22:56:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.107+0000 7f80f2a00640 1 -- 172.21.15.111:0/4134941819 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8100005650 con 0x560fa1d97260 2024-09-15T22:56:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.107+0000 7f8116e82740 10 client.25790 fetch_fsmap learned FSMap version 30 2024-09-15T22:56:08.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.107+0000 7f8116e82740 1 -- 172.21.15.111:0/4134941819 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560fa1dab820 con 0x560fa1d97260 2024-09-15T22:56:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.107+0000 7f80f2a00640 1 -- 172.21.15.111:0/4134941819 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81040119e0 con 0x560fa1d97260 2024-09-15T22:56:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.107+0000 7f80f2a00640 1 -- 172.21.15.111:0/4134941819 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f8100031be0 con 0x560fa1d97260 2024-09-15T22:56:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.107+0000 7f8116e82740 10 client.25790 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:56:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.107+0000 7f8116e82740 10 client.25790 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:56:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.107+0000 7f8116e82740 20 client.25790 populate_metadata read hostname 'smithi111' 2024-09-15T22:56:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.107+0000 7f8116e82740 1 --2- 172.21.15.111:0/4134941819 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560fa1dacef0 0x560fa1daf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.107+0000 7f8116e82740 4 client.25790 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:56:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.107+0000 7f8116e82740 1 -- 172.21.15.111:0/4134941819 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fa1d09830 con 0x560fa1dacef0 2024-09-15T22:56:08.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.108+0000 7f8110c00640 1 --2- 172.21.15.111:0/4134941819 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560fa1dacef0 0x560fa1daf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:08.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.108+0000 7f8110c00640 1 --2- 172.21.15.111:0/4134941819 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560fa1dacef0 0x560fa1daf320 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=985cb9787db8ae29 server_cookie=f6ba6f55b8e10014 in_seq=0 out_seq=0 2024-09-15T22:56:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.108+0000 7f80f2a00640 10 client.25790 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:56:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.109+0000 7f80f2a00640 1 -- 172.21.15.111:0/4134941819 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560fa1d09830 con 0x560fa1dacef0 2024-09-15T22:56:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.109+0000 7f80f2a00640 10 client.25790 handle_command_reply: tid=0 2024-09-15T22:56:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.109+0000 7f8116e82740 1 client.25790 shutdown 2024-09-15T22:56:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.109+0000 7f8116e82740 1 -- 172.21.15.111:0/4134941819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fa1d97260 msgr2=0x560fa1da1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.109+0000 7f8116e82740 1 --2- 172.21.15.111:0/4134941819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fa1d97260 0x560fa1da1420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f81000314e0 tx=0x7f81000043b0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.109+0000 7f8116e82740 1 -- 172.21.15.111:0/4134941819 shutdown_connections 2024-09-15T22:56:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.109+0000 7f8116e82740 1 --2- 172.21.15.111:0/4134941819 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560fa1dacef0 0x560fa1daf320 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.109+0000 7f8116e82740 1 --2- 172.21.15.111:0/4134941819 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa1d9cc30 0x560fa1da89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.109+0000 7f80f2a00640 0 client.25790 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:56:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.109+0000 7f8116e82740 1 --2- 172.21.15.111:0/4134941819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560fa1d97ba0 0x560fa1da1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.109+0000 7f8116e82740 1 --2- 172.21.15.111:0/4134941819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fa1d97260 0x560fa1da1420 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.109+0000 7f8116e82740 1 -- 172.21.15.111:0/4134941819 >> 172.21.15.111:0/4134941819 conn(0x560fa1d928d0 msgr2=0x560fa1d940e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:56:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.109+0000 7f8116e82740 1 -- 172.21.15.111:0/4134941819 shutdown_connections 2024-09-15T22:56:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.110+0000 7f8116e82740 1 -- 172.21.15.111:0/4134941819 wait complete. 2024-09-15T22:56:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.110+0000 7f8116e82740 20 client.25790 trim_cache size 0 max 16384 2024-09-15T22:56:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.128+0000 7f8116e82740 1 Processor -- start 2024-09-15T22:56:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.128+0000 7f8116e82740 1 -- start start 2024-09-15T22:56:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.129+0000 7f8116e82740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fa1d928d0 0x560fa1d93750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.129+0000 7f8116e82740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa1d97260 0x560fa1da8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:08.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.129+0000 7f8116e82740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560fa1d97ba0 0x560fa1d93c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.129+0000 7f8116e82740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f810c1ad6a0 con 0x560fa1d97260 2024-09-15T22:56:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.129+0000 7f8116e82740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f810c1aaf70 con 0x560fa1d928d0 2024-09-15T22:56:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.129+0000 7f8116e82740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f810c1a8dc0 con 0x560fa1d97ba0 2024-09-15T22:56:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.129+0000 7f8111600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fa1d928d0 0x560fa1d93750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.129+0000 7f8111600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fa1d928d0 0x560fa1d93750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:45134/0 (socket says 172.21.15.111:45134) 2024-09-15T22:56:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.129+0000 7f8111600640 1 -- 172.21.15.111:0/1819259129 learned_addr learned my addr 172.21.15.111:0/1819259129 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:56:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.129+0000 7f8110c00640 1 --2- 172.21.15.111:0/1819259129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa1d97260 0x560fa1da8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.129+0000 7f8112000640 1 --2- 172.21.15.111:0/1819259129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560fa1d97ba0 0x560fa1d93c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.129+0000 7f8111600640 1 -- 172.21.15.111:0/1819259129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560fa1d97ba0 msgr2=0x560fa1d93c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.129+0000 7f8111600640 1 --2- 172.21.15.111:0/1819259129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560fa1d97ba0 0x560fa1d93c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.129+0000 7f8111600640 1 -- 172.21.15.111:0/1819259129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa1d97260 msgr2=0x560fa1da8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.129+0000 7f8111600640 1 --2- 172.21.15.111:0/1819259129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa1d97260 0x560fa1da8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.129+0000 7f8111600640 1 -- 172.21.15.111:0/1819259129 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fa1d98080 con 0x560fa1d928d0 2024-09-15T22:56:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.129+0000 7f8110c00640 1 --2- 172.21.15.111:0/1819259129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa1d97260 0x560fa1da8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:56:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.130+0000 7f8111600640 1 --2- 172.21.15.111:0/1819259129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fa1d928d0 0x560fa1d93750 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f810002aa00 tx=0x7f810002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c29c1a5822b0f79 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:56:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.130+0000 7f80fdc00640 1 -- 172.21.15.111:0/1819259129 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100035030 con 0x560fa1d928d0 2024-09-15T22:56:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.130+0000 7f80fdc00640 1 -- 172.21.15.111:0/1819259129 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8100041070 con 0x560fa1d928d0 2024-09-15T22:56:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.130+0000 7f80fdc00640 1 -- 172.21.15.111:0/1819259129 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100004d00 con 0x560fa1d928d0 2024-09-15T22:56:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.131+0000 7f8116e82740 1 -- 172.21.15.111:0/1819259129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fa1d928d0 msgr2=0x560fa1d93750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.131+0000 7f8116e82740 1 --2- 172.21.15.111:0/1819259129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fa1d928d0 0x560fa1d93750 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f810002aa00 tx=0x7f810002a410 comp rx=0 tx=0).stop 2024-09-15T22:56:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.131+0000 7f8116e82740 1 -- 172.21.15.111:0/1819259129 shutdown_connections 2024-09-15T22:56:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.131+0000 7f8116e82740 1 --2- 172.21.15.111:0/1819259129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560fa1d97ba0 0x560fa1d93c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.131+0000 7f8116e82740 1 --2- 172.21.15.111:0/1819259129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa1d97260 0x560fa1da8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.131+0000 7f8116e82740 1 --2- 172.21.15.111:0/1819259129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fa1d928d0 0x560fa1d93750 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.131+0000 7f8116e82740 1 -- 172.21.15.111:0/1819259129 >> 172.21.15.111:0/1819259129 conn(0x560fa1d9c750 msgr2=0x560fa1da11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:56:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.131+0000 7f8116e82740 1 -- 172.21.15.111:0/1819259129 shutdown_connections 2024-09-15T22:56:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.131+0000 7f8116e82740 1 -- 172.21.15.111:0/1819259129 wait complete. 2024-09-15T22:56:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.131+0000 7f8116e82740 1 -- start start 2024-09-15T22:56:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.131+0000 7f8116e82740 1 Processor -- start 2024-09-15T22:56:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.132+0000 7f8116e82740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fa1d928d0 0x560fa1da61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.132+0000 7f8116e82740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa1d97260 0x560fa1da6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.132+0000 7f8116e82740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560fa1d97ba0 0x560fa1da76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.132+0000 7f8116e82740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f810c1a9430 con 0x560fa1d97260 2024-09-15T22:56:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.132+0000 7f8116e82740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f810c1aa8a0 con 0x560fa1d928d0 2024-09-15T22:56:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.132+0000 7f8116e82740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f810c1a9b00 con 0x560fa1d97ba0 2024-09-15T22:56:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.132+0000 7f8116e82740 10 client.0 _ll_register_callbacks cb 0x7f8100009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:56:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.132+0000 7f8111600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fa1d928d0 0x560fa1da61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.132+0000 7f8111600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fa1d928d0 0x560fa1da61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:45148/0 (socket says 172.21.15.111:45148) 2024-09-15T22:56:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.132+0000 7f8111600640 1 -- 172.21.15.111:0/2174632092 learned_addr learned my addr 172.21.15.111:0/2174632092 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:56:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.132+0000 7f8112000640 1 --2- 172.21.15.111:0/2174632092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560fa1d97ba0 0x560fa1da76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.132+0000 7f8110c00640 1 --2- 172.21.15.111:0/2174632092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa1d97260 0x560fa1da6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.132+0000 7f8111600640 1 -- 172.21.15.111:0/2174632092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560fa1d97ba0 msgr2=0x560fa1da76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.132+0000 7f8111600640 1 --2- 172.21.15.111:0/2174632092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560fa1d97ba0 0x560fa1da76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.132+0000 7f8111600640 1 -- 172.21.15.111:0/2174632092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa1d97260 msgr2=0x560fa1da6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.132+0000 7f8111600640 1 --2- 172.21.15.111:0/2174632092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa1d97260 0x560fa1da6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.132+0000 7f8111600640 1 -- 172.21.15.111:0/2174632092 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fa1da7cf0 con 0x560fa1d928d0 2024-09-15T22:56:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.133+0000 7f8111600640 1 --2- 172.21.15.111:0/2174632092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fa1d928d0 0x560fa1da61e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f810002a930 tx=0x7f81000095a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffa2ffad4df78902 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:56:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.133+0000 7f80f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:56:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.133+0000 7f80f2a00640 1 -- 172.21.15.111:0/2174632092 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100066060 con 0x560fa1d928d0 2024-09-15T22:56:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.133+0000 7f80f2a00640 1 -- 172.21.15.111:0/2174632092 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f8100003b80 con 0x560fa1d928d0 2024-09-15T22:56:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.133+0000 7f8116e82740 1 -- 172.21.15.111:0/2174632092 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8100041070 con 0x560fa1d928d0 2024-09-15T22:56:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.133+0000 7f80f2a00640 1 -- 172.21.15.111:0/2174632092 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8100004e90 con 0x560fa1d928d0 2024-09-15T22:56:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.133+0000 7f8116e82740 10 client.25796 fetch_fsmap learned FSMap version 30 2024-09-15T22:56:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.133+0000 7f8116e82740 1 -- 172.21.15.111:0/2174632092 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560fa1e17d20 con 0x560fa1d928d0 2024-09-15T22:56:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.134+0000 7f80f2a00640 1 -- 172.21.15.111:0/2174632092 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8100041070 con 0x560fa1d928d0 2024-09-15T22:56:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.134+0000 7f80f2a00640 1 -- 172.21.15.111:0/2174632092 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f8100046470 con 0x560fa1d928d0 2024-09-15T22:56:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.134+0000 7f8116e82740 10 client.25796 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:56:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.134+0000 7f8116e82740 10 client.25796 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:56:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.134+0000 7f8116e82740 20 client.25796 populate_metadata read hostname 'smithi111' 2024-09-15T22:56:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.134+0000 7f8116e82740 1 --2- 172.21.15.111:0/2174632092 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560fa1d9cc30 0x560fa1e1a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.134+0000 7f8116e82740 4 client.25796 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:56:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.134+0000 7f8116e82740 1 -- 172.21.15.111:0/2174632092 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560fa1d03b80 con 0x560fa1d9cc30 2024-09-15T22:56:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.134+0000 7f8110c00640 1 --2- 172.21.15.111:0/2174632092 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560fa1d9cc30 0x560fa1e1a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.135+0000 7f80f2a00640 10 client.25796 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:56:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.135+0000 7f8110c00640 1 --2- 172.21.15.111:0/2174632092 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560fa1d9cc30 0x560fa1e1a620 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dfcb7863cf8a9db server_cookie=cbb03dc7109211f2 in_seq=0 out_seq=0 2024-09-15T22:56:08.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:07 smithi121 ceph-mon[34815]: pgmap v656: 97 pgs: 97 active+clean; 273 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 2.8 MiB/s wr, 225 op/s 2024-09-15T22:56:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.145+0000 7f80f2a00640 1 -- 172.21.15.111:0/2174632092 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560fa1d03b80 con 0x560fa1d9cc30 2024-09-15T22:56:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.145+0000 7f80f2a00640 10 client.25796 handle_command_reply: tid=0 2024-09-15T22:56:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.145+0000 7f8116e82740 1 client.25796 shutdown 2024-09-15T22:56:08.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.145+0000 7f8116e82740 1 -- 172.21.15.111:0/2174632092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fa1d928d0 msgr2=0x560fa1da61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.145+0000 7f8116e82740 1 --2- 172.21.15.111:0/2174632092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fa1d928d0 0x560fa1da61e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f810002a930 tx=0x7f81000095a0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.145+0000 7f8116e82740 1 -- 172.21.15.111:0/2174632092 shutdown_connections 2024-09-15T22:56:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.145+0000 7f8116e82740 1 --2- 172.21.15.111:0/2174632092 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560fa1d9cc30 0x560fa1e1a620 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.145+0000 7f8116e82740 1 --2- 172.21.15.111:0/2174632092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560fa1d97ba0 0x560fa1da76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.145+0000 7f80f2a00640 0 client.25796 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:56:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.145+0000 7f8116e82740 1 --2- 172.21.15.111:0/2174632092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560fa1d97260 0x560fa1da6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.145+0000 7f8116e82740 1 --2- 172.21.15.111:0/2174632092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560fa1d928d0 0x560fa1da61e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.145+0000 7f8116e82740 1 -- 172.21.15.111:0/2174632092 >> 172.21.15.111:0/2174632092 conn(0x560fa1d9c750 msgr2=0x560fa1d911f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:56:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.145+0000 7f8116e82740 1 -- 172.21.15.111:0/2174632092 shutdown_connections 2024-09-15T22:56:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.145+0000 7f8116e82740 1 -- 172.21.15.111:0/2174632092 wait complete. 2024-09-15T22:56:08.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.146+0000 7f8116e82740 20 client.25796 trim_cache size 0 max 16384 2024-09-15T22:56:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:56:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:56:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:56:08.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:56:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.146+0000 7f8112a00640 1 -- 172.21.15.111:0/2837189486 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f80f40a3340 msgr2=0x7f80f40a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.146+0000 7f8112a00640 1 --2- 172.21.15.111:0/2837189486 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f80f40a3340 0x7f80f40a5800 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8104011010 tx=0x7f810401d040 comp rx=0 tx=0).stop 2024-09-15T22:56:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.146+0000 7f8112a00640 1 -- 172.21.15.111:0/2837189486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f810c100b50 msgr2=0x7f810c19a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.146+0000 7f8112a00640 1 --2- 172.21.15.111:0/2837189486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f810c100b50 0x7f810c19a390 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f80f800e9f0 tx=0x7f80f800edf0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.146+0000 7f8112a00640 1 -- 172.21.15.111:0/2837189486 shutdown_connections 2024-09-15T22:56:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.146+0000 7f8112a00640 1 --2- 172.21.15.111:0/2837189486 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f810c1014f0 0x7f810c1a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.146+0000 7f8112a00640 1 --2- 172.21.15.111:0/2837189486 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f80f40a3340 0x7f80f40a5800 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.146+0000 7f8112a00640 1 --2- 172.21.15.111:0/2837189486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f810c100b50 0x7f810c19a390 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.146+0000 7f8112a00640 1 --2- 172.21.15.111:0/2837189486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f810c100210 0x7f810c199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.146+0000 7f8112a00640 1 -- 172.21.15.111:0/2837189486 >> 172.21.15.111:0/2837189486 conn(0x7f810c0fbfa0 msgr2=0x7f810c0fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:56:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.146+0000 7f8112a00640 1 -- 172.21.15.111:0/2837189486 shutdown_connections 2024-09-15T22:56:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:08.147+0000 7f8112a00640 1 -- 172.21.15.111:0/2837189486 wait complete. 2024-09-15T22:56:08.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:07 smithi111 ceph-mon[30629]: pgmap v656: 97 pgs: 97 active+clean; 273 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 2.8 MiB/s wr, 225 op/s 2024-09-15T22:56:08.315 INFO:tasks.workunit.client.0.smithi111.stdout: 1 962830 9.96 MB/sec execute 396 sec latency 11.563 ms 2024-09-15T22:56:09.315 INFO:tasks.workunit.client.0.smithi111.stdout: 1 965203 9.95 MB/sec execute 397 sec latency 17.661 ms 2024-09-15T22:56:10.006 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:09 smithi111 ceph-mon[30629]: pgmap v657: 97 pgs: 97 active+clean; 273 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 2.4 MiB/s wr, 179 op/s 2024-09-15T22:56:10.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:09 smithi192 ceph-mon[35427]: pgmap v657: 97 pgs: 97 active+clean; 273 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 2.4 MiB/s wr, 179 op/s 2024-09-15T22:56:10.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:09 smithi121 ceph-mon[34815]: pgmap v657: 97 pgs: 97 active+clean; 273 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 2.4 MiB/s wr, 179 op/s 2024-09-15T22:56:10.315 INFO:tasks.workunit.client.0.smithi111.stdout: 1 965749 9.94 MB/sec execute 398 sec latency 822.570 ms 2024-09-15T22:56:11.315 INFO:tasks.workunit.client.0.smithi111.stdout: 1 966043 9.92 MB/sec execute 399 sec latency 1285.582 ms 2024-09-15T22:56:12.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:11 smithi192 ceph-mon[35427]: pgmap v658: 97 pgs: 97 active+clean; 277 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 2.5 MiB/s wr, 200 op/s 2024-09-15T22:56:12.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:11 smithi121 ceph-mon[34815]: pgmap v658: 97 pgs: 97 active+clean; 277 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 2.5 MiB/s wr, 200 op/s 2024-09-15T22:56:12.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:11 smithi111 ceph-mon[30629]: pgmap v658: 97 pgs: 97 active+clean; 277 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 2.5 MiB/s wr, 200 op/s 2024-09-15T22:56:12.315 INFO:tasks.workunit.client.0.smithi111.stdout: 1 968869 9.93 MB/sec execute 400 sec latency 12.307 ms 2024-09-15T22:56:13.315 INFO:tasks.workunit.client.0.smithi111.stdout: 1 971700 9.94 MB/sec execute 401 sec latency 16.444 ms 2024-09-15T22:56:14.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:13 smithi192 ceph-mon[35427]: pgmap v659: 97 pgs: 97 active+clean; 280 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 2.7 MiB/s wr, 215 op/s 2024-09-15T22:56:14.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:13 smithi121 ceph-mon[34815]: pgmap v659: 97 pgs: 97 active+clean; 280 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 2.7 MiB/s wr, 215 op/s 2024-09-15T22:56:14.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:13 smithi111 ceph-mon[30629]: pgmap v659: 97 pgs: 97 active+clean; 280 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 2.7 MiB/s wr, 215 op/s 2024-09-15T22:56:14.315 INFO:tasks.workunit.client.0.smithi111.stdout: 1 974377 9.96 MB/sec execute 402 sec latency 15.153 ms 2024-09-15T22:56:15.316 INFO:tasks.workunit.client.0.smithi111.stdout: 1 975172 9.94 MB/sec execute 403 sec latency 570.415 ms 2024-09-15T22:56:16.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:15 smithi192 ceph-mon[35427]: pgmap v660: 97 pgs: 97 active+clean; 280 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 2.5 MiB/s wr, 169 op/s 2024-09-15T22:56:16.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:15 smithi121 ceph-mon[34815]: pgmap v660: 97 pgs: 97 active+clean; 280 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 2.5 MiB/s wr, 169 op/s 2024-09-15T22:56:16.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:15 smithi111 ceph-mon[30629]: pgmap v660: 97 pgs: 97 active+clean; 280 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 2.5 MiB/s wr, 169 op/s 2024-09-15T22:56:16.316 INFO:tasks.workunit.client.0.smithi111.stdout: 1 977012 9.95 MB/sec execute 404 sec latency 1109.843 ms 2024-09-15T22:56:17.316 INFO:tasks.workunit.client.0.smithi111.stdout: 1 979779 9.95 MB/sec execute 405 sec latency 11.160 ms 2024-09-15T22:56:18.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:17 smithi192 ceph-mon[35427]: pgmap v661: 97 pgs: 97 active+clean; 266 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 2.5 MiB/s wr, 176 op/s 2024-09-15T22:56:18.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:17 smithi121 ceph-mon[34815]: pgmap v661: 97 pgs: 97 active+clean; 266 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 2.5 MiB/s wr, 176 op/s 2024-09-15T22:56:18.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:17 smithi111 ceph-mon[30629]: pgmap v661: 97 pgs: 97 active+clean; 266 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 2.5 MiB/s wr, 176 op/s 2024-09-15T22:56:18.316 INFO:tasks.workunit.client.0.smithi111.stdout: 1 982466 9.96 MB/sec execute 406 sec latency 11.753 ms 2024-09-15T22:56:19.316 INFO:tasks.workunit.client.0.smithi111.stdout: 1 985053 9.97 MB/sec execute 407 sec latency 12.384 ms 2024-09-15T22:56:19.960 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:19 smithi111 ceph-mon[30629]: pgmap v662: 97 pgs: 97 active+clean; 256 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 2.6 MiB/s wr, 217 op/s 2024-09-15T22:56:20.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:19 smithi192 ceph-mon[35427]: pgmap v662: 97 pgs: 97 active+clean; 256 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 2.6 MiB/s wr, 217 op/s 2024-09-15T22:56:20.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:19 smithi121 ceph-mon[34815]: pgmap v662: 97 pgs: 97 active+clean; 256 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 2.6 MiB/s wr, 217 op/s 2024-09-15T22:56:20.316 INFO:tasks.workunit.client.0.smithi111.stdout: 1 986593 9.97 MB/sec execute 408 sec latency 342.025 ms 2024-09-15T22:56:21.261 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.259+0000 7fb509600640 1 -- 172.21.15.111:0/2671503288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb504103d60 msgr2=0x7fb504104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.259+0000 7fb509600640 1 --2- 172.21.15.111:0/2671503288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb504103d60 0x7fb504104160 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009900 tx=0x7fb4f402a410 comp rx=0 tx=0).stop 2024-09-15T22:56:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.260+0000 7fb509600640 1 -- 172.21.15.111:0/2671503288 shutdown_connections 2024-09-15T22:56:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.260+0000 7fb509600640 1 --2- 172.21.15.111:0/2671503288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb504105910 0x7fb50410c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.260+0000 7fb509600640 1 --2- 172.21.15.111:0/2671503288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb504104f50 0x7fb5041053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.260+0000 7fb509600640 1 --2- 172.21.15.111:0/2671503288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb504103d60 0x7fb504104160 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.260+0000 7fb509600640 1 -- 172.21.15.111:0/2671503288 >> 172.21.15.111:0/2671503288 conn(0x7fb5040ff4e0 msgr2=0x7fb504101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:56:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.260+0000 7fb509600640 1 -- 172.21.15.111:0/2671503288 shutdown_connections 2024-09-15T22:56:21.262 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.260+0000 7fb509600640 1 -- 172.21.15.111:0/2671503288 wait complete. 2024-09-15T22:56:21.263 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.261+0000 7fb509600640 1 Processor -- start 2024-09-15T22:56:21.263 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.261+0000 7fb509600640 1 -- start start 2024-09-15T22:56:21.263 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.261+0000 7fb509600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb504103d60 0x7fb504199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:21.263 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.261+0000 7fb509600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb504104f50 0x7fb50419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:21.263 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.261+0000 7fb509600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb504105910 0x7fb5041a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:21.269 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.261+0000 7fb509600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb50410fa30 con 0x7fb504104f50 2024-09-15T22:56:21.269 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.261+0000 7fb509600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb50410f8b0 con 0x7fb504103d60 2024-09-15T22:56:21.269 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.261+0000 7fb509600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb50410fbb0 con 0x7fb504105910 2024-09-15T22:56:21.269 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.261+0000 7fb502000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb504104f50 0x7fb50419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:21.277 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.261+0000 7fb502000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb504104f50 0x7fb50419a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34900/0 (socket says 172.21.15.111:34900) 2024-09-15T22:56:21.277 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.261+0000 7fb502000640 1 -- 172.21.15.111:0/3022614754 learned_addr learned my addr 172.21.15.111:0/3022614754 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:56:21.277 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.261+0000 7fb502a00640 1 --2- 172.21.15.111:0/3022614754 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb504103d60 0x7fb504199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:21.277 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.262+0000 7fb502000640 1 -- 172.21.15.111:0/3022614754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb504105910 msgr2=0x7fb5041a1570 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:56:21.277 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.262+0000 7fb502000640 1 --2- 172.21.15.111:0/3022614754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb504105910 0x7fb5041a1570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.277 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.262+0000 7fb502000640 1 -- 172.21.15.111:0/3022614754 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb504103d60 msgr2=0x7fb504199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.262+0000 7fb502000640 1 --2- 172.21.15.111:0/3022614754 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb504103d60 0x7fb504199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.262+0000 7fb502000640 1 -- 172.21.15.111:0/3022614754 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5041a1be0 con 0x7fb504104f50 2024-09-15T22:56:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.262+0000 7fb502000640 1 --2- 172.21.15.111:0/3022614754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb504104f50 0x7fb50419a500 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb4f000e9f0 tx=0x7fb4f000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:56:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.262+0000 7fb4fb400640 1 -- 172.21.15.111:0/3022614754 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f000cd80 con 0x7fb504104f50 2024-09-15T22:56:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.262+0000 7fb4fb400640 1 -- 172.21.15.111:0/3022614754 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f0005500 con 0x7fb504104f50 2024-09-15T22:56:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.262+0000 7fb509600640 1 -- 172.21.15.111:0/3022614754 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5041a1ed0 con 0x7fb504104f50 2024-09-15T22:56:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.262+0000 7fb4fb400640 1 -- 172.21.15.111:0/3022614754 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0010690 con 0x7fb504104f50 2024-09-15T22:56:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.262+0000 7fb509600640 1 -- 172.21.15.111:0/3022614754 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5041a2340 con 0x7fb504104f50 2024-09-15T22:56:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.265+0000 7fb50c41a740 1 Processor -- start 2024-09-15T22:56:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.265+0000 7fb50c41a740 1 -- start start 2024-09-15T22:56:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.265+0000 7fb50c41a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 0x5617702e9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.265+0000 7fb50c41a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617702e99d0 0x5617702ee490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.265+0000 7fb50c41a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617702eea60 0x5617702f0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.265+0000 7fb50c41a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56177024e100 con 0x5617702eea60 2024-09-15T22:56:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.266+0000 7fb50c41a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5617701f61b0 con 0x5617702e9090 2024-09-15T22:56:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.266+0000 7fb50c41a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56177024a0c0 con 0x5617702e99d0 2024-09-15T22:56:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.266+0000 7fb503400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 0x5617702e9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.266+0000 7fb502a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617702e99d0 0x5617702ee490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.266+0000 7fb503400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 0x5617702e9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:42478/0 (socket says 172.21.15.111:42478) 2024-09-15T22:56:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.266+0000 7fb503400640 1 -- 172.21.15.111:0/4233331413 learned_addr learned my addr 172.21.15.111:0/4233331413 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:56:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.266+0000 7fb503400640 1 -- 172.21.15.111:0/4233331413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617702e99d0 msgr2=0x5617702ee490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.266+0000 7fb503400640 1 --2- 172.21.15.111:0/4233331413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617702e99d0 0x5617702ee490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.266+0000 7fb503400640 1 -- 172.21.15.111:0/4233331413 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617702eea60 msgr2=0x5617702f0f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:56:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.266+0000 7fb503400640 1 --2- 172.21.15.111:0/4233331413 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617702eea60 0x5617702f0f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.266+0000 7fb503400640 1 -- 172.21.15.111:0/4233331413 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56176ff61210 con 0x5617702e9090 2024-09-15T22:56:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.266+0000 7fb503400640 1 --2- 172.21.15.111:0/4233331413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 0x5617702e9490 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009e30 tx=0x7fb4fc00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f36df44fa1d2a34 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:56:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.267+0000 7fb4ef400640 1 -- 172.21.15.111:0/4233331413 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc004d20 con 0x5617702e9090 2024-09-15T22:56:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.267+0000 7fb4ef400640 1 -- 172.21.15.111:0/4233331413 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4fc00ee70 con 0x5617702e9090 2024-09-15T22:56:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.267+0000 7fb4ef400640 1 -- 172.21.15.111:0/4233331413 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc0107f0 con 0x5617702e9090 2024-09-15T22:56:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.267+0000 7fb4fb400640 1 -- 172.21.15.111:0/3022614754 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fb4f00046c0 con 0x7fb504104f50 2024-09-15T22:56:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.267+0000 7fb50c41a740 1 -- 172.21.15.111:0/4233331413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 msgr2=0x5617702e9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.267+0000 7fb50c41a740 1 --2- 172.21.15.111:0/4233331413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 0x5617702e9490 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009e30 tx=0x7fb4fc00e720 comp rx=0 tx=0).stop 2024-09-15T22:56:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.267+0000 7fb50c41a740 1 -- 172.21.15.111:0/4233331413 shutdown_connections 2024-09-15T22:56:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.268+0000 7fb50c41a740 1 --2- 172.21.15.111:0/4233331413 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617702eea60 0x5617702f0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.268+0000 7fb50c41a740 1 --2- 172.21.15.111:0/4233331413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617702e99d0 0x5617702ee490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.268+0000 7fb50c41a740 1 --2- 172.21.15.111:0/4233331413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 0x5617702e9490 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.268+0000 7fb50c41a740 1 -- 172.21.15.111:0/4233331413 >> 172.21.15.111:0/4233331413 conn(0x5617702e4700 msgr2=0x5617702e4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:56:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.268+0000 7fb50c41a740 1 -- 172.21.15.111:0/4233331413 shutdown_connections 2024-09-15T22:56:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.268+0000 7fb50c41a740 1 -- 172.21.15.111:0/4233331413 wait complete. 2024-09-15T22:56:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.268+0000 7fb50c41a740 1 -- start start 2024-09-15T22:56:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.269+0000 7fb50c41a740 1 Processor -- start 2024-09-15T22:56:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.269+0000 7fb4fb400640 1 --2- 172.21.15.111:0/3022614754 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb4e80a3200 0x7fb4e80a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.269+0000 7fb4fb400640 1 -- 172.21.15.111:0/3022614754 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fb4f00cfde0 con 0x7fb504104f50 2024-09-15T22:56:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.269+0000 7fb50c41a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 0x5617702f3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.269+0000 7fb50c41a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617702e99d0 0x5617702f3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.269+0000 7fb503400640 1 --2- 172.21.15.111:0/3022614754 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb4e80a3200 0x7fb4e80a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.269+0000 7fb50c41a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617702eea60 0x5617702fa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.269+0000 7fb50c41a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5617702f1a30 con 0x5617702eea60 2024-09-15T22:56:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.269+0000 7fb50c41a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5617701d7090 con 0x5617702e9090 2024-09-15T22:56:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.269+0000 7fb50c41a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5617702f1bb0 con 0x5617702e99d0 2024-09-15T22:56:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.269+0000 7fb50c41a740 10 client.0 _ll_register_callbacks cb 0x56177022bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:56:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.269+0000 7fb502a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 0x5617702f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.270+0000 7fb503400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617702eea60 0x5617702fa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.270+0000 7fb502a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 0x5617702f3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:42486/0 (socket says 172.21.15.111:42486) 2024-09-15T22:56:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.270+0000 7fb502a00640 1 -- 172.21.15.111:0/453025443 learned_addr learned my addr 172.21.15.111:0/453025443 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:56:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.270+0000 7fb502000640 1 --2- 172.21.15.111:0/453025443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617702e99d0 0x5617702f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.270+0000 7fb502a00640 1 -- 172.21.15.111:0/453025443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617702e99d0 msgr2=0x5617702f3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.270+0000 7fb502a00640 1 --2- 172.21.15.111:0/453025443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617702e99d0 0x5617702f3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.270+0000 7fb502a00640 1 -- 172.21.15.111:0/453025443 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617702eea60 msgr2=0x5617702fa7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.270+0000 7fb502a00640 1 --2- 172.21.15.111:0/453025443 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617702eea60 0x5617702fa7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.270+0000 7fb502a00640 1 -- 172.21.15.111:0/453025443 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56177002d620 con 0x5617702e9090 2024-09-15T22:56:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.270+0000 7fb502000640 1 --2- 172.21.15.111:0/453025443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617702e99d0 0x5617702f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:56:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.270+0000 7fb503400640 1 --2- 172.21.15.111:0/3022614754 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb4e80a3200 0x7fb4e80a56c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009c70 tx=0x7fb4fc000f40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:56:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.270+0000 7fb502a00640 1 --2- 172.21.15.111:0/453025443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 0x5617702f3250 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4000c00 tx=0x7fb4f40043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fee05f977dc9ffde server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:56:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.270+0000 7fb4e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:56:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.271+0000 7fb4e3e00640 1 -- 172.21.15.111:0/453025443 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4035040 con 0x5617702e9090 2024-09-15T22:56:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.271+0000 7fb50c41a740 1 -- 172.21.15.111:0/453025443 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4fc00ee70 con 0x5617702e9090 2024-09-15T22:56:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.271+0000 7fb4e3e00640 1 -- 172.21.15.111:0/453025443 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fb4f4005450 con 0x5617702e9090 2024-09-15T22:56:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.271+0000 7fb4e3e00640 1 -- 172.21.15.111:0/453025443 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb4f4005650 con 0x5617702e9090 2024-09-15T22:56:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.271+0000 7fb50c41a740 10 client.25808 fetch_fsmap learned FSMap version 30 2024-09-15T22:56:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.271+0000 7fb50c41a740 1 -- 172.21.15.111:0/453025443 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5617702fd680 con 0x5617702e9090 2024-09-15T22:56:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.271+0000 7fb4e3e00640 1 -- 172.21.15.111:0/453025443 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4fc00ee70 con 0x5617702e9090 2024-09-15T22:56:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.271+0000 7fb4e3e00640 1 -- 172.21.15.111:0/453025443 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fb4f4031ac0 con 0x5617702e9090 2024-09-15T22:56:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.271+0000 7fb50c41a740 10 client.25808 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:56:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.271+0000 7fb50c41a740 10 client.25808 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:56:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.271+0000 7fb50c41a740 20 client.25808 populate_metadata read hostname 'smithi111' 2024-09-15T22:56:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.271+0000 7fb50c41a740 1 --2- 172.21.15.111:0/453025443 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5617702fece0 0x5617703010d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.272+0000 7fb50c41a740 4 client.25808 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:56:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.272+0000 7fb50c41a740 1 -- 172.21.15.111:0/453025443 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56177025b830 con 0x5617702fece0 2024-09-15T22:56:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.272+0000 7fb502000640 1 --2- 172.21.15.111:0/453025443 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5617702fece0 0x5617703010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.272+0000 7fb502000640 1 --2- 172.21.15.111:0/453025443 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5617702fece0 0x5617703010d0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91642d984a4c9843 server_cookie=d8a5e9460657aef3 in_seq=0 out_seq=0 2024-09-15T22:56:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.272+0000 7fb4e3e00640 10 client.25808 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:56:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.273+0000 7fb4e3e00640 1 -- 172.21.15.111:0/453025443 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56177025b830 con 0x5617702fece0 2024-09-15T22:56:21.285 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.273+0000 7fb4e3e00640 10 client.25808 handle_command_reply: tid=0 2024-09-15T22:56:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.273+0000 7fb50c41a740 1 client.25808 shutdown 2024-09-15T22:56:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.273+0000 7fb50c41a740 1 -- 172.21.15.111:0/453025443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 msgr2=0x5617702f3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.273+0000 7fb50c41a740 1 --2- 172.21.15.111:0/453025443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 0x5617702f3250 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4000c00 tx=0x7fb4f40043b0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.273+0000 7fb50c41a740 1 -- 172.21.15.111:0/453025443 shutdown_connections 2024-09-15T22:56:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.273+0000 7fb50c41a740 1 --2- 172.21.15.111:0/453025443 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5617702fece0 0x5617703010d0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.273+0000 7fb50c41a740 1 --2- 172.21.15.111:0/453025443 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617702eea60 0x5617702fa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.273+0000 7fb50c41a740 1 --2- 172.21.15.111:0/453025443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617702e99d0 0x5617702f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.273+0000 7fb4e3e00640 0 client.25808 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:56:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.273+0000 7fb50c41a740 1 --2- 172.21.15.111:0/453025443 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 0x5617702f3250 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.286 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.273+0000 7fb50c41a740 1 -- 172.21.15.111:0/453025443 >> 172.21.15.111:0/453025443 conn(0x5617702e4700 msgr2=0x5617702e5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:56:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.273+0000 7fb50c41a740 1 -- 172.21.15.111:0/453025443 shutdown_connections 2024-09-15T22:56:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.273+0000 7fb50c41a740 1 -- 172.21.15.111:0/453025443 wait complete. 2024-09-15T22:56:21.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.274+0000 7fb50c41a740 20 client.25808 trim_cache size 0 max 16384 2024-09-15T22:56:21.290 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.288+0000 7fb50c41a740 1 Processor -- start 2024-09-15T22:56:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.288+0000 7fb50c41a740 1 -- start start 2024-09-15T22:56:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.288+0000 7fb50c41a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 0x5617702e3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.288+0000 7fb50c41a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617702e99d0 0x5617702fa790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.288+0000 7fb50c41a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617702eea60 0x5617702e35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.288+0000 7fb50c41a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041ad8f0 con 0x5617702e99d0 2024-09-15T22:56:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.288+0000 7fb50c41a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041ab1c0 con 0x5617702e9090 2024-09-15T22:56:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.288+0000 7fb50c41a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041a9030 con 0x5617702eea60 2024-09-15T22:56:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.289+0000 7fb502000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617702e99d0 0x5617702fa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.289+0000 7fb502a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 0x5617702e3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.289+0000 7fb502a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 0x5617702e3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:42494/0 (socket says 172.21.15.111:42494) 2024-09-15T22:56:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.289+0000 7fb502a00640 1 -- 172.21.15.111:0/2190679769 learned_addr learned my addr 172.21.15.111:0/2190679769 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:56:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.289+0000 7fb503400640 1 --2- 172.21.15.111:0/2190679769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617702eea60 0x5617702e35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.289+0000 7fb502a00640 1 -- 172.21.15.111:0/2190679769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617702eea60 msgr2=0x5617702e35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.289+0000 7fb502a00640 1 --2- 172.21.15.111:0/2190679769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617702eea60 0x5617702e35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.289+0000 7fb502a00640 1 -- 172.21.15.111:0/2190679769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617702e99d0 msgr2=0x5617702fa790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.289+0000 7fb502a00640 1 --2- 172.21.15.111:0/2190679769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617702e99d0 0x5617702fa790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.289+0000 7fb502a00640 1 -- 172.21.15.111:0/2190679769 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561770301440 con 0x5617702e9090 2024-09-15T22:56:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.289+0000 7fb502a00640 1 --2- 172.21.15.111:0/2190679769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 0x5617702e3060 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4031440 tx=0x7fb4f402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92331fb6ec043168 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:56:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.290+0000 7fb4eea00640 1 -- 172.21.15.111:0/2190679769 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4035030 con 0x5617702e9090 2024-09-15T22:56:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.290+0000 7fb4eea00640 1 -- 172.21.15.111:0/2190679769 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f4055070 con 0x5617702e9090 2024-09-15T22:56:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.290+0000 7fb4eea00640 1 -- 172.21.15.111:0/2190679769 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4000e60 con 0x5617702e9090 2024-09-15T22:56:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.290+0000 7fb50c41a740 1 -- 172.21.15.111:0/2190679769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 msgr2=0x5617702e3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:21.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.290+0000 7fb50c41a740 1 --2- 172.21.15.111:0/2190679769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 0x5617702e3060 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4031440 tx=0x7fb4f402a410 comp rx=0 tx=0).stop 2024-09-15T22:56:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.290+0000 7fb50c41a740 1 -- 172.21.15.111:0/2190679769 shutdown_connections 2024-09-15T22:56:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.290+0000 7fb50c41a740 1 --2- 172.21.15.111:0/2190679769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617702eea60 0x5617702e35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.290+0000 7fb50c41a740 1 --2- 172.21.15.111:0/2190679769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617702e99d0 0x5617702fa790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.290+0000 7fb50c41a740 1 --2- 172.21.15.111:0/2190679769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 0x5617702e3060 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.290+0000 7fb50c41a740 1 -- 172.21.15.111:0/2190679769 >> 172.21.15.111:0/2190679769 conn(0x5617702e4700 msgr2=0x5617702f3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:56:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.290+0000 7fb50c41a740 1 -- 172.21.15.111:0/2190679769 shutdown_connections 2024-09-15T22:56:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.290+0000 7fb50c41a740 1 -- 172.21.15.111:0/2190679769 wait complete. 2024-09-15T22:56:21.296 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.291+0000 7fb50c41a740 1 -- start start 2024-09-15T22:56:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.291+0000 7fb50c41a740 1 Processor -- start 2024-09-15T22:56:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.291+0000 7fb50c41a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 0x5617702f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.291+0000 7fb50c41a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617702e99d0 0x5617702f8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.291+0000 7fb50c41a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617702eea60 0x5617702f9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.291+0000 7fb50c41a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041a9680 con 0x5617702eea60 2024-09-15T22:56:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.291+0000 7fb50c41a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041aaaf0 con 0x5617702e9090 2024-09-15T22:56:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.291+0000 7fb50c41a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041a9d50 con 0x5617702e99d0 2024-09-15T22:56:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.291+0000 7fb50c41a740 10 client.0 _ll_register_callbacks cb 0x7fb4f4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:56:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.292+0000 7fb503400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617702eea60 0x5617702f9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.292+0000 7fb502a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 0x5617702f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.292+0000 7fb502a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 0x5617702f7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:42504/0 (socket says 172.21.15.111:42504) 2024-09-15T22:56:21.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.292+0000 7fb502a00640 1 -- 172.21.15.111:0/890282302 learned_addr learned my addr 172.21.15.111:0/890282302 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:56:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.292+0000 7fb502000640 1 --2- 172.21.15.111:0/890282302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617702e99d0 0x5617702f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.292+0000 7fb502a00640 1 -- 172.21.15.111:0/890282302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617702e99d0 msgr2=0x5617702f8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.292+0000 7fb502a00640 1 --2- 172.21.15.111:0/890282302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617702e99d0 0x5617702f8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.292+0000 7fb502a00640 1 -- 172.21.15.111:0/890282302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617702eea60 msgr2=0x5617702f9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.292+0000 7fb502a00640 1 --2- 172.21.15.111:0/890282302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617702eea60 0x5617702f9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.292+0000 7fb502a00640 1 -- 172.21.15.111:0/890282302 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617702f9730 con 0x5617702e9090 2024-09-15T22:56:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.292+0000 7fb502a00640 1 --2- 172.21.15.111:0/890282302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 0x5617702f7c20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4004150 tx=0x7fb4f40319e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=538053004d929be5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:56:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.292+0000 7fb4e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:56:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.292+0000 7fb4e3e00640 1 -- 172.21.15.111:0/890282302 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4035050 con 0x5617702e9090 2024-09-15T22:56:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.292+0000 7fb50c41a740 1 -- 172.21.15.111:0/890282302 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4f4055070 con 0x5617702e9090 2024-09-15T22:56:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.292+0000 7fb4e3e00640 1 -- 172.21.15.111:0/890282302 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fb4f4004970 con 0x5617702e9090 2024-09-15T22:56:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.293+0000 7fb4e3e00640 1 -- 172.21.15.111:0/890282302 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb4f4004b70 con 0x5617702e9090 2024-09-15T22:56:21.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.293+0000 7fb50c41a740 10 client.25814 fetch_fsmap learned FSMap version 30 2024-09-15T22:56:21.299 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.293+0000 7fb50c41a740 1 -- 172.21.15.111:0/890282302 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5617702faa40 con 0x5617702e9090 2024-09-15T22:56:21.299 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.293+0000 7fb4e3e00640 1 -- 172.21.15.111:0/890282302 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f4055070 con 0x5617702e9090 2024-09-15T22:56:21.299 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.293+0000 7fb4e3e00640 1 -- 172.21.15.111:0/890282302 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fb4f4046600 con 0x5617702e9090 2024-09-15T22:56:21.299 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.293+0000 7fb50c41a740 10 client.25814 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:56:21.299 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.293+0000 7fb50c41a740 10 client.25814 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:56:21.299 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.293+0000 7fb50c41a740 20 client.25814 populate_metadata read hostname 'smithi111' 2024-09-15T22:56:21.300 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.293+0000 7fb50c41a740 1 --2- 172.21.15.111:0/890282302 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561770369fa0 0x56177036c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:21.300 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.293+0000 7fb50c41a740 4 client.25814 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:56:21.300 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.293+0000 7fb50c41a740 1 -- 172.21.15.111:0/890282302 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561770255b80 con 0x561770369fa0 2024-09-15T22:56:21.300 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.294+0000 7fb502000640 1 --2- 172.21.15.111:0/890282302 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561770369fa0 0x56177036c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:21.301 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.294+0000 7fb502000640 1 --2- 172.21.15.111:0/890282302 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561770369fa0 0x56177036c3c0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=876f1c52819ebdd0 server_cookie=a5b175493452f940 in_seq=0 out_seq=0 2024-09-15T22:56:21.301 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.294+0000 7fb4e3e00640 10 client.25814 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:56:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.307+0000 7fb4e3e00640 1 -- 172.21.15.111:0/890282302 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561770255b80 con 0x561770369fa0 2024-09-15T22:56:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.307+0000 7fb4e3e00640 10 client.25814 handle_command_reply: tid=0 2024-09-15T22:56:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.307+0000 7fb50c41a740 1 client.25814 shutdown 2024-09-15T22:56:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.307+0000 7fb50c41a740 1 -- 172.21.15.111:0/890282302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 msgr2=0x5617702f7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.307+0000 7fb50c41a740 1 --2- 172.21.15.111:0/890282302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 0x5617702f7c20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4004150 tx=0x7fb4f40319e0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.307+0000 7fb50c41a740 1 -- 172.21.15.111:0/890282302 shutdown_connections 2024-09-15T22:56:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.307+0000 7fb50c41a740 1 --2- 172.21.15.111:0/890282302 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561770369fa0 0x56177036c3c0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.307+0000 7fb50c41a740 1 --2- 172.21.15.111:0/890282302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5617702eea60 0x5617702f9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.307+0000 7fb50c41a740 1 --2- 172.21.15.111:0/890282302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5617702e99d0 0x5617702f8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.309 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.307+0000 7fb4e3e00640 0 client.25814 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:56:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.307+0000 7fb50c41a740 1 --2- 172.21.15.111:0/890282302 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617702e9090 0x5617702f7c20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.307+0000 7fb50c41a740 1 -- 172.21.15.111:0/890282302 >> 172.21.15.111:0/890282302 conn(0x5617702e4700 msgr2=0x5617702e3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:56:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.307+0000 7fb50c41a740 1 -- 172.21.15.111:0/890282302 shutdown_connections 2024-09-15T22:56:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.307+0000 7fb50c41a740 1 -- 172.21.15.111:0/890282302 wait complete. 2024-09-15T22:56:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.307+0000 7fb50c41a740 20 client.25814 trim_cache size 0 max 16384 2024-09-15T22:56:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:56:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:56:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:56:21.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:56:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.308+0000 7fb509600640 1 -- 172.21.15.111:0/3022614754 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb4e80a3200 msgr2=0x7fb4e80a56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.308+0000 7fb509600640 1 --2- 172.21.15.111:0/3022614754 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb4e80a3200 0x7fb4e80a56c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc009c70 tx=0x7fb4fc000f40 comp rx=0 tx=0).stop 2024-09-15T22:56:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.308+0000 7fb509600640 1 -- 172.21.15.111:0/3022614754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb504104f50 msgr2=0x7fb50419a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.308+0000 7fb509600640 1 --2- 172.21.15.111:0/3022614754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb504104f50 0x7fb50419a500 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb4f000e9f0 tx=0x7fb4f000edf0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.308+0000 7fb509600640 1 -- 172.21.15.111:0/3022614754 shutdown_connections 2024-09-15T22:56:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.308+0000 7fb509600640 1 --2- 172.21.15.111:0/3022614754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb504105910 0x7fb5041a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.308+0000 7fb509600640 1 --2- 172.21.15.111:0/3022614754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb504104f50 0x7fb50419a500 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.308+0000 7fb509600640 1 --2- 172.21.15.111:0/3022614754 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fb4e80a3200 0x7fb4e80a56c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.308+0000 7fb509600640 1 --2- 172.21.15.111:0/3022614754 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb504103d60 0x7fb504199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.308+0000 7fb509600640 1 -- 172.21.15.111:0/3022614754 >> 172.21.15.111:0/3022614754 conn(0x7fb5040ff4e0 msgr2=0x7fb504108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:56:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.308+0000 7fb509600640 1 -- 172.21.15.111:0/3022614754 shutdown_connections 2024-09-15T22:56:21.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:21.308+0000 7fb509600640 1 -- 172.21.15.111:0/3022614754 wait complete. 2024-09-15T22:56:21.316 INFO:tasks.workunit.client.0.smithi111.stdout: 1 986598 9.94 MB/sec execute 409 sec latency 1277.824 ms 2024-09-15T22:56:22.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:21 smithi192 ceph-mon[35427]: pgmap v663: 97 pgs: 97 active+clean; 262 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 3.3 MiB/s wr, 219 op/s 2024-09-15T22:56:22.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:21 smithi121 ceph-mon[34815]: pgmap v663: 97 pgs: 97 active+clean; 262 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 3.3 MiB/s wr, 219 op/s 2024-09-15T22:56:22.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:21 smithi111 ceph-mon[30629]: pgmap v663: 97 pgs: 97 active+clean; 262 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 3.3 MiB/s wr, 219 op/s 2024-09-15T22:56:22.316 INFO:tasks.workunit.client.0.smithi111.stdout: 1 989036 9.96 MB/sec execute 410 sec latency 24.947 ms 2024-09-15T22:56:23.316 INFO:tasks.workunit.client.0.smithi111.stdout: 1 991831 9.97 MB/sec execute 411 sec latency 13.855 ms 2024-09-15T22:56:24.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:23 smithi192 ceph-mon[35427]: pgmap v664: 97 pgs: 97 active+clean; 268 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 3.5 MiB/s wr, 179 op/s 2024-09-15T22:56:24.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:23 smithi121 ceph-mon[34815]: pgmap v664: 97 pgs: 97 active+clean; 268 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 3.5 MiB/s wr, 179 op/s 2024-09-15T22:56:24.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:23 smithi111 ceph-mon[30629]: pgmap v664: 97 pgs: 97 active+clean; 268 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 3.5 MiB/s wr, 179 op/s 2024-09-15T22:56:24.316 INFO:tasks.workunit.client.0.smithi111.stdout: 1 994893 9.99 MB/sec execute 412 sec latency 46.268 ms 2024-09-15T22:56:25.316 INFO:tasks.workunit.client.0.smithi111.stdout: 1 995507 9.97 MB/sec execute 413 sec latency 523.221 ms 2024-09-15T22:56:25.885 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:25 smithi111 ceph-mon[30629]: pgmap v665: 97 pgs: 97 active+clean; 277 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 4.0 MiB/s wr, 186 op/s 2024-09-15T22:56:26.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:25 smithi192 ceph-mon[35427]: pgmap v665: 97 pgs: 97 active+clean; 277 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 4.0 MiB/s wr, 186 op/s 2024-09-15T22:56:26.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:25 smithi121 ceph-mon[34815]: pgmap v665: 97 pgs: 97 active+clean; 277 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 4.0 MiB/s wr, 186 op/s 2024-09-15T22:56:26.316 INFO:tasks.workunit.client.0.smithi111.stdout: 1 995507 9.94 MB/sec execute 414 sec latency 1523.285 ms 2024-09-15T22:56:27.317 INFO:tasks.workunit.client.0.smithi111.stdout: 1 998076 9.95 MB/sec execute 415 sec latency 1659.766 ms 2024-09-15T22:56:28.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:27 smithi192 ceph-mon[35427]: pgmap v666: 97 pgs: 97 active+clean; 278 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 195 op/s 2024-09-15T22:56:28.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:27 smithi121 ceph-mon[34815]: pgmap v666: 97 pgs: 97 active+clean; 278 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 195 op/s 2024-09-15T22:56:28.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:27 smithi111 ceph-mon[30629]: pgmap v666: 97 pgs: 97 active+clean; 278 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 195 op/s 2024-09-15T22:56:28.316 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1000415 9.95 MB/sec execute 416 sec latency 12.128 ms 2024-09-15T22:56:29.317 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1002691 9.96 MB/sec execute 417 sec latency 10.840 ms 2024-09-15T22:56:30.036 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:29 smithi111 ceph-mon[30629]: pgmap v667: 97 pgs: 97 active+clean; 280 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 227 op/s 2024-09-15T22:56:30.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:29 smithi192 ceph-mon[35427]: pgmap v667: 97 pgs: 97 active+clean; 280 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 227 op/s 2024-09-15T22:56:30.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:29 smithi121 ceph-mon[34815]: pgmap v667: 97 pgs: 97 active+clean; 280 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 227 op/s 2024-09-15T22:56:30.317 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1005053 9.97 MB/sec execute 418 sec latency 304.650 ms 2024-09-15T22:56:31.317 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1005053 9.95 MB/sec execute 419 sec latency 1304.718 ms 2024-09-15T22:56:32.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:31 smithi192 ceph-mon[35427]: pgmap v668: 97 pgs: 97 active+clean; 278 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 167 op/s 2024-09-15T22:56:32.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:31 smithi121 ceph-mon[34815]: pgmap v668: 97 pgs: 97 active+clean; 278 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 167 op/s 2024-09-15T22:56:32.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:31 smithi111 ceph-mon[30629]: pgmap v668: 97 pgs: 97 active+clean; 278 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 167 op/s 2024-09-15T22:56:32.317 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1006920 9.95 MB/sec execute 420 sec latency 1360.744 ms 2024-09-15T22:56:33.317 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1009624 9.96 MB/sec execute 421 sec latency 10.532 ms 2024-09-15T22:56:34.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:33 smithi192 ceph-mon[35427]: pgmap v669: 97 pgs: 97 active+clean; 281 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 190 op/s 2024-09-15T22:56:34.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:33 smithi121 ceph-mon[34815]: pgmap v669: 97 pgs: 97 active+clean; 281 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 190 op/s 2024-09-15T22:56:34.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:33 smithi111 ceph-mon[30629]: pgmap v669: 97 pgs: 97 active+clean; 281 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 190 op/s 2024-09-15T22:56:34.317 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1012766 9.98 MB/sec execute 422 sec latency 12.020 ms 2024-09-15T22:56:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.425+0000 7fd3c0800640 1 -- 172.21.15.111:0/2355875762 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3b8105910 msgr2=0x7fd3b810c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.425+0000 7fd3c0800640 1 --2- 172.21.15.111:0/2355875762 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3b8105910 0x7fd3b810c1a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009be0 tx=0x7fd3b402a6c0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.425+0000 7fd3c0800640 1 -- 172.21.15.111:0/2355875762 shutdown_connections 2024-09-15T22:56:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.425+0000 7fd3c0800640 1 --2- 172.21.15.111:0/2355875762 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3b8105910 0x7fd3b810c1a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.427 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.425+0000 7fd3c0800640 1 --2- 172.21.15.111:0/2355875762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3b8104f50 0x7fd3b81053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.425+0000 7fd3c0800640 1 --2- 172.21.15.111:0/2355875762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3b8103d60 0x7fd3b8104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.425+0000 7fd3c0800640 1 -- 172.21.15.111:0/2355875762 >> 172.21.15.111:0/2355875762 conn(0x7fd3b80ff4e0 msgr2=0x7fd3b8101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:56:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.425+0000 7fd3c0800640 1 -- 172.21.15.111:0/2355875762 shutdown_connections 2024-09-15T22:56:34.428 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.426+0000 7fd3c0800640 1 -- 172.21.15.111:0/2355875762 wait complete. 2024-09-15T22:56:34.429 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.426+0000 7fd3c0800640 1 Processor -- start 2024-09-15T22:56:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.427+0000 7fd3c0800640 1 -- start start 2024-09-15T22:56:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.427+0000 7fd3c0800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3b8103d60 0x7fd3b8199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.427+0000 7fd3c0800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3b8104f50 0x7fd3b819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.427+0000 7fd3c0800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3b8105910 0x7fd3b81a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.427+0000 7fd3c0800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b810f9e0 con 0x7fd3b8103d60 2024-09-15T22:56:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.427+0000 7fd3c0800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b810f860 con 0x7fd3b8104f50 2024-09-15T22:56:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.427+0000 7fd3c0800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b810fb60 con 0x7fd3b8105910 2024-09-15T22:56:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.427+0000 7fd3bde00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3b8103d60 0x7fd3b8199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.427+0000 7fd3bd400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3b8104f50 0x7fd3b819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.427+0000 7fd3bd400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3b8104f50 0x7fd3b819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:41822/0 (socket says 172.21.15.111:41822) 2024-09-15T22:56:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.427+0000 7fd3bd400640 1 -- 172.21.15.111:0/1329302920 learned_addr learned my addr 172.21.15.111:0/1329302920 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:56:34.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.427+0000 7fd3be800640 1 --2- 172.21.15.111:0/1329302920 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3b8105910 0x7fd3b81a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.427+0000 7fd3bd400640 1 -- 172.21.15.111:0/1329302920 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3b8105910 msgr2=0x7fd3b81a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.427+0000 7fd3bd400640 1 --2- 172.21.15.111:0/1329302920 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3b8105910 0x7fd3b81a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.427+0000 7fd3bd400640 1 -- 172.21.15.111:0/1329302920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3b8103d60 msgr2=0x7fd3b8199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.427+0000 7fd3bd400640 1 --2- 172.21.15.111:0/1329302920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3b8103d60 0x7fd3b8199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.427+0000 7fd3bd400640 1 -- 172.21.15.111:0/1329302920 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b81a1b70 con 0x7fd3b8104f50 2024-09-15T22:56:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.428+0000 7fd3bd400640 1 --2- 172.21.15.111:0/1329302920 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3b8104f50 0x7fd3b819a490 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800e9c0 tx=0x7fd3a800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:56:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.428+0000 7fd3b2a00640 1 -- 172.21.15.111:0/1329302920 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a800ccc0 con 0x7fd3b8104f50 2024-09-15T22:56:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.428+0000 7fd3b2a00640 1 -- 172.21.15.111:0/1329302920 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3a8005500 con 0x7fd3b8104f50 2024-09-15T22:56:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.428+0000 7fd3c0800640 1 -- 172.21.15.111:0/1329302920 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3b81a1e60 con 0x7fd3b8104f50 2024-09-15T22:56:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.428+0000 7fd3b2a00640 1 -- 172.21.15.111:0/1329302920 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8010430 con 0x7fd3b8104f50 2024-09-15T22:56:34.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.428+0000 7fd3c0800640 1 -- 172.21.15.111:0/1329302920 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3b81a2370 con 0x7fd3b8104f50 2024-09-15T22:56:34.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.431+0000 7fd3c3853740 1 Processor -- start 2024-09-15T22:56:34.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.431+0000 7fd3c3853740 1 -- start start 2024-09-15T22:56:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.431+0000 7fd3c3853740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587bf353090 0x5587bf353490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.431+0000 7fd3c3853740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5587bf3539d0 0x5587bf358490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.431+0000 7fd3b2a00640 1 -- 172.21.15.111:0/1329302920 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd3a80046c0 con 0x7fd3b8104f50 2024-09-15T22:56:34.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.431+0000 7fd3c3853740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587bf358a60 0x5587bf35af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.431+0000 7fd3c3853740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5587bf2b8100 con 0x5587bf3539d0 2024-09-15T22:56:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.431+0000 7fd3c3853740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5587bf2601b0 con 0x5587bf353090 2024-09-15T22:56:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.431+0000 7fd3c3853740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5587bf2b40c0 con 0x5587bf358a60 2024-09-15T22:56:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.431+0000 7fd3bd400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587bf358a60 0x5587bf35af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.432+0000 7fd3bde00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5587bf3539d0 0x5587bf358490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.432+0000 7fd3be800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587bf353090 0x5587bf353490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.432+0000 7fd3bd400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587bf358a60 0x5587bf35af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:34100/0 (socket says 172.21.15.111:34100) 2024-09-15T22:56:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.432+0000 7fd3bd400640 1 -- 172.21.15.111:0/2448971051 learned_addr learned my addr 172.21.15.111:0/2448971051 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:56:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.432+0000 7fd3bde00640 1 -- 172.21.15.111:0/2448971051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587bf358a60 msgr2=0x5587bf35af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.432+0000 7fd3bde00640 1 --2- 172.21.15.111:0/2448971051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587bf358a60 0x5587bf35af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.432+0000 7fd3bde00640 1 -- 172.21.15.111:0/2448971051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587bf353090 msgr2=0x5587bf353490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:34.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.432+0000 7fd3bde00640 1 --2- 172.21.15.111:0/2448971051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587bf353090 0x5587bf353490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.432+0000 7fd3bde00640 1 -- 172.21.15.111:0/2448971051 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587befcb210 con 0x5587bf3539d0 2024-09-15T22:56:34.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.432+0000 7fd3bde00640 1 --2- 172.21.15.111:0/2448971051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5587bf3539d0 0x5587bf358490 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009bd0 tx=0x7fd3ac011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31ff99703fa25c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:56:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.433+0000 7fd3a7000640 1 -- 172.21.15.111:0/2448971051 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac004230 con 0x5587bf3539d0 2024-09-15T22:56:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.433+0000 7fd3a7000640 1 -- 172.21.15.111:0/2448971051 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3ac009e40 con 0x5587bf3539d0 2024-09-15T22:56:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.433+0000 7fd3a7000640 1 -- 172.21.15.111:0/2448971051 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac0197e0 con 0x5587bf3539d0 2024-09-15T22:56:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.433+0000 7fd3b2a00640 1 --2- 172.21.15.111:0/1329302920 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd3a00a3370 0x7fd3a00a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.433+0000 7fd3b2a00640 1 -- 172.21.15.111:0/1329302920 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fd3a80cfae0 con 0x7fd3b8104f50 2024-09-15T22:56:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.433+0000 7fd3be800640 1 --2- 172.21.15.111:0/1329302920 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd3a00a3370 0x7fd3a00a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.433+0000 7fd3c3853740 1 -- 172.21.15.111:0/2448971051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5587bf3539d0 msgr2=0x5587bf358490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.433+0000 7fd3c3853740 1 --2- 172.21.15.111:0/2448971051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5587bf3539d0 0x5587bf358490 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009bd0 tx=0x7fd3ac011810 comp rx=0 tx=0).stop 2024-09-15T22:56:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.433+0000 7fd3c3853740 1 -- 172.21.15.111:0/2448971051 shutdown_connections 2024-09-15T22:56:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.433+0000 7fd3c3853740 1 --2- 172.21.15.111:0/2448971051 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587bf358a60 0x5587bf35af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.433+0000 7fd3c3853740 1 --2- 172.21.15.111:0/2448971051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5587bf3539d0 0x5587bf358490 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.433+0000 7fd3c3853740 1 --2- 172.21.15.111:0/2448971051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587bf353090 0x5587bf353490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.433+0000 7fd3c3853740 1 -- 172.21.15.111:0/2448971051 >> 172.21.15.111:0/2448971051 conn(0x5587bf34e700 msgr2=0x5587bf34eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:56:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.434+0000 7fd3c3853740 1 -- 172.21.15.111:0/2448971051 shutdown_connections 2024-09-15T22:56:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.434+0000 7fd3c3853740 1 -- 172.21.15.111:0/2448971051 wait complete. 2024-09-15T22:56:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.434+0000 7fd3be800640 1 --2- 172.21.15.111:0/1329302920 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd3a00a3370 0x7fd3a00a5830 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd3b402abe0 tx=0x7fd3b4005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:56:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.434+0000 7fd3c3853740 1 -- start start 2024-09-15T22:56:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.434+0000 7fd3c3853740 1 Processor -- start 2024-09-15T22:56:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.435+0000 7fd3c3853740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587bf353090 0x5587bf35d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.435+0000 7fd3c3853740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587bf3539d0 0x5587bf35d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.435+0000 7fd3c3853740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5587bf358a60 0x5587bf364760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.435+0000 7fd3c3853740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5587bf35ba30 con 0x5587bf358a60 2024-09-15T22:56:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.435+0000 7fd3c3853740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5587bf241090 con 0x5587bf353090 2024-09-15T22:56:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.435+0000 7fd3c3853740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5587bf35bbb0 con 0x5587bf3539d0 2024-09-15T22:56:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.435+0000 7fd3c3853740 10 client.0 _ll_register_callbacks cb 0x5587bf295f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:56:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.435+0000 7fd3bde00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587bf353090 0x5587bf35d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.435+0000 7fd3bde00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587bf353090 0x5587bf35d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:41836/0 (socket says 172.21.15.111:41836) 2024-09-15T22:56:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.435+0000 7fd3bde00640 1 -- 172.21.15.111:0/1606160776 learned_addr learned my addr 172.21.15.111:0/1606160776 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:56:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.435+0000 7fd3be800640 1 --2- 172.21.15.111:0/1606160776 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5587bf358a60 0x5587bf364760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.435+0000 7fd3bd400640 1 --2- 172.21.15.111:0/1606160776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587bf3539d0 0x5587bf35d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.435+0000 7fd3bde00640 1 -- 172.21.15.111:0/1606160776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587bf3539d0 msgr2=0x5587bf35d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.435+0000 7fd3bde00640 1 --2- 172.21.15.111:0/1606160776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587bf3539d0 0x5587bf35d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.435+0000 7fd3bde00640 1 -- 172.21.15.111:0/1606160776 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5587bf358a60 msgr2=0x5587bf364760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.435+0000 7fd3bde00640 1 --2- 172.21.15.111:0/1606160776 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5587bf358a60 0x5587bf364760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.435+0000 7fd3bde00640 1 -- 172.21.15.111:0/1606160776 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587bf097620 con 0x5587bf353090 2024-09-15T22:56:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.436+0000 7fd3bde00640 1 --2- 172.21.15.111:0/1606160776 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587bf353090 0x5587bf35d1e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac011d10 tx=0x7fd3ac003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14f06f931698d09f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:56:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.436+0000 7fd39b400640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:56:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.436+0000 7fd39b400640 1 -- 172.21.15.111:0/1606160776 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac028070 con 0x5587bf353090 2024-09-15T22:56:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.436+0000 7fd3c3853740 1 -- 172.21.15.111:0/1606160776 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5587bf2c5830 con 0x5587bf353090 2024-09-15T22:56:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.436+0000 7fd39b400640 1 -- 172.21.15.111:0/1606160776 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fd3ac004480 con 0x5587bf353090 2024-09-15T22:56:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.436+0000 7fd39b400640 1 -- 172.21.15.111:0/1606160776 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd3ac034050 con 0x5587bf353090 2024-09-15T22:56:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.436+0000 7fd3c3853740 10 client.25826 fetch_fsmap learned FSMap version 30 2024-09-15T22:56:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.436+0000 7fd3c3853740 1 -- 172.21.15.111:0/1606160776 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5587bf367550 con 0x5587bf353090 2024-09-15T22:56:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.436+0000 7fd39b400640 1 -- 172.21.15.111:0/1606160776 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5587bf2c5830 con 0x5587bf353090 2024-09-15T22:56:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.437+0000 7fd39b400640 1 -- 172.21.15.111:0/1606160776 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fd3ac0049c0 con 0x5587bf353090 2024-09-15T22:56:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.437+0000 7fd3c3853740 10 client.25826 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:56:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.437+0000 7fd3c3853740 10 client.25826 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:56:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.437+0000 7fd3c3853740 20 client.25826 populate_metadata read hostname 'smithi111' 2024-09-15T22:56:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.437+0000 7fd3c3853740 1 --2- 172.21.15.111:0/1606160776 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5587bf368a30 0x5587bf36aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.437+0000 7fd3c3853740 4 client.25826 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:56:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.437+0000 7fd3c3853740 1 -- 172.21.15.111:0/1606160776 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587bf2bfb80 con 0x5587bf368a30 2024-09-15T22:56:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.437+0000 7fd3bd400640 1 --2- 172.21.15.111:0/1606160776 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5587bf368a30 0x5587bf36aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.437+0000 7fd3bd400640 1 --2- 172.21.15.111:0/1606160776 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5587bf368a30 0x5587bf36aea0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ade0694418bef94 server_cookie=a57a05afc6312a28 in_seq=0 out_seq=0 2024-09-15T22:56:34.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.437+0000 7fd39b400640 10 client.25826 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:56:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.438+0000 7fd39b400640 1 -- 172.21.15.111:0/1606160776 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587bf2bfb80 con 0x5587bf368a30 2024-09-15T22:56:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.438+0000 7fd39b400640 10 client.25826 handle_command_reply: tid=0 2024-09-15T22:56:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.438+0000 7fd3c3853740 1 client.25826 shutdown 2024-09-15T22:56:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.438+0000 7fd3c3853740 1 -- 172.21.15.111:0/1606160776 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587bf353090 msgr2=0x5587bf35d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.438+0000 7fd3c3853740 1 --2- 172.21.15.111:0/1606160776 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587bf353090 0x5587bf35d1e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac011d10 tx=0x7fd3ac003820 comp rx=0 tx=0).stop 2024-09-15T22:56:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.438+0000 7fd3c3853740 1 -- 172.21.15.111:0/1606160776 shutdown_connections 2024-09-15T22:56:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.438+0000 7fd3c3853740 1 --2- 172.21.15.111:0/1606160776 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5587bf368a30 0x5587bf36aea0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.438+0000 7fd3c3853740 1 --2- 172.21.15.111:0/1606160776 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5587bf358a60 0x5587bf364760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.438+0000 7fd39b400640 0 client.25826 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:56:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.438+0000 7fd3c3853740 1 --2- 172.21.15.111:0/1606160776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587bf3539d0 0x5587bf35d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.438+0000 7fd3c3853740 1 --2- 172.21.15.111:0/1606160776 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587bf353090 0x5587bf35d1e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.438+0000 7fd3c3853740 1 -- 172.21.15.111:0/1606160776 >> 172.21.15.111:0/1606160776 conn(0x5587bf34e700 msgr2=0x5587bf34ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:56:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.438+0000 7fd3c3853740 1 -- 172.21.15.111:0/1606160776 shutdown_connections 2024-09-15T22:56:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.439+0000 7fd3c3853740 1 -- 172.21.15.111:0/1606160776 wait complete. 2024-09-15T22:56:34.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.439+0000 7fd3c3853740 20 client.25826 trim_cache size 0 max 16384 2024-09-15T22:56:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.453+0000 7fd3c3853740 1 Processor -- start 2024-09-15T22:56:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.453+0000 7fd3c3853740 1 -- start start 2024-09-15T22:56:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.454+0000 7fd3c3853740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587bf353090 0x5587bf35cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.454+0000 7fd3c3853740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5587bf3539d0 0x5587bf364680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.454+0000 7fd3c3853740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587bf358a60 0x5587bf35d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.454+0000 7fd3c3853740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b81ad950 con 0x5587bf3539d0 2024-09-15T22:56:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.454+0000 7fd3c3853740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b81ab220 con 0x5587bf358a60 2024-09-15T22:56:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.454+0000 7fd3c3853740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b81a9070 con 0x5587bf353090 2024-09-15T22:56:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.454+0000 7fd3bd400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5587bf3539d0 0x5587bf364680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.454+0000 7fd3be800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587bf358a60 0x5587bf35d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.454+0000 7fd3be800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587bf358a60 0x5587bf35d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:41846/0 (socket says 172.21.15.111:41846) 2024-09-15T22:56:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.454+0000 7fd3be800640 1 -- 172.21.15.111:0/3014576288 learned_addr learned my addr 172.21.15.111:0/3014576288 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:56:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.454+0000 7fd3bde00640 1 --2- 172.21.15.111:0/3014576288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587bf353090 0x5587bf35cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.454+0000 7fd3be800640 1 -- 172.21.15.111:0/3014576288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587bf353090 msgr2=0x5587bf35cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.454+0000 7fd3be800640 1 --2- 172.21.15.111:0/3014576288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587bf353090 0x5587bf35cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.454+0000 7fd3be800640 1 -- 172.21.15.111:0/3014576288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5587bf3539d0 msgr2=0x5587bf364680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.454+0000 7fd3be800640 1 --2- 172.21.15.111:0/3014576288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5587bf3539d0 0x5587bf364680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.454+0000 7fd3be800640 1 -- 172.21.15.111:0/3014576288 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587bf34e520 con 0x5587bf358a60 2024-09-15T22:56:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.454+0000 7fd3bd400640 1 --2- 172.21.15.111:0/3014576288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5587bf3539d0 0x5587bf364680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:56:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.455+0000 7fd3be800640 1 --2- 172.21.15.111:0/3014576288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587bf358a60 0x5587bf35d4d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4033000 tx=0x7fd3b4040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=551d3e341149f4ef server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:56:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.455+0000 7fd3a6600640 1 -- 172.21.15.111:0/3014576288 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4036040 con 0x5587bf358a60 2024-09-15T22:56:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.455+0000 7fd3a6600640 1 -- 172.21.15.111:0/3014576288 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3b4050070 con 0x5587bf358a60 2024-09-15T22:56:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.455+0000 7fd3a6600640 1 -- 172.21.15.111:0/3014576288 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b40559b0 con 0x5587bf358a60 2024-09-15T22:56:34.461 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.456+0000 7fd3c3853740 1 -- 172.21.15.111:0/3014576288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587bf358a60 msgr2=0x5587bf35d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:34.461 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.456+0000 7fd3c3853740 1 --2- 172.21.15.111:0/3014576288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587bf358a60 0x5587bf35d4d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4033000 tx=0x7fd3b4040000 comp rx=0 tx=0).stop 2024-09-15T22:56:34.461 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.456+0000 7fd3c3853740 1 -- 172.21.15.111:0/3014576288 shutdown_connections 2024-09-15T22:56:34.461 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.456+0000 7fd3c3853740 1 --2- 172.21.15.111:0/3014576288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587bf358a60 0x5587bf35d4d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.456+0000 7fd3c3853740 1 --2- 172.21.15.111:0/3014576288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5587bf3539d0 0x5587bf364680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.456+0000 7fd3c3853740 1 --2- 172.21.15.111:0/3014576288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587bf353090 0x5587bf35cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.456+0000 7fd3c3853740 1 -- 172.21.15.111:0/3014576288 >> 172.21.15.111:0/3014576288 conn(0x5587bf34e700 msgr2=0x5587bf34eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:56:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.456+0000 7fd3c3853740 1 -- 172.21.15.111:0/3014576288 shutdown_connections 2024-09-15T22:56:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.456+0000 7fd3c3853740 1 -- 172.21.15.111:0/3014576288 wait complete. 2024-09-15T22:56:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.456+0000 7fd3c3853740 1 -- start start 2024-09-15T22:56:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.456+0000 7fd3c3853740 1 Processor -- start 2024-09-15T22:56:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.457+0000 7fd3c3853740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5587bf353090 0x5587bf364450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.457+0000 7fd3c3853740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587bf3539d0 0x5587bf360f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.457+0000 7fd3c3853740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587bf358a60 0x5587bf361480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.457+0000 7fd3c3853740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b81a96e0 con 0x5587bf353090 2024-09-15T22:56:34.462 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.457+0000 7fd3c3853740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b81aab50 con 0x5587bf358a60 2024-09-15T22:56:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.457+0000 7fd3c3853740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b81a9db0 con 0x5587bf3539d0 2024-09-15T22:56:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.457+0000 7fd3c3853740 10 client.0 _ll_register_callbacks cb 0x7fd3a8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:56:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.457+0000 7fd3bd400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587bf3539d0 0x5587bf360f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.457+0000 7fd3be800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587bf358a60 0x5587bf361480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.457+0000 7fd3bd400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587bf3539d0 0x5587bf360f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:34126/0 (socket says 172.21.15.111:34126) 2024-09-15T22:56:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.457+0000 7fd3bd400640 1 -- 172.21.15.111:0/2333262513 learned_addr learned my addr 172.21.15.111:0/2333262513 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:56:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.457+0000 7fd3bde00640 1 --2- 172.21.15.111:0/2333262513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5587bf353090 0x5587bf364450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.457+0000 7fd3be800640 1 -- 172.21.15.111:0/2333262513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587bf3539d0 msgr2=0x5587bf360f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.457+0000 7fd3be800640 1 --2- 172.21.15.111:0/2333262513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587bf3539d0 0x5587bf360f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.457+0000 7fd3be800640 1 -- 172.21.15.111:0/2333262513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5587bf353090 msgr2=0x5587bf364450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.457+0000 7fd3be800640 1 --2- 172.21.15.111:0/2333262513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5587bf353090 0x5587bf364450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.457+0000 7fd3be800640 1 -- 172.21.15.111:0/2333262513 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587bf3619f0 con 0x5587bf358a60 2024-09-15T22:56:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.457+0000 7fd3bd400640 1 --2- 172.21.15.111:0/2333262513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587bf3539d0 0x5587bf360f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:56:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.457+0000 7fd3be800640 1 --2- 172.21.15.111:0/2333262513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587bf358a60 0x5587bf361480 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009be0 tx=0x7fd3b402a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55b98aa2b994daa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:56:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.457+0000 7fd39b400640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:56:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.458+0000 7fd39b400640 1 -- 172.21.15.111:0/2333262513 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b4063060 con 0x5587bf358a60 2024-09-15T22:56:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.458+0000 7fd3c3853740 1 -- 172.21.15.111:0/2333262513 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3b4050070 con 0x5587bf358a60 2024-09-15T22:56:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.458+0000 7fd39b400640 1 -- 172.21.15.111:0/2333262513 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fd3b4003cf0 con 0x5587bf358a60 2024-09-15T22:56:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.458+0000 7fd39b400640 1 -- 172.21.15.111:0/2333262513 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd3b4036040 con 0x5587bf358a60 2024-09-15T22:56:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.458+0000 7fd3c3853740 10 client.25832 fetch_fsmap learned FSMap version 30 2024-09-15T22:56:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.458+0000 7fd3c3853740 1 -- 172.21.15.111:0/2333262513 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5587bf3d2730 con 0x5587bf358a60 2024-09-15T22:56:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.458+0000 7fd39b400640 1 -- 172.21.15.111:0/2333262513 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3b4050070 con 0x5587bf358a60 2024-09-15T22:56:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.458+0000 7fd39b400640 1 -- 172.21.15.111:0/2333262513 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fd3b405e480 con 0x5587bf358a60 2024-09-15T22:56:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.458+0000 7fd3c3853740 10 client.25832 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:56:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.458+0000 7fd3c3853740 10 client.25832 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:56:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.458+0000 7fd3c3853740 20 client.25832 populate_metadata read hostname 'smithi111' 2024-09-15T22:56:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.458+0000 7fd3c3853740 1 --2- 172.21.15.111:0/2333262513 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5587bf3d3030 0x5587bf3d5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.458+0000 7fd3c3853740 4 client.25832 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:56:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.458+0000 7fd3c3853740 1 -- 172.21.15.111:0/2333262513 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5587bf23c7c0 con 0x5587bf3d3030 2024-09-15T22:56:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.459+0000 7fd3bde00640 1 --2- 172.21.15.111:0/2333262513 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5587bf3d3030 0x5587bf3d5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.459+0000 7fd3bde00640 1 --2- 172.21.15.111:0/2333262513 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5587bf3d3030 0x5587bf3d5450 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3a51b3bcaac06bf server_cookie=9f8ece9cb3047f98 in_seq=0 out_seq=0 2024-09-15T22:56:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.459+0000 7fd39b400640 10 client.25832 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:56:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.470+0000 7fd39b400640 1 -- 172.21.15.111:0/2333262513 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5587bf23c7c0 con 0x5587bf3d3030 2024-09-15T22:56:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.470+0000 7fd39b400640 10 client.25832 handle_command_reply: tid=0 2024-09-15T22:56:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.470+0000 7fd3c3853740 1 client.25832 shutdown 2024-09-15T22:56:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.470+0000 7fd3c3853740 1 -- 172.21.15.111:0/2333262513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587bf358a60 msgr2=0x5587bf361480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.470+0000 7fd3c3853740 1 --2- 172.21.15.111:0/2333262513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587bf358a60 0x5587bf361480 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4009be0 tx=0x7fd3b402a6c0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.470+0000 7fd3c3853740 1 -- 172.21.15.111:0/2333262513 shutdown_connections 2024-09-15T22:56:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.470+0000 7fd3c3853740 1 --2- 172.21.15.111:0/2333262513 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5587bf3d3030 0x5587bf3d5450 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.470+0000 7fd3c3853740 1 --2- 172.21.15.111:0/2333262513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5587bf358a60 0x5587bf361480 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.470+0000 7fd3c3853740 1 --2- 172.21.15.111:0/2333262513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5587bf3539d0 0x5587bf360f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.470+0000 7fd39b400640 0 client.25832 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:56:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.470+0000 7fd3c3853740 1 --2- 172.21.15.111:0/2333262513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5587bf353090 0x5587bf364450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.470+0000 7fd3c3853740 1 -- 172.21.15.111:0/2333262513 >> 172.21.15.111:0/2333262513 conn(0x5587bf34e700 msgr2=0x5587bf34d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:56:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.470+0000 7fd3c3853740 1 -- 172.21.15.111:0/2333262513 shutdown_connections 2024-09-15T22:56:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.470+0000 7fd3c3853740 1 -- 172.21.15.111:0/2333262513 wait complete. 2024-09-15T22:56:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.471+0000 7fd3c3853740 20 client.25832 trim_cache size 0 max 16384 2024-09-15T22:56:34.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:56:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:56:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:56:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:56:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.471+0000 7fd3c0800640 1 -- 172.21.15.111:0/1329302920 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd3a00a3370 msgr2=0x7fd3a00a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.471+0000 7fd3c0800640 1 --2- 172.21.15.111:0/1329302920 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd3a00a3370 0x7fd3a00a5830 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd3b402abe0 tx=0x7fd3b4005fd0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.471+0000 7fd3c0800640 1 -- 172.21.15.111:0/1329302920 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3b8104f50 msgr2=0x7fd3b819a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.471+0000 7fd3c0800640 1 --2- 172.21.15.111:0/1329302920 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3b8104f50 0x7fd3b819a490 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800e9c0 tx=0x7fd3a800ee80 comp rx=0 tx=0).stop 2024-09-15T22:56:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.471+0000 7fd3c0800640 1 -- 172.21.15.111:0/1329302920 shutdown_connections 2024-09-15T22:56:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.471+0000 7fd3c0800640 1 --2- 172.21.15.111:0/1329302920 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3b8105910 0x7fd3b81a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.471+0000 7fd3c0800640 1 --2- 172.21.15.111:0/1329302920 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd3a00a3370 0x7fd3a00a5830 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.471+0000 7fd3c0800640 1 --2- 172.21.15.111:0/1329302920 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3b8104f50 0x7fd3b819a490 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.471+0000 7fd3c0800640 1 --2- 172.21.15.111:0/1329302920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3b8103d60 0x7fd3b8199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.471+0000 7fd3c0800640 1 -- 172.21.15.111:0/1329302920 >> 172.21.15.111:0/1329302920 conn(0x7fd3b80ff4e0 msgr2=0x7fd3b8108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:56:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.471+0000 7fd3c0800640 1 -- 172.21.15.111:0/1329302920 shutdown_connections 2024-09-15T22:56:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:34.471+0000 7fd3c0800640 1 -- 172.21.15.111:0/1329302920 wait complete. 2024-09-15T22:56:35.317 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1014398 9.97 MB/sec execute 423 sec latency 239.435 ms 2024-09-15T22:56:35.914 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:35 smithi111 ceph-mon[30629]: pgmap v670: 97 pgs: 97 active+clean; 283 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 223 op/s 2024-09-15T22:56:36.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:35 smithi192 ceph-mon[35427]: pgmap v670: 97 pgs: 97 active+clean; 283 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 223 op/s 2024-09-15T22:56:36.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:35 smithi121 ceph-mon[34815]: pgmap v670: 97 pgs: 97 active+clean; 283 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 223 op/s 2024-09-15T22:56:36.317 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1014398 9.95 MB/sec execute 424 sec latency 1239.483 ms 2024-09-15T22:56:37.317 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1014764 9.92 MB/sec execute 425 sec latency 1277.017 ms 2024-09-15T22:56:38.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:37 smithi192 ceph-mon[35427]: pgmap v671: 97 pgs: 97 active+clean; 285 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 198 op/s 2024-09-15T22:56:38.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:37 smithi121 ceph-mon[34815]: pgmap v671: 97 pgs: 97 active+clean; 285 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 198 op/s 2024-09-15T22:56:38.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:37 smithi111 ceph-mon[30629]: pgmap v671: 97 pgs: 97 active+clean; 285 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 198 op/s 2024-09-15T22:56:38.317 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1016760 9.93 MB/sec execute 426 sec latency 110.325 ms 2024-09-15T22:56:39.317 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1019664 9.95 MB/sec execute 427 sec latency 207.953 ms 2024-09-15T22:56:40.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:39 smithi192 ceph-mon[35427]: pgmap v672: 97 pgs: 97 active+clean; 286 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 216 op/s 2024-09-15T22:56:40.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:39 smithi121 ceph-mon[34815]: pgmap v672: 97 pgs: 97 active+clean; 286 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 216 op/s 2024-09-15T22:56:40.167 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:39 smithi111 ceph-mon[30629]: pgmap v672: 97 pgs: 97 active+clean; 286 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 216 op/s 2024-09-15T22:56:40.317 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1021238 9.94 MB/sec execute 428 sec latency 123.957 ms 2024-09-15T22:56:41.317 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1021238 9.92 MB/sec execute 429 sec latency 1124.001 ms 2024-09-15T22:56:41.881 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:41 smithi111 ceph-mon[30629]: pgmap v673: 97 pgs: 97 active+clean; 285 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 180 op/s 2024-09-15T22:56:42.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:41 smithi192 ceph-mon[35427]: pgmap v673: 97 pgs: 97 active+clean; 285 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 180 op/s 2024-09-15T22:56:42.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:41 smithi121 ceph-mon[34815]: pgmap v673: 97 pgs: 97 active+clean; 285 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 180 op/s 2024-09-15T22:56:42.318 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1021690 9.90 MB/sec execute 430 sec latency 1143.602 ms 2024-09-15T22:56:43.317 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1023929 9.91 MB/sec execute 431 sec latency 12.195 ms 2024-09-15T22:56:44.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:43 smithi192 ceph-mon[35427]: pgmap v674: 97 pgs: 97 active+clean; 286 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 210 op/s 2024-09-15T22:56:44.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:43 smithi121 ceph-mon[34815]: pgmap v674: 97 pgs: 97 active+clean; 286 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 210 op/s 2024-09-15T22:56:44.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:43 smithi111 ceph-mon[30629]: pgmap v674: 97 pgs: 97 active+clean; 286 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 210 op/s 2024-09-15T22:56:44.317 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1026934 9.93 MB/sec execute 432 sec latency 11.439 ms 2024-09-15T22:56:45.318 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1029453 9.93 MB/sec execute 433 sec latency 35.679 ms 2024-09-15T22:56:45.856 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:45 smithi111 ceph-mon[30629]: pgmap v675: 97 pgs: 97 active+clean; 286 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 182 op/s 2024-09-15T22:56:46.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:45 smithi192 ceph-mon[35427]: pgmap v675: 97 pgs: 97 active+clean; 286 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 182 op/s 2024-09-15T22:56:46.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:45 smithi121 ceph-mon[34815]: pgmap v675: 97 pgs: 97 active+clean; 286 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 182 op/s 2024-09-15T22:56:46.318 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1029455 9.91 MB/sec execute 434 sec latency 929.322 ms 2024-09-15T22:56:47.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:56:47.082 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:56:47.121 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:56:47.318 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1030477 9.90 MB/sec execute 435 sec latency 664.038 ms 2024-09-15T22:56:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.589+0000 7fa6ae200640 1 -- 172.21.15.111:0/1454250500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6a8101950 msgr2=0x7fa6a8101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.589+0000 7fa6ae200640 1 --2- 172.21.15.111:0/1454250500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6a8101950 0x7fa6a8101d50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009900 tx=0x7fa69c02a400 comp rx=0 tx=0).stop 2024-09-15T22:56:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.589+0000 7fa6ae200640 1 -- 172.21.15.111:0/1454250500 shutdown_connections 2024-09-15T22:56:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.589+0000 7fa6ae200640 1 --2- 172.21.15.111:0/1454250500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6a8103500 0x7fa6a8109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.589+0000 7fa6ae200640 1 --2- 172.21.15.111:0/1454250500 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6a8102b40 0x7fa6a8102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.589+0000 7fa6ae200640 1 --2- 172.21.15.111:0/1454250500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6a8101950 0x7fa6a8101d50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.589+0000 7fa6ae200640 1 -- 172.21.15.111:0/1454250500 >> 172.21.15.111:0/1454250500 conn(0x7fa6a80fd0d0 msgr2=0x7fa6a80ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:56:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.589+0000 7fa6ae200640 1 -- 172.21.15.111:0/1454250500 shutdown_connections 2024-09-15T22:56:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.589+0000 7fa6ae200640 1 -- 172.21.15.111:0/1454250500 wait complete. 2024-09-15T22:56:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.590+0000 7fa6ae200640 1 Processor -- start 2024-09-15T22:56:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.590+0000 7fa6ae200640 1 -- start start 2024-09-15T22:56:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.590+0000 7fa6ae200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6a8101950 0x7fa6a8197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.591+0000 7fa6ae200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6a8102b40 0x7fa6a81981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.591+0000 7fa6ae200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6a8103500 0x7fa6a819f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.591+0000 7fa6ae200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a810d5d0 con 0x7fa6a8103500 2024-09-15T22:56:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.591+0000 7fa6ae200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a810d450 con 0x7fa6a8101950 2024-09-15T22:56:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.591+0000 7fa6ae200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a810d750 con 0x7fa6a8102b40 2024-09-15T22:56:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.591+0000 7fa6ace00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6a8101950 0x7fa6a8197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.591+0000 7fa6ace00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6a8101950 0x7fa6a8197cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:46580/0 (socket says 172.21.15.111:46580) 2024-09-15T22:56:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.591+0000 7fa6ace00640 1 -- 172.21.15.111:0/677640427 learned_addr learned my addr 172.21.15.111:0/677640427 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:56:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.591+0000 7fa6ad800640 1 --2- 172.21.15.111:0/677640427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6a8103500 0x7fa6a819f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.591+0000 7fa6a7e00640 1 --2- 172.21.15.111:0/677640427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6a8102b40 0x7fa6a81981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.591+0000 7fa6ace00640 1 -- 172.21.15.111:0/677640427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6a8102b40 msgr2=0x7fa6a81981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.591+0000 7fa6ace00640 1 --2- 172.21.15.111:0/677640427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6a8102b40 0x7fa6a81981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.591+0000 7fa6ace00640 1 -- 172.21.15.111:0/677640427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6a8103500 msgr2=0x7fa6a819f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.591+0000 7fa6ace00640 1 --2- 172.21.15.111:0/677640427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6a8103500 0x7fa6a819f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.591+0000 7fa6ace00640 1 -- 172.21.15.111:0/677640427 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a819f8f0 con 0x7fa6a8101950 2024-09-15T22:56:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.591+0000 7fa6ace00640 1 --2- 172.21.15.111:0/677640427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6a8101950 0x7fa6a8197cb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa69c031610 tx=0x7fa69c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:56:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.592+0000 7fa6a5600640 1 -- 172.21.15.111:0/677640427 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c03f030 con 0x7fa6a8101950 2024-09-15T22:56:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.592+0000 7fa6a5600640 1 -- 172.21.15.111:0/677640427 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa69c003ad0 con 0x7fa6a8101950 2024-09-15T22:56:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.592+0000 7fa6ae200640 1 -- 172.21.15.111:0/677640427 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a819fb80 con 0x7fa6a8101950 2024-09-15T22:56:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.592+0000 7fa6a5600640 1 -- 172.21.15.111:0/677640427 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c035070 con 0x7fa6a8101950 2024-09-15T22:56:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.592+0000 7fa6ae200640 1 -- 172.21.15.111:0/677640427 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a81a0070 con 0x7fa6a8101950 2024-09-15T22:56:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.595+0000 7fa6b283a740 1 Processor -- start 2024-09-15T22:56:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.595+0000 7fa6b283a740 1 -- start start 2024-09-15T22:56:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.595+0000 7fa6a5600640 1 -- 172.21.15.111:0/677640427 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa69c042030 con 0x7fa6a8101950 2024-09-15T22:56:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.595+0000 7fa6b283a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d647fb2670 0x55d647fb2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.595+0000 7fa6b283a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d647fb2fb0 0x55d647fb7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.595+0000 7fa6b283a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d647fb8040 0x55d647fba500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.595+0000 7fa6b283a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d647f13100 con 0x55d647fb2670 2024-09-15T22:56:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.595+0000 7fa6b283a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d647ebb1b0 con 0x55d647fb2fb0 2024-09-15T22:56:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.595+0000 7fa6b283a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d647f0f0c0 con 0x55d647fb8040 2024-09-15T22:56:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.595+0000 7fa6ad800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d647fb2670 0x55d647fb2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.596+0000 7fa6ace00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d647fb2fb0 0x55d647fb7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.596+0000 7fa6ad800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d647fb2670 0x55d647fb2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45966/0 (socket says 172.21.15.111:45966) 2024-09-15T22:56:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.596+0000 7fa6ad800640 1 -- 172.21.15.111:0/2460376803 learned_addr learned my addr 172.21.15.111:0/2460376803 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:56:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.596+0000 7fa6a7e00640 1 --2- 172.21.15.111:0/2460376803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d647fb8040 0x55d647fba500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.596+0000 7fa6ad800640 1 -- 172.21.15.111:0/2460376803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d647fb8040 msgr2=0x55d647fba500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.596+0000 7fa6ad800640 1 --2- 172.21.15.111:0/2460376803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d647fb8040 0x55d647fba500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.596+0000 7fa6ad800640 1 -- 172.21.15.111:0/2460376803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d647fb2fb0 msgr2=0x55d647fb7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.596+0000 7fa6ad800640 1 --2- 172.21.15.111:0/2460376803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d647fb2fb0 0x55d647fb7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.596+0000 7fa6ad800640 1 -- 172.21.15.111:0/2460376803 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d647c26210 con 0x55d647fb2670 2024-09-15T22:56:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.596+0000 7fa6ad800640 1 --2- 172.21.15.111:0/2460376803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d647fb2670 0x55d647fb2a70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009fd0 tx=0x7fa6a00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36518228e92b55de server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:56:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.596+0000 7fa69a000640 1 -- 172.21.15.111:0/2460376803 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0004d20 con 0x55d647fb2670 2024-09-15T22:56:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.597+0000 7fa69a000640 1 -- 172.21.15.111:0/2460376803 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a0004050 con 0x55d647fb2670 2024-09-15T22:56:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.597+0000 7fa69a000640 1 -- 172.21.15.111:0/2460376803 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0005220 con 0x55d647fb2670 2024-09-15T22:56:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.597+0000 7fa6a5600640 1 --2- 172.21.15.111:0/677640427 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa6900a3240 0x7fa6900a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.597+0000 7fa6a5600640 1 -- 172.21.15.111:0/677640427 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fa69c0efe00 con 0x7fa6a8101950 2024-09-15T22:56:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.597+0000 7fa6ace00640 1 --2- 172.21.15.111:0/677640427 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa6900a3240 0x7fa6900a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.597+0000 7fa6b283a740 1 -- 172.21.15.111:0/2460376803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d647fb2670 msgr2=0x55d647fb2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.597+0000 7fa6b283a740 1 --2- 172.21.15.111:0/2460376803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d647fb2670 0x55d647fb2a70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009fd0 tx=0x7fa6a00118d0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.597+0000 7fa6b283a740 1 -- 172.21.15.111:0/2460376803 shutdown_connections 2024-09-15T22:56:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.597+0000 7fa6b283a740 1 --2- 172.21.15.111:0/2460376803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d647fb8040 0x55d647fba500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.597+0000 7fa6b283a740 1 --2- 172.21.15.111:0/2460376803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d647fb2fb0 0x55d647fb7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.597+0000 7fa6b283a740 1 --2- 172.21.15.111:0/2460376803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d647fb2670 0x55d647fb2a70 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.597+0000 7fa6b283a740 1 -- 172.21.15.111:0/2460376803 >> 172.21.15.111:0/2460376803 conn(0x55d647fa98d0 msgr2=0x55d647fa9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:56:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.597+0000 7fa6b283a740 1 -- 172.21.15.111:0/2460376803 shutdown_connections 2024-09-15T22:56:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.597+0000 7fa6b283a740 1 -- 172.21.15.111:0/2460376803 wait complete. 2024-09-15T22:56:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.598+0000 7fa6ace00640 1 --2- 172.21.15.111:0/677640427 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa6900a3240 0x7fa6900a5700 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0bc270 tx=0x7fa69c0f3920 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:56:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.598+0000 7fa6b283a740 1 -- start start 2024-09-15T22:56:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.598+0000 7fa6b283a740 1 Processor -- start 2024-09-15T22:56:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.598+0000 7fa6b283a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d647fb2670 0x55d647fbc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.598+0000 7fa6b283a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d647fb2fb0 0x55d647fbccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.598+0000 7fa6b283a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d647fb8040 0x55d647fc3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.598+0000 7fa6b283a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d647fbafb0 con 0x55d647fb2670 2024-09-15T22:56:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.598+0000 7fa6b283a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d647e9c090 con 0x55d647fb8040 2024-09-15T22:56:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.598+0000 7fa6b283a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d647fbb130 con 0x55d647fb2fb0 2024-09-15T22:56:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.599+0000 7fa6b283a740 10 client.0 _ll_register_callbacks cb 0x55d647ef0f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:56:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.599+0000 7fa6a7e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d647fb2fb0 0x55d647fbccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.599+0000 7fa6ad800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d647fb2670 0x55d647fbc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.599+0000 7fa6a7e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d647fb2fb0 0x55d647fbccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:48212/0 (socket says 172.21.15.111:48212) 2024-09-15T22:56:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.599+0000 7fa6a7e00640 1 -- 172.21.15.111:0/3474458817 learned_addr learned my addr 172.21.15.111:0/3474458817 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:56:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.599+0000 7fa6ad800640 1 --2- 172.21.15.111:0/3474458817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d647fb8040 0x55d647fc3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.599+0000 7fa6a7e00640 1 -- 172.21.15.111:0/3474458817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d647fb8040 msgr2=0x55d647fc3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.599+0000 7fa6a7e00640 1 --2- 172.21.15.111:0/3474458817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d647fb8040 0x55d647fc3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.599+0000 7fa6a7e00640 1 -- 172.21.15.111:0/3474458817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d647fb2670 msgr2=0x55d647fbc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.599+0000 7fa6a7e00640 1 --2- 172.21.15.111:0/3474458817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d647fb2670 0x55d647fbc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.599+0000 7fa6a7e00640 1 -- 172.21.15.111:0/3474458817 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d647cf2620 con 0x55d647fb2fb0 2024-09-15T22:56:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.599+0000 7fa6ad800640 1 --2- 172.21.15.111:0/3474458817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d647fb2670 0x55d647fbc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:56:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.599+0000 7fa6a7e00640 1 --2- 172.21.15.111:0/3474458817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d647fb2fb0 0x55d647fbccc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa694012700 tx=0x7fa694012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74645f7f81e398fd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:56:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.599+0000 7fa68ea00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:56:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.600+0000 7fa68ea00640 1 -- 172.21.15.111:0/3474458817 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69400f8c0 con 0x55d647fb2fb0 2024-09-15T22:56:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.600+0000 7fa6b283a740 1 -- 172.21.15.111:0/3474458817 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6a0004050 con 0x55d647fb2fb0 2024-09-15T22:56:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.600+0000 7fa68ea00640 1 -- 172.21.15.111:0/3474458817 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fa69400fa60 con 0x55d647fb2fb0 2024-09-15T22:56:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.600+0000 7fa68ea00640 1 -- 172.21.15.111:0/3474458817 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa69400fc60 con 0x55d647fb2fb0 2024-09-15T22:56:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.600+0000 7fa6b283a740 10 client.25867 fetch_fsmap learned FSMap version 30 2024-09-15T22:56:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.600+0000 7fa6b283a740 1 -- 172.21.15.111:0/3474458817 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d647fc6c10 con 0x55d647fb2fb0 2024-09-15T22:56:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.600+0000 7fa68ea00640 1 -- 172.21.15.111:0/3474458817 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a0004050 con 0x55d647fb2fb0 2024-09-15T22:56:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.600+0000 7fa68ea00640 1 -- 172.21.15.111:0/3474458817 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fa69401c420 con 0x55d647fb2fb0 2024-09-15T22:56:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.600+0000 7fa6b283a740 10 client.25867 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:56:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.600+0000 7fa6b283a740 10 client.25867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:56:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.600+0000 7fa6b283a740 20 client.25867 populate_metadata read hostname 'smithi111' 2024-09-15T22:56:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.600+0000 7fa6b283a740 1 --2- 172.21.15.111:0/3474458817 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d647fc8310 0x55d647fca770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.600+0000 7fa6b283a740 4 client.25867 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:56:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.600+0000 7fa6b283a740 1 -- 172.21.15.111:0/3474458817 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d647f20830 con 0x55d647fc8310 2024-09-15T22:56:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.601+0000 7fa6ad800640 1 --2- 172.21.15.111:0/3474458817 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d647fc8310 0x55d647fca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.601+0000 7fa6ad800640 1 --2- 172.21.15.111:0/3474458817 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d647fc8310 0x55d647fca770 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=439ca11931b7a82d server_cookie=d02c8ca0f8da6f90 in_seq=0 out_seq=0 2024-09-15T22:56:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.601+0000 7fa68ea00640 10 client.25867 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:56:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.602+0000 7fa68ea00640 1 -- 172.21.15.111:0/3474458817 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d647f20830 con 0x55d647fc8310 2024-09-15T22:56:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.602+0000 7fa68ea00640 10 client.25867 handle_command_reply: tid=0 2024-09-15T22:56:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.602+0000 7fa6b283a740 1 client.25867 shutdown 2024-09-15T22:56:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.602+0000 7fa6b283a740 1 -- 172.21.15.111:0/3474458817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d647fb2fb0 msgr2=0x55d647fbccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.602+0000 7fa6b283a740 1 --2- 172.21.15.111:0/3474458817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d647fb2fb0 0x55d647fbccc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa694012700 tx=0x7fa694012bc0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.602+0000 7fa6b283a740 1 -- 172.21.15.111:0/3474458817 shutdown_connections 2024-09-15T22:56:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.602+0000 7fa6b283a740 1 --2- 172.21.15.111:0/3474458817 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d647fc8310 0x55d647fca770 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.602+0000 7fa6b283a740 1 --2- 172.21.15.111:0/3474458817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d647fb8040 0x55d647fc3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.602+0000 7fa68ea00640 0 client.25867 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:56:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.602+0000 7fa6b283a740 1 --2- 172.21.15.111:0/3474458817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d647fb2fb0 0x55d647fbccc0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.602+0000 7fa6b283a740 1 --2- 172.21.15.111:0/3474458817 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d647fb2670 0x55d647fbc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.602+0000 7fa6b283a740 1 -- 172.21.15.111:0/3474458817 >> 172.21.15.111:0/3474458817 conn(0x55d647fa98d0 msgr2=0x55d647fab0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:56:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.602+0000 7fa6b283a740 1 -- 172.21.15.111:0/3474458817 shutdown_connections 2024-09-15T22:56:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.602+0000 7fa6b283a740 1 -- 172.21.15.111:0/3474458817 wait complete. 2024-09-15T22:56:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.602+0000 7fa6b283a740 20 client.25867 trim_cache size 0 max 16384 2024-09-15T22:56:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.618+0000 7fa6b283a740 1 Processor -- start 2024-09-15T22:56:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.618+0000 7fa6b283a740 1 -- start start 2024-09-15T22:56:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.618+0000 7fa6b283a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d647fb7b90 0x55d647fbbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.618+0000 7fa6b283a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d647fa98d0 0x55d647fc3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.618+0000 7fa6b283a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d647fb2670 0x55d647fa8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.618+0000 7fa6b283a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a81a9360 con 0x55d647fb7b90 2024-09-15T22:56:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.618+0000 7fa6b283a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a81a6c30 con 0x55d647fa98d0 2024-09-15T22:56:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.618+0000 7fa6b283a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a81a4aa0 con 0x55d647fb2670 2024-09-15T22:56:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.618+0000 7fa6ad800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d647fb7b90 0x55d647fbbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.618+0000 7fa6ad800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d647fb7b90 0x55d647fbbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45990/0 (socket says 172.21.15.111:45990) 2024-09-15T22:56:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.618+0000 7fa6ad800640 1 -- 172.21.15.111:0/3352222945 learned_addr learned my addr 172.21.15.111:0/3352222945 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:56:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.618+0000 7fa6a7e00640 1 --2- 172.21.15.111:0/3352222945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d647fa98d0 0x55d647fc3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.618+0000 7fa6ad800640 1 --2- 172.21.15.111:0/3352222945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d647fb2670 0x55d647fa8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.619+0000 7fa6ad800640 1 -- 172.21.15.111:0/3352222945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d647fb2670 msgr2=0x55d647fa8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.619+0000 7fa6ad800640 1 --2- 172.21.15.111:0/3352222945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d647fb2670 0x55d647fa8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.619+0000 7fa6ad800640 1 -- 172.21.15.111:0/3352222945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d647fa98d0 msgr2=0x55d647fc3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.619+0000 7fa6ad800640 1 --2- 172.21.15.111:0/3352222945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d647fa98d0 0x55d647fc3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.619+0000 7fa6ad800640 1 -- 172.21.15.111:0/3352222945 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d647fa96f0 con 0x55d647fb7b90 2024-09-15T22:56:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.619+0000 7fa6ad800640 1 --2- 172.21.15.111:0/3352222945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d647fb7b90 0x55d647fbbcb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0004b80 tx=0x7fa6a0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7953a0e4e145f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:56:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.619+0000 7fa699600640 1 -- 172.21.15.111:0/3352222945 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0004250 con 0x55d647fb7b90 2024-09-15T22:56:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.619+0000 7fa699600640 1 -- 172.21.15.111:0/3352222945 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a00043f0 con 0x55d647fb7b90 2024-09-15T22:56:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.619+0000 7fa699600640 1 -- 172.21.15.111:0/3352222945 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0027440 con 0x55d647fb7b90 2024-09-15T22:56:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.620+0000 7fa6b283a740 1 -- 172.21.15.111:0/3352222945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d647fb7b90 msgr2=0x55d647fbbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.620+0000 7fa6b283a740 1 --2- 172.21.15.111:0/3352222945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d647fb7b90 0x55d647fbbcb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0004b80 tx=0x7fa6a0005fd0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.620+0000 7fa6b283a740 1 -- 172.21.15.111:0/3352222945 shutdown_connections 2024-09-15T22:56:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.620+0000 7fa6b283a740 1 --2- 172.21.15.111:0/3352222945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d647fb2670 0x55d647fa8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.620+0000 7fa6b283a740 1 --2- 172.21.15.111:0/3352222945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d647fa98d0 0x55d647fc3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.620+0000 7fa6b283a740 1 --2- 172.21.15.111:0/3352222945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d647fb7b90 0x55d647fbbcb0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.620+0000 7fa6b283a740 1 -- 172.21.15.111:0/3352222945 >> 172.21.15.111:0/3352222945 conn(0x55d647faa020 msgr2=0x55d647fa81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:56:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.620+0000 7fa6b283a740 1 -- 172.21.15.111:0/3352222945 shutdown_connections 2024-09-15T22:56:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.620+0000 7fa6b283a740 1 -- 172.21.15.111:0/3352222945 wait complete. 2024-09-15T22:56:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.620+0000 7fa6b283a740 1 -- start start 2024-09-15T22:56:47.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.620+0000 7fa6b283a740 1 Processor -- start 2024-09-15T22:56:47.625 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.621+0000 7fa6b283a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d647fa98d0 0x55d647fc1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:47.625 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.621+0000 7fa6b283a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d647fb2670 0x55d647fc1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.621+0000 7fa6b283a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d647fb7b90 0x55d647fc2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.621+0000 7fa6b283a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a81a50f0 con 0x55d647fb2670 2024-09-15T22:56:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.621+0000 7fa6b283a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a81a6560 con 0x55d647fa98d0 2024-09-15T22:56:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.621+0000 7fa6b283a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a81a57c0 con 0x55d647fb7b90 2024-09-15T22:56:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.621+0000 7fa6b283a740 10 client.0 _ll_register_callbacks cb 0x7fa694009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:56:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.621+0000 7fa6ad800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d647fa98d0 0x55d647fc1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.621+0000 7fa6a7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d647fb2670 0x55d647fc1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.621+0000 7fa6ad800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d647fa98d0 0x55d647fc1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:46614/0 (socket says 172.21.15.111:46614) 2024-09-15T22:56:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.621+0000 7fa6ad800640 1 -- 172.21.15.111:0/198058814 learned_addr learned my addr 172.21.15.111:0/198058814 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:56:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.621+0000 7fa6ad800640 1 --2- 172.21.15.111:0/198058814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d647fb7b90 0x55d647fc2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.621+0000 7fa6a7e00640 1 -- 172.21.15.111:0/198058814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d647fb7b90 msgr2=0x55d647fc2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.622+0000 7fa6a7e00640 1 --2- 172.21.15.111:0/198058814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d647fb7b90 0x55d647fc2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.622+0000 7fa6a7e00640 1 -- 172.21.15.111:0/198058814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d647fa98d0 msgr2=0x55d647fc1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:56:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.622+0000 7fa6a7e00640 1 --2- 172.21.15.111:0/198058814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d647fa98d0 0x55d647fc1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.622+0000 7fa6a7e00640 1 -- 172.21.15.111:0/198058814 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d647fc31f0 con 0x55d647fb2670 2024-09-15T22:56:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.622+0000 7fa6ad800640 1 --2- 172.21.15.111:0/198058814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d647fa98d0 0x55d647fc1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:56:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.622+0000 7fa6a7e00640 1 --2- 172.21.15.111:0/198058814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d647fb2670 0x55d647fc1bd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa694000c00 tx=0x7fa6940094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7446eb29e70b3fc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:56:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.622+0000 7fa68ea00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:56:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.622+0000 7fa68ea00640 1 -- 172.21.15.111:0/198058814 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69400f9a0 con 0x55d647fb2670 2024-09-15T22:56:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.622+0000 7fa6b283a740 1 -- 172.21.15.111:0/198058814 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6a00043f0 con 0x55d647fb2670 2024-09-15T22:56:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.622+0000 7fa68ea00640 1 -- 172.21.15.111:0/198058814 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fa69401cab0 con 0x55d647fb2670 2024-09-15T22:56:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.622+0000 7fa68ea00640 1 -- 172.21.15.111:0/198058814 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa694036050 con 0x55d647fb2670 2024-09-15T22:56:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.622+0000 7fa6b283a740 10 client.15870 fetch_fsmap learned FSMap version 30 2024-09-15T22:56:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.622+0000 7fa6b283a740 1 -- 172.21.15.111:0/198058814 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d6480333e0 con 0x55d647fb2670 2024-09-15T22:56:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.623+0000 7fa68ea00640 1 -- 172.21.15.111:0/198058814 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a00043f0 con 0x55d647fb2670 2024-09-15T22:56:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.623+0000 7fa68ea00640 1 -- 172.21.15.111:0/198058814 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fa6940035f0 con 0x55d647fb2670 2024-09-15T22:56:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.623+0000 7fa6b283a740 10 client.15870 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:56:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.623+0000 7fa6b283a740 10 client.15870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:56:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.623+0000 7fa6b283a740 20 client.15870 populate_metadata read hostname 'smithi111' 2024-09-15T22:56:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.623+0000 7fa6b283a740 1 --2- 172.21.15.111:0/198058814 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d647fb2fb0 0x55d648035da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:56:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.623+0000 7fa6b283a740 4 client.15870 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:56:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.623+0000 7fa6b283a740 1 -- 172.21.15.111:0/198058814 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d647f1ab80 con 0x55d647fb2fb0 2024-09-15T22:56:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.623+0000 7fa6ad800640 1 --2- 172.21.15.111:0/198058814 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d647fb2fb0 0x55d648035da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:56:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.624+0000 7fa6ad800640 1 --2- 172.21.15.111:0/198058814 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d647fb2fb0 0x55d648035da0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2d48265b02b0a96 server_cookie=8c38e2fd249b6f82 in_seq=0 out_seq=0 2024-09-15T22:56:47.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.624+0000 7fa68ea00640 10 client.15870 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:56:47.638 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.636+0000 7fa68ea00640 1 -- 172.21.15.111:0/198058814 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d647f1ab80 con 0x55d647fb2fb0 2024-09-15T22:56:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.636+0000 7fa68ea00640 10 client.15870 handle_command_reply: tid=0 2024-09-15T22:56:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.636+0000 7fa6b283a740 1 client.15870 shutdown 2024-09-15T22:56:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.637+0000 7fa6b283a740 1 -- 172.21.15.111:0/198058814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d647fb2670 msgr2=0x55d647fc1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.637+0000 7fa6b283a740 1 --2- 172.21.15.111:0/198058814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d647fb2670 0x55d647fc1bd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa694000c00 tx=0x7fa6940094a0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.637+0000 7fa6b283a740 1 -- 172.21.15.111:0/198058814 shutdown_connections 2024-09-15T22:56:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.637+0000 7fa6b283a740 1 --2- 172.21.15.111:0/198058814 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d647fb2fb0 0x55d648035da0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.637+0000 7fa6b283a740 1 --2- 172.21.15.111:0/198058814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d647fb7b90 0x55d647fc2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.637+0000 7fa6b283a740 1 --2- 172.21.15.111:0/198058814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d647fb2670 0x55d647fc1bd0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.637+0000 7fa68ea00640 0 client.15870 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:56:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.637+0000 7fa6b283a740 1 --2- 172.21.15.111:0/198058814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d647fa98d0 0x55d647fc1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.639 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.637+0000 7fa6b283a740 1 -- 172.21.15.111:0/198058814 >> 172.21.15.111:0/198058814 conn(0x55d647faa020 msgr2=0x55d647fbbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:56:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.637+0000 7fa6b283a740 1 -- 172.21.15.111:0/198058814 shutdown_connections 2024-09-15T22:56:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.637+0000 7fa6b283a740 1 -- 172.21.15.111:0/198058814 wait complete. 2024-09-15T22:56:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.637+0000 7fa6b283a740 20 client.15870 trim_cache size 0 max 16384 2024-09-15T22:56:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:56:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:56:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:56:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:56:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.638+0000 7fa6ae200640 1 -- 172.21.15.111:0/677640427 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa6900a3240 msgr2=0x7fa6900a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.638+0000 7fa6ae200640 1 --2- 172.21.15.111:0/677640427 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa6900a3240 0x7fa6900a5700 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0bc270 tx=0x7fa69c0f3920 comp rx=0 tx=0).stop 2024-09-15T22:56:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.638+0000 7fa6ae200640 1 -- 172.21.15.111:0/677640427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6a8101950 msgr2=0x7fa6a8197cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:56:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.638+0000 7fa6ae200640 1 --2- 172.21.15.111:0/677640427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6a8101950 0x7fa6a8197cb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa69c031610 tx=0x7fa69c004270 comp rx=0 tx=0).stop 2024-09-15T22:56:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.638+0000 7fa6ae200640 1 -- 172.21.15.111:0/677640427 shutdown_connections 2024-09-15T22:56:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.638+0000 7fa6ae200640 1 --2- 172.21.15.111:0/677640427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6a8103500 0x7fa6a819f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.638+0000 7fa6ae200640 1 --2- 172.21.15.111:0/677640427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6a8102b40 0x7fa6a81981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.638+0000 7fa6ae200640 1 --2- 172.21.15.111:0/677640427 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa6900a3240 0x7fa6900a5700 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.638+0000 7fa6ae200640 1 --2- 172.21.15.111:0/677640427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6a8101950 0x7fa6a8197cb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:56:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.638+0000 7fa6ae200640 1 -- 172.21.15.111:0/677640427 >> 172.21.15.111:0/677640427 conn(0x7fa6a80fd0d0 msgr2=0x7fa6a8105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:56:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.638+0000 7fa6ae200640 1 -- 172.21.15.111:0/677640427 shutdown_connections 2024-09-15T22:56:47.642 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:56:47.638+0000 7fa6ae200640 1 -- 172.21.15.111:0/677640427 wait complete. 2024-09-15T22:56:48.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:47 smithi192 ceph-mon[35427]: pgmap v676: 97 pgs: 97 active+clean; 288 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 186 op/s 2024-09-15T22:56:48.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:47 smithi121 ceph-mon[34815]: pgmap v676: 97 pgs: 97 active+clean; 288 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 186 op/s 2024-09-15T22:56:48.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:47 smithi111 ceph-mon[30629]: pgmap v676: 97 pgs: 97 active+clean; 288 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 186 op/s 2024-09-15T22:56:48.318 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1032586 9.90 MB/sec execute 436 sec latency 15.084 ms 2024-09-15T22:56:49.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:48 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:56:49.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:48 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:56:49.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:48 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:56:49.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:48 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:56:49.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:48 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:56:49.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:48 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:56:49.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:48 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:56:49.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:48 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:56:49.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:48 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:56:49.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:48 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:56:49.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:48 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:56:49.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:48 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:56:49.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:48 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:56:49.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:48 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:56:49.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:48 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:56:49.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:48 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:56:49.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:48 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:56:49.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:48 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:56:49.318 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1034661 9.90 MB/sec execute 437 sec latency 13.832 ms 2024-09-15T22:56:50.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:49 smithi121 ceph-mon[34815]: pgmap v677: 97 pgs: 97 active+clean; 288 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 187 op/s 2024-09-15T22:56:50.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:49 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:56:50.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:49 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:56:50.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:49 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:56:50.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:49 smithi111 ceph-mon[30629]: pgmap v677: 97 pgs: 97 active+clean; 288 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 187 op/s 2024-09-15T22:56:50.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:49 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:56:50.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:49 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:56:50.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:49 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:56:50.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:49 smithi192 ceph-mon[35427]: pgmap v677: 97 pgs: 97 active+clean; 288 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 187 op/s 2024-09-15T22:56:50.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:49 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:56:50.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:49 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:56:50.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:49 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:56:50.318 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1037617 9.92 MB/sec execute 438 sec latency 9.382 ms 2024-09-15T22:56:51.318 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1037724 9.90 MB/sec execute 439 sec latency 535.089 ms 2024-09-15T22:56:51.854 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:51 smithi111 ceph-mon[30629]: pgmap v678: 97 pgs: 97 active+clean; 294 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 209 op/s 2024-09-15T22:56:52.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:51 smithi121 ceph-mon[34815]: pgmap v678: 97 pgs: 97 active+clean; 294 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 209 op/s 2024-09-15T22:56:52.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:51 smithi192 ceph-mon[35427]: pgmap v678: 97 pgs: 97 active+clean; 294 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 209 op/s 2024-09-15T22:56:52.318 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1037724 9.88 MB/sec execute 440 sec latency 1535.157 ms 2024-09-15T22:56:53.318 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1040670 9.89 MB/sec execute 441 sec latency 1581.600 ms 2024-09-15T22:56:54.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:53 smithi121 ceph-mon[34815]: pgmap v679: 97 pgs: 97 active+clean; 293 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 206 op/s 2024-09-15T22:56:54.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:53 smithi111 ceph-mon[30629]: pgmap v679: 97 pgs: 97 active+clean; 293 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 206 op/s 2024-09-15T22:56:54.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:53 smithi192 ceph-mon[35427]: pgmap v679: 97 pgs: 97 active+clean; 293 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 206 op/s 2024-09-15T22:56:54.318 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1043259 9.89 MB/sec execute 442 sec latency 14.854 ms 2024-09-15T22:56:55.318 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1045299 9.90 MB/sec execute 443 sec latency 81.777 ms 2024-09-15T22:56:56.115 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:55 smithi111 ceph-mon[30629]: pgmap v680: 97 pgs: 97 active+clean; 295 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 197 op/s 2024-09-15T22:56:56.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:55 smithi121 ceph-mon[34815]: pgmap v680: 97 pgs: 97 active+clean; 295 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 197 op/s 2024-09-15T22:56:56.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:55 smithi192 ceph-mon[35427]: pgmap v680: 97 pgs: 97 active+clean; 295 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 197 op/s 2024-09-15T22:56:56.318 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1045903 9.89 MB/sec execute 444 sec latency 801.453 ms 2024-09-15T22:56:57.318 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1047345 9.88 MB/sec execute 445 sec latency 893.251 ms 2024-09-15T22:56:58.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:57 smithi121 ceph-mon[34815]: pgmap v681: 97 pgs: 97 active+clean; 294 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 205 op/s 2024-09-15T22:56:58.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:57 smithi111 ceph-mon[30629]: pgmap v681: 97 pgs: 97 active+clean; 294 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 205 op/s 2024-09-15T22:56:58.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:57 smithi192 ceph-mon[35427]: pgmap v681: 97 pgs: 97 active+clean; 294 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 205 op/s 2024-09-15T22:56:58.320 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1049509 9.88 MB/sec execute 446 sec latency 19.692 ms 2024-09-15T22:56:59.319 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1052170 9.89 MB/sec execute 447 sec latency 11.425 ms 2024-09-15T22:57:00.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:56:59 smithi121 ceph-mon[34815]: pgmap v682: 97 pgs: 97 active+clean; 294 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 170 op/s 2024-09-15T22:57:00.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:56:59 smithi111 ceph-mon[30629]: pgmap v682: 97 pgs: 97 active+clean; 294 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 170 op/s 2024-09-15T22:57:00.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:56:59 smithi192 ceph-mon[35427]: pgmap v682: 97 pgs: 97 active+clean; 294 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 170 op/s 2024-09-15T22:57:00.319 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1055213 9.91 MB/sec execute 448 sec latency 46.363 ms 2024-09-15T22:57:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.785+0000 7efcb2e00640 1 -- 172.21.15.111:0/2737546792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcac104f50 msgr2=0x7efcac1053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.785+0000 7efcb2e00640 1 --2- 172.21.15.111:0/2737546792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcac104f50 0x7efcac1053d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efc98009860 tx=0x7efc9802a360 comp rx=0 tx=0).stop 2024-09-15T22:57:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.785+0000 7efcb2e00640 1 -- 172.21.15.111:0/2737546792 shutdown_connections 2024-09-15T22:57:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.785+0000 7efcb2e00640 1 --2- 172.21.15.111:0/2737546792 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcac105910 0x7efcac10c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.785+0000 7efcb2e00640 1 --2- 172.21.15.111:0/2737546792 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcac104f50 0x7efcac1053d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.785+0000 7efcb2e00640 1 --2- 172.21.15.111:0/2737546792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcac103d60 0x7efcac104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.785+0000 7efcb2e00640 1 -- 172.21.15.111:0/2737546792 >> 172.21.15.111:0/2737546792 conn(0x7efcac0ff4e0 msgr2=0x7efcac101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.785+0000 7efcb2e00640 1 -- 172.21.15.111:0/2737546792 shutdown_connections 2024-09-15T22:57:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.785+0000 7efcb2e00640 1 -- 172.21.15.111:0/2737546792 wait complete. 2024-09-15T22:57:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.786+0000 7efcb2e00640 1 Processor -- start 2024-09-15T22:57:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.786+0000 7efcb2e00640 1 -- start start 2024-09-15T22:57:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.787+0000 7efcb2e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcac103d60 0x7efcac19a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.787+0000 7efcb2e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcac104f50 0x7efcac19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.787+0000 7efcb2e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcac105910 0x7efcac1a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.787+0000 7efcb2e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efcac10fa60 con 0x7efcac105910 2024-09-15T22:57:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.787+0000 7efcb2e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efcac10f8e0 con 0x7efcac104f50 2024-09-15T22:57:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.787+0000 7efcb2e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efcac10fbe0 con 0x7efcac103d60 2024-09-15T22:57:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.787+0000 7efcab400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcac104f50 0x7efcac19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:00.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.787+0000 7efcabe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcac103d60 0x7efcac19a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.787+0000 7efcab400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcac104f50 0x7efcac19a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:45534/0 (socket says 172.21.15.111:45534) 2024-09-15T22:57:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.787+0000 7efcab400640 1 -- 172.21.15.111:0/3229605720 learned_addr learned my addr 172.21.15.111:0/3229605720 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:57:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.787+0000 7efcb0e00640 1 --2- 172.21.15.111:0/3229605720 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcac105910 0x7efcac1a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.787+0000 7efcab400640 1 -- 172.21.15.111:0/3229605720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcac103d60 msgr2=0x7efcac19a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.787+0000 7efcab400640 1 --2- 172.21.15.111:0/3229605720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcac103d60 0x7efcac19a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.787+0000 7efcab400640 1 -- 172.21.15.111:0/3229605720 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcac105910 msgr2=0x7efcac1a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.787+0000 7efcab400640 1 --2- 172.21.15.111:0/3229605720 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcac105910 0x7efcac1a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.787+0000 7efcab400640 1 -- 172.21.15.111:0/3229605720 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcac1a1c40 con 0x7efcac104f50 2024-09-15T22:57:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.787+0000 7efcab400640 1 --2- 172.21.15.111:0/3229605720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcac104f50 0x7efcac19a560 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efc9802a950 tx=0x7efc98032830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.788+0000 7efca8c00640 1 -- 172.21.15.111:0/3229605720 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc98037040 con 0x7efcac104f50 2024-09-15T22:57:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.788+0000 7efca8c00640 1 -- 172.21.15.111:0/3229605720 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc98004050 con 0x7efcac104f50 2024-09-15T22:57:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.788+0000 7efca8c00640 1 -- 172.21.15.111:0/3229605720 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc980055b0 con 0x7efcac104f50 2024-09-15T22:57:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.788+0000 7efcb2e00640 1 -- 172.21.15.111:0/3229605720 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcac1a1ed0 con 0x7efcac104f50 2024-09-15T22:57:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.788+0000 7efcb2e00640 1 -- 172.21.15.111:0/3229605720 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcac1a22e0 con 0x7efcac104f50 2024-09-15T22:57:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.791+0000 7efca8c00640 1 -- 172.21.15.111:0/3229605720 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7efc98004200 con 0x7efcac104f50 2024-09-15T22:57:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.791+0000 7efcb5bcb740 1 Processor -- start 2024-09-15T22:57:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.791+0000 7efcb5bcb740 1 -- start start 2024-09-15T22:57:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.791+0000 7efcb5bcb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f456d4090 0x562f456d4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.791+0000 7efcb5bcb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562f456d49d0 0x562f456d9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.791+0000 7efcb5bcb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562f456d9a60 0x562f456dbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.792+0000 7efcb5bcb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562f45639100 con 0x562f456d9a60 2024-09-15T22:57:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.792+0000 7efcb5bcb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562f455e11b0 con 0x562f456d4090 2024-09-15T22:57:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.792+0000 7efcb5bcb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562f456350c0 con 0x562f456d49d0 2024-09-15T22:57:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.792+0000 7efcabe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562f456d49d0 0x562f456d9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.792+0000 7efcabe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562f456d49d0 0x562f456d9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:38534/0 (socket says 172.21.15.111:38534) 2024-09-15T22:57:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.792+0000 7efcb0e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f456d4090 0x562f456d4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.792+0000 7efcabe00640 1 -- 172.21.15.111:0/2328749068 learned_addr learned my addr 172.21.15.111:0/2328749068 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:57:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.792+0000 7efcab400640 1 --2- 172.21.15.111:0/2328749068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562f456d9a60 0x562f456dbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.792+0000 7efcabe00640 1 -- 172.21.15.111:0/2328749068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f456d4090 msgr2=0x562f456d4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.792+0000 7efcabe00640 1 --2- 172.21.15.111:0/2328749068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f456d4090 0x562f456d4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.792+0000 7efcabe00640 1 -- 172.21.15.111:0/2328749068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562f456d9a60 msgr2=0x562f456dbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.792+0000 7efcabe00640 1 --2- 172.21.15.111:0/2328749068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562f456d9a60 0x562f456dbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.792+0000 7efcabe00640 1 -- 172.21.15.111:0/2328749068 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f4534c210 con 0x562f456d49d0 2024-09-15T22:57:00.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.792+0000 7efcabe00640 1 --2- 172.21.15.111:0/2328749068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562f456d49d0 0x562f456d9490 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009c80 tx=0x7efc9c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4e06516dc9b687f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:00.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.793+0000 7efca8c00640 1 --2- 172.21.15.111:0/3229605720 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7efc900a3370 0x7efc900a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:00.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.793+0000 7efca1600640 1 -- 172.21.15.111:0/2328749068 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c005220 con 0x562f456d49d0 2024-09-15T22:57:00.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.793+0000 7efca1600640 1 -- 172.21.15.111:0/2328749068 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc9c009e30 con 0x562f456d49d0 2024-09-15T22:57:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.793+0000 7efca1600640 1 -- 172.21.15.111:0/2328749068 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c019890 con 0x562f456d49d0 2024-09-15T22:57:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.793+0000 7efca8c00640 1 -- 172.21.15.111:0/3229605720 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7efc980ef530 con 0x7efcac104f50 2024-09-15T22:57:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.793+0000 7efcb0e00640 1 --2- 172.21.15.111:0/3229605720 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7efc900a3370 0x7efc900a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.793+0000 7efcb5bcb740 1 -- 172.21.15.111:0/2328749068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562f456d49d0 msgr2=0x562f456d9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.793+0000 7efcb5bcb740 1 --2- 172.21.15.111:0/2328749068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562f456d49d0 0x562f456d9490 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009c80 tx=0x7efc9c011810 comp rx=0 tx=0).stop 2024-09-15T22:57:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.794+0000 7efcb5bcb740 1 -- 172.21.15.111:0/2328749068 shutdown_connections 2024-09-15T22:57:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.794+0000 7efcb5bcb740 1 --2- 172.21.15.111:0/2328749068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562f456d9a60 0x562f456dbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.794+0000 7efcb5bcb740 1 --2- 172.21.15.111:0/2328749068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562f456d49d0 0x562f456d9490 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.794+0000 7efcb5bcb740 1 --2- 172.21.15.111:0/2328749068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f456d4090 0x562f456d4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.794+0000 7efcb5bcb740 1 -- 172.21.15.111:0/2328749068 >> 172.21.15.111:0/2328749068 conn(0x562f456cf700 msgr2=0x562f456cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:00.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.794+0000 7efcb0e00640 1 --2- 172.21.15.111:0/3229605720 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7efc900a3370 0x7efc900a5830 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efca400a450 tx=0x7efca400d040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.794+0000 7efcb5bcb740 1 -- 172.21.15.111:0/2328749068 shutdown_connections 2024-09-15T22:57:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.794+0000 7efcb5bcb740 1 -- 172.21.15.111:0/2328749068 wait complete. 2024-09-15T22:57:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.794+0000 7efcb5bcb740 1 -- start start 2024-09-15T22:57:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.794+0000 7efcb5bcb740 1 Processor -- start 2024-09-15T22:57:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.795+0000 7efcb5bcb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562f456d4090 0x562f456de270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.795+0000 7efcb5bcb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562f456d49d0 0x562f456de7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.795+0000 7efcb5bcb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f456d9a60 0x562f456e5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.795+0000 7efcb5bcb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562f456dca30 con 0x562f456d49d0 2024-09-15T22:57:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.795+0000 7efcb5bcb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562f455c2090 con 0x562f456d9a60 2024-09-15T22:57:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.795+0000 7efcb5bcb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562f456dcbb0 con 0x562f456d4090 2024-09-15T22:57:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.795+0000 7efcb5bcb740 10 client.0 _ll_register_callbacks cb 0x562f45616f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:57:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.795+0000 7efcab400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562f456d49d0 0x562f456de7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.795+0000 7efcab400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562f456d49d0 0x562f456de7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57622/0 (socket says 172.21.15.111:57622) 2024-09-15T22:57:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.795+0000 7efcabe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562f456d4090 0x562f456de270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.795+0000 7efcab400640 1 -- 172.21.15.111:0/1114521113 learned_addr learned my addr 172.21.15.111:0/1114521113 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:57:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.795+0000 7efcab400640 1 -- 172.21.15.111:0/1114521113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562f456d4090 msgr2=0x562f456de270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.795+0000 7efcab400640 1 --2- 172.21.15.111:0/1114521113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562f456d4090 0x562f456de270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.795+0000 7efcab400640 1 -- 172.21.15.111:0/1114521113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f456d9a60 msgr2=0x562f456e5780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:57:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.795+0000 7efcab400640 1 --2- 172.21.15.111:0/1114521113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f456d9a60 0x562f456e5780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.795+0000 7efcab400640 1 -- 172.21.15.111:0/1114521113 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f45418620 con 0x562f456d49d0 2024-09-15T22:57:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.796+0000 7efcab400640 1 --2- 172.21.15.111:0/1114521113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562f456d49d0 0x562f456de7b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efc980c4290 tx=0x7efc980b7eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05836d41d91175b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.796+0000 7efc96000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:57:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.796+0000 7efc96000640 1 -- 172.21.15.111:0/1114521113 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc980f2e60 con 0x562f456d49d0 2024-09-15T22:57:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.796+0000 7efc96000640 1 -- 172.21.15.111:0/1114521113 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7efc980ef0d0 con 0x562f456d49d0 2024-09-15T22:57:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.796+0000 7efcb5bcb740 1 -- 172.21.15.111:0/1114521113 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc9c009e30 con 0x562f456d49d0 2024-09-15T22:57:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.796+0000 7efc96000640 1 -- 172.21.15.111:0/1114521113 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efc980d3020 con 0x562f456d49d0 2024-09-15T22:57:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.796+0000 7efcb5bcb740 10 client.15882 fetch_fsmap learned FSMap version 30 2024-09-15T22:57:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.796+0000 7efcb5bcb740 1 -- 172.21.15.111:0/1114521113 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562f456e8570 con 0x562f456d49d0 2024-09-15T22:57:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.796+0000 7efc96000640 1 -- 172.21.15.111:0/1114521113 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc9c009e30 con 0x562f456d49d0 2024-09-15T22:57:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.796+0000 7efc96000640 1 -- 172.21.15.111:0/1114521113 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7efc9803a070 con 0x562f456d49d0 2024-09-15T22:57:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.796+0000 7efcb5bcb740 10 client.15882 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:57:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.797+0000 7efcb5bcb740 10 client.15882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:57:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.797+0000 7efcb5bcb740 20 client.15882 populate_metadata read hostname 'smithi111' 2024-09-15T22:57:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.797+0000 7efcb5bcb740 1 --2- 172.21.15.111:0/1114521113 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x562f456e9c80 0x562f456ec070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.797+0000 7efcb5bcb740 4 client.15882 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:57:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.797+0000 7efcb5bcb740 1 -- 172.21.15.111:0/1114521113 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f45646830 con 0x562f456e9c80 2024-09-15T22:57:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.797+0000 7efcabe00640 1 --2- 172.21.15.111:0/1114521113 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x562f456e9c80 0x562f456ec070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.797+0000 7efcabe00640 1 --2- 172.21.15.111:0/1114521113 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x562f456e9c80 0x562f456ec070 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c1fbcaa3ebae91 server_cookie=340c225ac11737fe in_seq=0 out_seq=0 2024-09-15T22:57:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.797+0000 7efc96000640 10 client.15882 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:57:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.798+0000 7efc96000640 1 -- 172.21.15.111:0/1114521113 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f45646830 con 0x562f456e9c80 2024-09-15T22:57:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.798+0000 7efc96000640 10 client.15882 handle_command_reply: tid=0 2024-09-15T22:57:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.798+0000 7efcb5bcb740 1 client.15882 shutdown 2024-09-15T22:57:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.798+0000 7efcb5bcb740 1 -- 172.21.15.111:0/1114521113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562f456d49d0 msgr2=0x562f456de7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.798+0000 7efcb5bcb740 1 --2- 172.21.15.111:0/1114521113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562f456d49d0 0x562f456de7b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7efc980c4290 tx=0x7efc980b7eb0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.798+0000 7efcb5bcb740 1 -- 172.21.15.111:0/1114521113 shutdown_connections 2024-09-15T22:57:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.798+0000 7efcb5bcb740 1 --2- 172.21.15.111:0/1114521113 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x562f456e9c80 0x562f456ec070 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.798+0000 7efcb5bcb740 1 --2- 172.21.15.111:0/1114521113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f456d9a60 0x562f456e5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.798+0000 7efc96000640 0 client.15882 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:57:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.798+0000 7efcb5bcb740 1 --2- 172.21.15.111:0/1114521113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562f456d49d0 0x562f456de7b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.798+0000 7efcb5bcb740 1 --2- 172.21.15.111:0/1114521113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562f456d4090 0x562f456de270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.798+0000 7efcb5bcb740 1 -- 172.21.15.111:0/1114521113 >> 172.21.15.111:0/1114521113 conn(0x562f456cf700 msgr2=0x562f456d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.799+0000 7efcb5bcb740 1 -- 172.21.15.111:0/1114521113 shutdown_connections 2024-09-15T22:57:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.799+0000 7efcb5bcb740 1 -- 172.21.15.111:0/1114521113 wait complete. 2024-09-15T22:57:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.799+0000 7efcb5bcb740 20 client.15882 trim_cache size 0 max 16384 2024-09-15T22:57:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.813+0000 7efcb5bcb740 1 Processor -- start 2024-09-15T22:57:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.813+0000 7efcb5bcb740 1 -- start start 2024-09-15T22:57:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.814+0000 7efcb5bcb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562f456d4090 0x562f456de270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.814+0000 7efcb5bcb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f456d49d0 0x562f456de7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.814+0000 7efcb5bcb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562f456d9a60 0x562f456e5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.814+0000 7efcb5bcb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efcac1ad810 con 0x562f456d9a60 2024-09-15T22:57:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.814+0000 7efcb5bcb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efcac1ab0e0 con 0x562f456d49d0 2024-09-15T22:57:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.814+0000 7efcb5bcb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efcac1a8f30 con 0x562f456d4090 2024-09-15T22:57:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.814+0000 7efcb0e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562f456d9a60 0x562f456e5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.814+0000 7efcb0e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562f456d9a60 0x562f456e5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57634/0 (socket says 172.21.15.111:57634) 2024-09-15T22:57:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.814+0000 7efcb0e00640 1 -- 172.21.15.111:0/746147929 learned_addr learned my addr 172.21.15.111:0/746147929 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:57:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.814+0000 7efcabe00640 1 --2- 172.21.15.111:0/746147929 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562f456d4090 0x562f456de270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.814+0000 7efcb0e00640 1 -- 172.21.15.111:0/746147929 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562f456d4090 msgr2=0x562f456de270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.814+0000 7efcb0e00640 1 --2- 172.21.15.111:0/746147929 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562f456d4090 0x562f456de270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.814+0000 7efcb0e00640 1 -- 172.21.15.111:0/746147929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f456d49d0 msgr2=0x562f456de7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:57:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.814+0000 7efcb0e00640 1 --2- 172.21.15.111:0/746147929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f456d49d0 0x562f456de7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.814+0000 7efcb0e00640 1 -- 172.21.15.111:0/746147929 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f456cf520 con 0x562f456d9a60 2024-09-15T22:57:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.815+0000 7efcb0e00640 1 --2- 172.21.15.111:0/746147929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562f456d9a60 0x562f456e5740 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efca4009ef0 tx=0x7efca4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2b4bc0b84a1279 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.815+0000 7efca0c00640 1 -- 172.21.15.111:0/746147929 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4016a10 con 0x562f456d9a60 2024-09-15T22:57:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.815+0000 7efca0c00640 1 -- 172.21.15.111:0/746147929 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efca4027070 con 0x562f456d9a60 2024-09-15T22:57:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.815+0000 7efca0c00640 1 -- 172.21.15.111:0/746147929 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4017620 con 0x562f456d9a60 2024-09-15T22:57:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.815+0000 7efcb5bcb740 1 -- 172.21.15.111:0/746147929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562f456d9a60 msgr2=0x562f456e5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.815+0000 7efcb5bcb740 1 --2- 172.21.15.111:0/746147929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562f456d9a60 0x562f456e5740 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efca4009ef0 tx=0x7efca4014000 comp rx=0 tx=0).stop 2024-09-15T22:57:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.815+0000 7efcb5bcb740 1 -- 172.21.15.111:0/746147929 shutdown_connections 2024-09-15T22:57:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.815+0000 7efcb5bcb740 1 --2- 172.21.15.111:0/746147929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562f456d9a60 0x562f456e5740 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.815+0000 7efcb5bcb740 1 --2- 172.21.15.111:0/746147929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f456d49d0 0x562f456de7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.816+0000 7efcb5bcb740 1 --2- 172.21.15.111:0/746147929 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562f456d4090 0x562f456de270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.816+0000 7efcb5bcb740 1 -- 172.21.15.111:0/746147929 >> 172.21.15.111:0/746147929 conn(0x562f456cf700 msgr2=0x562f456d9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.816+0000 7efcb5bcb740 1 -- 172.21.15.111:0/746147929 shutdown_connections 2024-09-15T22:57:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.816+0000 7efcb5bcb740 1 -- 172.21.15.111:0/746147929 wait complete. 2024-09-15T22:57:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.816+0000 7efcb5bcb740 1 -- start start 2024-09-15T22:57:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.816+0000 7efcb5bcb740 1 Processor -- start 2024-09-15T22:57:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.817+0000 7efcb5bcb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562f456d4090 0x562f456e2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.817+0000 7efcb5bcb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562f456d49d0 0x562f456e3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.817+0000 7efcb5bcb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f456d9a60 0x562f456e4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.817+0000 7efcb5bcb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efcac1a95a0 con 0x562f456d4090 2024-09-15T22:57:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.817+0000 7efcb5bcb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efcac1aaa10 con 0x562f456d9a60 2024-09-15T22:57:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.817+0000 7efcb5bcb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efcac1a9c70 con 0x562f456d49d0 2024-09-15T22:57:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.817+0000 7efcb5bcb740 10 client.0 _ll_register_callbacks cb 0x7efc980097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:57:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.817+0000 7efcabe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562f456d4090 0x562f456e2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.817+0000 7efcabe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562f456d4090 0x562f456e2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57640/0 (socket says 172.21.15.111:57640) 2024-09-15T22:57:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.817+0000 7efcabe00640 1 -- 172.21.15.111:0/1639753767 learned_addr learned my addr 172.21.15.111:0/1639753767 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:57:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.817+0000 7efcab400640 1 --2- 172.21.15.111:0/1639753767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562f456d49d0 0x562f456e3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.817+0000 7efcabe00640 1 -- 172.21.15.111:0/1639753767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562f456d49d0 msgr2=0x562f456e3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.817+0000 7efcabe00640 1 --2- 172.21.15.111:0/1639753767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562f456d49d0 0x562f456e3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.817+0000 7efcabe00640 1 -- 172.21.15.111:0/1639753767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f456d9a60 msgr2=0x562f456e4110 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:57:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.817+0000 7efcabe00640 1 --2- 172.21.15.111:0/1639753767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f456d9a60 0x562f456e4110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.817+0000 7efcabe00640 1 -- 172.21.15.111:0/1639753767 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f456e4720 con 0x562f456d4090 2024-09-15T22:57:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.818+0000 7efcabe00640 1 --2- 172.21.15.111:0/1639753767 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562f456d4090 0x562f456e2c10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7efc9c01b040 tx=0x7efc9c009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f321eab0a51d19 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.818+0000 7efc96000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:57:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.818+0000 7efc96000640 1 -- 172.21.15.111:0/1639753767 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c0198d0 con 0x562f456d4090 2024-09-15T22:57:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.818+0000 7efcb5bcb740 1 -- 172.21.15.111:0/1639753767 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efca4027070 con 0x562f456d4090 2024-09-15T22:57:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.818+0000 7efc96000640 1 -- 172.21.15.111:0/1639753767 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7efc9c019a70 con 0x562f456d4090 2024-09-15T22:57:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.818+0000 7efc96000640 1 -- 172.21.15.111:0/1639753767 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efc9c019c90 con 0x562f456d4090 2024-09-15T22:57:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.818+0000 7efcb5bcb740 10 client.15888 fetch_fsmap learned FSMap version 30 2024-09-15T22:57:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.818+0000 7efcb5bcb740 1 -- 172.21.15.111:0/1639753767 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562f456e5aa0 con 0x562f456d4090 2024-09-15T22:57:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.818+0000 7efc96000640 1 -- 172.21.15.111:0/1639753767 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efca4027070 con 0x562f456d4090 2024-09-15T22:57:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.819+0000 7efc96000640 1 -- 172.21.15.111:0/1639753767 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7efc9c02b040 con 0x562f456d4090 2024-09-15T22:57:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.819+0000 7efcb5bcb740 10 client.15888 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:57:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.819+0000 7efcb5bcb740 10 client.15888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:57:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.819+0000 7efcb5bcb740 20 client.15888 populate_metadata read hostname 'smithi111' 2024-09-15T22:57:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.819+0000 7efcb5bcb740 1 --2- 172.21.15.111:0/1639753767 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x562f45754e80 0x562f457572a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.819+0000 7efcb5bcb740 4 client.15888 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:57:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.819+0000 7efcb5bcb740 1 -- 172.21.15.111:0/1639753767 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562f45640b80 con 0x562f45754e80 2024-09-15T22:57:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.819+0000 7efcab400640 1 --2- 172.21.15.111:0/1639753767 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x562f45754e80 0x562f457572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.820+0000 7efc96000640 10 client.15888 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:57:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.820+0000 7efcab400640 1 --2- 172.21.15.111:0/1639753767 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x562f45754e80 0x562f457572a0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=734b8648798da8f7 server_cookie=7a0fe038e855c0ed in_seq=0 out_seq=0 2024-09-15T22:57:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.831+0000 7efc96000640 1 -- 172.21.15.111:0/1639753767 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562f45640b80 con 0x562f45754e80 2024-09-15T22:57:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.831+0000 7efc96000640 10 client.15888 handle_command_reply: tid=0 2024-09-15T22:57:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.831+0000 7efcb5bcb740 1 client.15888 shutdown 2024-09-15T22:57:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.831+0000 7efcb5bcb740 1 -- 172.21.15.111:0/1639753767 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562f456d4090 msgr2=0x562f456e2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.831+0000 7efcb5bcb740 1 --2- 172.21.15.111:0/1639753767 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562f456d4090 0x562f456e2c10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7efc9c01b040 tx=0x7efc9c009590 comp rx=0 tx=0).stop 2024-09-15T22:57:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.832+0000 7efcb5bcb740 1 -- 172.21.15.111:0/1639753767 shutdown_connections 2024-09-15T22:57:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.832+0000 7efcb5bcb740 1 --2- 172.21.15.111:0/1639753767 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x562f45754e80 0x562f457572a0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.832+0000 7efcb5bcb740 1 --2- 172.21.15.111:0/1639753767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562f456d9a60 0x562f456e4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.832+0000 7efcb5bcb740 1 --2- 172.21.15.111:0/1639753767 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562f456d49d0 0x562f456e3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.832+0000 7efc96000640 0 client.15888 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:57:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.832+0000 7efcb5bcb740 1 --2- 172.21.15.111:0/1639753767 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562f456d4090 0x562f456e2c10 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.832+0000 7efcb5bcb740 1 -- 172.21.15.111:0/1639753767 >> 172.21.15.111:0/1639753767 conn(0x562f456cf700 msgr2=0x562f456d8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.832+0000 7efcb5bcb740 1 -- 172.21.15.111:0/1639753767 shutdown_connections 2024-09-15T22:57:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.832+0000 7efcb5bcb740 1 -- 172.21.15.111:0/1639753767 wait complete. 2024-09-15T22:57:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.832+0000 7efcb5bcb740 20 client.15888 trim_cache size 0 max 16384 2024-09-15T22:57:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:57:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:57:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:57:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:57:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.833+0000 7efcb2e00640 1 -- 172.21.15.111:0/3229605720 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7efc900a3370 msgr2=0x7efc900a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.833+0000 7efcb2e00640 1 --2- 172.21.15.111:0/3229605720 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7efc900a3370 0x7efc900a5830 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efca400a450 tx=0x7efca400d040 comp rx=0 tx=0).stop 2024-09-15T22:57:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.833+0000 7efcb2e00640 1 -- 172.21.15.111:0/3229605720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcac104f50 msgr2=0x7efcac19a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.833+0000 7efcb2e00640 1 --2- 172.21.15.111:0/3229605720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcac104f50 0x7efcac19a560 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efc9802a950 tx=0x7efc98032830 comp rx=0 tx=0).stop 2024-09-15T22:57:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.833+0000 7efcb2e00640 1 -- 172.21.15.111:0/3229605720 shutdown_connections 2024-09-15T22:57:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.833+0000 7efcb2e00640 1 --2- 172.21.15.111:0/3229605720 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcac105910 0x7efcac1a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.833+0000 7efcb2e00640 1 --2- 172.21.15.111:0/3229605720 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7efc900a3370 0x7efc900a5830 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.833+0000 7efcb2e00640 1 --2- 172.21.15.111:0/3229605720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcac104f50 0x7efcac19a560 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.833+0000 7efcb2e00640 1 --2- 172.21.15.111:0/3229605720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcac103d60 0x7efcac19a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.833+0000 7efcb2e00640 1 -- 172.21.15.111:0/3229605720 >> 172.21.15.111:0/3229605720 conn(0x7efcac0ff4e0 msgr2=0x7efcac1082d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.833+0000 7efcb2e00640 1 -- 172.21.15.111:0/3229605720 shutdown_connections 2024-09-15T22:57:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:00.833+0000 7efcb2e00640 1 -- 172.21.15.111:0/3229605720 wait complete. 2024-09-15T22:57:01.319 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1055675 9.89 MB/sec execute 449 sec latency 652.258 ms 2024-09-15T22:57:02.047 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:01 smithi111 ceph-mon[30629]: pgmap v683: 97 pgs: 97 active+clean; 296 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 203 op/s 2024-09-15T22:57:02.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:01 smithi121 ceph-mon[34815]: pgmap v683: 97 pgs: 97 active+clean; 296 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 203 op/s 2024-09-15T22:57:02.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:01 smithi192 ceph-mon[35427]: pgmap v683: 97 pgs: 97 active+clean; 296 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 203 op/s 2024-09-15T22:57:02.319 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1055722 9.87 MB/sec execute 450 sec latency 1641.820 ms 2024-09-15T22:57:03.319 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1058827 9.89 MB/sec execute 451 sec latency 13.330 ms 2024-09-15T22:57:04.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:03 smithi121 ceph-mon[34815]: pgmap v684: 97 pgs: 97 active+clean; 295 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 160 op/s 2024-09-15T22:57:04.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:03 smithi111 ceph-mon[30629]: pgmap v684: 97 pgs: 97 active+clean; 295 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 160 op/s 2024-09-15T22:57:04.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:03 smithi192 ceph-mon[35427]: pgmap v684: 97 pgs: 97 active+clean; 295 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 160 op/s 2024-09-15T22:57:04.319 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1061375 9.89 MB/sec execute 452 sec latency 15.494 ms 2024-09-15T22:57:05.319 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1063599 9.90 MB/sec execute 453 sec latency 19.493 ms 2024-09-15T22:57:06.032 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:05 smithi111 ceph-mon[30629]: pgmap v685: 97 pgs: 97 active+clean; 289 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 196 op/s 2024-09-15T22:57:06.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:05 smithi192 ceph-mon[35427]: pgmap v685: 97 pgs: 97 active+clean; 289 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 196 op/s 2024-09-15T22:57:06.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:05 smithi121 ceph-mon[34815]: pgmap v685: 97 pgs: 97 active+clean; 289 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 196 op/s 2024-09-15T22:57:06.319 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1065342 9.90 MB/sec execute 454 sec latency 567.076 ms 2024-09-15T22:57:07.319 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1065344 9.87 MB/sec execute 455 sec latency 1391.197 ms 2024-09-15T22:57:08.074 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:07 smithi111 ceph-mon[30629]: pgmap v686: 97 pgs: 97 active+clean; 276 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 177 op/s 2024-09-15T22:57:08.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:07 smithi121 ceph-mon[34815]: pgmap v686: 97 pgs: 97 active+clean; 276 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 177 op/s 2024-09-15T22:57:08.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:07 smithi192 ceph-mon[35427]: pgmap v686: 97 pgs: 97 active+clean; 276 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 177 op/s 2024-09-15T22:57:08.323 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1067123 9.88 MB/sec execute 456 sec latency 308.967 ms 2024-09-15T22:57:09.319 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1069835 9.89 MB/sec execute 457 sec latency 6.707 ms 2024-09-15T22:57:10.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:09 smithi121 ceph-mon[34815]: pgmap v687: 97 pgs: 97 active+clean; 271 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 215 op/s 2024-09-15T22:57:10.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:09 smithi111 ceph-mon[30629]: pgmap v687: 97 pgs: 97 active+clean; 271 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 215 op/s 2024-09-15T22:57:10.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:09 smithi192 ceph-mon[35427]: pgmap v687: 97 pgs: 97 active+clean; 271 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 215 op/s 2024-09-15T22:57:10.319 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1073025 9.91 MB/sec execute 458 sec latency 11.214 ms 2024-09-15T22:57:11.319 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1074166 9.90 MB/sec execute 459 sec latency 434.897 ms 2024-09-15T22:57:12.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:11 smithi121 ceph-mon[34815]: pgmap v688: 97 pgs: 97 active+clean; 271 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 216 op/s 2024-09-15T22:57:12.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:11 smithi111 ceph-mon[30629]: pgmap v688: 97 pgs: 97 active+clean; 271 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 216 op/s 2024-09-15T22:57:12.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:11 smithi192 ceph-mon[35427]: pgmap v688: 97 pgs: 97 active+clean; 271 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 216 op/s 2024-09-15T22:57:12.319 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1075282 9.88 MB/sec execute 460 sec latency 1116.212 ms 2024-09-15T22:57:13.319 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1077780 9.89 MB/sec execute 461 sec latency 12.348 ms 2024-09-15T22:57:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.957+0000 7f73df200640 1 -- 172.21.15.111:0/1896074027 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73e0102b40 msgr2=0x7f73e0102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.957+0000 7f73df200640 1 --2- 172.21.15.111:0/1896074027 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73e0102b40 0x7f73e0102fc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009900 tx=0x7f73cc02a3f0 comp rx=0 tx=0).stop 2024-09-15T22:57:13.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.957+0000 7f73df200640 1 -- 172.21.15.111:0/1896074027 shutdown_connections 2024-09-15T22:57:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.957+0000 7f73df200640 1 --2- 172.21.15.111:0/1896074027 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73e0103500 0x7f73e0109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.957+0000 7f73df200640 1 --2- 172.21.15.111:0/1896074027 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73e0102b40 0x7f73e0102fc0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.957+0000 7f73df200640 1 --2- 172.21.15.111:0/1896074027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73e0101950 0x7f73e0101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.957+0000 7f73df200640 1 -- 172.21.15.111:0/1896074027 >> 172.21.15.111:0/1896074027 conn(0x7f73e00fd0d0 msgr2=0x7f73e00ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.958+0000 7f73df200640 1 -- 172.21.15.111:0/1896074027 shutdown_connections 2024-09-15T22:57:13.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.958+0000 7f73df200640 1 -- 172.21.15.111:0/1896074027 wait complete. 2024-09-15T22:57:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.958+0000 7f73df200640 1 Processor -- start 2024-09-15T22:57:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.959+0000 7f73df200640 1 -- start start 2024-09-15T22:57:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.959+0000 7f73df200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73e0101950 0x7f73e0197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.959+0000 7f73df200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73e0102b40 0x7f73e01981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.959+0000 7f73df200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73e0103500 0x7f73e019f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.959+0000 7f73df200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f73e010d620 con 0x7f73e0103500 2024-09-15T22:57:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.959+0000 7f73df200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f73e010d4a0 con 0x7f73e0102b40 2024-09-15T22:57:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.959+0000 7f73df200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f73e010d7a0 con 0x7f73e0101950 2024-09-15T22:57:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.959+0000 7f73dde00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73e0101950 0x7f73e0197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.959+0000 7f73dd400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73e0102b40 0x7f73e01981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.959+0000 7f73dde00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73e0101950 0x7f73e0197cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:50416/0 (socket says 172.21.15.111:50416) 2024-09-15T22:57:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.959+0000 7f73dde00640 1 -- 172.21.15.111:0/1595342624 learned_addr learned my addr 172.21.15.111:0/1595342624 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:57:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.959+0000 7f73dd400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73e0102b40 0x7f73e01981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:46326/0 (socket says 172.21.15.111:46326) 2024-09-15T22:57:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.959+0000 7f73de800640 1 --2- 172.21.15.111:0/1595342624 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73e0103500 0x7f73e019f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.959+0000 7f73dde00640 1 -- 172.21.15.111:0/1595342624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73e0102b40 msgr2=0x7f73e01981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.959+0000 7f73dde00640 1 --2- 172.21.15.111:0/1595342624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73e0102b40 0x7f73e01981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.959+0000 7f73dde00640 1 -- 172.21.15.111:0/1595342624 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73e0103500 msgr2=0x7f73e019f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.959+0000 7f73dde00640 1 --2- 172.21.15.111:0/1595342624 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73e0103500 0x7f73e019f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.959+0000 7f73dde00640 1 -- 172.21.15.111:0/1595342624 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e019f8f0 con 0x7f73e0101950 2024-09-15T22:57:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.959+0000 7f73de800640 1 --2- 172.21.15.111:0/1595342624 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73e0103500 0x7f73e019f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:57:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.960+0000 7f73dde00640 1 --2- 172.21.15.111:0/1595342624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73e0101950 0x7f73e0197cb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f73d000e9c0 tx=0x7f73d000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.960+0000 7f73d6a00640 1 -- 172.21.15.111:0/1595342624 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d000cd80 con 0x7f73e0101950 2024-09-15T22:57:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.960+0000 7f73d6a00640 1 -- 172.21.15.111:0/1595342624 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d00035f0 con 0x7f73e0101950 2024-09-15T22:57:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.960+0000 7f73d6a00640 1 -- 172.21.15.111:0/1595342624 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0010690 con 0x7f73e0101950 2024-09-15T22:57:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.960+0000 7f73df200640 1 -- 172.21.15.111:0/1595342624 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e019fbe0 con 0x7f73e0101950 2024-09-15T22:57:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.960+0000 7f73df200640 1 -- 172.21.15.111:0/1595342624 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e01a00b0 con 0x7f73e0101950 2024-09-15T22:57:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.963+0000 7f73e7a7c740 1 Processor -- start 2024-09-15T22:57:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.963+0000 7f73e7a7c740 1 -- start start 2024-09-15T22:57:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.963+0000 7f73e7a7c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df469b9260 0x55df469b9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.963+0000 7f73e7a7c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df469b9ba0 0x55df469be660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.963+0000 7f73e7a7c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df469bec30 0x55df469c10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.963+0000 7f73e7a7c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55df4691e100 con 0x55df469b9ba0 2024-09-15T22:57:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.963+0000 7f73e7a7c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55df468c61b0 con 0x55df469bec30 2024-09-15T22:57:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.963+0000 7f73e7a7c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55df4691a0c0 con 0x55df469b9260 2024-09-15T22:57:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.964+0000 7f73d6a00640 1 -- 172.21.15.111:0/1595342624 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f73d00037a0 con 0x7f73e0101950 2024-09-15T22:57:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.964+0000 7f73de800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df469b9260 0x55df469b9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.964+0000 7f73dd400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df469bec30 0x55df469c10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.964+0000 7f73de800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df469b9260 0x55df469b9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:50432/0 (socket says 172.21.15.111:50432) 2024-09-15T22:57:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.964+0000 7f73de800640 1 -- 172.21.15.111:0/2064695913 learned_addr learned my addr 172.21.15.111:0/2064695913 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:57:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.964+0000 7f73dde00640 1 --2- 172.21.15.111:0/2064695913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df469b9ba0 0x55df469be660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.964+0000 7f73de800640 1 -- 172.21.15.111:0/2064695913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df469bec30 msgr2=0x55df469c10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.964+0000 7f73de800640 1 --2- 172.21.15.111:0/2064695913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df469bec30 0x55df469c10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.964+0000 7f73de800640 1 -- 172.21.15.111:0/2064695913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df469b9ba0 msgr2=0x55df469be660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.964+0000 7f73de800640 1 --2- 172.21.15.111:0/2064695913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df469b9ba0 0x55df469be660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.964+0000 7f73de800640 1 -- 172.21.15.111:0/2064695913 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df46631210 con 0x55df469b9260 2024-09-15T22:57:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.964+0000 7f73dde00640 1 --2- 172.21.15.111:0/2064695913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df469b9ba0 0x55df469be660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:57:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.964+0000 7f73de800640 1 --2- 172.21.15.111:0/2064695913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df469b9260 0x55df469b9660 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f73d800f410 tx=0x7f73d800f8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28dd558d12c395a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.965+0000 7f73cb000640 1 -- 172.21.15.111:0/2064695913 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8015070 con 0x55df469b9260 2024-09-15T22:57:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.965+0000 7f73cb000640 1 -- 172.21.15.111:0/2064695913 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d8011be0 con 0x55df469b9260 2024-09-15T22:57:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.965+0000 7f73cb000640 1 -- 172.21.15.111:0/2064695913 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8010d30 con 0x55df469b9260 2024-09-15T22:57:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.965+0000 7f73d6a00640 1 --2- 172.21.15.111:0/1595342624 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f73c40a32c0 0x7f73c40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.965+0000 7f73e7a7c740 1 -- 172.21.15.111:0/2064695913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df469b9260 msgr2=0x55df469b9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.965+0000 7f73e7a7c740 1 --2- 172.21.15.111:0/2064695913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df469b9260 0x55df469b9660 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f73d800f410 tx=0x7f73d800f8d0 comp rx=0 tx=0).stop 2024-09-15T22:57:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.965+0000 7f73d6a00640 1 -- 172.21.15.111:0/1595342624 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f73d00cfca0 con 0x7f73e0101950 2024-09-15T22:57:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.965+0000 7f73e7a7c740 1 -- 172.21.15.111:0/2064695913 shutdown_connections 2024-09-15T22:57:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.965+0000 7f73e7a7c740 1 --2- 172.21.15.111:0/2064695913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df469bec30 0x55df469c10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.965+0000 7f73e7a7c740 1 --2- 172.21.15.111:0/2064695913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df469b9ba0 0x55df469be660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.965+0000 7f73e7a7c740 1 --2- 172.21.15.111:0/2064695913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df469b9260 0x55df469b9660 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.965+0000 7f73e7a7c740 1 -- 172.21.15.111:0/2064695913 >> 172.21.15.111:0/2064695913 conn(0x55df469b48d0 msgr2=0x55df469b4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.965+0000 7f73dde00640 1 --2- 172.21.15.111:0/1595342624 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f73c40a32c0 0x7f73c40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.966+0000 7f73e7a7c740 1 -- 172.21.15.111:0/2064695913 shutdown_connections 2024-09-15T22:57:13.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.966+0000 7f73e7a7c740 1 -- 172.21.15.111:0/2064695913 wait complete. 2024-09-15T22:57:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.966+0000 7f73e7a7c740 1 -- start start 2024-09-15T22:57:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.966+0000 7f73dde00640 1 --2- 172.21.15.111:0/1595342624 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f73c40a32c0 0x7f73c40a5780 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f73d0098c90 tx=0x7f73d0003f10 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.966+0000 7f73e7a7c740 1 Processor -- start 2024-09-15T22:57:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.967+0000 7f73e7a7c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df469b9260 0x55df469c7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.967+0000 7f73e7a7c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df469b9ba0 0x55df469c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:13.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.967+0000 7f73e7a7c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df469bec30 0x55df469cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.967+0000 7f73e7a7c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55df469c1c00 con 0x55df469bec30 2024-09-15T22:57:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.967+0000 7f73e7a7c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55df468a7090 con 0x55df469b9260 2024-09-15T22:57:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.967+0000 7f73e7a7c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55df469c1d80 con 0x55df469b9ba0 2024-09-15T22:57:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.967+0000 7f73e7a7c740 10 client.0 _ll_register_callbacks cb 0x55df468fbf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:57:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.967+0000 7f73dd400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df469b9ba0 0x55df469c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.967+0000 7f73de800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df469b9260 0x55df469c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.967+0000 7f73dd400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df469b9ba0 0x55df469c7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:50442/0 (socket says 172.21.15.111:50442) 2024-09-15T22:57:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.967+0000 7f73dd400640 1 -- 172.21.15.111:0/2240902112 learned_addr learned my addr 172.21.15.111:0/2240902112 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:57:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.967+0000 7f73de800640 1 --2- 172.21.15.111:0/2240902112 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df469bec30 0x55df469cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.967+0000 7f73dd400640 1 -- 172.21.15.111:0/2240902112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df469b9260 msgr2=0x55df469c7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.967+0000 7f73dd400640 1 --2- 172.21.15.111:0/2240902112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df469b9260 0x55df469c7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:13.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.967+0000 7f73dd400640 1 -- 172.21.15.111:0/2240902112 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df469bec30 msgr2=0x55df469cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.967+0000 7f73dd400640 1 --2- 172.21.15.111:0/2240902112 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df469bec30 0x55df469cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.967+0000 7f73dd400640 1 -- 172.21.15.111:0/2240902112 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df466fd620 con 0x55df469b9ba0 2024-09-15T22:57:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.968+0000 7f73dd400640 1 --2- 172.21.15.111:0/2240902112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df469b9ba0 0x55df469c7d70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f73cc0314e0 tx=0x7f73cc0044f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40a1ab34695861ec server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.968+0000 7f73bf400640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:57:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.968+0000 7f73bf400640 1 -- 172.21.15.111:0/2240902112 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc035040 con 0x55df469b9ba0 2024-09-15T22:57:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.968+0000 7f73bf400640 1 -- 172.21.15.111:0/2240902112 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f73cc0054d0 con 0x55df469b9ba0 2024-09-15T22:57:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.968+0000 7f73e7a7c740 1 -- 172.21.15.111:0/2240902112 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73d8011be0 con 0x55df469b9ba0 2024-09-15T22:57:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.968+0000 7f73bf400640 1 -- 172.21.15.111:0/2240902112 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f73cc004770 con 0x55df469b9ba0 2024-09-15T22:57:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.968+0000 7f73e7a7c740 10 client.25900 fetch_fsmap learned FSMap version 30 2024-09-15T22:57:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.968+0000 7f73e7a7c740 1 -- 172.21.15.111:0/2240902112 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55df469d1b80 con 0x55df469b9ba0 2024-09-15T22:57:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.968+0000 7f73bf400640 1 -- 172.21.15.111:0/2240902112 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d8011be0 con 0x55df469b9ba0 2024-09-15T22:57:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.969+0000 7f73bf400640 1 -- 172.21.15.111:0/2240902112 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f73cc031ce0 con 0x55df469b9ba0 2024-09-15T22:57:13.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.969+0000 7f73e7a7c740 10 client.25900 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:57:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.969+0000 7f73e7a7c740 10 client.25900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:57:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.969+0000 7f73e7a7c740 20 client.25900 populate_metadata read hostname 'smithi111' 2024-09-15T22:57:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.969+0000 7f73e7a7c740 1 --2- 172.21.15.111:0/2240902112 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55df469d3270 0x55df469d5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.969+0000 7f73e7a7c740 4 client.25900 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:57:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.969+0000 7f73e7a7c740 1 -- 172.21.15.111:0/2240902112 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df4692b830 con 0x55df469d3270 2024-09-15T22:57:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.969+0000 7f73de800640 1 --2- 172.21.15.111:0/2240902112 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55df469d3270 0x55df469d5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.969+0000 7f73de800640 1 --2- 172.21.15.111:0/2240902112 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55df469d3270 0x55df469d5660 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=208152d612c61880 server_cookie=fc5bee93fdabc2e in_seq=0 out_seq=0 2024-09-15T22:57:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.969+0000 7f73bf400640 10 client.25900 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:57:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.970+0000 7f73bf400640 1 -- 172.21.15.111:0/2240902112 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55df4692b830 con 0x55df469d3270 2024-09-15T22:57:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.970+0000 7f73bf400640 10 client.25900 handle_command_reply: tid=0 2024-09-15T22:57:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.970+0000 7f73e7a7c740 1 client.25900 shutdown 2024-09-15T22:57:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.970+0000 7f73e7a7c740 1 -- 172.21.15.111:0/2240902112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df469b9ba0 msgr2=0x55df469c7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:13.976 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.970+0000 7f73e7a7c740 1 --2- 172.21.15.111:0/2240902112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df469b9ba0 0x55df469c7d70 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f73cc0314e0 tx=0x7f73cc0044f0 comp rx=0 tx=0).stop 2024-09-15T22:57:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.970+0000 7f73e7a7c740 1 -- 172.21.15.111:0/2240902112 shutdown_connections 2024-09-15T22:57:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.970+0000 7f73e7a7c740 1 --2- 172.21.15.111:0/2240902112 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55df469d3270 0x55df469d5660 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.970+0000 7f73e7a7c740 1 --2- 172.21.15.111:0/2240902112 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df469bec30 0x55df469cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.970+0000 7f73e7a7c740 1 --2- 172.21.15.111:0/2240902112 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df469b9ba0 0x55df469c7d70 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.970+0000 7f73e7a7c740 1 --2- 172.21.15.111:0/2240902112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df469b9260 0x55df469c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.970+0000 7f73e7a7c740 1 -- 172.21.15.111:0/2240902112 >> 172.21.15.111:0/2240902112 conn(0x55df469b48d0 msgr2=0x55df469b60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.970+0000 7f73bf400640 0 client.25900 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:57:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.970+0000 7f73e7a7c740 1 -- 172.21.15.111:0/2240902112 shutdown_connections 2024-09-15T22:57:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.970+0000 7f73e7a7c740 1 -- 172.21.15.111:0/2240902112 wait complete. 2024-09-15T22:57:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.971+0000 7f73e7a7c740 20 client.25900 trim_cache size 0 max 16384 2024-09-15T22:57:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.987+0000 7f73e7a7c740 1 Processor -- start 2024-09-15T22:57:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.987+0000 7f73e7a7c740 1 -- start start 2024-09-15T22:57:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.988+0000 7f73e7a7c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df469b9260 0x55df469b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.988+0000 7f73e7a7c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df469b9ba0 0x55df469ced70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.988+0000 7f73e7a7c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df469bec30 0x55df469d9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.988+0000 7f73e7a7c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f73e01a93c0 con 0x55df469b9ba0 2024-09-15T22:57:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.988+0000 7f73e7a7c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f73e01a6c90 con 0x55df469b9260 2024-09-15T22:57:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.988+0000 7f73e7a7c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f73e01a4ae0 con 0x55df469bec30 2024-09-15T22:57:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.988+0000 7f73de800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df469b9260 0x55df469b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.988+0000 7f73de800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df469b9260 0x55df469b5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:46370/0 (socket says 172.21.15.111:46370) 2024-09-15T22:57:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.988+0000 7f73de800640 1 -- 172.21.15.111:0/3890507017 learned_addr learned my addr 172.21.15.111:0/3890507017 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:57:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.988+0000 7f73dd400640 1 --2- 172.21.15.111:0/3890507017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df469b9ba0 0x55df469ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.988+0000 7f73de800640 1 --2- 172.21.15.111:0/3890507017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df469bec30 0x55df469d9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.988+0000 7f73de800640 1 -- 172.21.15.111:0/3890507017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df469bec30 msgr2=0x55df469d9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.988+0000 7f73de800640 1 --2- 172.21.15.111:0/3890507017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df469bec30 0x55df469d9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.988+0000 7f73de800640 1 -- 172.21.15.111:0/3890507017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df469b9ba0 msgr2=0x55df469ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.988+0000 7f73de800640 1 --2- 172.21.15.111:0/3890507017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df469b9ba0 0x55df469ced70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.988+0000 7f73de800640 1 -- 172.21.15.111:0/3890507017 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df469ba080 con 0x55df469b9260 2024-09-15T22:57:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.989+0000 7f73de800640 1 --2- 172.21.15.111:0/3890507017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df469b9260 0x55df469b5ef0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009f70 tx=0x7f73d8009940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7ea1651d319e814 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:13.991 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.989+0000 7f73ca600640 1 -- 172.21.15.111:0/3890507017 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8015040 con 0x55df469b9260 2024-09-15T22:57:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.989+0000 7f73ca600640 1 -- 172.21.15.111:0/3890507017 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d8030070 con 0x55df469b9260 2024-09-15T22:57:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.989+0000 7f73ca600640 1 -- 172.21.15.111:0/3890507017 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d8010a80 con 0x55df469b9260 2024-09-15T22:57:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.990+0000 7f73e7a7c740 1 -- 172.21.15.111:0/3890507017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df469b9260 msgr2=0x55df469b5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.990+0000 7f73e7a7c740 1 --2- 172.21.15.111:0/3890507017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df469b9260 0x55df469b5ef0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009f70 tx=0x7f73d8009940 comp rx=0 tx=0).stop 2024-09-15T22:57:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.990+0000 7f73e7a7c740 1 -- 172.21.15.111:0/3890507017 shutdown_connections 2024-09-15T22:57:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.990+0000 7f73e7a7c740 1 --2- 172.21.15.111:0/3890507017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df469bec30 0x55df469d9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.990+0000 7f73e7a7c740 1 --2- 172.21.15.111:0/3890507017 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df469b9ba0 0x55df469ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.990+0000 7f73e7a7c740 1 --2- 172.21.15.111:0/3890507017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df469b9260 0x55df469b5ef0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.990+0000 7f73e7a7c740 1 -- 172.21.15.111:0/3890507017 >> 172.21.15.111:0/3890507017 conn(0x55df469b48d0 msgr2=0x55df469b57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.990+0000 7f73e7a7c740 1 -- 172.21.15.111:0/3890507017 shutdown_connections 2024-09-15T22:57:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.990+0000 7f73e7a7c740 1 -- 172.21.15.111:0/3890507017 wait complete. 2024-09-15T22:57:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.990+0000 7f73e7a7c740 1 -- start start 2024-09-15T22:57:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.990+0000 7f73e7a7c740 1 Processor -- start 2024-09-15T22:57:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.991+0000 7f73e7a7c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df469b9260 0x55df469ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.991+0000 7f73e7a7c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df469b9ba0 0x55df469ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.991+0000 7f73e7a7c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df469bec30 0x55df469ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.991+0000 7f73e7a7c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f73e01a5150 con 0x55df469b9ba0 2024-09-15T22:57:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.991+0000 7f73e7a7c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f73e01a65c0 con 0x55df469bec30 2024-09-15T22:57:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.991+0000 7f73e7a7c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f73e01a5820 con 0x55df469b9260 2024-09-15T22:57:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.991+0000 7f73e7a7c740 10 client.0 _ll_register_callbacks cb 0x7f73cc004180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:57:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.991+0000 7f73dd400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df469b9ba0 0x55df469ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.991+0000 7f73de800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df469bec30 0x55df469ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.991+0000 7f73de800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df469bec30 0x55df469ccb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:46386/0 (socket says 172.21.15.111:46386) 2024-09-15T22:57:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.991+0000 7f73de800640 1 -- 172.21.15.111:0/1654988487 learned_addr learned my addr 172.21.15.111:0/1654988487 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:57:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.991+0000 7f73de800640 1 --2- 172.21.15.111:0/1654988487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df469b9260 0x55df469ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.991+0000 7f73dd400640 1 -- 172.21.15.111:0/1654988487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df469b9260 msgr2=0x55df469ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.991+0000 7f73dd400640 1 --2- 172.21.15.111:0/1654988487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df469b9260 0x55df469ce570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.991+0000 7f73dd400640 1 -- 172.21.15.111:0/1654988487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df469bec30 msgr2=0x55df469ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.991+0000 7f73dd400640 1 --2- 172.21.15.111:0/1654988487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df469bec30 0x55df469ccb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.991+0000 7f73dd400640 1 -- 172.21.15.111:0/1654988487 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df469be780 con 0x55df469b9ba0 2024-09-15T22:57:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.992+0000 7f73dd400640 1 --2- 172.21.15.111:0/1654988487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df469b9ba0 0x55df469ceab0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009850 tx=0x7f73cc031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce822f1c4fc5fab server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.992+0000 7f73bf400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:57:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.992+0000 7f73bf400640 1 -- 172.21.15.111:0/1654988487 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc035070 con 0x55df469b9ba0 2024-09-15T22:57:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.992+0000 7f73bf400640 1 -- 172.21.15.111:0/1654988487 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f73cc0052d0 con 0x55df469b9ba0 2024-09-15T22:57:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.992+0000 7f73e7a7c740 1 -- 172.21.15.111:0/1654988487 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73d8030070 con 0x55df469b9ba0 2024-09-15T22:57:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.992+0000 7f73bf400640 1 -- 172.21.15.111:0/1654988487 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f73cc0054d0 con 0x55df469b9ba0 2024-09-15T22:57:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.992+0000 7f73e7a7c740 10 client.15903 fetch_fsmap learned FSMap version 30 2024-09-15T22:57:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.992+0000 7f73e7a7c740 1 -- 172.21.15.111:0/1654988487 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55df46a42d30 con 0x55df469b9ba0 2024-09-15T22:57:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.993+0000 7f73bf400640 1 -- 172.21.15.111:0/1654988487 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d8030070 con 0x55df469b9ba0 2024-09-15T22:57:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.993+0000 7f73bf400640 1 -- 172.21.15.111:0/1654988487 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f73cc031aa0 con 0x55df469b9ba0 2024-09-15T22:57:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.993+0000 7f73e7a7c740 10 client.15903 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:57:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.993+0000 7f73e7a7c740 10 client.15903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:57:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.993+0000 7f73e7a7c740 20 client.15903 populate_metadata read hostname 'smithi111' 2024-09-15T22:57:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.993+0000 7f73e7a7c740 1 --2- 172.21.15.111:0/1654988487 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55df46a43710 0x55df46a45b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.993+0000 7f73e7a7c740 4 client.15903 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:57:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.993+0000 7f73e7a7c740 1 -- 172.21.15.111:0/1654988487 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55df46925b80 con 0x55df46a43710 2024-09-15T22:57:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.993+0000 7f73de800640 1 --2- 172.21.15.111:0/1654988487 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55df46a43710 0x55df46a45b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.993+0000 7f73de800640 1 --2- 172.21.15.111:0/1654988487 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55df46a43710 0x55df46a45b30 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bab78ea3e715152 server_cookie=4fb9271d4a8abf67 in_seq=0 out_seq=0 2024-09-15T22:57:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:13.993+0000 7f73bf400640 10 client.15903 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:57:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:14.005+0000 7f73bf400640 1 -- 172.21.15.111:0/1654988487 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55df46925b80 con 0x55df46a43710 2024-09-15T22:57:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:14.005+0000 7f73bf400640 10 client.15903 handle_command_reply: tid=0 2024-09-15T22:57:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:14.005+0000 7f73e7a7c740 1 client.15903 shutdown 2024-09-15T22:57:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:14.005+0000 7f73e7a7c740 1 -- 172.21.15.111:0/1654988487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df469b9ba0 msgr2=0x55df469ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:14.005+0000 7f73e7a7c740 1 --2- 172.21.15.111:0/1654988487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df469b9ba0 0x55df469ceab0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009850 tx=0x7f73cc031400 comp rx=0 tx=0).stop 2024-09-15T22:57:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:14.005+0000 7f73e7a7c740 1 -- 172.21.15.111:0/1654988487 shutdown_connections 2024-09-15T22:57:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:14.005+0000 7f73e7a7c740 1 --2- 172.21.15.111:0/1654988487 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55df46a43710 0x55df46a45b30 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:14.005+0000 7f73e7a7c740 1 --2- 172.21.15.111:0/1654988487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55df469bec30 0x55df469ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:14.005+0000 7f73e7a7c740 1 --2- 172.21.15.111:0/1654988487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55df469b9ba0 0x55df469ceab0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:14.005+0000 7f73e7a7c740 1 --2- 172.21.15.111:0/1654988487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55df469b9260 0x55df469ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:14.005+0000 7f73bf400640 0 client.15903 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:57:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:14.005+0000 7f73e7a7c740 1 -- 172.21.15.111:0/1654988487 >> 172.21.15.111:0/1654988487 conn(0x55df469b48d0 msgr2=0x55df469b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:14.006+0000 7f73e7a7c740 1 -- 172.21.15.111:0/1654988487 shutdown_connections 2024-09-15T22:57:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:14.006+0000 7f73e7a7c740 1 -- 172.21.15.111:0/1654988487 wait complete. 2024-09-15T22:57:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:14.006+0000 7f73e7a7c740 20 client.15903 trim_cache size 0 max 16384 2024-09-15T22:57:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:57:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:57:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:57:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:57:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:14.006+0000 7f73df200640 1 -- 172.21.15.111:0/1595342624 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f73c40a32c0 msgr2=0x7f73c40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:14.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:14.006+0000 7f73df200640 1 --2- 172.21.15.111:0/1595342624 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f73c40a32c0 0x7f73c40a5780 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f73d0098c90 tx=0x7f73d0003f10 comp rx=0 tx=0).stop 2024-09-15T22:57:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:14.006+0000 7f73df200640 1 -- 172.21.15.111:0/1595342624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73e0101950 msgr2=0x7f73e0197cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:14.006+0000 7f73df200640 1 --2- 172.21.15.111:0/1595342624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73e0101950 0x7f73e0197cb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f73d000e9c0 tx=0x7f73d000ee80 comp rx=0 tx=0).stop 2024-09-15T22:57:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:14.006+0000 7f73df200640 1 -- 172.21.15.111:0/1595342624 shutdown_connections 2024-09-15T22:57:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:14.006+0000 7f73df200640 1 --2- 172.21.15.111:0/1595342624 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73e0103500 0x7f73e019f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:14.006+0000 7f73df200640 1 --2- 172.21.15.111:0/1595342624 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f73c40a32c0 0x7f73c40a5780 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:14.006+0000 7f73df200640 1 --2- 172.21.15.111:0/1595342624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73e0102b40 0x7f73e01981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:14.006+0000 7f73df200640 1 --2- 172.21.15.111:0/1595342624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73e0101950 0x7f73e0197cb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:14.006+0000 7f73df200640 1 -- 172.21.15.111:0/1595342624 >> 172.21.15.111:0/1595342624 conn(0x7f73e00fd0d0 msgr2=0x7f73e0105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:14.007+0000 7f73df200640 1 -- 172.21.15.111:0/1595342624 shutdown_connections 2024-09-15T22:57:14.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:14.007+0000 7f73df200640 1 -- 172.21.15.111:0/1595342624 wait complete. 2024-09-15T22:57:14.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:13 smithi121 ceph-mon[34815]: pgmap v689: 97 pgs: 97 active+clean; 273 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 177 op/s 2024-09-15T22:57:14.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:13 smithi111 ceph-mon[30629]: pgmap v689: 97 pgs: 97 active+clean; 273 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 177 op/s 2024-09-15T22:57:14.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:13 smithi192 ceph-mon[35427]: pgmap v689: 97 pgs: 97 active+clean; 273 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 177 op/s 2024-09-15T22:57:14.319 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1080491 9.90 MB/sec execute 462 sec latency 12.204 ms 2024-09-15T22:57:15.319 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1083660 9.92 MB/sec execute 463 sec latency 13.338 ms 2024-09-15T22:57:16.136 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:15 smithi111 ceph-mon[30629]: pgmap v690: 97 pgs: 97 active+clean; 276 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 219 op/s 2024-09-15T22:57:16.139 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:15 smithi121 ceph-mon[34815]: pgmap v690: 97 pgs: 97 active+clean; 276 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 219 op/s 2024-09-15T22:57:16.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:15 smithi192 ceph-mon[35427]: pgmap v690: 97 pgs: 97 active+clean; 276 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 219 op/s 2024-09-15T22:57:16.319 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1085292 9.91 MB/sec execute 464 sec latency 257.849 ms 2024-09-15T22:57:17.319 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1085292 9.89 MB/sec execute 465 sec latency 1257.891 ms 2024-09-15T22:57:18.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:17 smithi121 ceph-mon[34815]: pgmap v691: 97 pgs: 97 active+clean; 276 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 185 op/s 2024-09-15T22:57:18.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:17 smithi111 ceph-mon[30629]: pgmap v691: 97 pgs: 97 active+clean; 276 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 185 op/s 2024-09-15T22:57:18.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:17 smithi192 ceph-mon[35427]: pgmap v691: 97 pgs: 97 active+clean; 276 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 185 op/s 2024-09-15T22:57:18.319 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1087540 9.90 MB/sec execute 466 sec latency 1399.468 ms 2024-09-15T22:57:19.320 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1090718 9.92 MB/sec execute 467 sec latency 13.177 ms 2024-09-15T22:57:20.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:19 smithi121 ceph-mon[34815]: pgmap v692: 97 pgs: 97 active+clean; 279 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 231 op/s 2024-09-15T22:57:20.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:19 smithi111 ceph-mon[30629]: pgmap v692: 97 pgs: 97 active+clean; 279 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 231 op/s 2024-09-15T22:57:20.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:19 smithi192 ceph-mon[35427]: pgmap v692: 97 pgs: 97 active+clean; 279 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 231 op/s 2024-09-15T22:57:20.320 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1093408 9.92 MB/sec execute 468 sec latency 14.824 ms 2024-09-15T22:57:21.320 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1094937 9.92 MB/sec execute 469 sec latency 203.988 ms 2024-09-15T22:57:21.995 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:21 smithi111 ceph-mon[30629]: pgmap v693: 97 pgs: 97 active+clean; 279 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 189 op/s 2024-09-15T22:57:22.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:21 smithi121 ceph-mon[34815]: pgmap v693: 97 pgs: 97 active+clean; 279 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 189 op/s 2024-09-15T22:57:22.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:21 smithi192 ceph-mon[35427]: pgmap v693: 97 pgs: 97 active+clean; 279 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 189 op/s 2024-09-15T22:57:22.320 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1094956 9.90 MB/sec execute 470 sec latency 926.439 ms 2024-09-15T22:57:23.320 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1096489 9.89 MB/sec execute 471 sec latency 867.700 ms 2024-09-15T22:57:24.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:23 smithi121 ceph-mon[34815]: pgmap v694: 97 pgs: 97 active+clean; 281 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 206 op/s 2024-09-15T22:57:24.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:23 smithi111 ceph-mon[30629]: pgmap v694: 97 pgs: 97 active+clean; 281 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 206 op/s 2024-09-15T22:57:24.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:23 smithi192 ceph-mon[35427]: pgmap v694: 97 pgs: 97 active+clean; 281 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 206 op/s 2024-09-15T22:57:24.320 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1098590 9.90 MB/sec execute 472 sec latency 21.741 ms 2024-09-15T22:57:25.320 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1101539 9.92 MB/sec execute 473 sec latency 7.894 ms 2024-09-15T22:57:26.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:25 smithi121 ceph-mon[34815]: pgmap v695: 97 pgs: 97 active+clean; 281 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 210 op/s 2024-09-15T22:57:26.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:25 smithi111 ceph-mon[30629]: pgmap v695: 97 pgs: 97 active+clean; 281 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 210 op/s 2024-09-15T22:57:26.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:25 smithi192 ceph-mon[35427]: pgmap v695: 97 pgs: 97 active+clean; 281 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 210 op/s 2024-09-15T22:57:26.320 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1104114 9.92 MB/sec execute 474 sec latency 66.589 ms 2024-09-15T22:57:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.130+0000 7f9549000640 1 -- 172.21.15.111:0/3488267400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9544104f50 msgr2=0x7f95441053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.130+0000 7f9549000640 1 --2- 172.21.15.111:0/3488267400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9544104f50 0x7f95441053d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9530009900 tx=0x7f953002a3f0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.130+0000 7f9549000640 1 -- 172.21.15.111:0/3488267400 shutdown_connections 2024-09-15T22:57:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.130+0000 7f9549000640 1 --2- 172.21.15.111:0/3488267400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9544105910 0x7f954410c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.130+0000 7f9549000640 1 --2- 172.21.15.111:0/3488267400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9544104f50 0x7f95441053d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.130+0000 7f9549000640 1 --2- 172.21.15.111:0/3488267400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9544103d60 0x7f9544104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.130+0000 7f9549000640 1 -- 172.21.15.111:0/3488267400 >> 172.21.15.111:0/3488267400 conn(0x7f95440ff4e0 msgr2=0x7f9544101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.131+0000 7f9549000640 1 -- 172.21.15.111:0/3488267400 shutdown_connections 2024-09-15T22:57:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.131+0000 7f9549000640 1 -- 172.21.15.111:0/3488267400 wait complete. 2024-09-15T22:57:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.132+0000 7f9549000640 1 Processor -- start 2024-09-15T22:57:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.132+0000 7f9549000640 1 -- start start 2024-09-15T22:57:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.132+0000 7f9549000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9544103d60 0x7f9544199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.132+0000 7f9549000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9544104f50 0x7f954419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.132+0000 7f9549000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9544105910 0x7f95441a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.132+0000 7f9549000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f954410f9c0 con 0x7f9544103d60 2024-09-15T22:57:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.132+0000 7f9549000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f954410f840 con 0x7f9544105910 2024-09-15T22:57:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.132+0000 7f9549000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f954410fb40 con 0x7f9544104f50 2024-09-15T22:57:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.132+0000 7f9541400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9544104f50 0x7f954419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.132+0000 7f9541400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9544104f50 0x7f954419a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:34482/0 (socket says 172.21.15.111:34482) 2024-09-15T22:57:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.132+0000 7f9541400640 1 -- 172.21.15.111:0/2937268668 learned_addr learned my addr 172.21.15.111:0/2937268668 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:57:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.133+0000 7f9542800640 1 --2- 172.21.15.111:0/2937268668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9544105910 0x7f95441a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.133+0000 7f9541e00640 1 --2- 172.21.15.111:0/2937268668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9544103d60 0x7f9544199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.134+0000 7f9541400640 1 -- 172.21.15.111:0/2937268668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9544105910 msgr2=0x7f95441a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.134+0000 7f9541400640 1 --2- 172.21.15.111:0/2937268668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9544105910 0x7f95441a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.134+0000 7f9541400640 1 -- 172.21.15.111:0/2937268668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9544103d60 msgr2=0x7f9544199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.134+0000 7f9541400640 1 --2- 172.21.15.111:0/2937268668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9544103d60 0x7f9544199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.134+0000 7f9541400640 1 -- 172.21.15.111:0/2937268668 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95441a1bd0 con 0x7f9544104f50 2024-09-15T22:57:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.134+0000 7f9541400640 1 --2- 172.21.15.111:0/2937268668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9544104f50 0x7f954419a4f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9530006010 tx=0x7f9530004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.135+0000 7f953aa00640 1 -- 172.21.15.111:0/2937268668 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953003f030 con 0x7f9544104f50 2024-09-15T22:57:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.135+0000 7f953aa00640 1 -- 172.21.15.111:0/2937268668 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9530003ce0 con 0x7f9544104f50 2024-09-15T22:57:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.135+0000 7f9549000640 1 -- 172.21.15.111:0/2937268668 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95441a1e60 con 0x7f9544104f50 2024-09-15T22:57:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.135+0000 7f953aa00640 1 -- 172.21.15.111:0/2937268668 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530035070 con 0x7f9544104f50 2024-09-15T22:57:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.135+0000 7f9549000640 1 -- 172.21.15.111:0/2937268668 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95441a22c0 con 0x7f9544104f50 2024-09-15T22:57:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.138+0000 7f954beac740 1 Processor -- start 2024-09-15T22:57:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.138+0000 7f954beac740 1 -- start start 2024-09-15T22:57:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.139+0000 7f954beac740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555e8163d8a0 0x555e8163dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.139+0000 7f954beac740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e8163e1e0 0x555e81642ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.139+0000 7f954beac740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e81643270 0x555e81645730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.139+0000 7f954beac740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555e8159a100 con 0x555e8163e1e0 2024-09-15T22:57:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.139+0000 7f954beac740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x555e815421b0 con 0x555e81643270 2024-09-15T22:57:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.139+0000 7f954beac740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555e815960c0 con 0x555e8163d8a0 2024-09-15T22:57:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.139+0000 7f953aa00640 1 -- 172.21.15.111:0/2937268668 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f9530042030 con 0x7f9544104f50 2024-09-15T22:57:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.139+0000 7f9542800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555e8163d8a0 0x555e8163dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.139+0000 7f9542800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555e8163d8a0 0x555e8163dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:34492/0 (socket says 172.21.15.111:34492) 2024-09-15T22:57:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.139+0000 7f9542800640 1 -- 172.21.15.111:0/56663780 learned_addr learned my addr 172.21.15.111:0/56663780 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:57:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.140+0000 7f9541e00640 1 --2- 172.21.15.111:0/56663780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e8163e1e0 0x555e81642ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.140+0000 7f9541400640 1 --2- 172.21.15.111:0/56663780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e81643270 0x555e81645730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.140+0000 7f9542800640 1 -- 172.21.15.111:0/56663780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e81643270 msgr2=0x555e81645730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.140+0000 7f9542800640 1 --2- 172.21.15.111:0/56663780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e81643270 0x555e81645730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.140+0000 7f9542800640 1 -- 172.21.15.111:0/56663780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e8163e1e0 msgr2=0x555e81642ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.140+0000 7f9542800640 1 --2- 172.21.15.111:0/56663780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e8163e1e0 0x555e81642ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.140+0000 7f9542800640 1 -- 172.21.15.111:0/56663780 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e812ad210 con 0x555e8163d8a0 2024-09-15T22:57:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.140+0000 7f9541e00640 1 --2- 172.21.15.111:0/56663780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e8163e1e0 0x555e81642ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:57:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.140+0000 7f9542800640 1 --2- 172.21.15.111:0/56663780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555e8163d8a0 0x555e8163dca0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f953c0039f0 tx=0x7f953c003eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbba7b6d31148a80 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.141+0000 7f952f400640 1 -- 172.21.15.111:0/56663780 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c01a470 con 0x555e8163d8a0 2024-09-15T22:57:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.141+0000 7f952f400640 1 -- 172.21.15.111:0/56663780 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f953c015070 con 0x555e8163d8a0 2024-09-15T22:57:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.141+0000 7f952f400640 1 -- 172.21.15.111:0/56663780 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c010900 con 0x555e8163d8a0 2024-09-15T22:57:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.141+0000 7f953aa00640 1 --2- 172.21.15.111:0/2937268668 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f95280a32f0 0x7f95280a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.141+0000 7f953aa00640 1 -- 172.21.15.111:0/2937268668 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f95300f0120 con 0x7f9544104f50 2024-09-15T22:57:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.141+0000 7f9541e00640 1 --2- 172.21.15.111:0/2937268668 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f95280a32f0 0x7f95280a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.142+0000 7f954beac740 1 -- 172.21.15.111:0/56663780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555e8163d8a0 msgr2=0x555e8163dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.142+0000 7f954beac740 1 --2- 172.21.15.111:0/56663780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555e8163d8a0 0x555e8163dca0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f953c0039f0 tx=0x7f953c003eb0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.142+0000 7f954beac740 1 -- 172.21.15.111:0/56663780 shutdown_connections 2024-09-15T22:57:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.142+0000 7f954beac740 1 --2- 172.21.15.111:0/56663780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e81643270 0x555e81645730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.142+0000 7f954beac740 1 --2- 172.21.15.111:0/56663780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e8163e1e0 0x555e81642ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.142+0000 7f954beac740 1 --2- 172.21.15.111:0/56663780 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555e8163d8a0 0x555e8163dca0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.142+0000 7f954beac740 1 -- 172.21.15.111:0/56663780 >> 172.21.15.111:0/56663780 conn(0x555e81630700 msgr2=0x555e81630b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.142+0000 7f954beac740 1 -- 172.21.15.111:0/56663780 shutdown_connections 2024-09-15T22:57:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.142+0000 7f9541e00640 1 --2- 172.21.15.111:0/2937268668 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f95280a32f0 0x7f95280a57b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x555e8163ec10 tx=0x7f9534005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.143+0000 7f954beac740 1 -- 172.21.15.111:0/56663780 wait complete. 2024-09-15T22:57:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.143+0000 7f954beac740 1 -- start start 2024-09-15T22:57:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.143+0000 7f954beac740 1 Processor -- start 2024-09-15T22:57:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.144+0000 7f954beac740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555e8163d8a0 0x555e81647a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.144+0000 7f954beac740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e8163e1e0 0x555e81647fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.144+0000 7f954beac740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e81643270 0x555e8164efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.144+0000 7f954beac740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555e81646240 con 0x555e81643270 2024-09-15T22:57:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.144+0000 7f954beac740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x555e81523090 con 0x555e8163e1e0 2024-09-15T22:57:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.144+0000 7f954beac740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555e816463c0 con 0x555e8163d8a0 2024-09-15T22:57:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.144+0000 7f954beac740 10 client.0 _ll_register_callbacks cb 0x555e81577f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:57:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.144+0000 7f9541400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e8163e1e0 0x555e81647fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.144+0000 7f9541400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e8163e1e0 0x555e81647fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:52644/0 (socket says 172.21.15.111:52644) 2024-09-15T22:57:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.145+0000 7f9541400640 1 -- 172.21.15.111:0/2502277955 learned_addr learned my addr 172.21.15.111:0/2502277955 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:57:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.145+0000 7f9542800640 1 --2- 172.21.15.111:0/2502277955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e81643270 0x555e8164efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.145+0000 7f9541e00640 1 --2- 172.21.15.111:0/2502277955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555e8163d8a0 0x555e81647a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.145+0000 7f9541400640 1 -- 172.21.15.111:0/2502277955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555e8163d8a0 msgr2=0x555e81647a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.145+0000 7f9541400640 1 --2- 172.21.15.111:0/2502277955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555e8163d8a0 0x555e81647a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.145+0000 7f9541400640 1 -- 172.21.15.111:0/2502277955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e81643270 msgr2=0x555e8164efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.145+0000 7f9541400640 1 --2- 172.21.15.111:0/2502277955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e81643270 0x555e8164efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.145+0000 7f9541400640 1 -- 172.21.15.111:0/2502277955 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e81379620 con 0x555e8163e1e0 2024-09-15T22:57:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.145+0000 7f9542800640 1 --2- 172.21.15.111:0/2502277955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e81643270 0x555e8164efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:57:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.145+0000 7f9541e00640 1 --2- 172.21.15.111:0/2502277955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555e8163d8a0 0x555e81647a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:57:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.145+0000 7f9541400640 1 --2- 172.21.15.111:0/2502277955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e8163e1e0 0x555e81647fa0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f95300bc800 tx=0x7f95300b8510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80fbed983369c5c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.145+0000 7f9523e00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:57:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.146+0000 7f9523e00640 1 -- 172.21.15.111:0/2502277955 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530049070 con 0x555e8163e1e0 2024-09-15T22:57:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.146+0000 7f954beac740 1 -- 172.21.15.111:0/2502277955 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f953c015070 con 0x555e8163e1e0 2024-09-15T22:57:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.146+0000 7f9523e00640 1 -- 172.21.15.111:0/2502277955 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f95300b8ee0 con 0x555e8163e1e0 2024-09-15T22:57:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.146+0000 7f9523e00640 1 -- 172.21.15.111:0/2502277955 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f95300bc640 con 0x555e8163e1e0 2024-09-15T22:57:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.146+0000 7f954beac740 10 client.25886 fetch_fsmap learned FSMap version 30 2024-09-15T22:57:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.146+0000 7f954beac740 1 -- 172.21.15.111:0/2502277955 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555e81651dd0 con 0x555e8163e1e0 2024-09-15T22:57:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.147+0000 7f9523e00640 1 -- 172.21.15.111:0/2502277955 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f953c015070 con 0x555e8163e1e0 2024-09-15T22:57:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.148+0000 7f9523e00640 1 -- 172.21.15.111:0/2502277955 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f95300d0ce0 con 0x555e8163e1e0 2024-09-15T22:57:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.149+0000 7f954beac740 10 client.25886 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:57:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.149+0000 7f954beac740 10 client.25886 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:57:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.150+0000 7f954beac740 20 client.25886 populate_metadata read hostname 'smithi111' 2024-09-15T22:57:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.150+0000 7f954beac740 1 --2- 172.21.15.111:0/2502277955 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555e816535a0 0x555e816559d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.150+0000 7f954beac740 4 client.25886 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:57:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.150+0000 7f954beac740 1 -- 172.21.15.111:0/2502277955 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e815a7830 con 0x555e816535a0 2024-09-15T22:57:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.150+0000 7f9541e00640 1 --2- 172.21.15.111:0/2502277955 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555e816535a0 0x555e816559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.152+0000 7f9541e00640 1 --2- 172.21.15.111:0/2502277955 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555e816535a0 0x555e816559d0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e80be6902a7015c8 server_cookie=dc7e366dd2597875 in_seq=0 out_seq=0 2024-09-15T22:57:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.152+0000 7f9523e00640 10 client.25886 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:57:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.153+0000 7f9523e00640 1 -- 172.21.15.111:0/2502277955 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555e815a7830 con 0x555e816535a0 2024-09-15T22:57:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.153+0000 7f9523e00640 10 client.25886 handle_command_reply: tid=0 2024-09-15T22:57:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.154+0000 7f954beac740 1 client.25886 shutdown 2024-09-15T22:57:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.154+0000 7f954beac740 1 -- 172.21.15.111:0/2502277955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e8163e1e0 msgr2=0x555e81647fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.154+0000 7f954beac740 1 --2- 172.21.15.111:0/2502277955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e8163e1e0 0x555e81647fa0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f95300bc800 tx=0x7f95300b8510 comp rx=0 tx=0).stop 2024-09-15T22:57:27.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.154+0000 7f954beac740 1 -- 172.21.15.111:0/2502277955 shutdown_connections 2024-09-15T22:57:27.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.154+0000 7f954beac740 1 --2- 172.21.15.111:0/2502277955 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555e816535a0 0x555e816559d0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.155+0000 7f954beac740 1 --2- 172.21.15.111:0/2502277955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e81643270 0x555e8164efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.155+0000 7f954beac740 1 --2- 172.21.15.111:0/2502277955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e8163e1e0 0x555e81647fa0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.155+0000 7f954beac740 1 --2- 172.21.15.111:0/2502277955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555e8163d8a0 0x555e81647a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.155+0000 7f954beac740 1 -- 172.21.15.111:0/2502277955 >> 172.21.15.111:0/2502277955 conn(0x555e81630700 msgr2=0x555e81631f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:27.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.155+0000 7f9523e00640 0 client.25886 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:57:27.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.155+0000 7f954beac740 1 -- 172.21.15.111:0/2502277955 shutdown_connections 2024-09-15T22:57:27.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.155+0000 7f954beac740 1 -- 172.21.15.111:0/2502277955 wait complete. 2024-09-15T22:57:27.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.155+0000 7f954beac740 20 client.25886 trim_cache size 0 max 16384 2024-09-15T22:57:27.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.181+0000 7f954beac740 1 Processor -- start 2024-09-15T22:57:27.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.181+0000 7f954beac740 1 -- start start 2024-09-15T22:57:27.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.182+0000 7f954beac740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555e8163d8a0 0x555e8162b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:27.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.182+0000 7f954beac740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e8163e1e0 0x555e8164efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:27.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.182+0000 7f954beac740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e81643270 0x555e816315b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:27.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.182+0000 7f954beac740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f95441a9480 con 0x555e81643270 2024-09-15T22:57:27.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.182+0000 7f954beac740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f95441a6d50 con 0x555e8163e1e0 2024-09-15T22:57:27.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.182+0000 7f954beac740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f95441a4ba0 con 0x555e8163d8a0 2024-09-15T22:57:27.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.182+0000 7f9541e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555e8163d8a0 0x555e8162b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:27.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.182+0000 7f9541e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555e8163d8a0 0x555e8162b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:34500/0 (socket says 172.21.15.111:34500) 2024-09-15T22:57:27.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.182+0000 7f9541400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e8163e1e0 0x555e8164efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:27.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.182+0000 7f9541e00640 1 -- 172.21.15.111:0/209691717 learned_addr learned my addr 172.21.15.111:0/209691717 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:57:27.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.182+0000 7f9542800640 1 --2- 172.21.15.111:0/209691717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e81643270 0x555e816315b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:27.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.182+0000 7f9541e00640 1 -- 172.21.15.111:0/209691717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e8163e1e0 msgr2=0x555e8164efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:27.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.182+0000 7f9541e00640 1 --2- 172.21.15.111:0/209691717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e8163e1e0 0x555e8164efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.182+0000 7f9541e00640 1 -- 172.21.15.111:0/209691717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e81643270 msgr2=0x555e816315b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:27.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.182+0000 7f9541e00640 1 --2- 172.21.15.111:0/209691717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e81643270 0x555e816315b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.182+0000 7f9541e00640 1 -- 172.21.15.111:0/209691717 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e8163e6c0 con 0x555e8163d8a0 2024-09-15T22:57:27.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.182+0000 7f9541400640 1 --2- 172.21.15.111:0/209691717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e8163e1e0 0x555e8164efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:57:27.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.183+0000 7f9541e00640 1 --2- 172.21.15.111:0/209691717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555e8163d8a0 0x555e8162b8d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x555e81631580 tx=0x7f9534015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dff77b63e1b14d21 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.183+0000 7f952ea00640 1 -- 172.21.15.111:0/209691717 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534003c00 con 0x555e8163d8a0 2024-09-15T22:57:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.183+0000 7f952ea00640 1 -- 172.21.15.111:0/209691717 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9534003da0 con 0x555e8163d8a0 2024-09-15T22:57:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.183+0000 7f952ea00640 1 -- 172.21.15.111:0/209691717 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953401c950 con 0x555e8163d8a0 2024-09-15T22:57:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.184+0000 7f954beac740 1 -- 172.21.15.111:0/209691717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555e8163d8a0 msgr2=0x555e8162b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.184+0000 7f954beac740 1 --2- 172.21.15.111:0/209691717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555e8163d8a0 0x555e8162b8d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x555e81631580 tx=0x7f9534015000 comp rx=0 tx=0).stop 2024-09-15T22:57:27.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.184+0000 7f954beac740 1 -- 172.21.15.111:0/209691717 shutdown_connections 2024-09-15T22:57:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.184+0000 7f954beac740 1 --2- 172.21.15.111:0/209691717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e81643270 0x555e816315b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.184+0000 7f954beac740 1 --2- 172.21.15.111:0/209691717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e8163e1e0 0x555e8164efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.184+0000 7f954beac740 1 --2- 172.21.15.111:0/209691717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555e8163d8a0 0x555e8162b8d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.184+0000 7f954beac740 1 -- 172.21.15.111:0/209691717 >> 172.21.15.111:0/209691717 conn(0x555e81630700 msgr2=0x555e81647a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.184+0000 7f954beac740 1 -- 172.21.15.111:0/209691717 shutdown_connections 2024-09-15T22:57:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.184+0000 7f954beac740 1 -- 172.21.15.111:0/209691717 wait complete. 2024-09-15T22:57:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.184+0000 7f954beac740 1 -- start start 2024-09-15T22:57:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.184+0000 7f954beac740 1 Processor -- start 2024-09-15T22:57:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.185+0000 7f954beac740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555e8163d8a0 0x555e8164c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.185+0000 7f954beac740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e8163e1e0 0x555e8164cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:27.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.185+0000 7f954beac740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e81643270 0x555e8164dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.185+0000 7f954beac740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f95441a5210 con 0x555e8163e1e0 2024-09-15T22:57:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.185+0000 7f954beac740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f95441a6680 con 0x555e81643270 2024-09-15T22:57:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.185+0000 7f954beac740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f95441a58e0 con 0x555e8163d8a0 2024-09-15T22:57:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.185+0000 7f954beac740 10 client.0 _ll_register_callbacks cb 0x7f9530004550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:57:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.185+0000 7f9541e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555e8163d8a0 0x555e8164c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.185+0000 7f9542800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e81643270 0x555e8164dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.185+0000 7f9541e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555e8163d8a0 0x555e8164c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:34502/0 (socket says 172.21.15.111:34502) 2024-09-15T22:57:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.185+0000 7f9541e00640 1 -- 172.21.15.111:0/1256206704 learned_addr learned my addr 172.21.15.111:0/1256206704 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:57:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.185+0000 7f9541400640 1 --2- 172.21.15.111:0/1256206704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e8163e1e0 0x555e8164cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.185+0000 7f9541e00640 1 -- 172.21.15.111:0/1256206704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e81643270 msgr2=0x555e8164dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.186+0000 7f9541e00640 1 --2- 172.21.15.111:0/1256206704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e81643270 0x555e8164dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.186+0000 7f9541e00640 1 -- 172.21.15.111:0/1256206704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e8163e1e0 msgr2=0x555e8164cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.186+0000 7f9541e00640 1 --2- 172.21.15.111:0/1256206704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e8163e1e0 0x555e8164cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.186+0000 7f9541e00640 1 -- 172.21.15.111:0/1256206704 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e8164e220 con 0x555e8163d8a0 2024-09-15T22:57:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.186+0000 7f9541400640 1 --2- 172.21.15.111:0/1256206704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e8163e1e0 0x555e8164cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:57:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.186+0000 7f9542800640 1 --2- 172.21.15.111:0/1256206704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e81643270 0x555e8164dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:57:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.186+0000 7f9541e00640 1 --2- 172.21.15.111:0/1256206704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555e8163d8a0 0x555e8164c7d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9534011010 tx=0x7f953400a820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89eff56c29f58e40 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.186+0000 7f9523e00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:57:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.186+0000 7f9523e00640 1 -- 172.21.15.111:0/1256206704 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9534003c00 con 0x555e8163d8a0 2024-09-15T22:57:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.186+0000 7f954beac740 1 -- 172.21.15.111:0/1256206704 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9534003da0 con 0x555e8163d8a0 2024-09-15T22:57:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.186+0000 7f9523e00640 1 -- 172.21.15.111:0/1256206704 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f9534028680 con 0x555e8163d8a0 2024-09-15T22:57:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.187+0000 7f9523e00640 1 -- 172.21.15.111:0/1256206704 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f95340288a0 con 0x555e8163d8a0 2024-09-15T22:57:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.187+0000 7f954beac740 10 client.25921 fetch_fsmap learned FSMap version 30 2024-09-15T22:57:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.187+0000 7f954beac740 1 -- 172.21.15.111:0/1256206704 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555e816be280 con 0x555e8163d8a0 2024-09-15T22:57:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.187+0000 7f9523e00640 1 -- 172.21.15.111:0/1256206704 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9534003da0 con 0x555e8163d8a0 2024-09-15T22:57:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.187+0000 7f9523e00640 1 -- 172.21.15.111:0/1256206704 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f9534030070 con 0x555e8163d8a0 2024-09-15T22:57:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.187+0000 7f954beac740 10 client.25921 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:57:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.187+0000 7f954beac740 10 client.25921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:57:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.187+0000 7f954beac740 20 client.25921 populate_metadata read hostname 'smithi111' 2024-09-15T22:57:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.187+0000 7f954beac740 1 --2- 172.21.15.111:0/1256206704 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555e816beb20 0x555e816c0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.187+0000 7f954beac740 4 client.25921 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:57:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.187+0000 7f954beac740 1 -- 172.21.15.111:0/1256206704 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555e815a1b80 con 0x555e816beb20 2024-09-15T22:57:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.187+0000 7f9541400640 1 --2- 172.21.15.111:0/1256206704 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555e816beb20 0x555e816c0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.188+0000 7f9523e00640 10 client.25921 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:57:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.188+0000 7f9541400640 1 --2- 172.21.15.111:0/1256206704 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555e816beb20 0x555e816c0f40 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd0b10b64ec3075b server_cookie=1e7f8cc015e59215 in_seq=0 out_seq=0 2024-09-15T22:57:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.201+0000 7f9523e00640 1 -- 172.21.15.111:0/1256206704 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555e815a1b80 con 0x555e816beb20 2024-09-15T22:57:27.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.201+0000 7f9523e00640 10 client.25921 handle_command_reply: tid=0 2024-09-15T22:57:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.202+0000 7f954beac740 1 client.25921 shutdown 2024-09-15T22:57:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.202+0000 7f954beac740 1 -- 172.21.15.111:0/1256206704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555e8163d8a0 msgr2=0x555e8164c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.202+0000 7f954beac740 1 --2- 172.21.15.111:0/1256206704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555e8163d8a0 0x555e8164c7d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9534011010 tx=0x7f953400a820 comp rx=0 tx=0).stop 2024-09-15T22:57:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.202+0000 7f954beac740 1 -- 172.21.15.111:0/1256206704 shutdown_connections 2024-09-15T22:57:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.202+0000 7f954beac740 1 --2- 172.21.15.111:0/1256206704 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555e816beb20 0x555e816c0f40 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.202+0000 7f954beac740 1 --2- 172.21.15.111:0/1256206704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555e81643270 0x555e8164dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.202+0000 7f9523e00640 0 client.25921 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:57:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.202+0000 7f954beac740 1 --2- 172.21.15.111:0/1256206704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555e8163e1e0 0x555e8164cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.202+0000 7f954beac740 1 --2- 172.21.15.111:0/1256206704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555e8163d8a0 0x555e8164c7d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.202+0000 7f954beac740 1 -- 172.21.15.111:0/1256206704 >> 172.21.15.111:0/1256206704 conn(0x555e81630700 msgr2=0x555e8162f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.202+0000 7f954beac740 1 -- 172.21.15.111:0/1256206704 shutdown_connections 2024-09-15T22:57:27.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.202+0000 7f954beac740 1 -- 172.21.15.111:0/1256206704 wait complete. 2024-09-15T22:57:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.202+0000 7f954beac740 20 client.25921 trim_cache size 0 max 16384 2024-09-15T22:57:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:57:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:57:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:57:27.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:57:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.203+0000 7f9549000640 1 -- 172.21.15.111:0/2937268668 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f95280a32f0 msgr2=0x7f95280a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.203+0000 7f9549000640 1 --2- 172.21.15.111:0/2937268668 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f95280a32f0 0x7f95280a57b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x555e8163ec10 tx=0x7f9534005f90 comp rx=0 tx=0).stop 2024-09-15T22:57:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.203+0000 7f9549000640 1 -- 172.21.15.111:0/2937268668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9544104f50 msgr2=0x7f954419a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.203+0000 7f9549000640 1 --2- 172.21.15.111:0/2937268668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9544104f50 0x7f954419a4f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9530006010 tx=0x7f9530004360 comp rx=0 tx=0).stop 2024-09-15T22:57:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.203+0000 7f9549000640 1 -- 172.21.15.111:0/2937268668 shutdown_connections 2024-09-15T22:57:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.203+0000 7f9549000640 1 --2- 172.21.15.111:0/2937268668 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f95280a32f0 0x7f95280a57b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.203+0000 7f9549000640 1 --2- 172.21.15.111:0/2937268668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9544105910 0x7f95441a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.203+0000 7f9549000640 1 --2- 172.21.15.111:0/2937268668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9544104f50 0x7f954419a4f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.203+0000 7f9549000640 1 --2- 172.21.15.111:0/2937268668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9544103d60 0x7f9544199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.203+0000 7f9549000640 1 -- 172.21.15.111:0/2937268668 >> 172.21.15.111:0/2937268668 conn(0x7f95440ff4e0 msgr2=0x7f95441081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.203+0000 7f9549000640 1 -- 172.21.15.111:0/2937268668 shutdown_connections 2024-09-15T22:57:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:27.203+0000 7f9549000640 1 -- 172.21.15.111:0/2937268668 wait complete. 2024-09-15T22:57:27.320 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1104114 9.90 MB/sec execute 475 sec latency 1010.088 ms 2024-09-15T22:57:28.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:27 smithi121 ceph-mon[34815]: pgmap v696: 97 pgs: 97 active+clean; 282 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 167 op/s 2024-09-15T22:57:28.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:27 smithi111 ceph-mon[30629]: pgmap v696: 97 pgs: 97 active+clean; 282 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 167 op/s 2024-09-15T22:57:28.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:27 smithi192 ceph-mon[35427]: pgmap v696: 97 pgs: 97 active+clean; 282 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 167 op/s 2024-09-15T22:57:28.320 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1105370 9.89 MB/sec execute 476 sec latency 1294.279 ms 2024-09-15T22:57:29.320 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1108447 9.91 MB/sec execute 477 sec latency 13.285 ms 2024-09-15T22:57:30.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:29 smithi121 ceph-mon[34815]: pgmap v697: 97 pgs: 97 active+clean; 284 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 209 op/s 2024-09-15T22:57:30.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:29 smithi111 ceph-mon[30629]: pgmap v697: 97 pgs: 97 active+clean; 284 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 209 op/s 2024-09-15T22:57:30.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:29 smithi192 ceph-mon[35427]: pgmap v697: 97 pgs: 97 active+clean; 284 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 209 op/s 2024-09-15T22:57:30.320 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1110993 9.91 MB/sec execute 478 sec latency 13.336 ms 2024-09-15T22:57:31.321 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1113187 9.92 MB/sec execute 479 sec latency 14.105 ms 2024-09-15T22:57:31.940 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:31 smithi111 ceph-mon[30629]: pgmap v698: 97 pgs: 97 active+clean; 284 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 160 op/s 2024-09-15T22:57:32.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:31 smithi121 ceph-mon[34815]: pgmap v698: 97 pgs: 97 active+clean; 284 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 160 op/s 2024-09-15T22:57:32.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:31 smithi192 ceph-mon[35427]: pgmap v698: 97 pgs: 97 active+clean; 284 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 160 op/s 2024-09-15T22:57:32.320 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1113277 9.90 MB/sec execute 480 sec latency 872.836 ms 2024-09-15T22:57:33.321 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1114343 9.89 MB/sec execute 481 sec latency 820.255 ms 2024-09-15T22:57:34.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:33 smithi121 ceph-mon[34815]: pgmap v699: 97 pgs: 97 active+clean; 287 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 203 op/s 2024-09-15T22:57:34.156 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:33 smithi111 ceph-mon[30629]: pgmap v699: 97 pgs: 97 active+clean; 287 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 203 op/s 2024-09-15T22:57:34.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:33 smithi192 ceph-mon[35427]: pgmap v699: 97 pgs: 97 active+clean; 287 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 203 op/s 2024-09-15T22:57:34.320 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1116399 9.89 MB/sec execute 482 sec latency 13.365 ms 2024-09-15T22:57:35.321 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1119286 9.91 MB/sec execute 483 sec latency 8.142 ms 2024-09-15T22:57:36.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:35 smithi121 ceph-mon[34815]: pgmap v700: 97 pgs: 97 active+clean; 288 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 184 op/s 2024-09-15T22:57:36.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:35 smithi111 ceph-mon[30629]: pgmap v700: 97 pgs: 97 active+clean; 288 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 184 op/s 2024-09-15T22:57:36.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:35 smithi192 ceph-mon[35427]: pgmap v700: 97 pgs: 97 active+clean; 288 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 184 op/s 2024-09-15T22:57:36.321 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1122070 9.92 MB/sec execute 484 sec latency 15.558 ms 2024-09-15T22:57:37.321 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1122527 9.90 MB/sec execute 485 sec latency 820.192 ms 2024-09-15T22:57:37.926 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:37 smithi111 ceph-mon[30629]: pgmap v701: 97 pgs: 97 active+clean; 287 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 184 op/s 2024-09-15T22:57:38.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:37 smithi121 ceph-mon[34815]: pgmap v701: 97 pgs: 97 active+clean; 287 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 184 op/s 2024-09-15T22:57:38.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:37 smithi192 ceph-mon[35427]: pgmap v701: 97 pgs: 97 active+clean; 287 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 184 op/s 2024-09-15T22:57:38.321 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1122527 9.88 MB/sec execute 486 sec latency 1820.233 ms 2024-09-15T22:57:39.321 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1123218 9.87 MB/sec execute 487 sec latency 2266.463 ms 2024-09-15T22:57:40.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:39 smithi121 ceph-mon[34815]: pgmap v702: 97 pgs: 97 active+clean; 294 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 220 op/s 2024-09-15T22:57:40.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:39 smithi111 ceph-mon[30629]: pgmap v702: 97 pgs: 97 active+clean; 294 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 220 op/s 2024-09-15T22:57:40.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:39 smithi192 ceph-mon[35427]: pgmap v702: 97 pgs: 97 active+clean; 294 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 220 op/s 2024-09-15T22:57:40.321 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1126225 9.88 MB/sec execute 488 sec latency 12.988 ms 2024-09-15T22:57:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.328+0000 7f38cf200640 1 -- 172.21.15.111:0/3276918824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38c8103050 msgr2=0x7f38c8103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.328+0000 7f38cf200640 1 --2- 172.21.15.111:0/3276918824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38c8103050 0x7f38c8103450 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009900 tx=0x7f38bc02a410 comp rx=0 tx=0).stop 2024-09-15T22:57:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.328+0000 7f38cf200640 1 -- 172.21.15.111:0/3276918824 shutdown_connections 2024-09-15T22:57:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.328+0000 7f38cf200640 1 --2- 172.21.15.111:0/3276918824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38c8101f40 0x7f38c8109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.328+0000 7f38cf200640 1 --2- 172.21.15.111:0/3276918824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38c8101580 0x7f38c8101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.328+0000 7f38cf200640 1 --2- 172.21.15.111:0/3276918824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38c8103050 0x7f38c8103450 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.328+0000 7f38cf200640 1 -- 172.21.15.111:0/3276918824 >> 172.21.15.111:0/3276918824 conn(0x7f38c80fd0d0 msgr2=0x7f38c80ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.328+0000 7f38cf200640 1 -- 172.21.15.111:0/3276918824 shutdown_connections 2024-09-15T22:57:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.328+0000 7f38cf200640 1 -- 172.21.15.111:0/3276918824 wait complete. 2024-09-15T22:57:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.329+0000 7f38cf200640 1 Processor -- start 2024-09-15T22:57:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.329+0000 7f38cf200640 1 -- start start 2024-09-15T22:57:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.329+0000 7f38cf200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38c8101580 0x7f38c8075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.329+0000 7f38cf200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38c8101f40 0x7f38c80745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.329+0000 7f38cf200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38c8103050 0x7f38c8074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.329+0000 7f38cf200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f38c810d6d0 con 0x7f38c8101580 2024-09-15T22:57:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.329+0000 7f38cf200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f38c810d550 con 0x7f38c8101f40 2024-09-15T22:57:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.329+0000 7f38cf200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f38c810d850 con 0x7f38c8103050 2024-09-15T22:57:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.330+0000 7f38ce800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38c8103050 0x7f38c8074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.330+0000 7f38cde00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38c8101580 0x7f38c8075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.330+0000 7f38ce800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38c8103050 0x7f38c8074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:45078/0 (socket says 172.21.15.111:45078) 2024-09-15T22:57:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.330+0000 7f38ce800640 1 -- 172.21.15.111:0/2914229168 learned_addr learned my addr 172.21.15.111:0/2914229168 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:57:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.330+0000 7f38cd400640 1 --2- 172.21.15.111:0/2914229168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38c8101f40 0x7f38c80745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.330+0000 7f38ce800640 1 -- 172.21.15.111:0/2914229168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38c8101f40 msgr2=0x7f38c80745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.330+0000 7f38ce800640 1 --2- 172.21.15.111:0/2914229168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38c8101f40 0x7f38c80745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.330+0000 7f38ce800640 1 -- 172.21.15.111:0/2914229168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38c8101580 msgr2=0x7f38c8075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.330+0000 7f38ce800640 1 --2- 172.21.15.111:0/2914229168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38c8101580 0x7f38c8075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.330+0000 7f38ce800640 1 -- 172.21.15.111:0/2914229168 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c80753b0 con 0x7f38c8103050 2024-09-15T22:57:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.331+0000 7f38ce800640 1 --2- 172.21.15.111:0/2914229168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38c8103050 0x7f38c8074b20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f38c400eb10 tx=0x7f38c400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.331+0000 7f38c2a00640 1 -- 172.21.15.111:0/2914229168 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c4004b20 con 0x7f38c8103050 2024-09-15T22:57:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.331+0000 7f38c2a00640 1 -- 172.21.15.111:0/2914229168 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38c4004cc0 con 0x7f38c8103050 2024-09-15T22:57:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.331+0000 7f38c2a00640 1 -- 172.21.15.111:0/2914229168 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c4010760 con 0x7f38c8103050 2024-09-15T22:57:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.331+0000 7f38cf200640 1 -- 172.21.15.111:0/2914229168 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c81a1ee0 con 0x7f38c8103050 2024-09-15T22:57:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.331+0000 7f38cf200640 1 -- 172.21.15.111:0/2914229168 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c81a2290 con 0x7f38c8103050 2024-09-15T22:57:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.334+0000 7f38c2a00640 1 -- 172.21.15.111:0/2914229168 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f38c4005020 con 0x7f38c8103050 2024-09-15T22:57:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.334+0000 7f38d36bd740 1 Processor -- start 2024-09-15T22:57:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.334+0000 7f38d36bd740 1 -- start start 2024-09-15T22:57:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.335+0000 7f38d36bd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563218b51260 0x563218b51660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.335+0000 7f38d36bd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563218b51ba0 0x563218b56660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.335+0000 7f38d36bd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563218b56c30 0x563218b590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.335+0000 7f38d36bd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563218ab6100 con 0x563218b51260 2024-09-15T22:57:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.335+0000 7f38d36bd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563218a5e1b0 con 0x563218b51ba0 2024-09-15T22:57:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.335+0000 7f38d36bd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563218ab20c0 con 0x563218b56c30 2024-09-15T22:57:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.335+0000 7f38cd400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563218b51260 0x563218b51660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.335+0000 7f38cd400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563218b51260 0x563218b51660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41444/0 (socket says 172.21.15.111:41444) 2024-09-15T22:57:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.335+0000 7f38cd400640 1 -- 172.21.15.111:0/1557386334 learned_addr learned my addr 172.21.15.111:0/1557386334 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:57:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.335+0000 7f38cde00640 1 --2- 172.21.15.111:0/1557386334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563218b51ba0 0x563218b56660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.335+0000 7f38cd400640 1 --2- 172.21.15.111:0/1557386334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563218b56c30 0x563218b590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.335+0000 7f38cde00640 1 -- 172.21.15.111:0/1557386334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563218b56c30 msgr2=0x563218b590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.335+0000 7f38cde00640 1 --2- 172.21.15.111:0/1557386334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563218b56c30 0x563218b590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.335+0000 7f38cde00640 1 -- 172.21.15.111:0/1557386334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563218b51260 msgr2=0x563218b51660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.335+0000 7f38cde00640 1 --2- 172.21.15.111:0/1557386334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563218b51260 0x563218b51660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.335+0000 7f38cde00640 1 -- 172.21.15.111:0/1557386334 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632187c9210 con 0x563218b51ba0 2024-09-15T22:57:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.335+0000 7f38cd400640 1 --2- 172.21.15.111:0/1557386334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563218b51260 0x563218b51660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:57:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.336+0000 7f38cde00640 1 --2- 172.21.15.111:0/1557386334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563218b51ba0 0x563218b56660 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009230 tx=0x7f38bc005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2112ed9e3f42eec server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.336+0000 7f38c2a00640 1 --2- 172.21.15.111:0/2914229168 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f38b00a32c0 0x7f38b00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:40.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.336+0000 7f38c2a00640 1 -- 172.21.15.111:0/2914229168 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f38c40cfe40 con 0x7f38c8103050 2024-09-15T22:57:40.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.336+0000 7f38b7000640 1 -- 172.21.15.111:0/1557386334 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc040070 con 0x563218b51ba0 2024-09-15T22:57:40.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.336+0000 7f38b7000640 1 -- 172.21.15.111:0/1557386334 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38bc004030 con 0x563218b51ba0 2024-09-15T22:57:40.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.336+0000 7f38b7000640 1 -- 172.21.15.111:0/1557386334 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc035040 con 0x563218b51ba0 2024-09-15T22:57:40.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.336+0000 7f38cd400640 1 --2- 172.21.15.111:0/2914229168 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f38b00a32c0 0x7f38b00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:40.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.337+0000 7f38d36bd740 1 -- 172.21.15.111:0/1557386334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563218b51ba0 msgr2=0x563218b56660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:40.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.337+0000 7f38cd400640 1 --2- 172.21.15.111:0/2914229168 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f38b00a32c0 0x7f38b00a5780 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f38b8005bc0 tx=0x7f38b8009850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:40.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.337+0000 7f38d36bd740 1 --2- 172.21.15.111:0/1557386334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563218b51ba0 0x563218b56660 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009230 tx=0x7f38bc005e20 comp rx=0 tx=0).stop 2024-09-15T22:57:40.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.337+0000 7f38d36bd740 1 -- 172.21.15.111:0/1557386334 shutdown_connections 2024-09-15T22:57:40.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.337+0000 7f38d36bd740 1 --2- 172.21.15.111:0/1557386334 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563218b56c30 0x563218b590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.337+0000 7f38d36bd740 1 --2- 172.21.15.111:0/1557386334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563218b51ba0 0x563218b56660 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.337+0000 7f38d36bd740 1 --2- 172.21.15.111:0/1557386334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563218b51260 0x563218b51660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.337+0000 7f38d36bd740 1 -- 172.21.15.111:0/1557386334 >> 172.21.15.111:0/1557386334 conn(0x563218b4c8d0 msgr2=0x563218b4ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:40.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.337+0000 7f38d36bd740 1 -- 172.21.15.111:0/1557386334 shutdown_connections 2024-09-15T22:57:40.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.337+0000 7f38d36bd740 1 -- 172.21.15.111:0/1557386334 wait complete. 2024-09-15T22:57:40.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.337+0000 7f38d36bd740 1 -- start start 2024-09-15T22:57:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.338+0000 7f38d36bd740 1 Processor -- start 2024-09-15T22:57:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.338+0000 7f38d36bd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563218b51260 0x563218b5b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.338+0000 7f38d36bd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563218b51ba0 0x563218b5b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.338+0000 7f38d36bd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563218b56c30 0x563218b62970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:40.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.338+0000 7f38d36bd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563218b59ba0 con 0x563218b56c30 2024-09-15T22:57:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.338+0000 7f38d36bd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563218a3f090 con 0x563218b51ba0 2024-09-15T22:57:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.338+0000 7f38d36bd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563218b59d20 con 0x563218b51260 2024-09-15T22:57:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.338+0000 7f38d36bd740 10 client.0 _ll_register_callbacks cb 0x563218a93f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:57:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.338+0000 7f38cd400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563218b51ba0 0x563218b5b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.338+0000 7f38cd400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563218b51ba0 0x563218b5b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:58804/0 (socket says 172.21.15.111:58804) 2024-09-15T22:57:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.338+0000 7f38cd400640 1 -- 172.21.15.111:0/323367439 learned_addr learned my addr 172.21.15.111:0/323367439 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:57:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.338+0000 7f38ce800640 1 --2- 172.21.15.111:0/323367439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563218b56c30 0x563218b62970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.338+0000 7f38cde00640 1 --2- 172.21.15.111:0/323367439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563218b51260 0x563218b5b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.339+0000 7f38cd400640 1 -- 172.21.15.111:0/323367439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563218b51260 msgr2=0x563218b5b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.339+0000 7f38cd400640 1 --2- 172.21.15.111:0/323367439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563218b51260 0x563218b5b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.339+0000 7f38cd400640 1 -- 172.21.15.111:0/323367439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563218b56c30 msgr2=0x563218b62970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.339+0000 7f38cd400640 1 --2- 172.21.15.111:0/323367439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563218b56c30 0x563218b62970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.339+0000 7f38cd400640 1 -- 172.21.15.111:0/323367439 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563218895620 con 0x563218b51ba0 2024-09-15T22:57:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.339+0000 7f38cd400640 1 --2- 172.21.15.111:0/323367439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563218b51ba0 0x563218b5b930 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f38b800fd20 tx=0x7f38b8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb8e78361eb570d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.339+0000 7f38ab400640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:57:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.339+0000 7f38ab400640 1 -- 172.21.15.111:0/323367439 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b8023420 con 0x563218b51ba0 2024-09-15T22:57:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.340+0000 7f38d36bd740 1 -- 172.21.15.111:0/323367439 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38bc004030 con 0x563218b51ba0 2024-09-15T22:57:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.340+0000 7f38ab400640 1 -- 172.21.15.111:0/323367439 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f38b80235c0 con 0x563218b51ba0 2024-09-15T22:57:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.340+0000 7f38ab400640 1 -- 172.21.15.111:0/323367439 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f38b80237c0 con 0x563218b51ba0 2024-09-15T22:57:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.340+0000 7f38d36bd740 10 client.25904 fetch_fsmap learned FSMap version 30 2024-09-15T22:57:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.340+0000 7f38d36bd740 1 -- 172.21.15.111:0/323367439 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563218b6e030 con 0x563218b51ba0 2024-09-15T22:57:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.340+0000 7f38ab400640 1 -- 172.21.15.111:0/323367439 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38bc004030 con 0x563218b51ba0 2024-09-15T22:57:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.340+0000 7f38ab400640 1 -- 172.21.15.111:0/323367439 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f38b80208d0 con 0x563218b51ba0 2024-09-15T22:57:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.341+0000 7f38d36bd740 10 client.25904 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:57:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.341+0000 7f38d36bd740 10 client.25904 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:57:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.341+0000 7f38d36bd740 20 client.25904 populate_metadata read hostname 'smithi111' 2024-09-15T22:57:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.341+0000 7f38d36bd740 1 --2- 172.21.15.111:0/323367439 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563218b6f700 0x563218b71b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.341+0000 7f38d36bd740 4 client.25904 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:57:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.341+0000 7f38d36bd740 1 -- 172.21.15.111:0/323367439 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563218ac3830 con 0x563218b6f700 2024-09-15T22:57:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.341+0000 7f38cde00640 1 --2- 172.21.15.111:0/323367439 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563218b6f700 0x563218b71b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.342+0000 7f38cde00640 1 --2- 172.21.15.111:0/323367439 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563218b6f700 0x563218b71b50 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=590b3d94fdc46e01 server_cookie=61b268b10926829f in_seq=0 out_seq=0 2024-09-15T22:57:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.342+0000 7f38ab400640 10 client.25904 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:57:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.342+0000 7f38ab400640 1 -- 172.21.15.111:0/323367439 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563218ac3830 con 0x563218b6f700 2024-09-15T22:57:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.342+0000 7f38ab400640 10 client.25904 handle_command_reply: tid=0 2024-09-15T22:57:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.342+0000 7f38d36bd740 1 client.25904 shutdown 2024-09-15T22:57:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.343+0000 7f38d36bd740 1 -- 172.21.15.111:0/323367439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563218b51ba0 msgr2=0x563218b5b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.343+0000 7f38d36bd740 1 --2- 172.21.15.111:0/323367439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563218b51ba0 0x563218b5b930 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f38b800fd20 tx=0x7f38b8004730 comp rx=0 tx=0).stop 2024-09-15T22:57:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.343+0000 7f38d36bd740 1 -- 172.21.15.111:0/323367439 shutdown_connections 2024-09-15T22:57:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.343+0000 7f38d36bd740 1 --2- 172.21.15.111:0/323367439 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563218b6f700 0x563218b71b50 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.343+0000 7f38d36bd740 1 --2- 172.21.15.111:0/323367439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563218b56c30 0x563218b62970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.343+0000 7f38d36bd740 1 --2- 172.21.15.111:0/323367439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563218b51ba0 0x563218b5b930 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.343+0000 7f38d36bd740 1 --2- 172.21.15.111:0/323367439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563218b51260 0x563218b5b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.343+0000 7f38d36bd740 1 -- 172.21.15.111:0/323367439 >> 172.21.15.111:0/323367439 conn(0x563218b4c8d0 msgr2=0x563218b4e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.343+0000 7f38ab400640 0 client.25904 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:57:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.343+0000 7f38d36bd740 1 -- 172.21.15.111:0/323367439 shutdown_connections 2024-09-15T22:57:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.343+0000 7f38d36bd740 1 -- 172.21.15.111:0/323367439 wait complete. 2024-09-15T22:57:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.343+0000 7f38d36bd740 20 client.25904 trim_cache size 0 max 16384 2024-09-15T22:57:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.359+0000 7f38d36bd740 1 Processor -- start 2024-09-15T22:57:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.359+0000 7f38d36bd740 1 -- start start 2024-09-15T22:57:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.359+0000 7f38d36bd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563218b51260 0x563218b478d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.359+0000 7f38d36bd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563218b51ba0 0x563218b62930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.359+0000 7f38d36bd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563218b56c30 0x563218b4d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.359+0000 7f38d36bd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f38c81a93e0 con 0x563218b56c30 2024-09-15T22:57:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.359+0000 7f38d36bd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f38c81a6cb0 con 0x563218b51260 2024-09-15T22:57:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.359+0000 7f38d36bd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f38c81a4b00 con 0x563218b51ba0 2024-09-15T22:57:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.360+0000 7f38cde00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563218b51260 0x563218b478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.360+0000 7f38cd400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563218b51ba0 0x563218b62930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.360+0000 7f38cd400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563218b51ba0 0x563218b62930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:45098/0 (socket says 172.21.15.111:45098) 2024-09-15T22:57:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.360+0000 7f38cde00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563218b51260 0x563218b478d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:58820/0 (socket says 172.21.15.111:58820) 2024-09-15T22:57:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.360+0000 7f38cd400640 1 -- 172.21.15.111:0/380904405 learned_addr learned my addr 172.21.15.111:0/380904405 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:57:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.360+0000 7f38ce800640 1 --2- 172.21.15.111:0/380904405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563218b56c30 0x563218b4d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.360+0000 7f38cde00640 1 -- 172.21.15.111:0/380904405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563218b51ba0 msgr2=0x563218b62930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.360+0000 7f38cde00640 1 --2- 172.21.15.111:0/380904405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563218b51ba0 0x563218b62930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.360+0000 7f38cde00640 1 -- 172.21.15.111:0/380904405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563218b56c30 msgr2=0x563218b4d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.360+0000 7f38cde00640 1 --2- 172.21.15.111:0/380904405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563218b56c30 0x563218b4d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.360+0000 7f38cde00640 1 -- 172.21.15.111:0/380904405 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563218b4c6f0 con 0x563218b51260 2024-09-15T22:57:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.360+0000 7f38cd400640 1 --2- 172.21.15.111:0/380904405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563218b51ba0 0x563218b62930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:57:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.360+0000 7f38ce800640 1 --2- 172.21.15.111:0/380904405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563218b56c30 0x563218b4d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:57:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.360+0000 7f38cde00640 1 --2- 172.21.15.111:0/380904405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563218b51260 0x563218b478d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f38bc02a930 tx=0x7f38bc02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c43ee270e828392 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.360+0000 7f38b6600640 1 -- 172.21.15.111:0/380904405 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc040040 con 0x563218b51260 2024-09-15T22:57:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.361+0000 7f38b6600640 1 -- 172.21.15.111:0/380904405 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38bc0318b0 con 0x563218b51260 2024-09-15T22:57:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.361+0000 7f38b6600640 1 -- 172.21.15.111:0/380904405 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc035040 con 0x563218b51260 2024-09-15T22:57:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.361+0000 7f38d36bd740 1 -- 172.21.15.111:0/380904405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563218b51260 msgr2=0x563218b478d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.361+0000 7f38d36bd740 1 --2- 172.21.15.111:0/380904405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563218b51260 0x563218b478d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f38bc02a930 tx=0x7f38bc02a410 comp rx=0 tx=0).stop 2024-09-15T22:57:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.361+0000 7f38d36bd740 1 -- 172.21.15.111:0/380904405 shutdown_connections 2024-09-15T22:57:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.361+0000 7f38d36bd740 1 --2- 172.21.15.111:0/380904405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563218b56c30 0x563218b4d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.361+0000 7f38d36bd740 1 --2- 172.21.15.111:0/380904405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563218b51ba0 0x563218b62930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.361+0000 7f38d36bd740 1 --2- 172.21.15.111:0/380904405 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563218b51260 0x563218b478d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.362+0000 7f38d36bd740 1 -- 172.21.15.111:0/380904405 >> 172.21.15.111:0/380904405 conn(0x563218b4c8d0 msgr2=0x563218b5b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:40.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.362+0000 7f38d36bd740 1 -- 172.21.15.111:0/380904405 shutdown_connections 2024-09-15T22:57:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.362+0000 7f38d36bd740 1 -- 172.21.15.111:0/380904405 wait complete. 2024-09-15T22:57:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.362+0000 7f38d36bd740 1 -- start start 2024-09-15T22:57:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.362+0000 7f38d36bd740 1 Processor -- start 2024-09-15T22:57:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.363+0000 7f38d36bd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563218b51260 0x563218b5fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.363+0000 7f38d36bd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563218b51ba0 0x563218b603a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.363+0000 7f38d36bd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563218b56c30 0x563218b61310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.363+0000 7f38d36bd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f38c81a5170 con 0x563218b51ba0 2024-09-15T22:57:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.363+0000 7f38d36bd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f38c81a65e0 con 0x563218b56c30 2024-09-15T22:57:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.363+0000 7f38d36bd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f38c81a5840 con 0x563218b51260 2024-09-15T22:57:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.363+0000 7f38d36bd740 10 client.0 _ll_register_callbacks cb 0x7f38b8010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:57:40.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.363+0000 7f38cde00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563218b51260 0x563218b5fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.363+0000 7f38ce800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563218b56c30 0x563218b61310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.363+0000 7f38cde00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563218b51260 0x563218b5fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:45106/0 (socket says 172.21.15.111:45106) 2024-09-15T22:57:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.363+0000 7f38cde00640 1 -- 172.21.15.111:0/2382588460 learned_addr learned my addr 172.21.15.111:0/2382588460 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:57:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.363+0000 7f38ce800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563218b56c30 0x563218b61310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:58834/0 (socket says 172.21.15.111:58834) 2024-09-15T22:57:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.363+0000 7f38cd400640 1 --2- 172.21.15.111:0/2382588460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563218b51ba0 0x563218b603a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.363+0000 7f38cde00640 1 -- 172.21.15.111:0/2382588460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563218b56c30 msgr2=0x563218b61310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.363+0000 7f38cde00640 1 --2- 172.21.15.111:0/2382588460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563218b56c30 0x563218b61310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.363+0000 7f38cde00640 1 -- 172.21.15.111:0/2382588460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563218b51ba0 msgr2=0x563218b603a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.363+0000 7f38cde00640 1 --2- 172.21.15.111:0/2382588460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563218b51ba0 0x563218b603a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.363+0000 7f38cde00640 1 -- 172.21.15.111:0/2382588460 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563218b61920 con 0x563218b51260 2024-09-15T22:57:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.364+0000 7f38cde00640 1 --2- 172.21.15.111:0/2382588460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563218b51260 0x563218b5fe60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f38bc005b90 tx=0x7f38bc005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a48f0fbc466a7654 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.364+0000 7f38ab400640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:57:40.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.364+0000 7f38ab400640 1 -- 172.21.15.111:0/2382588460 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc040050 con 0x563218b51260 2024-09-15T22:57:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.364+0000 7f38d36bd740 1 -- 172.21.15.111:0/2382588460 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38bc0318b0 con 0x563218b51260 2024-09-15T22:57:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.364+0000 7f38ab400640 1 -- 172.21.15.111:0/2382588460 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f38bc004040 con 0x563218b51260 2024-09-15T22:57:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.364+0000 7f38ab400640 1 -- 172.21.15.111:0/2382588460 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f38bc035040 con 0x563218b51260 2024-09-15T22:57:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.364+0000 7f38d36bd740 10 client.25939 fetch_fsmap learned FSMap version 30 2024-09-15T22:57:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.364+0000 7f38d36bd740 1 -- 172.21.15.111:0/2382588460 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563218b62c30 con 0x563218b51260 2024-09-15T22:57:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.364+0000 7f38ab400640 1 -- 172.21.15.111:0/2382588460 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38bc0318b0 con 0x563218b51260 2024-09-15T22:57:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.365+0000 7f38ab400640 1 -- 172.21.15.111:0/2382588460 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f38bc064480 con 0x563218b51260 2024-09-15T22:57:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.365+0000 7f38d36bd740 10 client.25939 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:57:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.365+0000 7f38d36bd740 10 client.25939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:57:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.365+0000 7f38d36bd740 20 client.25939 populate_metadata read hostname 'smithi111' 2024-09-15T22:57:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.365+0000 7f38d36bd740 1 --2- 172.21.15.111:0/2382588460 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563218bda690 0x563218bdca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.365+0000 7f38d36bd740 4 client.25939 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:57:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.365+0000 7f38d36bd740 1 -- 172.21.15.111:0/2382588460 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563218abdb80 con 0x563218bda690 2024-09-15T22:57:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.365+0000 7f38cd400640 1 --2- 172.21.15.111:0/2382588460 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563218bda690 0x563218bdca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.365+0000 7f38cd400640 1 --2- 172.21.15.111:0/2382588460 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563218bda690 0x563218bdca80 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a1ff440a73a9cb6 server_cookie=e0b5d9a682f4d643 in_seq=0 out_seq=0 2024-09-15T22:57:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.365+0000 7f38ab400640 10 client.25939 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:57:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.376+0000 7f38ab400640 1 -- 172.21.15.111:0/2382588460 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563218abdb80 con 0x563218bda690 2024-09-15T22:57:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.376+0000 7f38ab400640 10 client.25939 handle_command_reply: tid=0 2024-09-15T22:57:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.376+0000 7f38d36bd740 1 client.25939 shutdown 2024-09-15T22:57:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.377+0000 7f38d36bd740 1 -- 172.21.15.111:0/2382588460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563218b51260 msgr2=0x563218b5fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.377+0000 7f38d36bd740 1 --2- 172.21.15.111:0/2382588460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563218b51260 0x563218b5fe60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f38bc005b90 tx=0x7f38bc005de0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.377+0000 7f38d36bd740 1 -- 172.21.15.111:0/2382588460 shutdown_connections 2024-09-15T22:57:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.377+0000 7f38d36bd740 1 --2- 172.21.15.111:0/2382588460 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563218bda690 0x563218bdca80 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.377+0000 7f38d36bd740 1 --2- 172.21.15.111:0/2382588460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563218b56c30 0x563218b61310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.377+0000 7f38d36bd740 1 --2- 172.21.15.111:0/2382588460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563218b51ba0 0x563218b603a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.377+0000 7f38d36bd740 1 --2- 172.21.15.111:0/2382588460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563218b51260 0x563218b5fe60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.377+0000 7f38ab400640 0 client.25939 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:57:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.377+0000 7f38d36bd740 1 -- 172.21.15.111:0/2382588460 >> 172.21.15.111:0/2382588460 conn(0x563218b4c8d0 msgr2=0x563218b4b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.377+0000 7f38d36bd740 1 -- 172.21.15.111:0/2382588460 shutdown_connections 2024-09-15T22:57:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.377+0000 7f38d36bd740 1 -- 172.21.15.111:0/2382588460 wait complete. 2024-09-15T22:57:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.377+0000 7f38d36bd740 20 client.25939 trim_cache size 0 max 16384 2024-09-15T22:57:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:57:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:57:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:57:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:57:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.378+0000 7f38cf200640 1 -- 172.21.15.111:0/2914229168 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f38b00a32c0 msgr2=0x7f38b00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.378+0000 7f38cf200640 1 --2- 172.21.15.111:0/2914229168 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f38b00a32c0 0x7f38b00a5780 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f38b8005bc0 tx=0x7f38b8009850 comp rx=0 tx=0).stop 2024-09-15T22:57:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.378+0000 7f38cf200640 1 -- 172.21.15.111:0/2914229168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38c8103050 msgr2=0x7f38c8074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.378+0000 7f38cf200640 1 --2- 172.21.15.111:0/2914229168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38c8103050 0x7f38c8074b20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f38c400eb10 tx=0x7f38c400c2d0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.378+0000 7f38cf200640 1 -- 172.21.15.111:0/2914229168 shutdown_connections 2024-09-15T22:57:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.378+0000 7f38cf200640 1 --2- 172.21.15.111:0/2914229168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38c8103050 0x7f38c8074b20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.378+0000 7f38cf200640 1 --2- 172.21.15.111:0/2914229168 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f38b00a32c0 0x7f38b00a5780 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.378+0000 7f38cf200640 1 --2- 172.21.15.111:0/2914229168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38c8101f40 0x7f38c80745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.378+0000 7f38cf200640 1 --2- 172.21.15.111:0/2914229168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f38c8101580 0x7f38c8075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.378+0000 7f38cf200640 1 -- 172.21.15.111:0/2914229168 >> 172.21.15.111:0/2914229168 conn(0x7f38c80fd0d0 msgr2=0x7f38c8105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.378+0000 7f38cf200640 1 -- 172.21.15.111:0/2914229168 shutdown_connections 2024-09-15T22:57:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:40.378+0000 7f38cf200640 1 -- 172.21.15.111:0/2914229168 wait complete. 2024-09-15T22:57:41.321 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1128975 9.89 MB/sec execute 489 sec latency 15.327 ms 2024-09-15T22:57:42.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:41 smithi121 ceph-mon[34815]: pgmap v703: 97 pgs: 97 active+clean; 294 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 178 op/s 2024-09-15T22:57:42.144 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:41 smithi111 ceph-mon[30629]: pgmap v703: 97 pgs: 97 active+clean; 294 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 178 op/s 2024-09-15T22:57:42.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:41 smithi192 ceph-mon[35427]: pgmap v703: 97 pgs: 97 active+clean; 294 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 178 op/s 2024-09-15T22:57:42.321 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1129752 9.88 MB/sec execute 490 sec latency 689.049 ms 2024-09-15T22:57:43.321 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1130167 9.86 MB/sec execute 491 sec latency 1391.216 ms 2024-09-15T22:57:44.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:43 smithi121 ceph-mon[34815]: pgmap v704: 97 pgs: 97 active+clean; 294 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 203 op/s 2024-09-15T22:57:44.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:43 smithi111 ceph-mon[30629]: pgmap v704: 97 pgs: 97 active+clean; 294 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 203 op/s 2024-09-15T22:57:44.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:43 smithi192 ceph-mon[35427]: pgmap v704: 97 pgs: 97 active+clean; 294 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 203 op/s 2024-09-15T22:57:44.321 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1133356 9.88 MB/sec execute 492 sec latency 10.881 ms 2024-09-15T22:57:45.321 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1136123 9.89 MB/sec execute 493 sec latency 11.080 ms 2024-09-15T22:57:46.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:45 smithi111 ceph-mon[30629]: pgmap v705: 97 pgs: 97 active+clean; 292 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 175 op/s 2024-09-15T22:57:46.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:45 smithi192 ceph-mon[35427]: pgmap v705: 97 pgs: 97 active+clean; 292 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 175 op/s 2024-09-15T22:57:46.321 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1137712 9.89 MB/sec execute 494 sec latency 213.871 ms 2024-09-15T22:57:46.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:45 smithi121 ceph-mon[34815]: pgmap v705: 97 pgs: 97 active+clean; 292 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 175 op/s 2024-09-15T22:57:47.322 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1138632 9.87 MB/sec execute 495 sec latency 411.261 ms 2024-09-15T22:57:48.144 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:47 smithi111 ceph-mon[30629]: pgmap v706: 97 pgs: 97 active+clean; 292 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 175 op/s 2024-09-15T22:57:48.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:47 smithi192 ceph-mon[35427]: pgmap v706: 97 pgs: 97 active+clean; 292 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 175 op/s 2024-09-15T22:57:48.321 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1138632 9.85 MB/sec execute 496 sec latency 1411.343 ms 2024-09-15T22:57:48.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:47 smithi121 ceph-mon[34815]: pgmap v706: 97 pgs: 97 active+clean; 292 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 175 op/s 2024-09-15T22:57:49.322 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1139625 9.85 MB/sec execute 497 sec latency 2086.133 ms 2024-09-15T22:57:50.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:49 smithi111 ceph-mon[30629]: pgmap v707: 97 pgs: 97 active+clean; 298 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 208 op/s 2024-09-15T22:57:50.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:49 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:57:50.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:49 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:57:50.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:49 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:57:50.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:49 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:57:50.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:49 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:57:50.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:49 smithi192 ceph-mon[35427]: pgmap v707: 97 pgs: 97 active+clean; 298 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 208 op/s 2024-09-15T22:57:50.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:49 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:57:50.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:49 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:57:50.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:49 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:57:50.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:49 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:57:50.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:49 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:57:50.322 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1141309 9.84 MB/sec execute 498 sec latency 93.785 ms 2024-09-15T22:57:50.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:49 smithi121 ceph-mon[34815]: pgmap v707: 97 pgs: 97 active+clean; 298 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 208 op/s 2024-09-15T22:57:50.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:49 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:57:50.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:49 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:57:50.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:49 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:57:50.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:49 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:57:50.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:49 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:57:51.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:57:51.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:57:51.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:57:51.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:57:51.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:57:51.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:50 smithi111 ceph-mon[30629]: pgmap v708: 97 pgs: 97 active+clean; 299 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-09-15T22:57:51.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:57:51.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:57:51.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:57:51.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:57:51.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:57:51.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:50 smithi192 ceph-mon[35427]: pgmap v708: 97 pgs: 97 active+clean; 299 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-09-15T22:57:51.322 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1144182 9.86 MB/sec execute 499 sec latency 9.363 ms 2024-09-15T22:57:51.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:57:51.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:57:51.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:57:51.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:57:51.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:57:51.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:50 smithi121 ceph-mon[34815]: pgmap v708: 97 pgs: 97 active+clean; 299 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-09-15T22:57:52.322 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1145552 9.85 MB/sec execute 500 sec latency 244.703 ms 2024-09-15T22:57:53.322 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1145552 9.83 MB/sec execute 501 sec latency 1240.324 ms 2024-09-15T22:57:53.498 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.496+0000 7f9ec2c00640 1 -- 172.21.15.111:0/1457506653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ebc0761d0 msgr2=0x7f9ebc074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:53.498 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.496+0000 7f9ec2c00640 1 --2- 172.21.15.111:0/1457506653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ebc0761d0 0x7f9ebc074620 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009900 tx=0x7f9eac02a480 comp rx=0 tx=0).stop 2024-09-15T22:57:53.498 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.497+0000 7f9ec2c00640 1 -- 172.21.15.111:0/1457506653 shutdown_connections 2024-09-15T22:57:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.497+0000 7f9ec2c00640 1 --2- 172.21.15.111:0/1457506653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ebc1099f0 0x7f9ebc10beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.497+0000 7f9ec2c00640 1 --2- 172.21.15.111:0/1457506653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ebc074bf0 0x7f9ebc075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.497+0000 7f9ec2c00640 1 --2- 172.21.15.111:0/1457506653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ebc0761d0 0x7f9ebc074620 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.497+0000 7f9ec2c00640 1 -- 172.21.15.111:0/1457506653 >> 172.21.15.111:0/1457506653 conn(0x7f9ebc0fd2e0 msgr2=0x7f9ebc0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.497+0000 7f9ec2c00640 1 -- 172.21.15.111:0/1457506653 shutdown_connections 2024-09-15T22:57:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.497+0000 7f9ec2c00640 1 -- 172.21.15.111:0/1457506653 wait complete. 2024-09-15T22:57:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.497+0000 7f9ec2c00640 1 Processor -- start 2024-09-15T22:57:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.498+0000 7f9ec2c00640 1 -- start start 2024-09-15T22:57:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.498+0000 7f9ec2c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ebc074bf0 0x7f9ebc199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.498+0000 7f9ec2c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ebc0761d0 0x7f9ebc19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.498+0000 7f9ec2c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ebc1099f0 0x7f9ebc1a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.498+0000 7f9ec2c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc10f9d0 con 0x7f9ebc0761d0 2024-09-15T22:57:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.498+0000 7f9ec2c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc10f850 con 0x7f9ebc074bf0 2024-09-15T22:57:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.498+0000 7f9ec2c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc10fb50 con 0x7f9ebc1099f0 2024-09-15T22:57:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.498+0000 7f9ebbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ebc074bf0 0x7f9ebc199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.498+0000 7f9ebb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ebc0761d0 0x7f9ebc19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.498+0000 7f9ebbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ebc074bf0 0x7f9ebc199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:60926/0 (socket says 172.21.15.111:60926) 2024-09-15T22:57:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.498+0000 7f9ebbe00640 1 -- 172.21.15.111:0/1733856093 learned_addr learned my addr 172.21.15.111:0/1733856093 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:57:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.498+0000 7f9ec0c00640 1 --2- 172.21.15.111:0/1733856093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ebc1099f0 0x7f9ebc1a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.499+0000 7f9ebbe00640 1 -- 172.21.15.111:0/1733856093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ebc1099f0 msgr2=0x7f9ebc1a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.499+0000 7f9ebbe00640 1 --2- 172.21.15.111:0/1733856093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ebc1099f0 0x7f9ebc1a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.499+0000 7f9ebbe00640 1 -- 172.21.15.111:0/1733856093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ebc0761d0 msgr2=0x7f9ebc19a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.499+0000 7f9ebbe00640 1 --2- 172.21.15.111:0/1733856093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ebc0761d0 0x7f9ebc19a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.499+0000 7f9ebbe00640 1 -- 172.21.15.111:0/1733856093 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ebc1a1c10 con 0x7f9ebc074bf0 2024-09-15T22:57:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.499+0000 7f9ebbe00640 1 --2- 172.21.15.111:0/1733856093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ebc074bf0 0x7f9ebc199ff0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009230 tx=0x7f9eac0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.499+0000 7f9eb8c00640 1 -- 172.21.15.111:0/1733856093 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac03f030 con 0x7f9ebc074bf0 2024-09-15T22:57:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.499+0000 7f9eb8c00640 1 -- 172.21.15.111:0/1733856093 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eac003ce0 con 0x7f9ebc074bf0 2024-09-15T22:57:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.499+0000 7f9ec2c00640 1 -- 172.21.15.111:0/1733856093 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ebc1a1ea0 con 0x7f9ebc074bf0 2024-09-15T22:57:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.499+0000 7f9eb8c00640 1 -- 172.21.15.111:0/1733856093 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eac035070 con 0x7f9ebc074bf0 2024-09-15T22:57:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.499+0000 7f9ec2c00640 1 -- 172.21.15.111:0/1733856093 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ebc1a22b0 con 0x7f9ebc074bf0 2024-09-15T22:57:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.503+0000 7f9ec5c31740 1 Processor -- start 2024-09-15T22:57:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.504+0000 7f9ec5c31740 1 -- start start 2024-09-15T22:57:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.504+0000 7f9ec5c31740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afcb5ad090 0x55afcb5ad490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.504+0000 7f9ec5c31740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afcb5ad9d0 0x55afcb5b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.504+0000 7f9eb8c00640 1 -- 172.21.15.111:0/1733856093 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f9eac042030 con 0x7f9ebc074bf0 2024-09-15T22:57:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.504+0000 7f9ec5c31740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afcb5b2a60 0x55afcb5b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.504+0000 7f9ec5c31740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55afcb512100 con 0x55afcb5ad9d0 2024-09-15T22:57:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.504+0000 7f9ec5c31740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55afcb4ba1b0 con 0x55afcb5ad090 2024-09-15T22:57:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.504+0000 7f9ec5c31740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55afcb50e0c0 con 0x55afcb5b2a60 2024-09-15T22:57:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.504+0000 7f9ec0c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afcb5ad090 0x55afcb5ad490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.504+0000 7f9ec0c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afcb5ad090 0x55afcb5ad490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:60930/0 (socket says 172.21.15.111:60930) 2024-09-15T22:57:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.504+0000 7f9ec0c00640 1 -- 172.21.15.111:0/3676100982 learned_addr learned my addr 172.21.15.111:0/3676100982 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:57:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.505+0000 7f9ebbe00640 1 --2- 172.21.15.111:0/3676100982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afcb5ad9d0 0x55afcb5b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.505+0000 7f9ebb400640 1 --2- 172.21.15.111:0/3676100982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afcb5b2a60 0x55afcb5b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.505+0000 7f9ec0c00640 1 -- 172.21.15.111:0/3676100982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afcb5b2a60 msgr2=0x55afcb5b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.505+0000 7f9ec0c00640 1 --2- 172.21.15.111:0/3676100982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afcb5b2a60 0x55afcb5b4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.505+0000 7f9ec0c00640 1 -- 172.21.15.111:0/3676100982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afcb5ad9d0 msgr2=0x55afcb5b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.505+0000 7f9ec0c00640 1 --2- 172.21.15.111:0/3676100982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afcb5ad9d0 0x55afcb5b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.505+0000 7f9ec0c00640 1 -- 172.21.15.111:0/3676100982 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afcb225210 con 0x55afcb5ad090 2024-09-15T22:57:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.505+0000 7f9ebbe00640 1 --2- 172.21.15.111:0/3676100982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afcb5ad9d0 0x55afcb5b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:57:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.505+0000 7f9ec0c00640 1 --2- 172.21.15.111:0/3676100982 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afcb5ad090 0x55afcb5ad490 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009fd0 tx=0x7f9eb40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de114362067c149a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.506+0000 7f9eb1200640 1 -- 172.21.15.111:0/3676100982 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4004d20 con 0x55afcb5ad090 2024-09-15T22:57:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.506+0000 7f9eb1200640 1 -- 172.21.15.111:0/3676100982 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eb4004050 con 0x55afcb5ad090 2024-09-15T22:57:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.506+0000 7f9eb1200640 1 -- 172.21.15.111:0/3676100982 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4005260 con 0x55afcb5ad090 2024-09-15T22:57:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.506+0000 7f9eb8c00640 1 --2- 172.21.15.111:0/1733856093 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9ea00a32f0 0x7f9ea00a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.506+0000 7f9eb8c00640 1 -- 172.21.15.111:0/1733856093 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f9eac0efce0 con 0x7f9ebc074bf0 2024-09-15T22:57:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.506+0000 7f9ebbe00640 1 --2- 172.21.15.111:0/1733856093 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9ea00a32f0 0x7f9ea00a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.506+0000 7f9ec5c31740 1 -- 172.21.15.111:0/3676100982 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afcb5ad090 msgr2=0x55afcb5ad490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.507+0000 7f9ec5c31740 1 --2- 172.21.15.111:0/3676100982 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afcb5ad090 0x55afcb5ad490 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009fd0 tx=0x7f9eb40118d0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.507+0000 7f9ec5c31740 1 -- 172.21.15.111:0/3676100982 shutdown_connections 2024-09-15T22:57:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.507+0000 7f9ec5c31740 1 --2- 172.21.15.111:0/3676100982 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afcb5b2a60 0x55afcb5b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.507+0000 7f9ec5c31740 1 --2- 172.21.15.111:0/3676100982 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afcb5ad9d0 0x55afcb5b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.507+0000 7f9ec5c31740 1 --2- 172.21.15.111:0/3676100982 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afcb5ad090 0x55afcb5ad490 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.507+0000 7f9ec5c31740 1 -- 172.21.15.111:0/3676100982 >> 172.21.15.111:0/3676100982 conn(0x55afcb5a8700 msgr2=0x55afcb5a8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.507+0000 7f9ec5c31740 1 -- 172.21.15.111:0/3676100982 shutdown_connections 2024-09-15T22:57:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.507+0000 7f9ec5c31740 1 -- 172.21.15.111:0/3676100982 wait complete. 2024-09-15T22:57:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.507+0000 7f9ebbe00640 1 --2- 172.21.15.111:0/1733856093 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9ea00a32f0 0x7f9ea00a57b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0bc150 tx=0x7f9eac009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.507+0000 7f9ec5c31740 1 -- start start 2024-09-15T22:57:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.507+0000 7f9ec5c31740 1 Processor -- start 2024-09-15T22:57:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.508+0000 7f9ec5c31740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afcb5ad090 0x55afcb5b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.508+0000 7f9ec5c31740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afcb5ad9d0 0x55afcb5b7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.508+0000 7f9ec5c31740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afcb5b2a60 0x55afcb5be770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.508+0000 7f9ec5c31740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55afcb5b5a30 con 0x55afcb5ad9d0 2024-09-15T22:57:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.508+0000 7f9ec5c31740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55afcb49b090 con 0x55afcb5b2a60 2024-09-15T22:57:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.508+0000 7f9ec5c31740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55afcb5b5bb0 con 0x55afcb5ad090 2024-09-15T22:57:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.508+0000 7f9ec5c31740 10 client.0 _ll_register_callbacks cb 0x55afcb4eff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:57:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.508+0000 7f9ec0c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afcb5ad090 0x55afcb5b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.508+0000 7f9ec0c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afcb5ad090 0x55afcb5b72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:51622/0 (socket says 172.21.15.111:51622) 2024-09-15T22:57:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.508+0000 7f9ec0c00640 1 -- 172.21.15.111:0/1414702273 learned_addr learned my addr 172.21.15.111:0/1414702273 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:57:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.508+0000 7f9ebb400640 1 --2- 172.21.15.111:0/1414702273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afcb5ad9d0 0x55afcb5b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.508+0000 7f9ec0c00640 1 --2- 172.21.15.111:0/1414702273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afcb5b2a60 0x55afcb5be770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.509+0000 7f9ec0c00640 1 -- 172.21.15.111:0/1414702273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afcb5b2a60 msgr2=0x55afcb5be770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.509+0000 7f9ec0c00640 1 --2- 172.21.15.111:0/1414702273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afcb5b2a60 0x55afcb5be770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.509+0000 7f9ec0c00640 1 -- 172.21.15.111:0/1414702273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afcb5ad9d0 msgr2=0x55afcb5b7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.509+0000 7f9ec0c00640 1 --2- 172.21.15.111:0/1414702273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afcb5ad9d0 0x55afcb5b7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.509+0000 7f9ec0c00640 1 -- 172.21.15.111:0/1414702273 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afcb2f1620 con 0x55afcb5ad090 2024-09-15T22:57:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.509+0000 7f9ec0c00640 1 --2- 172.21.15.111:0/1414702273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afcb5ad090 0x55afcb5b72d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009370 tx=0x7f9eb4000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5830cfc878ac475 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.509+0000 7f9ea5600640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:57:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.509+0000 7f9ea5600640 1 -- 172.21.15.111:0/1414702273 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb400f6e0 con 0x55afcb5ad090 2024-09-15T22:57:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.509+0000 7f9ec5c31740 1 -- 172.21.15.111:0/1414702273 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9eb4004050 con 0x55afcb5ad090 2024-09-15T22:57:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.509+0000 7f9ea5600640 1 -- 172.21.15.111:0/1414702273 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f9eb400f880 con 0x55afcb5ad090 2024-09-15T22:57:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.510+0000 7f9ea5600640 1 -- 172.21.15.111:0/1414702273 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9eb4004680 con 0x55afcb5ad090 2024-09-15T22:57:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.510+0000 7f9ec5c31740 10 client.25951 fetch_fsmap learned FSMap version 30 2024-09-15T22:57:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.510+0000 7f9ec5c31740 1 -- 172.21.15.111:0/1414702273 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55afcb5c1560 con 0x55afcb5ad090 2024-09-15T22:57:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.510+0000 7f9ea5600640 1 -- 172.21.15.111:0/1414702273 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eb4004050 con 0x55afcb5ad090 2024-09-15T22:57:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.510+0000 7f9ea5600640 1 -- 172.21.15.111:0/1414702273 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f9eb4025040 con 0x55afcb5ad090 2024-09-15T22:57:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.510+0000 7f9ec5c31740 10 client.25951 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:57:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.510+0000 7f9ec5c31740 10 client.25951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:57:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.510+0000 7f9ec5c31740 20 client.25951 populate_metadata read hostname 'smithi111' 2024-09-15T22:57:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.510+0000 7f9ec5c31740 1 --2- 172.21.15.111:0/1414702273 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55afcb5c2a80 0x55afcb5c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.510+0000 7f9ec5c31740 4 client.25951 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:57:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.510+0000 7f9ec5c31740 1 -- 172.21.15.111:0/1414702273 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afcb51f830 con 0x55afcb5c2a80 2024-09-15T22:57:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.510+0000 7f9ebb400640 1 --2- 172.21.15.111:0/1414702273 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55afcb5c2a80 0x55afcb5c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.511+0000 7f9ebb400640 1 --2- 172.21.15.111:0/1414702273 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55afcb5c2a80 0x55afcb5c4ea0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdd8e8705fa8988a server_cookie=6c23e979a2dc60ca in_seq=0 out_seq=0 2024-09-15T22:57:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.511+0000 7f9ea5600640 10 client.25951 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:57:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.511+0000 7f9ea5600640 1 -- 172.21.15.111:0/1414702273 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55afcb51f830 con 0x55afcb5c2a80 2024-09-15T22:57:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.511+0000 7f9ea5600640 10 client.25951 handle_command_reply: tid=0 2024-09-15T22:57:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.512+0000 7f9ec5c31740 1 client.25951 shutdown 2024-09-15T22:57:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.512+0000 7f9ec5c31740 1 -- 172.21.15.111:0/1414702273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afcb5ad090 msgr2=0x55afcb5b72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.512+0000 7f9ec5c31740 1 --2- 172.21.15.111:0/1414702273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afcb5ad090 0x55afcb5b72d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009370 tx=0x7f9eb4000f40 comp rx=0 tx=0).stop 2024-09-15T22:57:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.512+0000 7f9ec5c31740 1 -- 172.21.15.111:0/1414702273 shutdown_connections 2024-09-15T22:57:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.512+0000 7f9ec5c31740 1 --2- 172.21.15.111:0/1414702273 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55afcb5c2a80 0x55afcb5c4ea0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.512+0000 7f9ec5c31740 1 --2- 172.21.15.111:0/1414702273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afcb5b2a60 0x55afcb5be770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.512+0000 7f9ec5c31740 1 --2- 172.21.15.111:0/1414702273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afcb5ad9d0 0x55afcb5b7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.512+0000 7f9ea5600640 0 client.25951 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:57:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.512+0000 7f9ec5c31740 1 --2- 172.21.15.111:0/1414702273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afcb5ad090 0x55afcb5b72d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.512+0000 7f9ec5c31740 1 -- 172.21.15.111:0/1414702273 >> 172.21.15.111:0/1414702273 conn(0x55afcb5a8700 msgr2=0x55afcb5a9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.512+0000 7f9ec5c31740 1 -- 172.21.15.111:0/1414702273 shutdown_connections 2024-09-15T22:57:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.512+0000 7f9ec5c31740 1 -- 172.21.15.111:0/1414702273 wait complete. 2024-09-15T22:57:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.512+0000 7f9ec5c31740 20 client.25951 trim_cache size 0 max 16384 2024-09-15T22:57:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.527+0000 7f9ec5c31740 1 Processor -- start 2024-09-15T22:57:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.527+0000 7f9ec5c31740 1 -- start start 2024-09-15T22:57:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.527+0000 7f9ec5c31740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afcb5ad090 0x55afcb5ad470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.527+0000 7f9ec5c31740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afcb5ad9d0 0x55afcb5be690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.527+0000 7f9ec5c31740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afcb5b2a60 0x55afcb5a7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.527+0000 7f9ec5c31740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc1ad9a0 con 0x55afcb5ad9d0 2024-09-15T22:57:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.527+0000 7f9ec5c31740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc1ab270 con 0x55afcb5ad090 2024-09-15T22:57:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.527+0000 7f9ec5c31740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc1a90e0 con 0x55afcb5b2a60 2024-09-15T22:57:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.527+0000 7f9ebb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afcb5ad9d0 0x55afcb5be690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.527+0000 7f9ec0c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afcb5ad090 0x55afcb5ad470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.528+0000 7f9ec0c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afcb5ad090 0x55afcb5ad470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:60946/0 (socket says 172.21.15.111:60946) 2024-09-15T22:57:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.528+0000 7f9ebb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afcb5ad9d0 0x55afcb5be690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42490/0 (socket says 172.21.15.111:42490) 2024-09-15T22:57:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.528+0000 7f9ec0c00640 1 -- 172.21.15.111:0/2623658836 learned_addr learned my addr 172.21.15.111:0/2623658836 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:57:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.528+0000 7f9ec0c00640 1 --2- 172.21.15.111:0/2623658836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afcb5b2a60 0x55afcb5a7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.528+0000 7f9ebb400640 1 -- 172.21.15.111:0/2623658836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afcb5b2a60 msgr2=0x55afcb5a7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.528+0000 7f9ebb400640 1 --2- 172.21.15.111:0/2623658836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afcb5b2a60 0x55afcb5a7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.528+0000 7f9ebb400640 1 -- 172.21.15.111:0/2623658836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afcb5ad090 msgr2=0x55afcb5ad470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.528+0000 7f9ebb400640 1 --2- 172.21.15.111:0/2623658836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afcb5ad090 0x55afcb5ad470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.528+0000 7f9ebb400640 1 -- 172.21.15.111:0/2623658836 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afcb5a8520 con 0x55afcb5ad9d0 2024-09-15T22:57:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.528+0000 7f9ebb400640 1 --2- 172.21.15.111:0/2623658836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afcb5ad9d0 0x55afcb5be690 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55afcb5c56b0 tx=0x7f9ea8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c323144d5d0419 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.528+0000 7f9ea7e00640 1 -- 172.21.15.111:0/2623658836 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8022070 con 0x55afcb5ad9d0 2024-09-15T22:57:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.529+0000 7f9ea7e00640 1 -- 172.21.15.111:0/2623658836 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea80047a0 con 0x55afcb5ad9d0 2024-09-15T22:57:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.529+0000 7f9ea7e00640 1 -- 172.21.15.111:0/2623658836 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea801e9e0 con 0x55afcb5ad9d0 2024-09-15T22:57:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.529+0000 7f9ec5c31740 1 -- 172.21.15.111:0/2623658836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afcb5ad9d0 msgr2=0x55afcb5be690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.529+0000 7f9ec5c31740 1 --2- 172.21.15.111:0/2623658836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afcb5ad9d0 0x55afcb5be690 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x55afcb5c56b0 tx=0x7f9ea8005f90 comp rx=0 tx=0).stop 2024-09-15T22:57:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.529+0000 7f9ec5c31740 1 -- 172.21.15.111:0/2623658836 shutdown_connections 2024-09-15T22:57:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.529+0000 7f9ec5c31740 1 --2- 172.21.15.111:0/2623658836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afcb5b2a60 0x55afcb5a7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.529+0000 7f9ec5c31740 1 --2- 172.21.15.111:0/2623658836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afcb5ad9d0 0x55afcb5be690 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.529+0000 7f9ec5c31740 1 --2- 172.21.15.111:0/2623658836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afcb5ad090 0x55afcb5ad470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.529+0000 7f9ec5c31740 1 -- 172.21.15.111:0/2623658836 >> 172.21.15.111:0/2623658836 conn(0x55afcb5a8700 msgr2=0x55afcb5b7080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.529+0000 7f9ec5c31740 1 -- 172.21.15.111:0/2623658836 shutdown_connections 2024-09-15T22:57:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.530+0000 7f9ec5c31740 1 -- 172.21.15.111:0/2623658836 wait complete. 2024-09-15T22:57:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.530+0000 7f9ec5c31740 1 -- start start 2024-09-15T22:57:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.530+0000 7f9ec5c31740 1 Processor -- start 2024-09-15T22:57:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.530+0000 7f9ec5c31740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afcb5ad090 0x55afcb5bb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.530+0000 7f9ec5c31740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afcb5ad9d0 0x55afcb5bbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.530+0000 7f9ec5c31740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afcb5b2a60 0x55afcb62b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.530+0000 7f9ec5c31740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc1a9730 con 0x55afcb5ad090 2024-09-15T22:57:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.530+0000 7f9ec5c31740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc1aaba0 con 0x55afcb5b2a60 2024-09-15T22:57:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.530+0000 7f9ec5c31740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc1a9e00 con 0x55afcb5ad9d0 2024-09-15T22:57:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.530+0000 7f9ec5c31740 10 client.0 _ll_register_callbacks cb 0x7f9ea80097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:57:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.531+0000 7f9ec0c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afcb5ad090 0x55afcb5bb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.531+0000 7f9ec0c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afcb5ad090 0x55afcb5bb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42506/0 (socket says 172.21.15.111:42506) 2024-09-15T22:57:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.531+0000 7f9ec0c00640 1 -- 172.21.15.111:0/1132048868 learned_addr learned my addr 172.21.15.111:0/1132048868 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:57:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.531+0000 7f9ebb400640 1 --2- 172.21.15.111:0/1132048868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afcb5ad9d0 0x55afcb5bbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.531+0000 7f9ec0c00640 1 --2- 172.21.15.111:0/1132048868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afcb5b2a60 0x55afcb62b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.531+0000 7f9ec0c00640 1 -- 172.21.15.111:0/1132048868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afcb5ad9d0 msgr2=0x55afcb5bbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.531+0000 7f9ec0c00640 1 --2- 172.21.15.111:0/1132048868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afcb5ad9d0 0x55afcb5bbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.531+0000 7f9ec0c00640 1 -- 172.21.15.111:0/1132048868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afcb5b2a60 msgr2=0x55afcb62b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:57:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.531+0000 7f9ec0c00640 1 --2- 172.21.15.111:0/1132048868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afcb5b2a60 0x55afcb62b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.531+0000 7f9ec0c00640 1 -- 172.21.15.111:0/1132048868 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afcb62bcf0 con 0x55afcb5ad090 2024-09-15T22:57:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.531+0000 7f9ec0c00640 1 --2- 172.21.15.111:0/1132048868 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afcb5ad090 0x55afcb5bb800 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4011dd0 tx=0x7f9eb40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5420ed3f2b23085 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:57:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.531+0000 7f9ea5600640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:57:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.531+0000 7f9ea5600640 1 -- 172.21.15.111:0/1132048868 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4028750 con 0x55afcb5ad090 2024-09-15T22:57:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.532+0000 7f9ea5600640 1 -- 172.21.15.111:0/1132048868 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f9eb40288f0 con 0x55afcb5ad090 2024-09-15T22:57:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.532+0000 7f9ec5c31740 1 -- 172.21.15.111:0/1132048868 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ea80047a0 con 0x55afcb5ad090 2024-09-15T22:57:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.532+0000 7f9ea5600640 1 -- 172.21.15.111:0/1132048868 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9eb4047050 con 0x55afcb5ad090 2024-09-15T22:57:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.532+0000 7f9ec5c31740 10 client.15957 fetch_fsmap learned FSMap version 30 2024-09-15T22:57:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.532+0000 7f9ec5c31740 1 -- 172.21.15.111:0/1132048868 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55afcb62d000 con 0x55afcb5ad090 2024-09-15T22:57:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.532+0000 7f9ea5600640 1 -- 172.21.15.111:0/1132048868 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea80047a0 con 0x55afcb5ad090 2024-09-15T22:57:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.532+0000 7f9ea5600640 1 -- 172.21.15.111:0/1132048868 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f9eb400f6a0 con 0x55afcb5ad090 2024-09-15T22:57:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.532+0000 7f9ec5c31740 10 client.15957 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:57:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.532+0000 7f9ec5c31740 10 client.15957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:57:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.532+0000 7f9ec5c31740 20 client.15957 populate_metadata read hostname 'smithi111' 2024-09-15T22:57:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.532+0000 7f9ec5c31740 1 --2- 172.21.15.111:0/1132048868 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55afcb62d9e0 0x55afcb62fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:57:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.532+0000 7f9ec5c31740 4 client.15957 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:57:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.532+0000 7f9ec5c31740 1 -- 172.21.15.111:0/1132048868 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55afcb519b80 con 0x55afcb62d9e0 2024-09-15T22:57:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.533+0000 7f9ebb400640 1 --2- 172.21.15.111:0/1132048868 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55afcb62d9e0 0x55afcb62fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:57:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.533+0000 7f9ebb400640 1 --2- 172.21.15.111:0/1132048868 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55afcb62d9e0 0x55afcb62fe00 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d00e57667ae21d08 server_cookie=c4aa5692838d5811 in_seq=0 out_seq=0 2024-09-15T22:57:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.533+0000 7f9ea5600640 10 client.15957 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:57:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.546+0000 7f9ea5600640 1 -- 172.21.15.111:0/1132048868 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55afcb519b80 con 0x55afcb62d9e0 2024-09-15T22:57:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.546+0000 7f9ea5600640 10 client.15957 handle_command_reply: tid=0 2024-09-15T22:57:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.546+0000 7f9ec5c31740 1 client.15957 shutdown 2024-09-15T22:57:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.546+0000 7f9ec5c31740 1 -- 172.21.15.111:0/1132048868 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afcb5ad090 msgr2=0x55afcb5bb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.546+0000 7f9ec5c31740 1 --2- 172.21.15.111:0/1132048868 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afcb5ad090 0x55afcb5bb800 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4011dd0 tx=0x7f9eb40118d0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.546+0000 7f9ec5c31740 1 -- 172.21.15.111:0/1132048868 shutdown_connections 2024-09-15T22:57:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.546+0000 7f9ec5c31740 1 --2- 172.21.15.111:0/1132048868 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55afcb62d9e0 0x55afcb62fe00 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.546+0000 7f9ec5c31740 1 --2- 172.21.15.111:0/1132048868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55afcb5b2a60 0x55afcb62b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.546+0000 7f9ec5c31740 1 --2- 172.21.15.111:0/1132048868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55afcb5ad9d0 0x55afcb5bbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.546+0000 7f9ec5c31740 1 --2- 172.21.15.111:0/1132048868 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55afcb5ad090 0x55afcb5bb800 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.546+0000 7f9ec5c31740 1 -- 172.21.15.111:0/1132048868 >> 172.21.15.111:0/1132048868 conn(0x55afcb5a8700 msgr2=0x55afcb5a7060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.546+0000 7f9ea5600640 0 client.15957 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:57:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.546+0000 7f9ec5c31740 1 -- 172.21.15.111:0/1132048868 shutdown_connections 2024-09-15T22:57:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.546+0000 7f9ec5c31740 1 -- 172.21.15.111:0/1132048868 wait complete. 2024-09-15T22:57:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.546+0000 7f9ec5c31740 20 client.15957 trim_cache size 0 max 16384 2024-09-15T22:57:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:57:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:57:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:57:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:57:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.547+0000 7f9ec2c00640 1 -- 172.21.15.111:0/1733856093 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9ea00a32f0 msgr2=0x7f9ea00a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.547+0000 7f9ec2c00640 1 --2- 172.21.15.111:0/1733856093 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9ea00a32f0 0x7f9ea00a57b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0bc150 tx=0x7f9eac009260 comp rx=0 tx=0).stop 2024-09-15T22:57:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.547+0000 7f9ec2c00640 1 -- 172.21.15.111:0/1733856093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ebc074bf0 msgr2=0x7f9ebc199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:57:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.547+0000 7f9ec2c00640 1 --2- 172.21.15.111:0/1733856093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ebc074bf0 0x7f9ebc199ff0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009230 tx=0x7f9eac0042f0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.547+0000 7f9ec2c00640 1 -- 172.21.15.111:0/1733856093 shutdown_connections 2024-09-15T22:57:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.547+0000 7f9ec2c00640 1 --2- 172.21.15.111:0/1733856093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ebc1099f0 0x7f9ebc1a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.547+0000 7f9ec2c00640 1 --2- 172.21.15.111:0/1733856093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ebc0761d0 0x7f9ebc19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.547+0000 7f9ec2c00640 1 --2- 172.21.15.111:0/1733856093 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9ea00a32f0 0x7f9ea00a57b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.547+0000 7f9ec2c00640 1 --2- 172.21.15.111:0/1733856093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ebc074bf0 0x7f9ebc199ff0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:57:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.547+0000 7f9ec2c00640 1 -- 172.21.15.111:0/1733856093 >> 172.21.15.111:0/1733856093 conn(0x7f9ebc0fd2e0 msgr2=0x7f9ebc105980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:57:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.547+0000 7f9ec2c00640 1 -- 172.21.15.111:0/1733856093 shutdown_connections 2024-09-15T22:57:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:57:53.547+0000 7f9ec2c00640 1 -- 172.21.15.111:0/1733856093 wait complete. 2024-09-15T22:57:54.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:53 smithi192 ceph-mon[35427]: pgmap v709: 97 pgs: 97 active+clean; 290 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 184 op/s 2024-09-15T22:57:54.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:53 smithi121 ceph-mon[34815]: pgmap v709: 97 pgs: 97 active+clean; 290 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 184 op/s 2024-09-15T22:57:54.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:53 smithi111 ceph-mon[30629]: pgmap v709: 97 pgs: 97 active+clean; 290 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 184 op/s 2024-09-15T22:57:54.322 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1147750 9.84 MB/sec execute 502 sec latency 1372.601 ms 2024-09-15T22:57:55.322 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1150849 9.85 MB/sec execute 503 sec latency 17.035 ms 2024-09-15T22:57:55.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:55 smithi192 ceph-mon[35427]: pgmap v710: 97 pgs: 97 active+clean; 273 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 161 op/s 2024-09-15T22:57:55.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:55 smithi121 ceph-mon[34815]: pgmap v710: 97 pgs: 97 active+clean; 273 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 161 op/s 2024-09-15T22:57:55.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:55 smithi111 ceph-mon[30629]: pgmap v710: 97 pgs: 97 active+clean; 273 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 161 op/s 2024-09-15T22:57:56.322 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1153225 9.85 MB/sec execute 504 sec latency 13.571 ms 2024-09-15T22:57:57.322 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1154709 9.85 MB/sec execute 505 sec latency 370.042 ms 2024-09-15T22:57:58.035 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:57 smithi111 ceph-mon[30629]: pgmap v711: 97 pgs: 97 active+clean; 276 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 186 op/s 2024-09-15T22:57:58.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:57 smithi192 ceph-mon[35427]: pgmap v711: 97 pgs: 97 active+clean; 276 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 186 op/s 2024-09-15T22:57:58.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:57 smithi121 ceph-mon[34815]: pgmap v711: 97 pgs: 97 active+clean; 276 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 186 op/s 2024-09-15T22:57:58.322 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1154709 9.84 MB/sec execute 506 sec latency 1370.083 ms 2024-09-15T22:57:59.322 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1155505 9.82 MB/sec execute 507 sec latency 1840.378 ms 2024-09-15T22:58:00.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:57:59 smithi192 ceph-mon[35427]: pgmap v712: 97 pgs: 97 active+clean; 278 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 189 op/s 2024-09-15T22:58:00.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:57:59 smithi121 ceph-mon[34815]: pgmap v712: 97 pgs: 97 active+clean; 278 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 189 op/s 2024-09-15T22:58:00.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:57:59 smithi111 ceph-mon[30629]: pgmap v712: 97 pgs: 97 active+clean; 278 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 189 op/s 2024-09-15T22:58:00.322 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1158317 9.83 MB/sec execute 508 sec latency 7.763 ms 2024-09-15T22:58:01.323 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1161133 9.84 MB/sec execute 509 sec latency 12.334 ms 2024-09-15T22:58:02.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:01 smithi192 ceph-mon[35427]: pgmap v713: 97 pgs: 97 active+clean; 281 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 192 op/s 2024-09-15T22:58:02.103 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:01 smithi111 ceph-mon[30629]: pgmap v713: 97 pgs: 97 active+clean; 281 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 192 op/s 2024-09-15T22:58:02.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:01 smithi121 ceph-mon[34815]: pgmap v713: 97 pgs: 97 active+clean; 281 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 192 op/s 2024-09-15T22:58:02.322 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1162408 9.84 MB/sec execute 510 sec latency 215.424 ms 2024-09-15T22:58:03.322 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1162411 9.82 MB/sec execute 511 sec latency 912.839 ms 2024-09-15T22:58:04.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:03 smithi192 ceph-mon[35427]: pgmap v714: 97 pgs: 97 active+clean; 278 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 175 op/s 2024-09-15T22:58:04.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:03 smithi121 ceph-mon[34815]: pgmap v714: 97 pgs: 97 active+clean; 278 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 175 op/s 2024-09-15T22:58:04.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:03 smithi111 ceph-mon[30629]: pgmap v714: 97 pgs: 97 active+clean; 278 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 175 op/s 2024-09-15T22:58:04.323 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1163300 9.81 MB/sec execute 512 sec latency 985.216 ms 2024-09-15T22:58:05.323 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1165830 9.81 MB/sec execute 513 sec latency 13.705 ms 2024-09-15T22:58:06.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:05 smithi192 ceph-mon[35427]: pgmap v715: 97 pgs: 97 active+clean; 276 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-09-15T22:58:06.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:05 smithi121 ceph-mon[34815]: pgmap v715: 97 pgs: 97 active+clean; 276 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-09-15T22:58:06.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:05 smithi111 ceph-mon[30629]: pgmap v715: 97 pgs: 97 active+clean; 276 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-09-15T22:58:06.323 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1168907 9.83 MB/sec execute 514 sec latency 11.618 ms 2024-09-15T22:58:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.664+0000 7f2574e00640 1 -- 172.21.15.111:0/1149794067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2570102480 msgr2=0x7f2570104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.664+0000 7f2574e00640 1 --2- 172.21.15.111:0/1149794067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2570102480 0x7f2570104870 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2560009900 tx=0x7f256002a410 comp rx=0 tx=0).stop 2024-09-15T22:58:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.664+0000 7f2574e00640 1 -- 172.21.15.111:0/1149794067 shutdown_connections 2024-09-15T22:58:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.664+0000 7f2574e00640 1 --2- 172.21.15.111:0/1149794067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f25701076e0 0x7f2570109ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.664+0000 7f2574e00640 1 --2- 172.21.15.111:0/1149794067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2570104db0 0x7f25701071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.664+0000 7f2574e00640 1 --2- 172.21.15.111:0/1149794067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2570102480 0x7f2570104870 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.664+0000 7f2574e00640 1 -- 172.21.15.111:0/1149794067 >> 172.21.15.111:0/1149794067 conn(0x7f25700fbfa0 msgr2=0x7f25700fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.664+0000 7f2574e00640 1 -- 172.21.15.111:0/1149794067 shutdown_connections 2024-09-15T22:58:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.664+0000 7f2574e00640 1 -- 172.21.15.111:0/1149794067 wait complete. 2024-09-15T22:58:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.665+0000 7f2574e00640 1 Processor -- start 2024-09-15T22:58:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.665+0000 7f2574e00640 1 -- start start 2024-09-15T22:58:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.665+0000 7f2574e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2570102480 0x7f2570197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.665+0000 7f2574e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2570104db0 0x7f2570198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.665+0000 7f2574e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25701076e0 0x7f257019f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.665+0000 7f2574e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f257010d5f0 con 0x7f25701076e0 2024-09-15T22:58:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.665+0000 7f2574e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f257010d470 con 0x7f2570104db0 2024-09-15T22:58:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.665+0000 7f2574e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f257010d770 con 0x7f2570102480 2024-09-15T22:58:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.665+0000 7f256ea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2570104db0 0x7f2570198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.665+0000 7f256fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25701076e0 0x7f257019f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.665+0000 7f256ea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2570104db0 0x7f2570198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:56240/0 (socket says 172.21.15.111:56240) 2024-09-15T22:58:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.665+0000 7f256ea00640 1 -- 172.21.15.111:0/400842123 learned_addr learned my addr 172.21.15.111:0/400842123 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:58:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.665+0000 7f256f400640 1 --2- 172.21.15.111:0/400842123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2570102480 0x7f2570197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.666+0000 7f256ea00640 1 -- 172.21.15.111:0/400842123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2570102480 msgr2=0x7f2570197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.666+0000 7f256ea00640 1 --2- 172.21.15.111:0/400842123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2570102480 0x7f2570197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.666+0000 7f256ea00640 1 -- 172.21.15.111:0/400842123 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25701076e0 msgr2=0x7f257019f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.666+0000 7f256ea00640 1 --2- 172.21.15.111:0/400842123 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25701076e0 0x7f257019f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.666+0000 7f256ea00640 1 -- 172.21.15.111:0/400842123 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f257019f960 con 0x7f2570104db0 2024-09-15T22:58:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.666+0000 7f256ea00640 1 --2- 172.21.15.111:0/400842123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2570104db0 0x7f2570198290 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f255c00e9f0 tx=0x7f255c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.666+0000 7f2567e00640 1 -- 172.21.15.111:0/400842123 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c00ccc0 con 0x7f2570104db0 2024-09-15T22:58:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.666+0000 7f2574e00640 1 -- 172.21.15.111:0/400842123 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f257019fc50 con 0x7f2570104db0 2024-09-15T22:58:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.666+0000 7f2567e00640 1 -- 172.21.15.111:0/400842123 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f255c005500 con 0x7f2570104db0 2024-09-15T22:58:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.666+0000 7f2567e00640 1 -- 172.21.15.111:0/400842123 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c010690 con 0x7f2570104db0 2024-09-15T22:58:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.667+0000 7f2574e00640 1 -- 172.21.15.111:0/400842123 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25701a01a0 con 0x7f2570104db0 2024-09-15T22:58:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.670+0000 7f257928c740 1 Processor -- start 2024-09-15T22:58:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.670+0000 7f257928c740 1 -- start start 2024-09-15T22:58:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.670+0000 7f257928c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d75fceb670 0x55d75fceba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.670+0000 7f257928c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fcebfb0 0x55d75fcf0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.670+0000 7f257928c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d75fcf1040 0x55d75fcf3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.670+0000 7f257928c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d75fc4c100 con 0x55d75fceb670 2024-09-15T22:58:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.670+0000 7f257928c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d75fbf41b0 con 0x55d75fcebfb0 2024-09-15T22:58:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.670+0000 7f257928c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d75fc480c0 con 0x55d75fcf1040 2024-09-15T22:58:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.670+0000 7f2567e00640 1 -- 172.21.15.111:0/400842123 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f255c0046c0 con 0x7f2570104db0 2024-09-15T22:58:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.670+0000 7f256f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fcebfb0 0x55d75fcf0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.671+0000 7f256f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fcebfb0 0x55d75fcf0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:56252/0 (socket says 172.21.15.111:56252) 2024-09-15T22:58:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.671+0000 7f256f400640 1 -- 172.21.15.111:0/1862139816 learned_addr learned my addr 172.21.15.111:0/1862139816 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:58:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.671+0000 7f256fe00640 1 --2- 172.21.15.111:0/1862139816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d75fceb670 0x55d75fceba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.671+0000 7f256ea00640 1 --2- 172.21.15.111:0/1862139816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d75fcf1040 0x55d75fcf3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.671+0000 7f256f400640 1 -- 172.21.15.111:0/1862139816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d75fcf1040 msgr2=0x55d75fcf3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.671+0000 7f256f400640 1 --2- 172.21.15.111:0/1862139816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d75fcf1040 0x55d75fcf3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.671+0000 7f256f400640 1 -- 172.21.15.111:0/1862139816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d75fceb670 msgr2=0x55d75fceba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.671+0000 7f256f400640 1 --2- 172.21.15.111:0/1862139816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d75fceb670 0x55d75fceba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.671+0000 7f256f400640 1 -- 172.21.15.111:0/1862139816 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d75f95f210 con 0x55d75fcebfb0 2024-09-15T22:58:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.671+0000 7f256f400640 1 --2- 172.21.15.111:0/1862139816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fcebfb0 0x55d75fcf0a70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f25600314e0 tx=0x7f2560005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74b7f4dc5f7b5d35 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.671+0000 7f2564c00640 1 -- 172.21.15.111:0/1862139816 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2560040070 con 0x55d75fcebfb0 2024-09-15T22:58:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.672+0000 7f2564c00640 1 -- 172.21.15.111:0/1862139816 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2560004030 con 0x55d75fcebfb0 2024-09-15T22:58:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.672+0000 7f2564c00640 1 -- 172.21.15.111:0/1862139816 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2560035040 con 0x55d75fcebfb0 2024-09-15T22:58:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.672+0000 7f2567e00640 1 --2- 172.21.15.111:0/400842123 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f25540a3240 0x7f25540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.672+0000 7f257928c740 1 -- 172.21.15.111:0/1862139816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fcebfb0 msgr2=0x55d75fcf0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.672+0000 7f257928c740 1 --2- 172.21.15.111:0/1862139816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fcebfb0 0x55d75fcf0a70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f25600314e0 tx=0x7f2560005de0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.672+0000 7f2567e00640 1 -- 172.21.15.111:0/400842123 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f255c0cfd30 con 0x7f2570104db0 2024-09-15T22:58:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.673+0000 7f257928c740 1 -- 172.21.15.111:0/1862139816 shutdown_connections 2024-09-15T22:58:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.673+0000 7f257928c740 1 --2- 172.21.15.111:0/1862139816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d75fcf1040 0x55d75fcf3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.673+0000 7f257928c740 1 --2- 172.21.15.111:0/1862139816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fcebfb0 0x55d75fcf0a70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.673+0000 7f257928c740 1 --2- 172.21.15.111:0/1862139816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d75fceb670 0x55d75fceba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.673+0000 7f257928c740 1 -- 172.21.15.111:0/1862139816 >> 172.21.15.111:0/1862139816 conn(0x55d75fce28d0 msgr2=0x55d75fce2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.673+0000 7f256fe00640 1 --2- 172.21.15.111:0/400842123 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f25540a3240 0x7f25540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.673+0000 7f257928c740 1 -- 172.21.15.111:0/1862139816 shutdown_connections 2024-09-15T22:58:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.673+0000 7f257928c740 1 -- 172.21.15.111:0/1862139816 wait complete. 2024-09-15T22:58:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.673+0000 7f257928c740 1 -- start start 2024-09-15T22:58:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.673+0000 7f256fe00640 1 --2- 172.21.15.111:0/400842123 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f25540a3240 0x7f25540a5700 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2568005fd0 tx=0x7f2568009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.674+0000 7f257928c740 1 Processor -- start 2024-09-15T22:58:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.674+0000 7f257928c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fceb670 0x55d75fcf5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.674+0000 7f257928c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d75fcebfb0 0x55d75fcf5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.674+0000 7f257928c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d75fcf1040 0x55d75fcfcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.674+0000 7f257928c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d75fcf3fb0 con 0x55d75fcf1040 2024-09-15T22:58:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.674+0000 7f257928c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d75fbd5090 con 0x55d75fceb670 2024-09-15T22:58:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.674+0000 7f257928c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d75fcf4130 con 0x55d75fcebfb0 2024-09-15T22:58:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.674+0000 7f257928c740 10 client.0 _ll_register_callbacks cb 0x55d75fc29f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:58:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.674+0000 7f256f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fceb670 0x55d75fcf5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.674+0000 7f256f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fceb670 0x55d75fcf5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:56256/0 (socket says 172.21.15.111:56256) 2024-09-15T22:58:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.674+0000 7f256f400640 1 -- 172.21.15.111:0/1537929974 learned_addr learned my addr 172.21.15.111:0/1537929974 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:58:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.674+0000 7f256ea00640 1 --2- 172.21.15.111:0/1537929974 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d75fcebfb0 0x55d75fcf5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.675+0000 7f256fe00640 1 --2- 172.21.15.111:0/1537929974 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d75fcf1040 0x55d75fcfcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.675+0000 7f256f400640 1 -- 172.21.15.111:0/1537929974 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d75fcebfb0 msgr2=0x55d75fcf5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.675+0000 7f256f400640 1 --2- 172.21.15.111:0/1537929974 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d75fcebfb0 0x55d75fcf5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.675+0000 7f256f400640 1 -- 172.21.15.111:0/1537929974 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d75fcf1040 msgr2=0x55d75fcfcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.675+0000 7f256f400640 1 --2- 172.21.15.111:0/1537929974 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d75fcf1040 0x55d75fcfcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.675+0000 7f256f400640 1 -- 172.21.15.111:0/1537929974 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d75fa2b620 con 0x55d75fceb670 2024-09-15T22:58:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.675+0000 7f256f400640 1 --2- 172.21.15.111:0/1537929974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fceb670 0x55d75fcf5780 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2560031440 tx=0x7f25600095f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9105b1e355af3690 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.675+0000 7f2559600640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:58:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.675+0000 7f2559600640 1 -- 172.21.15.111:0/1537929974 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256004c020 con 0x55d75fceb670 2024-09-15T22:58:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.675+0000 7f257928c740 1 -- 172.21.15.111:0/1537929974 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2560004030 con 0x55d75fceb670 2024-09-15T22:58:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.675+0000 7f2559600640 1 -- 172.21.15.111:0/1537929974 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f25600316b0 con 0x55d75fceb670 2024-09-15T22:58:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.676+0000 7f2559600640 1 -- 172.21.15.111:0/1537929974 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2560040070 con 0x55d75fceb670 2024-09-15T22:58:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.676+0000 7f257928c740 10 client.25940 fetch_fsmap learned FSMap version 30 2024-09-15T22:58:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.676+0000 7f257928c740 1 -- 172.21.15.111:0/1537929974 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d75fcffaf0 con 0x55d75fceb670 2024-09-15T22:58:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.676+0000 7f2559600640 1 -- 172.21.15.111:0/1537929974 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2560004030 con 0x55d75fceb670 2024-09-15T22:58:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.676+0000 7f2559600640 1 -- 172.21.15.111:0/1537929974 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f2560004780 con 0x55d75fceb670 2024-09-15T22:58:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.676+0000 7f257928c740 10 client.25940 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:58:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.676+0000 7f257928c740 10 client.25940 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:58:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.676+0000 7f257928c740 20 client.25940 populate_metadata read hostname 'smithi111' 2024-09-15T22:58:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.676+0000 7f257928c740 1 --2- 172.21.15.111:0/1537929974 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d75fd00e40 0x55d75fd03230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.676+0000 7f257928c740 4 client.25940 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:58:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.676+0000 7f257928c740 1 -- 172.21.15.111:0/1537929974 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d75fc59830 con 0x55d75fd00e40 2024-09-15T22:58:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.676+0000 7f256ea00640 1 --2- 172.21.15.111:0/1537929974 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d75fd00e40 0x55d75fd03230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.677+0000 7f256ea00640 1 --2- 172.21.15.111:0/1537929974 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d75fd00e40 0x55d75fd03230 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4a66c6fe19b6ede server_cookie=f177a67106c5b97c in_seq=0 out_seq=0 2024-09-15T22:58:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.677+0000 7f2559600640 10 client.25940 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:58:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.678+0000 7f2559600640 1 -- 172.21.15.111:0/1537929974 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d75fc59830 con 0x55d75fd00e40 2024-09-15T22:58:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.678+0000 7f2559600640 10 client.25940 handle_command_reply: tid=0 2024-09-15T22:58:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.678+0000 7f257928c740 1 client.25940 shutdown 2024-09-15T22:58:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.678+0000 7f257928c740 1 -- 172.21.15.111:0/1537929974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fceb670 msgr2=0x55d75fcf5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.678+0000 7f257928c740 1 --2- 172.21.15.111:0/1537929974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fceb670 0x55d75fcf5780 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2560031440 tx=0x7f25600095f0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.678+0000 7f257928c740 1 -- 172.21.15.111:0/1537929974 shutdown_connections 2024-09-15T22:58:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.678+0000 7f257928c740 1 --2- 172.21.15.111:0/1537929974 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d75fd00e40 0x55d75fd03230 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.678+0000 7f257928c740 1 --2- 172.21.15.111:0/1537929974 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d75fcf1040 0x55d75fcfcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.678+0000 7f257928c740 1 --2- 172.21.15.111:0/1537929974 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d75fcebfb0 0x55d75fcf5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.678+0000 7f257928c740 1 --2- 172.21.15.111:0/1537929974 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fceb670 0x55d75fcf5780 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.678+0000 7f2559600640 0 client.25940 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:58:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.678+0000 7f257928c740 1 -- 172.21.15.111:0/1537929974 >> 172.21.15.111:0/1537929974 conn(0x55d75fce28d0 msgr2=0x55d75fce40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.678+0000 7f257928c740 1 -- 172.21.15.111:0/1537929974 shutdown_connections 2024-09-15T22:58:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.678+0000 7f257928c740 1 -- 172.21.15.111:0/1537929974 wait complete. 2024-09-15T22:58:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.678+0000 7f257928c740 20 client.25940 trim_cache size 0 max 16384 2024-09-15T22:58:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.693+0000 7f257928c740 1 Processor -- start 2024-09-15T22:58:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.693+0000 7f257928c740 1 -- start start 2024-09-15T22:58:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.693+0000 7f257928c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fceb670 0x55d75fcf5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.693+0000 7f257928c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d75fcebfb0 0x55d75fcfccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.693+0000 7f257928c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d75fcf1040 0x55d75fce1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.693+0000 7f257928c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f25701a9320 con 0x55d75fcf1040 2024-09-15T22:58:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.693+0000 7f257928c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f25701a6bf0 con 0x55d75fceb670 2024-09-15T22:58:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.693+0000 7f257928c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f25701a4a40 con 0x55d75fcebfb0 2024-09-15T22:58:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.694+0000 7f256f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fceb670 0x55d75fcf5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.694+0000 7f256fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d75fcf1040 0x55d75fce1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.694+0000 7f256f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fceb670 0x55d75fcf5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:56270/0 (socket says 172.21.15.111:56270) 2024-09-15T22:58:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.694+0000 7f256f400640 1 -- 172.21.15.111:0/2589887902 learned_addr learned my addr 172.21.15.111:0/2589887902 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:58:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.694+0000 7f256ea00640 1 --2- 172.21.15.111:0/2589887902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d75fcebfb0 0x55d75fcfccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.694+0000 7f256f400640 1 -- 172.21.15.111:0/2589887902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d75fcebfb0 msgr2=0x55d75fcfccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.694+0000 7f256f400640 1 --2- 172.21.15.111:0/2589887902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d75fcebfb0 0x55d75fcfccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.694+0000 7f256f400640 1 -- 172.21.15.111:0/2589887902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d75fcf1040 msgr2=0x55d75fce1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.694+0000 7f256f400640 1 --2- 172.21.15.111:0/2589887902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d75fcf1040 0x55d75fce1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.694+0000 7f256f400640 1 -- 172.21.15.111:0/2589887902 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d75fce26f0 con 0x55d75fceb670 2024-09-15T22:58:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.694+0000 7f256ea00640 1 --2- 172.21.15.111:0/2589887902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d75fcebfb0 0x55d75fcfccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:58:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.694+0000 7f256f400640 1 --2- 172.21.15.111:0/2589887902 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fceb670 0x55d75fcf5d50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2560033000 tx=0x7f2560005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f51cb5800b30c48f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.695+0000 7f255be00640 1 -- 172.21.15.111:0/2589887902 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256004c070 con 0x55d75fceb670 2024-09-15T22:58:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.695+0000 7f255be00640 1 -- 172.21.15.111:0/2589887902 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2560031b90 con 0x55d75fceb670 2024-09-15T22:58:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.695+0000 7f255be00640 1 -- 172.21.15.111:0/2589887902 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2560035040 con 0x55d75fceb670 2024-09-15T22:58:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.695+0000 7f257928c740 1 -- 172.21.15.111:0/2589887902 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fceb670 msgr2=0x55d75fcf5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.695+0000 7f257928c740 1 --2- 172.21.15.111:0/2589887902 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fceb670 0x55d75fcf5d50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2560033000 tx=0x7f2560005fd0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.695+0000 7f257928c740 1 -- 172.21.15.111:0/2589887902 shutdown_connections 2024-09-15T22:58:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.695+0000 7f257928c740 1 --2- 172.21.15.111:0/2589887902 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d75fcf1040 0x55d75fce1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.695+0000 7f257928c740 1 --2- 172.21.15.111:0/2589887902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d75fcebfb0 0x55d75fcfccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.695+0000 7f257928c740 1 --2- 172.21.15.111:0/2589887902 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fceb670 0x55d75fcf5d50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.695+0000 7f257928c740 1 -- 172.21.15.111:0/2589887902 >> 172.21.15.111:0/2589887902 conn(0x55d75fce28d0 msgr2=0x55d75fce11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.695+0000 7f257928c740 1 -- 172.21.15.111:0/2589887902 shutdown_connections 2024-09-15T22:58:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.695+0000 7f257928c740 1 -- 172.21.15.111:0/2589887902 wait complete. 2024-09-15T22:58:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.696+0000 7f257928c740 1 -- start start 2024-09-15T22:58:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.696+0000 7f257928c740 1 Processor -- start 2024-09-15T22:58:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.696+0000 7f257928c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d75fceb670 0x55d75fcf9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.696+0000 7f257928c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fcebfb0 0x55d75fcfa360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.696+0000 7f257928c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d75fcf1040 0x55d75fcfb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.696+0000 7f257928c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f25701a50b0 con 0x55d75fcf1040 2024-09-15T22:58:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.696+0000 7f257928c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f25701a6520 con 0x55d75fcebfb0 2024-09-15T22:58:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.696+0000 7f257928c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f25701a5780 con 0x55d75fceb670 2024-09-15T22:58:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.696+0000 7f257928c740 10 client.0 _ll_register_callbacks cb 0x7f2568009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:58:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.697+0000 7f256ea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fcebfb0 0x55d75fcfa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.697+0000 7f256ea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fcebfb0 0x55d75fcfa360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:56278/0 (socket says 172.21.15.111:56278) 2024-09-15T22:58:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.697+0000 7f256ea00640 1 -- 172.21.15.111:0/2852008815 learned_addr learned my addr 172.21.15.111:0/2852008815 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:58:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.697+0000 7f256f400640 1 --2- 172.21.15.111:0/2852008815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d75fceb670 0x55d75fcf9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.697+0000 7f256fe00640 1 --2- 172.21.15.111:0/2852008815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d75fcf1040 0x55d75fcfb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.697+0000 7f256ea00640 1 -- 172.21.15.111:0/2852008815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d75fceb670 msgr2=0x55d75fcf9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.697+0000 7f256ea00640 1 --2- 172.21.15.111:0/2852008815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d75fceb670 0x55d75fcf9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.697+0000 7f256ea00640 1 -- 172.21.15.111:0/2852008815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d75fcf1040 msgr2=0x55d75fcfb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.697+0000 7f256ea00640 1 --2- 172.21.15.111:0/2852008815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d75fcf1040 0x55d75fcfb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.697+0000 7f256ea00640 1 -- 172.21.15.111:0/2852008815 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d75fcfb980 con 0x55d75fcebfb0 2024-09-15T22:58:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.697+0000 7f256fe00640 1 --2- 172.21.15.111:0/2852008815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d75fcf1040 0x55d75fcfb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:58:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.697+0000 7f256ea00640 1 --2- 172.21.15.111:0/2852008815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fcebfb0 0x55d75fcfa360 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f255c0cd830 tx=0x7f255c0baf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a4ab5dd14aa6e91 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.697+0000 7f2559600640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:58:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.698+0000 7f2559600640 1 -- 172.21.15.111:0/2852008815 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c09ca50 con 0x55d75fcebfb0 2024-09-15T22:58:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.698+0000 7f257928c740 1 -- 172.21.15.111:0/2852008815 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2560031b90 con 0x55d75fcebfb0 2024-09-15T22:58:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.698+0000 7f2559600640 1 -- 172.21.15.111:0/2852008815 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f255c0b51b0 con 0x55d75fcebfb0 2024-09-15T22:58:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.698+0000 7f2559600640 1 -- 172.21.15.111:0/2852008815 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f255c09c350 con 0x55d75fcebfb0 2024-09-15T22:58:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.698+0000 7f257928c740 10 client.25946 fetch_fsmap learned FSMap version 30 2024-09-15T22:58:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.698+0000 7f257928c740 1 -- 172.21.15.111:0/2852008815 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d75fcfcd00 con 0x55d75fcebfb0 2024-09-15T22:58:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.698+0000 7f2559600640 1 -- 172.21.15.111:0/2852008815 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2560031b90 con 0x55d75fcebfb0 2024-09-15T22:58:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.698+0000 7f2559600640 1 -- 172.21.15.111:0/2852008815 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f255c09b7a0 con 0x55d75fcebfb0 2024-09-15T22:58:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.698+0000 7f257928c740 10 client.25946 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:58:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.698+0000 7f257928c740 10 client.25946 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:58:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.698+0000 7f257928c740 20 client.25946 populate_metadata read hostname 'smithi111' 2024-09-15T22:58:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.698+0000 7f257928c740 1 --2- 172.21.15.111:0/2852008815 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d75fd6be30 0x55d75fd6e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.698+0000 7f257928c740 4 client.25946 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:58:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.698+0000 7f257928c740 1 -- 172.21.15.111:0/2852008815 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d75fc53b80 con 0x55d75fd6be30 2024-09-15T22:58:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.699+0000 7f256f400640 1 --2- 172.21.15.111:0/2852008815 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d75fd6be30 0x55d75fd6e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.699+0000 7f256f400640 1 --2- 172.21.15.111:0/2852008815 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d75fd6be30 0x55d75fd6e220 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b40e443fd90f4062 server_cookie=b76d746988386c56 in_seq=0 out_seq=0 2024-09-15T22:58:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.699+0000 7f2559600640 10 client.25946 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.711+0000 7f2559600640 1 -- 172.21.15.111:0/2852008815 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d75fc53b80 con 0x55d75fd6be30 2024-09-15T22:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.711+0000 7f2559600640 10 client.25946 handle_command_reply: tid=0 2024-09-15T22:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.711+0000 7f257928c740 1 client.25946 shutdown 2024-09-15T22:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.711+0000 7f257928c740 1 -- 172.21.15.111:0/2852008815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fcebfb0 msgr2=0x55d75fcfa360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.711+0000 7f257928c740 1 --2- 172.21.15.111:0/2852008815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fcebfb0 0x55d75fcfa360 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f255c0cd830 tx=0x7f255c0baf90 comp rx=0 tx=0).stop 2024-09-15T22:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.711+0000 7f257928c740 1 -- 172.21.15.111:0/2852008815 shutdown_connections 2024-09-15T22:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.711+0000 7f257928c740 1 --2- 172.21.15.111:0/2852008815 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d75fd6be30 0x55d75fd6e220 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.711+0000 7f257928c740 1 --2- 172.21.15.111:0/2852008815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d75fcf1040 0x55d75fcfb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.711+0000 7f257928c740 1 --2- 172.21.15.111:0/2852008815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d75fcebfb0 0x55d75fcfa360 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.711+0000 7f257928c740 1 --2- 172.21.15.111:0/2852008815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d75fceb670 0x55d75fcf9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.711+0000 7f2559600640 0 client.25946 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.711+0000 7f257928c740 1 -- 172.21.15.111:0/2852008815 >> 172.21.15.111:0/2852008815 conn(0x55d75fce28d0 msgr2=0x55d75fce1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.711+0000 7f257928c740 1 -- 172.21.15.111:0/2852008815 shutdown_connections 2024-09-15T22:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.711+0000 7f257928c740 1 -- 172.21.15.111:0/2852008815 wait complete. 2024-09-15T22:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.712+0000 7f257928c740 20 client.25946 trim_cache size 0 max 16384 2024-09-15T22:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.712+0000 7f2574e00640 1 -- 172.21.15.111:0/400842123 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f25540a3240 msgr2=0x7f25540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.712+0000 7f2574e00640 1 --2- 172.21.15.111:0/400842123 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f25540a3240 0x7f25540a5700 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2568005fd0 tx=0x7f2568009500 comp rx=0 tx=0).stop 2024-09-15T22:58:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.712+0000 7f2574e00640 1 -- 172.21.15.111:0/400842123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2570104db0 msgr2=0x7f2570198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.712+0000 7f2574e00640 1 --2- 172.21.15.111:0/400842123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2570104db0 0x7f2570198290 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f255c00e9f0 tx=0x7f255c00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.712+0000 7f2574e00640 1 -- 172.21.15.111:0/400842123 shutdown_connections 2024-09-15T22:58:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.712+0000 7f2574e00640 1 --2- 172.21.15.111:0/400842123 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25701076e0 0x7f257019f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.713+0000 7f2574e00640 1 --2- 172.21.15.111:0/400842123 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f25540a3240 0x7f25540a5700 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.713+0000 7f2574e00640 1 --2- 172.21.15.111:0/400842123 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2570104db0 0x7f2570198290 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.713+0000 7f2574e00640 1 --2- 172.21.15.111:0/400842123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2570102480 0x7f2570197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.713+0000 7f2574e00640 1 -- 172.21.15.111:0/400842123 >> 172.21.15.111:0/400842123 conn(0x7f25700fbfa0 msgr2=0x7f2570102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.713+0000 7f2574e00640 1 -- 172.21.15.111:0/400842123 shutdown_connections 2024-09-15T22:58:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:06.713+0000 7f2574e00640 1 -- 172.21.15.111:0/400842123 wait complete. 2024-09-15T22:58:07.323 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1171285 9.83 MB/sec execute 515 sec latency 130.351 ms 2024-09-15T22:58:08.038 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:07 smithi111 ceph-mon[30629]: pgmap v716: 97 pgs: 97 active+clean; 278 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 195 op/s 2024-09-15T22:58:08.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:07 smithi192 ceph-mon[35427]: pgmap v716: 97 pgs: 97 active+clean; 278 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 195 op/s 2024-09-15T22:58:08.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:07 smithi121 ceph-mon[34815]: pgmap v716: 97 pgs: 97 active+clean; 278 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 195 op/s 2024-09-15T22:58:08.323 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1171285 9.81 MB/sec execute 516 sec latency 1130.391 ms 2024-09-15T22:58:09.323 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1172430 9.81 MB/sec execute 517 sec latency 1589.580 ms 2024-09-15T22:58:10.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:09 smithi192 ceph-mon[35427]: pgmap v717: 97 pgs: 97 active+clean; 283 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 189 op/s 2024-09-15T22:58:10.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:09 smithi121 ceph-mon[34815]: pgmap v717: 97 pgs: 97 active+clean; 283 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 189 op/s 2024-09-15T22:58:10.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:09 smithi111 ceph-mon[30629]: pgmap v717: 97 pgs: 97 active+clean; 283 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 189 op/s 2024-09-15T22:58:10.323 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1175305 9.82 MB/sec execute 518 sec latency 11.446 ms 2024-09-15T22:58:11.323 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1177551 9.82 MB/sec execute 519 sec latency 15.472 ms 2024-09-15T22:58:12.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:11 smithi192 ceph-mon[35427]: pgmap v718: 97 pgs: 97 active+clean; 283 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-09-15T22:58:12.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:11 smithi121 ceph-mon[34815]: pgmap v718: 97 pgs: 97 active+clean; 283 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-09-15T22:58:12.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:11 smithi111 ceph-mon[30629]: pgmap v718: 97 pgs: 97 active+clean; 283 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-09-15T22:58:12.323 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1180006 9.83 MB/sec execute 520 sec latency 15.638 ms 2024-09-15T22:58:13.324 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1180023 9.81 MB/sec execute 521 sec latency 959.868 ms 2024-09-15T22:58:14.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:13 smithi121 ceph-mon[34815]: pgmap v719: 97 pgs: 97 active+clean; 284 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 151 op/s 2024-09-15T22:58:14.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:13 smithi111 ceph-mon[30629]: pgmap v719: 97 pgs: 97 active+clean; 284 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 151 op/s 2024-09-15T22:58:14.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:13 smithi192 ceph-mon[35427]: pgmap v719: 97 pgs: 97 active+clean; 284 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 151 op/s 2024-09-15T22:58:14.324 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1181782 9.81 MB/sec execute 522 sec latency 1148.892 ms 2024-09-15T22:58:15.324 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1184158 9.82 MB/sec execute 523 sec latency 15.406 ms 2024-09-15T22:58:16.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:15 smithi121 ceph-mon[34815]: pgmap v720: 97 pgs: 97 active+clean; 285 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 188 op/s 2024-09-15T22:58:16.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:15 smithi111 ceph-mon[30629]: pgmap v720: 97 pgs: 97 active+clean; 285 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 188 op/s 2024-09-15T22:58:16.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:15 smithi192 ceph-mon[35427]: pgmap v720: 97 pgs: 97 active+clean; 285 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 188 op/s 2024-09-15T22:58:16.324 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1186888 9.83 MB/sec execute 524 sec latency 9.540 ms 2024-09-15T22:58:17.324 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1189999 9.84 MB/sec execute 525 sec latency 13.392 ms 2024-09-15T22:58:18.018 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:17 smithi111 ceph-mon[30629]: pgmap v721: 97 pgs: 97 active+clean; 285 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 195 op/s 2024-09-15T22:58:18.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:17 smithi121 ceph-mon[34815]: pgmap v721: 97 pgs: 97 active+clean; 285 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 195 op/s 2024-09-15T22:58:18.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:17 smithi192 ceph-mon[35427]: pgmap v721: 97 pgs: 97 active+clean; 285 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 195 op/s 2024-09-15T22:58:18.324 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1190159 9.82 MB/sec execute 526 sec latency 843.230 ms 2024-09-15T22:58:19.324 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1190642 9.81 MB/sec execute 527 sec latency 1270.071 ms 2024-09-15T22:58:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.829+0000 7f1903c00640 1 -- 172.21.15.111:0/1542428016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18fc103d60 msgr2=0x7f18fc104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.829+0000 7f1903c00640 1 --2- 172.21.15.111:0/1542428016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18fc103d60 0x7f18fc104160 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f18f0009900 tx=0x7f18f002a3f0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.830+0000 7f1903c00640 1 -- 172.21.15.111:0/1542428016 shutdown_connections 2024-09-15T22:58:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.830+0000 7f1903c00640 1 --2- 172.21.15.111:0/1542428016 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18fc105910 0x7f18fc10c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.830+0000 7f1903c00640 1 --2- 172.21.15.111:0/1542428016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18fc104f50 0x7f18fc1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.830+0000 7f1903c00640 1 --2- 172.21.15.111:0/1542428016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18fc103d60 0x7f18fc104160 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.830+0000 7f1903c00640 1 -- 172.21.15.111:0/1542428016 >> 172.21.15.111:0/1542428016 conn(0x7f18fc0ff4e0 msgr2=0x7f18fc101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.830+0000 7f1903c00640 1 -- 172.21.15.111:0/1542428016 shutdown_connections 2024-09-15T22:58:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.830+0000 7f1903c00640 1 -- 172.21.15.111:0/1542428016 wait complete. 2024-09-15T22:58:19.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.830+0000 7f1903c00640 1 Processor -- start 2024-09-15T22:58:19.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.831+0000 7f1903c00640 1 -- start start 2024-09-15T22:58:19.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.831+0000 7f1903c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18fc103d60 0x7f18fc199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.831+0000 7f1903c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18fc104f50 0x7f18fc19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.831+0000 7f1903c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18fc105910 0x7f18fc1a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.831+0000 7f1903c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc10fa30 con 0x7f18fc104f50 2024-09-15T22:58:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.831+0000 7f1903c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc10f8b0 con 0x7f18fc105910 2024-09-15T22:58:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.831+0000 7f1903c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc10fbb0 con 0x7f18fc103d60 2024-09-15T22:58:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.831+0000 7f1901200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18fc103d60 0x7f18fc199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.831+0000 7f18fbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18fc104f50 0x7f18fc19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:19.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.831+0000 7f1901200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18fc103d60 0x7f18fc199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:49942/0 (socket says 172.21.15.111:49942) 2024-09-15T22:58:19.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.831+0000 7f1901200640 1 -- 172.21.15.111:0/1033187080 learned_addr learned my addr 172.21.15.111:0/1033187080 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:58:19.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.831+0000 7f1901c00640 1 --2- 172.21.15.111:0/1033187080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18fc105910 0x7f18fc1a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:19.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.832+0000 7f1901200640 1 -- 172.21.15.111:0/1033187080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18fc105910 msgr2=0x7f18fc1a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:19.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.832+0000 7f1901200640 1 --2- 172.21.15.111:0/1033187080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18fc105910 0x7f18fc1a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.832+0000 7f1901200640 1 -- 172.21.15.111:0/1033187080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18fc104f50 msgr2=0x7f18fc19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:19.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.832+0000 7f1901200640 1 --2- 172.21.15.111:0/1033187080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18fc104f50 0x7f18fc19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.832+0000 7f1901200640 1 -- 172.21.15.111:0/1033187080 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18fc1a1be0 con 0x7f18fc103d60 2024-09-15T22:58:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.832+0000 7f1901200640 1 --2- 172.21.15.111:0/1033187080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18fc103d60 0x7f18fc199fc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f18f0031440 tx=0x7f18f0032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.832+0000 7f18f9600640 1 -- 172.21.15.111:0/1033187080 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f003f030 con 0x7f18fc103d60 2024-09-15T22:58:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.832+0000 7f18f9600640 1 -- 172.21.15.111:0/1033187080 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18f0003ce0 con 0x7f18fc103d60 2024-09-15T22:58:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.832+0000 7f1903c00640 1 -- 172.21.15.111:0/1033187080 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18fc1a1e70 con 0x7f18fc103d60 2024-09-15T22:58:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.832+0000 7f18f9600640 1 -- 172.21.15.111:0/1033187080 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f0035070 con 0x7f18fc103d60 2024-09-15T22:58:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.832+0000 7f1903c00640 1 -- 172.21.15.111:0/1033187080 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18fc1a2280 con 0x7f18fc103d60 2024-09-15T22:58:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.835+0000 7f18f9600640 1 -- 172.21.15.111:0/1033187080 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f18f0042030 con 0x7f18fc103d60 2024-09-15T22:58:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.835+0000 7f1906c44740 1 Processor -- start 2024-09-15T22:58:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.835+0000 7f1906c44740 1 -- start start 2024-09-15T22:58:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.836+0000 7f1906c44740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558690d41090 0x558690d41490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.836+0000 7f1906c44740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558690d419d0 0x558690d46490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.836+0000 7f1906c44740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558690d46a60 0x558690d48f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.836+0000 7f1906c44740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x558690ca6100 con 0x558690d419d0 2024-09-15T22:58:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.836+0000 7f1906c44740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558690c4e1b0 con 0x558690d41090 2024-09-15T22:58:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.836+0000 7f1906c44740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558690ca20c0 con 0x558690d46a60 2024-09-15T22:58:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.836+0000 7f1901c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558690d41090 0x558690d41490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.836+0000 7f1901c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558690d41090 0x558690d41490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:48590/0 (socket says 172.21.15.111:48590) 2024-09-15T22:58:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.836+0000 7f1901c00640 1 -- 172.21.15.111:0/3348347113 learned_addr learned my addr 172.21.15.111:0/3348347113 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:58:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.836+0000 7f1901200640 1 --2- 172.21.15.111:0/3348347113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558690d419d0 0x558690d46490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.836+0000 7f18fbe00640 1 --2- 172.21.15.111:0/3348347113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558690d46a60 0x558690d48f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.836+0000 7f1901c00640 1 -- 172.21.15.111:0/3348347113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558690d46a60 msgr2=0x558690d48f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.836+0000 7f1901c00640 1 --2- 172.21.15.111:0/3348347113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558690d46a60 0x558690d48f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.836+0000 7f1901c00640 1 -- 172.21.15.111:0/3348347113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558690d419d0 msgr2=0x558690d46490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.836+0000 7f1901c00640 1 --2- 172.21.15.111:0/3348347113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558690d419d0 0x558690d46490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.836+0000 7f1901c00640 1 -- 172.21.15.111:0/3348347113 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586909b9210 con 0x558690d41090 2024-09-15T22:58:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.837+0000 7f18f9600640 1 --2- 172.21.15.111:0/1033187080 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f18e40a32f0 0x7f18e40a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.837+0000 7f1901c00640 1 --2- 172.21.15.111:0/3348347113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558690d41090 0x558690d41490 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f18f4009fd0 tx=0x7f18f40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd046b6f26448fee server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.837+0000 7f18f9600640 1 -- 172.21.15.111:0/1033187080 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f18f00efa10 con 0x7f18fc103d60 2024-09-15T22:58:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.837+0000 7f18ee000640 1 -- 172.21.15.111:0/3348347113 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4004d20 con 0x558690d41090 2024-09-15T22:58:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.837+0000 7f18ee000640 1 -- 172.21.15.111:0/3348347113 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18f4004050 con 0x558690d41090 2024-09-15T22:58:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.837+0000 7f18ee000640 1 -- 172.21.15.111:0/3348347113 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4005280 con 0x558690d41090 2024-09-15T22:58:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.837+0000 7f1901200640 1 --2- 172.21.15.111:0/1033187080 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f18e40a32f0 0x7f18e40a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.838+0000 7f1901200640 1 --2- 172.21.15.111:0/1033187080 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f18e40a32f0 0x7f18e40a57b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f18f0032fd0 tx=0x7f18f00f54b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.838+0000 7f1906c44740 1 -- 172.21.15.111:0/3348347113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558690d41090 msgr2=0x558690d41490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.838+0000 7f1906c44740 1 --2- 172.21.15.111:0/3348347113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558690d41090 0x558690d41490 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f18f4009fd0 tx=0x7f18f40118d0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.838+0000 7f1906c44740 1 -- 172.21.15.111:0/3348347113 shutdown_connections 2024-09-15T22:58:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.838+0000 7f1906c44740 1 --2- 172.21.15.111:0/3348347113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558690d46a60 0x558690d48f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.838+0000 7f1906c44740 1 --2- 172.21.15.111:0/3348347113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558690d419d0 0x558690d46490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.838+0000 7f1906c44740 1 --2- 172.21.15.111:0/3348347113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558690d41090 0x558690d41490 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.838+0000 7f1906c44740 1 -- 172.21.15.111:0/3348347113 >> 172.21.15.111:0/3348347113 conn(0x558690d3c700 msgr2=0x558690d3cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.838+0000 7f1906c44740 1 -- 172.21.15.111:0/3348347113 shutdown_connections 2024-09-15T22:58:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.838+0000 7f1906c44740 1 -- 172.21.15.111:0/3348347113 wait complete. 2024-09-15T22:58:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.838+0000 7f1906c44740 1 -- start start 2024-09-15T22:58:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.839+0000 7f1906c44740 1 Processor -- start 2024-09-15T22:58:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.839+0000 7f1906c44740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558690d41090 0x558690d4b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.839+0000 7f1906c44740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558690d419d0 0x558690d4b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.839+0000 7f1906c44740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558690d46a60 0x558690d527f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.839+0000 7f1906c44740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x558690d49a30 con 0x558690d46a60 2024-09-15T22:58:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.839+0000 7f1906c44740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558690c2f090 con 0x558690d41090 2024-09-15T22:58:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.839+0000 7f1906c44740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558690d49bb0 con 0x558690d419d0 2024-09-15T22:58:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.839+0000 7f1906c44740 10 client.0 _ll_register_callbacks cb 0x558690c83f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:58:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.839+0000 7f1901c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558690d41090 0x558690d4b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.839+0000 7f1901c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558690d46a60 0x558690d527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.839+0000 7f18fbe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558690d419d0 0x558690d4b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.839+0000 7f18fbe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558690d419d0 0x558690d4b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:49966/0 (socket says 172.21.15.111:49966) 2024-09-15T22:58:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.839+0000 7f18fbe00640 1 -- 172.21.15.111:0/1204093706 learned_addr learned my addr 172.21.15.111:0/1204093706 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:58:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.839+0000 7f1901c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558690d46a60 0x558690d527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43550/0 (socket says 172.21.15.111:43550) 2024-09-15T22:58:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.840+0000 7f18fbe00640 1 -- 172.21.15.111:0/1204093706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558690d41090 msgr2=0x558690d4b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.840+0000 7f18fbe00640 1 --2- 172.21.15.111:0/1204093706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558690d41090 0x558690d4b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.840+0000 7f18fbe00640 1 -- 172.21.15.111:0/1204093706 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558690d46a60 msgr2=0x558690d527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.840+0000 7f18fbe00640 1 --2- 172.21.15.111:0/1204093706 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558690d46a60 0x558690d527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.840+0000 7f18fbe00640 1 -- 172.21.15.111:0/1204093706 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558690a85620 con 0x558690d419d0 2024-09-15T22:58:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.840+0000 7f18fbe00640 1 --2- 172.21.15.111:0/1204093706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558690d419d0 0x558690d4b7b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f18e8012700 tx=0x7f18e8012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d07e3a20dc3f16e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.840+0000 7f18e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:58:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.840+0000 7f18e2a00640 1 -- 172.21.15.111:0/1204093706 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e800f8c0 con 0x558690d419d0 2024-09-15T22:58:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.840+0000 7f1906c44740 1 -- 172.21.15.111:0/1204093706 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18f4004050 con 0x558690d419d0 2024-09-15T22:58:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.840+0000 7f18e2a00640 1 -- 172.21.15.111:0/1204093706 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f18e800fa60 con 0x558690d419d0 2024-09-15T22:58:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.840+0000 7f18e2a00640 1 -- 172.21.15.111:0/1204093706 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f18e800fca0 con 0x558690d419d0 2024-09-15T22:58:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.840+0000 7f1906c44740 10 client.25987 fetch_fsmap learned FSMap version 30 2024-09-15T22:58:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.840+0000 7f1906c44740 1 -- 172.21.15.111:0/1204093706 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558690d55700 con 0x558690d419d0 2024-09-15T22:58:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.841+0000 7f18e2a00640 1 -- 172.21.15.111:0/1204093706 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18f4004050 con 0x558690d419d0 2024-09-15T22:58:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.841+0000 7f18e2a00640 1 -- 172.21.15.111:0/1204093706 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f18e8010450 con 0x558690d419d0 2024-09-15T22:58:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.841+0000 7f1906c44740 10 client.25987 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:58:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.841+0000 7f1906c44740 10 client.25987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:58:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.841+0000 7f1906c44740 20 client.25987 populate_metadata read hostname 'smithi111' 2024-09-15T22:58:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.841+0000 7f1906c44740 1 --2- 172.21.15.111:0/1204093706 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x558690d56e20 0x558690d59230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.841+0000 7f1906c44740 4 client.25987 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:58:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.841+0000 7f1906c44740 1 -- 172.21.15.111:0/1204093706 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558690cb3830 con 0x558690d56e20 2024-09-15T22:58:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.841+0000 7f1901c00640 1 --2- 172.21.15.111:0/1204093706 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x558690d56e20 0x558690d59230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.842+0000 7f18e2a00640 10 client.25987 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:58:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.842+0000 7f1901c00640 1 --2- 172.21.15.111:0/1204093706 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x558690d56e20 0x558690d59230 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=209ae32f2f075d54 server_cookie=e47e4ace2cccc60f in_seq=0 out_seq=0 2024-09-15T22:58:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.842+0000 7f18e2a00640 1 -- 172.21.15.111:0/1204093706 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558690cb3830 con 0x558690d56e20 2024-09-15T22:58:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.842+0000 7f18e2a00640 10 client.25987 handle_command_reply: tid=0 2024-09-15T22:58:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.843+0000 7f1906c44740 1 client.25987 shutdown 2024-09-15T22:58:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.843+0000 7f1906c44740 1 -- 172.21.15.111:0/1204093706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558690d419d0 msgr2=0x558690d4b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.843+0000 7f1906c44740 1 --2- 172.21.15.111:0/1204093706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558690d419d0 0x558690d4b7b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f18e8012700 tx=0x7f18e8012bc0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.843+0000 7f1906c44740 1 -- 172.21.15.111:0/1204093706 shutdown_connections 2024-09-15T22:58:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.843+0000 7f1906c44740 1 --2- 172.21.15.111:0/1204093706 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x558690d56e20 0x558690d59230 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.843+0000 7f1906c44740 1 --2- 172.21.15.111:0/1204093706 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558690d46a60 0x558690d527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.843+0000 7f1906c44740 1 --2- 172.21.15.111:0/1204093706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558690d419d0 0x558690d4b7b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.843+0000 7f1906c44740 1 --2- 172.21.15.111:0/1204093706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558690d41090 0x558690d4b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.843+0000 7f18e2a00640 0 client.25987 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:58:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.843+0000 7f1906c44740 1 -- 172.21.15.111:0/1204093706 >> 172.21.15.111:0/1204093706 conn(0x558690d3c700 msgr2=0x558690d3df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.843+0000 7f1906c44740 1 -- 172.21.15.111:0/1204093706 shutdown_connections 2024-09-15T22:58:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.843+0000 7f1906c44740 1 -- 172.21.15.111:0/1204093706 wait complete. 2024-09-15T22:58:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.843+0000 7f1906c44740 20 client.25987 trim_cache size 0 max 16384 2024-09-15T22:58:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.858+0000 7f1906c44740 1 Processor -- start 2024-09-15T22:58:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.859+0000 7f1906c44740 1 -- start start 2024-09-15T22:58:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.859+0000 7f1906c44740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558690d41090 0x558690d59210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:19.860 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.859+0000 7f1906c44740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558690d419d0 0x558690d527b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.859+0000 7f1906c44740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558690d46a60 0x558690d3d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.859+0000 7f1906c44740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc1ad870 con 0x558690d41090 2024-09-15T22:58:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.859+0000 7f1906c44740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc1ab140 con 0x558690d419d0 2024-09-15T22:58:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.859+0000 7f1906c44740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc1a8f90 con 0x558690d46a60 2024-09-15T22:58:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.859+0000 7f1901c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558690d41090 0x558690d59210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.859+0000 7f18fbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558690d419d0 0x558690d527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.859+0000 7f1901c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558690d41090 0x558690d59210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43552/0 (socket says 172.21.15.111:43552) 2024-09-15T22:58:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.859+0000 7f1901c00640 1 -- 172.21.15.111:0/2879908245 learned_addr learned my addr 172.21.15.111:0/2879908245 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:58:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.859+0000 7f1901c00640 1 --2- 172.21.15.111:0/2879908245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558690d46a60 0x558690d3d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.859+0000 7f18fbe00640 1 -- 172.21.15.111:0/2879908245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558690d46a60 msgr2=0x558690d3d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.859+0000 7f18fbe00640 1 --2- 172.21.15.111:0/2879908245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558690d46a60 0x558690d3d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.859+0000 7f18fbe00640 1 -- 172.21.15.111:0/2879908245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558690d41090 msgr2=0x558690d59210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.859+0000 7f18fbe00640 1 --2- 172.21.15.111:0/2879908245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558690d41090 0x558690d59210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.859+0000 7f18fbe00640 1 -- 172.21.15.111:0/2879908245 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558690d3d390 con 0x558690d419d0 2024-09-15T22:58:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.860+0000 7f18fbe00640 1 --2- 172.21.15.111:0/2879908245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558690d419d0 0x558690d527b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f18e8010a60 tx=0x7f18e8009f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c189839b5020d3d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.860+0000 7f18ed600640 1 -- 172.21.15.111:0/2879908245 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e802b070 con 0x558690d419d0 2024-09-15T22:58:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.860+0000 7f18ed600640 1 -- 172.21.15.111:0/2879908245 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18e8010430 con 0x558690d419d0 2024-09-15T22:58:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.860+0000 7f18ed600640 1 -- 172.21.15.111:0/2879908245 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e80053c0 con 0x558690d419d0 2024-09-15T22:58:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.861+0000 7f1906c44740 1 -- 172.21.15.111:0/2879908245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558690d419d0 msgr2=0x558690d527b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.861+0000 7f1906c44740 1 --2- 172.21.15.111:0/2879908245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558690d419d0 0x558690d527b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f18e8010a60 tx=0x7f18e8009f20 comp rx=0 tx=0).stop 2024-09-15T22:58:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.861+0000 7f1906c44740 1 -- 172.21.15.111:0/2879908245 shutdown_connections 2024-09-15T22:58:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.861+0000 7f1906c44740 1 --2- 172.21.15.111:0/2879908245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558690d46a60 0x558690d3d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.861+0000 7f1906c44740 1 --2- 172.21.15.111:0/2879908245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558690d419d0 0x558690d527b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.861+0000 7f1906c44740 1 --2- 172.21.15.111:0/2879908245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558690d41090 0x558690d59210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.861+0000 7f1906c44740 1 -- 172.21.15.111:0/2879908245 >> 172.21.15.111:0/2879908245 conn(0x558690d3c700 msgr2=0x558690d42c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.861+0000 7f1906c44740 1 -- 172.21.15.111:0/2879908245 shutdown_connections 2024-09-15T22:58:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.861+0000 7f1906c44740 1 -- 172.21.15.111:0/2879908245 wait complete. 2024-09-15T22:58:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.861+0000 7f1906c44740 1 -- start start 2024-09-15T22:58:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.861+0000 7f1906c44740 1 Processor -- start 2024-09-15T22:58:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.862+0000 7f1906c44740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558690d41090 0x558690d4fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.862+0000 7f1906c44740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558690d419d0 0x558690d502d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.862+0000 7f1906c44740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558690d46a60 0x558690d51290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.862+0000 7f1906c44740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc1a9600 con 0x558690d46a60 2024-09-15T22:58:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.862+0000 7f1906c44740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc1aaa70 con 0x558690d41090 2024-09-15T22:58:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.862+0000 7f1906c44740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc1a9cd0 con 0x558690d419d0 2024-09-15T22:58:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.862+0000 7f1906c44740 10 client.0 _ll_register_callbacks cb 0x7f18e8009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:58:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.862+0000 7f1901c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558690d41090 0x558690d4fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.862+0000 7f18fbe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558690d419d0 0x558690d502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.862+0000 7f1901c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558690d41090 0x558690d4fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:48598/0 (socket says 172.21.15.111:48598) 2024-09-15T22:58:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.862+0000 7f18fbe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558690d419d0 0x558690d502d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:49994/0 (socket says 172.21.15.111:49994) 2024-09-15T22:58:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.862+0000 7f1901c00640 1 -- 172.21.15.111:0/3960275637 learned_addr learned my addr 172.21.15.111:0/3960275637 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:58:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.862+0000 7f1901c00640 1 --2- 172.21.15.111:0/3960275637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558690d46a60 0x558690d51290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:19.867 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.862+0000 7f18fbe00640 1 -- 172.21.15.111:0/3960275637 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558690d41090 msgr2=0x558690d4fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.862+0000 7f18fbe00640 1 --2- 172.21.15.111:0/3960275637 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558690d41090 0x558690d4fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.862+0000 7f18fbe00640 1 -- 172.21.15.111:0/3960275637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558690d46a60 msgr2=0x558690d51290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.862+0000 7f18fbe00640 1 --2- 172.21.15.111:0/3960275637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558690d46a60 0x558690d51290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.862+0000 7f18fbe00640 1 -- 172.21.15.111:0/3960275637 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558690d518a0 con 0x558690d419d0 2024-09-15T22:58:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.862+0000 7f1901c00640 1 --2- 172.21.15.111:0/3960275637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558690d46a60 0x558690d51290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:58:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.863+0000 7f18fbe00640 1 --2- 172.21.15.111:0/3960275637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558690d419d0 0x558690d502d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x558690d490a0 tx=0x7f18e8012050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=625ea12adbd502ee server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.863+0000 7f18e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:58:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.863+0000 7f18e2a00640 1 -- 172.21.15.111:0/3960275637 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e802b070 con 0x558690d419d0 2024-09-15T22:58:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.863+0000 7f1906c44740 1 -- 172.21.15.111:0/3960275637 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18e8010430 con 0x558690d419d0 2024-09-15T22:58:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.863+0000 7f18e2a00640 1 -- 172.21.15.111:0/3960275637 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f18e800f9a0 con 0x558690d419d0 2024-09-15T22:58:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.863+0000 7f18e2a00640 1 -- 172.21.15.111:0/3960275637 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f18e8005120 con 0x558690d419d0 2024-09-15T22:58:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.863+0000 7f1906c44740 10 client.25993 fetch_fsmap learned FSMap version 30 2024-09-15T22:58:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.863+0000 7f1906c44740 1 -- 172.21.15.111:0/3960275637 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558690dc1b10 con 0x558690d419d0 2024-09-15T22:58:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.863+0000 7f18e2a00640 1 -- 172.21.15.111:0/3960275637 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18e8010430 con 0x558690d419d0 2024-09-15T22:58:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.863+0000 7f18e2a00640 1 -- 172.21.15.111:0/3960275637 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f18e801c470 con 0x558690d419d0 2024-09-15T22:58:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.864+0000 7f1906c44740 10 client.25993 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:58:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.864+0000 7f1906c44740 10 client.25993 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:58:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.864+0000 7f1906c44740 20 client.25993 populate_metadata read hostname 'smithi111' 2024-09-15T22:58:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.864+0000 7f1906c44740 1 --2- 172.21.15.111:0/3960275637 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x558690dc22a0 0x558690dc46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.864+0000 7f1906c44740 4 client.25993 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:58:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.864+0000 7f1906c44740 1 -- 172.21.15.111:0/3960275637 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558690d37d80 con 0x558690dc22a0 2024-09-15T22:58:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.864+0000 7f1901c00640 1 --2- 172.21.15.111:0/3960275637 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x558690dc22a0 0x558690dc46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.864+0000 7f1901c00640 1 --2- 172.21.15.111:0/3960275637 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x558690dc22a0 0x558690dc46c0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=617887af5a4771cf server_cookie=832aa07f3a75b3cc in_seq=0 out_seq=0 2024-09-15T22:58:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.864+0000 7f18e2a00640 10 client.25993 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:58:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.875+0000 7f18e2a00640 1 -- 172.21.15.111:0/3960275637 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558690d37d80 con 0x558690dc22a0 2024-09-15T22:58:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.875+0000 7f18e2a00640 10 client.25993 handle_command_reply: tid=0 2024-09-15T22:58:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.875+0000 7f1906c44740 1 client.25993 shutdown 2024-09-15T22:58:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.875+0000 7f1906c44740 1 -- 172.21.15.111:0/3960275637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558690d419d0 msgr2=0x558690d502d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.875+0000 7f1906c44740 1 --2- 172.21.15.111:0/3960275637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558690d419d0 0x558690d502d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x558690d490a0 tx=0x7f18e8012050 comp rx=0 tx=0).stop 2024-09-15T22:58:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.876+0000 7f1906c44740 1 -- 172.21.15.111:0/3960275637 shutdown_connections 2024-09-15T22:58:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.876+0000 7f1906c44740 1 --2- 172.21.15.111:0/3960275637 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x558690dc22a0 0x558690dc46c0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.876+0000 7f1906c44740 1 --2- 172.21.15.111:0/3960275637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558690d46a60 0x558690d51290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.876+0000 7f18e2a00640 0 client.25993 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:58:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.876+0000 7f1906c44740 1 --2- 172.21.15.111:0/3960275637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558690d419d0 0x558690d502d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.876+0000 7f1906c44740 1 --2- 172.21.15.111:0/3960275637 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558690d41090 0x558690d4fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.876+0000 7f1906c44740 1 -- 172.21.15.111:0/3960275637 >> 172.21.15.111:0/3960275637 conn(0x558690d3c700 msgr2=0x558690d3b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.876+0000 7f1906c44740 1 -- 172.21.15.111:0/3960275637 shutdown_connections 2024-09-15T22:58:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.876+0000 7f1906c44740 1 -- 172.21.15.111:0/3960275637 wait complete. 2024-09-15T22:58:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.876+0000 7f1906c44740 20 client.25993 trim_cache size 0 max 16384 2024-09-15T22:58:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:58:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:58:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:58:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:58:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.877+0000 7f1903c00640 1 -- 172.21.15.111:0/1033187080 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f18e40a32f0 msgr2=0x7f18e40a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.877+0000 7f1903c00640 1 --2- 172.21.15.111:0/1033187080 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f18e40a32f0 0x7f18e40a57b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f18f0032fd0 tx=0x7f18f00f54b0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.877+0000 7f1903c00640 1 -- 172.21.15.111:0/1033187080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18fc103d60 msgr2=0x7f18fc199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.877+0000 7f1903c00640 1 --2- 172.21.15.111:0/1033187080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18fc103d60 0x7f18fc199fc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f18f0031440 tx=0x7f18f0032ce0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.877+0000 7f1903c00640 1 -- 172.21.15.111:0/1033187080 shutdown_connections 2024-09-15T22:58:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.877+0000 7f1903c00640 1 --2- 172.21.15.111:0/1033187080 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f18e40a32f0 0x7f18e40a57b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.877+0000 7f1903c00640 1 --2- 172.21.15.111:0/1033187080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18fc105910 0x7f18fc1a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.877+0000 7f1903c00640 1 --2- 172.21.15.111:0/1033187080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f18fc104f50 0x7f18fc19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.877+0000 7f1903c00640 1 --2- 172.21.15.111:0/1033187080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18fc103d60 0x7f18fc199fc0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.877+0000 7f1903c00640 1 -- 172.21.15.111:0/1033187080 >> 172.21.15.111:0/1033187080 conn(0x7f18fc0ff4e0 msgr2=0x7f18fc108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.877+0000 7f1903c00640 1 -- 172.21.15.111:0/1033187080 shutdown_connections 2024-09-15T22:58:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:19.877+0000 7f1903c00640 1 -- 172.21.15.111:0/1033187080 wait complete. 2024-09-15T22:58:20.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:19 smithi121 ceph-mon[34815]: pgmap v722: 97 pgs: 97 active+clean; 287 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 215 op/s 2024-09-15T22:58:20.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:19 smithi111 ceph-mon[30629]: pgmap v722: 97 pgs: 97 active+clean; 287 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 215 op/s 2024-09-15T22:58:20.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:19 smithi192 ceph-mon[35427]: pgmap v722: 97 pgs: 97 active+clean; 287 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 215 op/s 2024-09-15T22:58:20.324 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1193693 9.82 MB/sec execute 528 sec latency 9.252 ms 2024-09-15T22:58:21.324 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1196379 9.83 MB/sec execute 529 sec latency 10.933 ms 2024-09-15T22:58:22.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:21 smithi121 ceph-mon[34815]: pgmap v723: 97 pgs: 97 active+clean; 289 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 202 op/s 2024-09-15T22:58:22.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:21 smithi111 ceph-mon[30629]: pgmap v723: 97 pgs: 97 active+clean; 289 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 202 op/s 2024-09-15T22:58:22.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:21 smithi192 ceph-mon[35427]: pgmap v723: 97 pgs: 97 active+clean; 289 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 202 op/s 2024-09-15T22:58:22.324 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1198568 9.83 MB/sec execute 530 sec latency 11.861 ms 2024-09-15T22:58:23.324 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1199427 9.82 MB/sec execute 531 sec latency 448.738 ms 2024-09-15T22:58:24.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:23 smithi121 ceph-mon[34815]: pgmap v724: 97 pgs: 97 active+clean; 289 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 201 op/s 2024-09-15T22:58:24.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:23 smithi111 ceph-mon[30629]: pgmap v724: 97 pgs: 97 active+clean; 289 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 201 op/s 2024-09-15T22:58:24.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:23 smithi192 ceph-mon[35427]: pgmap v724: 97 pgs: 97 active+clean; 289 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 201 op/s 2024-09-15T22:58:24.324 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1200842 9.82 MB/sec execute 532 sec latency 862.643 ms 2024-09-15T22:58:25.324 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1203682 9.83 MB/sec execute 533 sec latency 13.421 ms 2024-09-15T22:58:26.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:25 smithi121 ceph-mon[34815]: pgmap v725: 97 pgs: 97 active+clean; 289 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 195 op/s 2024-09-15T22:58:26.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:25 smithi111 ceph-mon[30629]: pgmap v725: 97 pgs: 97 active+clean; 289 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 195 op/s 2024-09-15T22:58:26.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:25 smithi192 ceph-mon[35427]: pgmap v725: 97 pgs: 97 active+clean; 289 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 195 op/s 2024-09-15T22:58:26.325 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1206083 9.83 MB/sec execute 534 sec latency 15.400 ms 2024-09-15T22:58:27.325 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1208437 9.84 MB/sec execute 535 sec latency 13.254 ms 2024-09-15T22:58:27.925 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:27 smithi111 ceph-mon[30629]: pgmap v726: 97 pgs: 97 active+clean; 292 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 191 op/s 2024-09-15T22:58:28.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:27 smithi121 ceph-mon[34815]: pgmap v726: 97 pgs: 97 active+clean; 292 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 191 op/s 2024-09-15T22:58:28.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:27 smithi192 ceph-mon[35427]: pgmap v726: 97 pgs: 97 active+clean; 292 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 191 op/s 2024-09-15T22:58:28.327 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1209909 9.83 MB/sec execute 536 sec latency 647.244 ms 2024-09-15T22:58:29.327 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1209917 9.81 MB/sec execute 537 sec latency 893.223 ms 2024-09-15T22:58:30.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:29 smithi121 ceph-mon[34815]: pgmap v727: 97 pgs: 97 active+clean; 285 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 227 op/s 2024-09-15T22:58:30.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:29 smithi111 ceph-mon[30629]: pgmap v727: 97 pgs: 97 active+clean; 285 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 227 op/s 2024-09-15T22:58:30.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:29 smithi192 ceph-mon[35427]: pgmap v727: 97 pgs: 97 active+clean; 285 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 227 op/s 2024-09-15T22:58:30.327 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1211936 9.82 MB/sec execute 538 sec latency 911.547 ms 2024-09-15T22:58:31.327 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1215037 9.84 MB/sec execute 539 sec latency 7.019 ms 2024-09-15T22:58:32.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:31 smithi121 ceph-mon[34815]: pgmap v728: 97 pgs: 97 active+clean; 277 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 184 op/s 2024-09-15T22:58:32.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:31 smithi111 ceph-mon[30629]: pgmap v728: 97 pgs: 97 active+clean; 277 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 184 op/s 2024-09-15T22:58:32.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:31 smithi192 ceph-mon[35427]: pgmap v728: 97 pgs: 97 active+clean; 277 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 184 op/s 2024-09-15T22:58:32.327 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1217857 9.84 MB/sec execute 540 sec latency 12.555 ms 2024-09-15T22:58:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.990+0000 7fcd9d600640 1 -- 172.21.15.111:0/1695920725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd981025d0 msgr2=0x7fcd981049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.990+0000 7fcd9d600640 1 --2- 172.21.15.111:0/1695920725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd981025d0 0x7fcd981049c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcd88009900 tx=0x7fcd8802a410 comp rx=0 tx=0).stop 2024-09-15T22:58:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.991+0000 7fcd9d600640 1 -- 172.21.15.111:0/1695920725 shutdown_connections 2024-09-15T22:58:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.991+0000 7fcd9d600640 1 --2- 172.21.15.111:0/1695920725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd98107830 0x7fcd98109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.991+0000 7fcd9d600640 1 --2- 172.21.15.111:0/1695920725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd98104f00 0x7fcd981072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.991+0000 7fcd9d600640 1 --2- 172.21.15.111:0/1695920725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd981025d0 0x7fcd981049c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.991+0000 7fcd9d600640 1 -- 172.21.15.111:0/1695920725 >> 172.21.15.111:0/1695920725 conn(0x7fcd980fc1b0 msgr2=0x7fcd980fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.991+0000 7fcd9d600640 1 -- 172.21.15.111:0/1695920725 shutdown_connections 2024-09-15T22:58:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.991+0000 7fcd9d600640 1 -- 172.21.15.111:0/1695920725 wait complete. 2024-09-15T22:58:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.992+0000 7fcd9d600640 1 Processor -- start 2024-09-15T22:58:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.992+0000 7fcd9d600640 1 -- start start 2024-09-15T22:58:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.992+0000 7fcd9d600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd981025d0 0x7fcd98101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.992+0000 7fcd9d600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd98104f00 0x7fcd98100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.992+0000 7fcd9d600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd98107830 0x7fcd98100ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.992+0000 7fcd9d600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9810d8e0 con 0x7fcd98104f00 2024-09-15T22:58:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.992+0000 7fcd9d600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9810d760 con 0x7fcd98107830 2024-09-15T22:58:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.992+0000 7fcd9d600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9810da60 con 0x7fcd981025d0 2024-09-15T22:58:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.992+0000 7fcd96000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd98104f00 0x7fcd98100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.992+0000 7fcd96000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd98104f00 0x7fcd98100580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39214/0 (socket says 172.21.15.111:39214) 2024-09-15T22:58:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.993+0000 7fcd96000640 1 -- 172.21.15.111:0/311605041 learned_addr learned my addr 172.21.15.111:0/311605041 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:58:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.993+0000 7fcd96a00640 1 --2- 172.21.15.111:0/311605041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd981025d0 0x7fcd98101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.993+0000 7fcd96000640 1 -- 172.21.15.111:0/311605041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd981025d0 msgr2=0x7fcd98101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.993+0000 7fcd96000640 1 --2- 172.21.15.111:0/311605041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd981025d0 0x7fcd98101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.993+0000 7fcd96000640 1 -- 172.21.15.111:0/311605041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd98107830 msgr2=0x7fcd98100ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:58:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.993+0000 7fcd96000640 1 --2- 172.21.15.111:0/311605041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd98107830 0x7fcd98100ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.993+0000 7fcd96000640 1 -- 172.21.15.111:0/311605041 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd98101350 con 0x7fcd98104f00 2024-09-15T22:58:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.993+0000 7fcd96000640 1 --2- 172.21.15.111:0/311605041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd98104f00 0x7fcd98100580 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400e9a0 tx=0x7fcd8400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.994+0000 7fcd8f400640 1 -- 172.21.15.111:0/311605041 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8400cd80 con 0x7fcd98104f00 2024-09-15T22:58:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.994+0000 7fcd8f400640 1 -- 172.21.15.111:0/311605041 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd840035f0 con 0x7fcd98104f00 2024-09-15T22:58:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.994+0000 7fcd9d600640 1 -- 172.21.15.111:0/311605041 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd981a1fc0 con 0x7fcd98104f00 2024-09-15T22:58:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.994+0000 7fcd8f400640 1 -- 172.21.15.111:0/311605041 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd84010670 con 0x7fcd98104f00 2024-09-15T22:58:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.994+0000 7fcd9d600640 1 -- 172.21.15.111:0/311605041 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd981a2500 con 0x7fcd98104f00 2024-09-15T22:58:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.998+0000 7fcda047d740 1 Processor -- start 2024-09-15T22:58:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.998+0000 7fcda047d740 1 -- start start 2024-09-15T22:58:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.999+0000 7fcda047d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e5505c090 0x563e5505c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.999+0000 7fcda047d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e5505c9d0 0x563e55061490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.999+0000 7fcda047d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563e55061a60 0x563e55063f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.999+0000 7fcda047d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563e54fc1100 con 0x563e5505c090 2024-09-15T22:58:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.999+0000 7fcda047d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563e54f691b0 con 0x563e5505c9d0 2024-09-15T22:58:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.999+0000 7fcda047d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563e54fbd0c0 con 0x563e55061a60 2024-09-15T22:58:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.999+0000 7fcd8f400640 1 -- 172.21.15.111:0/311605041 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fcd840037a0 con 0x7fcd98104f00 2024-09-15T22:58:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.999+0000 7fcd96a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e5505c9d0 0x563e55061490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.999+0000 7fcd96a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e5505c9d0 0x563e55061490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:52918/0 (socket says 172.21.15.111:52918) 2024-09-15T22:58:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.999+0000 7fcd96a00640 1 -- 172.21.15.111:0/117362969 learned_addr learned my addr 172.21.15.111:0/117362969 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:58:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.999+0000 7fcd97400640 1 --2- 172.21.15.111:0/117362969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e5505c090 0x563e5505c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.999+0000 7fcd96000640 1 --2- 172.21.15.111:0/117362969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563e55061a60 0x563e55063f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.999+0000 7fcd96a00640 1 -- 172.21.15.111:0/117362969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563e55061a60 msgr2=0x563e55063f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.999+0000 7fcd96a00640 1 --2- 172.21.15.111:0/117362969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563e55061a60 0x563e55063f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.999+0000 7fcd96a00640 1 -- 172.21.15.111:0/117362969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e5505c090 msgr2=0x563e5505c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.999+0000 7fcd96a00640 1 --2- 172.21.15.111:0/117362969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e5505c090 0x563e5505c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.999+0000 7fcd96a00640 1 -- 172.21.15.111:0/117362969 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e54cd4210 con 0x563e5505c9d0 2024-09-15T22:58:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.999+0000 7fcd96000640 1 --2- 172.21.15.111:0/117362969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563e55061a60 0x563e55063f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:58:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:32.999+0000 7fcd97400640 1 --2- 172.21.15.111:0/117362969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e5505c090 0x563e5505c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:58:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.000+0000 7fcd96a00640 1 --2- 172.21.15.111:0/117362969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e5505c9d0 0x563e55061490 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcd88005ab0 tx=0x7fcd88004170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79ee7be548580628 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.000+0000 7fcd83400640 1 -- 172.21.15.111:0/117362969 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd88035040 con 0x563e5505c9d0 2024-09-15T22:58:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.000+0000 7fcd83400640 1 -- 172.21.15.111:0/117362969 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd88005520 con 0x563e5505c9d0 2024-09-15T22:58:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.000+0000 7fcd83400640 1 -- 172.21.15.111:0/117362969 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd88031910 con 0x563e5505c9d0 2024-09-15T22:58:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.000+0000 7fcd8f400640 1 --2- 172.21.15.111:0/311605041 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fcd7c0a32f0 0x7fcd7c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.000+0000 7fcd8f400640 1 -- 172.21.15.111:0/311605041 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fcd840cfca0 con 0x7fcd98104f00 2024-09-15T22:58:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.001+0000 7fcd97400640 1 --2- 172.21.15.111:0/311605041 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fcd7c0a32f0 0x7fcd7c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.001+0000 7fcda047d740 1 -- 172.21.15.111:0/117362969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e5505c9d0 msgr2=0x563e55061490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.001+0000 7fcda047d740 1 --2- 172.21.15.111:0/117362969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e5505c9d0 0x563e55061490 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcd88005ab0 tx=0x7fcd88004170 comp rx=0 tx=0).stop 2024-09-15T22:58:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.001+0000 7fcda047d740 1 -- 172.21.15.111:0/117362969 shutdown_connections 2024-09-15T22:58:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.001+0000 7fcda047d740 1 --2- 172.21.15.111:0/117362969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563e55061a60 0x563e55063f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.001+0000 7fcda047d740 1 --2- 172.21.15.111:0/117362969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e5505c9d0 0x563e55061490 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.001+0000 7fcda047d740 1 --2- 172.21.15.111:0/117362969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e5505c090 0x563e5505c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.001+0000 7fcda047d740 1 -- 172.21.15.111:0/117362969 >> 172.21.15.111:0/117362969 conn(0x563e55057700 msgr2=0x563e55057b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.001+0000 7fcda047d740 1 -- 172.21.15.111:0/117362969 shutdown_connections 2024-09-15T22:58:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.001+0000 7fcda047d740 1 -- 172.21.15.111:0/117362969 wait complete. 2024-09-15T22:58:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.001+0000 7fcd97400640 1 --2- 172.21.15.111:0/311605041 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fcd7c0a32f0 0x7fcd7c0a57b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcd9806b3e0 tx=0x7fcd90009470 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.002+0000 7fcda047d740 1 -- start start 2024-09-15T22:58:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.002+0000 7fcda047d740 1 Processor -- start 2024-09-15T22:58:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.002+0000 7fcda047d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563e5505c090 0x563e55066240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.002+0000 7fcda047d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e5505c9d0 0x563e55066780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.002+0000 7fcda047d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e55061a60 0x563e5506d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.002+0000 7fcda047d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563e550649d0 con 0x563e55061a60 2024-09-15T22:58:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.002+0000 7fcda047d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563e54f4a090 con 0x563e5505c9d0 2024-09-15T22:58:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.002+0000 7fcda047d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563e55064b50 con 0x563e5505c090 2024-09-15T22:58:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.002+0000 7fcda047d740 10 client.0 _ll_register_callbacks cb 0x563e54f9ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:58:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.003+0000 7fcd96a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563e5505c090 0x563e55066240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.003+0000 7fcd96000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e5505c9d0 0x563e55066780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.003+0000 7fcd96a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563e5505c090 0x563e55066240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:41104/0 (socket says 172.21.15.111:41104) 2024-09-15T22:58:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.003+0000 7fcd96a00640 1 -- 172.21.15.111:0/1440813068 learned_addr learned my addr 172.21.15.111:0/1440813068 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:58:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.003+0000 7fcd97400640 1 --2- 172.21.15.111:0/1440813068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e55061a60 0x563e5506d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.003+0000 7fcd96a00640 1 -- 172.21.15.111:0/1440813068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e5505c9d0 msgr2=0x563e55066780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.003+0000 7fcd96a00640 1 --2- 172.21.15.111:0/1440813068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e5505c9d0 0x563e55066780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.003+0000 7fcd96a00640 1 -- 172.21.15.111:0/1440813068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e55061a60 msgr2=0x563e5506d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.003+0000 7fcd96a00640 1 --2- 172.21.15.111:0/1440813068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e55061a60 0x563e5506d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.003+0000 7fcd96a00640 1 -- 172.21.15.111:0/1440813068 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e54da0620 con 0x563e5505c090 2024-09-15T22:58:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.003+0000 7fcd97400640 1 --2- 172.21.15.111:0/1440813068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e55061a60 0x563e5506d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:58:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.003+0000 7fcd96000640 1 --2- 172.21.15.111:0/1440813068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e5505c9d0 0x563e55066780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:58:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.010+0000 7fcd96a00640 1 --2- 172.21.15.111:0/1440813068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563e5505c090 0x563e55066240 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcd880314e0 tx=0x7fcd880058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3648819d4b49cb02 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.010+0000 7fcd77e00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:58:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.010+0000 7fcd77e00640 1 -- 172.21.15.111:0/1440813068 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd88035070 con 0x563e5505c090 2024-09-15T22:58:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.011+0000 7fcd77e00640 1 -- 172.21.15.111:0/1440813068 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fcd88004a10 con 0x563e5505c090 2024-09-15T22:58:33.012 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.011+0000 7fcda047d740 1 -- 172.21.15.111:0/1440813068 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd88005520 con 0x563e5505c090 2024-09-15T22:58:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.011+0000 7fcd77e00640 1 -- 172.21.15.111:0/1440813068 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcd88004ec0 con 0x563e5505c090 2024-09-15T22:58:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.011+0000 7fcda047d740 10 client.25999 fetch_fsmap learned FSMap version 30 2024-09-15T22:58:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.011+0000 7fcda047d740 1 -- 172.21.15.111:0/1440813068 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563e550649d0 con 0x563e5505c090 2024-09-15T22:58:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.011+0000 7fcd77e00640 1 -- 172.21.15.111:0/1440813068 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd88005520 con 0x563e5505c090 2024-09-15T22:58:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.011+0000 7fcd77e00640 1 -- 172.21.15.111:0/1440813068 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fcd8803c440 con 0x563e5505c090 2024-09-15T22:58:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.011+0000 7fcda047d740 10 client.25999 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:58:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.011+0000 7fcda047d740 10 client.25999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:58:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.011+0000 7fcda047d740 20 client.25999 populate_metadata read hostname 'smithi111' 2024-09-15T22:58:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.011+0000 7fcda047d740 1 --2- 172.21.15.111:0/1440813068 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563e550717b0 0x563e55073ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.011+0000 7fcda047d740 4 client.25999 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:58:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.011+0000 7fcda047d740 1 -- 172.21.15.111:0/1440813068 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e54fce830 con 0x563e550717b0 2024-09-15T22:58:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.012+0000 7fcd96000640 1 --2- 172.21.15.111:0/1440813068 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563e550717b0 0x563e55073ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.012+0000 7fcd96000640 1 --2- 172.21.15.111:0/1440813068 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563e550717b0 0x563e55073ba0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4a487faa927e884 server_cookie=235f6d110e65e17f in_seq=0 out_seq=0 2024-09-15T22:58:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.012+0000 7fcd77e00640 10 client.25999 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:58:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.013+0000 7fcd77e00640 1 -- 172.21.15.111:0/1440813068 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e54fce830 con 0x563e550717b0 2024-09-15T22:58:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.013+0000 7fcd77e00640 10 client.25999 handle_command_reply: tid=0 2024-09-15T22:58:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.013+0000 7fcda047d740 1 client.25999 shutdown 2024-09-15T22:58:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.013+0000 7fcda047d740 1 -- 172.21.15.111:0/1440813068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563e5505c090 msgr2=0x563e55066240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.013+0000 7fcda047d740 1 --2- 172.21.15.111:0/1440813068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563e5505c090 0x563e55066240 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcd880314e0 tx=0x7fcd880058e0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.013+0000 7fcda047d740 1 -- 172.21.15.111:0/1440813068 shutdown_connections 2024-09-15T22:58:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.013+0000 7fcda047d740 1 --2- 172.21.15.111:0/1440813068 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563e550717b0 0x563e55073ba0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.013+0000 7fcda047d740 1 --2- 172.21.15.111:0/1440813068 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e55061a60 0x563e5506d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.013+0000 7fcda047d740 1 --2- 172.21.15.111:0/1440813068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e5505c9d0 0x563e55066780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.013+0000 7fcda047d740 1 --2- 172.21.15.111:0/1440813068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563e5505c090 0x563e55066240 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.013+0000 7fcda047d740 1 -- 172.21.15.111:0/1440813068 >> 172.21.15.111:0/1440813068 conn(0x563e55057700 msgr2=0x563e55058f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.013+0000 7fcd77e00640 0 client.25999 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:58:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.013+0000 7fcda047d740 1 -- 172.21.15.111:0/1440813068 shutdown_connections 2024-09-15T22:58:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.013+0000 7fcda047d740 1 -- 172.21.15.111:0/1440813068 wait complete. 2024-09-15T22:58:33.017 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.013+0000 7fcda047d740 20 client.25999 trim_cache size 0 max 16384 2024-09-15T22:58:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.028+0000 7fcda047d740 1 Processor -- start 2024-09-15T22:58:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.028+0000 7fcda047d740 1 -- start start 2024-09-15T22:58:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.029+0000 7fcda047d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563e5505c090 0x563e55066810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.029+0000 7fcda047d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e5505c9d0 0x563e5506d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.029+0000 7fcda047d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e55061a60 0x563e55057ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.029+0000 7fcda047d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcd981ada90 con 0x563e55061a60 2024-09-15T22:58:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.029+0000 7fcda047d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcd981ab360 con 0x563e5505c9d0 2024-09-15T22:58:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.029+0000 7fcda047d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcd981a91d0 con 0x563e5505c090 2024-09-15T22:58:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.029+0000 7fcd97400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e55061a60 0x563e55057ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.029+0000 7fcd96a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563e5505c090 0x563e55066810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.029+0000 7fcd97400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e55061a60 0x563e55057ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39240/0 (socket says 172.21.15.111:39240) 2024-09-15T22:58:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.029+0000 7fcd97400640 1 -- 172.21.15.111:0/1155431117 learned_addr learned my addr 172.21.15.111:0/1155431117 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:58:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.029+0000 7fcd96000640 1 --2- 172.21.15.111:0/1155431117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e5505c9d0 0x563e5506d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.029+0000 7fcd96a00640 1 -- 172.21.15.111:0/1155431117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e5505c9d0 msgr2=0x563e5506d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.029+0000 7fcd96a00640 1 --2- 172.21.15.111:0/1155431117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e5505c9d0 0x563e5506d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.029+0000 7fcd96a00640 1 -- 172.21.15.111:0/1155431117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e55061a60 msgr2=0x563e55057ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.029+0000 7fcd96a00640 1 --2- 172.21.15.111:0/1155431117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e55061a60 0x563e55057ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.029+0000 7fcd96a00640 1 -- 172.21.15.111:0/1155431117 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e55073f10 con 0x563e5505c090 2024-09-15T22:58:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.029+0000 7fcd97400640 1 --2- 172.21.15.111:0/1155431117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e55061a60 0x563e55057ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:58:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.030+0000 7fcd96a00640 1 --2- 172.21.15.111:0/1155431117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563e5505c090 0x563e55066810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcd88004650 tx=0x7fcd88005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2a0015642f30f0b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.030+0000 7fcd82a00640 1 -- 172.21.15.111:0/1155431117 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8805e070 con 0x563e5505c090 2024-09-15T22:58:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.030+0000 7fcd82a00640 1 -- 172.21.15.111:0/1155431117 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd88005540 con 0x563e5505c090 2024-09-15T22:58:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.030+0000 7fcd82a00640 1 -- 172.21.15.111:0/1155431117 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd88035040 con 0x563e5505c090 2024-09-15T22:58:33.036 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.030+0000 7fcda047d740 1 -- 172.21.15.111:0/1155431117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563e5505c090 msgr2=0x563e55066810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:33.036 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.030+0000 7fcda047d740 1 --2- 172.21.15.111:0/1155431117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563e5505c090 0x563e55066810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcd88004650 tx=0x7fcd88005fd0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.036 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.030+0000 7fcda047d740 1 -- 172.21.15.111:0/1155431117 shutdown_connections 2024-09-15T22:58:33.036 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.030+0000 7fcda047d740 1 --2- 172.21.15.111:0/1155431117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e55061a60 0x563e55057ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.036 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.030+0000 7fcda047d740 1 --2- 172.21.15.111:0/1155431117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e5505c9d0 0x563e5506d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.036 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.030+0000 7fcda047d740 1 --2- 172.21.15.111:0/1155431117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563e5505c090 0x563e55066810 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.036 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.030+0000 7fcda047d740 1 -- 172.21.15.111:0/1155431117 >> 172.21.15.111:0/1155431117 conn(0x563e55057700 msgr2=0x563e550526a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:33.036 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.031+0000 7fcda047d740 1 -- 172.21.15.111:0/1155431117 shutdown_connections 2024-09-15T22:58:33.036 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.031+0000 7fcda047d740 1 -- 172.21.15.111:0/1155431117 wait complete. 2024-09-15T22:58:33.036 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.031+0000 7fcda047d740 1 -- start start 2024-09-15T22:58:33.037 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.031+0000 7fcda047d740 1 Processor -- start 2024-09-15T22:58:33.037 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.031+0000 7fcda047d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e5505c090 0x563e5506a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:33.037 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.031+0000 7fcda047d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563e5505c9d0 0x563e5506aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:33.037 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.032+0000 7fcda047d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e55061a60 0x563e550da420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:33.037 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.032+0000 7fcda047d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcd981a9820 con 0x563e55061a60 2024-09-15T22:58:33.037 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.032+0000 7fcda047d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcd981aac90 con 0x563e5505c090 2024-09-15T22:58:33.037 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.032+0000 7fcda047d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcd981a9ef0 con 0x563e5505c9d0 2024-09-15T22:58:33.037 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.032+0000 7fcda047d740 10 client.0 _ll_register_callbacks cb 0x7fcd88004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:58:33.037 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.032+0000 7fcd97400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e55061a60 0x563e550da420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:33.037 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.032+0000 7fcd96a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e5505c090 0x563e5506a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:33.037 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.032+0000 7fcd96000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563e5505c9d0 0x563e5506aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:33.037 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.032+0000 7fcd96a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e5505c090 0x563e5506a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:52944/0 (socket says 172.21.15.111:52944) 2024-09-15T22:58:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.032+0000 7fcd97400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e55061a60 0x563e550da420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39250/0 (socket says 172.21.15.111:39250) 2024-09-15T22:58:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.032+0000 7fcd96000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563e5505c9d0 0x563e5506aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:41128/0 (socket says 172.21.15.111:41128) 2024-09-15T22:58:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.032+0000 7fcd96a00640 1 -- 172.21.15.111:0/1458886182 learned_addr learned my addr 172.21.15.111:0/1458886182 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:58:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.032+0000 7fcd96a00640 1 -- 172.21.15.111:0/1458886182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563e5505c9d0 msgr2=0x563e5506aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.032+0000 7fcd96a00640 1 --2- 172.21.15.111:0/1458886182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563e5505c9d0 0x563e5506aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.032+0000 7fcd96a00640 1 -- 172.21.15.111:0/1458886182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e55061a60 msgr2=0x563e550da420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.032+0000 7fcd96a00640 1 --2- 172.21.15.111:0/1458886182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e55061a60 0x563e550da420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.032+0000 7fcd96a00640 1 -- 172.21.15.111:0/1458886182 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e550da960 con 0x563e5505c090 2024-09-15T22:58:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.032+0000 7fcd96000640 1 --2- 172.21.15.111:0/1458886182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563e5505c9d0 0x563e5506aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:58:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.032+0000 7fcd96a00640 1 --2- 172.21.15.111:0/1458886182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e5505c090 0x563e5506a5b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcd8802a930 tx=0x7fcd880315b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dda373e0358ed43b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.032+0000 7fcd77e00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:58:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.033+0000 7fcd77e00640 1 -- 172.21.15.111:0/1458886182 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8805e070 con 0x563e5505c090 2024-09-15T22:58:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.033+0000 7fcda047d740 1 -- 172.21.15.111:0/1458886182 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd88005540 con 0x563e5505c090 2024-09-15T22:58:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.033+0000 7fcd77e00640 1 -- 172.21.15.111:0/1458886182 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fcd88005060 con 0x563e5505c090 2024-09-15T22:58:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.033+0000 7fcd77e00640 1 -- 172.21.15.111:0/1458886182 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcd88035040 con 0x563e5505c090 2024-09-15T22:58:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.033+0000 7fcda047d740 10 client.25979 fetch_fsmap learned FSMap version 30 2024-09-15T22:58:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.033+0000 7fcda047d740 1 -- 172.21.15.111:0/1458886182 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563e550dbb90 con 0x563e5505c090 2024-09-15T22:58:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.033+0000 7fcd77e00640 1 -- 172.21.15.111:0/1458886182 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd88005540 con 0x563e5505c090 2024-09-15T22:58:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.033+0000 7fcd77e00640 1 -- 172.21.15.111:0/1458886182 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fcd880035f0 con 0x563e5505c090 2024-09-15T22:58:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.033+0000 7fcda047d740 10 client.25979 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:58:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.033+0000 7fcda047d740 10 client.25979 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:58:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.033+0000 7fcda047d740 20 client.25979 populate_metadata read hostname 'smithi111' 2024-09-15T22:58:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.033+0000 7fcda047d740 1 --2- 172.21.15.111:0/1458886182 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563e550dc2f0 0x563e550de6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.033+0000 7fcda047d740 4 client.25979 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:58:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.033+0000 7fcda047d740 1 -- 172.21.15.111:0/1458886182 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563e54fc8b80 con 0x563e550dc2f0 2024-09-15T22:58:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.034+0000 7fcd96000640 1 --2- 172.21.15.111:0/1458886182 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563e550dc2f0 0x563e550de6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:33.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.034+0000 7fcd96000640 1 --2- 172.21.15.111:0/1458886182 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563e550dc2f0 0x563e550de6e0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db71f984c1a4a108 server_cookie=28b7b052c453dc69 in_seq=0 out_seq=0 2024-09-15T22:58:33.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.034+0000 7fcd77e00640 10 client.25979 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:58:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.045+0000 7fcd77e00640 1 -- 172.21.15.111:0/1458886182 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563e54fc8b80 con 0x563e550dc2f0 2024-09-15T22:58:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.045+0000 7fcd77e00640 10 client.25979 handle_command_reply: tid=0 2024-09-15T22:58:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.045+0000 7fcda047d740 1 client.25979 shutdown 2024-09-15T22:58:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.045+0000 7fcda047d740 1 -- 172.21.15.111:0/1458886182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e5505c090 msgr2=0x563e5506a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.045+0000 7fcda047d740 1 --2- 172.21.15.111:0/1458886182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e5505c090 0x563e5506a5b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcd8802a930 tx=0x7fcd880315b0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.046+0000 7fcda047d740 1 -- 172.21.15.111:0/1458886182 shutdown_connections 2024-09-15T22:58:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.046+0000 7fcda047d740 1 --2- 172.21.15.111:0/1458886182 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563e550dc2f0 0x563e550de6e0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.046+0000 7fcda047d740 1 --2- 172.21.15.111:0/1458886182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563e55061a60 0x563e550da420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.046+0000 7fcd77e00640 0 client.25979 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:58:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.046+0000 7fcda047d740 1 --2- 172.21.15.111:0/1458886182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563e5505c9d0 0x563e5506aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.046+0000 7fcda047d740 1 --2- 172.21.15.111:0/1458886182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e5505c090 0x563e5506a5b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.046+0000 7fcda047d740 1 -- 172.21.15.111:0/1458886182 >> 172.21.15.111:0/1458886182 conn(0x563e55057700 msgr2=0x563e55056060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.046+0000 7fcda047d740 1 -- 172.21.15.111:0/1458886182 shutdown_connections 2024-09-15T22:58:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.046+0000 7fcda047d740 1 -- 172.21.15.111:0/1458886182 wait complete. 2024-09-15T22:58:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.046+0000 7fcda047d740 20 client.25979 trim_cache size 0 max 16384 2024-09-15T22:58:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:58:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:58:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:58:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:58:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.046+0000 7fcd9d600640 1 -- 172.21.15.111:0/311605041 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fcd7c0a32f0 msgr2=0x7fcd7c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.046+0000 7fcd9d600640 1 --2- 172.21.15.111:0/311605041 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fcd7c0a32f0 0x7fcd7c0a57b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcd9806b3e0 tx=0x7fcd90009470 comp rx=0 tx=0).stop 2024-09-15T22:58:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.046+0000 7fcd9d600640 1 -- 172.21.15.111:0/311605041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd98104f00 msgr2=0x7fcd98100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.046+0000 7fcd9d600640 1 --2- 172.21.15.111:0/311605041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd98104f00 0x7fcd98100580 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400e9a0 tx=0x7fcd8400ee60 comp rx=0 tx=0).stop 2024-09-15T22:58:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.047+0000 7fcd9d600640 1 -- 172.21.15.111:0/311605041 shutdown_connections 2024-09-15T22:58:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.047+0000 7fcd9d600640 1 --2- 172.21.15.111:0/311605041 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fcd7c0a32f0 0x7fcd7c0a57b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.047+0000 7fcd9d600640 1 --2- 172.21.15.111:0/311605041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd98107830 0x7fcd98100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.047+0000 7fcd9d600640 1 --2- 172.21.15.111:0/311605041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcd98104f00 0x7fcd98100580 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.047+0000 7fcd9d600640 1 --2- 172.21.15.111:0/311605041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd981025d0 0x7fcd98101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.047+0000 7fcd9d600640 1 -- 172.21.15.111:0/311605041 >> 172.21.15.111:0/311605041 conn(0x7fcd980fc1b0 msgr2=0x7fcd98102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.047+0000 7fcd9d600640 1 -- 172.21.15.111:0/311605041 shutdown_connections 2024-09-15T22:58:33.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:33.047+0000 7fcd9d600640 1 -- 172.21.15.111:0/311605041 wait complete. 2024-09-15T22:58:33.327 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1218679 9.84 MB/sec execute 541 sec latency 535.478 ms 2024-09-15T22:58:33.957 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:33 smithi111 ceph-mon[30629]: pgmap v729: 97 pgs: 97 active+clean; 270 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 170 op/s 2024-09-15T22:58:34.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:33 smithi121 ceph-mon[34815]: pgmap v729: 97 pgs: 97 active+clean; 270 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 170 op/s 2024-09-15T22:58:34.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:33 smithi192 ceph-mon[35427]: pgmap v729: 97 pgs: 97 active+clean; 270 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 170 op/s 2024-09-15T22:58:34.327 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1219094 9.82 MB/sec execute 542 sec latency 1260.389 ms 2024-09-15T22:58:35.327 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1222144 9.83 MB/sec execute 543 sec latency 7.198 ms 2024-09-15T22:58:36.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:35 smithi121 ceph-mon[34815]: pgmap v730: 97 pgs: 97 active+clean; 273 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 189 op/s 2024-09-15T22:58:36.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:35 smithi111 ceph-mon[30629]: pgmap v730: 97 pgs: 97 active+clean; 273 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 189 op/s 2024-09-15T22:58:36.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:35 smithi192 ceph-mon[35427]: pgmap v730: 97 pgs: 97 active+clean; 273 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 189 op/s 2024-09-15T22:58:36.327 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1225114 9.84 MB/sec execute 544 sec latency 13.896 ms 2024-09-15T22:58:37.327 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1227670 9.85 MB/sec execute 545 sec latency 15.301 ms 2024-09-15T22:58:38.108 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:37 smithi111 ceph-mon[30629]: pgmap v731: 97 pgs: 97 active+clean; 273 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 194 op/s 2024-09-15T22:58:38.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:37 smithi192 ceph-mon[35427]: pgmap v731: 97 pgs: 97 active+clean; 273 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 194 op/s 2024-09-15T22:58:38.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:37 smithi121 ceph-mon[34815]: pgmap v731: 97 pgs: 97 active+clean; 273 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 194 op/s 2024-09-15T22:58:38.327 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1229184 9.85 MB/sec execute 546 sec latency 391.485 ms 2024-09-15T22:58:39.327 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1229184 9.83 MB/sec execute 547 sec latency 1391.522 ms 2024-09-15T22:58:40.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:39 smithi121 ceph-mon[34815]: pgmap v732: 97 pgs: 97 active+clean; 276 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 211 op/s 2024-09-15T22:58:40.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:39 smithi111 ceph-mon[30629]: pgmap v732: 97 pgs: 97 active+clean; 276 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 211 op/s 2024-09-15T22:58:40.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:39 smithi192 ceph-mon[35427]: pgmap v732: 97 pgs: 97 active+clean; 276 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 211 op/s 2024-09-15T22:58:40.327 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1231261 9.83 MB/sec execute 548 sec latency 1598.188 ms 2024-09-15T22:58:41.327 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1233069 9.83 MB/sec execute 549 sec latency 105.191 ms 2024-09-15T22:58:42.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:41 smithi121 ceph-mon[34815]: pgmap v733: 97 pgs: 97 active+clean; 273 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-09-15T22:58:42.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:41 smithi111 ceph-mon[30629]: pgmap v733: 97 pgs: 97 active+clean; 273 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-09-15T22:58:42.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:41 smithi192 ceph-mon[35427]: pgmap v733: 97 pgs: 97 active+clean; 273 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-09-15T22:58:42.327 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1236197 9.85 MB/sec execute 550 sec latency 7.302 ms 2024-09-15T22:58:43.328 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1237574 9.84 MB/sec execute 551 sec latency 243.092 ms 2024-09-15T22:58:43.895 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:43 smithi111 ceph-mon[30629]: pgmap v734: 97 pgs: 97 active+clean; 273 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 178 op/s 2024-09-15T22:58:44.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:43 smithi121 ceph-mon[34815]: pgmap v734: 97 pgs: 97 active+clean; 273 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 178 op/s 2024-09-15T22:58:44.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:43 smithi192 ceph-mon[35427]: pgmap v734: 97 pgs: 97 active+clean; 273 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 178 op/s 2024-09-15T22:58:44.327 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1237583 9.82 MB/sec execute 552 sec latency 679.951 ms 2024-09-15T22:58:45.328 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1239937 9.83 MB/sec execute 553 sec latency 667.138 ms 2024-09-15T22:58:46.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:45 smithi121 ceph-mon[34815]: pgmap v735: 97 pgs: 97 active+clean; 274 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 208 op/s 2024-09-15T22:58:46.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:45 smithi111 ceph-mon[30629]: pgmap v735: 97 pgs: 97 active+clean; 274 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 208 op/s 2024-09-15T22:58:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.174+0000 7fe5ed200640 1 -- 172.21.15.111:0/2390017453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5e8107620 msgr2=0x7fe5e8109a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.174+0000 7fe5ed200640 1 --2- 172.21.15.111:0/2390017453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5e8107620 0x7fe5e8109a90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0009ca0 tx=0x7fe5e002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.174+0000 7fe5ed200640 1 -- 172.21.15.111:0/2390017453 shutdown_connections 2024-09-15T22:58:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.174+0000 7fe5ed200640 1 --2- 172.21.15.111:0/2390017453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5e8107620 0x7fe5e8109a90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.176 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.174+0000 7fe5ed200640 1 --2- 172.21.15.111:0/2390017453 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5e8104cf0 0x7fe5e81070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.174+0000 7fe5ed200640 1 --2- 172.21.15.111:0/2390017453 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5e81023c0 0x7fe5e81047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.174+0000 7fe5ed200640 1 -- 172.21.15.111:0/2390017453 >> 172.21.15.111:0/2390017453 conn(0x7fe5e80fbfa0 msgr2=0x7fe5e80fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.174+0000 7fe5ed200640 1 -- 172.21.15.111:0/2390017453 shutdown_connections 2024-09-15T22:58:46.177 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.174+0000 7fe5ed200640 1 -- 172.21.15.111:0/2390017453 wait complete. 2024-09-15T22:58:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.175+0000 7fe5ed200640 1 Processor -- start 2024-09-15T22:58:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.175+0000 7fe5ed200640 1 -- start start 2024-09-15T22:58:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.176+0000 7fe5ed200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5e81023c0 0x7fe5e8101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.176+0000 7fe5ed200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5e8104cf0 0x7fe5e8100370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.176+0000 7fe5ed200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5e8107620 0x7fe5e81008b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.176+0000 7fe5ed200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e810d600 con 0x7fe5e8104cf0 2024-09-15T22:58:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.176+0000 7fe5ed200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e810d480 con 0x7fe5e81023c0 2024-09-15T22:58:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.176+0000 7fe5ed200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e810d780 con 0x7fe5e8107620 2024-09-15T22:58:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.176+0000 7fe5e7e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5e8107620 0x7fe5e81008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:46.179 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.176+0000 7fe5e7400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5e81023c0 0x7fe5e8101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.176+0000 7fe5dfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5e8104cf0 0x7fe5e8100370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.176+0000 7fe5e7400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5e81023c0 0x7fe5e8101d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:55944/0 (socket says 172.21.15.111:55944) 2024-09-15T22:58:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.176+0000 7fe5e7400640 1 -- 172.21.15.111:0/197967400 learned_addr learned my addr 172.21.15.111:0/197967400 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:58:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.176+0000 7fe5e7e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5e8107620 0x7fe5e81008b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:34682/0 (socket says 172.21.15.111:34682) 2024-09-15T22:58:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.176+0000 7fe5e7e00640 1 -- 172.21.15.111:0/197967400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5e81023c0 msgr2=0x7fe5e8101d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.176+0000 7fe5e7e00640 1 --2- 172.21.15.111:0/197967400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5e81023c0 0x7fe5e8101d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.176+0000 7fe5e7e00640 1 -- 172.21.15.111:0/197967400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5e8104cf0 msgr2=0x7fe5e8100370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.176+0000 7fe5e7e00640 1 --2- 172.21.15.111:0/197967400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5e8104cf0 0x7fe5e8100370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.176+0000 7fe5e7e00640 1 -- 172.21.15.111:0/197967400 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e8100df0 con 0x7fe5e8107620 2024-09-15T22:58:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.176+0000 7fe5e7400640 1 --2- 172.21.15.111:0/197967400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5e81023c0 0x7fe5e8101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:58:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.177+0000 7fe5e7e00640 1 --2- 172.21.15.111:0/197967400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5e8107620 0x7fe5e81008b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe5e002ade0 tx=0x7fe5e0004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.177+0000 7fe5e4c00640 1 -- 172.21.15.111:0/197967400 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0003b70 con 0x7fe5e8107620 2024-09-15T22:58:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.177+0000 7fe5ed200640 1 -- 172.21.15.111:0/197967400 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e8101080 con 0x7fe5e8107620 2024-09-15T22:58:46.180 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.177+0000 7fe5e4c00640 1 -- 172.21.15.111:0/197967400 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5e0003d10 con 0x7fe5e8107620 2024-09-15T22:58:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.177+0000 7fe5e4c00640 1 -- 172.21.15.111:0/197967400 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e002fc10 con 0x7fe5e8107620 2024-09-15T22:58:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.177+0000 7fe5ed200640 1 -- 172.21.15.111:0/197967400 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e81a6710 con 0x7fe5e8107620 2024-09-15T22:58:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.180+0000 7fe5f17fc740 1 Processor -- start 2024-09-15T22:58:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.180+0000 7fe5e4c00640 1 -- 172.21.15.111:0/197967400 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe5e0004050 con 0x7fe5e8107620 2024-09-15T22:58:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.180+0000 7fe5f17fc740 1 -- start start 2024-09-15T22:58:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.181+0000 7fe5f17fc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55924578a260 0x55924578a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.181+0000 7fe5f17fc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55924578aba0 0x55924578f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.181+0000 7fe5f17fc740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55924578fc30 0x5592457920f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.181+0000 7fe5f17fc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5592456ef100 con 0x55924578a260 2024-09-15T22:58:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.181+0000 7fe5f17fc740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5592456971b0 con 0x55924578fc30 2024-09-15T22:58:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.181+0000 7fe5f17fc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5592456eb0c0 con 0x55924578aba0 2024-09-15T22:58:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.181+0000 7fe5dfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55924578a260 0x55924578a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.181+0000 7fe5dfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55924578a260 0x55924578a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33796/0 (socket says 172.21.15.111:33796) 2024-09-15T22:58:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.181+0000 7fe5e7400640 1 --2- 172.21.15.111:0/2633283545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55924578aba0 0x55924578f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.181+0000 7fe5dfe00640 1 -- 172.21.15.111:0/2633283545 learned_addr learned my addr 172.21.15.111:0/2633283545 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:58:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.181+0000 7fe5dfe00640 1 --2- 172.21.15.111:0/2633283545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55924578fc30 0x5592457920f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.181+0000 7fe5e7400640 1 -- 172.21.15.111:0/2633283545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55924578fc30 msgr2=0x5592457920f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.181+0000 7fe5e7400640 1 --2- 172.21.15.111:0/2633283545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55924578fc30 0x5592457920f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.181+0000 7fe5e7400640 1 -- 172.21.15.111:0/2633283545 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55924578a260 msgr2=0x55924578a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.181+0000 7fe5e7400640 1 --2- 172.21.15.111:0/2633283545 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55924578a260 0x55924578a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.181+0000 7fe5e7400640 1 -- 172.21.15.111:0/2633283545 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559245402210 con 0x55924578aba0 2024-09-15T22:58:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.182+0000 7fe5e7400640 1 --2- 172.21.15.111:0/2633283545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55924578aba0 0x55924578f660 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8011510 tx=0x7fe5d80119d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64806782c218ec8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.182+0000 7fe5e4c00640 1 --2- 172.21.15.111:0/197967400 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fe5cc0a3240 0x7fe5cc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:46.188 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.182+0000 7fe5dcc00640 1 -- 172.21.15.111:0/2633283545 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d801e070 con 0x55924578aba0 2024-09-15T22:58:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.182+0000 7fe5e4c00640 1 -- 172.21.15.111:0/197967400 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fe5e00f0ae0 con 0x7fe5e8107620 2024-09-15T22:58:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.182+0000 7fe5dcc00640 1 -- 172.21.15.111:0/2633283545 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d800fa20 con 0x55924578aba0 2024-09-15T22:58:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.182+0000 7fe5dcc00640 1 -- 172.21.15.111:0/2633283545 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8019bc0 con 0x55924578aba0 2024-09-15T22:58:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.182+0000 7fe5dfe00640 1 --2- 172.21.15.111:0/197967400 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fe5cc0a3240 0x7fe5cc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.183+0000 7fe5f17fc740 1 -- 172.21.15.111:0/2633283545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55924578aba0 msgr2=0x55924578f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.183+0000 7fe5dfe00640 1 --2- 172.21.15.111:0/197967400 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fe5cc0a3240 0x7fe5cc0a5700 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0009fd0 tx=0x7fe5d0005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.183+0000 7fe5f17fc740 1 --2- 172.21.15.111:0/2633283545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55924578aba0 0x55924578f660 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8011510 tx=0x7fe5d80119d0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.183+0000 7fe5f17fc740 1 -- 172.21.15.111:0/2633283545 shutdown_connections 2024-09-15T22:58:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.183+0000 7fe5f17fc740 1 --2- 172.21.15.111:0/2633283545 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55924578fc30 0x5592457920f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.183+0000 7fe5f17fc740 1 --2- 172.21.15.111:0/2633283545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55924578aba0 0x55924578f660 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.183+0000 7fe5f17fc740 1 --2- 172.21.15.111:0/2633283545 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55924578a260 0x55924578a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.183+0000 7fe5f17fc740 1 -- 172.21.15.111:0/2633283545 >> 172.21.15.111:0/2633283545 conn(0x5592457858d0 msgr2=0x559245785cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.183+0000 7fe5f17fc740 1 -- 172.21.15.111:0/2633283545 shutdown_connections 2024-09-15T22:58:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.183+0000 7fe5f17fc740 1 -- 172.21.15.111:0/2633283545 wait complete. 2024-09-15T22:58:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.184+0000 7fe5f17fc740 1 -- start start 2024-09-15T22:58:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.184+0000 7fe5f17fc740 1 Processor -- start 2024-09-15T22:58:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.184+0000 7fe5f17fc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55924578a260 0x559245794410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.184+0000 7fe5f17fc740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55924578aba0 0x559245794950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.184+0000 7fe5f17fc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55924578fc30 0x55924579b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.184+0000 7fe5f17fc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559245792c00 con 0x55924578fc30 2024-09-15T22:58:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.184+0000 7fe5f17fc740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559245678090 con 0x55924578aba0 2024-09-15T22:58:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.184+0000 7fe5f17fc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x559245792d80 con 0x55924578a260 2024-09-15T22:58:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.184+0000 7fe5f17fc740 10 client.0 _ll_register_callbacks cb 0x5592456ccf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:58:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.185+0000 7fe5dfe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55924578aba0 0x559245794950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.185+0000 7fe5e7400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55924578a260 0x559245794410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:46.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.185+0000 7fe5dfe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55924578aba0 0x559245794950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:55966/0 (socket says 172.21.15.111:55966) 2024-09-15T22:58:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.185+0000 7fe5dfe00640 1 -- 172.21.15.111:0/3018850266 learned_addr learned my addr 172.21.15.111:0/3018850266 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:58:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.185+0000 7fe5e7e00640 1 --2- 172.21.15.111:0/3018850266 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55924578fc30 0x55924579b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.185+0000 7fe5e7400640 1 -- 172.21.15.111:0/3018850266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55924578aba0 msgr2=0x559245794950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.185+0000 7fe5e7400640 1 --2- 172.21.15.111:0/3018850266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55924578aba0 0x559245794950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.185+0000 7fe5e7400640 1 -- 172.21.15.111:0/3018850266 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55924578fc30 msgr2=0x55924579b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.185+0000 7fe5e7400640 1 --2- 172.21.15.111:0/3018850266 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55924578fc30 0x55924579b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.185+0000 7fe5e7400640 1 -- 172.21.15.111:0/3018850266 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592454ce620 con 0x55924578a260 2024-09-15T22:58:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.185+0000 7fe5e7e00640 1 --2- 172.21.15.111:0/3018850266 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55924578fc30 0x55924579b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:58:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.185+0000 7fe5e7400640 1 --2- 172.21.15.111:0/3018850266 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55924578a260 0x559245794410 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe5d801c040 tx=0x7fe5d800f770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3fac2a49819dafa server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.185+0000 7fe5d5600640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:58:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.185+0000 7fe5d5600640 1 -- 172.21.15.111:0/3018850266 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d801e040 con 0x55924578a260 2024-09-15T22:58:46.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.185+0000 7fe5f17fc740 1 -- 172.21.15.111:0/3018850266 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5d800fa20 con 0x55924578a260 2024-09-15T22:58:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.185+0000 7fe5d5600640 1 -- 172.21.15.111:0/3018850266 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fe5d8019900 con 0x55924578a260 2024-09-15T22:58:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.186+0000 7fe5d5600640 1 -- 172.21.15.111:0/3018850266 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe5d8009cf0 con 0x55924578a260 2024-09-15T22:58:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.186+0000 7fe5f17fc740 10 client.26017 fetch_fsmap learned FSMap version 30 2024-09-15T22:58:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.186+0000 7fe5f17fc740 1 -- 172.21.15.111:0/3018850266 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55924579e780 con 0x55924578a260 2024-09-15T22:58:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.186+0000 7fe5d5600640 1 -- 172.21.15.111:0/3018850266 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d800fa20 con 0x55924578a260 2024-09-15T22:58:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.186+0000 7fe5d5600640 1 -- 172.21.15.111:0/3018850266 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fe5d8021030 con 0x55924578a260 2024-09-15T22:58:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.186+0000 7fe5f17fc740 10 client.26017 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:58:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.186+0000 7fe5f17fc740 10 client.26017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:58:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.186+0000 7fe5f17fc740 20 client.26017 populate_metadata read hostname 'smithi111' 2024-09-15T22:58:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.186+0000 7fe5f17fc740 1 --2- 172.21.15.111:0/3018850266 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55924579fca0 0x5592457a20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.186+0000 7fe5f17fc740 4 client.26017 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:58:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.186+0000 7fe5f17fc740 1 -- 172.21.15.111:0/3018850266 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592456fc830 con 0x55924579fca0 2024-09-15T22:58:46.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.187+0000 7fe5dfe00640 1 --2- 172.21.15.111:0/3018850266 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55924579fca0 0x5592457a20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.187+0000 7fe5dfe00640 1 --2- 172.21.15.111:0/3018850266 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55924579fca0 0x5592457a20b0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=779df3ae675c7ee2 server_cookie=68a1429d43b54bf3 in_seq=0 out_seq=0 2024-09-15T22:58:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.187+0000 7fe5d5600640 10 client.26017 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:58:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.188+0000 7fe5d5600640 1 -- 172.21.15.111:0/3018850266 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5592456fc830 con 0x55924579fca0 2024-09-15T22:58:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.188+0000 7fe5d5600640 10 client.26017 handle_command_reply: tid=0 2024-09-15T22:58:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.188+0000 7fe5f17fc740 1 client.26017 shutdown 2024-09-15T22:58:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.188+0000 7fe5f17fc740 1 -- 172.21.15.111:0/3018850266 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55924578a260 msgr2=0x559245794410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.188+0000 7fe5f17fc740 1 --2- 172.21.15.111:0/3018850266 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55924578a260 0x559245794410 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe5d801c040 tx=0x7fe5d800f770 comp rx=0 tx=0).stop 2024-09-15T22:58:46.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.188+0000 7fe5f17fc740 1 -- 172.21.15.111:0/3018850266 shutdown_connections 2024-09-15T22:58:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.188+0000 7fe5f17fc740 1 --2- 172.21.15.111:0/3018850266 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55924579fca0 0x5592457a20b0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.188+0000 7fe5f17fc740 1 --2- 172.21.15.111:0/3018850266 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55924578fc30 0x55924579b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.188+0000 7fe5d5600640 0 client.26017 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:58:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.188+0000 7fe5f17fc740 1 --2- 172.21.15.111:0/3018850266 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55924578aba0 0x559245794950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.188+0000 7fe5f17fc740 1 --2- 172.21.15.111:0/3018850266 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55924578a260 0x559245794410 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.188+0000 7fe5f17fc740 1 -- 172.21.15.111:0/3018850266 >> 172.21.15.111:0/3018850266 conn(0x5592457858d0 msgr2=0x5592457870e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.189+0000 7fe5f17fc740 1 -- 172.21.15.111:0/3018850266 shutdown_connections 2024-09-15T22:58:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.189+0000 7fe5f17fc740 1 -- 172.21.15.111:0/3018850266 wait complete. 2024-09-15T22:58:46.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.189+0000 7fe5f17fc740 20 client.26017 trim_cache size 0 max 16384 2024-09-15T22:58:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.205+0000 7fe5f17fc740 1 Processor -- start 2024-09-15T22:58:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.205+0000 7fe5f17fc740 1 -- start start 2024-09-15T22:58:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.205+0000 7fe5f17fc740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592457858d0 0x559245785cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.205+0000 7fe5f17fc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55924578a260 0x55924579b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.205+0000 7fe5f17fc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55924578aba0 0x5592457861f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.205+0000 7fe5f17fc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e81b1b20 con 0x55924578a260 2024-09-15T22:58:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.205+0000 7fe5f17fc740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e81af3f0 con 0x5592457858d0 2024-09-15T22:58:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.205+0000 7fe5f17fc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e81ad240 con 0x55924578aba0 2024-09-15T22:58:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.205+0000 7fe5e7400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592457858d0 0x559245785cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.205+0000 7fe5dfe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55924578a260 0x55924579b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.205+0000 7fe5e7400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592457858d0 0x559245785cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:55972/0 (socket says 172.21.15.111:55972) 2024-09-15T22:58:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.205+0000 7fe5e7400640 1 -- 172.21.15.111:0/1673986042 learned_addr learned my addr 172.21.15.111:0/1673986042 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:58:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.206+0000 7fe5e7400640 1 -- 172.21.15.111:0/1673986042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55924578aba0 msgr2=0x5592457861f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.206+0000 7fe5e7400640 1 --2- 172.21.15.111:0/1673986042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55924578aba0 0x5592457861f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.206+0000 7fe5e7400640 1 -- 172.21.15.111:0/1673986042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55924578a260 msgr2=0x55924579b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.206+0000 7fe5e7400640 1 --2- 172.21.15.111:0/1673986042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55924578a260 0x55924579b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.206+0000 7fe5e7400640 1 -- 172.21.15.111:0/1673986042 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592457a2730 con 0x5592457858d0 2024-09-15T22:58:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.206+0000 7fe5e7400640 1 --2- 172.21.15.111:0/1673986042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592457858d0 0x559245785cb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8011460 tx=0x7fe5d8012f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae0fa00f4f6ef2b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:46.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.206+0000 7fe5d7e00640 1 -- 172.21.15.111:0/1673986042 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8019790 con 0x5592457858d0 2024-09-15T22:58:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.206+0000 7fe5d7e00640 1 -- 172.21.15.111:0/1673986042 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d8029070 con 0x5592457858d0 2024-09-15T22:58:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.206+0000 7fe5d7e00640 1 -- 172.21.15.111:0/1673986042 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8024ac0 con 0x5592457858d0 2024-09-15T22:58:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.207+0000 7fe5f17fc740 1 -- 172.21.15.111:0/1673986042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592457858d0 msgr2=0x559245785cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.207+0000 7fe5f17fc740 1 --2- 172.21.15.111:0/1673986042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592457858d0 0x559245785cb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8011460 tx=0x7fe5d8012f90 comp rx=0 tx=0).stop 2024-09-15T22:58:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.207+0000 7fe5f17fc740 1 -- 172.21.15.111:0/1673986042 shutdown_connections 2024-09-15T22:58:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.207+0000 7fe5f17fc740 1 --2- 172.21.15.111:0/1673986042 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55924578aba0 0x5592457861f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.207+0000 7fe5f17fc740 1 --2- 172.21.15.111:0/1673986042 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55924578a260 0x55924579b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.207+0000 7fe5f17fc740 1 --2- 172.21.15.111:0/1673986042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592457858d0 0x559245785cb0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.208+0000 7fe5f17fc740 1 -- 172.21.15.111:0/1673986042 >> 172.21.15.111:0/1673986042 conn(0x55924578f780 msgr2=0x5592457941c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.208+0000 7fe5f17fc740 1 -- 172.21.15.111:0/1673986042 shutdown_connections 2024-09-15T22:58:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.208+0000 7fe5f17fc740 1 -- 172.21.15.111:0/1673986042 wait complete. 2024-09-15T22:58:46.213 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.208+0000 7fe5f17fc740 1 -- start start 2024-09-15T22:58:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.208+0000 7fe5f17fc740 1 Processor -- start 2024-09-15T22:58:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.209+0000 7fe5f17fc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5592457858d0 0x5592457986c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.209+0000 7fe5f17fc740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55924578a260 0x559245798c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.209+0000 7fe5f17fc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55924578aba0 0x559245808980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.209+0000 7fe5f17fc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e81ad8b0 con 0x5592457858d0 2024-09-15T22:58:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.209+0000 7fe5f17fc740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e81aed20 con 0x55924578a260 2024-09-15T22:58:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.209+0000 7fe5f17fc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e81adf80 con 0x55924578aba0 2024-09-15T22:58:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.209+0000 7fe5f17fc740 10 client.0 _ll_register_callbacks cb 0x7fe5d801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:58:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.209+0000 7fe5dfe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55924578a260 0x559245798c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.209+0000 7fe5e7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5592457858d0 0x5592457986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.209+0000 7fe5dfe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55924578a260 0x559245798c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:55976/0 (socket says 172.21.15.111:55976) 2024-09-15T22:58:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.209+0000 7fe5dfe00640 1 -- 172.21.15.111:0/127442864 learned_addr learned my addr 172.21.15.111:0/127442864 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:58:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.209+0000 7fe5e7e00640 1 --2- 172.21.15.111:0/127442864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55924578aba0 0x559245808980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.209+0000 7fe5dfe00640 1 -- 172.21.15.111:0/127442864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55924578aba0 msgr2=0x559245808980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.209+0000 7fe5dfe00640 1 --2- 172.21.15.111:0/127442864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55924578aba0 0x559245808980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.210+0000 7fe5dfe00640 1 -- 172.21.15.111:0/127442864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5592457858d0 msgr2=0x5592457986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.210+0000 7fe5dfe00640 1 --2- 172.21.15.111:0/127442864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5592457858d0 0x5592457986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.210+0000 7fe5dfe00640 1 -- 172.21.15.111:0/127442864 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559245808ec0 con 0x55924578a260 2024-09-15T22:58:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.210+0000 7fe5dfe00640 1 --2- 172.21.15.111:0/127442864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55924578a260 0x559245798c00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x5592457a2930 tx=0x7fe5d0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa367eabb6967483 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.210+0000 7fe5d5600640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:58:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.210+0000 7fe5d5600640 1 -- 172.21.15.111:0/127442864 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d000fbc0 con 0x55924578a260 2024-09-15T22:58:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.210+0000 7fe5d5600640 1 -- 172.21.15.111:0/127442864 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fe5d0009040 con 0x55924578a260 2024-09-15T22:58:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.210+0000 7fe5f17fc740 1 -- 172.21.15.111:0/127442864 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5d8029070 con 0x55924578a260 2024-09-15T22:58:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.210+0000 7fe5d5600640 1 -- 172.21.15.111:0/127442864 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe5d000fd60 con 0x55924578a260 2024-09-15T22:58:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.210+0000 7fe5f17fc740 10 client.25997 fetch_fsmap learned FSMap version 30 2024-09-15T22:58:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.210+0000 7fe5f17fc740 1 -- 172.21.15.111:0/127442864 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55924580a110 con 0x55924578a260 2024-09-15T22:58:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.211+0000 7fe5d5600640 1 -- 172.21.15.111:0/127442864 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d8029070 con 0x55924578a260 2024-09-15T22:58:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.211+0000 7fe5d5600640 1 -- 172.21.15.111:0/127442864 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fe5d002e410 con 0x55924578a260 2024-09-15T22:58:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.211+0000 7fe5f17fc740 10 client.25997 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:58:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.211+0000 7fe5f17fc740 10 client.25997 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:58:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.211+0000 7fe5f17fc740 20 client.25997 populate_metadata read hostname 'smithi111' 2024-09-15T22:58:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.211+0000 7fe5f17fc740 1 --2- 172.21.15.111:0/127442864 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55924578fc30 0x55924580cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.211+0000 7fe5f17fc740 4 client.25997 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:58:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.211+0000 7fe5f17fc740 1 -- 172.21.15.111:0/127442864 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5592456f6b80 con 0x55924578fc30 2024-09-15T22:58:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.211+0000 7fe5e7400640 1 --2- 172.21.15.111:0/127442864 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55924578fc30 0x55924580cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.212+0000 7fe5e7400640 1 --2- 172.21.15.111:0/127442864 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55924578fc30 0x55924580cb60 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4bcf7ec71807672 server_cookie=8883902ac8d0ca0 in_seq=0 out_seq=0 2024-09-15T22:58:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.212+0000 7fe5d5600640 10 client.25997 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:58:46.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.224+0000 7fe5d5600640 1 -- 172.21.15.111:0/127442864 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5592456f6b80 con 0x55924578fc30 2024-09-15T22:58:46.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.224+0000 7fe5d5600640 10 client.25997 handle_command_reply: tid=0 2024-09-15T22:58:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.224+0000 7fe5f17fc740 1 client.25997 shutdown 2024-09-15T22:58:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.225+0000 7fe5f17fc740 1 -- 172.21.15.111:0/127442864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55924578a260 msgr2=0x559245798c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.225+0000 7fe5f17fc740 1 --2- 172.21.15.111:0/127442864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55924578a260 0x559245798c00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x5592457a2930 tx=0x7fe5d0005f90 comp rx=0 tx=0).stop 2024-09-15T22:58:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.225+0000 7fe5f17fc740 1 -- 172.21.15.111:0/127442864 shutdown_connections 2024-09-15T22:58:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.225+0000 7fe5f17fc740 1 --2- 172.21.15.111:0/127442864 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55924578fc30 0x55924580cb60 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.225+0000 7fe5f17fc740 1 --2- 172.21.15.111:0/127442864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55924578aba0 0x559245808980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.225+0000 7fe5f17fc740 1 --2- 172.21.15.111:0/127442864 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55924578a260 0x559245798c00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.225+0000 7fe5d5600640 0 client.25997 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:58:46.227 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.225+0000 7fe5f17fc740 1 --2- 172.21.15.111:0/127442864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5592457858d0 0x5592457986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.225+0000 7fe5f17fc740 1 -- 172.21.15.111:0/127442864 >> 172.21.15.111:0/127442864 conn(0x55924578f780 msgr2=0x5592457841f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.225+0000 7fe5f17fc740 1 -- 172.21.15.111:0/127442864 shutdown_connections 2024-09-15T22:58:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.225+0000 7fe5f17fc740 1 -- 172.21.15.111:0/127442864 wait complete. 2024-09-15T22:58:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.225+0000 7fe5f17fc740 20 client.25997 trim_cache size 0 max 16384 2024-09-15T22:58:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:58:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:58:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:58:46.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:58:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.226+0000 7fe5ed200640 1 -- 172.21.15.111:0/197967400 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fe5cc0a3240 msgr2=0x7fe5cc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.226+0000 7fe5ed200640 1 --2- 172.21.15.111:0/197967400 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fe5cc0a3240 0x7fe5cc0a5700 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0009fd0 tx=0x7fe5d0005850 comp rx=0 tx=0).stop 2024-09-15T22:58:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.226+0000 7fe5ed200640 1 -- 172.21.15.111:0/197967400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5e8107620 msgr2=0x7fe5e81008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.226+0000 7fe5ed200640 1 --2- 172.21.15.111:0/197967400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5e8107620 0x7fe5e81008b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe5e002ade0 tx=0x7fe5e0004ab0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.226+0000 7fe5ed200640 1 -- 172.21.15.111:0/197967400 shutdown_connections 2024-09-15T22:58:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.226+0000 7fe5ed200640 1 --2- 172.21.15.111:0/197967400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5e8107620 0x7fe5e81008b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.226+0000 7fe5ed200640 1 --2- 172.21.15.111:0/197967400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5e8104cf0 0x7fe5e8100370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.226+0000 7fe5ed200640 1 --2- 172.21.15.111:0/197967400 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fe5cc0a3240 0x7fe5cc0a5700 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.226+0000 7fe5ed200640 1 --2- 172.21.15.111:0/197967400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5e81023c0 0x7fe5e8101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.226+0000 7fe5ed200640 1 -- 172.21.15.111:0/197967400 >> 172.21.15.111:0/197967400 conn(0x7fe5e80fbfa0 msgr2=0x7fe5e8102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.226+0000 7fe5ed200640 1 -- 172.21.15.111:0/197967400 shutdown_connections 2024-09-15T22:58:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:46.226+0000 7fe5ed200640 1 -- 172.21.15.111:0/197967400 wait complete. 2024-09-15T22:58:46.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:45 smithi192 ceph-mon[35427]: pgmap v735: 97 pgs: 97 active+clean; 274 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 208 op/s 2024-09-15T22:58:46.328 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1243114 9.84 MB/sec execute 554 sec latency 11.094 ms 2024-09-15T22:58:47.328 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1245475 9.84 MB/sec execute 555 sec latency 38.430 ms 2024-09-15T22:58:48.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:47 smithi121 ceph-mon[34815]: pgmap v736: 97 pgs: 97 active+clean; 274 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 189 op/s 2024-09-15T22:58:48.178 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:47 smithi111 ceph-mon[30629]: pgmap v736: 97 pgs: 97 active+clean; 274 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 189 op/s 2024-09-15T22:58:48.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:47 smithi192 ceph-mon[35427]: pgmap v736: 97 pgs: 97 active+clean; 274 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 189 op/s 2024-09-15T22:58:48.328 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1247271 9.85 MB/sec execute 556 sec latency 124.852 ms 2024-09-15T22:58:49.328 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1247271 9.83 MB/sec execute 557 sec latency 1124.902 ms 2024-09-15T22:58:50.123 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:49 smithi111 ceph-mon[30629]: pgmap v737: 97 pgs: 97 active+clean; 279 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 235 op/s 2024-09-15T22:58:50.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:49 smithi121 ceph-mon[34815]: pgmap v737: 97 pgs: 97 active+clean; 279 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 235 op/s 2024-09-15T22:58:50.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:49 smithi192 ceph-mon[35427]: pgmap v737: 97 pgs: 97 active+clean; 279 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 235 op/s 2024-09-15T22:58:50.328 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1247271 9.81 MB/sec execute 558 sec latency 2124.947 ms 2024-09-15T22:58:50.977 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:50 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:58:51.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:50 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:58:51.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:50 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:58:51.328 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1248092 9.80 MB/sec execute 559 sec latency 2584.477 ms 2024-09-15T22:58:52.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:51 smithi121 ceph-mon[34815]: pgmap v738: 97 pgs: 97 active+clean; 282 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 180 op/s 2024-09-15T22:58:52.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:51 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:58:52.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:51 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:58:52.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:51 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:58:52.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:51 smithi111 ceph-mon[30629]: pgmap v738: 97 pgs: 97 active+clean; 282 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 180 op/s 2024-09-15T22:58:52.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:51 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:58:52.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:51 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:58:52.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:51 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:58:52.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:51 smithi192 ceph-mon[35427]: pgmap v738: 97 pgs: 97 active+clean; 282 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 180 op/s 2024-09-15T22:58:52.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:51 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:58:52.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:51 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:58:52.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:51 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:58:52.328 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1250680 9.81 MB/sec execute 560 sec latency 15.286 ms 2024-09-15T22:58:53.328 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1253653 9.82 MB/sec execute 561 sec latency 42.028 ms 2024-09-15T22:58:54.135 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:53 smithi111 ceph-mon[30629]: pgmap v739: 97 pgs: 97 active+clean; 282 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 180 op/s 2024-09-15T22:58:54.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:53 smithi121 ceph-mon[34815]: pgmap v739: 97 pgs: 97 active+clean; 282 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 180 op/s 2024-09-15T22:58:54.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:53 smithi192 ceph-mon[35427]: pgmap v739: 97 pgs: 97 active+clean; 282 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 180 op/s 2024-09-15T22:58:54.328 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1253658 9.81 MB/sec execute 562 sec latency 936.805 ms 2024-09-15T22:58:55.328 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1254253 9.79 MB/sec execute 563 sec latency 1508.871 ms 2024-09-15T22:58:56.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:55 smithi121 ceph-mon[34815]: pgmap v740: 97 pgs: 97 active+clean; 286 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 200 op/s 2024-09-15T22:58:56.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:55 smithi111 ceph-mon[30629]: pgmap v740: 97 pgs: 97 active+clean; 286 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 200 op/s 2024-09-15T22:58:56.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:55 smithi192 ceph-mon[35427]: pgmap v740: 97 pgs: 97 active+clean; 286 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 200 op/s 2024-09-15T22:58:56.328 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1257347 9.80 MB/sec execute 564 sec latency 39.597 ms 2024-09-15T22:58:57.328 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1260101 9.81 MB/sec execute 565 sec latency 18.542 ms 2024-09-15T22:58:58.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:57 smithi121 ceph-mon[34815]: pgmap v741: 97 pgs: 97 active+clean; 281 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-09-15T22:58:58.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:57 smithi111 ceph-mon[30629]: pgmap v741: 97 pgs: 97 active+clean; 281 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-09-15T22:58:58.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:57 smithi192 ceph-mon[35427]: pgmap v741: 97 pgs: 97 active+clean; 281 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-09-15T22:58:58.329 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1261682 9.81 MB/sec execute 566 sec latency 94.418 ms 2024-09-15T22:58:59.328 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1261891 9.79 MB/sec execute 567 sec latency 792.403 ms 2024-09-15T22:58:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.361+0000 7f3e0b600640 1 -- 172.21.15.111:0/2830501292 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e04103d70 msgr2=0x7f3e04104170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.361+0000 7f3e0b600640 1 --2- 172.21.15.111:0/2830501292 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e04103d70 0x7f3e04104170 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3df8009900 tx=0x7f3df802a3f0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.361+0000 7f3e0b600640 1 -- 172.21.15.111:0/2830501292 shutdown_connections 2024-09-15T22:58:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.361+0000 7f3e0b600640 1 --2- 172.21.15.111:0/2830501292 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e04105920 0x7f3e0410c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.361+0000 7f3e0b600640 1 --2- 172.21.15.111:0/2830501292 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e04104f60 0x7f3e041053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.361+0000 7f3e0b600640 1 --2- 172.21.15.111:0/2830501292 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e04103d70 0x7f3e04104170 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.361+0000 7f3e0b600640 1 -- 172.21.15.111:0/2830501292 >> 172.21.15.111:0/2830501292 conn(0x7f3e040ff4d0 msgr2=0x7f3e04101910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.361+0000 7f3e0b600640 1 -- 172.21.15.111:0/2830501292 shutdown_connections 2024-09-15T22:58:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.361+0000 7f3e0b600640 1 -- 172.21.15.111:0/2830501292 wait complete. 2024-09-15T22:58:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.362+0000 7f3e0b600640 1 Processor -- start 2024-09-15T22:58:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.362+0000 7f3e0b600640 1 -- start start 2024-09-15T22:58:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.362+0000 7f3e0b600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e04103d70 0x7f3e04195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.362+0000 7f3e0b600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e04104f60 0x7f3e04196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.362+0000 7f3e0b600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e04105920 0x7f3e0419d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.362+0000 7f3e0b600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3e040792b0 con 0x7f3e04104f60 2024-09-15T22:58:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.362+0000 7f3e0b600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3e04079130 con 0x7f3e04105920 2024-09-15T22:58:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.362+0000 7f3e0b600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3e04079430 con 0x7f3e04103d70 2024-09-15T22:58:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.362+0000 7f3e08c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e04103d70 0x7f3e04195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.362+0000 7f3e08c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e04103d70 0x7f3e04195c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:51422/0 (socket says 172.21.15.111:51422) 2024-09-15T22:58:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.362+0000 7f3e08c00640 1 -- 172.21.15.111:0/3062055210 learned_addr learned my addr 172.21.15.111:0/3062055210 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:58:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.362+0000 7f3e09600640 1 --2- 172.21.15.111:0/3062055210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e04105920 0x7f3e0419d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.362+0000 7f3e03e00640 1 --2- 172.21.15.111:0/3062055210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e04104f60 0x7f3e04196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.362+0000 7f3e08c00640 1 -- 172.21.15.111:0/3062055210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e04105920 msgr2=0x7f3e0419d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.362+0000 7f3e08c00640 1 --2- 172.21.15.111:0/3062055210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e04105920 0x7f3e0419d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.362+0000 7f3e08c00640 1 -- 172.21.15.111:0/3062055210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e04104f60 msgr2=0x7f3e04196170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.362+0000 7f3e08c00640 1 --2- 172.21.15.111:0/3062055210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e04104f60 0x7f3e04196170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.362+0000 7f3e08c00640 1 -- 172.21.15.111:0/3062055210 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e0419d840 con 0x7f3e04103d70 2024-09-15T22:58:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.362+0000 7f3e08c00640 1 --2- 172.21.15.111:0/3062055210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e04103d70 0x7f3e04195c30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3df8031440 tx=0x7f3df8032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.362+0000 7f3e01600640 1 -- 172.21.15.111:0/3062055210 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df803f030 con 0x7f3e04103d70 2024-09-15T22:58:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.362+0000 7f3e0b600640 1 -- 172.21.15.111:0/3062055210 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e0419dad0 con 0x7f3e04103d70 2024-09-15T22:58:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.362+0000 7f3e01600640 1 -- 172.21.15.111:0/3062055210 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3df8003ce0 con 0x7f3e04103d70 2024-09-15T22:58:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.362+0000 7f3e01600640 1 -- 172.21.15.111:0/3062055210 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8035070 con 0x7f3e04103d70 2024-09-15T22:58:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.362+0000 7f3e0b600640 1 -- 172.21.15.111:0/3062055210 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e0419e010 con 0x7f3e04103d70 2024-09-15T22:58:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.366+0000 7f3e0e3c2740 1 Processor -- start 2024-09-15T22:58:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.366+0000 7f3e01600640 1 -- 172.21.15.111:0/3062055210 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f3df8042030 con 0x7f3e04103d70 2024-09-15T22:58:59.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.366+0000 7f3e0e3c2740 1 -- start start 2024-09-15T22:58:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.366+0000 7f3e0e3c2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557229db9090 0x557229db9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.366+0000 7f3e0e3c2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557229db99d0 0x557229dbe490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.366+0000 7f3e0e3c2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557229dbea60 0x557229dc0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.366+0000 7f3e0e3c2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557229d1e100 con 0x557229dbea60 2024-09-15T22:58:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.366+0000 7f3e0e3c2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x557229cc61b0 con 0x557229db99d0 2024-09-15T22:58:59.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.366+0000 7f3e0e3c2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557229d1a0c0 con 0x557229db9090 2024-09-15T22:58:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.366+0000 7f3e09600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557229db9090 0x557229db9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.366+0000 7f3e09600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557229db9090 0x557229db9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:51434/0 (socket says 172.21.15.111:51434) 2024-09-15T22:58:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.366+0000 7f3e09600640 1 -- 172.21.15.111:0/2939103257 learned_addr learned my addr 172.21.15.111:0/2939103257 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:58:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.366+0000 7f3e03e00640 1 --2- 172.21.15.111:0/2939103257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557229dbea60 0x557229dc0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.366+0000 7f3e01600640 1 --2- 172.21.15.111:0/3062055210 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3dec0a3370 0x7f3dec0a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.366+0000 7f3e01600640 1 -- 172.21.15.111:0/3062055210 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f3df80eff00 con 0x7f3e04103d70 2024-09-15T22:58:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e08c00640 1 --2- 172.21.15.111:0/2939103257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557229db99d0 0x557229dbe490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e09600640 1 --2- 172.21.15.111:0/3062055210 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3dec0a3370 0x7f3dec0a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e03e00640 1 -- 172.21.15.111:0/2939103257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557229db9090 msgr2=0x557229db9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e03e00640 1 --2- 172.21.15.111:0/2939103257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557229db9090 0x557229db9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e03e00640 1 -- 172.21.15.111:0/2939103257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557229db99d0 msgr2=0x557229dbe490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e03e00640 1 --2- 172.21.15.111:0/2939103257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557229db99d0 0x557229dbe490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e03e00640 1 -- 172.21.15.111:0/2939103257 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557229a31210 con 0x557229dbea60 2024-09-15T22:58:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e03e00640 1 --2- 172.21.15.111:0/2939103257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557229dbea60 0x557229dc0f20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3df00043e0 tx=0x7f3df0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3439bb56a462ba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3df5c00640 1 -- 172.21.15.111:0/2939103257 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df0022030 con 0x557229dbea60 2024-09-15T22:58:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e09600640 1 --2- 172.21.15.111:0/3062055210 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3dec0a3370 0x7f3dec0a5830 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc005f40 tx=0x7f3dfc004ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3df5c00640 1 -- 172.21.15.111:0/2939103257 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3df0005500 con 0x557229dbea60 2024-09-15T22:58:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3df5c00640 1 -- 172.21.15.111:0/2939103257 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df0022030 con 0x557229dbea60 2024-09-15T22:58:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e0e3c2740 1 -- 172.21.15.111:0/2939103257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557229dbea60 msgr2=0x557229dc0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e0e3c2740 1 --2- 172.21.15.111:0/2939103257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557229dbea60 0x557229dc0f20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3df00043e0 tx=0x7f3df0005f90 comp rx=0 tx=0).stop 2024-09-15T22:58:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e0e3c2740 1 -- 172.21.15.111:0/2939103257 shutdown_connections 2024-09-15T22:58:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e0e3c2740 1 --2- 172.21.15.111:0/2939103257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557229dbea60 0x557229dc0f20 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e0e3c2740 1 --2- 172.21.15.111:0/2939103257 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557229db99d0 0x557229dbe490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e0e3c2740 1 --2- 172.21.15.111:0/2939103257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557229db9090 0x557229db9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e0e3c2740 1 -- 172.21.15.111:0/2939103257 >> 172.21.15.111:0/2939103257 conn(0x557229db4700 msgr2=0x557229db4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e0e3c2740 1 -- 172.21.15.111:0/2939103257 shutdown_connections 2024-09-15T22:58:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e0e3c2740 1 -- 172.21.15.111:0/2939103257 wait complete. 2024-09-15T22:58:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e0e3c2740 1 -- start start 2024-09-15T22:58:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e0e3c2740 1 Processor -- start 2024-09-15T22:58:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e0e3c2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557229db9090 0x557229dc3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e0e3c2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557229db99d0 0x557229dc36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e0e3c2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557229dbea60 0x557229dca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e0e3c2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557229dc1a30 con 0x557229dbea60 2024-09-15T22:58:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e0e3c2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x557229ca7090 con 0x557229db99d0 2024-09-15T22:58:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e0e3c2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557229dc1bb0 con 0x557229db9090 2024-09-15T22:58:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e08c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557229db9090 0x557229dc3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e0e3c2740 10 client.0 _ll_register_callbacks cb 0x557229cfbf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:58:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e03e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557229db99d0 0x557229dc36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e08c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557229db9090 0x557229dc3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:51448/0 (socket says 172.21.15.111:51448) 2024-09-15T22:58:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e08c00640 1 -- 172.21.15.111:0/291793032 learned_addr learned my addr 172.21.15.111:0/291793032 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:58:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.371+0000 7f3e09600640 1 --2- 172.21.15.111:0/291793032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557229dbea60 0x557229dca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.376+0000 7f3e08c00640 1 -- 172.21.15.111:0/291793032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557229db99d0 msgr2=0x557229dc36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.376+0000 7f3e08c00640 1 --2- 172.21.15.111:0/291793032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557229db99d0 0x557229dc36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.376+0000 7f3e08c00640 1 -- 172.21.15.111:0/291793032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557229dbea60 msgr2=0x557229dca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.376+0000 7f3e08c00640 1 --2- 172.21.15.111:0/291793032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557229dbea60 0x557229dca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.376+0000 7f3e08c00640 1 -- 172.21.15.111:0/291793032 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557229afd620 con 0x557229db9090 2024-09-15T22:58:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.376+0000 7f3e09600640 1 --2- 172.21.15.111:0/291793032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557229dbea60 0x557229dca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:58:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.376+0000 7f3e08c00640 1 --2- 172.21.15.111:0/291793032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557229db9090 0x557229dc3180 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3df802a910 tx=0x7f3df80bc4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=120bfac1d8c8163e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.376+0000 7f3dea000640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:58:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.376+0000 7f3dea000640 1 -- 172.21.15.111:0/291793032 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8049040 con 0x557229db9090 2024-09-15T22:58:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.376+0000 7f3dea000640 1 -- 172.21.15.111:0/291793032 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f3df803bd10 con 0x557229db9090 2024-09-15T22:58:59.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.376+0000 7f3e0e3c2740 1 -- 172.21.15.111:0/291793032 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3df0005500 con 0x557229db9090 2024-09-15T22:58:59.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.376+0000 7f3dea000640 1 -- 172.21.15.111:0/291793032 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3df8005d50 con 0x557229db9090 2024-09-15T22:58:59.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.376+0000 7f3e0e3c2740 10 client.26032 fetch_fsmap learned FSMap version 30 2024-09-15T22:58:59.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.376+0000 7f3e0e3c2740 1 -- 172.21.15.111:0/291793032 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557229dc1aa0 con 0x557229db9090 2024-09-15T22:58:59.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.376+0000 7f3dea000640 1 -- 172.21.15.111:0/291793032 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3df0005500 con 0x557229db9090 2024-09-15T22:58:59.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.376+0000 7f3dea000640 1 -- 172.21.15.111:0/291793032 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f3df8003800 con 0x557229db9090 2024-09-15T22:58:59.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.376+0000 7f3e0e3c2740 10 client.26032 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:58:59.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.376+0000 7f3e0e3c2740 10 client.26032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:58:59.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.376+0000 7f3e0e3c2740 20 client.26032 populate_metadata read hostname 'smithi111' 2024-09-15T22:58:59.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.376+0000 7f3e0e3c2740 1 --2- 172.21.15.111:0/291793032 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557229ddfc00 0x557229de2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:59.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.376+0000 7f3e0e3c2740 4 client.26032 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:58:59.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.376+0000 7f3e0e3c2740 1 -- 172.21.15.111:0/291793032 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557229d2b830 con 0x557229ddfc00 2024-09-15T22:58:59.384 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.376+0000 7f3e03e00640 1 --2- 172.21.15.111:0/291793032 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557229ddfc00 0x557229de2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:59.384 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.379+0000 7f3dea000640 10 client.26032 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:58:59.384 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.379+0000 7f3e03e00640 1 --2- 172.21.15.111:0/291793032 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557229ddfc00 0x557229de2030 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf593883f7b8b39f server_cookie=300cc72f6d9f96d1 in_seq=0 out_seq=0 2024-09-15T22:58:59.384 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.379+0000 7f3dea000640 1 -- 172.21.15.111:0/291793032 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557229d2b830 con 0x557229ddfc00 2024-09-15T22:58:59.384 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.379+0000 7f3dea000640 10 client.26032 handle_command_reply: tid=0 2024-09-15T22:58:59.384 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.379+0000 7f3e0e3c2740 1 client.26032 shutdown 2024-09-15T22:58:59.384 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.379+0000 7f3e0e3c2740 1 -- 172.21.15.111:0/291793032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557229db9090 msgr2=0x557229dc3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:59.384 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.379+0000 7f3e0e3c2740 1 --2- 172.21.15.111:0/291793032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557229db9090 0x557229dc3180 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3df802a910 tx=0x7f3df80bc4d0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.384 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.379+0000 7f3e0e3c2740 1 -- 172.21.15.111:0/291793032 shutdown_connections 2024-09-15T22:58:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.379+0000 7f3e0e3c2740 1 --2- 172.21.15.111:0/291793032 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557229ddfc00 0x557229de2030 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.379+0000 7f3e0e3c2740 1 --2- 172.21.15.111:0/291793032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557229dbea60 0x557229dca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.379+0000 7f3e0e3c2740 1 --2- 172.21.15.111:0/291793032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557229db99d0 0x557229dc36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.379+0000 7f3e0e3c2740 1 --2- 172.21.15.111:0/291793032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557229db9090 0x557229dc3180 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.379+0000 7f3e0e3c2740 1 -- 172.21.15.111:0/291793032 >> 172.21.15.111:0/291793032 conn(0x557229db4700 msgr2=0x557229db5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.379+0000 7f3dea000640 0 client.26032 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:58:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.379+0000 7f3e0e3c2740 1 -- 172.21.15.111:0/291793032 shutdown_connections 2024-09-15T22:58:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.379+0000 7f3e0e3c2740 1 -- 172.21.15.111:0/291793032 wait complete. 2024-09-15T22:58:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.379+0000 7f3e0e3c2740 20 client.26032 trim_cache size 0 max 16384 2024-09-15T22:58:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e0e3c2740 1 Processor -- start 2024-09-15T22:58:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e0e3c2740 1 -- start start 2024-09-15T22:58:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e0e3c2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557229db9090 0x557229dc3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e0e3c2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557229db99d0 0x557229dca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e0e3c2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557229dbea60 0x557229db55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e0e3c2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041a9480 con 0x557229db99d0 2024-09-15T22:58:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e0e3c2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041a6d50 con 0x557229dbea60 2024-09-15T22:58:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e0e3c2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041a4ba0 con 0x557229db9090 2024-09-15T22:58:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e08c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557229db9090 0x557229dc3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e09600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557229dbea60 0x557229db55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e09600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557229dbea60 0x557229db55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:50356/0 (socket says 172.21.15.111:50356) 2024-09-15T22:58:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e09600640 1 -- 172.21.15.111:0/3012001347 learned_addr learned my addr 172.21.15.111:0/3012001347 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:58:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e03e00640 1 --2- 172.21.15.111:0/3012001347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557229db99d0 0x557229dca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e09600640 1 -- 172.21.15.111:0/3012001347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557229db9090 msgr2=0x557229dc3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e09600640 1 --2- 172.21.15.111:0/3012001347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557229db9090 0x557229dc3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e09600640 1 -- 172.21.15.111:0/3012001347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557229db99d0 msgr2=0x557229dca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e09600640 1 --2- 172.21.15.111:0/3012001347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557229db99d0 0x557229dca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e09600640 1 -- 172.21.15.111:0/3012001347 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557229db5390 con 0x557229dbea60 2024-09-15T22:58:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e03e00640 1 --2- 172.21.15.111:0/3012001347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557229db99d0 0x557229dca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:58:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e08c00640 1 --2- 172.21.15.111:0/3012001347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557229db9090 0x557229dc3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:58:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e09600640 1 --2- 172.21.15.111:0/3012001347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557229dbea60 0x557229db55b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc003c80 tx=0x7f3dfc009a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d409b640c3155bc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3df5200640 1 -- 172.21.15.111:0/3012001347 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc012db0 con 0x557229dbea60 2024-09-15T22:58:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3df5200640 1 -- 172.21.15.111:0/3012001347 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dfc026070 con 0x557229dbea60 2024-09-15T22:58:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3df5200640 1 -- 172.21.15.111:0/3012001347 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc02b9f0 con 0x557229dbea60 2024-09-15T22:58:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e0e3c2740 1 -- 172.21.15.111:0/3012001347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557229dbea60 msgr2=0x557229db55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e0e3c2740 1 --2- 172.21.15.111:0/3012001347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557229dbea60 0x557229db55b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc003c80 tx=0x7f3dfc009a70 comp rx=0 tx=0).stop 2024-09-15T22:58:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e0e3c2740 1 -- 172.21.15.111:0/3012001347 shutdown_connections 2024-09-15T22:58:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e0e3c2740 1 --2- 172.21.15.111:0/3012001347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557229dbea60 0x557229db55b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e0e3c2740 1 --2- 172.21.15.111:0/3012001347 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557229db99d0 0x557229dca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e0e3c2740 1 --2- 172.21.15.111:0/3012001347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557229db9090 0x557229dc3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e0e3c2740 1 -- 172.21.15.111:0/3012001347 >> 172.21.15.111:0/3012001347 conn(0x557229db4700 msgr2=0x557229dc2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.403+0000 7f3e0e3c2740 1 -- 172.21.15.111:0/3012001347 shutdown_connections 2024-09-15T22:58:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3e0e3c2740 1 -- 172.21.15.111:0/3012001347 wait complete. 2024-09-15T22:58:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3e0e3c2740 1 -- start start 2024-09-15T22:58:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3e0e3c2740 1 Processor -- start 2024-09-15T22:58:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3e0e3c2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557229db9090 0x557229dc78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3e0e3c2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557229db99d0 0x557229dc7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3e0e3c2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557229dbea60 0x557229dc8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3e0e3c2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041a5210 con 0x557229db99d0 2024-09-15T22:58:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3e0e3c2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041a6680 con 0x557229db9090 2024-09-15T22:58:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3e0e3c2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3e041a58e0 con 0x557229dbea60 2024-09-15T22:58:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3e0e3c2740 10 client.0 _ll_register_callbacks cb 0x7f3df80095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:58:59.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3e03e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557229db99d0 0x557229dc7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3e03e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557229db99d0 0x557229dc7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:54636/0 (socket says 172.21.15.111:54636) 2024-09-15T22:58:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3e03e00640 1 -- 172.21.15.111:0/3593458628 learned_addr learned my addr 172.21.15.111:0/3593458628 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:58:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3e09600640 1 --2- 172.21.15.111:0/3593458628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557229dbea60 0x557229dc8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3e08c00640 1 --2- 172.21.15.111:0/3593458628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557229db9090 0x557229dc78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3e03e00640 1 -- 172.21.15.111:0/3593458628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557229dbea60 msgr2=0x557229dc8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3e03e00640 1 --2- 172.21.15.111:0/3593458628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557229dbea60 0x557229dc8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3e03e00640 1 -- 172.21.15.111:0/3593458628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557229db9090 msgr2=0x557229dc78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:58:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3e03e00640 1 --2- 172.21.15.111:0/3593458628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557229db9090 0x557229dc78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3e03e00640 1 -- 172.21.15.111:0/3593458628 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557229dc9450 con 0x557229db99d0 2024-09-15T22:58:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3e08c00640 1 --2- 172.21.15.111:0/3593458628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557229db9090 0x557229dc78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:58:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3e03e00640 1 --2- 172.21.15.111:0/3593458628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557229db99d0 0x557229dc7e30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x557229db5580 tx=0x7f3df0004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b24433346f6edd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:58:59.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3dea000640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T22:58:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3dea000640 1 -- 172.21.15.111:0/3593458628 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df001e530 con 0x557229db99d0 2024-09-15T22:58:59.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3dea000640 1 -- 172.21.15.111:0/3593458628 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f3df001e6d0 con 0x557229db99d0 2024-09-15T22:58:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3e0e3c2740 1 -- 172.21.15.111:0/3593458628 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dfc026070 con 0x557229db99d0 2024-09-15T22:58:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3dea000640 1 -- 172.21.15.111:0/3593458628 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3df001e8d0 con 0x557229db99d0 2024-09-15T22:58:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3e0e3c2740 10 client.16047 fetch_fsmap learned FSMap version 30 2024-09-15T22:58:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3e0e3c2740 1 -- 172.21.15.111:0/3593458628 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557229dca760 con 0x557229db99d0 2024-09-15T22:58:59.416 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.408+0000 7f3dea000640 1 -- 172.21.15.111:0/3593458628 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dfc026070 con 0x557229db99d0 2024-09-15T22:58:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.413+0000 7f3dea000640 1 -- 172.21.15.111:0/3593458628 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f3df000f040 con 0x557229db99d0 2024-09-15T22:58:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.413+0000 7f3e0e3c2740 10 client.16047 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:58:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.413+0000 7f3e0e3c2740 10 client.16047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:58:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.413+0000 7f3e0e3c2740 20 client.16047 populate_metadata read hostname 'smithi111' 2024-09-15T22:58:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.413+0000 7f3e0e3c2740 1 --2- 172.21.15.111:0/3593458628 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557229e4aad0 0x557229e4cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:58:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.413+0000 7f3e0e3c2740 4 client.16047 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:58:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.413+0000 7f3e0e3c2740 1 -- 172.21.15.111:0/3593458628 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557229d25b80 con 0x557229e4aad0 2024-09-15T22:58:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.413+0000 7f3e08c00640 1 --2- 172.21.15.111:0/3593458628 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557229e4aad0 0x557229e4cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:58:59.417 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.413+0000 7f3e08c00640 1 --2- 172.21.15.111:0/3593458628 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557229e4aad0 0x557229e4cec0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6d65a516a318efa server_cookie=c8a376a224f88f1f in_seq=0 out_seq=0 2024-09-15T22:58:59.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.413+0000 7f3dea000640 10 client.16047 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:58:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.427+0000 7f3dea000640 1 -- 172.21.15.111:0/3593458628 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557229d25b80 con 0x557229e4aad0 2024-09-15T22:58:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.427+0000 7f3dea000640 10 client.16047 handle_command_reply: tid=0 2024-09-15T22:58:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.427+0000 7f3e0e3c2740 1 client.16047 shutdown 2024-09-15T22:58:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.427+0000 7f3e0e3c2740 1 -- 172.21.15.111:0/3593458628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557229db99d0 msgr2=0x557229dc7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:59.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.427+0000 7f3e0e3c2740 1 --2- 172.21.15.111:0/3593458628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557229db99d0 0x557229dc7e30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x557229db5580 tx=0x7f3df0004130 comp rx=0 tx=0).stop 2024-09-15T22:58:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.427+0000 7f3e0e3c2740 1 -- 172.21.15.111:0/3593458628 shutdown_connections 2024-09-15T22:58:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.427+0000 7f3e0e3c2740 1 --2- 172.21.15.111:0/3593458628 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557229e4aad0 0x557229e4cec0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.427+0000 7f3e0e3c2740 1 --2- 172.21.15.111:0/3593458628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557229dbea60 0x557229dc8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.427+0000 7f3e0e3c2740 1 --2- 172.21.15.111:0/3593458628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557229db99d0 0x557229dc7e30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.427+0000 7f3e0e3c2740 1 --2- 172.21.15.111:0/3593458628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557229db9090 0x557229dc78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.427+0000 7f3e0e3c2740 1 -- 172.21.15.111:0/3593458628 >> 172.21.15.111:0/3593458628 conn(0x557229db4700 msgr2=0x557229dc2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.427+0000 7f3dea000640 0 client.16047 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:58:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.427+0000 7f3e0e3c2740 1 -- 172.21.15.111:0/3593458628 shutdown_connections 2024-09-15T22:58:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.427+0000 7f3e0e3c2740 1 -- 172.21.15.111:0/3593458628 wait complete. 2024-09-15T22:58:59.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.427+0000 7f3e0e3c2740 20 client.16047 trim_cache size 0 max 16384 2024-09-15T22:58:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:58:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:58:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:58:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:58:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.432+0000 7f3e0b600640 1 -- 172.21.15.111:0/3062055210 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3dec0a3370 msgr2=0x7f3dec0a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.432+0000 7f3e0b600640 1 --2- 172.21.15.111:0/3062055210 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3dec0a3370 0x7f3dec0a5830 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc005f40 tx=0x7f3dfc004ab0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.432+0000 7f3e0b600640 1 -- 172.21.15.111:0/3062055210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e04103d70 msgr2=0x7f3e04195c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:58:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.432+0000 7f3e0b600640 1 --2- 172.21.15.111:0/3062055210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e04103d70 0x7f3e04195c30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3df8031440 tx=0x7f3df8032ce0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.432+0000 7f3e0b600640 1 -- 172.21.15.111:0/3062055210 shutdown_connections 2024-09-15T22:58:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.432+0000 7f3e0b600640 1 --2- 172.21.15.111:0/3062055210 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3dec0a3370 0x7f3dec0a5830 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.432+0000 7f3e0b600640 1 --2- 172.21.15.111:0/3062055210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e04105920 0x7f3e0419d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.432+0000 7f3e0b600640 1 --2- 172.21.15.111:0/3062055210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e04104f60 0x7f3e04196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.432+0000 7f3e0b600640 1 --2- 172.21.15.111:0/3062055210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e04103d70 0x7f3e04195c30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:58:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.432+0000 7f3e0b600640 1 -- 172.21.15.111:0/3062055210 >> 172.21.15.111:0/3062055210 conn(0x7f3e040ff4d0 msgr2=0x7f3e040ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:58:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.432+0000 7f3e0b600640 1 -- 172.21.15.111:0/3062055210 shutdown_connections 2024-09-15T22:58:59.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:58:59.432+0000 7f3e0b600640 1 -- 172.21.15.111:0/3062055210 wait complete. 2024-09-15T22:59:00.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:58:59 smithi111 ceph-mon[30629]: pgmap v742: 97 pgs: 97 active+clean; 284 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 209 op/s 2024-09-15T22:59:00.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:58:59 smithi192 ceph-mon[35427]: pgmap v742: 97 pgs: 97 active+clean; 284 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 209 op/s 2024-09-15T22:59:00.328 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1262598 9.78 MB/sec execute 568 sec latency 1589.009 ms 2024-09-15T22:59:00.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:58:59 smithi121 ceph-mon[34815]: pgmap v742: 97 pgs: 97 active+clean; 284 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 209 op/s 2024-09-15T22:59:01.329 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1265166 9.79 MB/sec execute 569 sec latency 13.073 ms 2024-09-15T22:59:02.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:01 smithi111 ceph-mon[30629]: pgmap v743: 97 pgs: 97 active+clean; 283 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 166 op/s 2024-09-15T22:59:02.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:01 smithi192 ceph-mon[35427]: pgmap v743: 97 pgs: 97 active+clean; 283 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 166 op/s 2024-09-15T22:59:02.329 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1268218 9.80 MB/sec execute 570 sec latency 8.995 ms 2024-09-15T22:59:02.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:01 smithi121 ceph-mon[34815]: pgmap v743: 97 pgs: 97 active+clean; 283 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 166 op/s 2024-09-15T22:59:03.329 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1271395 9.82 MB/sec execute 571 sec latency 13.242 ms 2024-09-15T22:59:04.076 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:03 smithi111 ceph-mon[30629]: pgmap v744: 97 pgs: 97 active+clean; 283 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 171 op/s 2024-09-15T22:59:04.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:03 smithi192 ceph-mon[35427]: pgmap v744: 97 pgs: 97 active+clean; 283 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 171 op/s 2024-09-15T22:59:04.336 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1271683 9.80 MB/sec execute 572 sec latency 733.262 ms 2024-09-15T22:59:04.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:03 smithi121 ceph-mon[34815]: pgmap v744: 97 pgs: 97 active+clean; 283 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 171 op/s 2024-09-15T22:59:05.336 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1272121 9.79 MB/sec execute 573 sec latency 1385.348 ms 2024-09-15T22:59:06.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:05 smithi111 ceph-mon[30629]: pgmap v745: 97 pgs: 97 active+clean; 285 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 181 op/s 2024-09-15T22:59:06.192 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:05 smithi121 ceph-mon[34815]: pgmap v745: 97 pgs: 97 active+clean; 285 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 181 op/s 2024-09-15T22:59:06.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:05 smithi192 ceph-mon[35427]: pgmap v745: 97 pgs: 97 active+clean; 285 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 181 op/s 2024-09-15T22:59:06.338 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1275255 9.80 MB/sec execute 574 sec latency 25.489 ms 2024-09-15T22:59:07.336 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1277909 9.81 MB/sec execute 575 sec latency 83.277 ms 2024-09-15T22:59:08.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:07 smithi111 ceph-mon[30629]: pgmap v746: 97 pgs: 97 active+clean; 288 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 194 op/s 2024-09-15T22:59:08.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:07 smithi192 ceph-mon[35427]: pgmap v746: 97 pgs: 97 active+clean; 288 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 194 op/s 2024-09-15T22:59:08.336 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1279905 9.81 MB/sec execute 576 sec latency 19.353 ms 2024-09-15T22:59:08.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:07 smithi121 ceph-mon[34815]: pgmap v746: 97 pgs: 97 active+clean; 288 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 194 op/s 2024-09-15T22:59:09.336 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1280978 9.80 MB/sec execute 577 sec latency 664.874 ms 2024-09-15T22:59:10.070 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:09 smithi111 ceph-mon[30629]: pgmap v747: 97 pgs: 97 active+clean; 287 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 191 op/s 2024-09-15T22:59:10.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:09 smithi192 ceph-mon[35427]: pgmap v747: 97 pgs: 97 active+clean; 287 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 191 op/s 2024-09-15T22:59:10.336 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1280979 9.78 MB/sec execute 578 sec latency 868.495 ms 2024-09-15T22:59:10.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:09 smithi121 ceph-mon[34815]: pgmap v747: 97 pgs: 97 active+clean; 287 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 191 op/s 2024-09-15T22:59:11.336 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1282911 9.79 MB/sec execute 579 sec latency 956.946 ms 2024-09-15T22:59:12.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:11 smithi192 ceph-mon[35427]: pgmap v748: 97 pgs: 97 active+clean; 292 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 184 op/s 2024-09-15T22:59:12.337 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1285782 9.80 MB/sec execute 580 sec latency 16.537 ms 2024-09-15T22:59:12.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:11 smithi121 ceph-mon[34815]: pgmap v748: 97 pgs: 97 active+clean; 292 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 184 op/s 2024-09-15T22:59:12.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:11 smithi111 ceph-mon[30629]: pgmap v748: 97 pgs: 97 active+clean; 292 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 184 op/s 2024-09-15T22:59:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.555+0000 7f6c71400640 1 -- 172.21.15.111:0/1873425383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c6c104f50 msgr2=0x7f6c6c1053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.555+0000 7f6c71400640 1 --2- 172.21.15.111:0/1873425383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c6c104f50 0x7f6c6c1053d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009860 tx=0x7f6c5802a340 comp rx=0 tx=0).stop 2024-09-15T22:59:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.555+0000 7f6c71400640 1 -- 172.21.15.111:0/1873425383 shutdown_connections 2024-09-15T22:59:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.555+0000 7f6c71400640 1 --2- 172.21.15.111:0/1873425383 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c6c105910 0x7f6c6c10c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.555+0000 7f6c71400640 1 --2- 172.21.15.111:0/1873425383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c6c104f50 0x7f6c6c1053d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.555+0000 7f6c71400640 1 --2- 172.21.15.111:0/1873425383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c6c103d60 0x7f6c6c104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.555+0000 7f6c71400640 1 -- 172.21.15.111:0/1873425383 >> 172.21.15.111:0/1873425383 conn(0x7f6c6c0ff4e0 msgr2=0x7f6c6c101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:59:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.555+0000 7f6c71400640 1 -- 172.21.15.111:0/1873425383 shutdown_connections 2024-09-15T22:59:12.558 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.556+0000 7f6c71400640 1 -- 172.21.15.111:0/1873425383 wait complete. 2024-09-15T22:59:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.556+0000 7f6c71400640 1 Processor -- start 2024-09-15T22:59:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.557+0000 7f6c71400640 1 -- start start 2024-09-15T22:59:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.557+0000 7f6c71400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c6c103d60 0x7f6c6c199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.557+0000 7f6c71400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c6c104f50 0x7f6c6c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.557+0000 7f6c71400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c6c105910 0x7f6c6c1a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.557+0000 7f6c71400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6c10f9e0 con 0x7f6c6c104f50 2024-09-15T22:59:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.557+0000 7f6c71400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6c10f860 con 0x7f6c6c103d60 2024-09-15T22:59:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.557+0000 7f6c71400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6c10fb60 con 0x7f6c6c105910 2024-09-15T22:59:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.557+0000 7f6c6a800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c6c103d60 0x7f6c6c199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.557+0000 7f6c69e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c6c104f50 0x7f6c6c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.557+0000 7f6c6a800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c6c103d60 0x7f6c6c199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:58456/0 (socket says 172.21.15.111:58456) 2024-09-15T22:59:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.557+0000 7f6c6a800640 1 -- 172.21.15.111:0/1184539139 learned_addr learned my addr 172.21.15.111:0/1184539139 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:59:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.557+0000 7f6c6b200640 1 --2- 172.21.15.111:0/1184539139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c6c105910 0x7f6c6c1a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.557+0000 7f6c6a800640 1 -- 172.21.15.111:0/1184539139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c6c105910 msgr2=0x7f6c6c1a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.557+0000 7f6c6a800640 1 --2- 172.21.15.111:0/1184539139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c6c105910 0x7f6c6c1a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.557+0000 7f6c6a800640 1 -- 172.21.15.111:0/1184539139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c6c104f50 msgr2=0x7f6c6c19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.557+0000 7f6c6a800640 1 --2- 172.21.15.111:0/1184539139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c6c104f50 0x7f6c6c19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.557+0000 7f6c6a800640 1 -- 172.21.15.111:0/1184539139 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c6c1a1b90 con 0x7f6c6c103d60 2024-09-15T22:59:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.558+0000 7f6c6a800640 1 --2- 172.21.15.111:0/1184539139 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c6c103d60 0x7f6c6c199f70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00e9f0 tx=0x7f6c5c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:59:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.558+0000 7f6c63400640 1 -- 172.21.15.111:0/1184539139 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c00ccc0 con 0x7f6c6c103d60 2024-09-15T22:59:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.558+0000 7f6c71400640 1 -- 172.21.15.111:0/1184539139 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c6c1a1e80 con 0x7f6c6c103d60 2024-09-15T22:59:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.558+0000 7f6c63400640 1 -- 172.21.15.111:0/1184539139 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c5c0035f0 con 0x7f6c6c103d60 2024-09-15T22:59:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.558+0000 7f6c63400640 1 -- 172.21.15.111:0/1184539139 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5c00ce60 con 0x7f6c6c103d60 2024-09-15T22:59:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.558+0000 7f6c71400640 1 -- 172.21.15.111:0/1184539139 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c6c1a22f0 con 0x7f6c6c103d60 2024-09-15T22:59:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.561+0000 7f6c743e8740 1 Processor -- start 2024-09-15T22:59:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.561+0000 7f6c743e8740 1 -- start start 2024-09-15T22:59:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.562+0000 7f6c743e8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f6c1938a0 0x556f6c193ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.562+0000 7f6c743e8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f6c1941e0 0x556f6c198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.562+0000 7f6c63400640 1 -- 172.21.15.111:0/1184539139 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f6c5c0037a0 con 0x7f6c6c103d60 2024-09-15T22:59:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.562+0000 7f6c743e8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f6c199270 0x556f6c19b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.562+0000 7f6c743e8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556f6c0f0100 con 0x556f6c199270 2024-09-15T22:59:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.562+0000 7f6c743e8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556f6c0981b0 con 0x556f6c1941e0 2024-09-15T22:59:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.562+0000 7f6c743e8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556f6c0ec0c0 con 0x556f6c1938a0 2024-09-15T22:59:12.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.562+0000 7f6c6b200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f6c1938a0 0x556f6c193ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.562+0000 7f6c6b200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f6c1938a0 0x556f6c193ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:36426/0 (socket says 172.21.15.111:36426) 2024-09-15T22:59:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.562+0000 7f6c6b200640 1 -- 172.21.15.111:0/2337406744 learned_addr learned my addr 172.21.15.111:0/2337406744 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:59:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.562+0000 7f6c69e00640 1 --2- 172.21.15.111:0/2337406744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f6c199270 0x556f6c19b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.562+0000 7f6c6a800640 1 --2- 172.21.15.111:0/2337406744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f6c1941e0 0x556f6c198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.562+0000 7f6c6b200640 1 -- 172.21.15.111:0/2337406744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f6c1941e0 msgr2=0x556f6c198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.562+0000 7f6c6b200640 1 --2- 172.21.15.111:0/2337406744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f6c1941e0 0x556f6c198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.562+0000 7f6c6b200640 1 -- 172.21.15.111:0/2337406744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f6c199270 msgr2=0x556f6c19b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.562+0000 7f6c6b200640 1 --2- 172.21.15.111:0/2337406744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f6c199270 0x556f6c19b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.562+0000 7f6c6b200640 1 -- 172.21.15.111:0/2337406744 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f6be03210 con 0x556f6c1938a0 2024-09-15T22:59:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.562+0000 7f6c6a800640 1 --2- 172.21.15.111:0/2337406744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f6c1941e0 0x556f6c198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:59:12.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.563+0000 7f6c6b200640 1 --2- 172.21.15.111:0/2337406744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f6c1938a0 0x556f6c193ca0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009fd0 tx=0x7f6c640118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76ebdfa593a21891 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:59:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.563+0000 7f6c57400640 1 -- 172.21.15.111:0/2337406744 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c64004d20 con 0x556f6c1938a0 2024-09-15T22:59:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.563+0000 7f6c57400640 1 -- 172.21.15.111:0/2337406744 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c64004050 con 0x556f6c1938a0 2024-09-15T22:59:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.563+0000 7f6c57400640 1 -- 172.21.15.111:0/2337406744 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c64005260 con 0x556f6c1938a0 2024-09-15T22:59:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.563+0000 7f6c63400640 1 --2- 172.21.15.111:0/1184539139 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6c500a32f0 0x7f6c500a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.564+0000 7f6c63400640 1 -- 172.21.15.111:0/1184539139 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f6c5c0cfa90 con 0x7f6c6c103d60 2024-09-15T22:59:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.564+0000 7f6c6a800640 1 --2- 172.21.15.111:0/1184539139 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6c500a32f0 0x7f6c500a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.564+0000 7f6c743e8740 1 -- 172.21.15.111:0/2337406744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f6c1938a0 msgr2=0x556f6c193ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.564+0000 7f6c743e8740 1 --2- 172.21.15.111:0/2337406744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f6c1938a0 0x556f6c193ca0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009fd0 tx=0x7f6c640118d0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.564+0000 7f6c743e8740 1 -- 172.21.15.111:0/2337406744 shutdown_connections 2024-09-15T22:59:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.564+0000 7f6c743e8740 1 --2- 172.21.15.111:0/2337406744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f6c199270 0x556f6c19b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.564+0000 7f6c743e8740 1 --2- 172.21.15.111:0/2337406744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f6c1941e0 0x556f6c198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.564+0000 7f6c743e8740 1 --2- 172.21.15.111:0/2337406744 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f6c1938a0 0x556f6c193ca0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.564+0000 7f6c743e8740 1 -- 172.21.15.111:0/2337406744 >> 172.21.15.111:0/2337406744 conn(0x556f6c18ef10 msgr2=0x556f6c18f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:59:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.564+0000 7f6c743e8740 1 -- 172.21.15.111:0/2337406744 shutdown_connections 2024-09-15T22:59:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.564+0000 7f6c743e8740 1 -- 172.21.15.111:0/2337406744 wait complete. 2024-09-15T22:59:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.564+0000 7f6c6a800640 1 --2- 172.21.15.111:0/1184539139 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6c500a32f0 0x7f6c500a57b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c0a4890 tx=0x7f6c5c003f10 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:59:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.565+0000 7f6c743e8740 1 -- start start 2024-09-15T22:59:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.565+0000 7f6c743e8740 1 Processor -- start 2024-09-15T22:59:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.565+0000 7f6c743e8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f6c1938a0 0x556f6c19d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.565+0000 7f6c743e8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f6c1941e0 0x556f6c19df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.565+0000 7f6c743e8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f6c199270 0x556f6c1a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.565+0000 7f6c743e8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556f6c19c240 con 0x556f6c199270 2024-09-15T22:59:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.565+0000 7f6c743e8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556f6c079090 con 0x556f6c1938a0 2024-09-15T22:59:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.565+0000 7f6c743e8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556f6c19c3c0 con 0x556f6c1941e0 2024-09-15T22:59:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.565+0000 7f6c743e8740 10 client.0 _ll_register_callbacks cb 0x556f6c0cdf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:59:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.566+0000 7f6c69e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f6c1941e0 0x556f6c19df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.566+0000 7f6c6b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f6c1938a0 0x556f6c19d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.566+0000 7f6c69e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f6c1941e0 0x556f6c19df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:36438/0 (socket says 172.21.15.111:36438) 2024-09-15T22:59:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.566+0000 7f6c6b200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f6c199270 0x556f6c1a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.566+0000 7f6c69e00640 1 -- 172.21.15.111:0/1585241213 learned_addr learned my addr 172.21.15.111:0/1585241213 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:59:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.566+0000 7f6c69e00640 1 -- 172.21.15.111:0/1585241213 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f6c1938a0 msgr2=0x556f6c19d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.566+0000 7f6c69e00640 1 --2- 172.21.15.111:0/1585241213 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f6c1938a0 0x556f6c19d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.566+0000 7f6c69e00640 1 -- 172.21.15.111:0/1585241213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f6c199270 msgr2=0x556f6c1a4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.566+0000 7f6c69e00640 1 --2- 172.21.15.111:0/1585241213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f6c199270 0x556f6c1a4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.566+0000 7f6c69e00640 1 -- 172.21.15.111:0/1585241213 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f6becf620 con 0x556f6c1941e0 2024-09-15T22:59:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.566+0000 7f6c69e00640 1 --2- 172.21.15.111:0/1585241213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f6c1941e0 0x556f6c19df10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6c5802a930 tx=0x7f6c580095a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3a7a0b595b5109e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:59:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.566+0000 7f6c4be00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:59:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.567+0000 7f6c4be00640 1 -- 172.21.15.111:0/1585241213 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58037040 con 0x556f6c1941e0 2024-09-15T22:59:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.567+0000 7f6c743e8740 1 -- 172.21.15.111:0/1585241213 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c64004050 con 0x556f6c1941e0 2024-09-15T22:59:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.567+0000 7f6c4be00640 1 -- 172.21.15.111:0/1585241213 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f6c58004240 con 0x556f6c1941e0 2024-09-15T22:59:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.567+0000 7f6c4be00640 1 -- 172.21.15.111:0/1585241213 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6c5804c050 con 0x556f6c1941e0 2024-09-15T22:59:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.567+0000 7f6c743e8740 10 client.26050 fetch_fsmap learned FSMap version 30 2024-09-15T22:59:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.567+0000 7f6c743e8740 1 -- 172.21.15.111:0/1585241213 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556f6c1a7e00 con 0x556f6c1941e0 2024-09-15T22:59:12.572 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.567+0000 7f6c4be00640 1 -- 172.21.15.111:0/1585241213 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c64004050 con 0x556f6c1941e0 2024-09-15T22:59:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.567+0000 7f6c4be00640 1 -- 172.21.15.111:0/1585241213 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f6c58004a90 con 0x556f6c1941e0 2024-09-15T22:59:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.567+0000 7f6c743e8740 10 client.26050 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:59:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.567+0000 7f6c743e8740 10 client.26050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:59:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.567+0000 7f6c743e8740 20 client.26050 populate_metadata read hostname 'smithi111' 2024-09-15T22:59:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.567+0000 7f6c743e8740 1 --2- 172.21.15.111:0/1585241213 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556f6c1a94f0 0x556f6c1ab900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.568+0000 7f6c743e8740 4 client.26050 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:59:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.568+0000 7f6c743e8740 1 -- 172.21.15.111:0/1585241213 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f6c0fd830 con 0x556f6c1a94f0 2024-09-15T22:59:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.568+0000 7f6c6b200640 1 --2- 172.21.15.111:0/1585241213 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556f6c1a94f0 0x556f6c1ab900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.568+0000 7f6c6b200640 1 --2- 172.21.15.111:0/1585241213 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556f6c1a94f0 0x556f6c1ab900 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb0bcaeb1a41f7a1 server_cookie=af43492b97e72a0a in_seq=0 out_seq=0 2024-09-15T22:59:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.568+0000 7f6c4be00640 10 client.26050 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:59:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.569+0000 7f6c4be00640 1 -- 172.21.15.111:0/1585241213 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556f6c0fd830 con 0x556f6c1a94f0 2024-09-15T22:59:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.569+0000 7f6c4be00640 10 client.26050 handle_command_reply: tid=0 2024-09-15T22:59:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.569+0000 7f6c743e8740 1 client.26050 shutdown 2024-09-15T22:59:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.569+0000 7f6c743e8740 1 -- 172.21.15.111:0/1585241213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f6c1941e0 msgr2=0x556f6c19df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.569+0000 7f6c743e8740 1 --2- 172.21.15.111:0/1585241213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f6c1941e0 0x556f6c19df10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6c5802a930 tx=0x7f6c580095a0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.569+0000 7f6c743e8740 1 -- 172.21.15.111:0/1585241213 shutdown_connections 2024-09-15T22:59:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.569+0000 7f6c743e8740 1 --2- 172.21.15.111:0/1585241213 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556f6c1a94f0 0x556f6c1ab900 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.569+0000 7f6c743e8740 1 --2- 172.21.15.111:0/1585241213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f6c199270 0x556f6c1a4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.569+0000 7f6c743e8740 1 --2- 172.21.15.111:0/1585241213 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f6c1941e0 0x556f6c19df10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.569+0000 7f6c743e8740 1 --2- 172.21.15.111:0/1585241213 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f6c1938a0 0x556f6c19d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.569+0000 7f6c4be00640 0 client.26050 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:59:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.569+0000 7f6c743e8740 1 -- 172.21.15.111:0/1585241213 >> 172.21.15.111:0/1585241213 conn(0x556f6c18ef10 msgr2=0x556f6c190720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:59:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.569+0000 7f6c743e8740 1 -- 172.21.15.111:0/1585241213 shutdown_connections 2024-09-15T22:59:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.570+0000 7f6c743e8740 1 -- 172.21.15.111:0/1585241213 wait complete. 2024-09-15T22:59:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.570+0000 7f6c743e8740 20 client.26050 trim_cache size 0 max 16384 2024-09-15T22:59:12.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.587+0000 7f6c743e8740 1 Processor -- start 2024-09-15T22:59:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.587+0000 7f6c743e8740 1 -- start start 2024-09-15T22:59:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.587+0000 7f6c743e8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f6c1938a0 0x556f6c190300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.587+0000 7f6c743e8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f6c1941e0 0x556f6c1a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.587+0000 7f6c743e8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f6c199270 0x556f6c190840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.587+0000 7f6c743e8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6c1a9510 con 0x556f6c1941e0 2024-09-15T22:59:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.587+0000 7f6c743e8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6c1a6de0 con 0x556f6c199270 2024-09-15T22:59:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.587+0000 7f6c743e8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6c1a4c30 con 0x556f6c1938a0 2024-09-15T22:59:12.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.587+0000 7f6c6b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f6c199270 0x556f6c190840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.587+0000 7f6c6b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f6c199270 0x556f6c190840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:58480/0 (socket says 172.21.15.111:58480) 2024-09-15T22:59:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.587+0000 7f6c6b200640 1 -- 172.21.15.111:0/2564206613 learned_addr learned my addr 172.21.15.111:0/2564206613 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:59:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.587+0000 7f6c69e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f6c1941e0 0x556f6c1a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.587+0000 7f6c6b200640 1 --2- 172.21.15.111:0/2564206613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f6c1938a0 0x556f6c190300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.588+0000 7f6c6b200640 1 -- 172.21.15.111:0/2564206613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f6c1938a0 msgr2=0x556f6c190300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.588+0000 7f6c6b200640 1 --2- 172.21.15.111:0/2564206613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f6c1938a0 0x556f6c190300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.588+0000 7f6c6b200640 1 -- 172.21.15.111:0/2564206613 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f6c1941e0 msgr2=0x556f6c1a4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.588+0000 7f6c6b200640 1 --2- 172.21.15.111:0/2564206613 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f6c1941e0 0x556f6c1a4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.588+0000 7f6c6b200640 1 -- 172.21.15.111:0/2564206613 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f6c1946c0 con 0x556f6c199270 2024-09-15T22:59:12.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.588+0000 7f6c6b200640 1 --2- 172.21.15.111:0/2564206613 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f6c199270 0x556f6c190840 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009f70 tx=0x7f6c64004ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6371ad96eab1df40 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:59:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.588+0000 7f6c56a00640 1 -- 172.21.15.111:0/2564206613 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c64003d20 con 0x556f6c199270 2024-09-15T22:59:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.588+0000 7f6c56a00640 1 -- 172.21.15.111:0/2564206613 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c640044e0 con 0x556f6c199270 2024-09-15T22:59:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.588+0000 7f6c56a00640 1 -- 172.21.15.111:0/2564206613 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6400fc10 con 0x556f6c199270 2024-09-15T22:59:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.589+0000 7f6c743e8740 1 -- 172.21.15.111:0/2564206613 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f6c199270 msgr2=0x556f6c190840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.589+0000 7f6c743e8740 1 --2- 172.21.15.111:0/2564206613 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f6c199270 0x556f6c190840 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009f70 tx=0x7f6c64004ba0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.589+0000 7f6c743e8740 1 -- 172.21.15.111:0/2564206613 shutdown_connections 2024-09-15T22:59:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.589+0000 7f6c743e8740 1 --2- 172.21.15.111:0/2564206613 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f6c199270 0x556f6c190840 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.589+0000 7f6c743e8740 1 --2- 172.21.15.111:0/2564206613 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f6c1941e0 0x556f6c1a4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.589+0000 7f6c743e8740 1 --2- 172.21.15.111:0/2564206613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f6c1938a0 0x556f6c190300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.589+0000 7f6c743e8740 1 -- 172.21.15.111:0/2564206613 >> 172.21.15.111:0/2564206613 conn(0x556f6c18ef10 msgr2=0x556f6c185880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:59:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.589+0000 7f6c743e8740 1 -- 172.21.15.111:0/2564206613 shutdown_connections 2024-09-15T22:59:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.589+0000 7f6c743e8740 1 -- 172.21.15.111:0/2564206613 wait complete. 2024-09-15T22:59:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.590+0000 7f6c743e8740 1 -- start start 2024-09-15T22:59:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.590+0000 7f6c743e8740 1 Processor -- start 2024-09-15T22:59:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.590+0000 7f6c743e8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f6c1938a0 0x556f6c1a2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.590+0000 7f6c743e8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f6c1941e0 0x556f6c1a2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.590+0000 7f6c743e8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f6c199270 0x556f6c1a3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.590+0000 7f6c743e8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6c1a52a0 con 0x556f6c199270 2024-09-15T22:59:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.590+0000 7f6c743e8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6c1a6710 con 0x556f6c1938a0 2024-09-15T22:59:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.590+0000 7f6c743e8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6c1a5970 con 0x556f6c1941e0 2024-09-15T22:59:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.591+0000 7f6c743e8740 10 client.0 _ll_register_callbacks cb 0x7f6c640093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:59:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.591+0000 7f6c6b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f6c1938a0 0x556f6c1a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.591+0000 7f6c6b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f6c1938a0 0x556f6c1a2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:58488/0 (socket says 172.21.15.111:58488) 2024-09-15T22:59:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.591+0000 7f6c6b200640 1 -- 172.21.15.111:0/3701583025 learned_addr learned my addr 172.21.15.111:0/3701583025 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:59:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.591+0000 7f6c69e00640 1 --2- 172.21.15.111:0/3701583025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f6c1941e0 0x556f6c1a2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.591+0000 7f6c6b200640 1 --2- 172.21.15.111:0/3701583025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f6c199270 0x556f6c1a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.591+0000 7f6c69e00640 1 -- 172.21.15.111:0/3701583025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f6c1938a0 msgr2=0x556f6c1a2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.591+0000 7f6c69e00640 1 --2- 172.21.15.111:0/3701583025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f6c1938a0 0x556f6c1a2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.591+0000 7f6c69e00640 1 -- 172.21.15.111:0/3701583025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f6c199270 msgr2=0x556f6c1a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.591+0000 7f6c69e00640 1 --2- 172.21.15.111:0/3701583025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f6c199270 0x556f6c1a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.591+0000 7f6c69e00640 1 -- 172.21.15.111:0/3701583025 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f6c1a3e90 con 0x556f6c1941e0 2024-09-15T22:59:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.591+0000 7f6c69e00640 1 --2- 172.21.15.111:0/3701583025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f6c1941e0 0x556f6c1a2870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6c58004f00 tx=0x7f6c580048d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6556d2c238f3e245 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:59:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.591+0000 7f6c4be00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:59:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.592+0000 7f6c4be00640 1 -- 172.21.15.111:0/3701583025 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58054070 con 0x556f6c1941e0 2024-09-15T22:59:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.592+0000 7f6c4be00640 1 -- 172.21.15.111:0/3701583025 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f6c580051d0 con 0x556f6c1941e0 2024-09-15T22:59:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.592+0000 7f6c743e8740 1 -- 172.21.15.111:0/3701583025 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c640044e0 con 0x556f6c1941e0 2024-09-15T22:59:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.592+0000 7f6c4be00640 1 -- 172.21.15.111:0/3701583025 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6c58060050 con 0x556f6c1941e0 2024-09-15T22:59:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.592+0000 7f6c743e8740 10 client.26056 fetch_fsmap learned FSMap version 30 2024-09-15T22:59:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.592+0000 7f6c743e8740 1 -- 172.21.15.111:0/3701583025 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556f6c1a5210 con 0x556f6c1941e0 2024-09-15T22:59:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.592+0000 7f6c4be00640 1 -- 172.21.15.111:0/3701583025 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c640044e0 con 0x556f6c1941e0 2024-09-15T22:59:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.592+0000 7f6c4be00640 1 -- 172.21.15.111:0/3701583025 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7f6c580053d0 con 0x556f6c1941e0 2024-09-15T22:59:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.592+0000 7f6c743e8740 10 client.26056 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:59:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.592+0000 7f6c743e8740 10 client.26056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:59:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.592+0000 7f6c743e8740 20 client.26056 populate_metadata read hostname 'smithi111' 2024-09-15T22:59:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.592+0000 7f6c743e8740 1 --2- 172.21.15.111:0/3701583025 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556f6c214920 0x556f6c216d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.592+0000 7f6c743e8740 4 client.26056 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:59:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.592+0000 7f6c743e8740 1 -- 172.21.15.111:0/3701583025 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556f6c0f7b80 con 0x556f6c214920 2024-09-15T22:59:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.593+0000 7f6c6b200640 1 --2- 172.21.15.111:0/3701583025 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556f6c214920 0x556f6c216d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.593+0000 7f6c4be00640 10 client.26056 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:59:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.593+0000 7f6c6b200640 1 --2- 172.21.15.111:0/3701583025 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556f6c214920 0x556f6c216d40 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9767d530e61e1a03 server_cookie=68a5cb93171c0d6 in_seq=0 out_seq=0 2024-09-15T22:59:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.605+0000 7f6c4be00640 1 -- 172.21.15.111:0/3701583025 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556f6c0f7b80 con 0x556f6c214920 2024-09-15T22:59:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.605+0000 7f6c4be00640 10 client.26056 handle_command_reply: tid=0 2024-09-15T22:59:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.605+0000 7f6c743e8740 1 client.26056 shutdown 2024-09-15T22:59:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.605+0000 7f6c743e8740 1 -- 172.21.15.111:0/3701583025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f6c1941e0 msgr2=0x556f6c1a2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.605+0000 7f6c743e8740 1 --2- 172.21.15.111:0/3701583025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f6c1941e0 0x556f6c1a2870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6c58004f00 tx=0x7f6c580048d0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.605+0000 7f6c743e8740 1 -- 172.21.15.111:0/3701583025 shutdown_connections 2024-09-15T22:59:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.605+0000 7f6c743e8740 1 --2- 172.21.15.111:0/3701583025 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556f6c214920 0x556f6c216d40 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.605+0000 7f6c743e8740 1 --2- 172.21.15.111:0/3701583025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f6c199270 0x556f6c1a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.605+0000 7f6c743e8740 1 --2- 172.21.15.111:0/3701583025 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f6c1941e0 0x556f6c1a2870 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.605+0000 7f6c4be00640 0 client.26056 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:59:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.605+0000 7f6c743e8740 1 --2- 172.21.15.111:0/3701583025 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f6c1938a0 0x556f6c1a2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.605+0000 7f6c743e8740 1 -- 172.21.15.111:0/3701583025 >> 172.21.15.111:0/3701583025 conn(0x556f6c18ef10 msgr2=0x556f6c18f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:59:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.605+0000 7f6c743e8740 1 -- 172.21.15.111:0/3701583025 shutdown_connections 2024-09-15T22:59:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.605+0000 7f6c743e8740 1 -- 172.21.15.111:0/3701583025 wait complete. 2024-09-15T22:59:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.606+0000 7f6c743e8740 20 client.26056 trim_cache size 0 max 16384 2024-09-15T22:59:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:59:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:59:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:59:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:59:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.606+0000 7f6c71400640 1 -- 172.21.15.111:0/1184539139 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6c500a32f0 msgr2=0x7f6c500a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.606+0000 7f6c71400640 1 --2- 172.21.15.111:0/1184539139 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6c500a32f0 0x7f6c500a57b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c0a4890 tx=0x7f6c5c003f10 comp rx=0 tx=0).stop 2024-09-15T22:59:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.606+0000 7f6c71400640 1 -- 172.21.15.111:0/1184539139 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c6c103d60 msgr2=0x7f6c6c199f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.606+0000 7f6c71400640 1 --2- 172.21.15.111:0/1184539139 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c6c103d60 0x7f6c6c199f70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00e9f0 tx=0x7f6c5c00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.606+0000 7f6c71400640 1 -- 172.21.15.111:0/1184539139 shutdown_connections 2024-09-15T22:59:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.606+0000 7f6c71400640 1 --2- 172.21.15.111:0/1184539139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c6c105910 0x7f6c6c1a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.606+0000 7f6c71400640 1 --2- 172.21.15.111:0/1184539139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c6c104f50 0x7f6c6c19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.606+0000 7f6c71400640 1 --2- 172.21.15.111:0/1184539139 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6c500a32f0 0x7f6c500a57b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.606+0000 7f6c71400640 1 --2- 172.21.15.111:0/1184539139 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c6c103d60 0x7f6c6c199f70 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.606+0000 7f6c71400640 1 -- 172.21.15.111:0/1184539139 >> 172.21.15.111:0/1184539139 conn(0x7f6c6c0ff4e0 msgr2=0x7f6c6c108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:59:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.606+0000 7f6c71400640 1 -- 172.21.15.111:0/1184539139 shutdown_connections 2024-09-15T22:59:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:12.606+0000 7f6c71400640 1 -- 172.21.15.111:0/1184539139 wait complete. 2024-09-15T22:59:13.337 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1288130 9.80 MB/sec execute 581 sec latency 11.803 ms 2024-09-15T22:59:14.025 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:13 smithi111 ceph-mon[30629]: pgmap v749: 97 pgs: 97 active+clean; 288 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 182 op/s 2024-09-15T22:59:14.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:13 smithi192 ceph-mon[35427]: pgmap v749: 97 pgs: 97 active+clean; 288 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 182 op/s 2024-09-15T22:59:14.337 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1289345 9.80 MB/sec execute 582 sec latency 583.476 ms 2024-09-15T22:59:14.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:13 smithi121 ceph-mon[34815]: pgmap v749: 97 pgs: 97 active+clean; 288 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 182 op/s 2024-09-15T22:59:15.337 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1289511 9.78 MB/sec execute 583 sec latency 1408.232 ms 2024-09-15T22:59:16.316 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:15 smithi192 ceph-mon[35427]: pgmap v750: 97 pgs: 97 active+clean; 279 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 177 op/s 2024-09-15T22:59:16.337 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1292491 9.79 MB/sec execute 584 sec latency 14.425 ms 2024-09-15T22:59:16.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:15 smithi121 ceph-mon[34815]: pgmap v750: 97 pgs: 97 active+clean; 279 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 177 op/s 2024-09-15T22:59:16.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:15 smithi111 ceph-mon[30629]: pgmap v750: 97 pgs: 97 active+clean; 279 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 177 op/s 2024-09-15T22:59:17.337 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1294842 9.79 MB/sec execute 585 sec latency 13.265 ms 2024-09-15T22:59:18.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:17 smithi192 ceph-mon[35427]: pgmap v751: 97 pgs: 97 active+clean; 266 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 212 op/s 2024-09-15T22:59:18.320 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:17 smithi111 ceph-mon[30629]: pgmap v751: 97 pgs: 97 active+clean; 266 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 212 op/s 2024-09-15T22:59:18.337 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1296983 9.80 MB/sec execute 586 sec latency 27.119 ms 2024-09-15T22:59:18.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:17 smithi121 ceph-mon[34815]: pgmap v751: 97 pgs: 97 active+clean; 266 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 212 op/s 2024-09-15T22:59:19.337 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1298751 9.79 MB/sec execute 587 sec latency 422.038 ms 2024-09-15T22:59:20.027 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:19 smithi111 ceph-mon[30629]: pgmap v752: 97 pgs: 97 active+clean; 268 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 199 op/s 2024-09-15T22:59:20.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:19 smithi192 ceph-mon[35427]: pgmap v752: 97 pgs: 97 active+clean; 268 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 199 op/s 2024-09-15T22:59:20.337 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1298751 9.78 MB/sec execute 588 sec latency 1007.920 ms 2024-09-15T22:59:20.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:19 smithi121 ceph-mon[34815]: pgmap v752: 97 pgs: 97 active+clean; 268 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 199 op/s 2024-09-15T22:59:21.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:20 smithi192 ceph-mon[35427]: pgmap v753: 97 pgs: 97 active+clean; 271 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 195 op/s 2024-09-15T22:59:21.337 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1300547 9.78 MB/sec execute 589 sec latency 1136.156 ms 2024-09-15T22:59:21.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:20 smithi121 ceph-mon[34815]: pgmap v753: 97 pgs: 97 active+clean; 271 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 195 op/s 2024-09-15T22:59:21.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:20 smithi111 ceph-mon[30629]: pgmap v753: 97 pgs: 97 active+clean; 271 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 195 op/s 2024-09-15T22:59:22.337 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1303573 9.80 MB/sec execute 590 sec latency 15.020 ms 2024-09-15T22:59:23.337 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1305921 9.79 MB/sec execute 591 sec latency 11.902 ms 2024-09-15T22:59:24.068 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:23 smithi111 ceph-mon[30629]: pgmap v754: 97 pgs: 97 active+clean; 274 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 193 op/s 2024-09-15T22:59:24.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:23 smithi121 ceph-mon[34815]: pgmap v754: 97 pgs: 97 active+clean; 274 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 193 op/s 2024-09-15T22:59:24.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:23 smithi192 ceph-mon[35427]: pgmap v754: 97 pgs: 97 active+clean; 274 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 193 op/s 2024-09-15T22:59:24.337 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1307321 9.80 MB/sec execute 592 sec latency 289.950 ms 2024-09-15T22:59:25.337 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1307574 9.78 MB/sec execute 593 sec latency 830.038 ms 2024-09-15T22:59:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.729+0000 7fa500600640 1 -- 172.21.15.111:0/360701326 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f8103710 msgr2=0x7fa4f8109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.729+0000 7fa500600640 1 --2- 172.21.15.111:0/360701326 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f8103710 0x7fa4f8109fa0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0009be0 tx=0x7fa4f002a6c0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.729+0000 7fa500600640 1 -- 172.21.15.111:0/360701326 shutdown_connections 2024-09-15T22:59:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.729+0000 7fa500600640 1 --2- 172.21.15.111:0/360701326 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f8103710 0x7fa4f8109fa0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.729+0000 7fa500600640 1 --2- 172.21.15.111:0/360701326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4f8102d50 0x7fa4f81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.729+0000 7fa500600640 1 --2- 172.21.15.111:0/360701326 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4f8101b60 0x7fa4f8101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.729+0000 7fa500600640 1 -- 172.21.15.111:0/360701326 >> 172.21.15.111:0/360701326 conn(0x7fa4f80fd2e0 msgr2=0x7fa4f80ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:59:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.729+0000 7fa500600640 1 -- 172.21.15.111:0/360701326 shutdown_connections 2024-09-15T22:59:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.729+0000 7fa500600640 1 -- 172.21.15.111:0/360701326 wait complete. 2024-09-15T22:59:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.730+0000 7fa500600640 1 Processor -- start 2024-09-15T22:59:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.731+0000 7fa500600640 1 -- start start 2024-09-15T22:59:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.731+0000 7fa500600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4f8101b60 0x7fa4f8199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.731+0000 7fa500600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4f8102d50 0x7fa4f819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.731+0000 7fa500600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f8103710 0x7fa4f81a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.731+0000 7fa500600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f810d7c0 con 0x7fa4f8102d50 2024-09-15T22:59:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.731+0000 7fa500600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f810d640 con 0x7fa4f8103710 2024-09-15T22:59:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.731+0000 7fa500600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f810d940 con 0x7fa4f8101b60 2024-09-15T22:59:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.731+0000 7fa4fdc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4f8101b60 0x7fa4f8199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.731+0000 7fa4fdc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4f8101b60 0x7fa4f8199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:60778/0 (socket says 172.21.15.111:60778) 2024-09-15T22:59:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.731+0000 7fa4fd200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4f8102d50 0x7fa4f819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.731+0000 7fa4fdc00640 1 -- 172.21.15.111:0/246474834 learned_addr learned my addr 172.21.15.111:0/246474834 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:59:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.731+0000 7fa4fe600640 1 --2- 172.21.15.111:0/246474834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f8103710 0x7fa4f81a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.731+0000 7fa4fe600640 1 -- 172.21.15.111:0/246474834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4f8101b60 msgr2=0x7fa4f8199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.732+0000 7fa4fe600640 1 --2- 172.21.15.111:0/246474834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4f8101b60 0x7fa4f8199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.732+0000 7fa4fe600640 1 -- 172.21.15.111:0/246474834 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4f8102d50 msgr2=0x7fa4f819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.732+0000 7fa4fe600640 1 --2- 172.21.15.111:0/246474834 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4f8102d50 0x7fa4f819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.732+0000 7fa4fe600640 1 -- 172.21.15.111:0/246474834 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f81a1b70 con 0x7fa4f8103710 2024-09-15T22:59:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.732+0000 7fa4fdc00640 1 --2- 172.21.15.111:0/246474834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4f8101b60 0x7fa4f8199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:59:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.732+0000 7fa4fe600640 1 --2- 172.21.15.111:0/246474834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f8103710 0x7fa4f81a1500 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0034040 tx=0x7fa4f00305b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:59:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.732+0000 7fa4f6000640 1 -- 172.21.15.111:0/246474834 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0041020 con 0x7fa4f8103710 2024-09-15T22:59:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.732+0000 7fa4f6000640 1 -- 172.21.15.111:0/246474834 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f0030720 con 0x7fa4f8103710 2024-09-15T22:59:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.732+0000 7fa500600640 1 -- 172.21.15.111:0/246474834 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f81a1e00 con 0x7fa4f8103710 2024-09-15T22:59:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.732+0000 7fa4f6000640 1 -- 172.21.15.111:0/246474834 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0036040 con 0x7fa4f8103710 2024-09-15T22:59:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.732+0000 7fa500600640 1 -- 172.21.15.111:0/246474834 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f81a2260 con 0x7fa4f8103710 2024-09-15T22:59:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.735+0000 7fa5033fe740 1 Processor -- start 2024-09-15T22:59:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.735+0000 7fa4f6000640 1 -- 172.21.15.111:0/246474834 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa4f0044020 con 0x7fa4f8103710 2024-09-15T22:59:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.736+0000 7fa5033fe740 1 -- start start 2024-09-15T22:59:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.736+0000 7fa5033fe740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b291090 0x560a6b291490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.736+0000 7fa5033fe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560a6b2919d0 0x560a6b296490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.736+0000 7fa5033fe740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a6b296a60 0x560a6b298f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.736+0000 7fa5033fe740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560a6b1f6100 con 0x560a6b296a60 2024-09-15T22:59:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.736+0000 7fa5033fe740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560a6b19e1b0 con 0x560a6b291090 2024-09-15T22:59:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.736+0000 7fa5033fe740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x560a6b1f20c0 con 0x560a6b2919d0 2024-09-15T22:59:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.736+0000 7fa4fd200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b291090 0x560a6b291490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.736+0000 7fa4fd200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a6b296a60 0x560a6b298f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.736+0000 7fa4fd200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b291090 0x560a6b291490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:34286/0 (socket says 172.21.15.111:34286) 2024-09-15T22:59:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.736+0000 7fa4fd200640 1 -- 172.21.15.111:0/2584927629 learned_addr learned my addr 172.21.15.111:0/2584927629 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:59:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.736+0000 7fa4fdc00640 1 --2- 172.21.15.111:0/2584927629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560a6b2919d0 0x560a6b296490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.737+0000 7fa4fd200640 1 -- 172.21.15.111:0/2584927629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560a6b2919d0 msgr2=0x560a6b296490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.737+0000 7fa4fd200640 1 --2- 172.21.15.111:0/2584927629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560a6b2919d0 0x560a6b296490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.737+0000 7fa4fd200640 1 -- 172.21.15.111:0/2584927629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a6b296a60 msgr2=0x560a6b298f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.737+0000 7fa4fd200640 1 --2- 172.21.15.111:0/2584927629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a6b296a60 0x560a6b298f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.737+0000 7fa4fd200640 1 -- 172.21.15.111:0/2584927629 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a6af09210 con 0x560a6b291090 2024-09-15T22:59:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.737+0000 7fa4fd200640 1 --2- 172.21.15.111:0/2584927629 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b291090 0x560a6b291490 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa4e40137d0 tx=0x7fa4e40097b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aefe8766d4e22a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:59:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.738+0000 7fa4f6000640 1 --2- 172.21.15.111:0/246474834 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa4e00a32f0 0x7fa4e00a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.738+0000 7fa4eaa00640 1 -- 172.21.15.111:0/2584927629 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e40130d0 con 0x560a6b291090 2024-09-15T22:59:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.738+0000 7fa4eaa00640 1 -- 172.21.15.111:0/2584927629 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4e4013270 con 0x560a6b291090 2024-09-15T22:59:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.738+0000 7fa4eaa00640 1 -- 172.21.15.111:0/2584927629 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e40047a0 con 0x560a6b291090 2024-09-15T22:59:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.738+0000 7fa4f6000640 1 -- 172.21.15.111:0/246474834 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fa4f00f05e0 con 0x7fa4f8103710 2024-09-15T22:59:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.738+0000 7fa4fdc00640 1 --2- 172.21.15.111:0/246474834 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa4e00a32f0 0x7fa4e00a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.738+0000 7fa5033fe740 1 -- 172.21.15.111:0/2584927629 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b291090 msgr2=0x560a6b291490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.738+0000 7fa5033fe740 1 --2- 172.21.15.111:0/2584927629 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b291090 0x560a6b291490 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa4e40137d0 tx=0x7fa4e40097b0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.739+0000 7fa4fdc00640 1 --2- 172.21.15.111:0/246474834 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa4e00a32f0 0x7fa4e00a57b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00bfd0 tx=0x7fa4ec00b4a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:59:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.739+0000 7fa5033fe740 1 -- 172.21.15.111:0/2584927629 shutdown_connections 2024-09-15T22:59:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.739+0000 7fa5033fe740 1 --2- 172.21.15.111:0/2584927629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a6b296a60 0x560a6b298f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.739+0000 7fa5033fe740 1 --2- 172.21.15.111:0/2584927629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560a6b2919d0 0x560a6b296490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.739+0000 7fa5033fe740 1 --2- 172.21.15.111:0/2584927629 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b291090 0x560a6b291490 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.739+0000 7fa5033fe740 1 -- 172.21.15.111:0/2584927629 >> 172.21.15.111:0/2584927629 conn(0x560a6b28c700 msgr2=0x560a6b28cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:59:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.739+0000 7fa5033fe740 1 -- 172.21.15.111:0/2584927629 shutdown_connections 2024-09-15T22:59:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.739+0000 7fa5033fe740 1 -- 172.21.15.111:0/2584927629 wait complete. 2024-09-15T22:59:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.739+0000 7fa5033fe740 1 -- start start 2024-09-15T22:59:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.739+0000 7fa5033fe740 1 Processor -- start 2024-09-15T22:59:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.740+0000 7fa5033fe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560a6b291090 0x560a6b29b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.740+0000 7fa5033fe740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a6b2919d0 0x560a6b29b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.740+0000 7fa5033fe740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b296a60 0x560a6b2a27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.740+0000 7fa5033fe740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560a6b2999d0 con 0x560a6b2919d0 2024-09-15T22:59:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.740+0000 7fa5033fe740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560a6b17f090 con 0x560a6b296a60 2024-09-15T22:59:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.740+0000 7fa5033fe740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x560a6b299b50 con 0x560a6b291090 2024-09-15T22:59:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.740+0000 7fa5033fe740 10 client.0 _ll_register_callbacks cb 0x560a6b1d3f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:59:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.740+0000 7fa4fe600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b296a60 0x560a6b2a27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.740+0000 7fa4fdc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560a6b291090 0x560a6b29b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.740+0000 7fa4fe600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b296a60 0x560a6b2a27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:34302/0 (socket says 172.21.15.111:34302) 2024-09-15T22:59:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.740+0000 7fa4fe600640 1 -- 172.21.15.111:0/1968266998 learned_addr learned my addr 172.21.15.111:0/1968266998 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:59:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.740+0000 7fa4fd200640 1 --2- 172.21.15.111:0/1968266998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a6b2919d0 0x560a6b29b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.740+0000 7fa4fe600640 1 -- 172.21.15.111:0/1968266998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560a6b291090 msgr2=0x560a6b29b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.740+0000 7fa4fe600640 1 --2- 172.21.15.111:0/1968266998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560a6b291090 0x560a6b29b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.740+0000 7fa4fe600640 1 -- 172.21.15.111:0/1968266998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a6b2919d0 msgr2=0x560a6b29b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.740+0000 7fa4fe600640 1 --2- 172.21.15.111:0/1968266998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a6b2919d0 0x560a6b29b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.740+0000 7fa4fe600640 1 -- 172.21.15.111:0/1968266998 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a6afd5620 con 0x560a6b296a60 2024-09-15T22:59:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.740+0000 7fa4fdc00640 1 --2- 172.21.15.111:0/1968266998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560a6b291090 0x560a6b29b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:59:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.741+0000 7fa4fe600640 1 --2- 172.21.15.111:0/1968266998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b296a60 0x560a6b2a27c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa4f00bc900 tx=0x7fa4f00bc560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78ba032495625a21 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:59:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.741+0000 7fa4df400640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:59:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.741+0000 7fa4df400640 1 -- 172.21.15.111:0/1968266998 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f00bcc00 con 0x560a6b296a60 2024-09-15T22:59:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.741+0000 7fa4df400640 1 -- 172.21.15.111:0/1968266998 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fa4f00bce60 con 0x560a6b296a60 2024-09-15T22:59:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.741+0000 7fa5033fe740 1 -- 172.21.15.111:0/1968266998 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4e4013270 con 0x560a6b296a60 2024-09-15T22:59:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.741+0000 7fa4df400640 1 -- 172.21.15.111:0/1968266998 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa4f00c0fb0 con 0x560a6b296a60 2024-09-15T22:59:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.741+0000 7fa5033fe740 10 client.26045 fetch_fsmap learned FSMap version 30 2024-09-15T22:59:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.741+0000 7fa5033fe740 1 -- 172.21.15.111:0/1968266998 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560a6b2a55b0 con 0x560a6b296a60 2024-09-15T22:59:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.741+0000 7fa4df400640 1 -- 172.21.15.111:0/1968266998 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4e4013270 con 0x560a6b296a60 2024-09-15T22:59:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.741+0000 7fa4df400640 1 -- 172.21.15.111:0/1968266998 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fa4f00f00e0 con 0x560a6b296a60 2024-09-15T22:59:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.742+0000 7fa5033fe740 10 client.26045 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:59:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.742+0000 7fa5033fe740 10 client.26045 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:59:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.742+0000 7fa5033fe740 20 client.26045 populate_metadata read hostname 'smithi111' 2024-09-15T22:59:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.742+0000 7fa5033fe740 1 --2- 172.21.15.111:0/1968266998 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560a6b2a6d30 0x560a6b2a9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.742+0000 7fa5033fe740 4 client.26045 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:59:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.742+0000 7fa5033fe740 1 -- 172.21.15.111:0/1968266998 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a6b203830 con 0x560a6b2a6d30 2024-09-15T22:59:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.742+0000 7fa4fdc00640 1 --2- 172.21.15.111:0/1968266998 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560a6b2a6d30 0x560a6b2a9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.742+0000 7fa4df400640 10 client.26045 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:59:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.742+0000 7fa4fdc00640 1 --2- 172.21.15.111:0/1968266998 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560a6b2a6d30 0x560a6b2a9180 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1558d41ea060c1c server_cookie=9f20e36953e327c4 in_seq=0 out_seq=0 2024-09-15T22:59:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.743+0000 7fa4df400640 1 -- 172.21.15.111:0/1968266998 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a6b203830 con 0x560a6b2a6d30 2024-09-15T22:59:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.743+0000 7fa4df400640 10 client.26045 handle_command_reply: tid=0 2024-09-15T22:59:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.743+0000 7fa5033fe740 1 client.26045 shutdown 2024-09-15T22:59:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.743+0000 7fa5033fe740 1 -- 172.21.15.111:0/1968266998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b296a60 msgr2=0x560a6b2a27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.743+0000 7fa5033fe740 1 --2- 172.21.15.111:0/1968266998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b296a60 0x560a6b2a27c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa4f00bc900 tx=0x7fa4f00bc560 comp rx=0 tx=0).stop 2024-09-15T22:59:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.744+0000 7fa5033fe740 1 -- 172.21.15.111:0/1968266998 shutdown_connections 2024-09-15T22:59:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.744+0000 7fa5033fe740 1 --2- 172.21.15.111:0/1968266998 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560a6b2a6d30 0x560a6b2a9180 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.744+0000 7fa5033fe740 1 --2- 172.21.15.111:0/1968266998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b296a60 0x560a6b2a27c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.744+0000 7fa5033fe740 1 --2- 172.21.15.111:0/1968266998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a6b2919d0 0x560a6b29b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.744+0000 7fa5033fe740 1 --2- 172.21.15.111:0/1968266998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560a6b291090 0x560a6b29b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.744+0000 7fa5033fe740 1 -- 172.21.15.111:0/1968266998 >> 172.21.15.111:0/1968266998 conn(0x560a6b28c700 msgr2=0x560a6b28df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:59:25.751 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.744+0000 7fa4df400640 0 client.26045 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:59:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.744+0000 7fa5033fe740 1 -- 172.21.15.111:0/1968266998 shutdown_connections 2024-09-15T22:59:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.744+0000 7fa5033fe740 1 -- 172.21.15.111:0/1968266998 wait complete. 2024-09-15T22:59:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.744+0000 7fa5033fe740 20 client.26045 trim_cache size 0 max 16384 2024-09-15T22:59:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.764+0000 7fa5033fe740 1 Processor -- start 2024-09-15T22:59:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.764+0000 7fa5033fe740 1 -- start start 2024-09-15T22:59:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.764+0000 7fa5033fe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560a6b28c700 0x560a6b2a8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.764+0000 7fa5033fe740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b291090 0x560a6b2a2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.764+0000 7fa5033fe740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a6b2919d0 0x560a6b28d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.764+0000 7fa5033fe740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f81ad6f0 con 0x560a6b2919d0 2024-09-15T22:59:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.764+0000 7fa5033fe740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f81aafc0 con 0x560a6b291090 2024-09-15T22:59:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.764+0000 7fa5033fe740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f81a8e10 con 0x560a6b28c700 2024-09-15T22:59:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.765+0000 7fa4fdc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560a6b28c700 0x560a6b2a8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.765+0000 7fa4fd200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b291090 0x560a6b2a2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.765+0000 7fa4fdc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560a6b28c700 0x560a6b2a8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:60812/0 (socket says 172.21.15.111:60812) 2024-09-15T22:59:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.765+0000 7fa4fd200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b291090 0x560a6b2a2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:34316/0 (socket says 172.21.15.111:34316) 2024-09-15T22:59:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.765+0000 7fa4fdc00640 1 -- 172.21.15.111:0/1312469666 learned_addr learned my addr 172.21.15.111:0/1312469666 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:59:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.765+0000 7fa4fe600640 1 --2- 172.21.15.111:0/1312469666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a6b2919d0 0x560a6b28d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.765+0000 7fa4fdc00640 1 -- 172.21.15.111:0/1312469666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b291090 msgr2=0x560a6b2a2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.765+0000 7fa4fdc00640 1 --2- 172.21.15.111:0/1312469666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b291090 0x560a6b2a2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.765+0000 7fa4fdc00640 1 -- 172.21.15.111:0/1312469666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a6b2919d0 msgr2=0x560a6b28d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.765+0000 7fa4fdc00640 1 --2- 172.21.15.111:0/1312469666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a6b2919d0 0x560a6b28d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.765+0000 7fa4fdc00640 1 -- 172.21.15.111:0/1312469666 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a6b287d60 con 0x560a6b28c700 2024-09-15T22:59:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.765+0000 7fa4fd200640 1 --2- 172.21.15.111:0/1312469666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b291090 0x560a6b2a2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:59:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.765+0000 7fa4fdc00640 1 --2- 172.21.15.111:0/1312469666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560a6b28c700 0x560a6b2a8d40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x560a6b28d580 tx=0x7fa4ec016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f217e7f0b69ce77b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:59:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.766+0000 7fa4ea000640 1 -- 172.21.15.111:0/1312469666 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec020190 con 0x560a6b28c700 2024-09-15T22:59:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.766+0000 7fa4ea000640 1 -- 172.21.15.111:0/1312469666 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4ec004020 con 0x560a6b28c700 2024-09-15T22:59:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.766+0000 7fa4ea000640 1 -- 172.21.15.111:0/1312469666 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec003ad0 con 0x560a6b28c700 2024-09-15T22:59:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.766+0000 7fa5033fe740 1 -- 172.21.15.111:0/1312469666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560a6b28c700 msgr2=0x560a6b2a8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.766+0000 7fa5033fe740 1 --2- 172.21.15.111:0/1312469666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560a6b28c700 0x560a6b2a8d40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x560a6b28d580 tx=0x7fa4ec016000 comp rx=0 tx=0).stop 2024-09-15T22:59:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.766+0000 7fa5033fe740 1 -- 172.21.15.111:0/1312469666 shutdown_connections 2024-09-15T22:59:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.766+0000 7fa5033fe740 1 --2- 172.21.15.111:0/1312469666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a6b2919d0 0x560a6b28d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.766+0000 7fa5033fe740 1 --2- 172.21.15.111:0/1312469666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b291090 0x560a6b2a2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.766+0000 7fa5033fe740 1 --2- 172.21.15.111:0/1312469666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560a6b28c700 0x560a6b2a8d40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.766+0000 7fa5033fe740 1 -- 172.21.15.111:0/1312469666 >> 172.21.15.111:0/1312469666 conn(0x560a6b2965f0 msgr2=0x560a6b292c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:59:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.767+0000 7fa5033fe740 1 -- 172.21.15.111:0/1312469666 shutdown_connections 2024-09-15T22:59:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.767+0000 7fa5033fe740 1 -- 172.21.15.111:0/1312469666 wait complete. 2024-09-15T22:59:25.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.767+0000 7fa5033fe740 1 -- start start 2024-09-15T22:59:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.767+0000 7fa5033fe740 1 Processor -- start 2024-09-15T22:59:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.768+0000 7fa5033fe740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a6b28c700 0x560a6b29fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.768+0000 7fa5033fe740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560a6b291090 0x560a6b2a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.768+0000 7fa5033fe740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b2919d0 0x560a6b2a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.768+0000 7fa5033fe740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f81a9480 con 0x560a6b28c700 2024-09-15T22:59:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.768+0000 7fa5033fe740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f81aa8f0 con 0x560a6b2919d0 2024-09-15T22:59:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.768+0000 7fa5033fe740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f81a9b50 con 0x560a6b291090 2024-09-15T22:59:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.768+0000 7fa5033fe740 10 client.0 _ll_register_callbacks cb 0x7fa4f0003b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:59:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.768+0000 7fa4fe600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b2919d0 0x560a6b2a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.768+0000 7fa4fdc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a6b28c700 0x560a6b29fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:25.773 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.768+0000 7fa4fd200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560a6b291090 0x560a6b2a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.768+0000 7fa4fe600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b2919d0 0x560a6b2a10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:34322/0 (socket says 172.21.15.111:34322) 2024-09-15T22:59:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.768+0000 7fa4fe600640 1 -- 172.21.15.111:0/2776877662 learned_addr learned my addr 172.21.15.111:0/2776877662 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:59:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.768+0000 7fa4fe600640 1 -- 172.21.15.111:0/2776877662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560a6b291090 msgr2=0x560a6b2a01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.768+0000 7fa4fe600640 1 --2- 172.21.15.111:0/2776877662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560a6b291090 0x560a6b2a01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.768+0000 7fa4fe600640 1 -- 172.21.15.111:0/2776877662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a6b28c700 msgr2=0x560a6b29fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.768+0000 7fa4fe600640 1 --2- 172.21.15.111:0/2776877662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a6b28c700 0x560a6b29fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.768+0000 7fa4fe600640 1 -- 172.21.15.111:0/2776877662 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a6b2a16e0 con 0x560a6b2919d0 2024-09-15T22:59:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.768+0000 7fa4fd200640 1 --2- 172.21.15.111:0/2776877662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560a6b291090 0x560a6b2a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:59:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.769+0000 7fa4fe600640 1 --2- 172.21.15.111:0/2776877662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b2919d0 0x560a6b2a10d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0030960 tx=0x7fa4f00bb110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4af78c3f58545c5a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:59:25.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.769+0000 7fa4df400640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:59:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.769+0000 7fa4df400640 1 -- 172.21.15.111:0/2776877662 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f00bc9d0 con 0x560a6b2919d0 2024-09-15T22:59:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.769+0000 7fa5033fe740 1 -- 172.21.15.111:0/2776877662 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4ec004020 con 0x560a6b2919d0 2024-09-15T22:59:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.769+0000 7fa4df400640 1 -- 172.21.15.111:0/2776877662 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fa4f00bce60 con 0x560a6b2919d0 2024-09-15T22:59:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.769+0000 7fa4df400640 1 -- 172.21.15.111:0/2776877662 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa4f00b8ea0 con 0x560a6b2919d0 2024-09-15T22:59:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.769+0000 7fa5033fe740 10 client.26051 fetch_fsmap learned FSMap version 30 2024-09-15T22:59:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.769+0000 7fa5033fe740 1 -- 172.21.15.111:0/2776877662 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560a6b2a2a60 con 0x560a6b2919d0 2024-09-15T22:59:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.769+0000 7fa4df400640 1 -- 172.21.15.111:0/2776877662 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4ec004020 con 0x560a6b2919d0 2024-09-15T22:59:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.769+0000 7fa4df400640 1 -- 172.21.15.111:0/2776877662 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8346+0+0 (secure 0 0 0) 0x7fa4f00bc1f0 con 0x560a6b2919d0 2024-09-15T22:59:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.769+0000 7fa5033fe740 10 client.26051 fetch_fsmap finished waiting for FSMap version 30 2024-09-15T22:59:25.775 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.769+0000 7fa5033fe740 10 client.26051 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:59:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.769+0000 7fa5033fe740 20 client.26051 populate_metadata read hostname 'smithi111' 2024-09-15T22:59:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.769+0000 7fa5033fe740 1 --2- 172.21.15.111:0/2776877662 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560a6b296a60 0x560a6b31c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.769+0000 7fa5033fe740 4 client.26051 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:59:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.769+0000 7fa5033fe740 1 -- 172.21.15.111:0/2776877662 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560a6b2a9510 con 0x560a6b296a60 2024-09-15T22:59:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.770+0000 7fa4fdc00640 1 --2- 172.21.15.111:0/2776877662 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560a6b296a60 0x560a6b31c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.770+0000 7fa4fdc00640 1 --2- 172.21.15.111:0/2776877662 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560a6b296a60 0x560a6b31c770 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92470d95247f8da8 server_cookie=26d605ddbedeb5a9 in_seq=0 out_seq=0 2024-09-15T22:59:25.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.770+0000 7fa4df400640 10 client.26051 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:59:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.782+0000 7fa4df400640 1 -- 172.21.15.111:0/2776877662 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560a6b2a9510 con 0x560a6b296a60 2024-09-15T22:59:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.782+0000 7fa4df400640 10 client.26051 handle_command_reply: tid=0 2024-09-15T22:59:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.782+0000 7fa5033fe740 1 client.26051 shutdown 2024-09-15T22:59:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.782+0000 7fa5033fe740 1 -- 172.21.15.111:0/2776877662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b2919d0 msgr2=0x560a6b2a10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.782+0000 7fa5033fe740 1 --2- 172.21.15.111:0/2776877662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b2919d0 0x560a6b2a10d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0030960 tx=0x7fa4f00bb110 comp rx=0 tx=0).stop 2024-09-15T22:59:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.782+0000 7fa5033fe740 1 -- 172.21.15.111:0/2776877662 shutdown_connections 2024-09-15T22:59:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.782+0000 7fa5033fe740 1 --2- 172.21.15.111:0/2776877662 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560a6b296a60 0x560a6b31c770 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.782+0000 7fa5033fe740 1 --2- 172.21.15.111:0/2776877662 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560a6b2919d0 0x560a6b2a10d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.782+0000 7fa5033fe740 1 --2- 172.21.15.111:0/2776877662 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560a6b291090 0x560a6b2a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.782+0000 7fa5033fe740 1 --2- 172.21.15.111:0/2776877662 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560a6b28c700 0x560a6b29fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.782+0000 7fa4df400640 0 client.26051 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:59:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.782+0000 7fa5033fe740 1 -- 172.21.15.111:0/2776877662 >> 172.21.15.111:0/2776877662 conn(0x560a6b2965f0 msgr2=0x560a6b28b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:59:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.782+0000 7fa5033fe740 1 -- 172.21.15.111:0/2776877662 shutdown_connections 2024-09-15T22:59:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.782+0000 7fa5033fe740 1 -- 172.21.15.111:0/2776877662 wait complete. 2024-09-15T22:59:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.782+0000 7fa5033fe740 20 client.26051 trim_cache size 0 max 16384 2024-09-15T22:59:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:59:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:59:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:59:25.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:59:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.783+0000 7fa500600640 1 -- 172.21.15.111:0/246474834 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa4e00a32f0 msgr2=0x7fa4e00a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.783+0000 7fa500600640 1 --2- 172.21.15.111:0/246474834 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa4e00a32f0 0x7fa4e00a57b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00bfd0 tx=0x7fa4ec00b4a0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.783+0000 7fa500600640 1 -- 172.21.15.111:0/246474834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f8103710 msgr2=0x7fa4f81a1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.783+0000 7fa500600640 1 --2- 172.21.15.111:0/246474834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f8103710 0x7fa4f81a1500 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0034040 tx=0x7fa4f00305b0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.783+0000 7fa500600640 1 -- 172.21.15.111:0/246474834 shutdown_connections 2024-09-15T22:59:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.783+0000 7fa500600640 1 --2- 172.21.15.111:0/246474834 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa4e00a32f0 0x7fa4e00a57b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.783+0000 7fa500600640 1 --2- 172.21.15.111:0/246474834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f8103710 0x7fa4f81a1500 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.783+0000 7fa500600640 1 --2- 172.21.15.111:0/246474834 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa4f8102d50 0x7fa4f819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.783+0000 7fa500600640 1 --2- 172.21.15.111:0/246474834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4f8101b60 0x7fa4f8199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.783+0000 7fa500600640 1 -- 172.21.15.111:0/246474834 >> 172.21.15.111:0/246474834 conn(0x7fa4f80fd2e0 msgr2=0x7fa4f8074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:59:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.783+0000 7fa500600640 1 -- 172.21.15.111:0/246474834 shutdown_connections 2024-09-15T22:59:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:25.783+0000 7fa500600640 1 -- 172.21.15.111:0/246474834 wait complete. 2024-09-15T22:59:26.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:25 smithi121 ceph-mon[34815]: pgmap v755: 97 pgs: 97 active+clean; 273 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 188 op/s 2024-09-15T22:59:26.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:25 smithi111 ceph-mon[30629]: pgmap v755: 97 pgs: 97 active+clean; 273 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 188 op/s 2024-09-15T22:59:26.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:25 smithi192 ceph-mon[35427]: pgmap v755: 97 pgs: 97 active+clean; 273 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 188 op/s 2024-09-15T22:59:26.337 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1310523 9.79 MB/sec execute 594 sec latency 20.988 ms 2024-09-15T22:59:27.337 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1312655 9.79 MB/sec execute 595 sec latency 19.467 ms 2024-09-15T22:59:28.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:27 smithi121 ceph-mon[34815]: pgmap v756: 97 pgs: 97 active+clean; 274 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 225 op/s 2024-09-15T22:59:28.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:27 smithi111 ceph-mon[30629]: pgmap v756: 97 pgs: 97 active+clean; 274 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 225 op/s 2024-09-15T22:59:28.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:27 smithi192 ceph-mon[35427]: pgmap v756: 97 pgs: 97 active+clean; 274 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 225 op/s 2024-09-15T22:59:28.337 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1314664 9.79 MB/sec execute 596 sec latency 20.102 ms 2024-09-15T22:59:29.337 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1317336 9.80 MB/sec execute 597 sec latency 161.352 ms 2024-09-15T22:59:29.945 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:29 smithi111 ceph-mon[30629]: pgmap v757: 97 pgs: 97 active+clean; 274 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 187 op/s 2024-09-15T22:59:30.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:29 smithi121 ceph-mon[34815]: pgmap v757: 97 pgs: 97 active+clean; 274 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 187 op/s 2024-09-15T22:59:30.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:29 smithi192 ceph-mon[35427]: pgmap v757: 97 pgs: 97 active+clean; 274 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 187 op/s 2024-09-15T22:59:30.338 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1317336 9.79 MB/sec execute 598 sec latency 1161.422 ms 2024-09-15T22:59:31.338 INFO:tasks.workunit.client.0.smithi111.stdout: 1 1318998 9.79 MB/sec execute 599 sec latency 1291.135 ms 2024-09-15T22:59:32.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:31 smithi121 ceph-mon[34815]: pgmap v758: 97 pgs: 97 active+clean; 272 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 166 op/s 2024-09-15T22:59:32.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:31 smithi111 ceph-mon[30629]: pgmap v758: 97 pgs: 97 active+clean; 272 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 166 op/s 2024-09-15T22:59:32.322 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:31 smithi192 ceph-mon[35427]: pgmap v758: 97 pgs: 97 active+clean; 272 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 166 op/s 2024-09-15T22:59:32.338 INFO:tasks.workunit.client.0.smithi111.stdout: 1 cleanup 600 sec 2024-09-15T22:59:32.476 INFO:tasks.workunit.client.0.smithi111.stdout: 0 cleanup 600 sec 2024-09-15T22:59:32.476 INFO:tasks.workunit.client.0.smithi111.stdout: 2024-09-15T22:59:32.476 INFO:tasks.workunit.client.0.smithi111.stdout: Operation Count AvgLat MaxLat 2024-09-15T22:59:32.476 INFO:tasks.workunit.client.0.smithi111.stdout: ---------------------------------------- 2024-09-15T22:59:32.477 INFO:tasks.workunit.client.0.smithi111.stdout: NTCreateX 187449 1.211 3292.340 2024-09-15T22:59:32.477 INFO:tasks.workunit.client.0.smithi111.stdout: Close 137641 0.004 0.515 2024-09-15T22:59:32.477 INFO:tasks.workunit.client.0.smithi111.stdout: Rename 7934 4.285 1734.399 2024-09-15T22:59:32.477 INFO:tasks.workunit.client.0.smithi111.stdout: Unlink 37890 2.384 2137.314 2024-09-15T22:59:32.477 INFO:tasks.workunit.client.0.smithi111.stdout: Deltree 4 68.328 98.529 2024-09-15T22:59:32.477 INFO:tasks.workunit.client.0.smithi111.stdout: Mkdir 2 0.032 0.040 2024-09-15T22:59:32.477 INFO:tasks.workunit.client.0.smithi111.stdout: Qpathinfo 169900 0.924 2266.453 2024-09-15T22:59:32.477 INFO:tasks.workunit.client.0.smithi111.stdout: Qfileinfo 29732 0.002 0.070 2024-09-15T22:59:32.477 INFO:tasks.workunit.client.0.smithi111.stdout: Qfsinfo 31143 0.201 12.510 2024-09-15T22:59:32.477 INFO:tasks.workunit.client.0.smithi111.stdout: Sfileinfo 15258 0.027 1.167 2024-09-15T22:59:32.478 INFO:tasks.workunit.client.0.smithi111.stdout: Find 65657 0.188 1457.197 2024-09-15T22:59:32.478 INFO:tasks.workunit.client.0.smithi111.stdout: WriteX 93277 0.028 54.382 2024-09-15T22:59:32.478 INFO:tasks.workunit.client.0.smithi111.stdout: ReadX 293672 0.018 41.166 2024-09-15T22:59:32.478 INFO:tasks.workunit.client.0.smithi111.stdout: LockX 610 0.755 2.663 2024-09-15T22:59:32.478 INFO:tasks.workunit.client.0.smithi111.stdout: UnlockX 610 0.623 1.872 2024-09-15T22:59:32.478 INFO:tasks.workunit.client.0.smithi111.stdout: Flush 13127 4.706 1783.961 2024-09-15T22:59:32.478 INFO:tasks.workunit.client.0.smithi111.stdout: 2024-09-15T22:59:32.478 INFO:tasks.workunit.client.0.smithi111.stdout:Throughput 9.78536 MB/sec 1 clients 1 procs max_latency=3292.347 ms 2024-09-15T22:59:32.478 INFO:tasks.workunit.client.0.smithi111.stderr:+ dbench 10 2024-09-15T22:59:32.479 INFO:tasks.workunit.client.0.smithi111.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-15T22:59:32.479 INFO:tasks.workunit.client.0.smithi111.stdout: 2024-09-15T22:59:32.479 INFO:tasks.workunit.client.0.smithi111.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-15T22:59:32.480 INFO:tasks.workunit.client.0.smithi111.stdout:9 of 10 processes prepared for launch 0 sec 2024-09-15T22:59:32.581 INFO:tasks.workunit.client.0.smithi111.stdout:10 of 10 processes prepared for launch 0 sec 2024-09-15T22:59:32.581 INFO:tasks.workunit.client.0.smithi111.stdout:releasing clients 2024-09-15T22:59:33.581 INFO:tasks.workunit.client.0.smithi111.stdout: 10 461 161.92 MB/sec warmup 1 sec latency 35.780 ms 2024-09-15T22:59:34.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:33 smithi121 ceph-mon[34815]: pgmap v759: 97 pgs: 97 active+clean; 275 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 195 op/s 2024-09-15T22:59:34.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:33 smithi111 ceph-mon[30629]: pgmap v759: 97 pgs: 97 active+clean; 275 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 195 op/s 2024-09-15T22:59:34.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:33 smithi192 ceph-mon[35427]: pgmap v759: 97 pgs: 97 active+clean; 275 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 195 op/s 2024-09-15T22:59:34.581 INFO:tasks.workunit.client.0.smithi111.stdout: 10 855 107.88 MB/sec warmup 2 sec latency 349.539 ms 2024-09-15T22:59:35.581 INFO:tasks.workunit.client.0.smithi111.stdout: 10 856 71.93 MB/sec warmup 3 sec latency 1321.218 ms 2024-09-15T22:59:35.985 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:35 smithi111 ceph-mon[30629]: pgmap v760: 97 pgs: 97 active+clean; 275 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 199 op/s 2024-09-15T22:59:36.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:35 smithi121 ceph-mon[34815]: pgmap v760: 97 pgs: 97 active+clean; 275 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 199 op/s 2024-09-15T22:59:36.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:35 smithi192 ceph-mon[35427]: pgmap v760: 97 pgs: 97 active+clean; 275 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 199 op/s 2024-09-15T22:59:36.581 INFO:tasks.workunit.client.0.smithi111.stdout: 10 856 53.95 MB/sec warmup 4 sec latency 2321.257 ms 2024-09-15T22:59:37.581 INFO:tasks.workunit.client.0.smithi111.stdout: 10 1236 46.14 MB/sec warmup 5 sec latency 2567.656 ms 2024-09-15T22:59:38.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:37 smithi192 ceph-mon[35427]: pgmap v761: 97 pgs: 97 active+clean; 268 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 209 op/s 2024-09-15T22:59:38.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:37 smithi121 ceph-mon[34815]: pgmap v761: 97 pgs: 97 active+clean; 268 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 209 op/s 2024-09-15T22:59:38.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:37 smithi111 ceph-mon[30629]: pgmap v761: 97 pgs: 97 active+clean; 268 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 209 op/s 2024-09-15T22:59:38.581 INFO:tasks.workunit.client.0.smithi111.stdout: 10 2435 47.97 MB/sec warmup 6 sec latency 173.980 ms 2024-09-15T22:59:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.917+0000 7f0cfc800640 1 -- 172.21.15.111:0/1426843297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0cf4103d60 msgr2=0x7f0cf4104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.917+0000 7f0cfc800640 1 --2- 172.21.15.111:0/1426843297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0cf4103d60 0x7f0cf4104160 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009900 tx=0x7f0ce802a410 comp rx=0 tx=0).stop 2024-09-15T22:59:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.917+0000 7f0cfc800640 1 -- 172.21.15.111:0/1426843297 shutdown_connections 2024-09-15T22:59:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.917+0000 7f0cfc800640 1 --2- 172.21.15.111:0/1426843297 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cf4105910 0x7f0cf410c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.917+0000 7f0cfc800640 1 --2- 172.21.15.111:0/1426843297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cf4104f50 0x7f0cf41053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.917+0000 7f0cfc800640 1 --2- 172.21.15.111:0/1426843297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0cf4103d60 0x7f0cf4104160 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.917+0000 7f0cfc800640 1 -- 172.21.15.111:0/1426843297 >> 172.21.15.111:0/1426843297 conn(0x7f0cf40ff4e0 msgr2=0x7f0cf4101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:59:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.917+0000 7f0cfc800640 1 -- 172.21.15.111:0/1426843297 shutdown_connections 2024-09-15T22:59:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.918+0000 7f0cfc800640 1 -- 172.21.15.111:0/1426843297 wait complete. 2024-09-15T22:59:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.918+0000 7f0cfc800640 1 Processor -- start 2024-09-15T22:59:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.919+0000 7f0cfc800640 1 -- start start 2024-09-15T22:59:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.919+0000 7f0cfc800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cf4103d60 0x7f0cf4199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.919+0000 7f0cfc800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cf4104f50 0x7f0cf419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.919+0000 7f0cfc800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0cf4105910 0x7f0cf41a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.919+0000 7f0cfc800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf410fa10 con 0x7f0cf4103d60 2024-09-15T22:59:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.919+0000 7f0cfc800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf410f890 con 0x7f0cf4105910 2024-09-15T22:59:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.919+0000 7f0cfc800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf410fb90 con 0x7f0cf4104f50 2024-09-15T22:59:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.919+0000 7f0cf9e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cf4103d60 0x7f0cf4199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.919+0000 7f0cf9e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cf4103d60 0x7f0cf4199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56518/0 (socket says 172.21.15.111:56518) 2024-09-15T22:59:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.919+0000 7f0cfa800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0cf4105910 0x7f0cf41a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.919+0000 7f0cf9e00640 1 -- 172.21.15.111:0/1375029148 learned_addr learned my addr 172.21.15.111:0/1375029148 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:59:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.919+0000 7f0cf9400640 1 --2- 172.21.15.111:0/1375029148 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cf4104f50 0x7f0cf419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.920+0000 7f0cfa800640 1 -- 172.21.15.111:0/1375029148 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cf4104f50 msgr2=0x7f0cf419a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.920+0000 7f0cfa800640 1 --2- 172.21.15.111:0/1375029148 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cf4104f50 0x7f0cf419a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.920+0000 7f0cfa800640 1 -- 172.21.15.111:0/1375029148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cf4103d60 msgr2=0x7f0cf4199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.920+0000 7f0cfa800640 1 --2- 172.21.15.111:0/1375029148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cf4103d60 0x7f0cf4199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.920+0000 7f0cfa800640 1 -- 172.21.15.111:0/1375029148 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf41a1bf0 con 0x7f0cf4105910 2024-09-15T22:59:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.920+0000 7f0cfa800640 1 --2- 172.21.15.111:0/1375029148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0cf4105910 0x7f0cf41a1580 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009fd0 tx=0x7f0cf000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:59:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.920+0000 7f0ceea00640 1 -- 172.21.15.111:0/1375029148 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf000ce20 con 0x7f0cf4105910 2024-09-15T22:59:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.920+0000 7f0ceea00640 1 -- 172.21.15.111:0/1375029148 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf0005020 con 0x7f0cf4105910 2024-09-15T22:59:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.920+0000 7f0cfc800640 1 -- 172.21.15.111:0/1375029148 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf41a1ee0 con 0x7f0cf4105910 2024-09-15T22:59:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.920+0000 7f0ceea00640 1 -- 172.21.15.111:0/1375029148 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0010720 con 0x7f0cf4105910 2024-09-15T22:59:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.921+0000 7f0cfc800640 1 -- 172.21.15.111:0/1375029148 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf41a2350 con 0x7f0cf4105910 2024-09-15T22:59:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.925+0000 7f0cff87f740 1 Processor -- start 2024-09-15T22:59:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.925+0000 7f0ceea00640 1 -- 172.21.15.111:0/1375029148 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f0cf0005540 con 0x7f0cf4105910 2024-09-15T22:59:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.926+0000 7f0cff87f740 1 -- start start 2024-09-15T22:59:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.926+0000 7f0cff87f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555bda5d7090 0x555bda5d7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.926+0000 7f0cff87f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555bda5d79d0 0x555bda5dc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.926+0000 7f0cff87f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5dca60 0x555bda5def20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.926+0000 7f0cff87f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555bda53c100 con 0x555bda5d79d0 2024-09-15T22:59:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.926+0000 7f0cff87f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x555bda4e41b0 con 0x555bda5d7090 2024-09-15T22:59:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.926+0000 7f0cff87f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555bda5380c0 con 0x555bda5dca60 2024-09-15T22:59:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.927+0000 7f0cf9400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5dca60 0x555bda5def20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.927+0000 7f0cf9400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5dca60 0x555bda5def20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:53146/0 (socket says 172.21.15.111:53146) 2024-09-15T22:59:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.927+0000 7f0cf9400640 1 -- 172.21.15.111:0/2600470694 learned_addr learned my addr 172.21.15.111:0/2600470694 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:59:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.928+0000 7f0cf9400640 1 --2- 172.21.15.111:0/2600470694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555bda5d7090 0x555bda5d7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.928+0000 7f0cf9400640 1 -- 172.21.15.111:0/2600470694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555bda5d7090 msgr2=0x555bda5d7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.928+0000 7f0cf9400640 1 --2- 172.21.15.111:0/2600470694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555bda5d7090 0x555bda5d7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.928+0000 7f0cf9400640 1 -- 172.21.15.111:0/2600470694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555bda5d79d0 msgr2=0x555bda5dc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.928+0000 7f0ceea00640 1 --2- 172.21.15.111:0/1375029148 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0cdc0a32f0 0x7f0cdc0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.928+0000 7f0ceea00640 1 -- 172.21.15.111:0/1375029148 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f0cf00cfee0 con 0x7f0cf4105910 2024-09-15T22:59:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.929+0000 7f0cf9e00640 1 --2- 172.21.15.111:0/2600470694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555bda5d79d0 0x555bda5dc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.929+0000 7f0cf9400640 1 --2- 172.21.15.111:0/2600470694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555bda5d79d0 0x555bda5dc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.929+0000 7f0cf9400640 1 -- 172.21.15.111:0/2600470694 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bda24f210 con 0x555bda5dca60 2024-09-15T22:59:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.929+0000 7f0cf9e00640 1 --2- 172.21.15.111:0/2600470694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555bda5d79d0 0x555bda5dc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:59:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.929+0000 7f0cf9400640 1 --2- 172.21.15.111:0/2600470694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5dca60 0x555bda5def20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4012810 tx=0x7f0ce40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb3236d1ce941be9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:59:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.929+0000 7f0cf9400640 1 --2- 172.21.15.111:0/1375029148 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0cdc0a32f0 0x7f0cdc0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.930+0000 7f0ce3000640 1 -- 172.21.15.111:0/2600470694 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4012d90 con 0x555bda5dca60 2024-09-15T22:59:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.930+0000 7f0ce3000640 1 -- 172.21.15.111:0/2600470694 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ce4010c50 con 0x555bda5dca60 2024-09-15T22:59:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.930+0000 7f0ce3000640 1 -- 172.21.15.111:0/2600470694 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce401d370 con 0x555bda5dca60 2024-09-15T22:59:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.931+0000 7f0cff87f740 1 -- 172.21.15.111:0/2600470694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5dca60 msgr2=0x555bda5def20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.931+0000 7f0cff87f740 1 --2- 172.21.15.111:0/2600470694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5dca60 0x555bda5def20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4012810 tx=0x7f0ce40046c0 comp rx=0 tx=0).stop 2024-09-15T22:59:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.931+0000 7f0cf9400640 1 --2- 172.21.15.111:0/1375029148 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0cdc0a32f0 0x7f0cdc0a57b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0ce401e000 tx=0x7f0ce401fe90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:59:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.932+0000 7f0cff87f740 1 -- 172.21.15.111:0/2600470694 shutdown_connections 2024-09-15T22:59:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.932+0000 7f0cff87f740 1 --2- 172.21.15.111:0/2600470694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5dca60 0x555bda5def20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:38.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.932+0000 7f0cff87f740 1 --2- 172.21.15.111:0/2600470694 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555bda5d79d0 0x555bda5dc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.932+0000 7f0cff87f740 1 --2- 172.21.15.111:0/2600470694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555bda5d7090 0x555bda5d7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.932+0000 7f0cff87f740 1 -- 172.21.15.111:0/2600470694 >> 172.21.15.111:0/2600470694 conn(0x555bda5d2700 msgr2=0x555bda5d2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:59:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.932+0000 7f0cff87f740 1 -- 172.21.15.111:0/2600470694 shutdown_connections 2024-09-15T22:59:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.932+0000 7f0cff87f740 1 -- 172.21.15.111:0/2600470694 wait complete. 2024-09-15T22:59:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.932+0000 7f0cff87f740 1 -- start start 2024-09-15T22:59:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.933+0000 7f0cff87f740 1 Processor -- start 2024-09-15T22:59:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.933+0000 7f0cff87f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5d7090 0x555bda5e1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.933+0000 7f0cff87f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555bda5d79d0 0x555bda5e1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.933+0000 7f0cff87f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555bda5dca60 0x555bda5e8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.933+0000 7f0cff87f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555bda5dfa30 con 0x555bda5dca60 2024-09-15T22:59:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.933+0000 7f0cff87f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x555bda4c5090 con 0x555bda5d79d0 2024-09-15T22:59:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.933+0000 7f0cff87f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555bda5dfbb0 con 0x555bda5d7090 2024-09-15T22:59:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.933+0000 7f0cff87f740 10 client.0 _ll_register_callbacks cb 0x555bda519f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:59:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.934+0000 7f0cf9e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5d7090 0x555bda5e1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.934+0000 7f0cf9e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5d7090 0x555bda5e1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:53160/0 (socket says 172.21.15.111:53160) 2024-09-15T22:59:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.934+0000 7f0cf9e00640 1 -- 172.21.15.111:0/2335175905 learned_addr learned my addr 172.21.15.111:0/2335175905 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:59:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.934+0000 7f0cfa800640 1 --2- 172.21.15.111:0/2335175905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555bda5dca60 0x555bda5e8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.934+0000 7f0cf9400640 1 --2- 172.21.15.111:0/2335175905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555bda5d79d0 0x555bda5e1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.934+0000 7f0cf9e00640 1 -- 172.21.15.111:0/2335175905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555bda5d79d0 msgr2=0x555bda5e1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.934+0000 7f0cf9e00640 1 --2- 172.21.15.111:0/2335175905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555bda5d79d0 0x555bda5e1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.934+0000 7f0cf9e00640 1 -- 172.21.15.111:0/2335175905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555bda5dca60 msgr2=0x555bda5e8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.934+0000 7f0cf9e00640 1 --2- 172.21.15.111:0/2335175905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555bda5dca60 0x555bda5e8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.934+0000 7f0cf9e00640 1 -- 172.21.15.111:0/2335175905 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bda31b620 con 0x555bda5d7090 2024-09-15T22:59:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.935+0000 7f0cf9e00640 1 --2- 172.21.15.111:0/2335175905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5d7090 0x555bda5e1210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0ce802a930 tx=0x7f0ce80043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cade3a4d5c15e37c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:59:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.935+0000 7f0cd7400640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:59:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.935+0000 7f0cd7400640 1 -- 172.21.15.111:0/2335175905 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8040070 con 0x555bda5d7090 2024-09-15T22:59:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.935+0000 7f0cd7400640 1 -- 172.21.15.111:0/2335175905 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f0ce8031ac0 con 0x555bda5d7090 2024-09-15T22:59:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.936+0000 7f0cff87f740 1 -- 172.21.15.111:0/2335175905 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ce4010c50 con 0x555bda5d7090 2024-09-15T22:59:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.936+0000 7f0cd7400640 1 -- 172.21.15.111:0/2335175905 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0ce8035040 con 0x555bda5d7090 2024-09-15T22:59:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.936+0000 7f0cff87f740 10 client.26086 fetch_fsmap learned FSMap version 31 2024-09-15T22:59:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.936+0000 7f0cff87f740 1 -- 172.21.15.111:0/2335175905 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555bda5eb560 con 0x555bda5d7090 2024-09-15T22:59:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.936+0000 7f0cd7400640 1 -- 172.21.15.111:0/2335175905 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ce4010c50 con 0x555bda5d7090 2024-09-15T22:59:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.937+0000 7f0cd7400640 1 -- 172.21.15.111:0/2335175905 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f0ce8004bb0 con 0x555bda5d7090 2024-09-15T22:59:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.937+0000 7f0cff87f740 10 client.26086 fetch_fsmap finished waiting for FSMap version 31 2024-09-15T22:59:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.937+0000 7f0cff87f740 10 client.26086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:59:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.937+0000 7f0cff87f740 20 client.26086 populate_metadata read hostname 'smithi111' 2024-09-15T22:59:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.937+0000 7f0cff87f740 1 --2- 172.21.15.111:0/2335175905 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555bda5ecca0 0x555bda5ef140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.937+0000 7f0cff87f740 4 client.26086 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:59:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.937+0000 7f0cff87f740 1 -- 172.21.15.111:0/2335175905 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bda549830 con 0x555bda5ecca0 2024-09-15T22:59:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.937+0000 7f0cf9400640 1 --2- 172.21.15.111:0/2335175905 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555bda5ecca0 0x555bda5ef140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.938+0000 7f0cf9400640 1 --2- 172.21.15.111:0/2335175905 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555bda5ecca0 0x555bda5ef140 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ead48b13a49ef126 server_cookie=b6c23669b5fec14c in_seq=0 out_seq=0 2024-09-15T22:59:38.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.938+0000 7f0cd7400640 10 client.26086 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:59:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.943+0000 7f0cd7400640 1 -- 172.21.15.111:0/2335175905 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555bda549830 con 0x555bda5ecca0 2024-09-15T22:59:38.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.943+0000 7f0cd7400640 10 client.26086 handle_command_reply: tid=0 2024-09-15T22:59:38.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.943+0000 7f0cff87f740 1 client.26086 shutdown 2024-09-15T22:59:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.944+0000 7f0cff87f740 1 -- 172.21.15.111:0/2335175905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5d7090 msgr2=0x555bda5e1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.944+0000 7f0cff87f740 1 --2- 172.21.15.111:0/2335175905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5d7090 0x555bda5e1210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0ce802a930 tx=0x7f0ce80043b0 comp rx=0 tx=0).stop 2024-09-15T22:59:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.944+0000 7f0cff87f740 1 -- 172.21.15.111:0/2335175905 shutdown_connections 2024-09-15T22:59:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.944+0000 7f0cff87f740 1 --2- 172.21.15.111:0/2335175905 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555bda5ecca0 0x555bda5ef140 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.944+0000 7f0cff87f740 1 --2- 172.21.15.111:0/2335175905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555bda5dca60 0x555bda5e8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.944+0000 7f0cff87f740 1 --2- 172.21.15.111:0/2335175905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555bda5d79d0 0x555bda5e1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.944+0000 7f0cff87f740 1 --2- 172.21.15.111:0/2335175905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5d7090 0x555bda5e1210 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.944+0000 7f0cd7400640 0 client.26086 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:59:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.944+0000 7f0cff87f740 1 -- 172.21.15.111:0/2335175905 >> 172.21.15.111:0/2335175905 conn(0x555bda5d2700 msgr2=0x555bda5d3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:59:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.944+0000 7f0cff87f740 1 -- 172.21.15.111:0/2335175905 shutdown_connections 2024-09-15T22:59:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.944+0000 7f0cff87f740 1 -- 172.21.15.111:0/2335175905 wait complete. 2024-09-15T22:59:38.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.944+0000 7f0cff87f740 20 client.26086 trim_cache size 0 max 16384 2024-09-15T22:59:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.961+0000 7f0cff87f740 1 Processor -- start 2024-09-15T22:59:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.961+0000 7f0cff87f740 1 -- start start 2024-09-15T22:59:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.961+0000 7f0cff87f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5d2700 0x555bda5e1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.961+0000 7f0cff87f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555bda5d7090 0x555bda5e8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.961+0000 7f0cff87f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555bda5d79d0 0x555bda5d8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.961+0000 7f0cff87f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf41ad950 con 0x555bda5d79d0 2024-09-15T22:59:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.961+0000 7f0cff87f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf41ab1c0 con 0x555bda5d7090 2024-09-15T22:59:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.961+0000 7f0cff87f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf41a9010 con 0x555bda5d2700 2024-09-15T22:59:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.962+0000 7f0cf9e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5d2700 0x555bda5e1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.962+0000 7f0cf9e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5d2700 0x555bda5e1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:53176/0 (socket says 172.21.15.111:53176) 2024-09-15T22:59:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.962+0000 7f0cf9e00640 1 -- 172.21.15.111:0/1697298176 learned_addr learned my addr 172.21.15.111:0/1697298176 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:59:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.962+0000 7f0cf9400640 1 --2- 172.21.15.111:0/1697298176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555bda5d7090 0x555bda5e8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.962+0000 7f0cfa800640 1 --2- 172.21.15.111:0/1697298176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555bda5d79d0 0x555bda5d8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.962+0000 7f0cf9400640 1 -- 172.21.15.111:0/1697298176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5d2700 msgr2=0x555bda5e1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.962+0000 7f0cf9400640 1 --2- 172.21.15.111:0/1697298176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5d2700 0x555bda5e1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.962+0000 7f0cf9400640 1 -- 172.21.15.111:0/1697298176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555bda5d79d0 msgr2=0x555bda5d8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.962+0000 7f0cf9400640 1 --2- 172.21.15.111:0/1697298176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555bda5d79d0 0x555bda5d8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.962+0000 7f0cf9400640 1 -- 172.21.15.111:0/1697298176 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bda5cdd60 con 0x555bda5d7090 2024-09-15T22:59:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.962+0000 7f0cf9e00640 1 --2- 172.21.15.111:0/1697298176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5d2700 0x555bda5e1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:59:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.962+0000 7f0cf9400640 1 --2- 172.21.15.111:0/1697298176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555bda5d7090 0x555bda5e8750 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400bfd0 tx=0x7f0ce4028000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e68107462808aacf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:59:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.963+0000 7f0ce2600640 1 -- 172.21.15.111:0/1697298176 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce401d370 con 0x555bda5d7090 2024-09-15T22:59:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.963+0000 7f0ce2600640 1 -- 172.21.15.111:0/1697298176 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ce4016040 con 0x555bda5d7090 2024-09-15T22:59:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.963+0000 7f0ce2600640 1 -- 172.21.15.111:0/1697298176 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce401e950 con 0x555bda5d7090 2024-09-15T22:59:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.964+0000 7f0cff87f740 1 -- 172.21.15.111:0/1697298176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555bda5d7090 msgr2=0x555bda5e8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.964+0000 7f0cff87f740 1 --2- 172.21.15.111:0/1697298176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555bda5d7090 0x555bda5e8750 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400bfd0 tx=0x7f0ce4028000 comp rx=0 tx=0).stop 2024-09-15T22:59:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.964+0000 7f0cff87f740 1 -- 172.21.15.111:0/1697298176 shutdown_connections 2024-09-15T22:59:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.964+0000 7f0cff87f740 1 --2- 172.21.15.111:0/1697298176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555bda5d79d0 0x555bda5d8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.964+0000 7f0cff87f740 1 --2- 172.21.15.111:0/1697298176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555bda5d7090 0x555bda5e8750 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.964+0000 7f0cff87f740 1 --2- 172.21.15.111:0/1697298176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5d2700 0x555bda5e1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.964+0000 7f0cff87f740 1 -- 172.21.15.111:0/1697298176 >> 172.21.15.111:0/1697298176 conn(0x555bda5dc580 msgr2=0x555bda5d1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:59:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.964+0000 7f0cff87f740 1 -- 172.21.15.111:0/1697298176 shutdown_connections 2024-09-15T22:59:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.964+0000 7f0cff87f740 1 -- 172.21.15.111:0/1697298176 wait complete. 2024-09-15T22:59:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.964+0000 7f0cff87f740 1 -- start start 2024-09-15T22:59:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.964+0000 7f0cff87f740 1 Processor -- start 2024-09-15T22:59:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.965+0000 7f0cff87f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5d2700 0x555bda5e6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.965+0000 7f0cff87f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555bda5d7090 0x555bda5e6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.965+0000 7f0cff87f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555bda5d79d0 0x555bda5e7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.965+0000 7f0cff87f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf41a9680 con 0x555bda5d7090 2024-09-15T22:59:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.965+0000 7f0cff87f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf41aaaf0 con 0x555bda5d79d0 2024-09-15T22:59:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.965+0000 7f0cff87f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf41a9d50 con 0x555bda5d2700 2024-09-15T22:59:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.965+0000 7f0cff87f740 10 client.0 _ll_register_callbacks cb 0x7f0ce8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:59:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.965+0000 7f0cf9e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5d2700 0x555bda5e6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.965+0000 7f0cf9e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5d2700 0x555bda5e6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:53178/0 (socket says 172.21.15.111:53178) 2024-09-15T22:59:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.965+0000 7f0cf9e00640 1 -- 172.21.15.111:0/3348029460 learned_addr learned my addr 172.21.15.111:0/3348029460 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:59:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.965+0000 7f0cfa800640 1 --2- 172.21.15.111:0/3348029460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555bda5d79d0 0x555bda5e7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.965+0000 7f0cf9400640 1 --2- 172.21.15.111:0/3348029460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555bda5d7090 0x555bda5e6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.965+0000 7f0cf9e00640 1 -- 172.21.15.111:0/3348029460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555bda5d79d0 msgr2=0x555bda5e7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.965+0000 7f0cf9e00640 1 --2- 172.21.15.111:0/3348029460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555bda5d79d0 0x555bda5e7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.965+0000 7f0cf9e00640 1 -- 172.21.15.111:0/3348029460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555bda5d7090 msgr2=0x555bda5e6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.965+0000 7f0cf9e00640 1 --2- 172.21.15.111:0/3348029460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555bda5d7090 0x555bda5e6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.965+0000 7f0cf9e00640 1 -- 172.21.15.111:0/3348029460 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bda5e7b60 con 0x555bda5d2700 2024-09-15T22:59:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.966+0000 7f0cf9e00640 1 --2- 172.21.15.111:0/3348029460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5d2700 0x555bda5e6000 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8000c00 tx=0x7f0ce8009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fca5125b8030e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:59:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.966+0000 7f0cd7400640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:59:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.966+0000 7f0cd7400640 1 -- 172.21.15.111:0/3348029460 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8040040 con 0x555bda5d2700 2024-09-15T22:59:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.966+0000 7f0cff87f740 1 -- 172.21.15.111:0/3348029460 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ce4016040 con 0x555bda5d2700 2024-09-15T22:59:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.966+0000 7f0cd7400640 1 -- 172.21.15.111:0/3348029460 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f0ce80318c0 con 0x555bda5d2700 2024-09-15T22:59:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.966+0000 7f0cd7400640 1 -- 172.21.15.111:0/3348029460 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0ce8031ac0 con 0x555bda5d2700 2024-09-15T22:59:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.966+0000 7f0cff87f740 10 client.26092 fetch_fsmap learned FSMap version 31 2024-09-15T22:59:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.966+0000 7f0cff87f740 1 -- 172.21.15.111:0/3348029460 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555bda657c60 con 0x555bda5d2700 2024-09-15T22:59:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.966+0000 7f0cd7400640 1 -- 172.21.15.111:0/3348029460 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ce4016040 con 0x555bda5d2700 2024-09-15T22:59:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.967+0000 7f0cd7400640 1 -- 172.21.15.111:0/3348029460 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f0ce8004fa0 con 0x555bda5d2700 2024-09-15T22:59:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.967+0000 7f0cff87f740 10 client.26092 fetch_fsmap finished waiting for FSMap version 31 2024-09-15T22:59:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.967+0000 7f0cff87f740 10 client.26092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:59:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.967+0000 7f0cff87f740 20 client.26092 populate_metadata read hostname 'smithi111' 2024-09-15T22:59:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.967+0000 7f0cff87f740 1 --2- 172.21.15.111:0/3348029460 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555bda5dca60 0x555bda65a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.967+0000 7f0cff87f740 4 client.26092 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:59:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.967+0000 7f0cff87f740 1 -- 172.21.15.111:0/3348029460 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555bda543b80 con 0x555bda5dca60 2024-09-15T22:59:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.967+0000 7f0cf9400640 1 --2- 172.21.15.111:0/3348029460 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555bda5dca60 0x555bda65a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.968+0000 7f0cf9400640 1 --2- 172.21.15.111:0/3348029460 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555bda5dca60 0x555bda65a680 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d4b8a24df879fde server_cookie=19ebf4853c555b55 in_seq=0 out_seq=0 2024-09-15T22:59:38.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:38.968+0000 7f0cd7400640 10 client.26092 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:59:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:39.084+0000 7f0cd7400640 1 -- 172.21.15.111:0/3348029460 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555bda543b80 con 0x555bda5dca60 2024-09-15T22:59:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:39.084+0000 7f0cd7400640 10 client.26092 handle_command_reply: tid=0 2024-09-15T22:59:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:39.084+0000 7f0cff87f740 1 client.26092 shutdown 2024-09-15T22:59:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:39.084+0000 7f0cff87f740 1 -- 172.21.15.111:0/3348029460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5d2700 msgr2=0x555bda5e6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:39.084+0000 7f0cff87f740 1 --2- 172.21.15.111:0/3348029460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5d2700 0x555bda5e6000 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8000c00 tx=0x7f0ce8009680 comp rx=0 tx=0).stop 2024-09-15T22:59:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:39.084+0000 7f0cff87f740 1 -- 172.21.15.111:0/3348029460 shutdown_connections 2024-09-15T22:59:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:39.084+0000 7f0cff87f740 1 --2- 172.21.15.111:0/3348029460 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555bda5dca60 0x555bda65a680 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:39.084+0000 7f0cff87f740 1 --2- 172.21.15.111:0/3348029460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555bda5d79d0 0x555bda5e7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:39.084+0000 7f0cd7400640 0 client.26092 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:59:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:39.084+0000 7f0cff87f740 1 --2- 172.21.15.111:0/3348029460 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555bda5d7090 0x555bda5e6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:39.084+0000 7f0cff87f740 1 --2- 172.21.15.111:0/3348029460 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555bda5d2700 0x555bda5e6000 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:39.084+0000 7f0cff87f740 1 -- 172.21.15.111:0/3348029460 >> 172.21.15.111:0/3348029460 conn(0x555bda5dc580 msgr2=0x555bda5d1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:59:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:39.084+0000 7f0cff87f740 1 -- 172.21.15.111:0/3348029460 shutdown_connections 2024-09-15T22:59:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:39.084+0000 7f0cff87f740 1 -- 172.21.15.111:0/3348029460 wait complete. 2024-09-15T22:59:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:39.084+0000 7f0cff87f740 20 client.26092 trim_cache size 0 max 16384 2024-09-15T22:59:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:59:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:59:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:59:39.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:59:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:39.085+0000 7f0cfc800640 1 -- 172.21.15.111:0/1375029148 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0cdc0a32f0 msgr2=0x7f0cdc0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:39.085+0000 7f0cfc800640 1 --2- 172.21.15.111:0/1375029148 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0cdc0a32f0 0x7f0cdc0a57b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0ce401e000 tx=0x7f0ce401fe90 comp rx=0 tx=0).stop 2024-09-15T22:59:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:39.085+0000 7f0cfc800640 1 -- 172.21.15.111:0/1375029148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0cf4105910 msgr2=0x7f0cf41a1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:39.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:39.085+0000 7f0cfc800640 1 --2- 172.21.15.111:0/1375029148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0cf4105910 0x7f0cf41a1580 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009fd0 tx=0x7f0cf000ef10 comp rx=0 tx=0).stop 2024-09-15T22:59:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:39.085+0000 7f0cfc800640 1 -- 172.21.15.111:0/1375029148 shutdown_connections 2024-09-15T22:59:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:39.085+0000 7f0cfc800640 1 --2- 172.21.15.111:0/1375029148 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0cdc0a32f0 0x7f0cdc0a57b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:39.085+0000 7f0cfc800640 1 --2- 172.21.15.111:0/1375029148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0cf4105910 0x7f0cf41a1580 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:39.085+0000 7f0cfc800640 1 --2- 172.21.15.111:0/1375029148 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0cf4104f50 0x7f0cf419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:39.085+0000 7f0cfc800640 1 --2- 172.21.15.111:0/1375029148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0cf4103d60 0x7f0cf4199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:39.085+0000 7f0cfc800640 1 -- 172.21.15.111:0/1375029148 >> 172.21.15.111:0/1375029148 conn(0x7f0cf40ff4e0 msgr2=0x7f0cf4108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:59:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:39.085+0000 7f0cfc800640 1 -- 172.21.15.111:0/1375029148 shutdown_connections 2024-09-15T22:59:39.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:39.085+0000 7f0cfc800640 1 -- 172.21.15.111:0/1375029148 wait complete. 2024-09-15T22:59:39.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:38 smithi121 ceph-mon[34815]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:59:39.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:38 smithi111 ceph-mon[30629]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:59:39.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:38 smithi192 ceph-mon[35427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:59:39.581 INFO:tasks.workunit.client.0.smithi111.stdout: 10 3269 49.82 MB/sec warmup 7 sec latency 205.003 ms 2024-09-15T22:59:39.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:39 smithi111 ceph-mon[30629]: pgmap v762: 97 pgs: 97 active+clean; 273 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 240 op/s 2024-09-15T22:59:40.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:39 smithi121 ceph-mon[34815]: pgmap v762: 97 pgs: 97 active+clean; 273 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 240 op/s 2024-09-15T22:59:40.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:39 smithi192 ceph-mon[35427]: pgmap v762: 97 pgs: 97 active+clean; 273 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 240 op/s 2024-09-15T22:59:40.581 INFO:tasks.workunit.client.0.smithi111.stdout: 10 3284 43.64 MB/sec warmup 8 sec latency 1197.283 ms 2024-09-15T22:59:41.581 INFO:tasks.workunit.client.0.smithi111.stdout: 10 3725 42.40 MB/sec warmup 9 sec latency 1684.930 ms 2024-09-15T22:59:42.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:41 smithi121 ceph-mon[34815]: pgmap v763: 97 pgs: 97 active+clean; 270 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 257 op/s 2024-09-15T22:59:42.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:41 smithi111 ceph-mon[30629]: pgmap v763: 97 pgs: 97 active+clean; 270 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 257 op/s 2024-09-15T22:59:42.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:41 smithi192 ceph-mon[35427]: pgmap v763: 97 pgs: 97 active+clean; 270 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 257 op/s 2024-09-15T22:59:42.582 INFO:tasks.workunit.client.0.smithi111.stdout: 10 4274 41.59 MB/sec warmup 10 sec latency 487.397 ms 2024-09-15T22:59:43.582 INFO:tasks.workunit.client.0.smithi111.stdout: 10 4782 39.31 MB/sec warmup 11 sec latency 75.227 ms 2024-09-15T22:59:44.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:43 smithi121 ceph-mon[34815]: pgmap v764: 97 pgs: 97 active+clean; 274 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 285 op/s 2024-09-15T22:59:44.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:43 smithi111 ceph-mon[30629]: pgmap v764: 97 pgs: 97 active+clean; 274 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 285 op/s 2024-09-15T22:59:44.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:43 smithi192 ceph-mon[35427]: pgmap v764: 97 pgs: 97 active+clean; 274 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 285 op/s 2024-09-15T22:59:44.582 INFO:tasks.workunit.client.0.smithi111.stdout: 10 5588 39.51 MB/sec warmup 12 sec latency 140.057 ms 2024-09-15T22:59:45.582 INFO:tasks.workunit.client.0.smithi111.stdout: 10 5834 37.66 MB/sec warmup 13 sec latency 1127.395 ms 2024-09-15T22:59:45.895 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:45 smithi111 ceph-mon[30629]: pgmap v765: 97 pgs: 97 active+clean; 278 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 329 op/s 2024-09-15T22:59:46.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:45 smithi121 ceph-mon[34815]: pgmap v765: 97 pgs: 97 active+clean; 278 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 329 op/s 2024-09-15T22:59:46.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:45 smithi192 ceph-mon[35427]: pgmap v765: 97 pgs: 97 active+clean; 278 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 329 op/s 2024-09-15T22:59:46.582 INFO:tasks.workunit.client.0.smithi111.stdout: 10 6321 36.49 MB/sec warmup 14 sec latency 1562.509 ms 2024-09-15T22:59:47.582 INFO:tasks.workunit.client.0.smithi111.stdout: 10 7127 36.47 MB/sec warmup 15 sec latency 118.907 ms 2024-09-15T22:59:48.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:47 smithi121 ceph-mon[34815]: pgmap v766: 97 pgs: 97 active+clean; 283 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 342 op/s 2024-09-15T22:59:48.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:47 smithi111 ceph-mon[30629]: pgmap v766: 97 pgs: 97 active+clean; 283 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 342 op/s 2024-09-15T22:59:48.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:47 smithi192 ceph-mon[35427]: pgmap v766: 97 pgs: 97 active+clean; 283 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 342 op/s 2024-09-15T22:59:48.582 INFO:tasks.workunit.client.0.smithi111.stdout: 10 8173 38.13 MB/sec warmup 16 sec latency 222.710 ms 2024-09-15T22:59:49.582 INFO:tasks.workunit.client.0.smithi111.stdout: 10 9021 39.25 MB/sec warmup 17 sec latency 47.913 ms 2024-09-15T22:59:49.888 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:49 smithi111 ceph-mon[30629]: pgmap v767: 97 pgs: 97 active+clean; 290 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 416 op/s 2024-09-15T22:59:50.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:49 smithi121 ceph-mon[34815]: pgmap v767: 97 pgs: 97 active+clean; 290 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 416 op/s 2024-09-15T22:59:50.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:49 smithi192 ceph-mon[35427]: pgmap v767: 97 pgs: 97 active+clean; 290 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 416 op/s 2024-09-15T22:59:50.582 INFO:tasks.workunit.client.0.smithi111.stdout: 10 9144 37.68 MB/sec warmup 18 sec latency 949.140 ms 2024-09-15T22:59:51.582 INFO:tasks.workunit.client.0.smithi111.stdout: 10 9375 36.18 MB/sec warmup 19 sec latency 1613.631 ms 2024-09-15T22:59:52.001 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:51 smithi111 ceph-mon[30629]: pgmap v768: 97 pgs: 97 active+clean; 286 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 402 op/s 2024-09-15T22:59:52.001 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:51 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:59:52.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:51 smithi121 ceph-mon[34815]: pgmap v768: 97 pgs: 97 active+clean; 286 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 402 op/s 2024-09-15T22:59:52.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:51 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:59:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.214+0000 7f2727200640 1 -- 172.21.15.111:0/2579173356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27281023c0 msgr2=0x7f27281047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.214+0000 7f2727200640 1 --2- 172.21.15.111:0/2579173356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27281023c0 0x7f27281047b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2718009900 tx=0x7f271802a410 comp rx=0 tx=0).stop 2024-09-15T22:59:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.214+0000 7f2727200640 1 -- 172.21.15.111:0/2579173356 shutdown_connections 2024-09-15T22:59:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.214+0000 7f2727200640 1 --2- 172.21.15.111:0/2579173356 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2728107620 0x7f2728109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.214+0000 7f2727200640 1 --2- 172.21.15.111:0/2579173356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2728104cf0 0x7f27281070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.214+0000 7f2727200640 1 --2- 172.21.15.111:0/2579173356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27281023c0 0x7f27281047b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.214+0000 7f2727200640 1 -- 172.21.15.111:0/2579173356 >> 172.21.15.111:0/2579173356 conn(0x7f27280fbfa0 msgr2=0x7f27280fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:59:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.214+0000 7f2727200640 1 -- 172.21.15.111:0/2579173356 shutdown_connections 2024-09-15T22:59:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.214+0000 7f2727200640 1 -- 172.21.15.111:0/2579173356 wait complete. 2024-09-15T22:59:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.215+0000 7f2727200640 1 Processor -- start 2024-09-15T22:59:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.216+0000 7f2727200640 1 -- start start 2024-09-15T22:59:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.216+0000 7f2727200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27281023c0 0x7f2728199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.216+0000 7f2727200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2728104cf0 0x7f272819a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.216+0000 7f2727200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2728107620 0x7f27281a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.216+0000 7f2727200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f272810d650 con 0x7f2728107620 2024-09-15T22:59:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.216+0000 7f2727200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f272810d4d0 con 0x7f2728104cf0 2024-09-15T22:59:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.216+0000 7f2727200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f272810d7d0 con 0x7f27281023c0 2024-09-15T22:59:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.216+0000 7f2725400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2728104cf0 0x7f272819a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.216+0000 7f2725e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27281023c0 0x7f2728199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.216+0000 7f2725400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2728104cf0 0x7f272819a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:47046/0 (socket says 172.21.15.111:47046) 2024-09-15T22:59:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.216+0000 7f2725400640 1 -- 172.21.15.111:0/4195802048 learned_addr learned my addr 172.21.15.111:0/4195802048 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:59:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.217+0000 7f2726800640 1 --2- 172.21.15.111:0/4195802048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2728107620 0x7f27281a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.217+0000 7f2725400640 1 -- 172.21.15.111:0/4195802048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27281023c0 msgr2=0x7f2728199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.217+0000 7f2725400640 1 --2- 172.21.15.111:0/4195802048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27281023c0 0x7f2728199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.217+0000 7f2725400640 1 -- 172.21.15.111:0/4195802048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2728107620 msgr2=0x7f27281a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.217+0000 7f2725400640 1 --2- 172.21.15.111:0/4195802048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2728107620 0x7f27281a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.217+0000 7f2725400640 1 -- 172.21.15.111:0/4195802048 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27281a1a90 con 0x7f2728104cf0 2024-09-15T22:59:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.217+0000 7f2726800640 1 --2- 172.21.15.111:0/4195802048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2728107620 0x7f27281a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:59:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.217+0000 7f2725e00640 1 --2- 172.21.15.111:0/4195802048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27281023c0 0x7f2728199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:59:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.217+0000 7f2725400640 1 --2- 172.21.15.111:0/4195802048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2728104cf0 0x7f272819a3b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f271400e9c0 tx=0x7f271400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:59:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.217+0000 7f271ea00640 1 -- 172.21.15.111:0/4195802048 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271400cd80 con 0x7f2728104cf0 2024-09-15T22:59:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.217+0000 7f2727200640 1 -- 172.21.15.111:0/4195802048 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27281a1d80 con 0x7f2728104cf0 2024-09-15T22:59:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.217+0000 7f271ea00640 1 -- 172.21.15.111:0/4195802048 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2714005500 con 0x7f2728104cf0 2024-09-15T22:59:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.218+0000 7f271ea00640 1 -- 172.21.15.111:0/4195802048 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2714010690 con 0x7f2728104cf0 2024-09-15T22:59:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.218+0000 7f2727200640 1 -- 172.21.15.111:0/4195802048 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27281a2250 con 0x7f2728104cf0 2024-09-15T22:59:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.221+0000 7f272fdbc740 1 Processor -- start 2024-09-15T22:59:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.221+0000 7f272fdbc740 1 -- start start 2024-09-15T22:59:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.221+0000 7f271ea00640 1 -- 172.21.15.111:0/4195802048 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f2714010830 con 0x7f2728104cf0 2024-09-15T22:59:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.221+0000 7f272fdbc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d735260 0x55833d735660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.221+0000 7f272fdbc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55833d735ba0 0x55833d73a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.221+0000 7f272fdbc740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833d73ac30 0x55833d73d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.221+0000 7f272fdbc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55833d69a100 con 0x55833d735ba0 2024-09-15T22:59:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.221+0000 7f272fdbc740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55833d6421b0 con 0x55833d73ac30 2024-09-15T22:59:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.221+0000 7f272fdbc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55833d6960c0 con 0x55833d735260 2024-09-15T22:59:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.221+0000 7f2726800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d735260 0x55833d735660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.221+0000 7f2726800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d735260 0x55833d735660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:59538/0 (socket says 172.21.15.111:59538) 2024-09-15T22:59:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.221+0000 7f2726800640 1 -- 172.21.15.111:0/495056533 learned_addr learned my addr 172.21.15.111:0/495056533 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:59:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.221+0000 7f2725400640 1 --2- 172.21.15.111:0/495056533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833d73ac30 0x55833d73d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.222+0000 7f2725e00640 1 --2- 172.21.15.111:0/495056533 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55833d735ba0 0x55833d73a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.222+0000 7f2726800640 1 -- 172.21.15.111:0/495056533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833d73ac30 msgr2=0x55833d73d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.222+0000 7f2726800640 1 --2- 172.21.15.111:0/495056533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833d73ac30 0x55833d73d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.222+0000 7f2726800640 1 -- 172.21.15.111:0/495056533 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55833d735ba0 msgr2=0x55833d73a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.222+0000 7f2726800640 1 --2- 172.21.15.111:0/495056533 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55833d735ba0 0x55833d73a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.222+0000 7f2726800640 1 -- 172.21.15.111:0/495056533 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55833d3ad210 con 0x55833d735260 2024-09-15T22:59:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.222+0000 7f2726800640 1 --2- 172.21.15.111:0/495056533 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d735260 0x55833d735660 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2720007ed0 tx=0x7f272000e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b270721d1ee8979b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:59:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.222+0000 7f2713400640 1 -- 172.21.15.111:0/495056533 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2720014070 con 0x55833d735260 2024-09-15T22:59:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.223+0000 7f2713400640 1 -- 172.21.15.111:0/495056533 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2720004d20 con 0x55833d735260 2024-09-15T22:59:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.223+0000 7f2713400640 1 -- 172.21.15.111:0/495056533 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2720005560 con 0x55833d735260 2024-09-15T22:59:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.223+0000 7f271ea00640 1 --2- 172.21.15.111:0/4195802048 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f270c0a3240 0x7f270c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.223+0000 7f271ea00640 1 -- 172.21.15.111:0/4195802048 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f27140cfcc0 con 0x7f2728104cf0 2024-09-15T22:59:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.223+0000 7f2725e00640 1 --2- 172.21.15.111:0/4195802048 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f270c0a3240 0x7f270c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.223+0000 7f272fdbc740 1 -- 172.21.15.111:0/495056533 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d735260 msgr2=0x55833d735660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.223+0000 7f272fdbc740 1 --2- 172.21.15.111:0/495056533 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d735260 0x55833d735660 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2720007ed0 tx=0x7f272000e720 comp rx=0 tx=0).stop 2024-09-15T22:59:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.223+0000 7f272fdbc740 1 -- 172.21.15.111:0/495056533 shutdown_connections 2024-09-15T22:59:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.223+0000 7f272fdbc740 1 --2- 172.21.15.111:0/495056533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833d73ac30 0x55833d73d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.223+0000 7f272fdbc740 1 --2- 172.21.15.111:0/495056533 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55833d735ba0 0x55833d73a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.223+0000 7f272fdbc740 1 --2- 172.21.15.111:0/495056533 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d735260 0x55833d735660 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.224+0000 7f272fdbc740 1 -- 172.21.15.111:0/495056533 >> 172.21.15.111:0/495056533 conn(0x55833d7308d0 msgr2=0x55833d730cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:59:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.224+0000 7f272fdbc740 1 -- 172.21.15.111:0/495056533 shutdown_connections 2024-09-15T22:59:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.224+0000 7f272fdbc740 1 -- 172.21.15.111:0/495056533 wait complete. 2024-09-15T22:59:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.224+0000 7f2725e00640 1 --2- 172.21.15.111:0/4195802048 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f270c0a3240 0x7f270c0a5700 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2718031440 tx=0x7f27180315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:59:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.224+0000 7f272fdbc740 1 -- start start 2024-09-15T22:59:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.224+0000 7f272fdbc740 1 Processor -- start 2024-09-15T22:59:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.225+0000 7f272fdbc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d735260 0x55833d73f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.225+0000 7f272fdbc740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833d735ba0 0x55833d73f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.225+0000 7f272fdbc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55833d73ac30 0x55833d746960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.225+0000 7f272fdbc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55833d73dc00 con 0x55833d73ac30 2024-09-15T22:59:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.225+0000 7f272fdbc740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55833d623090 con 0x55833d735ba0 2024-09-15T22:59:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.225+0000 7f272fdbc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55833d73dd80 con 0x55833d735260 2024-09-15T22:59:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.225+0000 7f272fdbc740 10 client.0 _ll_register_callbacks cb 0x55833d677f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:59:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.225+0000 7f2725e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d735260 0x55833d73f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.225+0000 7f2725e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d735260 0x55833d73f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:59548/0 (socket says 172.21.15.111:59548) 2024-09-15T22:59:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.225+0000 7f2725e00640 1 -- 172.21.15.111:0/691002690 learned_addr learned my addr 172.21.15.111:0/691002690 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:59:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.225+0000 7f2725400640 1 --2- 172.21.15.111:0/691002690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833d735ba0 0x55833d73f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.225+0000 7f2726800640 1 --2- 172.21.15.111:0/691002690 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55833d73ac30 0x55833d746960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.226+0000 7f2725400640 1 -- 172.21.15.111:0/691002690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d735260 msgr2=0x55833d73f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.226+0000 7f2725400640 1 --2- 172.21.15.111:0/691002690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d735260 0x55833d73f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.226+0000 7f2725400640 1 -- 172.21.15.111:0/691002690 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55833d73ac30 msgr2=0x55833d746960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.226+0000 7f2725400640 1 --2- 172.21.15.111:0/691002690 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55833d73ac30 0x55833d746960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.226+0000 7f2725400640 1 -- 172.21.15.111:0/691002690 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55833d479620 con 0x55833d735ba0 2024-09-15T22:59:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.226+0000 7f2725e00640 1 --2- 172.21.15.111:0/691002690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d735260 0x55833d73f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:59:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.226+0000 7f2725400640 1 --2- 172.21.15.111:0/691002690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833d735ba0 0x55833d73f990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f27140cd8b0 tx=0x7f27140d3310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=614ec0c6ed6af8ef server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:59:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.226+0000 7f2707e00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T22:59:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.226+0000 7f2707e00640 1 -- 172.21.15.111:0/691002690 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27140d1030 con 0x55833d735ba0 2024-09-15T22:59:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.226+0000 7f2707e00640 1 -- 172.21.15.111:0/691002690 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f271409c5e0 con 0x55833d735ba0 2024-09-15T22:59:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.226+0000 7f272fdbc740 1 -- 172.21.15.111:0/691002690 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2720004d20 con 0x55833d735ba0 2024-09-15T22:59:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.226+0000 7f2707e00640 1 -- 172.21.15.111:0/691002690 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f271409c150 con 0x55833d735ba0 2024-09-15T22:59:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.226+0000 7f272fdbc740 10 client.26081 fetch_fsmap learned FSMap version 31 2024-09-15T22:59:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.226+0000 7f272fdbc740 1 -- 172.21.15.111:0/691002690 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55833d749870 con 0x55833d735ba0 2024-09-15T22:59:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.227+0000 7f2707e00640 1 -- 172.21.15.111:0/691002690 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2720004d20 con 0x55833d735ba0 2024-09-15T22:59:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.227+0000 7f2707e00640 1 -- 172.21.15.111:0/691002690 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f27140b50f0 con 0x55833d735ba0 2024-09-15T22:59:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.227+0000 7f272fdbc740 10 client.26081 fetch_fsmap finished waiting for FSMap version 31 2024-09-15T22:59:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.227+0000 7f272fdbc740 10 client.26081 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:59:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.227+0000 7f272fdbc740 20 client.26081 populate_metadata read hostname 'smithi111' 2024-09-15T22:59:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.227+0000 7f272fdbc740 1 --2- 172.21.15.111:0/691002690 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55833d74aef0 0x55833d74d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.227+0000 7f272fdbc740 4 client.26081 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:59:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.227+0000 7f272fdbc740 1 -- 172.21.15.111:0/691002690 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55833d6a7830 con 0x55833d74aef0 2024-09-15T22:59:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.227+0000 7f2725e00640 1 --2- 172.21.15.111:0/691002690 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55833d74aef0 0x55833d74d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.228+0000 7f2725e00640 1 --2- 172.21.15.111:0/691002690 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55833d74aef0 0x55833d74d300 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=302c55921c9b26ea server_cookie=e0ac774c32ea1890 in_seq=0 out_seq=0 2024-09-15T22:59:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.228+0000 7f2707e00640 10 client.26081 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:59:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.229+0000 7f2707e00640 1 -- 172.21.15.111:0/691002690 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55833d6a7830 con 0x55833d74aef0 2024-09-15T22:59:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.229+0000 7f2707e00640 10 client.26081 handle_command_reply: tid=0 2024-09-15T22:59:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.229+0000 7f272fdbc740 1 client.26081 shutdown 2024-09-15T22:59:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.229+0000 7f272fdbc740 1 -- 172.21.15.111:0/691002690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833d735ba0 msgr2=0x55833d73f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.229+0000 7f272fdbc740 1 --2- 172.21.15.111:0/691002690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833d735ba0 0x55833d73f990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f27140cd8b0 tx=0x7f27140d3310 comp rx=0 tx=0).stop 2024-09-15T22:59:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.229+0000 7f272fdbc740 1 -- 172.21.15.111:0/691002690 shutdown_connections 2024-09-15T22:59:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.229+0000 7f272fdbc740 1 --2- 172.21.15.111:0/691002690 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55833d74aef0 0x55833d74d300 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.229+0000 7f272fdbc740 1 --2- 172.21.15.111:0/691002690 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55833d73ac30 0x55833d746960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.229+0000 7f272fdbc740 1 --2- 172.21.15.111:0/691002690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833d735ba0 0x55833d73f990 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.229+0000 7f272fdbc740 1 --2- 172.21.15.111:0/691002690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d735260 0x55833d73f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.229+0000 7f272fdbc740 1 -- 172.21.15.111:0/691002690 >> 172.21.15.111:0/691002690 conn(0x55833d7308d0 msgr2=0x55833d7320e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:59:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.229+0000 7f2707e00640 0 client.26081 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:59:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.229+0000 7f272fdbc740 1 -- 172.21.15.111:0/691002690 shutdown_connections 2024-09-15T22:59:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.229+0000 7f272fdbc740 1 -- 172.21.15.111:0/691002690 wait complete. 2024-09-15T22:59:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.230+0000 7f272fdbc740 20 client.26081 trim_cache size 0 max 16384 2024-09-15T22:59:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.248+0000 7f272fdbc740 1 Processor -- start 2024-09-15T22:59:52.249 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.248+0000 7f272fdbc740 1 -- start start 2024-09-15T22:59:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.248+0000 7f272fdbc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55833d735260 0x55833d72f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.248+0000 7f272fdbc740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833d735ba0 0x55833d746920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.248+0000 7f272fdbc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d73ac30 0x55833d72f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.248+0000 7f272fdbc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f27281ad780 con 0x55833d735260 2024-09-15T22:59:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.248+0000 7f272fdbc740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f27281aaff0 con 0x55833d735ba0 2024-09-15T22:59:52.250 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.248+0000 7f272fdbc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f27281a8e60 con 0x55833d73ac30 2024-09-15T22:59:52.251 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.248+0000 7f2725400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833d735ba0 0x55833d746920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.248+0000 7f2726800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d73ac30 0x55833d72f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.248+0000 7f2725400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833d735ba0 0x55833d746920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:47070/0 (socket says 172.21.15.111:47070) 2024-09-15T22:59:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.248+0000 7f2725e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55833d735260 0x55833d72f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.248+0000 7f2726800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d73ac30 0x55833d72f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:59558/0 (socket says 172.21.15.111:59558) 2024-09-15T22:59:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.249+0000 7f2725400640 1 -- 172.21.15.111:0/4292979052 learned_addr learned my addr 172.21.15.111:0/4292979052 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:59:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.249+0000 7f2725400640 1 -- 172.21.15.111:0/4292979052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d73ac30 msgr2=0x55833d72f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.249+0000 7f2725400640 1 --2- 172.21.15.111:0/4292979052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d73ac30 0x55833d72f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.249+0000 7f2725400640 1 -- 172.21.15.111:0/4292979052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55833d735260 msgr2=0x55833d72f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.249+0000 7f2725400640 1 --2- 172.21.15.111:0/4292979052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55833d735260 0x55833d72f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.249+0000 7f2725400640 1 -- 172.21.15.111:0/4292979052 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55833d736080 con 0x55833d735ba0 2024-09-15T22:59:52.252 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.249+0000 7f2725e00640 1 --2- 172.21.15.111:0/4292979052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55833d735260 0x55833d72f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:59:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.249+0000 7f2726800640 1 --2- 172.21.15.111:0/4292979052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d73ac30 0x55833d72f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:59:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.249+0000 7f2725400640 1 --2- 172.21.15.111:0/4292979052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833d735ba0 0x55833d746920 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2714098d50 tx=0x7f271409a890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc66072b4b2c75a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:59:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.249+0000 7f2712a00640 1 -- 172.21.15.111:0/4292979052 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271409bce0 con 0x55833d735ba0 2024-09-15T22:59:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.249+0000 7f2712a00640 1 -- 172.21.15.111:0/4292979052 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27140d2070 con 0x55833d735ba0 2024-09-15T22:59:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.249+0000 7f2712a00640 1 -- 172.21.15.111:0/4292979052 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271409bec0 con 0x55833d735ba0 2024-09-15T22:59:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.250+0000 7f272fdbc740 1 -- 172.21.15.111:0/4292979052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833d735ba0 msgr2=0x55833d746920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.250+0000 7f272fdbc740 1 --2- 172.21.15.111:0/4292979052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833d735ba0 0x55833d746920 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2714098d50 tx=0x7f271409a890 comp rx=0 tx=0).stop 2024-09-15T22:59:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.250+0000 7f272fdbc740 1 -- 172.21.15.111:0/4292979052 shutdown_connections 2024-09-15T22:59:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.250+0000 7f272fdbc740 1 --2- 172.21.15.111:0/4292979052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d73ac30 0x55833d72f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.250+0000 7f272fdbc740 1 --2- 172.21.15.111:0/4292979052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833d735ba0 0x55833d746920 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.250+0000 7f272fdbc740 1 --2- 172.21.15.111:0/4292979052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55833d735260 0x55833d72f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.250+0000 7f272fdbc740 1 -- 172.21.15.111:0/4292979052 >> 172.21.15.111:0/4292979052 conn(0x55833d7308d0 msgr2=0x55833d73f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:59:52.253 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.250+0000 7f272fdbc740 1 -- 172.21.15.111:0/4292979052 shutdown_connections 2024-09-15T22:59:52.254 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.250+0000 7f272fdbc740 1 -- 172.21.15.111:0/4292979052 wait complete. 2024-09-15T22:59:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.251+0000 7f272fdbc740 1 -- start start 2024-09-15T22:59:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.251+0000 7f272fdbc740 1 Processor -- start 2024-09-15T22:59:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.251+0000 7f272fdbc740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833d735260 0x55833d743ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.251+0000 7f272fdbc740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d735ba0 0x55833d743ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.251+0000 7f272fdbc740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55833d73ac30 0x55833d744fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:52.256 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.251+0000 7f272fdbc740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f27281a94b0 con 0x55833d73ac30 2024-09-15T22:59:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.251+0000 7f272fdbc740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f27281aa920 con 0x55833d735260 2024-09-15T22:59:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.251+0000 7f272fdbc740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f27281a9b80 con 0x55833d735ba0 2024-09-15T22:59:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.252+0000 7f272fdbc740 10 client.0 _ll_register_callbacks cb 0x7f2714009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:59:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.252+0000 7f2725400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d735ba0 0x55833d743ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.252+0000 7f2725e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833d735260 0x55833d743ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.252+0000 7f2725400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d735ba0 0x55833d743ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:59564/0 (socket says 172.21.15.111:59564) 2024-09-15T22:59:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.252+0000 7f2725400640 1 -- 172.21.15.111:0/3068557574 learned_addr learned my addr 172.21.15.111:0/3068557574 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T22:59:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.252+0000 7f2726800640 1 --2- 172.21.15.111:0/3068557574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55833d73ac30 0x55833d744fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.252+0000 7f2725400640 1 -- 172.21.15.111:0/3068557574 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833d735260 msgr2=0x55833d743ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.252+0000 7f2725400640 1 --2- 172.21.15.111:0/3068557574 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833d735260 0x55833d743ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.252+0000 7f2725400640 1 -- 172.21.15.111:0/3068557574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55833d73ac30 msgr2=0x55833d744fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:59:52.257 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.252+0000 7f2725400640 1 --2- 172.21.15.111:0/3068557574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55833d73ac30 0x55833d744fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.252+0000 7f2725400640 1 -- 172.21.15.111:0/3068557574 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55833d7455c0 con 0x55833d735ba0 2024-09-15T22:59:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.252+0000 7f2726800640 1 --2- 172.21.15.111:0/3068557574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55833d73ac30 0x55833d744fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:59:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.252+0000 7f2725400640 1 --2- 172.21.15.111:0/3068557574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d735ba0 0x55833d743ff0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2714005820 tx=0x7f2714098020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=429b5c1141069ad5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:59:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.252+0000 7f2707e00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T22:59:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.252+0000 7f2707e00640 1 -- 172.21.15.111:0/3068557574 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27140b9020 con 0x55833d735ba0 2024-09-15T22:59:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.253+0000 7f2707e00640 1 -- 172.21.15.111:0/3068557574 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f271409b7d0 con 0x55833d735ba0 2024-09-15T22:59:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.253+0000 7f272fdbc740 1 -- 172.21.15.111:0/3068557574 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27140d2070 con 0x55833d735ba0 2024-09-15T22:59:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.253+0000 7f2707e00640 1 -- 172.21.15.111:0/3068557574 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f271409bec0 con 0x55833d735ba0 2024-09-15T22:59:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.253+0000 7f272fdbc740 10 client.26110 fetch_fsmap learned FSMap version 31 2024-09-15T22:59:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.253+0000 7f272fdbc740 1 -- 172.21.15.111:0/3068557574 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55833d7468d0 con 0x55833d735ba0 2024-09-15T22:59:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.253+0000 7f2707e00640 1 -- 172.21.15.111:0/3068557574 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27140d2070 con 0x55833d735ba0 2024-09-15T22:59:52.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.253+0000 7f2707e00640 1 -- 172.21.15.111:0/3068557574 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f27140671f0 con 0x55833d735ba0 2024-09-15T22:59:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.253+0000 7f272fdbc740 10 client.26110 fetch_fsmap finished waiting for FSMap version 31 2024-09-15T22:59:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.253+0000 7f272fdbc740 10 client.26110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T22:59:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.253+0000 7f272fdbc740 20 client.26110 populate_metadata read hostname 'smithi111' 2024-09-15T22:59:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.253+0000 7f272fdbc740 1 --2- 172.21.15.111:0/3068557574 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55833d7b5de0 0x55833d7b8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:59:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.253+0000 7f272fdbc740 4 client.26110 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T22:59:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.253+0000 7f272fdbc740 1 -- 172.21.15.111:0/3068557574 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55833d6a1b80 con 0x55833d7b5de0 2024-09-15T22:59:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.254+0000 7f2725e00640 1 --2- 172.21.15.111:0/3068557574 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55833d7b5de0 0x55833d7b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:59:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.254+0000 7f2725e00640 1 --2- 172.21.15.111:0/3068557574 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55833d7b5de0 0x55833d7b8200 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc92b1e413732ec4 server_cookie=1762258a7b95132c in_seq=0 out_seq=0 2024-09-15T22:59:52.259 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.254+0000 7f2707e00640 10 client.26110 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T22:59:52.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:51 smithi192 ceph-mon[35427]: pgmap v768: 97 pgs: 97 active+clean; 286 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 402 op/s 2024-09-15T22:59:52.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:51 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:59:52.453 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.451+0000 7f2707e00640 1 -- 172.21.15.111:0/3068557574 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55833d6a1b80 con 0x55833d7b5de0 2024-09-15T22:59:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.451+0000 7f2707e00640 10 client.26110 handle_command_reply: tid=0 2024-09-15T22:59:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.451+0000 7f272fdbc740 1 client.26110 shutdown 2024-09-15T22:59:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.452+0000 7f272fdbc740 1 -- 172.21.15.111:0/3068557574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d735ba0 msgr2=0x55833d743ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.452+0000 7f272fdbc740 1 --2- 172.21.15.111:0/3068557574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d735ba0 0x55833d743ff0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2714005820 tx=0x7f2714098020 comp rx=0 tx=0).stop 2024-09-15T22:59:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.452+0000 7f272fdbc740 1 -- 172.21.15.111:0/3068557574 shutdown_connections 2024-09-15T22:59:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.452+0000 7f272fdbc740 1 --2- 172.21.15.111:0/3068557574 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55833d7b5de0 0x55833d7b8200 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.452+0000 7f272fdbc740 1 --2- 172.21.15.111:0/3068557574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55833d73ac30 0x55833d744fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.452+0000 7f272fdbc740 1 --2- 172.21.15.111:0/3068557574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55833d735ba0 0x55833d743ff0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.452+0000 7f272fdbc740 1 --2- 172.21.15.111:0/3068557574 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55833d735260 0x55833d743ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.452+0000 7f272fdbc740 1 -- 172.21.15.111:0/3068557574 >> 172.21.15.111:0/3068557574 conn(0x55833d7308d0 msgr2=0x55833d73f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:59:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.452+0000 7f2707e00640 0 client.26110 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T22:59:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.452+0000 7f272fdbc740 1 -- 172.21.15.111:0/3068557574 shutdown_connections 2024-09-15T22:59:52.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.452+0000 7f272fdbc740 1 -- 172.21.15.111:0/3068557574 wait complete. 2024-09-15T22:59:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.452+0000 7f272fdbc740 20 client.26110 trim_cache size 0 max 16384 2024-09-15T22:59:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T22:59:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T22:59:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T22:59:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T22:59:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.453+0000 7f2727200640 1 -- 172.21.15.111:0/4195802048 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f270c0a3240 msgr2=0x7f270c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.453+0000 7f2727200640 1 --2- 172.21.15.111:0/4195802048 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f270c0a3240 0x7f270c0a5700 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2718031440 tx=0x7f27180315b0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.453+0000 7f2727200640 1 -- 172.21.15.111:0/4195802048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2728104cf0 msgr2=0x7f272819a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:59:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.453+0000 7f2727200640 1 --2- 172.21.15.111:0/4195802048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2728104cf0 0x7f272819a3b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f271400e9c0 tx=0x7f271400ee80 comp rx=0 tx=0).stop 2024-09-15T22:59:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.453+0000 7f2727200640 1 -- 172.21.15.111:0/4195802048 shutdown_connections 2024-09-15T22:59:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.453+0000 7f2727200640 1 --2- 172.21.15.111:0/4195802048 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2728107620 0x7f27281a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.453+0000 7f2727200640 1 --2- 172.21.15.111:0/4195802048 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f270c0a3240 0x7f270c0a5700 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.453+0000 7f2727200640 1 --2- 172.21.15.111:0/4195802048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2728104cf0 0x7f272819a3b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.453+0000 7f2727200640 1 --2- 172.21.15.111:0/4195802048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27281023c0 0x7f2728199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:59:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.453+0000 7f2727200640 1 -- 172.21.15.111:0/4195802048 >> 172.21.15.111:0/4195802048 conn(0x7f27280fbfa0 msgr2=0x7f27281019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:59:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.453+0000 7f2727200640 1 -- 172.21.15.111:0/4195802048 shutdown_connections 2024-09-15T22:59:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T22:59:52.453+0000 7f2727200640 1 -- 172.21.15.111:0/4195802048 wait complete. 2024-09-15T22:59:52.582 INFO:tasks.workunit.client.0.smithi111.stdout: 10 9839 35.60 MB/sec warmup 20 sec latency 168.103 ms 2024-09-15T22:59:53.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:52 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:59:53.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:52 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:59:53.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:52 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:59:53.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:52 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:59:53.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:52 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:59:53.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:52 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:59:53.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:52 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:59:53.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:52 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:59:53.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:52 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T22:59:53.582 INFO:tasks.workunit.client.0.smithi111.stdout: 10 10520 35.17 MB/sec warmup 21 sec latency 64.018 ms 2024-09-15T22:59:54.128 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:53 smithi111 ceph-mon[30629]: pgmap v769: 97 pgs: 97 active+clean; 277 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.5 MiB/s wr, 434 op/s 2024-09-15T22:59:54.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:53 smithi121 ceph-mon[34815]: pgmap v769: 97 pgs: 97 active+clean; 277 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.5 MiB/s wr, 434 op/s 2024-09-15T22:59:54.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:53 smithi192 ceph-mon[35427]: pgmap v769: 97 pgs: 97 active+clean; 277 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.5 MiB/s wr, 434 op/s 2024-09-15T22:59:54.582 INFO:tasks.workunit.client.0.smithi111.stdout: 10 11434 35.46 MB/sec warmup 22 sec latency 52.429 ms 2024-09-15T22:59:55.280 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:55 smithi111 ceph-mon[30629]: pgmap v770: 97 pgs: 97 active+clean; 275 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.7 MiB/s wr, 491 op/s 2024-09-15T22:59:55.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:55 smithi192 ceph-mon[35427]: pgmap v770: 97 pgs: 97 active+clean; 275 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.7 MiB/s wr, 491 op/s 2024-09-15T22:59:55.582 INFO:tasks.workunit.client.0.smithi111.stdout: 10 11621 34.39 MB/sec warmup 23 sec latency 835.202 ms 2024-09-15T22:59:55.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:55 smithi121 ceph-mon[34815]: pgmap v770: 97 pgs: 97 active+clean; 275 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.7 MiB/s wr, 491 op/s 2024-09-15T22:59:56.583 INFO:tasks.workunit.client.0.smithi111.stdout: 10 11622 32.96 MB/sec warmup 24 sec latency 1835.268 ms 2024-09-15T22:59:57.583 INFO:tasks.workunit.client.0.smithi111.stdout: 10 11968 32.17 MB/sec warmup 25 sec latency 2468.977 ms 2024-09-15T22:59:58.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:57 smithi121 ceph-mon[34815]: pgmap v771: 97 pgs: 97 active+clean; 283 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.1 MiB/s wr, 513 op/s 2024-09-15T22:59:58.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:57 smithi111 ceph-mon[30629]: pgmap v771: 97 pgs: 97 active+clean; 283 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.1 MiB/s wr, 513 op/s 2024-09-15T22:59:58.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:57 smithi192 ceph-mon[35427]: pgmap v771: 97 pgs: 97 active+clean; 283 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.1 MiB/s wr, 513 op/s 2024-09-15T22:59:58.583 INFO:tasks.workunit.client.0.smithi111.stdout: 10 13170 34.25 MB/sec warmup 26 sec latency 42.246 ms 2024-09-15T22:59:59.583 INFO:tasks.workunit.client.0.smithi111.stdout: 10 13795 34.16 MB/sec warmup 27 sec latency 163.780 ms 2024-09-15T23:00:00.079 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 22:59:59 smithi111 ceph-mon[30629]: pgmap v772: 97 pgs: 97 active+clean; 286 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.1 MiB/s wr, 534 op/s 2024-09-15T23:00:00.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 22:59:59 smithi121 ceph-mon[34815]: pgmap v772: 97 pgs: 97 active+clean; 286 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.1 MiB/s wr, 534 op/s 2024-09-15T23:00:00.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 22:59:59 smithi192 ceph-mon[35427]: pgmap v772: 97 pgs: 97 active+clean; 286 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.1 MiB/s wr, 534 op/s 2024-09-15T23:00:00.588 INFO:tasks.workunit.client.0.smithi111.stdout: 10 14085 33.58 MB/sec warmup 28 sec latency 762.242 ms 2024-09-15T23:00:00.931 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:00 smithi111 ceph-mon[30629]: overall HEALTH_OK 2024-09-15T23:00:01.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:00 smithi121 ceph-mon[34815]: overall HEALTH_OK 2024-09-15T23:00:01.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:00 smithi192 ceph-mon[35427]: overall HEALTH_OK 2024-09-15T23:00:01.588 INFO:tasks.workunit.client.0.smithi111.stdout: 10 14085 32.43 MB/sec warmup 29 sec latency 1752.116 ms 2024-09-15T23:00:02.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:01 smithi121 ceph-mon[34815]: pgmap v773: 97 pgs: 97 active+clean; 281 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 461 op/s 2024-09-15T23:00:02.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:01 smithi111 ceph-mon[30629]: pgmap v773: 97 pgs: 97 active+clean; 281 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 461 op/s 2024-09-15T23:00:02.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:01 smithi192 ceph-mon[35427]: pgmap v773: 97 pgs: 97 active+clean; 281 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 461 op/s 2024-09-15T23:00:02.588 INFO:tasks.workunit.client.0.smithi111.stdout: 10 14695 32.16 MB/sec warmup 30 sec latency 1783.659 ms 2024-09-15T23:00:03.588 INFO:tasks.workunit.client.0.smithi111.stdout: 10 15503 32.58 MB/sec warmup 31 sec latency 81.167 ms 2024-09-15T23:00:04.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:03 smithi121 ceph-mon[34815]: pgmap v774: 97 pgs: 97 active+clean; 286 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 451 op/s 2024-09-15T23:00:04.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:03 smithi111 ceph-mon[30629]: pgmap v774: 97 pgs: 97 active+clean; 286 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 451 op/s 2024-09-15T23:00:04.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:03 smithi192 ceph-mon[35427]: pgmap v774: 97 pgs: 97 active+clean; 286 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 451 op/s 2024-09-15T23:00:04.588 INFO:tasks.workunit.client.0.smithi111.stdout: 10 16323 32.67 MB/sec warmup 32 sec latency 82.004 ms 2024-09-15T23:00:05.589 INFO:tasks.workunit.client.0.smithi111.stdout: 10 16677 32.30 MB/sec warmup 33 sec latency 621.742 ms 2024-09-15T23:00:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.671+0000 7fa611e00640 1 -- 172.21.15.111:0/3854836823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa60c074bf0 msgr2=0x7fa60c075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.671+0000 7fa611e00640 1 --2- 172.21.15.111:0/3854836823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa60c074bf0 0x7fa60c075050 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8009900 tx=0x7fa5f802a4d0 comp rx=0 tx=0).stop 2024-09-15T23:00:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.671+0000 7fa611e00640 1 -- 172.21.15.111:0/3854836823 shutdown_connections 2024-09-15T23:00:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.671+0000 7fa611e00640 1 --2- 172.21.15.111:0/3854836823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa60c1099f0 0x7fa60c10beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.671+0000 7fa611e00640 1 --2- 172.21.15.111:0/3854836823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa60c074bf0 0x7fa60c075050 secure :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8009900 tx=0x7fa5f802a4d0 comp rx=0 tx=0).stop 2024-09-15T23:00:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.671+0000 7fa611e00640 1 --2- 172.21.15.111:0/3854836823 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa60c0761d0 0x7fa60c074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.671+0000 7fa611e00640 1 -- 172.21.15.111:0/3854836823 >> 172.21.15.111:0/3854836823 conn(0x7fa60c0fd2c0 msgr2=0x7fa60c0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:05.678 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.676+0000 7fa611e00640 1 -- 172.21.15.111:0/3854836823 shutdown_connections 2024-09-15T23:00:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.676+0000 7fa611e00640 1 -- 172.21.15.111:0/3854836823 wait complete. 2024-09-15T23:00:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.678+0000 7fa611e00640 1 Processor -- start 2024-09-15T23:00:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.678+0000 7fa611e00640 1 -- start start 2024-09-15T23:00:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.678+0000 7fa611e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa60c0761d0 0x7fa60c195f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.678+0000 7fa611e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa60c1099f0 0x7fa60c1964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.678+0000 7fa611e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa60c19b500 0x7fa60c1aea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.678+0000 7fa611e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c10fb60 con 0x7fa60c1099f0 2024-09-15T23:00:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.678+0000 7fa611e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c10f9e0 con 0x7fa60c0761d0 2024-09-15T23:00:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.678+0000 7fa611e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c10fce0 con 0x7fa60c19b500 2024-09-15T23:00:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.678+0000 7fa60a000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa60c1099f0 0x7fa60c1964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.678+0000 7fa60a000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa60c1099f0 0x7fa60c1964a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55186/0 (socket says 172.21.15.111:55186) 2024-09-15T23:00:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.678+0000 7fa60a000640 1 -- 172.21.15.111:0/1574844659 learned_addr learned my addr 172.21.15.111:0/1574844659 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:00:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.678+0000 7fa60a000640 1 -- 172.21.15.111:0/1574844659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa60c19b500 msgr2=0x7fa60c1aea60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T23:00:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.678+0000 7fa60a000640 1 --2- 172.21.15.111:0/1574844659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa60c19b500 0x7fa60c1aea60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.678+0000 7fa60a000640 1 -- 172.21.15.111:0/1574844659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa60c0761d0 msgr2=0x7fa60c195f60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T23:00:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.678+0000 7fa60a000640 1 --2- 172.21.15.111:0/1574844659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa60c0761d0 0x7fa60c195f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.678+0000 7fa60a000640 1 -- 172.21.15.111:0/1574844659 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa60c1aefa0 con 0x7fa60c1099f0 2024-09-15T23:00:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.678+0000 7fa60a000640 1 --2- 172.21.15.111:0/1574844659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa60c1099f0 0x7fa60c1964a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8006010 tx=0x7fa5f8032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.684+0000 7fa610a00640 1 -- 172.21.15.111:0/1574844659 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f803f030 con 0x7fa60c1099f0 2024-09-15T23:00:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.684+0000 7fa610a00640 1 -- 172.21.15.111:0/1574844659 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f8003d60 con 0x7fa60c1099f0 2024-09-15T23:00:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.684+0000 7fa610a00640 1 -- 172.21.15.111:0/1574844659 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8035070 con 0x7fa60c1099f0 2024-09-15T23:00:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.684+0000 7fa611e00640 1 -- 172.21.15.111:0/1574844659 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa60c1af230 con 0x7fa60c1099f0 2024-09-15T23:00:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.684+0000 7fa611e00640 1 -- 172.21.15.111:0/1574844659 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa60c1af720 con 0x7fa60c1099f0 2024-09-15T23:00:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.688+0000 7fa614e4a740 1 Processor -- start 2024-09-15T23:00:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.688+0000 7fa614e4a740 1 -- start start 2024-09-15T23:00:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.688+0000 7fa614e4a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4a72090 0x55d4f4a72490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.688+0000 7fa614e4a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d4f4a729d0 0x55d4f4a77490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.691+0000 7fa610a00640 1 -- 172.21.15.111:0/1574844659 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa5f8042030 con 0x7fa60c1099f0 2024-09-15T23:00:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.691+0000 7fa614e4a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d4f4a77a60 0x55d4f4a79f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.691+0000 7fa614e4a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d4f49d7100 con 0x55d4f4a729d0 2024-09-15T23:00:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.691+0000 7fa614e4a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d4f497f1b0 con 0x55d4f4a72090 2024-09-15T23:00:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.691+0000 7fa614e4a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d4f49d30c0 con 0x55d4f4a77a60 2024-09-15T23:00:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.691+0000 7fa60aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d4f4a729d0 0x55d4f4a77490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.692+0000 7fa60b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4a72090 0x55d4f4a72490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.692+0000 7fa60a000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d4f4a77a60 0x55d4f4a79f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.692+0000 7fa60a000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d4f4a77a60 0x55d4f4a79f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:40842/0 (socket says 172.21.15.111:40842) 2024-09-15T23:00:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.692+0000 7fa60a000640 1 -- 172.21.15.111:0/2037230810 learned_addr learned my addr 172.21.15.111:0/2037230810 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:00:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.692+0000 7fa60a000640 1 -- 172.21.15.111:0/2037230810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4a72090 msgr2=0x55d4f4a72490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.692+0000 7fa60a000640 1 --2- 172.21.15.111:0/2037230810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4a72090 0x55d4f4a72490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.692+0000 7fa60a000640 1 -- 172.21.15.111:0/2037230810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d4f4a729d0 msgr2=0x55d4f4a77490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.692+0000 7fa60a000640 1 --2- 172.21.15.111:0/2037230810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d4f4a729d0 0x55d4f4a77490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.692+0000 7fa60a000640 1 -- 172.21.15.111:0/2037230810 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4f46ea210 con 0x55d4f4a77a60 2024-09-15T23:00:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.693+0000 7fa60b400640 1 --2- 172.21.15.111:0/2037230810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4a72090 0x55d4f4a72490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:00:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.693+0000 7fa60a000640 1 --2- 172.21.15.111:0/2037230810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d4f4a77a60 0x55d4f4a79f20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80f3fd0 tx=0x7fa5f80f8320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb1e04e273301e19 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.693+0000 7fa5f7e00640 1 -- 172.21.15.111:0/2037230810 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f80f8b50 con 0x55d4f4a77a60 2024-09-15T23:00:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.693+0000 7fa5f7e00640 1 -- 172.21.15.111:0/2037230810 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f80fd030 con 0x55d4f4a77a60 2024-09-15T23:00:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.693+0000 7fa5f7e00640 1 -- 172.21.15.111:0/2037230810 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8101630 con 0x55d4f4a77a60 2024-09-15T23:00:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.695+0000 7fa614e4a740 1 -- 172.21.15.111:0/2037230810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d4f4a77a60 msgr2=0x55d4f4a79f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.695+0000 7fa614e4a740 1 --2- 172.21.15.111:0/2037230810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d4f4a77a60 0x55d4f4a79f20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80f3fd0 tx=0x7fa5f80f8320 comp rx=0 tx=0).stop 2024-09-15T23:00:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.697+0000 7fa614e4a740 1 -- 172.21.15.111:0/2037230810 shutdown_connections 2024-09-15T23:00:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.697+0000 7fa614e4a740 1 --2- 172.21.15.111:0/2037230810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d4f4a77a60 0x55d4f4a79f20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.697+0000 7fa614e4a740 1 --2- 172.21.15.111:0/2037230810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d4f4a729d0 0x55d4f4a77490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.697+0000 7fa614e4a740 1 --2- 172.21.15.111:0/2037230810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4a72090 0x55d4f4a72490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.697+0000 7fa614e4a740 1 -- 172.21.15.111:0/2037230810 >> 172.21.15.111:0/2037230810 conn(0x55d4f4a6d700 msgr2=0x55d4f4a6db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.697+0000 7fa614e4a740 1 -- 172.21.15.111:0/2037230810 shutdown_connections 2024-09-15T23:00:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.697+0000 7fa614e4a740 1 -- 172.21.15.111:0/2037230810 wait complete. 2024-09-15T23:00:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.697+0000 7fa614e4a740 1 -- start start 2024-09-15T23:00:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.698+0000 7fa614e4a740 1 Processor -- start 2024-09-15T23:00:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.699+0000 7fa614e4a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d4f4a72090 0x55d4f4a7bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.699+0000 7fa614e4a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4a729d0 0x55d4f4a7c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.699+0000 7fa614e4a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d4f4a77a60 0x55d4f4a834e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.699+0000 7fa614e4a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d4f4a7a7f0 con 0x55d4f4a77a60 2024-09-15T23:00:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.699+0000 7fa614e4a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d4f4960090 con 0x55d4f4a729d0 2024-09-15T23:00:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.699+0000 7fa614e4a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d4f4a7a970 con 0x55d4f4a72090 2024-09-15T23:00:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.699+0000 7fa614e4a740 10 client.0 _ll_register_callbacks cb 0x55d4f49b4f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:00:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.704+0000 7fa60a000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d4f4a77a60 0x55d4f4a834e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.704+0000 7fa60a000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d4f4a77a60 0x55d4f4a834e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55200/0 (socket says 172.21.15.111:55200) 2024-09-15T23:00:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.704+0000 7fa60a000640 1 -- 172.21.15.111:0/615170882 learned_addr learned my addr 172.21.15.111:0/615170882 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:00:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.705+0000 7fa60aa00640 1 --2- 172.21.15.111:0/615170882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4a729d0 0x55d4f4a7c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.706+0000 7fa60b400640 1 --2- 172.21.15.111:0/615170882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d4f4a72090 0x55d4f4a7bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.706+0000 7fa60b400640 1 -- 172.21.15.111:0/615170882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4a729d0 msgr2=0x55d4f4a7c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.706+0000 7fa60b400640 1 --2- 172.21.15.111:0/615170882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4a729d0 0x55d4f4a7c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.706+0000 7fa60b400640 1 -- 172.21.15.111:0/615170882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d4f4a77a60 msgr2=0x55d4f4a834e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.706+0000 7fa60b400640 1 --2- 172.21.15.111:0/615170882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d4f4a77a60 0x55d4f4a834e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.706+0000 7fa60b400640 1 -- 172.21.15.111:0/615170882 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4f47b6620 con 0x55d4f4a72090 2024-09-15T23:00:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.707+0000 7fa60b400640 1 --2- 172.21.15.111:0/615170882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d4f4a72090 0x55d4f4a7bf60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa60400ca60 tx=0x7fa60400cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=883eb6777984cce server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.707+0000 7fa5f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:00:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.709+0000 7fa610a00640 1 --2- 172.21.15.111:0/1574844659 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa5f00a3280 0x7fa5f00a5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.709+0000 7fa610a00640 1 -- 172.21.15.111:0/1574844659 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fa5f80f0000 con 0x7fa60c1099f0 2024-09-15T23:00:05.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.710+0000 7fa60aa00640 1 --2- 172.21.15.111:0/1574844659 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa5f00a3280 0x7fa5f00a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:05.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.712+0000 7fa60aa00640 1 --2- 172.21.15.111:0/1574844659 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa5f00a3280 0x7fa5f00a5740 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55d4f4a7d440 tx=0x7fa5fc008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.707+0000 7fa5f4c00640 1 -- 172.21.15.111:0/615170882 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60401a060 con 0x55d4f4a72090 2024-09-15T23:00:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.736+0000 7fa5f4c00640 1 -- 172.21.15.111:0/615170882 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fa604015440 con 0x55d4f4a72090 2024-09-15T23:00:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.736+0000 7fa614e4a740 1 -- 172.21.15.111:0/615170882 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5f80fd030 con 0x55d4f4a72090 2024-09-15T23:00:05.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.737+0000 7fa5f4c00640 1 -- 172.21.15.111:0/615170882 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa604015640 con 0x55d4f4a72090 2024-09-15T23:00:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.739+0000 7fa614e4a740 10 client.26116 fetch_fsmap learned FSMap version 31 2024-09-15T23:00:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.739+0000 7fa614e4a740 1 -- 172.21.15.111:0/615170882 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d4f4a86360 con 0x55d4f4a72090 2024-09-15T23:00:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.749+0000 7fa5f4c00640 1 -- 172.21.15.111:0/615170882 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f80fd030 con 0x55d4f4a72090 2024-09-15T23:00:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.749+0000 7fa5f4c00640 1 -- 172.21.15.111:0/615170882 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fa60401f070 con 0x55d4f4a72090 2024-09-15T23:00:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.750+0000 7fa614e4a740 10 client.26116 fetch_fsmap finished waiting for FSMap version 31 2024-09-15T23:00:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.751+0000 7fa614e4a740 10 client.26116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:00:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.751+0000 7fa614e4a740 20 client.26116 populate_metadata read hostname 'smithi111' 2024-09-15T23:00:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.751+0000 7fa614e4a740 1 --2- 172.21.15.111:0/615170882 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d4f4a87a30 0x55d4f4a89e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.752+0000 7fa60a000640 1 --2- 172.21.15.111:0/615170882 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d4f4a87a30 0x55d4f4a89e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.752+0000 7fa614e4a740 4 client.26116 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:00:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.753+0000 7fa614e4a740 1 -- 172.21.15.111:0/615170882 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4f49e4830 con 0x55d4f4a87a30 2024-09-15T23:00:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.753+0000 7fa60a000640 1 --2- 172.21.15.111:0/615170882 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d4f4a87a30 0x55d4f4a89e20 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45e58aa6aa4e95ec server_cookie=4b7360ddd9ecee2a in_seq=0 out_seq=0 2024-09-15T23:00:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.753+0000 7fa5f4c00640 10 client.26116 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:00:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.754+0000 7fa5f4c00640 1 -- 172.21.15.111:0/615170882 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d4f49e4830 con 0x55d4f4a87a30 2024-09-15T23:00:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.754+0000 7fa5f4c00640 10 client.26116 handle_command_reply: tid=0 2024-09-15T23:00:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.754+0000 7fa614e4a740 1 client.26116 shutdown 2024-09-15T23:00:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.755+0000 7fa614e4a740 1 -- 172.21.15.111:0/615170882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d4f4a72090 msgr2=0x55d4f4a7bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:05.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.755+0000 7fa614e4a740 1 --2- 172.21.15.111:0/615170882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d4f4a72090 0x55d4f4a7bf60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa60400ca60 tx=0x7fa60400cf20 comp rx=0 tx=0).stop 2024-09-15T23:00:05.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.755+0000 7fa614e4a740 1 -- 172.21.15.111:0/615170882 shutdown_connections 2024-09-15T23:00:05.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.755+0000 7fa614e4a740 1 --2- 172.21.15.111:0/615170882 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d4f4a87a30 0x55d4f4a89e20 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:05.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.755+0000 7fa614e4a740 1 --2- 172.21.15.111:0/615170882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d4f4a77a60 0x55d4f4a834e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:05.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.755+0000 7fa614e4a740 1 --2- 172.21.15.111:0/615170882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4a729d0 0x55d4f4a7c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:05.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.755+0000 7fa614e4a740 1 --2- 172.21.15.111:0/615170882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d4f4a72090 0x55d4f4a7bf60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:05.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.755+0000 7fa614e4a740 1 -- 172.21.15.111:0/615170882 >> 172.21.15.111:0/615170882 conn(0x55d4f4a6d700 msgr2=0x55d4f4a6ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:05.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.755+0000 7fa5f4c00640 0 client.26116 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:00:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.757+0000 7fa614e4a740 1 -- 172.21.15.111:0/615170882 shutdown_connections 2024-09-15T23:00:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.757+0000 7fa614e4a740 1 -- 172.21.15.111:0/615170882 wait complete. 2024-09-15T23:00:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.758+0000 7fa614e4a740 20 client.26116 trim_cache size 0 max 16384 2024-09-15T23:00:05.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.788+0000 7fa614e4a740 1 Processor -- start 2024-09-15T23:00:05.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.788+0000 7fa614e4a740 1 -- start start 2024-09-15T23:00:05.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.789+0000 7fa614e4a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4a72090 0x55d4f4a6c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:05.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.789+0000 7fa614e4a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d4f4a729d0 0x55d4f4a834a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:05.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.789+0000 7fa614e4a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d4f4a77a60 0x55d4f4a6c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:05.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.789+0000 7fa614e4a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c1bad80 con 0x55d4f4a729d0 2024-09-15T23:00:05.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.789+0000 7fa614e4a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c1b85f0 con 0x55d4f4a72090 2024-09-15T23:00:05.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.789+0000 7fa614e4a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c1b6420 con 0x55d4f4a77a60 2024-09-15T23:00:05.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.789+0000 7fa60a000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d4f4a729d0 0x55d4f4a834a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:05.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.789+0000 7fa60aa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4a72090 0x55d4f4a6c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:05.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.789+0000 7fa60aa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4a72090 0x55d4f4a6c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:60694/0 (socket says 172.21.15.111:60694) 2024-09-15T23:00:05.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.789+0000 7fa60aa00640 1 -- 172.21.15.111:0/2120429917 learned_addr learned my addr 172.21.15.111:0/2120429917 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:00:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.789+0000 7fa60a000640 1 -- 172.21.15.111:0/2120429917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d4f4a77a60 msgr2=0x55d4f4a6c5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:00:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.789+0000 7fa60a000640 1 --2- 172.21.15.111:0/2120429917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d4f4a77a60 0x55d4f4a6c5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.789+0000 7fa60a000640 1 -- 172.21.15.111:0/2120429917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4a72090 msgr2=0x55d4f4a6c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.789+0000 7fa60a000640 1 --2- 172.21.15.111:0/2120429917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4a72090 0x55d4f4a6c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.789+0000 7fa60a000640 1 -- 172.21.15.111:0/2120429917 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4f4a86180 con 0x55d4f4a729d0 2024-09-15T23:00:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.789+0000 7fa60aa00640 1 --2- 172.21.15.111:0/2120429917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4a72090 0x55d4f4a6c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:00:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.790+0000 7fa60a000640 1 --2- 172.21.15.111:0/2120429917 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d4f4a729d0 0x55d4f4a834a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80b9090 tx=0x7fa5f8032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a8c5332db26c7b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:05.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.790+0000 7fa5f7400640 1 -- 172.21.15.111:0/2120429917 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8049040 con 0x55d4f4a729d0 2024-09-15T23:00:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.790+0000 7fa5f7400640 1 -- 172.21.15.111:0/2120429917 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f80fd040 con 0x55d4f4a729d0 2024-09-15T23:00:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.790+0000 7fa5f7400640 1 -- 172.21.15.111:0/2120429917 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f80f3d70 con 0x55d4f4a729d0 2024-09-15T23:00:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.791+0000 7fa614e4a740 1 -- 172.21.15.111:0/2120429917 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d4f4a729d0 msgr2=0x55d4f4a834a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:05.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.791+0000 7fa614e4a740 1 --2- 172.21.15.111:0/2120429917 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d4f4a729d0 0x55d4f4a834a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80b9090 tx=0x7fa5f8032f90 comp rx=0 tx=0).stop 2024-09-15T23:00:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.791+0000 7fa614e4a740 1 -- 172.21.15.111:0/2120429917 shutdown_connections 2024-09-15T23:00:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.791+0000 7fa614e4a740 1 --2- 172.21.15.111:0/2120429917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d4f4a77a60 0x55d4f4a6c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.791+0000 7fa614e4a740 1 --2- 172.21.15.111:0/2120429917 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d4f4a729d0 0x55d4f4a834a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.791+0000 7fa614e4a740 1 --2- 172.21.15.111:0/2120429917 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4a72090 0x55d4f4a6c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.791+0000 7fa614e4a740 1 -- 172.21.15.111:0/2120429917 >> 172.21.15.111:0/2120429917 conn(0x55d4f4a6d700 msgr2=0x55d4f4a7c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.791+0000 7fa614e4a740 1 -- 172.21.15.111:0/2120429917 shutdown_connections 2024-09-15T23:00:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.791+0000 7fa614e4a740 1 -- 172.21.15.111:0/2120429917 wait complete. 2024-09-15T23:00:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.791+0000 7fa614e4a740 1 -- start start 2024-09-15T23:00:05.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.792+0000 7fa614e4a740 1 Processor -- start 2024-09-15T23:00:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.792+0000 7fa614e4a740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d4f4a72090 0x55d4f4a805a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.792+0000 7fa614e4a740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d4f4a729d0 0x55d4f4a80ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.792+0000 7fa614e4a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4a77a60 0x55d4f4a81af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.792+0000 7fa614e4a740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c1b6ab0 con 0x55d4f4a72090 2024-09-15T23:00:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.792+0000 7fa614e4a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c1b7f20 con 0x55d4f4a77a60 2024-09-15T23:00:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.792+0000 7fa614e4a740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c1b7180 con 0x55d4f4a729d0 2024-09-15T23:00:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.792+0000 7fa614e4a740 10 client.0 _ll_register_callbacks cb 0x7fa604007600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:00:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.792+0000 7fa60aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d4f4a72090 0x55d4f4a805a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.792+0000 7fa60b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4a77a60 0x55d4f4a81af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.792+0000 7fa60b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4a77a60 0x55d4f4a81af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:60704/0 (socket says 172.21.15.111:60704) 2024-09-15T23:00:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.792+0000 7fa60b400640 1 -- 172.21.15.111:0/2872335210 learned_addr learned my addr 172.21.15.111:0/2872335210 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:00:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.792+0000 7fa60a000640 1 --2- 172.21.15.111:0/2872335210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d4f4a729d0 0x55d4f4a80ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.793+0000 7fa60b400640 1 -- 172.21.15.111:0/2872335210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d4f4a729d0 msgr2=0x55d4f4a80ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.793+0000 7fa60b400640 1 --2- 172.21.15.111:0/2872335210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d4f4a729d0 0x55d4f4a80ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.793+0000 7fa60b400640 1 -- 172.21.15.111:0/2872335210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d4f4a72090 msgr2=0x55d4f4a805a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.793+0000 7fa60b400640 1 --2- 172.21.15.111:0/2872335210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d4f4a72090 0x55d4f4a805a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.793+0000 7fa60b400640 1 -- 172.21.15.111:0/2872335210 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4f4a82100 con 0x55d4f4a77a60 2024-09-15T23:00:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.793+0000 7fa60b400640 1 --2- 172.21.15.111:0/2872335210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4a77a60 0x55d4f4a81af0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa604001220 tx=0x7fa60400aee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63d3c978bb5294f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.793+0000 7fa5f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:00:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.793+0000 7fa5f4c00640 1 -- 172.21.15.111:0/2872335210 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa604025070 con 0x55d4f4a77a60 2024-09-15T23:00:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.793+0000 7fa614e4a740 1 -- 172.21.15.111:0/2872335210 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5f80fd040 con 0x55d4f4a77a60 2024-09-15T23:00:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.793+0000 7fa5f4c00640 1 -- 172.21.15.111:0/2872335210 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fa604007e40 con 0x55d4f4a77a60 2024-09-15T23:00:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.793+0000 7fa5f4c00640 1 -- 172.21.15.111:0/2872335210 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa604015640 con 0x55d4f4a77a60 2024-09-15T23:00:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.793+0000 7fa614e4a740 10 client.26096 fetch_fsmap learned FSMap version 31 2024-09-15T23:00:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.793+0000 7fa614e4a740 1 -- 172.21.15.111:0/2872335210 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d4f4a83470 con 0x55d4f4a77a60 2024-09-15T23:00:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.794+0000 7fa5f4c00640 1 -- 172.21.15.111:0/2872335210 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f80fd040 con 0x55d4f4a77a60 2024-09-15T23:00:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.794+0000 7fa5f4c00640 1 -- 172.21.15.111:0/2872335210 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fa60401f070 con 0x55d4f4a77a60 2024-09-15T23:00:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.794+0000 7fa614e4a740 10 client.26096 fetch_fsmap finished waiting for FSMap version 31 2024-09-15T23:00:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.794+0000 7fa614e4a740 10 client.26096 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:00:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.794+0000 7fa614e4a740 20 client.26096 populate_metadata read hostname 'smithi111' 2024-09-15T23:00:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.794+0000 7fa614e4a740 1 --2- 172.21.15.111:0/2872335210 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d4f4af2b50 0x55d4f4af4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.794+0000 7fa614e4a740 4 client.26096 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:00:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.794+0000 7fa614e4a740 1 -- 172.21.15.111:0/2872335210 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d4f49deb80 con 0x55d4f4af2b50 2024-09-15T23:00:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.794+0000 7fa60aa00640 1 --2- 172.21.15.111:0/2872335210 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d4f4af2b50 0x55d4f4af4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.795+0000 7fa60aa00640 1 --2- 172.21.15.111:0/2872335210 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d4f4af2b50 0x55d4f4af4f70 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95ab997413231c99 server_cookie=f3066b5888cd1c80 in_seq=0 out_seq=0 2024-09-15T23:00:05.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:05.795+0000 7fa5f4c00640 10 client.26096 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:00:05.987 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:05 smithi111 ceph-mon[30629]: pgmap v775: 97 pgs: 97 active+clean; 292 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 456 op/s 2024-09-15T23:00:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:06.035+0000 7fa5f4c00640 1 -- 172.21.15.111:0/2872335210 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d4f49deb80 con 0x55d4f4af2b50 2024-09-15T23:00:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:06.035+0000 7fa5f4c00640 10 client.26096 handle_command_reply: tid=0 2024-09-15T23:00:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:06.036+0000 7fa614e4a740 1 client.26096 shutdown 2024-09-15T23:00:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:06.036+0000 7fa614e4a740 1 -- 172.21.15.111:0/2872335210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4a77a60 msgr2=0x55d4f4a81af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:06.036+0000 7fa614e4a740 1 --2- 172.21.15.111:0/2872335210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4a77a60 0x55d4f4a81af0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa604001220 tx=0x7fa60400aee0 comp rx=0 tx=0).stop 2024-09-15T23:00:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:06.036+0000 7fa614e4a740 1 -- 172.21.15.111:0/2872335210 shutdown_connections 2024-09-15T23:00:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:06.036+0000 7fa614e4a740 1 --2- 172.21.15.111:0/2872335210 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d4f4af2b50 0x55d4f4af4f70 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:06.036+0000 7fa614e4a740 1 --2- 172.21.15.111:0/2872335210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d4f4a77a60 0x55d4f4a81af0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:06.036+0000 7fa614e4a740 1 --2- 172.21.15.111:0/2872335210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d4f4a729d0 0x55d4f4a80ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:06.036+0000 7fa5f4c00640 0 client.26096 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:00:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:06.036+0000 7fa614e4a740 1 --2- 172.21.15.111:0/2872335210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d4f4a72090 0x55d4f4a805a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:06.036+0000 7fa614e4a740 1 -- 172.21.15.111:0/2872335210 >> 172.21.15.111:0/2872335210 conn(0x55d4f4a6d700 msgr2=0x55d4f4a6e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:06.036+0000 7fa614e4a740 1 -- 172.21.15.111:0/2872335210 shutdown_connections 2024-09-15T23:00:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:06.036+0000 7fa614e4a740 1 -- 172.21.15.111:0/2872335210 wait complete. 2024-09-15T23:00:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:06.036+0000 7fa614e4a740 20 client.26096 trim_cache size 0 max 16384 2024-09-15T23:00:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:00:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:00:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:00:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:00:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:06.037+0000 7fa611e00640 1 -- 172.21.15.111:0/1574844659 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa5f00a3280 msgr2=0x7fa5f00a5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:06.037+0000 7fa611e00640 1 --2- 172.21.15.111:0/1574844659 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa5f00a3280 0x7fa5f00a5740 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55d4f4a7d440 tx=0x7fa5fc008040 comp rx=0 tx=0).stop 2024-09-15T23:00:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:06.037+0000 7fa611e00640 1 -- 172.21.15.111:0/1574844659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa60c1099f0 msgr2=0x7fa60c1964a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:06.037+0000 7fa611e00640 1 --2- 172.21.15.111:0/1574844659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa60c1099f0 0x7fa60c1964a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8006010 tx=0x7fa5f8032e20 comp rx=0 tx=0).stop 2024-09-15T23:00:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:06.037+0000 7fa611e00640 1 -- 172.21.15.111:0/1574844659 shutdown_connections 2024-09-15T23:00:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:06.037+0000 7fa611e00640 1 --2- 172.21.15.111:0/1574844659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa60c19b500 0x7fa60c1aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:06.037+0000 7fa611e00640 1 --2- 172.21.15.111:0/1574844659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa60c1099f0 0x7fa60c1964a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:06.037+0000 7fa611e00640 1 --2- 172.21.15.111:0/1574844659 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa5f00a3280 0x7fa5f00a5740 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:06.037+0000 7fa611e00640 1 --2- 172.21.15.111:0/1574844659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa60c0761d0 0x7fa60c195f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:06.037+0000 7fa611e00640 1 -- 172.21.15.111:0/1574844659 >> 172.21.15.111:0/1574844659 conn(0x7fa60c0fd2c0 msgr2=0x7fa60c101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:06.037+0000 7fa611e00640 1 -- 172.21.15.111:0/1574844659 shutdown_connections 2024-09-15T23:00:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:06.037+0000 7fa611e00640 1 -- 172.21.15.111:0/1574844659 wait complete. 2024-09-15T23:00:06.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:05 smithi121 ceph-mon[34815]: pgmap v775: 97 pgs: 97 active+clean; 292 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 456 op/s 2024-09-15T23:00:06.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:05 smithi192 ceph-mon[35427]: pgmap v775: 97 pgs: 97 active+clean; 292 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 456 op/s 2024-09-15T23:00:06.588 INFO:tasks.workunit.client.0.smithi111.stdout: 10 16976 31.79 MB/sec warmup 34 sec latency 1199.923 ms 2024-09-15T23:00:07.588 INFO:tasks.workunit.client.0.smithi111.stdout: 10 17932 32.59 MB/sec warmup 35 sec latency 77.004 ms 2024-09-15T23:00:08.116 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:07 smithi192 ceph-mon[35427]: pgmap v776: 97 pgs: 97 active+clean; 364 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 10 MiB/s wr, 454 op/s 2024-09-15T23:00:08.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:07 smithi121 ceph-mon[34815]: pgmap v776: 97 pgs: 97 active+clean; 364 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 10 MiB/s wr, 454 op/s 2024-09-15T23:00:08.143 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:07 smithi111 ceph-mon[30629]: pgmap v776: 97 pgs: 97 active+clean; 364 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 10 MiB/s wr, 454 op/s 2024-09-15T23:00:08.589 INFO:tasks.workunit.client.0.smithi111.stdout: 10 18587 32.69 MB/sec warmup 36 sec latency 162.632 ms 2024-09-15T23:00:09.588 INFO:tasks.workunit.client.0.smithi111.stdout: 10 19303 32.81 MB/sec warmup 37 sec latency 81.794 ms 2024-09-15T23:00:10.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:09 smithi121 ceph-mon[34815]: pgmap v777: 97 pgs: 97 active+clean; 453 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 17 MiB/s wr, 488 op/s 2024-09-15T23:00:10.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:09 smithi111 ceph-mon[30629]: pgmap v777: 97 pgs: 97 active+clean; 453 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 17 MiB/s wr, 488 op/s 2024-09-15T23:00:10.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:09 smithi192 ceph-mon[35427]: pgmap v777: 97 pgs: 97 active+clean; 453 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 17 MiB/s wr, 488 op/s 2024-09-15T23:00:10.588 INFO:tasks.workunit.client.0.smithi111.stdout: 10 20021 33.08 MB/sec warmup 38 sec latency 183.076 ms 2024-09-15T23:00:11.588 INFO:tasks.workunit.client.0.smithi111.stdout: 10 20391 32.68 MB/sec warmup 39 sec latency 465.002 ms 2024-09-15T23:00:11.998 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:11 smithi111 ceph-mon[30629]: pgmap v778: 97 pgs: 97 active+clean; 493 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 20 MiB/s wr, 505 op/s 2024-09-15T23:00:12.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:11 smithi121 ceph-mon[34815]: pgmap v778: 97 pgs: 97 active+clean; 493 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 20 MiB/s wr, 505 op/s 2024-09-15T23:00:12.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:11 smithi192 ceph-mon[35427]: pgmap v778: 97 pgs: 97 active+clean; 493 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 20 MiB/s wr, 505 op/s 2024-09-15T23:00:12.588 INFO:tasks.workunit.client.0.smithi111.stdout: 10 20391 31.87 MB/sec warmup 40 sec latency 1465.069 ms 2024-09-15T23:00:13.588 INFO:tasks.workunit.client.0.smithi111.stdout: 10 20392 31.09 MB/sec warmup 41 sec latency 2465.135 ms 2024-09-15T23:00:14.049 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:13 smithi111 ceph-mon[30629]: pgmap v779: 97 pgs: 97 active+clean; 503 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 21 MiB/s wr, 584 op/s 2024-09-15T23:00:14.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:13 smithi121 ceph-mon[34815]: pgmap v779: 97 pgs: 97 active+clean; 503 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 21 MiB/s wr, 584 op/s 2024-09-15T23:00:14.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:13 smithi192 ceph-mon[35427]: pgmap v779: 97 pgs: 97 active+clean; 503 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 21 MiB/s wr, 584 op/s 2024-09-15T23:00:14.589 INFO:tasks.workunit.client.0.smithi111.stdout: 10 20392 30.35 MB/sec warmup 42 sec latency 3465.202 ms 2024-09-15T23:00:15.589 INFO:tasks.workunit.client.0.smithi111.stdout: 10 20434 29.72 MB/sec warmup 43 sec latency 4465.243 ms 2024-09-15T23:00:16.104 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:15 smithi111 ceph-mon[30629]: pgmap v780: 97 pgs: 97 active+clean; 514 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 22 MiB/s wr, 594 op/s 2024-09-15T23:00:16.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:15 smithi121 ceph-mon[34815]: pgmap v780: 97 pgs: 97 active+clean; 514 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 22 MiB/s wr, 594 op/s 2024-09-15T23:00:16.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:15 smithi192 ceph-mon[35427]: pgmap v780: 97 pgs: 97 active+clean; 514 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 22 MiB/s wr, 594 op/s 2024-09-15T23:00:16.591 INFO:tasks.workunit.client.0.smithi111.stdout: 10 20640 29.31 MB/sec warmup 44 sec latency 4788.362 ms 2024-09-15T23:00:17.591 INFO:tasks.workunit.client.0.smithi111.stdout: 10 20643 28.66 MB/sec warmup 45 sec latency 1282.261 ms 2024-09-15T23:00:18.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:17 smithi121 ceph-mon[34815]: pgmap v781: 97 pgs: 97 active+clean; 514 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 21 MiB/s wr, 548 op/s 2024-09-15T23:00:18.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:17 smithi111 ceph-mon[30629]: pgmap v781: 97 pgs: 97 active+clean; 514 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 21 MiB/s wr, 548 op/s 2024-09-15T23:00:18.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:17 smithi192 ceph-mon[35427]: pgmap v781: 97 pgs: 97 active+clean; 514 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 21 MiB/s wr, 548 op/s 2024-09-15T23:00:18.591 INFO:tasks.workunit.client.0.smithi111.stdout: 10 20643 28.03 MB/sec warmup 46 sec latency 2282.330 ms 2024-09-15T23:00:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.153+0000 7f344b200640 1 -- 172.21.15.111:0/1182221076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3444107620 msgr2=0x7f3444109a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.153+0000 7f344b200640 1 --2- 172.21.15.111:0/1182221076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3444107620 0x7f3444109a90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3440009ca0 tx=0x7f344002a8e0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.153+0000 7f344b200640 1 -- 172.21.15.111:0/1182221076 shutdown_connections 2024-09-15T23:00:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.153+0000 7f344b200640 1 --2- 172.21.15.111:0/1182221076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3444107620 0x7f3444109a90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.153+0000 7f344b200640 1 --2- 172.21.15.111:0/1182221076 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3444104cf0 0x7f34441070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.153+0000 7f344b200640 1 --2- 172.21.15.111:0/1182221076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f34441023c0 0x7f34441047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.153+0000 7f344b200640 1 -- 172.21.15.111:0/1182221076 >> 172.21.15.111:0/1182221076 conn(0x7f34440fbfa0 msgr2=0x7f34440fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.153+0000 7f344b200640 1 -- 172.21.15.111:0/1182221076 shutdown_connections 2024-09-15T23:00:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.153+0000 7f344b200640 1 -- 172.21.15.111:0/1182221076 wait complete. 2024-09-15T23:00:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.154+0000 7f344b200640 1 Processor -- start 2024-09-15T23:00:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.154+0000 7f344b200640 1 -- start start 2024-09-15T23:00:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.154+0000 7f344b200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34441023c0 0x7f3444197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.154+0000 7f344b200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3444104cf0 0x7f34441982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.154+0000 7f344b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3444107620 0x7f344419f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.154+0000 7f344b200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f344410d650 con 0x7f3444104cf0 2024-09-15T23:00:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.154+0000 7f344b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f344410d4d0 con 0x7f3444107620 2024-09-15T23:00:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.154+0000 7f344b200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f344410d7d0 con 0x7f34441023c0 2024-09-15T23:00:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.155+0000 7f3449400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3444104cf0 0x7f34441982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.155+0000 7f3449400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3444104cf0 0x7f34441982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43494/0 (socket says 172.21.15.111:43494) 2024-09-15T23:00:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.155+0000 7f3449400640 1 -- 172.21.15.111:0/2654914638 learned_addr learned my addr 172.21.15.111:0/2654914638 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:00:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.155+0000 7f3449e00640 1 --2- 172.21.15.111:0/2654914638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34441023c0 0x7f3444197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.155+0000 7f344a800640 1 --2- 172.21.15.111:0/2654914638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3444107620 0x7f344419f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.155+0000 7f344a800640 1 -- 172.21.15.111:0/2654914638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34441023c0 msgr2=0x7f3444197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.155+0000 7f344a800640 1 --2- 172.21.15.111:0/2654914638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34441023c0 0x7f3444197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.155+0000 7f344a800640 1 -- 172.21.15.111:0/2654914638 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3444104cf0 msgr2=0x7f34441982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.155+0000 7f344a800640 1 --2- 172.21.15.111:0/2654914638 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3444104cf0 0x7f34441982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.155+0000 7f344a800640 1 -- 172.21.15.111:0/2654914638 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f344419f980 con 0x7f3444107620 2024-09-15T23:00:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.155+0000 7f3449400640 1 --2- 172.21.15.111:0/2654914638 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3444104cf0 0x7f34441982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:00:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.155+0000 7f344a800640 1 --2- 172.21.15.111:0/2654914638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3444107620 0x7f344419f310 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3440031930 tx=0x7f3440003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:19.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.156+0000 7f343ea00640 1 -- 172.21.15.111:0/2654914638 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3440004040 con 0x7f3444107620 2024-09-15T23:00:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.156+0000 7f343ea00640 1 -- 172.21.15.111:0/2654914638 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34400046c0 con 0x7f3444107620 2024-09-15T23:00:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.156+0000 7f344b200640 1 -- 172.21.15.111:0/2654914638 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f344419fc10 con 0x7f3444107620 2024-09-15T23:00:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.156+0000 7f343ea00640 1 -- 172.21.15.111:0/2654914638 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344003b400 con 0x7f3444107620 2024-09-15T23:00:19.159 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.156+0000 7f344b200640 1 -- 172.21.15.111:0/2654914638 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34441a0160 con 0x7f3444107620 2024-09-15T23:00:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.159+0000 7f344f6b6740 1 Processor -- start 2024-09-15T23:00:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.159+0000 7f343ea00640 1 -- 172.21.15.111:0/2654914638 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f3440030670 con 0x7f3444107620 2024-09-15T23:00:19.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.159+0000 7f344f6b6740 1 -- start start 2024-09-15T23:00:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.159+0000 7f344f6b6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5630b111f670 0x5630b111fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.159+0000 7f344f6b6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5630b111ffb0 0x5630b1124a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.159+0000 7f344f6b6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630b1125040 0x5630b1127500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.159+0000 7f344f6b6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5630b1080100 con 0x5630b1125040 2024-09-15T23:00:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.159+0000 7f344f6b6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5630b10281b0 con 0x5630b111ffb0 2024-09-15T23:00:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.159+0000 7f344f6b6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5630b107c0c0 con 0x5630b111f670 2024-09-15T23:00:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.160+0000 7f3449400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5630b111f670 0x5630b111fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:19.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.160+0000 7f3449400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5630b111f670 0x5630b111fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:58584/0 (socket says 172.21.15.111:58584) 2024-09-15T23:00:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.160+0000 7f3449e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5630b111ffb0 0x5630b1124a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.160+0000 7f3449400640 1 -- 172.21.15.111:0/1470089111 learned_addr learned my addr 172.21.15.111:0/1470089111 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:00:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.160+0000 7f3449400640 1 --2- 172.21.15.111:0/1470089111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630b1125040 0x5630b1127500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.160+0000 7f3449e00640 1 -- 172.21.15.111:0/1470089111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5630b111f670 msgr2=0x5630b111fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.160+0000 7f3449e00640 1 --2- 172.21.15.111:0/1470089111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5630b111f670 0x5630b111fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.160+0000 7f3449e00640 1 -- 172.21.15.111:0/1470089111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630b1125040 msgr2=0x5630b1127500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.160+0000 7f3449e00640 1 --2- 172.21.15.111:0/1470089111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630b1125040 0x5630b1127500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.160+0000 7f3449e00640 1 -- 172.21.15.111:0/1470089111 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630b0d93210 con 0x5630b111ffb0 2024-09-15T23:00:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.161+0000 7f3449e00640 1 --2- 172.21.15.111:0/1470089111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5630b111ffb0 0x5630b1124a70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3438009c20 tx=0x7f3438011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ae3ad9c47a9af29 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:19.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.161+0000 7f343ea00640 1 --2- 172.21.15.111:0/2654914638 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f342c0a3240 0x7f342c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.161+0000 7f3433000640 1 -- 172.21.15.111:0/1470089111 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343801e070 con 0x5630b111ffb0 2024-09-15T23:00:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.161+0000 7f343ea00640 1 -- 172.21.15.111:0/2654914638 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f34400f0bd0 con 0x7f3444107620 2024-09-15T23:00:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.161+0000 7f3449400640 1 --2- 172.21.15.111:0/2654914638 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f342c0a3240 0x7f342c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.161+0000 7f3433000640 1 -- 172.21.15.111:0/1470089111 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3438009dd0 con 0x5630b111ffb0 2024-09-15T23:00:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.161+0000 7f3433000640 1 -- 172.21.15.111:0/1470089111 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34380187b0 con 0x5630b111ffb0 2024-09-15T23:00:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.162+0000 7f3449400640 1 --2- 172.21.15.111:0/2654914638 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f342c0a3240 0x7f342c0a5700 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3434015a40 tx=0x7f3434008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.162+0000 7f344f6b6740 1 -- 172.21.15.111:0/1470089111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5630b111ffb0 msgr2=0x5630b1124a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.162+0000 7f344f6b6740 1 --2- 172.21.15.111:0/1470089111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5630b111ffb0 0x5630b1124a70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3438009c20 tx=0x7f3438011810 comp rx=0 tx=0).stop 2024-09-15T23:00:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.162+0000 7f344f6b6740 1 -- 172.21.15.111:0/1470089111 shutdown_connections 2024-09-15T23:00:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.162+0000 7f344f6b6740 1 --2- 172.21.15.111:0/1470089111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630b1125040 0x5630b1127500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.162+0000 7f344f6b6740 1 --2- 172.21.15.111:0/1470089111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5630b111ffb0 0x5630b1124a70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.162+0000 7f344f6b6740 1 --2- 172.21.15.111:0/1470089111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5630b111f670 0x5630b111fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.162+0000 7f344f6b6740 1 -- 172.21.15.111:0/1470089111 >> 172.21.15.111:0/1470089111 conn(0x5630b111ace0 msgr2=0x5630b111b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.162+0000 7f344f6b6740 1 -- 172.21.15.111:0/1470089111 shutdown_connections 2024-09-15T23:00:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.162+0000 7f344f6b6740 1 -- 172.21.15.111:0/1470089111 wait complete. 2024-09-15T23:00:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.162+0000 7f344f6b6740 1 -- start start 2024-09-15T23:00:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.163+0000 7f344f6b6740 1 Processor -- start 2024-09-15T23:00:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.163+0000 7f344f6b6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630b111f670 0x5630b1129810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.163+0000 7f344f6b6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5630b111ffb0 0x5630b1129d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.163+0000 7f344f6b6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5630b1125040 0x5630b1130d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.163+0000 7f344f6b6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5630b1127fb0 con 0x5630b111f670 2024-09-15T23:00:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.163+0000 7f344f6b6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5630b1009090 con 0x5630b1125040 2024-09-15T23:00:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.163+0000 7f344f6b6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5630b1128130 con 0x5630b111ffb0 2024-09-15T23:00:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.163+0000 7f344f6b6740 10 client.0 _ll_register_callbacks cb 0x5630b105df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:00:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.164+0000 7f3449e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630b111f670 0x5630b1129810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.164+0000 7f3449400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5630b111ffb0 0x5630b1129d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.164+0000 7f3449400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5630b111ffb0 0x5630b1129d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:58586/0 (socket says 172.21.15.111:58586) 2024-09-15T23:00:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.164+0000 7f3449e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630b111f670 0x5630b1129810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43506/0 (socket says 172.21.15.111:43506) 2024-09-15T23:00:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.164+0000 7f3449400640 1 -- 172.21.15.111:0/1705296280 learned_addr learned my addr 172.21.15.111:0/1705296280 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:00:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.164+0000 7f3449e00640 1 -- 172.21.15.111:0/1705296280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5630b111ffb0 msgr2=0x5630b1129d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.164+0000 7f3449e00640 1 --2- 172.21.15.111:0/1705296280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5630b111ffb0 0x5630b1129d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.164+0000 7f3449e00640 1 -- 172.21.15.111:0/1705296280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5630b1125040 msgr2=0x5630b1130d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:00:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.164+0000 7f3449e00640 1 --2- 172.21.15.111:0/1705296280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5630b1125040 0x5630b1130d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.164+0000 7f3449e00640 1 -- 172.21.15.111:0/1705296280 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630b0e5f620 con 0x5630b111f670 2024-09-15T23:00:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.164+0000 7f3449e00640 1 --2- 172.21.15.111:0/1705296280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630b111f670 0x5630b1129810 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3438009bf0 tx=0x7f34380052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84704cefc000aa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.164+0000 7f3427400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:00:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.165+0000 7f3427400640 1 -- 172.21.15.111:0/1705296280 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343801e040 con 0x5630b111f670 2024-09-15T23:00:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.165+0000 7f3427400640 1 -- 172.21.15.111:0/1705296280 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f3438016040 con 0x5630b111f670 2024-09-15T23:00:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.165+0000 7f344f6b6740 1 -- 172.21.15.111:0/1705296280 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3438009dd0 con 0x5630b111f670 2024-09-15T23:00:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.165+0000 7f3427400640 1 -- 172.21.15.111:0/1705296280 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3438004490 con 0x5630b111f670 2024-09-15T23:00:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.165+0000 7f344f6b6740 10 client.16143 fetch_fsmap learned FSMap version 31 2024-09-15T23:00:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.165+0000 7f344f6b6740 1 -- 172.21.15.111:0/1705296280 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5630b1133bd0 con 0x5630b111f670 2024-09-15T23:00:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.165+0000 7f3427400640 1 -- 172.21.15.111:0/1705296280 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3438009dd0 con 0x5630b111f670 2024-09-15T23:00:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.165+0000 7f3427400640 1 -- 172.21.15.111:0/1705296280 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f343802d020 con 0x5630b111f670 2024-09-15T23:00:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.165+0000 7f344f6b6740 10 client.16143 fetch_fsmap finished waiting for FSMap version 31 2024-09-15T23:00:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.165+0000 7f344f6b6740 10 client.16143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:00:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.165+0000 7f344f6b6740 20 client.16143 populate_metadata read hostname 'smithi111' 2024-09-15T23:00:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.165+0000 7f344f6b6740 1 --2- 172.21.15.111:0/1705296280 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5630b11350a0 0x5630b11374b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.165+0000 7f344f6b6740 4 client.16143 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:00:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.165+0000 7f344f6b6740 1 -- 172.21.15.111:0/1705296280 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630b108d830 con 0x5630b11350a0 2024-09-15T23:00:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.166+0000 7f3449400640 1 --2- 172.21.15.111:0/1705296280 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5630b11350a0 0x5630b11374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.166+0000 7f3449400640 1 --2- 172.21.15.111:0/1705296280 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5630b11350a0 0x5630b11374b0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=295669572ae2a5d9 server_cookie=467a85a391ce9aa7 in_seq=0 out_seq=0 2024-09-15T23:00:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.166+0000 7f3427400640 10 client.16143 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:00:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.167+0000 7f3427400640 1 -- 172.21.15.111:0/1705296280 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5630b108d830 con 0x5630b11350a0 2024-09-15T23:00:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.167+0000 7f3427400640 10 client.16143 handle_command_reply: tid=0 2024-09-15T23:00:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.167+0000 7f344f6b6740 1 client.16143 shutdown 2024-09-15T23:00:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.167+0000 7f344f6b6740 1 -- 172.21.15.111:0/1705296280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630b111f670 msgr2=0x5630b1129810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.167+0000 7f344f6b6740 1 --2- 172.21.15.111:0/1705296280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630b111f670 0x5630b1129810 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3438009bf0 tx=0x7f34380052f0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.167+0000 7f344f6b6740 1 -- 172.21.15.111:0/1705296280 shutdown_connections 2024-09-15T23:00:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.167+0000 7f344f6b6740 1 --2- 172.21.15.111:0/1705296280 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5630b11350a0 0x5630b11374b0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.167+0000 7f344f6b6740 1 --2- 172.21.15.111:0/1705296280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5630b1125040 0x5630b1130d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.167+0000 7f344f6b6740 1 --2- 172.21.15.111:0/1705296280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5630b111ffb0 0x5630b1129d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.167+0000 7f344f6b6740 1 --2- 172.21.15.111:0/1705296280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630b111f670 0x5630b1129810 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.167+0000 7f3427400640 0 client.16143 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:00:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.167+0000 7f344f6b6740 1 -- 172.21.15.111:0/1705296280 >> 172.21.15.111:0/1705296280 conn(0x5630b111ace0 msgr2=0x5630b111c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.168+0000 7f344f6b6740 1 -- 172.21.15.111:0/1705296280 shutdown_connections 2024-09-15T23:00:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.168+0000 7f344f6b6740 1 -- 172.21.15.111:0/1705296280 wait complete. 2024-09-15T23:00:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.168+0000 7f344f6b6740 20 client.16143 trim_cache size 0 max 16384 2024-09-15T23:00:19.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.195+0000 7f344f6b6740 1 Processor -- start 2024-09-15T23:00:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.195+0000 7f344f6b6740 1 -- start start 2024-09-15T23:00:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.195+0000 7f344f6b6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5630b111f670 0x5630b1115900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.195+0000 7f344f6b6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630b111ffb0 0x5630b1130cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.195+0000 7f344f6b6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5630b1125040 0x5630b111c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.195+0000 7f344f6b6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f34441a9420 con 0x5630b111ffb0 2024-09-15T23:00:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.195+0000 7f344f6b6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f34441a6c90 con 0x5630b1125040 2024-09-15T23:00:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.195+0000 7f344f6b6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f34441a4ae0 con 0x5630b111f670 2024-09-15T23:00:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.196+0000 7f3449400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630b111ffb0 0x5630b1130cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.196+0000 7f3449e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5630b111f670 0x5630b1115900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.196+0000 7f3449400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630b111ffb0 0x5630b1130cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43514/0 (socket says 172.21.15.111:43514) 2024-09-15T23:00:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.196+0000 7f3449400640 1 -- 172.21.15.111:0/3449089973 learned_addr learned my addr 172.21.15.111:0/3449089973 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:00:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.196+0000 7f344a800640 1 --2- 172.21.15.111:0/3449089973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5630b1125040 0x5630b111c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.196+0000 7f3449e00640 1 -- 172.21.15.111:0/3449089973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5630b1125040 msgr2=0x5630b111c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.196+0000 7f3449e00640 1 --2- 172.21.15.111:0/3449089973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5630b1125040 0x5630b111c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.196+0000 7f3449e00640 1 -- 172.21.15.111:0/3449089973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630b111ffb0 msgr2=0x5630b1130cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.196+0000 7f3449e00640 1 --2- 172.21.15.111:0/3449089973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630b111ffb0 0x5630b1130cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.196+0000 7f3449e00640 1 -- 172.21.15.111:0/3449089973 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630b1137b30 con 0x5630b111f670 2024-09-15T23:00:19.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.196+0000 7f3449400640 1 --2- 172.21.15.111:0/3449089973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630b111ffb0 0x5630b1130cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:00:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.196+0000 7f3449e00640 1 --2- 172.21.15.111:0/3449089973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5630b111f670 0x5630b1115900 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f34380048a0 tx=0x7f3438011d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=761feda47bd823d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.197+0000 7f3432600640 1 -- 172.21.15.111:0/3449089973 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3438009dd0 con 0x5630b111f670 2024-09-15T23:00:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.197+0000 7f3432600640 1 -- 172.21.15.111:0/3449089973 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3438004dc0 con 0x5630b111f670 2024-09-15T23:00:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.197+0000 7f3432600640 1 -- 172.21.15.111:0/3449089973 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3438019b50 con 0x5630b111f670 2024-09-15T23:00:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.197+0000 7f344f6b6740 1 -- 172.21.15.111:0/3449089973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5630b111f670 msgr2=0x5630b1115900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.197+0000 7f344f6b6740 1 --2- 172.21.15.111:0/3449089973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5630b111f670 0x5630b1115900 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f34380048a0 tx=0x7f3438011d10 comp rx=0 tx=0).stop 2024-09-15T23:00:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.197+0000 7f344f6b6740 1 -- 172.21.15.111:0/3449089973 shutdown_connections 2024-09-15T23:00:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.197+0000 7f344f6b6740 1 --2- 172.21.15.111:0/3449089973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5630b1125040 0x5630b111c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.197+0000 7f344f6b6740 1 --2- 172.21.15.111:0/3449089973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630b111ffb0 0x5630b1130cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.197+0000 7f344f6b6740 1 --2- 172.21.15.111:0/3449089973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5630b111f670 0x5630b1115900 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.197+0000 7f344f6b6740 1 -- 172.21.15.111:0/3449089973 >> 172.21.15.111:0/3449089973 conn(0x5630b111ace0 msgr2=0x5630b11151f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.197+0000 7f344f6b6740 1 -- 172.21.15.111:0/3449089973 shutdown_connections 2024-09-15T23:00:19.200 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.197+0000 7f344f6b6740 1 -- 172.21.15.111:0/3449089973 wait complete. 2024-09-15T23:00:19.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.198+0000 7f344f6b6740 1 -- start start 2024-09-15T23:00:19.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.198+0000 7f344f6b6740 1 Processor -- start 2024-09-15T23:00:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.198+0000 7f344f6b6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5630b111f670 0x5630b112dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.198+0000 7f344f6b6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630b111ffb0 0x5630b112e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.198+0000 7f344f6b6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5630b1125040 0x5630b112f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.198+0000 7f344f6b6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f34441a5150 con 0x5630b111ffb0 2024-09-15T23:00:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.198+0000 7f344f6b6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f34441a65c0 con 0x5630b111f670 2024-09-15T23:00:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.198+0000 7f344f6b6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f34441a5820 con 0x5630b1125040 2024-09-15T23:00:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.198+0000 7f344f6b6740 10 client.0 _ll_register_callbacks cb 0x7f343801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:00:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.198+0000 7f3449400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630b111ffb0 0x5630b112e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.198+0000 7f3449400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630b111ffb0 0x5630b112e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43522/0 (socket says 172.21.15.111:43522) 2024-09-15T23:00:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.198+0000 7f3449400640 1 -- 172.21.15.111:0/1706587061 learned_addr learned my addr 172.21.15.111:0/1706587061 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:00:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.199+0000 7f3449e00640 1 --2- 172.21.15.111:0/1706587061 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5630b111f670 0x5630b112dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.199+0000 7f3449400640 1 -- 172.21.15.111:0/1706587061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5630b1125040 msgr2=0x5630b112f460 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:00:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.199+0000 7f3449400640 1 --2- 172.21.15.111:0/1706587061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5630b1125040 0x5630b112f460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.199+0000 7f3449400640 1 -- 172.21.15.111:0/1706587061 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5630b111f670 msgr2=0x5630b112dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.199+0000 7f3449400640 1 --2- 172.21.15.111:0/1706587061 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5630b111f670 0x5630b112dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.199+0000 7f3449400640 1 -- 172.21.15.111:0/1706587061 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630b112fa70 con 0x5630b111ffb0 2024-09-15T23:00:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.199+0000 7f3449400640 1 --2- 172.21.15.111:0/1706587061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630b111ffb0 0x5630b112e4f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3434015870 tx=0x7f34340046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f452cb2cfdfdca7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.199+0000 7f3427400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:00:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.199+0000 7f3427400640 1 -- 172.21.15.111:0/1706587061 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3434026d30 con 0x5630b111ffb0 2024-09-15T23:00:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.199+0000 7f3427400640 1 -- 172.21.15.111:0/1706587061 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f3434018450 con 0x5630b111ffb0 2024-09-15T23:00:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.199+0000 7f344f6b6740 1 -- 172.21.15.111:0/1706587061 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3438004dc0 con 0x5630b111ffb0 2024-09-15T23:00:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.200+0000 7f3427400640 1 -- 172.21.15.111:0/1706587061 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3434018650 con 0x5630b111ffb0 2024-09-15T23:00:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.200+0000 7f344f6b6740 10 client.16149 fetch_fsmap learned FSMap version 31 2024-09-15T23:00:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.200+0000 7f344f6b6740 1 -- 172.21.15.111:0/1706587061 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5630b1130e50 con 0x5630b111ffb0 2024-09-15T23:00:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.200+0000 7f3427400640 1 -- 172.21.15.111:0/1706587061 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3438004dc0 con 0x5630b111ffb0 2024-09-15T23:00:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.200+0000 7f3427400640 1 -- 172.21.15.111:0/1706587061 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f3434023040 con 0x5630b111ffb0 2024-09-15T23:00:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.200+0000 7f344f6b6740 10 client.16149 fetch_fsmap finished waiting for FSMap version 31 2024-09-15T23:00:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.200+0000 7f344f6b6740 10 client.16149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:00:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.200+0000 7f344f6b6740 20 client.16149 populate_metadata read hostname 'smithi111' 2024-09-15T23:00:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.200+0000 7f344f6b6740 1 --2- 172.21.15.111:0/1706587061 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5630b11a0360 0x5630b11a2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.200+0000 7f344f6b6740 4 client.16149 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:00:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.200+0000 7f344f6b6740 1 -- 172.21.15.111:0/1706587061 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5630b1087b80 con 0x5630b11a0360 2024-09-15T23:00:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.201+0000 7f3449e00640 1 --2- 172.21.15.111:0/1706587061 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5630b11a0360 0x5630b11a2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.201+0000 7f3449e00640 1 --2- 172.21.15.111:0/1706587061 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5630b11a0360 0x5630b11a2780 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=985200a57e0b4820 server_cookie=cffb2376a1c14994 in_seq=0 out_seq=0 2024-09-15T23:00:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.201+0000 7f3427400640 10 client.16149 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:00:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.331+0000 7f3427400640 1 -- 172.21.15.111:0/1706587061 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5630b1087b80 con 0x5630b11a0360 2024-09-15T23:00:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.331+0000 7f3427400640 10 client.16149 handle_command_reply: tid=0 2024-09-15T23:00:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.331+0000 7f344f6b6740 1 client.16149 shutdown 2024-09-15T23:00:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.331+0000 7f344f6b6740 1 -- 172.21.15.111:0/1706587061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630b111ffb0 msgr2=0x5630b112e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.331+0000 7f344f6b6740 1 --2- 172.21.15.111:0/1706587061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630b111ffb0 0x5630b112e4f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3434015870 tx=0x7f34340046e0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.331+0000 7f344f6b6740 1 -- 172.21.15.111:0/1706587061 shutdown_connections 2024-09-15T23:00:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.331+0000 7f344f6b6740 1 --2- 172.21.15.111:0/1706587061 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5630b11a0360 0x5630b11a2780 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.332+0000 7f344f6b6740 1 --2- 172.21.15.111:0/1706587061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5630b1125040 0x5630b112f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.332+0000 7f3427400640 0 client.16149 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:00:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.332+0000 7f344f6b6740 1 --2- 172.21.15.111:0/1706587061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5630b111ffb0 0x5630b112e4f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.332+0000 7f344f6b6740 1 --2- 172.21.15.111:0/1706587061 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5630b111f670 0x5630b112dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.332+0000 7f344f6b6740 1 -- 172.21.15.111:0/1706587061 >> 172.21.15.111:0/1706587061 conn(0x5630b111ace0 msgr2=0x5630b1129ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.332+0000 7f344f6b6740 1 -- 172.21.15.111:0/1706587061 shutdown_connections 2024-09-15T23:00:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.332+0000 7f344f6b6740 1 -- 172.21.15.111:0/1706587061 wait complete. 2024-09-15T23:00:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.332+0000 7f344f6b6740 20 client.16149 trim_cache size 0 max 16384 2024-09-15T23:00:19.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:00:19.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:00:19.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:00:19.335 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:00:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.332+0000 7f344b200640 1 -- 172.21.15.111:0/2654914638 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f342c0a3240 msgr2=0x7f342c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.332+0000 7f344b200640 1 --2- 172.21.15.111:0/2654914638 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f342c0a3240 0x7f342c0a5700 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3434015a40 tx=0x7f3434008040 comp rx=0 tx=0).stop 2024-09-15T23:00:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.332+0000 7f344b200640 1 -- 172.21.15.111:0/2654914638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3444107620 msgr2=0x7f344419f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.332+0000 7f344b200640 1 --2- 172.21.15.111:0/2654914638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3444107620 0x7f344419f310 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3440031930 tx=0x7f3440003ba0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.333+0000 7f344b200640 1 -- 172.21.15.111:0/2654914638 shutdown_connections 2024-09-15T23:00:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.333+0000 7f344b200640 1 --2- 172.21.15.111:0/2654914638 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f342c0a3240 0x7f342c0a5700 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.333+0000 7f344b200640 1 --2- 172.21.15.111:0/2654914638 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3444107620 0x7f344419f310 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.333+0000 7f344b200640 1 --2- 172.21.15.111:0/2654914638 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3444104cf0 0x7f34441982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.333+0000 7f344b200640 1 --2- 172.21.15.111:0/2654914638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34441023c0 0x7f3444197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.333+0000 7f344b200640 1 -- 172.21.15.111:0/2654914638 >> 172.21.15.111:0/2654914638 conn(0x7f34440fbfa0 msgr2=0x7f3444102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.333+0000 7f344b200640 1 -- 172.21.15.111:0/2654914638 shutdown_connections 2024-09-15T23:00:19.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:19.333+0000 7f344b200640 1 -- 172.21.15.111:0/2654914638 wait complete. 2024-09-15T23:00:19.591 INFO:tasks.workunit.client.0.smithi111.stdout: 10 20643 27.44 MB/sec warmup 47 sec latency 3282.386 ms 2024-09-15T23:00:19.979 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:19 smithi111 ceph-mon[30629]: pgmap v782: 97 pgs: 97 active+clean; 510 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 16 MiB/s wr, 493 op/s 2024-09-15T23:00:20.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:19 smithi121 ceph-mon[34815]: pgmap v782: 97 pgs: 97 active+clean; 510 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 16 MiB/s wr, 493 op/s 2024-09-15T23:00:20.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:19 smithi192 ceph-mon[35427]: pgmap v782: 97 pgs: 97 active+clean; 510 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 16 MiB/s wr, 493 op/s 2024-09-15T23:00:20.591 INFO:tasks.workunit.client.0.smithi111.stdout: 10 21109 27.22 MB/sec warmup 48 sec latency 4006.896 ms 2024-09-15T23:00:21.595 INFO:tasks.workunit.client.0.smithi111.stdout: 10 21647 27.27 MB/sec warmup 49 sec latency 195.920 ms 2024-09-15T23:00:21.937 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:21 smithi111 ceph-mon[30629]: pgmap v783: 97 pgs: 97 active+clean; 499 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.7 MiB/s wr, 413 op/s 2024-09-15T23:00:22.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:21 smithi192 ceph-mon[35427]: pgmap v783: 97 pgs: 97 active+clean; 499 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.7 MiB/s wr, 413 op/s 2024-09-15T23:00:22.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:21 smithi121 ceph-mon[34815]: pgmap v783: 97 pgs: 97 active+clean; 499 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.7 MiB/s wr, 413 op/s 2024-09-15T23:00:22.595 INFO:tasks.workunit.client.0.smithi111.stdout: 10 21648 26.73 MB/sec warmup 50 sec latency 1168.886 ms 2024-09-15T23:00:23.595 INFO:tasks.workunit.client.0.smithi111.stdout: 10 21650 26.20 MB/sec warmup 51 sec latency 2168.971 ms 2024-09-15T23:00:24.120 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:23 smithi111 ceph-mon[30629]: pgmap v784: 97 pgs: 97 active+clean; 502 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.7 MiB/s wr, 382 op/s 2024-09-15T23:00:24.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:23 smithi192 ceph-mon[35427]: pgmap v784: 97 pgs: 97 active+clean; 502 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.7 MiB/s wr, 382 op/s 2024-09-15T23:00:24.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:23 smithi121 ceph-mon[34815]: pgmap v784: 97 pgs: 97 active+clean; 502 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.7 MiB/s wr, 382 op/s 2024-09-15T23:00:24.595 INFO:tasks.workunit.client.0.smithi111.stdout: 10 21651 25.70 MB/sec warmup 52 sec latency 3164.014 ms 2024-09-15T23:00:25.595 INFO:tasks.workunit.client.0.smithi111.stdout: 10 21651 25.22 MB/sec warmup 53 sec latency 4164.063 ms 2024-09-15T23:00:26.081 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:25 smithi111 ceph-mon[30629]: pgmap v785: 97 pgs: 97 active+clean; 512 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.4 MiB/s wr, 343 op/s 2024-09-15T23:00:26.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:25 smithi192 ceph-mon[35427]: pgmap v785: 97 pgs: 97 active+clean; 512 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.4 MiB/s wr, 343 op/s 2024-09-15T23:00:26.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:25 smithi121 ceph-mon[34815]: pgmap v785: 97 pgs: 97 active+clean; 512 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.4 MiB/s wr, 343 op/s 2024-09-15T23:00:26.595 INFO:tasks.workunit.client.0.smithi111.stdout: 10 21729 24.80 MB/sec warmup 54 sec latency 5068.911 ms 2024-09-15T23:00:27.595 INFO:tasks.workunit.client.0.smithi111.stdout: 10 21730 24.35 MB/sec warmup 55 sec latency 6068.967 ms 2024-09-15T23:00:27.997 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:27 smithi111 ceph-mon[30629]: pgmap v786: 97 pgs: 97 active+clean; 520 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.3 MiB/s wr, 305 op/s 2024-09-15T23:00:28.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:27 smithi192 ceph-mon[35427]: pgmap v786: 97 pgs: 97 active+clean; 520 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.3 MiB/s wr, 305 op/s 2024-09-15T23:00:28.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:27 smithi121 ceph-mon[34815]: pgmap v786: 97 pgs: 97 active+clean; 520 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.3 MiB/s wr, 305 op/s 2024-09-15T23:00:28.595 INFO:tasks.workunit.client.0.smithi111.stdout: 10 21731 23.91 MB/sec warmup 56 sec latency 7069.042 ms 2024-09-15T23:00:29.595 INFO:tasks.workunit.client.0.smithi111.stdout: 10 21738 23.51 MB/sec warmup 57 sec latency 7393.181 ms 2024-09-15T23:00:29.965 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:29 smithi111 ceph-mon[30629]: pgmap v787: 97 pgs: 97 active+clean; 517 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.4 MiB/s wr, 315 op/s 2024-09-15T23:00:30.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:29 smithi192 ceph-mon[35427]: pgmap v787: 97 pgs: 97 active+clean; 517 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.4 MiB/s wr, 315 op/s 2024-09-15T23:00:30.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:29 smithi121 ceph-mon[34815]: pgmap v787: 97 pgs: 97 active+clean; 517 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.4 MiB/s wr, 315 op/s 2024-09-15T23:00:30.595 INFO:tasks.workunit.client.0.smithi111.stdout: 10 21738 23.11 MB/sec warmup 58 sec latency 4610.657 ms 2024-09-15T23:00:31.596 INFO:tasks.workunit.client.0.smithi111.stdout: 10 21738 22.72 MB/sec warmup 59 sec latency 5610.712 ms 2024-09-15T23:00:31.980 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:31 smithi111 ceph-mon[30629]: pgmap v788: 97 pgs: 97 active+clean; 505 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.9 MiB/s wr, 300 op/s 2024-09-15T23:00:32.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:31 smithi192 ceph-mon[35427]: pgmap v788: 97 pgs: 97 active+clean; 505 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.9 MiB/s wr, 300 op/s 2024-09-15T23:00:32.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:31 smithi121 ceph-mon[34815]: pgmap v788: 97 pgs: 97 active+clean; 505 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.9 MiB/s wr, 300 op/s 2024-09-15T23:00:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f6750400640 1 -- 172.21.15.111:0/2197004193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6748105910 msgr2=0x7f674810c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f6750400640 1 --2- 172.21.15.111:0/2197004193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6748105910 0x7f674810c1a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6740009ca0 tx=0x7f674002a8c0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f6750400640 1 -- 172.21.15.111:0/2197004193 shutdown_connections 2024-09-15T23:00:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f6750400640 1 --2- 172.21.15.111:0/2197004193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6748105910 0x7f674810c1a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f6750400640 1 --2- 172.21.15.111:0/2197004193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6748104f50 0x7f67481053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f6750400640 1 --2- 172.21.15.111:0/2197004193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6748103d60 0x7f6748104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f6750400640 1 -- 172.21.15.111:0/2197004193 >> 172.21.15.111:0/2197004193 conn(0x7f67480ff4e0 msgr2=0x7f6748101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f6750400640 1 -- 172.21.15.111:0/2197004193 shutdown_connections 2024-09-15T23:00:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f6750400640 1 -- 172.21.15.111:0/2197004193 wait complete. 2024-09-15T23:00:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f6750400640 1 Processor -- start 2024-09-15T23:00:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f6750400640 1 -- start start 2024-09-15T23:00:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f6750400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6748103d60 0x7f6748199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f6750400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6748104f50 0x7f674819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f6750400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6748105910 0x7f67481a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f6750400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f674810fa90 con 0x7f6748104f50 2024-09-15T23:00:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f6750400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f674810f910 con 0x7f6748103d60 2024-09-15T23:00:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f6750400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f674810fc10 con 0x7f6748105910 2024-09-15T23:00:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f674d000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6748104f50 0x7f674819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f674d000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6748104f50 0x7f674819a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34500/0 (socket says 172.21.15.111:34500) 2024-09-15T23:00:32.456 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f674d000640 1 -- 172.21.15.111:0/4166478973 learned_addr learned my addr 172.21.15.111:0/4166478973 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:00:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f674da00640 1 --2- 172.21.15.111:0/4166478973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6748103d60 0x7f6748199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f674e400640 1 --2- 172.21.15.111:0/4166478973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6748105910 0x7f67481a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f674d000640 1 -- 172.21.15.111:0/4166478973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6748105910 msgr2=0x7f67481a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f674d000640 1 --2- 172.21.15.111:0/4166478973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6748105910 0x7f67481a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f674d000640 1 -- 172.21.15.111:0/4166478973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6748103d60 msgr2=0x7f6748199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f674d000640 1 --2- 172.21.15.111:0/4166478973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6748103d60 0x7f6748199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f674d000640 1 -- 172.21.15.111:0/4166478973 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67481a1be0 con 0x7f6748104f50 2024-09-15T23:00:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f674d000640 1 --2- 172.21.15.111:0/4166478973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6748104f50 0x7f674819a500 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f673c00b4a0 tx=0x7f673c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f6746000640 1 -- 172.21.15.111:0/4166478973 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c014070 con 0x7f6748104f50 2024-09-15T23:00:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f6746000640 1 -- 172.21.15.111:0/4166478973 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f673c010af0 con 0x7f6748104f50 2024-09-15T23:00:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f6750400640 1 -- 172.21.15.111:0/4166478973 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67481a1ed0 con 0x7f6748104f50 2024-09-15T23:00:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f6746000640 1 -- 172.21.15.111:0/4166478973 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c017d10 con 0x7f6748104f50 2024-09-15T23:00:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.450+0000 7f6750400640 1 -- 172.21.15.111:0/4166478973 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67481a2410 con 0x7f6748104f50 2024-09-15T23:00:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.456+0000 7f6746000640 1 -- 172.21.15.111:0/4166478973 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f673c017070 con 0x7f6748104f50 2024-09-15T23:00:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.456+0000 7f675344e740 1 Processor -- start 2024-09-15T23:00:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.456+0000 7f675344e740 1 -- start start 2024-09-15T23:00:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.456+0000 7f675344e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55faa65ad090 0x55faa65ad490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.456+0000 7f675344e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa65ad9d0 0x55faa65b2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.456+0000 7f675344e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55faa65b2a60 0x55faa65b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.456+0000 7f675344e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55faa6512100 con 0x55faa65ad9d0 2024-09-15T23:00:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.456+0000 7f675344e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55faa64ba1b0 con 0x55faa65ad090 2024-09-15T23:00:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.456+0000 7f675344e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55faa650e0c0 con 0x55faa65b2a60 2024-09-15T23:00:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.456+0000 7f674da00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa65ad9d0 0x55faa65b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.456+0000 7f674da00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa65ad9d0 0x55faa65b2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34514/0 (socket says 172.21.15.111:34514) 2024-09-15T23:00:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.456+0000 7f674da00640 1 -- 172.21.15.111:0/1758742960 learned_addr learned my addr 172.21.15.111:0/1758742960 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:00:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.456+0000 7f674e400640 1 --2- 172.21.15.111:0/1758742960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55faa65ad090 0x55faa65ad490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.456+0000 7f674d000640 1 --2- 172.21.15.111:0/1758742960 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55faa65b2a60 0x55faa65b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.456+0000 7f674e400640 1 -- 172.21.15.111:0/1758742960 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55faa65b2a60 msgr2=0x55faa65b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.456+0000 7f674e400640 1 --2- 172.21.15.111:0/1758742960 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55faa65b2a60 0x55faa65b4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.456+0000 7f674e400640 1 -- 172.21.15.111:0/1758742960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa65ad9d0 msgr2=0x55faa65b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.456+0000 7f674e400640 1 --2- 172.21.15.111:0/1758742960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa65ad9d0 0x55faa65b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.456+0000 7f674e400640 1 -- 172.21.15.111:0/1758742960 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faa6225210 con 0x55faa65ad090 2024-09-15T23:00:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.456+0000 7f674da00640 1 --2- 172.21.15.111:0/1758742960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa65ad9d0 0x55faa65b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:00:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.456+0000 7f674e400640 1 --2- 172.21.15.111:0/1758742960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55faa65ad090 0x55faa65ad490 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f674002ade0 tx=0x7f6740009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7906fba68eccb604 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.456+0000 7f6746000640 1 --2- 172.21.15.111:0/4166478973 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f67300a32f0 0x7f67300a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.456+0000 7f6746000640 1 -- 172.21.15.111:0/4166478973 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f673c0cec70 con 0x7f6748104f50 2024-09-15T23:00:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.456+0000 7f673aa00640 1 -- 172.21.15.111:0/1758742960 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6740031710 con 0x55faa65ad090 2024-09-15T23:00:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.456+0000 7f673aa00640 1 -- 172.21.15.111:0/1758742960 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6740036030 con 0x55faa65ad090 2024-09-15T23:00:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.456+0000 7f673aa00640 1 -- 172.21.15.111:0/1758742960 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674002f610 con 0x55faa65ad090 2024-09-15T23:00:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.456+0000 7f674da00640 1 --2- 172.21.15.111:0/4166478973 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f67300a32f0 0x7f67300a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.461+0000 7f675344e740 1 -- 172.21.15.111:0/1758742960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55faa65ad090 msgr2=0x55faa65ad490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.461+0000 7f675344e740 1 --2- 172.21.15.111:0/1758742960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55faa65ad090 0x55faa65ad490 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f674002ade0 tx=0x7f6740009800 comp rx=0 tx=0).stop 2024-09-15T23:00:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.461+0000 7f674da00640 1 --2- 172.21.15.111:0/4166478973 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f67300a32f0 0x7f67300a57b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55faa65ae400 tx=0x7f6734005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:32.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.461+0000 7f675344e740 1 -- 172.21.15.111:0/1758742960 shutdown_connections 2024-09-15T23:00:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.461+0000 7f675344e740 1 --2- 172.21.15.111:0/1758742960 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55faa65b2a60 0x55faa65b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.461+0000 7f675344e740 1 --2- 172.21.15.111:0/1758742960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa65ad9d0 0x55faa65b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.461+0000 7f675344e740 1 --2- 172.21.15.111:0/1758742960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55faa65ad090 0x55faa65ad490 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.461+0000 7f675344e740 1 -- 172.21.15.111:0/1758742960 >> 172.21.15.111:0/1758742960 conn(0x55faa65a8700 msgr2=0x55faa65a8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.461+0000 7f675344e740 1 -- 172.21.15.111:0/1758742960 shutdown_connections 2024-09-15T23:00:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.461+0000 7f675344e740 1 -- 172.21.15.111:0/1758742960 wait complete. 2024-09-15T23:00:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.461+0000 7f675344e740 1 -- start start 2024-09-15T23:00:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.461+0000 7f675344e740 1 Processor -- start 2024-09-15T23:00:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55faa65ad090 0x55faa65b7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa65ad9d0 0x55faa65b75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55faa65b2a60 0x55faa65be5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55faa65b5850 con 0x55faa65ad9d0 2024-09-15T23:00:32.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55faa649b090 con 0x55faa65ad090 2024-09-15T23:00:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55faa65b59d0 con 0x55faa65b2a60 2024-09-15T23:00:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 10 client.0 _ll_register_callbacks cb 0x55faa64eff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:00:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f674d000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa65ad9d0 0x55faa65b75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f674e400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55faa65b2a60 0x55faa65be5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f674d000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa65ad9d0 0x55faa65b75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34526/0 (socket says 172.21.15.111:34526) 2024-09-15T23:00:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f674e400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55faa65b2a60 0x55faa65be5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:42080/0 (socket says 172.21.15.111:42080) 2024-09-15T23:00:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f674d000640 1 -- 172.21.15.111:0/1416929431 learned_addr learned my addr 172.21.15.111:0/1416929431 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:00:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f674e400640 1 -- 172.21.15.111:0/1416929431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55faa65ad090 msgr2=0x55faa65b7070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:00:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f674e400640 1 --2- 172.21.15.111:0/1416929431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55faa65ad090 0x55faa65b7070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f674e400640 1 -- 172.21.15.111:0/1416929431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa65ad9d0 msgr2=0x55faa65b75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f674e400640 1 --2- 172.21.15.111:0/1416929431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa65ad9d0 0x55faa65b75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f674e400640 1 -- 172.21.15.111:0/1416929431 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faa62f1620 con 0x55faa65b2a60 2024-09-15T23:00:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f674d000640 1 --2- 172.21.15.111:0/1416929431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa65ad9d0 0x55faa65b75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:00:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f674e400640 1 --2- 172.21.15.111:0/1416929431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55faa65b2a60 0x55faa65be5f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6740031ae0 tx=0x7f67400317c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c3888445613325f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f672f400640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:00:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f672f400640 1 -- 172.21.15.111:0/1416929431 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674004a040 con 0x55faa65b2a60 2024-09-15T23:00:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f672f400640 1 -- 172.21.15.111:0/1416929431 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f674002f810 con 0x55faa65b2a60 2024-09-15T23:00:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 1 -- 172.21.15.111:0/1416929431 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6740036030 con 0x55faa65b2a60 2024-09-15T23:00:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f672f400640 1 -- 172.21.15.111:0/1416929431 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f674002fa30 con 0x55faa65b2a60 2024-09-15T23:00:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 10 client.26140 fetch_fsmap learned FSMap version 31 2024-09-15T23:00:32.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 1 -- 172.21.15.111:0/1416929431 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55faa65b5850 con 0x55faa65b2a60 2024-09-15T23:00:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f672f400640 1 -- 172.21.15.111:0/1416929431 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6740036030 con 0x55faa65b2a60 2024-09-15T23:00:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f672f400640 1 -- 172.21.15.111:0/1416929431 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f674004d030 con 0x55faa65b2a60 2024-09-15T23:00:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 10 client.26140 fetch_fsmap finished waiting for FSMap version 31 2024-09-15T23:00:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 10 client.26140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:00:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 20 client.26140 populate_metadata read hostname 'smithi111' 2024-09-15T23:00:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 1 --2- 172.21.15.111:0/1416929431 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55faa65c2830 0x55faa65c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 4 client.26140 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:00:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 1 -- 172.21.15.111:0/1416929431 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55faa651f830 con 0x55faa65c2830 2024-09-15T23:00:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f674da00640 1 --2- 172.21.15.111:0/1416929431 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55faa65c2830 0x55faa65c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:32.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f674da00640 1 --2- 172.21.15.111:0/1416929431 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55faa65c2830 0x55faa65c4c20 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90bee42085ecf2e9 server_cookie=d2275a2d6288c1b5 in_seq=0 out_seq=0 2024-09-15T23:00:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f672f400640 10 client.26140 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:00:32.475 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f672f400640 1 -- 172.21.15.111:0/1416929431 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55faa651f830 con 0x55faa65c2830 2024-09-15T23:00:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f672f400640 10 client.26140 handle_command_reply: tid=0 2024-09-15T23:00:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 1 client.26140 shutdown 2024-09-15T23:00:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 1 -- 172.21.15.111:0/1416929431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55faa65b2a60 msgr2=0x55faa65be5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 1 --2- 172.21.15.111:0/1416929431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55faa65b2a60 0x55faa65be5f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6740031ae0 tx=0x7f67400317c0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 1 -- 172.21.15.111:0/1416929431 shutdown_connections 2024-09-15T23:00:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 1 --2- 172.21.15.111:0/1416929431 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55faa65c2830 0x55faa65c4c20 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 1 --2- 172.21.15.111:0/1416929431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55faa65b2a60 0x55faa65be5f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 1 --2- 172.21.15.111:0/1416929431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa65ad9d0 0x55faa65b75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f672f400640 0 client.26140 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:00:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 1 --2- 172.21.15.111:0/1416929431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55faa65ad090 0x55faa65b7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.476 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 1 -- 172.21.15.111:0/1416929431 >> 172.21.15.111:0/1416929431 conn(0x55faa65a8700 msgr2=0x55faa65a9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 1 -- 172.21.15.111:0/1416929431 shutdown_connections 2024-09-15T23:00:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 1 -- 172.21.15.111:0/1416929431 wait complete. 2024-09-15T23:00:32.477 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.462+0000 7f675344e740 20 client.26140 trim_cache size 0 max 16384 2024-09-15T23:00:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.480+0000 7f675344e740 1 Processor -- start 2024-09-15T23:00:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.480+0000 7f675344e740 1 -- start start 2024-09-15T23:00:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.480+0000 7f675344e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55faa65ad090 0x55faa65b7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.480+0000 7f675344e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55faa65ad9d0 0x55faa65be5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.480+0000 7f675344e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa65b2a60 0x55faa65b75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.480+0000 7f675344e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f67481ad940 con 0x55faa65b2a60 2024-09-15T23:00:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.480+0000 7f675344e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f67481ab1b0 con 0x55faa65ad9d0 2024-09-15T23:00:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.480+0000 7f675344e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f67481a9000 con 0x55faa65ad090 2024-09-15T23:00:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.480+0000 7f674e400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa65b2a60 0x55faa65b75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.480+0000 7f674da00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55faa65ad090 0x55faa65b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.480+0000 7f674e400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa65b2a60 0x55faa65b75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34536/0 (socket says 172.21.15.111:34536) 2024-09-15T23:00:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.480+0000 7f674e400640 1 -- 172.21.15.111:0/2038268756 learned_addr learned my addr 172.21.15.111:0/2038268756 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:00:32.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.480+0000 7f674da00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55faa65ad090 0x55faa65b7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:42094/0 (socket says 172.21.15.111:42094) 2024-09-15T23:00:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.480+0000 7f674d000640 1 --2- 172.21.15.111:0/2038268756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55faa65ad9d0 0x55faa65be5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.480+0000 7f674da00640 1 -- 172.21.15.111:0/2038268756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55faa65ad9d0 msgr2=0x55faa65be5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.480+0000 7f674da00640 1 --2- 172.21.15.111:0/2038268756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55faa65ad9d0 0x55faa65be5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.480+0000 7f674da00640 1 -- 172.21.15.111:0/2038268756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa65b2a60 msgr2=0x55faa65b75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.480+0000 7f674da00640 1 --2- 172.21.15.111:0/2038268756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa65b2a60 0x55faa65b75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.480+0000 7f674da00640 1 -- 172.21.15.111:0/2038268756 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faa65a8520 con 0x55faa65ad090 2024-09-15T23:00:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.480+0000 7f674e400640 1 --2- 172.21.15.111:0/2038268756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa65b2a60 0x55faa65b75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:00:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.480+0000 7f674da00640 1 --2- 172.21.15.111:0/2038268756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55faa65ad090 0x55faa65b7070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x55faa65c5430 tx=0x7f673401a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61e2e1cee5239dd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.480+0000 7f673a000640 1 -- 172.21.15.111:0/2038268756 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67340210d0 con 0x55faa65ad090 2024-09-15T23:00:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.480+0000 7f673a000640 1 -- 172.21.15.111:0/2038268756 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6734021270 con 0x55faa65ad090 2024-09-15T23:00:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.480+0000 7f673a000640 1 -- 172.21.15.111:0/2038268756 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67340049c0 con 0x55faa65ad090 2024-09-15T23:00:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.485+0000 7f675344e740 1 -- 172.21.15.111:0/2038268756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55faa65ad090 msgr2=0x55faa65b7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.485+0000 7f675344e740 1 --2- 172.21.15.111:0/2038268756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55faa65ad090 0x55faa65b7070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x55faa65c5430 tx=0x7f673401a000 comp rx=0 tx=0).stop 2024-09-15T23:00:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.485+0000 7f675344e740 1 -- 172.21.15.111:0/2038268756 shutdown_connections 2024-09-15T23:00:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.485+0000 7f675344e740 1 --2- 172.21.15.111:0/2038268756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa65b2a60 0x55faa65b75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.485+0000 7f675344e740 1 --2- 172.21.15.111:0/2038268756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55faa65ad9d0 0x55faa65be5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.485+0000 7f675344e740 1 --2- 172.21.15.111:0/2038268756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55faa65ad090 0x55faa65b7070 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.485+0000 7f675344e740 1 -- 172.21.15.111:0/2038268756 >> 172.21.15.111:0/2038268756 conn(0x55faa65a8700 msgr2=0x55faa65b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.485+0000 7f675344e740 1 -- 172.21.15.111:0/2038268756 shutdown_connections 2024-09-15T23:00:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.485+0000 7f675344e740 1 -- 172.21.15.111:0/2038268756 wait complete. 2024-09-15T23:00:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.485+0000 7f675344e740 1 -- start start 2024-09-15T23:00:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.485+0000 7f675344e740 1 Processor -- start 2024-09-15T23:00:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f675344e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55faa65ad090 0x55faa65bb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f675344e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa65ad9d0 0x55faa65bbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f675344e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55faa65b2a60 0x55faa65bccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f675344e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f67481a9670 con 0x55faa65ad9d0 2024-09-15T23:00:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f675344e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f67481aaae0 con 0x55faa65ad090 2024-09-15T23:00:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f675344e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f67481a9d40 con 0x55faa65b2a60 2024-09-15T23:00:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f675344e740 10 client.0 _ll_register_callbacks cb 0x7f67340096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:00:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f674da00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55faa65ad090 0x55faa65bb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f674da00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55faa65ad090 0x55faa65bb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:58892/0 (socket says 172.21.15.111:58892) 2024-09-15T23:00:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f674da00640 1 -- 172.21.15.111:0/1639324028 learned_addr learned my addr 172.21.15.111:0/1639324028 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:00:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f674d000640 1 --2- 172.21.15.111:0/1639324028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa65ad9d0 0x55faa65bbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f674e400640 1 --2- 172.21.15.111:0/1639324028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55faa65b2a60 0x55faa65bccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f674da00640 1 -- 172.21.15.111:0/1639324028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55faa65b2a60 msgr2=0x55faa65bccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f674da00640 1 --2- 172.21.15.111:0/1639324028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55faa65b2a60 0x55faa65bccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f674da00640 1 -- 172.21.15.111:0/1639324028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa65ad9d0 msgr2=0x55faa65bbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f674da00640 1 --2- 172.21.15.111:0/1639324028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa65ad9d0 0x55faa65bbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f674da00640 1 -- 172.21.15.111:0/1639324028 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faa65bd2c0 con 0x55faa65ad090 2024-09-15T23:00:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f674d000640 1 --2- 172.21.15.111:0/1639324028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa65ad9d0 0x55faa65bbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:00:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f674da00640 1 --2- 172.21.15.111:0/1639324028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55faa65ad090 0x55faa65bb760 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6734019040 tx=0x7f67340211d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69e1449c613b9f97 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f672f400640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:00:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f672f400640 1 -- 172.21.15.111:0/1639324028 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6734038070 con 0x55faa65ad090 2024-09-15T23:00:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f672f400640 1 -- 172.21.15.111:0/1639324028 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f6734021ce0 con 0x55faa65ad090 2024-09-15T23:00:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f675344e740 1 -- 172.21.15.111:0/1639324028 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6734021270 con 0x55faa65ad090 2024-09-15T23:00:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f672f400640 1 -- 172.21.15.111:0/1639324028 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6734044050 con 0x55faa65ad090 2024-09-15T23:00:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f675344e740 10 client.26126 fetch_fsmap learned FSMap version 31 2024-09-15T23:00:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f675344e740 1 -- 172.21.15.111:0/1639324028 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55faa65be5d0 con 0x55faa65ad090 2024-09-15T23:00:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f672f400640 1 -- 172.21.15.111:0/1639324028 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6734021270 con 0x55faa65ad090 2024-09-15T23:00:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f672f400640 1 -- 172.21.15.111:0/1639324028 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f673403d670 con 0x55faa65ad090 2024-09-15T23:00:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f675344e740 10 client.26126 fetch_fsmap finished waiting for FSMap version 31 2024-09-15T23:00:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f675344e740 10 client.26126 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:00:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f675344e740 20 client.26126 populate_metadata read hostname 'smithi111' 2024-09-15T23:00:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f675344e740 1 --2- 172.21.15.111:0/1639324028 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55faa662d950 0x55faa662fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f675344e740 4 client.26126 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:00:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.486+0000 7f675344e740 1 -- 172.21.15.111:0/1639324028 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55faa6519b80 con 0x55faa662d950 2024-09-15T23:00:32.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.509+0000 7f674d000640 1 --2- 172.21.15.111:0/1639324028 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55faa662d950 0x55faa662fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.556+0000 7f672f400640 10 client.26126 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:00:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.556+0000 7f674d000640 1 --2- 172.21.15.111:0/1639324028 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55faa662d950 0x55faa662fd70 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80436da3149a693a server_cookie=dcd9f67ae025a395 in_seq=0 out_seq=0 2024-09-15T23:00:32.621 INFO:tasks.workunit.client.0.smithi111.stdout: 10 21743 22.33 MB/sec warmup 60 sec latency 6377.898 ms 2024-09-15T23:00:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.663+0000 7f672f400640 1 -- 172.21.15.111:0/1639324028 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55faa6519b80 con 0x55faa662d950 2024-09-15T23:00:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.663+0000 7f672f400640 10 client.26126 handle_command_reply: tid=0 2024-09-15T23:00:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.663+0000 7f675344e740 1 client.26126 shutdown 2024-09-15T23:00:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.663+0000 7f675344e740 1 -- 172.21.15.111:0/1639324028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55faa65ad090 msgr2=0x55faa65bb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.663+0000 7f675344e740 1 --2- 172.21.15.111:0/1639324028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55faa65ad090 0x55faa65bb760 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6734019040 tx=0x7f67340211d0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.663+0000 7f675344e740 1 -- 172.21.15.111:0/1639324028 shutdown_connections 2024-09-15T23:00:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.663+0000 7f675344e740 1 --2- 172.21.15.111:0/1639324028 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55faa662d950 0x55faa662fd70 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.665 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.663+0000 7f675344e740 1 --2- 172.21.15.111:0/1639324028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55faa65b2a60 0x55faa65bccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.663+0000 7f672f400640 0 client.26126 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:00:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.663+0000 7f675344e740 1 --2- 172.21.15.111:0/1639324028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55faa65ad9d0 0x55faa65bbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.663+0000 7f675344e740 1 --2- 172.21.15.111:0/1639324028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55faa65ad090 0x55faa65bb760 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.663+0000 7f675344e740 1 -- 172.21.15.111:0/1639324028 >> 172.21.15.111:0/1639324028 conn(0x55faa65a8700 msgr2=0x55faa65b7010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.663+0000 7f675344e740 1 -- 172.21.15.111:0/1639324028 shutdown_connections 2024-09-15T23:00:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.663+0000 7f675344e740 1 -- 172.21.15.111:0/1639324028 wait complete. 2024-09-15T23:00:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.663+0000 7f675344e740 20 client.26126 trim_cache size 0 max 16384 2024-09-15T23:00:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:00:32.666 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:00:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:00:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:00:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.663+0000 7f6750400640 1 -- 172.21.15.111:0/4166478973 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f67300a32f0 msgr2=0x7f67300a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:32.667 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.663+0000 7f6750400640 1 --2- 172.21.15.111:0/4166478973 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f67300a32f0 0x7f67300a57b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55faa65ae400 tx=0x7f6734005f90 comp rx=0 tx=0).stop 2024-09-15T23:00:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.663+0000 7f6750400640 1 -- 172.21.15.111:0/4166478973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6748104f50 msgr2=0x7f674819a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.663+0000 7f6750400640 1 --2- 172.21.15.111:0/4166478973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6748104f50 0x7f674819a500 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f673c00b4a0 tx=0x7f673c00b960 comp rx=0 tx=0).stop 2024-09-15T23:00:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.663+0000 7f6750400640 1 -- 172.21.15.111:0/4166478973 shutdown_connections 2024-09-15T23:00:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.663+0000 7f6750400640 1 --2- 172.21.15.111:0/4166478973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6748105910 0x7f67481a1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.663+0000 7f6750400640 1 --2- 172.21.15.111:0/4166478973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6748104f50 0x7f674819a500 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.663+0000 7f6750400640 1 --2- 172.21.15.111:0/4166478973 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f67300a32f0 0x7f67300a57b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.663+0000 7f6750400640 1 --2- 172.21.15.111:0/4166478973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6748103d60 0x7f6748199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.663+0000 7f6750400640 1 -- 172.21.15.111:0/4166478973 >> 172.21.15.111:0/4166478973 conn(0x7f67480ff4e0 msgr2=0x7f6748108180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.663+0000 7f6750400640 1 -- 172.21.15.111:0/4166478973 shutdown_connections 2024-09-15T23:00:32.668 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:32.663+0000 7f6750400640 1 -- 172.21.15.111:0/4166478973 wait complete. 2024-09-15T23:00:33.619 INFO:tasks.workunit.client.0.smithi111.stdout: 10 21743 21.96 MB/sec warmup 61 sec latency 4399.135 ms 2024-09-15T23:00:33.984 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:33 smithi111 ceph-mon[30629]: pgmap v789: 97 pgs: 97 active+clean; 505 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.0 MiB/s wr, 264 op/s 2024-09-15T23:00:34.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:33 smithi192 ceph-mon[35427]: pgmap v789: 97 pgs: 97 active+clean; 505 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.0 MiB/s wr, 264 op/s 2024-09-15T23:00:34.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:33 smithi121 ceph-mon[34815]: pgmap v789: 97 pgs: 97 active+clean; 505 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.0 MiB/s wr, 264 op/s 2024-09-15T23:00:34.620 INFO:tasks.workunit.client.0.smithi111.stdout: 10 21783 21.63 MB/sec warmup 62 sec latency 5399.178 ms 2024-09-15T23:00:35.619 INFO:tasks.workunit.client.0.smithi111.stdout: 10 21831 21.38 MB/sec warmup 63 sec latency 6399.230 ms 2024-09-15T23:00:36.001 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:35 smithi111 ceph-mon[30629]: pgmap v790: 97 pgs: 97 active+clean; 498 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 4.6 MiB/s wr, 227 op/s 2024-09-15T23:00:36.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:35 smithi192 ceph-mon[35427]: pgmap v790: 97 pgs: 97 active+clean; 498 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 4.6 MiB/s wr, 227 op/s 2024-09-15T23:00:36.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:35 smithi121 ceph-mon[34815]: pgmap v790: 97 pgs: 97 active+clean; 498 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 4.6 MiB/s wr, 227 op/s 2024-09-15T23:00:36.621 INFO:tasks.workunit.client.0.smithi111.stdout: 10 22448 21.56 MB/sec warmup 64 sec latency 6805.027 ms 2024-09-15T23:00:36.995 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:36 smithi111 ceph-mon[30629]: pgmap v791: 97 pgs: 97 active+clean; 479 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 4.3 MiB/s wr, 201 op/s 2024-09-15T23:00:37.291 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:36 smithi121 ceph-mon[34815]: pgmap v791: 97 pgs: 97 active+clean; 479 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 4.3 MiB/s wr, 201 op/s 2024-09-15T23:00:37.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:36 smithi192 ceph-mon[35427]: pgmap v791: 97 pgs: 97 active+clean; 479 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 362 KiB/s rd, 4.3 MiB/s wr, 201 op/s 2024-09-15T23:00:37.619 INFO:tasks.workunit.client.0.smithi111.stdout: 10 22558 21.38 MB/sec warmup 65 sec latency 928.067 ms 2024-09-15T23:00:38.619 INFO:tasks.workunit.client.0.smithi111.stdout: 10 22561 21.06 MB/sec warmup 66 sec latency 1872.862 ms 2024-09-15T23:00:39.619 INFO:tasks.workunit.client.0.smithi111.stdout: 10 22561 20.74 MB/sec warmup 67 sec latency 2730.676 ms 2024-09-15T23:00:40.112 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:39 smithi111 ceph-mon[30629]: pgmap v792: 97 pgs: 97 active+clean; 468 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 4.5 MiB/s wr, 246 op/s 2024-09-15T23:00:40.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:39 smithi121 ceph-mon[34815]: pgmap v792: 97 pgs: 97 active+clean; 468 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 4.5 MiB/s wr, 246 op/s 2024-09-15T23:00:40.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:39 smithi192 ceph-mon[35427]: pgmap v792: 97 pgs: 97 active+clean; 468 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 4.5 MiB/s wr, 246 op/s 2024-09-15T23:00:40.619 INFO:tasks.workunit.client.0.smithi111.stdout: 10 22561 20.44 MB/sec warmup 68 sec latency 3730.760 ms 2024-09-15T23:00:41.619 INFO:tasks.workunit.client.0.smithi111.stdout: 10 22561 20.14 MB/sec warmup 69 sec latency 4730.807 ms 2024-09-15T23:00:42.153 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:41 smithi111 ceph-mon[30629]: pgmap v793: 97 pgs: 97 active+clean; 467 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 4.7 MiB/s wr, 258 op/s 2024-09-15T23:00:42.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:41 smithi192 ceph-mon[35427]: pgmap v793: 97 pgs: 97 active+clean; 467 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 4.7 MiB/s wr, 258 op/s 2024-09-15T23:00:42.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:41 smithi121 ceph-mon[34815]: pgmap v793: 97 pgs: 97 active+clean; 467 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 960 KiB/s rd, 4.7 MiB/s wr, 258 op/s 2024-09-15T23:00:42.619 INFO:tasks.workunit.client.0.smithi111.stdout: 10 22743 20.08 MB/sec warmup 70 sec latency 4818.391 ms 2024-09-15T23:00:43.619 INFO:tasks.workunit.client.0.smithi111.stdout: 10 22743 19.80 MB/sec warmup 71 sec latency 1800.030 ms 2024-09-15T23:00:44.132 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:43 smithi111 ceph-mon[30629]: pgmap v794: 97 pgs: 97 active+clean; 466 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 953 KiB/s rd, 5.3 MiB/s wr, 257 op/s 2024-09-15T23:00:44.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:43 smithi192 ceph-mon[35427]: pgmap v794: 97 pgs: 97 active+clean; 466 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 953 KiB/s rd, 5.3 MiB/s wr, 257 op/s 2024-09-15T23:00:44.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:43 smithi121 ceph-mon[34815]: pgmap v794: 97 pgs: 97 active+clean; 466 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 953 KiB/s rd, 5.3 MiB/s wr, 257 op/s 2024-09-15T23:00:44.620 INFO:tasks.workunit.client.0.smithi111.stdout: 10 22743 19.52 MB/sec warmup 72 sec latency 2800.087 ms 2024-09-15T23:00:45.624 INFO:tasks.workunit.client.0.smithi111.stdout: 10 22810 19.28 MB/sec warmup 73 sec latency 3739.259 ms 2024-09-15T23:00:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6826a00640 1 -- 172.21.15.111:0/2430292462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6820103b70 msgr2=0x7f6820103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6826a00640 1 --2- 172.21.15.111:0/2430292462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6820103b70 0x7f6820103f70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6814009900 tx=0x7f681402a410 comp rx=0 tx=0).stop 2024-09-15T23:00:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6826a00640 1 -- 172.21.15.111:0/2430292462 shutdown_connections 2024-09-15T23:00:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6826a00640 1 --2- 172.21.15.111:0/2430292462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6820105720 0x7f682010bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6826a00640 1 --2- 172.21.15.111:0/2430292462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6820104d60 0x7f68201051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6826a00640 1 --2- 172.21.15.111:0/2430292462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6820103b70 0x7f6820103f70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6826a00640 1 -- 172.21.15.111:0/2430292462 >> 172.21.15.111:0/2430292462 conn(0x7f68200ff2f0 msgr2=0x7f6820101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:45.799 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6826a00640 1 -- 172.21.15.111:0/2430292462 shutdown_connections 2024-09-15T23:00:45.800 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6826a00640 1 -- 172.21.15.111:0/2430292462 wait complete. 2024-09-15T23:00:45.801 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6826a00640 1 Processor -- start 2024-09-15T23:00:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6826a00640 1 -- start start 2024-09-15T23:00:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6826a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6820103b70 0x7f6820199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6826a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6820104d60 0x7f682019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6826a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6820105720 0x7f68201a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6826a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f682010f840 con 0x7f6820104d60 2024-09-15T23:00:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6826a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f682010f6c0 con 0x7f6820103b70 2024-09-15T23:00:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6826a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f682010f9c0 con 0x7f6820105720 2024-09-15T23:00:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6825600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6820103b70 0x7f6820199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6825600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6820103b70 0x7f6820199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:36164/0 (socket says 172.21.15.111:36164) 2024-09-15T23:00:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6825600640 1 -- 172.21.15.111:0/2626199922 learned_addr learned my addr 172.21.15.111:0/2626199922 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:00:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6826000640 1 --2- 172.21.15.111:0/2626199922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6820105720 0x7f68201a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6824c00640 1 --2- 172.21.15.111:0/2626199922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6820104d60 0x7f682019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6825600640 1 -- 172.21.15.111:0/2626199922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6820105720 msgr2=0x7f68201a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6825600640 1 --2- 172.21.15.111:0/2626199922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6820105720 0x7f68201a1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6825600640 1 -- 172.21.15.111:0/2626199922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6820104d60 msgr2=0x7f682019a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6825600640 1 --2- 172.21.15.111:0/2626199922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6820104d60 0x7f682019a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6825600640 1 -- 172.21.15.111:0/2626199922 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68201a1ae0 con 0x7f6820103b70 2024-09-15T23:00:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6826000640 1 --2- 172.21.15.111:0/2626199922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6820105720 0x7f68201a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:00:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6825600640 1 --2- 172.21.15.111:0/2626199922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6820103b70 0x7f6820199ec0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6814031440 tx=0x7f6814032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f681e000640 1 -- 172.21.15.111:0/2626199922 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681403f030 con 0x7f6820103b70 2024-09-15T23:00:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f681e000640 1 -- 172.21.15.111:0/2626199922 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6814003ce0 con 0x7f6820103b70 2024-09-15T23:00:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f681e000640 1 -- 172.21.15.111:0/2626199922 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6814035070 con 0x7f6820103b70 2024-09-15T23:00:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6826a00640 1 -- 172.21.15.111:0/2626199922 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68201a1d70 con 0x7f6820103b70 2024-09-15T23:00:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.797+0000 7f6826a00640 1 -- 172.21.15.111:0/2626199922 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68201a21b0 con 0x7f6820103b70 2024-09-15T23:00:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f682aff1740 1 Processor -- start 2024-09-15T23:00:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f682aff1740 1 -- start start 2024-09-15T23:00:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f682aff1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b3461a260 0x556b3461a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:45.807 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f682aff1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b3461aba0 0x556b3461f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f681e000640 1 -- 172.21.15.111:0/2626199922 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f6814042030 con 0x7f6820103b70 2024-09-15T23:00:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f682aff1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b3461fc30 0x556b346220f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f682aff1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556b3457f100 con 0x556b3461a260 2024-09-15T23:00:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f682aff1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556b345271b0 con 0x556b3461fc30 2024-09-15T23:00:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f682aff1740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556b3457b0c0 con 0x556b3461aba0 2024-09-15T23:00:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f6826000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b3461a260 0x556b3461a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f6826000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b3461a260 0x556b3461a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52008/0 (socket says 172.21.15.111:52008) 2024-09-15T23:00:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f6826000640 1 -- 172.21.15.111:0/4029913406 learned_addr learned my addr 172.21.15.111:0/4029913406 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:00:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f6825600640 1 --2- 172.21.15.111:0/4029913406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b3461aba0 0x556b3461f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f6824c00640 1 --2- 172.21.15.111:0/4029913406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b3461fc30 0x556b346220f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f6826000640 1 -- 172.21.15.111:0/4029913406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b3461aba0 msgr2=0x556b3461f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f6826000640 1 --2- 172.21.15.111:0/4029913406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b3461aba0 0x556b3461f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f6826000640 1 -- 172.21.15.111:0/4029913406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b3461fc30 msgr2=0x556b346220f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f6826000640 1 --2- 172.21.15.111:0/4029913406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b3461fc30 0x556b346220f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f6826000640 1 -- 172.21.15.111:0/4029913406 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b34292210 con 0x556b3461a260 2024-09-15T23:00:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f6825600640 1 --2- 172.21.15.111:0/4029913406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b3461aba0 0x556b3461f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:00:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f6824c00640 1 --2- 172.21.15.111:0/4029913406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b3461fc30 0x556b346220f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:00:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f6826000640 1 --2- 172.21.15.111:0/4029913406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b3461a260 0x556b3461a660 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6818017f20 tx=0x7f681800e840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9938c0aa552e7ba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f6812600640 1 -- 172.21.15.111:0/4029913406 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818003a30 con 0x556b3461a260 2024-09-15T23:00:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f6812600640 1 -- 172.21.15.111:0/4029913406 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6818003bd0 con 0x556b3461a260 2024-09-15T23:00:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f6812600640 1 -- 172.21.15.111:0/4029913406 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68180207c0 con 0x556b3461a260 2024-09-15T23:00:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f681e000640 1 --2- 172.21.15.111:0/2626199922 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f68080a3390 0x7f68080a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f681e000640 1 -- 172.21.15.111:0/2626199922 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f68140efd70 con 0x7f6820103b70 2024-09-15T23:00:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f682aff1740 1 -- 172.21.15.111:0/4029913406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b3461a260 msgr2=0x556b3461a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f682aff1740 1 --2- 172.21.15.111:0/4029913406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b3461a260 0x556b3461a660 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6818017f20 tx=0x7f681800e840 comp rx=0 tx=0).stop 2024-09-15T23:00:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f6825600640 1 --2- 172.21.15.111:0/2626199922 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f68080a3390 0x7f68080a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f682aff1740 1 -- 172.21.15.111:0/4029913406 shutdown_connections 2024-09-15T23:00:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f682aff1740 1 --2- 172.21.15.111:0/4029913406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b3461fc30 0x556b346220f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f682aff1740 1 --2- 172.21.15.111:0/4029913406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b3461aba0 0x556b3461f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f682aff1740 1 --2- 172.21.15.111:0/4029913406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b3461a260 0x556b3461a660 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f682aff1740 1 -- 172.21.15.111:0/4029913406 >> 172.21.15.111:0/4029913406 conn(0x556b346158d0 msgr2=0x556b34615cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f682aff1740 1 -- 172.21.15.111:0/4029913406 shutdown_connections 2024-09-15T23:00:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f682aff1740 1 -- 172.21.15.111:0/4029913406 wait complete. 2024-09-15T23:00:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f6825600640 1 --2- 172.21.15.111:0/2626199922 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f68080a3390 0x7f68080a5850 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f68140b8870 tx=0x7f68140b8040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f682aff1740 1 -- start start 2024-09-15T23:00:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f682aff1740 1 Processor -- start 2024-09-15T23:00:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f682aff1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b3461a260 0x556b346244f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.804+0000 7f682aff1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b3461aba0 0x556b34624a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f682aff1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b3461fc30 0x556b3462ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f682aff1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556b34622c50 con 0x556b3461fc30 2024-09-15T23:00:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f682aff1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556b34508090 con 0x556b3461a260 2024-09-15T23:00:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f682aff1740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556b34622dd0 con 0x556b3461aba0 2024-09-15T23:00:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f682aff1740 10 client.0 _ll_register_callbacks cb 0x556b3455cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:00:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f6826000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b3461a260 0x556b346244f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f6826000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b3461a260 0x556b346244f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:36168/0 (socket says 172.21.15.111:36168) 2024-09-15T23:00:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f6826000640 1 -- 172.21.15.111:0/1560326783 learned_addr learned my addr 172.21.15.111:0/1560326783 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:00:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f6826000640 1 --2- 172.21.15.111:0/1560326783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b3461fc30 0x556b3462ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f6826000640 1 -- 172.21.15.111:0/1560326783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b3461aba0 msgr2=0x556b34624a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:00:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f6826000640 1 --2- 172.21.15.111:0/1560326783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b3461aba0 0x556b34624a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f6826000640 1 -- 172.21.15.111:0/1560326783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b3461fc30 msgr2=0x556b3462ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f6826000640 1 --2- 172.21.15.111:0/1560326783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b3461fc30 0x556b3462ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f6826000640 1 -- 172.21.15.111:0/1560326783 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b3435e620 con 0x556b3461a260 2024-09-15T23:00:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f6826000640 1 --2- 172.21.15.111:0/1560326783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b3461a260 0x556b346244f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6818004050 tx=0x7f6818018fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e31c64892dabaab server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f6806a00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:00:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f6806a00640 1 -- 172.21.15.111:0/1560326783 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818035060 con 0x556b3461a260 2024-09-15T23:00:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f682aff1740 1 -- 172.21.15.111:0/1560326783 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6818003bd0 con 0x556b3461a260 2024-09-15T23:00:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f6806a00640 1 -- 172.21.15.111:0/1560326783 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f6818020580 con 0x556b3461a260 2024-09-15T23:00:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f6806a00640 1 -- 172.21.15.111:0/1560326783 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f68180207c0 con 0x556b3461a260 2024-09-15T23:00:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f682aff1740 10 client.26138 fetch_fsmap learned FSMap version 31 2024-09-15T23:00:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f682aff1740 1 -- 172.21.15.111:0/1560326783 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556b3462e8b0 con 0x556b3461a260 2024-09-15T23:00:45.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f6806a00640 1 -- 172.21.15.111:0/1560326783 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6818003bd0 con 0x556b3461a260 2024-09-15T23:00:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f6806a00640 1 -- 172.21.15.111:0/1560326783 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f6818021960 con 0x556b3461a260 2024-09-15T23:00:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f682aff1740 10 client.26138 fetch_fsmap finished waiting for FSMap version 31 2024-09-15T23:00:45.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f682aff1740 10 client.26138 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:00:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f682aff1740 20 client.26138 populate_metadata read hostname 'smithi111' 2024-09-15T23:00:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f682aff1740 1 --2- 172.21.15.111:0/1560326783 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556b3462fd20 0x556b34632110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f682aff1740 4 client.26138 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:00:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f682aff1740 1 -- 172.21.15.111:0/1560326783 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b3458c830 con 0x556b3462fd20 2024-09-15T23:00:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f6824c00640 1 --2- 172.21.15.111:0/1560326783 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556b3462fd20 0x556b34632110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f6824c00640 1 --2- 172.21.15.111:0/1560326783 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556b3462fd20 0x556b34632110 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7520db8dac1203e server_cookie=9a0e400052e12b36 in_seq=0 out_seq=0 2024-09-15T23:00:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.810+0000 7f6806a00640 10 client.26138 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:00:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.815+0000 7f6806a00640 1 -- 172.21.15.111:0/1560326783 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b3458c830 con 0x556b3462fd20 2024-09-15T23:00:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.815+0000 7f6806a00640 10 client.26138 handle_command_reply: tid=0 2024-09-15T23:00:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.815+0000 7f682aff1740 1 client.26138 shutdown 2024-09-15T23:00:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.816+0000 7f682aff1740 1 -- 172.21.15.111:0/1560326783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b3461a260 msgr2=0x556b346244f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.816+0000 7f682aff1740 1 --2- 172.21.15.111:0/1560326783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b3461a260 0x556b346244f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6818004050 tx=0x7f6818018fc0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.816+0000 7f682aff1740 1 -- 172.21.15.111:0/1560326783 shutdown_connections 2024-09-15T23:00:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.816+0000 7f682aff1740 1 --2- 172.21.15.111:0/1560326783 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556b3462fd20 0x556b34632110 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.816+0000 7f682aff1740 1 --2- 172.21.15.111:0/1560326783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b3461fc30 0x556b3462ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.816+0000 7f682aff1740 1 --2- 172.21.15.111:0/1560326783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b3461aba0 0x556b34624a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.816+0000 7f682aff1740 1 --2- 172.21.15.111:0/1560326783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b3461a260 0x556b346244f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.816+0000 7f682aff1740 1 -- 172.21.15.111:0/1560326783 >> 172.21.15.111:0/1560326783 conn(0x556b346158d0 msgr2=0x556b346170e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.816+0000 7f6806a00640 0 client.26138 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:00:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.816+0000 7f682aff1740 1 -- 172.21.15.111:0/1560326783 shutdown_connections 2024-09-15T23:00:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.816+0000 7f682aff1740 1 -- 172.21.15.111:0/1560326783 wait complete. 2024-09-15T23:00:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.816+0000 7f682aff1740 20 client.26138 trim_cache size 0 max 16384 2024-09-15T23:00:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.834+0000 7f682aff1740 1 Processor -- start 2024-09-15T23:00:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.834+0000 7f682aff1740 1 -- start start 2024-09-15T23:00:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.834+0000 7f682aff1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b346158d0 0x556b34616780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.834+0000 7f682aff1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b3461a260 0x556b3462b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.834+0000 7f682aff1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b3461aba0 0x556b34616cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.834+0000 7f682aff1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f68201a93b0 con 0x556b3461aba0 2024-09-15T23:00:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.834+0000 7f682aff1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f68201a6c20 con 0x556b3461a260 2024-09-15T23:00:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.834+0000 7f682aff1740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f68201a4ab0 con 0x556b346158d0 2024-09-15T23:00:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.834+0000 7f6826000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b346158d0 0x556b34616780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.834+0000 7f6824c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b3461a260 0x556b3462b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.834+0000 7f6826000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b346158d0 0x556b34616780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:45138/0 (socket says 172.21.15.111:45138) 2024-09-15T23:00:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.834+0000 7f6826000640 1 -- 172.21.15.111:0/2235155814 learned_addr learned my addr 172.21.15.111:0/2235155814 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:00:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.834+0000 7f6826000640 1 --2- 172.21.15.111:0/2235155814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b3461aba0 0x556b34616cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.834+0000 7f6824c00640 1 -- 172.21.15.111:0/2235155814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b346158d0 msgr2=0x556b34616780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.834+0000 7f6824c00640 1 --2- 172.21.15.111:0/2235155814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b346158d0 0x556b34616780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.834+0000 7f6824c00640 1 -- 172.21.15.111:0/2235155814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b3461aba0 msgr2=0x556b34616cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.834+0000 7f6824c00640 1 --2- 172.21.15.111:0/2235155814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b3461aba0 0x556b34616cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.834+0000 7f6824c00640 1 -- 172.21.15.111:0/2235155814 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b34622020 con 0x556b3461a260 2024-09-15T23:00:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f6824c00640 1 --2- 172.21.15.111:0/2235155814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b3461a260 0x556b3462b9e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x556b34632920 tx=0x7f680c00d580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53d3cbd31c874093 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f6811c00640 1 -- 172.21.15.111:0/2235155814 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c01e020 con 0x556b3461a260 2024-09-15T23:00:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f6811c00640 1 -- 172.21.15.111:0/2235155814 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f680c003d80 con 0x556b3461a260 2024-09-15T23:00:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f6811c00640 1 -- 172.21.15.111:0/2235155814 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c011800 con 0x556b3461a260 2024-09-15T23:00:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f682aff1740 1 -- 172.21.15.111:0/2235155814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b3461a260 msgr2=0x556b3462b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f682aff1740 1 --2- 172.21.15.111:0/2235155814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b3461a260 0x556b3462b9e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x556b34632920 tx=0x7f680c00d580 comp rx=0 tx=0).stop 2024-09-15T23:00:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f682aff1740 1 -- 172.21.15.111:0/2235155814 shutdown_connections 2024-09-15T23:00:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f682aff1740 1 --2- 172.21.15.111:0/2235155814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b3461aba0 0x556b34616cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f682aff1740 1 --2- 172.21.15.111:0/2235155814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b3461a260 0x556b3462b9e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f682aff1740 1 --2- 172.21.15.111:0/2235155814 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b346158d0 0x556b34616780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f682aff1740 1 -- 172.21.15.111:0/2235155814 >> 172.21.15.111:0/2235155814 conn(0x556b3461f7c0 msgr2=0x556b346242a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f682aff1740 1 -- 172.21.15.111:0/2235155814 shutdown_connections 2024-09-15T23:00:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f682aff1740 1 -- 172.21.15.111:0/2235155814 wait complete. 2024-09-15T23:00:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f682aff1740 1 -- start start 2024-09-15T23:00:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f682aff1740 1 Processor -- start 2024-09-15T23:00:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f682aff1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b346158d0 0x556b346290c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f682aff1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b3461a260 0x556b34629600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f682aff1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b3461aba0 0x556b3462a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f682aff1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f68201a5100 con 0x556b3461a260 2024-09-15T23:00:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f682aff1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f68201a6550 con 0x556b346158d0 2024-09-15T23:00:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f682aff1740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f68201a57b0 con 0x556b3461aba0 2024-09-15T23:00:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f682aff1740 10 client.0 _ll_register_callbacks cb 0x7f680c00d420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:00:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f6826000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b346158d0 0x556b346290c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f6826000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b346158d0 0x556b346290c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:36178/0 (socket says 172.21.15.111:36178) 2024-09-15T23:00:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f6826000640 1 -- 172.21.15.111:0/1560696183 learned_addr learned my addr 172.21.15.111:0/1560696183 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:00:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f6824c00640 1 --2- 172.21.15.111:0/1560696183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b3461a260 0x556b34629600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f6826000640 1 --2- 172.21.15.111:0/1560696183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b3461aba0 0x556b3462a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f6826000640 1 -- 172.21.15.111:0/1560696183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b3461aba0 msgr2=0x556b3462a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f6826000640 1 --2- 172.21.15.111:0/1560696183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b3461aba0 0x556b3462a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f6826000640 1 -- 172.21.15.111:0/1560696183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b3461a260 msgr2=0x556b34629600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f6826000640 1 --2- 172.21.15.111:0/1560696183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b3461a260 0x556b34629600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f6826000640 1 -- 172.21.15.111:0/1560696183 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b3462ac20 con 0x556b346158d0 2024-09-15T23:00:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f6826000640 1 --2- 172.21.15.111:0/1560696183 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b346158d0 0x556b346290c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6818021d80 tx=0x7f6818005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67db2002a6ba3aff server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f6806a00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:00:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f6806a00640 1 -- 172.21.15.111:0/1560696183 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681803f040 con 0x556b346158d0 2024-09-15T23:00:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f6806a00640 1 -- 172.21.15.111:0/1560696183 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f681801d040 con 0x556b346158d0 2024-09-15T23:00:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f682aff1740 1 -- 172.21.15.111:0/1560696183 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f680c003d80 con 0x556b346158d0 2024-09-15T23:00:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f6806a00640 1 -- 172.21.15.111:0/1560696183 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6818035020 con 0x556b346158d0 2024-09-15T23:00:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f682aff1740 10 client.26144 fetch_fsmap learned FSMap version 31 2024-09-15T23:00:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f682aff1740 1 -- 172.21.15.111:0/1560696183 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556b3469a980 con 0x556b346158d0 2024-09-15T23:00:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f6806a00640 1 -- 172.21.15.111:0/1560696183 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f680c003d80 con 0x556b346158d0 2024-09-15T23:00:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f6806a00640 1 -- 172.21.15.111:0/1560696183 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f6818045040 con 0x556b346158d0 2024-09-15T23:00:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f682aff1740 10 client.26144 fetch_fsmap finished waiting for FSMap version 31 2024-09-15T23:00:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f682aff1740 10 client.26144 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:00:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f682aff1740 20 client.26144 populate_metadata read hostname 'smithi111' 2024-09-15T23:00:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f682aff1740 1 --2- 172.21.15.111:0/1560696183 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556b3461fc30 0x556b3469d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f682aff1740 4 client.26144 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:00:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f682aff1740 1 -- 172.21.15.111:0/1560696183 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556b34586b80 con 0x556b3461fc30 2024-09-15T23:00:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.841+0000 7f6824c00640 1 --2- 172.21.15.111:0/1560696183 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556b3461fc30 0x556b3469d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.846+0000 7f6824c00640 1 --2- 172.21.15.111:0/1560696183 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556b3461fc30 0x556b3469d080 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fac006b7bef5ec88 server_cookie=605ab3ccc43b49b1 in_seq=0 out_seq=0 2024-09-15T23:00:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.846+0000 7f6806a00640 10 client.26144 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:00:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.923+0000 7f6806a00640 1 -- 172.21.15.111:0/1560696183 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556b34586b80 con 0x556b3461fc30 2024-09-15T23:00:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.923+0000 7f6806a00640 10 client.26144 handle_command_reply: tid=0 2024-09-15T23:00:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.923+0000 7f682aff1740 1 client.26144 shutdown 2024-09-15T23:00:45.926 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.923+0000 7f682aff1740 1 -- 172.21.15.111:0/1560696183 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b346158d0 msgr2=0x556b346290c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.923+0000 7f682aff1740 1 --2- 172.21.15.111:0/1560696183 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b346158d0 0x556b346290c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6818021d80 tx=0x7f6818005bd0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.923+0000 7f682aff1740 1 -- 172.21.15.111:0/1560696183 shutdown_connections 2024-09-15T23:00:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.923+0000 7f682aff1740 1 --2- 172.21.15.111:0/1560696183 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556b3461fc30 0x556b3469d080 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.923+0000 7f682aff1740 1 --2- 172.21.15.111:0/1560696183 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556b3461aba0 0x556b3462a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.923+0000 7f682aff1740 1 --2- 172.21.15.111:0/1560696183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556b3461a260 0x556b34629600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.923+0000 7f682aff1740 1 --2- 172.21.15.111:0/1560696183 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556b346158d0 0x556b346290c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.923+0000 7f6806a00640 0 client.26144 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:00:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.923+0000 7f682aff1740 1 -- 172.21.15.111:0/1560696183 >> 172.21.15.111:0/1560696183 conn(0x556b3461f7c0 msgr2=0x556b346141f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.923+0000 7f682aff1740 1 -- 172.21.15.111:0/1560696183 shutdown_connections 2024-09-15T23:00:45.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.923+0000 7f682aff1740 1 -- 172.21.15.111:0/1560696183 wait complete. 2024-09-15T23:00:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.923+0000 7f682aff1740 20 client.26144 trim_cache size 0 max 16384 2024-09-15T23:00:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:00:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:00:45.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:00:45.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:00:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.923+0000 7f6826a00640 1 -- 172.21.15.111:0/2626199922 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f68080a3390 msgr2=0x7f68080a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.923+0000 7f6826a00640 1 --2- 172.21.15.111:0/2626199922 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f68080a3390 0x7f68080a5850 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f68140b8870 tx=0x7f68140b8040 comp rx=0 tx=0).stop 2024-09-15T23:00:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.923+0000 7f6826a00640 1 -- 172.21.15.111:0/2626199922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6820103b70 msgr2=0x7f6820199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.923+0000 7f6826a00640 1 --2- 172.21.15.111:0/2626199922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6820103b70 0x7f6820199ec0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6814031440 tx=0x7f6814032ce0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.923+0000 7f6826a00640 1 -- 172.21.15.111:0/2626199922 shutdown_connections 2024-09-15T23:00:45.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.923+0000 7f6826a00640 1 --2- 172.21.15.111:0/2626199922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6820105720 0x7f68201a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.923+0000 7f6826a00640 1 --2- 172.21.15.111:0/2626199922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6820104d60 0x7f682019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.923+0000 7f6826a00640 1 --2- 172.21.15.111:0/2626199922 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f68080a3390 0x7f68080a5850 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.923+0000 7f6826a00640 1 --2- 172.21.15.111:0/2626199922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6820103b70 0x7f6820199ec0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.923+0000 7f6826a00640 1 -- 172.21.15.111:0/2626199922 >> 172.21.15.111:0/2626199922 conn(0x7f68200ff2f0 msgr2=0x7f6820108080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.923+0000 7f6826a00640 1 -- 172.21.15.111:0/2626199922 shutdown_connections 2024-09-15T23:00:45.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:45.923+0000 7f6826a00640 1 -- 172.21.15.111:0/2626199922 wait complete. 2024-09-15T23:00:46.092 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:45 smithi111 ceph-mon[30629]: pgmap v795: 97 pgs: 97 active+clean; 468 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.3 MiB/s wr, 264 op/s 2024-09-15T23:00:46.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:45 smithi192 ceph-mon[35427]: pgmap v795: 97 pgs: 97 active+clean; 468 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.3 MiB/s wr, 264 op/s 2024-09-15T23:00:46.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:45 smithi121 ceph-mon[34815]: pgmap v795: 97 pgs: 97 active+clean; 468 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.3 MiB/s wr, 264 op/s 2024-09-15T23:00:46.621 INFO:tasks.workunit.client.0.smithi111.stdout: 10 23256 19.29 MB/sec warmup 74 sec latency 4379.428 ms 2024-09-15T23:00:47.621 INFO:tasks.workunit.client.0.smithi111.stdout: 10 23545 19.17 MB/sec warmup 75 sec latency 582.632 ms 2024-09-15T23:00:48.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:47 smithi192 ceph-mon[35427]: pgmap v796: 97 pgs: 97 active+clean; 469 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.7 MiB/s wr, 259 op/s 2024-09-15T23:00:48.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:47 smithi121 ceph-mon[34815]: pgmap v796: 97 pgs: 97 active+clean; 469 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.7 MiB/s wr, 259 op/s 2024-09-15T23:00:48.391 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:47 smithi111 ceph-mon[30629]: pgmap v796: 97 pgs: 97 active+clean; 469 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.7 MiB/s wr, 259 op/s 2024-09-15T23:00:48.621 INFO:tasks.workunit.client.0.smithi111.stdout: 10 24374 19.45 MB/sec warmup 76 sec latency 114.761 ms 2024-09-15T23:00:49.625 INFO:tasks.workunit.client.0.smithi111.stdout: 10 25294 19.97 MB/sec warmup 77 sec latency 55.979 ms 2024-09-15T23:00:50.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:49 smithi192 ceph-mon[35427]: pgmap v797: 97 pgs: 97 active+clean; 460 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.6 MiB/s wr, 308 op/s 2024-09-15T23:00:50.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:49 smithi121 ceph-mon[34815]: pgmap v797: 97 pgs: 97 active+clean; 460 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.6 MiB/s wr, 308 op/s 2024-09-15T23:00:50.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:49 smithi111 ceph-mon[30629]: pgmap v797: 97 pgs: 97 active+clean; 460 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.6 MiB/s wr, 308 op/s 2024-09-15T23:00:50.625 INFO:tasks.workunit.client.0.smithi111.stdout: 10 26136 20.27 MB/sec warmup 78 sec latency 56.754 ms 2024-09-15T23:00:51.625 INFO:tasks.workunit.client.0.smithi111.stdout: 10 26971 20.64 MB/sec warmup 79 sec latency 72.101 ms 2024-09-15T23:00:52.209 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:51 smithi111 ceph-mon[30629]: pgmap v798: 97 pgs: 97 active+clean; 464 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.1 MiB/s wr, 321 op/s 2024-09-15T23:00:52.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:51 smithi192 ceph-mon[35427]: pgmap v798: 97 pgs: 97 active+clean; 464 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.1 MiB/s wr, 321 op/s 2024-09-15T23:00:52.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:51 smithi121 ceph-mon[34815]: pgmap v798: 97 pgs: 97 active+clean; 464 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.1 MiB/s wr, 321 op/s 2024-09-15T23:00:52.625 INFO:tasks.workunit.client.0.smithi111.stdout: 10 27237 20.60 MB/sec warmup 80 sec latency 569.384 ms 2024-09-15T23:00:52.988 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:52 smithi111 ceph-mon[30629]: pgmap v799: 97 pgs: 97 active+clean; 470 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.9 MiB/s wr, 319 op/s 2024-09-15T23:00:52.988 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:52 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:00:53.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:52 smithi192 ceph-mon[35427]: pgmap v799: 97 pgs: 97 active+clean; 470 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.9 MiB/s wr, 319 op/s 2024-09-15T23:00:53.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:52 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:00:53.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:52 smithi121 ceph-mon[34815]: pgmap v799: 97 pgs: 97 active+clean; 470 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.9 MiB/s wr, 319 op/s 2024-09-15T23:00:53.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:52 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:00:53.625 INFO:tasks.workunit.client.0.smithi111.stdout: 10 27479 20.47 MB/sec warmup 81 sec latency 1191.540 ms 2024-09-15T23:00:54.311 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:53 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:00:54.311 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:53 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:00:54.311 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:53 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:00:54.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:53 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:00:54.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:53 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:00:54.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:53 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:00:54.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:53 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:00:54.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:53 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:00:54.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:53 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:00:54.625 INFO:tasks.workunit.client.0.smithi111.stdout: 10 28148 20.58 MB/sec warmup 82 sec latency 66.852 ms 2024-09-15T23:00:55.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:54 smithi111 ceph-mon[30629]: pgmap v800: 97 pgs: 97 active+clean; 475 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.8 MiB/s wr, 374 op/s 2024-09-15T23:00:55.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:54 smithi192 ceph-mon[35427]: pgmap v800: 97 pgs: 97 active+clean; 475 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.8 MiB/s wr, 374 op/s 2024-09-15T23:00:55.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:54 smithi121 ceph-mon[34815]: pgmap v800: 97 pgs: 97 active+clean; 475 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.8 MiB/s wr, 374 op/s 2024-09-15T23:00:55.625 INFO:tasks.workunit.client.0.smithi111.stdout: 10 29031 20.82 MB/sec warmup 83 sec latency 65.608 ms 2024-09-15T23:00:56.625 INFO:tasks.workunit.client.0.smithi111.stdout: 10 30266 21.52 MB/sec warmup 84 sec latency 37.635 ms 2024-09-15T23:00:57.625 INFO:tasks.workunit.client.0.smithi111.stdout: 10 30863 21.67 MB/sec warmup 85 sec latency 188.072 ms 2024-09-15T23:00:58.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:57 smithi121 ceph-mon[34815]: pgmap v801: 97 pgs: 97 active+clean; 478 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.5 MiB/s wr, 397 op/s 2024-09-15T23:00:58.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:57 smithi111 ceph-mon[30629]: pgmap v801: 97 pgs: 97 active+clean; 478 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.5 MiB/s wr, 397 op/s 2024-09-15T23:00:58.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:57 smithi192 ceph-mon[35427]: pgmap v801: 97 pgs: 97 active+clean; 478 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.5 MiB/s wr, 397 op/s 2024-09-15T23:00:58.625 INFO:tasks.workunit.client.0.smithi111.stdout: 10 31460 21.84 MB/sec warmup 86 sec latency 113.204 ms 2024-09-15T23:00:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.071+0000 7fdfd4600640 1 -- 172.21.15.111:0/1284321799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfcc105910 msgr2=0x7fdfcc10c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:59.072 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.071+0000 7fdfd4600640 1 --2- 172.21.15.111:0/1284321799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfcc105910 0x7fdfcc10c1a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009be0 tx=0x7fdfc402a6c0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.071+0000 7fdfd4600640 1 -- 172.21.15.111:0/1284321799 shutdown_connections 2024-09-15T23:00:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.071+0000 7fdfd4600640 1 --2- 172.21.15.111:0/1284321799 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfcc105910 0x7fdfcc10c1a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.071+0000 7fdfd4600640 1 --2- 172.21.15.111:0/1284321799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfcc104f50 0x7fdfcc1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.071+0000 7fdfd4600640 1 --2- 172.21.15.111:0/1284321799 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdfcc103d60 0x7fdfcc104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.071+0000 7fdfd4600640 1 -- 172.21.15.111:0/1284321799 >> 172.21.15.111:0/1284321799 conn(0x7fdfcc0ff4e0 msgr2=0x7fdfcc101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.071+0000 7fdfd4600640 1 -- 172.21.15.111:0/1284321799 shutdown_connections 2024-09-15T23:00:59.073 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.071+0000 7fdfd4600640 1 -- 172.21.15.111:0/1284321799 wait complete. 2024-09-15T23:00:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.072+0000 7fdfd4600640 1 Processor -- start 2024-09-15T23:00:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.072+0000 7fdfd4600640 1 -- start start 2024-09-15T23:00:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.072+0000 7fdfd4600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdfcc103d60 0x7fdfcc19e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.072+0000 7fdfd4600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfcc104f50 0x7fdfcc19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.072+0000 7fdfd4600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfcc105910 0x7fdfcc1a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.072+0000 7fdfd4600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc076b00 con 0x7fdfcc103d60 2024-09-15T23:00:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.072+0000 7fdfd4600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc076980 con 0x7fdfcc105910 2024-09-15T23:00:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.072+0000 7fdfd4600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc076c80 con 0x7fdfcc104f50 2024-09-15T23:00:59.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.073+0000 7fdfd1200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfcc104f50 0x7fdfcc19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.073+0000 7fdfd1c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdfcc103d60 0x7fdfcc19e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.073+0000 7fdfd2600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfcc105910 0x7fdfcc1a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.073+0000 7fdfd1200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfcc104f50 0x7fdfcc19e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:59474/0 (socket says 172.21.15.111:59474) 2024-09-15T23:00:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.073+0000 7fdfd1200640 1 -- 172.21.15.111:0/735976913 learned_addr learned my addr 172.21.15.111:0/735976913 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:00:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.073+0000 7fdfd1200640 1 -- 172.21.15.111:0/735976913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfcc105910 msgr2=0x7fdfcc1a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.073+0000 7fdfd1200640 1 --2- 172.21.15.111:0/735976913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfcc105910 0x7fdfcc1a5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.073+0000 7fdfd1200640 1 -- 172.21.15.111:0/735976913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdfcc103d60 msgr2=0x7fdfcc19e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.073+0000 7fdfd1200640 1 --2- 172.21.15.111:0/735976913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdfcc103d60 0x7fdfcc19e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.073+0000 7fdfd1200640 1 -- 172.21.15.111:0/735976913 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfcc1a5fb0 con 0x7fdfcc104f50 2024-09-15T23:00:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.073+0000 7fdfd1c00640 1 --2- 172.21.15.111:0/735976913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdfcc103d60 0x7fdfcc19e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:00:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.073+0000 7fdfd1200640 1 --2- 172.21.15.111:0/735976913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfcc104f50 0x7fdfcc19e8d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdfb800e9c0 tx=0x7fdfb800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:59.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.073+0000 7fdfca000640 1 -- 172.21.15.111:0/735976913 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb800ccc0 con 0x7fdfcc104f50 2024-09-15T23:00:59.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.074+0000 7fdfca000640 1 -- 172.21.15.111:0/735976913 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfb8005500 con 0x7fdfcc104f50 2024-09-15T23:00:59.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.074+0000 7fdfca000640 1 -- 172.21.15.111:0/735976913 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8010430 con 0x7fdfcc104f50 2024-09-15T23:00:59.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.074+0000 7fdfd4600640 1 -- 172.21.15.111:0/735976913 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfcc1a62a0 con 0x7fdfcc104f50 2024-09-15T23:00:59.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.074+0000 7fdfd4600640 1 -- 172.21.15.111:0/735976913 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfcc1a6840 con 0x7fdfcc104f50 2024-09-15T23:00:59.078 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.077+0000 7fdfd740f740 1 Processor -- start 2024-09-15T23:00:59.078 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.077+0000 7fdfca000640 1 -- 172.21.15.111:0/735976913 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fdfb80046c0 con 0x7fdfcc104f50 2024-09-15T23:00:59.078 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.077+0000 7fdfd740f740 1 -- start start 2024-09-15T23:00:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.077+0000 7fdfd740f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55622e3ac090 0x55622e3ac490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.077+0000 7fdfd740f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55622e3ac9d0 0x55622e3b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.077+0000 7fdfd740f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55622e3b1a60 0x55622e3b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.077+0000 7fdfd740f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55622e311100 con 0x55622e3b1a60 2024-09-15T23:00:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.077+0000 7fdfd740f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55622e2b91b0 con 0x55622e3ac9d0 2024-09-15T23:00:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.077+0000 7fdfd740f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55622e30d0c0 con 0x55622e3ac090 2024-09-15T23:00:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.077+0000 7fdfd2600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55622e3ac090 0x55622e3ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.077+0000 7fdfd1c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55622e3ac9d0 0x55622e3b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.077+0000 7fdfd2600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55622e3ac090 0x55622e3ac490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:59490/0 (socket says 172.21.15.111:59490) 2024-09-15T23:00:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.078+0000 7fdfd2600640 1 -- 172.21.15.111:0/2332196342 learned_addr learned my addr 172.21.15.111:0/2332196342 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:00:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.078+0000 7fdfd1200640 1 --2- 172.21.15.111:0/2332196342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55622e3b1a60 0x55622e3b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.078+0000 7fdfd1c00640 1 -- 172.21.15.111:0/2332196342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55622e3ac090 msgr2=0x55622e3ac490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.078+0000 7fdfd1c00640 1 --2- 172.21.15.111:0/2332196342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55622e3ac090 0x55622e3ac490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.078+0000 7fdfd1c00640 1 -- 172.21.15.111:0/2332196342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55622e3b1a60 msgr2=0x55622e3b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.078+0000 7fdfd1c00640 1 --2- 172.21.15.111:0/2332196342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55622e3b1a60 0x55622e3b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.078+0000 7fdfd1c00640 1 -- 172.21.15.111:0/2332196342 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55622e024210 con 0x55622e3ac9d0 2024-09-15T23:00:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.078+0000 7fdfd1c00640 1 --2- 172.21.15.111:0/2332196342 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55622e3ac9d0 0x55622e3b1490 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0009fa0 tx=0x7fdfc00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=245b5bedc273c3be server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.078+0000 7fdfca000640 1 --2- 172.21.15.111:0/735976913 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fdfb40a3370 0x7fdfb40a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.078+0000 7fdfbea00640 1 -- 172.21.15.111:0/2332196342 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0004310 con 0x55622e3ac9d0 2024-09-15T23:00:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.078+0000 7fdfbea00640 1 -- 172.21.15.111:0/2332196342 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfc00044b0 con 0x55622e3ac9d0 2024-09-15T23:00:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.078+0000 7fdfbea00640 1 -- 172.21.15.111:0/2332196342 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc00037f0 con 0x55622e3ac9d0 2024-09-15T23:00:59.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.079+0000 7fdfca000640 1 -- 172.21.15.111:0/735976913 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fdfb80cfb60 con 0x7fdfcc104f50 2024-09-15T23:00:59.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.079+0000 7fdfd2600640 1 --2- 172.21.15.111:0/735976913 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fdfb40a3370 0x7fdfb40a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:59.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.079+0000 7fdfd740f740 1 -- 172.21.15.111:0/2332196342 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55622e3ac9d0 msgr2=0x55622e3b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:59.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.079+0000 7fdfd740f740 1 --2- 172.21.15.111:0/2332196342 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55622e3ac9d0 0x55622e3b1490 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0009fa0 tx=0x7fdfc00118d0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.079+0000 7fdfd740f740 1 -- 172.21.15.111:0/2332196342 shutdown_connections 2024-09-15T23:00:59.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.079+0000 7fdfd740f740 1 --2- 172.21.15.111:0/2332196342 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55622e3b1a60 0x55622e3b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.079+0000 7fdfd740f740 1 --2- 172.21.15.111:0/2332196342 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55622e3ac9d0 0x55622e3b1490 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.079+0000 7fdfd740f740 1 --2- 172.21.15.111:0/2332196342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55622e3ac090 0x55622e3ac490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.079+0000 7fdfd740f740 1 -- 172.21.15.111:0/2332196342 >> 172.21.15.111:0/2332196342 conn(0x55622e3a7700 msgr2=0x55622e3a7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:59.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.079+0000 7fdfd2600640 1 --2- 172.21.15.111:0/735976913 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fdfb40a3370 0x7fdfb40a5830 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdfc402acb0 tx=0x7fdfc4005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:59.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.079+0000 7fdfd740f740 1 -- 172.21.15.111:0/2332196342 shutdown_connections 2024-09-15T23:00:59.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.079+0000 7fdfd740f740 1 -- 172.21.15.111:0/2332196342 wait complete. 2024-09-15T23:00:59.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.080+0000 7fdfd740f740 1 -- start start 2024-09-15T23:00:59.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.080+0000 7fdfd740f740 1 Processor -- start 2024-09-15T23:00:59.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.080+0000 7fdfd740f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55622e3ac090 0x55622e3b6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:59.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.080+0000 7fdfd740f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55622e3ac9d0 0x55622e3b67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:59.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.081+0000 7fdfd740f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55622e3b1a60 0x55622e3bd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:59.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.081+0000 7fdfd740f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55622e3b4a30 con 0x55622e3b1a60 2024-09-15T23:00:59.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.081+0000 7fdfd740f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55622e29a090 con 0x55622e3ac9d0 2024-09-15T23:00:59.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.081+0000 7fdfd740f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55622e3b4bb0 con 0x55622e3ac090 2024-09-15T23:00:59.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.081+0000 7fdfd740f740 10 client.0 _ll_register_callbacks cb 0x55622e2eef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:00:59.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.081+0000 7fdfd1200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55622e3ac9d0 0x55622e3b67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:59.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.081+0000 7fdfd1200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55622e3ac9d0 0x55622e3b67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:37244/0 (socket says 172.21.15.111:37244) 2024-09-15T23:00:59.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.081+0000 7fdfd1200640 1 -- 172.21.15.111:0/1853138127 learned_addr learned my addr 172.21.15.111:0/1853138127 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:00:59.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.081+0000 7fdfd2600640 1 --2- 172.21.15.111:0/1853138127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55622e3b1a60 0x55622e3bd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:59.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.081+0000 7fdfd1c00640 1 --2- 172.21.15.111:0/1853138127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55622e3ac090 0x55622e3b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:59.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.081+0000 7fdfd1200640 1 -- 172.21.15.111:0/1853138127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55622e3ac090 msgr2=0x55622e3b6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:59.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.081+0000 7fdfd1200640 1 --2- 172.21.15.111:0/1853138127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55622e3ac090 0x55622e3b6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.081+0000 7fdfd1200640 1 -- 172.21.15.111:0/1853138127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55622e3b1a60 msgr2=0x55622e3bd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:59.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.081+0000 7fdfd1200640 1 --2- 172.21.15.111:0/1853138127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55622e3b1a60 0x55622e3bd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.081+0000 7fdfd1200640 1 -- 172.21.15.111:0/1853138127 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55622e0f0620 con 0x55622e3ac9d0 2024-09-15T23:00:59.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.081+0000 7fdfd1200640 1 --2- 172.21.15.111:0/1853138127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55622e3ac9d0 0x55622e3b67b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdfb809c110 tx=0x7fdfb809c010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b617fd08e9763fa server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:59.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.081+0000 7fdfb3400640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:00:59.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.082+0000 7fdfb3400640 1 -- 172.21.15.111:0/1853138127 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb809c180 con 0x55622e3ac9d0 2024-09-15T23:00:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.082+0000 7fdfb3400640 1 -- 172.21.15.111:0/1853138127 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fdfb80a0170 con 0x55622e3ac9d0 2024-09-15T23:00:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.082+0000 7fdfd740f740 1 -- 172.21.15.111:0/1853138127 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfc00044b0 con 0x55622e3ac9d0 2024-09-15T23:00:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.082+0000 7fdfb3400640 1 -- 172.21.15.111:0/1853138127 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdfb80b5140 con 0x55622e3ac9d0 2024-09-15T23:00:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.082+0000 7fdfd740f740 10 client.26156 fetch_fsmap learned FSMap version 31 2024-09-15T23:00:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.082+0000 7fdfd740f740 1 -- 172.21.15.111:0/1853138127 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55622e3c0630 con 0x55622e3ac9d0 2024-09-15T23:00:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.082+0000 7fdfb3400640 1 -- 172.21.15.111:0/1853138127 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfc00044b0 con 0x55622e3ac9d0 2024-09-15T23:00:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.082+0000 7fdfb3400640 1 -- 172.21.15.111:0/1853138127 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fdfb80a4960 con 0x55622e3ac9d0 2024-09-15T23:00:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.082+0000 7fdfd740f740 10 client.26156 fetch_fsmap finished waiting for FSMap version 31 2024-09-15T23:00:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.082+0000 7fdfd740f740 10 client.26156 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:00:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.082+0000 7fdfd740f740 20 client.26156 populate_metadata read hostname 'smithi111' 2024-09-15T23:00:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.082+0000 7fdfd740f740 1 --2- 172.21.15.111:0/1853138127 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55622e3c1c90 0x55622e3c4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.082+0000 7fdfd740f740 4 client.26156 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:00:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.082+0000 7fdfd740f740 1 -- 172.21.15.111:0/1853138127 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55622e31e830 con 0x55622e3c1c90 2024-09-15T23:00:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.083+0000 7fdfd1c00640 1 --2- 172.21.15.111:0/1853138127 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55622e3c1c90 0x55622e3c4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.084+0000 7fdfd1c00640 1 --2- 172.21.15.111:0/1853138127 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55622e3c1c90 0x55622e3c4080 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3935d9db980d09ee server_cookie=4f78c91a47d40947 in_seq=0 out_seq=0 2024-09-15T23:00:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.084+0000 7fdfb3400640 10 client.26156 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:00:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.085+0000 7fdfb3400640 1 -- 172.21.15.111:0/1853138127 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55622e31e830 con 0x55622e3c1c90 2024-09-15T23:00:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.085+0000 7fdfb3400640 10 client.26156 handle_command_reply: tid=0 2024-09-15T23:00:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.085+0000 7fdfd740f740 1 client.26156 shutdown 2024-09-15T23:00:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.085+0000 7fdfd740f740 1 -- 172.21.15.111:0/1853138127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55622e3ac9d0 msgr2=0x55622e3b67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.085+0000 7fdfd740f740 1 --2- 172.21.15.111:0/1853138127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55622e3ac9d0 0x55622e3b67b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdfb809c110 tx=0x7fdfb809c010 comp rx=0 tx=0).stop 2024-09-15T23:00:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.086+0000 7fdfd740f740 1 -- 172.21.15.111:0/1853138127 shutdown_connections 2024-09-15T23:00:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.086+0000 7fdfd740f740 1 --2- 172.21.15.111:0/1853138127 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55622e3c1c90 0x55622e3c4080 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.086+0000 7fdfd740f740 1 --2- 172.21.15.111:0/1853138127 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55622e3b1a60 0x55622e3bd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.086+0000 7fdfd740f740 1 --2- 172.21.15.111:0/1853138127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55622e3ac9d0 0x55622e3b67b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.086+0000 7fdfd740f740 1 --2- 172.21.15.111:0/1853138127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55622e3ac090 0x55622e3b6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.086+0000 7fdfd740f740 1 -- 172.21.15.111:0/1853138127 >> 172.21.15.111:0/1853138127 conn(0x55622e3a7700 msgr2=0x55622e3a8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.086+0000 7fdfb3400640 0 client.26156 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:00:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.086+0000 7fdfd740f740 1 -- 172.21.15.111:0/1853138127 shutdown_connections 2024-09-15T23:00:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.087+0000 7fdfd740f740 1 -- 172.21.15.111:0/1853138127 wait complete. 2024-09-15T23:00:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.087+0000 7fdfd740f740 20 client.26156 trim_cache size 0 max 16384 2024-09-15T23:00:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.106+0000 7fdfd740f740 1 Processor -- start 2024-09-15T23:00:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.106+0000 7fdfd740f740 1 -- start start 2024-09-15T23:00:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.106+0000 7fdfd740f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55622e3ac090 0x55622e3c8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.106+0000 7fdfd740f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55622e3ac9d0 0x55622e3bd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.106+0000 7fdfd740f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55622e3b1a60 0x55622e3c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.106+0000 7fdfd740f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1b1d60 con 0x55622e3ac9d0 2024-09-15T23:00:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.106+0000 7fdfd740f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1af5d0 con 0x55622e3ac090 2024-09-15T23:00:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.106+0000 7fdfd740f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1ad420 con 0x55622e3b1a60 2024-09-15T23:00:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.106+0000 7fdfd1c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55622e3ac090 0x55622e3c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.107+0000 7fdfd1c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55622e3ac090 0x55622e3c8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:37252/0 (socket says 172.21.15.111:37252) 2024-09-15T23:00:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.107+0000 7fdfd1200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55622e3ac9d0 0x55622e3bd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.107+0000 7fdfd1c00640 1 -- 172.21.15.111:0/2602915248 learned_addr learned my addr 172.21.15.111:0/2602915248 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:00:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.107+0000 7fdfd2600640 1 --2- 172.21.15.111:0/2602915248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55622e3b1a60 0x55622e3c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.107+0000 7fdfd1c00640 1 -- 172.21.15.111:0/2602915248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55622e3b1a60 msgr2=0x55622e3c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.107+0000 7fdfd1c00640 1 --2- 172.21.15.111:0/2602915248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55622e3b1a60 0x55622e3c8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.107+0000 7fdfd1c00640 1 -- 172.21.15.111:0/2602915248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55622e3ac9d0 msgr2=0x55622e3bd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.107+0000 7fdfd1c00640 1 --2- 172.21.15.111:0/2602915248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55622e3ac9d0 0x55622e3bd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.107+0000 7fdfd1c00640 1 -- 172.21.15.111:0/2602915248 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55622e3c46b0 con 0x55622e3ac090 2024-09-15T23:00:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.107+0000 7fdfd1c00640 1 --2- 172.21.15.111:0/2602915248 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55622e3ac090 0x55622e3c8910 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdfc00058c0 tx=0x7fdfc00096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=442bf67433d3e60e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.108+0000 7fdfbe000640 1 -- 172.21.15.111:0/2602915248 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0031070 con 0x55622e3ac090 2024-09-15T23:00:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.108+0000 7fdfbe000640 1 -- 172.21.15.111:0/2602915248 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfc0009b50 con 0x55622e3ac090 2024-09-15T23:00:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.108+0000 7fdfbe000640 1 -- 172.21.15.111:0/2602915248 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc001d040 con 0x55622e3ac090 2024-09-15T23:00:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.109+0000 7fdfd740f740 1 -- 172.21.15.111:0/2602915248 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55622e3ac090 msgr2=0x55622e3c8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.109+0000 7fdfd740f740 1 --2- 172.21.15.111:0/2602915248 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55622e3ac090 0x55622e3c8910 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdfc00058c0 tx=0x7fdfc00096c0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.109+0000 7fdfd740f740 1 -- 172.21.15.111:0/2602915248 shutdown_connections 2024-09-15T23:00:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.109+0000 7fdfd740f740 1 --2- 172.21.15.111:0/2602915248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55622e3b1a60 0x55622e3c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.109+0000 7fdfd740f740 1 --2- 172.21.15.111:0/2602915248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55622e3ac9d0 0x55622e3bd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.109+0000 7fdfd740f740 1 --2- 172.21.15.111:0/2602915248 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55622e3ac090 0x55622e3c8910 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.109+0000 7fdfd740f740 1 -- 172.21.15.111:0/2602915248 >> 172.21.15.111:0/2602915248 conn(0x55622e3a7700 msgr2=0x55622e3a6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.109+0000 7fdfd740f740 1 -- 172.21.15.111:0/2602915248 shutdown_connections 2024-09-15T23:00:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.109+0000 7fdfd740f740 1 -- 172.21.15.111:0/2602915248 wait complete. 2024-09-15T23:00:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.110+0000 7fdfd740f740 1 -- start start 2024-09-15T23:00:59.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.110+0000 7fdfd740f740 1 Processor -- start 2024-09-15T23:00:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.110+0000 7fdfd740f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55622e3ac090 0x55622e3bb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.111+0000 7fdfd740f740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55622e3ac9d0 0x55622e3bbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.111+0000 7fdfd740f740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55622e3b1a60 0x55622e3bcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.111+0000 7fdfd740f740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1ada90 con 0x55622e3b1a60 2024-09-15T23:00:59.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.111+0000 7fdfd740f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1aef00 con 0x55622e3ac090 2024-09-15T23:00:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.111+0000 7fdfd740f740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fdfcc1ae160 con 0x55622e3ac9d0 2024-09-15T23:00:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.111+0000 7fdfd740f740 10 client.0 _ll_register_callbacks cb 0x7fdfb80056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:00:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.111+0000 7fdfd2600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55622e3b1a60 0x55622e3bcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.111+0000 7fdfd2600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55622e3b1a60 0x55622e3bcae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40762/0 (socket says 172.21.15.111:40762) 2024-09-15T23:00:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.111+0000 7fdfd2600640 1 -- 172.21.15.111:0/2218200955 learned_addr learned my addr 172.21.15.111:0/2218200955 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:00:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.111+0000 7fdfd1c00640 1 --2- 172.21.15.111:0/2218200955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55622e3ac090 0x55622e3bb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.111+0000 7fdfd2600640 1 -- 172.21.15.111:0/2218200955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55622e3ac9d0 msgr2=0x55622e3bbad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T23:00:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.111+0000 7fdfd2600640 1 --2- 172.21.15.111:0/2218200955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55622e3ac9d0 0x55622e3bbad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.111+0000 7fdfd2600640 1 -- 172.21.15.111:0/2218200955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55622e3ac090 msgr2=0x55622e3bb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:00:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.111+0000 7fdfd2600640 1 --2- 172.21.15.111:0/2218200955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55622e3ac090 0x55622e3bb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.111+0000 7fdfd2600640 1 -- 172.21.15.111:0/2218200955 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55622e3bd0f0 con 0x55622e3b1a60 2024-09-15T23:00:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.112+0000 7fdfd2600640 1 --2- 172.21.15.111:0/2218200955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55622e3b1a60 0x55622e3bcae0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40099a0 tx=0x7fdfc4009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae73c34e2a6a1eae server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:00:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.112+0000 7fdfb3400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:00:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.112+0000 7fdfb3400640 1 -- 172.21.15.111:0/2218200955 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4036040 con 0x55622e3b1a60 2024-09-15T23:00:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.112+0000 7fdfd740f740 1 -- 172.21.15.111:0/2218200955 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfc0009b50 con 0x55622e3b1a60 2024-09-15T23:00:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.112+0000 7fdfb3400640 1 -- 172.21.15.111:0/2218200955 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fdfc4004240 con 0x55622e3b1a60 2024-09-15T23:00:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.113+0000 7fdfb3400640 1 -- 172.21.15.111:0/2218200955 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdfc4004440 con 0x55622e3b1a60 2024-09-15T23:00:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.113+0000 7fdfd740f740 10 client.16203 fetch_fsmap learned FSMap version 31 2024-09-15T23:00:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.113+0000 7fdfd740f740 1 -- 172.21.15.111:0/2218200955 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55622e42d140 con 0x55622e3b1a60 2024-09-15T23:00:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.113+0000 7fdfb3400640 1 -- 172.21.15.111:0/2218200955 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfc0009b50 con 0x55622e3b1a60 2024-09-15T23:00:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.114+0000 7fdfb3400640 1 -- 172.21.15.111:0/2218200955 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fdfc4031440 con 0x55622e3b1a60 2024-09-15T23:00:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.114+0000 7fdfd740f740 10 client.16203 fetch_fsmap finished waiting for FSMap version 31 2024-09-15T23:00:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.114+0000 7fdfd740f740 10 client.16203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:00:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.114+0000 7fdfd740f740 20 client.16203 populate_metadata read hostname 'smithi111' 2024-09-15T23:00:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.114+0000 7fdfd740f740 1 --2- 172.21.15.111:0/2218200955 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55622e42dad0 0x55622e42fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:00:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.114+0000 7fdfd740f740 4 client.16203 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:00:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.114+0000 7fdfd740f740 1 -- 172.21.15.111:0/2218200955 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55622e318b80 con 0x55622e42dad0 2024-09-15T23:00:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.114+0000 7fdfd1c00640 1 --2- 172.21.15.111:0/2218200955 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55622e42dad0 0x55622e42fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:00:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.114+0000 7fdfd1c00640 1 --2- 172.21.15.111:0/2218200955 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55622e42dad0 0x55622e42fef0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee6333372be5b98f server_cookie=b3edbff9abf8f05f in_seq=0 out_seq=0 2024-09-15T23:00:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.114+0000 7fdfb3400640 10 client.16203 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:00:59.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.334+0000 7fdfb3400640 1 -- 172.21.15.111:0/2218200955 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55622e318b80 con 0x55622e42dad0 2024-09-15T23:00:59.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.334+0000 7fdfb3400640 10 client.16203 handle_command_reply: tid=0 2024-09-15T23:00:59.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.334+0000 7fdfd740f740 1 client.16203 shutdown 2024-09-15T23:00:59.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.335+0000 7fdfd740f740 1 -- 172.21.15.111:0/2218200955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55622e3b1a60 msgr2=0x55622e3bcae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:59.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.335+0000 7fdfd740f740 1 --2- 172.21.15.111:0/2218200955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55622e3b1a60 0x55622e3bcae0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40099a0 tx=0x7fdfc4009500 comp rx=0 tx=0).stop 2024-09-15T23:00:59.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.335+0000 7fdfd740f740 1 -- 172.21.15.111:0/2218200955 shutdown_connections 2024-09-15T23:00:59.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.335+0000 7fdfd740f740 1 --2- 172.21.15.111:0/2218200955 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55622e42dad0 0x55622e42fef0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.338 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.335+0000 7fdfd740f740 1 --2- 172.21.15.111:0/2218200955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55622e3b1a60 0x55622e3bcae0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.335+0000 7fdfd740f740 1 --2- 172.21.15.111:0/2218200955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55622e3ac9d0 0x55622e3bbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.335+0000 7fdfd740f740 1 --2- 172.21.15.111:0/2218200955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55622e3ac090 0x55622e3bb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.335+0000 7fdfb3400640 0 client.16203 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:00:59.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.335+0000 7fdfd740f740 1 -- 172.21.15.111:0/2218200955 >> 172.21.15.111:0/2218200955 conn(0x55622e3a7700 msgr2=0x55622e3a8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:59.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.335+0000 7fdfd740f740 1 -- 172.21.15.111:0/2218200955 shutdown_connections 2024-09-15T23:00:59.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.335+0000 7fdfd740f740 1 -- 172.21.15.111:0/2218200955 wait complete. 2024-09-15T23:00:59.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.335+0000 7fdfd740f740 20 client.16203 trim_cache size 0 max 16384 2024-09-15T23:00:59.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:00:59.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:00:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:00:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:00:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.336+0000 7fdfd4600640 1 -- 172.21.15.111:0/735976913 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fdfb40a3370 msgr2=0x7fdfb40a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.336+0000 7fdfd4600640 1 --2- 172.21.15.111:0/735976913 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fdfb40a3370 0x7fdfb40a5830 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdfc402acb0 tx=0x7fdfc4005fd0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.336+0000 7fdfd4600640 1 -- 172.21.15.111:0/735976913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfcc104f50 msgr2=0x7fdfcc19e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:00:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.336+0000 7fdfd4600640 1 --2- 172.21.15.111:0/735976913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfcc104f50 0x7fdfcc19e8d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdfb800e9c0 tx=0x7fdfb800ee80 comp rx=0 tx=0).stop 2024-09-15T23:00:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.336+0000 7fdfd4600640 1 -- 172.21.15.111:0/735976913 shutdown_connections 2024-09-15T23:00:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.336+0000 7fdfd4600640 1 --2- 172.21.15.111:0/735976913 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fdfb40a3370 0x7fdfb40a5830 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.336+0000 7fdfd4600640 1 --2- 172.21.15.111:0/735976913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfcc105910 0x7fdfcc1a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.336+0000 7fdfd4600640 1 --2- 172.21.15.111:0/735976913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdfcc104f50 0x7fdfcc19e8d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.336+0000 7fdfd4600640 1 --2- 172.21.15.111:0/735976913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdfcc103d60 0x7fdfcc19e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:00:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.336+0000 7fdfd4600640 1 -- 172.21.15.111:0/735976913 >> 172.21.15.111:0/735976913 conn(0x7fdfcc0ff4e0 msgr2=0x7fdfcc0795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:00:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.336+0000 7fdfd4600640 1 -- 172.21.15.111:0/735976913 shutdown_connections 2024-09-15T23:00:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:00:59.336+0000 7fdfd4600640 1 -- 172.21.15.111:0/735976913 wait complete. 2024-09-15T23:00:59.625 INFO:tasks.workunit.client.0.smithi111.stdout: 10 31957 21.80 MB/sec warmup 87 sec latency 148.524 ms 2024-09-15T23:01:00.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:00:59 smithi121 ceph-mon[34815]: pgmap v802: 97 pgs: 97 active+clean; 487 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.6 MiB/s wr, 494 op/s 2024-09-15T23:01:00.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:00:59 smithi111 ceph-mon[30629]: pgmap v802: 97 pgs: 97 active+clean; 487 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.6 MiB/s wr, 494 op/s 2024-09-15T23:01:00.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:00:59 smithi192 ceph-mon[35427]: pgmap v802: 97 pgs: 97 active+clean; 487 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.6 MiB/s wr, 494 op/s 2024-09-15T23:01:00.625 INFO:tasks.workunit.client.0.smithi111.stdout: 10 32799 21.97 MB/sec warmup 88 sec latency 63.877 ms 2024-09-15T23:01:01.626 INFO:tasks.workunit.client.0.smithi111.stdout: 10 33648 22.29 MB/sec warmup 89 sec latency 85.537 ms 2024-09-15T23:01:02.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:01 smithi121 ceph-mon[34815]: pgmap v803: 97 pgs: 97 active+clean; 496 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.9 MiB/s wr, 570 op/s 2024-09-15T23:01:02.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:01 smithi111 ceph-mon[30629]: pgmap v803: 97 pgs: 97 active+clean; 496 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.9 MiB/s wr, 570 op/s 2024-09-15T23:01:02.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:01 smithi192 ceph-mon[35427]: pgmap v803: 97 pgs: 97 active+clean; 496 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.9 MiB/s wr, 570 op/s 2024-09-15T23:01:02.625 INFO:tasks.workunit.client.0.smithi111.stdout: 10 34609 22.58 MB/sec warmup 90 sec latency 56.217 ms 2024-09-15T23:01:03.626 INFO:tasks.workunit.client.0.smithi111.stdout: 10 35324 22.89 MB/sec warmup 91 sec latency 40.748 ms 2024-09-15T23:01:04.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:03 smithi121 ceph-mon[34815]: pgmap v804: 97 pgs: 97 active+clean; 499 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.7 MiB/s wr, 561 op/s 2024-09-15T23:01:04.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:03 smithi111 ceph-mon[30629]: pgmap v804: 97 pgs: 97 active+clean; 499 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.7 MiB/s wr, 561 op/s 2024-09-15T23:01:04.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:03 smithi192 ceph-mon[35427]: pgmap v804: 97 pgs: 97 active+clean; 499 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.7 MiB/s wr, 561 op/s 2024-09-15T23:01:04.626 INFO:tasks.workunit.client.0.smithi111.stdout: 10 35933 22.97 MB/sec warmup 92 sec latency 47.602 ms 2024-09-15T23:01:05.626 INFO:tasks.workunit.client.0.smithi111.stdout: 10 36685 23.20 MB/sec warmup 93 sec latency 51.605 ms 2024-09-15T23:01:06.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:05 smithi121 ceph-mon[34815]: pgmap v805: 97 pgs: 97 active+clean; 494 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.4 MiB/s wr, 574 op/s 2024-09-15T23:01:06.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:05 smithi111 ceph-mon[30629]: pgmap v805: 97 pgs: 97 active+clean; 494 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.4 MiB/s wr, 574 op/s 2024-09-15T23:01:06.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:05 smithi192 ceph-mon[35427]: pgmap v805: 97 pgs: 97 active+clean; 494 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.4 MiB/s wr, 574 op/s 2024-09-15T23:01:06.626 INFO:tasks.workunit.client.0.smithi111.stdout: 10 37336 23.17 MB/sec warmup 94 sec latency 105.502 ms 2024-09-15T23:01:07.626 INFO:tasks.workunit.client.0.smithi111.stdout: 10 37986 23.33 MB/sec warmup 95 sec latency 224.716 ms 2024-09-15T23:01:07.963 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:07 smithi111 ceph-mon[30629]: pgmap v806: 97 pgs: 97 active+clean; 490 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.4 MiB/s wr, 590 op/s 2024-09-15T23:01:08.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:07 smithi192 ceph-mon[35427]: pgmap v806: 97 pgs: 97 active+clean; 490 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.4 MiB/s wr, 590 op/s 2024-09-15T23:01:08.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:07 smithi121 ceph-mon[34815]: pgmap v806: 97 pgs: 97 active+clean; 490 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.4 MiB/s wr, 590 op/s 2024-09-15T23:01:08.626 INFO:tasks.workunit.client.0.smithi111.stdout: 10 38191 23.29 MB/sec warmup 96 sec latency 1010.949 ms 2024-09-15T23:01:09.626 INFO:tasks.workunit.client.0.smithi111.stdout: 10 39034 23.41 MB/sec warmup 97 sec latency 79.357 ms 2024-09-15T23:01:10.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:09 smithi121 ceph-mon[34815]: pgmap v807: 97 pgs: 97 active+clean; 487 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 5.2 MiB/s wr, 674 op/s 2024-09-15T23:01:10.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:09 smithi111 ceph-mon[30629]: pgmap v807: 97 pgs: 97 active+clean; 487 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 5.2 MiB/s wr, 674 op/s 2024-09-15T23:01:10.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:09 smithi192 ceph-mon[35427]: pgmap v807: 97 pgs: 97 active+clean; 487 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 5.2 MiB/s wr, 674 op/s 2024-09-15T23:01:10.626 INFO:tasks.workunit.client.0.smithi111.stdout: 10 39959 23.75 MB/sec warmup 98 sec latency 77.801 ms 2024-09-15T23:01:11.629 INFO:tasks.workunit.client.0.smithi111.stdout: 10 40603 23.93 MB/sec warmup 99 sec latency 81.115 ms 2024-09-15T23:01:12.141 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:11 smithi121 ceph-mon[34815]: pgmap v808: 97 pgs: 97 active+clean; 490 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 4.8 MiB/s wr, 594 op/s 2024-09-15T23:01:12.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:11 smithi111 ceph-mon[30629]: pgmap v808: 97 pgs: 97 active+clean; 490 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 4.8 MiB/s wr, 594 op/s 2024-09-15T23:01:12.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:11 smithi192 ceph-mon[35427]: pgmap v808: 97 pgs: 97 active+clean; 490 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 4.8 MiB/s wr, 594 op/s 2024-09-15T23:01:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.467+0000 7fce61200640 1 -- 172.21.15.111:0/363911510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce5c104f50 msgr2=0x7fce5c1053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.467+0000 7fce61200640 1 --2- 172.21.15.111:0/363911510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce5c104f50 0x7fce5c1053d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fce48009900 tx=0x7fce4802a410 comp rx=0 tx=0).stop 2024-09-15T23:01:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.467+0000 7fce61200640 1 -- 172.21.15.111:0/363911510 shutdown_connections 2024-09-15T23:01:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.467+0000 7fce61200640 1 --2- 172.21.15.111:0/363911510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce5c105910 0x7fce5c10c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.467+0000 7fce61200640 1 --2- 172.21.15.111:0/363911510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce5c104f50 0x7fce5c1053d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.467+0000 7fce61200640 1 --2- 172.21.15.111:0/363911510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce5c103d60 0x7fce5c104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.467+0000 7fce61200640 1 -- 172.21.15.111:0/363911510 >> 172.21.15.111:0/363911510 conn(0x7fce5c0ff4e0 msgr2=0x7fce5c101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:01:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.467+0000 7fce61200640 1 -- 172.21.15.111:0/363911510 shutdown_connections 2024-09-15T23:01:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.467+0000 7fce61200640 1 -- 172.21.15.111:0/363911510 wait complete. 2024-09-15T23:01:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.468+0000 7fce61200640 1 Processor -- start 2024-09-15T23:01:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.468+0000 7fce61200640 1 -- start start 2024-09-15T23:01:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.468+0000 7fce61200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce5c103d60 0x7fce5c199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.468+0000 7fce61200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce5c104f50 0x7fce5c19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.468+0000 7fce61200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce5c105910 0x7fce5c1a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.468+0000 7fce61200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c10fa10 con 0x7fce5c105910 2024-09-15T23:01:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.468+0000 7fce61200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c10f890 con 0x7fce5c103d60 2024-09-15T23:01:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.468+0000 7fce61200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c10fb90 con 0x7fce5c104f50 2024-09-15T23:01:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.469+0000 7fce59e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce5c103d60 0x7fce5c199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.469+0000 7fce59e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce5c103d60 0x7fce5c199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:35636/0 (socket says 172.21.15.111:35636) 2024-09-15T23:01:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.469+0000 7fce59e00640 1 -- 172.21.15.111:0/3167095573 learned_addr learned my addr 172.21.15.111:0/3167095573 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:01:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.469+0000 7fce59400640 1 --2- 172.21.15.111:0/3167095573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce5c104f50 0x7fce5c19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.469+0000 7fce5a800640 1 --2- 172.21.15.111:0/3167095573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce5c105910 0x7fce5c1a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.469+0000 7fce59e00640 1 -- 172.21.15.111:0/3167095573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce5c104f50 msgr2=0x7fce5c19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.469+0000 7fce59e00640 1 --2- 172.21.15.111:0/3167095573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce5c104f50 0x7fce5c19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.469+0000 7fce59e00640 1 -- 172.21.15.111:0/3167095573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce5c105910 msgr2=0x7fce5c1a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.469+0000 7fce59e00640 1 --2- 172.21.15.111:0/3167095573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce5c105910 0x7fce5c1a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.469+0000 7fce59e00640 1 -- 172.21.15.111:0/3167095573 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5c1a1bd0 con 0x7fce5c103d60 2024-09-15T23:01:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.469+0000 7fce59e00640 1 --2- 172.21.15.111:0/3167095573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce5c103d60 0x7fce5c199fb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00e9c0 tx=0x7fce4c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:01:12.472 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.470+0000 7fce52a00640 1 -- 172.21.15.111:0/3167095573 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c00cd80 con 0x7fce5c103d60 2024-09-15T23:01:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.470+0000 7fce52a00640 1 -- 172.21.15.111:0/3167095573 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce4c0035f0 con 0x7fce5c103d60 2024-09-15T23:01:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.470+0000 7fce52a00640 1 -- 172.21.15.111:0/3167095573 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c010690 con 0x7fce5c103d60 2024-09-15T23:01:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.470+0000 7fce61200640 1 -- 172.21.15.111:0/3167095573 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce5c1a1ec0 con 0x7fce5c103d60 2024-09-15T23:01:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.470+0000 7fce61200640 1 -- 172.21.15.111:0/3167095573 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce5c1a22d0 con 0x7fce5c103d60 2024-09-15T23:01:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.473+0000 7fce64214740 1 Processor -- start 2024-09-15T23:01:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.473+0000 7fce64214740 1 -- start start 2024-09-15T23:01:12.476 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.474+0000 7fce52a00640 1 -- 172.21.15.111:0/3167095573 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fce4c0037a0 con 0x7fce5c103d60 2024-09-15T23:01:12.476 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.474+0000 7fce64214740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f309e68a0 0x556f309e6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.474+0000 7fce64214740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f309e71e0 0x556f309ebca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.474+0000 7fce64214740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309ec270 0x556f309ee730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.474+0000 7fce64214740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556f30943100 con 0x556f309e71e0 2024-09-15T23:01:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.474+0000 7fce64214740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556f308eb1b0 con 0x556f309e68a0 2024-09-15T23:01:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.474+0000 7fce64214740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556f3093f0c0 con 0x556f309ec270 2024-09-15T23:01:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.474+0000 7fce5a800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f309e68a0 0x556f309e6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.474+0000 7fce59400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309ec270 0x556f309ee730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.474+0000 7fce5a800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f309e68a0 0x556f309e6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:35640/0 (socket says 172.21.15.111:35640) 2024-09-15T23:01:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.474+0000 7fce5a800640 1 -- 172.21.15.111:0/4211044677 learned_addr learned my addr 172.21.15.111:0/4211044677 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:01:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.474+0000 7fce59400640 1 --2- 172.21.15.111:0/4211044677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309ec270 0x556f309ee730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:60020/0 (socket says 172.21.15.111:60020) 2024-09-15T23:01:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.474+0000 7fce59e00640 1 --2- 172.21.15.111:0/4211044677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f309e71e0 0x556f309ebca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:12.477 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.474+0000 7fce59400640 1 -- 172.21.15.111:0/4211044677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f309e68a0 msgr2=0x556f309e6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:12.478 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.474+0000 7fce59400640 1 --2- 172.21.15.111:0/4211044677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f309e68a0 0x556f309e6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.478 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.475+0000 7fce59400640 1 -- 172.21.15.111:0/4211044677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f309e71e0 msgr2=0x556f309ebca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:12.478 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.475+0000 7fce59400640 1 --2- 172.21.15.111:0/4211044677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f309e71e0 0x556f309ebca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.478 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.475+0000 7fce59400640 1 -- 172.21.15.111:0/4211044677 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f30656210 con 0x556f309ec270 2024-09-15T23:01:12.478 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.475+0000 7fce5a800640 1 --2- 172.21.15.111:0/4211044677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f309e68a0 0x556f309e6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:01:12.478 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.475+0000 7fce59400640 1 --2- 172.21.15.111:0/4211044677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309ec270 0x556f309ee730 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fce4802a930 tx=0x7fce48005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caafadee7cccb2d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:01:12.478 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.475+0000 7fce47400640 1 -- 172.21.15.111:0/4211044677 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce48035040 con 0x556f309ec270 2024-09-15T23:01:12.478 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.475+0000 7fce52a00640 1 --2- 172.21.15.111:0/3167095573 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fce400a3370 0x7fce400a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:12.478 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.475+0000 7fce47400640 1 -- 172.21.15.111:0/4211044677 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce48005520 con 0x556f309ec270 2024-09-15T23:01:12.478 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.475+0000 7fce47400640 1 -- 172.21.15.111:0/4211044677 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce480316b0 con 0x556f309ec270 2024-09-15T23:01:12.478 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.475+0000 7fce52a00640 1 -- 172.21.15.111:0/3167095573 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fce4c0cfc80 con 0x7fce5c103d60 2024-09-15T23:01:12.481 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.476+0000 7fce5a800640 1 --2- 172.21.15.111:0/3167095573 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fce400a3370 0x7fce400a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.476+0000 7fce64214740 1 -- 172.21.15.111:0/4211044677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309ec270 msgr2=0x556f309ee730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.476+0000 7fce64214740 1 --2- 172.21.15.111:0/4211044677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309ec270 0x556f309ee730 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fce4802a930 tx=0x7fce48005c00 comp rx=0 tx=0).stop 2024-09-15T23:01:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.476+0000 7fce64214740 1 -- 172.21.15.111:0/4211044677 shutdown_connections 2024-09-15T23:01:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.476+0000 7fce64214740 1 --2- 172.21.15.111:0/4211044677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309ec270 0x556f309ee730 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.476+0000 7fce64214740 1 --2- 172.21.15.111:0/4211044677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f309e71e0 0x556f309ebca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.476+0000 7fce64214740 1 --2- 172.21.15.111:0/4211044677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f309e68a0 0x556f309e6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.476+0000 7fce64214740 1 -- 172.21.15.111:0/4211044677 >> 172.21.15.111:0/4211044677 conn(0x556f309d9700 msgr2=0x556f309d9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:01:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.476+0000 7fce5a800640 1 --2- 172.21.15.111:0/3167095573 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fce400a3370 0x7fce400a5830 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fce54011890 tx=0x7fce54009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:01:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.476+0000 7fce64214740 1 -- 172.21.15.111:0/4211044677 shutdown_connections 2024-09-15T23:01:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.476+0000 7fce64214740 1 -- 172.21.15.111:0/4211044677 wait complete. 2024-09-15T23:01:12.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.477+0000 7fce64214740 1 -- start start 2024-09-15T23:01:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.477+0000 7fce64214740 1 Processor -- start 2024-09-15T23:01:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.478+0000 7fce64214740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f309e68a0 0x556f309f09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.478+0000 7fce64214740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309e71e0 0x556f309f0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.478+0000 7fce64214740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f309ec270 0x556f309f7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.478+0000 7fce64214740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x556f309ef1e0 con 0x556f309e68a0 2024-09-15T23:01:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.478+0000 7fce64214740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556f308cc090 con 0x556f309ec270 2024-09-15T23:01:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.478+0000 7fce64214740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x556f309ef360 con 0x556f309e71e0 2024-09-15T23:01:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.478+0000 7fce64214740 10 client.0 _ll_register_callbacks cb 0x556f30920f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:01:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.478+0000 7fce59400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309e71e0 0x556f309f0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.478+0000 7fce59e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f309e68a0 0x556f309f09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.478+0000 7fce59400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309e71e0 0x556f309f0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:60026/0 (socket says 172.21.15.111:60026) 2024-09-15T23:01:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.478+0000 7fce59400640 1 -- 172.21.15.111:0/1980779983 learned_addr learned my addr 172.21.15.111:0/1980779983 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:01:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.478+0000 7fce5a800640 1 --2- 172.21.15.111:0/1980779983 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f309ec270 0x556f309f7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:12.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.478+0000 7fce59400640 1 -- 172.21.15.111:0/1980779983 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f309ec270 msgr2=0x556f309f7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.478+0000 7fce59400640 1 --2- 172.21.15.111:0/1980779983 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f309ec270 0x556f309f7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.478+0000 7fce59400640 1 -- 172.21.15.111:0/1980779983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f309e68a0 msgr2=0x556f309f09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.478+0000 7fce59400640 1 --2- 172.21.15.111:0/1980779983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f309e68a0 0x556f309f09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.478+0000 7fce59400640 1 -- 172.21.15.111:0/1980779983 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f30722620 con 0x556f309e71e0 2024-09-15T23:01:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.478+0000 7fce5a800640 1 --2- 172.21.15.111:0/1980779983 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f309ec270 0x556f309f7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:01:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.479+0000 7fce59400640 1 --2- 172.21.15.111:0/1980779983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309e71e0 0x556f309f0f30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fce480314e0 tx=0x7fce48004530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d57789eee026978a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:01:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.479+0000 7fce3be00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:01:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.479+0000 7fce3be00640 1 -- 172.21.15.111:0/1980779983 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce48052060 con 0x556f309e71e0 2024-09-15T23:01:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.479+0000 7fce64214740 1 -- 172.21.15.111:0/1980779983 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce48005520 con 0x556f309e71e0 2024-09-15T23:01:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.479+0000 7fce3be00640 1 -- 172.21.15.111:0/1980779983 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fce48004f60 con 0x556f309e71e0 2024-09-15T23:01:12.484 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.479+0000 7fce3be00640 1 -- 172.21.15.111:0/1980779983 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fce48035050 con 0x556f309e71e0 2024-09-15T23:01:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.479+0000 7fce64214740 10 client.26188 fetch_fsmap learned FSMap version 31 2024-09-15T23:01:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.479+0000 7fce64214740 1 -- 172.21.15.111:0/1980779983 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556f309ef1e0 con 0x556f309e71e0 2024-09-15T23:01:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.480+0000 7fce3be00640 1 -- 172.21.15.111:0/1980779983 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce48005520 con 0x556f309e71e0 2024-09-15T23:01:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.480+0000 7fce3be00640 1 -- 172.21.15.111:0/1980779983 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fce48057070 con 0x556f309e71e0 2024-09-15T23:01:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.480+0000 7fce64214740 10 client.26188 fetch_fsmap finished waiting for FSMap version 31 2024-09-15T23:01:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.480+0000 7fce64214740 10 client.26188 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:01:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.480+0000 7fce64214740 20 client.26188 populate_metadata read hostname 'smithi111' 2024-09-15T23:01:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.480+0000 7fce64214740 1 --2- 172.21.15.111:0/1980779983 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556f309fbf80 0x556f309fe370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.480+0000 7fce64214740 4 client.26188 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:01:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.480+0000 7fce64214740 1 -- 172.21.15.111:0/1980779983 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f30950830 con 0x556f309fbf80 2024-09-15T23:01:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.480+0000 7fce59e00640 1 --2- 172.21.15.111:0/1980779983 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556f309fbf80 0x556f309fe370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.481+0000 7fce59e00640 1 --2- 172.21.15.111:0/1980779983 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556f309fbf80 0x556f309fe370 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71f9e3185172ebbd server_cookie=d7b110baf4c1f4ca in_seq=0 out_seq=0 2024-09-15T23:01:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.481+0000 7fce3be00640 10 client.26188 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:01:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.482+0000 7fce3be00640 1 -- 172.21.15.111:0/1980779983 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556f30950830 con 0x556f309fbf80 2024-09-15T23:01:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.482+0000 7fce3be00640 10 client.26188 handle_command_reply: tid=0 2024-09-15T23:01:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.482+0000 7fce64214740 1 client.26188 shutdown 2024-09-15T23:01:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.482+0000 7fce64214740 1 -- 172.21.15.111:0/1980779983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309e71e0 msgr2=0x556f309f0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.482+0000 7fce64214740 1 --2- 172.21.15.111:0/1980779983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309e71e0 0x556f309f0f30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fce480314e0 tx=0x7fce48004530 comp rx=0 tx=0).stop 2024-09-15T23:01:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.482+0000 7fce64214740 1 -- 172.21.15.111:0/1980779983 shutdown_connections 2024-09-15T23:01:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.482+0000 7fce64214740 1 --2- 172.21.15.111:0/1980779983 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556f309fbf80 0x556f309fe370 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.482+0000 7fce64214740 1 --2- 172.21.15.111:0/1980779983 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f309ec270 0x556f309f7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.482+0000 7fce64214740 1 --2- 172.21.15.111:0/1980779983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309e71e0 0x556f309f0f30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.482+0000 7fce3be00640 0 client.26188 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:01:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.482+0000 7fce64214740 1 --2- 172.21.15.111:0/1980779983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f309e68a0 0x556f309f09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.482+0000 7fce64214740 1 -- 172.21.15.111:0/1980779983 >> 172.21.15.111:0/1980779983 conn(0x556f309d9700 msgr2=0x556f309daf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:01:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.482+0000 7fce64214740 1 -- 172.21.15.111:0/1980779983 shutdown_connections 2024-09-15T23:01:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.482+0000 7fce64214740 1 -- 172.21.15.111:0/1980779983 wait complete. 2024-09-15T23:01:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.483+0000 7fce64214740 20 client.26188 trim_cache size 0 max 16384 2024-09-15T23:01:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.500+0000 7fce64214740 1 Processor -- start 2024-09-15T23:01:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.500+0000 7fce64214740 1 -- start start 2024-09-15T23:01:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.500+0000 7fce64214740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f309e68a0 0x556f309d8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.500+0000 7fce64214740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309e71e0 0x556f309f7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.500+0000 7fce64214740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f309ec270 0x556f309daaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.500+0000 7fce64214740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c1a9530 con 0x556f309ec270 2024-09-15T23:01:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.500+0000 7fce64214740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c1a6da0 con 0x556f309e68a0 2024-09-15T23:01:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.500+0000 7fce64214740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c1a4c40 con 0x556f309e71e0 2024-09-15T23:01:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.501+0000 7fce5a800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f309ec270 0x556f309daaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:12.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.501+0000 7fce59400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309e71e0 0x556f309f7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.501+0000 7fce5a800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f309ec270 0x556f309daaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42596/0 (socket says 172.21.15.111:42596) 2024-09-15T23:01:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.501+0000 7fce5a800640 1 -- 172.21.15.111:0/4167254078 learned_addr learned my addr 172.21.15.111:0/4167254078 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:01:12.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.501+0000 7fce59e00640 1 --2- 172.21.15.111:0/4167254078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f309e68a0 0x556f309d8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.501+0000 7fce59400640 1 -- 172.21.15.111:0/4167254078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f309e68a0 msgr2=0x556f309d8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.501+0000 7fce59400640 1 --2- 172.21.15.111:0/4167254078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f309e68a0 0x556f309d8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.501+0000 7fce59400640 1 -- 172.21.15.111:0/4167254078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f309ec270 msgr2=0x556f309daaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:12.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.501+0000 7fce59400640 1 --2- 172.21.15.111:0/4167254078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f309ec270 0x556f309daaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.501+0000 7fce59400640 1 -- 172.21.15.111:0/4167254078 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f309fe6e0 con 0x556f309e71e0 2024-09-15T23:01:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.501+0000 7fce59400640 1 --2- 172.21.15.111:0/4167254078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309e71e0 0x556f309f7f30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fce4802a930 tx=0x7fce48005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0a2bd4a55ea9969 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:01:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.501+0000 7fce46a00640 1 -- 172.21.15.111:0/4167254078 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4805d030 con 0x556f309e71e0 2024-09-15T23:01:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.502+0000 7fce46a00640 1 -- 172.21.15.111:0/4167254078 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce48005620 con 0x556f309e71e0 2024-09-15T23:01:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.502+0000 7fce46a00640 1 -- 172.21.15.111:0/4167254078 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce480316b0 con 0x556f309e71e0 2024-09-15T23:01:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.502+0000 7fce64214740 1 -- 172.21.15.111:0/4167254078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309e71e0 msgr2=0x556f309f7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.502+0000 7fce64214740 1 --2- 172.21.15.111:0/4167254078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309e71e0 0x556f309f7f30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fce4802a930 tx=0x7fce48005d50 comp rx=0 tx=0).stop 2024-09-15T23:01:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.502+0000 7fce64214740 1 -- 172.21.15.111:0/4167254078 shutdown_connections 2024-09-15T23:01:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.502+0000 7fce64214740 1 --2- 172.21.15.111:0/4167254078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f309ec270 0x556f309daaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.502+0000 7fce64214740 1 --2- 172.21.15.111:0/4167254078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309e71e0 0x556f309f7f30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.502+0000 7fce64214740 1 --2- 172.21.15.111:0/4167254078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f309e68a0 0x556f309d8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.502+0000 7fce64214740 1 -- 172.21.15.111:0/4167254078 >> 172.21.15.111:0/4167254078 conn(0x556f309d9700 msgr2=0x556f309d8060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:01:12.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.503+0000 7fce64214740 1 -- 172.21.15.111:0/4167254078 shutdown_connections 2024-09-15T23:01:12.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.503+0000 7fce64214740 1 -- 172.21.15.111:0/4167254078 wait complete. 2024-09-15T23:01:12.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.503+0000 7fce64214740 1 -- start start 2024-09-15T23:01:12.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.503+0000 7fce64214740 1 Processor -- start 2024-09-15T23:01:12.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.504+0000 7fce64214740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309e68a0 0x556f309f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:12.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.504+0000 7fce64214740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f309e71e0 0x556f309f5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:12.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.504+0000 7fce64214740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f309ec270 0x556f309f69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:12.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.504+0000 7fce64214740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c1a5260 con 0x556f309ec270 2024-09-15T23:01:12.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.504+0000 7fce64214740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c1a66d0 con 0x556f309e71e0 2024-09-15T23:01:12.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.504+0000 7fce64214740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c1a5930 con 0x556f309e68a0 2024-09-15T23:01:12.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.504+0000 7fce64214740 10 client.0 _ll_register_callbacks cb 0x7fce4c0d3590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:01:12.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.504+0000 7fce59400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f309e71e0 0x556f309f5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:12.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.504+0000 7fce59e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309e68a0 0x556f309f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:12.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.504+0000 7fce59400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f309e71e0 0x556f309f5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:35666/0 (socket says 172.21.15.111:35666) 2024-09-15T23:01:12.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.504+0000 7fce59400640 1 -- 172.21.15.111:0/2070383800 learned_addr learned my addr 172.21.15.111:0/2070383800 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:01:12.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.504+0000 7fce5a800640 1 --2- 172.21.15.111:0/2070383800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f309ec270 0x556f309f69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:12.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.504+0000 7fce59400640 1 -- 172.21.15.111:0/2070383800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309e68a0 msgr2=0x556f309f54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:12.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.504+0000 7fce59400640 1 --2- 172.21.15.111:0/2070383800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309e68a0 0x556f309f54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.504+0000 7fce59400640 1 -- 172.21.15.111:0/2070383800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f309ec270 msgr2=0x556f309f69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:12.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.504+0000 7fce59400640 1 --2- 172.21.15.111:0/2070383800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f309ec270 0x556f309f69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.504+0000 7fce59400640 1 -- 172.21.15.111:0/2070383800 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f309f6ff0 con 0x556f309e71e0 2024-09-15T23:01:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.504+0000 7fce5a800640 1 --2- 172.21.15.111:0/2070383800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f309ec270 0x556f309f69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:01:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.504+0000 7fce59e00640 1 --2- 172.21.15.111:0/2070383800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309e68a0 0x556f309f54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:01:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.505+0000 7fce59400640 1 --2- 172.21.15.111:0/2070383800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f309e71e0 0x556f309f5a20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fce48031e40 tx=0x7fce480046e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8417e8026accac7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:01:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.505+0000 7fce3be00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:01:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.505+0000 7fce3be00640 1 -- 172.21.15.111:0/2070383800 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4805d040 con 0x556f309e71e0 2024-09-15T23:01:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.505+0000 7fce64214740 1 -- 172.21.15.111:0/2070383800 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce48005620 con 0x556f309e71e0 2024-09-15T23:01:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.505+0000 7fce3be00640 1 -- 172.21.15.111:0/2070383800 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7fce48004770 con 0x556f309e71e0 2024-09-15T23:01:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.505+0000 7fce3be00640 1 -- 172.21.15.111:0/2070383800 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fce48009520 con 0x556f309e71e0 2024-09-15T23:01:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.505+0000 7fce64214740 10 client.26180 fetch_fsmap learned FSMap version 31 2024-09-15T23:01:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.505+0000 7fce64214740 1 -- 172.21.15.111:0/2070383800 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556f30a66be0 con 0x556f309e71e0 2024-09-15T23:01:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.505+0000 7fce3be00640 1 -- 172.21.15.111:0/2070383800 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce48005620 con 0x556f309e71e0 2024-09-15T23:01:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.506+0000 7fce3be00640 1 -- 172.21.15.111:0/2070383800 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7fce48057070 con 0x556f309e71e0 2024-09-15T23:01:12.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.506+0000 7fce64214740 10 client.26180 fetch_fsmap finished waiting for FSMap version 31 2024-09-15T23:01:12.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.506+0000 7fce64214740 10 client.26180 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:01:12.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.506+0000 7fce64214740 20 client.26180 populate_metadata read hostname 'smithi111' 2024-09-15T23:01:12.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.506+0000 7fce64214740 1 --2- 172.21.15.111:0/2070383800 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556f30a673a0 0x556f30a697c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:12.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.506+0000 7fce64214740 4 client.26180 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:01:12.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.506+0000 7fce64214740 1 -- 172.21.15.111:0/2070383800 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556f3094ab80 con 0x556f30a673a0 2024-09-15T23:01:12.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.506+0000 7fce59e00640 1 --2- 172.21.15.111:0/2070383800 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556f30a673a0 0x556f30a697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:12.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.507+0000 7fce59e00640 1 --2- 172.21.15.111:0/2070383800 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556f30a673a0 0x556f30a697c0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a725d41d2c7ce006 server_cookie=289281b9b2bc1909 in_seq=0 out_seq=0 2024-09-15T23:01:12.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.507+0000 7fce3be00640 10 client.26180 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:01:12.626 INFO:tasks.workunit.client.0.smithi111.stdout: 10 41099 23.93 MB/sec warmup 100 sec latency 84.672 ms 2024-09-15T23:01:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.697+0000 7fce3be00640 1 -- 172.21.15.111:0/2070383800 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556f3094ab80 con 0x556f30a673a0 2024-09-15T23:01:12.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.697+0000 7fce3be00640 10 client.26180 handle_command_reply: tid=0 2024-09-15T23:01:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.697+0000 7fce64214740 1 client.26180 shutdown 2024-09-15T23:01:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.697+0000 7fce64214740 1 -- 172.21.15.111:0/2070383800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f309e71e0 msgr2=0x556f309f5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.697+0000 7fce64214740 1 --2- 172.21.15.111:0/2070383800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f309e71e0 0x556f309f5a20 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fce48031e40 tx=0x7fce480046e0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.697+0000 7fce64214740 1 -- 172.21.15.111:0/2070383800 shutdown_connections 2024-09-15T23:01:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.697+0000 7fce64214740 1 --2- 172.21.15.111:0/2070383800 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x556f30a673a0 0x556f30a697c0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.697+0000 7fce64214740 1 --2- 172.21.15.111:0/2070383800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556f309ec270 0x556f309f69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.697+0000 7fce64214740 1 --2- 172.21.15.111:0/2070383800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556f309e71e0 0x556f309f5a20 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.697+0000 7fce3be00640 0 client.26180 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:01:12.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.697+0000 7fce64214740 1 --2- 172.21.15.111:0/2070383800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556f309e68a0 0x556f309f54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.697+0000 7fce64214740 1 -- 172.21.15.111:0/2070383800 >> 172.21.15.111:0/2070383800 conn(0x556f309d9700 msgr2=0x556f309d8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:01:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.698+0000 7fce64214740 1 -- 172.21.15.111:0/2070383800 shutdown_connections 2024-09-15T23:01:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.698+0000 7fce64214740 1 -- 172.21.15.111:0/2070383800 wait complete. 2024-09-15T23:01:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.698+0000 7fce64214740 20 client.26180 trim_cache size 0 max 16384 2024-09-15T23:01:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:01:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:01:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:01:12.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:01:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.699+0000 7fce61200640 1 -- 172.21.15.111:0/3167095573 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fce400a3370 msgr2=0x7fce400a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.699+0000 7fce61200640 1 --2- 172.21.15.111:0/3167095573 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fce400a3370 0x7fce400a5830 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fce54011890 tx=0x7fce54009500 comp rx=0 tx=0).stop 2024-09-15T23:01:12.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.699+0000 7fce61200640 1 -- 172.21.15.111:0/3167095573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce5c103d60 msgr2=0x7fce5c199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.699+0000 7fce61200640 1 --2- 172.21.15.111:0/3167095573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce5c103d60 0x7fce5c199fb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00e9c0 tx=0x7fce4c00ee80 comp rx=0 tx=0).stop 2024-09-15T23:01:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.699+0000 7fce61200640 1 -- 172.21.15.111:0/3167095573 shutdown_connections 2024-09-15T23:01:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.699+0000 7fce61200640 1 --2- 172.21.15.111:0/3167095573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce5c105910 0x7fce5c1a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.699+0000 7fce61200640 1 --2- 172.21.15.111:0/3167095573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce5c104f50 0x7fce5c19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.699+0000 7fce61200640 1 --2- 172.21.15.111:0/3167095573 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fce400a3370 0x7fce400a5830 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.699+0000 7fce61200640 1 --2- 172.21.15.111:0/3167095573 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fce5c103d60 0x7fce5c199fb0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.699+0000 7fce61200640 1 -- 172.21.15.111:0/3167095573 >> 172.21.15.111:0/3167095573 conn(0x7fce5c0ff4e0 msgr2=0x7fce5c1081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:01:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.701+0000 7fce61200640 1 -- 172.21.15.111:0/3167095573 shutdown_connections 2024-09-15T23:01:12.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:12.701+0000 7fce61200640 1 -- 172.21.15.111:0/3167095573 wait complete. 2024-09-15T23:01:13.626 INFO:tasks.workunit.client.0.smithi111.stdout: 10 41653 23.88 MB/sec warmup 101 sec latency 296.210 ms 2024-09-15T23:01:14.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:13 smithi121 ceph-mon[34815]: pgmap v809: 97 pgs: 97 active+clean; 492 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.3 MiB/s wr, 493 op/s 2024-09-15T23:01:14.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:13 smithi111 ceph-mon[30629]: pgmap v809: 97 pgs: 97 active+clean; 492 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.3 MiB/s wr, 493 op/s 2024-09-15T23:01:14.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:13 smithi192 ceph-mon[35427]: pgmap v809: 97 pgs: 97 active+clean; 492 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.3 MiB/s wr, 493 op/s 2024-09-15T23:01:14.626 INFO:tasks.workunit.client.0.smithi111.stdout: 10 42505 24.08 MB/sec warmup 102 sec latency 49.402 ms 2024-09-15T23:01:15.626 INFO:tasks.workunit.client.0.smithi111.stdout: 10 43578 24.49 MB/sec warmup 103 sec latency 40.537 ms 2024-09-15T23:01:16.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:15 smithi111 ceph-mon[30629]: pgmap v810: 97 pgs: 97 active+clean; 499 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 5.0 MiB/s wr, 585 op/s 2024-09-15T23:01:16.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:15 smithi192 ceph-mon[35427]: pgmap v810: 97 pgs: 97 active+clean; 499 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 5.0 MiB/s wr, 585 op/s 2024-09-15T23:01:16.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:15 smithi121 ceph-mon[34815]: pgmap v810: 97 pgs: 97 active+clean; 499 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 5.0 MiB/s wr, 585 op/s 2024-09-15T23:01:16.626 INFO:tasks.workunit.client.0.smithi111.stdout: 10 44352 24.69 MB/sec warmup 104 sec latency 71.214 ms 2024-09-15T23:01:17.626 INFO:tasks.workunit.client.0.smithi111.stdout: 10 44954 24.77 MB/sec warmup 105 sec latency 67.142 ms 2024-09-15T23:01:17.950 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:17 smithi111 ceph-mon[30629]: pgmap v811: 97 pgs: 97 active+clean; 503 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.1 MiB/s wr, 609 op/s 2024-09-15T23:01:18.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:17 smithi192 ceph-mon[35427]: pgmap v811: 97 pgs: 97 active+clean; 503 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.1 MiB/s wr, 609 op/s 2024-09-15T23:01:18.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:17 smithi121 ceph-mon[34815]: pgmap v811: 97 pgs: 97 active+clean; 503 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.1 MiB/s wr, 609 op/s 2024-09-15T23:01:18.626 INFO:tasks.workunit.client.0.smithi111.stdout: 10 45004 24.55 MB/sec warmup 106 sec latency 839.536 ms 2024-09-15T23:01:19.626 INFO:tasks.workunit.client.0.smithi111.stdout: 10 45539 24.58 MB/sec warmup 107 sec latency 1023.485 ms 2024-09-15T23:01:20.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:19 smithi111 ceph-mon[30629]: pgmap v812: 97 pgs: 97 active+clean; 504 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.1 MiB/s wr, 594 op/s 2024-09-15T23:01:20.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:19 smithi192 ceph-mon[35427]: pgmap v812: 97 pgs: 97 active+clean; 504 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.1 MiB/s wr, 594 op/s 2024-09-15T23:01:20.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:19 smithi121 ceph-mon[34815]: pgmap v812: 97 pgs: 97 active+clean; 504 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 5.1 MiB/s wr, 594 op/s 2024-09-15T23:01:20.627 INFO:tasks.workunit.client.0.smithi111.stdout: 10 46336 24.68 MB/sec warmup 108 sec latency 46.295 ms 2024-09-15T23:01:21.627 INFO:tasks.workunit.client.0.smithi111.stdout: 10 47199 24.86 MB/sec warmup 109 sec latency 52.923 ms 2024-09-15T23:01:22.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:21 smithi111 ceph-mon[30629]: pgmap v813: 97 pgs: 97 active+clean; 511 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 5.1 MiB/s wr, 563 op/s 2024-09-15T23:01:22.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:21 smithi192 ceph-mon[35427]: pgmap v813: 97 pgs: 97 active+clean; 511 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 5.1 MiB/s wr, 563 op/s 2024-09-15T23:01:22.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:21 smithi121 ceph-mon[34815]: pgmap v813: 97 pgs: 97 active+clean; 511 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 5.1 MiB/s wr, 563 op/s 2024-09-15T23:01:22.627 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48064 25.11 MB/sec warmup 110 sec latency 49.964 ms 2024-09-15T23:01:23.627 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48220 25.01 MB/sec warmup 111 sec latency 853.187 ms 2024-09-15T23:01:24.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:23 smithi111 ceph-mon[30629]: pgmap v814: 97 pgs: 97 active+clean; 514 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.0 MiB/s wr, 565 op/s 2024-09-15T23:01:24.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:23 smithi192 ceph-mon[35427]: pgmap v814: 97 pgs: 97 active+clean; 514 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.0 MiB/s wr, 565 op/s 2024-09-15T23:01:24.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:23 smithi121 ceph-mon[34815]: pgmap v814: 97 pgs: 97 active+clean; 514 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.0 MiB/s wr, 565 op/s 2024-09-15T23:01:24.627 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48220 24.79 MB/sec warmup 112 sec latency 1853.263 ms 2024-09-15T23:01:25.627 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48220 24.57 MB/sec warmup 113 sec latency 2853.309 ms 2024-09-15T23:01:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e37000640 1 -- 172.21.15.111:0/180315479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e30078790 msgr2=0x7f9e30079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e37000640 1 --2- 172.21.15.111:0/180315479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e30078790 0x7f9e30079030 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9e28009be0 tx=0x7f9e2802a6c0 comp rx=0 tx=0).stop 2024-09-15T23:01:25.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e37000640 1 -- 172.21.15.111:0/180315479 shutdown_connections 2024-09-15T23:01:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e37000640 1 --2- 172.21.15.111:0/180315479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e30078790 0x7f9e30079030 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e37000640 1 --2- 172.21.15.111:0/180315479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e30077df0 0x7f9e30078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e37000640 1 --2- 172.21.15.111:0/180315479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e30076bb0 0x7f9e30076fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e37000640 1 -- 172.21.15.111:0/180315479 >> 172.21.15.111:0/180315479 conn(0x7f9e300ff2f0 msgr2=0x7f9e30101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:01:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e37000640 1 -- 172.21.15.111:0/180315479 shutdown_connections 2024-09-15T23:01:25.820 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e37000640 1 -- 172.21.15.111:0/180315479 wait complete. 2024-09-15T23:01:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e37000640 1 Processor -- start 2024-09-15T23:01:25.821 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e37000640 1 -- start start 2024-09-15T23:01:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e37000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e30076bb0 0x7f9e3019e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e37000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e30077df0 0x7f9e3019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e37000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e30078790 0x7f9e301a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e37000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9e30113c40 con 0x7f9e30077df0 2024-09-15T23:01:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e37000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9e30113ac0 con 0x7f9e30078790 2024-09-15T23:01:25.822 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e37000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9e30113dc0 con 0x7f9e30076bb0 2024-09-15T23:01:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e35200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e30077df0 0x7f9e3019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e35200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e30077df0 0x7f9e3019e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47412/0 (socket says 172.21.15.111:47412) 2024-09-15T23:01:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e35200640 1 -- 172.21.15.111:0/1731084109 learned_addr learned my addr 172.21.15.111:0/1731084109 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:01:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e36600640 1 --2- 172.21.15.111:0/1731084109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e30078790 0x7f9e301a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e36600640 1 -- 172.21.15.111:0/1731084109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e30076bb0 msgr2=0x7f9e3019e280 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T23:01:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e36600640 1 --2- 172.21.15.111:0/1731084109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e30076bb0 0x7f9e3019e280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e36600640 1 -- 172.21.15.111:0/1731084109 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e30077df0 msgr2=0x7f9e3019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e36600640 1 --2- 172.21.15.111:0/1731084109 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e30077df0 0x7f9e3019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e36600640 1 -- 172.21.15.111:0/1731084109 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e301a5ea0 con 0x7f9e30078790 2024-09-15T23:01:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e35200640 1 --2- 172.21.15.111:0/1731084109 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e30077df0 0x7f9e3019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:01:25.823 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e36600640 1 --2- 172.21.15.111:0/1731084109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e30078790 0x7f9e301a5830 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9e28034040 tx=0x7f9e280305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:01:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e2e000640 1 -- 172.21.15.111:0/1731084109 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e28041020 con 0x7f9e30078790 2024-09-15T23:01:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e2e000640 1 -- 172.21.15.111:0/1731084109 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e28030740 con 0x7f9e30078790 2024-09-15T23:01:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e37000640 1 -- 172.21.15.111:0/1731084109 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e301a6130 con 0x7f9e30078790 2024-09-15T23:01:25.824 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.816+0000 7f9e2e000640 1 -- 172.21.15.111:0/1731084109 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e28036040 con 0x7f9e30078790 2024-09-15T23:01:25.825 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.822+0000 7f9e37000640 1 -- 172.21.15.111:0/1731084109 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e301a65f0 con 0x7f9e30078790 2024-09-15T23:01:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e3b495740 1 Processor -- start 2024-09-15T23:01:25.828 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e2e000640 1 -- 172.21.15.111:0/1731084109 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f9e28044020 con 0x7f9e30078790 2024-09-15T23:01:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e3b495740 1 -- start start 2024-09-15T23:01:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e3b495740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee3516f260 0x55ee3516f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e3b495740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee3516fba0 0x55ee35174660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e3b495740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee35174c30 0x55ee351770f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e3b495740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ee350d4100 con 0x55ee3516f260 2024-09-15T23:01:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e3b495740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ee3507c1b0 con 0x55ee35174c30 2024-09-15T23:01:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e3b495740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ee350d00c0 con 0x55ee3516fba0 2024-09-15T23:01:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e35c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee3516fba0 0x55ee35174660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e35c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee3516fba0 0x55ee35174660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:58292/0 (socket says 172.21.15.111:58292) 2024-09-15T23:01:25.829 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e35c00640 1 -- 172.21.15.111:0/194788615 learned_addr learned my addr 172.21.15.111:0/194788615 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:01:25.830 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e35200640 1 --2- 172.21.15.111:0/194788615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee3516f260 0x55ee3516f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e35c00640 1 -- 172.21.15.111:0/194788615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee35174c30 msgr2=0x55ee351770f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e35c00640 1 --2- 172.21.15.111:0/194788615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee35174c30 0x55ee351770f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e35c00640 1 -- 172.21.15.111:0/194788615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee3516f260 msgr2=0x55ee3516f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e35c00640 1 --2- 172.21.15.111:0/194788615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee3516f260 0x55ee3516f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e35c00640 1 -- 172.21.15.111:0/194788615 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee34de7210 con 0x55ee3516fba0 2024-09-15T23:01:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e35200640 1 --2- 172.21.15.111:0/194788615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee3516f260 0x55ee3516f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:01:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e2e000640 1 --2- 172.21.15.111:0/1731084109 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9e180a3240 0x7f9e180a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e2e000640 1 -- 172.21.15.111:0/1731084109 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f9e280f01d0 con 0x7f9e30078790 2024-09-15T23:01:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e35200640 1 --2- 172.21.15.111:0/1731084109 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9e180a3240 0x7f9e180a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e35c00640 1 --2- 172.21.15.111:0/194788615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee3516fba0 0x55ee35174660 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9e24009bd0 tx=0x7f9e2400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55a829c84fff0beb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:01:25.831 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e22a00640 1 -- 172.21.15.111:0/194788615 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e24005020 con 0x55ee3516fba0 2024-09-15T23:01:25.832 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e22a00640 1 -- 172.21.15.111:0/194788615 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e240047a0 con 0x55ee3516fba0 2024-09-15T23:01:25.832 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e22a00640 1 -- 172.21.15.111:0/194788615 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e24010690 con 0x55ee3516fba0 2024-09-15T23:01:25.834 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e3b495740 1 -- 172.21.15.111:0/194788615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee3516fba0 msgr2=0x55ee35174660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e3b495740 1 --2- 172.21.15.111:0/194788615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee3516fba0 0x55ee35174660 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9e24009bd0 tx=0x7f9e2400e720 comp rx=0 tx=0).stop 2024-09-15T23:01:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e35200640 1 --2- 172.21.15.111:0/1731084109 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9e180a3240 0x7f9e180a5700 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c0096c0 tx=0x7f9e1c009e10 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:01:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e3b495740 1 -- 172.21.15.111:0/194788615 shutdown_connections 2024-09-15T23:01:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e3b495740 1 --2- 172.21.15.111:0/194788615 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee35174c30 0x55ee351770f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e3b495740 1 --2- 172.21.15.111:0/194788615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee3516fba0 0x55ee35174660 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e3b495740 1 --2- 172.21.15.111:0/194788615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee3516f260 0x55ee3516f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e3b495740 1 -- 172.21.15.111:0/194788615 >> 172.21.15.111:0/194788615 conn(0x55ee3516a8d0 msgr2=0x55ee3516acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:01:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e3b495740 1 -- 172.21.15.111:0/194788615 shutdown_connections 2024-09-15T23:01:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e3b495740 1 -- 172.21.15.111:0/194788615 wait complete. 2024-09-15T23:01:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e3b495740 1 -- start start 2024-09-15T23:01:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e3b495740 1 Processor -- start 2024-09-15T23:01:25.835 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e3b495740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee3516f260 0x55ee351793b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e3b495740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee3516fba0 0x55ee351798f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e3b495740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee35174c30 0x55ee35180930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e3b495740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ee35177c00 con 0x55ee35174c30 2024-09-15T23:01:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e3b495740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ee3505d090 con 0x55ee3516fba0 2024-09-15T23:01:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e3b495740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ee35177d80 con 0x55ee3516f260 2024-09-15T23:01:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e3b495740 10 client.0 _ll_register_callbacks cb 0x55ee350b1f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:01:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e35c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee3516f260 0x55ee351793b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.826+0000 7f9e36600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee35174c30 0x55ee35180930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e35c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee3516f260 0x55ee351793b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:58302/0 (socket says 172.21.15.111:58302) 2024-09-15T23:01:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e35c00640 1 -- 172.21.15.111:0/1791326288 learned_addr learned my addr 172.21.15.111:0/1791326288 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:01:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e36600640 1 -- 172.21.15.111:0/1791326288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee3516f260 msgr2=0x55ee351793b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e36600640 1 --2- 172.21.15.111:0/1791326288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee3516f260 0x55ee351793b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:25.836 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e36600640 1 -- 172.21.15.111:0/1791326288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee3516fba0 msgr2=0x55ee351798f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:01:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e36600640 1 --2- 172.21.15.111:0/1791326288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee3516fba0 0x55ee351798f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e36600640 1 -- 172.21.15.111:0/1791326288 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee34eb3620 con 0x55ee35174c30 2024-09-15T23:01:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e36600640 1 --2- 172.21.15.111:0/1791326288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee35174c30 0x55ee35180930 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9e280eddc0 tx=0x7f9e280bc1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941e1e89ba57d55e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:01:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e17400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:01:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e17400640 1 -- 172.21.15.111:0/1791326288 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e280a7070 con 0x55ee35174c30 2024-09-15T23:01:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e17400640 1 -- 172.21.15.111:0/1791326288 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f9e280a5be0 con 0x55ee35174c30 2024-09-15T23:01:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e3b495740 1 -- 172.21.15.111:0/1791326288 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e240047a0 con 0x55ee35174c30 2024-09-15T23:01:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e17400640 1 -- 172.21.15.111:0/1791326288 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9e280a5e20 con 0x55ee35174c30 2024-09-15T23:01:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e3b495740 10 client.16233 fetch_fsmap learned FSMap version 31 2024-09-15T23:01:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e3b495740 1 -- 172.21.15.111:0/1791326288 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ee35183720 con 0x55ee35174c30 2024-09-15T23:01:25.837 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e17400640 1 -- 172.21.15.111:0/1791326288 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e240047a0 con 0x55ee35174c30 2024-09-15T23:01:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e17400640 1 -- 172.21.15.111:0/1791326288 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f9e280c5070 con 0x55ee35174c30 2024-09-15T23:01:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e3b495740 10 client.16233 fetch_fsmap finished waiting for FSMap version 31 2024-09-15T23:01:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e3b495740 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:01:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e3b495740 20 client.16233 populate_metadata read hostname 'smithi111' 2024-09-15T23:01:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e3b495740 1 --2- 172.21.15.111:0/1791326288 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ee35184eb0 0x55ee35187370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:25.839 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e3b495740 4 client.16233 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:01:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e3b495740 1 -- 172.21.15.111:0/1791326288 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee350e1830 con 0x55ee35184eb0 2024-09-15T23:01:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e35c00640 1 --2- 172.21.15.111:0/1791326288 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ee35184eb0 0x55ee35187370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e35c00640 1 --2- 172.21.15.111:0/1791326288 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ee35184eb0 0x55ee35187370 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d277769bf83d8578 server_cookie=1e52dea0abe11e4c in_seq=0 out_seq=0 2024-09-15T23:01:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e17400640 10 client.16233 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:01:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e17400640 1 -- 172.21.15.111:0/1791326288 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee350e1830 con 0x55ee35184eb0 2024-09-15T23:01:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e17400640 10 client.16233 handle_command_reply: tid=0 2024-09-15T23:01:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e3b495740 1 client.16233 shutdown 2024-09-15T23:01:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e3b495740 1 -- 172.21.15.111:0/1791326288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee35174c30 msgr2=0x55ee35180930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e3b495740 1 --2- 172.21.15.111:0/1791326288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee35174c30 0x55ee35180930 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9e280eddc0 tx=0x7f9e280bc1f0 comp rx=0 tx=0).stop 2024-09-15T23:01:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e3b495740 1 -- 172.21.15.111:0/1791326288 shutdown_connections 2024-09-15T23:01:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e3b495740 1 --2- 172.21.15.111:0/1791326288 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ee35184eb0 0x55ee35187370 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e3b495740 1 --2- 172.21.15.111:0/1791326288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee35174c30 0x55ee35180930 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:25.840 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e3b495740 1 --2- 172.21.15.111:0/1791326288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee3516fba0 0x55ee351798f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e3b495740 1 --2- 172.21.15.111:0/1791326288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee3516f260 0x55ee351793b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e3b495740 1 -- 172.21.15.111:0/1791326288 >> 172.21.15.111:0/1791326288 conn(0x55ee3516a8d0 msgr2=0x55ee3516c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:01:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e17400640 0 client.16233 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:01:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e3b495740 1 -- 172.21.15.111:0/1791326288 shutdown_connections 2024-09-15T23:01:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.832+0000 7f9e3b495740 1 -- 172.21.15.111:0/1791326288 wait complete. 2024-09-15T23:01:25.841 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.837+0000 7f9e3b495740 20 client.16233 trim_cache size 0 max 16384 2024-09-15T23:01:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.855+0000 7f9e3b495740 1 Processor -- start 2024-09-15T23:01:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.855+0000 7f9e3b495740 1 -- start start 2024-09-15T23:01:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.860+0000 7f9e3b495740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee3516f260 0x55ee3516bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.860+0000 7f9e3b495740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee3516fba0 0x55ee351808f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.860+0000 7f9e3b495740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee35174c30 0x55ee3516c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.860+0000 7f9e3b495740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9e301b1b20 con 0x55ee3516fba0 2024-09-15T23:01:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.860+0000 7f9e3b495740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9e301af390 con 0x55ee3516f260 2024-09-15T23:01:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.860+0000 7f9e3b495740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9e301ad1e0 con 0x55ee35174c30 2024-09-15T23:01:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.860+0000 7f9e35c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee3516f260 0x55ee3516bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.860+0000 7f9e35c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee3516f260 0x55ee3516bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:49116/0 (socket says 172.21.15.111:49116) 2024-09-15T23:01:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.860+0000 7f9e35c00640 1 -- 172.21.15.111:0/3150446459 learned_addr learned my addr 172.21.15.111:0/3150446459 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:01:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.860+0000 7f9e35200640 1 --2- 172.21.15.111:0/3150446459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee3516fba0 0x55ee351808f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.860+0000 7f9e35c00640 1 -- 172.21.15.111:0/3150446459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee35174c30 msgr2=0x55ee3516c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.860+0000 7f9e36600640 1 --2- 172.21.15.111:0/3150446459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee35174c30 0x55ee3516c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.860+0000 7f9e35c00640 1 --2- 172.21.15.111:0/3150446459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee35174c30 0x55ee3516c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.860+0000 7f9e35c00640 1 -- 172.21.15.111:0/3150446459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee3516fba0 msgr2=0x55ee351808f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.860+0000 7f9e35c00640 1 --2- 172.21.15.111:0/3150446459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee3516fba0 0x55ee351808f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:25.863 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.860+0000 7f9e35c00640 1 -- 172.21.15.111:0/3150446459 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee3516a6f0 con 0x55ee3516f260 2024-09-15T23:01:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.860+0000 7f9e35200640 1 --2- 172.21.15.111:0/3150446459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee3516fba0 0x55ee351808f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:01:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.862+0000 7f9e35c00640 1 --2- 172.21.15.111:0/3150446459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee3516f260 0x55ee3516bcc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9e24004770 tx=0x7f9e240046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa352f0a0a7589ad server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:01:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.862+0000 7f9e22000640 1 -- 172.21.15.111:0/3150446459 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2401d040 con 0x55ee3516f260 2024-09-15T23:01:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.862+0000 7f9e22000640 1 -- 172.21.15.111:0/3150446459 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e24009cc0 con 0x55ee3516f260 2024-09-15T23:01:25.864 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.862+0000 7f9e22000640 1 -- 172.21.15.111:0/3150446459 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e24029550 con 0x55ee3516f260 2024-09-15T23:01:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.862+0000 7f9e3b495740 1 -- 172.21.15.111:0/3150446459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee3516f260 msgr2=0x55ee3516bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.862+0000 7f9e3b495740 1 --2- 172.21.15.111:0/3150446459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee3516f260 0x55ee3516bcc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9e24004770 tx=0x7f9e240046c0 comp rx=0 tx=0).stop 2024-09-15T23:01:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.862+0000 7f9e3b495740 1 -- 172.21.15.111:0/3150446459 shutdown_connections 2024-09-15T23:01:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.862+0000 7f9e3b495740 1 --2- 172.21.15.111:0/3150446459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee35174c30 0x55ee3516c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.862+0000 7f9e3b495740 1 --2- 172.21.15.111:0/3150446459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee3516fba0 0x55ee351808f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.862+0000 7f9e3b495740 1 --2- 172.21.15.111:0/3150446459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee3516f260 0x55ee3516bcc0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.862+0000 7f9e3b495740 1 -- 172.21.15.111:0/3150446459 >> 172.21.15.111:0/3150446459 conn(0x55ee3516a8d0 msgr2=0x55ee35169a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:01:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.862+0000 7f9e3b495740 1 -- 172.21.15.111:0/3150446459 shutdown_connections 2024-09-15T23:01:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.862+0000 7f9e3b495740 1 -- 172.21.15.111:0/3150446459 wait complete. 2024-09-15T23:01:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.862+0000 7f9e3b495740 1 -- start start 2024-09-15T23:01:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.867+0000 7f9e3b495740 1 Processor -- start 2024-09-15T23:01:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.867+0000 7f9e3b495740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee3516f260 0x55ee3517dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.867+0000 7f9e3b495740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee3516fba0 0x55ee3517e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.867+0000 7f9e3b495740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee35174c30 0x55ee3517f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.867+0000 7f9e3b495740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9e301ad850 con 0x55ee3516fba0 2024-09-15T23:01:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.867+0000 7f9e3b495740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9e301aecc0 con 0x55ee35174c30 2024-09-15T23:01:25.871 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.867+0000 7f9e3b495740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9e301adf20 con 0x55ee3516f260 2024-09-15T23:01:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.867+0000 7f9e35200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee3516fba0 0x55ee3517e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.867+0000 7f9e35200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee3516fba0 0x55ee3517e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47440/0 (socket says 172.21.15.111:47440) 2024-09-15T23:01:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.867+0000 7f9e35200640 1 -- 172.21.15.111:0/2480007150 learned_addr learned my addr 172.21.15.111:0/2480007150 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:01:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.867+0000 7f9e35c00640 1 --2- 172.21.15.111:0/2480007150 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee3516f260 0x55ee3517dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.867+0000 7f9e36600640 1 --2- 172.21.15.111:0/2480007150 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee35174c30 0x55ee3517f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.867+0000 7f9e3b495740 10 client.0 _ll_register_callbacks cb 0x7f9e280bc710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:01:25.872 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.870+0000 7f9e35200640 1 -- 172.21.15.111:0/2480007150 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee3516f260 msgr2=0x55ee3517dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.870+0000 7f9e35200640 1 --2- 172.21.15.111:0/2480007150 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee3516f260 0x55ee3517dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.870+0000 7f9e35200640 1 -- 172.21.15.111:0/2480007150 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee35174c30 msgr2=0x55ee3517f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.870+0000 7f9e35200640 1 --2- 172.21.15.111:0/2480007150 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee35174c30 0x55ee3517f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.870+0000 7f9e35200640 1 -- 172.21.15.111:0/2480007150 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee3517f6e0 con 0x55ee3516fba0 2024-09-15T23:01:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.870+0000 7f9e36600640 1 --2- 172.21.15.111:0/2480007150 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee35174c30 0x55ee3517f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:01:25.873 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.870+0000 7f9e35200640 1 --2- 172.21.15.111:0/2480007150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee3516fba0 0x55ee3517e160 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c005ab0 tx=0x7f9e1c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6197eb50f638f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:01:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.870+0000 7f9e17400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:01:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.872+0000 7f9e17400640 1 -- 172.21.15.111:0/2480007150 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c02d060 con 0x55ee3516fba0 2024-09-15T23:01:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.872+0000 7f9e17400640 1 -- 172.21.15.111:0/2480007150 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f9e1c018030 con 0x55ee3516fba0 2024-09-15T23:01:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.872+0000 7f9e3b495740 1 -- 172.21.15.111:0/2480007150 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e24009cc0 con 0x55ee3516fba0 2024-09-15T23:01:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.872+0000 7f9e17400640 1 -- 172.21.15.111:0/2480007150 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9e1c00fde0 con 0x55ee3516fba0 2024-09-15T23:01:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.872+0000 7f9e3b495740 10 client.16236 fetch_fsmap learned FSMap version 31 2024-09-15T23:01:25.874 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.872+0000 7f9e3b495740 1 -- 172.21.15.111:0/2480007150 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ee35180a60 con 0x55ee3516fba0 2024-09-15T23:01:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.872+0000 7f9e17400640 1 -- 172.21.15.111:0/2480007150 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e24009cc0 con 0x55ee3516fba0 2024-09-15T23:01:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.872+0000 7f9e17400640 1 -- 172.21.15.111:0/2480007150 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f9e1c032070 con 0x55ee3516fba0 2024-09-15T23:01:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.872+0000 7f9e3b495740 10 client.16236 fetch_fsmap finished waiting for FSMap version 31 2024-09-15T23:01:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.872+0000 7f9e3b495740 10 client.16236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:01:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.872+0000 7f9e3b495740 20 client.16236 populate_metadata read hostname 'smithi111' 2024-09-15T23:01:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.872+0000 7f9e3b495740 1 --2- 172.21.15.111:0/2480007150 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ee351f0280 0x55ee351f26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.872+0000 7f9e3b495740 4 client.16236 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:01:25.875 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.872+0000 7f9e3b495740 1 -- 172.21.15.111:0/2480007150 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ee350dbb80 con 0x55ee351f0280 2024-09-15T23:01:25.876 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.874+0000 7f9e35c00640 1 --2- 172.21.15.111:0/2480007150 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ee351f0280 0x55ee351f26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.886+0000 7f9e35c00640 1 --2- 172.21.15.111:0/2480007150 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ee351f0280 0x55ee351f26a0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d6bb58fdf0f851 server_cookie=2e2f634a998285b5 in_seq=0 out_seq=0 2024-09-15T23:01:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:25.886+0000 7f9e17400640 10 client.16236 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:01:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:26.072+0000 7f9e17400640 1 -- 172.21.15.111:0/2480007150 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ee350dbb80 con 0x55ee351f0280 2024-09-15T23:01:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:26.072+0000 7f9e17400640 10 client.16236 handle_command_reply: tid=0 2024-09-15T23:01:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:26.072+0000 7f9e3b495740 1 client.16236 shutdown 2024-09-15T23:01:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:26.072+0000 7f9e3b495740 1 -- 172.21.15.111:0/2480007150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee3516fba0 msgr2=0x55ee3517e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:26.072+0000 7f9e3b495740 1 --2- 172.21.15.111:0/2480007150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee3516fba0 0x55ee3517e160 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c005ab0 tx=0x7f9e1c004730 comp rx=0 tx=0).stop 2024-09-15T23:01:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:26.072+0000 7f9e3b495740 1 -- 172.21.15.111:0/2480007150 shutdown_connections 2024-09-15T23:01:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:26.072+0000 7f9e3b495740 1 --2- 172.21.15.111:0/2480007150 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ee351f0280 0x55ee351f26a0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:26.072+0000 7f9e3b495740 1 --2- 172.21.15.111:0/2480007150 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ee35174c30 0x55ee3517f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:26.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:26.072+0000 7f9e3b495740 1 --2- 172.21.15.111:0/2480007150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee3516fba0 0x55ee3517e160 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:26.072+0000 7f9e3b495740 1 --2- 172.21.15.111:0/2480007150 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ee3516f260 0x55ee3517dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:26.072+0000 7f9e3b495740 1 -- 172.21.15.111:0/2480007150 >> 172.21.15.111:0/2480007150 conn(0x55ee3516a8d0 msgr2=0x55ee3516acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:01:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:26.072+0000 7f9e17400640 0 client.16236 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:01:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:26.072+0000 7f9e3b495740 1 -- 172.21.15.111:0/2480007150 shutdown_connections 2024-09-15T23:01:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:26.072+0000 7f9e3b495740 1 -- 172.21.15.111:0/2480007150 wait complete. 2024-09-15T23:01:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:26.072+0000 7f9e3b495740 20 client.16236 trim_cache size 0 max 16384 2024-09-15T23:01:26.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:01:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:01:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:01:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:01:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:26.072+0000 7f9e37000640 1 -- 172.21.15.111:0/1731084109 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9e180a3240 msgr2=0x7f9e180a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:26.076 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:26.072+0000 7f9e37000640 1 --2- 172.21.15.111:0/1731084109 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9e180a3240 0x7f9e180a5700 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c0096c0 tx=0x7f9e1c009e10 comp rx=0 tx=0).stop 2024-09-15T23:01:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:26.072+0000 7f9e37000640 1 -- 172.21.15.111:0/1731084109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e30078790 msgr2=0x7f9e301a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:26.072+0000 7f9e37000640 1 --2- 172.21.15.111:0/1731084109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e30078790 0x7f9e301a5830 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9e28034040 tx=0x7f9e280305d0 comp rx=0 tx=0).stop 2024-09-15T23:01:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:26.072+0000 7f9e37000640 1 -- 172.21.15.111:0/1731084109 shutdown_connections 2024-09-15T23:01:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:26.072+0000 7f9e37000640 1 --2- 172.21.15.111:0/1731084109 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9e180a3240 0x7f9e180a5700 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:26.072+0000 7f9e37000640 1 --2- 172.21.15.111:0/1731084109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e30078790 0x7f9e301a5830 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:26.072+0000 7f9e37000640 1 --2- 172.21.15.111:0/1731084109 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e30077df0 0x7f9e3019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:26.072+0000 7f9e37000640 1 --2- 172.21.15.111:0/1731084109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e30076bb0 0x7f9e3019e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:26.072+0000 7f9e37000640 1 -- 172.21.15.111:0/1731084109 >> 172.21.15.111:0/1731084109 conn(0x7f9e300ff2f0 msgr2=0x7f9e30100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:01:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:26.072+0000 7f9e37000640 1 -- 172.21.15.111:0/1731084109 shutdown_connections 2024-09-15T23:01:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:26.072+0000 7f9e37000640 1 -- 172.21.15.111:0/1731084109 wait complete. 2024-09-15T23:01:26.146 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:25 smithi111 ceph-mon[30629]: pgmap v815: 97 pgs: 97 active+clean; 526 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 5.8 MiB/s wr, 603 op/s 2024-09-15T23:01:26.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:25 smithi192 ceph-mon[35427]: pgmap v815: 97 pgs: 97 active+clean; 526 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 5.8 MiB/s wr, 603 op/s 2024-09-15T23:01:26.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:25 smithi121 ceph-mon[34815]: pgmap v815: 97 pgs: 97 active+clean; 526 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 5.8 MiB/s wr, 603 op/s 2024-09-15T23:01:26.627 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48221 24.35 MB/sec warmup 114 sec latency 3813.371 ms 2024-09-15T23:01:27.627 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48221 24.14 MB/sec warmup 115 sec latency 4813.412 ms 2024-09-15T23:01:28.081 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:27 smithi111 ceph-mon[30629]: pgmap v816: 97 pgs: 97 active+clean; 535 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.6 MiB/s wr, 467 op/s 2024-09-15T23:01:28.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:27 smithi192 ceph-mon[35427]: pgmap v816: 97 pgs: 97 active+clean; 535 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.6 MiB/s wr, 467 op/s 2024-09-15T23:01:28.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:27 smithi121 ceph-mon[34815]: pgmap v816: 97 pgs: 97 active+clean; 535 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.6 MiB/s wr, 467 op/s 2024-09-15T23:01:28.634 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48231 23.93 MB/sec warmup 116 sec latency 5146.614 ms 2024-09-15T23:01:29.634 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48231 23.73 MB/sec warmup 117 sec latency 3533.515 ms 2024-09-15T23:01:30.084 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:29 smithi111 ceph-mon[30629]: pgmap v817: 97 pgs: 97 active+clean; 539 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.0 MiB/s wr, 504 op/s 2024-09-15T23:01:30.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:29 smithi192 ceph-mon[35427]: pgmap v817: 97 pgs: 97 active+clean; 539 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.0 MiB/s wr, 504 op/s 2024-09-15T23:01:30.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:29 smithi121 ceph-mon[34815]: pgmap v817: 97 pgs: 97 active+clean; 539 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.0 MiB/s wr, 504 op/s 2024-09-15T23:01:30.634 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48232 23.53 MB/sec warmup 118 sec latency 4101.606 ms 2024-09-15T23:01:31.634 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48239 23.33 MB/sec warmup 119 sec latency 3672.039 ms 2024-09-15T23:01:32.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:31 smithi192 ceph-mon[35427]: pgmap v818: 97 pgs: 97 active+clean; 529 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.3 MiB/s wr, 495 op/s 2024-09-15T23:01:32.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:31 smithi121 ceph-mon[34815]: pgmap v818: 97 pgs: 97 active+clean; 529 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.3 MiB/s wr, 495 op/s 2024-09-15T23:01:32.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:31 smithi111 ceph-mon[30629]: pgmap v818: 97 pgs: 97 active+clean; 529 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.3 MiB/s wr, 495 op/s 2024-09-15T23:01:33.634 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48239 0.00 MB/sec execute 1 sec latency 5672.107 ms 2024-09-15T23:01:34.212 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:33 smithi111 ceph-mon[30629]: pgmap v819: 97 pgs: 97 active+clean; 525 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.1 MiB/s wr, 432 op/s 2024-09-15T23:01:34.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:33 smithi192 ceph-mon[35427]: pgmap v819: 97 pgs: 97 active+clean; 525 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.1 MiB/s wr, 432 op/s 2024-09-15T23:01:34.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:33 smithi121 ceph-mon[34815]: pgmap v819: 97 pgs: 97 active+clean; 525 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.1 MiB/s wr, 432 op/s 2024-09-15T23:01:34.634 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48240 0.00 MB/sec execute 2 sec latency 6672.181 ms 2024-09-15T23:01:35.634 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48240 0.00 MB/sec execute 3 sec latency 7672.254 ms 2024-09-15T23:01:36.208 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:35 smithi111 ceph-mon[30629]: pgmap v820: 97 pgs: 97 active+clean; 522 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.0 MiB/s wr, 407 op/s 2024-09-15T23:01:36.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:35 smithi192 ceph-mon[35427]: pgmap v820: 97 pgs: 97 active+clean; 522 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.0 MiB/s wr, 407 op/s 2024-09-15T23:01:36.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:35 smithi121 ceph-mon[34815]: pgmap v820: 97 pgs: 97 active+clean; 522 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.0 MiB/s wr, 407 op/s 2024-09-15T23:01:36.634 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48240 0.00 MB/sec execute 4 sec latency 8672.322 ms 2024-09-15T23:01:37.047 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:36 smithi111 ceph-mon[30629]: pgmap v821: 97 pgs: 97 active+clean; 522 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 6.5 MiB/s wr, 335 op/s 2024-09-15T23:01:37.291 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:36 smithi121 ceph-mon[34815]: pgmap v821: 97 pgs: 97 active+clean; 522 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 6.5 MiB/s wr, 335 op/s 2024-09-15T23:01:37.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:36 smithi192 ceph-mon[35427]: pgmap v821: 97 pgs: 97 active+clean; 522 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 6.5 MiB/s wr, 335 op/s 2024-09-15T23:01:37.634 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48240 0.00 MB/sec execute 5 sec latency 9672.389 ms 2024-09-15T23:01:38.634 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48240 0.00 MB/sec execute 6 sec latency 10672.446 ms 2024-09-15T23:01:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0738a00640 1 -- 172.21.15.111:0/3593946503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0734105910 msgr2=0x7f073410c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0738a00640 1 --2- 172.21.15.111:0/3593946503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0734105910 0x7f073410c1a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f072c009be0 tx=0x7f072c02a6d0 comp rx=0 tx=0).stop 2024-09-15T23:01:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0738a00640 1 -- 172.21.15.111:0/3593946503 shutdown_connections 2024-09-15T23:01:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0738a00640 1 --2- 172.21.15.111:0/3593946503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0734105910 0x7f073410c1a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0738a00640 1 --2- 172.21.15.111:0/3593946503 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0734104f50 0x7f07341053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0738a00640 1 --2- 172.21.15.111:0/3593946503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0734103d60 0x7f0734104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0738a00640 1 -- 172.21.15.111:0/3593946503 >> 172.21.15.111:0/3593946503 conn(0x7f07340ff4e0 msgr2=0x7f0734101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:01:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0738a00640 1 -- 172.21.15.111:0/3593946503 shutdown_connections 2024-09-15T23:01:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0738a00640 1 -- 172.21.15.111:0/3593946503 wait complete. 2024-09-15T23:01:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0738a00640 1 Processor -- start 2024-09-15T23:01:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0738a00640 1 -- start start 2024-09-15T23:01:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0738a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0734103d60 0x7f0734199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0738a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0734104f50 0x7f073419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:39.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0738a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0734105910 0x7f07341a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0738a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f073410f9e0 con 0x7f0734103d60 2024-09-15T23:01:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0738a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f073410f860 con 0x7f0734105910 2024-09-15T23:01:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0738a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f073410fb60 con 0x7f0734104f50 2024-09-15T23:01:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0731400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0734104f50 0x7f073419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0731e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0734103d60 0x7f0734199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0731400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0734104f50 0x7f073419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:36936/0 (socket says 172.21.15.111:36936) 2024-09-15T23:01:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0731400640 1 -- 172.21.15.111:0/36395816 learned_addr learned my addr 172.21.15.111:0/36395816 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:01:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0732800640 1 --2- 172.21.15.111:0/36395816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0734105910 0x7f07341a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0731400640 1 -- 172.21.15.111:0/36395816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0734105910 msgr2=0x7f07341a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0731400640 1 --2- 172.21.15.111:0/36395816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0734105910 0x7f07341a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:39.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0731400640 1 -- 172.21.15.111:0/36395816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0734103d60 msgr2=0x7f0734199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0731400640 1 --2- 172.21.15.111:0/36395816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0734103d60 0x7f0734199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0731400640 1 -- 172.21.15.111:0/36395816 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07341a1b30 con 0x7f0734104f50 2024-09-15T23:01:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.189+0000 7f0731e00640 1 --2- 172.21.15.111:0/36395816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0734103d60 0x7f0734199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:01:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.192+0000 7f0731400640 1 --2- 172.21.15.111:0/36395816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0734104f50 0x7f073419a450 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f072000e9c0 tx=0x7f072000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:01:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.192+0000 7f072aa00640 1 -- 172.21.15.111:0/36395816 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072000cde0 con 0x7f0734104f50 2024-09-15T23:01:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.192+0000 7f072aa00640 1 -- 172.21.15.111:0/36395816 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07200035f0 con 0x7f0734104f50 2024-09-15T23:01:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.192+0000 7f0738a00640 1 -- 172.21.15.111:0/36395816 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07341a1e20 con 0x7f0734104f50 2024-09-15T23:01:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.192+0000 7f072aa00640 1 -- 172.21.15.111:0/36395816 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0720010690 con 0x7f0734104f50 2024-09-15T23:01:39.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.192+0000 7f0738a00640 1 -- 172.21.15.111:0/36395816 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0734076630 con 0x7f0734104f50 2024-09-15T23:01:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f073ba59740 1 Processor -- start 2024-09-15T23:01:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f073ba59740 1 -- start start 2024-09-15T23:01:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f072aa00640 1 -- 172.21.15.111:0/36395816 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f07200037a0 con 0x7f0734104f50 2024-09-15T23:01:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f073ba59740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56042d078090 0x56042d078490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f073ba59740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56042d0789d0 0x56042d07d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f073ba59740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042d07da60 0x56042d07ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f073ba59740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56042cfdd100 con 0x56042d07da60 2024-09-15T23:01:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f073ba59740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56042cf851b0 con 0x56042d078090 2024-09-15T23:01:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f073ba59740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56042cfd90c0 con 0x56042d0789d0 2024-09-15T23:01:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f0731e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56042d0789d0 0x56042d07d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f0731e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56042d0789d0 0x56042d07d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:36944/0 (socket says 172.21.15.111:36944) 2024-09-15T23:01:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f0731e00640 1 -- 172.21.15.111:0/1754186168 learned_addr learned my addr 172.21.15.111:0/1754186168 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:01:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f0731400640 1 --2- 172.21.15.111:0/1754186168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042d07da60 0x56042d07ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f0731e00640 1 -- 172.21.15.111:0/1754186168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56042d078090 msgr2=0x56042d078490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:01:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f0731e00640 1 --2- 172.21.15.111:0/1754186168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56042d078090 0x56042d078490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f0731e00640 1 -- 172.21.15.111:0/1754186168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042d07da60 msgr2=0x56042d07ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f0731e00640 1 --2- 172.21.15.111:0/1754186168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042d07da60 0x56042d07ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f0731e00640 1 -- 172.21.15.111:0/1754186168 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56042ccf0210 con 0x56042d0789d0 2024-09-15T23:01:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f0731e00640 1 --2- 172.21.15.111:0/1754186168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56042d0789d0 0x56042d07d490 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f07240114f0 tx=0x7f07240119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bba8f9fa36c63f98 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:01:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f071f000640 1 -- 172.21.15.111:0/1754186168 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072401e070 con 0x56042d0789d0 2024-09-15T23:01:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f071f000640 1 -- 172.21.15.111:0/1754186168 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07240035f0 con 0x56042d0789d0 2024-09-15T23:01:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f071f000640 1 -- 172.21.15.111:0/1754186168 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072400f780 con 0x56042d0789d0 2024-09-15T23:01:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f072aa00640 1 --2- 172.21.15.111:0/36395816 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f07180a32f0 0x7f07180a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f072aa00640 1 -- 172.21.15.111:0/36395816 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f07200cfc30 con 0x7f0734104f50 2024-09-15T23:01:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f073ba59740 1 -- 172.21.15.111:0/1754186168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56042d0789d0 msgr2=0x56042d07d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f073ba59740 1 --2- 172.21.15.111:0/1754186168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56042d0789d0 0x56042d07d490 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f07240114f0 tx=0x7f07240119b0 comp rx=0 tx=0).stop 2024-09-15T23:01:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f0732800640 1 --2- 172.21.15.111:0/36395816 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f07180a32f0 0x7f07180a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f073ba59740 1 -- 172.21.15.111:0/1754186168 shutdown_connections 2024-09-15T23:01:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f073ba59740 1 --2- 172.21.15.111:0/1754186168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042d07da60 0x56042d07ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f073ba59740 1 --2- 172.21.15.111:0/1754186168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56042d0789d0 0x56042d07d490 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f073ba59740 1 --2- 172.21.15.111:0/1754186168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56042d078090 0x56042d078490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f073ba59740 1 -- 172.21.15.111:0/1754186168 >> 172.21.15.111:0/1754186168 conn(0x56042d073700 msgr2=0x56042d073b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:01:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f073ba59740 1 -- 172.21.15.111:0/1754186168 shutdown_connections 2024-09-15T23:01:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f073ba59740 1 -- 172.21.15.111:0/1754186168 wait complete. 2024-09-15T23:01:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f0732800640 1 --2- 172.21.15.111:0/36395816 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f07180a32f0 0x7f07180a57b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f072c02abf0 tx=0x7f072c005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:01:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f073ba59740 1 -- start start 2024-09-15T23:01:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f073ba59740 1 Processor -- start 2024-09-15T23:01:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f073ba59740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042d078090 0x56042d0822a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f073ba59740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56042d0789d0 0x56042d0827e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f073ba59740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56042d07da60 0x56042d089820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f073ba59740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56042d080a30 con 0x56042d078090 2024-09-15T23:01:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f073ba59740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56042cf66090 con 0x56042d07da60 2024-09-15T23:01:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f073ba59740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56042d080bb0 con 0x56042d0789d0 2024-09-15T23:01:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f073ba59740 10 client.0 _ll_register_callbacks cb 0x56042cfbaf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:01:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f0731400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56042d0789d0 0x56042d0827e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f0731e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042d078090 0x56042d0822a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:39.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f0732800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56042d07da60 0x56042d089820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f0731400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56042d0789d0 0x56042d0827e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:36954/0 (socket says 172.21.15.111:36954) 2024-09-15T23:01:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f0731400640 1 -- 172.21.15.111:0/3363957736 learned_addr learned my addr 172.21.15.111:0/3363957736 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:01:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f0731400640 1 -- 172.21.15.111:0/3363957736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56042d07da60 msgr2=0x56042d089820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f0731400640 1 --2- 172.21.15.111:0/3363957736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56042d07da60 0x56042d089820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f0731400640 1 -- 172.21.15.111:0/3363957736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042d078090 msgr2=0x56042d0822a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f0731400640 1 --2- 172.21.15.111:0/3363957736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042d078090 0x56042d0822a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.195+0000 7f0731400640 1 -- 172.21.15.111:0/3363957736 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56042cdbc620 con 0x56042d0789d0 2024-09-15T23:01:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f0731400640 1 --2- 172.21.15.111:0/3363957736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56042d0789d0 0x56042d0827e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f07200a4990 tx=0x7f072009bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78853b3003ce6fbe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:01:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f0713400640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:01:39.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f0713400640 1 -- 172.21.15.111:0/3363957736 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072009c250 con 0x56042d0789d0 2024-09-15T23:01:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f0713400640 1 -- 172.21.15.111:0/3363957736 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f07200bc0f0 con 0x56042d0789d0 2024-09-15T23:01:39.208 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f073ba59740 1 -- 172.21.15.111:0/3363957736 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07240035f0 con 0x56042d0789d0 2024-09-15T23:01:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f0713400640 1 -- 172.21.15.111:0/3363957736 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f07200a0600 con 0x56042d0789d0 2024-09-15T23:01:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f073ba59740 10 client.26215 fetch_fsmap learned FSMap version 31 2024-09-15T23:01:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f073ba59740 1 -- 172.21.15.111:0/3363957736 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56042d08c6d0 con 0x56042d0789d0 2024-09-15T23:01:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f0713400640 1 -- 172.21.15.111:0/3363957736 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07240035f0 con 0x56042d0789d0 2024-09-15T23:01:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f0713400640 1 -- 172.21.15.111:0/3363957736 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f07200a0200 con 0x56042d0789d0 2024-09-15T23:01:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f073ba59740 10 client.26215 fetch_fsmap finished waiting for FSMap version 31 2024-09-15T23:01:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f073ba59740 10 client.26215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:01:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f073ba59740 20 client.26215 populate_metadata read hostname 'smithi111' 2024-09-15T23:01:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f073ba59740 1 --2- 172.21.15.111:0/3363957736 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56042d08dce0 0x56042d0900d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f073ba59740 4 client.26215 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:01:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f073ba59740 1 -- 172.21.15.111:0/3363957736 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56042cfea830 con 0x56042d08dce0 2024-09-15T23:01:39.209 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f0731e00640 1 --2- 172.21.15.111:0/3363957736 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56042d08dce0 0x56042d0900d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f0713400640 10 client.26215 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:01:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f0731e00640 1 --2- 172.21.15.111:0/3363957736 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56042d08dce0 0x56042d0900d0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbf40b29f6452a3f server_cookie=fc1a01ea2e836204 in_seq=0 out_seq=0 2024-09-15T23:01:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f0713400640 1 -- 172.21.15.111:0/3363957736 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56042cfea830 con 0x56042d08dce0 2024-09-15T23:01:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f0713400640 10 client.26215 handle_command_reply: tid=0 2024-09-15T23:01:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f073ba59740 1 client.26215 shutdown 2024-09-15T23:01:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f073ba59740 1 -- 172.21.15.111:0/3363957736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56042d0789d0 msgr2=0x56042d0827e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f073ba59740 1 --2- 172.21.15.111:0/3363957736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56042d0789d0 0x56042d0827e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f07200a4990 tx=0x7f072009bd90 comp rx=0 tx=0).stop 2024-09-15T23:01:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f073ba59740 1 -- 172.21.15.111:0/3363957736 shutdown_connections 2024-09-15T23:01:39.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f073ba59740 1 --2- 172.21.15.111:0/3363957736 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56042d08dce0 0x56042d0900d0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f073ba59740 1 --2- 172.21.15.111:0/3363957736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56042d07da60 0x56042d089820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f0713400640 0 client.26215 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:01:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f073ba59740 1 --2- 172.21.15.111:0/3363957736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56042d0789d0 0x56042d0827e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f073ba59740 1 --2- 172.21.15.111:0/3363957736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042d078090 0x56042d0822a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f073ba59740 1 -- 172.21.15.111:0/3363957736 >> 172.21.15.111:0/3363957736 conn(0x56042d073700 msgr2=0x56042d074f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:01:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f073ba59740 1 -- 172.21.15.111:0/3363957736 shutdown_connections 2024-09-15T23:01:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f073ba59740 1 -- 172.21.15.111:0/3363957736 wait complete. 2024-09-15T23:01:39.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.200+0000 7f073ba59740 20 client.26215 trim_cache size 0 max 16384 2024-09-15T23:01:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.218+0000 7f073ba59740 1 Processor -- start 2024-09-15T23:01:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.218+0000 7f073ba59740 1 -- start start 2024-09-15T23:01:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.218+0000 7f073ba59740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042d073700 0x56042d080f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.218+0000 7f073ba59740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56042d078090 0x56042d0897e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.218+0000 7f073ba59740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56042d0789d0 0x56042d0745b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.218+0000 7f073ba59740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f07341a93f0 con 0x56042d073700 2024-09-15T23:01:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.218+0000 7f073ba59740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f07341a6c60 con 0x56042d0789d0 2024-09-15T23:01:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.218+0000 7f073ba59740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f07341a4af0 con 0x56042d078090 2024-09-15T23:01:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0731e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042d073700 0x56042d080f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0731400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56042d078090 0x56042d0897e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:39.221 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0731e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042d073700 0x56042d080f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55568/0 (socket says 172.21.15.111:55568) 2024-09-15T23:01:39.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0731e00640 1 -- 172.21.15.111:0/1966905941 learned_addr learned my addr 172.21.15.111:0/1966905941 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:01:39.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0732800640 1 --2- 172.21.15.111:0/1966905941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56042d0789d0 0x56042d0745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:39.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0731e00640 1 -- 172.21.15.111:0/1966905941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56042d078090 msgr2=0x56042d0897e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:39.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0731e00640 1 --2- 172.21.15.111:0/1966905941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56042d078090 0x56042d0897e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:39.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0731e00640 1 -- 172.21.15.111:0/1966905941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56042d0789d0 msgr2=0x56042d0745b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:39.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0731e00640 1 --2- 172.21.15.111:0/1966905941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56042d0789d0 0x56042d0745b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:39.222 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0731e00640 1 -- 172.21.15.111:0/1966905941 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56042d07fe50 con 0x56042d073700 2024-09-15T23:01:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0731e00640 1 --2- 172.21.15.111:0/1966905941 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042d073700 0x56042d080f80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0724014040 tx=0x7f0724003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e44aeaf455a4f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:01:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f071e600640 1 -- 172.21.15.111:0/1966905941 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072401e040 con 0x56042d073700 2024-09-15T23:01:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f071e600640 1 -- 172.21.15.111:0/1966905941 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f072402a070 con 0x56042d073700 2024-09-15T23:01:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f071e600640 1 -- 172.21.15.111:0/1966905941 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072400fcd0 con 0x56042d073700 2024-09-15T23:01:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f073ba59740 1 -- 172.21.15.111:0/1966905941 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042d073700 msgr2=0x56042d080f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f073ba59740 1 --2- 172.21.15.111:0/1966905941 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042d073700 0x56042d080f80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0724014040 tx=0x7f0724003780 comp rx=0 tx=0).stop 2024-09-15T23:01:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f073ba59740 1 -- 172.21.15.111:0/1966905941 shutdown_connections 2024-09-15T23:01:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f073ba59740 1 --2- 172.21.15.111:0/1966905941 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56042d0789d0 0x56042d0745b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:39.225 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f073ba59740 1 --2- 172.21.15.111:0/1966905941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56042d078090 0x56042d0897e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:39.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f073ba59740 1 --2- 172.21.15.111:0/1966905941 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042d073700 0x56042d080f80 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:39.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f073ba59740 1 -- 172.21.15.111:0/1966905941 >> 172.21.15.111:0/1966905941 conn(0x56042d07d5f0 msgr2=0x56042d079c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:01:39.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f073ba59740 1 -- 172.21.15.111:0/1966905941 shutdown_connections 2024-09-15T23:01:39.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f073ba59740 1 -- 172.21.15.111:0/1966905941 wait complete. 2024-09-15T23:01:39.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f073ba59740 1 -- start start 2024-09-15T23:01:39.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f073ba59740 1 Processor -- start 2024-09-15T23:01:39.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f073ba59740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56042d073700 0x56042d086ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:39.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f073ba59740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042d078090 0x56042d0871e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:39.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f073ba59740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56042d0789d0 0x56042d0881a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:39.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f073ba59740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f07341a5140 con 0x56042d078090 2024-09-15T23:01:39.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f073ba59740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f07341a6590 con 0x56042d0789d0 2024-09-15T23:01:39.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f073ba59740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f07341a57f0 con 0x56042d073700 2024-09-15T23:01:39.226 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f073ba59740 10 client.0 _ll_register_callbacks cb 0x7f07200984f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:01:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0731e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56042d073700 0x56042d086ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0731400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042d078090 0x56042d0871e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0732800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56042d0789d0 0x56042d0881a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0731e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56042d073700 0x56042d086ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:36972/0 (socket says 172.21.15.111:36972) 2024-09-15T23:01:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0731e00640 1 -- 172.21.15.111:0/4219744117 learned_addr learned my addr 172.21.15.111:0/4219744117 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:01:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0731400640 1 -- 172.21.15.111:0/4219744117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56042d073700 msgr2=0x56042d086ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0731400640 1 --2- 172.21.15.111:0/4219744117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56042d073700 0x56042d086ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0731400640 1 -- 172.21.15.111:0/4219744117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56042d0789d0 msgr2=0x56042d0881a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0731400640 1 --2- 172.21.15.111:0/4219744117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56042d0789d0 0x56042d0881a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0731400640 1 -- 172.21.15.111:0/4219744117 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56042d0887b0 con 0x56042d078090 2024-09-15T23:01:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0731e00640 1 --2- 172.21.15.111:0/4219744117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56042d073700 0x56042d086ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:01:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0732800640 1 --2- 172.21.15.111:0/4219744117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56042d0789d0 0x56042d0881a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:01:39.227 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0731400640 1 --2- 172.21.15.111:0/4219744117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042d078090 0x56042d0871e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f072009c1e0 tx=0x7f0720005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e011d99e678879a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:01:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0713400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:01:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0713400640 1 -- 172.21.15.111:0/4219744117 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07200bc2f0 con 0x56042d078090 2024-09-15T23:01:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f073ba59740 1 -- 172.21.15.111:0/4219744117 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f072402a070 con 0x56042d078090 2024-09-15T23:01:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0713400640 1 -- 172.21.15.111:0/4219744117 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f072009b6a0 con 0x56042d078090 2024-09-15T23:01:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0713400640 1 -- 172.21.15.111:0/4219744117 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f072009c250 con 0x56042d078090 2024-09-15T23:01:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f073ba59740 10 client.16254 fetch_fsmap learned FSMap version 31 2024-09-15T23:01:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f073ba59740 1 -- 172.21.15.111:0/4219744117 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56042d089ac0 con 0x56042d078090 2024-09-15T23:01:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0713400640 1 -- 172.21.15.111:0/4219744117 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f072402a070 con 0x56042d078090 2024-09-15T23:01:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0713400640 1 -- 172.21.15.111:0/4219744117 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f07200980a0 con 0x56042d078090 2024-09-15T23:01:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f073ba59740 10 client.16254 fetch_fsmap finished waiting for FSMap version 31 2024-09-15T23:01:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f073ba59740 10 client.16254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:01:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f073ba59740 20 client.16254 populate_metadata read hostname 'smithi111' 2024-09-15T23:01:39.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f073ba59740 1 --2- 172.21.15.111:0/4219744117 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56042d07da60 0x56042d0fad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f073ba59740 4 client.16254 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:01:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f073ba59740 1 -- 172.21.15.111:0/4219744117 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56042d06ed80 con 0x56042d07da60 2024-09-15T23:01:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0731e00640 1 --2- 172.21.15.111:0/4219744117 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56042d07da60 0x56042d0fad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0731e00640 1 --2- 172.21.15.111:0/4219744117 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56042d07da60 0x56042d0fad20 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee71c491a47d0d47 server_cookie=1d7484a323906ee in_seq=0 out_seq=0 2024-09-15T23:01:39.229 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.220+0000 7f0713400640 10 client.16254 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:01:39.634 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48240 0.00 MB/sec execute 7 sec latency 11672.533 ms 2024-09-15T23:01:39.909 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:39 smithi111 ceph-mon[30629]: pgmap v822: 97 pgs: 97 active+clean; 536 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 6.9 MiB/s wr, 331 op/s 2024-09-15T23:01:39.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.991+0000 7f0713400640 1 -- 172.21.15.111:0/4219744117 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56042d06ed80 con 0x56042d07da60 2024-09-15T23:01:39.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.991+0000 7f0713400640 10 client.16254 handle_command_reply: tid=0 2024-09-15T23:01:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.991+0000 7f073ba59740 1 client.16254 shutdown 2024-09-15T23:01:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.996+0000 7f073ba59740 1 -- 172.21.15.111:0/4219744117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042d078090 msgr2=0x56042d0871e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.996+0000 7f073ba59740 1 --2- 172.21.15.111:0/4219744117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042d078090 0x56042d0871e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f072009c1e0 tx=0x7f0720005b70 comp rx=0 tx=0).stop 2024-09-15T23:01:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.996+0000 7f073ba59740 1 -- 172.21.15.111:0/4219744117 shutdown_connections 2024-09-15T23:01:39.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.996+0000 7f073ba59740 1 --2- 172.21.15.111:0/4219744117 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56042d07da60 0x56042d0fad20 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:40.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.996+0000 7f073ba59740 1 --2- 172.21.15.111:0/4219744117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56042d0789d0 0x56042d0881a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:40.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.996+0000 7f073ba59740 1 --2- 172.21.15.111:0/4219744117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56042d078090 0x56042d0871e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:40.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.996+0000 7f0713400640 0 client.16254 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:01:40.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.996+0000 7f073ba59740 1 --2- 172.21.15.111:0/4219744117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56042d073700 0x56042d086ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:40.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.996+0000 7f073ba59740 1 -- 172.21.15.111:0/4219744117 >> 172.21.15.111:0/4219744117 conn(0x56042d07d5f0 msgr2=0x56042d072060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:01:40.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.996+0000 7f073ba59740 1 -- 172.21.15.111:0/4219744117 shutdown_connections 2024-09-15T23:01:40.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.996+0000 7f073ba59740 1 -- 172.21.15.111:0/4219744117 wait complete. 2024-09-15T23:01:40.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.996+0000 7f073ba59740 20 client.16254 trim_cache size 0 max 16384 2024-09-15T23:01:40.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:01:40.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:01:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:01:40.002 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:01:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.996+0000 7f0738a00640 1 -- 172.21.15.111:0/36395816 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f07180a32f0 msgr2=0x7f07180a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.996+0000 7f0738a00640 1 --2- 172.21.15.111:0/36395816 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f07180a32f0 0x7f07180a57b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f072c02abf0 tx=0x7f072c005fd0 comp rx=0 tx=0).stop 2024-09-15T23:01:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.996+0000 7f0738a00640 1 -- 172.21.15.111:0/36395816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0734104f50 msgr2=0x7f073419a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.996+0000 7f0738a00640 1 --2- 172.21.15.111:0/36395816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0734104f50 0x7f073419a450 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f072000e9c0 tx=0x7f072000ee80 comp rx=0 tx=0).stop 2024-09-15T23:01:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.997+0000 7f0738a00640 1 -- 172.21.15.111:0/36395816 shutdown_connections 2024-09-15T23:01:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.997+0000 7f0738a00640 1 --2- 172.21.15.111:0/36395816 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f07180a32f0 0x7f07180a57b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:40.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.997+0000 7f0738a00640 1 --2- 172.21.15.111:0/36395816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0734105910 0x7f07341a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:40.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.997+0000 7f0738a00640 1 --2- 172.21.15.111:0/36395816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0734104f50 0x7f073419a450 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:40.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.997+0000 7f0738a00640 1 --2- 172.21.15.111:0/36395816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0734103d60 0x7f0734199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:40.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.997+0000 7f0738a00640 1 -- 172.21.15.111:0/36395816 >> 172.21.15.111:0/36395816 conn(0x7f07340ff4e0 msgr2=0x7f07341081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:01:40.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.997+0000 7f0738a00640 1 -- 172.21.15.111:0/36395816 shutdown_connections 2024-09-15T23:01:40.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:39.997+0000 7f0738a00640 1 -- 172.21.15.111:0/36395816 wait complete. 2024-09-15T23:01:40.140 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:39 smithi121 ceph-mon[34815]: pgmap v822: 97 pgs: 97 active+clean; 536 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 6.9 MiB/s wr, 331 op/s 2024-09-15T23:01:40.317 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:39 smithi192 ceph-mon[35427]: pgmap v822: 97 pgs: 97 active+clean; 536 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 6.9 MiB/s wr, 331 op/s 2024-09-15T23:01:40.635 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48240 0.00 MB/sec execute 8 sec latency 12672.603 ms 2024-09-15T23:01:41.634 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48240 0.00 MB/sec execute 9 sec latency 13672.673 ms 2024-09-15T23:01:41.906 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:41 smithi111 ceph-mon[30629]: pgmap v823: 97 pgs: 97 active+clean; 541 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 6.0 MiB/s wr, 240 op/s 2024-09-15T23:01:42.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:41 smithi192 ceph-mon[35427]: pgmap v823: 97 pgs: 97 active+clean; 541 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 6.0 MiB/s wr, 240 op/s 2024-09-15T23:01:42.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:41 smithi121 ceph-mon[34815]: pgmap v823: 97 pgs: 97 active+clean; 541 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 6.0 MiB/s wr, 240 op/s 2024-09-15T23:01:42.635 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48242 0.00 MB/sec execute 10 sec latency 14672.758 ms 2024-09-15T23:01:43.635 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48242 0.00 MB/sec execute 11 sec latency 15672.826 ms 2024-09-15T23:01:44.149 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:43 smithi111 ceph-mon[30629]: pgmap v824: 97 pgs: 97 active+clean; 535 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 5.5 MiB/s wr, 194 op/s 2024-09-15T23:01:44.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:43 smithi192 ceph-mon[35427]: pgmap v824: 97 pgs: 97 active+clean; 535 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 5.5 MiB/s wr, 194 op/s 2024-09-15T23:01:44.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:43 smithi121 ceph-mon[34815]: pgmap v824: 97 pgs: 97 active+clean; 535 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 5.5 MiB/s wr, 194 op/s 2024-09-15T23:01:44.635 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48242 0.00 MB/sec execute 12 sec latency 16672.900 ms 2024-09-15T23:01:45.635 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48242 0.00 MB/sec execute 13 sec latency 17672.968 ms 2024-09-15T23:01:46.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:45 smithi111 ceph-mon[30629]: pgmap v825: 97 pgs: 97 active+clean; 524 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 5.0 MiB/s wr, 193 op/s 2024-09-15T23:01:46.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:45 smithi192 ceph-mon[35427]: pgmap v825: 97 pgs: 97 active+clean; 524 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 5.0 MiB/s wr, 193 op/s 2024-09-15T23:01:46.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:45 smithi121 ceph-mon[34815]: pgmap v825: 97 pgs: 97 active+clean; 524 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 5.0 MiB/s wr, 193 op/s 2024-09-15T23:01:46.635 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48243 0.00 MB/sec execute 14 sec latency 18673.036 ms 2024-09-15T23:01:47.635 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48243 0.00 MB/sec execute 15 sec latency 19673.097 ms 2024-09-15T23:01:48.139 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:47 smithi111 ceph-mon[30629]: pgmap v826: 97 pgs: 97 active+clean; 518 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 4.9 MiB/s wr, 194 op/s 2024-09-15T23:01:48.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:47 smithi192 ceph-mon[35427]: pgmap v826: 97 pgs: 97 active+clean; 518 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 4.9 MiB/s wr, 194 op/s 2024-09-15T23:01:48.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:47 smithi121 ceph-mon[34815]: pgmap v826: 97 pgs: 97 active+clean; 518 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 4.9 MiB/s wr, 194 op/s 2024-09-15T23:01:48.635 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48243 0.00 MB/sec execute 16 sec latency 20673.140 ms 2024-09-15T23:01:49.635 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48243 0.00 MB/sec execute 17 sec latency 21673.189 ms 2024-09-15T23:01:50.195 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:49 smithi111 ceph-mon[30629]: pgmap v827: 97 pgs: 97 active+clean; 524 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 5.9 MiB/s wr, 213 op/s 2024-09-15T23:01:50.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:49 smithi192 ceph-mon[35427]: pgmap v827: 97 pgs: 97 active+clean; 524 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 5.9 MiB/s wr, 213 op/s 2024-09-15T23:01:50.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:49 smithi121 ceph-mon[34815]: pgmap v827: 97 pgs: 97 active+clean; 524 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 5.9 MiB/s wr, 213 op/s 2024-09-15T23:01:50.635 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48243 0.00 MB/sec execute 18 sec latency 22673.236 ms 2024-09-15T23:01:51.635 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48243 0.00 MB/sec execute 19 sec latency 23673.306 ms 2024-09-15T23:01:51.999 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:51 smithi111 ceph-mon[30629]: pgmap v828: 97 pgs: 97 active+clean; 527 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 5.4 MiB/s wr, 214 op/s 2024-09-15T23:01:52.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:51 smithi192 ceph-mon[35427]: pgmap v828: 97 pgs: 97 active+clean; 527 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 5.4 MiB/s wr, 214 op/s 2024-09-15T23:01:52.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:51 smithi121 ceph-mon[34815]: pgmap v828: 97 pgs: 97 active+clean; 527 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 5.4 MiB/s wr, 214 op/s 2024-09-15T23:01:52.635 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48243 0.00 MB/sec execute 20 sec latency 24673.375 ms 2024-09-15T23:01:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.110+0000 7f0bfd800640 1 -- 172.21.15.111:0/2956066535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf8104f50 msgr2=0x7f0bf81053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.110+0000 7f0bfd800640 1 --2- 172.21.15.111:0/2956066535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf8104f50 0x7f0bf81053d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0be4009900 tx=0x7f0be402a3f0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.110+0000 7f0bfd800640 1 -- 172.21.15.111:0/2956066535 shutdown_connections 2024-09-15T23:01:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.110+0000 7f0bfd800640 1 --2- 172.21.15.111:0/2956066535 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bf8105910 0x7f0bf810c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.110+0000 7f0bfd800640 1 --2- 172.21.15.111:0/2956066535 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf8104f50 0x7f0bf81053d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.110+0000 7f0bfd800640 1 --2- 172.21.15.111:0/2956066535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bf8103d60 0x7f0bf8104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.110+0000 7f0bfd800640 1 -- 172.21.15.111:0/2956066535 >> 172.21.15.111:0/2956066535 conn(0x7f0bf80ff4e0 msgr2=0x7f0bf8101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:01:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.110+0000 7f0bfd800640 1 -- 172.21.15.111:0/2956066535 shutdown_connections 2024-09-15T23:01:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.110+0000 7f0bfd800640 1 -- 172.21.15.111:0/2956066535 wait complete. 2024-09-15T23:01:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.110+0000 7f0bfd800640 1 Processor -- start 2024-09-15T23:01:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bfd800640 1 -- start start 2024-09-15T23:01:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bfd800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf8103d60 0x7f0bf8199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bfd800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bf8104f50 0x7f0bf819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bfd800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bf8105910 0x7f0bf81a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bfd800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf810fa30 con 0x7f0bf8103d60 2024-09-15T23:01:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bfd800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf810f8b0 con 0x7f0bf8105910 2024-09-15T23:01:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bfd800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf810fbb0 con 0x7f0bf8104f50 2024-09-15T23:01:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bf6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf8103d60 0x7f0bf8199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bf6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf8103d60 0x7f0bf8199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45876/0 (socket says 172.21.15.111:45876) 2024-09-15T23:01:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bf6a00640 1 -- 172.21.15.111:0/2737076648 learned_addr learned my addr 172.21.15.111:0/2737076648 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:01:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bf7400640 1 --2- 172.21.15.111:0/2737076648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bf8105910 0x7f0bf81a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bf6000640 1 --2- 172.21.15.111:0/2737076648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bf8104f50 0x7f0bf819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bf6a00640 1 -- 172.21.15.111:0/2737076648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bf8104f50 msgr2=0x7f0bf819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bf6a00640 1 --2- 172.21.15.111:0/2737076648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bf8104f50 0x7f0bf819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bf6a00640 1 -- 172.21.15.111:0/2737076648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bf8105910 msgr2=0x7f0bf81a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bf6a00640 1 --2- 172.21.15.111:0/2737076648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bf8105910 0x7f0bf81a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bf6a00640 1 -- 172.21.15.111:0/2737076648 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf81a1bc0 con 0x7f0bf8103d60 2024-09-15T23:01:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bf7400640 1 --2- 172.21.15.111:0/2737076648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bf8105910 0x7f0bf81a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:01:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bf6a00640 1 --2- 172.21.15.111:0/2737076648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf8103d60 0x7f0bf8199fa0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0be800e9c0 tx=0x7f0be800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:01:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bef400640 1 -- 172.21.15.111:0/2737076648 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be800cd80 con 0x7f0bf8103d60 2024-09-15T23:01:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bef400640 1 -- 172.21.15.111:0/2737076648 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0be80035f0 con 0x7f0bf8103d60 2024-09-15T23:01:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bfd800640 1 -- 172.21.15.111:0/2737076648 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf81a1eb0 con 0x7f0bf8103d60 2024-09-15T23:01:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bef400640 1 -- 172.21.15.111:0/2737076648 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be8010690 con 0x7f0bf8103d60 2024-09-15T23:01:53.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bfd800640 1 -- 172.21.15.111:0/2737076648 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf81a22c0 con 0x7f0bf8103d60 2024-09-15T23:01:53.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0c00651740 1 Processor -- start 2024-09-15T23:01:53.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0c00651740 1 -- start start 2024-09-15T23:01:53.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0c00651740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e9f256090 0x564e9f256490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:53.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0c00651740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e9f2569d0 0x564e9f25b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0c00651740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e9f25ba60 0x564e9f25df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0c00651740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564e9f1bb100 con 0x564e9f25ba60 2024-09-15T23:01:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0c00651740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564e9f1631b0 con 0x564e9f2569d0 2024-09-15T23:01:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0c00651740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564e9f1b70c0 con 0x564e9f256090 2024-09-15T23:01:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bf6000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e9f25ba60 0x564e9f25df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bf7400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e9f256090 0x564e9f256490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bf6000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e9f25ba60 0x564e9f25df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45884/0 (socket says 172.21.15.111:45884) 2024-09-15T23:01:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.112+0000 7f0bf6000640 1 -- 172.21.15.111:0/3523314158 learned_addr learned my addr 172.21.15.111:0/3523314158 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:01:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0bef400640 1 -- 172.21.15.111:0/2737076648 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f0be80037a0 con 0x7f0bf8103d60 2024-09-15T23:01:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0bf7400640 1 -- 172.21.15.111:0/3523314158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e9f2569d0 msgr2=0x564e9f25b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0bf7400640 1 --2- 172.21.15.111:0/3523314158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e9f2569d0 0x564e9f25b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0bf7400640 1 -- 172.21.15.111:0/3523314158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e9f25ba60 msgr2=0x564e9f25df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0bf7400640 1 --2- 172.21.15.111:0/3523314158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e9f25ba60 0x564e9f25df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0bf7400640 1 -- 172.21.15.111:0/3523314158 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e9eece210 con 0x564e9f256090 2024-09-15T23:01:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0bf6000640 1 --2- 172.21.15.111:0/3523314158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e9f25ba60 0x564e9f25df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:01:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0bf7400640 1 --2- 172.21.15.111:0/3523314158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e9f256090 0x564e9f256490 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0009ee0 tx=0x7f0bf0010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba67fcb81864433b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:01:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0be3e00640 1 -- 172.21.15.111:0/3523314158 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0016070 con 0x564e9f256090 2024-09-15T23:01:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0be3e00640 1 -- 172.21.15.111:0/3523314158 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bf001bb40 con 0x564e9f256090 2024-09-15T23:01:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0be3e00640 1 -- 172.21.15.111:0/3523314158 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0012ca0 con 0x564e9f256090 2024-09-15T23:01:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0bef400640 1 --2- 172.21.15.111:0/2737076648 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0bdc0a32f0 0x7f0bdc0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0bef400640 1 -- 172.21.15.111:0/2737076648 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f0be80cfd10 con 0x7f0bf8103d60 2024-09-15T23:01:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0c00651740 1 -- 172.21.15.111:0/3523314158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e9f256090 msgr2=0x564e9f256490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0c00651740 1 --2- 172.21.15.111:0/3523314158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e9f256090 0x564e9f256490 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0009ee0 tx=0x7f0bf0010740 comp rx=0 tx=0).stop 2024-09-15T23:01:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0bf6a00640 1 --2- 172.21.15.111:0/2737076648 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0bdc0a32f0 0x7f0bdc0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0c00651740 1 -- 172.21.15.111:0/3523314158 shutdown_connections 2024-09-15T23:01:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0c00651740 1 --2- 172.21.15.111:0/3523314158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e9f25ba60 0x564e9f25df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0c00651740 1 --2- 172.21.15.111:0/3523314158 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e9f2569d0 0x564e9f25b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0c00651740 1 --2- 172.21.15.111:0/3523314158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e9f256090 0x564e9f256490 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0c00651740 1 -- 172.21.15.111:0/3523314158 >> 172.21.15.111:0/3523314158 conn(0x564e9f251700 msgr2=0x564e9f251b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:01:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0c00651740 1 -- 172.21.15.111:0/3523314158 shutdown_connections 2024-09-15T23:01:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0c00651740 1 -- 172.21.15.111:0/3523314158 wait complete. 2024-09-15T23:01:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0bf6a00640 1 --2- 172.21.15.111:0/2737076648 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0bdc0a32f0 0x7f0bdc0a57b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0be8098d00 tx=0x7f0be8097f60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:01:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0c00651740 1 -- start start 2024-09-15T23:01:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0c00651740 1 Processor -- start 2024-09-15T23:01:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0c00651740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e9f256090 0x564e9f2602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0c00651740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e9f2569d0 0x564e9f2607e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0c00651740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e9f25ba60 0x564e9f267820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0c00651740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564e9f25ea30 con 0x564e9f256090 2024-09-15T23:01:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0c00651740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564e9f144090 con 0x564e9f25ba60 2024-09-15T23:01:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0c00651740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564e9f25ebb0 con 0x564e9f2569d0 2024-09-15T23:01:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0c00651740 10 client.0 _ll_register_callbacks cb 0x564e9f198f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:01:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0bf7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e9f256090 0x564e9f2602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0bf7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e9f256090 0x564e9f2602a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45892/0 (socket says 172.21.15.111:45892) 2024-09-15T23:01:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0bf7400640 1 -- 172.21.15.111:0/4068496653 learned_addr learned my addr 172.21.15.111:0/4068496653 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:01:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0bf6000640 1 --2- 172.21.15.111:0/4068496653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e9f2569d0 0x564e9f2607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0bf7400640 1 --2- 172.21.15.111:0/4068496653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e9f25ba60 0x564e9f267820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0bf6000640 1 -- 172.21.15.111:0/4068496653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e9f25ba60 msgr2=0x564e9f267820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0bf6000640 1 --2- 172.21.15.111:0/4068496653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e9f25ba60 0x564e9f267820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0bf6000640 1 -- 172.21.15.111:0/4068496653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e9f256090 msgr2=0x564e9f2602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0bf6000640 1 --2- 172.21.15.111:0/4068496653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e9f256090 0x564e9f2602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0bf6000640 1 -- 172.21.15.111:0/4068496653 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e9ef9a620 con 0x564e9f2569d0 2024-09-15T23:01:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0bf6000640 1 --2- 172.21.15.111:0/4068496653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e9f2569d0 0x564e9f2607e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0be4006010 tx=0x7f0be4004420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bb0426412eeb7bc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:01:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.118+0000 7f0be0c00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:01:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.123+0000 7f0be0c00640 1 -- 172.21.15.111:0/4068496653 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be4035040 con 0x564e9f2569d0 2024-09-15T23:01:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.123+0000 7f0be0c00640 1 -- 172.21.15.111:0/4068496653 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f0be4005510 con 0x564e9f2569d0 2024-09-15T23:01:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.123+0000 7f0c00651740 1 -- 172.21.15.111:0/4068496653 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bf001bb40 con 0x564e9f2569d0 2024-09-15T23:01:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.123+0000 7f0be0c00640 1 -- 172.21.15.111:0/4068496653 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0be4004770 con 0x564e9f2569d0 2024-09-15T23:01:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.123+0000 7f0c00651740 10 client.26233 fetch_fsmap learned FSMap version 31 2024-09-15T23:01:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.123+0000 7f0c00651740 1 -- 172.21.15.111:0/4068496653 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564e9f26a680 con 0x564e9f2569d0 2024-09-15T23:01:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.123+0000 7f0be0c00640 1 -- 172.21.15.111:0/4068496653 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bf001bb40 con 0x564e9f2569d0 2024-09-15T23:01:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.123+0000 7f0be0c00640 1 -- 172.21.15.111:0/4068496653 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f0be4031c10 con 0x564e9f2569d0 2024-09-15T23:01:53.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.123+0000 7f0c00651740 10 client.26233 fetch_fsmap finished waiting for FSMap version 31 2024-09-15T23:01:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.123+0000 7f0c00651740 10 client.26233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:01:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.124+0000 7f0c00651740 20 client.26233 populate_metadata read hostname 'smithi111' 2024-09-15T23:01:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.124+0000 7f0c00651740 1 --2- 172.21.15.111:0/4068496653 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564e9f26bce0 0x564e9f26e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.124+0000 7f0c00651740 4 client.26233 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:01:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.124+0000 7f0c00651740 1 -- 172.21.15.111:0/4068496653 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e9f1c8830 con 0x564e9f26bce0 2024-09-15T23:01:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.124+0000 7f0bf7400640 1 --2- 172.21.15.111:0/4068496653 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564e9f26bce0 0x564e9f26e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.125+0000 7f0be0c00640 10 client.26233 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:01:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.125+0000 7f0bf7400640 1 --2- 172.21.15.111:0/4068496653 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564e9f26bce0 0x564e9f26e0d0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=192b44e1eac1eacc server_cookie=52b3efa17ba4449d in_seq=0 out_seq=0 2024-09-15T23:01:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.125+0000 7f0be0c00640 1 -- 172.21.15.111:0/4068496653 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e9f1c8830 con 0x564e9f26bce0 2024-09-15T23:01:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.125+0000 7f0be0c00640 10 client.26233 handle_command_reply: tid=0 2024-09-15T23:01:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.126+0000 7f0c00651740 1 client.26233 shutdown 2024-09-15T23:01:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.126+0000 7f0c00651740 1 -- 172.21.15.111:0/4068496653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e9f2569d0 msgr2=0x564e9f2607e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:53.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.126+0000 7f0c00651740 1 --2- 172.21.15.111:0/4068496653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e9f2569d0 0x564e9f2607e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0be4006010 tx=0x7f0be4004420 comp rx=0 tx=0).stop 2024-09-15T23:01:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.126+0000 7f0c00651740 1 -- 172.21.15.111:0/4068496653 shutdown_connections 2024-09-15T23:01:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.126+0000 7f0c00651740 1 --2- 172.21.15.111:0/4068496653 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564e9f26bce0 0x564e9f26e0d0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.126+0000 7f0c00651740 1 --2- 172.21.15.111:0/4068496653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e9f25ba60 0x564e9f267820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.126+0000 7f0be0c00640 0 client.26233 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:01:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.126+0000 7f0c00651740 1 --2- 172.21.15.111:0/4068496653 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e9f2569d0 0x564e9f2607e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.126+0000 7f0c00651740 1 --2- 172.21.15.111:0/4068496653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e9f256090 0x564e9f2602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.126+0000 7f0c00651740 1 -- 172.21.15.111:0/4068496653 >> 172.21.15.111:0/4068496653 conn(0x564e9f251700 msgr2=0x564e9f252f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:01:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.126+0000 7f0c00651740 1 -- 172.21.15.111:0/4068496653 shutdown_connections 2024-09-15T23:01:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.126+0000 7f0c00651740 1 -- 172.21.15.111:0/4068496653 wait complete. 2024-09-15T23:01:53.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.126+0000 7f0c00651740 20 client.26233 trim_cache size 0 max 16384 2024-09-15T23:01:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 1 Processor -- start 2024-09-15T23:01:53.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 1 -- start start 2024-09-15T23:01:53.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e9f256090 0x564e9f250770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:53.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e9f2569d0 0x564e9f2677e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:53.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e9f25ba60 0x564e9f272930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:53.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf81ad8f0 con 0x564e9f2569d0 2024-09-15T23:01:53.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf81ab160 con 0x564e9f256090 2024-09-15T23:01:53.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf81a8fb0 con 0x564e9f25ba60 2024-09-15T23:01:53.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0bf6000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e9f2569d0 0x564e9f2677e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:53.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0bf6000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e9f2569d0 0x564e9f2677e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45900/0 (socket says 172.21.15.111:45900) 2024-09-15T23:01:53.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0bf6000640 1 -- 172.21.15.111:0/3964813175 learned_addr learned my addr 172.21.15.111:0/3964813175 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:01:53.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0bf7400640 1 --2- 172.21.15.111:0/3964813175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e9f256090 0x564e9f250770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0bf7400640 1 --2- 172.21.15.111:0/3964813175 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e9f25ba60 0x564e9f272930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0bf6000640 1 -- 172.21.15.111:0/3964813175 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e9f25ba60 msgr2=0x564e9f272930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0bf6000640 1 --2- 172.21.15.111:0/3964813175 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e9f25ba60 0x564e9f272930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0bf6000640 1 -- 172.21.15.111:0/3964813175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e9f256090 msgr2=0x564e9f250770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0bf6000640 1 --2- 172.21.15.111:0/3964813175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e9f256090 0x564e9f250770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0bf6000640 1 -- 172.21.15.111:0/3964813175 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e9f257690 con 0x564e9f2569d0 2024-09-15T23:01:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0bf7400640 1 --2- 172.21.15.111:0/3964813175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e9f256090 0x564e9f250770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:01:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0bf6000640 1 --2- 172.21.15.111:0/3964813175 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e9f2569d0 0x564e9f2677e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0be402a910 tx=0x7f0be4031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a16708d863c8a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:01:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0be3400640 1 -- 172.21.15.111:0/3964813175 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be4035030 con 0x564e9f2569d0 2024-09-15T23:01:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0be3400640 1 -- 172.21.15.111:0/3964813175 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0be4041070 con 0x564e9f2569d0 2024-09-15T23:01:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0be3400640 1 -- 172.21.15.111:0/3964813175 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be40316b0 con 0x564e9f2569d0 2024-09-15T23:01:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 1 -- 172.21.15.111:0/3964813175 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e9f2569d0 msgr2=0x564e9f2677e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 1 --2- 172.21.15.111:0/3964813175 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e9f2569d0 0x564e9f2677e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0be402a910 tx=0x7f0be4031400 comp rx=0 tx=0).stop 2024-09-15T23:01:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 1 -- 172.21.15.111:0/3964813175 shutdown_connections 2024-09-15T23:01:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 1 --2- 172.21.15.111:0/3964813175 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e9f25ba60 0x564e9f272930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 1 --2- 172.21.15.111:0/3964813175 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e9f2569d0 0x564e9f2677e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 1 --2- 172.21.15.111:0/3964813175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e9f256090 0x564e9f250770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 1 -- 172.21.15.111:0/3964813175 >> 172.21.15.111:0/3964813175 conn(0x564e9f251700 msgr2=0x564e9f250060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:01:53.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 1 -- 172.21.15.111:0/3964813175 shutdown_connections 2024-09-15T23:01:53.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 1 -- 172.21.15.111:0/3964813175 wait complete. 2024-09-15T23:01:53.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 1 -- start start 2024-09-15T23:01:53.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 1 Processor -- start 2024-09-15T23:01:53.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e9f256090 0x564e9f265360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:53.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e9f2569d0 0x564e9f2658a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:53.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e9f25ba60 0x564e9f2668b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:53.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf81a9620 con 0x564e9f256090 2024-09-15T23:01:53.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf81aaa90 con 0x564e9f25ba60 2024-09-15T23:01:53.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf81a9cf0 con 0x564e9f2569d0 2024-09-15T23:01:53.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 10 client.0 _ll_register_callbacks cb 0x7f0be40040b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:01:53.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0bf7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e9f256090 0x564e9f265360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:53.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0bf7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e9f256090 0x564e9f265360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:45910/0 (socket says 172.21.15.111:45910) 2024-09-15T23:01:53.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0bf7400640 1 -- 172.21.15.111:0/584484272 learned_addr learned my addr 172.21.15.111:0/584484272 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:01:53.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0bf6000640 1 --2- 172.21.15.111:0/584484272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e9f2569d0 0x564e9f2658a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:53.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0bf7400640 1 --2- 172.21.15.111:0/584484272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e9f25ba60 0x564e9f2668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:53.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0bf7400640 1 -- 172.21.15.111:0/584484272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e9f2569d0 msgr2=0x564e9f2658a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:53.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0bf7400640 1 --2- 172.21.15.111:0/584484272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e9f2569d0 0x564e9f2658a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0bf7400640 1 -- 172.21.15.111:0/584484272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e9f25ba60 msgr2=0x564e9f2668b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:01:53.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0bf7400640 1 --2- 172.21.15.111:0/584484272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e9f25ba60 0x564e9f2668b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0bf7400640 1 -- 172.21.15.111:0/584484272 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e9f266ec0 con 0x564e9f256090 2024-09-15T23:01:53.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0bf7400640 1 --2- 172.21.15.111:0/584484272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e9f256090 0x564e9f265360 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0bf00095f0 tx=0x7f0bf0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4137bc71df8bd6cd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:01:53.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0be0c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:01:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0be0c00640 1 -- 172.21.15.111:0/584484272 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0016040 con 0x564e9f256090 2024-09-15T23:01:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 1 -- 172.21.15.111:0/584484272 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0be4041070 con 0x564e9f256090 2024-09-15T23:01:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0be0c00640 1 -- 172.21.15.111:0/584484272 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8153+0+0 (secure 0 0 0) 0x7f0bf001b660 con 0x564e9f256090 2024-09-15T23:01:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0be0c00640 1 -- 172.21.15.111:0/584484272 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0bf003d050 con 0x564e9f256090 2024-09-15T23:01:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 10 client.16272 fetch_fsmap learned FSMap version 31 2024-09-15T23:01:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.141+0000 7f0c00651740 1 -- 172.21.15.111:0/584484272 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564e9f2d6da0 con 0x564e9f256090 2024-09-15T23:01:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.146+0000 7f0be0c00640 1 -- 172.21.15.111:0/584484272 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0be4041070 con 0x564e9f256090 2024-09-15T23:01:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.146+0000 7f0be0c00640 1 -- 172.21.15.111:0/584484272 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 31) ==== 8354+0+0 (secure 0 0 0) 0x7f0bf002a070 con 0x564e9f256090 2024-09-15T23:01:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.146+0000 7f0c00651740 10 client.16272 fetch_fsmap finished waiting for FSMap version 31 2024-09-15T23:01:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.146+0000 7f0c00651740 10 client.16272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:01:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.146+0000 7f0c00651740 20 client.16272 populate_metadata read hostname 'smithi111' 2024-09-15T23:01:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.146+0000 7f0c00651740 1 --2- 172.21.15.111:0/584484272 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564e9f2d74d0 0x564e9f2d98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:01:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.146+0000 7f0c00651740 4 client.16272 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:01:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.146+0000 7f0c00651740 1 -- 172.21.15.111:0/584484272 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564e9f1c2b80 con 0x564e9f2d74d0 2024-09-15T23:01:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.146+0000 7f0bf6000640 1 --2- 172.21.15.111:0/584484272 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564e9f2d74d0 0x564e9f2d98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:01:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.146+0000 7f0be0c00640 10 client.16272 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:01:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.146+0000 7f0bf6000640 1 --2- 172.21.15.111:0/584484272 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564e9f2d74d0 0x564e9f2d98f0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9865e8fc33d7f0f server_cookie=9808c844710758cb in_seq=0 out_seq=0 2024-09-15T23:01:53.213 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.212+0000 7f0be0c00640 1 -- 172.21.15.111:0/584484272 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564e9f1c2b80 con 0x564e9f2d74d0 2024-09-15T23:01:53.213 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.212+0000 7f0be0c00640 10 client.16272 handle_command_reply: tid=0 2024-09-15T23:01:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.212+0000 7f0c00651740 1 client.16272 shutdown 2024-09-15T23:01:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.212+0000 7f0c00651740 1 -- 172.21.15.111:0/584484272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e9f256090 msgr2=0x564e9f265360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.212+0000 7f0c00651740 1 --2- 172.21.15.111:0/584484272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e9f256090 0x564e9f265360 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0bf00095f0 tx=0x7f0bf0010740 comp rx=0 tx=0).stop 2024-09-15T23:01:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.212+0000 7f0c00651740 1 -- 172.21.15.111:0/584484272 shutdown_connections 2024-09-15T23:01:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.212+0000 7f0c00651740 1 --2- 172.21.15.111:0/584484272 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564e9f2d74d0 0x564e9f2d98f0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.212+0000 7f0c00651740 1 --2- 172.21.15.111:0/584484272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564e9f25ba60 0x564e9f2668b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.212+0000 7f0c00651740 1 --2- 172.21.15.111:0/584484272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564e9f2569d0 0x564e9f2658a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.212+0000 7f0c00651740 1 --2- 172.21.15.111:0/584484272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564e9f256090 0x564e9f265360 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.212+0000 7f0c00651740 1 -- 172.21.15.111:0/584484272 >> 172.21.15.111:0/584484272 conn(0x564e9f251700 msgr2=0x564e9f250880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:01:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.212+0000 7f0be0c00640 0 client.16272 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:01:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.212+0000 7f0c00651740 1 -- 172.21.15.111:0/584484272 shutdown_connections 2024-09-15T23:01:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.212+0000 7f0c00651740 1 -- 172.21.15.111:0/584484272 wait complete. 2024-09-15T23:01:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.212+0000 7f0c00651740 20 client.16272 trim_cache size 0 max 16384 2024-09-15T23:01:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:01:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:01:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:01:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:01:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.212+0000 7f0bfd800640 1 -- 172.21.15.111:0/2737076648 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0bdc0a32f0 msgr2=0x7f0bdc0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.212+0000 7f0bfd800640 1 --2- 172.21.15.111:0/2737076648 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0bdc0a32f0 0x7f0bdc0a57b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0be8098d00 tx=0x7f0be8097f60 comp rx=0 tx=0).stop 2024-09-15T23:01:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.212+0000 7f0bfd800640 1 -- 172.21.15.111:0/2737076648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf8103d60 msgr2=0x7f0bf8199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:01:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.212+0000 7f0bfd800640 1 --2- 172.21.15.111:0/2737076648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf8103d60 0x7f0bf8199fa0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0be800e9c0 tx=0x7f0be800ee80 comp rx=0 tx=0).stop 2024-09-15T23:01:53.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.212+0000 7f0bfd800640 1 -- 172.21.15.111:0/2737076648 shutdown_connections 2024-09-15T23:01:53.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.212+0000 7f0bfd800640 1 --2- 172.21.15.111:0/2737076648 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0bdc0a32f0 0x7f0bdc0a57b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.212+0000 7f0bfd800640 1 --2- 172.21.15.111:0/2737076648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bf8105910 0x7f0bf81a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.212+0000 7f0bfd800640 1 --2- 172.21.15.111:0/2737076648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0bf8104f50 0x7f0bf819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.212+0000 7f0bfd800640 1 --2- 172.21.15.111:0/2737076648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0bf8103d60 0x7f0bf8199fa0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:01:53.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.212+0000 7f0bfd800640 1 -- 172.21.15.111:0/2737076648 >> 172.21.15.111:0/2737076648 conn(0x7f0bf80ff4e0 msgr2=0x7f0bf8108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:01:53.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.212+0000 7f0bfd800640 1 -- 172.21.15.111:0/2737076648 shutdown_connections 2024-09-15T23:01:53.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:01:53.212+0000 7f0bfd800640 1 -- 172.21.15.111:0/2737076648 wait complete. 2024-09-15T23:01:53.564 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 15 23:01:53 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-c[65843]: ignoring --setuser ceph since I am not root 2024-09-15T23:01:53.565 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 15 23:01:53 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-c[65843]: ignoring --setgroup ceph since I am not root 2024-09-15T23:01:53.565 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 15 23:01:53 smithi192 ceph-mds[65847]: -- 172.21.15.192:0/3203997778 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b2d7cdf860 con 0x55b2d7cbc800 2024-09-15T23:01:53.565 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 15 23:01:53 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-c[65843]: starting mds.c at 2024-09-15T23:01:53.636 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48246 0.01 MB/sec execute 21 sec latency 25673.979 ms 2024-09-15T23:01:53.981 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:53 smithi111 ceph-mon[30629]: pgmap v829: 97 pgs: 97 active+clean; 535 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 5.6 MiB/s wr, 212 op/s 2024-09-15T23:01:53.981 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:53 smithi111 ceph-mon[30629]: daemon mds.c restarted 2024-09-15T23:01:54.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:53 smithi192 ceph-mon[35427]: pgmap v829: 97 pgs: 97 active+clean; 535 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 5.6 MiB/s wr, 212 op/s 2024-09-15T23:01:54.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:53 smithi192 ceph-mon[35427]: daemon mds.c restarted 2024-09-15T23:01:54.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:53 smithi121 ceph-mon[34815]: pgmap v829: 97 pgs: 97 active+clean; 535 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 5.6 MiB/s wr, 212 op/s 2024-09-15T23:01:54.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:53 smithi121 ceph-mon[34815]: daemon mds.c restarted 2024-09-15T23:01:54.636 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48248 0.01 MB/sec execute 22 sec latency 26674.071 ms 2024-09-15T23:01:55.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:54 smithi192 ceph-mon[35427]: osdmap e89: 12 total, 12 up, 12 in 2024-09-15T23:01:55.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:54 smithi192 ceph-mon[35427]: mds.? [v2:172.21.15.192:6836/1721868112,v1:172.21.15.192:6837/1721868112] up:boot 2024-09-15T23:01:55.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:54 smithi192 ceph-mon[35427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-09-15T23:01:55.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:54 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T23:01:55.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:54 smithi192 ceph-mon[35427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T23:01:55.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:54 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:01:55.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:54 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:01:55.316 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:54 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:01:55.316 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:54 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:01:55.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:54 smithi121 ceph-mon[34815]: osdmap e89: 12 total, 12 up, 12 in 2024-09-15T23:01:55.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:54 smithi121 ceph-mon[34815]: mds.? [v2:172.21.15.192:6836/1721868112,v1:172.21.15.192:6837/1721868112] up:boot 2024-09-15T23:01:55.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:54 smithi121 ceph-mon[34815]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-09-15T23:01:55.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:54 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T23:01:55.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:54 smithi121 ceph-mon[34815]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T23:01:55.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:54 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:01:55.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:54 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:01:55.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:54 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:01:55.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:54 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:01:55.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:54 smithi111 ceph-mon[30629]: osdmap e89: 12 total, 12 up, 12 in 2024-09-15T23:01:55.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:54 smithi111 ceph-mon[30629]: mds.? [v2:172.21.15.192:6836/1721868112,v1:172.21.15.192:6837/1721868112] up:boot 2024-09-15T23:01:55.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:54 smithi111 ceph-mon[30629]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-09-15T23:01:55.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:54 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T23:01:55.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:54 smithi111 ceph-mon[30629]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T23:01:55.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:54 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:01:55.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:54 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:01:55.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:54 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:01:55.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:54 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:01:55.636 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48248 0.01 MB/sec execute 23 sec latency 27674.120 ms 2024-09-15T23:01:56.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:55 smithi192 ceph-mon[35427]: pgmap v831: 97 pgs: 97 active+clean; 537 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 5.7 MiB/s wr, 189 op/s 2024-09-15T23:01:56.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:55 smithi121 ceph-mon[34815]: pgmap v831: 97 pgs: 97 active+clean; 537 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 5.7 MiB/s wr, 189 op/s 2024-09-15T23:01:56.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:55 smithi111 ceph-mon[30629]: pgmap v831: 97 pgs: 97 active+clean; 537 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 5.7 MiB/s wr, 189 op/s 2024-09-15T23:01:56.636 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48248 0.01 MB/sec execute 24 sec latency 28674.162 ms 2024-09-15T23:01:57.636 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48248 0.01 MB/sec execute 25 sec latency 29674.208 ms 2024-09-15T23:01:57.983 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:57 smithi111 ceph-mon[30629]: pgmap v832: 97 pgs: 97 active+clean; 530 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 6.3 MiB/s wr, 186 op/s 2024-09-15T23:01:58.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:57 smithi192 ceph-mon[35427]: pgmap v832: 97 pgs: 97 active+clean; 530 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 6.3 MiB/s wr, 186 op/s 2024-09-15T23:01:58.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:57 smithi121 ceph-mon[34815]: pgmap v832: 97 pgs: 97 active+clean; 530 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 6.3 MiB/s wr, 186 op/s 2024-09-15T23:01:58.636 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48274 0.06 MB/sec execute 26 sec latency 30674.252 ms 2024-09-15T23:01:59.636 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48296 0.06 MB/sec execute 27 sec latency 31674.315 ms 2024-09-15T23:02:00.215 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:01:59 smithi111 ceph-mon[30629]: pgmap v833: 97 pgs: 97 active+clean; 511 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 6.1 MiB/s wr, 192 op/s 2024-09-15T23:02:00.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:01:59 smithi192 ceph-mon[35427]: pgmap v833: 97 pgs: 97 active+clean; 511 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 6.1 MiB/s wr, 192 op/s 2024-09-15T23:02:00.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:01:59 smithi121 ceph-mon[34815]: pgmap v833: 97 pgs: 97 active+clean; 511 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 6.1 MiB/s wr, 192 op/s 2024-09-15T23:02:00.636 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48296 0.06 MB/sec execute 28 sec latency 32674.366 ms 2024-09-15T23:02:01.636 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48296 0.06 MB/sec execute 29 sec latency 33674.441 ms 2024-09-15T23:02:02.221 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:01 smithi111 ceph-mon[30629]: pgmap v834: 97 pgs: 97 active+clean; 497 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 5.1 MiB/s wr, 185 op/s 2024-09-15T23:02:02.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:01 smithi192 ceph-mon[35427]: pgmap v834: 97 pgs: 97 active+clean; 497 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 5.1 MiB/s wr, 185 op/s 2024-09-15T23:02:02.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:01 smithi121 ceph-mon[34815]: pgmap v834: 97 pgs: 97 active+clean; 497 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 5.1 MiB/s wr, 185 op/s 2024-09-15T23:02:02.636 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48298 0.05 MB/sec execute 30 sec latency 34519.642 ms 2024-09-15T23:02:03.636 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48298 0.05 MB/sec execute 31 sec latency 10132.738 ms 2024-09-15T23:02:04.291 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:03 smithi111 ceph-mon[30629]: pgmap v835: 97 pgs: 97 active+clean; 477 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 3.7 MiB/s wr, 174 op/s 2024-09-15T23:02:04.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:03 smithi192 ceph-mon[35427]: pgmap v835: 97 pgs: 97 active+clean; 477 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 3.7 MiB/s wr, 174 op/s 2024-09-15T23:02:04.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:03 smithi121 ceph-mon[34815]: pgmap v835: 97 pgs: 97 active+clean; 477 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 3.7 MiB/s wr, 174 op/s 2024-09-15T23:02:04.636 INFO:tasks.workunit.client.0.smithi111.stdout: 10 48575 0.54 MB/sec execute 32 sec latency 10705.017 ms 2024-09-15T23:02:05.636 INFO:tasks.workunit.client.0.smithi111.stdout: 10 49254 1.66 MB/sec execute 33 sec latency 97.953 ms 2024-09-15T23:02:06.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:05 smithi111 ceph-mon[30629]: pgmap v836: 97 pgs: 97 active+clean; 477 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 3.2 MiB/s wr, 153 op/s 2024-09-15T23:02:06.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:05 smithi192 ceph-mon[35427]: pgmap v836: 97 pgs: 97 active+clean; 477 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 3.2 MiB/s wr, 153 op/s 2024-09-15T23:02:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.343+0000 7f786f200640 1 -- 172.21.15.111:0/1450309406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7870104cf0 msgr2=0x7f78701070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.343+0000 7f786f200640 1 --2- 172.21.15.111:0/1450309406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7870104cf0 0x7f78701070e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7858009900 tx=0x7f785802a410 comp rx=0 tx=0).stop 2024-09-15T23:02:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.343+0000 7f786f200640 1 -- 172.21.15.111:0/1450309406 shutdown_connections 2024-09-15T23:02:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.343+0000 7f786f200640 1 --2- 172.21.15.111:0/1450309406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7870107620 0x7f7870109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.343+0000 7f786f200640 1 --2- 172.21.15.111:0/1450309406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7870104cf0 0x7f78701070e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.343+0000 7f786f200640 1 --2- 172.21.15.111:0/1450309406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78701023c0 0x7f78701047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.343+0000 7f786f200640 1 -- 172.21.15.111:0/1450309406 >> 172.21.15.111:0/1450309406 conn(0x7f78700fbfa0 msgr2=0x7f78700fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.343+0000 7f786f200640 1 -- 172.21.15.111:0/1450309406 shutdown_connections 2024-09-15T23:02:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.344+0000 7f786f200640 1 -- 172.21.15.111:0/1450309406 wait complete. 2024-09-15T23:02:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.344+0000 7f786f200640 1 Processor -- start 2024-09-15T23:02:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.345+0000 7f786f200640 1 -- start start 2024-09-15T23:02:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.345+0000 7f786f200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78701023c0 0x7f7870199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.345+0000 7f786f200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7870104cf0 0x7f787019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.345+0000 7f786f200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7870107620 0x7f78701a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.345+0000 7f786f200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f787010d670 con 0x7f7870107620 2024-09-15T23:02:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.345+0000 7f786f200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f787010d4f0 con 0x7f7870104cf0 2024-09-15T23:02:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.345+0000 7f786f200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f787010d7f0 con 0x7f78701023c0 2024-09-15T23:02:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.345+0000 7f786de00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78701023c0 0x7f7870199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.345+0000 7f786de00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78701023c0 0x7f7870199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:47026/0 (socket says 172.21.15.111:47026) 2024-09-15T23:02:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.345+0000 7f786de00640 1 -- 172.21.15.111:0/3155695704 learned_addr learned my addr 172.21.15.111:0/3155695704 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:02:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.345+0000 7f786e800640 1 --2- 172.21.15.111:0/3155695704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7870107620 0x7f78701a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.345+0000 7f7867e00640 1 --2- 172.21.15.111:0/3155695704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7870104cf0 0x7f787019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.346+0000 7f786de00640 1 -- 172.21.15.111:0/3155695704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7870104cf0 msgr2=0x7f787019a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.346+0000 7f786de00640 1 --2- 172.21.15.111:0/3155695704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7870104cf0 0x7f787019a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.346+0000 7f786de00640 1 -- 172.21.15.111:0/3155695704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7870107620 msgr2=0x7f78701a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.346+0000 7f786de00640 1 --2- 172.21.15.111:0/3155695704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7870107620 0x7f78701a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.346+0000 7f786de00640 1 -- 172.21.15.111:0/3155695704 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78701a1bf0 con 0x7f78701023c0 2024-09-15T23:02:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.346+0000 7f786de00640 1 --2- 172.21.15.111:0/3155695704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78701023c0 0x7f7870199fd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f786000b450 tx=0x7f786000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.346+0000 7f7866a00640 1 -- 172.21.15.111:0/3155695704 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7860015020 con 0x7f78701023c0 2024-09-15T23:02:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.346+0000 7f7866a00640 1 -- 172.21.15.111:0/3155695704 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7860011d70 con 0x7f78701023c0 2024-09-15T23:02:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.346+0000 7f786f200640 1 -- 172.21.15.111:0/3155695704 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78701a1ee0 con 0x7f78701023c0 2024-09-15T23:02:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.346+0000 7f7866a00640 1 -- 172.21.15.111:0/3155695704 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786000f340 con 0x7f78701023c0 2024-09-15T23:02:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.346+0000 7f786f200640 1 -- 172.21.15.111:0/3155695704 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78701a2360 con 0x7f78701023c0 2024-09-15T23:02:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.349+0000 7f7877a26740 1 Processor -- start 2024-09-15T23:02:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.349+0000 7f7877a26740 1 -- start start 2024-09-15T23:02:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.349+0000 7f7866a00640 1 -- 172.21.15.111:0/3155695704 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f7860018020 con 0x7f78701023c0 2024-09-15T23:02:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.350+0000 7f7877a26740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564cb0142260 0x564cb0142660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.350+0000 7f7877a26740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cb0142ba0 0x564cb0147660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.350+0000 7f7877a26740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564cb0147c30 0x564cb014a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.350+0000 7f7877a26740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564cb00a7100 con 0x564cb0147c30 2024-09-15T23:02:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.350+0000 7f7877a26740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564cb004f1b0 con 0x564cb0142260 2024-09-15T23:02:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.350+0000 7f7877a26740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564cb00a30c0 con 0x564cb0142ba0 2024-09-15T23:02:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.350+0000 7f786e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564cb0142260 0x564cb0142660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.350+0000 7f786e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564cb0142260 0x564cb0142660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:39424/0 (socket says 172.21.15.111:39424) 2024-09-15T23:02:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.350+0000 7f786e800640 1 -- 172.21.15.111:0/3933455192 learned_addr learned my addr 172.21.15.111:0/3933455192 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:02:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.350+0000 7f786de00640 1 --2- 172.21.15.111:0/3933455192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cb0142ba0 0x564cb0147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.350+0000 7f7867e00640 1 --2- 172.21.15.111:0/3933455192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564cb0147c30 0x564cb014a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.350+0000 7f786e800640 1 -- 172.21.15.111:0/3933455192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cb0142ba0 msgr2=0x564cb0147660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.350+0000 7f786e800640 1 --2- 172.21.15.111:0/3933455192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cb0142ba0 0x564cb0147660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.350+0000 7f786e800640 1 -- 172.21.15.111:0/3933455192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564cb0147c30 msgr2=0x564cb014a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.350+0000 7f786e800640 1 --2- 172.21.15.111:0/3933455192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564cb0147c30 0x564cb014a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.350+0000 7f786e800640 1 -- 172.21.15.111:0/3933455192 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cafdba210 con 0x564cb0142260 2024-09-15T23:02:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.350+0000 7f7867e00640 1 --2- 172.21.15.111:0/3933455192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564cb0147c30 0x564cb014a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:02:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.351+0000 7f786e800640 1 --2- 172.21.15.111:0/3933455192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564cb0142260 0x564cb0142660 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7868009f90 tx=0x7f78680118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0be6cdd68c340f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.351+0000 7f785f000640 1 -- 172.21.15.111:0/3933455192 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7868004d20 con 0x564cb0142260 2024-09-15T23:02:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.351+0000 7f785f000640 1 -- 172.21.15.111:0/3933455192 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7868004050 con 0x564cb0142260 2024-09-15T23:02:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.351+0000 7f785f000640 1 -- 172.21.15.111:0/3933455192 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7868005260 con 0x564cb0142260 2024-09-15T23:02:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.351+0000 7f7866a00640 1 --2- 172.21.15.111:0/3155695704 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f78540a3240 0x7f78540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.352+0000 7f7877a26740 1 -- 172.21.15.111:0/3933455192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564cb0142260 msgr2=0x564cb0142660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.352+0000 7f7877a26740 1 --2- 172.21.15.111:0/3933455192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564cb0142260 0x564cb0142660 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7868009f90 tx=0x7f78680118d0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.352+0000 7f786de00640 1 --2- 172.21.15.111:0/3155695704 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f78540a3240 0x7f78540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.352+0000 7f7866a00640 1 -- 172.21.15.111:0/3155695704 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f78600cf360 con 0x7f78701023c0 2024-09-15T23:02:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.352+0000 7f7877a26740 1 -- 172.21.15.111:0/3933455192 shutdown_connections 2024-09-15T23:02:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.352+0000 7f7877a26740 1 --2- 172.21.15.111:0/3933455192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564cb0147c30 0x564cb014a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.352+0000 7f7877a26740 1 --2- 172.21.15.111:0/3933455192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cb0142ba0 0x564cb0147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.352+0000 7f7877a26740 1 --2- 172.21.15.111:0/3933455192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564cb0142260 0x564cb0142660 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.352+0000 7f7877a26740 1 -- 172.21.15.111:0/3933455192 >> 172.21.15.111:0/3933455192 conn(0x564cb013d8d0 msgr2=0x564cb013dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.352+0000 7f7877a26740 1 -- 172.21.15.111:0/3933455192 shutdown_connections 2024-09-15T23:02:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.352+0000 7f7877a26740 1 -- 172.21.15.111:0/3933455192 wait complete. 2024-09-15T23:02:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.353+0000 7f7877a26740 1 -- start start 2024-09-15T23:02:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.353+0000 7f786de00640 1 --2- 172.21.15.111:0/3155695704 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f78540a3240 0x7f78540a5700 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7860097de0 tx=0x7f78600107f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.353+0000 7f7877a26740 1 Processor -- start 2024-09-15T23:02:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.353+0000 7f7877a26740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564cb0142260 0x564cb014c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.354+0000 7f7877a26740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564cb0142ba0 0x564cb014c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.354+0000 7f7877a26740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cb0147c30 0x564cb01539a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.354+0000 7f7877a26740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564cb014ac00 con 0x564cb0142ba0 2024-09-15T23:02:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.354+0000 7f7877a26740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564cb0030090 con 0x564cb0142260 2024-09-15T23:02:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.354+0000 7f7877a26740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x564cb014ad80 con 0x564cb0147c30 2024-09-15T23:02:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.354+0000 7f7877a26740 10 client.0 _ll_register_callbacks cb 0x564cb0084f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:02:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.354+0000 7f786e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564cb0142260 0x564cb014c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.354+0000 7f786e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564cb0142260 0x564cb014c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:39432/0 (socket says 172.21.15.111:39432) 2024-09-15T23:02:06.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.354+0000 7f786e800640 1 -- 172.21.15.111:0/766236024 learned_addr learned my addr 172.21.15.111:0/766236024 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:02:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.354+0000 7f786e800640 1 --2- 172.21.15.111:0/766236024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cb0147c30 0x564cb01539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.354+0000 7f7867e00640 1 --2- 172.21.15.111:0/766236024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564cb0142ba0 0x564cb014c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.354+0000 7f786e800640 1 -- 172.21.15.111:0/766236024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564cb0142260 msgr2=0x564cb014c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.354+0000 7f786e800640 1 --2- 172.21.15.111:0/766236024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564cb0142260 0x564cb014c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.354+0000 7f786e800640 1 -- 172.21.15.111:0/766236024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564cb0142ba0 msgr2=0x564cb014c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.354+0000 7f786e800640 1 --2- 172.21.15.111:0/766236024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564cb0142ba0 0x564cb014c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.354+0000 7f786e800640 1 -- 172.21.15.111:0/766236024 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cafe86620 con 0x564cb0147c30 2024-09-15T23:02:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.355+0000 7f786e800640 1 --2- 172.21.15.111:0/766236024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cb0147c30 0x564cb01539a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7868005fd0 tx=0x7f7868000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9f4ed3fb6377280 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.355+0000 7f7853400640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:02:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.355+0000 7f7853400640 1 -- 172.21.15.111:0/766236024 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786802c060 con 0x564cb0147c30 2024-09-15T23:02:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.355+0000 7f7853400640 1 -- 172.21.15.111:0/766236024 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f7868004a40 con 0x564cb0147c30 2024-09-15T23:02:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.355+0000 7f7877a26740 1 -- 172.21.15.111:0/766236024 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7868004050 con 0x564cb0147c30 2024-09-15T23:02:06.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.355+0000 7f7853400640 1 -- 172.21.15.111:0/766236024 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7868004c40 con 0x564cb0147c30 2024-09-15T23:02:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.355+0000 7f7877a26740 10 client.26254 fetch_fsmap learned FSMap version 33 2024-09-15T23:02:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.355+0000 7f7877a26740 1 -- 172.21.15.111:0/766236024 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564cb0156790 con 0x564cb0147c30 2024-09-15T23:02:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.355+0000 7f7853400640 1 -- 172.21.15.111:0/766236024 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7868004050 con 0x564cb0147c30 2024-09-15T23:02:06.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.356+0000 7f7853400640 1 -- 172.21.15.111:0/766236024 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f786800f570 con 0x564cb0147c30 2024-09-15T23:02:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.356+0000 7f7877a26740 10 client.26254 fetch_fsmap finished waiting for FSMap version 33 2024-09-15T23:02:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.356+0000 7f7877a26740 10 client.26254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:02:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.356+0000 7f7877a26740 20 client.26254 populate_metadata read hostname 'smithi111' 2024-09-15T23:02:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.356+0000 7f7877a26740 1 --2- 172.21.15.111:0/766236024 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564cb0157bf0 0x564cb0159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.356+0000 7f7877a26740 4 client.26254 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:02:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.356+0000 7f7877a26740 1 -- 172.21.15.111:0/766236024 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cb00b4830 con 0x564cb0157bf0 2024-09-15T23:02:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.356+0000 7f7867e00640 1 --2- 172.21.15.111:0/766236024 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564cb0157bf0 0x564cb0159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.357+0000 7f7867e00640 1 --2- 172.21.15.111:0/766236024 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564cb0157bf0 0x564cb0159fe0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fff0b82ca9d81cad server_cookie=a45ae5158582ee7e in_seq=0 out_seq=0 2024-09-15T23:02:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.357+0000 7f7853400640 10 client.26254 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:02:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.359+0000 7f7853400640 1 -- 172.21.15.111:0/766236024 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564cb00b4830 con 0x564cb0157bf0 2024-09-15T23:02:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.359+0000 7f7853400640 10 client.26254 handle_command_reply: tid=0 2024-09-15T23:02:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.359+0000 7f7877a26740 1 client.26254 shutdown 2024-09-15T23:02:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.360+0000 7f7877a26740 1 -- 172.21.15.111:0/766236024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cb0147c30 msgr2=0x564cb01539a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.360+0000 7f7877a26740 1 --2- 172.21.15.111:0/766236024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cb0147c30 0x564cb01539a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7868005fd0 tx=0x7f7868000f40 comp rx=0 tx=0).stop 2024-09-15T23:02:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.360+0000 7f7877a26740 1 -- 172.21.15.111:0/766236024 shutdown_connections 2024-09-15T23:02:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.360+0000 7f7877a26740 1 --2- 172.21.15.111:0/766236024 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564cb0157bf0 0x564cb0159fe0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.360+0000 7f7877a26740 1 --2- 172.21.15.111:0/766236024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cb0147c30 0x564cb01539a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.360+0000 7f7877a26740 1 --2- 172.21.15.111:0/766236024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564cb0142ba0 0x564cb014c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.360+0000 7f7877a26740 1 --2- 172.21.15.111:0/766236024 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564cb0142260 0x564cb014c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.360+0000 7f7877a26740 1 -- 172.21.15.111:0/766236024 >> 172.21.15.111:0/766236024 conn(0x564cb013d8d0 msgr2=0x564cb013f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.360+0000 7f7853400640 0 client.26254 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:02:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.360+0000 7f7877a26740 1 -- 172.21.15.111:0/766236024 shutdown_connections 2024-09-15T23:02:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.360+0000 7f7877a26740 1 -- 172.21.15.111:0/766236024 wait complete. 2024-09-15T23:02:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.360+0000 7f7877a26740 20 client.26254 trim_cache size 0 max 16384 2024-09-15T23:02:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.378+0000 7f7877a26740 1 Processor -- start 2024-09-15T23:02:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.378+0000 7f7877a26740 1 -- start start 2024-09-15T23:02:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.379+0000 7f7877a26740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cb0142260 0x564cb01473d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.379+0000 7f7877a26740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564cb0142ba0 0x564cb01538c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.379+0000 7f7877a26740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564cb0147c30 0x564cb014c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.379+0000 7f7877a26740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f78701ad920 con 0x564cb0147c30 2024-09-15T23:02:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.379+0000 7f7877a26740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f78701ab1f0 con 0x564cb0142ba0 2024-09-15T23:02:06.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.379+0000 7f7877a26740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f78701a9040 con 0x564cb0142260 2024-09-15T23:02:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.379+0000 7f7867e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564cb0142ba0 0x564cb01538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:06.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.379+0000 7f786e800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cb0142260 0x564cb01473d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.379+0000 7f7867e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564cb0142ba0 0x564cb01538c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:39446/0 (socket says 172.21.15.111:39446) 2024-09-15T23:02:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.379+0000 7f7867e00640 1 -- 172.21.15.111:0/29924338 learned_addr learned my addr 172.21.15.111:0/29924338 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:02:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.379+0000 7f786e800640 1 --2- 172.21.15.111:0/29924338 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564cb0147c30 0x564cb014c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.379+0000 7f786e800640 1 -- 172.21.15.111:0/29924338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564cb0142ba0 msgr2=0x564cb01538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.379+0000 7f786e800640 1 --2- 172.21.15.111:0/29924338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564cb0142ba0 0x564cb01538c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.379+0000 7f786e800640 1 -- 172.21.15.111:0/29924338 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564cb0147c30 msgr2=0x564cb014c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.379+0000 7f786e800640 1 --2- 172.21.15.111:0/29924338 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564cb0147c30 0x564cb014c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.379+0000 7f786e800640 1 -- 172.21.15.111:0/29924338 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cb0143080 con 0x564cb0142260 2024-09-15T23:02:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.379+0000 7f7867e00640 1 --2- 172.21.15.111:0/29924338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564cb0142ba0 0x564cb01538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:02:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.380+0000 7f786e800640 1 --2- 172.21.15.111:0/29924338 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cb0142260 0x564cb01473d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f786802a040 tx=0x7f7868005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16df6d63f1902405 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.380+0000 7f785e600640 1 -- 172.21.15.111:0/29924338 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7868036070 con 0x564cb0142260 2024-09-15T23:02:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.380+0000 7f785e600640 1 -- 172.21.15.111:0/29924338 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f786800fb10 con 0x564cb0142260 2024-09-15T23:02:06.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.380+0000 7f785e600640 1 -- 172.21.15.111:0/29924338 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78680052a0 con 0x564cb0142260 2024-09-15T23:02:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.381+0000 7f7877a26740 1 -- 172.21.15.111:0/29924338 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cb0142260 msgr2=0x564cb01473d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.381+0000 7f7877a26740 1 --2- 172.21.15.111:0/29924338 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cb0142260 0x564cb01473d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f786802a040 tx=0x7f7868005f10 comp rx=0 tx=0).stop 2024-09-15T23:02:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.381+0000 7f7877a26740 1 -- 172.21.15.111:0/29924338 shutdown_connections 2024-09-15T23:02:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.381+0000 7f7877a26740 1 --2- 172.21.15.111:0/29924338 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564cb0147c30 0x564cb014c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.381+0000 7f7877a26740 1 --2- 172.21.15.111:0/29924338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564cb0142ba0 0x564cb01538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.381+0000 7f7877a26740 1 --2- 172.21.15.111:0/29924338 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cb0142260 0x564cb01473d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.381+0000 7f7877a26740 1 -- 172.21.15.111:0/29924338 >> 172.21.15.111:0/29924338 conn(0x564cb013d8d0 msgr2=0x564cb0143de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.381+0000 7f7877a26740 1 -- 172.21.15.111:0/29924338 shutdown_connections 2024-09-15T23:02:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.381+0000 7f7877a26740 1 -- 172.21.15.111:0/29924338 wait complete. 2024-09-15T23:02:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.381+0000 7f7877a26740 1 -- start start 2024-09-15T23:02:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.381+0000 7f7877a26740 1 Processor -- start 2024-09-15T23:02:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.382+0000 7f7877a26740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564cb0142260 0x564cb0153830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.382+0000 7f7877a26740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564cb0142ba0 0x564cb0150320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.382+0000 7f7877a26740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cb0147c30 0x564cb0150860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.382+0000 7f7877a26740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f78701a96b0 con 0x564cb0142260 2024-09-15T23:02:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.382+0000 7f7877a26740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f78701aab20 con 0x564cb0142ba0 2024-09-15T23:02:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.382+0000 7f7877a26740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f78701a9d80 con 0x564cb0147c30 2024-09-15T23:02:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.382+0000 7f7877a26740 10 client.0 _ll_register_callbacks cb 0x7f7868009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:02:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.382+0000 7f7867e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564cb0142ba0 0x564cb0150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.382+0000 7f7867e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564cb0142ba0 0x564cb0150320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:39452/0 (socket says 172.21.15.111:39452) 2024-09-15T23:02:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.382+0000 7f786e800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564cb0142260 0x564cb0153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.382+0000 7f7867e00640 1 -- 172.21.15.111:0/2696198317 learned_addr learned my addr 172.21.15.111:0/2696198317 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:02:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.382+0000 7f786e800640 1 --2- 172.21.15.111:0/2696198317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cb0147c30 0x564cb0150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.383+0000 7f7867e00640 1 -- 172.21.15.111:0/2696198317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cb0147c30 msgr2=0x564cb0150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.383+0000 7f7867e00640 1 --2- 172.21.15.111:0/2696198317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cb0147c30 0x564cb0150860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.383+0000 7f7867e00640 1 -- 172.21.15.111:0/2696198317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564cb0142260 msgr2=0x564cb0153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.383+0000 7f7867e00640 1 --2- 172.21.15.111:0/2696198317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564cb0142260 0x564cb0153830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.383+0000 7f7867e00640 1 -- 172.21.15.111:0/2696198317 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cb01c08b0 con 0x564cb0142ba0 2024-09-15T23:02:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.383+0000 7f7867e00640 1 --2- 172.21.15.111:0/2696198317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564cb0142ba0 0x564cb0150320 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7858005920 tx=0x7f7858009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c549aef73bd85341 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.383+0000 7f7853400640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:02:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.383+0000 7f7853400640 1 -- 172.21.15.111:0/2696198317 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7858035040 con 0x564cb0142ba0 2024-09-15T23:02:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.383+0000 7f7877a26740 1 -- 172.21.15.111:0/2696198317 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f786800fb10 con 0x564cb0142ba0 2024-09-15T23:02:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.383+0000 7f7853400640 1 -- 172.21.15.111:0/2696198317 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f7858005040 con 0x564cb0142ba0 2024-09-15T23:02:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.383+0000 7f7853400640 1 -- 172.21.15.111:0/2696198317 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7858054050 con 0x564cb0142ba0 2024-09-15T23:02:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.383+0000 7f7877a26740 10 client.26243 fetch_fsmap learned FSMap version 33 2024-09-15T23:02:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.384+0000 7f7877a26740 1 -- 172.21.15.111:0/2696198317 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564cb01c1c40 con 0x564cb0142ba0 2024-09-15T23:02:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.384+0000 7f7853400640 1 -- 172.21.15.111:0/2696198317 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f786800fb10 con 0x564cb0142ba0 2024-09-15T23:02:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.384+0000 7f7853400640 1 -- 172.21.15.111:0/2696198317 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f78580316b0 con 0x564cb0142ba0 2024-09-15T23:02:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.384+0000 7f7877a26740 10 client.26243 fetch_fsmap finished waiting for FSMap version 33 2024-09-15T23:02:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.384+0000 7f7877a26740 10 client.26243 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:02:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.384+0000 7f7877a26740 20 client.26243 populate_metadata read hostname 'smithi111' 2024-09-15T23:02:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.384+0000 7f7877a26740 1 --2- 172.21.15.111:0/2696198317 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564cb01c25a0 0x564cb01c49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.384+0000 7f7877a26740 4 client.26243 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:02:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.384+0000 7f7877a26740 1 -- 172.21.15.111:0/2696198317 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564cb00aeb80 con 0x564cb01c25a0 2024-09-15T23:02:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.384+0000 7f786e800640 1 --2- 172.21.15.111:0/2696198317 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564cb01c25a0 0x564cb01c49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.385+0000 7f786e800640 1 --2- 172.21.15.111:0/2696198317 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564cb01c25a0 0x564cb01c49c0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35122502202f7250 server_cookie=3463d8543c907ef7 in_seq=0 out_seq=0 2024-09-15T23:02:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.385+0000 7f7853400640 10 client.26243 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:02:06.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:05 smithi121 ceph-mon[34815]: pgmap v836: 97 pgs: 97 active+clean; 477 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 3.2 MiB/s wr, 153 op/s 2024-09-15T23:02:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.509+0000 7f7853400640 1 -- 172.21.15.111:0/2696198317 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564cb00aeb80 con 0x564cb01c25a0 2024-09-15T23:02:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.509+0000 7f7853400640 10 client.26243 handle_command_reply: tid=0 2024-09-15T23:02:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.509+0000 7f7877a26740 1 client.26243 shutdown 2024-09-15T23:02:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.509+0000 7f7877a26740 1 -- 172.21.15.111:0/2696198317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564cb0142ba0 msgr2=0x564cb0150320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.509+0000 7f7877a26740 1 --2- 172.21.15.111:0/2696198317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564cb0142ba0 0x564cb0150320 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7858005920 tx=0x7f7858009610 comp rx=0 tx=0).stop 2024-09-15T23:02:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.509+0000 7f7877a26740 1 -- 172.21.15.111:0/2696198317 shutdown_connections 2024-09-15T23:02:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.509+0000 7f7877a26740 1 --2- 172.21.15.111:0/2696198317 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x564cb01c25a0 0x564cb01c49c0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.509+0000 7f7877a26740 1 --2- 172.21.15.111:0/2696198317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x564cb0147c30 0x564cb0150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.509+0000 7f7877a26740 1 --2- 172.21.15.111:0/2696198317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564cb0142ba0 0x564cb0150320 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.509+0000 7f7877a26740 1 --2- 172.21.15.111:0/2696198317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564cb0142260 0x564cb0153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.509+0000 7f7853400640 0 client.26243 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:02:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.509+0000 7f7877a26740 1 -- 172.21.15.111:0/2696198317 >> 172.21.15.111:0/2696198317 conn(0x564cb013d8d0 msgr2=0x564cb0143de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.509+0000 7f7877a26740 1 -- 172.21.15.111:0/2696198317 shutdown_connections 2024-09-15T23:02:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.509+0000 7f7877a26740 1 -- 172.21.15.111:0/2696198317 wait complete. 2024-09-15T23:02:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.509+0000 7f7877a26740 20 client.26243 trim_cache size 0 max 16384 2024-09-15T23:02:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:02:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:02:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:02:06.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:02:06.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.510+0000 7f786f200640 1 -- 172.21.15.111:0/3155695704 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f78540a3240 msgr2=0x7f78540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:06.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.510+0000 7f786f200640 1 --2- 172.21.15.111:0/3155695704 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f78540a3240 0x7f78540a5700 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7860097de0 tx=0x7f78600107f0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.510+0000 7f786f200640 1 -- 172.21.15.111:0/3155695704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78701023c0 msgr2=0x7f7870199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:06.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.510+0000 7f786f200640 1 --2- 172.21.15.111:0/3155695704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78701023c0 0x7f7870199fd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f786000b450 tx=0x7f786000b910 comp rx=0 tx=0).stop 2024-09-15T23:02:06.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.510+0000 7f786f200640 1 -- 172.21.15.111:0/3155695704 shutdown_connections 2024-09-15T23:02:06.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.510+0000 7f786f200640 1 --2- 172.21.15.111:0/3155695704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7870107620 0x7f78701a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.510+0000 7f786f200640 1 --2- 172.21.15.111:0/3155695704 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f78540a3240 0x7f78540a5700 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.510+0000 7f786f200640 1 --2- 172.21.15.111:0/3155695704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7870104cf0 0x7f787019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.510+0000 7f786f200640 1 --2- 172.21.15.111:0/3155695704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78701023c0 0x7f7870199fd0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:06.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.510+0000 7f786f200640 1 -- 172.21.15.111:0/3155695704 >> 172.21.15.111:0/3155695704 conn(0x7f78700fbfa0 msgr2=0x7f7870101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:06.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.510+0000 7f786f200640 1 -- 172.21.15.111:0/3155695704 shutdown_connections 2024-09-15T23:02:06.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:06.510+0000 7f786f200640 1 -- 172.21.15.111:0/3155695704 wait complete. 2024-09-15T23:02:06.636 INFO:tasks.workunit.client.0.smithi111.stdout: 10 49852 2.63 MB/sec execute 34 sec latency 75.519 ms 2024-09-15T23:02:07.636 INFO:tasks.workunit.client.0.smithi111.stdout: 10 50539 3.40 MB/sec execute 35 sec latency 69.431 ms 2024-09-15T23:02:08.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:07 smithi111 ceph-mon[30629]: pgmap v837: 97 pgs: 97 active+clean; 476 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 2.7 MiB/s wr, 141 op/s 2024-09-15T23:02:08.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:07 smithi192 ceph-mon[35427]: pgmap v837: 97 pgs: 97 active+clean; 476 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 2.7 MiB/s wr, 141 op/s 2024-09-15T23:02:08.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:07 smithi121 ceph-mon[34815]: pgmap v837: 97 pgs: 97 active+clean; 476 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 2.7 MiB/s wr, 141 op/s 2024-09-15T23:02:08.637 INFO:tasks.workunit.client.0.smithi111.stdout: 10 51321 4.49 MB/sec execute 36 sec latency 90.573 ms 2024-09-15T23:02:09.637 INFO:tasks.workunit.client.0.smithi111.stdout: 10 52125 5.40 MB/sec execute 37 sec latency 70.538 ms 2024-09-15T23:02:10.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:09 smithi192 ceph-mon[35427]: pgmap v838: 97 pgs: 97 active+clean; 479 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 303 op/s 2024-09-15T23:02:10.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:09 smithi121 ceph-mon[34815]: pgmap v838: 97 pgs: 97 active+clean; 479 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 303 op/s 2024-09-15T23:02:10.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:09 smithi111 ceph-mon[30629]: pgmap v838: 97 pgs: 97 active+clean; 479 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 303 op/s 2024-09-15T23:02:10.637 INFO:tasks.workunit.client.0.smithi111.stdout: 10 52834 6.40 MB/sec execute 38 sec latency 81.204 ms 2024-09-15T23:02:11.637 INFO:tasks.workunit.client.0.smithi111.stdout: 10 53594 7.47 MB/sec execute 39 sec latency 48.581 ms 2024-09-15T23:02:12.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:11 smithi192 ceph-mon[35427]: pgmap v839: 97 pgs: 97 active+clean; 475 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.9 MiB/s wr, 328 op/s 2024-09-15T23:02:12.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:11 smithi121 ceph-mon[34815]: pgmap v839: 97 pgs: 97 active+clean; 475 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.9 MiB/s wr, 328 op/s 2024-09-15T23:02:12.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:11 smithi111 ceph-mon[30629]: pgmap v839: 97 pgs: 97 active+clean; 475 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.9 MiB/s wr, 328 op/s 2024-09-15T23:02:12.637 INFO:tasks.workunit.client.0.smithi111.stdout: 10 54378 8.24 MB/sec execute 40 sec latency 58.444 ms 2024-09-15T23:02:13.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:13 smithi192 ceph-mon[35427]: pgmap v840: 97 pgs: 97 active+clean; 478 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.4 MiB/s wr, 373 op/s 2024-09-15T23:02:13.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:13 smithi192 ceph-mon[35427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T23:02:13.348 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:13 smithi111 ceph-mon[30629]: pgmap v840: 97 pgs: 97 active+clean; 478 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.4 MiB/s wr, 373 op/s 2024-09-15T23:02:13.348 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:13 smithi111 ceph-mon[30629]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T23:02:13.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:13 smithi121 ceph-mon[34815]: pgmap v840: 97 pgs: 97 active+clean; 478 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.4 MiB/s wr, 373 op/s 2024-09-15T23:02:13.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:13 smithi121 ceph-mon[34815]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T23:02:13.637 INFO:tasks.workunit.client.0.smithi111.stdout: 10 54984 8.78 MB/sec execute 41 sec latency 81.435 ms 2024-09-15T23:02:14.637 INFO:tasks.workunit.client.0.smithi111.stdout: 10 55526 9.17 MB/sec execute 42 sec latency 122.643 ms 2024-09-15T23:02:15.637 INFO:tasks.workunit.client.0.smithi111.stdout: 10 56327 10.01 MB/sec execute 43 sec latency 99.803 ms 2024-09-15T23:02:16.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:15 smithi111 ceph-mon[30629]: pgmap v841: 97 pgs: 97 active+clean; 483 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.9 MiB/s wr, 418 op/s 2024-09-15T23:02:16.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:15 smithi192 ceph-mon[35427]: pgmap v841: 97 pgs: 97 active+clean; 483 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.9 MiB/s wr, 418 op/s 2024-09-15T23:02:16.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:15 smithi121 ceph-mon[34815]: pgmap v841: 97 pgs: 97 active+clean; 483 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.9 MiB/s wr, 418 op/s 2024-09-15T23:02:16.637 INFO:tasks.workunit.client.0.smithi111.stdout: 10 57222 10.80 MB/sec execute 44 sec latency 50.476 ms 2024-09-15T23:02:17.640 INFO:tasks.workunit.client.0.smithi111.stdout: 10 58117 11.86 MB/sec execute 45 sec latency 44.136 ms 2024-09-15T23:02:18.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:17 smithi111 ceph-mon[30629]: pgmap v842: 97 pgs: 97 active+clean; 486 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.3 MiB/s wr, 458 op/s 2024-09-15T23:02:18.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:17 smithi192 ceph-mon[35427]: pgmap v842: 97 pgs: 97 active+clean; 486 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.3 MiB/s wr, 458 op/s 2024-09-15T23:02:18.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:17 smithi121 ceph-mon[34815]: pgmap v842: 97 pgs: 97 active+clean; 486 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.3 MiB/s wr, 458 op/s 2024-09-15T23:02:18.637 INFO:tasks.workunit.client.0.smithi111.stdout: 10 58752 12.38 MB/sec execute 46 sec latency 91.951 ms 2024-09-15T23:02:19.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:18 smithi111 ceph-mon[30629]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T23:02:19.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:18 smithi192 ceph-mon[35427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T23:02:19.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:18 smithi121 ceph-mon[34815]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T23:02:19.637 INFO:tasks.workunit.client.0.smithi111.stdout: 10 59468 12.84 MB/sec execute 47 sec latency 100.078 ms 2024-09-15T23:02:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.640+0000 7f26c7200640 1 -- 172.21.15.111:0/708355614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26c0103d60 msgr2=0x7f26c0104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.640+0000 7f26c7200640 1 --2- 172.21.15.111:0/708355614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26c0103d60 0x7f26c0104160 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009900 tx=0x7f26b002a3f0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.640+0000 7f26c7200640 1 -- 172.21.15.111:0/708355614 shutdown_connections 2024-09-15T23:02:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.640+0000 7f26c7200640 1 --2- 172.21.15.111:0/708355614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26c0105910 0x7f26c010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.640+0000 7f26c7200640 1 --2- 172.21.15.111:0/708355614 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26c0104f50 0x7f26c01053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.640+0000 7f26c7200640 1 --2- 172.21.15.111:0/708355614 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26c0103d60 0x7f26c0104160 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.640+0000 7f26c7200640 1 -- 172.21.15.111:0/708355614 >> 172.21.15.111:0/708355614 conn(0x7f26c00ff4e0 msgr2=0x7f26c0101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.640+0000 7f26c7200640 1 -- 172.21.15.111:0/708355614 shutdown_connections 2024-09-15T23:02:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.640+0000 7f26c7200640 1 -- 172.21.15.111:0/708355614 wait complete. 2024-09-15T23:02:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.641+0000 7f26c7200640 1 Processor -- start 2024-09-15T23:02:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.641+0000 7f26c7200640 1 -- start start 2024-09-15T23:02:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.641+0000 7f26c7200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26c0103d60 0x7f26c019a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.641+0000 7f26c7200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26c0104f50 0x7f26c019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.641+0000 7f26c7200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26c0105910 0x7f26c01a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.641+0000 7f26c7200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f26c010fa30 con 0x7f26c0104f50 2024-09-15T23:02:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.641+0000 7f26c7200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f26c010f8b0 con 0x7f26c0103d60 2024-09-15T23:02:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.641+0000 7f26c7200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f26c010fbb0 con 0x7f26c0105910 2024-09-15T23:02:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.642+0000 7f26bfe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26c0103d60 0x7f26c019a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.642+0000 7f26bfe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26c0103d60 0x7f26c019a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:33826/0 (socket says 172.21.15.111:33826) 2024-09-15T23:02:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.642+0000 7f26bfe00640 1 -- 172.21.15.111:0/3079319641 learned_addr learned my addr 172.21.15.111:0/3079319641 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:02:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.642+0000 7f26bf400640 1 --2- 172.21.15.111:0/3079319641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26c0104f50 0x7f26c019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.642+0000 7f26c5200640 1 --2- 172.21.15.111:0/3079319641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26c0105910 0x7f26c01a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.642+0000 7f26bfe00640 1 -- 172.21.15.111:0/3079319641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26c0105910 msgr2=0x7f26c01a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.642+0000 7f26bfe00640 1 --2- 172.21.15.111:0/3079319641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26c0105910 0x7f26c01a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.642+0000 7f26bfe00640 1 -- 172.21.15.111:0/3079319641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26c0104f50 msgr2=0x7f26c019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.642+0000 7f26bfe00640 1 --2- 172.21.15.111:0/3079319641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26c0104f50 0x7f26c019a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.642+0000 7f26bfe00640 1 -- 172.21.15.111:0/3079319641 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c01a1c20 con 0x7f26c0103d60 2024-09-15T23:02:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.643+0000 7f26bfe00640 1 --2- 172.21.15.111:0/3079319641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26c0103d60 0x7f26c019a000 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f26b0031440 tx=0x7f26b0032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.643+0000 7f26bcc00640 1 -- 172.21.15.111:0/3079319641 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b003f030 con 0x7f26c0103d60 2024-09-15T23:02:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.643+0000 7f26bcc00640 1 -- 172.21.15.111:0/3079319641 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b0003ce0 con 0x7f26c0103d60 2024-09-15T23:02:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.643+0000 7f26c7200640 1 -- 172.21.15.111:0/3079319641 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c01a1eb0 con 0x7f26c0103d60 2024-09-15T23:02:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.643+0000 7f26bcc00640 1 -- 172.21.15.111:0/3079319641 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0035070 con 0x7f26c0103d60 2024-09-15T23:02:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.643+0000 7f26c7200640 1 -- 172.21.15.111:0/3079319641 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c01a22c0 con 0x7f26c0103d60 2024-09-15T23:02:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.647+0000 7f26bcc00640 1 -- 172.21.15.111:0/3079319641 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f26b0042030 con 0x7f26c0103d60 2024-09-15T23:02:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.647+0000 7f26ca0ac740 1 Processor -- start 2024-09-15T23:02:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.648+0000 7f26ca0ac740 1 -- start start 2024-09-15T23:02:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.648+0000 7f26bcc00640 1 --2- 172.21.15.111:0/3079319641 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f26a40a3460 0x7f26a40a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.648+0000 7f26ca0ac740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b85a74090 0x555b85a74490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.648+0000 7f26ca0ac740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b85a749d0 0x555b85a79490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.648+0000 7f26ca0ac740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b85a79a60 0x555b85a7bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.648+0000 7f26ca0ac740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555b859d9100 con 0x555b85a74090 2024-09-15T23:02:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.648+0000 7f26ca0ac740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x555b859811b0 con 0x555b85a79a60 2024-09-15T23:02:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.648+0000 7f26ca0ac740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555b859d50c0 con 0x555b85a749d0 2024-09-15T23:02:19.651 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.648+0000 7f26bf400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b85a749d0 0x555b85a79490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:19.651 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.648+0000 7f26c5200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b85a74090 0x555b85a74490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:19.651 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.648+0000 7f26bf400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b85a749d0 0x555b85a79490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:42364/0 (socket says 172.21.15.111:42364) 2024-09-15T23:02:19.651 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.648+0000 7f26bf400640 1 -- 172.21.15.111:0/2041409060 learned_addr learned my addr 172.21.15.111:0/2041409060 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:02:19.651 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.648+0000 7f26bcc00640 1 -- 172.21.15.111:0/3079319641 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f26b00effd0 con 0x7f26c0103d60 2024-09-15T23:02:19.651 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.649+0000 7f26c5200640 1 --2- 172.21.15.111:0/2041409060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b85a79a60 0x555b85a7bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:19.651 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.649+0000 7f26bf400640 1 -- 172.21.15.111:0/2041409060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b85a79a60 msgr2=0x555b85a7bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:19.651 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.649+0000 7f26bf400640 1 --2- 172.21.15.111:0/2041409060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b85a79a60 0x555b85a7bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.649+0000 7f26bf400640 1 -- 172.21.15.111:0/2041409060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b85a74090 msgr2=0x555b85a74490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.649+0000 7f26bf400640 1 --2- 172.21.15.111:0/2041409060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b85a74090 0x555b85a74490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.649+0000 7f26bf400640 1 -- 172.21.15.111:0/2041409060 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b856ec210 con 0x555b85a749d0 2024-09-15T23:02:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.649+0000 7f26c5200640 1 --2- 172.21.15.111:0/2041409060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b85a79a60 0x555b85a7bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:02:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.649+0000 7f26bfe00640 1 --2- 172.21.15.111:0/3079319641 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f26a40a3460 0x7f26a40a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.649+0000 7f26bf400640 1 --2- 172.21.15.111:0/2041409060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b85a749d0 0x555b85a79490 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009c90 tx=0x7f26ac011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a67e7a72d95c4c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.650+0000 7f26b5600640 1 -- 172.21.15.111:0/2041409060 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac01e070 con 0x555b85a749d0 2024-09-15T23:02:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.650+0000 7f26b5600640 1 -- 172.21.15.111:0/2041409060 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ac019b10 con 0x555b85a749d0 2024-09-15T23:02:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.650+0000 7f26b5600640 1 -- 172.21.15.111:0/2041409060 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac00eac0 con 0x555b85a749d0 2024-09-15T23:02:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.650+0000 7f26ca0ac740 1 -- 172.21.15.111:0/2041409060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b85a749d0 msgr2=0x555b85a79490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.650+0000 7f26bfe00640 1 --2- 172.21.15.111:0/3079319641 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f26a40a3460 0x7f26a40a5920 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f26b00b8b00 tx=0x7f26b0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.650+0000 7f26ca0ac740 1 --2- 172.21.15.111:0/2041409060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b85a749d0 0x555b85a79490 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f26ac009c90 tx=0x7f26ac011810 comp rx=0 tx=0).stop 2024-09-15T23:02:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.651+0000 7f26ca0ac740 1 -- 172.21.15.111:0/2041409060 shutdown_connections 2024-09-15T23:02:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.651+0000 7f26ca0ac740 1 --2- 172.21.15.111:0/2041409060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b85a79a60 0x555b85a7bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.651+0000 7f26ca0ac740 1 --2- 172.21.15.111:0/2041409060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b85a749d0 0x555b85a79490 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.651+0000 7f26ca0ac740 1 --2- 172.21.15.111:0/2041409060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b85a74090 0x555b85a74490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.651+0000 7f26ca0ac740 1 -- 172.21.15.111:0/2041409060 >> 172.21.15.111:0/2041409060 conn(0x555b85a6f700 msgr2=0x555b85a6fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.651+0000 7f26ca0ac740 1 -- 172.21.15.111:0/2041409060 shutdown_connections 2024-09-15T23:02:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.651+0000 7f26ca0ac740 1 -- 172.21.15.111:0/2041409060 wait complete. 2024-09-15T23:02:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.651+0000 7f26ca0ac740 1 -- start start 2024-09-15T23:02:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.651+0000 7f26ca0ac740 1 Processor -- start 2024-09-15T23:02:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.652+0000 7f26ca0ac740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b85a74090 0x555b85a7e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.652+0000 7f26ca0ac740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b85a749d0 0x555b85a7e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.652+0000 7f26ca0ac740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b85a79a60 0x555b85a85860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.652+0000 7f26ca0ac740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x555b85a7ca30 con 0x555b85a749d0 2024-09-15T23:02:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.652+0000 7f26ca0ac740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x555b85962090 con 0x555b85a79a60 2024-09-15T23:02:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.652+0000 7f26ca0ac740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555b85a7cbb0 con 0x555b85a74090 2024-09-15T23:02:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.652+0000 7f26c5200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b85a74090 0x555b85a7e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.652+0000 7f26c5200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b85a74090 0x555b85a7e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:42376/0 (socket says 172.21.15.111:42376) 2024-09-15T23:02:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.652+0000 7f26c5200640 1 -- 172.21.15.111:0/529019606 learned_addr learned my addr 172.21.15.111:0/529019606 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:02:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.652+0000 7f26ca0ac740 10 client.0 _ll_register_callbacks cb 0x555b859b6f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:02:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.652+0000 7f26c5200640 1 --2- 172.21.15.111:0/529019606 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b85a79a60 0x555b85a85860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.652+0000 7f26bf400640 1 --2- 172.21.15.111:0/529019606 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b85a749d0 0x555b85a7e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.653+0000 7f26c5200640 1 -- 172.21.15.111:0/529019606 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b85a79a60 msgr2=0x555b85a85860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.653+0000 7f26c5200640 1 --2- 172.21.15.111:0/529019606 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b85a79a60 0x555b85a85860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.653+0000 7f26c5200640 1 -- 172.21.15.111:0/529019606 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b85a749d0 msgr2=0x555b85a7e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.653+0000 7f26c5200640 1 --2- 172.21.15.111:0/529019606 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b85a749d0 0x555b85a7e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.653+0000 7f26c5200640 1 -- 172.21.15.111:0/529019606 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b857b8620 con 0x555b85a74090 2024-09-15T23:02:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.653+0000 7f26c5200640 1 --2- 172.21.15.111:0/529019606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b85a74090 0x555b85a7e2e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f26b8010430 tx=0x7f26b8009e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a99e23d8f418d46 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.653+0000 7f26aa000640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:02:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.654+0000 7f26aa000640 1 -- 172.21.15.111:0/529019606 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b801c040 con 0x555b85a74090 2024-09-15T23:02:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.654+0000 7f26aa000640 1 -- 172.21.15.111:0/529019606 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f26b800f5c0 con 0x555b85a74090 2024-09-15T23:02:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.654+0000 7f26ca0ac740 1 -- 172.21.15.111:0/529019606 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26ac019b10 con 0x555b85a74090 2024-09-15T23:02:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.654+0000 7f26aa000640 1 -- 172.21.15.111:0/529019606 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f26b800f7c0 con 0x555b85a74090 2024-09-15T23:02:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.654+0000 7f26ca0ac740 10 client.26272 fetch_fsmap learned FSMap version 35 2024-09-15T23:02:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.654+0000 7f26ca0ac740 1 -- 172.21.15.111:0/529019606 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555b85a88770 con 0x555b85a74090 2024-09-15T23:02:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.655+0000 7f26aa000640 1 -- 172.21.15.111:0/529019606 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ac019b10 con 0x555b85a74090 2024-09-15T23:02:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.655+0000 7f26aa000640 1 -- 172.21.15.111:0/529019606 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f26b80229f0 con 0x555b85a74090 2024-09-15T23:02:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.655+0000 7f26ca0ac740 10 client.26272 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:02:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.655+0000 7f26ca0ac740 10 client.26272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:02:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.655+0000 7f26ca0ac740 20 client.26272 populate_metadata read hostname 'smithi111' 2024-09-15T23:02:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.655+0000 7f26ca0ac740 1 --2- 172.21.15.111:0/529019606 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555b85a89e20 0x555b85a8c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.655+0000 7f26ca0ac740 4 client.26272 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:02:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.655+0000 7f26ca0ac740 1 -- 172.21.15.111:0/529019606 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b859e6830 con 0x555b85a89e20 2024-09-15T23:02:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.656+0000 7f26bf400640 1 --2- 172.21.15.111:0/529019606 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555b85a89e20 0x555b85a8c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.656+0000 7f26bf400640 1 --2- 172.21.15.111:0/529019606 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555b85a89e20 0x555b85a8c270 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cf74b6e0b16cfd0 server_cookie=dab06231793f279e in_seq=0 out_seq=0 2024-09-15T23:02:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.656+0000 7f26aa000640 10 client.26272 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:02:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.657+0000 7f26aa000640 1 -- 172.21.15.111:0/529019606 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b859e6830 con 0x555b85a89e20 2024-09-15T23:02:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.657+0000 7f26aa000640 10 client.26272 handle_command_reply: tid=0 2024-09-15T23:02:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.657+0000 7f26ca0ac740 1 client.26272 shutdown 2024-09-15T23:02:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.658+0000 7f26ca0ac740 1 -- 172.21.15.111:0/529019606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b85a74090 msgr2=0x555b85a7e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.658+0000 7f26ca0ac740 1 --2- 172.21.15.111:0/529019606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b85a74090 0x555b85a7e2e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f26b8010430 tx=0x7f26b8009e80 comp rx=0 tx=0).stop 2024-09-15T23:02:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.658+0000 7f26ca0ac740 1 -- 172.21.15.111:0/529019606 shutdown_connections 2024-09-15T23:02:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.658+0000 7f26ca0ac740 1 --2- 172.21.15.111:0/529019606 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555b85a89e20 0x555b85a8c270 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.658+0000 7f26ca0ac740 1 --2- 172.21.15.111:0/529019606 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b85a79a60 0x555b85a85860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.658+0000 7f26ca0ac740 1 --2- 172.21.15.111:0/529019606 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b85a749d0 0x555b85a7e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.658+0000 7f26ca0ac740 1 --2- 172.21.15.111:0/529019606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b85a74090 0x555b85a7e2e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.658+0000 7f26ca0ac740 1 -- 172.21.15.111:0/529019606 >> 172.21.15.111:0/529019606 conn(0x555b85a6f700 msgr2=0x555b85a70f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.658+0000 7f26aa000640 0 client.26272 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:02:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.658+0000 7f26ca0ac740 1 -- 172.21.15.111:0/529019606 shutdown_connections 2024-09-15T23:02:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.658+0000 7f26ca0ac740 1 -- 172.21.15.111:0/529019606 wait complete. 2024-09-15T23:02:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.658+0000 7f26ca0ac740 20 client.26272 trim_cache size 0 max 16384 2024-09-15T23:02:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.675+0000 7f26ca0ac740 1 Processor -- start 2024-09-15T23:02:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.675+0000 7f26ca0ac740 1 -- start start 2024-09-15T23:02:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.676+0000 7f26ca0ac740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b85a74090 0x555b85a7e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.676+0000 7f26ca0ac740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b85a749d0 0x555b85a857f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.676+0000 7f26ca0ac740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b85a79a60 0x555b85a705b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.676+0000 7f26ca0ac740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f26c01ad8e0 con 0x555b85a74090 2024-09-15T23:02:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.676+0000 7f26ca0ac740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f26c01ab1b0 con 0x555b85a79a60 2024-09-15T23:02:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.676+0000 7f26ca0ac740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f26c01a9000 con 0x555b85a749d0 2024-09-15T23:02:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.676+0000 7f26c5200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b85a74090 0x555b85a7e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:19.678 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.676+0000 7f26bf400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b85a749d0 0x555b85a857f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:19.679 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.676+0000 7f26c5200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b85a74090 0x555b85a7e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52552/0 (socket says 172.21.15.111:52552) 2024-09-15T23:02:19.679 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.676+0000 7f26bf400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b85a749d0 0x555b85a857f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:42380/0 (socket says 172.21.15.111:42380) 2024-09-15T23:02:19.679 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.676+0000 7f26c5200640 1 -- 172.21.15.111:0/2326829052 learned_addr learned my addr 172.21.15.111:0/2326829052 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:02:19.679 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.676+0000 7f26c5200640 1 --2- 172.21.15.111:0/2326829052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b85a79a60 0x555b85a705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:19.679 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.676+0000 7f26bf400640 1 -- 172.21.15.111:0/2326829052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b85a79a60 msgr2=0x555b85a705b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:19.679 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.676+0000 7f26bf400640 1 --2- 172.21.15.111:0/2326829052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b85a79a60 0x555b85a705b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.679 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.676+0000 7f26bf400640 1 -- 172.21.15.111:0/2326829052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b85a74090 msgr2=0x555b85a7e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:19.679 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.676+0000 7f26bf400640 1 --2- 172.21.15.111:0/2326829052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b85a74090 0x555b85a7e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.679 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.676+0000 7f26bf400640 1 -- 172.21.15.111:0/2326829052 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b85a6f520 con 0x555b85a749d0 2024-09-15T23:02:19.679 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.676+0000 7f26c5200640 1 --2- 172.21.15.111:0/2326829052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b85a79a60 0x555b85a705b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:02:19.679 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.677+0000 7f26bf400640 1 --2- 172.21.15.111:0/2326829052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b85a749d0 0x555b85a857f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f26ac01c040 tx=0x7f26ac005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1efab34f3e71490 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.677+0000 7f26b4c00640 1 -- 172.21.15.111:0/2326829052 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac01e040 con 0x555b85a749d0 2024-09-15T23:02:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.677+0000 7f26b4c00640 1 -- 172.21.15.111:0/2326829052 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ac0054d0 con 0x555b85a749d0 2024-09-15T23:02:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.677+0000 7f26b4c00640 1 -- 172.21.15.111:0/2326829052 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac003850 con 0x555b85a749d0 2024-09-15T23:02:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.678+0000 7f26ca0ac740 1 -- 172.21.15.111:0/2326829052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b85a749d0 msgr2=0x555b85a857f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.678+0000 7f26ca0ac740 1 --2- 172.21.15.111:0/2326829052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b85a749d0 0x555b85a857f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f26ac01c040 tx=0x7f26ac005850 comp rx=0 tx=0).stop 2024-09-15T23:02:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.678+0000 7f26ca0ac740 1 -- 172.21.15.111:0/2326829052 shutdown_connections 2024-09-15T23:02:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.678+0000 7f26ca0ac740 1 --2- 172.21.15.111:0/2326829052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b85a79a60 0x555b85a705b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.678+0000 7f26ca0ac740 1 --2- 172.21.15.111:0/2326829052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b85a749d0 0x555b85a857f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.678+0000 7f26ca0ac740 1 --2- 172.21.15.111:0/2326829052 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b85a74090 0x555b85a7e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.678+0000 7f26ca0ac740 1 -- 172.21.15.111:0/2326829052 >> 172.21.15.111:0/2326829052 conn(0x555b85a6f700 msgr2=0x555b85a7e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.678+0000 7f26ca0ac740 1 -- 172.21.15.111:0/2326829052 shutdown_connections 2024-09-15T23:02:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.678+0000 7f26ca0ac740 1 -- 172.21.15.111:0/2326829052 wait complete. 2024-09-15T23:02:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.678+0000 7f26ca0ac740 1 -- start start 2024-09-15T23:02:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.679+0000 7f26ca0ac740 1 Processor -- start 2024-09-15T23:02:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.679+0000 7f26ca0ac740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b85a74090 0x555b85a82b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.679+0000 7f26ca0ac740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b85a749d0 0x555b85a83050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.679+0000 7f26ca0ac740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b85a79a60 0x555b85a84060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.679+0000 7f26ca0ac740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f26c01a9670 con 0x555b85a79a60 2024-09-15T23:02:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.679+0000 7f26ca0ac740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f26c01aaae0 con 0x555b85a74090 2024-09-15T23:02:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.679+0000 7f26ca0ac740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f26c01a9d40 con 0x555b85a749d0 2024-09-15T23:02:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.679+0000 7f26ca0ac740 10 client.0 _ll_register_callbacks cb 0x7f26b8013c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:02:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.679+0000 7f26c5200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b85a74090 0x555b85a82b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.679+0000 7f26bf400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b85a749d0 0x555b85a83050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.680+0000 7f26c5200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b85a74090 0x555b85a82b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:33842/0 (socket says 172.21.15.111:33842) 2024-09-15T23:02:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.680+0000 7f26c5200640 1 -- 172.21.15.111:0/2711000019 learned_addr learned my addr 172.21.15.111:0/2711000019 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:02:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.680+0000 7f26c5200640 1 --2- 172.21.15.111:0/2711000019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b85a79a60 0x555b85a84060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.680+0000 7f26bf400640 1 -- 172.21.15.111:0/2711000019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b85a74090 msgr2=0x555b85a82b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.680+0000 7f26bf400640 1 --2- 172.21.15.111:0/2711000019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b85a74090 0x555b85a82b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.680+0000 7f26bf400640 1 -- 172.21.15.111:0/2711000019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b85a79a60 msgr2=0x555b85a84060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.680+0000 7f26bf400640 1 --2- 172.21.15.111:0/2711000019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b85a79a60 0x555b85a84060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.680+0000 7f26bf400640 1 -- 172.21.15.111:0/2711000019 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b85a84670 con 0x555b85a749d0 2024-09-15T23:02:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.680+0000 7f26bf400640 1 --2- 172.21.15.111:0/2711000019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b85a749d0 0x555b85a83050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x555b85a6f2b0 tx=0x7f26ac005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59143d011b128742 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.680+0000 7f26aa000640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:02:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.681+0000 7f26aa000640 1 -- 172.21.15.111:0/2711000019 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac01e040 con 0x555b85a749d0 2024-09-15T23:02:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.681+0000 7f26aa000640 1 -- 172.21.15.111:0/2711000019 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f26ac005240 con 0x555b85a749d0 2024-09-15T23:02:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.681+0000 7f26ca0ac740 1 -- 172.21.15.111:0/2711000019 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26ac0054d0 con 0x555b85a749d0 2024-09-15T23:02:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.681+0000 7f26aa000640 1 -- 172.21.15.111:0/2711000019 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f26ac019430 con 0x555b85a749d0 2024-09-15T23:02:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.681+0000 7f26ca0ac740 10 client.26278 fetch_fsmap learned FSMap version 35 2024-09-15T23:02:19.685 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.681+0000 7f26ca0ac740 1 -- 172.21.15.111:0/2711000019 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x555b85a858f0 con 0x555b85a749d0 2024-09-15T23:02:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.681+0000 7f26aa000640 1 -- 172.21.15.111:0/2711000019 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ac0054d0 con 0x555b85a749d0 2024-09-15T23:02:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.681+0000 7f26aa000640 1 -- 172.21.15.111:0/2711000019 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f26ac035030 con 0x555b85a749d0 2024-09-15T23:02:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.682+0000 7f26ca0ac740 10 client.26278 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:02:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.682+0000 7f26ca0ac740 10 client.26278 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:02:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.682+0000 7f26ca0ac740 20 client.26278 populate_metadata read hostname 'smithi111' 2024-09-15T23:02:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.682+0000 7f26ca0ac740 1 --2- 172.21.15.111:0/2711000019 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555b85af4eb0 0x555b85af72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.682+0000 7f26ca0ac740 4 client.26278 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:02:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.682+0000 7f26ca0ac740 1 -- 172.21.15.111:0/2711000019 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555b859e0b80 con 0x555b85af4eb0 2024-09-15T23:02:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.682+0000 7f26c5200640 1 --2- 172.21.15.111:0/2711000019 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555b85af4eb0 0x555b85af72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.682+0000 7f26aa000640 10 client.26278 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:02:19.686 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.682+0000 7f26c5200640 1 --2- 172.21.15.111:0/2711000019 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555b85af4eb0 0x555b85af72d0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf85768aa84036d3 server_cookie=5eb7fa638650f4aa in_seq=0 out_seq=0 2024-09-15T23:02:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.957+0000 7f26aa000640 1 -- 172.21.15.111:0/2711000019 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555b859e0b80 con 0x555b85af4eb0 2024-09-15T23:02:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.957+0000 7f26aa000640 10 client.26278 handle_command_reply: tid=0 2024-09-15T23:02:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.957+0000 7f26ca0ac740 1 client.26278 shutdown 2024-09-15T23:02:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.958+0000 7f26ca0ac740 1 -- 172.21.15.111:0/2711000019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b85a749d0 msgr2=0x555b85a83050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.958+0000 7f26ca0ac740 1 --2- 172.21.15.111:0/2711000019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b85a749d0 0x555b85a83050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x555b85a6f2b0 tx=0x7f26ac005d30 comp rx=0 tx=0).stop 2024-09-15T23:02:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.958+0000 7f26ca0ac740 1 -- 172.21.15.111:0/2711000019 shutdown_connections 2024-09-15T23:02:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.958+0000 7f26ca0ac740 1 --2- 172.21.15.111:0/2711000019 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x555b85af4eb0 0x555b85af72d0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.958+0000 7f26ca0ac740 1 --2- 172.21.15.111:0/2711000019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555b85a79a60 0x555b85a84060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.958+0000 7f26ca0ac740 1 --2- 172.21.15.111:0/2711000019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555b85a749d0 0x555b85a83050 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.958+0000 7f26ca0ac740 1 --2- 172.21.15.111:0/2711000019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555b85a74090 0x555b85a82b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.958+0000 7f26ca0ac740 1 -- 172.21.15.111:0/2711000019 >> 172.21.15.111:0/2711000019 conn(0x555b85a6f700 msgr2=0x555b85a6e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:19.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.958+0000 7f26aa000640 0 client.26278 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:02:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.958+0000 7f26ca0ac740 1 -- 172.21.15.111:0/2711000019 shutdown_connections 2024-09-15T23:02:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.958+0000 7f26ca0ac740 1 -- 172.21.15.111:0/2711000019 wait complete. 2024-09-15T23:02:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.958+0000 7f26ca0ac740 20 client.26278 trim_cache size 0 max 16384 2024-09-15T23:02:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:02:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:02:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:02:19.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:02:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.959+0000 7f26c7200640 1 -- 172.21.15.111:0/3079319641 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f26a40a3460 msgr2=0x7f26a40a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.959+0000 7f26c7200640 1 --2- 172.21.15.111:0/3079319641 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f26a40a3460 0x7f26a40a5920 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f26b00b8b00 tx=0x7f26b0009260 comp rx=0 tx=0).stop 2024-09-15T23:02:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.959+0000 7f26c7200640 1 -- 172.21.15.111:0/3079319641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26c0103d60 msgr2=0x7f26c019a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.959+0000 7f26c7200640 1 --2- 172.21.15.111:0/3079319641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26c0103d60 0x7f26c019a000 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f26b0031440 tx=0x7f26b0032ce0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.959+0000 7f26c7200640 1 -- 172.21.15.111:0/3079319641 shutdown_connections 2024-09-15T23:02:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.959+0000 7f26c7200640 1 --2- 172.21.15.111:0/3079319641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26c0105910 0x7f26c01a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.959+0000 7f26c7200640 1 --2- 172.21.15.111:0/3079319641 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26c0104f50 0x7f26c019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.959+0000 7f26c7200640 1 --2- 172.21.15.111:0/3079319641 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f26a40a3460 0x7f26a40a5920 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.959+0000 7f26c7200640 1 --2- 172.21.15.111:0/3079319641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26c0103d60 0x7f26c019a000 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.959+0000 7f26c7200640 1 -- 172.21.15.111:0/3079319641 >> 172.21.15.111:0/3079319641 conn(0x7f26c00ff4e0 msgr2=0x7f26c0108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.960+0000 7f26c7200640 1 -- 172.21.15.111:0/3079319641 shutdown_connections 2024-09-15T23:02:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:19.960+0000 7f26c7200640 1 -- 172.21.15.111:0/3079319641 wait complete. 2024-09-15T23:02:20.046 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:19 smithi111 ceph-mon[30629]: pgmap v843: 97 pgs: 97 active+clean; 499 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.5 MiB/s wr, 599 op/s 2024-09-15T23:02:20.316 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:19 smithi192 ceph-mon[35427]: pgmap v843: 97 pgs: 97 active+clean; 499 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.5 MiB/s wr, 599 op/s 2024-09-15T23:02:20.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:19 smithi121 ceph-mon[34815]: pgmap v843: 97 pgs: 97 active+clean; 499 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.5 MiB/s wr, 599 op/s 2024-09-15T23:02:20.637 INFO:tasks.workunit.client.0.smithi111.stdout: 10 60009 13.11 MB/sec execute 48 sec latency 156.076 ms 2024-09-15T23:02:21.637 INFO:tasks.workunit.client.0.smithi111.stdout: 10 60981 13.80 MB/sec execute 49 sec latency 43.754 ms 2024-09-15T23:02:22.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:21 smithi111 ceph-mon[30629]: pgmap v844: 97 pgs: 97 active+clean; 504 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.9 MiB/s wr, 506 op/s 2024-09-15T23:02:22.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:21 smithi192 ceph-mon[35427]: pgmap v844: 97 pgs: 97 active+clean; 504 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.9 MiB/s wr, 506 op/s 2024-09-15T23:02:22.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:21 smithi121 ceph-mon[34815]: pgmap v844: 97 pgs: 97 active+clean; 504 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.9 MiB/s wr, 506 op/s 2024-09-15T23:02:22.638 INFO:tasks.workunit.client.0.smithi111.stdout: 10 61864 14.61 MB/sec execute 50 sec latency 53.992 ms 2024-09-15T23:02:23.637 INFO:tasks.workunit.client.0.smithi111.stdout: 10 62499 15.07 MB/sec execute 51 sec latency 65.588 ms 2024-09-15T23:02:24.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:23 smithi111 ceph-mon[30629]: pgmap v845: 97 pgs: 97 active+clean; 506 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.0 MiB/s wr, 533 op/s 2024-09-15T23:02:24.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:23 smithi192 ceph-mon[35427]: pgmap v845: 97 pgs: 97 active+clean; 506 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.0 MiB/s wr, 533 op/s 2024-09-15T23:02:24.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:23 smithi121 ceph-mon[34815]: pgmap v845: 97 pgs: 97 active+clean; 506 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.0 MiB/s wr, 533 op/s 2024-09-15T23:02:24.637 INFO:tasks.workunit.client.0.smithi111.stdout: 10 63117 15.50 MB/sec execute 52 sec latency 88.602 ms 2024-09-15T23:02:25.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:25 smithi192 ceph-mon[35427]: pgmap v846: 97 pgs: 97 active+clean; 508 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.0 MiB/s wr, 575 op/s 2024-09-15T23:02:25.637 INFO:tasks.workunit.client.0.smithi111.stdout: 10 63739 15.68 MB/sec execute 53 sec latency 71.068 ms 2024-09-15T23:02:25.639 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:25 smithi121 ceph-mon[34815]: pgmap v846: 97 pgs: 97 active+clean; 508 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.0 MiB/s wr, 575 op/s 2024-09-15T23:02:25.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:25 smithi111 ceph-mon[30629]: pgmap v846: 97 pgs: 97 active+clean; 508 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.0 MiB/s wr, 575 op/s 2024-09-15T23:02:26.638 INFO:tasks.workunit.client.0.smithi111.stdout: 10 64439 16.05 MB/sec execute 54 sec latency 144.169 ms 2024-09-15T23:02:27.638 INFO:tasks.workunit.client.0.smithi111.stdout: 10 65164 16.47 MB/sec execute 55 sec latency 194.056 ms 2024-09-15T23:02:28.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:27 smithi111 ceph-mon[30629]: pgmap v847: 97 pgs: 97 active+clean; 507 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.3 MiB/s wr, 604 op/s 2024-09-15T23:02:28.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:27 smithi192 ceph-mon[35427]: pgmap v847: 97 pgs: 97 active+clean; 507 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.3 MiB/s wr, 604 op/s 2024-09-15T23:02:28.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:27 smithi121 ceph-mon[34815]: pgmap v847: 97 pgs: 97 active+clean; 507 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.3 MiB/s wr, 604 op/s 2024-09-15T23:02:28.638 INFO:tasks.workunit.client.0.smithi111.stdout: 10 65905 16.88 MB/sec execute 56 sec latency 111.826 ms 2024-09-15T23:02:29.638 INFO:tasks.workunit.client.0.smithi111.stdout: 10 66606 17.23 MB/sec execute 57 sec latency 58.824 ms 2024-09-15T23:02:30.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:29 smithi111 ceph-mon[30629]: pgmap v848: 97 pgs: 97 active+clean; 499 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 662 op/s 2024-09-15T23:02:30.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:29 smithi192 ceph-mon[35427]: pgmap v848: 97 pgs: 97 active+clean; 499 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 662 op/s 2024-09-15T23:02:30.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:29 smithi121 ceph-mon[34815]: pgmap v848: 97 pgs: 97 active+clean; 499 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 662 op/s 2024-09-15T23:02:30.638 INFO:tasks.workunit.client.0.smithi111.stdout: 10 67318 17.74 MB/sec execute 58 sec latency 51.595 ms 2024-09-15T23:02:31.638 INFO:tasks.workunit.client.0.smithi111.stdout: 10 67946 17.97 MB/sec execute 59 sec latency 136.148 ms 2024-09-15T23:02:32.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:31 smithi111 ceph-mon[30629]: pgmap v849: 97 pgs: 97 active+clean; 505 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.2 MiB/s wr, 590 op/s 2024-09-15T23:02:32.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:31 smithi192 ceph-mon[35427]: pgmap v849: 97 pgs: 97 active+clean; 505 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.2 MiB/s wr, 590 op/s 2024-09-15T23:02:32.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:31 smithi121 ceph-mon[34815]: pgmap v849: 97 pgs: 97 active+clean; 505 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.2 MiB/s wr, 590 op/s 2024-09-15T23:02:32.638 INFO:tasks.workunit.client.0.smithi111.stdout: 10 68555 18.19 MB/sec execute 60 sec latency 141.949 ms 2024-09-15T23:02:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.116+0000 7f9d89200640 1 -- 172.21.15.111:0/104906195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d84074bd0 msgr2=0x7f9d84075030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.116+0000 7f9d89200640 1 --2- 172.21.15.111:0/104906195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d84074bd0 0x7f9d84075030 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9d70009900 tx=0x7f9d7002a480 comp rx=0 tx=0).stop 2024-09-15T23:02:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.116+0000 7f9d89200640 1 -- 172.21.15.111:0/104906195 shutdown_connections 2024-09-15T23:02:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.116+0000 7f9d89200640 1 --2- 172.21.15.111:0/104906195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d84109a10 0x7f9d8410bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.116+0000 7f9d89200640 1 --2- 172.21.15.111:0/104906195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d84074bd0 0x7f9d84075030 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.116+0000 7f9d89200640 1 --2- 172.21.15.111:0/104906195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d840761b0 0x7f9d84074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.116+0000 7f9d89200640 1 -- 172.21.15.111:0/104906195 >> 172.21.15.111:0/104906195 conn(0x7f9d840fd2e0 msgr2=0x7f9d840ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.116+0000 7f9d89200640 1 -- 172.21.15.111:0/104906195 shutdown_connections 2024-09-15T23:02:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.116+0000 7f9d89200640 1 -- 172.21.15.111:0/104906195 wait complete. 2024-09-15T23:02:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.117+0000 7f9d89200640 1 Processor -- start 2024-09-15T23:02:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.117+0000 7f9d89200640 1 -- start start 2024-09-15T23:02:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.117+0000 7f9d89200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d84074bd0 0x7f9d8419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.117+0000 7f9d89200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d840761b0 0x7f9d8419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.117+0000 7f9d89200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d84109a10 0x7f9d841a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.117+0000 7f9d89200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8410f9e0 con 0x7f9d840761b0 2024-09-15T23:02:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.117+0000 7f9d89200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8410f860 con 0x7f9d84074bd0 2024-09-15T23:02:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.117+0000 7f9d89200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9d8410fb60 con 0x7f9d84109a10 2024-09-15T23:02:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.118+0000 7f9d81e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d84074bd0 0x7f9d8419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.118+0000 7f9d81e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d84074bd0 0x7f9d8419e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:50462/0 (socket says 172.21.15.111:50462) 2024-09-15T23:02:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.118+0000 7f9d81e00640 1 -- 172.21.15.111:0/3114626100 learned_addr learned my addr 172.21.15.111:0/3114626100 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:02:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.118+0000 7f9d82800640 1 --2- 172.21.15.111:0/3114626100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d84109a10 0x7f9d841a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.118+0000 7f9d81400640 1 --2- 172.21.15.111:0/3114626100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d840761b0 0x7f9d8419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.118+0000 7f9d81e00640 1 -- 172.21.15.111:0/3114626100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d84109a10 msgr2=0x7f9d841a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.118+0000 7f9d81e00640 1 --2- 172.21.15.111:0/3114626100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d84109a10 0x7f9d841a59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.118+0000 7f9d81e00640 1 -- 172.21.15.111:0/3114626100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d840761b0 msgr2=0x7f9d8419e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.118+0000 7f9d81e00640 1 --2- 172.21.15.111:0/3114626100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d840761b0 0x7f9d8419e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.118+0000 7f9d81e00640 1 -- 172.21.15.111:0/3114626100 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d841a6020 con 0x7f9d84074bd0 2024-09-15T23:02:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.118+0000 7f9d81400640 1 --2- 172.21.15.111:0/3114626100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d840761b0 0x7f9d8419e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:02:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.118+0000 7f9d81e00640 1 --2- 172.21.15.111:0/3114626100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d84074bd0 0x7f9d8419e400 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9d7400e9f0 tx=0x7f9d7400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.119+0000 7f9d7aa00640 1 -- 172.21.15.111:0/3114626100 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7400cd80 con 0x7f9d84074bd0 2024-09-15T23:02:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.119+0000 7f9d89200640 1 -- 172.21.15.111:0/3114626100 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d841a6310 con 0x7f9d84074bd0 2024-09-15T23:02:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.119+0000 7f9d7aa00640 1 -- 172.21.15.111:0/3114626100 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d74010910 con 0x7f9d84074bd0 2024-09-15T23:02:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.119+0000 7f9d7aa00640 1 -- 172.21.15.111:0/3114626100 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d740188f0 con 0x7f9d84074bd0 2024-09-15T23:02:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.119+0000 7f9d89200640 1 -- 172.21.15.111:0/3114626100 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d841a6880 con 0x7f9d84074bd0 2024-09-15T23:02:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.122+0000 7f9d8c0d9740 1 Processor -- start 2024-09-15T23:02:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.122+0000 7f9d8c0d9740 1 -- start start 2024-09-15T23:02:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.123+0000 7f9d8c0d9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558261328090 0x558261328490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.123+0000 7f9d8c0d9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5582613289d0 0x55826132d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.123+0000 7f9d8c0d9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55826132da60 0x55826132ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.123+0000 7f9d8c0d9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55826128d100 con 0x55826132da60 2024-09-15T23:02:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.123+0000 7f9d8c0d9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5582612351b0 con 0x5582613289d0 2024-09-15T23:02:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.123+0000 7f9d8c0d9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5582612890c0 con 0x558261328090 2024-09-15T23:02:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.123+0000 7f9d7aa00640 1 -- 172.21.15.111:0/3114626100 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f9d74010ac0 con 0x7f9d84074bd0 2024-09-15T23:02:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.123+0000 7f9d82800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558261328090 0x558261328490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.123+0000 7f9d82800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558261328090 0x558261328490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:33886/0 (socket says 172.21.15.111:33886) 2024-09-15T23:02:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.123+0000 7f9d82800640 1 -- 172.21.15.111:0/3223513440 learned_addr learned my addr 172.21.15.111:0/3223513440 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:02:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.123+0000 7f9d81400640 1 --2- 172.21.15.111:0/3223513440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55826132da60 0x55826132ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.123+0000 7f9d81e00640 1 --2- 172.21.15.111:0/3223513440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5582613289d0 0x55826132d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.123+0000 7f9d82800640 1 -- 172.21.15.111:0/3223513440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5582613289d0 msgr2=0x55826132d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.123+0000 7f9d82800640 1 --2- 172.21.15.111:0/3223513440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5582613289d0 0x55826132d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.123+0000 7f9d82800640 1 -- 172.21.15.111:0/3223513440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55826132da60 msgr2=0x55826132ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.123+0000 7f9d82800640 1 --2- 172.21.15.111:0/3223513440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55826132da60 0x55826132ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.123+0000 7f9d82800640 1 -- 172.21.15.111:0/3223513440 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558260fa0210 con 0x558261328090 2024-09-15T23:02:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.124+0000 7f9d82800640 1 --2- 172.21.15.111:0/3223513440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558261328090 0x558261328490 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009fd0 tx=0x7f9d7c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9722aef8ef92658c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.124+0000 7f9d6f400640 1 -- 172.21.15.111:0/3223513440 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c004d20 con 0x558261328090 2024-09-15T23:02:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.124+0000 7f9d6f400640 1 -- 172.21.15.111:0/3223513440 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d7c004050 con 0x558261328090 2024-09-15T23:02:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.124+0000 7f9d6f400640 1 -- 172.21.15.111:0/3223513440 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c005260 con 0x558261328090 2024-09-15T23:02:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.125+0000 7f9d8c0d9740 1 -- 172.21.15.111:0/3223513440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558261328090 msgr2=0x558261328490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.125+0000 7f9d8c0d9740 1 --2- 172.21.15.111:0/3223513440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558261328090 0x558261328490 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c009fd0 tx=0x7f9d7c0118d0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.125+0000 7f9d8c0d9740 1 -- 172.21.15.111:0/3223513440 shutdown_connections 2024-09-15T23:02:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.125+0000 7f9d8c0d9740 1 --2- 172.21.15.111:0/3223513440 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55826132da60 0x55826132ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.125+0000 7f9d8c0d9740 1 --2- 172.21.15.111:0/3223513440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5582613289d0 0x55826132d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.125+0000 7f9d8c0d9740 1 --2- 172.21.15.111:0/3223513440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558261328090 0x558261328490 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.125+0000 7f9d8c0d9740 1 -- 172.21.15.111:0/3223513440 >> 172.21.15.111:0/3223513440 conn(0x558261323700 msgr2=0x558261323b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.128+0000 7f9d7aa00640 1 --2- 172.21.15.111:0/3114626100 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9d680a32f0 0x7f9d680a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.128+0000 7f9d8c0d9740 1 -- 172.21.15.111:0/3223513440 shutdown_connections 2024-09-15T23:02:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.129+0000 7f9d8c0d9740 1 -- 172.21.15.111:0/3223513440 wait complete. 2024-09-15T23:02:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.129+0000 7f9d7aa00640 1 -- 172.21.15.111:0/3114626100 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f9d740cfcb0 con 0x7f9d84074bd0 2024-09-15T23:02:33.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.129+0000 7f9d81400640 1 --2- 172.21.15.111:0/3114626100 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9d680a32f0 0x7f9d680a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.129+0000 7f9d8c0d9740 1 -- start start 2024-09-15T23:02:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.130+0000 7f9d81400640 1 --2- 172.21.15.111:0/3114626100 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9d680a32f0 0x7f9d680a57b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9d70000c00 tx=0x7f9d700315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:33.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.130+0000 7f9d8c0d9740 1 Processor -- start 2024-09-15T23:02:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.130+0000 7f9d8c0d9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558261328090 0x5582613321b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.130+0000 7f9d8c0d9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582613289d0 0x5582613326f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.130+0000 7f9d8c0d9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826132da60 0x558261339730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.130+0000 7f9d8c0d9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x558261330a30 con 0x5582613289d0 2024-09-15T23:02:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.130+0000 7f9d8c0d9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558261216090 con 0x558261328090 2024-09-15T23:02:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.130+0000 7f9d8c0d9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558261330bb0 con 0x55826132da60 2024-09-15T23:02:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.130+0000 7f9d8c0d9740 10 client.0 _ll_register_callbacks cb 0x55826126af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:02:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.130+0000 7f9d82800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826132da60 0x558261339730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.130+0000 7f9d81e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558261328090 0x5582613321b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.131+0000 7f9d82800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826132da60 0x558261339730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:33902/0 (socket says 172.21.15.111:33902) 2024-09-15T23:02:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.131+0000 7f9d82800640 1 -- 172.21.15.111:0/2452389089 learned_addr learned my addr 172.21.15.111:0/2452389089 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:02:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.131+0000 7f9d81400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582613289d0 0x5582613326f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.131+0000 7f9d82800640 1 -- 172.21.15.111:0/2452389089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558261328090 msgr2=0x5582613321b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.131+0000 7f9d82800640 1 --2- 172.21.15.111:0/2452389089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558261328090 0x5582613321b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.131+0000 7f9d82800640 1 -- 172.21.15.111:0/2452389089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582613289d0 msgr2=0x5582613326f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.131+0000 7f9d82800640 1 --2- 172.21.15.111:0/2452389089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582613289d0 0x5582613326f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.131+0000 7f9d82800640 1 -- 172.21.15.111:0/2452389089 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55826106c620 con 0x55826132da60 2024-09-15T23:02:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.131+0000 7f9d81400640 1 --2- 172.21.15.111:0/2452389089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582613289d0 0x5582613326f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:02:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.131+0000 7f9d82800640 1 --2- 172.21.15.111:0/2452389089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826132da60 0x558261339730 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c005c40 tx=0x7f9d7c004e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f51e68776a43269c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.131+0000 7f9d63e00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:02:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.132+0000 7f9d63e00640 1 -- 172.21.15.111:0/2452389089 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c028020 con 0x55826132da60 2024-09-15T23:02:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.132+0000 7f9d63e00640 1 -- 172.21.15.111:0/2452389089 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f9d7c004490 con 0x55826132da60 2024-09-15T23:02:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.132+0000 7f9d8c0d9740 1 -- 172.21.15.111:0/2452389089 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d7c004050 con 0x55826132da60 2024-09-15T23:02:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.132+0000 7f9d63e00640 1 -- 172.21.15.111:0/2452389089 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9d7c004690 con 0x55826132da60 2024-09-15T23:02:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.132+0000 7f9d8c0d9740 10 client.26290 fetch_fsmap learned FSMap version 35 2024-09-15T23:02:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.132+0000 7f9d8c0d9740 1 -- 172.21.15.111:0/2452389089 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55826133c520 con 0x55826132da60 2024-09-15T23:02:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.132+0000 7f9d63e00640 1 -- 172.21.15.111:0/2452389089 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d7c004050 con 0x55826132da60 2024-09-15T23:02:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.132+0000 7f9d63e00640 1 -- 172.21.15.111:0/2452389089 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f9d7c00f7e0 con 0x55826132da60 2024-09-15T23:02:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.133+0000 7f9d8c0d9740 10 client.26290 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:02:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.133+0000 7f9d8c0d9740 10 client.26290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:02:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.133+0000 7f9d8c0d9740 20 client.26290 populate_metadata read hostname 'smithi111' 2024-09-15T23:02:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.133+0000 7f9d8c0d9740 1 --2- 172.21.15.111:0/2452389089 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55826133da30 0x55826133fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.133+0000 7f9d8c0d9740 4 client.26290 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:02:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.133+0000 7f9d8c0d9740 1 -- 172.21.15.111:0/2452389089 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55826129a830 con 0x55826133da30 2024-09-15T23:02:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.133+0000 7f9d81e00640 1 --2- 172.21.15.111:0/2452389089 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55826133da30 0x55826133fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.133+0000 7f9d81e00640 1 --2- 172.21.15.111:0/2452389089 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55826133da30 0x55826133fe20 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69d376044c85ddf3 server_cookie=6cabaf2fc447a9f1 in_seq=0 out_seq=0 2024-09-15T23:02:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.133+0000 7f9d63e00640 10 client.26290 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:02:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.134+0000 7f9d63e00640 1 -- 172.21.15.111:0/2452389089 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55826129a830 con 0x55826133da30 2024-09-15T23:02:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.134+0000 7f9d63e00640 10 client.26290 handle_command_reply: tid=0 2024-09-15T23:02:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.134+0000 7f9d8c0d9740 1 client.26290 shutdown 2024-09-15T23:02:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.135+0000 7f9d8c0d9740 1 -- 172.21.15.111:0/2452389089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826132da60 msgr2=0x558261339730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.135+0000 7f9d8c0d9740 1 --2- 172.21.15.111:0/2452389089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826132da60 0x558261339730 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c005c40 tx=0x7f9d7c004e30 comp rx=0 tx=0).stop 2024-09-15T23:02:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.135+0000 7f9d8c0d9740 1 -- 172.21.15.111:0/2452389089 shutdown_connections 2024-09-15T23:02:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.135+0000 7f9d8c0d9740 1 --2- 172.21.15.111:0/2452389089 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55826133da30 0x55826133fe20 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.135+0000 7f9d63e00640 0 client.26290 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:02:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.135+0000 7f9d8c0d9740 1 --2- 172.21.15.111:0/2452389089 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826132da60 0x558261339730 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.135+0000 7f9d8c0d9740 1 --2- 172.21.15.111:0/2452389089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582613289d0 0x5582613326f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.135+0000 7f9d8c0d9740 1 --2- 172.21.15.111:0/2452389089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558261328090 0x5582613321b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.135+0000 7f9d8c0d9740 1 -- 172.21.15.111:0/2452389089 >> 172.21.15.111:0/2452389089 conn(0x558261323700 msgr2=0x558261324f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.135+0000 7f9d8c0d9740 1 -- 172.21.15.111:0/2452389089 shutdown_connections 2024-09-15T23:02:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.135+0000 7f9d8c0d9740 1 -- 172.21.15.111:0/2452389089 wait complete. 2024-09-15T23:02:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.135+0000 7f9d8c0d9740 20 client.26290 trim_cache size 0 max 16384 2024-09-15T23:02:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.152+0000 7f9d8c0d9740 1 Processor -- start 2024-09-15T23:02:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.152+0000 7f9d8c0d9740 1 -- start start 2024-09-15T23:02:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.152+0000 7f9d8c0d9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558261328090 0x558261328470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.152+0000 7f9d8c0d9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582613289d0 0x558261339650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.152+0000 7f9d8c0d9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826132da60 0x558261322770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.152+0000 7f9d8c0d9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841b1dc0 con 0x5582613289d0 2024-09-15T23:02:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.152+0000 7f9d8c0d9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841af690 con 0x558261328090 2024-09-15T23:02:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.152+0000 7f9d8c0d9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841ad4e0 con 0x55826132da60 2024-09-15T23:02:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.153+0000 7f9d82800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826132da60 0x558261322770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.153+0000 7f9d81e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558261328090 0x558261328470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.153+0000 7f9d81400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582613289d0 0x558261339650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.153+0000 7f9d82800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826132da60 0x558261322770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:33908/0 (socket says 172.21.15.111:33908) 2024-09-15T23:02:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.153+0000 7f9d82800640 1 -- 172.21.15.111:0/31713678 learned_addr learned my addr 172.21.15.111:0/31713678 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:02:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.153+0000 7f9d82800640 1 -- 172.21.15.111:0/31713678 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558261328090 msgr2=0x558261328470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.153+0000 7f9d82800640 1 --2- 172.21.15.111:0/31713678 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558261328090 0x558261328470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.153+0000 7f9d82800640 1 -- 172.21.15.111:0/31713678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582613289d0 msgr2=0x558261339650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.153+0000 7f9d82800640 1 --2- 172.21.15.111:0/31713678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582613289d0 0x558261339650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.153+0000 7f9d82800640 1 -- 172.21.15.111:0/31713678 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558261323520 con 0x55826132da60 2024-09-15T23:02:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.153+0000 7f9d82800640 1 --2- 172.21.15.111:0/31713678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826132da60 0x558261322770 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c00fd70 tx=0x7f9d7c004a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f955dae001d5b6a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.154+0000 7f9d6ea00640 1 -- 172.21.15.111:0/31713678 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c03c070 con 0x55826132da60 2024-09-15T23:02:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.154+0000 7f9d6ea00640 1 -- 172.21.15.111:0/31713678 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d7c004000 con 0x55826132da60 2024-09-15T23:02:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.154+0000 7f9d6ea00640 1 -- 172.21.15.111:0/31713678 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c023750 con 0x55826132da60 2024-09-15T23:02:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.155+0000 7f9d8c0d9740 1 -- 172.21.15.111:0/31713678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826132da60 msgr2=0x558261322770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.155+0000 7f9d8c0d9740 1 --2- 172.21.15.111:0/31713678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826132da60 0x558261322770 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c00fd70 tx=0x7f9d7c004a90 comp rx=0 tx=0).stop 2024-09-15T23:02:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.155+0000 7f9d8c0d9740 1 -- 172.21.15.111:0/31713678 shutdown_connections 2024-09-15T23:02:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.155+0000 7f9d8c0d9740 1 --2- 172.21.15.111:0/31713678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826132da60 0x558261322770 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.155+0000 7f9d8c0d9740 1 --2- 172.21.15.111:0/31713678 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582613289d0 0x558261339650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.155+0000 7f9d8c0d9740 1 --2- 172.21.15.111:0/31713678 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558261328090 0x558261328470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.155+0000 7f9d8c0d9740 1 -- 172.21.15.111:0/31713678 >> 172.21.15.111:0/31713678 conn(0x558261323700 msgr2=0x558261322060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.155+0000 7f9d8c0d9740 1 -- 172.21.15.111:0/31713678 shutdown_connections 2024-09-15T23:02:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.155+0000 7f9d8c0d9740 1 -- 172.21.15.111:0/31713678 wait complete. 2024-09-15T23:02:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.155+0000 7f9d8c0d9740 1 -- start start 2024-09-15T23:02:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.156+0000 7f9d8c0d9740 1 Processor -- start 2024-09-15T23:02:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.156+0000 7f9d8c0d9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558261328090 0x558261339580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.156+0000 7f9d8c0d9740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582613289d0 0x558261336020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.156+0000 7f9d8c0d9740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826132da60 0x558261336560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.156+0000 7f9d8c0d9740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841adb50 con 0x5582613289d0 2024-09-15T23:02:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.156+0000 7f9d8c0d9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841aefc0 con 0x558261328090 2024-09-15T23:02:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.156+0000 7f9d8c0d9740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9d841ae220 con 0x55826132da60 2024-09-15T23:02:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.156+0000 7f9d8c0d9740 10 client.0 _ll_register_callbacks cb 0x7f9d7c009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:02:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.156+0000 7f9d81e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558261328090 0x558261339580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.156+0000 7f9d81e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558261328090 0x558261339580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:50484/0 (socket says 172.21.15.111:50484) 2024-09-15T23:02:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.156+0000 7f9d81e00640 1 -- 172.21.15.111:0/798797119 learned_addr learned my addr 172.21.15.111:0/798797119 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:02:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.157+0000 7f9d82800640 1 --2- 172.21.15.111:0/798797119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826132da60 0x558261336560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.157+0000 7f9d81e00640 1 -- 172.21.15.111:0/798797119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826132da60 msgr2=0x558261336560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.157+0000 7f9d81e00640 1 --2- 172.21.15.111:0/798797119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826132da60 0x558261336560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.157+0000 7f9d81e00640 1 -- 172.21.15.111:0/798797119 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582613289d0 msgr2=0x558261336020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T23:02:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.157+0000 7f9d81e00640 1 --2- 172.21.15.111:0/798797119 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582613289d0 0x558261336020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.157+0000 7f9d81e00640 1 -- 172.21.15.111:0/798797119 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582613a66d0 con 0x558261328090 2024-09-15T23:02:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.157+0000 7f9d82800640 1 --2- 172.21.15.111:0/798797119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826132da60 0x558261336560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:02:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.157+0000 7f9d81e00640 1 --2- 172.21.15.111:0/798797119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558261328090 0x558261339580 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9d7400e9c0 tx=0x7f9d7401f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef46918446e0a560 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.157+0000 7f9d63e00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:02:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.157+0000 7f9d63e00640 1 -- 172.21.15.111:0/798797119 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d740a4fe0 con 0x558261328090 2024-09-15T23:02:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.157+0000 7f9d8c0d9740 1 -- 172.21.15.111:0/798797119 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d7c004000 con 0x558261328090 2024-09-15T23:02:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.157+0000 7f9d63e00640 1 -- 172.21.15.111:0/798797119 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f9d740989e0 con 0x558261328090 2024-09-15T23:02:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.157+0000 7f9d63e00640 1 -- 172.21.15.111:0/798797119 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9d740b0050 con 0x558261328090 2024-09-15T23:02:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.158+0000 7f9d8c0d9740 10 client.26279 fetch_fsmap learned FSMap version 35 2024-09-15T23:02:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.158+0000 7f9d8c0d9740 1 -- 172.21.15.111:0/798797119 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5582613a7900 con 0x558261328090 2024-09-15T23:02:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.158+0000 7f9d63e00640 1 -- 172.21.15.111:0/798797119 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d740ae020 con 0x558261328090 2024-09-15T23:02:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.158+0000 7f9d63e00640 1 -- 172.21.15.111:0/798797119 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f9d740981e0 con 0x558261328090 2024-09-15T23:02:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.158+0000 7f9d8c0d9740 10 client.26279 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:02:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.158+0000 7f9d8c0d9740 10 client.26279 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:02:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.158+0000 7f9d8c0d9740 20 client.26279 populate_metadata read hostname 'smithi111' 2024-09-15T23:02:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.158+0000 7f9d8c0d9740 1 --2- 172.21.15.111:0/798797119 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5582613a8230 0x5582613aa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.158+0000 7f9d8c0d9740 4 client.26279 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:02:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.158+0000 7f9d8c0d9740 1 -- 172.21.15.111:0/798797119 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558261294b80 con 0x5582613a8230 2024-09-15T23:02:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.159+0000 7f9d81400640 1 --2- 172.21.15.111:0/798797119 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5582613a8230 0x5582613aa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.159+0000 7f9d81400640 1 --2- 172.21.15.111:0/798797119 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5582613a8230 0x5582613aa650 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48539f67b2bf5b34 server_cookie=4617ccf8f0c5f979 in_seq=0 out_seq=0 2024-09-15T23:02:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.159+0000 7f9d63e00640 10 client.26279 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:02:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.373+0000 7f9d63e00640 1 -- 172.21.15.111:0/798797119 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558261294b80 con 0x5582613a8230 2024-09-15T23:02:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.373+0000 7f9d63e00640 10 client.26279 handle_command_reply: tid=0 2024-09-15T23:02:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.373+0000 7f9d8c0d9740 1 client.26279 shutdown 2024-09-15T23:02:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.373+0000 7f9d8c0d9740 1 -- 172.21.15.111:0/798797119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558261328090 msgr2=0x558261339580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.373+0000 7f9d8c0d9740 1 --2- 172.21.15.111:0/798797119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558261328090 0x558261339580 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9d7400e9c0 tx=0x7f9d7401f000 comp rx=0 tx=0).stop 2024-09-15T23:02:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.374+0000 7f9d8c0d9740 1 -- 172.21.15.111:0/798797119 shutdown_connections 2024-09-15T23:02:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.374+0000 7f9d8c0d9740 1 --2- 172.21.15.111:0/798797119 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5582613a8230 0x5582613aa650 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.374+0000 7f9d8c0d9740 1 --2- 172.21.15.111:0/798797119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55826132da60 0x558261336560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.374+0000 7f9d8c0d9740 1 --2- 172.21.15.111:0/798797119 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5582613289d0 0x558261336020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.374+0000 7f9d63e00640 0 client.26279 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:02:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.374+0000 7f9d8c0d9740 1 --2- 172.21.15.111:0/798797119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558261328090 0x558261339580 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.374+0000 7f9d8c0d9740 1 -- 172.21.15.111:0/798797119 >> 172.21.15.111:0/798797119 conn(0x558261323700 msgr2=0x558261322060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.374+0000 7f9d8c0d9740 1 -- 172.21.15.111:0/798797119 shutdown_connections 2024-09-15T23:02:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.374+0000 7f9d8c0d9740 1 -- 172.21.15.111:0/798797119 wait complete. 2024-09-15T23:02:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.374+0000 7f9d8c0d9740 20 client.26279 trim_cache size 0 max 16384 2024-09-15T23:02:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:02:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:02:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:02:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:02:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.375+0000 7f9d89200640 1 -- 172.21.15.111:0/3114626100 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9d680a32f0 msgr2=0x7f9d680a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.375+0000 7f9d89200640 1 --2- 172.21.15.111:0/3114626100 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9d680a32f0 0x7f9d680a57b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9d70000c00 tx=0x7f9d700315b0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.375+0000 7f9d89200640 1 -- 172.21.15.111:0/3114626100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d84074bd0 msgr2=0x7f9d8419e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.375+0000 7f9d89200640 1 --2- 172.21.15.111:0/3114626100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d84074bd0 0x7f9d8419e400 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9d7400e9f0 tx=0x7f9d7400eeb0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.375+0000 7f9d89200640 1 -- 172.21.15.111:0/3114626100 shutdown_connections 2024-09-15T23:02:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.375+0000 7f9d89200640 1 --2- 172.21.15.111:0/3114626100 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d84109a10 0x7f9d841a59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.375+0000 7f9d89200640 1 --2- 172.21.15.111:0/3114626100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d840761b0 0x7f9d8419e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.375+0000 7f9d89200640 1 --2- 172.21.15.111:0/3114626100 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9d680a32f0 0x7f9d680a57b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.375+0000 7f9d89200640 1 --2- 172.21.15.111:0/3114626100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d84074bd0 0x7f9d8419e400 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.375+0000 7f9d89200640 1 -- 172.21.15.111:0/3114626100 >> 172.21.15.111:0/3114626100 conn(0x7f9d840fd2e0 msgr2=0x7f9d84101590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.375+0000 7f9d89200640 1 -- 172.21.15.111:0/3114626100 shutdown_connections 2024-09-15T23:02:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:33.376+0000 7f9d89200640 1 -- 172.21.15.111:0/3114626100 wait complete. 2024-09-15T23:02:33.641 INFO:tasks.workunit.client.0.smithi111.stdout: 10 69183 18.44 MB/sec execute 61 sec latency 134.383 ms 2024-09-15T23:02:34.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:33 smithi111 ceph-mon[30629]: pgmap v850: 97 pgs: 97 active+clean; 508 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.0 MiB/s wr, 550 op/s 2024-09-15T23:02:34.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:33 smithi192 ceph-mon[35427]: pgmap v850: 97 pgs: 97 active+clean; 508 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.0 MiB/s wr, 550 op/s 2024-09-15T23:02:34.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:33 smithi121 ceph-mon[34815]: pgmap v850: 97 pgs: 97 active+clean; 508 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.0 MiB/s wr, 550 op/s 2024-09-15T23:02:34.638 INFO:tasks.workunit.client.0.smithi111.stdout: 10 69912 18.76 MB/sec execute 62 sec latency 94.511 ms 2024-09-15T23:02:35.638 INFO:tasks.workunit.client.0.smithi111.stdout: 10 70249 18.70 MB/sec execute 63 sec latency 654.733 ms 2024-09-15T23:02:36.086 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:35 smithi111 ceph-mon[30629]: pgmap v851: 97 pgs: 97 active+clean; 510 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.3 MiB/s wr, 591 op/s 2024-09-15T23:02:36.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:35 smithi192 ceph-mon[35427]: pgmap v851: 97 pgs: 97 active+clean; 510 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.3 MiB/s wr, 591 op/s 2024-09-15T23:02:36.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:35 smithi121 ceph-mon[34815]: pgmap v851: 97 pgs: 97 active+clean; 510 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.3 MiB/s wr, 591 op/s 2024-09-15T23:02:36.638 INFO:tasks.workunit.client.0.smithi111.stdout: 10 71131 19.18 MB/sec execute 64 sec latency 705.118 ms 2024-09-15T23:02:37.638 INFO:tasks.workunit.client.0.smithi111.stdout: 10 71862 19.58 MB/sec execute 65 sec latency 56.184 ms 2024-09-15T23:02:38.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:37 smithi111 ceph-mon[30629]: pgmap v852: 97 pgs: 97 active+clean; 513 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.2 MiB/s wr, 541 op/s 2024-09-15T23:02:38.192 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:37 smithi121 ceph-mon[34815]: pgmap v852: 97 pgs: 97 active+clean; 513 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.2 MiB/s wr, 541 op/s 2024-09-15T23:02:38.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:37 smithi192 ceph-mon[35427]: pgmap v852: 97 pgs: 97 active+clean; 513 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.2 MiB/s wr, 541 op/s 2024-09-15T23:02:38.638 INFO:tasks.workunit.client.0.smithi111.stdout: 10 72456 19.79 MB/sec execute 66 sec latency 49.996 ms 2024-09-15T23:02:39.638 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73113 19.98 MB/sec execute 67 sec latency 64.306 ms 2024-09-15T23:02:40.172 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:39 smithi111 ceph-mon[30629]: pgmap v853: 97 pgs: 97 active+clean; 523 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.8 MiB/s wr, 604 op/s 2024-09-15T23:02:40.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:39 smithi192 ceph-mon[35427]: pgmap v853: 97 pgs: 97 active+clean; 523 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.8 MiB/s wr, 604 op/s 2024-09-15T23:02:40.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:39 smithi121 ceph-mon[34815]: pgmap v853: 97 pgs: 97 active+clean; 523 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.8 MiB/s wr, 604 op/s 2024-09-15T23:02:40.638 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73421 19.90 MB/sec execute 68 sec latency 556.408 ms 2024-09-15T23:02:41.638 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73426 19.62 MB/sec execute 69 sec latency 1556.484 ms 2024-09-15T23:02:42.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:41 smithi111 ceph-mon[30629]: pgmap v854: 97 pgs: 97 active+clean; 535 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 5.7 MiB/s wr, 590 op/s 2024-09-15T23:02:42.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:41 smithi192 ceph-mon[35427]: pgmap v854: 97 pgs: 97 active+clean; 535 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 5.7 MiB/s wr, 590 op/s 2024-09-15T23:02:42.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:41 smithi121 ceph-mon[34815]: pgmap v854: 97 pgs: 97 active+clean; 535 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 5.7 MiB/s wr, 590 op/s 2024-09-15T23:02:42.638 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73436 19.34 MB/sec execute 70 sec latency 2556.552 ms 2024-09-15T23:02:43.641 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73436 19.07 MB/sec execute 71 sec latency 3556.640 ms 2024-09-15T23:02:44.184 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:43 smithi111 ceph-mon[30629]: pgmap v855: 97 pgs: 97 active+clean; 548 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.3 MiB/s wr, 576 op/s 2024-09-15T23:02:44.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:43 smithi192 ceph-mon[35427]: pgmap v855: 97 pgs: 97 active+clean; 548 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.3 MiB/s wr, 576 op/s 2024-09-15T23:02:44.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:43 smithi121 ceph-mon[34815]: pgmap v855: 97 pgs: 97 active+clean; 548 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.3 MiB/s wr, 576 op/s 2024-09-15T23:02:44.638 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73436 18.80 MB/sec execute 72 sec latency 4556.699 ms 2024-09-15T23:02:45.639 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73438 18.54 MB/sec execute 73 sec latency 5522.565 ms 2024-09-15T23:02:46.160 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:45 smithi111 ceph-mon[30629]: pgmap v856: 97 pgs: 97 active+clean; 548 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.6 MiB/s wr, 567 op/s 2024-09-15T23:02:46.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:45 smithi192 ceph-mon[35427]: pgmap v856: 97 pgs: 97 active+clean; 548 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.6 MiB/s wr, 567 op/s 2024-09-15T23:02:46.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:45 smithi121 ceph-mon[34815]: pgmap v856: 97 pgs: 97 active+clean; 548 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.6 MiB/s wr, 567 op/s 2024-09-15T23:02:46.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.502+0000 7f3ac5200640 1 -- 172.21.15.111:0/2355894530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ac0074c50 msgr2=0x7f3ac00750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.502+0000 7f3ac5200640 1 --2- 172.21.15.111:0/2355894530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ac0074c50 0x7f3ac00750b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009900 tx=0x7f3aac02a3f0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.502+0000 7f3ac5200640 1 -- 172.21.15.111:0/2355894530 shutdown_connections 2024-09-15T23:02:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.502+0000 7f3ac5200640 1 --2- 172.21.15.111:0/2355894530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ac010e130 0x7f3ac0110520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.502+0000 7f3ac5200640 1 --2- 172.21.15.111:0/2355894530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ac0074c50 0x7f3ac00750b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.502+0000 7f3ac5200640 1 --2- 172.21.15.111:0/2355894530 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ac0076230 0x7f3ac0074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.502+0000 7f3ac5200640 1 -- 172.21.15.111:0/2355894530 >> 172.21.15.111:0/2355894530 conn(0x7f3ac00fd2e0 msgr2=0x7f3ac00ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.502+0000 7f3ac5200640 1 -- 172.21.15.111:0/2355894530 shutdown_connections 2024-09-15T23:02:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.502+0000 7f3ac5200640 1 -- 172.21.15.111:0/2355894530 wait complete. 2024-09-15T23:02:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.503+0000 7f3ac5200640 1 Processor -- start 2024-09-15T23:02:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.503+0000 7f3ac5200640 1 -- start start 2024-09-15T23:02:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.503+0000 7f3ac5200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ac0074c50 0x7f3ac019e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.503+0000 7f3ac5200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ac0076230 0x7f3ac019e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.503+0000 7f3ac5200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ac010e130 0x7f3ac01a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.503+0000 7f3ac5200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac0113e40 con 0x7f3ac010e130 2024-09-15T23:02:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.503+0000 7f3ac5200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac0113cc0 con 0x7f3ac0074c50 2024-09-15T23:02:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.503+0000 7f3ac5200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac0113fc0 con 0x7f3ac0076230 2024-09-15T23:02:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.503+0000 7f3abde00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ac0074c50 0x7f3ac019e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.503+0000 7f3abe800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ac010e130 0x7f3ac01a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.503+0000 7f3abde00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ac0074c50 0x7f3ac019e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:39268/0 (socket says 172.21.15.111:39268) 2024-09-15T23:02:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.504+0000 7f3abde00640 1 -- 172.21.15.111:0/644219178 learned_addr learned my addr 172.21.15.111:0/644219178 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:02:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.504+0000 7f3abde00640 1 -- 172.21.15.111:0/644219178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ac0076230 msgr2=0x7f3ac019e910 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T23:02:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.504+0000 7f3abde00640 1 --2- 172.21.15.111:0/644219178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ac0076230 0x7f3ac019e910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.504+0000 7f3abde00640 1 -- 172.21.15.111:0/644219178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ac010e130 msgr2=0x7f3ac01a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.504+0000 7f3abde00640 1 --2- 172.21.15.111:0/644219178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ac010e130 0x7f3ac01a5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.504+0000 7f3abde00640 1 -- 172.21.15.111:0/644219178 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac01a5ff0 con 0x7f3ac0074c50 2024-09-15T23:02:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.504+0000 7f3abe800640 1 --2- 172.21.15.111:0/644219178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ac010e130 0x7f3ac01a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:02:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.504+0000 7f3abde00640 1 --2- 172.21.15.111:0/644219178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ac0074c50 0x7f3ac019e3d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000ea60 tx=0x7f3ab000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.504+0000 7f3ab6a00640 1 -- 172.21.15.111:0/644219178 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab000cde0 con 0x7f3ac0074c50 2024-09-15T23:02:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.504+0000 7f3ac5200640 1 -- 172.21.15.111:0/644219178 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ac01a62e0 con 0x7f3ac0074c50 2024-09-15T23:02:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.505+0000 7f3ab6a00640 1 -- 172.21.15.111:0/644219178 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ab00035f0 con 0x7f3ac0074c50 2024-09-15T23:02:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.505+0000 7f3ab6a00640 1 -- 172.21.15.111:0/644219178 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0010670 con 0x7f3ac0074c50 2024-09-15T23:02:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.505+0000 7f3ac5200640 1 -- 172.21.15.111:0/644219178 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ac01a6880 con 0x7f3ac0074c50 2024-09-15T23:02:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.507+0000 7f3ac8056740 1 Processor -- start 2024-09-15T23:02:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.507+0000 7f3ac8056740 1 -- start start 2024-09-15T23:02:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.508+0000 7f3ac8056740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ddbde52090 0x55ddbde52490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.508+0000 7f3ac8056740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ddbde529d0 0x55ddbde57490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.508+0000 7f3ac8056740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ddbde57a60 0x55ddbde59f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.508+0000 7f3ac8056740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ddbddb7100 con 0x55ddbde57a60 2024-09-15T23:02:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.508+0000 7f3ac8056740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ddbdd5f1b0 con 0x55ddbde529d0 2024-09-15T23:02:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.508+0000 7f3ac8056740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ddbddb30c0 con 0x55ddbde52090 2024-09-15T23:02:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.508+0000 7f3abd400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ddbde57a60 0x55ddbde59f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.508+0000 7f3abd400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ddbde57a60 0x55ddbde59f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46488/0 (socket says 172.21.15.111:46488) 2024-09-15T23:02:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.508+0000 7f3abd400640 1 -- 172.21.15.111:0/2855910769 learned_addr learned my addr 172.21.15.111:0/2855910769 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:02:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.508+0000 7f3abe800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ddbde52090 0x55ddbde52490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.509+0000 7f3abd400640 1 -- 172.21.15.111:0/2855910769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ddbde52090 msgr2=0x55ddbde52490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.509+0000 7f3abd400640 1 --2- 172.21.15.111:0/2855910769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ddbde52090 0x55ddbde52490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.509+0000 7f3abd400640 1 -- 172.21.15.111:0/2855910769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ddbde529d0 msgr2=0x55ddbde57490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T23:02:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.509+0000 7f3abd400640 1 --2- 172.21.15.111:0/2855910769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ddbde529d0 0x55ddbde57490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.509+0000 7f3abd400640 1 -- 172.21.15.111:0/2855910769 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddbdaca210 con 0x55ddbde57a60 2024-09-15T23:02:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.509+0000 7f3ab6a00640 1 -- 172.21.15.111:0/644219178 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f3ab00037a0 con 0x7f3ac0074c50 2024-09-15T23:02:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.509+0000 7f3abe800640 1 --2- 172.21.15.111:0/2855910769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ddbde52090 0x55ddbde52490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:02:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.509+0000 7f3abd400640 1 --2- 172.21.15.111:0/2855910769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ddbde57a60 0x55ddbde59f20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3aac031440 tx=0x7f3aac0095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5bcdd6eadeb9d4a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.509+0000 7f3aab400640 1 -- 172.21.15.111:0/2855910769 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac03e070 con 0x55ddbde57a60 2024-09-15T23:02:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.509+0000 7f3aab400640 1 -- 172.21.15.111:0/2855910769 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aac004400 con 0x55ddbde57a60 2024-09-15T23:02:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.509+0000 7f3aab400640 1 -- 172.21.15.111:0/2855910769 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac035040 con 0x55ddbde57a60 2024-09-15T23:02:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.510+0000 7f3ac8056740 1 -- 172.21.15.111:0/2855910769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ddbde57a60 msgr2=0x55ddbde59f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.510+0000 7f3ac8056740 1 --2- 172.21.15.111:0/2855910769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ddbde57a60 0x55ddbde59f20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3aac031440 tx=0x7f3aac0095b0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.510+0000 7f3ac8056740 1 -- 172.21.15.111:0/2855910769 shutdown_connections 2024-09-15T23:02:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.510+0000 7f3ac8056740 1 --2- 172.21.15.111:0/2855910769 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ddbde57a60 0x55ddbde59f20 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.510+0000 7f3ac8056740 1 --2- 172.21.15.111:0/2855910769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ddbde529d0 0x55ddbde57490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.510+0000 7f3ac8056740 1 --2- 172.21.15.111:0/2855910769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ddbde52090 0x55ddbde52490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.510+0000 7f3ac8056740 1 -- 172.21.15.111:0/2855910769 >> 172.21.15.111:0/2855910769 conn(0x55ddbde4d700 msgr2=0x55ddbde4db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.510+0000 7f3ab6a00640 1 --2- 172.21.15.111:0/644219178 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3aa40a32f0 0x7f3aa40a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.510+0000 7f3ac8056740 1 -- 172.21.15.111:0/2855910769 shutdown_connections 2024-09-15T23:02:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.510+0000 7f3ab6a00640 1 -- 172.21.15.111:0/644219178 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f3ab00cfc70 con 0x7f3ac0074c50 2024-09-15T23:02:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.510+0000 7f3ac8056740 1 -- 172.21.15.111:0/2855910769 wait complete. 2024-09-15T23:02:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.510+0000 7f3abd400640 1 --2- 172.21.15.111:0/644219178 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3aa40a32f0 0x7f3aa40a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.511+0000 7f3ac8056740 1 -- start start 2024-09-15T23:02:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.511+0000 7f3ac8056740 1 Processor -- start 2024-09-15T23:02:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.511+0000 7f3abd400640 1 --2- 172.21.15.111:0/644219178 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3aa40a32f0 0x7f3aa40a57b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3aac000c00 tx=0x7f3aac005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.511+0000 7f3ac8056740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ddbde52090 0x55ddbde5c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.511+0000 7f3ac8056740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ddbde529d0 0x55ddbde5c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.512+0000 7f3ac8056740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ddbde57a60 0x55ddbde63780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.512+0000 7f3ac8056740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ddbde5a9d0 con 0x55ddbde529d0 2024-09-15T23:02:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.512+0000 7f3ac8056740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ddbdd40090 con 0x55ddbde52090 2024-09-15T23:02:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.512+0000 7f3ac8056740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ddbde5ab50 con 0x55ddbde57a60 2024-09-15T23:02:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.512+0000 7f3ac8056740 10 client.0 _ll_register_callbacks cb 0x55ddbdd94f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:02:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.512+0000 7f3abd400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ddbde529d0 0x55ddbde5c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.512+0000 7f3abd400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ddbde529d0 0x55ddbde5c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46490/0 (socket says 172.21.15.111:46490) 2024-09-15T23:02:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.512+0000 7f3abd400640 1 -- 172.21.15.111:0/1212737315 learned_addr learned my addr 172.21.15.111:0/1212737315 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:02:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.512+0000 7f3abe800640 1 --2- 172.21.15.111:0/1212737315 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ddbde57a60 0x55ddbde63780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.512+0000 7f3abd400640 1 -- 172.21.15.111:0/1212737315 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ddbde57a60 msgr2=0x55ddbde63780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.512+0000 7f3abd400640 1 --2- 172.21.15.111:0/1212737315 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ddbde57a60 0x55ddbde63780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.512+0000 7f3abd400640 1 -- 172.21.15.111:0/1212737315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ddbde52090 msgr2=0x55ddbde5c200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:02:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.512+0000 7f3abd400640 1 --2- 172.21.15.111:0/1212737315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ddbde52090 0x55ddbde5c200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.512+0000 7f3abd400640 1 -- 172.21.15.111:0/1212737315 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddbdb96620 con 0x55ddbde529d0 2024-09-15T23:02:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.513+0000 7f3abd400640 1 --2- 172.21.15.111:0/1212737315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ddbde529d0 0x55ddbde5c740 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3aac031440 tx=0x7f3aac0095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4abb15095603948 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.513+0000 7f3a9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:02:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.513+0000 7f3a9fe00640 1 -- 172.21.15.111:0/1212737315 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac03e030 con 0x55ddbde529d0 2024-09-15T23:02:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.513+0000 7f3a9fe00640 1 -- 172.21.15.111:0/1212737315 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f3aac0316b0 con 0x55ddbde529d0 2024-09-15T23:02:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.513+0000 7f3ac8056740 1 -- 172.21.15.111:0/1212737315 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aac004400 con 0x55ddbde529d0 2024-09-15T23:02:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.513+0000 7f3a9fe00640 1 -- 172.21.15.111:0/1212737315 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3aac035040 con 0x55ddbde529d0 2024-09-15T23:02:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.513+0000 7f3ac8056740 10 client.16338 fetch_fsmap learned FSMap version 35 2024-09-15T23:02:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.513+0000 7f3ac8056740 1 -- 172.21.15.111:0/1212737315 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ddbde5a9d0 con 0x55ddbde529d0 2024-09-15T23:02:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.513+0000 7f3a9fe00640 1 -- 172.21.15.111:0/1212737315 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aac004400 con 0x55ddbde529d0 2024-09-15T23:02:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.514+0000 7f3a9fe00640 1 -- 172.21.15.111:0/1212737315 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f3aac003bc0 con 0x55ddbde529d0 2024-09-15T23:02:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.514+0000 7f3ac8056740 10 client.16338 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:02:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.514+0000 7f3ac8056740 10 client.16338 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:02:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.514+0000 7f3ac8056740 20 client.16338 populate_metadata read hostname 'smithi111' 2024-09-15T23:02:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.514+0000 7f3ac8056740 1 --2- 172.21.15.111:0/1212737315 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ddbde67900 0x55ddbde69d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:46.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.514+0000 7f3ac8056740 4 client.16338 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:02:46.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.514+0000 7f3ac8056740 1 -- 172.21.15.111:0/1212737315 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddbddc4830 con 0x55ddbde67900 2024-09-15T23:02:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.536+0000 7f3abde00640 1 --2- 172.21.15.111:0/1212737315 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ddbde67900 0x55ddbde69d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.595+0000 7f3abde00640 1 --2- 172.21.15.111:0/1212737315 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ddbde67900 0x55ddbde69d10 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa4968234984befa server_cookie=290062f40cc023e3 in_seq=0 out_seq=0 2024-09-15T23:02:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.595+0000 7f3a9fe00640 10 client.16338 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:02:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.607+0000 7f3a9fe00640 1 -- 172.21.15.111:0/1212737315 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ddbddc4830 con 0x55ddbde67900 2024-09-15T23:02:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.607+0000 7f3a9fe00640 10 client.16338 handle_command_reply: tid=0 2024-09-15T23:02:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.607+0000 7f3ac8056740 1 client.16338 shutdown 2024-09-15T23:02:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.607+0000 7f3ac8056740 1 -- 172.21.15.111:0/1212737315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ddbde529d0 msgr2=0x55ddbde5c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.607+0000 7f3ac8056740 1 --2- 172.21.15.111:0/1212737315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ddbde529d0 0x55ddbde5c740 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3aac031440 tx=0x7f3aac0095b0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.607+0000 7f3ac8056740 1 -- 172.21.15.111:0/1212737315 shutdown_connections 2024-09-15T23:02:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.607+0000 7f3ac8056740 1 --2- 172.21.15.111:0/1212737315 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ddbde67900 0x55ddbde69d10 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.607+0000 7f3ac8056740 1 --2- 172.21.15.111:0/1212737315 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ddbde57a60 0x55ddbde63780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.607+0000 7f3ac8056740 1 --2- 172.21.15.111:0/1212737315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ddbde529d0 0x55ddbde5c740 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.607+0000 7f3ac8056740 1 --2- 172.21.15.111:0/1212737315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ddbde52090 0x55ddbde5c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.607+0000 7f3ac8056740 1 -- 172.21.15.111:0/1212737315 >> 172.21.15.111:0/1212737315 conn(0x55ddbde4d700 msgr2=0x55ddbde4ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.607+0000 7f3a9fe00640 0 client.16338 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:02:46.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.607+0000 7f3ac8056740 1 -- 172.21.15.111:0/1212737315 shutdown_connections 2024-09-15T23:02:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.607+0000 7f3ac8056740 1 -- 172.21.15.111:0/1212737315 wait complete. 2024-09-15T23:02:46.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.607+0000 7f3ac8056740 20 client.16338 trim_cache size 0 max 16384 2024-09-15T23:02:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.627+0000 7f3ac8056740 1 Processor -- start 2024-09-15T23:02:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.627+0000 7f3ac8056740 1 -- start start 2024-09-15T23:02:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.627+0000 7f3ac8056740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ddbde52090 0x55ddbde4dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.627+0000 7f3ac8056740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ddbde529d0 0x55ddbde63740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.627+0000 7f3ac8056740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ddbde57a60 0x55ddbde4e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.627+0000 7f3ac8056740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac01b1c80 con 0x55ddbde57a60 2024-09-15T23:02:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.627+0000 7f3ac8056740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac01af550 con 0x55ddbde52090 2024-09-15T23:02:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.627+0000 7f3ac8056740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac01ad3a0 con 0x55ddbde529d0 2024-09-15T23:02:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.627+0000 7f3abe800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ddbde57a60 0x55ddbde4e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.627+0000 7f3abde00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ddbde52090 0x55ddbde4dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.627+0000 7f3abe800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ddbde57a60 0x55ddbde4e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46494/0 (socket says 172.21.15.111:46494) 2024-09-15T23:02:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.627+0000 7f3abe800640 1 -- 172.21.15.111:0/1896033028 learned_addr learned my addr 172.21.15.111:0/1896033028 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:02:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.627+0000 7f3abde00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ddbde52090 0x55ddbde4dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:39288/0 (socket says 172.21.15.111:39288) 2024-09-15T23:02:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.628+0000 7f3abde00640 1 -- 172.21.15.111:0/1896033028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ddbde529d0 msgr2=0x55ddbde63740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:02:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.628+0000 7f3abde00640 1 --2- 172.21.15.111:0/1896033028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ddbde529d0 0x55ddbde63740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.628+0000 7f3abde00640 1 -- 172.21.15.111:0/1896033028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ddbde57a60 msgr2=0x55ddbde4e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.628+0000 7f3abde00640 1 --2- 172.21.15.111:0/1896033028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ddbde57a60 0x55ddbde4e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.628+0000 7f3abde00640 1 -- 172.21.15.111:0/1896033028 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddbde4d520 con 0x55ddbde52090 2024-09-15T23:02:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.628+0000 7f3abde00640 1 --2- 172.21.15.111:0/1896033028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ddbde52090 0x55ddbde4dae0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55ddbde53400 tx=0x7f3ab0003f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=663dc1bda7ffaa65 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.628+0000 7f3aaaa00640 1 -- 172.21.15.111:0/1896033028 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab009c290 con 0x55ddbde52090 2024-09-15T23:02:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.628+0000 7f3aaaa00640 1 -- 172.21.15.111:0/1896033028 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ab00a0640 con 0x55ddbde52090 2024-09-15T23:02:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.628+0000 7f3aaaa00640 1 -- 172.21.15.111:0/1896033028 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab00a0170 con 0x55ddbde52090 2024-09-15T23:02:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.629+0000 7f3ac8056740 1 -- 172.21.15.111:0/1896033028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ddbde52090 msgr2=0x55ddbde4dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.629+0000 7f3ac8056740 1 --2- 172.21.15.111:0/1896033028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ddbde52090 0x55ddbde4dae0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55ddbde53400 tx=0x7f3ab0003f30 comp rx=0 tx=0).stop 2024-09-15T23:02:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.629+0000 7f3ac8056740 1 -- 172.21.15.111:0/1896033028 shutdown_connections 2024-09-15T23:02:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.629+0000 7f3ac8056740 1 --2- 172.21.15.111:0/1896033028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ddbde57a60 0x55ddbde4e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.629+0000 7f3ac8056740 1 --2- 172.21.15.111:0/1896033028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ddbde529d0 0x55ddbde63740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.629+0000 7f3ac8056740 1 --2- 172.21.15.111:0/1896033028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ddbde52090 0x55ddbde4dae0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.629+0000 7f3ac8056740 1 -- 172.21.15.111:0/1896033028 >> 172.21.15.111:0/1896033028 conn(0x55ddbde4d700 msgr2=0x55ddbde5c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.629+0000 7f3ac8056740 1 -- 172.21.15.111:0/1896033028 shutdown_connections 2024-09-15T23:02:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.629+0000 7f3ac8056740 1 -- 172.21.15.111:0/1896033028 wait complete. 2024-09-15T23:02:46.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.630+0000 7f3ac8056740 1 -- start start 2024-09-15T23:02:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.630+0000 7f3ac8056740 1 Processor -- start 2024-09-15T23:02:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.630+0000 7f3ac8056740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ddbde52090 0x55ddbde60350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.630+0000 7f3ac8056740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ddbde529d0 0x55ddbde60890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.630+0000 7f3ac8056740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ddbde57a60 0x55ddbded05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.630+0000 7f3ac8056740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac01ada10 con 0x55ddbde529d0 2024-09-15T23:02:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.630+0000 7f3ac8056740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac01aee80 con 0x55ddbde52090 2024-09-15T23:02:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.630+0000 7f3ac8056740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac01ae0e0 con 0x55ddbde57a60 2024-09-15T23:02:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.630+0000 7f3ac8056740 10 client.0 _ll_register_callbacks cb 0x7f3ab8009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:02:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.631+0000 7f3abd400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ddbde529d0 0x55ddbde60890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.631+0000 7f3abd400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ddbde529d0 0x55ddbde60890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:46508/0 (socket says 172.21.15.111:46508) 2024-09-15T23:02:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.631+0000 7f3abd400640 1 -- 172.21.15.111:0/3227998237 learned_addr learned my addr 172.21.15.111:0/3227998237 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:02:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.631+0000 7f3abde00640 1 --2- 172.21.15.111:0/3227998237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ddbde52090 0x55ddbde60350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.631+0000 7f3abe800640 1 --2- 172.21.15.111:0/3227998237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ddbde57a60 0x55ddbded05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.631+0000 7f3abde00640 1 -- 172.21.15.111:0/3227998237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ddbde57a60 msgr2=0x55ddbded05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.631+0000 7f3abde00640 1 --2- 172.21.15.111:0/3227998237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ddbde57a60 0x55ddbded05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.631+0000 7f3abde00640 1 -- 172.21.15.111:0/3227998237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ddbde529d0 msgr2=0x55ddbde60890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.631+0000 7f3abde00640 1 --2- 172.21.15.111:0/3227998237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ddbde529d0 0x55ddbde60890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.631+0000 7f3abde00640 1 -- 172.21.15.111:0/3227998237 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddbde63980 con 0x55ddbde52090 2024-09-15T23:02:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.631+0000 7f3abd400640 1 --2- 172.21.15.111:0/3227998237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ddbde529d0 0x55ddbde60890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:02:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.631+0000 7f3abde00640 1 --2- 172.21.15.111:0/3227998237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ddbde52090 0x55ddbde60350 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55ddbde53400 tx=0x7f3ab0003ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=981fda0345c73719 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.631+0000 7f3a9fe00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:02:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.632+0000 7f3a9fe00640 1 -- 172.21.15.111:0/3227998237 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab009a590 con 0x55ddbde52090 2024-09-15T23:02:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.632+0000 7f3a9fe00640 1 -- 172.21.15.111:0/3227998237 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f3ab00a0170 con 0x55ddbde52090 2024-09-15T23:02:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.632+0000 7f3ac8056740 1 -- 172.21.15.111:0/3227998237 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ab00a0640 con 0x55ddbde52090 2024-09-15T23:02:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.632+0000 7f3a9fe00640 1 -- 172.21.15.111:0/3227998237 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3ab009c290 con 0x55ddbde52090 2024-09-15T23:02:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.632+0000 7f3ac8056740 10 client.26291 fetch_fsmap learned FSMap version 35 2024-09-15T23:02:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.632+0000 7f3ac8056740 1 -- 172.21.15.111:0/3227998237 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ddbded1b60 con 0x55ddbde52090 2024-09-15T23:02:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.632+0000 7f3a9fe00640 1 -- 172.21.15.111:0/3227998237 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ab00a0640 con 0x55ddbde52090 2024-09-15T23:02:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.632+0000 7f3a9fe00640 1 -- 172.21.15.111:0/3227998237 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f3ab00bc140 con 0x55ddbde52090 2024-09-15T23:02:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.632+0000 7f3ac8056740 10 client.26291 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:02:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.632+0000 7f3ac8056740 10 client.26291 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:02:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.632+0000 7f3ac8056740 20 client.26291 populate_metadata read hostname 'smithi111' 2024-09-15T23:02:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.632+0000 7f3ac8056740 1 --2- 172.21.15.111:0/3227998237 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ddbded2440 0x55ddbded4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.632+0000 7f3ac8056740 4 client.26291 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:02:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.633+0000 7f3ac8056740 1 -- 172.21.15.111:0/3227998237 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ddbddbeb80 con 0x55ddbded2440 2024-09-15T23:02:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.633+0000 7f3abd400640 1 --2- 172.21.15.111:0/3227998237 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ddbded2440 0x55ddbded4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.633+0000 7f3abd400640 1 --2- 172.21.15.111:0/3227998237 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ddbded2440 0x55ddbded4860 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9d30ebc578fd5da server_cookie=33ac84e8b7e25ea8 in_seq=0 out_seq=0 2024-09-15T23:02:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.633+0000 7f3a9fe00640 10 client.26291 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:02:46.638 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73440 18.29 MB/sec execute 74 sec latency 6522.632 ms 2024-09-15T23:02:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.781+0000 7f3a9fe00640 1 -- 172.21.15.111:0/3227998237 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ddbddbeb80 con 0x55ddbded2440 2024-09-15T23:02:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.781+0000 7f3a9fe00640 10 client.26291 handle_command_reply: tid=0 2024-09-15T23:02:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.781+0000 7f3ac8056740 1 client.26291 shutdown 2024-09-15T23:02:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.781+0000 7f3ac8056740 1 -- 172.21.15.111:0/3227998237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ddbde52090 msgr2=0x55ddbde60350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.781+0000 7f3ac8056740 1 --2- 172.21.15.111:0/3227998237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ddbde52090 0x55ddbde60350 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55ddbde53400 tx=0x7f3ab0003ca0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.781+0000 7f3ac8056740 1 -- 172.21.15.111:0/3227998237 shutdown_connections 2024-09-15T23:02:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.781+0000 7f3ac8056740 1 --2- 172.21.15.111:0/3227998237 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ddbded2440 0x55ddbded4860 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.781+0000 7f3ac8056740 1 --2- 172.21.15.111:0/3227998237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ddbde57a60 0x55ddbded05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.781+0000 7f3ac8056740 1 --2- 172.21.15.111:0/3227998237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ddbde529d0 0x55ddbde60890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.781+0000 7f3ac8056740 1 --2- 172.21.15.111:0/3227998237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ddbde52090 0x55ddbde60350 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.781+0000 7f3ac8056740 1 -- 172.21.15.111:0/3227998237 >> 172.21.15.111:0/3227998237 conn(0x55ddbde4d700 msgr2=0x55ddbde4c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.781+0000 7f3a9fe00640 0 client.26291 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:02:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.781+0000 7f3ac8056740 1 -- 172.21.15.111:0/3227998237 shutdown_connections 2024-09-15T23:02:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.781+0000 7f3ac8056740 1 -- 172.21.15.111:0/3227998237 wait complete. 2024-09-15T23:02:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.782+0000 7f3ac8056740 20 client.26291 trim_cache size 0 max 16384 2024-09-15T23:02:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:02:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:02:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:02:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:02:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.782+0000 7f3ac5200640 1 -- 172.21.15.111:0/644219178 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3aa40a32f0 msgr2=0x7f3aa40a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.782+0000 7f3ac5200640 1 --2- 172.21.15.111:0/644219178 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3aa40a32f0 0x7f3aa40a57b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3aac000c00 tx=0x7f3aac005fd0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.782+0000 7f3ac5200640 1 -- 172.21.15.111:0/644219178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ac0074c50 msgr2=0x7f3ac019e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.782+0000 7f3ac5200640 1 --2- 172.21.15.111:0/644219178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ac0074c50 0x7f3ac019e3d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000ea60 tx=0x7f3ab000ef20 comp rx=0 tx=0).stop 2024-09-15T23:02:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.783+0000 7f3ac5200640 1 -- 172.21.15.111:0/644219178 shutdown_connections 2024-09-15T23:02:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.783+0000 7f3ac5200640 1 --2- 172.21.15.111:0/644219178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ac010e130 0x7f3ac01a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.783+0000 7f3ac5200640 1 --2- 172.21.15.111:0/644219178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ac0076230 0x7f3ac019e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.783+0000 7f3ac5200640 1 --2- 172.21.15.111:0/644219178 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3aa40a32f0 0x7f3aa40a57b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.783+0000 7f3ac5200640 1 --2- 172.21.15.111:0/644219178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ac0074c50 0x7f3ac019e3d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.783+0000 7f3ac5200640 1 -- 172.21.15.111:0/644219178 >> 172.21.15.111:0/644219178 conn(0x7f3ac00fd2e0 msgr2=0x7f3ac01014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.783+0000 7f3ac5200640 1 -- 172.21.15.111:0/644219178 shutdown_connections 2024-09-15T23:02:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:46.783+0000 7f3ac5200640 1 -- 172.21.15.111:0/644219178 wait complete. 2024-09-15T23:02:47.639 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73440 18.05 MB/sec execute 75 sec latency 7522.698 ms 2024-09-15T23:02:48.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:47 smithi192 ceph-mon[35427]: pgmap v857: 97 pgs: 97 active+clean; 545 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.3 MiB/s wr, 455 op/s 2024-09-15T23:02:48.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:47 smithi121 ceph-mon[34815]: pgmap v857: 97 pgs: 97 active+clean; 545 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.3 MiB/s wr, 455 op/s 2024-09-15T23:02:48.412 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:47 smithi111 ceph-mon[30629]: pgmap v857: 97 pgs: 97 active+clean; 545 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.3 MiB/s wr, 455 op/s 2024-09-15T23:02:48.639 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73440 17.81 MB/sec execute 76 sec latency 8522.791 ms 2024-09-15T23:02:49.639 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73444 17.58 MB/sec execute 77 sec latency 8851.317 ms 2024-09-15T23:02:50.148 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:49 smithi111 ceph-mon[30629]: pgmap v858: 97 pgs: 97 active+clean; 538 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.3 MiB/s wr, 482 op/s 2024-09-15T23:02:50.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:49 smithi192 ceph-mon[35427]: pgmap v858: 97 pgs: 97 active+clean; 538 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.3 MiB/s wr, 482 op/s 2024-09-15T23:02:50.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:49 smithi121 ceph-mon[34815]: pgmap v858: 97 pgs: 97 active+clean; 538 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.3 MiB/s wr, 482 op/s 2024-09-15T23:02:50.639 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73444 17.36 MB/sec execute 78 sec latency 5329.289 ms 2024-09-15T23:02:51.639 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73445 17.14 MB/sec execute 79 sec latency 6209.572 ms 2024-09-15T23:02:52.153 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:51 smithi111 ceph-mon[30629]: pgmap v859: 97 pgs: 97 active+clean; 538 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.1 MiB/s wr, 383 op/s 2024-09-15T23:02:52.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:51 smithi192 ceph-mon[35427]: pgmap v859: 97 pgs: 97 active+clean; 538 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.1 MiB/s wr, 383 op/s 2024-09-15T23:02:52.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:51 smithi121 ceph-mon[34815]: pgmap v859: 97 pgs: 97 active+clean; 538 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.1 MiB/s wr, 383 op/s 2024-09-15T23:02:52.639 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73445 16.92 MB/sec execute 80 sec latency 7209.644 ms 2024-09-15T23:02:53.639 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73445 16.72 MB/sec execute 81 sec latency 7819.991 ms 2024-09-15T23:02:54.052 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:53 smithi111 ceph-mon[30629]: pgmap v860: 97 pgs: 97 active+clean; 541 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.5 MiB/s wr, 325 op/s 2024-09-15T23:02:54.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:53 smithi192 ceph-mon[35427]: pgmap v860: 97 pgs: 97 active+clean; 541 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.5 MiB/s wr, 325 op/s 2024-09-15T23:02:54.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:53 smithi121 ceph-mon[34815]: pgmap v860: 97 pgs: 97 active+clean; 541 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.5 MiB/s wr, 325 op/s 2024-09-15T23:02:54.639 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73445 16.51 MB/sec execute 82 sec latency 8334.540 ms 2024-09-15T23:02:55.639 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73449 16.31 MB/sec execute 83 sec latency 9334.580 ms 2024-09-15T23:02:56.065 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:55 smithi111 ceph-mon[30629]: pgmap v861: 97 pgs: 97 active+clean; 543 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 5.9 MiB/s wr, 279 op/s 2024-09-15T23:02:56.065 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:55 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:02:56.065 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:55 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:02:56.065 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:55 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:02:56.065 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:55 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:02:56.065 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:55 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:02:56.066 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:55 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:02:56.066 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:55 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:02:56.066 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:55 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:02:56.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:55 smithi192 ceph-mon[35427]: pgmap v861: 97 pgs: 97 active+clean; 543 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 5.9 MiB/s wr, 279 op/s 2024-09-15T23:02:56.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:55 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:02:56.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:55 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:02:56.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:55 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:02:56.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:55 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:02:56.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:55 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:02:56.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:55 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:02:56.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:55 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:02:56.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:55 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:02:56.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:55 smithi121 ceph-mon[34815]: pgmap v861: 97 pgs: 97 active+clean; 543 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 5.9 MiB/s wr, 279 op/s 2024-09-15T23:02:56.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:55 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:02:56.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:55 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:02:56.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:55 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:02:56.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:55 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:02:56.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:55 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:02:56.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:55 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:02:56.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:55 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:02:56.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:55 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:02:56.639 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73453 16.12 MB/sec execute 84 sec latency 10334.620 ms 2024-09-15T23:02:57.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:56 smithi111 ceph-mon[30629]: Adjusting osd_memory_target on smithi111 to 176.4M 2024-09-15T23:02:57.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:56 smithi111 ceph-mon[30629]: Unable to set osd_memory_target on smithi111 to 185037619: error parsing value: Value '185037619' is below minimum 939524096 2024-09-15T23:02:57.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:56 smithi192 ceph-mon[35427]: Adjusting osd_memory_target on smithi111 to 176.4M 2024-09-15T23:02:57.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:56 smithi192 ceph-mon[35427]: Unable to set osd_memory_target on smithi111 to 185037619: error parsing value: Value '185037619' is below minimum 939524096 2024-09-15T23:02:57.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:56 smithi121 ceph-mon[34815]: Adjusting osd_memory_target on smithi111 to 176.4M 2024-09-15T23:02:57.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:56 smithi121 ceph-mon[34815]: Unable to set osd_memory_target on smithi111 to 185037619: error parsing value: Value '185037619' is below minimum 939524096 2024-09-15T23:02:57.639 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73453 15.93 MB/sec execute 85 sec latency 11334.692 ms 2024-09-15T23:02:58.258 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:57 smithi111 ceph-mon[30629]: pgmap v862: 97 pgs: 97 active+clean; 538 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 5.0 MiB/s wr, 239 op/s 2024-09-15T23:02:58.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:57 smithi192 ceph-mon[35427]: pgmap v862: 97 pgs: 97 active+clean; 538 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 5.0 MiB/s wr, 239 op/s 2024-09-15T23:02:58.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:57 smithi121 ceph-mon[34815]: pgmap v862: 97 pgs: 97 active+clean; 538 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 5.0 MiB/s wr, 239 op/s 2024-09-15T23:02:58.639 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73453 15.75 MB/sec execute 86 sec latency 12334.768 ms 2024-09-15T23:02:59.295 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:02:59 smithi111 ceph-mon[30629]: pgmap v863: 97 pgs: 97 active+clean; 540 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 6.4 MiB/s wr, 269 op/s 2024-09-15T23:02:59.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:02:59 smithi192 ceph-mon[35427]: pgmap v863: 97 pgs: 97 active+clean; 540 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 6.4 MiB/s wr, 269 op/s 2024-09-15T23:02:59.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:02:59 smithi121 ceph-mon[34815]: pgmap v863: 97 pgs: 97 active+clean; 540 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 6.4 MiB/s wr, 269 op/s 2024-09-15T23:02:59.639 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73453 15.56 MB/sec execute 87 sec latency 13334.836 ms 2024-09-15T23:02:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1e000640 1 -- 172.21.15.111:0/587749736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d18104cf0 msgr2=0x7f4d181070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1e000640 1 --2- 172.21.15.111:0/587749736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d18104cf0 0x7f4d181070e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4d04009900 tx=0x7f4d0402a3f0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1e000640 1 -- 172.21.15.111:0/587749736 shutdown_connections 2024-09-15T23:02:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1e000640 1 --2- 172.21.15.111:0/587749736 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d18107620 0x7f4d18109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1e000640 1 --2- 172.21.15.111:0/587749736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d18104cf0 0x7f4d181070e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1e000640 1 --2- 172.21.15.111:0/587749736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d181023c0 0x7f4d181047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1e000640 1 -- 172.21.15.111:0/587749736 >> 172.21.15.111:0/587749736 conn(0x7f4d180fbfa0 msgr2=0x7f4d180fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1e000640 1 -- 172.21.15.111:0/587749736 shutdown_connections 2024-09-15T23:02:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1e000640 1 -- 172.21.15.111:0/587749736 wait complete. 2024-09-15T23:02:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1e000640 1 Processor -- start 2024-09-15T23:02:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1e000640 1 -- start start 2024-09-15T23:02:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1e000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d181023c0 0x7f4d18199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1e000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d18104cf0 0x7f4d1819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:59.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1e000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d18107620 0x7f4d181a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1e000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1810d640 con 0x7f4d18107620 2024-09-15T23:02:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1e000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1810d4c0 con 0x7f4d181023c0 2024-09-15T23:02:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1e000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1810d7c0 con 0x7f4d18104cf0 2024-09-15T23:02:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1d600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d18107620 0x7f4d181a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1cc00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d181023c0 0x7f4d18199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1d600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d18107620 0x7f4d181a14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47060/0 (socket says 172.21.15.111:47060) 2024-09-15T23:02:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1d600640 1 -- 172.21.15.111:0/1741135531 learned_addr learned my addr 172.21.15.111:0/1741135531 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:02:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d17e00640 1 --2- 172.21.15.111:0/1741135531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d18104cf0 0x7f4d1819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1d600640 1 -- 172.21.15.111:0/1741135531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d18104cf0 msgr2=0x7f4d1819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1d600640 1 --2- 172.21.15.111:0/1741135531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d18104cf0 0x7f4d1819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.904 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1d600640 1 -- 172.21.15.111:0/1741135531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d181023c0 msgr2=0x7f4d18199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1d600640 1 --2- 172.21.15.111:0/1741135531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d181023c0 0x7f4d18199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1d600640 1 -- 172.21.15.111:0/1741135531 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d181a1b20 con 0x7f4d18107620 2024-09-15T23:02:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1cc00640 1 --2- 172.21.15.111:0/1741135531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d181023c0 0x7f4d18199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:02:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d1d600640 1 --2- 172.21.15.111:0/1741135531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d18107620 0x7f4d181a14b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000eb40 tx=0x7f4d1000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.897+0000 7f4d15600640 1 -- 172.21.15.111:0/1741135531 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d10004b20 con 0x7f4d18107620 2024-09-15T23:02:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.902+0000 7f4d15600640 1 -- 172.21.15.111:0/1741135531 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d10004cc0 con 0x7f4d18107620 2024-09-15T23:02:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.902+0000 7f4d1e000640 1 -- 172.21.15.111:0/1741135531 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d181a1e10 con 0x7f4d18107620 2024-09-15T23:02:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.902+0000 7f4d15600640 1 -- 172.21.15.111:0/1741135531 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d100107f0 con 0x7f4d18107620 2024-09-15T23:02:59.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.902+0000 7f4d1e000640 1 -- 172.21.15.111:0/1741135531 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d181a2280 con 0x7f4d18107620 2024-09-15T23:02:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.902+0000 7f4d15600640 1 -- 172.21.15.111:0/1741135531 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f4d10005000 con 0x7f4d18107620 2024-09-15T23:02:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.902+0000 7f4d2265e740 1 Processor -- start 2024-09-15T23:02:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.902+0000 7f4d2265e740 1 -- start start 2024-09-15T23:02:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.902+0000 7f4d2265e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8689e0260 0x55c8689e0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.902+0000 7f4d2265e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c8689e0ba0 0x55c8689e5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.902+0000 7f4d2265e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8689e5c30 0x55c8689e80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.902+0000 7f4d2265e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c868945100 con 0x55c8689e0ba0 2024-09-15T23:02:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.902+0000 7f4d2265e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c8688ed1b0 con 0x55c8689e5c30 2024-09-15T23:02:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.902+0000 7f4d2265e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c8689410c0 con 0x55c8689e0260 2024-09-15T23:02:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.902+0000 7f4d17e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8689e0260 0x55c8689e0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.902+0000 7f4d17e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8689e0260 0x55c8689e0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:33038/0 (socket says 172.21.15.111:33038) 2024-09-15T23:02:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.902+0000 7f4d17e00640 1 -- 172.21.15.111:0/3874763310 learned_addr learned my addr 172.21.15.111:0/3874763310 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:02:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.902+0000 7f4d1cc00640 1 --2- 172.21.15.111:0/3874763310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c8689e0ba0 0x55c8689e5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.902+0000 7f4d17e00640 1 --2- 172.21.15.111:0/3874763310 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8689e5c30 0x55c8689e80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.902+0000 7f4d17e00640 1 -- 172.21.15.111:0/3874763310 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8689e5c30 msgr2=0x55c8689e80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.902+0000 7f4d17e00640 1 --2- 172.21.15.111:0/3874763310 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8689e5c30 0x55c8689e80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.902+0000 7f4d17e00640 1 -- 172.21.15.111:0/3874763310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c8689e0ba0 msgr2=0x55c8689e5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.902+0000 7f4d17e00640 1 --2- 172.21.15.111:0/3874763310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c8689e0ba0 0x55c8689e5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.902+0000 7f4d17e00640 1 -- 172.21.15.111:0/3874763310 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c868658210 con 0x55c8689e0260 2024-09-15T23:02:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.902+0000 7f4d15600640 1 --2- 172.21.15.111:0/1741135531 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4d000a3240 0x7f4d000a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.902+0000 7f4d17e00640 1 --2- 172.21.15.111:0/3874763310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8689e0260 0x55c8689e0660 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4d04000c00 tx=0x7f4d04005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a55c508e09edbdb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.902+0000 7f4d15600640 1 -- 172.21.15.111:0/1741135531 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f4d100d0070 con 0x7f4d18107620 2024-09-15T23:02:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.907+0000 7f4d1cc00640 1 --2- 172.21.15.111:0/1741135531 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4d000a3240 0x7f4d000a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.907+0000 7f4d09c00640 1 -- 172.21.15.111:0/3874763310 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d04035070 con 0x55c8689e0260 2024-09-15T23:02:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.907+0000 7f4d09c00640 1 -- 172.21.15.111:0/3874763310 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d040044c0 con 0x55c8689e0260 2024-09-15T23:02:59.910 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.907+0000 7f4d09c00640 1 -- 172.21.15.111:0/3874763310 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0403da40 con 0x55c8689e0260 2024-09-15T23:02:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.907+0000 7f4d2265e740 1 -- 172.21.15.111:0/3874763310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8689e0260 msgr2=0x55c8689e0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.907+0000 7f4d2265e740 1 --2- 172.21.15.111:0/3874763310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8689e0260 0x55c8689e0660 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4d04000c00 tx=0x7f4d04005780 comp rx=0 tx=0).stop 2024-09-15T23:02:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.907+0000 7f4d1cc00640 1 --2- 172.21.15.111:0/1741135531 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4d000a3240 0x7f4d000a5700 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55c8689e15d0 tx=0x7f4d0c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.907+0000 7f4d2265e740 1 -- 172.21.15.111:0/3874763310 shutdown_connections 2024-09-15T23:02:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.907+0000 7f4d2265e740 1 --2- 172.21.15.111:0/3874763310 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8689e5c30 0x55c8689e80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.907+0000 7f4d2265e740 1 --2- 172.21.15.111:0/3874763310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c8689e0ba0 0x55c8689e5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.907+0000 7f4d2265e740 1 --2- 172.21.15.111:0/3874763310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8689e0260 0x55c8689e0660 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.907+0000 7f4d2265e740 1 -- 172.21.15.111:0/3874763310 >> 172.21.15.111:0/3874763310 conn(0x55c8689db8d0 msgr2=0x55c8689dbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 1 -- 172.21.15.111:0/3874763310 shutdown_connections 2024-09-15T23:02:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 1 -- 172.21.15.111:0/3874763310 wait complete. 2024-09-15T23:02:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 1 -- start start 2024-09-15T23:02:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 1 Processor -- start 2024-09-15T23:02:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8689e0260 0x55c8689ea440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c8689e0ba0 0x55c8689ea980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8689e5c30 0x55c8689f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c8689e8ba0 con 0x55c8689e0ba0 2024-09-15T23:02:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c8688ce090 con 0x55c8689e5c30 2024-09-15T23:02:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c8689e8d20 con 0x55c8689e0260 2024-09-15T23:02:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 10 client.0 _ll_register_callbacks cb 0x55c868922f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:02:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d1cc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8689e0260 0x55c8689ea440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d17e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c8689e0ba0 0x55c8689ea980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d1cc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8689e0260 0x55c8689ea440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:33042/0 (socket says 172.21.15.111:33042) 2024-09-15T23:02:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d1cc00640 1 -- 172.21.15.111:0/32700702 learned_addr learned my addr 172.21.15.111:0/32700702 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:02:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d1d600640 1 --2- 172.21.15.111:0/32700702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8689e5c30 0x55c8689f19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d1cc00640 1 -- 172.21.15.111:0/32700702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8689e5c30 msgr2=0x55c8689f19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d1cc00640 1 --2- 172.21.15.111:0/32700702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8689e5c30 0x55c8689f19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d1cc00640 1 -- 172.21.15.111:0/32700702 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c8689e0ba0 msgr2=0x55c8689ea980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d1cc00640 1 --2- 172.21.15.111:0/32700702 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c8689e0ba0 0x55c8689ea980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d1cc00640 1 -- 172.21.15.111:0/32700702 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c868724620 con 0x55c8689e0260 2024-09-15T23:02:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d1cc00640 1 --2- 172.21.15.111:0/32700702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8689e0260 0x55c8689ea440 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c004650 tx=0x7f4d0c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d690b44c123fdf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4cfe000640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:02:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4cfe000640 1 -- 172.21.15.111:0/32700702 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c00f8b0 con 0x55c8689e0260 2024-09-15T23:02:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4cfe000640 1 -- 172.21.15.111:0/32700702 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f4d0c00fa50 con 0x55c8689e0260 2024-09-15T23:02:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 1 -- 172.21.15.111:0/32700702 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c868952830 con 0x55c8689e0260 2024-09-15T23:02:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4cfe000640 1 -- 172.21.15.111:0/32700702 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4d0c00fc50 con 0x55c8689e0260 2024-09-15T23:02:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 10 client.26320 fetch_fsmap learned FSMap version 35 2024-09-15T23:02:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 1 -- 172.21.15.111:0/32700702 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c8689f4880 con 0x55c8689e0260 2024-09-15T23:02:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4cfe000640 1 -- 172.21.15.111:0/32700702 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c868952830 con 0x55c8689e0260 2024-09-15T23:02:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4cfe000640 1 -- 172.21.15.111:0/32700702 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f4d0c027070 con 0x55c8689e0260 2024-09-15T23:02:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 10 client.26320 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:02:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 10 client.26320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:02:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 20 client.26320 populate_metadata read hostname 'smithi111' 2024-09-15T23:02:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 1 --2- 172.21.15.111:0/32700702 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c8689f5f70 0x55c8689f83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 4 client.26320 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:02:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 1 -- 172.21.15.111:0/32700702 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c86894cb80 con 0x55c8689f5f70 2024-09-15T23:02:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d17e00640 1 --2- 172.21.15.111:0/32700702 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c8689f5f70 0x55c8689f83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d17e00640 1 --2- 172.21.15.111:0/32700702 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c8689f5f70 0x55c8689f83f0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be8a738aed2857d1 server_cookie=84f07d8bfdc54260 in_seq=0 out_seq=0 2024-09-15T23:02:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4cfe000640 10 client.26320 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:02:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4cfe000640 1 -- 172.21.15.111:0/32700702 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c86894cb80 con 0x55c8689f5f70 2024-09-15T23:02:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4cfe000640 10 client.26320 handle_command_reply: tid=0 2024-09-15T23:02:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 1 client.26320 shutdown 2024-09-15T23:02:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 1 -- 172.21.15.111:0/32700702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8689e0260 msgr2=0x55c8689ea440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 1 --2- 172.21.15.111:0/32700702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8689e0260 0x55c8689ea440 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c004650 tx=0x7f4d0c004730 comp rx=0 tx=0).stop 2024-09-15T23:02:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 1 -- 172.21.15.111:0/32700702 shutdown_connections 2024-09-15T23:02:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 1 --2- 172.21.15.111:0/32700702 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c8689f5f70 0x55c8689f83f0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 1 --2- 172.21.15.111:0/32700702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8689e5c30 0x55c8689f19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 1 --2- 172.21.15.111:0/32700702 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c8689e0ba0 0x55c8689ea980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4cfe000640 0 client.26320 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:02:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 1 --2- 172.21.15.111:0/32700702 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8689e0260 0x55c8689ea440 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 1 -- 172.21.15.111:0/32700702 >> 172.21.15.111:0/32700702 conn(0x55c8689db8d0 msgr2=0x55c8689dd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 1 -- 172.21.15.111:0/32700702 shutdown_connections 2024-09-15T23:02:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 1 -- 172.21.15.111:0/32700702 wait complete. 2024-09-15T23:02:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.908+0000 7f4d2265e740 20 client.26320 trim_cache size 0 max 16384 2024-09-15T23:02:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.924+0000 7f4d2265e740 1 Processor -- start 2024-09-15T23:02:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.924+0000 7f4d2265e740 1 -- start start 2024-09-15T23:02:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.924+0000 7f4d2265e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8689e0260 0x55c8689dc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.924+0000 7f4d2265e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8689e0ba0 0x55c8689f1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.924+0000 7f4d2265e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c8689e5c30 0x55c8689dcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.924+0000 7f4d2265e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4d181ad7b0 con 0x55c8689e5c30 2024-09-15T23:02:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.924+0000 7f4d2265e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4d181ab080 con 0x55c8689e0ba0 2024-09-15T23:02:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.924+0000 7f4d2265e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4d181a8ef0 con 0x55c8689e0260 2024-09-15T23:02:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.924+0000 7f4d1d600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c8689e5c30 0x55c8689dcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.924+0000 7f4d1d600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c8689e5c30 0x55c8689dcc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47082/0 (socket says 172.21.15.111:47082) 2024-09-15T23:02:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.924+0000 7f4d1d600640 1 -- 172.21.15.111:0/1964218578 learned_addr learned my addr 172.21.15.111:0/1964218578 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:02:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.924+0000 7f4d1cc00640 1 --2- 172.21.15.111:0/1964218578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8689e0260 0x55c8689dc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.924+0000 7f4d17e00640 1 --2- 172.21.15.111:0/1964218578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8689e0ba0 0x55c8689f1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.924+0000 7f4d1d600640 1 -- 172.21.15.111:0/1964218578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8689e0260 msgr2=0x55c8689dc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.924+0000 7f4d1d600640 1 --2- 172.21.15.111:0/1964218578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8689e0260 0x55c8689dc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.924+0000 7f4d1d600640 1 -- 172.21.15.111:0/1964218578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8689e0ba0 msgr2=0x55c8689f1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.924+0000 7f4d1d600640 1 --2- 172.21.15.111:0/1964218578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8689e0ba0 0x55c8689f1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.924+0000 7f4d1d600640 1 -- 172.21.15.111:0/1964218578 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8689e1080 con 0x55c8689e5c30 2024-09-15T23:02:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d1d600640 1 --2- 172.21.15.111:0/1964218578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c8689e5c30 0x55c8689dcc60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4d10004af0 tx=0x7f4d100982f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea1df973037968e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d09200640 1 -- 172.21.15.111:0/1964218578 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d100ba070 con 0x55c8689e5c30 2024-09-15T23:02:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d09200640 1 -- 172.21.15.111:0/1964218578 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d1009c690 con 0x55c8689e5c30 2024-09-15T23:02:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d09200640 1 -- 172.21.15.111:0/1964218578 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d100a4e70 con 0x55c8689e5c30 2024-09-15T23:02:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d2265e740 1 -- 172.21.15.111:0/1964218578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c8689e5c30 msgr2=0x55c8689dcc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d2265e740 1 --2- 172.21.15.111:0/1964218578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c8689e5c30 0x55c8689dcc60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4d10004af0 tx=0x7f4d100982f0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d2265e740 1 -- 172.21.15.111:0/1964218578 shutdown_connections 2024-09-15T23:02:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d2265e740 1 --2- 172.21.15.111:0/1964218578 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c8689e5c30 0x55c8689dcc60 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d2265e740 1 --2- 172.21.15.111:0/1964218578 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8689e0ba0 0x55c8689f1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d2265e740 1 --2- 172.21.15.111:0/1964218578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8689e0260 0x55c8689dc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d2265e740 1 -- 172.21.15.111:0/1964218578 >> 172.21.15.111:0/1964218578 conn(0x55c8689db8d0 msgr2=0x55c8689ea1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d2265e740 1 -- 172.21.15.111:0/1964218578 shutdown_connections 2024-09-15T23:02:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d2265e740 1 -- 172.21.15.111:0/1964218578 wait complete. 2024-09-15T23:02:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d2265e740 1 -- start start 2024-09-15T23:02:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d2265e740 1 Processor -- start 2024-09-15T23:02:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d2265e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8689e0260 0x55c8689eef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d2265e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8689e0ba0 0x55c8689ef460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d2265e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c8689e5c30 0x55c8689f0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d2265e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4d181a9540 con 0x55c8689e5c30 2024-09-15T23:02:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d2265e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4d181aa9b0 con 0x55c8689e0ba0 2024-09-15T23:02:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d2265e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4d181a9c10 con 0x55c8689e0260 2024-09-15T23:02:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d2265e740 10 client.0 _ll_register_callbacks cb 0x7f4d0c009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:02:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d1cc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8689e0260 0x55c8689eef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d17e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8689e0ba0 0x55c8689ef460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d1d600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c8689e5c30 0x55c8689f0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d1cc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8689e0260 0x55c8689eef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:33052/0 (socket says 172.21.15.111:33052) 2024-09-15T23:02:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d1cc00640 1 -- 172.21.15.111:0/1959977179 learned_addr learned my addr 172.21.15.111:0/1959977179 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:02:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d17e00640 1 -- 172.21.15.111:0/1959977179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8689e0260 msgr2=0x55c8689eef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d17e00640 1 --2- 172.21.15.111:0/1959977179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8689e0260 0x55c8689eef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d17e00640 1 -- 172.21.15.111:0/1959977179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c8689e5c30 msgr2=0x55c8689f0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:02:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d17e00640 1 --2- 172.21.15.111:0/1959977179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c8689e5c30 0x55c8689f0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.929+0000 7f4d17e00640 1 -- 172.21.15.111:0/1959977179 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8689f0a80 con 0x55c8689e0ba0 2024-09-15T23:02:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.932+0000 7f4d1d600640 1 --2- 172.21.15.111:0/1959977179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c8689e5c30 0x55c8689f0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:02:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.932+0000 7f4d17e00640 1 --2- 172.21.15.111:0/1959977179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8689e0ba0 0x55c8689ef460 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4d04006010 tx=0x7f4d04005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff1f7413d781a19c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:02:59.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.932+0000 7f4cfe000640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:02:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.932+0000 7f4cfe000640 1 -- 172.21.15.111:0/1959977179 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0405c060 con 0x55c8689e0ba0 2024-09-15T23:02:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.932+0000 7f4d2265e740 1 -- 172.21.15.111:0/1959977179 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d1009c690 con 0x55c8689e0ba0 2024-09-15T23:02:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.932+0000 7f4cfe000640 1 -- 172.21.15.111:0/1959977179 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f4d0403dab0 con 0x55c8689e0ba0 2024-09-15T23:02:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.932+0000 7f4cfe000640 1 -- 172.21.15.111:0/1959977179 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4d04035040 con 0x55c8689e0ba0 2024-09-15T23:02:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.932+0000 7f4d2265e740 10 client.26309 fetch_fsmap learned FSMap version 35 2024-09-15T23:02:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.932+0000 7f4d2265e740 1 -- 172.21.15.111:0/1959977179 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c868a60cd0 con 0x55c8689e0ba0 2024-09-15T23:02:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.932+0000 7f4cfe000640 1 -- 172.21.15.111:0/1959977179 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d1009c690 con 0x55c8689e0ba0 2024-09-15T23:02:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.932+0000 7f4cfe000640 1 -- 172.21.15.111:0/1959977179 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f4d04061070 con 0x55c8689e0ba0 2024-09-15T23:02:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.932+0000 7f4d2265e740 10 client.26309 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:02:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.932+0000 7f4d2265e740 10 client.26309 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:02:59.941 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.932+0000 7f4d2265e740 20 client.26309 populate_metadata read hostname 'smithi111' 2024-09-15T23:02:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.932+0000 7f4d2265e740 1 --2- 172.21.15.111:0/1959977179 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c868a61060 0x55c868a63450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:02:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.932+0000 7f4d2265e740 4 client.26309 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:02:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.932+0000 7f4d2265e740 1 -- 172.21.15.111:0/1959977179 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c8688c97c0 con 0x55c868a61060 2024-09-15T23:02:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.932+0000 7f4d1cc00640 1 --2- 172.21.15.111:0/1959977179 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c868a61060 0x55c868a63450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:02:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.932+0000 7f4cfe000640 10 client.26309 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:02:59.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.932+0000 7f4d1cc00640 1 --2- 172.21.15.111:0/1959977179 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c868a61060 0x55c868a63450 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9961371b954f6952 server_cookie=ba3e7a224e91c1b4 in_seq=0 out_seq=0 2024-09-15T23:02:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.970+0000 7f4cfe000640 1 -- 172.21.15.111:0/1959977179 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c8688c97c0 con 0x55c868a61060 2024-09-15T23:02:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.970+0000 7f4cfe000640 10 client.26309 handle_command_reply: tid=0 2024-09-15T23:02:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.970+0000 7f4d2265e740 1 client.26309 shutdown 2024-09-15T23:02:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.970+0000 7f4d2265e740 1 -- 172.21.15.111:0/1959977179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8689e0ba0 msgr2=0x55c8689ef460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.970+0000 7f4d2265e740 1 --2- 172.21.15.111:0/1959977179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8689e0ba0 0x55c8689ef460 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4d04006010 tx=0x7f4d04005c00 comp rx=0 tx=0).stop 2024-09-15T23:02:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.970+0000 7f4d2265e740 1 -- 172.21.15.111:0/1959977179 shutdown_connections 2024-09-15T23:02:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.970+0000 7f4d2265e740 1 --2- 172.21.15.111:0/1959977179 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c868a61060 0x55c868a63450 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.970+0000 7f4d2265e740 1 --2- 172.21.15.111:0/1959977179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c8689e5c30 0x55c8689f0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.970+0000 7f4d2265e740 1 --2- 172.21.15.111:0/1959977179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c8689e0ba0 0x55c8689ef460 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.970+0000 7f4cfe000640 0 client.26309 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:02:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.970+0000 7f4d2265e740 1 --2- 172.21.15.111:0/1959977179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c8689e0260 0x55c8689eef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.970+0000 7f4d2265e740 1 -- 172.21.15.111:0/1959977179 >> 172.21.15.111:0/1959977179 conn(0x55c8689db8d0 msgr2=0x55c8689da1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.970+0000 7f4d2265e740 1 -- 172.21.15.111:0/1959977179 shutdown_connections 2024-09-15T23:02:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.971+0000 7f4d2265e740 1 -- 172.21.15.111:0/1959977179 wait complete. 2024-09-15T23:02:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.971+0000 7f4d2265e740 20 client.26309 trim_cache size 0 max 16384 2024-09-15T23:02:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:02:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:02:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:02:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:02:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.971+0000 7f4d1e000640 1 -- 172.21.15.111:0/1741135531 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4d000a3240 msgr2=0x7f4d000a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.971+0000 7f4d1e000640 1 --2- 172.21.15.111:0/1741135531 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4d000a3240 0x7f4d000a5700 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55c8689e15d0 tx=0x7f4d0c005f90 comp rx=0 tx=0).stop 2024-09-15T23:02:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.971+0000 7f4d1e000640 1 -- 172.21.15.111:0/1741135531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d18107620 msgr2=0x7f4d181a14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:02:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.971+0000 7f4d1e000640 1 --2- 172.21.15.111:0/1741135531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d18107620 0x7f4d181a14b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000eb40 tx=0x7f4d1000c2d0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.971+0000 7f4d1e000640 1 -- 172.21.15.111:0/1741135531 shutdown_connections 2024-09-15T23:02:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.971+0000 7f4d1e000640 1 --2- 172.21.15.111:0/1741135531 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d18107620 0x7f4d181a14b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.971+0000 7f4d1e000640 1 --2- 172.21.15.111:0/1741135531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d18104cf0 0x7f4d1819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.971+0000 7f4d1e000640 1 --2- 172.21.15.111:0/1741135531 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4d000a3240 0x7f4d000a5700 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.971+0000 7f4d1e000640 1 --2- 172.21.15.111:0/1741135531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d181023c0 0x7f4d18199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:02:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.971+0000 7f4d1e000640 1 -- 172.21.15.111:0/1741135531 >> 172.21.15.111:0/1741135531 conn(0x7f4d180fbfa0 msgr2=0x7f4d18101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:02:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.971+0000 7f4d1e000640 1 -- 172.21.15.111:0/1741135531 shutdown_connections 2024-09-15T23:02:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:02:59.971+0000 7f4d1e000640 1 -- 172.21.15.111:0/1741135531 wait complete. 2024-09-15T23:03:00.640 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73453 15.39 MB/sec execute 88 sec latency 14334.904 ms 2024-09-15T23:03:01.639 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73453 15.21 MB/sec execute 89 sec latency 15334.947 ms 2024-09-15T23:03:02.131 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:01 smithi111 ceph-mon[30629]: pgmap v864: 97 pgs: 97 active+clean; 537 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 5.7 MiB/s wr, 221 op/s 2024-09-15T23:03:02.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:01 smithi192 ceph-mon[35427]: pgmap v864: 97 pgs: 97 active+clean; 537 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 5.7 MiB/s wr, 221 op/s 2024-09-15T23:03:02.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:01 smithi121 ceph-mon[34815]: pgmap v864: 97 pgs: 97 active+clean; 537 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 5.7 MiB/s wr, 221 op/s 2024-09-15T23:03:02.640 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73457 15.05 MB/sec execute 90 sec latency 15695.936 ms 2024-09-15T23:03:03.640 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73458 14.88 MB/sec execute 91 sec latency 12453.393 ms 2024-09-15T23:03:04.160 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:03 smithi111 ceph-mon[30629]: pgmap v865: 97 pgs: 97 active+clean; 542 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 5.6 MiB/s wr, 193 op/s 2024-09-15T23:03:04.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:03 smithi192 ceph-mon[35427]: pgmap v865: 97 pgs: 97 active+clean; 542 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 5.6 MiB/s wr, 193 op/s 2024-09-15T23:03:04.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:03 smithi121 ceph-mon[34815]: pgmap v865: 97 pgs: 97 active+clean; 542 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 5.6 MiB/s wr, 193 op/s 2024-09-15T23:03:04.640 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73461 14.73 MB/sec execute 92 sec latency 13453.465 ms 2024-09-15T23:03:05.640 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73461 14.57 MB/sec execute 93 sec latency 14453.558 ms 2024-09-15T23:03:06.159 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:05 smithi111 ceph-mon[30629]: pgmap v866: 97 pgs: 97 active+clean; 537 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 5.1 MiB/s wr, 202 op/s 2024-09-15T23:03:06.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:05 smithi192 ceph-mon[35427]: pgmap v866: 97 pgs: 97 active+clean; 537 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 5.1 MiB/s wr, 202 op/s 2024-09-15T23:03:06.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:05 smithi121 ceph-mon[34815]: pgmap v866: 97 pgs: 97 active+clean; 537 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 5.1 MiB/s wr, 202 op/s 2024-09-15T23:03:06.640 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73462 14.42 MB/sec execute 94 sec latency 15138.707 ms 2024-09-15T23:03:07.640 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73462 14.26 MB/sec execute 95 sec latency 12439.959 ms 2024-09-15T23:03:08.137 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:07 smithi111 ceph-mon[30629]: pgmap v867: 97 pgs: 97 active+clean; 533 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 5.5 MiB/s wr, 218 op/s 2024-09-15T23:03:08.192 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:07 smithi121 ceph-mon[34815]: pgmap v867: 97 pgs: 97 active+clean; 533 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 5.5 MiB/s wr, 218 op/s 2024-09-15T23:03:08.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:07 smithi192 ceph-mon[35427]: pgmap v867: 97 pgs: 97 active+clean; 533 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 5.5 MiB/s wr, 218 op/s 2024-09-15T23:03:08.640 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73462 14.12 MB/sec execute 96 sec latency 13440.034 ms 2024-09-15T23:03:09.640 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73462 13.97 MB/sec execute 97 sec latency 14440.078 ms 2024-09-15T23:03:10.122 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:09 smithi111 ceph-mon[30629]: pgmap v868: 97 pgs: 97 active+clean; 521 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 7.0 MiB/s wr, 269 op/s 2024-09-15T23:03:10.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:09 smithi192 ceph-mon[35427]: pgmap v868: 97 pgs: 97 active+clean; 521 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 7.0 MiB/s wr, 269 op/s 2024-09-15T23:03:10.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:09 smithi121 ceph-mon[34815]: pgmap v868: 97 pgs: 97 active+clean; 521 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 7.0 MiB/s wr, 269 op/s 2024-09-15T23:03:10.640 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73462 13.83 MB/sec execute 98 sec latency 15440.147 ms 2024-09-15T23:03:11.640 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73462 13.69 MB/sec execute 99 sec latency 16440.207 ms 2024-09-15T23:03:12.123 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:11 smithi111 ceph-mon[30629]: pgmap v869: 97 pgs: 97 active+clean; 521 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 5.7 MiB/s wr, 239 op/s 2024-09-15T23:03:12.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:11 smithi192 ceph-mon[35427]: pgmap v869: 97 pgs: 97 active+clean; 521 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 5.7 MiB/s wr, 239 op/s 2024-09-15T23:03:12.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:11 smithi121 ceph-mon[34815]: pgmap v869: 97 pgs: 97 active+clean; 521 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 5.7 MiB/s wr, 239 op/s 2024-09-15T23:03:12.640 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73462 13.55 MB/sec execute 100 sec latency 17440.294 ms 2024-09-15T23:03:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5f07200640 1 -- 172.21.15.111:0/907411101 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f00104f50 msgr2=0x7f5f001053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5f07200640 1 --2- 172.21.15.111:0/907411101 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f00104f50 0x7f5f001053d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5eec009900 tx=0x7f5eec02a410 comp rx=0 tx=0).stop 2024-09-15T23:03:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5f07200640 1 -- 172.21.15.111:0/907411101 shutdown_connections 2024-09-15T23:03:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5f07200640 1 --2- 172.21.15.111:0/907411101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f00105910 0x7f5f0010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5f07200640 1 --2- 172.21.15.111:0/907411101 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f00104f50 0x7f5f001053d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5f07200640 1 --2- 172.21.15.111:0/907411101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f00103d60 0x7f5f00104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5f07200640 1 -- 172.21.15.111:0/907411101 >> 172.21.15.111:0/907411101 conn(0x7f5f000ff4e0 msgr2=0x7f5f00101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:03:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5f07200640 1 -- 172.21.15.111:0/907411101 shutdown_connections 2024-09-15T23:03:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5f07200640 1 -- 172.21.15.111:0/907411101 wait complete. 2024-09-15T23:03:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5f07200640 1 Processor -- start 2024-09-15T23:03:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5f07200640 1 -- start start 2024-09-15T23:03:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5f07200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f00103d60 0x7f5f0019a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5f07200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f00104f50 0x7f5f0019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5f07200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f00105910 0x7f5f001a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5f07200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0010fa30 con 0x7f5f00103d60 2024-09-15T23:03:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5f07200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0010f8b0 con 0x7f5f00105910 2024-09-15T23:03:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5f07200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0010fbb0 con 0x7f5f00104f50 2024-09-15T23:03:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5effe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f00103d60 0x7f5f0019a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5effe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f00103d60 0x7f5f0019a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57160/0 (socket says 172.21.15.111:57160) 2024-09-15T23:03:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5effe00640 1 -- 172.21.15.111:0/1719754098 learned_addr learned my addr 172.21.15.111:0/1719754098 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:03:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5eff400640 1 --2- 172.21.15.111:0/1719754098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f00104f50 0x7f5f0019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5f05200640 1 --2- 172.21.15.111:0/1719754098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f00105910 0x7f5f001a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5effe00640 1 -- 172.21.15.111:0/1719754098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f00104f50 msgr2=0x7f5f0019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5effe00640 1 --2- 172.21.15.111:0/1719754098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f00104f50 0x7f5f0019a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5effe00640 1 -- 172.21.15.111:0/1719754098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f00105910 msgr2=0x7f5f001a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5effe00640 1 --2- 172.21.15.111:0/1719754098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f00105910 0x7f5f001a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5effe00640 1 -- 172.21.15.111:0/1719754098 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f001a1c20 con 0x7f5f00103d60 2024-09-15T23:03:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5effe00640 1 --2- 172.21.15.111:0/1719754098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f00103d60 0x7f5f0019a000 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000e9c0 tx=0x7f5ef000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:03:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.082+0000 7f5efcc00640 1 -- 172.21.15.111:0/1719754098 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef000cd80 con 0x7f5f00103d60 2024-09-15T23:03:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.087+0000 7f5efcc00640 1 -- 172.21.15.111:0/1719754098 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ef00035f0 con 0x7f5f00103d60 2024-09-15T23:03:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.087+0000 7f5efcc00640 1 -- 172.21.15.111:0/1719754098 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0010690 con 0x7f5f00103d60 2024-09-15T23:03:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.087+0000 7f5f07200640 1 -- 172.21.15.111:0/1719754098 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f001a1f10 con 0x7f5f00103d60 2024-09-15T23:03:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.087+0000 7f5f07200640 1 -- 172.21.15.111:0/1719754098 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f001a2320 con 0x7f5f00103d60 2024-09-15T23:03:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.087+0000 7f5f0a034740 1 Processor -- start 2024-09-15T23:03:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.087+0000 7f5f0a034740 1 -- start start 2024-09-15T23:03:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.087+0000 7f5efcc00640 1 -- 172.21.15.111:0/1719754098 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5ef00037a0 con 0x7f5f00103d60 2024-09-15T23:03:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.087+0000 7f5f0a034740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c98512090 0x560c98512490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.087+0000 7f5f0a034740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c985129d0 0x560c98517490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.087+0000 7f5f0a034740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c98517a60 0x560c98519f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.087+0000 7f5f0a034740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560c98477100 con 0x560c985129d0 2024-09-15T23:03:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.087+0000 7f5f0a034740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560c9841f1b0 con 0x560c98517a60 2024-09-15T23:03:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.087+0000 7f5f0a034740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x560c984730c0 con 0x560c98512090 2024-09-15T23:03:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.087+0000 7f5effe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c985129d0 0x560c98517490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.087+0000 7f5effe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c985129d0 0x560c98517490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57174/0 (socket says 172.21.15.111:57174) 2024-09-15T23:03:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.087+0000 7f5effe00640 1 -- 172.21.15.111:0/205556921 learned_addr learned my addr 172.21.15.111:0/205556921 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:03:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.087+0000 7f5eff400640 1 --2- 172.21.15.111:0/205556921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c98517a60 0x560c98519f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.087+0000 7f5f05200640 1 --2- 172.21.15.111:0/205556921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c98512090 0x560c98512490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5eff400640 1 -- 172.21.15.111:0/205556921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c98512090 msgr2=0x560c98512490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5eff400640 1 --2- 172.21.15.111:0/205556921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c98512090 0x560c98512490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5eff400640 1 -- 172.21.15.111:0/205556921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c985129d0 msgr2=0x560c98517490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5eff400640 1 --2- 172.21.15.111:0/205556921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c985129d0 0x560c98517490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5eff400640 1 -- 172.21.15.111:0/205556921 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c9818a210 con 0x560c98517a60 2024-09-15T23:03:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5eff400640 1 --2- 172.21.15.111:0/205556921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c98517a60 0x560c98519f20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5eec02a930 tx=0x7f5eec004190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2954baf04e6388d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:03:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5ef5600640 1 -- 172.21.15.111:0/205556921 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec035040 con 0x560c98517a60 2024-09-15T23:03:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5efcc00640 1 --2- 172.21.15.111:0/1719754098 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5ee40a3370 0x7f5ee40a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5ef5600640 1 -- 172.21.15.111:0/205556921 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eec005520 con 0x560c98517a60 2024-09-15T23:03:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5ef5600640 1 -- 172.21.15.111:0/205556921 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec031910 con 0x560c98517a60 2024-09-15T23:03:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5efcc00640 1 -- 172.21.15.111:0/1719754098 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f5ef00cfb50 con 0x7f5f00103d60 2024-09-15T23:03:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5f05200640 1 --2- 172.21.15.111:0/1719754098 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5ee40a3370 0x7f5ee40a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5f0a034740 1 -- 172.21.15.111:0/205556921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c98517a60 msgr2=0x560c98519f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5f0a034740 1 --2- 172.21.15.111:0/205556921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c98517a60 0x560c98519f20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5eec02a930 tx=0x7f5eec004190 comp rx=0 tx=0).stop 2024-09-15T23:03:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5f0a034740 1 -- 172.21.15.111:0/205556921 shutdown_connections 2024-09-15T23:03:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5f0a034740 1 --2- 172.21.15.111:0/205556921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c98517a60 0x560c98519f20 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5f0a034740 1 --2- 172.21.15.111:0/205556921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c985129d0 0x560c98517490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5f0a034740 1 --2- 172.21.15.111:0/205556921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c98512090 0x560c98512490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.107 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5f0a034740 1 -- 172.21.15.111:0/205556921 >> 172.21.15.111:0/205556921 conn(0x560c9850d700 msgr2=0x560c9850db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:03:13.107 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5f0a034740 1 -- 172.21.15.111:0/205556921 shutdown_connections 2024-09-15T23:03:13.107 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5f05200640 1 --2- 172.21.15.111:0/1719754098 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5ee40a3370 0x7f5ee40a5830 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8005fd0 tx=0x7f5ef80096a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:03:13.107 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5f0a034740 1 -- 172.21.15.111:0/205556921 wait complete. 2024-09-15T23:03:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5f0a034740 1 -- start start 2024-09-15T23:03:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5f0a034740 1 Processor -- start 2024-09-15T23:03:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5f0a034740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c98512090 0x560c9851c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5f0a034740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c985129d0 0x560c9851c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5f0a034740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c98517a60 0x560c98523730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5f0a034740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560c9851a9d0 con 0x560c98517a60 2024-09-15T23:03:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5f0a034740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560c98400090 con 0x560c985129d0 2024-09-15T23:03:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5f0a034740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x560c9851ab50 con 0x560c98512090 2024-09-15T23:03:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5f0a034740 10 client.0 _ll_register_callbacks cb 0x560c98454f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:03:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5f05200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c98517a60 0x560c98523730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5f05200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c98517a60 0x560c98523730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57186/0 (socket says 172.21.15.111:57186) 2024-09-15T23:03:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5f05200640 1 -- 172.21.15.111:0/1548713870 learned_addr learned my addr 172.21.15.111:0/1548713870 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:03:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5effe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c98512090 0x560c9851c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5eff400640 1 --2- 172.21.15.111:0/1548713870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c985129d0 0x560c9851c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5effe00640 1 -- 172.21.15.111:0/1548713870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c985129d0 msgr2=0x560c9851c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5effe00640 1 --2- 172.21.15.111:0/1548713870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c985129d0 0x560c9851c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5effe00640 1 -- 172.21.15.111:0/1548713870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c98517a60 msgr2=0x560c98523730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5effe00640 1 --2- 172.21.15.111:0/1548713870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c98517a60 0x560c98523730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5effe00640 1 -- 172.21.15.111:0/1548713870 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c98256620 con 0x560c98512090 2024-09-15T23:03:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5effe00640 1 --2- 172.21.15.111:0/1548713870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c98512090 0x560c9851c1b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00a4950 tx=0x7f5ef00097a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f06f070350b29a46 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:03:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5eea000640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:03:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5eea000640 1 -- 172.21.15.111:0/1548713870 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef009bb70 con 0x560c98512090 2024-09-15T23:03:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5f0a034740 1 -- 172.21.15.111:0/1548713870 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5eec005520 con 0x560c98512090 2024-09-15T23:03:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5eea000640 1 -- 172.21.15.111:0/1548713870 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f5ef00b5180 con 0x560c98512090 2024-09-15T23:03:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5eea000640 1 -- 172.21.15.111:0/1548713870 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5ef009c910 con 0x560c98512090 2024-09-15T23:03:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5f0a034740 10 client.26338 fetch_fsmap learned FSMap version 35 2024-09-15T23:03:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5f0a034740 1 -- 172.21.15.111:0/1548713870 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560c98526640 con 0x560c98512090 2024-09-15T23:03:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.093+0000 7f5eea000640 1 -- 172.21.15.111:0/1548713870 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eec005520 con 0x560c98512090 2024-09-15T23:03:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.098+0000 7f5eea000640 1 -- 172.21.15.111:0/1548713870 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f5ef00a49f0 con 0x560c98512090 2024-09-15T23:03:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.098+0000 7f5f0a034740 10 client.26338 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:03:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.098+0000 7f5f0a034740 10 client.26338 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:03:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.098+0000 7f5f0a034740 20 client.26338 populate_metadata read hostname 'smithi111' 2024-09-15T23:03:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.098+0000 7f5f0a034740 1 --2- 172.21.15.111:0/1548713870 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560c98527cc0 0x560c9852a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.098+0000 7f5f0a034740 4 client.26338 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:03:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.098+0000 7f5f0a034740 1 -- 172.21.15.111:0/1548713870 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c98484830 con 0x560c98527cc0 2024-09-15T23:03:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.098+0000 7f5eff400640 1 --2- 172.21.15.111:0/1548713870 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560c98527cc0 0x560c9852a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.098+0000 7f5eff400640 1 --2- 172.21.15.111:0/1548713870 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560c98527cc0 0x560c9852a0b0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a073802ffbfe0e00 server_cookie=6747050fb2201729 in_seq=0 out_seq=0 2024-09-15T23:03:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.098+0000 7f5eea000640 10 client.26338 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:03:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.099+0000 7f5eea000640 1 -- 172.21.15.111:0/1548713870 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560c98484830 con 0x560c98527cc0 2024-09-15T23:03:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.099+0000 7f5eea000640 10 client.26338 handle_command_reply: tid=0 2024-09-15T23:03:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.099+0000 7f5f0a034740 1 client.26338 shutdown 2024-09-15T23:03:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.099+0000 7f5f0a034740 1 -- 172.21.15.111:0/1548713870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c98512090 msgr2=0x560c9851c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.099+0000 7f5f0a034740 1 --2- 172.21.15.111:0/1548713870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c98512090 0x560c9851c1b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00a4950 tx=0x7f5ef00097a0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.099+0000 7f5f0a034740 1 -- 172.21.15.111:0/1548713870 shutdown_connections 2024-09-15T23:03:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.099+0000 7f5f0a034740 1 --2- 172.21.15.111:0/1548713870 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560c98527cc0 0x560c9852a0b0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.099+0000 7f5f0a034740 1 --2- 172.21.15.111:0/1548713870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c98517a60 0x560c98523730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.099+0000 7f5eea000640 0 client.26338 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:03:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.099+0000 7f5f0a034740 1 --2- 172.21.15.111:0/1548713870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c985129d0 0x560c9851c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.099+0000 7f5f0a034740 1 --2- 172.21.15.111:0/1548713870 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c98512090 0x560c9851c1b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.099+0000 7f5f0a034740 1 -- 172.21.15.111:0/1548713870 >> 172.21.15.111:0/1548713870 conn(0x560c9850d700 msgr2=0x560c9850ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:03:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.099+0000 7f5f0a034740 1 -- 172.21.15.111:0/1548713870 shutdown_connections 2024-09-15T23:03:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.099+0000 7f5f0a034740 1 -- 172.21.15.111:0/1548713870 wait complete. 2024-09-15T23:03:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.099+0000 7f5f0a034740 20 client.26338 trim_cache size 0 max 16384 2024-09-15T23:03:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.111+0000 7f5f0a034740 1 Processor -- start 2024-09-15T23:03:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.111+0000 7f5f0a034740 1 -- start start 2024-09-15T23:03:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.111+0000 7f5f0a034740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c98512090 0x560c9851c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.111+0000 7f5f0a034740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c985129d0 0x560c985236f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.111+0000 7f5f0a034740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c98517a60 0x560c9851c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.111+0000 7f5f0a034740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5f001ad8d0 con 0x560c985129d0 2024-09-15T23:03:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.111+0000 7f5f0a034740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5f001ab1a0 con 0x560c98512090 2024-09-15T23:03:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.111+0000 7f5f0a034740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5f001a8ff0 con 0x560c98517a60 2024-09-15T23:03:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.111+0000 7f5eff400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c985129d0 0x560c985236f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.111+0000 7f5eff400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c985129d0 0x560c985236f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57190/0 (socket says 172.21.15.111:57190) 2024-09-15T23:03:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.111+0000 7f5eff400640 1 -- 172.21.15.111:0/1858015640 learned_addr learned my addr 172.21.15.111:0/1858015640 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:03:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.111+0000 7f5effe00640 1 --2- 172.21.15.111:0/1858015640 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c98512090 0x560c9851c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.111+0000 7f5f05200640 1 --2- 172.21.15.111:0/1858015640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c98517a60 0x560c9851c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.111+0000 7f5effe00640 1 -- 172.21.15.111:0/1858015640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c98517a60 msgr2=0x560c9851c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.111+0000 7f5effe00640 1 --2- 172.21.15.111:0/1858015640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c98517a60 0x560c9851c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.111+0000 7f5effe00640 1 -- 172.21.15.111:0/1858015640 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c985129d0 msgr2=0x560c985236f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.111+0000 7f5effe00640 1 --2- 172.21.15.111:0/1858015640 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c985129d0 0x560c985236f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.111+0000 7f5effe00640 1 -- 172.21.15.111:0/1858015640 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c98526460 con 0x560c98512090 2024-09-15T23:03:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.111+0000 7f5eff400640 1 --2- 172.21.15.111:0/1858015640 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c985129d0 0x560c985236f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:03:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5effe00640 1 --2- 172.21.15.111:0/1858015640 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c98512090 0x560c9851c170 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0098670 tx=0x7f5ef00d31f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69ae1e0da5b74d0c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:03:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5ef4c00640 1 -- 172.21.15.111:0/1858015640 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef009bb70 con 0x560c98512090 2024-09-15T23:03:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5ef4c00640 1 -- 172.21.15.111:0/1858015640 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ef00bae50 con 0x560c98512090 2024-09-15T23:03:13.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5ef4c00640 1 -- 172.21.15.111:0/1858015640 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef009bfe0 con 0x560c98512090 2024-09-15T23:03:13.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f0a034740 1 -- 172.21.15.111:0/1858015640 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c98512090 msgr2=0x560c9851c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:13.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f0a034740 1 --2- 172.21.15.111:0/1858015640 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c98512090 0x560c9851c170 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0098670 tx=0x7f5ef00d31f0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f0a034740 1 -- 172.21.15.111:0/1858015640 shutdown_connections 2024-09-15T23:03:13.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f0a034740 1 --2- 172.21.15.111:0/1858015640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c98517a60 0x560c9851c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f0a034740 1 --2- 172.21.15.111:0/1858015640 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c985129d0 0x560c985236f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f0a034740 1 --2- 172.21.15.111:0/1858015640 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c98512090 0x560c9851c170 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f0a034740 1 -- 172.21.15.111:0/1858015640 >> 172.21.15.111:0/1858015640 conn(0x560c9850d700 msgr2=0x560c98513b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:03:13.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f0a034740 1 -- 172.21.15.111:0/1858015640 shutdown_connections 2024-09-15T23:03:13.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f0a034740 1 -- 172.21.15.111:0/1858015640 wait complete. 2024-09-15T23:03:13.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f0a034740 1 -- start start 2024-09-15T23:03:13.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f0a034740 1 Processor -- start 2024-09-15T23:03:13.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f0a034740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c98512090 0x560c985206f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:13.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f0a034740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c985129d0 0x560c98520c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:13.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f0a034740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c98517a60 0x560c98590950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:13.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f0a034740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5f001a9660 con 0x560c985129d0 2024-09-15T23:03:13.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f0a034740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5f001aaad0 con 0x560c98512090 2024-09-15T23:03:13.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f0a034740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5f001a9d30 con 0x560c98517a60 2024-09-15T23:03:13.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f0a034740 10 client.0 _ll_register_callbacks cb 0x7f5ef0003e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:03:13.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5effe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c98512090 0x560c985206f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:13.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5eff400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c985129d0 0x560c98520c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:13.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5effe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c98512090 0x560c985206f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:57888/0 (socket says 172.21.15.111:57888) 2024-09-15T23:03:13.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5effe00640 1 -- 172.21.15.111:0/1129295227 learned_addr learned my addr 172.21.15.111:0/1129295227 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:03:13.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f05200640 1 --2- 172.21.15.111:0/1129295227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c98517a60 0x560c98590950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:13.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5effe00640 1 -- 172.21.15.111:0/1129295227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c98517a60 msgr2=0x560c98590950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:13.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5effe00640 1 --2- 172.21.15.111:0/1129295227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c98517a60 0x560c98590950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5effe00640 1 -- 172.21.15.111:0/1129295227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c985129d0 msgr2=0x560c98520c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:13.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5effe00640 1 --2- 172.21.15.111:0/1129295227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c985129d0 0x560c98520c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5effe00640 1 -- 172.21.15.111:0/1129295227 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c98590e90 con 0x560c98512090 2024-09-15T23:03:13.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f05200640 1 --2- 172.21.15.111:0/1129295227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c98517a60 0x560c98590950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:03:13.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5effe00640 1 --2- 172.21.15.111:0/1129295227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c98512090 0x560c985206f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00058c0 tx=0x7f5ef0003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a124dbd9f12a8bda server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:03:13.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5eea000640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:03:13.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5eea000640 1 -- 172.21.15.111:0/1129295227 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef009bfe0 con 0x560c98512090 2024-09-15T23:03:13.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5eea000640 1 -- 172.21.15.111:0/1129295227 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f5ef00bc0d0 con 0x560c98512090 2024-09-15T23:03:13.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f0a034740 1 -- 172.21.15.111:0/1129295227 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ef00bae50 con 0x560c98512090 2024-09-15T23:03:13.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5eea000640 1 -- 172.21.15.111:0/1129295227 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5ef009bb70 con 0x560c98512090 2024-09-15T23:03:13.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f0a034740 10 client.26327 fetch_fsmap learned FSMap version 35 2024-09-15T23:03:13.130 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f0a034740 1 -- 172.21.15.111:0/1129295227 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560c985921a0 con 0x560c98512090 2024-09-15T23:03:13.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5eea000640 1 -- 172.21.15.111:0/1129295227 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ef00bae50 con 0x560c98512090 2024-09-15T23:03:13.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5eea000640 1 -- 172.21.15.111:0/1129295227 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f5ef0004190 con 0x560c98512090 2024-09-15T23:03:13.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f0a034740 10 client.26327 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:03:13.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f0a034740 10 client.26327 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:03:13.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f0a034740 20 client.26327 populate_metadata read hostname 'smithi111' 2024-09-15T23:03:13.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f0a034740 1 --2- 172.21.15.111:0/1129295227 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560c985928b0 0x560c98594cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:13.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f0a034740 4 client.26327 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:03:13.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5f0a034740 1 -- 172.21.15.111:0/1129295227 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560c9847eb80 con 0x560c985928b0 2024-09-15T23:03:13.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5eff400640 1 --2- 172.21.15.111:0/1129295227 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560c985928b0 0x560c98594cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:13.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5eff400640 1 --2- 172.21.15.111:0/1129295227 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560c985928b0 0x560c98594cd0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dda4f29af9d6e5a5 server_cookie=ae284910f50b91bf in_seq=0 out_seq=0 2024-09-15T23:03:13.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.117+0000 7f5eea000640 10 client.26327 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:03:13.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.190+0000 7f5eea000640 1 -- 172.21.15.111:0/1129295227 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560c9847eb80 con 0x560c985928b0 2024-09-15T23:03:13.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.190+0000 7f5eea000640 10 client.26327 handle_command_reply: tid=0 2024-09-15T23:03:13.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.190+0000 7f5f0a034740 1 client.26327 shutdown 2024-09-15T23:03:13.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.190+0000 7f5f0a034740 1 -- 172.21.15.111:0/1129295227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c98512090 msgr2=0x560c985206f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:13.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.190+0000 7f5f0a034740 1 --2- 172.21.15.111:0/1129295227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c98512090 0x560c985206f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00058c0 tx=0x7f5ef0003f10 comp rx=0 tx=0).stop 2024-09-15T23:03:13.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.191+0000 7f5f0a034740 1 -- 172.21.15.111:0/1129295227 shutdown_connections 2024-09-15T23:03:13.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.191+0000 7f5f0a034740 1 --2- 172.21.15.111:0/1129295227 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560c985928b0 0x560c98594cd0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.191+0000 7f5f0a034740 1 --2- 172.21.15.111:0/1129295227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560c98517a60 0x560c98590950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.191+0000 7f5f0a034740 1 --2- 172.21.15.111:0/1129295227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560c985129d0 0x560c98520c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.191+0000 7f5eea000640 0 client.26327 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:03:13.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.191+0000 7f5f0a034740 1 --2- 172.21.15.111:0/1129295227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560c98512090 0x560c985206f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.191+0000 7f5f0a034740 1 -- 172.21.15.111:0/1129295227 >> 172.21.15.111:0/1129295227 conn(0x560c9850d700 msgr2=0x560c9851bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:03:13.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.191+0000 7f5f0a034740 1 -- 172.21.15.111:0/1129295227 shutdown_connections 2024-09-15T23:03:13.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.191+0000 7f5f0a034740 1 -- 172.21.15.111:0/1129295227 wait complete. 2024-09-15T23:03:13.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.191+0000 7f5f0a034740 20 client.26327 trim_cache size 0 max 16384 2024-09-15T23:03:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:03:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:03:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:03:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:03:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.191+0000 7f5f07200640 1 -- 172.21.15.111:0/1719754098 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5ee40a3370 msgr2=0x7f5ee40a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.191+0000 7f5f07200640 1 --2- 172.21.15.111:0/1719754098 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5ee40a3370 0x7f5ee40a5830 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8005fd0 tx=0x7f5ef80096a0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.192+0000 7f5f07200640 1 -- 172.21.15.111:0/1719754098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f00103d60 msgr2=0x7f5f0019a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.192+0000 7f5f07200640 1 --2- 172.21.15.111:0/1719754098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f00103d60 0x7f5f0019a000 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000e9c0 tx=0x7f5ef000ee80 comp rx=0 tx=0).stop 2024-09-15T23:03:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.192+0000 7f5f07200640 1 -- 172.21.15.111:0/1719754098 shutdown_connections 2024-09-15T23:03:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.192+0000 7f5f07200640 1 --2- 172.21.15.111:0/1719754098 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5ee40a3370 0x7f5ee40a5830 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.192+0000 7f5f07200640 1 --2- 172.21.15.111:0/1719754098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5f00105910 0x7f5f001a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.192+0000 7f5f07200640 1 --2- 172.21.15.111:0/1719754098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f00104f50 0x7f5f0019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.192+0000 7f5f07200640 1 --2- 172.21.15.111:0/1719754098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f00103d60 0x7f5f0019a000 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.192+0000 7f5f07200640 1 -- 172.21.15.111:0/1719754098 >> 172.21.15.111:0/1719754098 conn(0x7f5f000ff4e0 msgr2=0x7f5f00108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:03:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.192+0000 7f5f07200640 1 -- 172.21.15.111:0/1719754098 shutdown_connections 2024-09-15T23:03:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:13.192+0000 7f5f07200640 1 -- 172.21.15.111:0/1719754098 wait complete. 2024-09-15T23:03:13.642 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73462 13.42 MB/sec execute 101 sec latency 18440.363 ms 2024-09-15T23:03:14.082 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:13 smithi111 ceph-mon[30629]: pgmap v870: 97 pgs: 97 active+clean; 524 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 5.6 MiB/s wr, 218 op/s 2024-09-15T23:03:14.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:13 smithi192 ceph-mon[35427]: pgmap v870: 97 pgs: 97 active+clean; 524 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 5.6 MiB/s wr, 218 op/s 2024-09-15T23:03:14.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:13 smithi121 ceph-mon[34815]: pgmap v870: 97 pgs: 97 active+clean; 524 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 5.6 MiB/s wr, 218 op/s 2024-09-15T23:03:14.640 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73462 13.29 MB/sec execute 102 sec latency 19440.447 ms 2024-09-15T23:03:15.641 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73462 13.16 MB/sec execute 103 sec latency 20440.535 ms 2024-09-15T23:03:16.137 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:15 smithi111 ceph-mon[30629]: pgmap v871: 97 pgs: 97 active+clean; 528 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 5.2 MiB/s wr, 214 op/s 2024-09-15T23:03:16.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:15 smithi192 ceph-mon[35427]: pgmap v871: 97 pgs: 97 active+clean; 528 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 5.2 MiB/s wr, 214 op/s 2024-09-15T23:03:16.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:15 smithi121 ceph-mon[34815]: pgmap v871: 97 pgs: 97 active+clean; 528 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 5.2 MiB/s wr, 214 op/s 2024-09-15T23:03:16.641 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73466 13.03 MB/sec execute 104 sec latency 21362.195 ms 2024-09-15T23:03:17.641 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73545 12.98 MB/sec execute 105 sec latency 15640.220 ms 2024-09-15T23:03:18.078 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:17 smithi111 ceph-mon[30629]: pgmap v872: 97 pgs: 97 active+clean; 529 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 5.3 MiB/s wr, 193 op/s 2024-09-15T23:03:18.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:17 smithi192 ceph-mon[35427]: pgmap v872: 97 pgs: 97 active+clean; 529 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 5.3 MiB/s wr, 193 op/s 2024-09-15T23:03:18.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:17 smithi121 ceph-mon[34815]: pgmap v872: 97 pgs: 97 active+clean; 529 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 5.3 MiB/s wr, 193 op/s 2024-09-15T23:03:18.641 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73545 12.85 MB/sec execute 106 sec latency 16640.289 ms 2024-09-15T23:03:19.641 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73545 12.73 MB/sec execute 107 sec latency 17640.379 ms 2024-09-15T23:03:20.098 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:19 smithi111 ceph-mon[30629]: pgmap v873: 97 pgs: 97 active+clean; 502 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 4.3 MiB/s wr, 183 op/s 2024-09-15T23:03:20.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:19 smithi192 ceph-mon[35427]: pgmap v873: 97 pgs: 97 active+clean; 502 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 4.3 MiB/s wr, 183 op/s 2024-09-15T23:03:20.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:19 smithi121 ceph-mon[34815]: pgmap v873: 97 pgs: 97 active+clean; 502 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 4.3 MiB/s wr, 183 op/s 2024-09-15T23:03:20.641 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73545 12.62 MB/sec execute 108 sec latency 18640.424 ms 2024-09-15T23:03:21.641 INFO:tasks.workunit.client.0.smithi111.stdout: 10 73996 12.63 MB/sec execute 109 sec latency 19078.424 ms 2024-09-15T23:03:22.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:21 smithi192 ceph-mon[35427]: pgmap v874: 97 pgs: 97 active+clean; 485 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 2.9 MiB/s wr, 142 op/s 2024-09-15T23:03:22.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:21 smithi121 ceph-mon[34815]: pgmap v874: 97 pgs: 97 active+clean; 485 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 2.9 MiB/s wr, 142 op/s 2024-09-15T23:03:22.406 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:21 smithi111 ceph-mon[30629]: pgmap v874: 97 pgs: 97 active+clean; 485 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 2.9 MiB/s wr, 142 op/s 2024-09-15T23:03:22.641 INFO:tasks.workunit.client.0.smithi111.stdout: 10 74911 13.03 MB/sec execute 110 sec latency 63.015 ms 2024-09-15T23:03:23.641 INFO:tasks.workunit.client.0.smithi111.stdout: 10 75695 13.31 MB/sec execute 111 sec latency 84.437 ms 2024-09-15T23:03:24.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:23 smithi192 ceph-mon[35427]: pgmap v875: 97 pgs: 97 active+clean; 474 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 130 op/s 2024-09-15T23:03:24.345 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:23 smithi111 ceph-mon[30629]: pgmap v875: 97 pgs: 97 active+clean; 474 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 130 op/s 2024-09-15T23:03:24.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:23 smithi121 ceph-mon[34815]: pgmap v875: 97 pgs: 97 active+clean; 474 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.3 MiB/s wr, 130 op/s 2024-09-15T23:03:24.641 INFO:tasks.workunit.client.0.smithi111.stdout: 10 76411 13.57 MB/sec execute 112 sec latency 48.120 ms 2024-09-15T23:03:25.641 INFO:tasks.workunit.client.0.smithi111.stdout: 10 77025 13.73 MB/sec execute 113 sec latency 41.950 ms 2024-09-15T23:03:26.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:25 smithi111 ceph-mon[30629]: pgmap v876: 97 pgs: 97 active+clean; 478 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 186 op/s 2024-09-15T23:03:26.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:25 smithi192 ceph-mon[35427]: pgmap v876: 97 pgs: 97 active+clean; 478 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 186 op/s 2024-09-15T23:03:26.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.347+0000 7f808c800640 1 -- 172.21.15.111:0/3505478098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8084105900 msgr2=0x7f808410c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:26.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.347+0000 7f808c800640 1 --2- 172.21.15.111:0/3505478098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8084105900 0x7f808410c190 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f8080009be0 tx=0x7f808002a6d0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.347+0000 7f808c800640 1 -- 172.21.15.111:0/3505478098 shutdown_connections 2024-09-15T23:03:26.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.347+0000 7f808c800640 1 --2- 172.21.15.111:0/3505478098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8084105900 0x7f808410c190 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.347+0000 7f808c800640 1 --2- 172.21.15.111:0/3505478098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8084104f40 0x7f80841053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.347+0000 7f808c800640 1 --2- 172.21.15.111:0/3505478098 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8084103d50 0x7f8084104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.347+0000 7f808c800640 1 -- 172.21.15.111:0/3505478098 >> 172.21.15.111:0/3505478098 conn(0x7f80840ff4b0 msgr2=0x7f80841018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:03:26.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.347+0000 7f808c800640 1 -- 172.21.15.111:0/3505478098 shutdown_connections 2024-09-15T23:03:26.350 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.347+0000 7f808c800640 1 -- 172.21.15.111:0/3505478098 wait complete. 2024-09-15T23:03:26.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.349+0000 7f808c800640 1 Processor -- start 2024-09-15T23:03:26.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.349+0000 7f808c800640 1 -- start start 2024-09-15T23:03:26.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.349+0000 7f808c800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8084103d50 0x7f808419e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:26.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.349+0000 7f808c800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8084104f40 0x7f808419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:26.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.349+0000 7f808c800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8084105900 0x7f80841a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:26.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.349+0000 7f808c800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f808410f9b0 con 0x7f8084103d50 2024-09-15T23:03:26.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.349+0000 7f808c800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f808410f830 con 0x7f8084105900 2024-09-15T23:03:26.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.349+0000 7f808c800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f808410fb30 con 0x7f8084104f40 2024-09-15T23:03:26.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.349+0000 7f8089400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8084104f40 0x7f808419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:26.351 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.349+0000 7f8089400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8084104f40 0x7f808419e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:54388/0 (socket says 172.21.15.111:54388) 2024-09-15T23:03:26.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.349+0000 7f8089400640 1 -- 172.21.15.111:0/805872699 learned_addr learned my addr 172.21.15.111:0/805872699 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:03:26.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.349+0000 7f8089e00640 1 --2- 172.21.15.111:0/805872699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8084103d50 0x7f808419e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:26.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.350+0000 7f808a800640 1 --2- 172.21.15.111:0/805872699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8084105900 0x7f80841a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.350+0000 7f8089400640 1 -- 172.21.15.111:0/805872699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8084105900 msgr2=0x7f80841a5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.350+0000 7f8089400640 1 --2- 172.21.15.111:0/805872699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8084105900 0x7f80841a5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.350+0000 7f8089400640 1 -- 172.21.15.111:0/805872699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8084103d50 msgr2=0x7f808419e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.350+0000 7f8089400640 1 --2- 172.21.15.111:0/805872699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8084103d50 0x7f808419e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.350+0000 7f8089400640 1 -- 172.21.15.111:0/805872699 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80841a5f70 con 0x7f8084104f40 2024-09-15T23:03:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.350+0000 7f8089400640 1 --2- 172.21.15.111:0/805872699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8084104f40 0x7f808419e890 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f807400e9c0 tx=0x7f807400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:03:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.351+0000 7f807ea00640 1 -- 172.21.15.111:0/805872699 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807400cde0 con 0x7f8084104f40 2024-09-15T23:03:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.351+0000 7f807ea00640 1 -- 172.21.15.111:0/805872699 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80740035f0 con 0x7f8084104f40 2024-09-15T23:03:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.351+0000 7f807ea00640 1 -- 172.21.15.111:0/805872699 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8074010690 con 0x7f8084104f40 2024-09-15T23:03:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.351+0000 7f808c800640 1 -- 172.21.15.111:0/805872699 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80841a6260 con 0x7f8084104f40 2024-09-15T23:03:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.351+0000 7f808c800640 1 -- 172.21.15.111:0/805872699 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80841a6760 con 0x7f8084104f40 2024-09-15T23:03:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.356+0000 7f807ea00640 1 -- 172.21.15.111:0/805872699 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f80740037a0 con 0x7f8084104f40 2024-09-15T23:03:26.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.356+0000 7f808f7d5740 1 Processor -- start 2024-09-15T23:03:26.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.357+0000 7f808f7d5740 1 -- start start 2024-09-15T23:03:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.357+0000 7f808f7d5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f73845090 0x561f73845490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.357+0000 7f808f7d5740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f738459d0 0x561f7384a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.357+0000 7f808f7d5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561f7384aa60 0x561f7384cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.357+0000 7f808f7d5740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561f737aa100 con 0x561f738459d0 2024-09-15T23:03:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.357+0000 7f808f7d5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561f737521b0 con 0x561f7384aa60 2024-09-15T23:03:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.357+0000 7f808f7d5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561f737a60c0 con 0x561f73845090 2024-09-15T23:03:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.357+0000 7f808a800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f73845090 0x561f73845490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.357+0000 7f808a800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f73845090 0x561f73845490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:54396/0 (socket says 172.21.15.111:54396) 2024-09-15T23:03:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.357+0000 7f808a800640 1 -- 172.21.15.111:0/3635764470 learned_addr learned my addr 172.21.15.111:0/3635764470 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:03:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.358+0000 7f8089400640 1 --2- 172.21.15.111:0/3635764470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561f7384aa60 0x561f7384cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.358+0000 7f8089e00640 1 --2- 172.21.15.111:0/3635764470 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f738459d0 0x561f7384a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.358+0000 7f808a800640 1 -- 172.21.15.111:0/3635764470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561f7384aa60 msgr2=0x561f7384cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.358+0000 7f808a800640 1 --2- 172.21.15.111:0/3635764470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561f7384aa60 0x561f7384cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.358+0000 7f807ea00640 1 --2- 172.21.15.111:0/805872699 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f806c0a3370 0x7f806c0a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.358+0000 7f808a800640 1 -- 172.21.15.111:0/3635764470 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f738459d0 msgr2=0x561f7384a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:26.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.358+0000 7f808a800640 1 --2- 172.21.15.111:0/3635764470 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f738459d0 0x561f7384a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.358+0000 7f808a800640 1 -- 172.21.15.111:0/3635764470 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f734bd210 con 0x561f73845090 2024-09-15T23:03:26.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.358+0000 7f8089400640 1 --2- 172.21.15.111:0/3635764470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561f7384aa60 0x561f7384cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:03:26.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.358+0000 7f8089e00640 1 --2- 172.21.15.111:0/3635764470 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f738459d0 0x561f7384a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:03:26.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.358+0000 7f807ea00640 1 -- 172.21.15.111:0/805872699 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f80740cfc20 con 0x7f8084104f40 2024-09-15T23:03:26.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.358+0000 7f808a800640 1 --2- 172.21.15.111:0/805872699 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f806c0a3370 0x7f806c0a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:26.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.359+0000 7f808a800640 1 --2- 172.21.15.111:0/3635764470 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f73845090 0x561f73845490 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8080006010 tx=0x7f8080009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=525035cad6b7c8b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:03:26.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.359+0000 7f8073000640 1 -- 172.21.15.111:0/3635764470 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8080036040 con 0x561f73845090 2024-09-15T23:03:26.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.359+0000 7f8073000640 1 -- 172.21.15.111:0/3635764470 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8080003b30 con 0x561f73845090 2024-09-15T23:03:26.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.359+0000 7f8073000640 1 -- 172.21.15.111:0/3635764470 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80800315d0 con 0x561f73845090 2024-09-15T23:03:26.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.360+0000 7f808f7d5740 1 -- 172.21.15.111:0/3635764470 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f73845090 msgr2=0x561f73845490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:26.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.360+0000 7f808a800640 1 --2- 172.21.15.111:0/805872699 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f806c0a3370 0x7f806c0a5830 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8080004af0 tx=0x7f808004b000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:03:26.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.360+0000 7f808f7d5740 1 --2- 172.21.15.111:0/3635764470 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f73845090 0x561f73845490 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8080006010 tx=0x7f8080009310 comp rx=0 tx=0).stop 2024-09-15T23:03:26.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.360+0000 7f808f7d5740 1 -- 172.21.15.111:0/3635764470 shutdown_connections 2024-09-15T23:03:26.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.360+0000 7f808f7d5740 1 --2- 172.21.15.111:0/3635764470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561f7384aa60 0x561f7384cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.360+0000 7f808f7d5740 1 --2- 172.21.15.111:0/3635764470 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f738459d0 0x561f7384a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.360+0000 7f808f7d5740 1 --2- 172.21.15.111:0/3635764470 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f73845090 0x561f73845490 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.360+0000 7f808f7d5740 1 -- 172.21.15.111:0/3635764470 >> 172.21.15.111:0/3635764470 conn(0x561f73840700 msgr2=0x561f73840b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:03:26.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.360+0000 7f808f7d5740 1 -- 172.21.15.111:0/3635764470 shutdown_connections 2024-09-15T23:03:26.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.361+0000 7f808f7d5740 1 -- 172.21.15.111:0/3635764470 wait complete. 2024-09-15T23:03:26.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.361+0000 7f808f7d5740 1 -- start start 2024-09-15T23:03:26.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.361+0000 7f808f7d5740 1 Processor -- start 2024-09-15T23:03:26.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.362+0000 7f808f7d5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f73845090 0x561f7384f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:26.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.362+0000 7f808f7d5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561f738459d0 0x561f7384f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:26.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.362+0000 7f808f7d5740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f7384aa60 0x561f738567a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:26.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.362+0000 7f808f7d5740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561f7384da30 con 0x561f7384aa60 2024-09-15T23:03:26.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.362+0000 7f808f7d5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561f73733090 con 0x561f738459d0 2024-09-15T23:03:26.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.362+0000 7f808f7d5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561f7384dbb0 con 0x561f73845090 2024-09-15T23:03:26.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.362+0000 7f808f7d5740 10 client.0 _ll_register_callbacks cb 0x561f73787f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:03:26.366 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.362+0000 7f8089e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f73845090 0x561f7384f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:26.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.362+0000 7f8089e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f73845090 0x561f7384f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:54408/0 (socket says 172.21.15.111:54408) 2024-09-15T23:03:26.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.362+0000 7f8089e00640 1 -- 172.21.15.111:0/1227740103 learned_addr learned my addr 172.21.15.111:0/1227740103 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:03:26.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.362+0000 7f8089400640 1 --2- 172.21.15.111:0/1227740103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561f738459d0 0x561f7384f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:26.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.363+0000 7f808a800640 1 --2- 172.21.15.111:0/1227740103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f7384aa60 0x561f738567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:26.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.363+0000 7f8089e00640 1 -- 172.21.15.111:0/1227740103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561f738459d0 msgr2=0x561f7384f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:26.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.363+0000 7f8089e00640 1 --2- 172.21.15.111:0/1227740103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561f738459d0 0x561f7384f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.363+0000 7f8089e00640 1 -- 172.21.15.111:0/1227740103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f7384aa60 msgr2=0x561f738567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:26.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.363+0000 7f8089e00640 1 --2- 172.21.15.111:0/1227740103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f7384aa60 0x561f738567a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.363+0000 7f8089e00640 1 -- 172.21.15.111:0/1227740103 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f73589620 con 0x561f73845090 2024-09-15T23:03:26.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.363+0000 7f8089e00640 1 --2- 172.21.15.111:0/1227740103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f73845090 0x561f7384f220 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8078013790 tx=0x7f8078013b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85d9cb0123d88621 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:03:26.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.363+0000 7f8067400640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:03:26.367 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.364+0000 7f8067400640 1 -- 172.21.15.111:0/1227740103 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8078020060 con 0x561f73845090 2024-09-15T23:03:26.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.364+0000 7f8067400640 1 -- 172.21.15.111:0/1227740103 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f8078011dd0 con 0x561f73845090 2024-09-15T23:03:26.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.364+0000 7f808f7d5740 1 -- 172.21.15.111:0/1227740103 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8080003b30 con 0x561f73845090 2024-09-15T23:03:26.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.364+0000 7f8067400640 1 -- 172.21.15.111:0/1227740103 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f807800f450 con 0x561f73845090 2024-09-15T23:03:26.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.364+0000 7f808f7d5740 10 client.26356 fetch_fsmap learned FSMap version 35 2024-09-15T23:03:26.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.364+0000 7f808f7d5740 1 -- 172.21.15.111:0/1227740103 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561f73859660 con 0x561f73845090 2024-09-15T23:03:26.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.364+0000 7f8067400640 1 -- 172.21.15.111:0/1227740103 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8080003b30 con 0x561f73845090 2024-09-15T23:03:26.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.364+0000 7f8067400640 1 -- 172.21.15.111:0/1227740103 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f807801a030 con 0x561f73845090 2024-09-15T23:03:26.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.365+0000 7f808f7d5740 10 client.26356 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:03:26.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.365+0000 7f808f7d5740 10 client.26356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:03:26.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.365+0000 7f808f7d5740 20 client.26356 populate_metadata read hostname 'smithi111' 2024-09-15T23:03:26.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.365+0000 7f808f7d5740 1 --2- 172.21.15.111:0/1227740103 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561f7385ace0 0x561f7385d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:26.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.365+0000 7f808f7d5740 4 client.26356 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:03:26.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.365+0000 7f808f7d5740 1 -- 172.21.15.111:0/1227740103 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f737b7830 con 0x561f7385ace0 2024-09-15T23:03:26.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.365+0000 7f8089400640 1 --2- 172.21.15.111:0/1227740103 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561f7385ace0 0x561f7385d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:26.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.366+0000 7f8089400640 1 --2- 172.21.15.111:0/1227740103 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561f7385ace0 0x561f7385d110 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7f2d642cfd9b6c7 server_cookie=52db2e611c1866a0 in_seq=0 out_seq=0 2024-09-15T23:03:26.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.366+0000 7f8067400640 10 client.26356 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:03:26.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.367+0000 7f8067400640 1 -- 172.21.15.111:0/1227740103 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561f737b7830 con 0x561f7385ace0 2024-09-15T23:03:26.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.367+0000 7f8067400640 10 client.26356 handle_command_reply: tid=0 2024-09-15T23:03:26.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.367+0000 7f808f7d5740 1 client.26356 shutdown 2024-09-15T23:03:26.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.367+0000 7f808f7d5740 1 -- 172.21.15.111:0/1227740103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f73845090 msgr2=0x561f7384f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:26.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.367+0000 7f808f7d5740 1 --2- 172.21.15.111:0/1227740103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f73845090 0x561f7384f220 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8078013790 tx=0x7f8078013b90 comp rx=0 tx=0).stop 2024-09-15T23:03:26.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.367+0000 7f808f7d5740 1 -- 172.21.15.111:0/1227740103 shutdown_connections 2024-09-15T23:03:26.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.367+0000 7f808f7d5740 1 --2- 172.21.15.111:0/1227740103 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561f7385ace0 0x561f7385d110 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.367+0000 7f808f7d5740 1 --2- 172.21.15.111:0/1227740103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f7384aa60 0x561f738567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.367+0000 7f808f7d5740 1 --2- 172.21.15.111:0/1227740103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561f738459d0 0x561f7384f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.367+0000 7f808f7d5740 1 --2- 172.21.15.111:0/1227740103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f73845090 0x561f7384f220 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.367+0000 7f808f7d5740 1 -- 172.21.15.111:0/1227740103 >> 172.21.15.111:0/1227740103 conn(0x561f73840700 msgr2=0x561f73841f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:03:26.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.367+0000 7f8067400640 0 client.26356 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:03:26.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.368+0000 7f808f7d5740 1 -- 172.21.15.111:0/1227740103 shutdown_connections 2024-09-15T23:03:26.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.368+0000 7f808f7d5740 1 -- 172.21.15.111:0/1227740103 wait complete. 2024-09-15T23:03:26.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.368+0000 7f808f7d5740 20 client.26356 trim_cache size 0 max 16384 2024-09-15T23:03:26.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.385+0000 7f808f7d5740 1 Processor -- start 2024-09-15T23:03:26.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.385+0000 7f808f7d5740 1 -- start start 2024-09-15T23:03:26.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.385+0000 7f808f7d5740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f73845090 0x561f7384f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:26.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.385+0000 7f808f7d5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561f738459d0 0x561f738566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:26.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.385+0000 7f808f7d5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f7384aa60 0x561f73841af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:26.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.385+0000 7f808f7d5740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f80841b1d30 con 0x561f73845090 2024-09-15T23:03:26.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.385+0000 7f808f7d5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f80841af600 con 0x561f738459d0 2024-09-15T23:03:26.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.385+0000 7f808f7d5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f80841ad450 con 0x561f7384aa60 2024-09-15T23:03:26.388 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.386+0000 7f8089400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561f738459d0 0x561f738566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.386+0000 7f8089400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561f738459d0 0x561f738566c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:36604/0 (socket says 172.21.15.111:36604) 2024-09-15T23:03:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.386+0000 7f8089400640 1 -- 172.21.15.111:0/2885086214 learned_addr learned my addr 172.21.15.111:0/2885086214 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:03:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.386+0000 7f808a800640 1 --2- 172.21.15.111:0/2885086214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f7384aa60 0x561f73841af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.386+0000 7f8089e00640 1 --2- 172.21.15.111:0/2885086214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f73845090 0x561f7384f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.386+0000 7f8089400640 1 -- 172.21.15.111:0/2885086214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f7384aa60 msgr2=0x561f73841af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.386+0000 7f8089400640 1 --2- 172.21.15.111:0/2885086214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f7384aa60 0x561f73841af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.386+0000 7f8089400640 1 -- 172.21.15.111:0/2885086214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f73845090 msgr2=0x561f7384f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.386+0000 7f8089400640 1 --2- 172.21.15.111:0/2885086214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f73845090 0x561f7384f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.386+0000 7f8089400640 1 -- 172.21.15.111:0/2885086214 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f738578b0 con 0x561f738459d0 2024-09-15T23:03:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.386+0000 7f8089400640 1 --2- 172.21.15.111:0/2885086214 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561f738459d0 0x561f738566c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8074098740 tx=0x7f80740d2150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=685755f55a3ceb17 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:03:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.386+0000 7f8072600640 1 -- 172.21.15.111:0/2885086214 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807409c2e0 con 0x561f738459d0 2024-09-15T23:03:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.386+0000 7f8072600640 1 -- 172.21.15.111:0/2885086214 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8074096e50 con 0x561f738459d0 2024-09-15T23:03:26.389 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.386+0000 7f8072600640 1 -- 172.21.15.111:0/2885086214 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807401fda0 con 0x561f738459d0 2024-09-15T23:03:26.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:25 smithi121 ceph-mon[34815]: pgmap v876: 97 pgs: 97 active+clean; 478 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 186 op/s 2024-09-15T23:03:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.387+0000 7f808f7d5740 1 -- 172.21.15.111:0/2885086214 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561f738459d0 msgr2=0x561f738566c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.387+0000 7f808f7d5740 1 --2- 172.21.15.111:0/2885086214 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561f738459d0 0x561f738566c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8074098740 tx=0x7f80740d2150 comp rx=0 tx=0).stop 2024-09-15T23:03:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.387+0000 7f808f7d5740 1 -- 172.21.15.111:0/2885086214 shutdown_connections 2024-09-15T23:03:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.387+0000 7f808f7d5740 1 --2- 172.21.15.111:0/2885086214 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f7384aa60 0x561f73841af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.387+0000 7f808f7d5740 1 --2- 172.21.15.111:0/2885086214 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561f738459d0 0x561f738566c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.387+0000 7f808f7d5740 1 --2- 172.21.15.111:0/2885086214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f73845090 0x561f7384f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.387+0000 7f808f7d5740 1 -- 172.21.15.111:0/2885086214 >> 172.21.15.111:0/2885086214 conn(0x561f73840700 msgr2=0x561f7383f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:03:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.387+0000 7f808f7d5740 1 -- 172.21.15.111:0/2885086214 shutdown_connections 2024-09-15T23:03:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.387+0000 7f808f7d5740 1 -- 172.21.15.111:0/2885086214 wait complete. 2024-09-15T23:03:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.388+0000 7f808f7d5740 1 -- start start 2024-09-15T23:03:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.388+0000 7f808f7d5740 1 Processor -- start 2024-09-15T23:03:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.389+0000 7f808f7d5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561f73845090 0x561f738541b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.389+0000 7f808f7d5740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f738459d0 0x561f738546f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.389+0000 7f808f7d5740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f7384aa60 0x561f73855700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.389+0000 7f808f7d5740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f80841adac0 con 0x561f738459d0 2024-09-15T23:03:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.389+0000 7f808f7d5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f80841aef30 con 0x561f73845090 2024-09-15T23:03:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.389+0000 7f808f7d5740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f80841ae190 con 0x561f7384aa60 2024-09-15T23:03:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.389+0000 7f808f7d5740 10 client.0 _ll_register_callbacks cb 0x7f8078009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:03:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.389+0000 7f8089e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561f73845090 0x561f738541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.389+0000 7f8089400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f738459d0 0x561f738546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.389+0000 7f8089e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561f73845090 0x561f738541b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:36612/0 (socket says 172.21.15.111:36612) 2024-09-15T23:03:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.389+0000 7f8089e00640 1 -- 172.21.15.111:0/1624113215 learned_addr learned my addr 172.21.15.111:0/1624113215 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:03:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.389+0000 7f808a800640 1 --2- 172.21.15.111:0/1624113215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f7384aa60 0x561f73855700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.389+0000 7f8089400640 1 -- 172.21.15.111:0/1624113215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f7384aa60 msgr2=0x561f73855700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.389+0000 7f8089400640 1 --2- 172.21.15.111:0/1624113215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f7384aa60 0x561f73855700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.389+0000 7f8089400640 1 -- 172.21.15.111:0/1624113215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561f73845090 msgr2=0x561f738541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.389+0000 7f8089400640 1 --2- 172.21.15.111:0/1624113215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561f73845090 0x561f738541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.389+0000 7f8089400640 1 -- 172.21.15.111:0/1624113215 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f73855d10 con 0x561f738459d0 2024-09-15T23:03:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.389+0000 7f8089e00640 1 --2- 172.21.15.111:0/1624113215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561f73845090 0x561f738541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:03:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.389+0000 7f808a800640 1 --2- 172.21.15.111:0/1624113215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f7384aa60 0x561f73855700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:03:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.390+0000 7f8089400640 1 --2- 172.21.15.111:0/1624113215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f738459d0 0x561f738546f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f807400c6a0 tx=0x7f807409bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50159d97b5e56d09 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:03:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.390+0000 7f8067400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:03:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.390+0000 7f8067400640 1 -- 172.21.15.111:0/1624113215 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80740b9020 con 0x561f738459d0 2024-09-15T23:03:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.390+0000 7f8067400640 1 -- 172.21.15.111:0/1624113215 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f807401fda0 con 0x561f738459d0 2024-09-15T23:03:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.390+0000 7f808f7d5740 1 -- 172.21.15.111:0/1624113215 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8074096e50 con 0x561f738459d0 2024-09-15T23:03:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.390+0000 7f8067400640 1 -- 172.21.15.111:0/1624113215 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f80740b4070 con 0x561f738459d0 2024-09-15T23:03:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.390+0000 7f808f7d5740 10 client.16395 fetch_fsmap learned FSMap version 35 2024-09-15T23:03:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.390+0000 7f808f7d5740 1 -- 172.21.15.111:0/1624113215 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561f738c5da0 con 0x561f738459d0 2024-09-15T23:03:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.391+0000 7f8067400640 1 -- 172.21.15.111:0/1624113215 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8074096e50 con 0x561f738459d0 2024-09-15T23:03:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.391+0000 7f8067400640 1 -- 172.21.15.111:0/1624113215 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f80740a4ac0 con 0x561f738459d0 2024-09-15T23:03:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.392+0000 7f808f7d5740 10 client.16395 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:03:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.392+0000 7f808f7d5740 10 client.16395 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:03:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.392+0000 7f808f7d5740 20 client.16395 populate_metadata read hostname 'smithi111' 2024-09-15T23:03:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.392+0000 7f808f7d5740 1 --2- 172.21.15.111:0/1624113215 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561f738c66b0 0x561f738c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.392+0000 7f808f7d5740 4 client.16395 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:03:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.392+0000 7f808f7d5740 1 -- 172.21.15.111:0/1624113215 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561f737b1b80 con 0x561f738c66b0 2024-09-15T23:03:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.392+0000 7f8089e00640 1 --2- 172.21.15.111:0/1624113215 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561f738c66b0 0x561f738c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.393+0000 7f8089e00640 1 --2- 172.21.15.111:0/1624113215 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561f738c66b0 0x561f738c8ad0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c133761d2e67cf6 server_cookie=c6551deee2ef0e5e in_seq=0 out_seq=0 2024-09-15T23:03:26.398 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.393+0000 7f8067400640 10 client.16395 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:03:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.551+0000 7f8067400640 1 -- 172.21.15.111:0/1624113215 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561f737b1b80 con 0x561f738c66b0 2024-09-15T23:03:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.551+0000 7f8067400640 10 client.16395 handle_command_reply: tid=0 2024-09-15T23:03:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.551+0000 7f808f7d5740 1 client.16395 shutdown 2024-09-15T23:03:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.551+0000 7f808f7d5740 1 -- 172.21.15.111:0/1624113215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f738459d0 msgr2=0x561f738546f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.551+0000 7f808f7d5740 1 --2- 172.21.15.111:0/1624113215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f738459d0 0x561f738546f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f807400c6a0 tx=0x7f807409bef0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.551+0000 7f808f7d5740 1 -- 172.21.15.111:0/1624113215 shutdown_connections 2024-09-15T23:03:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.552+0000 7f808f7d5740 1 --2- 172.21.15.111:0/1624113215 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561f738c66b0 0x561f738c8ad0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.552+0000 7f808f7d5740 1 --2- 172.21.15.111:0/1624113215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561f7384aa60 0x561f73855700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.552+0000 7f808f7d5740 1 --2- 172.21.15.111:0/1624113215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561f738459d0 0x561f738546f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.552+0000 7f808f7d5740 1 --2- 172.21.15.111:0/1624113215 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561f73845090 0x561f738541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.552+0000 7f8067400640 0 client.16395 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:03:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.552+0000 7f808f7d5740 1 -- 172.21.15.111:0/1624113215 >> 172.21.15.111:0/1624113215 conn(0x561f73840700 msgr2=0x561f73840ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:03:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.552+0000 7f808f7d5740 1 -- 172.21.15.111:0/1624113215 shutdown_connections 2024-09-15T23:03:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.552+0000 7f808f7d5740 1 -- 172.21.15.111:0/1624113215 wait complete. 2024-09-15T23:03:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.552+0000 7f808f7d5740 20 client.16395 trim_cache size 0 max 16384 2024-09-15T23:03:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:03:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:03:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:03:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:03:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.553+0000 7f808c800640 1 -- 172.21.15.111:0/805872699 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f806c0a3370 msgr2=0x7f806c0a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.553+0000 7f808c800640 1 --2- 172.21.15.111:0/805872699 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f806c0a3370 0x7f806c0a5830 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8080004af0 tx=0x7f808004b000 comp rx=0 tx=0).stop 2024-09-15T23:03:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.553+0000 7f808c800640 1 -- 172.21.15.111:0/805872699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8084104f40 msgr2=0x7f808419e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.553+0000 7f808c800640 1 --2- 172.21.15.111:0/805872699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8084104f40 0x7f808419e890 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f807400e9c0 tx=0x7f807400ee80 comp rx=0 tx=0).stop 2024-09-15T23:03:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.553+0000 7f808c800640 1 -- 172.21.15.111:0/805872699 shutdown_connections 2024-09-15T23:03:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.553+0000 7f808c800640 1 --2- 172.21.15.111:0/805872699 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f806c0a3370 0x7f806c0a5830 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.553+0000 7f808c800640 1 --2- 172.21.15.111:0/805872699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8084105900 0x7f80841a5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.553+0000 7f808c800640 1 --2- 172.21.15.111:0/805872699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8084104f40 0x7f808419e890 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.553+0000 7f808c800640 1 --2- 172.21.15.111:0/805872699 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8084103d50 0x7f808419e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.553+0000 7f808c800640 1 -- 172.21.15.111:0/805872699 >> 172.21.15.111:0/805872699 conn(0x7f80840ff4b0 msgr2=0x7f8084078640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:03:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.553+0000 7f808c800640 1 -- 172.21.15.111:0/805872699 shutdown_connections 2024-09-15T23:03:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:26.553+0000 7f808c800640 1 -- 172.21.15.111:0/805872699 wait complete. 2024-09-15T23:03:26.641 INFO:tasks.workunit.client.0.smithi111.stdout: 10 77658 13.84 MB/sec execute 114 sec latency 76.732 ms 2024-09-15T23:03:27.641 INFO:tasks.workunit.client.0.smithi111.stdout: 10 78441 14.17 MB/sec execute 115 sec latency 80.464 ms 2024-09-15T23:03:28.290 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:27 smithi111 ceph-mon[30629]: pgmap v877: 97 pgs: 97 active+clean; 479 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 187 op/s 2024-09-15T23:03:28.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:27 smithi192 ceph-mon[35427]: pgmap v877: 97 pgs: 97 active+clean; 479 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 187 op/s 2024-09-15T23:03:28.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:27 smithi121 ceph-mon[34815]: pgmap v877: 97 pgs: 97 active+clean; 479 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 187 op/s 2024-09-15T23:03:28.641 INFO:tasks.workunit.client.0.smithi111.stdout: 10 79179 14.30 MB/sec execute 116 sec latency 69.470 ms 2024-09-15T23:03:29.641 INFO:tasks.workunit.client.0.smithi111.stdout: 10 80063 14.61 MB/sec execute 117 sec latency 54.631 ms 2024-09-15T23:03:30.013 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:29 smithi111 ceph-mon[30629]: pgmap v878: 97 pgs: 97 active+clean; 485 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.2 MiB/s wr, 357 op/s 2024-09-15T23:03:30.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:29 smithi192 ceph-mon[35427]: pgmap v878: 97 pgs: 97 active+clean; 485 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.2 MiB/s wr, 357 op/s 2024-09-15T23:03:30.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:29 smithi121 ceph-mon[34815]: pgmap v878: 97 pgs: 97 active+clean; 485 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.2 MiB/s wr, 357 op/s 2024-09-15T23:03:30.641 INFO:tasks.workunit.client.0.smithi111.stdout: 10 80836 14.90 MB/sec execute 118 sec latency 73.340 ms 2024-09-15T23:03:31.641 INFO:tasks.workunit.client.0.smithi111.stdout: 10 81650 15.15 MB/sec execute 119 sec latency 90.395 ms 2024-09-15T23:03:32.279 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:31 smithi111 ceph-mon[30629]: pgmap v879: 97 pgs: 97 active+clean; 492 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 2.9 MiB/s wr, 446 op/s 2024-09-15T23:03:32.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:31 smithi192 ceph-mon[35427]: pgmap v879: 97 pgs: 97 active+clean; 492 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 2.9 MiB/s wr, 446 op/s 2024-09-15T23:03:32.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:31 smithi121 ceph-mon[34815]: pgmap v879: 97 pgs: 97 active+clean; 492 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 2.9 MiB/s wr, 446 op/s 2024-09-15T23:03:32.641 INFO:tasks.workunit.client.0.smithi111.stdout: 10 82243 15.26 MB/sec execute 120 sec latency 72.802 ms 2024-09-15T23:03:33.641 INFO:tasks.workunit.client.0.smithi111.stdout: 10 82963 15.45 MB/sec execute 121 sec latency 54.187 ms 2024-09-15T23:03:34.285 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:33 smithi111 ceph-mon[30629]: pgmap v880: 97 pgs: 97 active+clean; 497 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.4 MiB/s wr, 489 op/s 2024-09-15T23:03:34.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:33 smithi192 ceph-mon[35427]: pgmap v880: 97 pgs: 97 active+clean; 497 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.4 MiB/s wr, 489 op/s 2024-09-15T23:03:34.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:33 smithi121 ceph-mon[34815]: pgmap v880: 97 pgs: 97 active+clean; 497 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.4 MiB/s wr, 489 op/s 2024-09-15T23:03:34.642 INFO:tasks.workunit.client.0.smithi111.stdout: 10 83802 15.64 MB/sec execute 122 sec latency 62.283 ms 2024-09-15T23:03:35.642 INFO:tasks.workunit.client.0.smithi111.stdout: 10 84646 15.87 MB/sec execute 123 sec latency 148.591 ms 2024-09-15T23:03:36.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:35 smithi192 ceph-mon[35427]: pgmap v881: 97 pgs: 97 active+clean; 503 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.0 MiB/s wr, 553 op/s 2024-09-15T23:03:36.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:35 smithi121 ceph-mon[34815]: pgmap v881: 97 pgs: 97 active+clean; 503 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.0 MiB/s wr, 553 op/s 2024-09-15T23:03:36.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:35 smithi111 ceph-mon[30629]: pgmap v881: 97 pgs: 97 active+clean; 503 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.0 MiB/s wr, 553 op/s 2024-09-15T23:03:36.642 INFO:tasks.workunit.client.0.smithi111.stdout: 10 85500 16.23 MB/sec execute 124 sec latency 47.689 ms 2024-09-15T23:03:37.642 INFO:tasks.workunit.client.0.smithi111.stdout: 10 86115 16.37 MB/sec execute 125 sec latency 69.546 ms 2024-09-15T23:03:38.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:37 smithi192 ceph-mon[35427]: pgmap v882: 97 pgs: 97 active+clean; 507 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 3.9 MiB/s wr, 522 op/s 2024-09-15T23:03:38.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:37 smithi121 ceph-mon[34815]: pgmap v882: 97 pgs: 97 active+clean; 507 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 3.9 MiB/s wr, 522 op/s 2024-09-15T23:03:38.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:37 smithi111 ceph-mon[30629]: pgmap v882: 97 pgs: 97 active+clean; 507 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 3.9 MiB/s wr, 522 op/s 2024-09-15T23:03:38.642 INFO:tasks.workunit.client.0.smithi111.stdout: 10 86669 16.41 MB/sec execute 126 sec latency 77.703 ms 2024-09-15T23:03:39.642 INFO:tasks.workunit.client.0.smithi111.stdout: 10 87390 16.58 MB/sec execute 127 sec latency 149.625 ms 2024-09-15T23:03:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.694+0000 7f2d19600640 1 -- 172.21.15.111:0/589030552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d14102d00 msgr2=0x7f2d14105170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.694+0000 7f2d19600640 1 --2- 172.21.15.111:0/589030552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d14102d00 0x7f2d14105170 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009860 tx=0x7f2d0002a340 comp rx=0 tx=0).stop 2024-09-15T23:03:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.694+0000 7f2d19600640 1 -- 172.21.15.111:0/589030552 shutdown_connections 2024-09-15T23:03:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.694+0000 7f2d19600640 1 --2- 172.21.15.111:0/589030552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d141056b0 0x7f2d14105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.694+0000 7f2d19600640 1 --2- 172.21.15.111:0/589030552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d14102d00 0x7f2d14105170 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.694+0000 7f2d19600640 1 --2- 172.21.15.111:0/589030552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d141003b0 0x7f2d141027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.694+0000 7f2d19600640 1 -- 172.21.15.111:0/589030552 >> 172.21.15.111:0/589030552 conn(0x7f2d140fc1e0 msgr2=0x7f2d140fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:03:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.695+0000 7f2d19600640 1 -- 172.21.15.111:0/589030552 shutdown_connections 2024-09-15T23:03:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.695+0000 7f2d19600640 1 -- 172.21.15.111:0/589030552 wait complete. 2024-09-15T23:03:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.696+0000 7f2d19600640 1 Processor -- start 2024-09-15T23:03:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.696+0000 7f2d19600640 1 -- start start 2024-09-15T23:03:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.696+0000 7f2d19600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d141003b0 0x7f2d1419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.696+0000 7f2d19600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d14102d00 0x7f2d1419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.696+0000 7f2d19600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d141056b0 0x7f2d141a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.696+0000 7f2d19600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2d14113df0 con 0x7f2d141056b0 2024-09-15T23:03:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.696+0000 7f2d19600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2d14113c70 con 0x7f2d14102d00 2024-09-15T23:03:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.696+0000 7f2d19600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2d14113f70 con 0x7f2d141003b0 2024-09-15T23:03:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.696+0000 7f2d12000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d14102d00 0x7f2d1419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.696+0000 7f2d12a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d141003b0 0x7f2d1419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.697+0000 7f2d12000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d14102d00 0x7f2d1419e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:52616/0 (socket says 172.21.15.111:52616) 2024-09-15T23:03:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.697+0000 7f2d12a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d141003b0 0x7f2d1419e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:38668/0 (socket says 172.21.15.111:38668) 2024-09-15T23:03:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.697+0000 7f2d12000640 1 -- 172.21.15.111:0/3731126182 learned_addr learned my addr 172.21.15.111:0/3731126182 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:03:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.697+0000 7f2d13400640 1 --2- 172.21.15.111:0/3731126182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d141056b0 0x7f2d141a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.697+0000 7f2d12000640 1 -- 172.21.15.111:0/3731126182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d141003b0 msgr2=0x7f2d1419e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.697+0000 7f2d12000640 1 --2- 172.21.15.111:0/3731126182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d141003b0 0x7f2d1419e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.697+0000 7f2d12000640 1 -- 172.21.15.111:0/3731126182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d141056b0 msgr2=0x7f2d141a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.697+0000 7f2d12000640 1 --2- 172.21.15.111:0/3731126182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d141056b0 0x7f2d141a59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.697+0000 7f2d12000640 1 -- 172.21.15.111:0/3731126182 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d141a6020 con 0x7f2d14102d00 2024-09-15T23:03:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.697+0000 7f2d12000640 1 --2- 172.21.15.111:0/3731126182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d14102d00 0x7f2d1419e940 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2d00035040 tx=0x7f2d00033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:03:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.698+0000 7f2d0b400640 1 -- 172.21.15.111:0/3731126182 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d00037040 con 0x7f2d14102d00 2024-09-15T23:03:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.698+0000 7f2d0b400640 1 -- 172.21.15.111:0/3731126182 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d00032770 con 0x7f2d14102d00 2024-09-15T23:03:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.698+0000 7f2d19600640 1 -- 172.21.15.111:0/3731126182 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d141a62b0 con 0x7f2d14102d00 2024-09-15T23:03:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.698+0000 7f2d0b400640 1 -- 172.21.15.111:0/3731126182 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d000046c0 con 0x7f2d14102d00 2024-09-15T23:03:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.698+0000 7f2d19600640 1 -- 172.21.15.111:0/3731126182 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d141a67f0 con 0x7f2d14102d00 2024-09-15T23:03:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.701+0000 7f2d1c641740 1 Processor -- start 2024-09-15T23:03:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.701+0000 7f2d1c641740 1 -- start start 2024-09-15T23:03:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.701+0000 7f2d1c641740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5614daf52090 0x5614daf52490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.701+0000 7f2d1c641740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614daf529d0 0x5614daf57490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.701+0000 7f2d1c641740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614daf57a60 0x5614daf59f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.701+0000 7f2d1c641740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5614daeb7100 con 0x5614daf529d0 2024-09-15T23:03:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.701+0000 7f2d1c641740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5614dae5f1b0 con 0x5614daf52090 2024-09-15T23:03:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.702+0000 7f2d1c641740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5614daeb30c0 con 0x5614daf57a60 2024-09-15T23:03:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.702+0000 7f2d13400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5614daf52090 0x5614daf52490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.702+0000 7f2d12a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614daf529d0 0x5614daf57490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.702+0000 7f2d13400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5614daf52090 0x5614daf52490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:52618/0 (socket says 172.21.15.111:52618) 2024-09-15T23:03:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.702+0000 7f2d13400640 1 -- 172.21.15.111:0/3548120442 learned_addr learned my addr 172.21.15.111:0/3548120442 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:03:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.702+0000 7f2d13400640 1 -- 172.21.15.111:0/3548120442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614daf57a60 msgr2=0x5614daf59f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T23:03:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.702+0000 7f2d13400640 1 --2- 172.21.15.111:0/3548120442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614daf57a60 0x5614daf59f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.702+0000 7f2d0b400640 1 -- 172.21.15.111:0/3731126182 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f2d00004050 con 0x7f2d14102d00 2024-09-15T23:03:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.702+0000 7f2d13400640 1 -- 172.21.15.111:0/3548120442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614daf529d0 msgr2=0x5614daf57490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.702+0000 7f2d13400640 1 --2- 172.21.15.111:0/3548120442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614daf529d0 0x5614daf57490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.702+0000 7f2d13400640 1 -- 172.21.15.111:0/3548120442 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614dabca210 con 0x5614daf52090 2024-09-15T23:03:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.702+0000 7f2d13400640 1 --2- 172.21.15.111:0/3548120442 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5614daf52090 0x5614daf52490 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009fc0 tx=0x7f2d0c010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1367a1c023ba570e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:03:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.703+0000 7f2cff400640 1 -- 172.21.15.111:0/3548120442 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c003b70 con 0x5614daf52090 2024-09-15T23:03:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.703+0000 7f2cff400640 1 -- 172.21.15.111:0/3548120442 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d0c003d10 con 0x5614daf52090 2024-09-15T23:03:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.703+0000 7f2cff400640 1 -- 172.21.15.111:0/3548120442 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c011aa0 con 0x5614daf52090 2024-09-15T23:03:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.703+0000 7f2d1c641740 1 -- 172.21.15.111:0/3548120442 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5614daf52090 msgr2=0x5614daf52490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.703+0000 7f2d1c641740 1 --2- 172.21.15.111:0/3548120442 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5614daf52090 0x5614daf52490 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009fc0 tx=0x7f2d0c010860 comp rx=0 tx=0).stop 2024-09-15T23:03:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.704+0000 7f2d1c641740 1 -- 172.21.15.111:0/3548120442 shutdown_connections 2024-09-15T23:03:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.704+0000 7f2d1c641740 1 --2- 172.21.15.111:0/3548120442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614daf57a60 0x5614daf59f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.704+0000 7f2d1c641740 1 --2- 172.21.15.111:0/3548120442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614daf529d0 0x5614daf57490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.704+0000 7f2d1c641740 1 --2- 172.21.15.111:0/3548120442 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5614daf52090 0x5614daf52490 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.704+0000 7f2d1c641740 1 -- 172.21.15.111:0/3548120442 >> 172.21.15.111:0/3548120442 conn(0x5614daf4d700 msgr2=0x5614daf4db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:03:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.704+0000 7f2d1c641740 1 -- 172.21.15.111:0/3548120442 shutdown_connections 2024-09-15T23:03:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.704+0000 7f2d1c641740 1 -- 172.21.15.111:0/3548120442 wait complete. 2024-09-15T23:03:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.704+0000 7f2d0b400640 1 --2- 172.21.15.111:0/3731126182 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2cf80a3280 0x7f2cf80a5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.704+0000 7f2d0b400640 1 -- 172.21.15.111:0/3731126182 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f2d000ef590 con 0x7f2d14102d00 2024-09-15T23:03:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.704+0000 7f2d1c641740 1 -- start start 2024-09-15T23:03:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.704+0000 7f2d12a00640 1 --2- 172.21.15.111:0/3731126182 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2cf80a3280 0x7f2cf80a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.704+0000 7f2d1c641740 1 Processor -- start 2024-09-15T23:03:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.705+0000 7f2d1c641740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614daf52090 0x5614daf5c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.705+0000 7f2d1c641740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5614daf529d0 0x5614daf5c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:39.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.705+0000 7f2d1c641740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614daf57a60 0x5614daf638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.705+0000 7f2d1c641740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5614daf5aa30 con 0x5614daf52090 2024-09-15T23:03:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.705+0000 7f2d1c641740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5614dae40090 con 0x5614daf529d0 2024-09-15T23:03:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.705+0000 7f2d1c641740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5614daf5abb0 con 0x5614daf57a60 2024-09-15T23:03:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.705+0000 7f2d1c641740 10 client.0 _ll_register_callbacks cb 0x5614dae94f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:03:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.705+0000 7f2d12000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5614daf529d0 0x5614daf5c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.705+0000 7f2d12000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5614daf529d0 0x5614daf5c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:52632/0 (socket says 172.21.15.111:52632) 2024-09-15T23:03:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.705+0000 7f2d13400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614daf57a60 0x5614daf638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:39.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.705+0000 7f2d12000640 1 -- 172.21.15.111:0/3119295507 learned_addr learned my addr 172.21.15.111:0/3119295507 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:03:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.705+0000 7f2d12a00640 1 --2- 172.21.15.111:0/3119295507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614daf52090 0x5614daf5c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.705+0000 7f2d12a00640 1 --2- 172.21.15.111:0/3731126182 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2cf80a3280 0x7f2cf80a5740 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x5614daf4e580 tx=0x7f2d04009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:03:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.706+0000 7f2d13400640 1 -- 172.21.15.111:0/3119295507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5614daf529d0 msgr2=0x5614daf5c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.706+0000 7f2d13400640 1 --2- 172.21.15.111:0/3119295507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5614daf529d0 0x5614daf5c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.706+0000 7f2d13400640 1 -- 172.21.15.111:0/3119295507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614daf52090 msgr2=0x5614daf5c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.706+0000 7f2d13400640 1 --2- 172.21.15.111:0/3119295507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614daf52090 0x5614daf5c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.706+0000 7f2d13400640 1 -- 172.21.15.111:0/3119295507 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614dac96620 con 0x5614daf57a60 2024-09-15T23:03:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.706+0000 7f2d12000640 1 --2- 172.21.15.111:0/3119295507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5614daf529d0 0x5614daf5c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:03:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.706+0000 7f2d13400640 1 --2- 172.21.15.111:0/3119295507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614daf57a60 0x5614daf638b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c010d60 tx=0x7f2d0c010fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=457700c9f9cee63e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:03:39.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.706+0000 7f2cf3e00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:03:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.706+0000 7f2cf3e00640 1 -- 172.21.15.111:0/3119295507 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c0117e0 con 0x5614daf57a60 2024-09-15T23:03:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.706+0000 7f2d1c641740 1 -- 172.21.15.111:0/3119295507 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d0c003d10 con 0x5614daf57a60 2024-09-15T23:03:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.706+0000 7f2cf3e00640 1 -- 172.21.15.111:0/3119295507 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f2d0c011980 con 0x5614daf57a60 2024-09-15T23:03:39.714 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.707+0000 7f2cf3e00640 1 -- 172.21.15.111:0/3119295507 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2d0c011b80 con 0x5614daf57a60 2024-09-15T23:03:39.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.707+0000 7f2d1c641740 10 client.26371 fetch_fsmap learned FSMap version 35 2024-09-15T23:03:39.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.707+0000 7f2d1c641740 1 -- 172.21.15.111:0/3119295507 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5614daf666a0 con 0x5614daf57a60 2024-09-15T23:03:39.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.707+0000 7f2cf3e00640 1 -- 172.21.15.111:0/3119295507 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d0c003d10 con 0x5614daf57a60 2024-09-15T23:03:39.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.707+0000 7f2cf3e00640 1 -- 172.21.15.111:0/3119295507 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f2d0c0050d0 con 0x5614daf57a60 2024-09-15T23:03:39.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.707+0000 7f2d1c641740 10 client.26371 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:03:39.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.707+0000 7f2d1c641740 10 client.26371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:03:39.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.707+0000 7f2d1c641740 20 client.26371 populate_metadata read hostname 'smithi111' 2024-09-15T23:03:39.715 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.707+0000 7f2d1c641740 1 --2- 172.21.15.111:0/3119295507 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5614daf67b00 0x5614daf69ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.708+0000 7f2d1c641740 4 client.26371 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:03:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.708+0000 7f2d1c641740 1 -- 172.21.15.111:0/3119295507 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614daec4830 con 0x5614daf67b00 2024-09-15T23:03:39.716 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.708+0000 7f2d12a00640 1 --2- 172.21.15.111:0/3119295507 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5614daf67b00 0x5614daf69ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:39.717 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.709+0000 7f2d12a00640 1 --2- 172.21.15.111:0/3119295507 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5614daf67b00 0x5614daf69ef0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14af795bc5d54b1 server_cookie=9d32ed5e003223f6 in_seq=0 out_seq=0 2024-09-15T23:03:39.717 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.709+0000 7f2cf3e00640 10 client.26371 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:03:39.717 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.710+0000 7f2cf3e00640 1 -- 172.21.15.111:0/3119295507 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5614daec4830 con 0x5614daf67b00 2024-09-15T23:03:39.717 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.710+0000 7f2cf3e00640 10 client.26371 handle_command_reply: tid=0 2024-09-15T23:03:39.717 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.710+0000 7f2d1c641740 1 client.26371 shutdown 2024-09-15T23:03:39.717 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.710+0000 7f2d1c641740 1 -- 172.21.15.111:0/3119295507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614daf57a60 msgr2=0x5614daf638b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:39.717 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.710+0000 7f2d1c641740 1 --2- 172.21.15.111:0/3119295507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614daf57a60 0x5614daf638b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c010d60 tx=0x7f2d0c010fd0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.717 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.710+0000 7f2d1c641740 1 -- 172.21.15.111:0/3119295507 shutdown_connections 2024-09-15T23:03:39.717 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.710+0000 7f2d1c641740 1 --2- 172.21.15.111:0/3119295507 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5614daf67b00 0x5614daf69ef0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.717 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.710+0000 7f2d1c641740 1 --2- 172.21.15.111:0/3119295507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614daf57a60 0x5614daf638b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.710+0000 7f2d1c641740 1 --2- 172.21.15.111:0/3119295507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5614daf529d0 0x5614daf5c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.710+0000 7f2d1c641740 1 --2- 172.21.15.111:0/3119295507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614daf52090 0x5614daf5c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.710+0000 7f2cf3e00640 0 client.26371 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:03:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.710+0000 7f2d1c641740 1 -- 172.21.15.111:0/3119295507 >> 172.21.15.111:0/3119295507 conn(0x5614daf4d700 msgr2=0x5614daf4ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:03:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.710+0000 7f2d1c641740 1 -- 172.21.15.111:0/3119295507 shutdown_connections 2024-09-15T23:03:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.710+0000 7f2d1c641740 1 -- 172.21.15.111:0/3119295507 wait complete. 2024-09-15T23:03:39.718 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.710+0000 7f2d1c641740 20 client.26371 trim_cache size 0 max 16384 2024-09-15T23:03:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.732+0000 7f2d1c641740 1 Processor -- start 2024-09-15T23:03:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.733+0000 7f2d1c641740 1 -- start start 2024-09-15T23:03:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.733+0000 7f2d1c641740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614daf52090 0x5614daf4c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.733+0000 7f2d1c641740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5614daf529d0 0x5614daf637d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.733+0000 7f2d1c641740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614daf57a60 0x5614daf6e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:39.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.733+0000 7f2d1c641740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141b1d30 con 0x5614daf57a60 2024-09-15T23:03:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.733+0000 7f2d1c641740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141af600 con 0x5614daf529d0 2024-09-15T23:03:39.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.733+0000 7f2d1c641740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141ad470 con 0x5614daf52090 2024-09-15T23:03:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.733+0000 7f2d12a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614daf52090 0x5614daf4c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.733+0000 7f2d12a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614daf52090 0x5614daf4c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:38682/0 (socket says 172.21.15.111:38682) 2024-09-15T23:03:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.733+0000 7f2d12a00640 1 -- 172.21.15.111:0/3877309243 learned_addr learned my addr 172.21.15.111:0/3877309243 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:03:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.733+0000 7f2d13400640 1 --2- 172.21.15.111:0/3877309243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614daf57a60 0x5614daf6e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.734+0000 7f2d12a00640 1 -- 172.21.15.111:0/3877309243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5614daf529d0 msgr2=0x5614daf637d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:39.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.734+0000 7f2d12a00640 1 --2- 172.21.15.111:0/3877309243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5614daf529d0 0x5614daf637d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.734+0000 7f2d12a00640 1 -- 172.21.15.111:0/3877309243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614daf57a60 msgr2=0x5614daf6e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.734+0000 7f2d12a00640 1 --2- 172.21.15.111:0/3877309243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614daf57a60 0x5614daf6e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.734+0000 7f2d12a00640 1 -- 172.21.15.111:0/3877309243 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614daf537a0 con 0x5614daf52090 2024-09-15T23:03:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.734+0000 7f2d12a00640 1 --2- 172.21.15.111:0/3877309243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614daf52090 0x5614daf4c770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2d04005fd0 tx=0x7f2d0401a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9249b64e019e53a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:03:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.735+0000 7f2cfea00640 1 -- 172.21.15.111:0/3877309243 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0402a430 con 0x5614daf52090 2024-09-15T23:03:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.735+0000 7f2cfea00640 1 -- 172.21.15.111:0/3877309243 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d0402aab0 con 0x5614daf52090 2024-09-15T23:03:39.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.735+0000 7f2cfea00640 1 -- 172.21.15.111:0/3877309243 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0401eac0 con 0x5614daf52090 2024-09-15T23:03:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.735+0000 7f2d1c641740 1 -- 172.21.15.111:0/3877309243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614daf52090 msgr2=0x5614daf4c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.735+0000 7f2d1c641740 1 --2- 172.21.15.111:0/3877309243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614daf52090 0x5614daf4c770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2d04005fd0 tx=0x7f2d0401a000 comp rx=0 tx=0).stop 2024-09-15T23:03:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.735+0000 7f2d1c641740 1 -- 172.21.15.111:0/3877309243 shutdown_connections 2024-09-15T23:03:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.735+0000 7f2d1c641740 1 --2- 172.21.15.111:0/3877309243 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614daf57a60 0x5614daf6e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.735+0000 7f2d1c641740 1 --2- 172.21.15.111:0/3877309243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5614daf529d0 0x5614daf637d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.735+0000 7f2d1c641740 1 --2- 172.21.15.111:0/3877309243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614daf52090 0x5614daf4c770 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.735+0000 7f2d1c641740 1 -- 172.21.15.111:0/3877309243 >> 172.21.15.111:0/3877309243 conn(0x5614daf4d700 msgr2=0x5614daf4c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:03:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.736+0000 7f2d1c641740 1 -- 172.21.15.111:0/3877309243 shutdown_connections 2024-09-15T23:03:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.736+0000 7f2d1c641740 1 -- 172.21.15.111:0/3877309243 wait complete. 2024-09-15T23:03:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.736+0000 7f2d1c641740 1 -- start start 2024-09-15T23:03:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.736+0000 7f2d1c641740 1 Processor -- start 2024-09-15T23:03:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.737+0000 7f2d1c641740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614daf52090 0x5614daf60e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.737+0000 7f2d1c641740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614daf529d0 0x5614daf61360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.737+0000 7f2d1c641740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5614daf57a60 0x5614daf622d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.737+0000 7f2d1c641740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141adac0 con 0x5614daf52090 2024-09-15T23:03:39.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.737+0000 7f2d1c641740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141aef30 con 0x5614daf57a60 2024-09-15T23:03:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.737+0000 7f2d1c641740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141ae190 con 0x5614daf529d0 2024-09-15T23:03:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.737+0000 7f2d1c641740 10 client.0 _ll_register_callbacks cb 0x7f2d040097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:03:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.737+0000 7f2d12000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614daf529d0 0x5614daf61360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.737+0000 7f2d12a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614daf52090 0x5614daf60e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.737+0000 7f2d12000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614daf529d0 0x5614daf61360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:38684/0 (socket says 172.21.15.111:38684) 2024-09-15T23:03:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.737+0000 7f2d12000640 1 -- 172.21.15.111:0/3162632567 learned_addr learned my addr 172.21.15.111:0/3162632567 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:03:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.737+0000 7f2d13400640 1 --2- 172.21.15.111:0/3162632567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5614daf57a60 0x5614daf622d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:39.741 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.738+0000 7f2d12000640 1 -- 172.21.15.111:0/3162632567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5614daf57a60 msgr2=0x5614daf622d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.738+0000 7f2d12000640 1 --2- 172.21.15.111:0/3162632567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5614daf57a60 0x5614daf622d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.738+0000 7f2d12000640 1 -- 172.21.15.111:0/3162632567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614daf52090 msgr2=0x5614daf60e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.738+0000 7f2d12000640 1 --2- 172.21.15.111:0/3162632567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614daf52090 0x5614daf60e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.738+0000 7f2d12000640 1 -- 172.21.15.111:0/3162632567 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614daf628e0 con 0x5614daf529d0 2024-09-15T23:03:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.738+0000 7f2d12a00640 1 --2- 172.21.15.111:0/3162632567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614daf52090 0x5614daf60e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:03:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.738+0000 7f2d13400640 1 --2- 172.21.15.111:0/3162632567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5614daf57a60 0x5614daf622d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:03:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.738+0000 7f2d12000640 1 --2- 172.21.15.111:0/3162632567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614daf529d0 0x5614daf61360 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2d000ed180 tx=0x7f2d000bb850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21f00d34a62654c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:03:39.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.738+0000 7f2cf3e00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:03:39.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.738+0000 7f2cf3e00640 1 -- 172.21.15.111:0/3162632567 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d000d1020 con 0x5614daf529d0 2024-09-15T23:03:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.738+0000 7f2cf3e00640 1 -- 172.21.15.111:0/3162632567 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f2d000c4630 con 0x5614daf529d0 2024-09-15T23:03:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.738+0000 7f2d1c641740 1 -- 172.21.15.111:0/3162632567 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d0402aab0 con 0x5614daf529d0 2024-09-15T23:03:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.738+0000 7f2cf3e00640 1 -- 172.21.15.111:0/3162632567 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2d000c0000 con 0x5614daf529d0 2024-09-15T23:03:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.739+0000 7f2d1c641740 10 client.26377 fetch_fsmap learned FSMap version 35 2024-09-15T23:03:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.739+0000 7f2d1c641740 1 -- 172.21.15.111:0/3162632567 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5614dafd2650 con 0x5614daf529d0 2024-09-15T23:03:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.739+0000 7f2cf3e00640 1 -- 172.21.15.111:0/3162632567 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d0402aab0 con 0x5614daf529d0 2024-09-15T23:03:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.739+0000 7f2cf3e00640 1 -- 172.21.15.111:0/3162632567 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f2d0003a070 con 0x5614daf529d0 2024-09-15T23:03:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.739+0000 7f2d1c641740 10 client.26377 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:03:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.739+0000 7f2d1c641740 10 client.26377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:03:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.739+0000 7f2d1c641740 20 client.26377 populate_metadata read hostname 'smithi111' 2024-09-15T23:03:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.739+0000 7f2d1c641740 1 --2- 172.21.15.111:0/3162632567 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5614dafd2fd0 0x5614dafd53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:39.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.739+0000 7f2d1c641740 4 client.26377 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:03:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.739+0000 7f2d1c641740 1 -- 172.21.15.111:0/3162632567 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5614daebeb80 con 0x5614dafd2fd0 2024-09-15T23:03:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.740+0000 7f2d12a00640 1 --2- 172.21.15.111:0/3162632567 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5614dafd2fd0 0x5614dafd53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.740+0000 7f2cf3e00640 10 client.26377 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:03:39.745 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.740+0000 7f2d12a00640 1 --2- 172.21.15.111:0/3162632567 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5614dafd2fd0 0x5614dafd53f0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=114f55a199c0b897 server_cookie=af22fec0887ce5fb in_seq=0 out_seq=0 2024-09-15T23:03:39.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.991+0000 7f2cf3e00640 1 -- 172.21.15.111:0/3162632567 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5614daebeb80 con 0x5614dafd2fd0 2024-09-15T23:03:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.991+0000 7f2cf3e00640 10 client.26377 handle_command_reply: tid=0 2024-09-15T23:03:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.991+0000 7f2d1c641740 1 client.26377 shutdown 2024-09-15T23:03:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.992+0000 7f2d1c641740 1 -- 172.21.15.111:0/3162632567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614daf529d0 msgr2=0x5614daf61360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.992+0000 7f2d1c641740 1 --2- 172.21.15.111:0/3162632567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614daf529d0 0x5614daf61360 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2d000ed180 tx=0x7f2d000bb850 comp rx=0 tx=0).stop 2024-09-15T23:03:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.992+0000 7f2d1c641740 1 -- 172.21.15.111:0/3162632567 shutdown_connections 2024-09-15T23:03:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.992+0000 7f2d1c641740 1 --2- 172.21.15.111:0/3162632567 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5614dafd2fd0 0x5614dafd53f0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.992+0000 7f2d1c641740 1 --2- 172.21.15.111:0/3162632567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5614daf57a60 0x5614daf622d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.992+0000 7f2d1c641740 1 --2- 172.21.15.111:0/3162632567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5614daf529d0 0x5614daf61360 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.992+0000 7f2cf3e00640 0 client.26377 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:03:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.992+0000 7f2d1c641740 1 --2- 172.21.15.111:0/3162632567 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5614daf52090 0x5614daf60e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.992+0000 7f2d1c641740 1 -- 172.21.15.111:0/3162632567 >> 172.21.15.111:0/3162632567 conn(0x5614daf4d700 msgr2=0x5614daf4edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:03:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.992+0000 7f2d1c641740 1 -- 172.21.15.111:0/3162632567 shutdown_connections 2024-09-15T23:03:39.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.992+0000 7f2d1c641740 1 -- 172.21.15.111:0/3162632567 wait complete. 2024-09-15T23:03:39.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.992+0000 7f2d1c641740 20 client.26377 trim_cache size 0 max 16384 2024-09-15T23:03:39.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:03:39.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:03:39.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:03:39.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:03:39.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.993+0000 7f2d19600640 1 -- 172.21.15.111:0/3731126182 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2cf80a3280 msgr2=0x7f2cf80a5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:39.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.993+0000 7f2d19600640 1 --2- 172.21.15.111:0/3731126182 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2cf80a3280 0x7f2cf80a5740 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x5614daf4e580 tx=0x7f2d04009260 comp rx=0 tx=0).stop 2024-09-15T23:03:39.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.993+0000 7f2d19600640 1 -- 172.21.15.111:0/3731126182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d14102d00 msgr2=0x7f2d1419e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:39.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.993+0000 7f2d19600640 1 --2- 172.21.15.111:0/3731126182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d14102d00 0x7f2d1419e940 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2d00035040 tx=0x7f2d00033d90 comp rx=0 tx=0).stop 2024-09-15T23:03:39.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.993+0000 7f2d19600640 1 -- 172.21.15.111:0/3731126182 shutdown_connections 2024-09-15T23:03:39.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.993+0000 7f2d19600640 1 --2- 172.21.15.111:0/3731126182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d141056b0 0x7f2d141a59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.993+0000 7f2d19600640 1 --2- 172.21.15.111:0/3731126182 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2cf80a3280 0x7f2cf80a5740 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.993+0000 7f2d19600640 1 --2- 172.21.15.111:0/3731126182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d14102d00 0x7f2d1419e940 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.993+0000 7f2d19600640 1 --2- 172.21.15.111:0/3731126182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d141003b0 0x7f2d1419e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:39.996 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.993+0000 7f2d19600640 1 -- 172.21.15.111:0/3731126182 >> 172.21.15.111:0/3731126182 conn(0x7f2d140fc1e0 msgr2=0x7f2d14100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:03:39.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.993+0000 7f2d19600640 1 -- 172.21.15.111:0/3731126182 shutdown_connections 2024-09-15T23:03:39.997 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:39.993+0000 7f2d19600640 1 -- 172.21.15.111:0/3731126182 wait complete. 2024-09-15T23:03:40.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:39 smithi192 ceph-mon[35427]: pgmap v883: 97 pgs: 97 active+clean; 518 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 4.8 MiB/s wr, 645 op/s 2024-09-15T23:03:40.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:39 smithi121 ceph-mon[34815]: pgmap v883: 97 pgs: 97 active+clean; 518 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 4.8 MiB/s wr, 645 op/s 2024-09-15T23:03:40.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:39 smithi111 ceph-mon[30629]: pgmap v883: 97 pgs: 97 active+clean; 518 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 4.8 MiB/s wr, 645 op/s 2024-09-15T23:03:40.642 INFO:tasks.workunit.client.0.smithi111.stdout: 10 88022 16.70 MB/sec execute 128 sec latency 191.403 ms 2024-09-15T23:03:41.642 INFO:tasks.workunit.client.0.smithi111.stdout: 10 88922 16.93 MB/sec execute 129 sec latency 52.218 ms 2024-09-15T23:03:42.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:41 smithi111 ceph-mon[30629]: pgmap v884: 97 pgs: 97 active+clean; 525 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.4 MiB/s wr, 567 op/s 2024-09-15T23:03:42.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:41 smithi192 ceph-mon[35427]: pgmap v884: 97 pgs: 97 active+clean; 525 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.4 MiB/s wr, 567 op/s 2024-09-15T23:03:42.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:41 smithi121 ceph-mon[34815]: pgmap v884: 97 pgs: 97 active+clean; 525 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.4 MiB/s wr, 567 op/s 2024-09-15T23:03:42.642 INFO:tasks.workunit.client.0.smithi111.stdout: 10 89739 17.23 MB/sec execute 130 sec latency 54.722 ms 2024-09-15T23:03:43.642 INFO:tasks.workunit.client.0.smithi111.stdout: 10 90338 17.33 MB/sec execute 131 sec latency 68.981 ms 2024-09-15T23:03:44.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:43 smithi192 ceph-mon[35427]: pgmap v885: 97 pgs: 97 active+clean; 527 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.2 MiB/s wr, 557 op/s 2024-09-15T23:03:44.367 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:43 smithi111 ceph-mon[30629]: pgmap v885: 97 pgs: 97 active+clean; 527 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.2 MiB/s wr, 557 op/s 2024-09-15T23:03:44.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:43 smithi121 ceph-mon[34815]: pgmap v885: 97 pgs: 97 active+clean; 527 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.2 MiB/s wr, 557 op/s 2024-09-15T23:03:44.642 INFO:tasks.workunit.client.0.smithi111.stdout: 10 91042 17.43 MB/sec execute 132 sec latency 122.490 ms 2024-09-15T23:03:45.642 INFO:tasks.workunit.client.0.smithi111.stdout: 10 91786 17.66 MB/sec execute 133 sec latency 50.833 ms 2024-09-15T23:03:46.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:45 smithi192 ceph-mon[35427]: pgmap v886: 97 pgs: 97 active+clean; 510 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.5 MiB/s wr, 629 op/s 2024-09-15T23:03:46.357 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:45 smithi111 ceph-mon[30629]: pgmap v886: 97 pgs: 97 active+clean; 510 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.5 MiB/s wr, 629 op/s 2024-09-15T23:03:46.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:45 smithi121 ceph-mon[34815]: pgmap v886: 97 pgs: 97 active+clean; 510 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.5 MiB/s wr, 629 op/s 2024-09-15T23:03:46.642 INFO:tasks.workunit.client.0.smithi111.stdout: 10 92403 17.72 MB/sec execute 134 sec latency 97.949 ms 2024-09-15T23:03:47.642 INFO:tasks.workunit.client.0.smithi111.stdout: 10 93182 17.93 MB/sec execute 135 sec latency 276.880 ms 2024-09-15T23:03:48.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:47 smithi111 ceph-mon[30629]: pgmap v887: 97 pgs: 97 active+clean; 511 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.3 MiB/s wr, 609 op/s 2024-09-15T23:03:48.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:47 smithi192 ceph-mon[35427]: pgmap v887: 97 pgs: 97 active+clean; 511 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.3 MiB/s wr, 609 op/s 2024-09-15T23:03:48.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:47 smithi121 ceph-mon[34815]: pgmap v887: 97 pgs: 97 active+clean; 511 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.3 MiB/s wr, 609 op/s 2024-09-15T23:03:48.642 INFO:tasks.workunit.client.0.smithi111.stdout: 10 94013 18.09 MB/sec execute 136 sec latency 49.983 ms 2024-09-15T23:03:49.642 INFO:tasks.workunit.client.0.smithi111.stdout: 10 94806 18.30 MB/sec execute 137 sec latency 48.861 ms 2024-09-15T23:03:50.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:49 smithi192 ceph-mon[35427]: pgmap v888: 97 pgs: 97 active+clean; 521 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.9 MiB/s wr, 665 op/s 2024-09-15T23:03:50.393 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:49 smithi121 ceph-mon[34815]: pgmap v888: 97 pgs: 97 active+clean; 521 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.9 MiB/s wr, 665 op/s 2024-09-15T23:03:50.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:49 smithi111 ceph-mon[30629]: pgmap v888: 97 pgs: 97 active+clean; 521 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.9 MiB/s wr, 665 op/s 2024-09-15T23:03:50.642 INFO:tasks.workunit.client.0.smithi111.stdout: 10 95490 18.45 MB/sec execute 138 sec latency 81.814 ms 2024-09-15T23:03:51.642 INFO:tasks.workunit.client.0.smithi111.stdout: 10 96142 18.54 MB/sec execute 139 sec latency 83.495 ms 2024-09-15T23:03:52.299 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:51 smithi111 ceph-mon[30629]: pgmap v889: 97 pgs: 97 active+clean; 527 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 4.5 MiB/s wr, 612 op/s 2024-09-15T23:03:52.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:51 smithi192 ceph-mon[35427]: pgmap v889: 97 pgs: 97 active+clean; 527 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 4.5 MiB/s wr, 612 op/s 2024-09-15T23:03:52.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:51 smithi121 ceph-mon[34815]: pgmap v889: 97 pgs: 97 active+clean; 527 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 4.5 MiB/s wr, 612 op/s 2024-09-15T23:03:52.645 INFO:tasks.workunit.client.0.smithi111.stdout: 10 96589 18.59 MB/sec execute 140 sec latency 314.555 ms 2024-09-15T23:03:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.109+0000 7f1d23c00640 1 -- 172.21.15.111:0/2150191867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d1c104f50 msgr2=0x7f1d1c1053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:53.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.109+0000 7f1d23c00640 1 --2- 172.21.15.111:0/2150191867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d1c104f50 0x7f1d1c1053d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1d08009900 tx=0x7f1d0802a410 comp rx=0 tx=0).stop 2024-09-15T23:03:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.110+0000 7f1d23c00640 1 -- 172.21.15.111:0/2150191867 shutdown_connections 2024-09-15T23:03:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.110+0000 7f1d23c00640 1 --2- 172.21.15.111:0/2150191867 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d1c105910 0x7f1d1c10c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.110+0000 7f1d23c00640 1 --2- 172.21.15.111:0/2150191867 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d1c104f50 0x7f1d1c1053d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.110+0000 7f1d23c00640 1 --2- 172.21.15.111:0/2150191867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d1c103d60 0x7f1d1c104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.110+0000 7f1d23c00640 1 -- 172.21.15.111:0/2150191867 >> 172.21.15.111:0/2150191867 conn(0x7f1d1c0ff4e0 msgr2=0x7f1d1c101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:03:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.110+0000 7f1d23c00640 1 -- 172.21.15.111:0/2150191867 shutdown_connections 2024-09-15T23:03:53.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.110+0000 7f1d23c00640 1 -- 172.21.15.111:0/2150191867 wait complete. 2024-09-15T23:03:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.111+0000 7f1d23c00640 1 Processor -- start 2024-09-15T23:03:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.112+0000 7f1d23c00640 1 -- start start 2024-09-15T23:03:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.112+0000 7f1d23c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d1c103d60 0x7f1d1c19e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.112+0000 7f1d23c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d1c104f50 0x7f1d1c19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:53.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.112+0000 7f1d23c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d1c105910 0x7f1d1c1a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.112+0000 7f1d23c00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c077080 con 0x7f1d1c103d60 2024-09-15T23:03:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.112+0000 7f1d23c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c076f00 con 0x7f1d1c105910 2024-09-15T23:03:53.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.112+0000 7f1d23c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c077200 con 0x7f1d1c104f50 2024-09-15T23:03:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.113+0000 7f1d21200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d1c103d60 0x7f1d1c19e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.113+0000 7f1d21c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d1c105910 0x7f1d1c1a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.113+0000 7f1d21200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d1c103d60 0x7f1d1c19e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:35498/0 (socket says 172.21.15.111:35498) 2024-09-15T23:03:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.113+0000 7f1d21200640 1 -- 172.21.15.111:0/2483431219 learned_addr learned my addr 172.21.15.111:0/2483431219 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:03:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.113+0000 7f1d1be00640 1 --2- 172.21.15.111:0/2483431219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d1c104f50 0x7f1d1c19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.113+0000 7f1d21c00640 1 -- 172.21.15.111:0/2483431219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d1c104f50 msgr2=0x7f1d1c19e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.113+0000 7f1d21c00640 1 --2- 172.21.15.111:0/2483431219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d1c104f50 0x7f1d1c19e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.113+0000 7f1d21c00640 1 -- 172.21.15.111:0/2483431219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d1c103d60 msgr2=0x7f1d1c19e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.113+0000 7f1d21c00640 1 --2- 172.21.15.111:0/2483431219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d1c103d60 0x7f1d1c19e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.113+0000 7f1d21c00640 1 -- 172.21.15.111:0/2483431219 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d1c1a5fb0 con 0x7f1d1c105910 2024-09-15T23:03:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.113+0000 7f1d21200640 1 --2- 172.21.15.111:0/2483431219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d1c103d60 0x7f1d1c19e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:03:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.114+0000 7f1d21c00640 1 --2- 172.21.15.111:0/2483431219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d1c105910 0x7f1d1c1a5940 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1d14009fd0 tx=0x7f1d1400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:03:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.114+0000 7f1d19600640 1 -- 172.21.15.111:0/2483431219 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1400ce20 con 0x7f1d1c105910 2024-09-15T23:03:53.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.115+0000 7f1d19600640 1 -- 172.21.15.111:0/2483431219 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d14005020 con 0x7f1d1c105910 2024-09-15T23:03:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.115+0000 7f1d19600640 1 -- 172.21.15.111:0/2483431219 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14010720 con 0x7f1d1c105910 2024-09-15T23:03:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.115+0000 7f1d23c00640 1 -- 172.21.15.111:0/2483431219 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d1c1a62a0 con 0x7f1d1c105910 2024-09-15T23:03:53.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.116+0000 7f1d23c00640 1 -- 172.21.15.111:0/2483431219 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d1c1a67c0 con 0x7f1d1c105910 2024-09-15T23:03:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.120+0000 7f1d26c00740 1 Processor -- start 2024-09-15T23:03:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.121+0000 7f1d19600640 1 -- 172.21.15.111:0/2483431219 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f1d140051d0 con 0x7f1d1c105910 2024-09-15T23:03:53.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.121+0000 7f1d26c00740 1 -- start start 2024-09-15T23:03:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.121+0000 7f1d26c00740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edefeb6090 0x55edefeb6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:53.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.121+0000 7f1d26c00740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edefeb69d0 0x55edefebb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.121+0000 7f1d26c00740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55edefebba60 0x55edefebdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.121+0000 7f1d26c00740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55edefe1b100 con 0x55edefebba60 2024-09-15T23:03:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.121+0000 7f1d26c00740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55edefdc31b0 con 0x55edefeb6090 2024-09-15T23:03:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.121+0000 7f1d26c00740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55edefe170c0 con 0x55edefeb69d0 2024-09-15T23:03:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.122+0000 7f1d1be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55edefebba60 0x55edefebdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.122+0000 7f1d1be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edefeb6090 0x55edefeb6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.122+0000 7f1d1be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edefeb6090 0x55edefeb6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:57472/0 (socket says 172.21.15.111:57472) 2024-09-15T23:03:53.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.122+0000 7f1d1be00640 1 -- 172.21.15.111:0/653944547 learned_addr learned my addr 172.21.15.111:0/653944547 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:03:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.123+0000 7f1d1be00640 1 -- 172.21.15.111:0/653944547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edefeb69d0 msgr2=0x55edefebb490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:03:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.123+0000 7f1d1be00640 1 --2- 172.21.15.111:0/653944547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edefeb69d0 0x55edefebb490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.123+0000 7f1d1be00640 1 -- 172.21.15.111:0/653944547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55edefebba60 msgr2=0x55edefebdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.123+0000 7f1d1be00640 1 --2- 172.21.15.111:0/653944547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55edefebba60 0x55edefebdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.123+0000 7f1d1be00640 1 -- 172.21.15.111:0/653944547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edefb2e210 con 0x55edefeb6090 2024-09-15T23:03:53.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.124+0000 7f1d1be00640 1 --2- 172.21.15.111:0/653944547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edefeb6090 0x55edefeb6490 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1d08005ec0 tx=0x7f1d0802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4692aba9747651 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:03:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.125+0000 7f1d0e000640 1 -- 172.21.15.111:0/653944547 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d08035070 con 0x55edefeb6090 2024-09-15T23:03:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.125+0000 7f1d0e000640 1 -- 172.21.15.111:0/653944547 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d080319d0 con 0x55edefeb6090 2024-09-15T23:03:53.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.125+0000 7f1d0e000640 1 -- 172.21.15.111:0/653944547 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d08004920 con 0x55edefeb6090 2024-09-15T23:03:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.125+0000 7f1d19600640 1 --2- 172.21.15.111:0/2483431219 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1d040a3330 0x7f1d040a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:53.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.125+0000 7f1d19600640 1 -- 172.21.15.111:0/2483431219 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f1d140cfe90 con 0x7f1d1c105910 2024-09-15T23:03:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.127+0000 7f1d26c00740 1 -- 172.21.15.111:0/653944547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edefeb6090 msgr2=0x55edefeb6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.127+0000 7f1d26c00740 1 --2- 172.21.15.111:0/653944547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edefeb6090 0x55edefeb6490 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1d08005ec0 tx=0x7f1d0802a410 comp rx=0 tx=0).stop 2024-09-15T23:03:53.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.127+0000 7f1d26c00740 1 -- 172.21.15.111:0/653944547 shutdown_connections 2024-09-15T23:03:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.127+0000 7f1d26c00740 1 --2- 172.21.15.111:0/653944547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55edefebba60 0x55edefebdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.127+0000 7f1d26c00740 1 --2- 172.21.15.111:0/653944547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edefeb69d0 0x55edefebb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.127+0000 7f1d26c00740 1 --2- 172.21.15.111:0/653944547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edefeb6090 0x55edefeb6490 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.127+0000 7f1d26c00740 1 -- 172.21.15.111:0/653944547 >> 172.21.15.111:0/653944547 conn(0x55edefeb1700 msgr2=0x55edefeb1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:03:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.127+0000 7f1d21200640 1 --2- 172.21.15.111:0/2483431219 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1d040a3330 0x7f1d040a57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:53.129 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.127+0000 7f1d26c00740 1 -- 172.21.15.111:0/653944547 shutdown_connections 2024-09-15T23:03:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.131+0000 7f1d26c00740 1 -- 172.21.15.111:0/653944547 wait complete. 2024-09-15T23:03:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.132+0000 7f1d26c00740 1 -- start start 2024-09-15T23:03:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.132+0000 7f1d26c00740 1 Processor -- start 2024-09-15T23:03:53.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.133+0000 7f1d21200640 1 --2- 172.21.15.111:0/2483431219 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1d040a3330 0x7f1d040a57f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1d10009fd0 tx=0x7f1d10005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:03:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.135+0000 7f1d26c00740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55edefeb6090 0x55edefec0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.135+0000 7f1d26c00740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edefeb69d0 0x55edefec07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.135+0000 7f1d26c00740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edefebba60 0x55edefec7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.135+0000 7f1d26c00740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55edefebe9d0 con 0x55edefeb6090 2024-09-15T23:03:53.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.135+0000 7f1d26c00740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55edefda4090 con 0x55edefeb69d0 2024-09-15T23:03:53.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.135+0000 7f1d26c00740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55edefebeb50 con 0x55edefebba60 2024-09-15T23:03:53.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.135+0000 7f1d26c00740 10 client.0 _ll_register_callbacks cb 0x55edefdf8f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:03:53.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.135+0000 7f1d21200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55edefeb6090 0x55edefec0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:53.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.135+0000 7f1d21200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55edefeb6090 0x55edefec0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:35526/0 (socket says 172.21.15.111:35526) 2024-09-15T23:03:53.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.135+0000 7f1d21c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edefebba60 0x55edefec7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:53.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.135+0000 7f1d21200640 1 -- 172.21.15.111:0/2911114755 learned_addr learned my addr 172.21.15.111:0/2911114755 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:03:53.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.135+0000 7f1d1be00640 1 --2- 172.21.15.111:0/2911114755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edefeb69d0 0x55edefec07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:53.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.136+0000 7f1d21200640 1 -- 172.21.15.111:0/2911114755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edefebba60 msgr2=0x55edefec7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:53.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.136+0000 7f1d21200640 1 --2- 172.21.15.111:0/2911114755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edefebba60 0x55edefec7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.136+0000 7f1d21200640 1 -- 172.21.15.111:0/2911114755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edefeb69d0 msgr2=0x55edefec07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:53.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.136+0000 7f1d21200640 1 --2- 172.21.15.111:0/2911114755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edefeb69d0 0x55edefec07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.136+0000 7f1d21200640 1 -- 172.21.15.111:0/2911114755 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edefbfa620 con 0x55edefeb6090 2024-09-15T23:03:53.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.136+0000 7f1d21c00640 1 --2- 172.21.15.111:0/2911114755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edefebba60 0x55edefec7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:03:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.136+0000 7f1d21200640 1 --2- 172.21.15.111:0/2911114755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55edefeb6090 0x55edefec0280 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1d1000fae0 tx=0x7f1d10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e1e02e5502eefd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:03:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.136+0000 7f1d02a00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:03:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.137+0000 7f1d02a00640 1 -- 172.21.15.111:0/2911114755 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1001cdd0 con 0x55edefeb6090 2024-09-15T23:03:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.137+0000 7f1d02a00640 1 -- 172.21.15.111:0/2911114755 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f1d1001d3f0 con 0x55edefeb6090 2024-09-15T23:03:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.137+0000 7f1d26c00740 1 -- 172.21.15.111:0/2911114755 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d080319d0 con 0x55edefeb6090 2024-09-15T23:03:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.137+0000 7f1d02a00640 1 -- 172.21.15.111:0/2911114755 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1d1001d5f0 con 0x55edefeb6090 2024-09-15T23:03:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.137+0000 7f1d26c00740 10 client.16425 fetch_fsmap learned FSMap version 35 2024-09-15T23:03:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.137+0000 7f1d26c00740 1 -- 172.21.15.111:0/2911114755 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55edefeca6c0 con 0x55edefeb6090 2024-09-15T23:03:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.138+0000 7f1d02a00640 1 -- 172.21.15.111:0/2911114755 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d080319d0 con 0x55edefeb6090 2024-09-15T23:03:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.138+0000 7f1d02a00640 1 -- 172.21.15.111:0/2911114755 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f1d1001a050 con 0x55edefeb6090 2024-09-15T23:03:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.138+0000 7f1d26c00740 10 client.16425 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:03:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.138+0000 7f1d26c00740 10 client.16425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:03:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.138+0000 7f1d26c00740 20 client.16425 populate_metadata read hostname 'smithi111' 2024-09-15T23:03:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.138+0000 7f1d26c00740 1 --2- 172.21.15.111:0/2911114755 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55edefecbd90 0x55edefece220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.138+0000 7f1d26c00740 4 client.16425 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:03:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.138+0000 7f1d26c00740 1 -- 172.21.15.111:0/2911114755 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edefe28830 con 0x55edefecbd90 2024-09-15T23:03:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.139+0000 7f1d1be00640 1 --2- 172.21.15.111:0/2911114755 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55edefecbd90 0x55edefece220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.140+0000 7f1d1be00640 1 --2- 172.21.15.111:0/2911114755 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55edefecbd90 0x55edefece220 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed5ef46434859686 server_cookie=8aa30a28815602f5 in_seq=0 out_seq=0 2024-09-15T23:03:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.140+0000 7f1d02a00640 10 client.16425 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:03:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.143+0000 7f1d02a00640 1 -- 172.21.15.111:0/2911114755 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55edefe28830 con 0x55edefecbd90 2024-09-15T23:03:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.143+0000 7f1d02a00640 10 client.16425 handle_command_reply: tid=0 2024-09-15T23:03:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.144+0000 7f1d26c00740 1 client.16425 shutdown 2024-09-15T23:03:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.144+0000 7f1d26c00740 1 -- 172.21.15.111:0/2911114755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55edefeb6090 msgr2=0x55edefec0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.144+0000 7f1d26c00740 1 --2- 172.21.15.111:0/2911114755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55edefeb6090 0x55edefec0280 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1d1000fae0 tx=0x7f1d10004730 comp rx=0 tx=0).stop 2024-09-15T23:03:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.144+0000 7f1d26c00740 1 -- 172.21.15.111:0/2911114755 shutdown_connections 2024-09-15T23:03:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.144+0000 7f1d26c00740 1 --2- 172.21.15.111:0/2911114755 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55edefecbd90 0x55edefece220 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.144+0000 7f1d26c00740 1 --2- 172.21.15.111:0/2911114755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edefebba60 0x55edefec7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.144+0000 7f1d26c00740 1 --2- 172.21.15.111:0/2911114755 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edefeb69d0 0x55edefec07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.144+0000 7f1d26c00740 1 --2- 172.21.15.111:0/2911114755 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55edefeb6090 0x55edefec0280 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.144+0000 7f1d26c00740 1 -- 172.21.15.111:0/2911114755 >> 172.21.15.111:0/2911114755 conn(0x55edefeb1700 msgr2=0x55edefeb2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:03:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.144+0000 7f1d02a00640 0 client.16425 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:03:53.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.145+0000 7f1d26c00740 1 -- 172.21.15.111:0/2911114755 shutdown_connections 2024-09-15T23:03:53.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.145+0000 7f1d26c00740 1 -- 172.21.15.111:0/2911114755 wait complete. 2024-09-15T23:03:53.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.145+0000 7f1d26c00740 20 client.16425 trim_cache size 0 max 16384 2024-09-15T23:03:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.182+0000 7f1d26c00740 1 Processor -- start 2024-09-15T23:03:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.182+0000 7f1d26c00740 1 -- start start 2024-09-15T23:03:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.182+0000 7f1d26c00740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55edefeb6090 0x55edefec0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.182+0000 7f1d26c00740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edefeb69d0 0x55edefec7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.182+0000 7f1d26c00740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edefebba60 0x55edefeb2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.182+0000 7f1d26c00740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c1b1b20 con 0x55edefeb6090 2024-09-15T23:03:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.182+0000 7f1d26c00740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c1af3f0 con 0x55edefeb69d0 2024-09-15T23:03:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.182+0000 7f1d26c00740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c1ad240 con 0x55edefebba60 2024-09-15T23:03:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.182+0000 7f1d1be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edefeb69d0 0x55edefec7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.182+0000 7f1d1be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edefeb69d0 0x55edefec7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:57492/0 (socket says 172.21.15.111:57492) 2024-09-15T23:03:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.182+0000 7f1d1be00640 1 -- 172.21.15.111:0/1760970326 learned_addr learned my addr 172.21.15.111:0/1760970326 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:03:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.182+0000 7f1d21200640 1 --2- 172.21.15.111:0/1760970326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55edefeb6090 0x55edefec0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.183+0000 7f1d21c00640 1 --2- 172.21.15.111:0/1760970326 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edefebba60 0x55edefeb2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.183+0000 7f1d1be00640 1 -- 172.21.15.111:0/1760970326 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edefebba60 msgr2=0x55edefeb2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.183+0000 7f1d1be00640 1 --2- 172.21.15.111:0/1760970326 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edefebba60 0x55edefeb2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.183+0000 7f1d1be00640 1 -- 172.21.15.111:0/1760970326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55edefeb6090 msgr2=0x55edefec0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.183+0000 7f1d1be00640 1 --2- 172.21.15.111:0/1760970326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55edefeb6090 0x55edefec0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.183+0000 7f1d1be00640 1 -- 172.21.15.111:0/1760970326 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edefeb1520 con 0x55edefeb69d0 2024-09-15T23:03:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.183+0000 7f1d1be00640 1 --2- 172.21.15.111:0/1760970326 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edefeb69d0 0x55edefec7720 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1d08006010 tx=0x7f1d080058f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d559a9548db9ca5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:03:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.183+0000 7f1d0d600640 1 -- 172.21.15.111:0/1760970326 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d08035030 con 0x55edefeb69d0 2024-09-15T23:03:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.183+0000 7f1d0d600640 1 -- 172.21.15.111:0/1760970326 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d0804d030 con 0x55edefeb69d0 2024-09-15T23:03:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.184+0000 7f1d0d600640 1 -- 172.21.15.111:0/1760970326 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d08004dd0 con 0x55edefeb69d0 2024-09-15T23:03:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.184+0000 7f1d26c00740 1 -- 172.21.15.111:0/1760970326 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edefeb69d0 msgr2=0x55edefec7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.184+0000 7f1d26c00740 1 --2- 172.21.15.111:0/1760970326 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edefeb69d0 0x55edefec7720 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1d08006010 tx=0x7f1d080058f0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.184+0000 7f1d26c00740 1 -- 172.21.15.111:0/1760970326 shutdown_connections 2024-09-15T23:03:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.184+0000 7f1d26c00740 1 --2- 172.21.15.111:0/1760970326 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edefebba60 0x55edefeb2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.184+0000 7f1d26c00740 1 --2- 172.21.15.111:0/1760970326 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edefeb69d0 0x55edefec7720 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.184+0000 7f1d26c00740 1 --2- 172.21.15.111:0/1760970326 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55edefeb6090 0x55edefec0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.184+0000 7f1d26c00740 1 -- 172.21.15.111:0/1760970326 >> 172.21.15.111:0/1760970326 conn(0x55edefeb1700 msgr2=0x55edefec0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:03:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.185+0000 7f1d26c00740 1 -- 172.21.15.111:0/1760970326 shutdown_connections 2024-09-15T23:03:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.185+0000 7f1d26c00740 1 -- 172.21.15.111:0/1760970326 wait complete. 2024-09-15T23:03:53.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.185+0000 7f1d26c00740 1 -- start start 2024-09-15T23:03:53.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.185+0000 7f1d26c00740 1 Processor -- start 2024-09-15T23:03:53.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.186+0000 7f1d26c00740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55edefeb6090 0x55edefec4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:53.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.186+0000 7f1d26c00740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edefeb69d0 0x55edefec49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:53.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.186+0000 7f1d26c00740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edefebba60 0x55edeff34af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:53.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.187+0000 7f1d26c00740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c1ad8b0 con 0x55edefeb6090 2024-09-15T23:03:53.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.187+0000 7f1d26c00740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c1aed20 con 0x55edefebba60 2024-09-15T23:03:53.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.187+0000 7f1d21200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55edefeb6090 0x55edefec4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:53.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.187+0000 7f1d26c00740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c1adf80 con 0x55edefeb69d0 2024-09-15T23:03:53.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.187+0000 7f1d1be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edefeb69d0 0x55edefec49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:53.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.187+0000 7f1d1be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edefeb69d0 0x55edefec49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:52044/0 (socket says 172.21.15.111:52044) 2024-09-15T23:03:53.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.187+0000 7f1d1be00640 1 -- 172.21.15.111:0/2141190409 learned_addr learned my addr 172.21.15.111:0/2141190409 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:03:53.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.187+0000 7f1d26c00740 10 client.0 _ll_register_callbacks cb 0x7f1d10009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:03:53.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.187+0000 7f1d21c00640 1 --2- 172.21.15.111:0/2141190409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edefebba60 0x55edeff34af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:53.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.187+0000 7f1d1be00640 1 -- 172.21.15.111:0/2141190409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edefebba60 msgr2=0x55edeff34af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:53.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.187+0000 7f1d1be00640 1 --2- 172.21.15.111:0/2141190409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edefebba60 0x55edeff34af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.187+0000 7f1d1be00640 1 -- 172.21.15.111:0/2141190409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55edefeb6090 msgr2=0x55edefec4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:03:53.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.188+0000 7f1d1be00640 1 --2- 172.21.15.111:0/2141190409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55edefeb6090 0x55edefec4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.188+0000 7f1d1be00640 1 -- 172.21.15.111:0/2141190409 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edefec7ac0 con 0x55edefeb69d0 2024-09-15T23:03:53.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.188+0000 7f1d21200640 1 --2- 172.21.15.111:0/2141190409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55edefeb6090 0x55edefec4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:03:53.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.188+0000 7f1d1be00640 1 --2- 172.21.15.111:0/2141190409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edefeb69d0 0x55edefec49d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1d08031bf0 tx=0x7f1d080316a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e903844027d6d6f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:03:53.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.188+0000 7f1d02a00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:03:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.188+0000 7f1d02a00640 1 -- 172.21.15.111:0/2141190409 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d08035040 con 0x55edefeb69d0 2024-09-15T23:03:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.188+0000 7f1d02a00640 1 -- 172.21.15.111:0/2141190409 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f1d08004b50 con 0x55edefeb69d0 2024-09-15T23:03:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.189+0000 7f1d26c00740 1 -- 172.21.15.111:0/2141190409 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d0804d030 con 0x55edefeb69d0 2024-09-15T23:03:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.189+0000 7f1d02a00640 1 -- 172.21.15.111:0/2141190409 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1d08004d50 con 0x55edefeb69d0 2024-09-15T23:03:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.189+0000 7f1d26c00740 10 client.26392 fetch_fsmap learned FSMap version 35 2024-09-15T23:03:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.189+0000 7f1d26c00740 1 -- 172.21.15.111:0/2141190409 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55edeff35ff0 con 0x55edefeb69d0 2024-09-15T23:03:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.189+0000 7f1d02a00640 1 -- 172.21.15.111:0/2141190409 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d0804d030 con 0x55edefeb69d0 2024-09-15T23:03:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.190+0000 7f1d02a00640 1 -- 172.21.15.111:0/2141190409 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f1d0803d7a0 con 0x55edefeb69d0 2024-09-15T23:03:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.190+0000 7f1d26c00740 10 client.26392 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:03:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.190+0000 7f1d26c00740 10 client.26392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:03:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.190+0000 7f1d26c00740 20 client.26392 populate_metadata read hostname 'smithi111' 2024-09-15T23:03:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.190+0000 7f1d26c00740 1 --2- 172.21.15.111:0/2141190409 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55edeff36440 0x55edeff38830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:03:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.190+0000 7f1d26c00740 4 client.26392 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:03:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.190+0000 7f1d26c00740 1 -- 172.21.15.111:0/2141190409 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55edefe22b80 con 0x55edeff36440 2024-09-15T23:03:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.191+0000 7f1d21200640 1 --2- 172.21.15.111:0/2141190409 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55edeff36440 0x55edeff38830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:03:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.191+0000 7f1d21200640 1 --2- 172.21.15.111:0/2141190409 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55edeff36440 0x55edeff38830 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35e2fe3ebbb875c9 server_cookie=3d3e673aa2ddca2d in_seq=0 out_seq=0 2024-09-15T23:03:53.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.191+0000 7f1d02a00640 10 client.26392 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:03:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.486+0000 7f1d02a00640 1 -- 172.21.15.111:0/2141190409 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55edefe22b80 con 0x55edeff36440 2024-09-15T23:03:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.486+0000 7f1d02a00640 10 client.26392 handle_command_reply: tid=0 2024-09-15T23:03:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.486+0000 7f1d26c00740 1 client.26392 shutdown 2024-09-15T23:03:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.487+0000 7f1d26c00740 1 -- 172.21.15.111:0/2141190409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edefeb69d0 msgr2=0x55edefec49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.487+0000 7f1d26c00740 1 --2- 172.21.15.111:0/2141190409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edefeb69d0 0x55edefec49d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1d08031bf0 tx=0x7f1d080316a0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.487+0000 7f1d26c00740 1 -- 172.21.15.111:0/2141190409 shutdown_connections 2024-09-15T23:03:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.487+0000 7f1d26c00740 1 --2- 172.21.15.111:0/2141190409 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55edeff36440 0x55edeff38830 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.487+0000 7f1d26c00740 1 --2- 172.21.15.111:0/2141190409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55edefebba60 0x55edeff34af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.487+0000 7f1d26c00740 1 --2- 172.21.15.111:0/2141190409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55edefeb69d0 0x55edefec49d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.487+0000 7f1d26c00740 1 --2- 172.21.15.111:0/2141190409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55edefeb6090 0x55edefec4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.487+0000 7f1d26c00740 1 -- 172.21.15.111:0/2141190409 >> 172.21.15.111:0/2141190409 conn(0x55edefeb1700 msgr2=0x55edefeb0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:03:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.487+0000 7f1d02a00640 0 client.26392 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:03:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.487+0000 7f1d26c00740 1 -- 172.21.15.111:0/2141190409 shutdown_connections 2024-09-15T23:03:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.487+0000 7f1d26c00740 1 -- 172.21.15.111:0/2141190409 wait complete. 2024-09-15T23:03:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.487+0000 7f1d26c00740 20 client.26392 trim_cache size 0 max 16384 2024-09-15T23:03:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:03:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:03:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:03:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:03:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.488+0000 7f1d23c00640 1 -- 172.21.15.111:0/2483431219 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1d040a3330 msgr2=0x7f1d040a57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.488+0000 7f1d23c00640 1 --2- 172.21.15.111:0/2483431219 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1d040a3330 0x7f1d040a57f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1d10009fd0 tx=0x7f1d10005f90 comp rx=0 tx=0).stop 2024-09-15T23:03:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.488+0000 7f1d23c00640 1 -- 172.21.15.111:0/2483431219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d1c105910 msgr2=0x7f1d1c1a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:03:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.488+0000 7f1d23c00640 1 --2- 172.21.15.111:0/2483431219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d1c105910 0x7f1d1c1a5940 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1d14009fd0 tx=0x7f1d1400ef10 comp rx=0 tx=0).stop 2024-09-15T23:03:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.488+0000 7f1d23c00640 1 -- 172.21.15.111:0/2483431219 shutdown_connections 2024-09-15T23:03:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.488+0000 7f1d23c00640 1 --2- 172.21.15.111:0/2483431219 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f1d040a3330 0x7f1d040a57f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.488+0000 7f1d23c00640 1 --2- 172.21.15.111:0/2483431219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d1c105910 0x7f1d1c1a5940 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.488+0000 7f1d23c00640 1 --2- 172.21.15.111:0/2483431219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d1c104f50 0x7f1d1c19e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.488+0000 7f1d23c00640 1 --2- 172.21.15.111:0/2483431219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1d1c103d60 0x7f1d1c19e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:03:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.488+0000 7f1d23c00640 1 -- 172.21.15.111:0/2483431219 >> 172.21.15.111:0/2483431219 conn(0x7f1d1c0ff4e0 msgr2=0x7f1d1c079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:03:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.489+0000 7f1d23c00640 1 -- 172.21.15.111:0/2483431219 shutdown_connections 2024-09-15T23:03:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:03:53.489+0000 7f1d23c00640 1 -- 172.21.15.111:0/2483431219 wait complete. 2024-09-15T23:03:53.643 INFO:tasks.workunit.client.0.smithi111.stdout: 10 96652 18.49 MB/sec execute 141 sec latency 1314.625 ms 2024-09-15T23:03:54.250 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:53 smithi111 ceph-mon[30629]: pgmap v890: 97 pgs: 97 active+clean; 532 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 4.3 MiB/s wr, 554 op/s 2024-09-15T23:03:54.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:53 smithi192 ceph-mon[35427]: pgmap v890: 97 pgs: 97 active+clean; 532 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 4.3 MiB/s wr, 554 op/s 2024-09-15T23:03:54.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:53 smithi121 ceph-mon[34815]: pgmap v890: 97 pgs: 97 active+clean; 532 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 4.3 MiB/s wr, 554 op/s 2024-09-15T23:03:54.643 INFO:tasks.workunit.client.0.smithi111.stdout: 10 96959 18.48 MB/sec execute 142 sec latency 1971.393 ms 2024-09-15T23:03:55.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:55 smithi192 ceph-mon[35427]: pgmap v891: 97 pgs: 97 active+clean; 537 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.5 MiB/s wr, 540 op/s 2024-09-15T23:03:55.598 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:55 smithi111 ceph-mon[30629]: pgmap v891: 97 pgs: 97 active+clean; 537 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.5 MiB/s wr, 540 op/s 2024-09-15T23:03:55.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:55 smithi121 ceph-mon[34815]: pgmap v891: 97 pgs: 97 active+clean; 537 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.5 MiB/s wr, 540 op/s 2024-09-15T23:03:55.642 INFO:tasks.workunit.client.0.smithi111.stdout: 10 97729 18.63 MB/sec execute 143 sec latency 74.717 ms 2024-09-15T23:03:56.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:56 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:03:56.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:56 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:03:56.642 INFO:tasks.workunit.client.0.smithi111.stdout: 10 98620 18.79 MB/sec execute 144 sec latency 75.717 ms 2024-09-15T23:03:56.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:56 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:03:57.532 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:03:57.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:03:57.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:03:57.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:03:57.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:57 smithi111 ceph-mon[30629]: Adjusting osd_memory_target on smithi192 to 176.0M 2024-09-15T23:03:57.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:57 smithi111 ceph-mon[30629]: Unable to set osd_memory_target on smithi192 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-15T23:03:57.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:03:57.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:03:57.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:03:57.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:03:57.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:57 smithi111 ceph-mon[30629]: Adjusting osd_memory_target on smithi121 to 175.9M 2024-09-15T23:03:57.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:57 smithi111 ceph-mon[30629]: Unable to set osd_memory_target on smithi121 to 184532275: error parsing value: Value '184532275' is below minimum 939524096 2024-09-15T23:03:57.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:57 smithi111 ceph-mon[30629]: pgmap v892: 97 pgs: 97 active+clean; 537 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.6 MiB/s wr, 496 op/s 2024-09-15T23:03:57.533 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:03:57.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:03:57.534 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:03:57.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:03:57.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:03:57.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:03:57.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:03:57.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:57 smithi192 ceph-mon[35427]: Adjusting osd_memory_target on smithi192 to 176.0M 2024-09-15T23:03:57.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:57 smithi192 ceph-mon[35427]: Unable to set osd_memory_target on smithi192 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-15T23:03:57.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:03:57.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:03:57.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:03:57.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:03:57.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:57 smithi192 ceph-mon[35427]: Adjusting osd_memory_target on smithi121 to 175.9M 2024-09-15T23:03:57.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:57 smithi192 ceph-mon[35427]: Unable to set osd_memory_target on smithi121 to 184532275: error parsing value: Value '184532275' is below minimum 939524096 2024-09-15T23:03:57.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:57 smithi192 ceph-mon[35427]: pgmap v892: 97 pgs: 97 active+clean; 537 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.6 MiB/s wr, 496 op/s 2024-09-15T23:03:57.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:03:57.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:03:57.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:03:57.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:03:57.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:03:57.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:03:57.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:03:57.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:57 smithi121 ceph-mon[34815]: Adjusting osd_memory_target on smithi192 to 176.0M 2024-09-15T23:03:57.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:57 smithi121 ceph-mon[34815]: Unable to set osd_memory_target on smithi192 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-15T23:03:57.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:03:57.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:03:57.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:03:57.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T23:03:57.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:57 smithi121 ceph-mon[34815]: Adjusting osd_memory_target on smithi121 to 175.9M 2024-09-15T23:03:57.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:57 smithi121 ceph-mon[34815]: Unable to set osd_memory_target on smithi121 to 184532275: error parsing value: Value '184532275' is below minimum 939524096 2024-09-15T23:03:57.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:57 smithi121 ceph-mon[34815]: pgmap v892: 97 pgs: 97 active+clean; 537 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.6 MiB/s wr, 496 op/s 2024-09-15T23:03:57.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:03:57.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:03:57.641 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:03:57.645 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99190 18.95 MB/sec execute 145 sec latency 252.431 ms 2024-09-15T23:03:58.645 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99194 18.82 MB/sec execute 146 sec latency 1236.509 ms 2024-09-15T23:03:59.645 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99194 18.69 MB/sec execute 147 sec latency 2236.556 ms 2024-09-15T23:04:00.102 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:03:59 smithi111 ceph-mon[30629]: pgmap v893: 97 pgs: 97 active+clean; 547 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.1 MiB/s wr, 577 op/s 2024-09-15T23:04:00.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:03:59 smithi192 ceph-mon[35427]: pgmap v893: 97 pgs: 97 active+clean; 547 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.1 MiB/s wr, 577 op/s 2024-09-15T23:04:00.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:03:59 smithi121 ceph-mon[34815]: pgmap v893: 97 pgs: 97 active+clean; 547 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.1 MiB/s wr, 577 op/s 2024-09-15T23:04:00.645 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99194 18.56 MB/sec execute 148 sec latency 3236.618 ms 2024-09-15T23:04:01.645 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99207 18.46 MB/sec execute 149 sec latency 4236.704 ms 2024-09-15T23:04:02.113 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:01 smithi111 ceph-mon[30629]: pgmap v894: 97 pgs: 97 active+clean; 552 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.6 MiB/s wr, 545 op/s 2024-09-15T23:04:02.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:01 smithi192 ceph-mon[35427]: pgmap v894: 97 pgs: 97 active+clean; 552 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.6 MiB/s wr, 545 op/s 2024-09-15T23:04:02.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:01 smithi121 ceph-mon[34815]: pgmap v894: 97 pgs: 97 active+clean; 552 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.6 MiB/s wr, 545 op/s 2024-09-15T23:04:02.645 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99207 18.33 MB/sec execute 150 sec latency 5236.756 ms 2024-09-15T23:04:03.645 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99207 18.21 MB/sec execute 151 sec latency 6236.808 ms 2024-09-15T23:04:04.113 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:03 smithi111 ceph-mon[30629]: pgmap v895: 97 pgs: 97 active+clean; 549 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.8 MiB/s wr, 490 op/s 2024-09-15T23:04:04.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:03 smithi192 ceph-mon[35427]: pgmap v895: 97 pgs: 97 active+clean; 549 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.8 MiB/s wr, 490 op/s 2024-09-15T23:04:04.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:03 smithi121 ceph-mon[34815]: pgmap v895: 97 pgs: 97 active+clean; 549 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 6.8 MiB/s wr, 490 op/s 2024-09-15T23:04:04.646 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99207 18.09 MB/sec execute 152 sec latency 7236.851 ms 2024-09-15T23:04:05.647 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99210 17.98 MB/sec execute 153 sec latency 8214.486 ms 2024-09-15T23:04:06.098 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:05 smithi111 ceph-mon[30629]: pgmap v896: 97 pgs: 97 active+clean; 548 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.0 MiB/s wr, 466 op/s 2024-09-15T23:04:06.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:05 smithi192 ceph-mon[35427]: pgmap v896: 97 pgs: 97 active+clean; 548 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.0 MiB/s wr, 466 op/s 2024-09-15T23:04:06.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:05 smithi121 ceph-mon[34815]: pgmap v896: 97 pgs: 97 active+clean; 548 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.0 MiB/s wr, 466 op/s 2024-09-15T23:04:06.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.602+0000 7f0393200640 1 -- 172.21.15.111:0/3228117222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0394074680 msgr2=0x7f0394074a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.602+0000 7f0393200640 1 --2- 172.21.15.111:0/3228117222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0394074680 0x7f0394074a80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0384009900 tx=0x7f038402a3e0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0393200640 1 -- 172.21.15.111:0/3228117222 shutdown_connections 2024-09-15T23:04:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0393200640 1 --2- 172.21.15.111:0/3228117222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0394109c90 0x7f039410c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0393200640 1 --2- 172.21.15.111:0/3228117222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0394074fc0 0x7f0394109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0393200640 1 --2- 172.21.15.111:0/3228117222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0394074680 0x7f0394074a80 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0393200640 1 -- 172.21.15.111:0/3228117222 >> 172.21.15.111:0/3228117222 conn(0x7f03940fd0d0 msgr2=0x7f03940ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:04:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0393200640 1 -- 172.21.15.111:0/3228117222 shutdown_connections 2024-09-15T23:04:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0393200640 1 -- 172.21.15.111:0/3228117222 wait complete. 2024-09-15T23:04:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0393200640 1 Processor -- start 2024-09-15T23:04:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0393200640 1 -- start start 2024-09-15T23:04:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0393200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0394074680 0x7f0394199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0393200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0394074fc0 0x7f039419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0393200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0394109c90 0x7f03941a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0393200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f039410f8f0 con 0x7f0394074fc0 2024-09-15T23:04:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0393200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f039410f770 con 0x7f0394109c90 2024-09-15T23:04:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0393200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f039410fa70 con 0x7f0394074680 2024-09-15T23:04:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0391400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0394074fc0 0x7f039419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0391e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0394074680 0x7f0394199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0391400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0394074fc0 0x7f039419a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:37642/0 (socket says 172.21.15.111:37642) 2024-09-15T23:04:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0391400640 1 -- 172.21.15.111:0/1219347722 learned_addr learned my addr 172.21.15.111:0/1219347722 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:04:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0392800640 1 --2- 172.21.15.111:0/1219347722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0394109c90 0x7f03941a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0391400640 1 -- 172.21.15.111:0/1219347722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0394074680 msgr2=0x7f0394199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0391400640 1 --2- 172.21.15.111:0/1219347722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0394074680 0x7f0394199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0391400640 1 -- 172.21.15.111:0/1219347722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0394109c90 msgr2=0x7f03941a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:06.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0391400640 1 --2- 172.21.15.111:0/1219347722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0394109c90 0x7f03941a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0391400640 1 -- 172.21.15.111:0/1219347722 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03941a1bd0 con 0x7f0394074fc0 2024-09-15T23:04:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0391400640 1 --2- 172.21.15.111:0/1219347722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0394074fc0 0x7f039419a4f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f038000e9f0 tx=0x7f038000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:04:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f038aa00640 1 -- 172.21.15.111:0/1219347722 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038000cd80 con 0x7f0394074fc0 2024-09-15T23:04:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f038aa00640 1 -- 172.21.15.111:0/1219347722 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0380005500 con 0x7f0394074fc0 2024-09-15T23:04:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0393200640 1 -- 172.21.15.111:0/1219347722 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03941a1ec0 con 0x7f0394074fc0 2024-09-15T23:04:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f038aa00640 1 -- 172.21.15.111:0/1219347722 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0380010690 con 0x7f0394074fc0 2024-09-15T23:04:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.603+0000 7f0393200640 1 -- 172.21.15.111:0/1219347722 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03941a2330 con 0x7f0394074fc0 2024-09-15T23:04:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.606+0000 7f038aa00640 1 -- 172.21.15.111:0/1219347722 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f03800046c0 con 0x7f0394074fc0 2024-09-15T23:04:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.606+0000 7f039bbdf740 1 Processor -- start 2024-09-15T23:04:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.606+0000 7f039bbdf740 1 -- start start 2024-09-15T23:04:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.606+0000 7f039bbdf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610eb263260 0x5610eb263660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.606+0000 7f039bbdf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610eb263ba0 0x5610eb268660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.606+0000 7f039bbdf740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5610eb268c30 0x5610eb26b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.606+0000 7f039bbdf740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5610eb1c8100 con 0x5610eb263260 2024-09-15T23:04:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.606+0000 7f039bbdf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5610eb1701b0 con 0x5610eb263ba0 2024-09-15T23:04:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.606+0000 7f039bbdf740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5610eb1c40c0 con 0x5610eb268c30 2024-09-15T23:04:06.613 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.606+0000 7f0392800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610eb263260 0x5610eb263660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.606+0000 7f0391400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5610eb268c30 0x5610eb26b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.606+0000 7f0391e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610eb263ba0 0x5610eb268660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.606+0000 7f0391400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5610eb268c30 0x5610eb26b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:34716/0 (socket says 172.21.15.111:34716) 2024-09-15T23:04:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.606+0000 7f0391400640 1 -- 172.21.15.111:0/1096780149 learned_addr learned my addr 172.21.15.111:0/1096780149 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:04:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f0391e00640 1 -- 172.21.15.111:0/1096780149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5610eb268c30 msgr2=0x5610eb26b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f0391e00640 1 --2- 172.21.15.111:0/1096780149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5610eb268c30 0x5610eb26b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f0391e00640 1 -- 172.21.15.111:0/1096780149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610eb263260 msgr2=0x5610eb263660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f0391e00640 1 --2- 172.21.15.111:0/1096780149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610eb263260 0x5610eb263660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f0391e00640 1 -- 172.21.15.111:0/1096780149 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610eaedb210 con 0x5610eb263ba0 2024-09-15T23:04:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f038aa00640 1 --2- 172.21.15.111:0/1219347722 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f03780a3240 0x7f03780a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f0391400640 1 --2- 172.21.15.111:0/1096780149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5610eb268c30 0x5610eb26b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:04:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f038aa00640 1 -- 172.21.15.111:0/1219347722 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f03800cfc60 con 0x7f0394074fc0 2024-09-15T23:04:06.614 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f0392800640 1 --2- 172.21.15.111:0/1219347722 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f03780a3240 0x7f03780a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f0391e00640 1 --2- 172.21.15.111:0/1096780149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610eb263ba0 0x5610eb268660 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0384031440 tx=0x7f0384005e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dc4a9efddcfa19f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:04:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f037f400640 1 -- 172.21.15.111:0/1096780149 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0384040070 con 0x5610eb263ba0 2024-09-15T23:04:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f037f400640 1 -- 172.21.15.111:0/1096780149 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0384004030 con 0x5610eb263ba0 2024-09-15T23:04:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f037f400640 1 -- 172.21.15.111:0/1096780149 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0384035040 con 0x5610eb263ba0 2024-09-15T23:04:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f039bbdf740 1 -- 172.21.15.111:0/1096780149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610eb263ba0 msgr2=0x5610eb268660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f0392800640 1 --2- 172.21.15.111:0/1219347722 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f03780a3240 0x7f03780a5700 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f038c005fd0 tx=0x7f038c009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:04:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f039bbdf740 1 --2- 172.21.15.111:0/1096780149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610eb263ba0 0x5610eb268660 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0384031440 tx=0x7f0384005e00 comp rx=0 tx=0).stop 2024-09-15T23:04:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f039bbdf740 1 -- 172.21.15.111:0/1096780149 shutdown_connections 2024-09-15T23:04:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f039bbdf740 1 --2- 172.21.15.111:0/1096780149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5610eb268c30 0x5610eb26b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f039bbdf740 1 --2- 172.21.15.111:0/1096780149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610eb263ba0 0x5610eb268660 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f039bbdf740 1 --2- 172.21.15.111:0/1096780149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610eb263260 0x5610eb263660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f039bbdf740 1 -- 172.21.15.111:0/1096780149 >> 172.21.15.111:0/1096780149 conn(0x5610eb25e8d0 msgr2=0x5610eb25ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:04:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f039bbdf740 1 -- 172.21.15.111:0/1096780149 shutdown_connections 2024-09-15T23:04:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f039bbdf740 1 -- 172.21.15.111:0/1096780149 wait complete. 2024-09-15T23:04:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f039bbdf740 1 -- start start 2024-09-15T23:04:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f039bbdf740 1 Processor -- start 2024-09-15T23:04:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f039bbdf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610eb263260 0x5610eb26d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f039bbdf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610eb263ba0 0x5610eb26d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f039bbdf740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5610eb268c30 0x5610eb274910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f039bbdf740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5610eb26bba0 con 0x5610eb263260 2024-09-15T23:04:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f039bbdf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5610eb151090 con 0x5610eb263ba0 2024-09-15T23:04:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f039bbdf740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5610eb26bd20 con 0x5610eb268c30 2024-09-15T23:04:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f039bbdf740 10 client.0 _ll_register_callbacks cb 0x5610eb1a5f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:04:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f0391400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610eb263ba0 0x5610eb26d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f0392800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5610eb268c30 0x5610eb274910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f0391400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610eb263ba0 0x5610eb26d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:37490/0 (socket says 172.21.15.111:37490) 2024-09-15T23:04:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f0391400640 1 -- 172.21.15.111:0/2805061028 learned_addr learned my addr 172.21.15.111:0/2805061028 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:04:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f0391e00640 1 --2- 172.21.15.111:0/2805061028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610eb263260 0x5610eb26d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f0391400640 1 -- 172.21.15.111:0/2805061028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5610eb268c30 msgr2=0x5610eb274910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f0391400640 1 --2- 172.21.15.111:0/2805061028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5610eb268c30 0x5610eb274910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f0391400640 1 -- 172.21.15.111:0/2805061028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610eb263260 msgr2=0x5610eb26d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f0391400640 1 --2- 172.21.15.111:0/2805061028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610eb263260 0x5610eb26d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f0391400640 1 -- 172.21.15.111:0/2805061028 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610eafa7620 con 0x5610eb263ba0 2024-09-15T23:04:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f0391400640 1 --2- 172.21.15.111:0/2805061028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610eb263ba0 0x5610eb26d8d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f03800a4a40 tx=0x7f0380009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f99e8cc649e52ae server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:04:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f0373e00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:04:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f0373e00640 1 -- 172.21.15.111:0/2805061028 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038009c0d0 con 0x5610eb263ba0 2024-09-15T23:04:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f0373e00640 1 -- 172.21.15.111:0/2805061028 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f03800a4ae0 con 0x5610eb263ba0 2024-09-15T23:04:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f039bbdf740 1 -- 172.21.15.111:0/2805061028 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0384004030 con 0x5610eb263ba0 2024-09-15T23:04:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f0373e00640 1 -- 172.21.15.111:0/2805061028 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f038009bc60 con 0x5610eb263ba0 2024-09-15T23:04:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f039bbdf740 10 client.26390 fetch_fsmap learned FSMap version 35 2024-09-15T23:04:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f039bbdf740 1 -- 172.21.15.111:0/2805061028 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5610eb2777d0 con 0x5610eb263ba0 2024-09-15T23:04:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f0373e00640 1 -- 172.21.15.111:0/2805061028 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0384004030 con 0x5610eb263ba0 2024-09-15T23:04:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f0373e00640 1 -- 172.21.15.111:0/2805061028 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f03800d5070 con 0x5610eb263ba0 2024-09-15T23:04:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f039bbdf740 10 client.26390 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:04:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f039bbdf740 10 client.26390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:04:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f039bbdf740 20 client.26390 populate_metadata read hostname 'smithi111' 2024-09-15T23:04:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f039bbdf740 1 --2- 172.21.15.111:0/2805061028 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5610eb278ea0 0x5610eb27b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f039bbdf740 4 client.26390 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:04:06.621 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.612+0000 7f039bbdf740 1 -- 172.21.15.111:0/2805061028 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610eb1d5830 con 0x5610eb278ea0 2024-09-15T23:04:06.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.617+0000 7f0391e00640 1 --2- 172.21.15.111:0/2805061028 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5610eb278ea0 0x5610eb27b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:06.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.617+0000 7f0391e00640 1 --2- 172.21.15.111:0/2805061028 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5610eb278ea0 0x5610eb27b290 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecc11dafc2072837 server_cookie=916c81300f7b4896 in_seq=0 out_seq=0 2024-09-15T23:04:06.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.617+0000 7f0373e00640 10 client.26390 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:04:06.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.617+0000 7f0373e00640 1 -- 172.21.15.111:0/2805061028 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5610eb1d5830 con 0x5610eb278ea0 2024-09-15T23:04:06.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.617+0000 7f0373e00640 10 client.26390 handle_command_reply: tid=0 2024-09-15T23:04:06.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.618+0000 7f039bbdf740 1 client.26390 shutdown 2024-09-15T23:04:06.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.618+0000 7f039bbdf740 1 -- 172.21.15.111:0/2805061028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610eb263ba0 msgr2=0x5610eb26d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.618+0000 7f039bbdf740 1 --2- 172.21.15.111:0/2805061028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610eb263ba0 0x5610eb26d8d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f03800a4a40 tx=0x7f0380009310 comp rx=0 tx=0).stop 2024-09-15T23:04:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.618+0000 7f039bbdf740 1 -- 172.21.15.111:0/2805061028 shutdown_connections 2024-09-15T23:04:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.618+0000 7f039bbdf740 1 --2- 172.21.15.111:0/2805061028 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5610eb278ea0 0x5610eb27b290 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.618+0000 7f039bbdf740 1 --2- 172.21.15.111:0/2805061028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5610eb268c30 0x5610eb274910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.618+0000 7f039bbdf740 1 --2- 172.21.15.111:0/2805061028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610eb263ba0 0x5610eb26d8d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.618+0000 7f0373e00640 0 client.26390 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:04:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.618+0000 7f039bbdf740 1 --2- 172.21.15.111:0/2805061028 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610eb263260 0x5610eb26d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.618+0000 7f039bbdf740 1 -- 172.21.15.111:0/2805061028 >> 172.21.15.111:0/2805061028 conn(0x5610eb25e8d0 msgr2=0x5610eb2600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:04:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.618+0000 7f039bbdf740 1 -- 172.21.15.111:0/2805061028 shutdown_connections 2024-09-15T23:04:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.618+0000 7f039bbdf740 1 -- 172.21.15.111:0/2805061028 wait complete. 2024-09-15T23:04:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.618+0000 7f039bbdf740 20 client.26390 trim_cache size 0 max 16384 2024-09-15T23:04:06.645 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99224 17.86 MB/sec execute 154 sec latency 9206.335 ms 2024-09-15T23:04:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.646+0000 7f039bbdf740 1 Processor -- start 2024-09-15T23:04:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.646+0000 7f039bbdf740 1 -- start start 2024-09-15T23:04:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.646+0000 7f039bbdf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610eb263260 0x5610eb25d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.646+0000 7f039bbdf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610eb263ba0 0x5610eb2748d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.646+0000 7f039bbdf740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5610eb268c30 0x5610eb27fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f039bbdf740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f03941ad720 con 0x5610eb263ba0 2024-09-15T23:04:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f039bbdf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f03941aaff0 con 0x5610eb263260 2024-09-15T23:04:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f039bbdf740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f03941a8e80 con 0x5610eb268c30 2024-09-15T23:04:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f0391400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610eb263ba0 0x5610eb2748d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f0391400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610eb263ba0 0x5610eb2748d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:37664/0 (socket says 172.21.15.111:37664) 2024-09-15T23:04:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f0391400640 1 -- 172.21.15.111:0/1752883180 learned_addr learned my addr 172.21.15.111:0/1752883180 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:04:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f0392800640 1 --2- 172.21.15.111:0/1752883180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5610eb268c30 0x5610eb27fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f0391e00640 1 --2- 172.21.15.111:0/1752883180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610eb263260 0x5610eb25d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f0391400640 1 -- 172.21.15.111:0/1752883180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5610eb268c30 msgr2=0x5610eb27fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f0391400640 1 --2- 172.21.15.111:0/1752883180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5610eb268c30 0x5610eb27fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f0391400640 1 -- 172.21.15.111:0/1752883180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610eb263260 msgr2=0x5610eb25d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f0391400640 1 --2- 172.21.15.111:0/1752883180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610eb263260 0x5610eb25d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f0391400640 1 -- 172.21.15.111:0/1752883180 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610eb27b600 con 0x5610eb263ba0 2024-09-15T23:04:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f0391400640 1 --2- 172.21.15.111:0/1752883180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610eb263ba0 0x5610eb2748d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0380098c80 tx=0x7f0380005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a835ce4a2820391f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:04:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f037ea00640 1 -- 172.21.15.111:0/1752883180 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038009be40 con 0x5610eb263ba0 2024-09-15T23:04:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f037ea00640 1 -- 172.21.15.111:0/1752883180 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f038009ca00 con 0x5610eb263ba0 2024-09-15T23:04:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f037ea00640 1 -- 172.21.15.111:0/1752883180 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03800a06b0 con 0x5610eb263ba0 2024-09-15T23:04:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f039bbdf740 1 -- 172.21.15.111:0/1752883180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610eb263ba0 msgr2=0x5610eb2748d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f039bbdf740 1 --2- 172.21.15.111:0/1752883180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610eb263ba0 0x5610eb2748d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0380098c80 tx=0x7f0380005e30 comp rx=0 tx=0).stop 2024-09-15T23:04:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f039bbdf740 1 -- 172.21.15.111:0/1752883180 shutdown_connections 2024-09-15T23:04:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f039bbdf740 1 --2- 172.21.15.111:0/1752883180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5610eb268c30 0x5610eb27fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f039bbdf740 1 --2- 172.21.15.111:0/1752883180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610eb263ba0 0x5610eb2748d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f039bbdf740 1 --2- 172.21.15.111:0/1752883180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610eb263260 0x5610eb25d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f039bbdf740 1 -- 172.21.15.111:0/1752883180 >> 172.21.15.111:0/1752883180 conn(0x5610eb25e8d0 msgr2=0x5610eb25d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:04:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f039bbdf740 1 -- 172.21.15.111:0/1752883180 shutdown_connections 2024-09-15T23:04:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f039bbdf740 1 -- 172.21.15.111:0/1752883180 wait complete. 2024-09-15T23:04:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f039bbdf740 1 -- start start 2024-09-15T23:04:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.647+0000 7f039bbdf740 1 Processor -- start 2024-09-15T23:04:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f039bbdf740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610eb263260 0x5610eb271c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f039bbdf740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5610eb263ba0 0x5610eb272180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f039bbdf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610eb268c30 0x5610eb273190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f039bbdf740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f03941a94d0 con 0x5610eb263260 2024-09-15T23:04:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f039bbdf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f03941aa920 con 0x5610eb268c30 2024-09-15T23:04:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f039bbdf740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f03941a9b80 con 0x5610eb263ba0 2024-09-15T23:04:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f039bbdf740 10 client.0 _ll_register_callbacks cb 0x7f038000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:04:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f0391e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610eb263260 0x5610eb271c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f0391400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5610eb263ba0 0x5610eb272180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f0391e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610eb263260 0x5610eb271c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:37680/0 (socket says 172.21.15.111:37680) 2024-09-15T23:04:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f0391e00640 1 -- 172.21.15.111:0/892665820 learned_addr learned my addr 172.21.15.111:0/892665820 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:04:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f0392800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610eb268c30 0x5610eb273190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f0391e00640 1 -- 172.21.15.111:0/892665820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5610eb263ba0 msgr2=0x5610eb272180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f0391e00640 1 --2- 172.21.15.111:0/892665820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5610eb263ba0 0x5610eb272180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f0391e00640 1 -- 172.21.15.111:0/892665820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610eb268c30 msgr2=0x5610eb273190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f0391e00640 1 --2- 172.21.15.111:0/892665820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610eb268c30 0x5610eb273190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f0391e00640 1 -- 172.21.15.111:0/892665820 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610eb2737a0 con 0x5610eb263260 2024-09-15T23:04:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f0391e00640 1 --2- 172.21.15.111:0/892665820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610eb263260 0x5610eb271c40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0384031440 tx=0x7f038402a3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6d44728d9fd13 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:04:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f0373e00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:04:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f0373e00640 1 -- 172.21.15.111:0/892665820 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0384058060 con 0x5610eb263260 2024-09-15T23:04:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f0373e00640 1 -- 172.21.15.111:0/892665820 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f03840318b0 con 0x5610eb263260 2024-09-15T23:04:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f039bbdf740 1 -- 172.21.15.111:0/892665820 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f038009ca00 con 0x5610eb263260 2024-09-15T23:04:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f0373e00640 1 -- 172.21.15.111:0/892665820 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0384040030 con 0x5610eb263260 2024-09-15T23:04:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f039bbdf740 10 client.16449 fetch_fsmap learned FSMap version 35 2024-09-15T23:04:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f039bbdf740 1 -- 172.21.15.111:0/892665820 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5610eb274b20 con 0x5610eb263260 2024-09-15T23:04:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f0373e00640 1 -- 172.21.15.111:0/892665820 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f038009ca00 con 0x5610eb263260 2024-09-15T23:04:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f0373e00640 1 -- 172.21.15.111:0/892665820 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f038405d070 con 0x5610eb263260 2024-09-15T23:04:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f039bbdf740 10 client.16449 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:04:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f039bbdf740 10 client.16449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:04:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f039bbdf740 20 client.16449 populate_metadata read hostname 'smithi111' 2024-09-15T23:04:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f039bbdf740 1 --2- 172.21.15.111:0/892665820 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5610eb2e3e70 0x5610eb2e6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f039bbdf740 4 client.16449 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:04:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f039bbdf740 1 -- 172.21.15.111:0/892665820 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5610eb1cfb80 con 0x5610eb2e3e70 2024-09-15T23:04:06.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f0391400640 1 --2- 172.21.15.111:0/892665820 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5610eb2e3e70 0x5610eb2e6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:06.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f0391400640 1 --2- 172.21.15.111:0/892665820 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5610eb2e3e70 0x5610eb2e6260 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dced26d13785363 server_cookie=57aad0fdc053c40 in_seq=0 out_seq=0 2024-09-15T23:04:06.656 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.650+0000 7f0373e00640 10 client.16449 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:04:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.782+0000 7f0373e00640 1 -- 172.21.15.111:0/892665820 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5610eb1cfb80 con 0x5610eb2e3e70 2024-09-15T23:04:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.782+0000 7f0373e00640 10 client.16449 handle_command_reply: tid=0 2024-09-15T23:04:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.782+0000 7f039bbdf740 1 client.16449 shutdown 2024-09-15T23:04:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.782+0000 7f039bbdf740 1 -- 172.21.15.111:0/892665820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610eb263260 msgr2=0x5610eb271c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.782+0000 7f039bbdf740 1 --2- 172.21.15.111:0/892665820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610eb263260 0x5610eb271c40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0384031440 tx=0x7f038402a3e0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.782+0000 7f039bbdf740 1 -- 172.21.15.111:0/892665820 shutdown_connections 2024-09-15T23:04:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.782+0000 7f039bbdf740 1 --2- 172.21.15.111:0/892665820 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5610eb2e3e70 0x5610eb2e6260 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.782+0000 7f039bbdf740 1 --2- 172.21.15.111:0/892665820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610eb268c30 0x5610eb273190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.782+0000 7f039bbdf740 1 --2- 172.21.15.111:0/892665820 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5610eb263ba0 0x5610eb272180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.782+0000 7f0373e00640 0 client.16449 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:04:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.782+0000 7f039bbdf740 1 --2- 172.21.15.111:0/892665820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5610eb263260 0x5610eb271c40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.782+0000 7f039bbdf740 1 -- 172.21.15.111:0/892665820 >> 172.21.15.111:0/892665820 conn(0x5610eb25e8d0 msgr2=0x5610eb25ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:04:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.783+0000 7f039bbdf740 1 -- 172.21.15.111:0/892665820 shutdown_connections 2024-09-15T23:04:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.783+0000 7f039bbdf740 1 -- 172.21.15.111:0/892665820 wait complete. 2024-09-15T23:04:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.783+0000 7f039bbdf740 20 client.16449 trim_cache size 0 max 16384 2024-09-15T23:04:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:04:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:04:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:04:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:04:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.783+0000 7f0393200640 1 -- 172.21.15.111:0/1219347722 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f03780a3240 msgr2=0x7f03780a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.783+0000 7f0393200640 1 --2- 172.21.15.111:0/1219347722 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f03780a3240 0x7f03780a5700 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f038c005fd0 tx=0x7f038c009500 comp rx=0 tx=0).stop 2024-09-15T23:04:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.783+0000 7f0393200640 1 -- 172.21.15.111:0/1219347722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0394074fc0 msgr2=0x7f039419a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.783+0000 7f0393200640 1 --2- 172.21.15.111:0/1219347722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0394074fc0 0x7f039419a4f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f038000e9f0 tx=0x7f038000edf0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.783+0000 7f0393200640 1 -- 172.21.15.111:0/1219347722 shutdown_connections 2024-09-15T23:04:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.783+0000 7f0393200640 1 --2- 172.21.15.111:0/1219347722 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f03780a3240 0x7f03780a5700 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.783+0000 7f0393200640 1 --2- 172.21.15.111:0/1219347722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0394109c90 0x7f03941a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.783+0000 7f0393200640 1 --2- 172.21.15.111:0/1219347722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0394074fc0 0x7f039419a4f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.783+0000 7f0393200640 1 --2- 172.21.15.111:0/1219347722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0394074680 0x7f0394199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.783+0000 7f0393200640 1 -- 172.21.15.111:0/1219347722 >> 172.21.15.111:0/1219347722 conn(0x7f03940fd0d0 msgr2=0x7f0394105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:04:06.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.784+0000 7f0393200640 1 -- 172.21.15.111:0/1219347722 shutdown_connections 2024-09-15T23:04:06.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:06.784+0000 7f0393200640 1 -- 172.21.15.111:0/1219347722 wait complete. 2024-09-15T23:04:07.645 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99224 17.75 MB/sec execute 155 sec latency 10206.395 ms 2024-09-15T23:04:08.076 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:07 smithi111 ceph-mon[30629]: pgmap v897: 97 pgs: 97 active+clean; 553 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.9 MiB/s wr, 396 op/s 2024-09-15T23:04:08.192 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:07 smithi121 ceph-mon[34815]: pgmap v897: 97 pgs: 97 active+clean; 553 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.9 MiB/s wr, 396 op/s 2024-09-15T23:04:08.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:07 smithi192 ceph-mon[35427]: pgmap v897: 97 pgs: 97 active+clean; 553 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.9 MiB/s wr, 396 op/s 2024-09-15T23:04:08.645 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99224 17.63 MB/sec execute 156 sec latency 11206.477 ms 2024-09-15T23:04:09.646 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99224 17.52 MB/sec execute 157 sec latency 12206.547 ms 2024-09-15T23:04:10.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:09 smithi111 ceph-mon[30629]: pgmap v898: 97 pgs: 97 active+clean; 554 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.5 MiB/s wr, 382 op/s 2024-09-15T23:04:10.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:09 smithi192 ceph-mon[35427]: pgmap v898: 97 pgs: 97 active+clean; 554 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.5 MiB/s wr, 382 op/s 2024-09-15T23:04:10.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:09 smithi121 ceph-mon[34815]: pgmap v898: 97 pgs: 97 active+clean; 554 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.5 MiB/s wr, 382 op/s 2024-09-15T23:04:10.646 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99224 17.41 MB/sec execute 158 sec latency 13206.611 ms 2024-09-15T23:04:11.646 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99224 17.30 MB/sec execute 159 sec latency 14206.700 ms 2024-09-15T23:04:12.158 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:11 smithi111 ceph-mon[30629]: pgmap v899: 97 pgs: 97 active+clean; 549 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.7 MiB/s wr, 300 op/s 2024-09-15T23:04:12.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:11 smithi192 ceph-mon[35427]: pgmap v899: 97 pgs: 97 active+clean; 549 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.7 MiB/s wr, 300 op/s 2024-09-15T23:04:12.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:11 smithi121 ceph-mon[34815]: pgmap v899: 97 pgs: 97 active+clean; 549 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.7 MiB/s wr, 300 op/s 2024-09-15T23:04:12.646 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99224 17.19 MB/sec execute 160 sec latency 15206.775 ms 2024-09-15T23:04:13.646 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99224 17.09 MB/sec execute 161 sec latency 16206.819 ms 2024-09-15T23:04:14.103 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:13 smithi111 ceph-mon[30629]: pgmap v900: 97 pgs: 97 active+clean; 548 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 5.9 MiB/s wr, 255 op/s 2024-09-15T23:04:14.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:13 smithi192 ceph-mon[35427]: pgmap v900: 97 pgs: 97 active+clean; 548 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 5.9 MiB/s wr, 255 op/s 2024-09-15T23:04:14.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:13 smithi121 ceph-mon[34815]: pgmap v900: 97 pgs: 97 active+clean; 548 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 5.9 MiB/s wr, 255 op/s 2024-09-15T23:04:14.646 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99227 16.98 MB/sec execute 162 sec latency 16475.283 ms 2024-09-15T23:04:15.646 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99227 16.88 MB/sec execute 163 sec latency 10023.078 ms 2024-09-15T23:04:16.097 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:15 smithi111 ceph-mon[30629]: pgmap v901: 97 pgs: 97 active+clean; 547 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 5.7 MiB/s wr, 248 op/s 2024-09-15T23:04:16.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:15 smithi192 ceph-mon[35427]: pgmap v901: 97 pgs: 97 active+clean; 547 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 5.7 MiB/s wr, 248 op/s 2024-09-15T23:04:16.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:15 smithi121 ceph-mon[34815]: pgmap v901: 97 pgs: 97 active+clean; 547 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 5.7 MiB/s wr, 248 op/s 2024-09-15T23:04:16.646 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99227 16.77 MB/sec execute 164 sec latency 11023.126 ms 2024-09-15T23:04:17.646 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99233 16.67 MB/sec execute 165 sec latency 12023.171 ms 2024-09-15T23:04:18.219 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:17 smithi111 ceph-mon[30629]: pgmap v902: 97 pgs: 97 active+clean; 546 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 5.9 MiB/s wr, 250 op/s 2024-09-15T23:04:18.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:17 smithi192 ceph-mon[35427]: pgmap v902: 97 pgs: 97 active+clean; 546 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 5.9 MiB/s wr, 250 op/s 2024-09-15T23:04:18.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:17 smithi121 ceph-mon[34815]: pgmap v902: 97 pgs: 97 active+clean; 546 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 5.9 MiB/s wr, 250 op/s 2024-09-15T23:04:18.646 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99233 16.57 MB/sec execute 166 sec latency 13023.262 ms 2024-09-15T23:04:19.646 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99233 16.47 MB/sec execute 167 sec latency 14023.305 ms 2024-09-15T23:04:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.893+0000 7fae58a00640 1 -- 172.21.15.111:0/852925969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae54102b00 msgr2=0x7fae54104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.893+0000 7fae58a00640 1 --2- 172.21.15.111:0/852925969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae54102b00 0x7fae54104f70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fae40009900 tx=0x7fae4002a3f0 comp rx=0 tx=0).stop 2024-09-15T23:04:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.893+0000 7fae58a00640 1 -- 172.21.15.111:0/852925969 shutdown_connections 2024-09-15T23:04:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.893+0000 7fae58a00640 1 --2- 172.21.15.111:0/852925969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae541054b0 0x7fae54105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.893+0000 7fae58a00640 1 --2- 172.21.15.111:0/852925969 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae54102b00 0x7fae54104f70 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.893+0000 7fae58a00640 1 --2- 172.21.15.111:0/852925969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae541001b0 0x7fae541025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.893+0000 7fae58a00640 1 -- 172.21.15.111:0/852925969 >> 172.21.15.111:0/852925969 conn(0x7fae540fbfa0 msgr2=0x7fae540fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:04:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.893+0000 7fae58a00640 1 -- 172.21.15.111:0/852925969 shutdown_connections 2024-09-15T23:04:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.893+0000 7fae58a00640 1 -- 172.21.15.111:0/852925969 wait complete. 2024-09-15T23:04:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.893+0000 7fae58a00640 1 Processor -- start 2024-09-15T23:04:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.893+0000 7fae58a00640 1 -- start start 2024-09-15T23:04:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.893+0000 7fae58a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae541001b0 0x7fae5419e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.893+0000 7fae58a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae54102b00 0x7fae5419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.893+0000 7fae58a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae541054b0 0x7fae541a5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.893+0000 7fae58a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fae54113b90 con 0x7fae541001b0 2024-09-15T23:04:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.893+0000 7fae58a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fae54113a10 con 0x7fae54102b00 2024-09-15T23:04:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.893+0000 7fae58a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fae54113d10 con 0x7fae541054b0 2024-09-15T23:04:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.893+0000 7fae52800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae541001b0 0x7fae5419e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.893+0000 7fae52800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae541001b0 0x7fae5419e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34380/0 (socket says 172.21.15.111:34380) 2024-09-15T23:04:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.893+0000 7fae52800640 1 -- 172.21.15.111:0/2225397399 learned_addr learned my addr 172.21.15.111:0/2225397399 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:04:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae51e00640 1 --2- 172.21.15.111:0/2225397399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae54102b00 0x7fae5419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae53200640 1 --2- 172.21.15.111:0/2225397399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae541054b0 0x7fae541a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae52800640 1 -- 172.21.15.111:0/2225397399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae541054b0 msgr2=0x7fae541a5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae52800640 1 --2- 172.21.15.111:0/2225397399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae541054b0 0x7fae541a5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae52800640 1 -- 172.21.15.111:0/2225397399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae54102b00 msgr2=0x7fae5419e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae52800640 1 --2- 172.21.15.111:0/2225397399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae54102b00 0x7fae5419e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae52800640 1 -- 172.21.15.111:0/2225397399 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae541a5e90 con 0x7fae541001b0 2024-09-15T23:04:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae53200640 1 --2- 172.21.15.111:0/2225397399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae541054b0 0x7fae541a5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:04:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae51e00640 1 --2- 172.21.15.111:0/2225397399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae54102b00 0x7fae5419e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:04:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae52800640 1 --2- 172.21.15.111:0/2225397399 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae541001b0 0x7fae5419e270 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fae4400e9f0 tx=0x7fae4400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:04:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae4b400640 1 -- 172.21.15.111:0/2225397399 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4400cd80 con 0x7fae541001b0 2024-09-15T23:04:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae58a00640 1 -- 172.21.15.111:0/2225397399 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae541a6180 con 0x7fae541001b0 2024-09-15T23:04:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae4b400640 1 -- 172.21.15.111:0/2225397399 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae44010910 con 0x7fae541001b0 2024-09-15T23:04:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae4b400640 1 -- 172.21.15.111:0/2225397399 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae44018990 con 0x7fae541001b0 2024-09-15T23:04:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae58a00640 1 -- 172.21.15.111:0/2225397399 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae541a66a0 con 0x7fae541001b0 2024-09-15T23:04:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae5c1fd740 1 Processor -- start 2024-09-15T23:04:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae5c1fd740 1 -- start start 2024-09-15T23:04:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae4b400640 1 -- 172.21.15.111:0/2225397399 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fae44010ac0 con 0x7fae541001b0 2024-09-15T23:04:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae5c1fd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56078dc20260 0x56078dc20660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae5c1fd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56078dc20ba0 0x56078dc25660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae5c1fd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56078dc25c30 0x56078dc280f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae5c1fd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56078db85100 con 0x56078dc20ba0 2024-09-15T23:04:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae5c1fd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56078db2d1b0 con 0x56078dc25c30 2024-09-15T23:04:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae5c1fd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56078db810c0 con 0x56078dc20260 2024-09-15T23:04:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae52800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56078dc20ba0 0x56078dc25660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae53200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56078dc20260 0x56078dc20660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae52800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56078dc20ba0 0x56078dc25660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34390/0 (socket says 172.21.15.111:34390) 2024-09-15T23:04:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae52800640 1 -- 172.21.15.111:0/2258464140 learned_addr learned my addr 172.21.15.111:0/2258464140 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:04:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.899+0000 7fae51e00640 1 --2- 172.21.15.111:0/2258464140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56078dc25c30 0x56078dc280f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae53200640 1 -- 172.21.15.111:0/2258464140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56078dc25c30 msgr2=0x56078dc280f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae53200640 1 --2- 172.21.15.111:0/2258464140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56078dc25c30 0x56078dc280f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae53200640 1 -- 172.21.15.111:0/2258464140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56078dc20ba0 msgr2=0x56078dc25660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae53200640 1 --2- 172.21.15.111:0/2258464140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56078dc20ba0 0x56078dc25660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae53200640 1 -- 172.21.15.111:0/2258464140 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56078d898210 con 0x56078dc20260 2024-09-15T23:04:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae53200640 1 --2- 172.21.15.111:0/2258464140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56078dc20260 0x56078dc20660 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fae4c009f90 tx=0x7fae4c010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b99090e01def8dee server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:04:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae3f400640 1 -- 172.21.15.111:0/2258464140 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c011700 con 0x56078dc20260 2024-09-15T23:04:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae3f400640 1 -- 172.21.15.111:0/2258464140 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae4c011d80 con 0x56078dc20260 2024-09-15T23:04:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae3f400640 1 -- 172.21.15.111:0/2258464140 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c01a380 con 0x56078dc20260 2024-09-15T23:04:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae4b400640 1 --2- 172.21.15.111:0/2225397399 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fae380a3240 0x7fae380a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae4b400640 1 -- 172.21.15.111:0/2225397399 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fae440cfd80 con 0x7fae541001b0 2024-09-15T23:04:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae52800640 1 --2- 172.21.15.111:0/2225397399 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fae380a3240 0x7fae380a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae5c1fd740 1 -- 172.21.15.111:0/2258464140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56078dc20260 msgr2=0x56078dc20660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae5c1fd740 1 --2- 172.21.15.111:0/2258464140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56078dc20260 0x56078dc20660 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fae4c009f90 tx=0x7fae4c010860 comp rx=0 tx=0).stop 2024-09-15T23:04:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae5c1fd740 1 -- 172.21.15.111:0/2258464140 shutdown_connections 2024-09-15T23:04:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae5c1fd740 1 --2- 172.21.15.111:0/2258464140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56078dc25c30 0x56078dc280f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae5c1fd740 1 --2- 172.21.15.111:0/2258464140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56078dc20ba0 0x56078dc25660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae5c1fd740 1 --2- 172.21.15.111:0/2258464140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56078dc20260 0x56078dc20660 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae5c1fd740 1 -- 172.21.15.111:0/2258464140 >> 172.21.15.111:0/2258464140 conn(0x56078dc1b8d0 msgr2=0x56078dc1bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:04:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae5c1fd740 1 -- 172.21.15.111:0/2258464140 shutdown_connections 2024-09-15T23:04:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae5c1fd740 1 -- 172.21.15.111:0/2258464140 wait complete. 2024-09-15T23:04:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae52800640 1 --2- 172.21.15.111:0/2225397399 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fae380a3240 0x7fae380a5700 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fae440d0020 tx=0x7fae440d37c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:04:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae5c1fd740 1 -- start start 2024-09-15T23:04:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae5c1fd740 1 Processor -- start 2024-09-15T23:04:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae5c1fd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56078dc20260 0x56078dc2a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae5c1fd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56078dc20ba0 0x56078dc2a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae5c1fd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56078dc25c30 0x56078dc319a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae5c1fd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56078dc28c00 con 0x56078dc20ba0 2024-09-15T23:04:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae5c1fd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56078db0e090 con 0x56078dc20260 2024-09-15T23:04:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae5c1fd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56078dc28d80 con 0x56078dc25c30 2024-09-15T23:04:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae5c1fd740 10 client.0 _ll_register_callbacks cb 0x56078db62f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:04:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae53200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56078dc20260 0x56078dc2a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae53200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56078dc20260 0x56078dc2a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:34836/0 (socket says 172.21.15.111:34836) 2024-09-15T23:04:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae51e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56078dc20ba0 0x56078dc2a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae53200640 1 -- 172.21.15.111:0/3563573003 learned_addr learned my addr 172.21.15.111:0/3563573003 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:04:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae53200640 1 --2- 172.21.15.111:0/3563573003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56078dc25c30 0x56078dc319a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae51e00640 1 -- 172.21.15.111:0/3563573003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56078dc25c30 msgr2=0x56078dc319a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae51e00640 1 --2- 172.21.15.111:0/3563573003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56078dc25c30 0x56078dc319a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae51e00640 1 -- 172.21.15.111:0/3563573003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56078dc20260 msgr2=0x56078dc2a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae51e00640 1 --2- 172.21.15.111:0/3563573003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56078dc20260 0x56078dc2a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae51e00640 1 -- 172.21.15.111:0/3563573003 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56078d964620 con 0x56078dc20ba0 2024-09-15T23:04:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae51e00640 1 --2- 172.21.15.111:0/3563573003 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56078dc20ba0 0x56078dc2a960 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fae400314e0 tx=0x7fae40031970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37947a618b8c00fe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:04:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae33e00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:04:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae33e00640 1 -- 172.21.15.111:0/3563573003 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae40035040 con 0x56078dc20ba0 2024-09-15T23:04:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae33e00640 1 -- 172.21.15.111:0/3563573003 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fae40004300 con 0x56078dc20ba0 2024-09-15T23:04:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae5c1fd740 1 -- 172.21.15.111:0/3563573003 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae4c011d80 con 0x56078dc20ba0 2024-09-15T23:04:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae33e00640 1 -- 172.21.15.111:0/3563573003 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fae40005030 con 0x56078dc20ba0 2024-09-15T23:04:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae5c1fd740 10 client.16461 fetch_fsmap learned FSMap version 35 2024-09-15T23:04:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae5c1fd740 1 -- 172.21.15.111:0/3563573003 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56078dc34800 con 0x56078dc20ba0 2024-09-15T23:04:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae33e00640 1 -- 172.21.15.111:0/3563573003 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae4c011d80 con 0x56078dc20ba0 2024-09-15T23:04:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae33e00640 1 -- 172.21.15.111:0/3563573003 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fae40004b70 con 0x56078dc20ba0 2024-09-15T23:04:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae5c1fd740 10 client.16461 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:04:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae5c1fd740 10 client.16461 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:04:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae5c1fd740 20 client.16461 populate_metadata read hostname 'smithi111' 2024-09-15T23:04:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae5c1fd740 1 --2- 172.21.15.111:0/3563573003 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56078dc35e80 0x56078dc38270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae5c1fd740 4 client.16461 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:04:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.903+0000 7fae5c1fd740 1 -- 172.21.15.111:0/3563573003 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56078db92830 con 0x56078dc35e80 2024-09-15T23:04:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.909+0000 7fae53200640 1 --2- 172.21.15.111:0/3563573003 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56078dc35e80 0x56078dc38270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.909+0000 7fae53200640 1 --2- 172.21.15.111:0/3563573003 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56078dc35e80 0x56078dc38270 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65630d9bfe9a7356 server_cookie=8bd751211ac3b186 in_seq=0 out_seq=0 2024-09-15T23:04:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.909+0000 7fae33e00640 10 client.16461 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:04:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.909+0000 7fae33e00640 1 -- 172.21.15.111:0/3563573003 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56078db92830 con 0x56078dc35e80 2024-09-15T23:04:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.909+0000 7fae33e00640 10 client.16461 handle_command_reply: tid=0 2024-09-15T23:04:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.909+0000 7fae5c1fd740 1 client.16461 shutdown 2024-09-15T23:04:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.909+0000 7fae5c1fd740 1 -- 172.21.15.111:0/3563573003 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56078dc20ba0 msgr2=0x56078dc2a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.909+0000 7fae5c1fd740 1 --2- 172.21.15.111:0/3563573003 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56078dc20ba0 0x56078dc2a960 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fae400314e0 tx=0x7fae40031970 comp rx=0 tx=0).stop 2024-09-15T23:04:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.909+0000 7fae5c1fd740 1 -- 172.21.15.111:0/3563573003 shutdown_connections 2024-09-15T23:04:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.909+0000 7fae5c1fd740 1 --2- 172.21.15.111:0/3563573003 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56078dc35e80 0x56078dc38270 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.909+0000 7fae5c1fd740 1 --2- 172.21.15.111:0/3563573003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56078dc25c30 0x56078dc319a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.909+0000 7fae5c1fd740 1 --2- 172.21.15.111:0/3563573003 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56078dc20ba0 0x56078dc2a960 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.909+0000 7fae5c1fd740 1 --2- 172.21.15.111:0/3563573003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56078dc20260 0x56078dc2a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.909+0000 7fae5c1fd740 1 -- 172.21.15.111:0/3563573003 >> 172.21.15.111:0/3563573003 conn(0x56078dc1b8d0 msgr2=0x56078dc1d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:04:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.909+0000 7fae33e00640 0 client.16461 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:04:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.909+0000 7fae5c1fd740 1 -- 172.21.15.111:0/3563573003 shutdown_connections 2024-09-15T23:04:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.909+0000 7fae5c1fd740 1 -- 172.21.15.111:0/3563573003 wait complete. 2024-09-15T23:04:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.909+0000 7fae5c1fd740 20 client.16461 trim_cache size 0 max 16384 2024-09-15T23:04:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.921+0000 7fae5c1fd740 1 Processor -- start 2024-09-15T23:04:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.921+0000 7fae5c1fd740 1 -- start start 2024-09-15T23:04:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.921+0000 7fae5c1fd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56078dc20260 0x56078dc1a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.921+0000 7fae5c1fd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56078dc20ba0 0x56078dc1ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.926+0000 7fae5c1fd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56078dc25c30 0x56078dc31960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.926+0000 7fae5c1fd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fae541b1be0 con 0x56078dc20ba0 2024-09-15T23:04:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.926+0000 7fae5c1fd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fae541af4b0 con 0x56078dc25c30 2024-09-15T23:04:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.926+0000 7fae5c1fd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fae541ad300 con 0x56078dc20260 2024-09-15T23:04:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.926+0000 7fae51e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56078dc20ba0 0x56078dc1ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.926+0000 7fae53200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56078dc20260 0x56078dc1a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.926+0000 7fae51e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56078dc20ba0 0x56078dc1ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34406/0 (socket says 172.21.15.111:34406) 2024-09-15T23:04:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.926+0000 7fae51e00640 1 -- 172.21.15.111:0/16432456 learned_addr learned my addr 172.21.15.111:0/16432456 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:04:19.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.926+0000 7fae53200640 1 --2- 172.21.15.111:0/16432456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56078dc25c30 0x56078dc31960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:19.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.926+0000 7fae51e00640 1 -- 172.21.15.111:0/16432456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56078dc20260 msgr2=0x56078dc1a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:19.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.926+0000 7fae51e00640 1 --2- 172.21.15.111:0/16432456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56078dc20260 0x56078dc1a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:19.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.926+0000 7fae51e00640 1 -- 172.21.15.111:0/16432456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56078dc25c30 msgr2=0x56078dc31960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:19.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.926+0000 7fae51e00640 1 --2- 172.21.15.111:0/16432456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56078dc25c30 0x56078dc31960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:19.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.926+0000 7fae51e00640 1 -- 172.21.15.111:0/16432456 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56078dc1b6f0 con 0x56078dc20ba0 2024-09-15T23:04:19.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.926+0000 7fae53200640 1 --2- 172.21.15.111:0/16432456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56078dc20260 0x56078dc1a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:04:19.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.926+0000 7fae51e00640 1 --2- 172.21.15.111:0/16432456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56078dc20ba0 0x56078dc1ae40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fae4002a910 tx=0x7fae400315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9039d3db4a8cf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:04:19.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.926+0000 7fae3ea00640 1 -- 172.21.15.111:0/16432456 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4005d070 con 0x56078dc20ba0 2024-09-15T23:04:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.926+0000 7fae3ea00640 1 -- 172.21.15.111:0/16432456 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae40003960 con 0x56078dc20ba0 2024-09-15T23:04:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae3ea00640 1 -- 172.21.15.111:0/16432456 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae40035070 con 0x56078dc20ba0 2024-09-15T23:04:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae5c1fd740 1 -- 172.21.15.111:0/16432456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56078dc20ba0 msgr2=0x56078dc1ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae5c1fd740 1 --2- 172.21.15.111:0/16432456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56078dc20ba0 0x56078dc1ae40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fae4002a910 tx=0x7fae400315b0 comp rx=0 tx=0).stop 2024-09-15T23:04:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae5c1fd740 1 -- 172.21.15.111:0/16432456 shutdown_connections 2024-09-15T23:04:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae5c1fd740 1 --2- 172.21.15.111:0/16432456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56078dc25c30 0x56078dc31960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae5c1fd740 1 --2- 172.21.15.111:0/16432456 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56078dc20ba0 0x56078dc1ae40 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae5c1fd740 1 --2- 172.21.15.111:0/16432456 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56078dc20260 0x56078dc1a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae5c1fd740 1 -- 172.21.15.111:0/16432456 >> 172.21.15.111:0/16432456 conn(0x56078dc1b8d0 msgr2=0x56078dc1a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:04:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae5c1fd740 1 -- 172.21.15.111:0/16432456 shutdown_connections 2024-09-15T23:04:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae5c1fd740 1 -- 172.21.15.111:0/16432456 wait complete. 2024-09-15T23:04:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae5c1fd740 1 -- start start 2024-09-15T23:04:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae5c1fd740 1 Processor -- start 2024-09-15T23:04:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae5c1fd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56078dc20260 0x56078dc2e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae5c1fd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56078dc20ba0 0x56078dc2eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae5c1fd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56078dc25c30 0x56078dc9eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae5c1fd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fae541ad970 con 0x56078dc20260 2024-09-15T23:04:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae5c1fd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fae541aede0 con 0x56078dc25c30 2024-09-15T23:04:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae5c1fd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fae541ae040 con 0x56078dc20ba0 2024-09-15T23:04:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae5c1fd740 10 client.0 _ll_register_callbacks cb 0x7fae44005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:04:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae51e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56078dc20ba0 0x56078dc2eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae51e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56078dc20ba0 0x56078dc2eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:40262/0 (socket says 172.21.15.111:40262) 2024-09-15T23:04:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae51e00640 1 -- 172.21.15.111:0/1405060940 learned_addr learned my addr 172.21.15.111:0/1405060940 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:04:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae53200640 1 --2- 172.21.15.111:0/1405060940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56078dc20260 0x56078dc2e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae53200640 1 --2- 172.21.15.111:0/1405060940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56078dc25c30 0x56078dc9eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae51e00640 1 -- 172.21.15.111:0/1405060940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56078dc25c30 msgr2=0x56078dc9eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae51e00640 1 --2- 172.21.15.111:0/1405060940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56078dc25c30 0x56078dc9eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae51e00640 1 -- 172.21.15.111:0/1405060940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56078dc20260 msgr2=0x56078dc2e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae51e00640 1 --2- 172.21.15.111:0/1405060940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56078dc20260 0x56078dc2e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae51e00640 1 -- 172.21.15.111:0/1405060940 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56078dc31b40 con 0x56078dc20ba0 2024-09-15T23:04:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae53200640 1 --2- 172.21.15.111:0/1405060940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56078dc25c30 0x56078dc9eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:04:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae51e00640 1 --2- 172.21.15.111:0/1405060940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56078dc20ba0 0x56078dc2eaa0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fae40005ec0 tx=0x7fae40001e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d52ab9947b68d32 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:04:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae33e00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:04:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae33e00640 1 -- 172.21.15.111:0/1405060940 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae40066060 con 0x56078dc20ba0 2024-09-15T23:04:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae33e00640 1 -- 172.21.15.111:0/1405060940 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fae400052b0 con 0x56078dc20ba0 2024-09-15T23:04:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae5c1fd740 1 -- 172.21.15.111:0/1405060940 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae40003960 con 0x56078dc20ba0 2024-09-15T23:04:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae33e00640 1 -- 172.21.15.111:0/1405060940 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fae4005d040 con 0x56078dc20ba0 2024-09-15T23:04:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae5c1fd740 10 client.26428 fetch_fsmap learned FSMap version 35 2024-09-15T23:04:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae5c1fd740 1 -- 172.21.15.111:0/1405060940 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56078dca0030 con 0x56078dc20ba0 2024-09-15T23:04:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae33e00640 1 -- 172.21.15.111:0/1405060940 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae40003960 con 0x56078dc20ba0 2024-09-15T23:04:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae33e00640 1 -- 172.21.15.111:0/1405060940 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fae40060070 con 0x56078dc20ba0 2024-09-15T23:04:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae5c1fd740 10 client.26428 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:04:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae5c1fd740 10 client.26428 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:04:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae5c1fd740 20 client.26428 populate_metadata read hostname 'smithi111' 2024-09-15T23:04:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae5c1fd740 1 --2- 172.21.15.111:0/1405060940 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56078dca0a10 0x56078dca2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae5c1fd740 4 client.26428 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:04:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae5c1fd740 1 -- 172.21.15.111:0/1405060940 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56078db8cb80 con 0x56078dca0a10 2024-09-15T23:04:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae53200640 1 --2- 172.21.15.111:0/1405060940 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56078dca0a10 0x56078dca2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae53200640 1 --2- 172.21.15.111:0/1405060940 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56078dca0a10 0x56078dca2e30 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10f4d79e311c80b7 server_cookie=8ec9f5749efdef2f in_seq=0 out_seq=0 2024-09-15T23:04:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:19.927+0000 7fae33e00640 10 client.26428 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:04:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:20.002+0000 7fae33e00640 1 -- 172.21.15.111:0/1405060940 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56078db8cb80 con 0x56078dca0a10 2024-09-15T23:04:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:20.002+0000 7fae33e00640 10 client.26428 handle_command_reply: tid=0 2024-09-15T23:04:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:20.002+0000 7fae5c1fd740 1 client.26428 shutdown 2024-09-15T23:04:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:20.002+0000 7fae5c1fd740 1 -- 172.21.15.111:0/1405060940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56078dc20ba0 msgr2=0x56078dc2eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:20.002+0000 7fae5c1fd740 1 --2- 172.21.15.111:0/1405060940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56078dc20ba0 0x56078dc2eaa0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fae40005ec0 tx=0x7fae40001e50 comp rx=0 tx=0).stop 2024-09-15T23:04:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:20.002+0000 7fae5c1fd740 1 -- 172.21.15.111:0/1405060940 shutdown_connections 2024-09-15T23:04:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:20.002+0000 7fae5c1fd740 1 --2- 172.21.15.111:0/1405060940 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56078dca0a10 0x56078dca2e30 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:20.002+0000 7fae5c1fd740 1 --2- 172.21.15.111:0/1405060940 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56078dc25c30 0x56078dc9eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:20.002+0000 7fae5c1fd740 1 --2- 172.21.15.111:0/1405060940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56078dc20ba0 0x56078dc2eaa0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:20.002+0000 7fae33e00640 0 client.26428 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:04:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:20.002+0000 7fae5c1fd740 1 --2- 172.21.15.111:0/1405060940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56078dc20260 0x56078dc2e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:20.002+0000 7fae5c1fd740 1 -- 172.21.15.111:0/1405060940 >> 172.21.15.111:0/1405060940 conn(0x56078dc1b8d0 msgr2=0x56078dc1aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:04:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:20.002+0000 7fae5c1fd740 1 -- 172.21.15.111:0/1405060940 shutdown_connections 2024-09-15T23:04:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:20.002+0000 7fae5c1fd740 1 -- 172.21.15.111:0/1405060940 wait complete. 2024-09-15T23:04:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:20.007+0000 7fae5c1fd740 20 client.26428 trim_cache size 0 max 16384 2024-09-15T23:04:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:04:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:04:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:04:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:04:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:20.007+0000 7fae58a00640 1 -- 172.21.15.111:0/2225397399 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fae380a3240 msgr2=0x7fae380a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:20.007+0000 7fae58a00640 1 --2- 172.21.15.111:0/2225397399 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fae380a3240 0x7fae380a5700 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fae440d0020 tx=0x7fae440d37c0 comp rx=0 tx=0).stop 2024-09-15T23:04:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:20.007+0000 7fae58a00640 1 -- 172.21.15.111:0/2225397399 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae541001b0 msgr2=0x7fae5419e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:20.007+0000 7fae58a00640 1 --2- 172.21.15.111:0/2225397399 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae541001b0 0x7fae5419e270 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fae4400e9f0 tx=0x7fae4400eeb0 comp rx=0 tx=0).stop 2024-09-15T23:04:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:20.007+0000 7fae58a00640 1 -- 172.21.15.111:0/2225397399 shutdown_connections 2024-09-15T23:04:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:20.007+0000 7fae58a00640 1 --2- 172.21.15.111:0/2225397399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae541054b0 0x7fae541a5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:20.007+0000 7fae58a00640 1 --2- 172.21.15.111:0/2225397399 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fae380a3240 0x7fae380a5700 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:20.007+0000 7fae58a00640 1 --2- 172.21.15.111:0/2225397399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae54102b00 0x7fae5419e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:20.007+0000 7fae58a00640 1 --2- 172.21.15.111:0/2225397399 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae541001b0 0x7fae5419e270 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:20.007+0000 7fae58a00640 1 -- 172.21.15.111:0/2225397399 >> 172.21.15.111:0/2225397399 conn(0x7fae540fbfa0 msgr2=0x7fae54100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:04:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:20.007+0000 7fae58a00640 1 -- 172.21.15.111:0/2225397399 shutdown_connections 2024-09-15T23:04:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:20.007+0000 7fae58a00640 1 -- 172.21.15.111:0/2225397399 wait complete. 2024-09-15T23:04:20.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:19 smithi192 ceph-mon[35427]: pgmap v903: 97 pgs: 97 active+clean; 534 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 6.6 MiB/s wr, 290 op/s 2024-09-15T23:04:20.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:19 smithi121 ceph-mon[34815]: pgmap v903: 97 pgs: 97 active+clean; 534 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 6.6 MiB/s wr, 290 op/s 2024-09-15T23:04:20.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:19 smithi111 ceph-mon[30629]: pgmap v903: 97 pgs: 97 active+clean; 534 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 6.6 MiB/s wr, 290 op/s 2024-09-15T23:04:20.646 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99233 16.38 MB/sec execute 168 sec latency 15023.348 ms 2024-09-15T23:04:21.646 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99233 16.28 MB/sec execute 169 sec latency 16023.418 ms 2024-09-15T23:04:22.240 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:21 smithi111 ceph-mon[30629]: pgmap v904: 97 pgs: 97 active+clean; 538 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 6.3 MiB/s wr, 254 op/s 2024-09-15T23:04:22.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:21 smithi192 ceph-mon[35427]: pgmap v904: 97 pgs: 97 active+clean; 538 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 6.3 MiB/s wr, 254 op/s 2024-09-15T23:04:22.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:21 smithi121 ceph-mon[34815]: pgmap v904: 97 pgs: 97 active+clean; 538 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 6.3 MiB/s wr, 254 op/s 2024-09-15T23:04:22.646 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99233 16.18 MB/sec execute 170 sec latency 17023.505 ms 2024-09-15T23:04:23.646 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99233 16.09 MB/sec execute 171 sec latency 18023.577 ms 2024-09-15T23:04:24.235 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:23 smithi111 ceph-mon[30629]: pgmap v905: 97 pgs: 97 active+clean; 539 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 5.5 MiB/s wr, 209 op/s 2024-09-15T23:04:24.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:23 smithi192 ceph-mon[35427]: pgmap v905: 97 pgs: 97 active+clean; 539 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 5.5 MiB/s wr, 209 op/s 2024-09-15T23:04:24.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:23 smithi121 ceph-mon[34815]: pgmap v905: 97 pgs: 97 active+clean; 539 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 5.5 MiB/s wr, 209 op/s 2024-09-15T23:04:24.646 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99233 15.99 MB/sec execute 172 sec latency 19023.619 ms 2024-09-15T23:04:25.646 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99233 15.90 MB/sec execute 173 sec latency 20023.666 ms 2024-09-15T23:04:26.207 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:25 smithi111 ceph-mon[30629]: pgmap v906: 97 pgs: 97 active+clean; 543 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 5.4 MiB/s wr, 212 op/s 2024-09-15T23:04:26.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:25 smithi192 ceph-mon[35427]: pgmap v906: 97 pgs: 97 active+clean; 543 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 5.4 MiB/s wr, 212 op/s 2024-09-15T23:04:26.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:25 smithi121 ceph-mon[34815]: pgmap v906: 97 pgs: 97 active+clean; 543 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 5.4 MiB/s wr, 212 op/s 2024-09-15T23:04:26.647 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99233 15.81 MB/sec execute 174 sec latency 21023.734 ms 2024-09-15T23:04:27.647 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99233 15.72 MB/sec execute 175 sec latency 22023.801 ms 2024-09-15T23:04:28.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:27 smithi111 ceph-mon[30629]: pgmap v907: 97 pgs: 97 active+clean; 552 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 5.5 MiB/s wr, 195 op/s 2024-09-15T23:04:28.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:27 smithi192 ceph-mon[35427]: pgmap v907: 97 pgs: 97 active+clean; 552 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 5.5 MiB/s wr, 195 op/s 2024-09-15T23:04:28.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:27 smithi121 ceph-mon[34815]: pgmap v907: 97 pgs: 97 active+clean; 552 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 5.5 MiB/s wr, 195 op/s 2024-09-15T23:04:28.659 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99233 15.63 MB/sec execute 176 sec latency 23035.568 ms 2024-09-15T23:04:29.659 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99233 15.54 MB/sec execute 177 sec latency 24035.640 ms 2024-09-15T23:04:30.060 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:29 smithi111 ceph-mon[30629]: pgmap v908: 97 pgs: 97 active+clean; 555 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 5.5 MiB/s wr, 199 op/s 2024-09-15T23:04:30.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:29 smithi192 ceph-mon[35427]: pgmap v908: 97 pgs: 97 active+clean; 555 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 5.5 MiB/s wr, 199 op/s 2024-09-15T23:04:30.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:29 smithi121 ceph-mon[34815]: pgmap v908: 97 pgs: 97 active+clean; 555 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 5.5 MiB/s wr, 199 op/s 2024-09-15T23:04:30.659 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99233 15.45 MB/sec execute 178 sec latency 25035.714 ms 2024-09-15T23:04:31.659 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99233 15.37 MB/sec execute 179 sec latency 26035.778 ms 2024-09-15T23:04:32.032 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:31 smithi111 ceph-mon[30629]: pgmap v909: 97 pgs: 97 active+clean; 556 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 5.3 MiB/s wr, 169 op/s 2024-09-15T23:04:32.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:31 smithi192 ceph-mon[35427]: pgmap v909: 97 pgs: 97 active+clean; 556 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 5.3 MiB/s wr, 169 op/s 2024-09-15T23:04:32.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:31 smithi121 ceph-mon[34815]: pgmap v909: 97 pgs: 97 active+clean; 556 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 5.3 MiB/s wr, 169 op/s 2024-09-15T23:04:32.659 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99277 15.29 MB/sec execute 180 sec latency 27035.867 ms 2024-09-15T23:04:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.119+0000 7f9d00a00640 1 -- 172.21.15.111:0/586519400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cfc104cf0 msgr2=0x7f9cfc1070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.119+0000 7f9d00a00640 1 --2- 172.21.15.111:0/586519400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cfc104cf0 0x7f9cfc1070e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4009860 tx=0x7f9ce402a340 comp rx=0 tx=0).stop 2024-09-15T23:04:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.119+0000 7f9d00a00640 1 -- 172.21.15.111:0/586519400 shutdown_connections 2024-09-15T23:04:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.119+0000 7f9d00a00640 1 --2- 172.21.15.111:0/586519400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cfc107620 0x7f9cfc109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.119+0000 7f9d00a00640 1 --2- 172.21.15.111:0/586519400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cfc104cf0 0x7f9cfc1070e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.119+0000 7f9d00a00640 1 --2- 172.21.15.111:0/586519400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cfc1023c0 0x7f9cfc1047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.119+0000 7f9d00a00640 1 -- 172.21.15.111:0/586519400 >> 172.21.15.111:0/586519400 conn(0x7f9cfc0fbfa0 msgr2=0x7f9cfc0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:04:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.119+0000 7f9d00a00640 1 -- 172.21.15.111:0/586519400 shutdown_connections 2024-09-15T23:04:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.122+0000 7f9d00a00640 1 -- 172.21.15.111:0/586519400 wait complete. 2024-09-15T23:04:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.123+0000 7f9d00a00640 1 Processor -- start 2024-09-15T23:04:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.123+0000 7f9d00a00640 1 -- start start 2024-09-15T23:04:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.123+0000 7f9d00a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cfc1023c0 0x7f9cfc101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.123+0000 7f9d00a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cfc104cf0 0x7f9cfc100370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.123+0000 7f9d00a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cfc107620 0x7f9cfc1008b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.123+0000 7f9d00a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc10d5d0 con 0x7f9cfc1023c0 2024-09-15T23:04:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.123+0000 7f9d00a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc10d450 con 0x7f9cfc104cf0 2024-09-15T23:04:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.123+0000 7f9d00a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc10d750 con 0x7f9cfc107620 2024-09-15T23:04:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.123+0000 7f9cf3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cfc104cf0 0x7f9cfc100370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.123+0000 7f9cfb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cfc1023c0 0x7f9cfc101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.123+0000 7f9cf3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cfc104cf0 0x7f9cfc100370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:40930/0 (socket says 172.21.15.111:40930) 2024-09-15T23:04:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.123+0000 7f9cf3e00640 1 -- 172.21.15.111:0/1453858547 learned_addr learned my addr 172.21.15.111:0/1453858547 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:04:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.123+0000 7f9cfbe00640 1 --2- 172.21.15.111:0/1453858547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cfc107620 0x7f9cfc1008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:33.127 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.123+0000 7f9cf3e00640 1 -- 172.21.15.111:0/1453858547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cfc107620 msgr2=0x7f9cfc1008b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.123+0000 7f9cf3e00640 1 --2- 172.21.15.111:0/1453858547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cfc107620 0x7f9cfc1008b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.123+0000 7f9cf3e00640 1 -- 172.21.15.111:0/1453858547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cfc1023c0 msgr2=0x7f9cfc101d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.123+0000 7f9cf3e00640 1 --2- 172.21.15.111:0/1453858547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cfc1023c0 0x7f9cfc101d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.123+0000 7f9cf3e00640 1 -- 172.21.15.111:0/1453858547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cfc101140 con 0x7f9cfc104cf0 2024-09-15T23:04:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.123+0000 7f9cfb400640 1 --2- 172.21.15.111:0/1453858547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cfc1023c0 0x7f9cfc101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:04:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.123+0000 7f9cf3e00640 1 --2- 172.21.15.111:0/1453858547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cfc104cf0 0x7f9cfc100370 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4035040 tx=0x7f9ce4033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:04:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.123+0000 7f9cf8c00640 1 -- 172.21.15.111:0/1453858547 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4037040 con 0x7f9cfc104cf0 2024-09-15T23:04:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.123+0000 7f9cf8c00640 1 -- 172.21.15.111:0/1453858547 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ce4032770 con 0x7f9cfc104cf0 2024-09-15T23:04:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.123+0000 7f9cf8c00640 1 -- 172.21.15.111:0/1453858547 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce40046c0 con 0x7f9cfc104cf0 2024-09-15T23:04:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.123+0000 7f9d00a00640 1 -- 172.21.15.111:0/1453858547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cfc1a1db0 con 0x7f9cfc104cf0 2024-09-15T23:04:33.128 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.123+0000 7f9d00a00640 1 -- 172.21.15.111:0/1453858547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cfc1a2270 con 0x7f9cfc104cf0 2024-09-15T23:04:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9d05016740 1 Processor -- start 2024-09-15T23:04:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9d05016740 1 -- start start 2024-09-15T23:04:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9d05016740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be91d2260 0x560be91d2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9d05016740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be91d2ba0 0x560be91d7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9d05016740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be91d7c30 0x560be91da0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9d05016740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560be9137100 con 0x560be91d7c30 2024-09-15T23:04:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9cf8c00640 1 -- 172.21.15.111:0/1453858547 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f9ce4004050 con 0x7f9cfc104cf0 2024-09-15T23:04:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9d05016740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560be90df1b0 con 0x560be91d2ba0 2024-09-15T23:04:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9d05016740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x560be91330c0 con 0x560be91d2260 2024-09-15T23:04:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9cfbe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be91d2260 0x560be91d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9cfb400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be91d2ba0 0x560be91d7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9cfbe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be91d2260 0x560be91d2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:40250/0 (socket says 172.21.15.111:40250) 2024-09-15T23:04:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9cfbe00640 1 -- 172.21.15.111:0/1496797113 learned_addr learned my addr 172.21.15.111:0/1496797113 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:04:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9cf3e00640 1 --2- 172.21.15.111:0/1496797113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be91d7c30 0x560be91da0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9cfbe00640 1 -- 172.21.15.111:0/1496797113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be91d2ba0 msgr2=0x560be91d7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:33.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9cfbe00640 1 --2- 172.21.15.111:0/1496797113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be91d2ba0 0x560be91d7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9cfbe00640 1 -- 172.21.15.111:0/1496797113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be91d7c30 msgr2=0x560be91da0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9cfbe00640 1 --2- 172.21.15.111:0/1496797113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be91d7c30 0x560be91da0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9cfbe00640 1 -- 172.21.15.111:0/1496797113 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560be8e4a210 con 0x560be91d2260 2024-09-15T23:04:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9cf3e00640 1 --2- 172.21.15.111:0/1496797113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be91d7c30 0x560be91da0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:04:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9cfbe00640 1 --2- 172.21.15.111:0/1496797113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be91d2260 0x560be91d2660 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4009f90 tx=0x7f9cf40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adfb8538ffe36d21 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:04:33.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9cebe00640 1 -- 172.21.15.111:0/1496797113 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4004d20 con 0x560be91d2260 2024-09-15T23:04:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9cebe00640 1 -- 172.21.15.111:0/1496797113 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cf4004050 con 0x560be91d2260 2024-09-15T23:04:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9cebe00640 1 -- 172.21.15.111:0/1496797113 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4005220 con 0x560be91d2260 2024-09-15T23:04:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9cf8c00640 1 --2- 172.21.15.111:0/1453858547 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9ce00a3240 0x7f9ce00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9cf8c00640 1 -- 172.21.15.111:0/1453858547 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f9ce40ef780 con 0x7f9cfc104cf0 2024-09-15T23:04:33.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9cfb400640 1 --2- 172.21.15.111:0/1453858547 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9ce00a3240 0x7f9ce00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9d05016740 1 -- 172.21.15.111:0/1496797113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be91d2260 msgr2=0x560be91d2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9d05016740 1 --2- 172.21.15.111:0/1496797113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be91d2260 0x560be91d2660 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4009f90 tx=0x7f9cf40118d0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9d05016740 1 -- 172.21.15.111:0/1496797113 shutdown_connections 2024-09-15T23:04:33.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9d05016740 1 --2- 172.21.15.111:0/1496797113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be91d7c30 0x560be91da0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9d05016740 1 --2- 172.21.15.111:0/1496797113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be91d2ba0 0x560be91d7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9d05016740 1 --2- 172.21.15.111:0/1496797113 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be91d2260 0x560be91d2660 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9d05016740 1 -- 172.21.15.111:0/1496797113 >> 172.21.15.111:0/1496797113 conn(0x560be91cd8d0 msgr2=0x560be91cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:04:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9d05016740 1 -- 172.21.15.111:0/1496797113 shutdown_connections 2024-09-15T23:04:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9d05016740 1 -- 172.21.15.111:0/1496797113 wait complete. 2024-09-15T23:04:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9cfb400640 1 --2- 172.21.15.111:0/1453858547 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9ce00a3240 0x7f9ce00a5700 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x560be91ce750 tx=0x7f9cec009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:04:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9d05016740 1 -- start start 2024-09-15T23:04:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9d05016740 1 Processor -- start 2024-09-15T23:04:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9d05016740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be91d2260 0x560be91dc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9d05016740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be91d2ba0 0x560be91dc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9d05016740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be91d7c30 0x560be91e3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9d05016740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x560be91dac00 con 0x560be91d7c30 2024-09-15T23:04:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9d05016740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560be90c0090 con 0x560be91d2260 2024-09-15T23:04:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9d05016740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x560be91dad80 con 0x560be91d2ba0 2024-09-15T23:04:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9d05016740 10 client.0 _ll_register_callbacks cb 0x560be9114f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:04:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9cfb400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be91d2260 0x560be91dc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9cf3e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be91d2ba0 0x560be91dc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9cfbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be91d7c30 0x560be91e3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9cfb400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be91d2260 0x560be91dc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:40944/0 (socket says 172.21.15.111:40944) 2024-09-15T23:04:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.127+0000 7f9cfb400640 1 -- 172.21.15.111:0/634213394 learned_addr learned my addr 172.21.15.111:0/634213394 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:04:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9cfb400640 1 -- 172.21.15.111:0/634213394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be91d2ba0 msgr2=0x560be91dc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9cfb400640 1 --2- 172.21.15.111:0/634213394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be91d2ba0 0x560be91dc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9cfb400640 1 -- 172.21.15.111:0/634213394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be91d7c30 msgr2=0x560be91e3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9cfb400640 1 --2- 172.21.15.111:0/634213394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be91d7c30 0x560be91e3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9cfb400640 1 -- 172.21.15.111:0/634213394 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560be8f16620 con 0x560be91d2260 2024-09-15T23:04:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9cfb400640 1 --2- 172.21.15.111:0/634213394 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be91d2260 0x560be91dc3e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9cec003640 tx=0x7f9cec0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ac7a9e189a62edd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:04:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9ce8c00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:04:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9ce8c00640 1 -- 172.21.15.111:0/634213394 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec00f880 con 0x560be91d2260 2024-09-15T23:04:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9d05016740 1 -- 172.21.15.111:0/634213394 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cf4004050 con 0x560be91d2260 2024-09-15T23:04:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9ce8c00640 1 -- 172.21.15.111:0/634213394 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f9cec00fa20 con 0x560be91d2260 2024-09-15T23:04:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9ce8c00640 1 -- 172.21.15.111:0/634213394 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9cec00fc20 con 0x560be91d2260 2024-09-15T23:04:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9d05016740 10 client.26423 fetch_fsmap learned FSMap version 35 2024-09-15T23:04:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9d05016740 1 -- 172.21.15.111:0/634213394 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560be91e6870 con 0x560be91d2260 2024-09-15T23:04:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9ce8c00640 1 -- 172.21.15.111:0/634213394 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cf4004050 con 0x560be91d2260 2024-09-15T23:04:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9ce8c00640 1 -- 172.21.15.111:0/634213394 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f9cec027070 con 0x560be91d2260 2024-09-15T23:04:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9d05016740 10 client.26423 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:04:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9d05016740 10 client.26423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:04:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9d05016740 20 client.26423 populate_metadata read hostname 'smithi111' 2024-09-15T23:04:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9d05016740 1 --2- 172.21.15.111:0/634213394 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560be91e7ed0 0x560be91ea340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9d05016740 4 client.26423 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:04:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9d05016740 1 -- 172.21.15.111:0/634213394 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560be9144830 con 0x560be91e7ed0 2024-09-15T23:04:33.151 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9cf3e00640 1 --2- 172.21.15.111:0/634213394 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560be91e7ed0 0x560be91ea340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9cf3e00640 1 --2- 172.21.15.111:0/634213394 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560be91e7ed0 0x560be91ea340 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19da86f64c3b82ed server_cookie=deca105a4cca7ead in_seq=0 out_seq=0 2024-09-15T23:04:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9ce8c00640 10 client.26423 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:04:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9ce8c00640 1 -- 172.21.15.111:0/634213394 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560be9144830 con 0x560be91e7ed0 2024-09-15T23:04:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9ce8c00640 10 client.26423 handle_command_reply: tid=0 2024-09-15T23:04:33.153 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9d05016740 1 client.26423 shutdown 2024-09-15T23:04:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9d05016740 1 -- 172.21.15.111:0/634213394 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be91d2260 msgr2=0x560be91dc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9d05016740 1 --2- 172.21.15.111:0/634213394 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be91d2260 0x560be91dc3e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9cec003640 tx=0x7f9cec0046c0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9d05016740 1 -- 172.21.15.111:0/634213394 shutdown_connections 2024-09-15T23:04:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9d05016740 1 --2- 172.21.15.111:0/634213394 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560be91e7ed0 0x560be91ea340 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9d05016740 1 --2- 172.21.15.111:0/634213394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be91d7c30 0x560be91e3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9d05016740 1 --2- 172.21.15.111:0/634213394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be91d2ba0 0x560be91dc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9d05016740 1 --2- 172.21.15.111:0/634213394 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be91d2260 0x560be91dc3e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9ce8c00640 0 client.26423 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:04:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9d05016740 1 -- 172.21.15.111:0/634213394 >> 172.21.15.111:0/634213394 conn(0x560be91cd8d0 msgr2=0x560be91cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:04:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9d05016740 1 -- 172.21.15.111:0/634213394 shutdown_connections 2024-09-15T23:04:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9d05016740 1 -- 172.21.15.111:0/634213394 wait complete. 2024-09-15T23:04:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.132+0000 7f9d05016740 20 client.26423 trim_cache size 0 max 16384 2024-09-15T23:04:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.149+0000 7f9d05016740 1 Processor -- start 2024-09-15T23:04:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.149+0000 7f9d05016740 1 -- start start 2024-09-15T23:04:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.149+0000 7f9d05016740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be91d2260 0x560be91cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.149+0000 7f9d05016740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be91d2ba0 0x560be91e3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.149+0000 7f9d05016740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be91d7c30 0x560be91eec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.149+0000 7f9d05016740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc1ad820 con 0x560be91d2260 2024-09-15T23:04:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.149+0000 7f9d05016740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc1ab0f0 con 0x560be91d2ba0 2024-09-15T23:04:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.149+0000 7f9d05016740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc1a8f60 con 0x560be91d7c30 2024-09-15T23:04:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.149+0000 7f9cfb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be91d2260 0x560be91cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.149+0000 7f9cf3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be91d2ba0 0x560be91e3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.149+0000 7f9cf3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be91d2ba0 0x560be91e3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:40952/0 (socket says 172.21.15.111:40952) 2024-09-15T23:04:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.149+0000 7f9cf3e00640 1 -- 172.21.15.111:0/2690526231 learned_addr learned my addr 172.21.15.111:0/2690526231 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:04:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.149+0000 7f9cfbe00640 1 --2- 172.21.15.111:0/2690526231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be91d7c30 0x560be91eec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9cf3e00640 1 -- 172.21.15.111:0/2690526231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be91d7c30 msgr2=0x560be91eec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9cf3e00640 1 --2- 172.21.15.111:0/2690526231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be91d7c30 0x560be91eec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9cf3e00640 1 -- 172.21.15.111:0/2690526231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be91d2260 msgr2=0x560be91cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9cf3e00640 1 --2- 172.21.15.111:0/2690526231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be91d2260 0x560be91cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9cf3e00640 1 -- 172.21.15.111:0/2690526231 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560be91d3080 con 0x560be91d2ba0 2024-09-15T23:04:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9cf3e00640 1 --2- 172.21.15.111:0/2690526231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be91d2ba0 0x560be91e3880 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40b8770 tx=0x7f9ce4005250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d1af6e97626a491 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:04:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9ceb400640 1 -- 172.21.15.111:0/2690526231 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce40bb700 con 0x560be91d2ba0 2024-09-15T23:04:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9ceb400640 1 -- 172.21.15.111:0/2690526231 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ce40bc4a0 con 0x560be91d2ba0 2024-09-15T23:04:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9ceb400640 1 -- 172.21.15.111:0/2690526231 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4004340 con 0x560be91d2ba0 2024-09-15T23:04:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9d05016740 1 -- 172.21.15.111:0/2690526231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be91d2ba0 msgr2=0x560be91e3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9d05016740 1 --2- 172.21.15.111:0/2690526231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be91d2ba0 0x560be91e3880 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40b8770 tx=0x7f9ce4005250 comp rx=0 tx=0).stop 2024-09-15T23:04:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9d05016740 1 -- 172.21.15.111:0/2690526231 shutdown_connections 2024-09-15T23:04:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9d05016740 1 --2- 172.21.15.111:0/2690526231 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be91d7c30 0x560be91eec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9d05016740 1 --2- 172.21.15.111:0/2690526231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be91d2ba0 0x560be91e3880 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9d05016740 1 --2- 172.21.15.111:0/2690526231 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be91d2260 0x560be91cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9d05016740 1 -- 172.21.15.111:0/2690526231 >> 172.21.15.111:0/2690526231 conn(0x560be91cd8d0 msgr2=0x560be91cc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:04:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9d05016740 1 -- 172.21.15.111:0/2690526231 shutdown_connections 2024-09-15T23:04:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9d05016740 1 -- 172.21.15.111:0/2690526231 wait complete. 2024-09-15T23:04:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9d05016740 1 -- start start 2024-09-15T23:04:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9d05016740 1 Processor -- start 2024-09-15T23:04:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9d05016740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be91d2260 0x560be91e16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9d05016740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be91d2ba0 0x560be91e1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9d05016740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be91d7c30 0x560be91e2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9d05016740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc1a95b0 con 0x560be91d2260 2024-09-15T23:04:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9d05016740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc1aaa20 con 0x560be91d2ba0 2024-09-15T23:04:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9d05016740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc1a9c80 con 0x560be91d7c30 2024-09-15T23:04:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9d05016740 10 client.0 _ll_register_callbacks cb 0x7f9cec009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:04:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9cfb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be91d2260 0x560be91e16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9cfb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be91d2260 0x560be91e16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50426/0 (socket says 172.21.15.111:50426) 2024-09-15T23:04:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9cfb400640 1 -- 172.21.15.111:0/2315507984 learned_addr learned my addr 172.21.15.111:0/2315507984 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:04:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9cfbe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be91d7c30 0x560be91e2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9cf3e00640 1 --2- 172.21.15.111:0/2315507984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be91d2ba0 0x560be91e1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9cfb400640 1 -- 172.21.15.111:0/2315507984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be91d7c30 msgr2=0x560be91e2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9cfb400640 1 --2- 172.21.15.111:0/2315507984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be91d7c30 0x560be91e2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9cfb400640 1 -- 172.21.15.111:0/2315507984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be91d2ba0 msgr2=0x560be91e1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9cfb400640 1 --2- 172.21.15.111:0/2315507984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be91d2ba0 0x560be91e1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9cfb400640 1 -- 172.21.15.111:0/2315507984 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560be91e3220 con 0x560be91d2260 2024-09-15T23:04:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9cf3e00640 1 --2- 172.21.15.111:0/2315507984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be91d2ba0 0x560be91e1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:04:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9cfb400640 1 --2- 172.21.15.111:0/2315507984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be91d2260 0x560be91e16c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9cec006010 tx=0x7f9cec00fdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f7c8d14763962b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:04:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9ce8c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:04:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9ce8c00640 1 -- 172.21.15.111:0/2315507984 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cec038030 con 0x560be91d2260 2024-09-15T23:04:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9ce8c00640 1 -- 172.21.15.111:0/2315507984 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f9cec018040 con 0x560be91d2260 2024-09-15T23:04:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9d05016740 1 -- 172.21.15.111:0/2315507984 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ce40bc4a0 con 0x560be91d2260 2024-09-15T23:04:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9ce8c00640 1 -- 172.21.15.111:0/2315507984 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9cec005030 con 0x560be91d2260 2024-09-15T23:04:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9d05016740 10 client.16485 fetch_fsmap learned FSMap version 35 2024-09-15T23:04:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9d05016740 1 -- 172.21.15.111:0/2315507984 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560be92532c0 con 0x560be91d2260 2024-09-15T23:04:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9ce8c00640 1 -- 172.21.15.111:0/2315507984 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ce40bc4a0 con 0x560be91d2260 2024-09-15T23:04:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9ce8c00640 1 -- 172.21.15.111:0/2315507984 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f9cec027040 con 0x560be91d2260 2024-09-15T23:04:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9d05016740 10 client.16485 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:04:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9d05016740 10 client.16485 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:04:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9d05016740 20 client.16485 populate_metadata read hostname 'smithi111' 2024-09-15T23:04:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9d05016740 1 --2- 172.21.15.111:0/2315507984 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560be9253cf0 0x560be9256110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9d05016740 4 client.16485 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:04:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9d05016740 1 -- 172.21.15.111:0/2315507984 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560be913eb80 con 0x560be9253cf0 2024-09-15T23:04:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9cf3e00640 1 --2- 172.21.15.111:0/2315507984 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560be9253cf0 0x560be9256110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9cf3e00640 1 --2- 172.21.15.111:0/2315507984 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560be9253cf0 0x560be9256110 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2842797ba761fccb server_cookie=8e3c8f8a1af8c1a in_seq=0 out_seq=0 2024-09-15T23:04:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.151+0000 7f9ce8c00640 10 client.16485 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:04:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.366+0000 7f9ce8c00640 1 -- 172.21.15.111:0/2315507984 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560be913eb80 con 0x560be9253cf0 2024-09-15T23:04:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.366+0000 7f9ce8c00640 10 client.16485 handle_command_reply: tid=0 2024-09-15T23:04:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.366+0000 7f9d05016740 1 client.16485 shutdown 2024-09-15T23:04:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.366+0000 7f9d05016740 1 -- 172.21.15.111:0/2315507984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be91d2260 msgr2=0x560be91e16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.366+0000 7f9d05016740 1 --2- 172.21.15.111:0/2315507984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be91d2260 0x560be91e16c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9cec006010 tx=0x7f9cec00fdc0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.366+0000 7f9d05016740 1 -- 172.21.15.111:0/2315507984 shutdown_connections 2024-09-15T23:04:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.366+0000 7f9d05016740 1 --2- 172.21.15.111:0/2315507984 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x560be9253cf0 0x560be9256110 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.366+0000 7f9d05016740 1 --2- 172.21.15.111:0/2315507984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x560be91d7c30 0x560be91e2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.366+0000 7f9d05016740 1 --2- 172.21.15.111:0/2315507984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560be91d2ba0 0x560be91e1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.366+0000 7f9ce8c00640 0 client.16485 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:04:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.366+0000 7f9d05016740 1 --2- 172.21.15.111:0/2315507984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x560be91d2260 0x560be91e16c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.366+0000 7f9d05016740 1 -- 172.21.15.111:0/2315507984 >> 172.21.15.111:0/2315507984 conn(0x560be91cd8d0 msgr2=0x560be91cc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:04:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.366+0000 7f9d05016740 1 -- 172.21.15.111:0/2315507984 shutdown_connections 2024-09-15T23:04:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.366+0000 7f9d05016740 1 -- 172.21.15.111:0/2315507984 wait complete. 2024-09-15T23:04:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.366+0000 7f9d05016740 20 client.16485 trim_cache size 0 max 16384 2024-09-15T23:04:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:04:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:04:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:04:33.370 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:04:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.366+0000 7f9d00a00640 1 -- 172.21.15.111:0/1453858547 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9ce00a3240 msgr2=0x7f9ce00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.366+0000 7f9d00a00640 1 --2- 172.21.15.111:0/1453858547 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9ce00a3240 0x7f9ce00a5700 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x560be91ce750 tx=0x7f9cec009260 comp rx=0 tx=0).stop 2024-09-15T23:04:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.366+0000 7f9d00a00640 1 -- 172.21.15.111:0/1453858547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cfc104cf0 msgr2=0x7f9cfc100370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.366+0000 7f9d00a00640 1 --2- 172.21.15.111:0/1453858547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cfc104cf0 0x7f9cfc100370 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4035040 tx=0x7f9ce4033d90 comp rx=0 tx=0).stop 2024-09-15T23:04:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.366+0000 7f9d00a00640 1 -- 172.21.15.111:0/1453858547 shutdown_connections 2024-09-15T23:04:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.366+0000 7f9d00a00640 1 --2- 172.21.15.111:0/1453858547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cfc107620 0x7f9cfc1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.366+0000 7f9d00a00640 1 --2- 172.21.15.111:0/1453858547 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9ce00a3240 0x7f9ce00a5700 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.366+0000 7f9d00a00640 1 --2- 172.21.15.111:0/1453858547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9cfc104cf0 0x7f9cfc100370 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.366+0000 7f9d00a00640 1 --2- 172.21.15.111:0/1453858547 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cfc1023c0 0x7f9cfc101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.366+0000 7f9d00a00640 1 -- 172.21.15.111:0/1453858547 >> 172.21.15.111:0/1453858547 conn(0x7f9cfc0fbfa0 msgr2=0x7f9cfc102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:04:33.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.366+0000 7f9d00a00640 1 -- 172.21.15.111:0/1453858547 shutdown_connections 2024-09-15T23:04:33.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:33.366+0000 7f9d00a00640 1 -- 172.21.15.111:0/1453858547 wait complete. 2024-09-15T23:04:33.659 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99278 15.21 MB/sec execute 181 sec latency 28035.956 ms 2024-09-15T23:04:34.266 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:33 smithi111 ceph-mon[30629]: pgmap v910: 97 pgs: 97 active+clean; 557 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 4.8 MiB/s wr, 149 op/s 2024-09-15T23:04:34.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:34 smithi192 ceph-mon[35427]: pgmap v910: 97 pgs: 97 active+clean; 557 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 4.8 MiB/s wr, 149 op/s 2024-09-15T23:04:34.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:34 smithi121 ceph-mon[34815]: pgmap v910: 97 pgs: 97 active+clean; 557 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 4.8 MiB/s wr, 149 op/s 2024-09-15T23:04:34.659 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99278 15.12 MB/sec execute 182 sec latency 29036.001 ms 2024-09-15T23:04:35.659 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99278 15.04 MB/sec execute 183 sec latency 30036.077 ms 2024-09-15T23:04:36.026 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:36 smithi111 ceph-mon[30629]: pgmap v911: 97 pgs: 97 active+clean; 549 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 4.6 MiB/s wr, 161 op/s 2024-09-15T23:04:36.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:36 smithi192 ceph-mon[35427]: pgmap v911: 97 pgs: 97 active+clean; 549 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 4.6 MiB/s wr, 161 op/s 2024-09-15T23:04:36.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:36 smithi121 ceph-mon[34815]: pgmap v911: 97 pgs: 97 active+clean; 549 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 4.6 MiB/s wr, 161 op/s 2024-09-15T23:04:36.659 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99278 14.96 MB/sec execute 184 sec latency 31036.124 ms 2024-09-15T23:04:37.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:37 smithi111 ceph-mon[30629]: pgmap v912: 97 pgs: 97 active+clean; 531 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 4.0 MiB/s wr, 167 op/s 2024-09-15T23:04:37.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:37 smithi192 ceph-mon[35427]: pgmap v912: 97 pgs: 97 active+clean; 531 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 4.0 MiB/s wr, 167 op/s 2024-09-15T23:04:37.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:37 smithi121 ceph-mon[34815]: pgmap v912: 97 pgs: 97 active+clean; 531 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 4.0 MiB/s wr, 167 op/s 2024-09-15T23:04:37.659 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99278 14.88 MB/sec execute 185 sec latency 32036.170 ms 2024-09-15T23:04:38.659 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99278 14.80 MB/sec execute 186 sec latency 33036.245 ms 2024-09-15T23:04:39.659 INFO:tasks.workunit.client.0.smithi111.stdout: 10 99475 14.77 MB/sec execute 187 sec latency 33736.024 ms 2024-09-15T23:04:40.100 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:39 smithi111 ceph-mon[30629]: pgmap v913: 97 pgs: 97 active+clean; 517 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 3.3 MiB/s wr, 187 op/s 2024-09-15T23:04:40.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:39 smithi192 ceph-mon[35427]: pgmap v913: 97 pgs: 97 active+clean; 517 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 3.3 MiB/s wr, 187 op/s 2024-09-15T23:04:40.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:39 smithi121 ceph-mon[34815]: pgmap v913: 97 pgs: 97 active+clean; 517 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 3.3 MiB/s wr, 187 op/s 2024-09-15T23:04:40.659 INFO:tasks.workunit.client.0.smithi111.stdout: 10 100081 14.87 MB/sec execute 188 sec latency 68.308 ms 2024-09-15T23:04:41.659 INFO:tasks.workunit.client.0.smithi111.stdout: 10 100816 14.98 MB/sec execute 189 sec latency 47.211 ms 2024-09-15T23:04:42.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:41 smithi111 ceph-mon[30629]: pgmap v914: 97 pgs: 97 active+clean; 518 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 2.5 MiB/s wr, 183 op/s 2024-09-15T23:04:42.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:41 smithi192 ceph-mon[35427]: pgmap v914: 97 pgs: 97 active+clean; 518 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 2.5 MiB/s wr, 183 op/s 2024-09-15T23:04:42.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:41 smithi121 ceph-mon[34815]: pgmap v914: 97 pgs: 97 active+clean; 518 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 760 KiB/s rd, 2.5 MiB/s wr, 183 op/s 2024-09-15T23:04:42.659 INFO:tasks.workunit.client.0.smithi111.stdout: 10 101641 15.10 MB/sec execute 190 sec latency 63.836 ms 2024-09-15T23:04:43.659 INFO:tasks.workunit.client.0.smithi111.stdout: 10 102519 15.31 MB/sec execute 191 sec latency 60.630 ms 2024-09-15T23:04:44.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:43 smithi192 ceph-mon[35427]: pgmap v915: 97 pgs: 97 active+clean; 516 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-09-15T23:04:44.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:43 smithi121 ceph-mon[34815]: pgmap v915: 97 pgs: 97 active+clean; 516 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-09-15T23:04:44.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:43 smithi111 ceph-mon[30629]: pgmap v915: 97 pgs: 97 active+clean; 516 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-09-15T23:04:44.661 INFO:tasks.workunit.client.0.smithi111.stdout: 10 103191 15.42 MB/sec execute 192 sec latency 62.452 ms 2024-09-15T23:04:45.659 INFO:tasks.workunit.client.0.smithi111.stdout: 10 103880 15.50 MB/sec execute 193 sec latency 60.396 ms 2024-09-15T23:04:46.210 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:45 smithi111 ceph-mon[30629]: pgmap v916: 97 pgs: 97 active+clean; 503 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.5 MiB/s wr, 233 op/s 2024-09-15T23:04:46.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:45 smithi192 ceph-mon[35427]: pgmap v916: 97 pgs: 97 active+clean; 503 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.5 MiB/s wr, 233 op/s 2024-09-15T23:04:46.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:45 smithi121 ceph-mon[34815]: pgmap v916: 97 pgs: 97 active+clean; 503 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.5 MiB/s wr, 233 op/s 2024-09-15T23:04:46.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.548+0000 7fa1ba800640 1 -- 172.21.15.111:0/561633735 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1b4074fc0 msgr2=0x7fa1b4109750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.548+0000 7fa1ba800640 1 --2- 172.21.15.111:0/561633735 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1b4074fc0 0x7fa1b4109750 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009900 tx=0x7fa1a002a3e0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.548+0000 7fa1ba800640 1 -- 172.21.15.111:0/561633735 shutdown_connections 2024-09-15T23:04:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.548+0000 7fa1ba800640 1 --2- 172.21.15.111:0/561633735 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1b4109c90 0x7fa1b410c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.548+0000 7fa1ba800640 1 --2- 172.21.15.111:0/561633735 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1b4074fc0 0x7fa1b4109750 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.548+0000 7fa1ba800640 1 --2- 172.21.15.111:0/561633735 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1b4074680 0x7fa1b4074a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.548+0000 7fa1ba800640 1 -- 172.21.15.111:0/561633735 >> 172.21.15.111:0/561633735 conn(0x7fa1b40fd0d0 msgr2=0x7fa1b40ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:04:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.548+0000 7fa1ba800640 1 -- 172.21.15.111:0/561633735 shutdown_connections 2024-09-15T23:04:46.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.548+0000 7fa1ba800640 1 -- 172.21.15.111:0/561633735 wait complete. 2024-09-15T23:04:46.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.549+0000 7fa1ba800640 1 Processor -- start 2024-09-15T23:04:46.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.549+0000 7fa1ba800640 1 -- start start 2024-09-15T23:04:46.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.549+0000 7fa1ba800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1b4074680 0x7fa1b419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:46.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.549+0000 7fa1ba800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1b4074fc0 0x7fa1b419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:46.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.549+0000 7fa1ba800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1b4109c90 0x7fa1b41a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:46.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.550+0000 7fa1ba800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b41030f0 con 0x7fa1b4109c90 2024-09-15T23:04:46.552 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.550+0000 7fa1ba800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b4102f70 con 0x7fa1b4074680 2024-09-15T23:04:46.553 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.550+0000 7fa1ba800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b4103270 con 0x7fa1b4074fc0 2024-09-15T23:04:46.553 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.550+0000 7fa1b8a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1b4074fc0 0x7fa1b419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:46.553 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.550+0000 7fa1b8a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1b4074fc0 0x7fa1b419e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:41424/0 (socket says 172.21.15.111:41424) 2024-09-15T23:04:46.553 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.550+0000 7fa1b8a00640 1 -- 172.21.15.111:0/1100044850 learned_addr learned my addr 172.21.15.111:0/1100044850 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:04:46.553 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.550+0000 7fa1b9400640 1 --2- 172.21.15.111:0/1100044850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1b4074680 0x7fa1b419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:46.553 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.550+0000 7fa1b9e00640 1 --2- 172.21.15.111:0/1100044850 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1b4109c90 0x7fa1b41a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:46.553 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.550+0000 7fa1b9400640 1 -- 172.21.15.111:0/1100044850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1b4074fc0 msgr2=0x7fa1b419e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:46.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.550+0000 7fa1b9400640 1 --2- 172.21.15.111:0/1100044850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1b4074fc0 0x7fa1b419e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.550+0000 7fa1b9400640 1 -- 172.21.15.111:0/1100044850 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1b4109c90 msgr2=0x7fa1b41a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:46.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.550+0000 7fa1b9400640 1 --2- 172.21.15.111:0/1100044850 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1b4109c90 0x7fa1b41a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.550+0000 7fa1b9400640 1 -- 172.21.15.111:0/1100044850 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b41a6000 con 0x7fa1b4074680 2024-09-15T23:04:46.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.550+0000 7fa1b9e00640 1 --2- 172.21.15.111:0/1100044850 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1b4109c90 0x7fa1b41a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:04:46.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.551+0000 7fa1b9400640 1 --2- 172.21.15.111:0/1100044850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1b4074680 0x7fa1b419e3e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa1a800e9f0 tx=0x7fa1a800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:04:46.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.551+0000 7fa1b2000640 1 -- 172.21.15.111:0/1100044850 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a800ccc0 con 0x7fa1b4074680 2024-09-15T23:04:46.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.551+0000 7fa1ba800640 1 -- 172.21.15.111:0/1100044850 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b41a62f0 con 0x7fa1b4074680 2024-09-15T23:04:46.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.551+0000 7fa1b2000640 1 -- 172.21.15.111:0/1100044850 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a8010910 con 0x7fa1b4074680 2024-09-15T23:04:46.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.551+0000 7fa1b2000640 1 -- 172.21.15.111:0/1100044850 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8018960 con 0x7fa1b4074680 2024-09-15T23:04:46.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.551+0000 7fa1ba800640 1 -- 172.21.15.111:0/1100044850 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b41a6890 con 0x7fa1b4074680 2024-09-15T23:04:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.557+0000 7fa1bec37740 1 Processor -- start 2024-09-15T23:04:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.557+0000 7fa1b2000640 1 -- 172.21.15.111:0/1100044850 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa1a8010ac0 con 0x7fa1b4074680 2024-09-15T23:04:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.557+0000 7fa1bec37740 1 -- start start 2024-09-15T23:04:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.557+0000 7fa1bec37740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562555288260 0x562555288660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.557+0000 7fa1bec37740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562555288ba0 0x56255528d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.557+0000 7fa1bec37740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56255528dc30 0x5625552900f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.557+0000 7fa1bec37740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5625551ed100 con 0x562555288ba0 2024-09-15T23:04:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.557+0000 7fa1bec37740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5625551951b0 con 0x562555288260 2024-09-15T23:04:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.557+0000 7fa1bec37740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5625551e90c0 con 0x56255528dc30 2024-09-15T23:04:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.558+0000 7fa1b8a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56255528dc30 0x5625552900f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.558+0000 7fa1b9e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562555288260 0x562555288660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.558+0000 7fa1b8a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56255528dc30 0x5625552900f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:41428/0 (socket says 172.21.15.111:41428) 2024-09-15T23:04:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.558+0000 7fa1b8a00640 1 -- 172.21.15.111:0/873410716 learned_addr learned my addr 172.21.15.111:0/873410716 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:04:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.558+0000 7fa1b9400640 1 --2- 172.21.15.111:0/873410716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562555288ba0 0x56255528d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:46.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.558+0000 7fa1b8a00640 1 -- 172.21.15.111:0/873410716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562555288260 msgr2=0x562555288660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.558+0000 7fa1b8a00640 1 --2- 172.21.15.111:0/873410716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562555288260 0x562555288660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.558+0000 7fa1b8a00640 1 -- 172.21.15.111:0/873410716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562555288ba0 msgr2=0x56255528d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.558+0000 7fa1b8a00640 1 --2- 172.21.15.111:0/873410716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562555288ba0 0x56255528d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.558+0000 7fa1b8a00640 1 -- 172.21.15.111:0/873410716 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562554f00210 con 0x56255528dc30 2024-09-15T23:04:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.558+0000 7fa1b9400640 1 --2- 172.21.15.111:0/873410716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562555288ba0 0x56255528d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:04:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.558+0000 7fa1b2000640 1 --2- 172.21.15.111:0/1100044850 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa19c0a32c0 0x7fa19c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.559+0000 7fa1b2000640 1 -- 172.21.15.111:0/1100044850 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fa1a80cfdf0 con 0x7fa1b4074680 2024-09-15T23:04:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.559+0000 7fa1b9e00640 1 --2- 172.21.15.111:0/1100044850 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa19c0a32c0 0x7fa19c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.559+0000 7fa1b8a00640 1 --2- 172.21.15.111:0/873410716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56255528dc30 0x5625552900f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa1a002a9d0 tx=0x7fa1a0004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=510cc3e7f5f1b2cc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:04:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.559+0000 7fa1a6600640 1 -- 172.21.15.111:0/873410716 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0040070 con 0x56255528dc30 2024-09-15T23:04:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.559+0000 7fa1a6600640 1 -- 172.21.15.111:0/873410716 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a0005500 con 0x56255528dc30 2024-09-15T23:04:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.559+0000 7fa1a6600640 1 -- 172.21.15.111:0/873410716 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0035040 con 0x56255528dc30 2024-09-15T23:04:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.560+0000 7fa1bec37740 1 -- 172.21.15.111:0/873410716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56255528dc30 msgr2=0x5625552900f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.560+0000 7fa1b9e00640 1 --2- 172.21.15.111:0/1100044850 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa19c0a32c0 0x7fa19c0a5780 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac005fd0 tx=0x7fa1ac009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:04:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.560+0000 7fa1bec37740 1 --2- 172.21.15.111:0/873410716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56255528dc30 0x5625552900f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa1a002a9d0 tx=0x7fa1a0004040 comp rx=0 tx=0).stop 2024-09-15T23:04:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.561+0000 7fa1bec37740 1 -- 172.21.15.111:0/873410716 shutdown_connections 2024-09-15T23:04:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.561+0000 7fa1bec37740 1 --2- 172.21.15.111:0/873410716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56255528dc30 0x5625552900f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.561+0000 7fa1bec37740 1 --2- 172.21.15.111:0/873410716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562555288ba0 0x56255528d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.561+0000 7fa1bec37740 1 --2- 172.21.15.111:0/873410716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562555288260 0x562555288660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.561+0000 7fa1bec37740 1 -- 172.21.15.111:0/873410716 >> 172.21.15.111:0/873410716 conn(0x5625552838d0 msgr2=0x562555283cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:04:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.561+0000 7fa1bec37740 1 -- 172.21.15.111:0/873410716 shutdown_connections 2024-09-15T23:04:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.561+0000 7fa1bec37740 1 -- 172.21.15.111:0/873410716 wait complete. 2024-09-15T23:04:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.561+0000 7fa1bec37740 1 -- start start 2024-09-15T23:04:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.561+0000 7fa1bec37740 1 Processor -- start 2024-09-15T23:04:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.562+0000 7fa1bec37740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562555288260 0x5625552a33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.562+0000 7fa1bec37740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562555288ba0 0x5625552a3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:46.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.562+0000 7fa1bec37740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56255528dc30 0x5625552aa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.562+0000 7fa1bec37740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5625552a1bb0 con 0x562555288260 2024-09-15T23:04:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.562+0000 7fa1bec37740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562555176090 con 0x562555288ba0 2024-09-15T23:04:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.562+0000 7fa1bec37740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5625552a1d30 con 0x56255528dc30 2024-09-15T23:04:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.562+0000 7fa1bec37740 10 client.0 _ll_register_callbacks cb 0x5625551caf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:04:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.562+0000 7fa1b9400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562555288260 0x5625552a33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.562+0000 7fa1b8a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562555288ba0 0x5625552a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.562+0000 7fa1b9400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562555288260 0x5625552a33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47588/0 (socket says 172.21.15.111:47588) 2024-09-15T23:04:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.562+0000 7fa1b9400640 1 -- 172.21.15.111:0/2539479746 learned_addr learned my addr 172.21.15.111:0/2539479746 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:04:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.562+0000 7fa1b8a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562555288ba0 0x5625552a3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:48072/0 (socket says 172.21.15.111:48072) 2024-09-15T23:04:46.566 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.562+0000 7fa1b9e00640 1 --2- 172.21.15.111:0/2539479746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56255528dc30 0x5625552aa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.563+0000 7fa1b9400640 1 -- 172.21.15.111:0/2539479746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56255528dc30 msgr2=0x5625552aa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.563+0000 7fa1b9400640 1 --2- 172.21.15.111:0/2539479746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56255528dc30 0x5625552aa950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.563+0000 7fa1b9400640 1 -- 172.21.15.111:0/2539479746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562555288ba0 msgr2=0x5625552a3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.563+0000 7fa1b9400640 1 --2- 172.21.15.111:0/2539479746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562555288ba0 0x5625552a3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.563+0000 7fa1b9400640 1 -- 172.21.15.111:0/2539479746 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562554fcc620 con 0x562555288260 2024-09-15T23:04:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.563+0000 7fa1b9e00640 1 --2- 172.21.15.111:0/2539479746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56255528dc30 0x5625552aa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:04:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.563+0000 7fa1b9400640 1 --2- 172.21.15.111:0/2539479746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562555288260 0x5625552a33d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8098910 tx=0x7fa1a80d00f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aff0e49d20a5cbc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:04:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.563+0000 7fa19aa00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:04:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.563+0000 7fa19aa00640 1 -- 172.21.15.111:0/2539479746 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a809bff0 con 0x562555288260 2024-09-15T23:04:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.563+0000 7fa19aa00640 1 -- 172.21.15.111:0/2539479746 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fa1a809bb00 con 0x562555288260 2024-09-15T23:04:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.563+0000 7fa1bec37740 1 -- 172.21.15.111:0/2539479746 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1a0005500 con 0x562555288260 2024-09-15T23:04:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.564+0000 7fa19aa00640 1 -- 172.21.15.111:0/2539479746 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa1a809be10 con 0x562555288260 2024-09-15T23:04:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.564+0000 7fa1bec37740 10 client.16494 fetch_fsmap learned FSMap version 35 2024-09-15T23:04:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.564+0000 7fa1bec37740 1 -- 172.21.15.111:0/2539479746 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5625552ad7b0 con 0x562555288260 2024-09-15T23:04:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.564+0000 7fa19aa00640 1 -- 172.21.15.111:0/2539479746 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a0005500 con 0x562555288260 2024-09-15T23:04:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.564+0000 7fa19aa00640 1 -- 172.21.15.111:0/2539479746 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fa1a8018b00 con 0x562555288260 2024-09-15T23:04:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.564+0000 7fa1bec37740 10 client.16494 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:04:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.564+0000 7fa1bec37740 10 client.16494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:04:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.564+0000 7fa1bec37740 20 client.16494 populate_metadata read hostname 'smithi111' 2024-09-15T23:04:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.564+0000 7fa1bec37740 1 --2- 172.21.15.111:0/2539479746 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5625552aee50 0x5625552b1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.564+0000 7fa1bec37740 4 client.16494 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:04:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.564+0000 7fa1bec37740 1 -- 172.21.15.111:0/2539479746 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625551fa830 con 0x5625552aee50 2024-09-15T23:04:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.565+0000 7fa1b8a00640 1 --2- 172.21.15.111:0/2539479746 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5625552aee50 0x5625552b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.565+0000 7fa1b8a00640 1 --2- 172.21.15.111:0/2539479746 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5625552aee50 0x5625552b1240 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b20974c74d809bfb server_cookie=24d33f753e89dfe4 in_seq=0 out_seq=0 2024-09-15T23:04:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.565+0000 7fa19aa00640 10 client.16494 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:04:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.566+0000 7fa19aa00640 1 -- 172.21.15.111:0/2539479746 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5625551fa830 con 0x5625552aee50 2024-09-15T23:04:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.566+0000 7fa19aa00640 10 client.16494 handle_command_reply: tid=0 2024-09-15T23:04:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.566+0000 7fa1bec37740 1 client.16494 shutdown 2024-09-15T23:04:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.566+0000 7fa1bec37740 1 -- 172.21.15.111:0/2539479746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562555288260 msgr2=0x5625552a33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.566+0000 7fa1bec37740 1 --2- 172.21.15.111:0/2539479746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562555288260 0x5625552a33d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8098910 tx=0x7fa1a80d00f0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.567+0000 7fa1bec37740 1 -- 172.21.15.111:0/2539479746 shutdown_connections 2024-09-15T23:04:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.567+0000 7fa1bec37740 1 --2- 172.21.15.111:0/2539479746 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5625552aee50 0x5625552b1240 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.567+0000 7fa1bec37740 1 --2- 172.21.15.111:0/2539479746 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56255528dc30 0x5625552aa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.567+0000 7fa1bec37740 1 --2- 172.21.15.111:0/2539479746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562555288ba0 0x5625552a3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.567+0000 7fa1bec37740 1 --2- 172.21.15.111:0/2539479746 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562555288260 0x5625552a33d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.567+0000 7fa1bec37740 1 -- 172.21.15.111:0/2539479746 >> 172.21.15.111:0/2539479746 conn(0x5625552838d0 msgr2=0x5625552850e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:04:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.567+0000 7fa19aa00640 0 client.16494 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:04:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.567+0000 7fa1bec37740 1 -- 172.21.15.111:0/2539479746 shutdown_connections 2024-09-15T23:04:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.567+0000 7fa1bec37740 1 -- 172.21.15.111:0/2539479746 wait complete. 2024-09-15T23:04:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.567+0000 7fa1bec37740 20 client.16494 trim_cache size 0 max 16384 2024-09-15T23:04:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.589+0000 7fa1bec37740 1 Processor -- start 2024-09-15T23:04:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.589+0000 7fa1bec37740 1 -- start start 2024-09-15T23:04:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.589+0000 7fa1bec37740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562555288260 0x5625552a2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.589+0000 7fa1bec37740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562555288ba0 0x5625552aa910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.589+0000 7fa1bec37740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56255528dc30 0x5625552a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.589+0000 7fa1bec37740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b41a93c0 con 0x562555288ba0 2024-09-15T23:04:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.589+0000 7fa1bec37740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b41a6c90 con 0x562555288260 2024-09-15T23:04:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.589+0000 7fa1bec37740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b4114370 con 0x56255528dc30 2024-09-15T23:04:46.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.589+0000 7fa1b8a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562555288ba0 0x5625552aa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.589+0000 7fa1b9e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56255528dc30 0x5625552a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.589+0000 7fa1b9400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562555288260 0x5625552a2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.589+0000 7fa1b9400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562555288260 0x5625552a2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:48082/0 (socket says 172.21.15.111:48082) 2024-09-15T23:04:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.589+0000 7fa1b8a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562555288ba0 0x5625552aa910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:47600/0 (socket says 172.21.15.111:47600) 2024-09-15T23:04:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.589+0000 7fa1b9400640 1 -- 172.21.15.111:0/2518741704 learned_addr learned my addr 172.21.15.111:0/2518741704 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:04:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.590+0000 7fa1b9e00640 1 -- 172.21.15.111:0/2518741704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562555288260 msgr2=0x5625552a2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.590+0000 7fa1b9e00640 1 --2- 172.21.15.111:0/2518741704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562555288260 0x5625552a2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.590+0000 7fa1b9e00640 1 -- 172.21.15.111:0/2518741704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562555288ba0 msgr2=0x5625552aa910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.590+0000 7fa1b9e00640 1 --2- 172.21.15.111:0/2518741704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562555288ba0 0x5625552aa910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.590+0000 7fa1b9e00640 1 -- 172.21.15.111:0/2518741704 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562555284560 con 0x56255528dc30 2024-09-15T23:04:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.590+0000 7fa1b8a00640 1 --2- 172.21.15.111:0/2518741704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562555288ba0 0x5625552aa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:04:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.590+0000 7fa1b9e00640 1 --2- 172.21.15.111:0/2518741704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56255528dc30 0x5625552a34f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac012de0 tx=0x7fa1ac015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25c24d24d7a4a414 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:04:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.590+0000 7fa1a5c00640 1 -- 172.21.15.111:0/2518741704 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac02b070 con 0x56255528dc30 2024-09-15T23:04:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.591+0000 7fa1a5c00640 1 -- 172.21.15.111:0/2518741704 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1ac01ecb0 con 0x56255528dc30 2024-09-15T23:04:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.591+0000 7fa1a5c00640 1 -- 172.21.15.111:0/2518741704 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac0263f0 con 0x56255528dc30 2024-09-15T23:04:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.591+0000 7fa1bec37740 1 -- 172.21.15.111:0/2518741704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56255528dc30 msgr2=0x5625552a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.591+0000 7fa1bec37740 1 --2- 172.21.15.111:0/2518741704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56255528dc30 0x5625552a34f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac012de0 tx=0x7fa1ac015000 comp rx=0 tx=0).stop 2024-09-15T23:04:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.592+0000 7fa1bec37740 1 -- 172.21.15.111:0/2518741704 shutdown_connections 2024-09-15T23:04:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.592+0000 7fa1bec37740 1 --2- 172.21.15.111:0/2518741704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56255528dc30 0x5625552a34f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.592+0000 7fa1bec37740 1 --2- 172.21.15.111:0/2518741704 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562555288ba0 0x5625552aa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.592+0000 7fa1bec37740 1 --2- 172.21.15.111:0/2518741704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562555288260 0x5625552a2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.592+0000 7fa1bec37740 1 -- 172.21.15.111:0/2518741704 >> 172.21.15.111:0/2518741704 conn(0x5625552838d0 msgr2=0x562555289d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:04:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.592+0000 7fa1bec37740 1 -- 172.21.15.111:0/2518741704 shutdown_connections 2024-09-15T23:04:46.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.592+0000 7fa1bec37740 1 -- 172.21.15.111:0/2518741704 wait complete. 2024-09-15T23:04:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.592+0000 7fa1bec37740 1 -- start start 2024-09-15T23:04:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.593+0000 7fa1bec37740 1 Processor -- start 2024-09-15T23:04:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.594+0000 7fa1bec37740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562555288260 0x5625552a7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.594+0000 7fa1bec37740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562555288ba0 0x5625552a7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.594+0000 7fa1bec37740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56255528dc30 0x562555317ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.594+0000 7fa1bec37740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b41149e0 con 0x562555288260 2024-09-15T23:04:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.594+0000 7fa1bec37740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b4115e50 con 0x56255528dc30 2024-09-15T23:04:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.594+0000 7fa1bec37740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b41150b0 con 0x562555288ba0 2024-09-15T23:04:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.594+0000 7fa1bec37740 10 client.0 _ll_register_callbacks cb 0x7fa1a80d3600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:04:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.594+0000 7fa1b8a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562555288ba0 0x5625552a7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.594+0000 7fa1b8a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562555288ba0 0x5625552a7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:41462/0 (socket says 172.21.15.111:41462) 2024-09-15T23:04:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.594+0000 7fa1b8a00640 1 -- 172.21.15.111:0/3098619575 learned_addr learned my addr 172.21.15.111:0/3098619575 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:04:46.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.594+0000 7fa1b9e00640 1 --2- 172.21.15.111:0/3098619575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56255528dc30 0x562555317ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.594+0000 7fa1b9400640 1 --2- 172.21.15.111:0/3098619575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562555288260 0x5625552a7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.595+0000 7fa1b9e00640 1 -- 172.21.15.111:0/3098619575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562555288ba0 msgr2=0x5625552a7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.595+0000 7fa1b9e00640 1 --2- 172.21.15.111:0/3098619575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562555288ba0 0x5625552a7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.595+0000 7fa1b9e00640 1 -- 172.21.15.111:0/3098619575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562555288260 msgr2=0x5625552a7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:04:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.595+0000 7fa1b9e00640 1 --2- 172.21.15.111:0/3098619575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562555288260 0x5625552a7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.595+0000 7fa1b9e00640 1 -- 172.21.15.111:0/3098619575 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562555318030 con 0x56255528dc30 2024-09-15T23:04:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.596+0000 7fa1b9e00640 1 --2- 172.21.15.111:0/3098619575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56255528dc30 0x562555317ac0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac029040 tx=0x7fa1ac01cf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6a3bacc73ed5cbc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:04:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.596+0000 7fa19aa00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:04:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.596+0000 7fa19aa00640 1 -- 172.21.15.111:0/3098619575 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac033070 con 0x56255528dc30 2024-09-15T23:04:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.596+0000 7fa1bec37740 1 -- 172.21.15.111:0/3098619575 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1ac01ecb0 con 0x56255528dc30 2024-09-15T23:04:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.596+0000 7fa19aa00640 1 -- 172.21.15.111:0/3098619575 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fa1ac003970 con 0x56255528dc30 2024-09-15T23:04:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.596+0000 7fa19aa00640 1 -- 172.21.15.111:0/3098619575 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa1ac02b040 con 0x56255528dc30 2024-09-15T23:04:46.600 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.596+0000 7fa1bec37740 10 client.26447 fetch_fsmap learned FSMap version 35 2024-09-15T23:04:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.596+0000 7fa1bec37740 1 -- 172.21.15.111:0/3098619575 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562555319370 con 0x56255528dc30 2024-09-15T23:04:46.601 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.597+0000 7fa19aa00640 1 -- 172.21.15.111:0/3098619575 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1ac01ecb0 con 0x56255528dc30 2024-09-15T23:04:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.597+0000 7fa19aa00640 1 -- 172.21.15.111:0/3098619575 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fa1ac02e050 con 0x56255528dc30 2024-09-15T23:04:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.597+0000 7fa1bec37740 10 client.26447 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:04:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.597+0000 7fa1bec37740 10 client.26447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:04:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.597+0000 7fa1bec37740 20 client.26447 populate_metadata read hostname 'smithi111' 2024-09-15T23:04:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.597+0000 7fa1bec37740 1 --2- 172.21.15.111:0/3098619575 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x562555319c50 0x56255531c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:04:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.597+0000 7fa1bec37740 4 client.26447 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:04:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.597+0000 7fa1bec37740 1 -- 172.21.15.111:0/3098619575 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5625551f4b80 con 0x562555319c50 2024-09-15T23:04:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.598+0000 7fa1b9400640 1 --2- 172.21.15.111:0/3098619575 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x562555319c50 0x56255531c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:04:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.598+0000 7fa19aa00640 10 client.26447 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:04:46.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.598+0000 7fa1b9400640 1 --2- 172.21.15.111:0/3098619575 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x562555319c50 0x56255531c070 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e6e92c241120fb8 server_cookie=86d81b48107010ac in_seq=0 out_seq=0 2024-09-15T23:04:46.661 INFO:tasks.workunit.client.0.smithi111.stdout: 10 104633 15.65 MB/sec execute 194 sec latency 83.038 ms 2024-09-15T23:04:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.780+0000 7fa19aa00640 1 -- 172.21.15.111:0/3098619575 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5625551f4b80 con 0x562555319c50 2024-09-15T23:04:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.780+0000 7fa19aa00640 10 client.26447 handle_command_reply: tid=0 2024-09-15T23:04:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.780+0000 7fa1bec37740 1 client.26447 shutdown 2024-09-15T23:04:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.780+0000 7fa1bec37740 1 -- 172.21.15.111:0/3098619575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56255528dc30 msgr2=0x562555317ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.780+0000 7fa1bec37740 1 --2- 172.21.15.111:0/3098619575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56255528dc30 0x562555317ac0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac029040 tx=0x7fa1ac01cf10 comp rx=0 tx=0).stop 2024-09-15T23:04:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.780+0000 7fa1bec37740 1 -- 172.21.15.111:0/3098619575 shutdown_connections 2024-09-15T23:04:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.780+0000 7fa1bec37740 1 --2- 172.21.15.111:0/3098619575 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x562555319c50 0x56255531c070 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.780+0000 7fa1bec37740 1 --2- 172.21.15.111:0/3098619575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56255528dc30 0x562555317ac0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.780+0000 7fa1bec37740 1 --2- 172.21.15.111:0/3098619575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562555288ba0 0x5625552a7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.780+0000 7fa1bec37740 1 --2- 172.21.15.111:0/3098619575 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562555288260 0x5625552a7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.780+0000 7fa1bec37740 1 -- 172.21.15.111:0/3098619575 >> 172.21.15.111:0/3098619575 conn(0x5625552838d0 msgr2=0x5625552a2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:04:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.781+0000 7fa19aa00640 0 client.26447 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:04:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.781+0000 7fa1bec37740 1 -- 172.21.15.111:0/3098619575 shutdown_connections 2024-09-15T23:04:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.781+0000 7fa1bec37740 1 -- 172.21.15.111:0/3098619575 wait complete. 2024-09-15T23:04:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.781+0000 7fa1bec37740 20 client.26447 trim_cache size 0 max 16384 2024-09-15T23:04:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:04:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:04:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:04:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:04:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.781+0000 7fa1ba800640 1 -- 172.21.15.111:0/1100044850 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa19c0a32c0 msgr2=0x7fa19c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.781+0000 7fa1ba800640 1 --2- 172.21.15.111:0/1100044850 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa19c0a32c0 0x7fa19c0a5780 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac005fd0 tx=0x7fa1ac009500 comp rx=0 tx=0).stop 2024-09-15T23:04:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.782+0000 7fa1ba800640 1 -- 172.21.15.111:0/1100044850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1b4074680 msgr2=0x7fa1b419e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:04:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.782+0000 7fa1ba800640 1 --2- 172.21.15.111:0/1100044850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1b4074680 0x7fa1b419e3e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa1a800e9f0 tx=0x7fa1a800eeb0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.782+0000 7fa1ba800640 1 -- 172.21.15.111:0/1100044850 shutdown_connections 2024-09-15T23:04:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.782+0000 7fa1ba800640 1 --2- 172.21.15.111:0/1100044850 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1b4109c90 0x7fa1b41a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.782+0000 7fa1ba800640 1 --2- 172.21.15.111:0/1100044850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1b4074fc0 0x7fa1b419e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.782+0000 7fa1ba800640 1 --2- 172.21.15.111:0/1100044850 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa19c0a32c0 0x7fa19c0a5780 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.782+0000 7fa1ba800640 1 --2- 172.21.15.111:0/1100044850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1b4074680 0x7fa1b419e3e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:04:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.782+0000 7fa1ba800640 1 -- 172.21.15.111:0/1100044850 >> 172.21.15.111:0/1100044850 conn(0x7fa1b40fd0d0 msgr2=0x7fa1b4103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:04:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.782+0000 7fa1ba800640 1 -- 172.21.15.111:0/1100044850 shutdown_connections 2024-09-15T23:04:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:46.782+0000 7fa1ba800640 1 -- 172.21.15.111:0/1100044850 wait complete. 2024-09-15T23:04:47.661 INFO:tasks.workunit.client.0.smithi111.stdout: 10 105355 15.78 MB/sec execute 195 sec latency 155.016 ms 2024-09-15T23:04:48.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:47 smithi192 ceph-mon[35427]: pgmap v917: 97 pgs: 97 active+clean; 499 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.3 MiB/s wr, 342 op/s 2024-09-15T23:04:48.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:47 smithi121 ceph-mon[34815]: pgmap v917: 97 pgs: 97 active+clean; 499 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.3 MiB/s wr, 342 op/s 2024-09-15T23:04:48.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:47 smithi111 ceph-mon[30629]: pgmap v917: 97 pgs: 97 active+clean; 499 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.3 MiB/s wr, 342 op/s 2024-09-15T23:04:48.662 INFO:tasks.workunit.client.0.smithi111.stdout: 10 106161 15.89 MB/sec execute 196 sec latency 49.744 ms 2024-09-15T23:04:49.662 INFO:tasks.workunit.client.0.smithi111.stdout: 10 106769 16.00 MB/sec execute 197 sec latency 85.910 ms 2024-09-15T23:04:50.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:49 smithi111 ceph-mon[30629]: pgmap v918: 97 pgs: 97 active+clean; 497 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.7 MiB/s wr, 394 op/s 2024-09-15T23:04:50.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:49 smithi192 ceph-mon[35427]: pgmap v918: 97 pgs: 97 active+clean; 497 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.7 MiB/s wr, 394 op/s 2024-09-15T23:04:50.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:49 smithi121 ceph-mon[34815]: pgmap v918: 97 pgs: 97 active+clean; 497 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.7 MiB/s wr, 394 op/s 2024-09-15T23:04:50.661 INFO:tasks.workunit.client.0.smithi111.stdout: 10 107531 16.12 MB/sec execute 198 sec latency 52.238 ms 2024-09-15T23:04:51.662 INFO:tasks.workunit.client.0.smithi111.stdout: 10 108392 16.26 MB/sec execute 199 sec latency 57.973 ms 2024-09-15T23:04:52.236 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:51 smithi111 ceph-mon[30629]: pgmap v919: 97 pgs: 97 active+clean; 506 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 3.8 MiB/s wr, 555 op/s 2024-09-15T23:04:52.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:51 smithi192 ceph-mon[35427]: pgmap v919: 97 pgs: 97 active+clean; 506 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 3.8 MiB/s wr, 555 op/s 2024-09-15T23:04:52.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:51 smithi121 ceph-mon[34815]: pgmap v919: 97 pgs: 97 active+clean; 506 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 3.8 MiB/s wr, 555 op/s 2024-09-15T23:04:52.662 INFO:tasks.workunit.client.0.smithi111.stdout: 10 108976 16.36 MB/sec execute 200 sec latency 123.304 ms 2024-09-15T23:04:53.662 INFO:tasks.workunit.client.0.smithi111.stdout: 10 109704 16.48 MB/sec execute 201 sec latency 76.292 ms 2024-09-15T23:04:54.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:53 smithi192 ceph-mon[35427]: pgmap v920: 97 pgs: 97 active+clean; 507 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.8 MiB/s wr, 538 op/s 2024-09-15T23:04:54.353 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:53 smithi111 ceph-mon[30629]: pgmap v920: 97 pgs: 97 active+clean; 507 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.8 MiB/s wr, 538 op/s 2024-09-15T23:04:54.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:53 smithi121 ceph-mon[34815]: pgmap v920: 97 pgs: 97 active+clean; 507 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.8 MiB/s wr, 538 op/s 2024-09-15T23:04:54.662 INFO:tasks.workunit.client.0.smithi111.stdout: 10 110416 16.58 MB/sec execute 202 sec latency 43.034 ms 2024-09-15T23:04:55.662 INFO:tasks.workunit.client.0.smithi111.stdout: 10 111188 16.67 MB/sec execute 203 sec latency 99.907 ms 2024-09-15T23:04:56.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:55 smithi111 ceph-mon[30629]: pgmap v921: 97 pgs: 97 active+clean; 517 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.5 MiB/s wr, 624 op/s 2024-09-15T23:04:56.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:55 smithi192 ceph-mon[35427]: pgmap v921: 97 pgs: 97 active+clean; 517 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.5 MiB/s wr, 624 op/s 2024-09-15T23:04:56.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:55 smithi121 ceph-mon[34815]: pgmap v921: 97 pgs: 97 active+clean; 517 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.5 MiB/s wr, 624 op/s 2024-09-15T23:04:56.662 INFO:tasks.workunit.client.0.smithi111.stdout: 10 111999 16.88 MB/sec execute 204 sec latency 48.456 ms 2024-09-15T23:04:57.662 INFO:tasks.workunit.client.0.smithi111.stdout: 10 112596 16.89 MB/sec execute 205 sec latency 79.073 ms 2024-09-15T23:04:58.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:57 smithi111 ceph-mon[30629]: pgmap v922: 97 pgs: 97 active+clean; 520 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 4.2 MiB/s wr, 599 op/s 2024-09-15T23:04:58.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:57 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:04:58.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:57 smithi192 ceph-mon[35427]: pgmap v922: 97 pgs: 97 active+clean; 520 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 4.2 MiB/s wr, 599 op/s 2024-09-15T23:04:58.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:57 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:04:58.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:57 smithi121 ceph-mon[34815]: pgmap v922: 97 pgs: 97 active+clean; 520 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 4.2 MiB/s wr, 599 op/s 2024-09-15T23:04:58.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:57 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:04:58.662 INFO:tasks.workunit.client.0.smithi111.stdout: 10 113323 17.00 MB/sec execute 206 sec latency 69.077 ms 2024-09-15T23:04:59.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:58 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:04:59.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:58 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:04:59.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:58 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:04:59.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:58 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:04:59.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:58 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:04:59.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:58 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:04:59.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:58 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:04:59.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:58 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:04:59.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:58 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:04:59.662 INFO:tasks.workunit.client.0.smithi111.stdout: 10 114163 17.16 MB/sec execute 207 sec latency 42.253 ms 2024-09-15T23:04:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.927+0000 7ff00d600640 1 -- 172.21.15.111:0/3528342467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0080761b0 msgr2=0x7ff008074600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.927+0000 7ff00d600640 1 --2- 172.21.15.111:0/3528342467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0080761b0 0x7ff008074600 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7feff8009900 tx=0x7feff802a490 comp rx=0 tx=0).stop 2024-09-15T23:05:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.927+0000 7ff00d600640 1 -- 172.21.15.111:0/3528342467 shutdown_connections 2024-09-15T23:05:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.927+0000 7ff00d600640 1 --2- 172.21.15.111:0/3528342467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff008109a10 0x7ff00810bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.927+0000 7ff00d600640 1 --2- 172.21.15.111:0/3528342467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff008074bd0 0x7ff008075030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.927+0000 7ff00d600640 1 --2- 172.21.15.111:0/3528342467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0080761b0 0x7ff008074600 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.927+0000 7ff00d600640 1 -- 172.21.15.111:0/3528342467 >> 172.21.15.111:0/3528342467 conn(0x7ff0080fd2e0 msgr2=0x7ff0080ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.927+0000 7ff00d600640 1 -- 172.21.15.111:0/3528342467 shutdown_connections 2024-09-15T23:05:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.927+0000 7ff00d600640 1 -- 172.21.15.111:0/3528342467 wait complete. 2024-09-15T23:05:00.416 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:04:59 smithi121 ceph-mon[34815]: pgmap v923: 97 pgs: 97 active+clean; 530 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.3 MiB/s wr, 574 op/s 2024-09-15T23:05:00.416 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:04:59 smithi192 ceph-mon[35427]: pgmap v923: 97 pgs: 97 active+clean; 530 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.3 MiB/s wr, 574 op/s 2024-09-15T23:05:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.929+0000 7ff00d600640 1 Processor -- start 2024-09-15T23:05:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.929+0000 7ff00d600640 1 -- start start 2024-09-15T23:05:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.929+0000 7ff00d600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff008074bd0 0x7ff008104f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.929+0000 7ff00d600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0080761b0 0x7ff008105440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.929+0000 7ff00d600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff008109a10 0x7ff008101a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:00.429 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.929+0000 7ff00d600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff00810fa10 con 0x7ff008074bd0 2024-09-15T23:05:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.929+0000 7ff00d600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff00810f890 con 0x7ff0080761b0 2024-09-15T23:05:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.929+0000 7ff00d600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff00810fb90 con 0x7ff008109a10 2024-09-15T23:05:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.929+0000 7ff006a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff008074bd0 0x7ff008104f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.929+0000 7ff007400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff008109a10 0x7ff008101a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.929+0000 7ff006000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0080761b0 0x7ff008105440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.929+0000 7ff007400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff008109a10 0x7ff008101a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:46016/0 (socket says 172.21.15.111:46016) 2024-09-15T23:05:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.929+0000 7ff007400640 1 -- 172.21.15.111:0/2272728041 learned_addr learned my addr 172.21.15.111:0/2272728041 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:05:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.929+0000 7ff006a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff008074bd0 0x7ff008104f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34252/0 (socket says 172.21.15.111:34252) 2024-09-15T23:05:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.930+0000 7ff007400640 1 -- 172.21.15.111:0/2272728041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0080761b0 msgr2=0x7ff008105440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.930+0000 7ff007400640 1 --2- 172.21.15.111:0/2272728041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0080761b0 0x7ff008105440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.930+0000 7ff007400640 1 -- 172.21.15.111:0/2272728041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff008074bd0 msgr2=0x7ff008104f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.930+0000 7ff007400640 1 --2- 172.21.15.111:0/2272728041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff008074bd0 0x7ff008104f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.930+0000 7ff007400640 1 -- 172.21.15.111:0/2272728041 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff008102330 con 0x7ff008109a10 2024-09-15T23:05:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.930+0000 7ff006000640 1 --2- 172.21.15.111:0/2272728041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0080761b0 0x7ff008105440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:05:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.930+0000 7ff007400640 1 --2- 172.21.15.111:0/2272728041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff008109a10 0x7ff008101a70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff00000eaa0 tx=0x7ff00000ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.931+0000 7fefff400640 1 -- 172.21.15.111:0/2272728041 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00000cdb0 con 0x7ff008109a10 2024-09-15T23:05:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.931+0000 7fefff400640 1 -- 172.21.15.111:0/2272728041 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff000005020 con 0x7ff008109a10 2024-09-15T23:05:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.931+0000 7fefff400640 1 -- 172.21.15.111:0/2272728041 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0000116b0 con 0x7ff008109a10 2024-09-15T23:05:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.931+0000 7ff00d600640 1 -- 172.21.15.111:0/2272728041 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0081a6420 con 0x7ff008109a10 2024-09-15T23:05:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.931+0000 7ff00d600640 1 -- 172.21.15.111:0/2272728041 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0081a6830 con 0x7ff008109a10 2024-09-15T23:05:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.935+0000 7fefff400640 1 -- 172.21.15.111:0/2272728041 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7ff000005560 con 0x7ff008109a10 2024-09-15T23:05:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.935+0000 7ff010664740 1 Processor -- start 2024-09-15T23:05:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.935+0000 7ff010664740 1 -- start start 2024-09-15T23:05:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.936+0000 7ff010664740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5588debd8090 0x5588debd8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.936+0000 7ff010664740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588debd89d0 0x5588debdd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.936+0000 7ff010664740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588debdda60 0x5588debdff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.936+0000 7ff010664740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5588deb3d100 con 0x5588debd89d0 2024-09-15T23:05:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.936+0000 7ff010664740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5588deae51b0 con 0x5588debd8090 2024-09-15T23:05:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.936+0000 7ff010664740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5588deb390c0 con 0x5588debdda60 2024-09-15T23:05:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.936+0000 7ff006000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5588debd8090 0x5588debd8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.936+0000 7ff006000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5588debd8090 0x5588debd8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:58050/0 (socket says 172.21.15.111:58050) 2024-09-15T23:05:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.936+0000 7ff006000640 1 -- 172.21.15.111:0/3050630143 learned_addr learned my addr 172.21.15.111:0/3050630143 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:05:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.936+0000 7ff006000640 1 --2- 172.21.15.111:0/3050630143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588debdda60 0x5588debdff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.936+0000 7fefff400640 1 --2- 172.21.15.111:0/2272728041 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fefec0a32f0 0x7fefec0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.936+0000 7fefff400640 1 -- 172.21.15.111:0/2272728041 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7ff0000cfad0 con 0x7ff008109a10 2024-09-15T23:05:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.936+0000 7ff006000640 1 -- 172.21.15.111:0/3050630143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588debdda60 msgr2=0x5588debdff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.936+0000 7ff006000640 1 --2- 172.21.15.111:0/3050630143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588debdda60 0x5588debdff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.936+0000 7ff006000640 1 -- 172.21.15.111:0/3050630143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588debd89d0 msgr2=0x5588debdd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.936+0000 7ff006a00640 1 --2- 172.21.15.111:0/3050630143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588debd89d0 0x5588debdd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.936+0000 7ff006000640 1 --2- 172.21.15.111:0/3050630143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588debd89d0 0x5588debdd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.936+0000 7ff006000640 1 -- 172.21.15.111:0/3050630143 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588de850210 con 0x5588debd8090 2024-09-15T23:05:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.937+0000 7ff007400640 1 --2- 172.21.15.111:0/2272728041 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fefec0a32f0 0x7fefec0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.937+0000 7ff006000640 1 --2- 172.21.15.111:0/3050630143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5588debd8090 0x5588debd8490 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7feff4011be0 tx=0x7feff4009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2417a64d88f06339 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.937+0000 7feff3400640 1 -- 172.21.15.111:0/3050630143 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff400fc50 con 0x5588debd8090 2024-09-15T23:05:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.937+0000 7feff3400640 1 -- 172.21.15.111:0/3050630143 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feff400fdf0 con 0x5588debd8090 2024-09-15T23:05:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.937+0000 7feff3400640 1 -- 172.21.15.111:0/3050630143 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff400e6a0 con 0x5588debd8090 2024-09-15T23:05:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.938+0000 7ff010664740 1 -- 172.21.15.111:0/3050630143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5588debd8090 msgr2=0x5588debd8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.938+0000 7ff007400640 1 --2- 172.21.15.111:0/2272728041 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fefec0a32f0 0x7fefec0a57b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff000098b70 tx=0x7ff000005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.938+0000 7ff010664740 1 --2- 172.21.15.111:0/3050630143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5588debd8090 0x5588debd8490 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7feff4011be0 tx=0x7feff4009ba0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.938+0000 7ff010664740 1 -- 172.21.15.111:0/3050630143 shutdown_connections 2024-09-15T23:05:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.938+0000 7ff010664740 1 --2- 172.21.15.111:0/3050630143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588debdda60 0x5588debdff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.938+0000 7ff010664740 1 --2- 172.21.15.111:0/3050630143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588debd89d0 0x5588debdd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.938+0000 7ff010664740 1 --2- 172.21.15.111:0/3050630143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5588debd8090 0x5588debd8490 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.938+0000 7ff010664740 1 -- 172.21.15.111:0/3050630143 >> 172.21.15.111:0/3050630143 conn(0x5588debd3700 msgr2=0x5588debd3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.938+0000 7ff010664740 1 -- 172.21.15.111:0/3050630143 shutdown_connections 2024-09-15T23:05:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.938+0000 7ff010664740 1 -- 172.21.15.111:0/3050630143 wait complete. 2024-09-15T23:05:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.939+0000 7ff010664740 1 -- start start 2024-09-15T23:05:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.939+0000 7ff010664740 1 Processor -- start 2024-09-15T23:05:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.939+0000 7ff010664740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588debd8090 0x5588debe2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.940+0000 7ff010664740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588debd89d0 0x5588debe2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.940+0000 7ff010664740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5588debdda60 0x5588debe97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.940+0000 7ff010664740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5588debe0a30 con 0x5588debd8090 2024-09-15T23:05:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.940+0000 7ff010664740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5588deac6090 con 0x5588debdda60 2024-09-15T23:05:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.940+0000 7ff010664740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5588debe0bb0 con 0x5588debd89d0 2024-09-15T23:05:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.940+0000 7ff010664740 10 client.0 _ll_register_callbacks cb 0x5588deb1af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:05:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.940+0000 7ff006a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588debd89d0 0x5588debe2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.940+0000 7ff006a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588debd89d0 0x5588debe2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:46036/0 (socket says 172.21.15.111:46036) 2024-09-15T23:05:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.940+0000 7ff006a00640 1 -- 172.21.15.111:0/1643502427 learned_addr learned my addr 172.21.15.111:0/1643502427 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:05:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.940+0000 7ff006000640 1 --2- 172.21.15.111:0/1643502427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5588debdda60 0x5588debe97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.940+0000 7ff006000640 1 --2- 172.21.15.111:0/1643502427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588debd8090 0x5588debe2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.940+0000 7ff006a00640 1 -- 172.21.15.111:0/1643502427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5588debdda60 msgr2=0x5588debe97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.940+0000 7ff006a00640 1 --2- 172.21.15.111:0/1643502427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5588debdda60 0x5588debe97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.940+0000 7ff006a00640 1 -- 172.21.15.111:0/1643502427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588debd8090 msgr2=0x5588debe2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.940+0000 7ff006a00640 1 --2- 172.21.15.111:0/1643502427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588debd8090 0x5588debe2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.940+0000 7ff006a00640 1 -- 172.21.15.111:0/1643502427 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588de91c620 con 0x5588debd89d0 2024-09-15T23:05:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.940+0000 7ff006000640 1 --2- 172.21.15.111:0/1643502427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5588debdda60 0x5588debe97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:05:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.941+0000 7ff006a00640 1 --2- 172.21.15.111:0/1643502427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588debd89d0 0x5588debe2790 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feff8000c00 tx=0x7feff8004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c76492831a0ebfe2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.941+0000 7fefe7e00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:05:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.941+0000 7fefe7e00640 1 -- 172.21.15.111:0/1643502427 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff8035040 con 0x5588debd89d0 2024-09-15T23:05:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.941+0000 7fefe7e00640 1 -- 172.21.15.111:0/1643502427 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7feff8005450 con 0x5588debd89d0 2024-09-15T23:05:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.941+0000 7ff010664740 1 -- 172.21.15.111:0/1643502427 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feff400fdf0 con 0x5588debd89d0 2024-09-15T23:05:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.942+0000 7fefe7e00640 1 -- 172.21.15.111:0/1643502427 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7feff8005650 con 0x5588debd89d0 2024-09-15T23:05:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.942+0000 7ff010664740 10 client.26473 fetch_fsmap learned FSMap version 35 2024-09-15T23:05:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.942+0000 7ff010664740 1 -- 172.21.15.111:0/1643502427 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5588debec630 con 0x5588debd89d0 2024-09-15T23:05:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.942+0000 7fefe7e00640 1 -- 172.21.15.111:0/1643502427 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feff400fdf0 con 0x5588debd89d0 2024-09-15T23:05:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.942+0000 7fefe7e00640 1 -- 172.21.15.111:0/1643502427 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7feff8031ac0 con 0x5588debd89d0 2024-09-15T23:05:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.942+0000 7ff010664740 10 client.26473 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:05:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.942+0000 7ff010664740 10 client.26473 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:05:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.942+0000 7ff010664740 20 client.26473 populate_metadata read hostname 'smithi111' 2024-09-15T23:05:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.942+0000 7ff010664740 1 --2- 172.21.15.111:0/1643502427 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5588debedcb0 0x5588debf00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.942+0000 7ff010664740 4 client.26473 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:05:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.942+0000 7ff010664740 1 -- 172.21.15.111:0/1643502427 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588deb4a830 con 0x5588debedcb0 2024-09-15T23:05:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.943+0000 7ff006000640 1 --2- 172.21.15.111:0/1643502427 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5588debedcb0 0x5588debf00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.943+0000 7ff006000640 1 --2- 172.21.15.111:0/1643502427 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5588debedcb0 0x5588debf00e0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e008bd39bb5ed70f server_cookie=3861f0856a649439 in_seq=0 out_seq=0 2024-09-15T23:05:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.943+0000 7fefe7e00640 10 client.26473 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:05:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.944+0000 7fefe7e00640 1 -- 172.21.15.111:0/1643502427 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5588deb4a830 con 0x5588debedcb0 2024-09-15T23:05:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.944+0000 7fefe7e00640 10 client.26473 handle_command_reply: tid=0 2024-09-15T23:05:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.945+0000 7ff010664740 1 client.26473 shutdown 2024-09-15T23:05:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.945+0000 7ff010664740 1 -- 172.21.15.111:0/1643502427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588debd89d0 msgr2=0x5588debe2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.945+0000 7ff010664740 1 --2- 172.21.15.111:0/1643502427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588debd89d0 0x5588debe2790 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feff8000c00 tx=0x7feff8004270 comp rx=0 tx=0).stop 2024-09-15T23:05:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.945+0000 7ff010664740 1 -- 172.21.15.111:0/1643502427 shutdown_connections 2024-09-15T23:05:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.945+0000 7ff010664740 1 --2- 172.21.15.111:0/1643502427 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5588debedcb0 0x5588debf00e0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.945+0000 7ff010664740 1 --2- 172.21.15.111:0/1643502427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5588debdda60 0x5588debe97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.945+0000 7ff010664740 1 --2- 172.21.15.111:0/1643502427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588debd89d0 0x5588debe2790 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.945+0000 7ff010664740 1 --2- 172.21.15.111:0/1643502427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588debd8090 0x5588debe2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.945+0000 7ff010664740 1 -- 172.21.15.111:0/1643502427 >> 172.21.15.111:0/1643502427 conn(0x5588debd3700 msgr2=0x5588debd4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.945+0000 7fefe7e00640 0 client.26473 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:05:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.945+0000 7ff010664740 1 -- 172.21.15.111:0/1643502427 shutdown_connections 2024-09-15T23:05:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.945+0000 7ff010664740 1 -- 172.21.15.111:0/1643502427 wait complete. 2024-09-15T23:05:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.946+0000 7ff010664740 20 client.26473 trim_cache size 0 max 16384 2024-09-15T23:05:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.965+0000 7ff010664740 1 Processor -- start 2024-09-15T23:05:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.965+0000 7ff010664740 1 -- start start 2024-09-15T23:05:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.966+0000 7ff010664740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5588debd8090 0x5588debd4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.966+0000 7ff010664740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588debd89d0 0x5588debe9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.966+0000 7ff010664740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588debdda60 0x5588debd5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.966+0000 7ff010664740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff0081b1e30 con 0x5588debd89d0 2024-09-15T23:05:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.966+0000 7ff010664740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff0081af700 con 0x5588debd8090 2024-09-15T23:05:00.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.966+0000 7ff010664740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff0081ad570 con 0x5588debdda60 2024-09-15T23:05:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.966+0000 7ff006a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588debd89d0 0x5588debe9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.966+0000 7ff006a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588debd89d0 0x5588debe9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34274/0 (socket says 172.21.15.111:34274) 2024-09-15T23:05:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.966+0000 7ff006a00640 1 -- 172.21.15.111:0/1320354796 learned_addr learned my addr 172.21.15.111:0/1320354796 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:05:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.966+0000 7ff006000640 1 --2- 172.21.15.111:0/1320354796 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5588debd8090 0x5588debd4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.966+0000 7ff006000640 1 --2- 172.21.15.111:0/1320354796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588debdda60 0x5588debd5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.967+0000 7ff006a00640 1 -- 172.21.15.111:0/1320354796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588debdda60 msgr2=0x5588debd5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.967+0000 7ff006a00640 1 --2- 172.21.15.111:0/1320354796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588debdda60 0x5588debd5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.967+0000 7ff006a00640 1 -- 172.21.15.111:0/1320354796 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5588debd8090 msgr2=0x5588debd4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.967+0000 7ff006a00640 1 --2- 172.21.15.111:0/1320354796 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5588debd8090 0x5588debd4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.967+0000 7ff006a00640 1 -- 172.21.15.111:0/1320354796 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588debced60 con 0x5588debd89d0 2024-09-15T23:05:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.967+0000 7ff006a00640 1 --2- 172.21.15.111:0/1320354796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588debd89d0 0x5588debe9790 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7feff802a9b0 tx=0x7feff8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f486de41b551b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.967+0000 7feff2a00640 1 -- 172.21.15.111:0/1320354796 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff8054070 con 0x5588debd89d0 2024-09-15T23:05:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.967+0000 7feff2a00640 1 -- 172.21.15.111:0/1320354796 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feff8005450 con 0x5588debd89d0 2024-09-15T23:05:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.967+0000 7feff2a00640 1 -- 172.21.15.111:0/1320354796 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff8035040 con 0x5588debd89d0 2024-09-15T23:05:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.968+0000 7ff010664740 1 -- 172.21.15.111:0/1320354796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588debd89d0 msgr2=0x5588debe9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.968+0000 7ff010664740 1 --2- 172.21.15.111:0/1320354796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588debd89d0 0x5588debe9790 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7feff802a9b0 tx=0x7feff8001e70 comp rx=0 tx=0).stop 2024-09-15T23:05:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.968+0000 7ff010664740 1 -- 172.21.15.111:0/1320354796 shutdown_connections 2024-09-15T23:05:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.968+0000 7ff010664740 1 --2- 172.21.15.111:0/1320354796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588debdda60 0x5588debd5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.968+0000 7ff010664740 1 --2- 172.21.15.111:0/1320354796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588debd89d0 0x5588debe9790 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.968+0000 7ff010664740 1 --2- 172.21.15.111:0/1320354796 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5588debd8090 0x5588debd4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.968+0000 7ff010664740 1 -- 172.21.15.111:0/1320354796 >> 172.21.15.111:0/1320354796 conn(0x5588debd3700 msgr2=0x5588debd2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.969+0000 7ff010664740 1 -- 172.21.15.111:0/1320354796 shutdown_connections 2024-09-15T23:05:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.969+0000 7ff010664740 1 -- 172.21.15.111:0/1320354796 wait complete. 2024-09-15T23:05:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.969+0000 7ff010664740 1 -- start start 2024-09-15T23:05:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.969+0000 7ff010664740 1 Processor -- start 2024-09-15T23:05:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.970+0000 7ff010664740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588debd8090 0x5588debe6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.970+0000 7ff010664740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5588debd89d0 0x5588debe7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.970+0000 7ff010664740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588debdda60 0x5588debe8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.970+0000 7ff010664740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ff0081adbc0 con 0x5588debdda60 2024-09-15T23:05:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.970+0000 7ff010664740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff0081af030 con 0x5588debd89d0 2024-09-15T23:05:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.970+0000 7ff010664740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff0081ae290 con 0x5588debd8090 2024-09-15T23:05:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.970+0000 7ff010664740 10 client.0 _ll_register_callbacks cb 0x7feff8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:05:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.970+0000 7ff006000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588debd8090 0x5588debe6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.970+0000 7ff006a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5588debd89d0 0x5588debe7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.970+0000 7ff006a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5588debd89d0 0x5588debe7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:58072/0 (socket says 172.21.15.111:58072) 2024-09-15T23:05:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.970+0000 7ff006a00640 1 -- 172.21.15.111:0/1046434743 learned_addr learned my addr 172.21.15.111:0/1046434743 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:05:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.970+0000 7ff006000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588debdda60 0x5588debe8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.971+0000 7ff006a00640 1 -- 172.21.15.111:0/1046434743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588debd8090 msgr2=0x5588debe6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.971+0000 7ff006a00640 1 --2- 172.21.15.111:0/1046434743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588debd8090 0x5588debe6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.971+0000 7ff006a00640 1 -- 172.21.15.111:0/1046434743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588debdda60 msgr2=0x5588debe8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.971+0000 7ff006a00640 1 --2- 172.21.15.111:0/1046434743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588debdda60 0x5588debe8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.971+0000 7ff006a00640 1 -- 172.21.15.111:0/1046434743 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588debe8820 con 0x5588debd89d0 2024-09-15T23:05:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.971+0000 7ff006a00640 1 --2- 172.21.15.111:0/1046434743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5588debd89d0 0x5588debe7250 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7feff80314e0 tx=0x7feff8005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48686c03dd765461 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.971+0000 7fefe7e00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:05:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.971+0000 7fefe7e00640 1 -- 172.21.15.111:0/1046434743 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff8065060 con 0x5588debd89d0 2024-09-15T23:05:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.971+0000 7fefe7e00640 1 -- 172.21.15.111:0/1046434743 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7feff8003c70 con 0x5588debd89d0 2024-09-15T23:05:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.971+0000 7ff010664740 1 -- 172.21.15.111:0/1046434743 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feff8005450 con 0x5588debd89d0 2024-09-15T23:05:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.972+0000 7fefe7e00640 1 -- 172.21.15.111:0/1046434743 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7feff8035040 con 0x5588debd89d0 2024-09-15T23:05:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.972+0000 7ff010664740 10 client.26465 fetch_fsmap learned FSMap version 35 2024-09-15T23:05:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.972+0000 7ff010664740 1 -- 172.21.15.111:0/1046434743 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5588debe9b30 con 0x5588debd89d0 2024-09-15T23:05:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.972+0000 7fefe7e00640 1 -- 172.21.15.111:0/1046434743 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feff8005450 con 0x5588debd89d0 2024-09-15T23:05:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.972+0000 7fefe7e00640 1 -- 172.21.15.111:0/1046434743 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7feff806a070 con 0x5588debd89d0 2024-09-15T23:05:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.973+0000 7ff010664740 10 client.26465 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:05:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.973+0000 7ff010664740 10 client.26465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:05:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.973+0000 7ff010664740 20 client.26465 populate_metadata read hostname 'smithi111' 2024-09-15T23:05:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.973+0000 7ff010664740 1 --2- 172.21.15.111:0/1046434743 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5588dec59070 0x5588dec5b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.973+0000 7ff010664740 4 client.26465 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:05:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.973+0000 7ff010664740 1 -- 172.21.15.111:0/1046434743 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5588deb44b80 con 0x5588dec59070 2024-09-15T23:05:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.973+0000 7ff006000640 1 --2- 172.21.15.111:0/1046434743 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5588dec59070 0x5588dec5b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.974+0000 7fefe7e00640 10 client.26465 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:05:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:04:59.974+0000 7ff006000640 1 --2- 172.21.15.111:0/1046434743 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5588dec59070 0x5588dec5b490 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c238d8c33b7e9c2 server_cookie=235f1c8fe51c578f in_seq=0 out_seq=0 2024-09-15T23:05:00.445 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:04:59 smithi111 ceph-mon[30629]: pgmap v923: 97 pgs: 97 active+clean; 530 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.3 MiB/s wr, 574 op/s 2024-09-15T23:05:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:00.221+0000 7fefe7e00640 1 -- 172.21.15.111:0/1046434743 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5588deb44b80 con 0x5588dec59070 2024-09-15T23:05:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:00.221+0000 7fefe7e00640 10 client.26465 handle_command_reply: tid=0 2024-09-15T23:05:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:00.221+0000 7ff010664740 1 client.26465 shutdown 2024-09-15T23:05:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:00.222+0000 7ff010664740 1 -- 172.21.15.111:0/1046434743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5588debd89d0 msgr2=0x5588debe7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:00.222+0000 7ff010664740 1 --2- 172.21.15.111:0/1046434743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5588debd89d0 0x5588debe7250 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7feff80314e0 tx=0x7feff8005850 comp rx=0 tx=0).stop 2024-09-15T23:05:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:00.222+0000 7ff010664740 1 -- 172.21.15.111:0/1046434743 shutdown_connections 2024-09-15T23:05:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:00.222+0000 7ff010664740 1 --2- 172.21.15.111:0/1046434743 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5588dec59070 0x5588dec5b490 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:00.222+0000 7ff010664740 1 --2- 172.21.15.111:0/1046434743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588debdda60 0x5588debe8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:00.222+0000 7ff010664740 1 --2- 172.21.15.111:0/1046434743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5588debd89d0 0x5588debe7250 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:00.222+0000 7ff010664740 1 --2- 172.21.15.111:0/1046434743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5588debd8090 0x5588debe6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:00.222+0000 7ff010664740 1 -- 172.21.15.111:0/1046434743 >> 172.21.15.111:0/1046434743 conn(0x5588debd3700 msgr2=0x5588debd3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:00.222+0000 7fefe7e00640 0 client.26465 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:05:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:00.222+0000 7ff010664740 1 -- 172.21.15.111:0/1046434743 shutdown_connections 2024-09-15T23:05:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:00.222+0000 7ff010664740 1 -- 172.21.15.111:0/1046434743 wait complete. 2024-09-15T23:05:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:00.222+0000 7ff010664740 20 client.26465 trim_cache size 0 max 16384 2024-09-15T23:05:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:05:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:05:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:05:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:05:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:00.223+0000 7ff00d600640 1 -- 172.21.15.111:0/2272728041 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fefec0a32f0 msgr2=0x7fefec0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:00.223+0000 7ff00d600640 1 --2- 172.21.15.111:0/2272728041 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fefec0a32f0 0x7fefec0a57b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff000098b70 tx=0x7ff000005850 comp rx=0 tx=0).stop 2024-09-15T23:05:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:00.223+0000 7ff00d600640 1 -- 172.21.15.111:0/2272728041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff008109a10 msgr2=0x7ff008101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:00.223+0000 7ff00d600640 1 --2- 172.21.15.111:0/2272728041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff008109a10 0x7ff008101a70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff00000eaa0 tx=0x7ff00000ef60 comp rx=0 tx=0).stop 2024-09-15T23:05:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:00.223+0000 7ff00d600640 1 -- 172.21.15.111:0/2272728041 shutdown_connections 2024-09-15T23:05:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:00.223+0000 7ff00d600640 1 --2- 172.21.15.111:0/2272728041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff008109a10 0x7ff008101a70 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:00.223+0000 7ff00d600640 1 --2- 172.21.15.111:0/2272728041 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fefec0a32f0 0x7fefec0a57b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:00.223+0000 7ff00d600640 1 --2- 172.21.15.111:0/2272728041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0080761b0 0x7ff008105440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:00.223+0000 7ff00d600640 1 --2- 172.21.15.111:0/2272728041 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff008074bd0 0x7ff008104f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:00.223+0000 7ff00d600640 1 -- 172.21.15.111:0/2272728041 >> 172.21.15.111:0/2272728041 conn(0x7ff0080fd2e0 msgr2=0x7ff0081059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:00.224+0000 7ff00d600640 1 -- 172.21.15.111:0/2272728041 shutdown_connections 2024-09-15T23:05:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:00.224+0000 7ff00d600640 1 -- 172.21.15.111:0/2272728041 wait complete. 2024-09-15T23:05:00.662 INFO:tasks.workunit.client.0.smithi111.stdout: 10 114852 17.25 MB/sec execute 208 sec latency 138.957 ms 2024-09-15T23:05:01.662 INFO:tasks.workunit.client.0.smithi111.stdout: 10 115531 17.36 MB/sec execute 209 sec latency 72.186 ms 2024-09-15T23:05:02.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:01 smithi111 ceph-mon[30629]: pgmap v924: 97 pgs: 97 active+clean; 536 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.6 MiB/s wr, 605 op/s 2024-09-15T23:05:02.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:01 smithi192 ceph-mon[35427]: pgmap v924: 97 pgs: 97 active+clean; 536 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.6 MiB/s wr, 605 op/s 2024-09-15T23:05:02.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:01 smithi121 ceph-mon[34815]: pgmap v924: 97 pgs: 97 active+clean; 536 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.6 MiB/s wr, 605 op/s 2024-09-15T23:05:02.662 INFO:tasks.workunit.client.0.smithi111.stdout: 10 116099 17.42 MB/sec execute 210 sec latency 84.037 ms 2024-09-15T23:05:03.662 INFO:tasks.workunit.client.0.smithi111.stdout: 10 116782 17.49 MB/sec execute 211 sec latency 78.459 ms 2024-09-15T23:05:04.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:03 smithi111 ceph-mon[30629]: pgmap v925: 97 pgs: 97 active+clean; 542 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.9 MiB/s wr, 506 op/s 2024-09-15T23:05:04.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:03 smithi192 ceph-mon[35427]: pgmap v925: 97 pgs: 97 active+clean; 542 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.9 MiB/s wr, 506 op/s 2024-09-15T23:05:04.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:03 smithi121 ceph-mon[34815]: pgmap v925: 97 pgs: 97 active+clean; 542 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.9 MiB/s wr, 506 op/s 2024-09-15T23:05:04.662 INFO:tasks.workunit.client.0.smithi111.stdout: 10 117509 17.59 MB/sec execute 212 sec latency 50.978 ms 2024-09-15T23:05:05.662 INFO:tasks.workunit.client.0.smithi111.stdout: 10 118393 17.70 MB/sec execute 213 sec latency 36.137 ms 2024-09-15T23:05:06.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:05 smithi192 ceph-mon[35427]: pgmap v926: 97 pgs: 97 active+clean; 539 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.3 MiB/s wr, 594 op/s 2024-09-15T23:05:06.327 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:05 smithi111 ceph-mon[30629]: pgmap v926: 97 pgs: 97 active+clean; 539 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.3 MiB/s wr, 594 op/s 2024-09-15T23:05:06.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:05 smithi121 ceph-mon[34815]: pgmap v926: 97 pgs: 97 active+clean; 539 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.3 MiB/s wr, 594 op/s 2024-09-15T23:05:06.662 INFO:tasks.workunit.client.0.smithi111.stdout: 10 119222 17.88 MB/sec execute 214 sec latency 55.110 ms 2024-09-15T23:05:07.662 INFO:tasks.workunit.client.0.smithi111.stdout: 10 119732 17.93 MB/sec execute 215 sec latency 144.981 ms 2024-09-15T23:05:08.214 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:07 smithi111 ceph-mon[30629]: pgmap v927: 97 pgs: 97 active+clean; 540 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.9 MiB/s wr, 554 op/s 2024-09-15T23:05:08.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:07 smithi192 ceph-mon[35427]: pgmap v927: 97 pgs: 97 active+clean; 540 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.9 MiB/s wr, 554 op/s 2024-09-15T23:05:08.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:07 smithi121 ceph-mon[34815]: pgmap v927: 97 pgs: 97 active+clean; 540 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.9 MiB/s wr, 554 op/s 2024-09-15T23:05:08.662 INFO:tasks.workunit.client.0.smithi111.stdout: 10 120209 17.94 MB/sec execute 216 sec latency 136.596 ms 2024-09-15T23:05:09.662 INFO:tasks.workunit.client.0.smithi111.stdout: 10 120997 18.05 MB/sec execute 217 sec latency 59.669 ms 2024-09-15T23:05:10.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:09 smithi192 ceph-mon[35427]: pgmap v928: 97 pgs: 97 active+clean; 530 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.5 MiB/s wr, 613 op/s 2024-09-15T23:05:10.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:09 smithi121 ceph-mon[34815]: pgmap v928: 97 pgs: 97 active+clean; 530 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.5 MiB/s wr, 613 op/s 2024-09-15T23:05:10.418 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:09 smithi111 ceph-mon[30629]: pgmap v928: 97 pgs: 97 active+clean; 530 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.5 MiB/s wr, 613 op/s 2024-09-15T23:05:10.663 INFO:tasks.workunit.client.0.smithi111.stdout: 10 121866 18.20 MB/sec execute 218 sec latency 40.492 ms 2024-09-15T23:05:11.663 INFO:tasks.workunit.client.0.smithi111.stdout: 10 122668 18.28 MB/sec execute 219 sec latency 62.234 ms 2024-09-15T23:05:12.268 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:11 smithi111 ceph-mon[30629]: pgmap v929: 97 pgs: 97 active+clean; 537 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 4.3 MiB/s wr, 602 op/s 2024-09-15T23:05:12.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:11 smithi192 ceph-mon[35427]: pgmap v929: 97 pgs: 97 active+clean; 537 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 4.3 MiB/s wr, 602 op/s 2024-09-15T23:05:12.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:11 smithi121 ceph-mon[34815]: pgmap v929: 97 pgs: 97 active+clean; 537 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 4.3 MiB/s wr, 602 op/s 2024-09-15T23:05:12.663 INFO:tasks.workunit.client.0.smithi111.stdout: 10 123346 18.41 MB/sec execute 220 sec latency 186.465 ms 2024-09-15T23:05:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.391+0000 7f8f6be00640 1 -- 172.21.15.111:0/2634488239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f6c100b50 msgr2=0x7f8f6c100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.391+0000 7f8f6be00640 1 --2- 172.21.15.111:0/2634488239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f6c100b50 0x7f8f6c100fb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009900 tx=0x7f8f5802a3f0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.391+0000 7f8f6be00640 1 -- 172.21.15.111:0/2634488239 shutdown_connections 2024-09-15T23:05:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.391+0000 7f8f6be00640 1 --2- 172.21.15.111:0/2634488239 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f6c1014f0 0x7f8f6c1019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.391+0000 7f8f6be00640 1 --2- 172.21.15.111:0/2634488239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f6c100b50 0x7f8f6c100fb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.391+0000 7f8f6be00640 1 --2- 172.21.15.111:0/2634488239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f6c100210 0x7f8f6c100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.391+0000 7f8f6be00640 1 -- 172.21.15.111:0/2634488239 >> 172.21.15.111:0/2634488239 conn(0x7f8f6c0fbfa0 msgr2=0x7f8f6c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.391+0000 7f8f6be00640 1 -- 172.21.15.111:0/2634488239 shutdown_connections 2024-09-15T23:05:13.393 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.391+0000 7f8f6be00640 1 -- 172.21.15.111:0/2634488239 wait complete. 2024-09-15T23:05:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.392+0000 7f8f6be00640 1 Processor -- start 2024-09-15T23:05:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.392+0000 7f8f6be00640 1 -- start start 2024-09-15T23:05:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.393+0000 7f8f6be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f6c100210 0x7f8f6c199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:13.395 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.393+0000 7f8f6be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f6c100b50 0x7f8f6c19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.393+0000 7f8f6be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f6c1014f0 0x7f8f6c1a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.393+0000 7f8f6be00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c10f780 con 0x7f8f6c100210 2024-09-15T23:05:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.393+0000 7f8f6be00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c10f600 con 0x7f8f6c100b50 2024-09-15T23:05:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.393+0000 7f8f6be00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c10f900 con 0x7f8f6c1014f0 2024-09-15T23:05:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.393+0000 7f8f6a000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f6c100b50 0x7f8f6c19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.393+0000 7f8f6a000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f6c100b50 0x7f8f6c19a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:47554/0 (socket says 172.21.15.111:47554) 2024-09-15T23:05:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.393+0000 7f8f6b400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f6c1014f0 0x7f8f6c1a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.393+0000 7f8f6a000640 1 -- 172.21.15.111:0/2919253734 learned_addr learned my addr 172.21.15.111:0/2919253734 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:05:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.393+0000 7f8f6aa00640 1 --2- 172.21.15.111:0/2919253734 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f6c100210 0x7f8f6c199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.393+0000 7f8f6a000640 1 -- 172.21.15.111:0/2919253734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f6c1014f0 msgr2=0x7f8f6c1a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:13.396 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.393+0000 7f8f6a000640 1 --2- 172.21.15.111:0/2919253734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f6c1014f0 0x7f8f6c1a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.393+0000 7f8f6a000640 1 -- 172.21.15.111:0/2919253734 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f6c100210 msgr2=0x7f8f6c199e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.393+0000 7f8f6a000640 1 --2- 172.21.15.111:0/2919253734 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f6c100210 0x7f8f6c199e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.393+0000 7f8f6a000640 1 -- 172.21.15.111:0/2919253734 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6c1a1a50 con 0x7f8f6c100b50 2024-09-15T23:05:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.394+0000 7f8f6a000640 1 --2- 172.21.15.111:0/2919253734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f6c100b50 0x7f8f6c19a370 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8f58031440 tx=0x7f8f58032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.394+0000 7f8f63400640 1 -- 172.21.15.111:0/2919253734 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5803f030 con 0x7f8f6c100b50 2024-09-15T23:05:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.394+0000 7f8f63400640 1 -- 172.21.15.111:0/2919253734 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f58003ce0 con 0x7f8f6c100b50 2024-09-15T23:05:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.394+0000 7f8f63400640 1 -- 172.21.15.111:0/2919253734 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58035070 con 0x7f8f6c100b50 2024-09-15T23:05:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.395+0000 7f8f6be00640 1 -- 172.21.15.111:0/2919253734 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f6c1a1ce0 con 0x7f8f6c100b50 2024-09-15T23:05:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.395+0000 7f8f6be00640 1 -- 172.21.15.111:0/2919253734 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f6c1a2180 con 0x7f8f6c100b50 2024-09-15T23:05:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.398+0000 7f8f74a9b740 1 Processor -- start 2024-09-15T23:05:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.398+0000 7f8f74a9b740 1 -- start start 2024-09-15T23:05:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.398+0000 7f8f74a9b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3c3c9f260 0x55a3c3c9f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.398+0000 7f8f74a9b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c3c9fba0 0x55a3c3ca4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.398+0000 7f8f74a9b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3c3ca4c30 0x55a3c3ca70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.398+0000 7f8f74a9b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a3c3c04100 con 0x55a3c3c9fba0 2024-09-15T23:05:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.398+0000 7f8f74a9b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a3c3bac1b0 con 0x55a3c3ca4c30 2024-09-15T23:05:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.398+0000 7f8f74a9b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a3c3c000c0 con 0x55a3c3c9f260 2024-09-15T23:05:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.399+0000 7f8f6aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c3c9fba0 0x55a3c3ca4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.399+0000 7f8f6b400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3c3c9f260 0x55a3c3c9f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.399+0000 7f8f6aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c3c9fba0 0x55a3c3ca4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:34710/0 (socket says 172.21.15.111:34710) 2024-09-15T23:05:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.399+0000 7f8f6aa00640 1 -- 172.21.15.111:0/1392680727 learned_addr learned my addr 172.21.15.111:0/1392680727 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:05:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.399+0000 7f8f63400640 1 -- 172.21.15.111:0/2919253734 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f8f58042030 con 0x7f8f6c100b50 2024-09-15T23:05:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.399+0000 7f8f6b400640 1 -- 172.21.15.111:0/1392680727 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3c3ca4c30 msgr2=0x55a3c3ca70f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T23:05:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.399+0000 7f8f6b400640 1 --2- 172.21.15.111:0/1392680727 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3c3ca4c30 0x55a3c3ca70f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.399+0000 7f8f6b400640 1 -- 172.21.15.111:0/1392680727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c3c9fba0 msgr2=0x55a3c3ca4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.399+0000 7f8f6b400640 1 --2- 172.21.15.111:0/1392680727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c3c9fba0 0x55a3c3ca4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.399+0000 7f8f6b400640 1 -- 172.21.15.111:0/1392680727 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3c3917210 con 0x55a3c3c9f260 2024-09-15T23:05:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.399+0000 7f8f6aa00640 1 --2- 172.21.15.111:0/1392680727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c3c9fba0 0x55a3c3ca4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:05:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.400+0000 7f8f6b400640 1 --2- 172.21.15.111:0/1392680727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3c3c9f260 0x55a3c3c9f660 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009f10 tx=0x7f8f64011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3567bc61858de61 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.400+0000 7f8f57e00640 1 -- 172.21.15.111:0/1392680727 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f64003c30 con 0x55a3c3c9f260 2024-09-15T23:05:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.400+0000 7f8f57e00640 1 -- 172.21.15.111:0/1392680727 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f64003dd0 con 0x55a3c3c9f260 2024-09-15T23:05:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.400+0000 7f8f57e00640 1 -- 172.21.15.111:0/1392680727 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6400fab0 con 0x55a3c3c9f260 2024-09-15T23:05:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.401+0000 7f8f74a9b740 1 -- 172.21.15.111:0/1392680727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3c3c9f260 msgr2=0x55a3c3c9f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.401+0000 7f8f74a9b740 1 --2- 172.21.15.111:0/1392680727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3c3c9f260 0x55a3c3c9f660 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009f10 tx=0x7f8f64011810 comp rx=0 tx=0).stop 2024-09-15T23:05:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.402+0000 7f8f63400640 1 --2- 172.21.15.111:0/2919253734 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8f500a32c0 0x7f8f500a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.402+0000 7f8f63400640 1 -- 172.21.15.111:0/2919253734 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f8f580f0070 con 0x7f8f6c100b50 2024-09-15T23:05:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.402+0000 7f8f74a9b740 1 -- 172.21.15.111:0/1392680727 shutdown_connections 2024-09-15T23:05:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.402+0000 7f8f74a9b740 1 --2- 172.21.15.111:0/1392680727 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3c3ca4c30 0x55a3c3ca70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.402+0000 7f8f74a9b740 1 --2- 172.21.15.111:0/1392680727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c3c9fba0 0x55a3c3ca4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.402+0000 7f8f74a9b740 1 --2- 172.21.15.111:0/1392680727 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3c3c9f260 0x55a3c3c9f660 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.402+0000 7f8f74a9b740 1 -- 172.21.15.111:0/1392680727 >> 172.21.15.111:0/1392680727 conn(0x55a3c3c9a8d0 msgr2=0x55a3c3c9acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.402+0000 7f8f6b400640 1 --2- 172.21.15.111:0/2919253734 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8f500a32c0 0x7f8f500a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.402+0000 7f8f74a9b740 1 -- 172.21.15.111:0/1392680727 shutdown_connections 2024-09-15T23:05:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.402+0000 7f8f74a9b740 1 -- 172.21.15.111:0/1392680727 wait complete. 2024-09-15T23:05:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.403+0000 7f8f74a9b740 1 -- start start 2024-09-15T23:05:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.403+0000 7f8f6b400640 1 --2- 172.21.15.111:0/2919253734 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8f500a32c0 0x7f8f500a5780 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009d50 tx=0x7f8f64000f40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.403+0000 7f8f74a9b740 1 Processor -- start 2024-09-15T23:05:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.404+0000 7f8f74a9b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3c3c9f260 0x55a3c3ca94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.404+0000 7f8f74a9b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c3c9fba0 0x55a3c3ca99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.404+0000 7f8f74a9b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3c3ca4c30 0x55a3c3cb0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.404+0000 7f8f74a9b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a3c3ca7c00 con 0x55a3c3c9fba0 2024-09-15T23:05:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.404+0000 7f8f74a9b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a3c3b8d090 con 0x55a3c3c9f260 2024-09-15T23:05:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.404+0000 7f8f74a9b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a3c3ca7d80 con 0x55a3c3ca4c30 2024-09-15T23:05:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.404+0000 7f8f74a9b740 10 client.0 _ll_register_callbacks cb 0x55a3c3be1f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:05:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.404+0000 7f8f6aa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3c3c9f260 0x55a3c3ca94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:13.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.404+0000 7f8f6a000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c3c9fba0 0x55a3c3ca99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.404+0000 7f8f6aa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3c3c9f260 0x55a3c3ca94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:47578/0 (socket says 172.21.15.111:47578) 2024-09-15T23:05:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.404+0000 7f8f6aa00640 1 -- 172.21.15.111:0/4141287926 learned_addr learned my addr 172.21.15.111:0/4141287926 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:05:13.407 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.404+0000 7f8f6b400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3c3ca4c30 0x55a3c3cb0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.404+0000 7f8f6a000640 1 -- 172.21.15.111:0/4141287926 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3c3ca4c30 msgr2=0x55a3c3cb0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.404+0000 7f8f6a000640 1 --2- 172.21.15.111:0/4141287926 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3c3ca4c30 0x55a3c3cb0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.405+0000 7f8f6a000640 1 -- 172.21.15.111:0/4141287926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3c3c9f260 msgr2=0x55a3c3ca94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.405+0000 7f8f6a000640 1 --2- 172.21.15.111:0/4141287926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3c3c9f260 0x55a3c3ca94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.405+0000 7f8f6a000640 1 -- 172.21.15.111:0/4141287926 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3c39e3620 con 0x55a3c3c9fba0 2024-09-15T23:05:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.405+0000 7f8f6b400640 1 --2- 172.21.15.111:0/4141287926 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3c3ca4c30 0x55a3c3cb0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:05:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.405+0000 7f8f6aa00640 1 --2- 172.21.15.111:0/4141287926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3c3c9f260 0x55a3c3ca94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:05:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.405+0000 7f8f6a000640 1 --2- 172.21.15.111:0/4141287926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c3c9fba0 0x55a3c3ca99e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8f580edc40 tx=0x7f8f58009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ddad54b1c67aa9a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.405+0000 7f8f54c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:05:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.405+0000 7f8f54c00640 1 -- 172.21.15.111:0/4141287926 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58049070 con 0x55a3c3c9fba0 2024-09-15T23:05:13.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.405+0000 7f8f54c00640 1 -- 172.21.15.111:0/4141287926 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f8f58003800 con 0x55a3c3c9fba0 2024-09-15T23:05:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.405+0000 7f8f74a9b740 1 -- 172.21.15.111:0/4141287926 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f64003dd0 con 0x55a3c3c9fba0 2024-09-15T23:05:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.406+0000 7f8f54c00640 1 -- 172.21.15.111:0/4141287926 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8f580bc070 con 0x55a3c3c9fba0 2024-09-15T23:05:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.406+0000 7f8f74a9b740 10 client.16527 fetch_fsmap learned FSMap version 35 2024-09-15T23:05:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.406+0000 7f8f74a9b740 1 -- 172.21.15.111:0/4141287926 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a3c3cb3810 con 0x55a3c3c9fba0 2024-09-15T23:05:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.406+0000 7f8f54c00640 1 -- 172.21.15.111:0/4141287926 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f64003dd0 con 0x55a3c3c9fba0 2024-09-15T23:05:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.407+0000 7f8f54c00640 1 -- 172.21.15.111:0/4141287926 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f8f580b8d80 con 0x55a3c3c9fba0 2024-09-15T23:05:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.407+0000 7f8f74a9b740 10 client.16527 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:05:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.407+0000 7f8f74a9b740 10 client.16527 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:05:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.407+0000 7f8f74a9b740 20 client.16527 populate_metadata read hostname 'smithi111' 2024-09-15T23:05:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.407+0000 7f8f74a9b740 1 --2- 172.21.15.111:0/4141287926 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a3c3cb4f40 0x55a3c3cb7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.407+0000 7f8f74a9b740 4 client.16527 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:05:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.407+0000 7f8f74a9b740 1 -- 172.21.15.111:0/4141287926 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3c3c11830 con 0x55a3c3cb4f40 2024-09-15T23:05:13.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.407+0000 7f8f6aa00640 1 --2- 172.21.15.111:0/4141287926 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a3c3cb4f40 0x55a3c3cb7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.408+0000 7f8f6aa00640 1 --2- 172.21.15.111:0/4141287926 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a3c3cb4f40 0x55a3c3cb7360 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90b8ef8bf80c4276 server_cookie=18d5e3a6a1529e67 in_seq=0 out_seq=0 2024-09-15T23:05:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.408+0000 7f8f54c00640 10 client.16527 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:05:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.409+0000 7f8f54c00640 1 -- 172.21.15.111:0/4141287926 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a3c3c11830 con 0x55a3c3cb4f40 2024-09-15T23:05:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.409+0000 7f8f54c00640 10 client.16527 handle_command_reply: tid=0 2024-09-15T23:05:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.409+0000 7f8f74a9b740 1 client.16527 shutdown 2024-09-15T23:05:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.409+0000 7f8f74a9b740 1 -- 172.21.15.111:0/4141287926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c3c9fba0 msgr2=0x55a3c3ca99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.409+0000 7f8f74a9b740 1 --2- 172.21.15.111:0/4141287926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c3c9fba0 0x55a3c3ca99e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8f580edc40 tx=0x7f8f58009620 comp rx=0 tx=0).stop 2024-09-15T23:05:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.409+0000 7f8f74a9b740 1 -- 172.21.15.111:0/4141287926 shutdown_connections 2024-09-15T23:05:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.409+0000 7f8f74a9b740 1 --2- 172.21.15.111:0/4141287926 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a3c3cb4f40 0x55a3c3cb7360 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.409+0000 7f8f74a9b740 1 --2- 172.21.15.111:0/4141287926 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3c3ca4c30 0x55a3c3cb0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.412 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.409+0000 7f8f74a9b740 1 --2- 172.21.15.111:0/4141287926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c3c9fba0 0x55a3c3ca99e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.409+0000 7f8f74a9b740 1 --2- 172.21.15.111:0/4141287926 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3c3c9f260 0x55a3c3ca94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.409+0000 7f8f54c00640 0 client.16527 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:05:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.409+0000 7f8f74a9b740 1 -- 172.21.15.111:0/4141287926 >> 172.21.15.111:0/4141287926 conn(0x55a3c3c9a8d0 msgr2=0x55a3c3c9c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.409+0000 7f8f74a9b740 1 -- 172.21.15.111:0/4141287926 shutdown_connections 2024-09-15T23:05:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.409+0000 7f8f74a9b740 1 -- 172.21.15.111:0/4141287926 wait complete. 2024-09-15T23:05:13.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.410+0000 7f8f74a9b740 20 client.16527 trim_cache size 0 max 16384 2024-09-15T23:05:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.429+0000 7f8f74a9b740 1 Processor -- start 2024-09-15T23:05:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.429+0000 7f8f74a9b740 1 -- start start 2024-09-15T23:05:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.430+0000 7f8f74a9b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3c3c9a8d0 0x55a3c3c9a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.430+0000 7f8f74a9b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c3c9f260 0x55a3c3cb09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.430+0000 7f8f74a9b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3c3c9fba0 0x55a3c3c9bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.430+0000 7f8f74a9b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c1ad760 con 0x55a3c3c9f260 2024-09-15T23:05:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.430+0000 7f8f74a9b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c1ab030 con 0x55a3c3c9fba0 2024-09-15T23:05:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.430+0000 7f8f74a9b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c1a8ea0 con 0x55a3c3c9a8d0 2024-09-15T23:05:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.430+0000 7f8f6aa00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3c3c9a8d0 0x55a3c3c9a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.430+0000 7f8f6a000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c3c9f260 0x55a3c3cb09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.430+0000 7f8f6aa00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3c3c9a8d0 0x55a3c3c9a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:53692/0 (socket says 172.21.15.111:53692) 2024-09-15T23:05:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.430+0000 7f8f6aa00640 1 -- 172.21.15.111:0/774844216 learned_addr learned my addr 172.21.15.111:0/774844216 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:05:13.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.430+0000 7f8f6b400640 1 --2- 172.21.15.111:0/774844216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3c3c9fba0 0x55a3c3c9bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.431+0000 7f8f6b400640 1 -- 172.21.15.111:0/774844216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3c3c9a8d0 msgr2=0x55a3c3c9a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.431+0000 7f8f6b400640 1 --2- 172.21.15.111:0/774844216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3c3c9a8d0 0x55a3c3c9a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.431+0000 7f8f6b400640 1 -- 172.21.15.111:0/774844216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c3c9f260 msgr2=0x55a3c3cb09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.431+0000 7f8f6b400640 1 --2- 172.21.15.111:0/774844216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c3c9f260 0x55a3c3cb09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.431+0000 7f8f6b400640 1 -- 172.21.15.111:0/774844216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3c3cb79e0 con 0x55a3c3c9fba0 2024-09-15T23:05:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.431+0000 7f8f6aa00640 1 --2- 172.21.15.111:0/774844216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3c3c9a8d0 0x55a3c3c9a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:05:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.431+0000 7f8f6b400640 1 --2- 172.21.15.111:0/774844216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3c3c9fba0 0x55a3c3c9bcc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8f640049d0 tx=0x7f8f64023000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26ed4f70303ff19a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.431+0000 7f8f57400640 1 -- 172.21.15.111:0/774844216 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6401f040 con 0x55a3c3c9fba0 2024-09-15T23:05:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.432+0000 7f8f57400640 1 -- 172.21.15.111:0/774844216 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f64037070 con 0x55a3c3c9fba0 2024-09-15T23:05:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.432+0000 7f8f57400640 1 -- 172.21.15.111:0/774844216 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f64005330 con 0x55a3c3c9fba0 2024-09-15T23:05:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.432+0000 7f8f74a9b740 1 -- 172.21.15.111:0/774844216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3c3c9fba0 msgr2=0x55a3c3c9bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:13.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.432+0000 7f8f74a9b740 1 --2- 172.21.15.111:0/774844216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3c3c9fba0 0x55a3c3c9bcc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8f640049d0 tx=0x7f8f64023000 comp rx=0 tx=0).stop 2024-09-15T23:05:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.432+0000 7f8f74a9b740 1 -- 172.21.15.111:0/774844216 shutdown_connections 2024-09-15T23:05:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.432+0000 7f8f74a9b740 1 --2- 172.21.15.111:0/774844216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3c3c9fba0 0x55a3c3c9bcc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.432+0000 7f8f74a9b740 1 --2- 172.21.15.111:0/774844216 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c3c9f260 0x55a3c3cb09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.432+0000 7f8f74a9b740 1 --2- 172.21.15.111:0/774844216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3c3c9a8d0 0x55a3c3c9a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.432+0000 7f8f74a9b740 1 -- 172.21.15.111:0/774844216 >> 172.21.15.111:0/774844216 conn(0x55a3c3ca47c0 msgr2=0x55a3c3c99a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.432+0000 7f8f74a9b740 1 -- 172.21.15.111:0/774844216 shutdown_connections 2024-09-15T23:05:13.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.433+0000 7f8f74a9b740 1 -- 172.21.15.111:0/774844216 wait complete. 2024-09-15T23:05:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.433+0000 7f8f74a9b740 1 -- start start 2024-09-15T23:05:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.433+0000 7f8f74a9b740 1 Processor -- start 2024-09-15T23:05:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.433+0000 7f8f74a9b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c3c9a8d0 0x55a3c3cadd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.433+0000 7f8f74a9b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3c3c9f260 0x55a3c3cae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.433+0000 7f8f74a9b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3c3c9fba0 0x55a3c3caf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.433+0000 7f8f74a9b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c1a94f0 con 0x55a3c3c9a8d0 2024-09-15T23:05:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.433+0000 7f8f74a9b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c1aa960 con 0x55a3c3c9f260 2024-09-15T23:05:13.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.433+0000 7f8f74a9b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c1a9bc0 con 0x55a3c3c9fba0 2024-09-15T23:05:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.433+0000 7f8f74a9b740 10 client.0 _ll_register_callbacks cb 0x7f8f58003f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:05:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.434+0000 7f8f6a000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3c3c9f260 0x55a3c3cae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.434+0000 7f8f6a000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3c3c9f260 0x55a3c3cae280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:47594/0 (socket says 172.21.15.111:47594) 2024-09-15T23:05:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.434+0000 7f8f6aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c3c9a8d0 0x55a3c3cadd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.434+0000 7f8f6a000640 1 -- 172.21.15.111:0/682005717 learned_addr learned my addr 172.21.15.111:0/682005717 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:05:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.434+0000 7f8f6b400640 1 --2- 172.21.15.111:0/682005717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3c3c9fba0 0x55a3c3caf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.434+0000 7f8f6a000640 1 -- 172.21.15.111:0/682005717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3c3c9fba0 msgr2=0x55a3c3caf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.434+0000 7f8f6a000640 1 --2- 172.21.15.111:0/682005717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3c3c9fba0 0x55a3c3caf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.434+0000 7f8f6a000640 1 -- 172.21.15.111:0/682005717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c3c9a8d0 msgr2=0x55a3c3cadd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.434+0000 7f8f6a000640 1 --2- 172.21.15.111:0/682005717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c3c9a8d0 0x55a3c3cadd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.434+0000 7f8f6a000640 1 -- 172.21.15.111:0/682005717 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3c3caf8a0 con 0x55a3c3c9f260 2024-09-15T23:05:13.439 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.434+0000 7f8f6aa00640 1 --2- 172.21.15.111:0/682005717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c3c9a8d0 0x55a3c3cadd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:05:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.434+0000 7f8f6a000640 1 --2- 172.21.15.111:0/682005717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3c3c9f260 0x55a3c3cae280 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8f580db2a0 tx=0x7f8f58005a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c416eb7f69256956 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.434+0000 7f8f54c00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:05:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.434+0000 7f8f54c00640 1 -- 172.21.15.111:0/682005717 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58049070 con 0x55a3c3c9f260 2024-09-15T23:05:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.435+0000 7f8f74a9b740 1 -- 172.21.15.111:0/682005717 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f64037070 con 0x55a3c3c9f260 2024-09-15T23:05:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.435+0000 7f8f54c00640 1 -- 172.21.15.111:0/682005717 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f8f5803bc80 con 0x55a3c3c9f260 2024-09-15T23:05:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.435+0000 7f8f54c00640 1 -- 172.21.15.111:0/682005717 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8f580ca050 con 0x55a3c3c9f260 2024-09-15T23:05:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.435+0000 7f8f74a9b740 10 client.26480 fetch_fsmap learned FSMap version 35 2024-09-15T23:05:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.435+0000 7f8f74a9b740 1 -- 172.21.15.111:0/682005717 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a3c3cb0c20 con 0x55a3c3c9f260 2024-09-15T23:05:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.435+0000 7f8f54c00640 1 -- 172.21.15.111:0/682005717 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f64037070 con 0x55a3c3c9f260 2024-09-15T23:05:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.435+0000 7f8f54c00640 1 -- 172.21.15.111:0/682005717 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f8f580bc9f0 con 0x55a3c3c9f260 2024-09-15T23:05:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.435+0000 7f8f74a9b740 10 client.26480 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:05:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.435+0000 7f8f74a9b740 10 client.26480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:05:13.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.435+0000 7f8f74a9b740 20 client.26480 populate_metadata read hostname 'smithi111' 2024-09-15T23:05:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.435+0000 7f8f74a9b740 1 --2- 172.21.15.111:0/682005717 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a3c3ca4c30 0x55a3c3d22190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.435+0000 7f8f74a9b740 4 client.26480 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:05:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.435+0000 7f8f74a9b740 1 -- 172.21.15.111:0/682005717 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a3c3c0bb80 con 0x55a3c3ca4c30 2024-09-15T23:05:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.436+0000 7f8f6aa00640 1 --2- 172.21.15.111:0/682005717 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a3c3ca4c30 0x55a3c3d22190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.436+0000 7f8f6aa00640 1 --2- 172.21.15.111:0/682005717 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a3c3ca4c30 0x55a3c3d22190 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8160c935ecf2da37 server_cookie=a1c31109141d3ab8 in_seq=0 out_seq=0 2024-09-15T23:05:13.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.436+0000 7f8f54c00640 10 client.26480 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:05:13.663 INFO:tasks.workunit.client.0.smithi111.stdout: 10 123889 18.46 MB/sec execute 221 sec latency 214.771 ms 2024-09-15T23:05:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.721+0000 7f8f54c00640 1 -- 172.21.15.111:0/682005717 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a3c3c0bb80 con 0x55a3c3ca4c30 2024-09-15T23:05:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.721+0000 7f8f54c00640 10 client.26480 handle_command_reply: tid=0 2024-09-15T23:05:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.721+0000 7f8f74a9b740 1 client.26480 shutdown 2024-09-15T23:05:13.723 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.721+0000 7f8f74a9b740 1 -- 172.21.15.111:0/682005717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3c3c9f260 msgr2=0x55a3c3cae280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.721+0000 7f8f74a9b740 1 --2- 172.21.15.111:0/682005717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3c3c9f260 0x55a3c3cae280 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8f580db2a0 tx=0x7f8f58005a40 comp rx=0 tx=0).stop 2024-09-15T23:05:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.722+0000 7f8f74a9b740 1 -- 172.21.15.111:0/682005717 shutdown_connections 2024-09-15T23:05:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.722+0000 7f8f74a9b740 1 --2- 172.21.15.111:0/682005717 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a3c3ca4c30 0x55a3c3d22190 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.722+0000 7f8f74a9b740 1 --2- 172.21.15.111:0/682005717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3c3c9fba0 0x55a3c3caf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.722+0000 7f8f74a9b740 1 --2- 172.21.15.111:0/682005717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3c3c9f260 0x55a3c3cae280 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.722+0000 7f8f54c00640 0 client.26480 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:05:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.722+0000 7f8f74a9b740 1 --2- 172.21.15.111:0/682005717 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3c3c9a8d0 0x55a3c3cadd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.722+0000 7f8f74a9b740 1 -- 172.21.15.111:0/682005717 >> 172.21.15.111:0/682005717 conn(0x55a3c3ca47c0 msgr2=0x55a3c3c9a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.722+0000 7f8f74a9b740 1 -- 172.21.15.111:0/682005717 shutdown_connections 2024-09-15T23:05:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.722+0000 7f8f74a9b740 1 -- 172.21.15.111:0/682005717 wait complete. 2024-09-15T23:05:13.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.722+0000 7f8f74a9b740 20 client.26480 trim_cache size 0 max 16384 2024-09-15T23:05:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:05:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:05:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:05:13.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:05:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.723+0000 7f8f6be00640 1 -- 172.21.15.111:0/2919253734 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8f500a32c0 msgr2=0x7f8f500a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.723+0000 7f8f6be00640 1 --2- 172.21.15.111:0/2919253734 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8f500a32c0 0x7f8f500a5780 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009d50 tx=0x7f8f64000f40 comp rx=0 tx=0).stop 2024-09-15T23:05:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.723+0000 7f8f6be00640 1 -- 172.21.15.111:0/2919253734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f6c100b50 msgr2=0x7f8f6c19a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.723+0000 7f8f6be00640 1 --2- 172.21.15.111:0/2919253734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f6c100b50 0x7f8f6c19a370 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8f58031440 tx=0x7f8f58032ce0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.723+0000 7f8f6be00640 1 -- 172.21.15.111:0/2919253734 shutdown_connections 2024-09-15T23:05:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.723+0000 7f8f6be00640 1 --2- 172.21.15.111:0/2919253734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f6c1014f0 0x7f8f6c1a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.723+0000 7f8f6be00640 1 --2- 172.21.15.111:0/2919253734 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8f500a32c0 0x7f8f500a5780 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.723+0000 7f8f6be00640 1 --2- 172.21.15.111:0/2919253734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f6c100b50 0x7f8f6c19a370 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.723+0000 7f8f6be00640 1 --2- 172.21.15.111:0/2919253734 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8f6c100210 0x7f8f6c199e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.723+0000 7f8f6be00640 1 -- 172.21.15.111:0/2919253734 >> 172.21.15.111:0/2919253734 conn(0x7f8f6c0fbfa0 msgr2=0x7f8f6c0fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.724+0000 7f8f6be00640 1 -- 172.21.15.111:0/2919253734 shutdown_connections 2024-09-15T23:05:13.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:13.724+0000 7f8f6be00640 1 -- 172.21.15.111:0/2919253734 wait complete. 2024-09-15T23:05:14.280 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:13 smithi111 ceph-mon[30629]: pgmap v930: 97 pgs: 97 active+clean; 542 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 4.1 MiB/s wr, 545 op/s 2024-09-15T23:05:14.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:13 smithi192 ceph-mon[35427]: pgmap v930: 97 pgs: 97 active+clean; 542 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 4.1 MiB/s wr, 545 op/s 2024-09-15T23:05:14.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:13 smithi121 ceph-mon[34815]: pgmap v930: 97 pgs: 97 active+clean; 542 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 4.1 MiB/s wr, 545 op/s 2024-09-15T23:05:14.663 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124299 18.46 MB/sec execute 222 sec latency 420.464 ms 2024-09-15T23:05:15.663 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124300 18.38 MB/sec execute 223 sec latency 1318.645 ms 2024-09-15T23:05:16.138 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:15 smithi111 ceph-mon[30629]: pgmap v931: 97 pgs: 97 active+clean; 551 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 4.4 MiB/s wr, 540 op/s 2024-09-15T23:05:16.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:15 smithi192 ceph-mon[35427]: pgmap v931: 97 pgs: 97 active+clean; 551 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 4.4 MiB/s wr, 540 op/s 2024-09-15T23:05:16.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:15 smithi121 ceph-mon[34815]: pgmap v931: 97 pgs: 97 active+clean; 551 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 4.4 MiB/s wr, 540 op/s 2024-09-15T23:05:16.663 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124300 18.30 MB/sec execute 224 sec latency 2318.717 ms 2024-09-15T23:05:17.663 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124300 18.21 MB/sec execute 225 sec latency 3318.758 ms 2024-09-15T23:05:18.136 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:17 smithi111 ceph-mon[30629]: pgmap v932: 97 pgs: 97 active+clean; 551 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 5.5 MiB/s wr, 549 op/s 2024-09-15T23:05:18.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:17 smithi192 ceph-mon[35427]: pgmap v932: 97 pgs: 97 active+clean; 551 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 5.5 MiB/s wr, 549 op/s 2024-09-15T23:05:18.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:17 smithi121 ceph-mon[34815]: pgmap v932: 97 pgs: 97 active+clean; 551 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 5.5 MiB/s wr, 549 op/s 2024-09-15T23:05:18.663 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124303 18.13 MB/sec execute 226 sec latency 4318.800 ms 2024-09-15T23:05:19.663 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124309 18.05 MB/sec execute 227 sec latency 5223.609 ms 2024-09-15T23:05:20.121 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:19 smithi111 ceph-mon[30629]: pgmap v933: 97 pgs: 97 active+clean; 560 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.6 MiB/s wr, 576 op/s 2024-09-15T23:05:20.316 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:19 smithi192 ceph-mon[35427]: pgmap v933: 97 pgs: 97 active+clean; 560 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.6 MiB/s wr, 576 op/s 2024-09-15T23:05:20.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:19 smithi121 ceph-mon[34815]: pgmap v933: 97 pgs: 97 active+clean; 560 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.6 MiB/s wr, 576 op/s 2024-09-15T23:05:20.663 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124319 17.98 MB/sec execute 228 sec latency 6223.737 ms 2024-09-15T23:05:21.663 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124319 17.90 MB/sec execute 229 sec latency 7223.793 ms 2024-09-15T23:05:22.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:21 smithi192 ceph-mon[35427]: pgmap v934: 97 pgs: 97 active+clean; 561 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.8 MiB/s wr, 511 op/s 2024-09-15T23:05:22.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:21 smithi121 ceph-mon[34815]: pgmap v934: 97 pgs: 97 active+clean; 561 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.8 MiB/s wr, 511 op/s 2024-09-15T23:05:22.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:21 smithi111 ceph-mon[30629]: pgmap v934: 97 pgs: 97 active+clean; 561 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 6.8 MiB/s wr, 511 op/s 2024-09-15T23:05:22.663 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124319 17.82 MB/sec execute 230 sec latency 8223.863 ms 2024-09-15T23:05:23.664 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124319 17.74 MB/sec execute 231 sec latency 9223.931 ms 2024-09-15T23:05:24.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:23 smithi192 ceph-mon[35427]: pgmap v935: 97 pgs: 97 active+clean; 556 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.6 MiB/s wr, 439 op/s 2024-09-15T23:05:24.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:23 smithi121 ceph-mon[34815]: pgmap v935: 97 pgs: 97 active+clean; 556 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.6 MiB/s wr, 439 op/s 2024-09-15T23:05:24.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:23 smithi111 ceph-mon[30629]: pgmap v935: 97 pgs: 97 active+clean; 556 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.6 MiB/s wr, 439 op/s 2024-09-15T23:05:24.664 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124319 17.67 MB/sec execute 232 sec latency 10224.006 ms 2024-09-15T23:05:25.664 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124319 17.59 MB/sec execute 233 sec latency 11224.061 ms 2024-09-15T23:05:26.184 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:26 smithi111 ceph-mon[30629]: pgmap v936: 97 pgs: 97 active+clean; 542 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.1 MiB/s wr, 424 op/s 2024-09-15T23:05:26.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:26 smithi192 ceph-mon[35427]: pgmap v936: 97 pgs: 97 active+clean; 542 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.1 MiB/s wr, 424 op/s 2024-09-15T23:05:26.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:26 smithi121 ceph-mon[34815]: pgmap v936: 97 pgs: 97 active+clean; 542 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.1 MiB/s wr, 424 op/s 2024-09-15T23:05:26.664 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124321 17.52 MB/sec execute 234 sec latency 11602.776 ms 2024-09-15T23:05:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.843+0000 7f87dca00640 1 -- 172.21.15.111:0/3523926176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87d8074fc0 msgr2=0x7f87d810df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.843+0000 7f87dca00640 1 --2- 172.21.15.111:0/3523926176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87d8074fc0 0x7f87d810df40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009900 tx=0x7f87c402a3e0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.843+0000 7f87dca00640 1 -- 172.21.15.111:0/3523926176 shutdown_connections 2024-09-15T23:05:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.843+0000 7f87dca00640 1 --2- 172.21.15.111:0/3523926176 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87d8105340 0x7f87d8110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.843+0000 7f87dca00640 1 --2- 172.21.15.111:0/3523926176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87d8074fc0 0x7f87d810df40 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.843+0000 7f87dca00640 1 --2- 172.21.15.111:0/3523926176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87d8074680 0x7f87d8074a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.843+0000 7f87dca00640 1 -- 172.21.15.111:0/3523926176 >> 172.21.15.111:0/3523926176 conn(0x7f87d80fd0d0 msgr2=0x7f87d80ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.843+0000 7f87dca00640 1 -- 172.21.15.111:0/3523926176 shutdown_connections 2024-09-15T23:05:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.843+0000 7f87dca00640 1 -- 172.21.15.111:0/3523926176 wait complete. 2024-09-15T23:05:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.844+0000 7f87dca00640 1 Processor -- start 2024-09-15T23:05:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.844+0000 7f87dca00640 1 -- start start 2024-09-15T23:05:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.844+0000 7f87dca00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87d8074680 0x7f87d819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.844+0000 7f87dca00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87d8074fc0 0x7f87d819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.844+0000 7f87dca00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87d81a3980 0x7f87d81a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.844+0000 7f87dca00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f87d8113d90 con 0x7f87d81a3980 2024-09-15T23:05:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.844+0000 7f87dca00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f87d8113c10 con 0x7f87d8074680 2024-09-15T23:05:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.844+0000 7f87dca00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f87d8113f10 con 0x7f87d8074fc0 2024-09-15T23:05:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.845+0000 7f87d7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87d81a3980 0x7f87d81a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.845+0000 7f87d7400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87d8074680 0x7f87d819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.845+0000 7f87d7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87d81a3980 0x7f87d81a5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52422/0 (socket says 172.21.15.111:52422) 2024-09-15T23:05:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.845+0000 7f87d7e00640 1 -- 172.21.15.111:0/3404985828 learned_addr learned my addr 172.21.15.111:0/3404985828 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:05:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.845+0000 7f87d7e00640 1 -- 172.21.15.111:0/3404985828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87d8074fc0 msgr2=0x7f87d819e920 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T23:05:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.845+0000 7f87d7e00640 1 --2- 172.21.15.111:0/3404985828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87d8074fc0 0x7f87d819e920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.845+0000 7f87d7e00640 1 -- 172.21.15.111:0/3404985828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87d8074680 msgr2=0x7f87d819e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.845+0000 7f87d7e00640 1 --2- 172.21.15.111:0/3404985828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87d8074680 0x7f87d819e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.845+0000 7f87d7e00640 1 -- 172.21.15.111:0/3404985828 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d81a63e0 con 0x7f87d81a3980 2024-09-15T23:05:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.845+0000 7f87d7400640 1 --2- 172.21.15.111:0/3404985828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87d8074680 0x7f87d819e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:05:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.845+0000 7f87d7e00640 1 --2- 172.21.15.111:0/3404985828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87d81a3980 0x7f87d81a5d70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f87d000eb10 tx=0x7f87d000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.846+0000 7f87cfe00640 1 -- 172.21.15.111:0/3404985828 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0004b20 con 0x7f87d81a3980 2024-09-15T23:05:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.846+0000 7f87cfe00640 1 -- 172.21.15.111:0/3404985828 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87d0004cc0 con 0x7f87d81a3980 2024-09-15T23:05:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.846+0000 7f87cfe00640 1 -- 172.21.15.111:0/3404985828 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0010760 con 0x7f87d81a3980 2024-09-15T23:05:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.846+0000 7f87dca00640 1 -- 172.21.15.111:0/3404985828 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d81a66d0 con 0x7f87d81a3980 2024-09-15T23:05:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.846+0000 7f87dca00640 1 -- 172.21.15.111:0/3404985828 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d81a6bd0 con 0x7f87d81a3980 2024-09-15T23:05:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.849+0000 7f87e0ec8740 1 Processor -- start 2024-09-15T23:05:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.850+0000 7f87cfe00640 1 -- 172.21.15.111:0/3404985828 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f87d0005020 con 0x7f87d81a3980 2024-09-15T23:05:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.850+0000 7f87e0ec8740 1 -- start start 2024-09-15T23:05:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.850+0000 7f87e0ec8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e7543b260 0x561e7543b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.850+0000 7f87e0ec8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561e7543bba0 0x561e75440660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.850+0000 7f87e0ec8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561e75440c30 0x561e754430f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.850+0000 7f87e0ec8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561e753a0100 con 0x561e75440c30 2024-09-15T23:05:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.850+0000 7f87e0ec8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561e753481b0 con 0x561e7543bba0 2024-09-15T23:05:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.850+0000 7f87e0ec8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561e7539c0c0 con 0x561e7543b260 2024-09-15T23:05:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.851+0000 7f87d7400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561e7543bba0 0x561e75440660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.851+0000 7f87d6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561e75440c30 0x561e754430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.851+0000 7f87d7400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561e7543bba0 0x561e75440660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:55144/0 (socket says 172.21.15.111:55144) 2024-09-15T23:05:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.851+0000 7f87d7400640 1 -- 172.21.15.111:0/435023144 learned_addr learned my addr 172.21.15.111:0/435023144 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:05:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.851+0000 7f87d7400640 1 -- 172.21.15.111:0/435023144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e7543b260 msgr2=0x561e7543b660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:05:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.851+0000 7f87d7400640 1 --2- 172.21.15.111:0/435023144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e7543b260 0x561e7543b660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.851+0000 7f87d7400640 1 -- 172.21.15.111:0/435023144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561e75440c30 msgr2=0x561e754430f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.851+0000 7f87d7400640 1 --2- 172.21.15.111:0/435023144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561e75440c30 0x561e754430f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.851+0000 7f87d7400640 1 -- 172.21.15.111:0/435023144 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e750b3210 con 0x561e7543bba0 2024-09-15T23:05:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.851+0000 7f87d7400640 1 --2- 172.21.15.111:0/435023144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561e7543bba0 0x561e75440660 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f87c80115f0 tx=0x7f87c8011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9450aab3a6ad9610 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.851+0000 7f87cfe00640 1 --2- 172.21.15.111:0/3404985828 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f87bc0a32c0 0x7f87bc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.852+0000 7f87c3e00640 1 -- 172.21.15.111:0/435023144 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c801e070 con 0x561e7543bba0 2024-09-15T23:05:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.852+0000 7f87c3e00640 1 -- 172.21.15.111:0/435023144 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87c800fb20 con 0x561e7543bba0 2024-09-15T23:05:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.852+0000 7f87c3e00640 1 -- 172.21.15.111:0/435023144 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8019c40 con 0x561e7543bba0 2024-09-15T23:05:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.852+0000 7f87cfe00640 1 -- 172.21.15.111:0/3404985828 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f87d00cfef0 con 0x7f87d81a3980 2024-09-15T23:05:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.852+0000 7f87d6a00640 1 --2- 172.21.15.111:0/3404985828 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f87bc0a32c0 0x7f87bc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.852+0000 7f87e0ec8740 1 -- 172.21.15.111:0/435023144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561e7543bba0 msgr2=0x561e75440660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.852+0000 7f87e0ec8740 1 --2- 172.21.15.111:0/435023144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561e7543bba0 0x561e75440660 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f87c80115f0 tx=0x7f87c8011ab0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.852+0000 7f87e0ec8740 1 -- 172.21.15.111:0/435023144 shutdown_connections 2024-09-15T23:05:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.852+0000 7f87e0ec8740 1 --2- 172.21.15.111:0/435023144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561e75440c30 0x561e754430f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.852+0000 7f87e0ec8740 1 --2- 172.21.15.111:0/435023144 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561e7543bba0 0x561e75440660 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.852+0000 7f87e0ec8740 1 --2- 172.21.15.111:0/435023144 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e7543b260 0x561e7543b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.852+0000 7f87e0ec8740 1 -- 172.21.15.111:0/435023144 >> 172.21.15.111:0/435023144 conn(0x561e754368d0 msgr2=0x561e75436cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.853+0000 7f87e0ec8740 1 -- 172.21.15.111:0/435023144 shutdown_connections 2024-09-15T23:05:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.853+0000 7f87d6a00640 1 --2- 172.21.15.111:0/3404985828 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f87bc0a32c0 0x7f87bc0a5780 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009230 tx=0x7f87c4031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.853+0000 7f87e0ec8740 1 -- 172.21.15.111:0/435023144 wait complete. 2024-09-15T23:05:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.853+0000 7f87e0ec8740 1 -- start start 2024-09-15T23:05:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.853+0000 7f87e0ec8740 1 Processor -- start 2024-09-15T23:05:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.854+0000 7f87e0ec8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e7543b260 0x561e75445470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.854+0000 7f87e0ec8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561e7543bba0 0x561e754459b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.854+0000 7f87e0ec8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561e75440c30 0x561e7544c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.854+0000 7f87e0ec8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x561e75443c00 con 0x561e7543bba0 2024-09-15T23:05:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.854+0000 7f87e0ec8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561e75329090 con 0x561e75440c30 2024-09-15T23:05:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.854+0000 7f87e0ec8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x561e75443d80 con 0x561e7543b260 2024-09-15T23:05:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.854+0000 7f87e0ec8740 10 client.0 _ll_register_callbacks cb 0x561e7537df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:05:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.854+0000 7f87d6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561e7543bba0 0x561e754459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.854+0000 7f87d7400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e7543b260 0x561e75445470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.854+0000 7f87d6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561e7543bba0 0x561e754459b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52436/0 (socket says 172.21.15.111:52436) 2024-09-15T23:05:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.854+0000 7f87d6a00640 1 -- 172.21.15.111:0/657005071 learned_addr learned my addr 172.21.15.111:0/657005071 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:05:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.854+0000 7f87d7e00640 1 --2- 172.21.15.111:0/657005071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561e75440c30 0x561e7544c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.854+0000 7f87d6a00640 1 -- 172.21.15.111:0/657005071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e7543b260 msgr2=0x561e75445470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.854+0000 7f87d6a00640 1 --2- 172.21.15.111:0/657005071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e7543b260 0x561e75445470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.854+0000 7f87d6a00640 1 -- 172.21.15.111:0/657005071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561e75440c30 msgr2=0x561e7544c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.854+0000 7f87d6a00640 1 --2- 172.21.15.111:0/657005071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561e75440c30 0x561e7544c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.859 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.854+0000 7f87d6a00640 1 -- 172.21.15.111:0/657005071 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e7517f620 con 0x561e7543bba0 2024-09-15T23:05:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.855+0000 7f87d6a00640 1 --2- 172.21.15.111:0/657005071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561e7543bba0 0x561e754459b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f87c4033000 tx=0x7f87c4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace374625e50ca81 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.855+0000 7f87c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:05:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.855+0000 7f87c0c00640 1 -- 172.21.15.111:0/657005071 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c404d020 con 0x561e7543bba0 2024-09-15T23:05:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.855+0000 7f87c0c00640 1 -- 172.21.15.111:0/657005071 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f87c40037f0 con 0x561e7543bba0 2024-09-15T23:05:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.855+0000 7f87e0ec8740 1 -- 172.21.15.111:0/657005071 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87c800fb20 con 0x561e7543bba0 2024-09-15T23:05:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.855+0000 7f87c0c00640 1 -- 172.21.15.111:0/657005071 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f87c4035040 con 0x561e7543bba0 2024-09-15T23:05:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.855+0000 7f87e0ec8740 10 client.16545 fetch_fsmap learned FSMap version 35 2024-09-15T23:05:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.855+0000 7f87e0ec8740 1 -- 172.21.15.111:0/657005071 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561e7544f850 con 0x561e7543bba0 2024-09-15T23:05:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.856+0000 7f87c0c00640 1 -- 172.21.15.111:0/657005071 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87c800fb20 con 0x561e7543bba0 2024-09-15T23:05:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.856+0000 7f87c0c00640 1 -- 172.21.15.111:0/657005071 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f87c403a040 con 0x561e7543bba0 2024-09-15T23:05:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.856+0000 7f87e0ec8740 10 client.16545 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:05:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.856+0000 7f87e0ec8740 10 client.16545 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:05:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.856+0000 7f87e0ec8740 20 client.16545 populate_metadata read hostname 'smithi111' 2024-09-15T23:05:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.856+0000 7f87e0ec8740 1 --2- 172.21.15.111:0/657005071 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561e75450f70 0x561e754533c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.856+0000 7f87e0ec8740 4 client.16545 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:05:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.856+0000 7f87e0ec8740 1 -- 172.21.15.111:0/657005071 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e753ad830 con 0x561e75450f70 2024-09-15T23:05:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.856+0000 7f87d7400640 1 --2- 172.21.15.111:0/657005071 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561e75450f70 0x561e754533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.857+0000 7f87d7400640 1 --2- 172.21.15.111:0/657005071 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561e75450f70 0x561e754533c0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7528cb95bdfa5bc8 server_cookie=b8539c1d1110c1a5 in_seq=0 out_seq=0 2024-09-15T23:05:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.857+0000 7f87c0c00640 10 client.16545 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:05:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.858+0000 7f87c0c00640 1 -- 172.21.15.111:0/657005071 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e753ad830 con 0x561e75450f70 2024-09-15T23:05:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.858+0000 7f87c0c00640 10 client.16545 handle_command_reply: tid=0 2024-09-15T23:05:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.858+0000 7f87e0ec8740 1 client.16545 shutdown 2024-09-15T23:05:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.858+0000 7f87e0ec8740 1 -- 172.21.15.111:0/657005071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561e7543bba0 msgr2=0x561e754459b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.858+0000 7f87e0ec8740 1 --2- 172.21.15.111:0/657005071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561e7543bba0 0x561e754459b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f87c4033000 tx=0x7f87c4009260 comp rx=0 tx=0).stop 2024-09-15T23:05:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.858+0000 7f87e0ec8740 1 -- 172.21.15.111:0/657005071 shutdown_connections 2024-09-15T23:05:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.858+0000 7f87e0ec8740 1 --2- 172.21.15.111:0/657005071 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561e75450f70 0x561e754533c0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.858+0000 7f87e0ec8740 1 --2- 172.21.15.111:0/657005071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561e75440c30 0x561e7544c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.858+0000 7f87e0ec8740 1 --2- 172.21.15.111:0/657005071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561e7543bba0 0x561e754459b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.858+0000 7f87e0ec8740 1 --2- 172.21.15.111:0/657005071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e7543b260 0x561e75445470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.858+0000 7f87e0ec8740 1 -- 172.21.15.111:0/657005071 >> 172.21.15.111:0/657005071 conn(0x561e754368d0 msgr2=0x561e754380e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.858+0000 7f87c0c00640 0 client.16545 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:05:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.858+0000 7f87e0ec8740 1 -- 172.21.15.111:0/657005071 shutdown_connections 2024-09-15T23:05:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.858+0000 7f87e0ec8740 1 -- 172.21.15.111:0/657005071 wait complete. 2024-09-15T23:05:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.858+0000 7f87e0ec8740 20 client.16545 trim_cache size 0 max 16384 2024-09-15T23:05:26.876 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.873+0000 7f87e0ec8740 1 Processor -- start 2024-09-15T23:05:26.876 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.873+0000 7f87e0ec8740 1 -- start start 2024-09-15T23:05:26.876 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.873+0000 7f87e0ec8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561e7543b260 0x561e75435900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:26.876 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.873+0000 7f87e0ec8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561e7543bba0 0x561e7544c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:26.876 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.873+0000 7f87e0ec8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e75440c30 0x561e75457cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:26.876 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.873+0000 7f87e0ec8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f87d81b1ca0 con 0x561e7543b260 2024-09-15T23:05:26.876 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.873+0000 7f87e0ec8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f87d81af570 con 0x561e7543bba0 2024-09-15T23:05:26.876 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.873+0000 7f87e0ec8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f87d81ad3c0 con 0x561e75440c30 2024-09-15T23:05:26.876 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.873+0000 7f87d7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561e7543b260 0x561e75435900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:26.876 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.873+0000 7f87d7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561e7543b260 0x561e75435900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52448/0 (socket says 172.21.15.111:52448) 2024-09-15T23:05:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.874+0000 7f87d7400640 1 -- 172.21.15.111:0/4249053904 learned_addr learned my addr 172.21.15.111:0/4249053904 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:05:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.874+0000 7f87d7e00640 1 --2- 172.21.15.111:0/4249053904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e75440c30 0x561e75457cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.874+0000 7f87d7400640 1 -- 172.21.15.111:0/4249053904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e75440c30 msgr2=0x561e75457cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.874+0000 7f87d7400640 1 --2- 172.21.15.111:0/4249053904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e75440c30 0x561e75457cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.874+0000 7f87d7400640 1 -- 172.21.15.111:0/4249053904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561e7543bba0 msgr2=0x561e7544c9b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:05:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.874+0000 7f87d7400640 1 --2- 172.21.15.111:0/4249053904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561e7543bba0 0x561e7544c9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.874+0000 7f87d7400640 1 -- 172.21.15.111:0/4249053904 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e7544db00 con 0x561e7543b260 2024-09-15T23:05:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.874+0000 7f87d7400640 1 --2- 172.21.15.111:0/4249053904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561e7543b260 0x561e75435900 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f87c8000c00 tx=0x7f87c8013fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92791ed2d1cb43c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.874+0000 7f87c3400640 1 -- 172.21.15.111:0/4249053904 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c801e040 con 0x561e7543b260 2024-09-15T23:05:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.874+0000 7f87c3400640 1 -- 172.21.15.111:0/4249053904 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87c8029030 con 0x561e7543b260 2024-09-15T23:05:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.874+0000 7f87c3400640 1 -- 172.21.15.111:0/4249053904 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c800f640 con 0x561e7543b260 2024-09-15T23:05:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.875+0000 7f87e0ec8740 1 -- 172.21.15.111:0/4249053904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561e7543b260 msgr2=0x561e75435900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.875+0000 7f87e0ec8740 1 --2- 172.21.15.111:0/4249053904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561e7543b260 0x561e75435900 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f87c8000c00 tx=0x7f87c8013fd0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.875+0000 7f87e0ec8740 1 -- 172.21.15.111:0/4249053904 shutdown_connections 2024-09-15T23:05:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.875+0000 7f87e0ec8740 1 --2- 172.21.15.111:0/4249053904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e75440c30 0x561e75457cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.875+0000 7f87e0ec8740 1 --2- 172.21.15.111:0/4249053904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561e7543bba0 0x561e7544c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.875+0000 7f87e0ec8740 1 --2- 172.21.15.111:0/4249053904 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561e7543b260 0x561e75435900 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.875+0000 7f87e0ec8740 1 -- 172.21.15.111:0/4249053904 >> 172.21.15.111:0/4249053904 conn(0x561e754368d0 msgr2=0x561e754351f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.875+0000 7f87e0ec8740 1 -- 172.21.15.111:0/4249053904 shutdown_connections 2024-09-15T23:05:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.875+0000 7f87e0ec8740 1 -- 172.21.15.111:0/4249053904 wait complete. 2024-09-15T23:05:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.876+0000 7f87e0ec8740 1 -- start start 2024-09-15T23:05:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.876+0000 7f87e0ec8740 1 Processor -- start 2024-09-15T23:05:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.876+0000 7f87e0ec8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561e7543b260 0x561e7544a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.876+0000 7f87e0ec8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e7543bba0 0x561e7544a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.876+0000 7f87e0ec8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561e75440c30 0x561e7544b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.876+0000 7f87e0ec8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f87d81ada30 con 0x561e7543b260 2024-09-15T23:05:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.876+0000 7f87e0ec8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f87d81aeea0 con 0x561e75440c30 2024-09-15T23:05:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.876+0000 7f87e0ec8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f87d81ae100 con 0x561e7543bba0 2024-09-15T23:05:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.876+0000 7f87e0ec8740 10 client.0 _ll_register_callbacks cb 0x7f87d00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:05:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.876+0000 7f87d7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561e7543b260 0x561e7544a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.876+0000 7f87d7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561e7543b260 0x561e7544a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52462/0 (socket says 172.21.15.111:52462) 2024-09-15T23:05:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.876+0000 7f87d7400640 1 -- 172.21.15.111:0/2386157576 learned_addr learned my addr 172.21.15.111:0/2386157576 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:05:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.877+0000 7f87d7e00640 1 --2- 172.21.15.111:0/2386157576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561e75440c30 0x561e7544b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.877+0000 7f87d6a00640 1 --2- 172.21.15.111:0/2386157576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e7543bba0 0x561e7544a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.877+0000 7f87d7400640 1 -- 172.21.15.111:0/2386157576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e7543bba0 msgr2=0x561e7544a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.877+0000 7f87d7400640 1 --2- 172.21.15.111:0/2386157576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e7543bba0 0x561e7544a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.877+0000 7f87d7400640 1 -- 172.21.15.111:0/2386157576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561e75440c30 msgr2=0x561e7544b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.877+0000 7f87d7400640 1 --2- 172.21.15.111:0/2386157576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561e75440c30 0x561e7544b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.877+0000 7f87d7400640 1 -- 172.21.15.111:0/2386157576 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e7544bd30 con 0x561e7543b260 2024-09-15T23:05:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.877+0000 7f87d7400640 1 --2- 172.21.15.111:0/2386157576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561e7543b260 0x561e7544a1d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f87c801c040 tx=0x7f87c8009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1fc1756d2d2b5d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.877+0000 7f87c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:05:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.877+0000 7f87c0c00640 1 -- 172.21.15.111:0/2386157576 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c801e040 con 0x561e7543b260 2024-09-15T23:05:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.877+0000 7f87c0c00640 1 -- 172.21.15.111:0/2386157576 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f87c8019860 con 0x561e7543b260 2024-09-15T23:05:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.878+0000 7f87e0ec8740 1 -- 172.21.15.111:0/2386157576 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87c8029030 con 0x561e7543b260 2024-09-15T23:05:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.878+0000 7f87c0c00640 1 -- 172.21.15.111:0/2386157576 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f87c803f050 con 0x561e7543b260 2024-09-15T23:05:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.878+0000 7f87e0ec8740 10 client.16551 fetch_fsmap learned FSMap version 35 2024-09-15T23:05:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.878+0000 7f87e0ec8740 1 -- 172.21.15.111:0/2386157576 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x561e754bbdc0 con 0x561e7543b260 2024-09-15T23:05:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.878+0000 7f87c0c00640 1 -- 172.21.15.111:0/2386157576 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87c8029030 con 0x561e7543b260 2024-09-15T23:05:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.878+0000 7f87c0c00640 1 -- 172.21.15.111:0/2386157576 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f87c80035f0 con 0x561e7543b260 2024-09-15T23:05:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.878+0000 7f87e0ec8740 10 client.16551 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:05:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.878+0000 7f87e0ec8740 10 client.16551 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:05:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.878+0000 7f87e0ec8740 20 client.16551 populate_metadata read hostname 'smithi111' 2024-09-15T23:05:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.878+0000 7f87e0ec8740 1 --2- 172.21.15.111:0/2386157576 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561e754bc570 0x561e754be990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.878+0000 7f87e0ec8740 4 client.16551 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:05:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.878+0000 7f87e0ec8740 1 -- 172.21.15.111:0/2386157576 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e754409b0 con 0x561e754bc570 2024-09-15T23:05:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.879+0000 7f87d6a00640 1 --2- 172.21.15.111:0/2386157576 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561e754bc570 0x561e754be990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.879+0000 7f87d6a00640 1 --2- 172.21.15.111:0/2386157576 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561e754bc570 0x561e754be990 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19143cb51aea86b2 server_cookie=b2a3b97f6b42a1c9 in_seq=0 out_seq=0 2024-09-15T23:05:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.879+0000 7f87c0c00640 10 client.16551 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:05:26.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.932+0000 7f87c0c00640 1 -- 172.21.15.111:0/2386157576 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561e754409b0 con 0x561e754bc570 2024-09-15T23:05:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.933+0000 7f87c0c00640 10 client.16551 handle_command_reply: tid=0 2024-09-15T23:05:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.933+0000 7f87e0ec8740 1 client.16551 shutdown 2024-09-15T23:05:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.933+0000 7f87e0ec8740 1 -- 172.21.15.111:0/2386157576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561e7543b260 msgr2=0x561e7544a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.933+0000 7f87e0ec8740 1 --2- 172.21.15.111:0/2386157576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561e7543b260 0x561e7544a1d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f87c801c040 tx=0x7f87c8009d40 comp rx=0 tx=0).stop 2024-09-15T23:05:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.933+0000 7f87e0ec8740 1 -- 172.21.15.111:0/2386157576 shutdown_connections 2024-09-15T23:05:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.933+0000 7f87e0ec8740 1 --2- 172.21.15.111:0/2386157576 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x561e754bc570 0x561e754be990 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.933+0000 7f87e0ec8740 1 --2- 172.21.15.111:0/2386157576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561e75440c30 0x561e7544b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.933+0000 7f87c0c00640 0 client.16551 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:05:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.933+0000 7f87e0ec8740 1 --2- 172.21.15.111:0/2386157576 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x561e7543bba0 0x561e7544a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.935 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.933+0000 7f87e0ec8740 1 --2- 172.21.15.111:0/2386157576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x561e7543b260 0x561e7544a1d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.933+0000 7f87e0ec8740 1 -- 172.21.15.111:0/2386157576 >> 172.21.15.111:0/2386157576 conn(0x561e754368d0 msgr2=0x561e75435a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.933+0000 7f87e0ec8740 1 -- 172.21.15.111:0/2386157576 shutdown_connections 2024-09-15T23:05:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.933+0000 7f87e0ec8740 1 -- 172.21.15.111:0/2386157576 wait complete. 2024-09-15T23:05:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.933+0000 7f87e0ec8740 20 client.16551 trim_cache size 0 max 16384 2024-09-15T23:05:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:05:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:05:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:05:26.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:05:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.934+0000 7f87dca00640 1 -- 172.21.15.111:0/3404985828 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f87bc0a32c0 msgr2=0x7f87bc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.934+0000 7f87dca00640 1 --2- 172.21.15.111:0/3404985828 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f87bc0a32c0 0x7f87bc0a5780 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009230 tx=0x7f87c4031400 comp rx=0 tx=0).stop 2024-09-15T23:05:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.934+0000 7f87dca00640 1 -- 172.21.15.111:0/3404985828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87d81a3980 msgr2=0x7f87d81a5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.934+0000 7f87dca00640 1 --2- 172.21.15.111:0/3404985828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87d81a3980 0x7f87d81a5d70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f87d000eb10 tx=0x7f87d000c2d0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.934+0000 7f87dca00640 1 -- 172.21.15.111:0/3404985828 shutdown_connections 2024-09-15T23:05:26.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.934+0000 7f87dca00640 1 --2- 172.21.15.111:0/3404985828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87d81a3980 0x7f87d81a5d70 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.934+0000 7f87dca00640 1 --2- 172.21.15.111:0/3404985828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87d8074fc0 0x7f87d819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.934+0000 7f87dca00640 1 --2- 172.21.15.111:0/3404985828 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f87bc0a32c0 0x7f87bc0a5780 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.934+0000 7f87dca00640 1 --2- 172.21.15.111:0/3404985828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87d8074680 0x7f87d819e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.934+0000 7f87dca00640 1 -- 172.21.15.111:0/3404985828 >> 172.21.15.111:0/3404985828 conn(0x7f87d80fd0d0 msgr2=0x7f87d8110150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.934+0000 7f87dca00640 1 -- 172.21.15.111:0/3404985828 shutdown_connections 2024-09-15T23:05:26.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:26.934+0000 7f87dca00640 1 -- 172.21.15.111:0/3404985828 wait complete. 2024-09-15T23:05:27.081 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:27 smithi111 ceph-mon[30629]: pgmap v937: 97 pgs: 97 active+clean; 549 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.2 MiB/s wr, 356 op/s 2024-09-15T23:05:27.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:27 smithi192 ceph-mon[35427]: pgmap v937: 97 pgs: 97 active+clean; 549 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.2 MiB/s wr, 356 op/s 2024-09-15T23:05:27.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:27 smithi121 ceph-mon[34815]: pgmap v937: 97 pgs: 97 active+clean; 549 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.2 MiB/s wr, 356 op/s 2024-09-15T23:05:27.664 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124321 17.44 MB/sec execute 235 sec latency 8130.970 ms 2024-09-15T23:05:28.664 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124322 17.37 MB/sec execute 236 sec latency 9131.020 ms 2024-09-15T23:05:29.664 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124322 17.29 MB/sec execute 237 sec latency 10131.103 ms 2024-09-15T23:05:30.159 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:29 smithi111 ceph-mon[30629]: pgmap v938: 97 pgs: 97 active+clean; 559 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 6.4 MiB/s wr, 279 op/s 2024-09-15T23:05:30.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:29 smithi192 ceph-mon[35427]: pgmap v938: 97 pgs: 97 active+clean; 559 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 6.4 MiB/s wr, 279 op/s 2024-09-15T23:05:30.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:29 smithi121 ceph-mon[34815]: pgmap v938: 97 pgs: 97 active+clean; 559 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 6.4 MiB/s wr, 279 op/s 2024-09-15T23:05:30.664 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124322 17.22 MB/sec execute 238 sec latency 11131.158 ms 2024-09-15T23:05:31.664 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124322 17.15 MB/sec execute 239 sec latency 12131.208 ms 2024-09-15T23:05:32.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:31 smithi111 ceph-mon[30629]: pgmap v939: 97 pgs: 97 active+clean; 568 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 5.7 MiB/s wr, 214 op/s 2024-09-15T23:05:32.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:31 smithi192 ceph-mon[35427]: pgmap v939: 97 pgs: 97 active+clean; 568 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 5.7 MiB/s wr, 214 op/s 2024-09-15T23:05:32.393 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:31 smithi121 ceph-mon[34815]: pgmap v939: 97 pgs: 97 active+clean; 568 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 5.7 MiB/s wr, 214 op/s 2024-09-15T23:05:32.664 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124322 17.08 MB/sec execute 240 sec latency 13131.262 ms 2024-09-15T23:05:33.664 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124322 17.01 MB/sec execute 241 sec latency 14131.313 ms 2024-09-15T23:05:34.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:33 smithi111 ceph-mon[30629]: pgmap v940: 97 pgs: 97 active+clean; 574 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 5.2 MiB/s wr, 189 op/s 2024-09-15T23:05:34.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:33 smithi192 ceph-mon[35427]: pgmap v940: 97 pgs: 97 active+clean; 574 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 5.2 MiB/s wr, 189 op/s 2024-09-15T23:05:34.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:33 smithi121 ceph-mon[34815]: pgmap v940: 97 pgs: 97 active+clean; 574 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 5.2 MiB/s wr, 189 op/s 2024-09-15T23:05:34.664 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124322 16.94 MB/sec execute 242 sec latency 15131.365 ms 2024-09-15T23:05:35.664 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124329 16.87 MB/sec execute 243 sec latency 15290.293 ms 2024-09-15T23:05:36.133 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:35 smithi111 ceph-mon[30629]: pgmap v941: 97 pgs: 97 active+clean; 576 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 5.8 MiB/s wr, 197 op/s 2024-09-15T23:05:36.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:35 smithi192 ceph-mon[35427]: pgmap v941: 97 pgs: 97 active+clean; 576 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 5.8 MiB/s wr, 197 op/s 2024-09-15T23:05:36.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:35 smithi121 ceph-mon[34815]: pgmap v941: 97 pgs: 97 active+clean; 576 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 5.8 MiB/s wr, 197 op/s 2024-09-15T23:05:36.664 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124329 16.80 MB/sec execute 244 sec latency 10810.839 ms 2024-09-15T23:05:37.664 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124329 16.73 MB/sec execute 245 sec latency 11810.906 ms 2024-09-15T23:05:38.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:37 smithi192 ceph-mon[35427]: pgmap v942: 97 pgs: 97 active+clean; 574 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 5.4 MiB/s wr, 178 op/s 2024-09-15T23:05:38.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:37 smithi121 ceph-mon[34815]: pgmap v942: 97 pgs: 97 active+clean; 574 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 5.4 MiB/s wr, 178 op/s 2024-09-15T23:05:38.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:37 smithi111 ceph-mon[30629]: pgmap v942: 97 pgs: 97 active+clean; 574 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 5.4 MiB/s wr, 178 op/s 2024-09-15T23:05:38.664 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124329 16.66 MB/sec execute 246 sec latency 12810.956 ms 2024-09-15T23:05:39.664 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124329 16.59 MB/sec execute 247 sec latency 13811.006 ms 2024-09-15T23:05:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.096+0000 7fd239000640 1 -- 172.21.15.111:0/571087721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd234103050 msgr2=0x7fd234103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.096+0000 7fd239000640 1 --2- 172.21.15.111:0/571087721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd234103050 0x7fd234103450 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd224009860 tx=0x7fd22402a360 comp rx=0 tx=0).stop 2024-09-15T23:05:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.096+0000 7fd239000640 1 -- 172.21.15.111:0/571087721 shutdown_connections 2024-09-15T23:05:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.096+0000 7fd239000640 1 --2- 172.21.15.111:0/571087721 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd234101f40 0x7fd234109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.096+0000 7fd239000640 1 --2- 172.21.15.111:0/571087721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd234101580 0x7fd234101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.096+0000 7fd239000640 1 --2- 172.21.15.111:0/571087721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd234103050 0x7fd234103450 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.096+0000 7fd239000640 1 -- 172.21.15.111:0/571087721 >> 172.21.15.111:0/571087721 conn(0x7fd2340fd0d0 msgr2=0x7fd2340ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.096+0000 7fd239000640 1 -- 172.21.15.111:0/571087721 shutdown_connections 2024-09-15T23:05:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.096+0000 7fd239000640 1 -- 172.21.15.111:0/571087721 wait complete. 2024-09-15T23:05:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.096+0000 7fd239000640 1 Processor -- start 2024-09-15T23:05:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.096+0000 7fd239000640 1 -- start start 2024-09-15T23:05:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.096+0000 7fd239000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd234101580 0x7fd234075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.096+0000 7fd239000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd234101f40 0x7fd2340745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.096+0000 7fd239000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd234103050 0x7fd234074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.096+0000 7fd239000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd23410d770 con 0x7fd234101f40 2024-09-15T23:05:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.096+0000 7fd239000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd23410d5f0 con 0x7fd234101580 2024-09-15T23:05:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.096+0000 7fd239000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd23410d8f0 con 0x7fd234103050 2024-09-15T23:05:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.096+0000 7fd233400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd234101580 0x7fd234075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.096+0000 7fd233400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd234101580 0x7fd234075fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:37156/0 (socket says 172.21.15.111:37156) 2024-09-15T23:05:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.096+0000 7fd233400640 1 -- 172.21.15.111:0/1801480972 learned_addr learned my addr 172.21.15.111:0/1801480972 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:05:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.096+0000 7fd232a00640 1 --2- 172.21.15.111:0/1801480972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd234101f40 0x7fd2340745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.096+0000 7fd233400640 1 -- 172.21.15.111:0/1801480972 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd234103050 msgr2=0x7fd234074b20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T23:05:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.096+0000 7fd233400640 1 --2- 172.21.15.111:0/1801480972 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd234103050 0x7fd234074b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.096+0000 7fd233400640 1 -- 172.21.15.111:0/1801480972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd234101f40 msgr2=0x7fd2340745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.096+0000 7fd233400640 1 --2- 172.21.15.111:0/1801480972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd234101f40 0x7fd2340745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.096+0000 7fd233400640 1 -- 172.21.15.111:0/1801480972 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2340753b0 con 0x7fd234101580 2024-09-15T23:05:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.096+0000 7fd232a00640 1 --2- 172.21.15.111:0/1801480972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd234101f40 0x7fd2340745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:05:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.101+0000 7fd233400640 1 --2- 172.21.15.111:0/1801480972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd234101580 0x7fd234075fa0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd2240304b0 tx=0x7fd224033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.101+0000 7fd22be00640 1 -- 172.21.15.111:0/1801480972 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd224037040 con 0x7fd234101580 2024-09-15T23:05:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.101+0000 7fd22be00640 1 -- 172.21.15.111:0/1801480972 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd224030880 con 0x7fd234101580 2024-09-15T23:05:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.101+0000 7fd239000640 1 -- 172.21.15.111:0/1801480972 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2341a1f20 con 0x7fd234101580 2024-09-15T23:05:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.101+0000 7fd22be00640 1 -- 172.21.15.111:0/1801480972 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2240046c0 con 0x7fd234101580 2024-09-15T23:05:40.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.101+0000 7fd239000640 1 -- 172.21.15.111:0/1801480972 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2341a22d0 con 0x7fd234101580 2024-09-15T23:05:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.101+0000 7fd22be00640 1 -- 172.21.15.111:0/1801480972 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd224004050 con 0x7fd234101580 2024-09-15T23:05:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.101+0000 7fd23d3ef740 1 Processor -- start 2024-09-15T23:05:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.101+0000 7fd23d3ef740 1 -- start start 2024-09-15T23:05:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.101+0000 7fd23d3ef740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559341d4e260 0x559341d4e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.101+0000 7fd23d3ef740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559341d4eba0 0x559341d53660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.101+0000 7fd23d3ef740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559341d53c30 0x559341d560f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.101+0000 7fd23d3ef740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559341cb3100 con 0x559341d4eba0 2024-09-15T23:05:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.101+0000 7fd23d3ef740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559341c5b1b0 con 0x559341d4e260 2024-09-15T23:05:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.101+0000 7fd23d3ef740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x559341caf0c0 con 0x559341d53c30 2024-09-15T23:05:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.101+0000 7fd233400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559341d4eba0 0x559341d53660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.101+0000 7fd233e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559341d4e260 0x559341d4e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.101+0000 7fd233e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559341d4e260 0x559341d4e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:37166/0 (socket says 172.21.15.111:37166) 2024-09-15T23:05:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.101+0000 7fd233e00640 1 -- 172.21.15.111:0/478793071 learned_addr learned my addr 172.21.15.111:0/478793071 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:05:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.106+0000 7fd232a00640 1 --2- 172.21.15.111:0/478793071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559341d53c30 0x559341d560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.106+0000 7fd22be00640 1 --2- 172.21.15.111:0/1801480972 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd2180a3240 0x7fd2180a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.106+0000 7fd233400640 1 -- 172.21.15.111:0/478793071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559341d53c30 msgr2=0x559341d560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.106+0000 7fd233400640 1 --2- 172.21.15.111:0/478793071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559341d53c30 0x559341d560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.106+0000 7fd233400640 1 -- 172.21.15.111:0/478793071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559341d4e260 msgr2=0x559341d4e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.106+0000 7fd233400640 1 --2- 172.21.15.111:0/478793071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559341d4e260 0x559341d4e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.106+0000 7fd233400640 1 -- 172.21.15.111:0/478793071 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593419c6210 con 0x559341d4eba0 2024-09-15T23:05:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.106+0000 7fd233e00640 1 --2- 172.21.15.111:0/478793071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559341d4e260 0x559341d4e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:05:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.106+0000 7fd22be00640 1 -- 172.21.15.111:0/1801480972 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fd2240ef360 con 0x7fd234101580 2024-09-15T23:05:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.106+0000 7fd233400640 1 --2- 172.21.15.111:0/478793071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559341d4eba0 0x559341d53660 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd2240ece00 tx=0x7fd224047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af5cbf9126fcb78 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.106+0000 7fd228c00640 1 -- 172.21.15.111:0/478793071 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2240bb4c0 con 0x559341d4eba0 2024-09-15T23:05:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.106+0000 7fd233e00640 1 --2- 172.21.15.111:0/1801480972 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd2180a3240 0x7fd2180a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd228c00640 1 -- 172.21.15.111:0/478793071 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2240bc080 con 0x559341d4eba0 2024-09-15T23:05:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd228c00640 1 -- 172.21.15.111:0/478793071 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2240a6450 con 0x559341d4eba0 2024-09-15T23:05:40.110 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:39 smithi111 ceph-mon[30629]: pgmap v943: 97 pgs: 97 active+clean; 584 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 5.8 MiB/s wr, 192 op/s 2024-09-15T23:05:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd23d3ef740 1 -- 172.21.15.111:0/478793071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559341d4eba0 msgr2=0x559341d53660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd23d3ef740 1 --2- 172.21.15.111:0/478793071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559341d4eba0 0x559341d53660 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd2240ece00 tx=0x7fd224047000 comp rx=0 tx=0).stop 2024-09-15T23:05:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd23d3ef740 1 -- 172.21.15.111:0/478793071 shutdown_connections 2024-09-15T23:05:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd23d3ef740 1 --2- 172.21.15.111:0/478793071 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559341d53c30 0x559341d560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd23d3ef740 1 --2- 172.21.15.111:0/478793071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559341d4eba0 0x559341d53660 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd23d3ef740 1 --2- 172.21.15.111:0/478793071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559341d4e260 0x559341d4e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd23d3ef740 1 -- 172.21.15.111:0/478793071 >> 172.21.15.111:0/478793071 conn(0x559341d498d0 msgr2=0x559341d49cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd233e00640 1 --2- 172.21.15.111:0/1801480972 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd2180a3240 0x7fd2180a5700 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd22c009ef0 tx=0x7fd22c004ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd23d3ef740 1 -- 172.21.15.111:0/478793071 shutdown_connections 2024-09-15T23:05:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd23d3ef740 1 -- 172.21.15.111:0/478793071 wait complete. 2024-09-15T23:05:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd23d3ef740 1 -- start start 2024-09-15T23:05:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd23d3ef740 1 Processor -- start 2024-09-15T23:05:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd23d3ef740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559341d4e260 0x559341d57f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd23d3ef740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559341d4eba0 0x559341d58450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd23d3ef740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559341d53c30 0x559341d5f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd23d3ef740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559341cb3100 con 0x559341d4eba0 2024-09-15T23:05:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd23d3ef740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559341c3c090 con 0x559341d4e260 2024-09-15T23:05:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd23d3ef740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x559341d569c0 con 0x559341d53c30 2024-09-15T23:05:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd23d3ef740 10 client.0 _ll_register_callbacks cb 0x559341c90f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:05:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd233400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559341d4e260 0x559341d57f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd232a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559341d4eba0 0x559341d58450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd232a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559341d4eba0 0x559341d58450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41788/0 (socket says 172.21.15.111:41788) 2024-09-15T23:05:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd232a00640 1 -- 172.21.15.111:0/2505026309 learned_addr learned my addr 172.21.15.111:0/2505026309 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:05:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd233e00640 1 --2- 172.21.15.111:0/2505026309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559341d53c30 0x559341d5f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd232a00640 1 -- 172.21.15.111:0/2505026309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559341d53c30 msgr2=0x559341d5f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd232a00640 1 --2- 172.21.15.111:0/2505026309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559341d53c30 0x559341d5f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd232a00640 1 -- 172.21.15.111:0/2505026309 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559341d4e260 msgr2=0x559341d57f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd232a00640 1 --2- 172.21.15.111:0/2505026309 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559341d4e260 0x559341d57f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd232a00640 1 -- 172.21.15.111:0/2505026309 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559341a92620 con 0x559341d4eba0 2024-09-15T23:05:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd232a00640 1 --2- 172.21.15.111:0/2505026309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559341d4eba0 0x559341d58450 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd22000df20 tx=0x7fd22000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd5cbb7bd0c6a6b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd21d600640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:05:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd21d600640 1 -- 172.21.15.111:0/2505026309 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220014670 con 0x559341d4eba0 2024-09-15T23:05:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd21d600640 1 -- 172.21.15.111:0/2505026309 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fd220014810 con 0x559341d4eba0 2024-09-15T23:05:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd23d3ef740 1 -- 172.21.15.111:0/2505026309 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2240bc080 con 0x559341d4eba0 2024-09-15T23:05:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd21d600640 1 -- 172.21.15.111:0/2505026309 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd220014a10 con 0x559341d4eba0 2024-09-15T23:05:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd23d3ef740 10 client.16563 fetch_fsmap learned FSMap version 35 2024-09-15T23:05:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd23d3ef740 1 -- 172.21.15.111:0/2505026309 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559341d62350 con 0x559341d4eba0 2024-09-15T23:05:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd21d600640 1 -- 172.21.15.111:0/2505026309 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2240bc080 con 0x559341d4eba0 2024-09-15T23:05:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd21d600640 1 -- 172.21.15.111:0/2505026309 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fd220026020 con 0x559341d4eba0 2024-09-15T23:05:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd23d3ef740 10 client.16563 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:05:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd23d3ef740 10 client.16563 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:05:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd23d3ef740 20 client.16563 populate_metadata read hostname 'smithi111' 2024-09-15T23:05:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd23d3ef740 1 --2- 172.21.15.111:0/2505026309 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559341d63a90 0x559341d65e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd23d3ef740 4 client.16563 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:05:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.108+0000 7fd23d3ef740 1 -- 172.21.15.111:0/2505026309 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559341cc0830 con 0x559341d63a90 2024-09-15T23:05:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.113+0000 7fd233400640 1 --2- 172.21.15.111:0/2505026309 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559341d63a90 0x559341d65e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.113+0000 7fd21d600640 10 client.16563 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:05:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.113+0000 7fd233400640 1 --2- 172.21.15.111:0/2505026309 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559341d63a90 0x559341d65e80 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=679573426b300b38 server_cookie=e1683ce1ff4cf8ad in_seq=0 out_seq=0 2024-09-15T23:05:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.113+0000 7fd21d600640 1 -- 172.21.15.111:0/2505026309 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559341cc0830 con 0x559341d63a90 2024-09-15T23:05:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.113+0000 7fd21d600640 10 client.16563 handle_command_reply: tid=0 2024-09-15T23:05:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.113+0000 7fd23d3ef740 1 client.16563 shutdown 2024-09-15T23:05:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.113+0000 7fd23d3ef740 1 -- 172.21.15.111:0/2505026309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559341d4eba0 msgr2=0x559341d58450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.113+0000 7fd23d3ef740 1 --2- 172.21.15.111:0/2505026309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559341d4eba0 0x559341d58450 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd22000df20 tx=0x7fd22000f670 comp rx=0 tx=0).stop 2024-09-15T23:05:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.113+0000 7fd23d3ef740 1 -- 172.21.15.111:0/2505026309 shutdown_connections 2024-09-15T23:05:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.113+0000 7fd23d3ef740 1 --2- 172.21.15.111:0/2505026309 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559341d63a90 0x559341d65e80 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.113+0000 7fd23d3ef740 1 --2- 172.21.15.111:0/2505026309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559341d53c30 0x559341d5f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.113+0000 7fd23d3ef740 1 --2- 172.21.15.111:0/2505026309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559341d4eba0 0x559341d58450 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.113+0000 7fd23d3ef740 1 --2- 172.21.15.111:0/2505026309 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559341d4e260 0x559341d57f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.113+0000 7fd23d3ef740 1 -- 172.21.15.111:0/2505026309 >> 172.21.15.111:0/2505026309 conn(0x559341d498d0 msgr2=0x559341d4b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.113+0000 7fd21d600640 0 client.16563 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:05:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.113+0000 7fd23d3ef740 1 -- 172.21.15.111:0/2505026309 shutdown_connections 2024-09-15T23:05:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.113+0000 7fd23d3ef740 1 -- 172.21.15.111:0/2505026309 wait complete. 2024-09-15T23:05:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.113+0000 7fd23d3ef740 20 client.16563 trim_cache size 0 max 16384 2024-09-15T23:05:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.130+0000 7fd23d3ef740 1 Processor -- start 2024-09-15T23:05:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.130+0000 7fd23d3ef740 1 -- start start 2024-09-15T23:05:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.130+0000 7fd23d3ef740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559341d4e260 0x559341d448d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.130+0000 7fd23d3ef740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559341d4eba0 0x559341d5f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.130+0000 7fd23d3ef740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559341d53c30 0x559341d48900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.130+0000 7fd23d3ef740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd2341ad870 con 0x559341d4e260 2024-09-15T23:05:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.130+0000 7fd23d3ef740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd2341ab140 con 0x559341d4eba0 2024-09-15T23:05:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.130+0000 7fd23d3ef740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd2341a8f90 con 0x559341d53c30 2024-09-15T23:05:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.130+0000 7fd232a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559341d4eba0 0x559341d5f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.130+0000 7fd233400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559341d4e260 0x559341d448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:40.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.130+0000 7fd232a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559341d4eba0 0x559341d5f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:37178/0 (socket says 172.21.15.111:37178) 2024-09-15T23:05:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.130+0000 7fd232a00640 1 -- 172.21.15.111:0/1734671202 learned_addr learned my addr 172.21.15.111:0/1734671202 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:05:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.130+0000 7fd233e00640 1 --2- 172.21.15.111:0/1734671202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559341d53c30 0x559341d48900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:40.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd232a00640 1 -- 172.21.15.111:0/1734671202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559341d53c30 msgr2=0x559341d48900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd232a00640 1 --2- 172.21.15.111:0/1734671202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559341d53c30 0x559341d48900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd232a00640 1 -- 172.21.15.111:0/1734671202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559341d4e260 msgr2=0x559341d448d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd232a00640 1 --2- 172.21.15.111:0/1734671202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559341d4e260 0x559341d448d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd232a00640 1 -- 172.21.15.111:0/1734671202 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559341d496f0 con 0x559341d4eba0 2024-09-15T23:05:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd233e00640 1 --2- 172.21.15.111:0/1734671202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559341d53c30 0x559341d48900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:05:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd232a00640 1 --2- 172.21.15.111:0/1734671202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559341d4eba0 0x559341d5f450 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd220000ea0 tx=0x7fd22000d590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11259502b01dad76 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd21fe00640 1 -- 172.21.15.111:0/1734671202 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22002d020 con 0x559341d4eba0 2024-09-15T23:05:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd21fe00640 1 -- 172.21.15.111:0/1734671202 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd220004520 con 0x559341d4eba0 2024-09-15T23:05:40.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd21fe00640 1 -- 172.21.15.111:0/1734671202 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220003690 con 0x559341d4eba0 2024-09-15T23:05:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd23d3ef740 1 -- 172.21.15.111:0/1734671202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559341d4eba0 msgr2=0x559341d5f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd23d3ef740 1 --2- 172.21.15.111:0/1734671202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559341d4eba0 0x559341d5f450 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd220000ea0 tx=0x7fd22000d590 comp rx=0 tx=0).stop 2024-09-15T23:05:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd23d3ef740 1 -- 172.21.15.111:0/1734671202 shutdown_connections 2024-09-15T23:05:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd23d3ef740 1 --2- 172.21.15.111:0/1734671202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559341d53c30 0x559341d48900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd23d3ef740 1 --2- 172.21.15.111:0/1734671202 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559341d4eba0 0x559341d5f450 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd23d3ef740 1 --2- 172.21.15.111:0/1734671202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559341d4e260 0x559341d448d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd23d3ef740 1 -- 172.21.15.111:0/1734671202 >> 172.21.15.111:0/1734671202 conn(0x559341d498d0 msgr2=0x559341d481f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd23d3ef740 1 -- 172.21.15.111:0/1734671202 shutdown_connections 2024-09-15T23:05:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd23d3ef740 1 -- 172.21.15.111:0/1734671202 wait complete. 2024-09-15T23:05:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd23d3ef740 1 -- start start 2024-09-15T23:05:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd23d3ef740 1 Processor -- start 2024-09-15T23:05:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd23d3ef740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559341d4e260 0x559341d5ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd23d3ef740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559341d4eba0 0x559341d5cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd23d3ef740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559341d53c30 0x559341d5df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd23d3ef740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd2341a9600 con 0x559341d4e260 2024-09-15T23:05:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd23d3ef740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd2341aaa70 con 0x559341d53c30 2024-09-15T23:05:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd23d3ef740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd2341a9cd0 con 0x559341d4eba0 2024-09-15T23:05:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd23d3ef740 10 client.0 _ll_register_callbacks cb 0x7fd22000d240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:05:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd233400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559341d4e260 0x559341d5ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd233400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559341d4e260 0x559341d5ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:41800/0 (socket says 172.21.15.111:41800) 2024-09-15T23:05:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd233400640 1 -- 172.21.15.111:0/4104230080 learned_addr learned my addr 172.21.15.111:0/4104230080 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:05:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd233e00640 1 --2- 172.21.15.111:0/4104230080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559341d53c30 0x559341d5df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd233400640 1 -- 172.21.15.111:0/4104230080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559341d4eba0 msgr2=0x559341d5cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd233400640 1 --2- 172.21.15.111:0/4104230080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559341d4eba0 0x559341d5cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd233400640 1 -- 172.21.15.111:0/4104230080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559341d53c30 msgr2=0x559341d5df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd233400640 1 --2- 172.21.15.111:0/4104230080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559341d53c30 0x559341d5df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd233400640 1 -- 172.21.15.111:0/4104230080 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559341d5e570 con 0x559341d4e260 2024-09-15T23:05:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd233400640 1 --2- 172.21.15.111:0/4104230080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559341d4e260 0x559341d5ca10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd2240bb680 tx=0x7fd22402a360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b230b766700a9e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd21d600640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:05:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd21d600640 1 -- 172.21.15.111:0/4104230080 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2240bb2e0 con 0x559341d4e260 2024-09-15T23:05:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd21d600640 1 -- 172.21.15.111:0/4104230080 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fd2240b9c80 con 0x559341d4e260 2024-09-15T23:05:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd23d3ef740 1 -- 172.21.15.111:0/4104230080 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd220004520 con 0x559341d4e260 2024-09-15T23:05:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd21d600640 1 -- 172.21.15.111:0/4104230080 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd2240bc080 con 0x559341d4e260 2024-09-15T23:05:40.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd23d3ef740 10 client.16569 fetch_fsmap learned FSMap version 35 2024-09-15T23:05:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd23d3ef740 1 -- 172.21.15.111:0/4104230080 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559341dce760 con 0x559341d4e260 2024-09-15T23:05:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd21d600640 1 -- 172.21.15.111:0/4104230080 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd220004520 con 0x559341d4e260 2024-09-15T23:05:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd21d600640 1 -- 172.21.15.111:0/4104230080 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fd22403a070 con 0x559341d4e260 2024-09-15T23:05:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd23d3ef740 10 client.16569 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:05:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd23d3ef740 10 client.16569 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:05:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd23d3ef740 20 client.16569 populate_metadata read hostname 'smithi111' 2024-09-15T23:05:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd23d3ef740 1 --2- 172.21.15.111:0/4104230080 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559341dcecd0 0x559341dd10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd23d3ef740 4 client.16569 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:05:40.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.131+0000 7fd23d3ef740 1 -- 172.21.15.111:0/4104230080 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559341cbab80 con 0x559341dcecd0 2024-09-15T23:05:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.155+0000 7fd232a00640 1 --2- 172.21.15.111:0/4104230080 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559341dcecd0 0x559341dd10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.213+0000 7fd21d600640 10 client.16569 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:05:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.213+0000 7fd232a00640 1 --2- 172.21.15.111:0/4104230080 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559341dcecd0 0x559341dd10c0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb131e285bf944e2 server_cookie=5e154b5097b39f85 in_seq=0 out_seq=0 2024-09-15T23:05:40.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:39 smithi192 ceph-mon[35427]: pgmap v943: 97 pgs: 97 active+clean; 584 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 5.8 MiB/s wr, 192 op/s 2024-09-15T23:05:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.356+0000 7fd21d600640 1 -- 172.21.15.111:0/4104230080 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559341cbab80 con 0x559341dcecd0 2024-09-15T23:05:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.356+0000 7fd21d600640 10 client.16569 handle_command_reply: tid=0 2024-09-15T23:05:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.356+0000 7fd23d3ef740 1 client.16569 shutdown 2024-09-15T23:05:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.356+0000 7fd23d3ef740 1 -- 172.21.15.111:0/4104230080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559341d4e260 msgr2=0x559341d5ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.356+0000 7fd23d3ef740 1 --2- 172.21.15.111:0/4104230080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559341d4e260 0x559341d5ca10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd2240bb680 tx=0x7fd22402a360 comp rx=0 tx=0).stop 2024-09-15T23:05:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.356+0000 7fd23d3ef740 1 -- 172.21.15.111:0/4104230080 shutdown_connections 2024-09-15T23:05:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.356+0000 7fd23d3ef740 1 --2- 172.21.15.111:0/4104230080 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559341dcecd0 0x559341dd10c0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.356+0000 7fd23d3ef740 1 --2- 172.21.15.111:0/4104230080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559341d53c30 0x559341d5df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.356+0000 7fd23d3ef740 1 --2- 172.21.15.111:0/4104230080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559341d4eba0 0x559341d5cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.356+0000 7fd23d3ef740 1 --2- 172.21.15.111:0/4104230080 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559341d4e260 0x559341d5ca10 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.356+0000 7fd23d3ef740 1 -- 172.21.15.111:0/4104230080 >> 172.21.15.111:0/4104230080 conn(0x559341d498d0 msgr2=0x559341d48a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.356+0000 7fd21d600640 0 client.16569 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:05:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.356+0000 7fd23d3ef740 1 -- 172.21.15.111:0/4104230080 shutdown_connections 2024-09-15T23:05:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.356+0000 7fd23d3ef740 1 -- 172.21.15.111:0/4104230080 wait complete. 2024-09-15T23:05:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.356+0000 7fd23d3ef740 20 client.16569 trim_cache size 0 max 16384 2024-09-15T23:05:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:05:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:05:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:05:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:05:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.356+0000 7fd239000640 1 -- 172.21.15.111:0/1801480972 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd2180a3240 msgr2=0x7fd2180a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.356+0000 7fd239000640 1 --2- 172.21.15.111:0/1801480972 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd2180a3240 0x7fd2180a5700 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd22c009ef0 tx=0x7fd22c004ab0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.356+0000 7fd239000640 1 -- 172.21.15.111:0/1801480972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd234101580 msgr2=0x7fd234075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.356+0000 7fd239000640 1 --2- 172.21.15.111:0/1801480972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd234101580 0x7fd234075fa0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd2240304b0 tx=0x7fd224033d90 comp rx=0 tx=0).stop 2024-09-15T23:05:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.362+0000 7fd239000640 1 -- 172.21.15.111:0/1801480972 shutdown_connections 2024-09-15T23:05:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.362+0000 7fd239000640 1 --2- 172.21.15.111:0/1801480972 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd234103050 0x7fd234074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.362+0000 7fd239000640 1 --2- 172.21.15.111:0/1801480972 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd234101f40 0x7fd2340745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.362+0000 7fd239000640 1 --2- 172.21.15.111:0/1801480972 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd2180a3240 0x7fd2180a5700 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.362+0000 7fd239000640 1 --2- 172.21.15.111:0/1801480972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd234101580 0x7fd234075fa0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.362+0000 7fd239000640 1 -- 172.21.15.111:0/1801480972 >> 172.21.15.111:0/1801480972 conn(0x7fd2340fd0d0 msgr2=0x7fd234105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.362+0000 7fd239000640 1 -- 172.21.15.111:0/1801480972 shutdown_connections 2024-09-15T23:05:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:40.362+0000 7fd239000640 1 -- 172.21.15.111:0/1801480972 wait complete. 2024-09-15T23:05:40.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:39 smithi121 ceph-mon[34815]: pgmap v943: 97 pgs: 97 active+clean; 584 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 5.8 MiB/s wr, 192 op/s 2024-09-15T23:05:40.664 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124330 16.53 MB/sec execute 248 sec latency 14811.057 ms 2024-09-15T23:05:41.665 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124330 16.46 MB/sec execute 249 sec latency 15811.113 ms 2024-09-15T23:05:42.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:41 smithi111 ceph-mon[30629]: pgmap v944: 97 pgs: 97 active+clean; 589 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 5.8 MiB/s wr, 189 op/s 2024-09-15T23:05:42.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:42 smithi192 ceph-mon[35427]: pgmap v944: 97 pgs: 97 active+clean; 589 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 5.8 MiB/s wr, 189 op/s 2024-09-15T23:05:42.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:42 smithi121 ceph-mon[34815]: pgmap v944: 97 pgs: 97 active+clean; 589 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 5.8 MiB/s wr, 189 op/s 2024-09-15T23:05:42.665 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124338 16.40 MB/sec execute 250 sec latency 16811.178 ms 2024-09-15T23:05:43.665 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124338 16.33 MB/sec execute 251 sec latency 17811.231 ms 2024-09-15T23:05:44.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:44 smithi192 ceph-mon[35427]: pgmap v945: 97 pgs: 97 active+clean; 576 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 5.5 MiB/s wr, 197 op/s 2024-09-15T23:05:44.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:44 smithi121 ceph-mon[34815]: pgmap v945: 97 pgs: 97 active+clean; 576 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 5.5 MiB/s wr, 197 op/s 2024-09-15T23:05:44.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:44 smithi111 ceph-mon[30629]: pgmap v945: 97 pgs: 97 active+clean; 576 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 5.5 MiB/s wr, 197 op/s 2024-09-15T23:05:44.665 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124338 16.27 MB/sec execute 252 sec latency 18811.281 ms 2024-09-15T23:05:45.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:45 smithi121 ceph-mon[34815]: pgmap v946: 97 pgs: 97 active+clean; 551 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 6.0 MiB/s wr, 255 op/s 2024-09-15T23:05:45.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:45 smithi111 ceph-mon[30629]: pgmap v946: 97 pgs: 97 active+clean; 551 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 6.0 MiB/s wr, 255 op/s 2024-09-15T23:05:45.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:45 smithi192 ceph-mon[35427]: pgmap v946: 97 pgs: 97 active+clean; 551 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 6.0 MiB/s wr, 255 op/s 2024-09-15T23:05:45.665 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124338 16.20 MB/sec execute 253 sec latency 19811.331 ms 2024-09-15T23:05:46.665 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124340 16.14 MB/sec execute 254 sec latency 20811.380 ms 2024-09-15T23:05:47.671 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124340 16.07 MB/sec execute 255 sec latency 21818.114 ms 2024-09-15T23:05:48.037 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:47 smithi111 ceph-mon[30629]: pgmap v947: 97 pgs: 97 active+clean; 534 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 5.2 MiB/s wr, 254 op/s 2024-09-15T23:05:48.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:47 smithi192 ceph-mon[35427]: pgmap v947: 97 pgs: 97 active+clean; 534 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 5.2 MiB/s wr, 254 op/s 2024-09-15T23:05:48.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:47 smithi121 ceph-mon[34815]: pgmap v947: 97 pgs: 97 active+clean; 534 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 5.2 MiB/s wr, 254 op/s 2024-09-15T23:05:48.671 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124340 16.01 MB/sec execute 256 sec latency 22818.183 ms 2024-09-15T23:05:49.672 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124341 15.95 MB/sec execute 257 sec latency 23818.243 ms 2024-09-15T23:05:50.009 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:49 smithi111 ceph-mon[30629]: pgmap v948: 97 pgs: 97 active+clean; 546 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 6.1 MiB/s wr, 258 op/s 2024-09-15T23:05:50.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:49 smithi192 ceph-mon[35427]: pgmap v948: 97 pgs: 97 active+clean; 546 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 6.1 MiB/s wr, 258 op/s 2024-09-15T23:05:50.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:49 smithi121 ceph-mon[34815]: pgmap v948: 97 pgs: 97 active+clean; 546 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 6.1 MiB/s wr, 258 op/s 2024-09-15T23:05:50.672 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124341 15.89 MB/sec execute 258 sec latency 24818.313 ms 2024-09-15T23:05:51.672 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124342 15.83 MB/sec execute 259 sec latency 24981.691 ms 2024-09-15T23:05:52.120 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:51 smithi111 ceph-mon[30629]: pgmap v949: 97 pgs: 97 active+clean; 548 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 4.9 MiB/s wr, 235 op/s 2024-09-15T23:05:52.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:51 smithi192 ceph-mon[35427]: pgmap v949: 97 pgs: 97 active+clean; 548 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 4.9 MiB/s wr, 235 op/s 2024-09-15T23:05:52.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:51 smithi121 ceph-mon[34815]: pgmap v949: 97 pgs: 97 active+clean; 548 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 4.9 MiB/s wr, 235 op/s 2024-09-15T23:05:52.672 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124342 15.76 MB/sec execute 260 sec latency 17897.817 ms 2024-09-15T23:05:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.489+0000 7ffa53e00640 1 -- 172.21.15.111:0/3880539311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa54074fc0 msgr2=0x7ffa5410df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.489+0000 7ffa53e00640 1 --2- 172.21.15.111:0/3880539311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa54074fc0 0x7ffa5410df40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009900 tx=0x7ffa4002a400 comp rx=0 tx=0).stop 2024-09-15T23:05:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.489+0000 7ffa53e00640 1 -- 172.21.15.111:0/3880539311 shutdown_connections 2024-09-15T23:05:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.489+0000 7ffa53e00640 1 --2- 172.21.15.111:0/3880539311 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa54105340 0x7ffa54110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.489+0000 7ffa53e00640 1 --2- 172.21.15.111:0/3880539311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa54074fc0 0x7ffa5410df40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.489+0000 7ffa53e00640 1 --2- 172.21.15.111:0/3880539311 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa54074680 0x7ffa54074a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.489+0000 7ffa53e00640 1 -- 172.21.15.111:0/3880539311 >> 172.21.15.111:0/3880539311 conn(0x7ffa540fd0d0 msgr2=0x7ffa540ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.489+0000 7ffa53e00640 1 -- 172.21.15.111:0/3880539311 shutdown_connections 2024-09-15T23:05:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.489+0000 7ffa53e00640 1 -- 172.21.15.111:0/3880539311 wait complete. 2024-09-15T23:05:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.489+0000 7ffa53e00640 1 Processor -- start 2024-09-15T23:05:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.489+0000 7ffa53e00640 1 -- start start 2024-09-15T23:05:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.489+0000 7ffa53e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa54074680 0x7ffa5419e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.489+0000 7ffa53e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa54074fc0 0x7ffa5419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.489+0000 7ffa53e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa541a39f0 0x7ffa541a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.489+0000 7ffa53e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffa54113db0 con 0x7ffa54074fc0 2024-09-15T23:05:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.489+0000 7ffa53e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffa54113c30 con 0x7ffa54074680 2024-09-15T23:05:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.489+0000 7ffa53e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ffa54113f30 con 0x7ffa541a39f0 2024-09-15T23:05:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.489+0000 7ffa52000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa54074fc0 0x7ffa5419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.489+0000 7ffa52000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa54074fc0 0x7ffa5419e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57810/0 (socket says 172.21.15.111:57810) 2024-09-15T23:05:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.489+0000 7ffa52000640 1 -- 172.21.15.111:0/3962140227 learned_addr learned my addr 172.21.15.111:0/3962140227 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:05:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.489+0000 7ffa52a00640 1 --2- 172.21.15.111:0/3962140227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa54074680 0x7ffa5419e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.489+0000 7ffa52000640 1 -- 172.21.15.111:0/3962140227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa541a39f0 msgr2=0x7ffa541a5de0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T23:05:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.489+0000 7ffa52000640 1 --2- 172.21.15.111:0/3962140227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa541a39f0 0x7ffa541a5de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.489+0000 7ffa52000640 1 -- 172.21.15.111:0/3962140227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa54074680 msgr2=0x7ffa5419e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.489+0000 7ffa52000640 1 --2- 172.21.15.111:0/3962140227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa54074680 0x7ffa5419e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.489+0000 7ffa52000640 1 -- 172.21.15.111:0/3962140227 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa541a6450 con 0x7ffa54074fc0 2024-09-15T23:05:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.494+0000 7ffa52000640 1 --2- 172.21.15.111:0/3962140227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa54074fc0 0x7ffa5419e990 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ffa40031440 tx=0x7ffa40032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.494+0000 7ffa4b400640 1 -- 172.21.15.111:0/3962140227 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4003f030 con 0x7ffa54074fc0 2024-09-15T23:05:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.494+0000 7ffa53e00640 1 -- 172.21.15.111:0/3962140227 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa541a66e0 con 0x7ffa54074fc0 2024-09-15T23:05:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.494+0000 7ffa4b400640 1 -- 172.21.15.111:0/3962140227 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa40003ce0 con 0x7ffa54074fc0 2024-09-15T23:05:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.494+0000 7ffa4b400640 1 -- 172.21.15.111:0/3962140227 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40035070 con 0x7ffa54074fc0 2024-09-15T23:05:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.494+0000 7ffa53e00640 1 -- 172.21.15.111:0/3962140227 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa541a6ca0 con 0x7ffa54074fc0 2024-09-15T23:05:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.494+0000 7ffa5ca32740 1 Processor -- start 2024-09-15T23:05:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.494+0000 7ffa5ca32740 1 -- start start 2024-09-15T23:05:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.494+0000 7ffa5ca32740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646473e7260 0x5646473e7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.494+0000 7ffa5ca32740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646473e7ba0 0x5646473ec660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.494+0000 7ffa5ca32740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646473ecc30 0x5646473ef0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.494+0000 7ffa5ca32740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56464734c100 con 0x5646473ecc30 2024-09-15T23:05:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.494+0000 7ffa5ca32740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5646472f41b0 con 0x5646473e7ba0 2024-09-15T23:05:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.494+0000 7ffa5ca32740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5646473480c0 con 0x5646473e7260 2024-09-15T23:05:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.494+0000 7ffa4b400640 1 -- 172.21.15.111:0/3962140227 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7ffa40042030 con 0x7ffa54074fc0 2024-09-15T23:05:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.494+0000 7ffa53400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646473e7260 0x5646473e7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.494+0000 7ffa53400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646473e7260 0x5646473e7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:35780/0 (socket says 172.21.15.111:35780) 2024-09-15T23:05:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.494+0000 7ffa53400640 1 -- 172.21.15.111:0/2520015663 learned_addr learned my addr 172.21.15.111:0/2520015663 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:05:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.499+0000 7ffa52000640 1 --2- 172.21.15.111:0/2520015663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646473ecc30 0x5646473ef0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.499+0000 7ffa53400640 1 -- 172.21.15.111:0/2520015663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646473e7ba0 msgr2=0x5646473ec660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:05:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.499+0000 7ffa53400640 1 --2- 172.21.15.111:0/2520015663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646473e7ba0 0x5646473ec660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.499+0000 7ffa53400640 1 -- 172.21.15.111:0/2520015663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646473ecc30 msgr2=0x5646473ef0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.499+0000 7ffa53400640 1 --2- 172.21.15.111:0/2520015663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646473ecc30 0x5646473ef0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.499+0000 7ffa53400640 1 -- 172.21.15.111:0/2520015663 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56464705f210 con 0x5646473e7260 2024-09-15T23:05:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.499+0000 7ffa4b400640 1 --2- 172.21.15.111:0/3962140227 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ffa380a32c0 0x7ffa380a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.499+0000 7ffa4b400640 1 -- 172.21.15.111:0/3962140227 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7ffa400efe00 con 0x7ffa54074fc0 2024-09-15T23:05:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.499+0000 7ffa52a00640 1 --2- 172.21.15.111:0/3962140227 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ffa380a32c0 0x7ffa380a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa53400640 1 --2- 172.21.15.111:0/2520015663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646473e7260 0x5646473e7660 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009e30 tx=0x7ffa4c00e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ce3559eee88e54f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa52a00640 1 --2- 172.21.15.111:0/3962140227 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ffa380a32c0 0x7ffa380a5780 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x5646473e85d0 tx=0x7ffa44009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa3fe00640 1 -- 172.21.15.111:0/2520015663 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c004d20 con 0x5646473e7260 2024-09-15T23:05:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa3fe00640 1 -- 172.21.15.111:0/2520015663 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa4c00ee70 con 0x5646473e7260 2024-09-15T23:05:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa3fe00640 1 -- 172.21.15.111:0/2520015663 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c0107f0 con 0x5646473e7260 2024-09-15T23:05:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa5ca32740 1 -- 172.21.15.111:0/2520015663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646473e7260 msgr2=0x5646473e7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa5ca32740 1 --2- 172.21.15.111:0/2520015663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646473e7260 0x5646473e7660 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009e30 tx=0x7ffa4c00e720 comp rx=0 tx=0).stop 2024-09-15T23:05:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa5ca32740 1 -- 172.21.15.111:0/2520015663 shutdown_connections 2024-09-15T23:05:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa5ca32740 1 --2- 172.21.15.111:0/2520015663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646473ecc30 0x5646473ef0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa5ca32740 1 --2- 172.21.15.111:0/2520015663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646473e7ba0 0x5646473ec660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa5ca32740 1 --2- 172.21.15.111:0/2520015663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646473e7260 0x5646473e7660 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa5ca32740 1 -- 172.21.15.111:0/2520015663 >> 172.21.15.111:0/2520015663 conn(0x5646473e28d0 msgr2=0x5646473e2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa5ca32740 1 -- 172.21.15.111:0/2520015663 shutdown_connections 2024-09-15T23:05:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa5ca32740 1 -- 172.21.15.111:0/2520015663 wait complete. 2024-09-15T23:05:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa5ca32740 1 -- start start 2024-09-15T23:05:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa5ca32740 1 Processor -- start 2024-09-15T23:05:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa5ca32740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646473e7260 0x5646473f1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa5ca32740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646473e7ba0 0x5646473f1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa5ca32740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646473ecc30 0x5646473f89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa5ca32740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5646473efc00 con 0x5646473e7260 2024-09-15T23:05:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa5ca32740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5646472d5090 con 0x5646473e7ba0 2024-09-15T23:05:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa5ca32740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5646473efd80 con 0x5646473ecc30 2024-09-15T23:05:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa5ca32740 10 client.0 _ll_register_callbacks cb 0x564647329f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:05:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa52a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646473e7260 0x5646473f1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa52a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646473e7260 0x5646473f1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57830/0 (socket says 172.21.15.111:57830) 2024-09-15T23:05:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa52a00640 1 -- 172.21.15.111:0/3898332656 learned_addr learned my addr 172.21.15.111:0/3898332656 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:05:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa52000640 1 --2- 172.21.15.111:0/3898332656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646473e7ba0 0x5646473f1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa52a00640 1 -- 172.21.15.111:0/3898332656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646473ecc30 msgr2=0x5646473f89c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:05:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa52a00640 1 --2- 172.21.15.111:0/3898332656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646473ecc30 0x5646473f89c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa52a00640 1 -- 172.21.15.111:0/3898332656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646473e7ba0 msgr2=0x5646473f1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa52a00640 1 --2- 172.21.15.111:0/3898332656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646473e7ba0 0x5646473f1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa52a00640 1 -- 172.21.15.111:0/3898332656 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56464712b620 con 0x5646473e7260 2024-09-15T23:05:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa52a00640 1 --2- 172.21.15.111:0/3898332656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646473e7260 0x5646473f1440 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ffa44015700 tx=0x7ffa440047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325a45afbfc1ebd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:05:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa3cc00640 1 -- 172.21.15.111:0/3898332656 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44023020 con 0x5646473e7260 2024-09-15T23:05:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa3cc00640 1 -- 172.21.15.111:0/3898332656 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7ffa44013590 con 0x5646473e7260 2024-09-15T23:05:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa5ca32740 1 -- 172.21.15.111:0/3898332656 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa4c00ee70 con 0x5646473e7260 2024-09-15T23:05:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa3cc00640 1 -- 172.21.15.111:0/3898332656 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ffa4401b040 con 0x5646473e7260 2024-09-15T23:05:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa5ca32740 10 client.16581 fetch_fsmap learned FSMap version 35 2024-09-15T23:05:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa5ca32740 1 -- 172.21.15.111:0/3898332656 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5646473fb8d0 con 0x5646473e7260 2024-09-15T23:05:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa3cc00640 1 -- 172.21.15.111:0/3898332656 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa4c00ee70 con 0x5646473e7260 2024-09-15T23:05:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa3cc00640 1 -- 172.21.15.111:0/3898332656 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7ffa4401e030 con 0x5646473e7260 2024-09-15T23:05:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa5ca32740 10 client.16581 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:05:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa5ca32740 10 client.16581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:05:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa5ca32740 20 client.16581 populate_metadata read hostname 'smithi111' 2024-09-15T23:05:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa5ca32740 1 --2- 172.21.15.111:0/3898332656 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5646473fd010 0x5646473ff4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa5ca32740 4 client.16581 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:05:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa5ca32740 1 -- 172.21.15.111:0/3898332656 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564647359830 con 0x5646473fd010 2024-09-15T23:05:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.501+0000 7ffa52000640 1 --2- 172.21.15.111:0/3898332656 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5646473fd010 0x5646473ff4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.506+0000 7ffa52000640 1 --2- 172.21.15.111:0/3898332656 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5646473fd010 0x5646473ff4d0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc9e9a542ed4a898 server_cookie=4e1497551b659442 in_seq=0 out_seq=0 2024-09-15T23:05:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.506+0000 7ffa3cc00640 10 client.16581 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:05:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.506+0000 7ffa3cc00640 1 -- 172.21.15.111:0/3898332656 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564647359830 con 0x5646473fd010 2024-09-15T23:05:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.506+0000 7ffa3cc00640 10 client.16581 handle_command_reply: tid=0 2024-09-15T23:05:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.506+0000 7ffa5ca32740 1 client.16581 shutdown 2024-09-15T23:05:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.506+0000 7ffa5ca32740 1 -- 172.21.15.111:0/3898332656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646473e7260 msgr2=0x5646473f1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.506+0000 7ffa5ca32740 1 --2- 172.21.15.111:0/3898332656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646473e7260 0x5646473f1440 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ffa44015700 tx=0x7ffa440047d0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.506+0000 7ffa5ca32740 1 -- 172.21.15.111:0/3898332656 shutdown_connections 2024-09-15T23:05:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.506+0000 7ffa5ca32740 1 --2- 172.21.15.111:0/3898332656 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5646473fd010 0x5646473ff4d0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.506+0000 7ffa5ca32740 1 --2- 172.21.15.111:0/3898332656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646473ecc30 0x5646473f89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.506+0000 7ffa5ca32740 1 --2- 172.21.15.111:0/3898332656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646473e7ba0 0x5646473f1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.506+0000 7ffa5ca32740 1 --2- 172.21.15.111:0/3898332656 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646473e7260 0x5646473f1440 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.506+0000 7ffa3cc00640 0 client.16581 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:05:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.506+0000 7ffa5ca32740 1 -- 172.21.15.111:0/3898332656 >> 172.21.15.111:0/3898332656 conn(0x5646473e28d0 msgr2=0x5646473e40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.506+0000 7ffa5ca32740 1 -- 172.21.15.111:0/3898332656 shutdown_connections 2024-09-15T23:05:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.506+0000 7ffa5ca32740 1 -- 172.21.15.111:0/3898332656 wait complete. 2024-09-15T23:05:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.506+0000 7ffa5ca32740 20 client.16581 trim_cache size 0 max 16384 2024-09-15T23:05:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.518+0000 7ffa5ca32740 1 Processor -- start 2024-09-15T23:05:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.518+0000 7ffa5ca32740 1 -- start start 2024-09-15T23:05:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.518+0000 7ffa5ca32740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646473e7260 0x5646473dd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.518+0000 7ffa5ca32740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646473e7ba0 0x5646473f8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.518+0000 7ffa5ca32740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646473ecc30 0x5646473e2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.518+0000 7ffa5ca32740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffa541b1be0 con 0x5646473ecc30 2024-09-15T23:05:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.518+0000 7ffa5ca32740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffa541af4b0 con 0x5646473e7ba0 2024-09-15T23:05:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.518+0000 7ffa5ca32740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ffa541ad340 con 0x5646473e7260 2024-09-15T23:05:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.518+0000 7ffa53400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646473ecc30 0x5646473e2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.518+0000 7ffa53400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646473ecc30 0x5646473e2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57846/0 (socket says 172.21.15.111:57846) 2024-09-15T23:05:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.518+0000 7ffa53400640 1 -- 172.21.15.111:0/3206269891 learned_addr learned my addr 172.21.15.111:0/3206269891 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:05:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.518+0000 7ffa52000640 1 --2- 172.21.15.111:0/3206269891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646473e7ba0 0x5646473f8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.523+0000 7ffa53400640 1 -- 172.21.15.111:0/3206269891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646473e7260 msgr2=0x5646473dd8d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:05:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.523+0000 7ffa53400640 1 --2- 172.21.15.111:0/3206269891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646473e7260 0x5646473dd8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.523+0000 7ffa53400640 1 -- 172.21.15.111:0/3206269891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646473e7ba0 msgr2=0x5646473f8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.523+0000 7ffa53400640 1 --2- 172.21.15.111:0/3206269891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646473e7ba0 0x5646473f8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.523+0000 7ffa53400640 1 -- 172.21.15.111:0/3206269891 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646473f04b0 con 0x5646473ecc30 2024-09-15T23:05:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.523+0000 7ffa52000640 1 --2- 172.21.15.111:0/3206269891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646473e7ba0 0x5646473f8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:05:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.523+0000 7ffa53400640 1 --2- 172.21.15.111:0/3206269891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646473ecc30 0x5646473e2cb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009e00 tx=0x7ffa4c00efb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a8a82c8efaa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.523+0000 7ffa3f400640 1 -- 172.21.15.111:0/3206269891 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c003970 con 0x5646473ecc30 2024-09-15T23:05:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.523+0000 7ffa3f400640 1 -- 172.21.15.111:0/3206269891 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa4c005020 con 0x5646473ecc30 2024-09-15T23:05:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.523+0000 7ffa3f400640 1 -- 172.21.15.111:0/3206269891 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c018690 con 0x5646473ecc30 2024-09-15T23:05:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa5ca32740 1 -- 172.21.15.111:0/3206269891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646473ecc30 msgr2=0x5646473e2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa5ca32740 1 --2- 172.21.15.111:0/3206269891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646473ecc30 0x5646473e2cb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009e00 tx=0x7ffa4c00efb0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa5ca32740 1 -- 172.21.15.111:0/3206269891 shutdown_connections 2024-09-15T23:05:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa5ca32740 1 --2- 172.21.15.111:0/3206269891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646473ecc30 0x5646473e2cb0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa5ca32740 1 --2- 172.21.15.111:0/3206269891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646473e7ba0 0x5646473f8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa5ca32740 1 --2- 172.21.15.111:0/3206269891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646473e7260 0x5646473dd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa5ca32740 1 -- 172.21.15.111:0/3206269891 >> 172.21.15.111:0/3206269891 conn(0x5646473e28d0 msgr2=0x5646473f1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa5ca32740 1 -- 172.21.15.111:0/3206269891 shutdown_connections 2024-09-15T23:05:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa5ca32740 1 -- 172.21.15.111:0/3206269891 wait complete. 2024-09-15T23:05:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa5ca32740 1 -- start start 2024-09-15T23:05:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa5ca32740 1 Processor -- start 2024-09-15T23:05:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa5ca32740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646473e7260 0x5646473f61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa5ca32740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646473e7ba0 0x5646473f6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa5ca32740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646473ecc30 0x5646473f7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa5ca32740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7ffa541ad990 con 0x5646473e7ba0 2024-09-15T23:05:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa5ca32740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffa541aede0 con 0x5646473ecc30 2024-09-15T23:05:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa5ca32740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ffa541ae040 con 0x5646473e7260 2024-09-15T23:05:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa5ca32740 10 client.0 _ll_register_callbacks cb 0x7ffa440095e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:05:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa52000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646473e7ba0 0x5646473f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa52000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646473e7ba0 0x5646473f6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57852/0 (socket says 172.21.15.111:57852) 2024-09-15T23:05:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa52000640 1 -- 172.21.15.111:0/3495212644 learned_addr learned my addr 172.21.15.111:0/3495212644 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:05:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa53400640 1 --2- 172.21.15.111:0/3495212644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646473ecc30 0x5646473f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa52000640 1 -- 172.21.15.111:0/3495212644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646473e7260 msgr2=0x5646473f61f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:05:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa52000640 1 --2- 172.21.15.111:0/3495212644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646473e7260 0x5646473f61f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa52000640 1 -- 172.21.15.111:0/3495212644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646473ecc30 msgr2=0x5646473f7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:05:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa52000640 1 --2- 172.21.15.111:0/3495212644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646473ecc30 0x5646473f7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa52000640 1 -- 172.21.15.111:0/3495212644 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646473f7d50 con 0x5646473e7ba0 2024-09-15T23:05:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa52000640 1 --2- 172.21.15.111:0/3495212644 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646473e7ba0 0x5646473f6730 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ffa400b8e70 tx=0x7ffa40009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38fb0f1a714daae8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:05:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa3cc00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:05:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa3cc00640 1 -- 172.21.15.111:0/3495212644 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40049040 con 0x5646473e7ba0 2024-09-15T23:05:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa3cc00640 1 -- 172.21.15.111:0/3495212644 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7ffa4003bcd0 con 0x5646473e7ba0 2024-09-15T23:05:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa5ca32740 1 -- 172.21.15.111:0/3495212644 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa4c005020 con 0x5646473e7ba0 2024-09-15T23:05:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa3cc00640 1 -- 172.21.15.111:0/3495212644 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ffa400bcba0 con 0x5646473e7ba0 2024-09-15T23:05:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa5ca32740 10 client.16587 fetch_fsmap learned FSMap version 35 2024-09-15T23:05:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa5ca32740 1 -- 172.21.15.111:0/3495212644 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564647467fd0 con 0x5646473e7ba0 2024-09-15T23:05:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa3cc00640 1 -- 172.21.15.111:0/3495212644 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa4c005020 con 0x5646473e7ba0 2024-09-15T23:05:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa3cc00640 1 -- 172.21.15.111:0/3495212644 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7ffa400b8b10 con 0x5646473e7ba0 2024-09-15T23:05:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa5ca32740 10 client.16587 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:05:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa5ca32740 10 client.16587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:05:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa5ca32740 20 client.16587 populate_metadata read hostname 'smithi111' 2024-09-15T23:05:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa5ca32740 1 --2- 172.21.15.111:0/3495212644 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5646474688e0 0x56464746ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:05:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa5ca32740 4 client.16587 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:05:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa5ca32740 1 -- 172.21.15.111:0/3495212644 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564647353b80 con 0x5646474688e0 2024-09-15T23:05:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa52a00640 1 --2- 172.21.15.111:0/3495212644 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5646474688e0 0x56464746ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:05:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa52a00640 1 --2- 172.21.15.111:0/3495212644 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5646474688e0 0x56464746ad00 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fa30d1a54d82454 server_cookie=cf53391a844a7650 in_seq=0 out_seq=0 2024-09-15T23:05:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.525+0000 7ffa3cc00640 10 client.16587 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:05:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.572+0000 7ffa3cc00640 1 -- 172.21.15.111:0/3495212644 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564647353b80 con 0x5646474688e0 2024-09-15T23:05:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.572+0000 7ffa3cc00640 10 client.16587 handle_command_reply: tid=0 2024-09-15T23:05:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.572+0000 7ffa5ca32740 1 client.16587 shutdown 2024-09-15T23:05:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.572+0000 7ffa5ca32740 1 -- 172.21.15.111:0/3495212644 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646473e7ba0 msgr2=0x5646473f6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.572+0000 7ffa5ca32740 1 --2- 172.21.15.111:0/3495212644 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646473e7ba0 0x5646473f6730 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ffa400b8e70 tx=0x7ffa40009260 comp rx=0 tx=0).stop 2024-09-15T23:05:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.572+0000 7ffa5ca32740 1 -- 172.21.15.111:0/3495212644 shutdown_connections 2024-09-15T23:05:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.572+0000 7ffa5ca32740 1 --2- 172.21.15.111:0/3495212644 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5646474688e0 0x56464746ad00 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.572+0000 7ffa5ca32740 1 --2- 172.21.15.111:0/3495212644 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646473ecc30 0x5646473f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.572+0000 7ffa5ca32740 1 --2- 172.21.15.111:0/3495212644 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5646473e7ba0 0x5646473f6730 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.572+0000 7ffa5ca32740 1 --2- 172.21.15.111:0/3495212644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5646473e7260 0x5646473f61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.572+0000 7ffa3cc00640 0 client.16587 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:05:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.572+0000 7ffa5ca32740 1 -- 172.21.15.111:0/3495212644 >> 172.21.15.111:0/3495212644 conn(0x5646473e28d0 msgr2=0x5646473f0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.572+0000 7ffa5ca32740 1 -- 172.21.15.111:0/3495212644 shutdown_connections 2024-09-15T23:05:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.572+0000 7ffa5ca32740 1 -- 172.21.15.111:0/3495212644 wait complete. 2024-09-15T23:05:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.572+0000 7ffa5ca32740 20 client.16587 trim_cache size 0 max 16384 2024-09-15T23:05:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:05:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:05:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:05:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:05:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.577+0000 7ffa53e00640 1 -- 172.21.15.111:0/3962140227 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ffa380a32c0 msgr2=0x7ffa380a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.577+0000 7ffa53e00640 1 --2- 172.21.15.111:0/3962140227 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ffa380a32c0 0x7ffa380a5780 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x5646473e85d0 tx=0x7ffa44009110 comp rx=0 tx=0).stop 2024-09-15T23:05:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.577+0000 7ffa53e00640 1 -- 172.21.15.111:0/3962140227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa54074fc0 msgr2=0x7ffa5419e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:05:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.577+0000 7ffa53e00640 1 --2- 172.21.15.111:0/3962140227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa54074fc0 0x7ffa5419e990 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ffa40031440 tx=0x7ffa40032ce0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.577+0000 7ffa53e00640 1 -- 172.21.15.111:0/3962140227 shutdown_connections 2024-09-15T23:05:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.577+0000 7ffa53e00640 1 --2- 172.21.15.111:0/3962140227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa541a39f0 0x7ffa541a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.577+0000 7ffa53e00640 1 --2- 172.21.15.111:0/3962140227 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ffa54074fc0 0x7ffa5419e990 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.577+0000 7ffa53e00640 1 --2- 172.21.15.111:0/3962140227 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7ffa380a32c0 0x7ffa380a5780 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.577+0000 7ffa53e00640 1 --2- 172.21.15.111:0/3962140227 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffa54074680 0x7ffa5419e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:05:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.577+0000 7ffa53e00640 1 -- 172.21.15.111:0/3962140227 >> 172.21.15.111:0/3962140227 conn(0x7ffa540fd0d0 msgr2=0x7ffa54110070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:05:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.577+0000 7ffa53e00640 1 -- 172.21.15.111:0/3962140227 shutdown_connections 2024-09-15T23:05:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:05:53.577+0000 7ffa53e00640 1 -- 172.21.15.111:0/3962140227 wait complete. 2024-09-15T23:05:53.672 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124342 15.70 MB/sec execute 261 sec latency 18897.863 ms 2024-09-15T23:05:54.116 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:53 smithi111 ceph-mon[30629]: pgmap v950: 97 pgs: 97 active+clean; 543 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 4.1 MiB/s wr, 205 op/s 2024-09-15T23:05:54.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:53 smithi192 ceph-mon[35427]: pgmap v950: 97 pgs: 97 active+clean; 543 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 4.1 MiB/s wr, 205 op/s 2024-09-15T23:05:54.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:53 smithi121 ceph-mon[34815]: pgmap v950: 97 pgs: 97 active+clean; 543 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 4.1 MiB/s wr, 205 op/s 2024-09-15T23:05:54.672 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124342 15.64 MB/sec execute 262 sec latency 19897.915 ms 2024-09-15T23:05:55.672 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124350 15.59 MB/sec execute 263 sec latency 20897.973 ms 2024-09-15T23:05:56.098 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:55 smithi111 ceph-mon[30629]: pgmap v951: 97 pgs: 97 active+clean; 541 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 3.4 MiB/s wr, 179 op/s 2024-09-15T23:05:56.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:55 smithi192 ceph-mon[35427]: pgmap v951: 97 pgs: 97 active+clean; 541 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 3.4 MiB/s wr, 179 op/s 2024-09-15T23:05:56.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:55 smithi121 ceph-mon[34815]: pgmap v951: 97 pgs: 97 active+clean; 541 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 3.4 MiB/s wr, 179 op/s 2024-09-15T23:05:56.672 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124350 15.53 MB/sec execute 264 sec latency 21898.029 ms 2024-09-15T23:05:57.672 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124350 15.47 MB/sec execute 265 sec latency 22898.092 ms 2024-09-15T23:05:58.096 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:57 smithi111 ceph-mon[30629]: pgmap v952: 97 pgs: 97 active+clean; 540 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 2.1 MiB/s wr, 110 op/s 2024-09-15T23:05:58.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:57 smithi192 ceph-mon[35427]: pgmap v952: 97 pgs: 97 active+clean; 540 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 2.1 MiB/s wr, 110 op/s 2024-09-15T23:05:58.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:57 smithi121 ceph-mon[34815]: pgmap v952: 97 pgs: 97 active+clean; 540 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 2.1 MiB/s wr, 110 op/s 2024-09-15T23:05:58.672 INFO:tasks.workunit.client.0.smithi111.stdout: 10 124440 15.42 MB/sec execute 266 sec latency 23758.829 ms 2024-09-15T23:05:59.241 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:58 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:05:59.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:58 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:05:59.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:58 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:05:59.672 INFO:tasks.workunit.client.0.smithi111.stdout: 10 125225 15.53 MB/sec execute 267 sec latency 64.171 ms 2024-09-15T23:06:00.266 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:59 smithi111 ceph-mon[30629]: pgmap v953: 97 pgs: 97 active+clean; 535 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.9 MiB/s wr, 91 op/s 2024-09-15T23:06:00.266 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:59 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:06:00.266 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:59 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:06:00.267 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:05:59 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:06:00.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:59 smithi192 ceph-mon[35427]: pgmap v953: 97 pgs: 97 active+clean; 535 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.9 MiB/s wr, 91 op/s 2024-09-15T23:06:00.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:59 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:06:00.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:59 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:06:00.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:05:59 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:06:00.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:59 smithi121 ceph-mon[34815]: pgmap v953: 97 pgs: 97 active+clean; 535 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 1.9 MiB/s wr, 91 op/s 2024-09-15T23:06:00.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:59 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:06:00.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:59 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:06:00.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:05:59 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:06:00.672 INFO:tasks.workunit.client.0.smithi111.stdout: 10 125827 15.61 MB/sec execute 268 sec latency 152.035 ms 2024-09-15T23:06:01.672 INFO:tasks.workunit.client.0.smithi111.stdout: 10 126632 15.69 MB/sec execute 269 sec latency 71.858 ms 2024-09-15T23:06:02.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:01 smithi192 ceph-mon[35427]: pgmap v954: 97 pgs: 97 active+clean; 531 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 892 KiB/s wr, 143 op/s 2024-09-15T23:06:02.374 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:01 smithi111 ceph-mon[30629]: pgmap v954: 97 pgs: 97 active+clean; 531 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 892 KiB/s wr, 143 op/s 2024-09-15T23:06:02.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:01 smithi121 ceph-mon[34815]: pgmap v954: 97 pgs: 97 active+clean; 531 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 892 KiB/s wr, 143 op/s 2024-09-15T23:06:02.672 INFO:tasks.workunit.client.0.smithi111.stdout: 10 127393 15.79 MB/sec execute 270 sec latency 56.189 ms 2024-09-15T23:06:03.672 INFO:tasks.workunit.client.0.smithi111.stdout: 10 128378 15.94 MB/sec execute 271 sec latency 59.318 ms 2024-09-15T23:06:04.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:03 smithi192 ceph-mon[35427]: pgmap v955: 97 pgs: 97 active+clean; 521 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 691 KiB/s wr, 169 op/s 2024-09-15T23:06:04.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:03 smithi121 ceph-mon[34815]: pgmap v955: 97 pgs: 97 active+clean; 521 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 691 KiB/s wr, 169 op/s 2024-09-15T23:06:04.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:03 smithi111 ceph-mon[30629]: pgmap v955: 97 pgs: 97 active+clean; 521 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 691 KiB/s wr, 169 op/s 2024-09-15T23:06:04.672 INFO:tasks.workunit.client.0.smithi111.stdout: 10 129033 16.02 MB/sec execute 272 sec latency 70.335 ms 2024-09-15T23:06:05.673 INFO:tasks.workunit.client.0.smithi111.stdout: 10 129541 16.04 MB/sec execute 273 sec latency 74.936 ms 2024-09-15T23:06:06.286 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:05 smithi111 ceph-mon[30629]: pgmap v956: 97 pgs: 97 active+clean; 523 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.5 MiB/s wr, 309 op/s 2024-09-15T23:06:06.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:05 smithi192 ceph-mon[35427]: pgmap v956: 97 pgs: 97 active+clean; 523 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.5 MiB/s wr, 309 op/s 2024-09-15T23:06:06.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:05 smithi121 ceph-mon[34815]: pgmap v956: 97 pgs: 97 active+clean; 523 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.5 MiB/s wr, 309 op/s 2024-09-15T23:06:06.673 INFO:tasks.workunit.client.0.smithi111.stdout: 10 130292 16.13 MB/sec execute 274 sec latency 84.859 ms 2024-09-15T23:06:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.723+0000 7fcff7e00640 1 -- 172.21.15.111:0/542084075 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcff8109af0 msgr2=0x7fcff8110380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.723+0000 7fcff7e00640 1 --2- 172.21.15.111:0/542084075 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcff8109af0 0x7fcff8110380 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcff0009ca0 tx=0x7fcff002a8c0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.723+0000 7fcff7e00640 1 -- 172.21.15.111:0/542084075 shutdown_connections 2024-09-15T23:06:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.723+0000 7fcff7e00640 1 --2- 172.21.15.111:0/542084075 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcff8109af0 0x7fcff8110380 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.723+0000 7fcff7e00640 1 --2- 172.21.15.111:0/542084075 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcff8109130 0x7fcff81095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.723+0000 7fcff7e00640 1 --2- 172.21.15.111:0/542084075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcff8108410 0x7fcff8108810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.723+0000 7fcff7e00640 1 -- 172.21.15.111:0/542084075 >> 172.21.15.111:0/542084075 conn(0x7fcff806b570 msgr2=0x7fcff80797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:06:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.723+0000 7fcff7e00640 1 -- 172.21.15.111:0/542084075 shutdown_connections 2024-09-15T23:06:06.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.723+0000 7fcff7e00640 1 -- 172.21.15.111:0/542084075 wait complete. 2024-09-15T23:06:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.724+0000 7fcff7e00640 1 Processor -- start 2024-09-15T23:06:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.725+0000 7fcff7e00640 1 -- start start 2024-09-15T23:06:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.725+0000 7fcff7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcff8108410 0x7fcff819e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.725+0000 7fcff7e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcff8109130 0x7fcff819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.725+0000 7fcff7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcff8109af0 0x7fcff81a57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.725+0000 7fcff7e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcff8113c30 con 0x7fcff8108410 2024-09-15T23:06:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.725+0000 7fcff7e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcff8113ab0 con 0x7fcff8109af0 2024-09-15T23:06:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.725+0000 7fcff7e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcff8113db0 con 0x7fcff8109130 2024-09-15T23:06:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.725+0000 7fcff6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcff8108410 0x7fcff819e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.725+0000 7fcff6000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcff8109130 0x7fcff819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.725+0000 7fcff6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcff8108410 0x7fcff819e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52434/0 (socket says 172.21.15.111:52434) 2024-09-15T23:06:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.725+0000 7fcff6a00640 1 -- 172.21.15.111:0/2843917690 learned_addr learned my addr 172.21.15.111:0/2843917690 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:06:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.725+0000 7fcff7400640 1 --2- 172.21.15.111:0/2843917690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcff8109af0 0x7fcff81a57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.726+0000 7fcff6000640 1 -- 172.21.15.111:0/2843917690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcff8109af0 msgr2=0x7fcff81a57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.726+0000 7fcff6000640 1 --2- 172.21.15.111:0/2843917690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcff8109af0 0x7fcff81a57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.726+0000 7fcff6000640 1 -- 172.21.15.111:0/2843917690 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcff8108410 msgr2=0x7fcff819e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.726+0000 7fcff6000640 1 --2- 172.21.15.111:0/2843917690 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcff8108410 0x7fcff819e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.726+0000 7fcff6000640 1 -- 172.21.15.111:0/2843917690 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff81a5e60 con 0x7fcff8109130 2024-09-15T23:06:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.726+0000 7fcff6a00640 1 --2- 172.21.15.111:0/2843917690 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcff8108410 0x7fcff819e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:06:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.726+0000 7fcff6000640 1 --2- 172.21.15.111:0/2843917690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcff8109130 0x7fcff819e780 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcfe400e9f0 tx=0x7fcfe400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:06:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.726+0000 7fcfef400640 1 -- 172.21.15.111:0/2843917690 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe400ccc0 con 0x7fcff8109130 2024-09-15T23:06:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.726+0000 7fcff7e00640 1 -- 172.21.15.111:0/2843917690 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff81a6150 con 0x7fcff8109130 2024-09-15T23:06:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.726+0000 7fcfef400640 1 -- 172.21.15.111:0/2843917690 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfe4005020 con 0x7fcff8109130 2024-09-15T23:06:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.726+0000 7fcfef400640 1 -- 172.21.15.111:0/2843917690 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe40049e0 con 0x7fcff8109130 2024-09-15T23:06:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.726+0000 7fcff7e00640 1 -- 172.21.15.111:0/2843917690 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff81a66a0 con 0x7fcff8109130 2024-09-15T23:06:06.732 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.730+0000 7fd000815740 1 Processor -- start 2024-09-15T23:06:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.730+0000 7fd000815740 1 -- start start 2024-09-15T23:06:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.731+0000 7fd000815740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653c67c1260 0x5653c67c1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.731+0000 7fd000815740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653c67c1ba0 0x5653c67c6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.731+0000 7fd000815740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5653c67c6c30 0x5653c67c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.731+0000 7fd000815740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5653c6726100 con 0x5653c67c6c30 2024-09-15T23:06:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.731+0000 7fd000815740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5653c66ce1b0 con 0x5653c67c1260 2024-09-15T23:06:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.731+0000 7fd000815740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5653c67220c0 con 0x5653c67c1ba0 2024-09-15T23:06:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.731+0000 7fcff7400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653c67c1260 0x5653c67c1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.731+0000 7fcff7400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653c67c1260 0x5653c67c1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:35974/0 (socket says 172.21.15.111:35974) 2024-09-15T23:06:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.731+0000 7fcff7400640 1 -- 172.21.15.111:0/3837019485 learned_addr learned my addr 172.21.15.111:0/3837019485 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:06:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.731+0000 7fcff6a00640 1 --2- 172.21.15.111:0/3837019485 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653c67c1ba0 0x5653c67c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:06.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.731+0000 7fcfef400640 1 -- 172.21.15.111:0/2843917690 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fcfe4005560 con 0x7fcff8109130 2024-09-15T23:06:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.731+0000 7fcff6a00640 1 -- 172.21.15.111:0/3837019485 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653c67c1260 msgr2=0x5653c67c1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:06:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.731+0000 7fcff6a00640 1 --2- 172.21.15.111:0/3837019485 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653c67c1260 0x5653c67c1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.731+0000 7fcff6a00640 1 -- 172.21.15.111:0/3837019485 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5653c67c6c30 msgr2=0x5653c67c90f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T23:06:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.732+0000 7fcff6a00640 1 --2- 172.21.15.111:0/3837019485 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5653c67c6c30 0x5653c67c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.732+0000 7fcff6a00640 1 -- 172.21.15.111:0/3837019485 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653c6439210 con 0x5653c67c1ba0 2024-09-15T23:06:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.732+0000 7fcff7400640 1 --2- 172.21.15.111:0/3837019485 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653c67c1260 0x5653c67c1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:06:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.732+0000 7fcff6a00640 1 --2- 172.21.15.111:0/3837019485 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653c67c1ba0 0x5653c67c6660 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcfe80114c0 tx=0x7fcfe8011980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8b37e159d6a2cf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:06:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.732+0000 7fcfe3e00640 1 -- 172.21.15.111:0/3837019485 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8005020 con 0x5653c67c1ba0 2024-09-15T23:06:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.732+0000 7fcfe3e00640 1 -- 172.21.15.111:0/3837019485 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfe80047a0 con 0x5653c67c1ba0 2024-09-15T23:06:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.732+0000 7fcfe3e00640 1 -- 172.21.15.111:0/3837019485 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe800f9d0 con 0x5653c67c1ba0 2024-09-15T23:06:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.733+0000 7fcfef400640 1 --2- 172.21.15.111:0/2843917690 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fcfdc0a3240 0x7fcfdc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.733+0000 7fd000815740 1 -- 172.21.15.111:0/3837019485 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653c67c1ba0 msgr2=0x5653c67c6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.733+0000 7fd000815740 1 --2- 172.21.15.111:0/3837019485 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653c67c1ba0 0x5653c67c6660 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcfe80114c0 tx=0x7fcfe8011980 comp rx=0 tx=0).stop 2024-09-15T23:06:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.733+0000 7fcfef400640 1 -- 172.21.15.111:0/2843917690 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fcfe40cf7e0 con 0x7fcff8109130 2024-09-15T23:06:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.734+0000 7fd000815740 1 -- 172.21.15.111:0/3837019485 shutdown_connections 2024-09-15T23:06:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.734+0000 7fd000815740 1 --2- 172.21.15.111:0/3837019485 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5653c67c6c30 0x5653c67c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.734+0000 7fd000815740 1 --2- 172.21.15.111:0/3837019485 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653c67c1ba0 0x5653c67c6660 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.734+0000 7fd000815740 1 --2- 172.21.15.111:0/3837019485 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653c67c1260 0x5653c67c1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.734+0000 7fd000815740 1 -- 172.21.15.111:0/3837019485 >> 172.21.15.111:0/3837019485 conn(0x5653c67bc8d0 msgr2=0x5653c67bccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:06:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.734+0000 7fcff7400640 1 --2- 172.21.15.111:0/2843917690 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fcfdc0a3240 0x7fcfdc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.734+0000 7fd000815740 1 -- 172.21.15.111:0/3837019485 shutdown_connections 2024-09-15T23:06:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.734+0000 7fd000815740 1 -- 172.21.15.111:0/3837019485 wait complete. 2024-09-15T23:06:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.734+0000 7fd000815740 1 -- start start 2024-09-15T23:06:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.735+0000 7fcff7400640 1 --2- 172.21.15.111:0/2843917690 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fcfdc0a3240 0x7fcfdc0a5700 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fcff0005e30 tx=0x7fcff003d040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:06:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.735+0000 7fd000815740 1 Processor -- start 2024-09-15T23:06:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.735+0000 7fd000815740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653c67c1260 0x5653c67cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.735+0000 7fd000815740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5653c67c1ba0 0x5653c67cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.735+0000 7fd000815740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653c67c6c30 0x5653c67d2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.735+0000 7fd000815740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5653c67c9c00 con 0x5653c67c1ba0 2024-09-15T23:06:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.735+0000 7fd000815740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5653c66af090 con 0x5653c67c1260 2024-09-15T23:06:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.735+0000 7fd000815740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5653c67c9d80 con 0x5653c67c6c30 2024-09-15T23:06:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.735+0000 7fd000815740 10 client.0 _ll_register_callbacks cb 0x5653c6703f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:06:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.735+0000 7fcff7400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653c67c6c30 0x5653c67d2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.735+0000 7fcff7400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653c67c6c30 0x5653c67d2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:56814/0 (socket says 172.21.15.111:56814) 2024-09-15T23:06:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.735+0000 7fcff7400640 1 -- 172.21.15.111:0/2997540319 learned_addr learned my addr 172.21.15.111:0/2997540319 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:06:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.735+0000 7fcff6000640 1 --2- 172.21.15.111:0/2997540319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5653c67c1ba0 0x5653c67cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.736+0000 7fcff6a00640 1 --2- 172.21.15.111:0/2997540319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653c67c1260 0x5653c67cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.736+0000 7fcff7400640 1 -- 172.21.15.111:0/2997540319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653c67c1260 msgr2=0x5653c67cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:06:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.736+0000 7fcff7400640 1 --2- 172.21.15.111:0/2997540319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653c67c1260 0x5653c67cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.736+0000 7fcff7400640 1 -- 172.21.15.111:0/2997540319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5653c67c1ba0 msgr2=0x5653c67cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:06:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.736+0000 7fcff7400640 1 --2- 172.21.15.111:0/2997540319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5653c67c1ba0 0x5653c67cb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.736+0000 7fcff7400640 1 -- 172.21.15.111:0/2997540319 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653c6505620 con 0x5653c67c6c30 2024-09-15T23:06:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.736+0000 7fcff6000640 1 --2- 172.21.15.111:0/2997540319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5653c67c1ba0 0x5653c67cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:06:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.736+0000 7fcff6a00640 1 --2- 172.21.15.111:0/2997540319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653c67c1260 0x5653c67cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:06:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.736+0000 7fcff7400640 1 --2- 172.21.15.111:0/2997540319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653c67c6c30 0x5653c67d2a30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcff0031a00 tx=0x7fcff002a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66f48fe1e853ef2e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:06:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.736+0000 7fcfe0c00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:06:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.736+0000 7fcfe0c00640 1 -- 172.21.15.111:0/2997540319 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff0055060 con 0x5653c67c6c30 2024-09-15T23:06:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.736+0000 7fd000815740 1 -- 172.21.15.111:0/2997540319 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfe80047a0 con 0x5653c67c6c30 2024-09-15T23:06:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.737+0000 7fcfe0c00640 1 -- 172.21.15.111:0/2997540319 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fcff0003e00 con 0x5653c67c6c30 2024-09-15T23:06:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.737+0000 7fcfe0c00640 1 -- 172.21.15.111:0/2997540319 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcff0004540 con 0x5653c67c6c30 2024-09-15T23:06:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.737+0000 7fd000815740 10 client.26533 fetch_fsmap learned FSMap version 35 2024-09-15T23:06:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.737+0000 7fd000815740 1 -- 172.21.15.111:0/2997540319 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5653c67d58e0 con 0x5653c67c6c30 2024-09-15T23:06:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.737+0000 7fcfe0c00640 1 -- 172.21.15.111:0/2997540319 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfe80047a0 con 0x5653c67c6c30 2024-09-15T23:06:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.737+0000 7fcfe0c00640 1 -- 172.21.15.111:0/2997540319 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fcff005a070 con 0x5653c67c6c30 2024-09-15T23:06:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.737+0000 7fd000815740 10 client.26533 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:06:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.738+0000 7fd000815740 10 client.26533 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:06:06.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.738+0000 7fd000815740 20 client.26533 populate_metadata read hostname 'smithi111' 2024-09-15T23:06:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.738+0000 7fd000815740 1 --2- 172.21.15.111:0/2997540319 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5653c67d6f60 0x5653c67d9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.738+0000 7fd000815740 4 client.26533 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:06:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.738+0000 7fd000815740 1 -- 172.21.15.111:0/2997540319 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653c6733830 con 0x5653c67d6f60 2024-09-15T23:06:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.738+0000 7fcff6a00640 1 --2- 172.21.15.111:0/2997540319 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5653c67d6f60 0x5653c67d9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.738+0000 7fcff6a00640 1 --2- 172.21.15.111:0/2997540319 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5653c67d6f60 0x5653c67d9350 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c7a5e8ab9c18894 server_cookie=aa190179f6fe96ef in_seq=0 out_seq=0 2024-09-15T23:06:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.738+0000 7fcfe0c00640 10 client.26533 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:06:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.739+0000 7fcfe0c00640 1 -- 172.21.15.111:0/2997540319 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5653c6733830 con 0x5653c67d6f60 2024-09-15T23:06:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.739+0000 7fcfe0c00640 10 client.26533 handle_command_reply: tid=0 2024-09-15T23:06:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.739+0000 7fd000815740 1 client.26533 shutdown 2024-09-15T23:06:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.739+0000 7fd000815740 1 -- 172.21.15.111:0/2997540319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653c67c6c30 msgr2=0x5653c67d2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.739+0000 7fd000815740 1 --2- 172.21.15.111:0/2997540319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653c67c6c30 0x5653c67d2a30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcff0031a00 tx=0x7fcff002a8c0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.739+0000 7fd000815740 1 -- 172.21.15.111:0/2997540319 shutdown_connections 2024-09-15T23:06:06.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.739+0000 7fd000815740 1 --2- 172.21.15.111:0/2997540319 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5653c67d6f60 0x5653c67d9350 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.739+0000 7fd000815740 1 --2- 172.21.15.111:0/2997540319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653c67c6c30 0x5653c67d2a30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.739+0000 7fd000815740 1 --2- 172.21.15.111:0/2997540319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5653c67c1ba0 0x5653c67cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.739+0000 7fcfe0c00640 0 client.26533 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:06:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.739+0000 7fd000815740 1 --2- 172.21.15.111:0/2997540319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653c67c1260 0x5653c67cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.740+0000 7fd000815740 1 -- 172.21.15.111:0/2997540319 >> 172.21.15.111:0/2997540319 conn(0x5653c67bc8d0 msgr2=0x5653c67be0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:06:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.740+0000 7fd000815740 1 -- 172.21.15.111:0/2997540319 shutdown_connections 2024-09-15T23:06:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.740+0000 7fd000815740 1 -- 172.21.15.111:0/2997540319 wait complete. 2024-09-15T23:06:06.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.740+0000 7fd000815740 20 client.26533 trim_cache size 0 max 16384 2024-09-15T23:06:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.761+0000 7fd000815740 1 Processor -- start 2024-09-15T23:06:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.761+0000 7fd000815740 1 -- start start 2024-09-15T23:06:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.761+0000 7fd000815740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653c67c1260 0x5653c67c1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.761+0000 7fd000815740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5653c67c1ba0 0x5653c67d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.761+0000 7fd000815740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653c67c6c30 0x5653c67bd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.761+0000 7fd000815740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81b1ba0 con 0x5653c67c1ba0 2024-09-15T23:06:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.761+0000 7fd000815740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81af470 con 0x5653c67c1260 2024-09-15T23:06:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.761+0000 7fd000815740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81ad2c0 con 0x5653c67c6c30 2024-09-15T23:06:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.761+0000 7fcff6a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653c67c1260 0x5653c67c1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.761+0000 7fcff7400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653c67c6c30 0x5653c67bd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.761+0000 7fcff6a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653c67c1260 0x5653c67c1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:36002/0 (socket says 172.21.15.111:36002) 2024-09-15T23:06:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.761+0000 7fcff6a00640 1 -- 172.21.15.111:0/3094336095 learned_addr learned my addr 172.21.15.111:0/3094336095 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:06:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.761+0000 7fcff6000640 1 --2- 172.21.15.111:0/3094336095 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5653c67c1ba0 0x5653c67d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.762+0000 7fcff6a00640 1 -- 172.21.15.111:0/3094336095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653c67c6c30 msgr2=0x5653c67bd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:06:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.762+0000 7fcff6a00640 1 --2- 172.21.15.111:0/3094336095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653c67c6c30 0x5653c67bd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.762+0000 7fcff6a00640 1 -- 172.21.15.111:0/3094336095 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5653c67c1ba0 msgr2=0x5653c67d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:06:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.762+0000 7fcff6a00640 1 --2- 172.21.15.111:0/3094336095 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5653c67c1ba0 0x5653c67d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.762+0000 7fcff6a00640 1 -- 172.21.15.111:0/3094336095 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653c67bc6f0 con 0x5653c67c1260 2024-09-15T23:06:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.762+0000 7fcff7400640 1 --2- 172.21.15.111:0/3094336095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653c67c6c30 0x5653c67bd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:06:06.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.762+0000 7fcff6a00640 1 --2- 172.21.15.111:0/3094336095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653c67c1260 0x5653c67c1640 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8011e80 tx=0x7fcfe8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a275fe2357508a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:06:06.765 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.762+0000 7fcfe3400640 1 -- 172.21.15.111:0/3094336095 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8032020 con 0x5653c67c1260 2024-09-15T23:06:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.762+0000 7fcfe3400640 1 -- 172.21.15.111:0/3094336095 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfe8005c50 con 0x5653c67c1260 2024-09-15T23:06:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.762+0000 7fcfe3400640 1 -- 172.21.15.111:0/3094336095 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe801d040 con 0x5653c67c1260 2024-09-15T23:06:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.763+0000 7fd000815740 1 -- 172.21.15.111:0/3094336095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653c67c1260 msgr2=0x5653c67c1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.763+0000 7fd000815740 1 --2- 172.21.15.111:0/3094336095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653c67c1260 0x5653c67c1640 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8011e80 tx=0x7fcfe8000bc0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.763+0000 7fd000815740 1 -- 172.21.15.111:0/3094336095 shutdown_connections 2024-09-15T23:06:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.763+0000 7fd000815740 1 --2- 172.21.15.111:0/3094336095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653c67c6c30 0x5653c67bd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.763+0000 7fd000815740 1 --2- 172.21.15.111:0/3094336095 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5653c67c1ba0 0x5653c67d29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.763+0000 7fd000815740 1 --2- 172.21.15.111:0/3094336095 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653c67c1260 0x5653c67c1640 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.763+0000 7fd000815740 1 -- 172.21.15.111:0/3094336095 >> 172.21.15.111:0/3094336095 conn(0x5653c67bc8d0 msgr2=0x5653c67cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:06:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.763+0000 7fd000815740 1 -- 172.21.15.111:0/3094336095 shutdown_connections 2024-09-15T23:06:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.763+0000 7fd000815740 1 -- 172.21.15.111:0/3094336095 wait complete. 2024-09-15T23:06:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.764+0000 7fd000815740 1 -- start start 2024-09-15T23:06:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.764+0000 7fd000815740 1 Processor -- start 2024-09-15T23:06:06.767 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.764+0000 7fd000815740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5653c67c1260 0x5653c67d29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:06.767 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.764+0000 7fd000815740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653c67c1ba0 0x5653c67cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:06.767 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.764+0000 7fd000815740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653c67c6c30 0x5653c67cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:06.767 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.764+0000 7fd000815740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81ad930 con 0x5653c67c1260 2024-09-15T23:06:06.767 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.764+0000 7fd000815740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81aeda0 con 0x5653c67c6c30 2024-09-15T23:06:06.767 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.764+0000 7fd000815740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcff81ae000 con 0x5653c67c1ba0 2024-09-15T23:06:06.767 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.764+0000 7fd000815740 10 client.0 _ll_register_callbacks cb 0x7fcff0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:06:06.767 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.764+0000 7fcff6000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653c67c1ba0 0x5653c67cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:06.767 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.764+0000 7fcff6000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653c67c1ba0 0x5653c67cf480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:56828/0 (socket says 172.21.15.111:56828) 2024-09-15T23:06:06.767 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.764+0000 7fcff6000640 1 -- 172.21.15.111:0/3716300783 learned_addr learned my addr 172.21.15.111:0/3716300783 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:06:06.767 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.764+0000 7fcff7400640 1 --2- 172.21.15.111:0/3716300783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653c67c6c30 0x5653c67cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:06.767 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.764+0000 7fcff6a00640 1 --2- 172.21.15.111:0/3716300783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5653c67c1260 0x5653c67d29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:06.767 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.765+0000 7fcff6000640 1 -- 172.21.15.111:0/3716300783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653c67c6c30 msgr2=0x5653c67cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:06:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.765+0000 7fcff6000640 1 --2- 172.21.15.111:0/3716300783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653c67c6c30 0x5653c67cfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.765+0000 7fcff6000640 1 -- 172.21.15.111:0/3716300783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5653c67c1260 msgr2=0x5653c67d29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:06:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.765+0000 7fcff6000640 1 --2- 172.21.15.111:0/3716300783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5653c67c1260 0x5653c67d29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.765+0000 7fcff6000640 1 -- 172.21.15.111:0/3716300783 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653c683fc60 con 0x5653c67c1ba0 2024-09-15T23:06:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.765+0000 7fcff6000640 1 --2- 172.21.15.111:0/3716300783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653c67c1ba0 0x5653c67cf480 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcfe40cd380 tx=0x7fcfe4098140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65df104817d13d8d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:06:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.765+0000 7fcfe0c00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:06:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.765+0000 7fcfe0c00640 1 -- 172.21.15.111:0/3716300783 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe401c040 con 0x5653c67c1ba0 2024-09-15T23:06:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.765+0000 7fd000815740 1 -- 172.21.15.111:0/3716300783 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfe8005c50 con 0x5653c67c1ba0 2024-09-15T23:06:06.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.765+0000 7fcfe0c00640 1 -- 172.21.15.111:0/3716300783 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fcfe4004bc0 con 0x5653c67c1ba0 2024-09-15T23:06:06.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.766+0000 7fcfe0c00640 1 -- 172.21.15.111:0/3716300783 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcfe407f050 con 0x5653c67c1ba0 2024-09-15T23:06:06.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.766+0000 7fd000815740 10 client.26539 fetch_fsmap learned FSMap version 35 2024-09-15T23:06:06.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.766+0000 7fd000815740 1 -- 172.21.15.111:0/3716300783 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5653c6841090 con 0x5653c67c1ba0 2024-09-15T23:06:06.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.766+0000 7fcfe0c00640 1 -- 172.21.15.111:0/3716300783 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfe8005c50 con 0x5653c67c1ba0 2024-09-15T23:06:06.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.766+0000 7fcfe0c00640 1 -- 172.21.15.111:0/3716300783 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fcfe409b2f0 con 0x5653c67c1ba0 2024-09-15T23:06:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.766+0000 7fd000815740 10 client.26539 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:06:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.766+0000 7fd000815740 10 client.26539 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:06:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.766+0000 7fd000815740 20 client.26539 populate_metadata read hostname 'smithi111' 2024-09-15T23:06:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.766+0000 7fd000815740 1 --2- 172.21.15.111:0/3716300783 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5653c6841a70 0x5653c6843e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.766+0000 7fd000815740 4 client.26539 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:06:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.766+0000 7fd000815740 1 -- 172.21.15.111:0/3716300783 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5653c672db80 con 0x5653c6841a70 2024-09-15T23:06:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.766+0000 7fcff6a00640 1 --2- 172.21.15.111:0/3716300783 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5653c6841a70 0x5653c6843e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.767+0000 7fcfe0c00640 10 client.26539 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:06:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.767+0000 7fcff6a00640 1 --2- 172.21.15.111:0/3716300783 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5653c6841a70 0x5653c6843e90 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff91a5f7680b7854 server_cookie=dcbc25f415d1c43 in_seq=0 out_seq=0 2024-09-15T23:06:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.946+0000 7fcfe0c00640 1 -- 172.21.15.111:0/3716300783 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5653c672db80 con 0x5653c6841a70 2024-09-15T23:06:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.946+0000 7fcfe0c00640 10 client.26539 handle_command_reply: tid=0 2024-09-15T23:06:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.946+0000 7fd000815740 1 client.26539 shutdown 2024-09-15T23:06:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.946+0000 7fd000815740 1 -- 172.21.15.111:0/3716300783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653c67c1ba0 msgr2=0x5653c67cf480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.946+0000 7fd000815740 1 --2- 172.21.15.111:0/3716300783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653c67c1ba0 0x5653c67cf480 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcfe40cd380 tx=0x7fcfe4098140 comp rx=0 tx=0).stop 2024-09-15T23:06:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.946+0000 7fd000815740 1 -- 172.21.15.111:0/3716300783 shutdown_connections 2024-09-15T23:06:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.946+0000 7fd000815740 1 --2- 172.21.15.111:0/3716300783 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5653c6841a70 0x5653c6843e90 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.946+0000 7fd000815740 1 --2- 172.21.15.111:0/3716300783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5653c67c6c30 0x5653c67cfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.946+0000 7fd000815740 1 --2- 172.21.15.111:0/3716300783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5653c67c1ba0 0x5653c67cf480 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.946+0000 7fd000815740 1 --2- 172.21.15.111:0/3716300783 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5653c67c1260 0x5653c67d29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.946+0000 7fcfe0c00640 0 client.26539 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:06:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.946+0000 7fd000815740 1 -- 172.21.15.111:0/3716300783 >> 172.21.15.111:0/3716300783 conn(0x5653c67bc8d0 msgr2=0x5653c67bb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:06:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.946+0000 7fd000815740 1 -- 172.21.15.111:0/3716300783 shutdown_connections 2024-09-15T23:06:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.946+0000 7fd000815740 1 -- 172.21.15.111:0/3716300783 wait complete. 2024-09-15T23:06:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.946+0000 7fd000815740 20 client.26539 trim_cache size 0 max 16384 2024-09-15T23:06:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:06:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:06:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:06:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:06:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.947+0000 7fcff7e00640 1 -- 172.21.15.111:0/2843917690 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fcfdc0a3240 msgr2=0x7fcfdc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.947+0000 7fcff7e00640 1 --2- 172.21.15.111:0/2843917690 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fcfdc0a3240 0x7fcfdc0a5700 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fcff0005e30 tx=0x7fcff003d040 comp rx=0 tx=0).stop 2024-09-15T23:06:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.947+0000 7fcff7e00640 1 -- 172.21.15.111:0/2843917690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcff8109130 msgr2=0x7fcff819e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.947+0000 7fcff7e00640 1 --2- 172.21.15.111:0/2843917690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcff8109130 0x7fcff819e780 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcfe400e9f0 tx=0x7fcfe400eeb0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.947+0000 7fcff7e00640 1 -- 172.21.15.111:0/2843917690 shutdown_connections 2024-09-15T23:06:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.947+0000 7fcff7e00640 1 --2- 172.21.15.111:0/2843917690 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fcfdc0a3240 0x7fcfdc0a5700 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.947+0000 7fcff7e00640 1 --2- 172.21.15.111:0/2843917690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcff8109af0 0x7fcff81a57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.947+0000 7fcff7e00640 1 --2- 172.21.15.111:0/2843917690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcff8109130 0x7fcff819e780 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.947+0000 7fcff7e00640 1 --2- 172.21.15.111:0/2843917690 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcff8108410 0x7fcff819e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.947+0000 7fcff7e00640 1 -- 172.21.15.111:0/2843917690 >> 172.21.15.111:0/2843917690 conn(0x7fcff806b570 msgr2=0x7fcff810c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:06:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.948+0000 7fcff7e00640 1 -- 172.21.15.111:0/2843917690 shutdown_connections 2024-09-15T23:06:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:06.948+0000 7fcff7e00640 1 -- 172.21.15.111:0/2843917690 wait complete. 2024-09-15T23:06:07.673 INFO:tasks.workunit.client.0.smithi111.stdout: 10 130972 16.19 MB/sec execute 275 sec latency 137.769 ms 2024-09-15T23:06:08.121 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:07 smithi111 ceph-mon[30629]: pgmap v957: 97 pgs: 97 active+clean; 517 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.0 MiB/s wr, 363 op/s 2024-09-15T23:06:08.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:07 smithi192 ceph-mon[35427]: pgmap v957: 97 pgs: 97 active+clean; 517 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.0 MiB/s wr, 363 op/s 2024-09-15T23:06:08.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:07 smithi121 ceph-mon[34815]: pgmap v957: 97 pgs: 97 active+clean; 517 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.0 MiB/s wr, 363 op/s 2024-09-15T23:06:08.673 INFO:tasks.workunit.client.0.smithi111.stdout: 10 131871 16.31 MB/sec execute 276 sec latency 51.519 ms 2024-09-15T23:06:09.673 INFO:tasks.workunit.client.0.smithi111.stdout: 10 132618 16.41 MB/sec execute 277 sec latency 56.386 ms 2024-09-15T23:06:10.092 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:09 smithi111 ceph-mon[30629]: pgmap v958: 97 pgs: 97 active+clean; 512 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.3 MiB/s wr, 548 op/s 2024-09-15T23:06:10.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:10 smithi192 ceph-mon[35427]: pgmap v958: 97 pgs: 97 active+clean; 512 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.3 MiB/s wr, 548 op/s 2024-09-15T23:06:10.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:09 smithi121 ceph-mon[34815]: pgmap v958: 97 pgs: 97 active+clean; 512 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.3 MiB/s wr, 548 op/s 2024-09-15T23:06:10.673 INFO:tasks.workunit.client.0.smithi111.stdout: 10 133219 16.49 MB/sec execute 278 sec latency 123.045 ms 2024-09-15T23:06:11.673 INFO:tasks.workunit.client.0.smithi111.stdout: 10 133823 16.53 MB/sec execute 279 sec latency 126.240 ms 2024-09-15T23:06:12.287 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:12 smithi111 ceph-mon[30629]: pgmap v959: 97 pgs: 97 active+clean; 517 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.7 MiB/s wr, 621 op/s 2024-09-15T23:06:12.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:12 smithi192 ceph-mon[35427]: pgmap v959: 97 pgs: 97 active+clean; 517 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.7 MiB/s wr, 621 op/s 2024-09-15T23:06:12.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:12 smithi121 ceph-mon[34815]: pgmap v959: 97 pgs: 97 active+clean; 517 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.7 MiB/s wr, 621 op/s 2024-09-15T23:06:12.681 INFO:tasks.workunit.client.0.smithi111.stdout: 10 134463 16.56 MB/sec execute 280 sec latency 91.248 ms 2024-09-15T23:06:13.673 INFO:tasks.workunit.client.0.smithi111.stdout: 10 135306 16.70 MB/sec execute 281 sec latency 56.203 ms 2024-09-15T23:06:14.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:14 smithi192 ceph-mon[35427]: pgmap v960: 97 pgs: 97 active+clean; 521 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.8 MiB/s wr, 578 op/s 2024-09-15T23:06:14.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:14 smithi121 ceph-mon[34815]: pgmap v960: 97 pgs: 97 active+clean; 521 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.8 MiB/s wr, 578 op/s 2024-09-15T23:06:14.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:14 smithi111 ceph-mon[30629]: pgmap v960: 97 pgs: 97 active+clean; 521 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.8 MiB/s wr, 578 op/s 2024-09-15T23:06:14.673 INFO:tasks.workunit.client.0.smithi111.stdout: 10 136023 16.76 MB/sec execute 282 sec latency 60.322 ms 2024-09-15T23:06:15.673 INFO:tasks.workunit.client.0.smithi111.stdout: 10 136689 16.83 MB/sec execute 283 sec latency 92.257 ms 2024-09-15T23:06:16.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:16 smithi192 ceph-mon[35427]: pgmap v961: 97 pgs: 97 active+clean; 529 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 4.5 MiB/s wr, 650 op/s 2024-09-15T23:06:16.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:16 smithi121 ceph-mon[34815]: pgmap v961: 97 pgs: 97 active+clean; 529 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 4.5 MiB/s wr, 650 op/s 2024-09-15T23:06:16.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:16 smithi111 ceph-mon[30629]: pgmap v961: 97 pgs: 97 active+clean; 529 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 4.5 MiB/s wr, 650 op/s 2024-09-15T23:06:16.673 INFO:tasks.workunit.client.0.smithi111.stdout: 10 137532 16.93 MB/sec execute 284 sec latency 75.036 ms 2024-09-15T23:06:17.673 INFO:tasks.workunit.client.0.smithi111.stdout: 10 138358 17.05 MB/sec execute 285 sec latency 107.208 ms 2024-09-15T23:06:18.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:18 smithi192 ceph-mon[35427]: pgmap v962: 97 pgs: 97 active+clean; 531 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.9 MiB/s wr, 529 op/s 2024-09-15T23:06:18.327 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:18 smithi111 ceph-mon[30629]: pgmap v962: 97 pgs: 97 active+clean; 531 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.9 MiB/s wr, 529 op/s 2024-09-15T23:06:18.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:18 smithi121 ceph-mon[34815]: pgmap v962: 97 pgs: 97 active+clean; 531 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.9 MiB/s wr, 529 op/s 2024-09-15T23:06:18.673 INFO:tasks.workunit.client.0.smithi111.stdout: 10 138908 17.08 MB/sec execute 286 sec latency 67.345 ms 2024-09-15T23:06:19.673 INFO:tasks.workunit.client.0.smithi111.stdout: 10 139631 17.15 MB/sec execute 287 sec latency 102.856 ms 2024-09-15T23:06:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.078+0000 7f65ebe00640 1 -- 172.21.15.111:0/1353501504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65ec103050 msgr2=0x7f65ec103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.078+0000 7f65ebe00640 1 --2- 172.21.15.111:0/1353501504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65ec103050 0x7f65ec103450 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f65dc009900 tx=0x7f65dc02a3f0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.078+0000 7f65ebe00640 1 -- 172.21.15.111:0/1353501504 shutdown_connections 2024-09-15T23:06:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.078+0000 7f65ebe00640 1 --2- 172.21.15.111:0/1353501504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65ec101f40 0x7f65ec109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.078+0000 7f65ebe00640 1 --2- 172.21.15.111:0/1353501504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65ec101580 0x7f65ec101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.078+0000 7f65ebe00640 1 --2- 172.21.15.111:0/1353501504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65ec103050 0x7f65ec103450 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.078+0000 7f65ebe00640 1 -- 172.21.15.111:0/1353501504 >> 172.21.15.111:0/1353501504 conn(0x7f65ec0fd0d0 msgr2=0x7f65ec0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:06:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.078+0000 7f65ebe00640 1 -- 172.21.15.111:0/1353501504 shutdown_connections 2024-09-15T23:06:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.078+0000 7f65ebe00640 1 -- 172.21.15.111:0/1353501504 wait complete. 2024-09-15T23:06:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.079+0000 7f65ebe00640 1 Processor -- start 2024-09-15T23:06:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.079+0000 7f65ebe00640 1 -- start start 2024-09-15T23:06:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.079+0000 7f65ebe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65ec101580 0x7f65ec075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.079+0000 7f65ebe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65ec101f40 0x7f65ec0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.079+0000 7f65ebe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65ec103050 0x7f65ec074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.079+0000 7f65ebe00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec10d6f0 con 0x7f65ec103050 2024-09-15T23:06:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.079+0000 7f65ebe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec10d570 con 0x7f65ec101f40 2024-09-15T23:06:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.079+0000 7f65ebe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec10d870 con 0x7f65ec101580 2024-09-15T23:06:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.080+0000 7f65ea000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65ec101f40 0x7f65ec0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.080+0000 7f65ea000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65ec101f40 0x7f65ec0745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:46104/0 (socket says 172.21.15.111:46104) 2024-09-15T23:06:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.080+0000 7f65ea000640 1 -- 172.21.15.111:0/951136709 learned_addr learned my addr 172.21.15.111:0/951136709 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:06:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.080+0000 7f65eb400640 1 --2- 172.21.15.111:0/951136709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65ec103050 0x7f65ec074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.080+0000 7f65eaa00640 1 --2- 172.21.15.111:0/951136709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65ec101580 0x7f65ec075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.080+0000 7f65ea000640 1 -- 172.21.15.111:0/951136709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65ec101580 msgr2=0x7f65ec075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.080+0000 7f65ea000640 1 --2- 172.21.15.111:0/951136709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65ec101580 0x7f65ec075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.080+0000 7f65ea000640 1 -- 172.21.15.111:0/951136709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65ec103050 msgr2=0x7f65ec074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.080+0000 7f65ea000640 1 --2- 172.21.15.111:0/951136709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65ec103050 0x7f65ec074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.080+0000 7f65ea000640 1 -- 172.21.15.111:0/951136709 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ec0753b0 con 0x7f65ec101f40 2024-09-15T23:06:20.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.080+0000 7f65ea000640 1 --2- 172.21.15.111:0/951136709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65ec101f40 0x7f65ec0745e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f65d800e9f0 tx=0x7f65d800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:06:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.081+0000 7f65f0a00640 1 -- 172.21.15.111:0/951136709 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d800cd80 con 0x7f65ec101f40 2024-09-15T23:06:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.081+0000 7f65ebe00640 1 -- 172.21.15.111:0/951136709 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65ec1a1f00 con 0x7f65ec101f40 2024-09-15T23:06:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.081+0000 7f65f0a00640 1 -- 172.21.15.111:0/951136709 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65d8005500 con 0x7f65ec101f40 2024-09-15T23:06:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.081+0000 7f65f0a00640 1 -- 172.21.15.111:0/951136709 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d8010690 con 0x7f65ec101f40 2024-09-15T23:06:20.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.081+0000 7f65ebe00640 1 -- 172.21.15.111:0/951136709 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65ec1a2310 con 0x7f65ec101f40 2024-09-15T23:06:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.084+0000 7f65f4dd4740 1 Processor -- start 2024-09-15T23:06:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.084+0000 7f65f4dd4740 1 -- start start 2024-09-15T23:06:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.084+0000 7f65f0a00640 1 -- 172.21.15.111:0/951136709 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f65d80046c0 con 0x7f65ec101f40 2024-09-15T23:06:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.084+0000 7f65f4dd4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56205af58260 0x56205af58660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.084+0000 7f65f4dd4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56205af58ba0 0x56205af5d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.084+0000 7f65f4dd4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56205af5dc30 0x56205af600f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.084+0000 7f65f4dd4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56205aebd100 con 0x56205af58ba0 2024-09-15T23:06:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.084+0000 7f65f4dd4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56205ae651b0 con 0x56205af58260 2024-09-15T23:06:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.084+0000 7f65f4dd4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56205aeb90c0 con 0x56205af5dc30 2024-09-15T23:06:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.085+0000 7f65eb400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56205af58260 0x56205af58660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.085+0000 7f65ea000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56205af5dc30 0x56205af600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.085+0000 7f65eb400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56205af58260 0x56205af58660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:46116/0 (socket says 172.21.15.111:46116) 2024-09-15T23:06:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.085+0000 7f65eb400640 1 -- 172.21.15.111:0/356578187 learned_addr learned my addr 172.21.15.111:0/356578187 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:06:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.085+0000 7f65eaa00640 1 --2- 172.21.15.111:0/356578187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56205af58ba0 0x56205af5d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.085+0000 7f65eb400640 1 -- 172.21.15.111:0/356578187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56205af5dc30 msgr2=0x56205af600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:06:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.085+0000 7f65eb400640 1 --2- 172.21.15.111:0/356578187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56205af5dc30 0x56205af600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.085+0000 7f65eb400640 1 -- 172.21.15.111:0/356578187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56205af58ba0 msgr2=0x56205af5d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:06:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.085+0000 7f65eb400640 1 --2- 172.21.15.111:0/356578187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56205af58ba0 0x56205af5d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.085+0000 7f65eb400640 1 -- 172.21.15.111:0/356578187 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56205abd0210 con 0x56205af58260 2024-09-15T23:06:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.085+0000 7f65ea000640 1 --2- 172.21.15.111:0/356578187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56205af5dc30 0x56205af600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:06:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.086+0000 7f65eb400640 1 --2- 172.21.15.111:0/356578187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56205af58260 0x56205af58660 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f65e4009fd0 tx=0x7f65e40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0a2d646c2bab5d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:06:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.086+0000 7f65d7e00640 1 -- 172.21.15.111:0/356578187 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4004d20 con 0x56205af58260 2024-09-15T23:06:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.086+0000 7f65d7e00640 1 -- 172.21.15.111:0/356578187 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65e4004050 con 0x56205af58260 2024-09-15T23:06:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.086+0000 7f65d7e00640 1 -- 172.21.15.111:0/356578187 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4005260 con 0x56205af58260 2024-09-15T23:06:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.086+0000 7f65f0a00640 1 --2- 172.21.15.111:0/951136709 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f65d00a3240 0x7f65d00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.086+0000 7f65f0a00640 1 -- 172.21.15.111:0/951136709 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f65d80cfce0 con 0x7f65ec101f40 2024-09-15T23:06:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.087+0000 7f65f4dd4740 1 -- 172.21.15.111:0/356578187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56205af58260 msgr2=0x56205af58660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.087+0000 7f65f4dd4740 1 --2- 172.21.15.111:0/356578187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56205af58260 0x56205af58660 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f65e4009fd0 tx=0x7f65e40118d0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.087+0000 7f65eaa00640 1 --2- 172.21.15.111:0/951136709 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f65d00a3240 0x7f65d00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:20.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.087+0000 7f65f4dd4740 1 -- 172.21.15.111:0/356578187 shutdown_connections 2024-09-15T23:06:20.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.087+0000 7f65f4dd4740 1 --2- 172.21.15.111:0/356578187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56205af5dc30 0x56205af600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.087+0000 7f65f4dd4740 1 --2- 172.21.15.111:0/356578187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56205af58ba0 0x56205af5d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.087+0000 7f65f4dd4740 1 --2- 172.21.15.111:0/356578187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56205af58260 0x56205af58660 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.087+0000 7f65f4dd4740 1 -- 172.21.15.111:0/356578187 >> 172.21.15.111:0/356578187 conn(0x56205af538d0 msgr2=0x56205af53cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:06:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.087+0000 7f65f4dd4740 1 -- 172.21.15.111:0/356578187 shutdown_connections 2024-09-15T23:06:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.087+0000 7f65f4dd4740 1 -- 172.21.15.111:0/356578187 wait complete. 2024-09-15T23:06:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.087+0000 7f65eaa00640 1 --2- 172.21.15.111:0/951136709 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f65d00a3240 0x7f65d00a5700 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f65dc005ab0 tx=0x7f65dc005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:06:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.088+0000 7f65f4dd4740 1 -- start start 2024-09-15T23:06:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.088+0000 7f65f4dd4740 1 Processor -- start 2024-09-15T23:06:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.088+0000 7f65f4dd4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56205af58260 0x56205af62410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.088+0000 7f65f4dd4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56205af58ba0 0x56205af62950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.088+0000 7f65f4dd4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56205af5dc30 0x56205af69920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:20.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.088+0000 7f65f4dd4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x56205af60c00 con 0x56205af5dc30 2024-09-15T23:06:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.088+0000 7f65f4dd4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56205ae46090 con 0x56205af58260 2024-09-15T23:06:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.088+0000 7f65f4dd4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56205af60d80 con 0x56205af58ba0 2024-09-15T23:06:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.088+0000 7f65f4dd4740 10 client.0 _ll_register_callbacks cb 0x56205ae9af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:06:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.088+0000 7f65eaa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56205af58260 0x56205af62410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.088+0000 7f65ea000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56205af58ba0 0x56205af62950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.089+0000 7f65eaa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56205af58260 0x56205af62410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:46130/0 (socket says 172.21.15.111:46130) 2024-09-15T23:06:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.089+0000 7f65eaa00640 1 -- 172.21.15.111:0/783069532 learned_addr learned my addr 172.21.15.111:0/783069532 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:06:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.089+0000 7f65ea000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56205af58ba0 0x56205af62950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:51514/0 (socket says 172.21.15.111:51514) 2024-09-15T23:06:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.089+0000 7f65eb400640 1 --2- 172.21.15.111:0/783069532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56205af5dc30 0x56205af69920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.089+0000 7f65ea000640 1 -- 172.21.15.111:0/783069532 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56205af58260 msgr2=0x56205af62410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:06:20.094 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.089+0000 7f65ea000640 1 --2- 172.21.15.111:0/783069532 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56205af58260 0x56205af62410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.089+0000 7f65ea000640 1 -- 172.21.15.111:0/783069532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56205af5dc30 msgr2=0x56205af69920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:06:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.089+0000 7f65ea000640 1 --2- 172.21.15.111:0/783069532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56205af5dc30 0x56205af69920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.089+0000 7f65ea000640 1 -- 172.21.15.111:0/783069532 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56205ac9c620 con 0x56205af58ba0 2024-09-15T23:06:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.089+0000 7f65eaa00640 1 --2- 172.21.15.111:0/783069532 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56205af58260 0x56205af62410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:06:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.089+0000 7f65ea000640 1 --2- 172.21.15.111:0/783069532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56205af58ba0 0x56205af62950 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f65d800c6a0 tx=0x7f65d8004b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bf25fe9fb25cc24 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:06:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.089+0000 7f65d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:06:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.089+0000 7f65d4c00640 1 -- 172.21.15.111:0/783069532 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d809bc60 con 0x56205af58ba0 2024-09-15T23:06:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.089+0000 7f65d4c00640 1 -- 172.21.15.111:0/783069532 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f65d80a4ae0 con 0x56205af58ba0 2024-09-15T23:06:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.089+0000 7f65f4dd4740 1 -- 172.21.15.111:0/783069532 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65e4004050 con 0x56205af58ba0 2024-09-15T23:06:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.090+0000 7f65d4c00640 1 -- 172.21.15.111:0/783069532 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f65d80b5050 con 0x56205af58ba0 2024-09-15T23:06:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.090+0000 7f65f4dd4740 10 client.26551 fetch_fsmap learned FSMap version 35 2024-09-15T23:06:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.090+0000 7f65f4dd4740 1 -- 172.21.15.111:0/783069532 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56205af6c7d0 con 0x56205af58ba0 2024-09-15T23:06:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.090+0000 7f65d4c00640 1 -- 172.21.15.111:0/783069532 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65e4004050 con 0x56205af58ba0 2024-09-15T23:06:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.090+0000 7f65d4c00640 1 -- 172.21.15.111:0/783069532 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f65d809b750 con 0x56205af58ba0 2024-09-15T23:06:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.090+0000 7f65f4dd4740 10 client.26551 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:06:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.090+0000 7f65f4dd4740 10 client.26551 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:06:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.090+0000 7f65f4dd4740 20 client.26551 populate_metadata read hostname 'smithi111' 2024-09-15T23:06:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.090+0000 7f65f4dd4740 1 --2- 172.21.15.111:0/783069532 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56205af6dde0 0x56205af701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.090+0000 7f65f4dd4740 4 client.26551 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:06:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.090+0000 7f65f4dd4740 1 -- 172.21.15.111:0/783069532 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56205aeca830 con 0x56205af6dde0 2024-09-15T23:06:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.091+0000 7f65eaa00640 1 --2- 172.21.15.111:0/783069532 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56205af6dde0 0x56205af701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.091+0000 7f65eaa00640 1 --2- 172.21.15.111:0/783069532 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56205af6dde0 0x56205af701d0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4233c89f22a2a577 server_cookie=a5010aa73ac33fb7 in_seq=0 out_seq=0 2024-09-15T23:06:20.096 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.091+0000 7f65d4c00640 10 client.26551 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:06:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.092+0000 7f65d4c00640 1 -- 172.21.15.111:0/783069532 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56205aeca830 con 0x56205af6dde0 2024-09-15T23:06:20.097 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.092+0000 7f65d4c00640 10 client.26551 handle_command_reply: tid=0 2024-09-15T23:06:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.092+0000 7f65f4dd4740 1 client.26551 shutdown 2024-09-15T23:06:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.092+0000 7f65f4dd4740 1 -- 172.21.15.111:0/783069532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56205af58ba0 msgr2=0x56205af62950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.092+0000 7f65f4dd4740 1 --2- 172.21.15.111:0/783069532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56205af58ba0 0x56205af62950 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f65d800c6a0 tx=0x7f65d8004b60 comp rx=0 tx=0).stop 2024-09-15T23:06:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.092+0000 7f65f4dd4740 1 -- 172.21.15.111:0/783069532 shutdown_connections 2024-09-15T23:06:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.092+0000 7f65f4dd4740 1 --2- 172.21.15.111:0/783069532 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56205af6dde0 0x56205af701d0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.092+0000 7f65f4dd4740 1 --2- 172.21.15.111:0/783069532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56205af5dc30 0x56205af69920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.092+0000 7f65f4dd4740 1 --2- 172.21.15.111:0/783069532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56205af58ba0 0x56205af62950 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.092+0000 7f65f4dd4740 1 --2- 172.21.15.111:0/783069532 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56205af58260 0x56205af62410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.092+0000 7f65d4c00640 0 client.26551 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:06:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.092+0000 7f65f4dd4740 1 -- 172.21.15.111:0/783069532 >> 172.21.15.111:0/783069532 conn(0x56205af538d0 msgr2=0x56205af550e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:06:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.092+0000 7f65f4dd4740 1 -- 172.21.15.111:0/783069532 shutdown_connections 2024-09-15T23:06:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.092+0000 7f65f4dd4740 1 -- 172.21.15.111:0/783069532 wait complete. 2024-09-15T23:06:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.092+0000 7f65f4dd4740 20 client.26551 trim_cache size 0 max 16384 2024-09-15T23:06:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.111+0000 7f65f4dd4740 1 Processor -- start 2024-09-15T23:06:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.111+0000 7f65f4dd4740 1 -- start start 2024-09-15T23:06:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.111+0000 7f65f4dd4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56205af538d0 0x56205af521f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.111+0000 7f65f4dd4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56205af58260 0x56205af698e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.111+0000 7f65f4dd4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56205af58ba0 0x56205af52730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.111+0000 7f65f4dd4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec1ad660 con 0x56205af58260 2024-09-15T23:06:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.111+0000 7f65f4dd4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec1aaf30 con 0x56205af538d0 2024-09-15T23:06:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.111+0000 7f65f4dd4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec1a8da0 con 0x56205af58ba0 2024-09-15T23:06:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.111+0000 7f65ea000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56205af58260 0x56205af698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.111+0000 7f65eaa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56205af538d0 0x56205af521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.111+0000 7f65eaa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56205af538d0 0x56205af521f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:46144/0 (socket says 172.21.15.111:46144) 2024-09-15T23:06:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.111+0000 7f65eaa00640 1 -- 172.21.15.111:0/2684241328 learned_addr learned my addr 172.21.15.111:0/2684241328 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:06:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.112+0000 7f65eb400640 1 --2- 172.21.15.111:0/2684241328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56205af58ba0 0x56205af52730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.112+0000 7f65ea000640 1 -- 172.21.15.111:0/2684241328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56205af58ba0 msgr2=0x56205af52730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:06:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.112+0000 7f65ea000640 1 --2- 172.21.15.111:0/2684241328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56205af58ba0 0x56205af52730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.112+0000 7f65ea000640 1 -- 172.21.15.111:0/2684241328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56205af538d0 msgr2=0x56205af521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:06:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.112+0000 7f65ea000640 1 --2- 172.21.15.111:0/2684241328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56205af538d0 0x56205af521f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.112+0000 7f65ea000640 1 -- 172.21.15.111:0/2684241328 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56205af70830 con 0x56205af58260 2024-09-15T23:06:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.112+0000 7f65eaa00640 1 --2- 172.21.15.111:0/2684241328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56205af538d0 0x56205af521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:06:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.112+0000 7f65ea000640 1 --2- 172.21.15.111:0/2684241328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56205af58260 0x56205af698e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f65d800e9c0 tx=0x7f65d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b32fe86f4fd0593 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:06:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.113+0000 7f65d7400640 1 -- 172.21.15.111:0/2684241328 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d80d2070 con 0x56205af58260 2024-09-15T23:06:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.113+0000 7f65d7400640 1 -- 172.21.15.111:0/2684241328 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65d809ca00 con 0x56205af58260 2024-09-15T23:06:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.113+0000 7f65d7400640 1 -- 172.21.15.111:0/2684241328 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d809c300 con 0x56205af58260 2024-09-15T23:06:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.114+0000 7f65f4dd4740 1 -- 172.21.15.111:0/2684241328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56205af58260 msgr2=0x56205af698e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.114+0000 7f65f4dd4740 1 --2- 172.21.15.111:0/2684241328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56205af58260 0x56205af698e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f65d800e9c0 tx=0x7f65d8004a40 comp rx=0 tx=0).stop 2024-09-15T23:06:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.114+0000 7f65f4dd4740 1 -- 172.21.15.111:0/2684241328 shutdown_connections 2024-09-15T23:06:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.114+0000 7f65f4dd4740 1 --2- 172.21.15.111:0/2684241328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56205af58ba0 0x56205af52730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.114+0000 7f65f4dd4740 1 --2- 172.21.15.111:0/2684241328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56205af58260 0x56205af698e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.114+0000 7f65f4dd4740 1 --2- 172.21.15.111:0/2684241328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56205af538d0 0x56205af521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.114+0000 7f65f4dd4740 1 -- 172.21.15.111:0/2684241328 >> 172.21.15.111:0/2684241328 conn(0x56205af5d780 msgr2=0x56205af62410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:06:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.114+0000 7f65f4dd4740 1 -- 172.21.15.111:0/2684241328 shutdown_connections 2024-09-15T23:06:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.114+0000 7f65f4dd4740 1 -- 172.21.15.111:0/2684241328 wait complete. 2024-09-15T23:06:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.114+0000 7f65f4dd4740 1 -- start start 2024-09-15T23:06:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.115+0000 7f65f4dd4740 1 Processor -- start 2024-09-15T23:06:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.115+0000 7f65f4dd4740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56205af538d0 0x56205af66cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.115+0000 7f65f4dd4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56205af58260 0x56205af67210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.115+0000 7f65f4dd4740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56205af58ba0 0x56205af68220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.115+0000 7f65f4dd4740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec1a93f0 con 0x56205af538d0 2024-09-15T23:06:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.115+0000 7f65f4dd4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec1aa860 con 0x56205af58260 2024-09-15T23:06:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.115+0000 7f65f4dd4740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec1a9ac0 con 0x56205af58ba0 2024-09-15T23:06:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.115+0000 7f65f4dd4740 10 client.0 _ll_register_callbacks cb 0x7f65d80d3410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:06:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.115+0000 7f65eb400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56205af58ba0 0x56205af68220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.115+0000 7f65eaa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56205af538d0 0x56205af66cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.115+0000 7f65eb400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56205af58ba0 0x56205af68220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:51530/0 (socket says 172.21.15.111:51530) 2024-09-15T23:06:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.115+0000 7f65eb400640 1 -- 172.21.15.111:0/1578536729 learned_addr learned my addr 172.21.15.111:0/1578536729 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:06:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.115+0000 7f65ea000640 1 --2- 172.21.15.111:0/1578536729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56205af58260 0x56205af67210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.116+0000 7f65eb400640 1 -- 172.21.15.111:0/1578536729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56205af58260 msgr2=0x56205af67210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:06:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.116+0000 7f65eb400640 1 --2- 172.21.15.111:0/1578536729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56205af58260 0x56205af67210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.116+0000 7f65eb400640 1 -- 172.21.15.111:0/1578536729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56205af538d0 msgr2=0x56205af66cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:06:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.116+0000 7f65eb400640 1 --2- 172.21.15.111:0/1578536729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56205af538d0 0x56205af66cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.116+0000 7f65eb400640 1 -- 172.21.15.111:0/1578536729 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56205af68830 con 0x56205af58ba0 2024-09-15T23:06:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.116+0000 7f65eb400640 1 --2- 172.21.15.111:0/1578536729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56205af58ba0 0x56205af68220 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f65e4009de0 tx=0x7f65e4005ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=340e9d4d06b897f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:06:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.116+0000 7f65d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:06:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.117+0000 7f65d4c00640 1 -- 172.21.15.111:0/1578536729 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4004c80 con 0x56205af58ba0 2024-09-15T23:06:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.117+0000 7f65f4dd4740 1 -- 172.21.15.111:0/1578536729 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65d809ca00 con 0x56205af58ba0 2024-09-15T23:06:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.117+0000 7f65d4c00640 1 -- 172.21.15.111:0/1578536729 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f65e4004050 con 0x56205af58ba0 2024-09-15T23:06:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.117+0000 7f65d4c00640 1 -- 172.21.15.111:0/1578536729 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f65e4004e20 con 0x56205af58ba0 2024-09-15T23:06:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.117+0000 7f65f4dd4740 10 client.26557 fetch_fsmap learned FSMap version 35 2024-09-15T23:06:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.117+0000 7f65f4dd4740 1 -- 172.21.15.111:0/1578536729 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56205af69b90 con 0x56205af58ba0 2024-09-15T23:06:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.117+0000 7f65d4c00640 1 -- 172.21.15.111:0/1578536729 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65d809ca00 con 0x56205af58ba0 2024-09-15T23:06:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.117+0000 7f65d4c00640 1 -- 172.21.15.111:0/1578536729 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f65e40057f0 con 0x56205af58ba0 2024-09-15T23:06:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.117+0000 7f65f4dd4740 10 client.26557 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:06:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.117+0000 7f65f4dd4740 10 client.26557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:06:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.117+0000 7f65f4dd4740 20 client.26557 populate_metadata read hostname 'smithi111' 2024-09-15T23:06:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.117+0000 7f65f4dd4740 1 --2- 172.21.15.111:0/1578536729 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56205af5dc30 0x56205afe3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.117+0000 7f65f4dd4740 4 client.26557 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:06:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.117+0000 7f65f4dd4740 1 -- 172.21.15.111:0/1578536729 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56205aec4b80 con 0x56205af5dc30 2024-09-15T23:06:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.118+0000 7f65eaa00640 1 --2- 172.21.15.111:0/1578536729 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56205af5dc30 0x56205afe3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.118+0000 7f65eaa00640 1 --2- 172.21.15.111:0/1578536729 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56205af5dc30 0x56205afe3870 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76183a3508cbb3b3 server_cookie=df77d422ea34b0e in_seq=0 out_seq=0 2024-09-15T23:06:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.118+0000 7f65d4c00640 10 client.26557 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:06:20.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:20 smithi192 ceph-mon[35427]: pgmap v963: 97 pgs: 97 active+clean; 543 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.6 MiB/s wr, 611 op/s 2024-09-15T23:06:20.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.355+0000 7f65d4c00640 1 -- 172.21.15.111:0/1578536729 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56205aec4b80 con 0x56205af5dc30 2024-09-15T23:06:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.355+0000 7f65d4c00640 10 client.26557 handle_command_reply: tid=0 2024-09-15T23:06:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.355+0000 7f65f4dd4740 1 client.26557 shutdown 2024-09-15T23:06:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.355+0000 7f65f4dd4740 1 -- 172.21.15.111:0/1578536729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56205af58ba0 msgr2=0x56205af68220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.355+0000 7f65f4dd4740 1 --2- 172.21.15.111:0/1578536729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56205af58ba0 0x56205af68220 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f65e4009de0 tx=0x7f65e4005ea0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.356+0000 7f65f4dd4740 1 -- 172.21.15.111:0/1578536729 shutdown_connections 2024-09-15T23:06:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.356+0000 7f65f4dd4740 1 --2- 172.21.15.111:0/1578536729 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56205af5dc30 0x56205afe3870 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.356+0000 7f65f4dd4740 1 --2- 172.21.15.111:0/1578536729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56205af58ba0 0x56205af68220 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.356+0000 7f65f4dd4740 1 --2- 172.21.15.111:0/1578536729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56205af58260 0x56205af67210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.356+0000 7f65f4dd4740 1 --2- 172.21.15.111:0/1578536729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56205af538d0 0x56205af66cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.356+0000 7f65f4dd4740 1 -- 172.21.15.111:0/1578536729 >> 172.21.15.111:0/1578536729 conn(0x56205af5d780 msgr2=0x56205af547e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:06:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.356+0000 7f65d4c00640 0 client.26557 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:06:20.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.356+0000 7f65f4dd4740 1 -- 172.21.15.111:0/1578536729 shutdown_connections 2024-09-15T23:06:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.356+0000 7f65f4dd4740 1 -- 172.21.15.111:0/1578536729 wait complete. 2024-09-15T23:06:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.356+0000 7f65f4dd4740 20 client.26557 trim_cache size 0 max 16384 2024-09-15T23:06:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:06:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:06:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:06:20.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:06:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.357+0000 7f65ebe00640 1 -- 172.21.15.111:0/951136709 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f65d00a3240 msgr2=0x7f65d00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.357+0000 7f65ebe00640 1 --2- 172.21.15.111:0/951136709 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f65d00a3240 0x7f65d00a5700 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f65dc005ab0 tx=0x7f65dc005fd0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.357+0000 7f65ebe00640 1 -- 172.21.15.111:0/951136709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65ec101f40 msgr2=0x7f65ec0745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.357+0000 7f65ebe00640 1 --2- 172.21.15.111:0/951136709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65ec101f40 0x7f65ec0745e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f65d800e9f0 tx=0x7f65d800edf0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.357+0000 7f65ebe00640 1 -- 172.21.15.111:0/951136709 shutdown_connections 2024-09-15T23:06:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.357+0000 7f65ebe00640 1 --2- 172.21.15.111:0/951136709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f65ec103050 0x7f65ec074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.360 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.358+0000 7f65ebe00640 1 --2- 172.21.15.111:0/951136709 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f65d00a3240 0x7f65d00a5700 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.358+0000 7f65ebe00640 1 --2- 172.21.15.111:0/951136709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f65ec101f40 0x7f65ec0745e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.358+0000 7f65ebe00640 1 --2- 172.21.15.111:0/951136709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f65ec101580 0x7f65ec075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.358+0000 7f65ebe00640 1 -- 172.21.15.111:0/951136709 >> 172.21.15.111:0/951136709 conn(0x7f65ec0fd0d0 msgr2=0x7f65ec106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:06:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.358+0000 7f65ebe00640 1 -- 172.21.15.111:0/951136709 shutdown_connections 2024-09-15T23:06:20.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:20.358+0000 7f65ebe00640 1 -- 172.21.15.111:0/951136709 wait complete. 2024-09-15T23:06:20.389 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:20 smithi121 ceph-mon[34815]: pgmap v963: 97 pgs: 97 active+clean; 543 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.6 MiB/s wr, 611 op/s 2024-09-15T23:06:20.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:20 smithi111 ceph-mon[30629]: pgmap v963: 97 pgs: 97 active+clean; 543 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.6 MiB/s wr, 611 op/s 2024-09-15T23:06:20.673 INFO:tasks.workunit.client.0.smithi111.stdout: 10 140199 17.21 MB/sec execute 288 sec latency 165.778 ms 2024-09-15T23:06:21.673 INFO:tasks.workunit.client.0.smithi111.stdout: 10 140973 17.27 MB/sec execute 289 sec latency 49.558 ms 2024-09-15T23:06:22.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:22 smithi192 ceph-mon[35427]: pgmap v964: 97 pgs: 97 active+clean; 550 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.1 MiB/s wr, 540 op/s 2024-09-15T23:06:22.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:22 smithi121 ceph-mon[34815]: pgmap v964: 97 pgs: 97 active+clean; 550 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.1 MiB/s wr, 540 op/s 2024-09-15T23:06:22.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:22 smithi111 ceph-mon[30629]: pgmap v964: 97 pgs: 97 active+clean; 550 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.1 MiB/s wr, 540 op/s 2024-09-15T23:06:22.673 INFO:tasks.workunit.client.0.smithi111.stdout: 10 141844 17.39 MB/sec execute 290 sec latency 63.963 ms 2024-09-15T23:06:23.673 INFO:tasks.workunit.client.0.smithi111.stdout: 10 142497 17.47 MB/sec execute 291 sec latency 113.026 ms 2024-09-15T23:06:24.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:24 smithi111 ceph-mon[30629]: pgmap v965: 97 pgs: 97 active+clean; 553 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.8 MiB/s wr, 497 op/s 2024-09-15T23:06:24.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:24 smithi192 ceph-mon[35427]: pgmap v965: 97 pgs: 97 active+clean; 553 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.8 MiB/s wr, 497 op/s 2024-09-15T23:06:24.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:24 smithi121 ceph-mon[34815]: pgmap v965: 97 pgs: 97 active+clean; 553 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.8 MiB/s wr, 497 op/s 2024-09-15T23:06:24.673 INFO:tasks.workunit.client.0.smithi111.stdout: 10 143073 17.48 MB/sec execute 292 sec latency 72.336 ms 2024-09-15T23:06:25.674 INFO:tasks.workunit.client.0.smithi111.stdout: 10 143764 17.57 MB/sec execute 293 sec latency 107.725 ms 2024-09-15T23:06:26.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:26 smithi192 ceph-mon[35427]: pgmap v966: 97 pgs: 97 active+clean; 554 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.5 MiB/s wr, 636 op/s 2024-09-15T23:06:26.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:26 smithi121 ceph-mon[34815]: pgmap v966: 97 pgs: 97 active+clean; 554 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.5 MiB/s wr, 636 op/s 2024-09-15T23:06:26.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:26 smithi111 ceph-mon[30629]: pgmap v966: 97 pgs: 97 active+clean; 554 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.5 MiB/s wr, 636 op/s 2024-09-15T23:06:26.674 INFO:tasks.workunit.client.0.smithi111.stdout: 10 144518 17.64 MB/sec execute 294 sec latency 155.491 ms 2024-09-15T23:06:27.674 INFO:tasks.workunit.client.0.smithi111.stdout: 10 145262 17.73 MB/sec execute 295 sec latency 50.662 ms 2024-09-15T23:06:28.294 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:28 smithi111 ceph-mon[30629]: pgmap v967: 97 pgs: 97 active+clean; 557 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.9 MiB/s wr, 558 op/s 2024-09-15T23:06:28.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:28 smithi192 ceph-mon[35427]: pgmap v967: 97 pgs: 97 active+clean; 557 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.9 MiB/s wr, 558 op/s 2024-09-15T23:06:28.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:28 smithi121 ceph-mon[34815]: pgmap v967: 97 pgs: 97 active+clean; 557 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.9 MiB/s wr, 558 op/s 2024-09-15T23:06:28.674 INFO:tasks.workunit.client.0.smithi111.stdout: 10 145940 17.77 MB/sec execute 296 sec latency 66.203 ms 2024-09-15T23:06:29.674 INFO:tasks.workunit.client.0.smithi111.stdout: 10 146656 17.85 MB/sec execute 297 sec latency 89.273 ms 2024-09-15T23:06:30.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:30 smithi192 ceph-mon[35427]: pgmap v968: 97 pgs: 97 active+clean; 563 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.3 MiB/s wr, 601 op/s 2024-09-15T23:06:30.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:30 smithi121 ceph-mon[34815]: pgmap v968: 97 pgs: 97 active+clean; 563 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.3 MiB/s wr, 601 op/s 2024-09-15T23:06:30.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:30 smithi111 ceph-mon[30629]: pgmap v968: 97 pgs: 97 active+clean; 563 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.3 MiB/s wr, 601 op/s 2024-09-15T23:06:30.674 INFO:tasks.workunit.client.0.smithi111.stdout: 10 147557 17.95 MB/sec execute 298 sec latency 51.831 ms 2024-09-15T23:06:31.674 INFO:tasks.workunit.client.0.smithi111.stdout: 10 147788 17.95 MB/sec execute 299 sec latency 678.780 ms 2024-09-15T23:06:32.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:32 smithi192 ceph-mon[35427]: pgmap v969: 97 pgs: 97 active+clean; 565 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.2 MiB/s wr, 592 op/s 2024-09-15T23:06:32.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:32 smithi121 ceph-mon[34815]: pgmap v969: 97 pgs: 97 active+clean; 565 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.2 MiB/s wr, 592 op/s 2024-09-15T23:06:32.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:32 smithi111 ceph-mon[30629]: pgmap v969: 97 pgs: 97 active+clean; 565 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.2 MiB/s wr, 592 op/s 2024-09-15T23:06:32.674 INFO:tasks.workunit.client.0.smithi111.stdout: 10 147789 17.89 MB/sec execute 300 sec latency 1678.883 ms 2024-09-15T23:06:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.493+0000 7fe1d2a00640 1 -- 172.21.15.111:0/2012017103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1cc076bb0 msgr2=0x7fe1cc076fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.493+0000 7fe1d2a00640 1 --2- 172.21.15.111:0/2012017103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1cc076bb0 0x7fe1cc076fb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4009900 tx=0x7fe1b402a440 comp rx=0 tx=0).stop 2024-09-15T23:06:33.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.493+0000 7fe1d2a00640 1 -- 172.21.15.111:0/2012017103 shutdown_connections 2024-09-15T23:06:33.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.493+0000 7fe1d2a00640 1 --2- 172.21.15.111:0/2012017103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1cc078790 0x7fe1cc079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.493+0000 7fe1d2a00640 1 --2- 172.21.15.111:0/2012017103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1cc077df0 0x7fe1cc078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.493+0000 7fe1d2a00640 1 --2- 172.21.15.111:0/2012017103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1cc076bb0 0x7fe1cc076fb0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.493+0000 7fe1d2a00640 1 -- 172.21.15.111:0/2012017103 >> 172.21.15.111:0/2012017103 conn(0x7fe1cc0ff4b0 msgr2=0x7fe1cc1018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:06:33.496 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.493+0000 7fe1d2a00640 1 -- 172.21.15.111:0/2012017103 shutdown_connections 2024-09-15T23:06:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.493+0000 7fe1d2a00640 1 -- 172.21.15.111:0/2012017103 wait complete. 2024-09-15T23:06:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.493+0000 7fe1d2a00640 1 Processor -- start 2024-09-15T23:06:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.496+0000 7fe1d2a00640 1 -- start start 2024-09-15T23:06:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.496+0000 7fe1d2a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1cc076bb0 0x7fe1cc19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.496+0000 7fe1d2a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1cc077df0 0x7fe1cc19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.496+0000 7fe1d2a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1cc078790 0x7fe1cc1a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.496+0000 7fe1d2a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc113f30 con 0x7fe1cc078790 2024-09-15T23:06:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.496+0000 7fe1d2a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc113db0 con 0x7fe1cc077df0 2024-09-15T23:06:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.496+0000 7fe1d2a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc1140b0 con 0x7fe1cc076bb0 2024-09-15T23:06:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.496+0000 7fe1cbe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1cc076bb0 0x7fe1cc19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.496+0000 7fe1cbe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1cc076bb0 0x7fe1cc19e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:52450/0 (socket says 172.21.15.111:52450) 2024-09-15T23:06:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.496+0000 7fe1cbe00640 1 -- 172.21.15.111:0/2281859884 learned_addr learned my addr 172.21.15.111:0/2281859884 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:06:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.496+0000 7fe1d0a00640 1 --2- 172.21.15.111:0/2281859884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1cc078790 0x7fe1cc1a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.496+0000 7fe1cbe00640 1 -- 172.21.15.111:0/2281859884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1cc077df0 msgr2=0x7fe1cc19eac0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T23:06:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.496+0000 7fe1cbe00640 1 --2- 172.21.15.111:0/2281859884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1cc077df0 0x7fe1cc19eac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.496+0000 7fe1cbe00640 1 -- 172.21.15.111:0/2281859884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1cc078790 msgr2=0x7fe1cc1a5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.496+0000 7fe1cbe00640 1 --2- 172.21.15.111:0/2281859884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1cc078790 0x7fe1cc1a5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.496+0000 7fe1cbe00640 1 -- 172.21.15.111:0/2281859884 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1cc1a61a0 con 0x7fe1cc076bb0 2024-09-15T23:06:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.496+0000 7fe1d0a00640 1 --2- 172.21.15.111:0/2281859884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1cc078790 0x7fe1cc1a5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:06:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.496+0000 7fe1cbe00640 1 --2- 172.21.15.111:0/2281859884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1cc076bb0 0x7fe1cc19e580 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4031440 tx=0x7fe1b4032f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:06:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.496+0000 7fe1c8c00640 1 -- 172.21.15.111:0/2281859884 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b403f030 con 0x7fe1cc076bb0 2024-09-15T23:06:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.496+0000 7fe1c8c00640 1 -- 172.21.15.111:0/2281859884 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1b4003d60 con 0x7fe1cc076bb0 2024-09-15T23:06:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.496+0000 7fe1d2a00640 1 -- 172.21.15.111:0/2281859884 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1cc1a6430 con 0x7fe1cc076bb0 2024-09-15T23:06:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.496+0000 7fe1d2a00640 1 -- 172.21.15.111:0/2281859884 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1cc1a69f0 con 0x7fe1cc076bb0 2024-09-15T23:06:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.496+0000 7fe1c8c00640 1 -- 172.21.15.111:0/2281859884 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4035070 con 0x7fe1cc076bb0 2024-09-15T23:06:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.500+0000 7fe1c8c00640 1 -- 172.21.15.111:0/2281859884 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe1b4042030 con 0x7fe1cc076bb0 2024-09-15T23:06:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.500+0000 7fe1d58d0740 1 Processor -- start 2024-09-15T23:06:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.506+0000 7fe1d58d0740 1 -- start start 2024-09-15T23:06:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.506+0000 7fe1d58d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562ffed22090 0x562ffed22490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.506+0000 7fe1d58d0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ffed229d0 0x562ffed27490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.506+0000 7fe1d58d0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ffed27a60 0x562ffed29f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.506+0000 7fe1d58d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562ffec87100 con 0x562ffed22090 2024-09-15T23:06:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.506+0000 7fe1d58d0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562ffec2f1b0 con 0x562ffed27a60 2024-09-15T23:06:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.506+0000 7fe1d58d0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562ffec830c0 con 0x562ffed229d0 2024-09-15T23:06:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.506+0000 7fe1c8c00640 1 --2- 172.21.15.111:0/2281859884 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fe1b00a33c0 0x7fe1b00a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.506+0000 7fe1c8c00640 1 -- 172.21.15.111:0/2281859884 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fe1b40f0170 con 0x7fe1cc076bb0 2024-09-15T23:06:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.506+0000 7fe1cbe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ffed229d0 0x562ffed27490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.506+0000 7fe1cbe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ffed229d0 0x562ffed27490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:52456/0 (socket says 172.21.15.111:52456) 2024-09-15T23:06:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.506+0000 7fe1cbe00640 1 -- 172.21.15.111:0/4164482880 learned_addr learned my addr 172.21.15.111:0/4164482880 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:06:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.506+0000 7fe1cbe00640 1 -- 172.21.15.111:0/4164482880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ffed27a60 msgr2=0x562ffed29f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:06:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.506+0000 7fe1cbe00640 1 --2- 172.21.15.111:0/4164482880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ffed27a60 0x562ffed29f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.506+0000 7fe1cbe00640 1 -- 172.21.15.111:0/4164482880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562ffed22090 msgr2=0x562ffed22490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:06:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.506+0000 7fe1cbe00640 1 --2- 172.21.15.111:0/4164482880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562ffed22090 0x562ffed22490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.506+0000 7fe1cbe00640 1 -- 172.21.15.111:0/4164482880 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ffe99a210 con 0x562ffed229d0 2024-09-15T23:06:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.506+0000 7fe1d0a00640 1 --2- 172.21.15.111:0/2281859884 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fe1b00a33c0 0x7fe1b00a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.506+0000 7fe1cbe00640 1 --2- 172.21.15.111:0/4164482880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ffed229d0 0x562ffed27490 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40c4f70 tx=0x7fe1b4009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4420d85b003667c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:06:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.506+0000 7fe1c1200640 1 -- 172.21.15.111:0/4164482880 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4049070 con 0x562ffed229d0 2024-09-15T23:06:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.506+0000 7fe1c1200640 1 -- 172.21.15.111:0/4164482880 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1b40c54a0 con 0x562ffed229d0 2024-09-15T23:06:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.506+0000 7fe1c1200640 1 -- 172.21.15.111:0/4164482880 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b403b710 con 0x562ffed229d0 2024-09-15T23:06:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.511+0000 7fe1d58d0740 1 -- 172.21.15.111:0/4164482880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ffed229d0 msgr2=0x562ffed27490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.511+0000 7fe1d58d0740 1 --2- 172.21.15.111:0/4164482880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ffed229d0 0x562ffed27490 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40c4f70 tx=0x7fe1b4009680 comp rx=0 tx=0).stop 2024-09-15T23:06:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.511+0000 7fe1d0a00640 1 --2- 172.21.15.111:0/2281859884 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fe1b00a33c0 0x7fe1b00a5880 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4009440 tx=0x7fe1c4009650 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:06:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.513+0000 7fe1d58d0740 1 -- 172.21.15.111:0/4164482880 shutdown_connections 2024-09-15T23:06:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.513+0000 7fe1d58d0740 1 --2- 172.21.15.111:0/4164482880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ffed27a60 0x562ffed29f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.513+0000 7fe1d58d0740 1 --2- 172.21.15.111:0/4164482880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ffed229d0 0x562ffed27490 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.513+0000 7fe1d58d0740 1 --2- 172.21.15.111:0/4164482880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562ffed22090 0x562ffed22490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.513+0000 7fe1d58d0740 1 -- 172.21.15.111:0/4164482880 >> 172.21.15.111:0/4164482880 conn(0x562ffed1d700 msgr2=0x562ffed1db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:06:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.513+0000 7fe1d58d0740 1 -- 172.21.15.111:0/4164482880 shutdown_connections 2024-09-15T23:06:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.513+0000 7fe1d58d0740 1 -- 172.21.15.111:0/4164482880 wait complete. 2024-09-15T23:06:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.513+0000 7fe1d58d0740 1 -- start start 2024-09-15T23:06:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.513+0000 7fe1d58d0740 1 Processor -- start 2024-09-15T23:06:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.513+0000 7fe1d58d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562ffed22090 0x562ffed2c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.513+0000 7fe1d58d0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ffed229d0 0x562ffed2c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.513+0000 7fe1d58d0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ffed27a60 0x562ffed335d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.513+0000 7fe1d58d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x562ffed2a7f0 con 0x562ffed22090 2024-09-15T23:06:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.513+0000 7fe1d58d0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562ffec10090 con 0x562ffed27a60 2024-09-15T23:06:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.513+0000 7fe1d58d0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562ffed2a970 con 0x562ffed229d0 2024-09-15T23:06:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.513+0000 7fe1d58d0740 10 client.0 _ll_register_callbacks cb 0x562ffec64f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:06:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.513+0000 7fe1cbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562ffed22090 0x562ffed2c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.513+0000 7fe1cbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562ffed22090 0x562ffed2c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36046/0 (socket says 172.21.15.111:36046) 2024-09-15T23:06:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.513+0000 7fe1cbe00640 1 -- 172.21.15.111:0/2686149921 learned_addr learned my addr 172.21.15.111:0/2686149921 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:06:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.518+0000 7fe1cbe00640 1 -- 172.21.15.111:0/2686149921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ffed229d0 msgr2=0x562ffed2c590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:06:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.518+0000 7fe1cbe00640 1 --2- 172.21.15.111:0/2686149921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ffed229d0 0x562ffed2c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.518+0000 7fe1cbe00640 1 -- 172.21.15.111:0/2686149921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ffed27a60 msgr2=0x562ffed335d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:06:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.518+0000 7fe1cbe00640 1 --2- 172.21.15.111:0/2686149921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ffed27a60 0x562ffed335d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.518+0000 7fe1cbe00640 1 -- 172.21.15.111:0/2686149921 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ffea66620 con 0x562ffed22090 2024-09-15T23:06:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.518+0000 7fe1cbe00640 1 --2- 172.21.15.111:0/2686149921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562ffed22090 0x562ffed2c050 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe1b402aa30 tx=0x7fe1b4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f010ff5417b07fc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:06:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.518+0000 7fe1b9600640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:06:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.518+0000 7fe1b9600640 1 -- 172.21.15.111:0/2686149921 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4049050 con 0x562ffed22090 2024-09-15T23:06:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.518+0000 7fe1b9600640 1 -- 172.21.15.111:0/2686149921 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fe1b40b8f30 con 0x562ffed22090 2024-09-15T23:06:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.518+0000 7fe1d58d0740 1 -- 172.21.15.111:0/2686149921 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1b40c54a0 con 0x562ffed22090 2024-09-15T23:06:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.518+0000 7fe1b9600640 1 -- 172.21.15.111:0/2686149921 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe1b40db1a0 con 0x562ffed22090 2024-09-15T23:06:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.518+0000 7fe1d58d0740 10 client.16629 fetch_fsmap learned FSMap version 35 2024-09-15T23:06:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.518+0000 7fe1d58d0740 1 -- 172.21.15.111:0/2686149921 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562ffed2a7f0 con 0x562ffed22090 2024-09-15T23:06:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.518+0000 7fe1b9600640 1 -- 172.21.15.111:0/2686149921 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1b40c54a0 con 0x562ffed22090 2024-09-15T23:06:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.518+0000 7fe1b9600640 1 -- 172.21.15.111:0/2686149921 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fe1b40bbd10 con 0x562ffed22090 2024-09-15T23:06:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.518+0000 7fe1d58d0740 10 client.16629 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:06:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.518+0000 7fe1d58d0740 10 client.16629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:06:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.518+0000 7fe1d58d0740 20 client.16629 populate_metadata read hostname 'smithi111' 2024-09-15T23:06:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.518+0000 7fe1d58d0740 1 --2- 172.21.15.111:0/2686149921 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x562ffed37810 0x562ffed39c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.518+0000 7fe1d58d0740 4 client.16629 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:06:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.518+0000 7fe1d58d0740 1 -- 172.21.15.111:0/2686149921 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ffec94830 con 0x562ffed37810 2024-09-15T23:06:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.537+0000 7fe1cb400640 1 --2- 172.21.15.111:0/2686149921 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x562ffed37810 0x562ffed39c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.549+0000 7fe1b9600640 10 client.16629 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:06:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.549+0000 7fe1cb400640 1 --2- 172.21.15.111:0/2686149921 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x562ffed37810 0x562ffed39c40 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15a22aa33cb2559b server_cookie=640903fe87a811ec in_seq=0 out_seq=0 2024-09-15T23:06:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.561+0000 7fe1b9600640 1 -- 172.21.15.111:0/2686149921 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562ffec94830 con 0x562ffed37810 2024-09-15T23:06:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.561+0000 7fe1b9600640 10 client.16629 handle_command_reply: tid=0 2024-09-15T23:06:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.561+0000 7fe1d58d0740 1 client.16629 shutdown 2024-09-15T23:06:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.561+0000 7fe1d58d0740 1 -- 172.21.15.111:0/2686149921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562ffed22090 msgr2=0x562ffed2c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.561+0000 7fe1d58d0740 1 --2- 172.21.15.111:0/2686149921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562ffed22090 0x562ffed2c050 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe1b402aa30 tx=0x7fe1b4003f10 comp rx=0 tx=0).stop 2024-09-15T23:06:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.561+0000 7fe1d58d0740 1 -- 172.21.15.111:0/2686149921 shutdown_connections 2024-09-15T23:06:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.561+0000 7fe1d58d0740 1 --2- 172.21.15.111:0/2686149921 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x562ffed37810 0x562ffed39c40 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.561+0000 7fe1d58d0740 1 --2- 172.21.15.111:0/2686149921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ffed27a60 0x562ffed335d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.561+0000 7fe1d58d0740 1 --2- 172.21.15.111:0/2686149921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ffed229d0 0x562ffed2c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.561+0000 7fe1d58d0740 1 --2- 172.21.15.111:0/2686149921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562ffed22090 0x562ffed2c050 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.561+0000 7fe1b9600640 0 client.16629 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:06:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.561+0000 7fe1d58d0740 1 -- 172.21.15.111:0/2686149921 >> 172.21.15.111:0/2686149921 conn(0x562ffed1d700 msgr2=0x562ffed1ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:06:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.561+0000 7fe1d58d0740 1 -- 172.21.15.111:0/2686149921 shutdown_connections 2024-09-15T23:06:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.561+0000 7fe1d58d0740 1 -- 172.21.15.111:0/2686149921 wait complete. 2024-09-15T23:06:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.561+0000 7fe1d58d0740 20 client.16629 trim_cache size 0 max 16384 2024-09-15T23:06:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1d58d0740 1 Processor -- start 2024-09-15T23:06:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1d58d0740 1 -- start start 2024-09-15T23:06:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1d58d0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ffed22090 0x562ffed2c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1d58d0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ffed229d0 0x562ffed334f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1d58d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562ffed27a60 0x562ffed1c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1d58d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc1b1f20 con 0x562ffed27a60 2024-09-15T23:06:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1d58d0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc1af7f0 con 0x562ffed229d0 2024-09-15T23:06:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1d58d0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc1ad640 con 0x562ffed22090 2024-09-15T23:06:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1cb400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ffed229d0 0x562ffed334f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1d0a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562ffed27a60 0x562ffed1c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1cbe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ffed22090 0x562ffed2c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1cbe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ffed22090 0x562ffed2c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:52482/0 (socket says 172.21.15.111:52482) 2024-09-15T23:06:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1cbe00640 1 -- 172.21.15.111:0/2336000265 learned_addr learned my addr 172.21.15.111:0/2336000265 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:06:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1cb400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ffed229d0 0x562ffed334f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:47570/0 (socket says 172.21.15.111:47570) 2024-09-15T23:06:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1cbe00640 1 -- 172.21.15.111:0/2336000265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ffed229d0 msgr2=0x562ffed334f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:06:33.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1cbe00640 1 --2- 172.21.15.111:0/2336000265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ffed229d0 0x562ffed334f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1cbe00640 1 -- 172.21.15.111:0/2336000265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562ffed27a60 msgr2=0x562ffed1c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:06:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1cbe00640 1 --2- 172.21.15.111:0/2336000265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562ffed27a60 0x562ffed1c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1cbe00640 1 -- 172.21.15.111:0/2336000265 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ffed3a2c0 con 0x562ffed22090 2024-09-15T23:06:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1cb400640 1 --2- 172.21.15.111:0/2336000265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ffed229d0 0x562ffed334f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:06:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1cbe00640 1 --2- 172.21.15.111:0/2336000265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ffed22090 0x562ffed2c620 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40b8d20 tx=0x7fe1b40bca40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbad88968d114cb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:06:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1bbe00640 1 -- 172.21.15.111:0/2336000265 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4049020 con 0x562ffed22090 2024-09-15T23:06:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1bbe00640 1 -- 172.21.15.111:0/2336000265 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1b40c3120 con 0x562ffed22090 2024-09-15T23:06:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1bbe00640 1 -- 172.21.15.111:0/2336000265 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b40bbd10 con 0x562ffed22090 2024-09-15T23:06:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1d58d0740 1 -- 172.21.15.111:0/2336000265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ffed22090 msgr2=0x562ffed2c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1d58d0740 1 --2- 172.21.15.111:0/2336000265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ffed22090 0x562ffed2c620 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40b8d20 tx=0x7fe1b40bca40 comp rx=0 tx=0).stop 2024-09-15T23:06:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1d58d0740 1 -- 172.21.15.111:0/2336000265 shutdown_connections 2024-09-15T23:06:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1d58d0740 1 --2- 172.21.15.111:0/2336000265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562ffed27a60 0x562ffed1c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1d58d0740 1 --2- 172.21.15.111:0/2336000265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ffed229d0 0x562ffed334f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1d58d0740 1 --2- 172.21.15.111:0/2336000265 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ffed22090 0x562ffed2c620 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1d58d0740 1 -- 172.21.15.111:0/2336000265 >> 172.21.15.111:0/2336000265 conn(0x562ffed1d700 msgr2=0x562ffed1c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:06:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1d58d0740 1 -- 172.21.15.111:0/2336000265 shutdown_connections 2024-09-15T23:06:33.588 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1d58d0740 1 -- 172.21.15.111:0/2336000265 wait complete. 2024-09-15T23:06:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.578+0000 7fe1d58d0740 1 -- start start 2024-09-15T23:06:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.583+0000 7fe1d58d0740 1 Processor -- start 2024-09-15T23:06:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.583+0000 7fe1d58d0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562ffed22090 0x562ffed30bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.583+0000 7fe1d58d0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ffed229d0 0x562ffed31100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.583+0000 7fe1d58d0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ffed27a60 0x562ffed32110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.583+0000 7fe1d58d0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc1adcb0 con 0x562ffed22090 2024-09-15T23:06:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.583+0000 7fe1d58d0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc1af120 con 0x562ffed27a60 2024-09-15T23:06:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.583+0000 7fe1d58d0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc1ae380 con 0x562ffed229d0 2024-09-15T23:06:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.583+0000 7fe1d58d0740 10 client.0 _ll_register_callbacks cb 0x7fe1b40bcfc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:06:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.583+0000 7fe1cbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562ffed22090 0x562ffed30bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.583+0000 7fe1cbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562ffed22090 0x562ffed30bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:36074/0 (socket says 172.21.15.111:36074) 2024-09-15T23:06:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.583+0000 7fe1d0a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ffed27a60 0x562ffed32110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.583+0000 7fe1cbe00640 1 -- 172.21.15.111:0/3326297922 learned_addr learned my addr 172.21.15.111:0/3326297922 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:06:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.583+0000 7fe1cb400640 1 --2- 172.21.15.111:0/3326297922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ffed229d0 0x562ffed31100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.583+0000 7fe1cbe00640 1 -- 172.21.15.111:0/3326297922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ffed229d0 msgr2=0x562ffed31100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:06:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.583+0000 7fe1cbe00640 1 --2- 172.21.15.111:0/3326297922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ffed229d0 0x562ffed31100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.583+0000 7fe1cbe00640 1 -- 172.21.15.111:0/3326297922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ffed27a60 msgr2=0x562ffed32110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:06:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.583+0000 7fe1cbe00640 1 --2- 172.21.15.111:0/3326297922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ffed27a60 0x562ffed32110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.583+0000 7fe1cbe00640 1 -- 172.21.15.111:0/3326297922 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ffed18d60 con 0x562ffed22090 2024-09-15T23:06:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.583+0000 7fe1cbe00640 1 --2- 172.21.15.111:0/3326297922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562ffed22090 0x562ffed30bc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40bc850 tx=0x7fe1b4009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89d4c7a3b9e4a01 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:06:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.583+0000 7fe1b9600640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:06:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.583+0000 7fe1b9600640 1 -- 172.21.15.111:0/3326297922 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4049050 con 0x562ffed22090 2024-09-15T23:06:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.585+0000 7fe1b9600640 1 -- 172.21.15.111:0/3326297922 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fe1b40bbd10 con 0x562ffed22090 2024-09-15T23:06:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.585+0000 7fe1d58d0740 1 -- 172.21.15.111:0/3326297922 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1b40c3120 con 0x562ffed22090 2024-09-15T23:06:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.585+0000 7fe1b9600640 1 -- 172.21.15.111:0/3326297922 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe1b40c54a0 con 0x562ffed22090 2024-09-15T23:06:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.585+0000 7fe1d58d0740 10 client.16635 fetch_fsmap learned FSMap version 35 2024-09-15T23:06:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.585+0000 7fe1d58d0740 1 -- 172.21.15.111:0/3326297922 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562ffed33860 con 0x562ffed22090 2024-09-15T23:06:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.585+0000 7fe1b9600640 1 -- 172.21.15.111:0/3326297922 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1b40c3120 con 0x562ffed22090 2024-09-15T23:06:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.585+0000 7fe1b9600640 1 -- 172.21.15.111:0/3326297922 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fe1b40b8610 con 0x562ffed22090 2024-09-15T23:06:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.585+0000 7fe1d58d0740 10 client.16635 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:06:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.585+0000 7fe1d58d0740 10 client.16635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:06:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.585+0000 7fe1d58d0740 20 client.16635 populate_metadata read hostname 'smithi111' 2024-09-15T23:06:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.585+0000 7fe1d58d0740 1 --2- 172.21.15.111:0/3326297922 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x562ffeda29b0 0x562ffeda4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.585+0000 7fe1d58d0740 4 client.16635 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:06:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.585+0000 7fe1d58d0740 1 -- 172.21.15.111:0/3326297922 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562ffec8eb80 con 0x562ffeda29b0 2024-09-15T23:06:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.585+0000 7fe1cb400640 1 --2- 172.21.15.111:0/3326297922 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x562ffeda29b0 0x562ffeda4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.585+0000 7fe1cb400640 1 --2- 172.21.15.111:0/3326297922 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x562ffeda29b0 0x562ffeda4da0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7fe9754f70c0e16 server_cookie=e2a25fcc99f91138 in_seq=0 out_seq=0 2024-09-15T23:06:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.585+0000 7fe1b9600640 10 client.16635 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:06:33.676 INFO:tasks.workunit.client.0.smithi111.stdout: 10 147863 17.84 MB/sec execute 301 sec latency 2680.414 ms 2024-09-15T23:06:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.840+0000 7fe1b9600640 1 -- 172.21.15.111:0/3326297922 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562ffec8eb80 con 0x562ffeda29b0 2024-09-15T23:06:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.840+0000 7fe1b9600640 10 client.16635 handle_command_reply: tid=0 2024-09-15T23:06:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.840+0000 7fe1d58d0740 1 client.16635 shutdown 2024-09-15T23:06:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.840+0000 7fe1d58d0740 1 -- 172.21.15.111:0/3326297922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562ffed22090 msgr2=0x562ffed30bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.840+0000 7fe1d58d0740 1 --2- 172.21.15.111:0/3326297922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562ffed22090 0x562ffed30bc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe1b40bc850 tx=0x7fe1b4009680 comp rx=0 tx=0).stop 2024-09-15T23:06:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.840+0000 7fe1d58d0740 1 -- 172.21.15.111:0/3326297922 shutdown_connections 2024-09-15T23:06:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.840+0000 7fe1d58d0740 1 --2- 172.21.15.111:0/3326297922 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x562ffeda29b0 0x562ffeda4da0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.840+0000 7fe1d58d0740 1 --2- 172.21.15.111:0/3326297922 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562ffed27a60 0x562ffed32110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.840+0000 7fe1d58d0740 1 --2- 172.21.15.111:0/3326297922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562ffed229d0 0x562ffed31100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.840+0000 7fe1d58d0740 1 --2- 172.21.15.111:0/3326297922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562ffed22090 0x562ffed30bc0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.840+0000 7fe1d58d0740 1 -- 172.21.15.111:0/3326297922 >> 172.21.15.111:0/3326297922 conn(0x562ffed1d700 msgr2=0x562ffed2c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:06:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.840+0000 7fe1b9600640 0 client.16635 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:06:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.840+0000 7fe1d58d0740 1 -- 172.21.15.111:0/3326297922 shutdown_connections 2024-09-15T23:06:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.840+0000 7fe1d58d0740 1 -- 172.21.15.111:0/3326297922 wait complete. 2024-09-15T23:06:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.840+0000 7fe1d58d0740 20 client.16635 trim_cache size 0 max 16384 2024-09-15T23:06:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:06:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:06:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:06:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:06:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.840+0000 7fe1d2a00640 1 -- 172.21.15.111:0/2281859884 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fe1b00a33c0 msgr2=0x7fe1b00a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.840+0000 7fe1d2a00640 1 --2- 172.21.15.111:0/2281859884 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fe1b00a33c0 0x7fe1b00a5880 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4009440 tx=0x7fe1c4009650 comp rx=0 tx=0).stop 2024-09-15T23:06:33.844 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.840+0000 7fe1d2a00640 1 -- 172.21.15.111:0/2281859884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1cc076bb0 msgr2=0x7fe1cc19e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.840+0000 7fe1d2a00640 1 --2- 172.21.15.111:0/2281859884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1cc076bb0 0x7fe1cc19e580 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4031440 tx=0x7fe1b4032f00 comp rx=0 tx=0).stop 2024-09-15T23:06:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.840+0000 7fe1d2a00640 1 -- 172.21.15.111:0/2281859884 shutdown_connections 2024-09-15T23:06:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.840+0000 7fe1d2a00640 1 --2- 172.21.15.111:0/2281859884 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe1cc078790 0x7fe1cc1a5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.840+0000 7fe1d2a00640 1 --2- 172.21.15.111:0/2281859884 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fe1b00a33c0 0x7fe1b00a5880 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.840+0000 7fe1d2a00640 1 --2- 172.21.15.111:0/2281859884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1cc077df0 0x7fe1cc19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.840+0000 7fe1d2a00640 1 --2- 172.21.15.111:0/2281859884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1cc076bb0 0x7fe1cc19e580 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.840+0000 7fe1d2a00640 1 -- 172.21.15.111:0/2281859884 >> 172.21.15.111:0/2281859884 conn(0x7fe1cc0ff4b0 msgr2=0x7fe1cc100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:06:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.840+0000 7fe1d2a00640 1 -- 172.21.15.111:0/2281859884 shutdown_connections 2024-09-15T23:06:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:33.840+0000 7fe1d2a00640 1 -- 172.21.15.111:0/2281859884 wait complete. 2024-09-15T23:06:34.067 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:34 smithi111 ceph-mon[30629]: pgmap v970: 97 pgs: 97 active+clean; 569 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.4 MiB/s wr, 548 op/s 2024-09-15T23:06:34.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:34 smithi192 ceph-mon[35427]: pgmap v970: 97 pgs: 97 active+clean; 569 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.4 MiB/s wr, 548 op/s 2024-09-15T23:06:34.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:34 smithi121 ceph-mon[34815]: pgmap v970: 97 pgs: 97 active+clean; 569 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.4 MiB/s wr, 548 op/s 2024-09-15T23:06:34.676 INFO:tasks.workunit.client.0.smithi111.stdout: 10 148268 17.84 MB/sec execute 302 sec latency 3150.606 ms 2024-09-15T23:06:35.676 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149035 17.91 MB/sec execute 303 sec latency 61.604 ms 2024-09-15T23:06:36.209 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:36 smithi111 ceph-mon[30629]: pgmap v971: 97 pgs: 97 active+clean; 553 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.0 MiB/s wr, 551 op/s 2024-09-15T23:06:36.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:36 smithi192 ceph-mon[35427]: pgmap v971: 97 pgs: 97 active+clean; 553 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.0 MiB/s wr, 551 op/s 2024-09-15T23:06:36.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:36 smithi121 ceph-mon[34815]: pgmap v971: 97 pgs: 97 active+clean; 553 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.0 MiB/s wr, 551 op/s 2024-09-15T23:06:36.676 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149309 17.91 MB/sec execute 304 sec latency 556.100 ms 2024-09-15T23:06:37.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:37 smithi192 ceph-mon[35427]: pgmap v972: 97 pgs: 97 active+clean; 554 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 5.0 MiB/s wr, 456 op/s 2024-09-15T23:06:37.318 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:37 smithi111 ceph-mon[30629]: pgmap v972: 97 pgs: 97 active+clean; 554 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 5.0 MiB/s wr, 456 op/s 2024-09-15T23:06:37.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:37 smithi121 ceph-mon[34815]: pgmap v972: 97 pgs: 97 active+clean; 554 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 5.0 MiB/s wr, 456 op/s 2024-09-15T23:06:37.676 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149310 17.86 MB/sec execute 305 sec latency 1533.291 ms 2024-09-15T23:06:38.676 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149310 17.80 MB/sec execute 306 sec latency 2533.340 ms 2024-09-15T23:06:39.676 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149311 17.74 MB/sec execute 307 sec latency 3533.380 ms 2024-09-15T23:06:40.316 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:39 smithi192 ceph-mon[35427]: pgmap v973: 97 pgs: 97 active+clean; 559 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.8 MiB/s wr, 572 op/s 2024-09-15T23:06:40.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:39 smithi121 ceph-mon[34815]: pgmap v973: 97 pgs: 97 active+clean; 559 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.8 MiB/s wr, 572 op/s 2024-09-15T23:06:40.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:39 smithi111 ceph-mon[30629]: pgmap v973: 97 pgs: 97 active+clean; 559 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.8 MiB/s wr, 572 op/s 2024-09-15T23:06:40.676 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149340 17.68 MB/sec execute 308 sec latency 4533.423 ms 2024-09-15T23:06:41.676 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149342 17.63 MB/sec execute 309 sec latency 5533.076 ms 2024-09-15T23:06:41.983 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:41 smithi111 ceph-mon[30629]: pgmap v974: 97 pgs: 97 active+clean; 561 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.1 MiB/s wr, 552 op/s 2024-09-15T23:06:42.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:41 smithi192 ceph-mon[35427]: pgmap v974: 97 pgs: 97 active+clean; 561 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.1 MiB/s wr, 552 op/s 2024-09-15T23:06:42.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:41 smithi121 ceph-mon[34815]: pgmap v974: 97 pgs: 97 active+clean; 561 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.1 MiB/s wr, 552 op/s 2024-09-15T23:06:42.676 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149342 17.57 MB/sec execute 310 sec latency 6533.132 ms 2024-09-15T23:06:43.676 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149347 17.51 MB/sec execute 311 sec latency 7349.548 ms 2024-09-15T23:06:44.223 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:43 smithi111 ceph-mon[30629]: pgmap v975: 97 pgs: 97 active+clean; 564 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.9 MiB/s wr, 437 op/s 2024-09-15T23:06:44.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:43 smithi192 ceph-mon[35427]: pgmap v975: 97 pgs: 97 active+clean; 564 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.9 MiB/s wr, 437 op/s 2024-09-15T23:06:44.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:43 smithi121 ceph-mon[34815]: pgmap v975: 97 pgs: 97 active+clean; 564 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.9 MiB/s wr, 437 op/s 2024-09-15T23:06:44.676 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149363 17.46 MB/sec execute 312 sec latency 4688.549 ms 2024-09-15T23:06:45.676 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149371 17.40 MB/sec execute 313 sec latency 4876.134 ms 2024-09-15T23:06:46.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:45 smithi192 ceph-mon[35427]: pgmap v976: 97 pgs: 97 active+clean; 570 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.9 MiB/s wr, 389 op/s 2024-09-15T23:06:46.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:45 smithi121 ceph-mon[34815]: pgmap v976: 97 pgs: 97 active+clean; 570 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.9 MiB/s wr, 389 op/s 2024-09-15T23:06:46.393 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:45 smithi111 ceph-mon[30629]: pgmap v976: 97 pgs: 97 active+clean; 570 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.9 MiB/s wr, 389 op/s 2024-09-15T23:06:46.676 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149371 17.35 MB/sec execute 314 sec latency 3207.272 ms 2024-09-15T23:06:46.958 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.955+0000 7f3d12a00640 1 -- 172.21.15.111:0/976512552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d0c104cf0 msgr2=0x7f3d0c1070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:46.958 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.955+0000 7f3d12a00640 1 --2- 172.21.15.111:0/976512552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d0c104cf0 0x7f3d0c1070e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009900 tx=0x7f3cf802a3f0 comp rx=0 tx=0).stop 2024-09-15T23:06:46.958 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.955+0000 7f3d12a00640 1 -- 172.21.15.111:0/976512552 shutdown_connections 2024-09-15T23:06:46.958 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.955+0000 7f3d12a00640 1 --2- 172.21.15.111:0/976512552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d0c107620 0x7f3d0c109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.955+0000 7f3d12a00640 1 --2- 172.21.15.111:0/976512552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d0c104cf0 0x7f3d0c1070e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.955+0000 7f3d12a00640 1 --2- 172.21.15.111:0/976512552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d0c1023c0 0x7f3d0c1047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.955+0000 7f3d12a00640 1 -- 172.21.15.111:0/976512552 >> 172.21.15.111:0/976512552 conn(0x7f3d0c0fbfa0 msgr2=0x7f3d0c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:06:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.955+0000 7f3d12a00640 1 -- 172.21.15.111:0/976512552 shutdown_connections 2024-09-15T23:06:46.959 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.955+0000 7f3d12a00640 1 -- 172.21.15.111:0/976512552 wait complete. 2024-09-15T23:06:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.958+0000 7f3d12a00640 1 Processor -- start 2024-09-15T23:06:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.958+0000 7f3d12a00640 1 -- start start 2024-09-15T23:06:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.958+0000 7f3d12a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d0c1023c0 0x7f3d0c197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:46.960 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.958+0000 7f3d12a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d0c104cf0 0x7f3d0c198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.958+0000 7f3d12a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d0c107620 0x7f3d0c19f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.958+0000 7f3d12a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c10d640 con 0x7f3d0c104cf0 2024-09-15T23:06:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.958+0000 7f3d12a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c10d4c0 con 0x7f3d0c107620 2024-09-15T23:06:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.958+0000 7f3d12a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c10d7c0 con 0x7f3d0c1023c0 2024-09-15T23:06:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.958+0000 7f3d10c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d0c104cf0 0x7f3d0c198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.958+0000 7f3d10c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d0c104cf0 0x7f3d0c198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:33616/0 (socket says 172.21.15.111:33616) 2024-09-15T23:06:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.958+0000 7f3d10c00640 1 -- 172.21.15.111:0/1462449772 learned_addr learned my addr 172.21.15.111:0/1462449772 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:06:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.958+0000 7f3d12000640 1 --2- 172.21.15.111:0/1462449772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d0c107620 0x7f3d0c19f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.958+0000 7f3d10c00640 1 -- 172.21.15.111:0/1462449772 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d0c1023c0 msgr2=0x7f3d0c197d50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T23:06:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.958+0000 7f3d10c00640 1 --2- 172.21.15.111:0/1462449772 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d0c1023c0 0x7f3d0c197d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.958+0000 7f3d10c00640 1 -- 172.21.15.111:0/1462449772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d0c107620 msgr2=0x7f3d0c19f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.958+0000 7f3d10c00640 1 --2- 172.21.15.111:0/1462449772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d0c107620 0x7f3d0c19f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:46.961 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.958+0000 7f3d10c00640 1 -- 172.21.15.111:0/1462449772 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0c19f960 con 0x7f3d0c104cf0 2024-09-15T23:06:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.958+0000 7f3d10c00640 1 --2- 172.21.15.111:0/1462449772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d0c104cf0 0x7f3d0c198290 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8031440 tx=0x7f3cf8032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:06:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.958+0000 7f3d0a000640 1 -- 172.21.15.111:0/1462449772 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf803f030 con 0x7f3d0c104cf0 2024-09-15T23:06:46.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.958+0000 7f3d12a00640 1 -- 172.21.15.111:0/1462449772 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d0c19fbf0 con 0x7f3d0c104cf0 2024-09-15T23:06:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.958+0000 7f3d0a000640 1 -- 172.21.15.111:0/1462449772 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cf8003ce0 con 0x7f3d0c104cf0 2024-09-15T23:06:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.958+0000 7f3d12a00640 1 -- 172.21.15.111:0/1462449772 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d0c1a0130 con 0x7f3d0c104cf0 2024-09-15T23:06:46.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.958+0000 7f3d0a000640 1 -- 172.21.15.111:0/1462449772 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8049060 con 0x7f3d0c104cf0 2024-09-15T23:06:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d16dfd740 1 Processor -- start 2024-09-15T23:06:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d16dfd740 1 -- start start 2024-09-15T23:06:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d16dfd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c7d7d81670 0x55c7d7d81a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d16dfd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7d7d81fb0 0x55c7d7d86a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d16dfd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7d7d87040 0x55c7d7d89500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d16dfd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c7d7ce2100 con 0x55c7d7d87040 2024-09-15T23:06:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d16dfd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c7d7c8a1b0 con 0x55c7d7d81fb0 2024-09-15T23:06:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d16dfd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c7d7cde0c0 con 0x55c7d7d81670 2024-09-15T23:06:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d12000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c7d7d81670 0x55c7d7d81a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d12000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c7d7d81670 0x55c7d7d81a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:48406/0 (socket says 172.21.15.111:48406) 2024-09-15T23:06:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d12000640 1 -- 172.21.15.111:0/3501854888 learned_addr learned my addr 172.21.15.111:0/3501854888 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:06:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d0a000640 1 -- 172.21.15.111:0/1462449772 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f3cf8042030 con 0x7f3d0c104cf0 2024-09-15T23:06:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d12000640 1 -- 172.21.15.111:0/3501854888 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7d7d81fb0 msgr2=0x55c7d7d86a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:06:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d12000640 1 --2- 172.21.15.111:0/3501854888 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7d7d81fb0 0x55c7d7d86a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d12000640 1 -- 172.21.15.111:0/3501854888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7d7d87040 msgr2=0x55c7d7d89500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:06:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d12000640 1 --2- 172.21.15.111:0/3501854888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7d7d87040 0x55c7d7d89500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d12000640 1 -- 172.21.15.111:0/3501854888 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7d79f5210 con 0x55c7d7d81670 2024-09-15T23:06:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d12000640 1 --2- 172.21.15.111:0/3501854888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c7d7d81670 0x55c7d7d81a70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009f90 tx=0x7f3d040118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1172ebde59eb5d5e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:06:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3cfe600640 1 -- 172.21.15.111:0/3501854888 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04004d20 con 0x55c7d7d81670 2024-09-15T23:06:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3cfe600640 1 -- 172.21.15.111:0/3501854888 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d04004050 con 0x55c7d7d81670 2024-09-15T23:06:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3cfe600640 1 -- 172.21.15.111:0/3501854888 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d04005260 con 0x55c7d7d81670 2024-09-15T23:06:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d0a000640 1 --2- 172.21.15.111:0/1462449772 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3cf40a3240 0x7f3cf40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d16dfd740 1 -- 172.21.15.111:0/3501854888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c7d7d81670 msgr2=0x55c7d7d81a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d16dfd740 1 --2- 172.21.15.111:0/3501854888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c7d7d81670 0x55c7d7d81a70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009f90 tx=0x7f3d040118d0 comp rx=0 tx=0).stop 2024-09-15T23:06:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d0a000640 1 -- 172.21.15.111:0/1462449772 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f3cf80f01f0 con 0x7f3d0c104cf0 2024-09-15T23:06:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d16dfd740 1 -- 172.21.15.111:0/3501854888 shutdown_connections 2024-09-15T23:06:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d16dfd740 1 --2- 172.21.15.111:0/3501854888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7d7d87040 0x55c7d7d89500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d16dfd740 1 --2- 172.21.15.111:0/3501854888 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7d7d81fb0 0x55c7d7d86a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d16dfd740 1 --2- 172.21.15.111:0/3501854888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c7d7d81670 0x55c7d7d81a70 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:46.969 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d16dfd740 1 -- 172.21.15.111:0/3501854888 >> 172.21.15.111:0/3501854888 conn(0x55c7d7d788d0 msgr2=0x55c7d7d78cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:06:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d11600640 1 --2- 172.21.15.111:0/1462449772 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3cf40a3240 0x7f3cf40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d16dfd740 1 -- 172.21.15.111:0/3501854888 shutdown_connections 2024-09-15T23:06:46.970 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d16dfd740 1 -- 172.21.15.111:0/3501854888 wait complete. 2024-09-15T23:06:46.972 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d16dfd740 1 -- start start 2024-09-15T23:06:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d16dfd740 1 Processor -- start 2024-09-15T23:06:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.963+0000 7f3d11600640 1 --2- 172.21.15.111:0/1462449772 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3cf40a3240 0x7f3cf40a5700 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x55c7d7d79750 tx=0x7f3d00005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:06:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.968+0000 7f3d16dfd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7d7d81670 0x55c7d7d8b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.968+0000 7f3d16dfd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c7d7d81fb0 0x55c7d7d8bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.968+0000 7f3d16dfd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7d7d87040 0x55c7d7d92dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.968+0000 7f3d16dfd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c7d7d89fb0 con 0x55c7d7d87040 2024-09-15T23:06:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.968+0000 7f3d16dfd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c7d7c6b090 con 0x55c7d7d81670 2024-09-15T23:06:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.968+0000 7f3d16dfd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c7d7d8a130 con 0x55c7d7d81fb0 2024-09-15T23:06:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.968+0000 7f3d16dfd740 10 client.0 _ll_register_callbacks cb 0x55c7d7cbff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:06:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.968+0000 7f3d11600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7d7d81670 0x55c7d7d8b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:46.973 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.968+0000 7f3d11600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7d7d81670 0x55c7d7d8b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:45894/0 (socket says 172.21.15.111:45894) 2024-09-15T23:06:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.968+0000 7f3d11600640 1 -- 172.21.15.111:0/3566274623 learned_addr learned my addr 172.21.15.111:0/3566274623 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:06:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.968+0000 7f3d10c00640 1 --2- 172.21.15.111:0/3566274623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c7d7d81fb0 0x55c7d7d8bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.968+0000 7f3d12000640 1 --2- 172.21.15.111:0/3566274623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7d7d87040 0x55c7d7d92dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.968+0000 7f3d11600640 1 -- 172.21.15.111:0/3566274623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c7d7d81fb0 msgr2=0x55c7d7d8bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:06:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.968+0000 7f3d11600640 1 --2- 172.21.15.111:0/3566274623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c7d7d81fb0 0x55c7d7d8bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.968+0000 7f3d11600640 1 -- 172.21.15.111:0/3566274623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7d7d87040 msgr2=0x55c7d7d92dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:06:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.968+0000 7f3d11600640 1 --2- 172.21.15.111:0/3566274623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7d7d87040 0x55c7d7d92dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.968+0000 7f3d11600640 1 -- 172.21.15.111:0/3566274623 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7d7ac1620 con 0x55c7d7d81670 2024-09-15T23:06:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.968+0000 7f3d11600640 1 --2- 172.21.15.111:0/3566274623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7d7d81670 0x55c7d7d8b850 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3d00004520 tx=0x7f3d00004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70917f72fe486bf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:06:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.968+0000 7f3cf2a00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:06:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.970+0000 7f3cf2a00640 1 -- 172.21.15.111:0/3566274623 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00021070 con 0x55c7d7d81670 2024-09-15T23:06:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.970+0000 7f3cf2a00640 1 -- 172.21.15.111:0/3566274623 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f3d00003cf0 con 0x55c7d7d81670 2024-09-15T23:06:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.970+0000 7f3d16dfd740 1 -- 172.21.15.111:0/3566274623 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d04004050 con 0x55c7d7d81670 2024-09-15T23:06:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.970+0000 7f3cf2a00640 1 -- 172.21.15.111:0/3566274623 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3d0002d050 con 0x55c7d7d81670 2024-09-15T23:06:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.970+0000 7f3d16dfd740 10 client.26579 fetch_fsmap learned FSMap version 35 2024-09-15T23:06:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.970+0000 7f3d16dfd740 1 -- 172.21.15.111:0/3566274623 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c7d7d95c90 con 0x55c7d7d81670 2024-09-15T23:06:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.970+0000 7f3cf2a00640 1 -- 172.21.15.111:0/3566274623 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d04004050 con 0x55c7d7d81670 2024-09-15T23:06:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.970+0000 7f3cf2a00640 1 -- 172.21.15.111:0/3566274623 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f3d00010660 con 0x55c7d7d81670 2024-09-15T23:06:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.970+0000 7f3d16dfd740 10 client.26579 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:06:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.970+0000 7f3d16dfd740 10 client.26579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:06:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.970+0000 7f3d16dfd740 20 client.26579 populate_metadata read hostname 'smithi111' 2024-09-15T23:06:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.970+0000 7f3d16dfd740 1 --2- 172.21.15.111:0/3566274623 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c7d7d97310 0x55c7d7d99740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.970+0000 7f3d16dfd740 4 client.26579 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:06:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:46.970+0000 7f3d16dfd740 1 -- 172.21.15.111:0/3566274623 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7d7cef830 con 0x55c7d7d97310 2024-09-15T23:06:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.004+0000 7f3d10c00640 1 --2- 172.21.15.111:0/3566274623 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c7d7d97310 0x55c7d7d99740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.017+0000 7f3d10c00640 1 --2- 172.21.15.111:0/3566274623 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c7d7d97310 0x55c7d7d99740 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d42b18ef623b2dda server_cookie=e4109f766e6d86 in_seq=0 out_seq=0 2024-09-15T23:06:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.017+0000 7f3cf2a00640 10 client.26579 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:06:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.017+0000 7f3cf2a00640 1 -- 172.21.15.111:0/3566274623 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c7d7cef830 con 0x55c7d7d97310 2024-09-15T23:06:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.017+0000 7f3cf2a00640 10 client.26579 handle_command_reply: tid=0 2024-09-15T23:06:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.017+0000 7f3d16dfd740 1 client.26579 shutdown 2024-09-15T23:06:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.017+0000 7f3d16dfd740 1 -- 172.21.15.111:0/3566274623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7d7d81670 msgr2=0x55c7d7d8b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.017+0000 7f3d16dfd740 1 --2- 172.21.15.111:0/3566274623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7d7d81670 0x55c7d7d8b850 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3d00004520 tx=0x7f3d00004730 comp rx=0 tx=0).stop 2024-09-15T23:06:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.017+0000 7f3d16dfd740 1 -- 172.21.15.111:0/3566274623 shutdown_connections 2024-09-15T23:06:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.017+0000 7f3d16dfd740 1 --2- 172.21.15.111:0/3566274623 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c7d7d97310 0x55c7d7d99740 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.017+0000 7f3d16dfd740 1 --2- 172.21.15.111:0/3566274623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7d7d87040 0x55c7d7d92dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.017+0000 7f3d16dfd740 1 --2- 172.21.15.111:0/3566274623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c7d7d81fb0 0x55c7d7d8bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.017+0000 7f3cf2a00640 0 client.26579 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:06:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.017+0000 7f3d16dfd740 1 --2- 172.21.15.111:0/3566274623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7d7d81670 0x55c7d7d8b850 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.017+0000 7f3d16dfd740 1 -- 172.21.15.111:0/3566274623 >> 172.21.15.111:0/3566274623 conn(0x55c7d7d788d0 msgr2=0x55c7d7d7a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:06:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.017+0000 7f3d16dfd740 1 -- 172.21.15.111:0/3566274623 shutdown_connections 2024-09-15T23:06:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.017+0000 7f3d16dfd740 1 -- 172.21.15.111:0/3566274623 wait complete. 2024-09-15T23:06:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.017+0000 7f3d16dfd740 20 client.26579 trim_cache size 0 max 16384 2024-09-15T23:06:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d16dfd740 1 Processor -- start 2024-09-15T23:06:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d16dfd740 1 -- start start 2024-09-15T23:06:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d16dfd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7d7d81670 0x55c7d7d79780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d16dfd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c7d7d81fb0 0x55c7d7d92d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d16dfd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7d7d87040 0x55c7d7d79cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d16dfd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c1a9590 con 0x55c7d7d81670 2024-09-15T23:06:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d16dfd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c1a6e60 con 0x55c7d7d87040 2024-09-15T23:06:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d16dfd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c1a4cb0 con 0x55c7d7d81fb0 2024-09-15T23:06:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d11600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7d7d81670 0x55c7d7d79780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d10c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c7d7d81fb0 0x55c7d7d92d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d10c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c7d7d81fb0 0x55c7d7d92d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:48414/0 (socket says 172.21.15.111:48414) 2024-09-15T23:06:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d10c00640 1 -- 172.21.15.111:0/555443209 learned_addr learned my addr 172.21.15.111:0/555443209 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:06:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d12000640 1 --2- 172.21.15.111:0/555443209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7d7d87040 0x55c7d7d79cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d10c00640 1 -- 172.21.15.111:0/555443209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7d7d87040 msgr2=0x55c7d7d79cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:06:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d10c00640 1 --2- 172.21.15.111:0/555443209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7d7d87040 0x55c7d7d79cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d10c00640 1 -- 172.21.15.111:0/555443209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7d7d81670 msgr2=0x55c7d7d79780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:06:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d10c00640 1 --2- 172.21.15.111:0/555443209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7d7d81670 0x55c7d7d79780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d10c00640 1 -- 172.21.15.111:0/555443209 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7d7d79580 con 0x55c7d7d81fb0 2024-09-15T23:06:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d10c00640 1 --2- 172.21.15.111:0/555443209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c7d7d81fb0 0x55c7d7d92d90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8003fb0 tx=0x7f3cf80a7040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc627580769a1ac8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:06:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3cfdc00640 1 -- 172.21.15.111:0/555443209 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8035040 con 0x55c7d7d81fb0 2024-09-15T23:06:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3cfdc00640 1 -- 172.21.15.111:0/555443209 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cf80bc810 con 0x55c7d7d81fb0 2024-09-15T23:06:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3cfdc00640 1 -- 172.21.15.111:0/555443209 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf80b8df0 con 0x55c7d7d81fb0 2024-09-15T23:06:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d16dfd740 1 -- 172.21.15.111:0/555443209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c7d7d81fb0 msgr2=0x55c7d7d92d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d16dfd740 1 --2- 172.21.15.111:0/555443209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c7d7d81fb0 0x55c7d7d92d90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8003fb0 tx=0x7f3cf80a7040 comp rx=0 tx=0).stop 2024-09-15T23:06:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d16dfd740 1 -- 172.21.15.111:0/555443209 shutdown_connections 2024-09-15T23:06:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d16dfd740 1 --2- 172.21.15.111:0/555443209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7d7d87040 0x55c7d7d79cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d16dfd740 1 --2- 172.21.15.111:0/555443209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c7d7d81fb0 0x55c7d7d92d90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d16dfd740 1 --2- 172.21.15.111:0/555443209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7d7d81670 0x55c7d7d79780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d16dfd740 1 -- 172.21.15.111:0/555443209 >> 172.21.15.111:0/555443209 conn(0x55c7d7d788d0 msgr2=0x55c7d7d8b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:06:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d16dfd740 1 -- 172.21.15.111:0/555443209 shutdown_connections 2024-09-15T23:06:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.040+0000 7f3d16dfd740 1 -- 172.21.15.111:0/555443209 wait complete. 2024-09-15T23:06:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d16dfd740 1 -- start start 2024-09-15T23:06:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d16dfd740 1 Processor -- start 2024-09-15T23:06:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d16dfd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7d7d81670 0x55c7d7d90700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d16dfd740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c7d7d81fb0 0x55c7d7d90c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d16dfd740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7d7d87040 0x55c7d7d91c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d16dfd740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c1a5320 con 0x55c7d7d87040 2024-09-15T23:06:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d16dfd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c1a6790 con 0x55c7d7d81670 2024-09-15T23:06:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d16dfd740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c1a59f0 con 0x55c7d7d81fb0 2024-09-15T23:06:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d16dfd740 10 client.0 _ll_register_callbacks cb 0x7f3d00014050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:06:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d11600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7d7d81670 0x55c7d7d90700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d11600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7d7d81670 0x55c7d7d90700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:45908/0 (socket says 172.21.15.111:45908) 2024-09-15T23:06:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d11600640 1 -- 172.21.15.111:0/1805513608 learned_addr learned my addr 172.21.15.111:0/1805513608 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:06:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d12000640 1 --2- 172.21.15.111:0/1805513608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7d7d87040 0x55c7d7d91c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d11600640 1 -- 172.21.15.111:0/1805513608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c7d7d81fb0 msgr2=0x55c7d7d90c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:06:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d11600640 1 --2- 172.21.15.111:0/1805513608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c7d7d81fb0 0x55c7d7d90c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d11600640 1 -- 172.21.15.111:0/1805513608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7d7d87040 msgr2=0x55c7d7d91c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:06:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d11600640 1 --2- 172.21.15.111:0/1805513608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7d7d87040 0x55c7d7d91c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d11600640 1 -- 172.21.15.111:0/1805513608 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7d7d92210 con 0x55c7d7d81670 2024-09-15T23:06:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d11600640 1 --2- 172.21.15.111:0/1805513608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7d7d81670 0x55c7d7d90700 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3d00004630 tx=0x7f3d00011e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e7b0da8e791276c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:06:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3cf2a00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:06:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3cf2a00640 1 -- 172.21.15.111:0/1805513608 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00021050 con 0x55c7d7d81670 2024-09-15T23:06:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3cf2a00640 1 -- 172.21.15.111:0/1805513608 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f3d000106c0 con 0x55c7d7d81670 2024-09-15T23:06:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d16dfd740 1 -- 172.21.15.111:0/1805513608 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cf80bc810 con 0x55c7d7d81670 2024-09-15T23:06:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3cf2a00640 1 -- 172.21.15.111:0/1805513608 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3d000039c0 con 0x55c7d7d81670 2024-09-15T23:06:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d16dfd740 10 client.26585 fetch_fsmap learned FSMap version 35 2024-09-15T23:06:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d16dfd740 1 -- 172.21.15.111:0/1805513608 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c7d7e02200 con 0x55c7d7d81670 2024-09-15T23:06:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3cf2a00640 1 -- 172.21.15.111:0/1805513608 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cf80bc810 con 0x55c7d7d81670 2024-09-15T23:06:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3cf2a00640 1 -- 172.21.15.111:0/1805513608 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f3d0001db50 con 0x55c7d7d81670 2024-09-15T23:06:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d16dfd740 10 client.26585 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:06:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d16dfd740 10 client.26585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:06:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d16dfd740 20 client.26585 populate_metadata read hostname 'smithi111' 2024-09-15T23:06:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d16dfd740 1 --2- 172.21.15.111:0/1805513608 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c7d7e02be0 0x55c7d7e05000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:06:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d16dfd740 4 client.26585 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:06:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.046+0000 7f3d16dfd740 1 -- 172.21.15.111:0/1805513608 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c7d7ce9b80 con 0x55c7d7e02be0 2024-09-15T23:06:47.064 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.063+0000 7f3d10c00640 1 --2- 172.21.15.111:0/1805513608 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c7d7e02be0 0x55c7d7e05000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:06:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.123+0000 7f3d10c00640 1 --2- 172.21.15.111:0/1805513608 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c7d7e02be0 0x55c7d7e05000 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d348f10ae671b765 server_cookie=af2e14e9b80848e0 in_seq=0 out_seq=0 2024-09-15T23:06:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.123+0000 7f3cf2a00640 10 client.26585 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:06:47.290 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.288+0000 7f3cf2a00640 1 -- 172.21.15.111:0/1805513608 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c7d7ce9b80 con 0x55c7d7e02be0 2024-09-15T23:06:47.290 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.288+0000 7f3cf2a00640 10 client.26585 handle_command_reply: tid=0 2024-09-15T23:06:47.290 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.288+0000 7f3d16dfd740 1 client.26585 shutdown 2024-09-15T23:06:47.290 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.288+0000 7f3d16dfd740 1 -- 172.21.15.111:0/1805513608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7d7d81670 msgr2=0x55c7d7d90700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:47.290 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.288+0000 7f3d16dfd740 1 --2- 172.21.15.111:0/1805513608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7d7d81670 0x55c7d7d90700 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3d00004630 tx=0x7f3d00011e90 comp rx=0 tx=0).stop 2024-09-15T23:06:47.290 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.288+0000 7f3d16dfd740 1 -- 172.21.15.111:0/1805513608 shutdown_connections 2024-09-15T23:06:47.290 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.288+0000 7f3d16dfd740 1 --2- 172.21.15.111:0/1805513608 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c7d7e02be0 0x55c7d7e05000 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.288+0000 7f3d16dfd740 1 --2- 172.21.15.111:0/1805513608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c7d7d87040 0x55c7d7d91c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.288+0000 7f3d16dfd740 1 --2- 172.21.15.111:0/1805513608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c7d7d81fb0 0x55c7d7d90c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.288+0000 7f3cf2a00640 0 client.26585 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:06:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.288+0000 7f3d16dfd740 1 --2- 172.21.15.111:0/1805513608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c7d7d81670 0x55c7d7d90700 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.288+0000 7f3d16dfd740 1 -- 172.21.15.111:0/1805513608 >> 172.21.15.111:0/1805513608 conn(0x55c7d7d788d0 msgr2=0x55c7d7d771f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:06:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.288+0000 7f3d16dfd740 1 -- 172.21.15.111:0/1805513608 shutdown_connections 2024-09-15T23:06:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.288+0000 7f3d16dfd740 1 -- 172.21.15.111:0/1805513608 wait complete. 2024-09-15T23:06:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.288+0000 7f3d16dfd740 20 client.26585 trim_cache size 0 max 16384 2024-09-15T23:06:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:06:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:06:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:06:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:06:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.288+0000 7f3d12a00640 1 -- 172.21.15.111:0/1462449772 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3cf40a3240 msgr2=0x7f3cf40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.288+0000 7f3d12a00640 1 --2- 172.21.15.111:0/1462449772 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3cf40a3240 0x7f3cf40a5700 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x55c7d7d79750 tx=0x7f3d00005f90 comp rx=0 tx=0).stop 2024-09-15T23:06:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.288+0000 7f3d12a00640 1 -- 172.21.15.111:0/1462449772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d0c104cf0 msgr2=0x7f3d0c198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:06:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.288+0000 7f3d12a00640 1 --2- 172.21.15.111:0/1462449772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d0c104cf0 0x7f3d0c198290 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8031440 tx=0x7f3cf8032ce0 comp rx=0 tx=0).stop 2024-09-15T23:06:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.288+0000 7f3d12a00640 1 -- 172.21.15.111:0/1462449772 shutdown_connections 2024-09-15T23:06:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.288+0000 7f3d12a00640 1 --2- 172.21.15.111:0/1462449772 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f3cf40a3240 0x7f3cf40a5700 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.288+0000 7f3d12a00640 1 --2- 172.21.15.111:0/1462449772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d0c107620 0x7f3d0c19f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.288+0000 7f3d12a00640 1 --2- 172.21.15.111:0/1462449772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3d0c104cf0 0x7f3d0c198290 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.288+0000 7f3d12a00640 1 --2- 172.21.15.111:0/1462449772 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d0c1023c0 0x7f3d0c197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:06:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.288+0000 7f3d12a00640 1 -- 172.21.15.111:0/1462449772 >> 172.21.15.111:0/1462449772 conn(0x7f3d0c0fbfa0 msgr2=0x7f3d0c102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:06:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.288+0000 7f3d12a00640 1 -- 172.21.15.111:0/1462449772 shutdown_connections 2024-09-15T23:06:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:06:47.288+0000 7f3d12a00640 1 -- 172.21.15.111:0/1462449772 wait complete. 2024-09-15T23:06:47.677 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149372 17.29 MB/sec execute 315 sec latency 4183.875 ms 2024-09-15T23:06:48.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:47 smithi192 ceph-mon[35427]: pgmap v977: 97 pgs: 97 active+clean; 565 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.7 MiB/s wr, 373 op/s 2024-09-15T23:06:48.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:47 smithi121 ceph-mon[34815]: pgmap v977: 97 pgs: 97 active+clean; 565 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.7 MiB/s wr, 373 op/s 2024-09-15T23:06:48.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:47 smithi111 ceph-mon[30629]: pgmap v977: 97 pgs: 97 active+clean; 565 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.7 MiB/s wr, 373 op/s 2024-09-15T23:06:48.677 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149372 17.24 MB/sec execute 316 sec latency 5183.964 ms 2024-09-15T23:06:49.677 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149372 17.18 MB/sec execute 317 sec latency 6184.047 ms 2024-09-15T23:06:50.234 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:49 smithi111 ceph-mon[30629]: pgmap v978: 97 pgs: 97 active+clean; 558 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.0 MiB/s wr, 361 op/s 2024-09-15T23:06:50.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:49 smithi192 ceph-mon[35427]: pgmap v978: 97 pgs: 97 active+clean; 558 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.0 MiB/s wr, 361 op/s 2024-09-15T23:06:50.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:49 smithi121 ceph-mon[34815]: pgmap v978: 97 pgs: 97 active+clean; 558 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.0 MiB/s wr, 361 op/s 2024-09-15T23:06:50.677 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149372 17.13 MB/sec execute 318 sec latency 7184.113 ms 2024-09-15T23:06:51.677 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149372 17.08 MB/sec execute 319 sec latency 8184.187 ms 2024-09-15T23:06:52.196 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:51 smithi111 ceph-mon[30629]: pgmap v979: 97 pgs: 97 active+clean; 549 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 6.0 MiB/s wr, 273 op/s 2024-09-15T23:06:52.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:51 smithi192 ceph-mon[35427]: pgmap v979: 97 pgs: 97 active+clean; 549 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 6.0 MiB/s wr, 273 op/s 2024-09-15T23:06:52.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:51 smithi121 ceph-mon[34815]: pgmap v979: 97 pgs: 97 active+clean; 549 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 6.0 MiB/s wr, 273 op/s 2024-09-15T23:06:52.677 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149378 17.02 MB/sec execute 320 sec latency 9184.046 ms 2024-09-15T23:06:53.677 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149378 16.97 MB/sec execute 321 sec latency 10184.100 ms 2024-09-15T23:06:54.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:53 smithi111 ceph-mon[30629]: pgmap v980: 97 pgs: 97 active+clean; 547 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 5.7 MiB/s wr, 253 op/s 2024-09-15T23:06:54.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:53 smithi192 ceph-mon[35427]: pgmap v980: 97 pgs: 97 active+clean; 547 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 5.7 MiB/s wr, 253 op/s 2024-09-15T23:06:54.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:53 smithi121 ceph-mon[34815]: pgmap v980: 97 pgs: 97 active+clean; 547 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 5.7 MiB/s wr, 253 op/s 2024-09-15T23:06:54.677 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149378 16.92 MB/sec execute 322 sec latency 11184.153 ms 2024-09-15T23:06:55.677 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149378 16.87 MB/sec execute 323 sec latency 12184.208 ms 2024-09-15T23:06:56.131 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:55 smithi111 ceph-mon[30629]: pgmap v981: 97 pgs: 97 active+clean; 549 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 5.6 MiB/s wr, 252 op/s 2024-09-15T23:06:56.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:55 smithi192 ceph-mon[35427]: pgmap v981: 97 pgs: 97 active+clean; 549 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 5.6 MiB/s wr, 252 op/s 2024-09-15T23:06:56.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:55 smithi121 ceph-mon[34815]: pgmap v981: 97 pgs: 97 active+clean; 549 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 5.6 MiB/s wr, 252 op/s 2024-09-15T23:06:56.678 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149378 16.81 MB/sec execute 324 sec latency 13185.040 ms 2024-09-15T23:06:57.678 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149378 16.76 MB/sec execute 325 sec latency 14185.103 ms 2024-09-15T23:06:58.103 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:57 smithi111 ceph-mon[30629]: pgmap v982: 97 pgs: 97 active+clean; 560 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 5.5 MiB/s wr, 239 op/s 2024-09-15T23:06:58.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:57 smithi192 ceph-mon[35427]: pgmap v982: 97 pgs: 97 active+clean; 560 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 5.5 MiB/s wr, 239 op/s 2024-09-15T23:06:58.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:57 smithi121 ceph-mon[34815]: pgmap v982: 97 pgs: 97 active+clean; 560 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 5.5 MiB/s wr, 239 op/s 2024-09-15T23:06:58.678 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149380 16.71 MB/sec execute 326 sec latency 15185.151 ms 2024-09-15T23:06:59.678 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149380 16.66 MB/sec execute 327 sec latency 16185.195 ms 2024-09-15T23:07:00.107 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:59 smithi111 ceph-mon[30629]: pgmap v983: 97 pgs: 97 active+clean; 574 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 6.2 MiB/s wr, 232 op/s 2024-09-15T23:07:00.107 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:06:59 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:07:00.251 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:59 smithi192 ceph-mon[35427]: pgmap v983: 97 pgs: 97 active+clean; 574 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 6.2 MiB/s wr, 232 op/s 2024-09-15T23:07:00.251 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:06:59 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:07:00.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:59 smithi121 ceph-mon[34815]: pgmap v983: 97 pgs: 97 active+clean; 574 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 6.2 MiB/s wr, 232 op/s 2024-09-15T23:07:00.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:06:59 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:07:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.410+0000 7f1211800640 1 -- 172.21.15.111:0/2108969053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f120c107620 msgr2=0x7f120c109a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.410+0000 7f1211800640 1 --2- 172.21.15.111:0/2108969053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f120c107620 0x7f120c109a90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1204009be0 tx=0x7f120402a6d0 comp rx=0 tx=0).stop 2024-09-15T23:07:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.410+0000 7f1211800640 1 -- 172.21.15.111:0/2108969053 shutdown_connections 2024-09-15T23:07:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.410+0000 7f1211800640 1 --2- 172.21.15.111:0/2108969053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f120c107620 0x7f120c109a90 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.410+0000 7f1211800640 1 --2- 172.21.15.111:0/2108969053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f120c104cf0 0x7f120c1070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.410+0000 7f1211800640 1 --2- 172.21.15.111:0/2108969053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f120c1023c0 0x7f120c1047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.410+0000 7f1211800640 1 -- 172.21.15.111:0/2108969053 >> 172.21.15.111:0/2108969053 conn(0x7f120c0fbfa0 msgr2=0x7f120c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.410+0000 7f1211800640 1 -- 172.21.15.111:0/2108969053 shutdown_connections 2024-09-15T23:07:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.410+0000 7f1211800640 1 -- 172.21.15.111:0/2108969053 wait complete. 2024-09-15T23:07:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.413+0000 7f1211800640 1 Processor -- start 2024-09-15T23:07:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.413+0000 7f1211800640 1 -- start start 2024-09-15T23:07:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.413+0000 7f1211800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f120c1023c0 0x7f120c199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.413+0000 7f1211800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f120c104cf0 0x7f120c19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.413+0000 7f1211800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f120c107620 0x7f120c1a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.413+0000 7f1211800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f120c10d5e0 con 0x7f120c104cf0 2024-09-15T23:07:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.413+0000 7f1211800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f120c10d460 con 0x7f120c107620 2024-09-15T23:07:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.413+0000 7f1211800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f120c10d760 con 0x7f120c1023c0 2024-09-15T23:07:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.413+0000 7f120b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f120c104cf0 0x7f120c19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.413+0000 7f120b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f120c104cf0 0x7f120c19a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42934/0 (socket says 172.21.15.111:42934) 2024-09-15T23:07:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.413+0000 7f120b400640 1 -- 172.21.15.111:0/1126576666 learned_addr learned my addr 172.21.15.111:0/1126576666 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:07:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.413+0000 7f1210e00640 1 --2- 172.21.15.111:0/1126576666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f120c107620 0x7f120c1a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.413+0000 7f1210e00640 1 -- 172.21.15.111:0/1126576666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f120c1023c0 msgr2=0x7f120c199ea0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T23:07:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.413+0000 7f1210e00640 1 --2- 172.21.15.111:0/1126576666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f120c1023c0 0x7f120c199ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.413+0000 7f1210e00640 1 -- 172.21.15.111:0/1126576666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f120c104cf0 msgr2=0x7f120c19a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.413+0000 7f1210e00640 1 --2- 172.21.15.111:0/1126576666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f120c104cf0 0x7f120c19a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.413+0000 7f1210e00640 1 -- 172.21.15.111:0/1126576666 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120c1a1ac0 con 0x7f120c107620 2024-09-15T23:07:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.413+0000 7f120b400640 1 --2- 172.21.15.111:0/1126576666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f120c104cf0 0x7f120c19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:07:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.425+0000 7f1210e00640 1 --2- 172.21.15.111:0/1126576666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f120c107620 0x7f120c1a1450 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1204030960 tx=0x7f1204030640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.425+0000 7f1208c00640 1 -- 172.21.15.111:0/1126576666 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204041020 con 0x7f120c107620 2024-09-15T23:07:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.425+0000 7f1211800640 1 -- 172.21.15.111:0/1126576666 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f120c1a1d50 con 0x7f120c107620 2024-09-15T23:07:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.425+0000 7f1208c00640 1 -- 172.21.15.111:0/1126576666 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1204031920 con 0x7f120c107620 2024-09-15T23:07:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.425+0000 7f1208c00640 1 -- 172.21.15.111:0/1126576666 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204036040 con 0x7f120c107620 2024-09-15T23:07:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.425+0000 7f1211800640 1 -- 172.21.15.111:0/1126576666 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f120c1a2200 con 0x7f120c107620 2024-09-15T23:07:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.425+0000 7f1215dee740 1 Processor -- start 2024-09-15T23:07:00.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.425+0000 7f1208c00640 1 -- 172.21.15.111:0/1126576666 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f1204044020 con 0x7f120c107620 2024-09-15T23:07:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.430+0000 7f1215dee740 1 -- start start 2024-09-15T23:07:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.430+0000 7f1215dee740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604b8e2ea70 0x5604b8e2ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:00.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.430+0000 7f1215dee740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604b8e2f3b0 0x5604b8e33e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.430+0000 7f1215dee740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604b8e34440 0x5604b8e36900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.430+0000 7f1215dee740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5604b8d8b100 con 0x5604b8e2ea70 2024-09-15T23:07:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.430+0000 7f1215dee740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5604b8d331b0 con 0x5604b8e34440 2024-09-15T23:07:00.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.430+0000 7f1215dee740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5604b8d870c0 con 0x5604b8e2f3b0 2024-09-15T23:07:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.430+0000 7f1208c00640 1 --2- 172.21.15.111:0/1126576666 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f11f00a3340 0x7f11f00a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:00.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.430+0000 7f1208c00640 1 -- 172.21.15.111:0/1126576666 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f12040f0640 con 0x7f120c107620 2024-09-15T23:07:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.430+0000 7f1210e00640 1 --2- 172.21.15.111:0/1126576666 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f11f00a3340 0x7f11f00a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:00.434 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.430+0000 7f120b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604b8e2ea70 0x5604b8e2ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.430+0000 7f120b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604b8e2ea70 0x5604b8e2ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42946/0 (socket says 172.21.15.111:42946) 2024-09-15T23:07:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.430+0000 7f120b400640 1 -- 172.21.15.111:0/696650046 learned_addr learned my addr 172.21.15.111:0/696650046 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:07:00.435 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.430+0000 7f1210e00640 1 --2- 172.21.15.111:0/1126576666 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f11f00a3340 0x7f11f00a5800 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1204006010 tx=0x7f120402a6d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.430+0000 7f120b400640 1 -- 172.21.15.111:0/696650046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604b8e2f3b0 msgr2=0x5604b8e33e70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:07:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.430+0000 7f120b400640 1 --2- 172.21.15.111:0/696650046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604b8e2f3b0 0x5604b8e33e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.435+0000 7f120b400640 1 -- 172.21.15.111:0/696650046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604b8e34440 msgr2=0x5604b8e36900 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:07:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.435+0000 7f120b400640 1 --2- 172.21.15.111:0/696650046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604b8e34440 0x5604b8e36900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:00.436 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.435+0000 7f120b400640 1 -- 172.21.15.111:0/696650046 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604b8a9e210 con 0x5604b8e2ea70 2024-09-15T23:07:00.437 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.437+0000 7f120b400640 1 --2- 172.21.15.111:0/696650046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604b8e2ea70 0x5604b8e2ee70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f11f8011570 tx=0x7f11f8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8342ec51b048563e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.437+0000 7f1201600640 1 -- 172.21.15.111:0/696650046 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f801e070 con 0x5604b8e2ea70 2024-09-15T23:07:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.437+0000 7f1201600640 1 -- 172.21.15.111:0/696650046 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11f800fb20 con 0x5604b8e2ea70 2024-09-15T23:07:00.438 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.437+0000 7f1201600640 1 -- 172.21.15.111:0/696650046 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f8019ce0 con 0x5604b8e2ea70 2024-09-15T23:07:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.437+0000 7f1215dee740 1 -- 172.21.15.111:0/696650046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604b8e2ea70 msgr2=0x5604b8e2ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.437+0000 7f1215dee740 1 --2- 172.21.15.111:0/696650046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604b8e2ea70 0x5604b8e2ee70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f11f8011570 tx=0x7f11f8011a30 comp rx=0 tx=0).stop 2024-09-15T23:07:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.437+0000 7f1215dee740 1 -- 172.21.15.111:0/696650046 shutdown_connections 2024-09-15T23:07:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.437+0000 7f1215dee740 1 --2- 172.21.15.111:0/696650046 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604b8e34440 0x5604b8e36900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.437+0000 7f1215dee740 1 --2- 172.21.15.111:0/696650046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604b8e2f3b0 0x5604b8e33e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.437+0000 7f1215dee740 1 --2- 172.21.15.111:0/696650046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604b8e2ea70 0x5604b8e2ee70 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:00.440 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.437+0000 7f1215dee740 1 -- 172.21.15.111:0/696650046 >> 172.21.15.111:0/696650046 conn(0x5604b8e218d0 msgr2=0x5604b8e21cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.437+0000 7f1215dee740 1 -- 172.21.15.111:0/696650046 shutdown_connections 2024-09-15T23:07:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.437+0000 7f1215dee740 1 -- 172.21.15.111:0/696650046 wait complete. 2024-09-15T23:07:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.442+0000 7f1215dee740 1 -- start start 2024-09-15T23:07:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.442+0000 7f1215dee740 1 Processor -- start 2024-09-15T23:07:00.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.442+0000 7f1215dee740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604b8e2f3b0 0x5604b8e38ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.442+0000 7f1215dee740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604b8e34440 0x5604b8e39410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.442+0000 7f1215dee740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604b8e3e440 0x5604b8e40830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.442+0000 7f1215dee740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5604b8e37410 con 0x5604b8e2f3b0 2024-09-15T23:07:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.442+0000 7f1215dee740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5604b8d14090 con 0x5604b8e3e440 2024-09-15T23:07:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.442+0000 7f1215dee740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5604b8e37590 con 0x5604b8e34440 2024-09-15T23:07:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.442+0000 7f1215dee740 10 client.0 _ll_register_callbacks cb 0x5604b8d68f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:07:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.442+0000 7f120be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604b8e34440 0x5604b8e39410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.442+0000 7f120be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604b8e34440 0x5604b8e39410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:44716/0 (socket says 172.21.15.111:44716) 2024-09-15T23:07:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.442+0000 7f120be00640 1 -- 172.21.15.111:0/2573112237 learned_addr learned my addr 172.21.15.111:0/2573112237 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:07:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.442+0000 7f120b400640 1 --2- 172.21.15.111:0/2573112237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604b8e3e440 0x5604b8e40830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.442+0000 7f120b400640 1 --2- 172.21.15.111:0/2573112237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604b8e2f3b0 0x5604b8e38ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.442+0000 7f120be00640 1 -- 172.21.15.111:0/2573112237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604b8e3e440 msgr2=0x5604b8e40830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.442+0000 7f120be00640 1 --2- 172.21.15.111:0/2573112237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604b8e3e440 0x5604b8e40830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.442+0000 7f120be00640 1 -- 172.21.15.111:0/2573112237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604b8e2f3b0 msgr2=0x5604b8e38ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.442+0000 7f120be00640 1 --2- 172.21.15.111:0/2573112237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604b8e2f3b0 0x5604b8e38ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.442+0000 7f120be00640 1 -- 172.21.15.111:0/2573112237 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604b8b6a620 con 0x5604b8e34440 2024-09-15T23:07:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.442+0000 7f120be00640 1 --2- 172.21.15.111:0/2573112237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604b8e34440 0x5604b8e39410 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00e350 tx=0x7f11fc00e810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bc783511fe11565 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.442+0000 7f11f6000640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:07:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.442+0000 7f11f6000640 1 -- 172.21.15.111:0/2573112237 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc004230 con 0x5604b8e34440 2024-09-15T23:07:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.447+0000 7f11f6000640 1 -- 172.21.15.111:0/2573112237 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f11fc0043d0 con 0x5604b8e34440 2024-09-15T23:07:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.447+0000 7f1215dee740 1 -- 172.21.15.111:0/2573112237 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11f800fb20 con 0x5604b8e34440 2024-09-15T23:07:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.447+0000 7f11f6000640 1 -- 172.21.15.111:0/2573112237 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f11fc010700 con 0x5604b8e34440 2024-09-15T23:07:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.447+0000 7f1215dee740 10 client.26587 fetch_fsmap learned FSMap version 35 2024-09-15T23:07:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.447+0000 7f1215dee740 1 -- 172.21.15.111:0/2573112237 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5604b8e27ab0 con 0x5604b8e34440 2024-09-15T23:07:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.447+0000 7f11f6000640 1 -- 172.21.15.111:0/2573112237 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11f800fb20 con 0x5604b8e34440 2024-09-15T23:07:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.447+0000 7f11f6000640 1 -- 172.21.15.111:0/2573112237 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f11fc003c60 con 0x5604b8e34440 2024-09-15T23:07:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.447+0000 7f1215dee740 10 client.26587 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:07:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.447+0000 7f1215dee740 10 client.26587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:07:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.447+0000 7f1215dee740 20 client.26587 populate_metadata read hostname 'smithi111' 2024-09-15T23:07:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.447+0000 7f1215dee740 1 --2- 172.21.15.111:0/2573112237 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5604b8e29180 0x5604b8e2b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.447+0000 7f1215dee740 4 client.26587 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:07:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.447+0000 7f1215dee740 1 -- 172.21.15.111:0/2573112237 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604b8d98830 con 0x5604b8e29180 2024-09-15T23:07:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.447+0000 7f120b400640 1 --2- 172.21.15.111:0/2573112237 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5604b8e29180 0x5604b8e2b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.447+0000 7f120b400640 1 --2- 172.21.15.111:0/2573112237 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5604b8e29180 0x5604b8e2b590 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c44af9511b6096e server_cookie=3e43bf73aa8e76b3 in_seq=0 out_seq=0 2024-09-15T23:07:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.447+0000 7f11f6000640 10 client.26587 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:07:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.447+0000 7f11f6000640 1 -- 172.21.15.111:0/2573112237 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5604b8d98830 con 0x5604b8e29180 2024-09-15T23:07:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.447+0000 7f11f6000640 10 client.26587 handle_command_reply: tid=0 2024-09-15T23:07:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.447+0000 7f1215dee740 1 client.26587 shutdown 2024-09-15T23:07:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.452+0000 7f1215dee740 1 -- 172.21.15.111:0/2573112237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604b8e34440 msgr2=0x5604b8e39410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.452+0000 7f1215dee740 1 --2- 172.21.15.111:0/2573112237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604b8e34440 0x5604b8e39410 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00e350 tx=0x7f11fc00e810 comp rx=0 tx=0).stop 2024-09-15T23:07:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.452+0000 7f1215dee740 1 -- 172.21.15.111:0/2573112237 shutdown_connections 2024-09-15T23:07:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.452+0000 7f1215dee740 1 --2- 172.21.15.111:0/2573112237 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5604b8e29180 0x5604b8e2b590 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.452+0000 7f1215dee740 1 --2- 172.21.15.111:0/2573112237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604b8e3e440 0x5604b8e40830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.452+0000 7f11f6000640 0 client.26587 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:07:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.452+0000 7f1215dee740 1 --2- 172.21.15.111:0/2573112237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604b8e34440 0x5604b8e39410 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.452+0000 7f1215dee740 1 --2- 172.21.15.111:0/2573112237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604b8e2f3b0 0x5604b8e38ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.452+0000 7f1215dee740 1 -- 172.21.15.111:0/2573112237 >> 172.21.15.111:0/2573112237 conn(0x5604b8e218d0 msgr2=0x5604b8e230e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.452+0000 7f1215dee740 1 -- 172.21.15.111:0/2573112237 shutdown_connections 2024-09-15T23:07:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.452+0000 7f1215dee740 1 -- 172.21.15.111:0/2573112237 wait complete. 2024-09-15T23:07:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.457+0000 7f1215dee740 20 client.26587 trim_cache size 0 max 16384 2024-09-15T23:07:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.489+0000 7f1215dee740 1 Processor -- start 2024-09-15T23:07:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.489+0000 7f1215dee740 1 -- start start 2024-09-15T23:07:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.489+0000 7f1215dee740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604b8e2f3b0 0x5604b8e227b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.489+0000 7f1215dee740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604b8e34440 0x5604b8e22cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.489+0000 7f1215dee740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604b8e3e440 0x5604b8e23230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.489+0000 7f1215dee740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f120c1a9430 con 0x5604b8e2f3b0 2024-09-15T23:07:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.489+0000 7f1215dee740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f120c1a6d00 con 0x5604b8e34440 2024-09-15T23:07:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.489+0000 7f1215dee740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f120c1a4b10 con 0x5604b8e3e440 2024-09-15T23:07:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.496+0000 7f120be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604b8e34440 0x5604b8e22cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.496+0000 7f120be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604b8e34440 0x5604b8e22cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:36670/0 (socket says 172.21.15.111:36670) 2024-09-15T23:07:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.496+0000 7f120be00640 1 -- 172.21.15.111:0/3373072228 learned_addr learned my addr 172.21.15.111:0/3373072228 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:07:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.496+0000 7f120b400640 1 --2- 172.21.15.111:0/3373072228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604b8e3e440 0x5604b8e23230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.496+0000 7f120be00640 1 -- 172.21.15.111:0/3373072228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604b8e3e440 msgr2=0x5604b8e23230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.496+0000 7f120be00640 1 --2- 172.21.15.111:0/3373072228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604b8e3e440 0x5604b8e23230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.496+0000 7f120be00640 1 -- 172.21.15.111:0/3373072228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604b8e2f3b0 msgr2=0x5604b8e227b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.496+0000 7f120be00640 1 --2- 172.21.15.111:0/3373072228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604b8e2f3b0 0x5604b8e227b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.496+0000 7f120be00640 1 -- 172.21.15.111:0/3373072228 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604b8e40ba0 con 0x5604b8e34440 2024-09-15T23:07:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.496+0000 7f120be00640 1 --2- 172.21.15.111:0/3373072228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604b8e34440 0x5604b8e22cf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f11fc005bc0 tx=0x7f11fc005cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bcef8bf731e4408 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.496+0000 7f1200c00640 1 -- 172.21.15.111:0/3373072228 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc010700 con 0x5604b8e34440 2024-09-15T23:07:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.496+0000 7f1200c00640 1 -- 172.21.15.111:0/3373072228 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11fc005450 con 0x5604b8e34440 2024-09-15T23:07:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.496+0000 7f1200c00640 1 -- 172.21.15.111:0/3373072228 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc018420 con 0x5604b8e34440 2024-09-15T23:07:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.496+0000 7f1215dee740 1 -- 172.21.15.111:0/3373072228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604b8e34440 msgr2=0x5604b8e22cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.496+0000 7f1215dee740 1 --2- 172.21.15.111:0/3373072228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604b8e34440 0x5604b8e22cf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f11fc005bc0 tx=0x7f11fc005cc0 comp rx=0 tx=0).stop 2024-09-15T23:07:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.501+0000 7f1215dee740 1 -- 172.21.15.111:0/3373072228 shutdown_connections 2024-09-15T23:07:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.501+0000 7f1215dee740 1 --2- 172.21.15.111:0/3373072228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604b8e3e440 0x5604b8e23230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.501+0000 7f1215dee740 1 --2- 172.21.15.111:0/3373072228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604b8e34440 0x5604b8e22cf0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.501+0000 7f1215dee740 1 --2- 172.21.15.111:0/3373072228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604b8e2f3b0 0x5604b8e227b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.501+0000 7f1215dee740 1 -- 172.21.15.111:0/3373072228 >> 172.21.15.111:0/3373072228 conn(0x5604b8e218d0 msgr2=0x5604b8e30770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.501+0000 7f1215dee740 1 -- 172.21.15.111:0/3373072228 shutdown_connections 2024-09-15T23:07:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.501+0000 7f1215dee740 1 -- 172.21.15.111:0/3373072228 wait complete. 2024-09-15T23:07:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.501+0000 7f1215dee740 1 -- start start 2024-09-15T23:07:00.505 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.501+0000 7f1215dee740 1 Processor -- start 2024-09-15T23:07:00.506 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.501+0000 7f1215dee740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604b8e2f3b0 0x5604b8e64100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.506+0000 7f1215dee740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604b8e34440 0x5604b8e60ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.506+0000 7f1215dee740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604b8e3e440 0x5604b8e610e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.506+0000 7f1215dee740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f120c1a51c0 con 0x5604b8e2f3b0 2024-09-15T23:07:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.506+0000 7f1215dee740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f120c1a6630 con 0x5604b8e34440 2024-09-15T23:07:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.506+0000 7f1215dee740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f120c1a5890 con 0x5604b8e3e440 2024-09-15T23:07:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.506+0000 7f1215dee740 10 client.0 _ll_register_callbacks cb 0x7f11fc009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:07:00.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.508+0000 7f120be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604b8e34440 0x5604b8e60ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.508+0000 7f120be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604b8e34440 0x5604b8e60ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:36684/0 (socket says 172.21.15.111:36684) 2024-09-15T23:07:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.508+0000 7f120b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604b8e2f3b0 0x5604b8e64100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.508+0000 7f120b400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604b8e2f3b0 0x5604b8e64100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:42976/0 (socket says 172.21.15.111:42976) 2024-09-15T23:07:00.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.508+0000 7f120b400640 1 -- 172.21.15.111:0/721170808 learned_addr learned my addr 172.21.15.111:0/721170808 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:07:00.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.508+0000 7f120be00640 1 -- 172.21.15.111:0/721170808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604b8e3e440 msgr2=0x5604b8e610e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:07:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.508+0000 7f120be00640 1 --2- 172.21.15.111:0/721170808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604b8e3e440 0x5604b8e610e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.508+0000 7f120be00640 1 -- 172.21.15.111:0/721170808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604b8e2f3b0 msgr2=0x5604b8e64100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.508+0000 7f120be00640 1 --2- 172.21.15.111:0/721170808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604b8e2f3b0 0x5604b8e64100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.508+0000 7f120be00640 1 -- 172.21.15.111:0/721170808 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604b8eb6a60 con 0x5604b8e34440 2024-09-15T23:07:00.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.508+0000 7f120be00640 1 --2- 172.21.15.111:0/721170808 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604b8e34440 0x5604b8e60ba0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x5604b8e39bd0 tx=0x7f11fc005600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4ba60109d0afcd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.513+0000 7f11f6000640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:07:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.513+0000 7f11f6000640 1 -- 172.21.15.111:0/721170808 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc021710 con 0x5604b8e34440 2024-09-15T23:07:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.513+0000 7f11f6000640 1 -- 172.21.15.111:0/721170808 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f11fc0218b0 con 0x5604b8e34440 2024-09-15T23:07:00.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.513+0000 7f1215dee740 1 -- 172.21.15.111:0/721170808 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11fc005450 con 0x5604b8e34440 2024-09-15T23:07:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.513+0000 7f11f6000640 1 -- 172.21.15.111:0/721170808 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f11fc021b10 con 0x5604b8e34440 2024-09-15T23:07:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.513+0000 7f1215dee740 10 client.26600 fetch_fsmap learned FSMap version 35 2024-09-15T23:07:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.513+0000 7f1215dee740 1 -- 172.21.15.111:0/721170808 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5604b8eb7cb0 con 0x5604b8e34440 2024-09-15T23:07:00.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.513+0000 7f11f6000640 1 -- 172.21.15.111:0/721170808 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11fc005450 con 0x5604b8e34440 2024-09-15T23:07:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.513+0000 7f11f6000640 1 -- 172.21.15.111:0/721170808 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f11fc018420 con 0x5604b8e34440 2024-09-15T23:07:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.513+0000 7f1215dee740 10 client.26600 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:07:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.513+0000 7f1215dee740 10 client.26600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:07:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.513+0000 7f1215dee740 20 client.26600 populate_metadata read hostname 'smithi111' 2024-09-15T23:07:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.513+0000 7f1215dee740 1 --2- 172.21.15.111:0/721170808 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5604b8eb84f0 0x5604b8eba8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.513+0000 7f1215dee740 4 client.26600 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:07:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.513+0000 7f1215dee740 1 -- 172.21.15.111:0/721170808 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5604b8d92b80 con 0x5604b8eb84f0 2024-09-15T23:07:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.513+0000 7f120b400640 1 --2- 172.21.15.111:0/721170808 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5604b8eb84f0 0x5604b8eba8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.513+0000 7f120b400640 1 --2- 172.21.15.111:0/721170808 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5604b8eb84f0 0x5604b8eba8e0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebf64cc1be7cf89b server_cookie=b7bf0eafe1c1188b in_seq=0 out_seq=0 2024-09-15T23:07:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:00.513+0000 7f11f6000640 10 client.26600 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:07:00.678 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149380 16.61 MB/sec execute 328 sec latency 17185.271 ms 2024-09-15T23:07:01.678 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149380 16.56 MB/sec execute 329 sec latency 18185.321 ms 2024-09-15T23:07:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:02.110+0000 7f11f6000640 1 -- 172.21.15.111:0/721170808 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5604b8d92b80 con 0x5604b8eb84f0 2024-09-15T23:07:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:02.110+0000 7f11f6000640 10 client.26600 handle_command_reply: tid=0 2024-09-15T23:07:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:02.110+0000 7f1215dee740 1 client.26600 shutdown 2024-09-15T23:07:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:02.110+0000 7f1215dee740 1 -- 172.21.15.111:0/721170808 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604b8e34440 msgr2=0x5604b8e60ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:02.110+0000 7f1215dee740 1 --2- 172.21.15.111:0/721170808 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604b8e34440 0x5604b8e60ba0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x5604b8e39bd0 tx=0x7f11fc005600 comp rx=0 tx=0).stop 2024-09-15T23:07:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:02.110+0000 7f1215dee740 1 -- 172.21.15.111:0/721170808 shutdown_connections 2024-09-15T23:07:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:02.110+0000 7f1215dee740 1 --2- 172.21.15.111:0/721170808 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5604b8eb84f0 0x5604b8eba8e0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:02.110+0000 7f1215dee740 1 --2- 172.21.15.111:0/721170808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5604b8e3e440 0x5604b8e610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:02.110+0000 7f1215dee740 1 --2- 172.21.15.111:0/721170808 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5604b8e34440 0x5604b8e60ba0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:02.110+0000 7f11f6000640 0 client.26600 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:07:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:02.110+0000 7f1215dee740 1 --2- 172.21.15.111:0/721170808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5604b8e2f3b0 0x5604b8e64100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:02.110+0000 7f1215dee740 1 -- 172.21.15.111:0/721170808 >> 172.21.15.111:0/721170808 conn(0x5604b8e218d0 msgr2=0x5604b8e201f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:02.110+0000 7f1215dee740 1 -- 172.21.15.111:0/721170808 shutdown_connections 2024-09-15T23:07:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:02.110+0000 7f1215dee740 1 -- 172.21.15.111:0/721170808 wait complete. 2024-09-15T23:07:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:02.110+0000 7f1215dee740 20 client.26600 trim_cache size 0 max 16384 2024-09-15T23:07:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:07:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:07:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:07:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:07:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:02.110+0000 7f1211800640 1 -- 172.21.15.111:0/1126576666 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f11f00a3340 msgr2=0x7f11f00a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:02.110+0000 7f1211800640 1 --2- 172.21.15.111:0/1126576666 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f11f00a3340 0x7f11f00a5800 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1204006010 tx=0x7f120402a6d0 comp rx=0 tx=0).stop 2024-09-15T23:07:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:02.110+0000 7f1211800640 1 -- 172.21.15.111:0/1126576666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f120c107620 msgr2=0x7f120c1a1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:02.110+0000 7f1211800640 1 --2- 172.21.15.111:0/1126576666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f120c107620 0x7f120c1a1450 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1204030960 tx=0x7f1204030640 comp rx=0 tx=0).stop 2024-09-15T23:07:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:02.110+0000 7f1211800640 1 -- 172.21.15.111:0/1126576666 shutdown_connections 2024-09-15T23:07:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:02.110+0000 7f1211800640 1 --2- 172.21.15.111:0/1126576666 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f11f00a3340 0x7f11f00a5800 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:02.110+0000 7f1211800640 1 --2- 172.21.15.111:0/1126576666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f120c107620 0x7f120c1a1450 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:02.110+0000 7f1211800640 1 --2- 172.21.15.111:0/1126576666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f120c104cf0 0x7f120c19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:02.110+0000 7f1211800640 1 --2- 172.21.15.111:0/1126576666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f120c1023c0 0x7f120c199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:02.110+0000 7f1211800640 1 -- 172.21.15.111:0/1126576666 >> 172.21.15.111:0/1126576666 conn(0x7f120c0fbfa0 msgr2=0x7f120c1019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:02.110+0000 7f1211800640 1 -- 172.21.15.111:0/1126576666 shutdown_connections 2024-09-15T23:07:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:02.110+0000 7f1211800640 1 -- 172.21.15.111:0/1126576666 wait complete. 2024-09-15T23:07:02.129 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:01 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:07:02.129 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:01 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:07:02.129 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:01 smithi111 ceph-mon[30629]: pgmap v984: 97 pgs: 97 active+clean; 576 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 6.0 MiB/s wr, 197 op/s 2024-09-15T23:07:02.129 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:01 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:07:02.130 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:01 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:07:02.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:01 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:07:02.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:01 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:07:02.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:01 smithi192 ceph-mon[35427]: pgmap v984: 97 pgs: 97 active+clean; 576 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 6.0 MiB/s wr, 197 op/s 2024-09-15T23:07:02.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:01 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:07:02.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:01 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:07:02.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:01 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:07:02.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:01 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:07:02.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:01 smithi121 ceph-mon[34815]: pgmap v984: 97 pgs: 97 active+clean; 576 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 6.0 MiB/s wr, 197 op/s 2024-09-15T23:07:02.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:01 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:07:02.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:01 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:07:02.678 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149386 16.51 MB/sec execute 330 sec latency 19185.374 ms 2024-09-15T23:07:02.978 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:02 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:07:02.978 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:02 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:07:03.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:02 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:07:03.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:02 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:07:03.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:02 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:07:03.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:02 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:07:03.678 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149386 16.46 MB/sec execute 331 sec latency 20185.419 ms 2024-09-15T23:07:04.097 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:03 smithi111 ceph-mon[30629]: pgmap v985: 97 pgs: 97 active+clean; 573 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 5.2 MiB/s wr, 160 op/s 2024-09-15T23:07:04.098 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:03 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:07:04.098 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:03 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:07:04.098 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:03 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:07:04.098 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:03 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:07:04.098 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:03 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:07:04.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:03 smithi192 ceph-mon[35427]: pgmap v985: 97 pgs: 97 active+clean; 573 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 5.2 MiB/s wr, 160 op/s 2024-09-15T23:07:04.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:03 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:07:04.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:03 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:07:04.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:03 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:07:04.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:03 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:07:04.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:03 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:07:04.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:03 smithi121 ceph-mon[34815]: pgmap v985: 97 pgs: 97 active+clean; 573 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 5.2 MiB/s wr, 160 op/s 2024-09-15T23:07:04.398 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:03 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:07:04.398 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:03 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:07:04.398 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:03 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:07:04.398 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:03 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:07:04.398 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:03 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:07:04.678 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149386 16.41 MB/sec execute 332 sec latency 21185.462 ms 2024-09-15T23:07:05.678 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149415 16.38 MB/sec execute 333 sec latency 22043.241 ms 2024-09-15T23:07:06.091 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:05 smithi111 ceph-mon[30629]: pgmap v986: 97 pgs: 97 active+clean; 562 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 5.6 MiB/s wr, 183 op/s 2024-09-15T23:07:06.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:05 smithi192 ceph-mon[35427]: pgmap v986: 97 pgs: 97 active+clean; 562 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 5.6 MiB/s wr, 183 op/s 2024-09-15T23:07:06.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:05 smithi121 ceph-mon[34815]: pgmap v986: 97 pgs: 97 active+clean; 562 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 5.6 MiB/s wr, 183 op/s 2024-09-15T23:07:06.678 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149430 16.34 MB/sec execute 334 sec latency 16456.097 ms 2024-09-15T23:07:07.678 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149430 16.29 MB/sec execute 335 sec latency 17236.698 ms 2024-09-15T23:07:08.168 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:07 smithi111 ceph-mon[30629]: pgmap v987: 97 pgs: 97 active+clean; 558 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 5.8 MiB/s wr, 200 op/s 2024-09-15T23:07:08.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:07 smithi192 ceph-mon[35427]: pgmap v987: 97 pgs: 97 active+clean; 558 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 5.8 MiB/s wr, 200 op/s 2024-09-15T23:07:08.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:07 smithi121 ceph-mon[34815]: pgmap v987: 97 pgs: 97 active+clean; 558 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 5.8 MiB/s wr, 200 op/s 2024-09-15T23:07:08.678 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149430 16.24 MB/sec execute 336 sec latency 6910.405 ms 2024-09-15T23:07:09.679 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149430 16.19 MB/sec execute 337 sec latency 7910.472 ms 2024-09-15T23:07:10.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:09 smithi192 ceph-mon[35427]: pgmap v988: 97 pgs: 97 active+clean; 547 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 6.1 MiB/s wr, 232 op/s 2024-09-15T23:07:10.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:09 smithi121 ceph-mon[34815]: pgmap v988: 97 pgs: 97 active+clean; 547 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 6.1 MiB/s wr, 232 op/s 2024-09-15T23:07:10.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:09 smithi111 ceph-mon[30629]: pgmap v988: 97 pgs: 97 active+clean; 547 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 6.1 MiB/s wr, 232 op/s 2024-09-15T23:07:10.679 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149430 16.14 MB/sec execute 338 sec latency 8910.539 ms 2024-09-15T23:07:11.679 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149430 16.10 MB/sec execute 339 sec latency 9910.623 ms 2024-09-15T23:07:12.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:11 smithi192 ceph-mon[35427]: pgmap v989: 97 pgs: 97 active+clean; 541 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 5.4 MiB/s wr, 231 op/s 2024-09-15T23:07:12.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:11 smithi121 ceph-mon[34815]: pgmap v989: 97 pgs: 97 active+clean; 541 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 5.4 MiB/s wr, 231 op/s 2024-09-15T23:07:12.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:11 smithi111 ceph-mon[30629]: pgmap v989: 97 pgs: 97 active+clean; 541 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 5.4 MiB/s wr, 231 op/s 2024-09-15T23:07:12.679 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149430 16.05 MB/sec execute 340 sec latency 10910.706 ms 2024-09-15T23:07:13.679 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149430 16.00 MB/sec execute 341 sec latency 11910.758 ms 2024-09-15T23:07:14.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:13 smithi192 ceph-mon[35427]: pgmap v990: 97 pgs: 97 active+clean; 536 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 4.3 MiB/s wr, 201 op/s 2024-09-15T23:07:14.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:13 smithi121 ceph-mon[34815]: pgmap v990: 97 pgs: 97 active+clean; 536 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 4.3 MiB/s wr, 201 op/s 2024-09-15T23:07:14.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:13 smithi111 ceph-mon[30629]: pgmap v990: 97 pgs: 97 active+clean; 536 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 4.3 MiB/s wr, 201 op/s 2024-09-15T23:07:14.679 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149430 15.95 MB/sec execute 342 sec latency 12910.811 ms 2024-09-15T23:07:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c8c400640 1 -- 172.21.15.111:0/2107232458 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c84105910 msgr2=0x7f8c8410c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c8c400640 1 --2- 172.21.15.111:0/2107232458 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c84105910 0x7f8c8410c1a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009ca0 tx=0x7f8c7c02a8c0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c8c400640 1 -- 172.21.15.111:0/2107232458 shutdown_connections 2024-09-15T23:07:15.227 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c8c400640 1 --2- 172.21.15.111:0/2107232458 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c84105910 0x7f8c8410c1a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c8c400640 1 --2- 172.21.15.111:0/2107232458 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c84104f50 0x7f8c841053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c8c400640 1 --2- 172.21.15.111:0/2107232458 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c84103d60 0x7f8c84104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c8c400640 1 -- 172.21.15.111:0/2107232458 >> 172.21.15.111:0/2107232458 conn(0x7f8c840ff4e0 msgr2=0x7f8c84101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c8c400640 1 -- 172.21.15.111:0/2107232458 shutdown_connections 2024-09-15T23:07:15.228 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c8c400640 1 -- 172.21.15.111:0/2107232458 wait complete. 2024-09-15T23:07:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c8c400640 1 Processor -- start 2024-09-15T23:07:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c8c400640 1 -- start start 2024-09-15T23:07:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c8c400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c84103d60 0x7f8c84199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c8c400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c84104f50 0x7f8c8419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c8c400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c84105910 0x7f8c841a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c8c400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8410f9e0 con 0x7f8c84103d60 2024-09-15T23:07:15.230 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c8c400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8410f860 con 0x7f8c84104f50 2024-09-15T23:07:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c8c400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8410fb60 con 0x7f8c84105910 2024-09-15T23:07:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c89a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c84103d60 0x7f8c84199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c89000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c84104f50 0x7f8c8419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c89a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c84103d60 0x7f8c84199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56614/0 (socket says 172.21.15.111:56614) 2024-09-15T23:07:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c89a00640 1 -- 172.21.15.111:0/3888810723 learned_addr learned my addr 172.21.15.111:0/3888810723 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:07:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c8a400640 1 --2- 172.21.15.111:0/3888810723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c84105910 0x7f8c841a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c89000640 1 -- 172.21.15.111:0/3888810723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c84105910 msgr2=0x7f8c841a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c89000640 1 --2- 172.21.15.111:0/3888810723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c84105910 0x7f8c841a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c89000640 1 -- 172.21.15.111:0/3888810723 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c84103d60 msgr2=0x7f8c84199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c89000640 1 --2- 172.21.15.111:0/3888810723 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c84103d60 0x7f8c84199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c89000640 1 -- 172.21.15.111:0/3888810723 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c841a1bf0 con 0x7f8c84104f50 2024-09-15T23:07:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c89000640 1 --2- 172.21.15.111:0/3888810723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c84104f50 0x7f8c8419a510 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000e9f0 tx=0x7f8c7000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c82000640 1 -- 172.21.15.111:0/3888810723 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7000cd80 con 0x7f8c84104f50 2024-09-15T23:07:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c82000640 1 -- 172.21.15.111:0/3888810723 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c700035f0 con 0x7f8c84104f50 2024-09-15T23:07:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c8c400640 1 -- 172.21.15.111:0/3888810723 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c841a1ee0 con 0x7f8c84104f50 2024-09-15T23:07:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c82000640 1 -- 172.21.15.111:0/3888810723 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c70010690 con 0x7f8c84104f50 2024-09-15T23:07:15.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.224+0000 7f8c8c400640 1 -- 172.21.15.111:0/3888810723 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c841a23a0 con 0x7f8c84104f50 2024-09-15T23:07:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c8f2aa740 1 Processor -- start 2024-09-15T23:07:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c8f2aa740 1 -- start start 2024-09-15T23:07:15.233 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c8f2aa740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f283910090 0x55f283910490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:15.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c8f2aa740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2839109d0 0x55f283915490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c8f2aa740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f283915a60 0x55f283917f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c82000640 1 -- 172.21.15.111:0/3888810723 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f8c70003b30 con 0x7f8c84104f50 2024-09-15T23:07:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c8f2aa740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f283875100 con 0x55f2839109d0 2024-09-15T23:07:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c8f2aa740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f28381d1b0 con 0x55f283915a60 2024-09-15T23:07:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c8f2aa740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f2838710c0 con 0x55f283910090 2024-09-15T23:07:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c8a400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f283910090 0x55f283910490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c8a400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f283910090 0x55f283910490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:56212/0 (socket says 172.21.15.111:56212) 2024-09-15T23:07:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c8a400640 1 -- 172.21.15.111:0/1546885607 learned_addr learned my addr 172.21.15.111:0/1546885607 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:07:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c89000640 1 --2- 172.21.15.111:0/1546885607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f283915a60 0x55f283917f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c89a00640 1 --2- 172.21.15.111:0/1546885607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2839109d0 0x55f283915490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c8a400640 1 -- 172.21.15.111:0/1546885607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f283915a60 msgr2=0x55f283917f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c8a400640 1 --2- 172.21.15.111:0/1546885607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f283915a60 0x55f283917f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c8a400640 1 -- 172.21.15.111:0/1546885607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2839109d0 msgr2=0x55f283915490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c8a400640 1 --2- 172.21.15.111:0/1546885607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2839109d0 0x55f283915490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c8a400640 1 -- 172.21.15.111:0/1546885607 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f283588210 con 0x55f283910090 2024-09-15T23:07:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c8a400640 1 --2- 172.21.15.111:0/1546885607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f283910090 0x55f283910490 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c035000 tx=0x7f8c7c005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aab91bbe12dae21 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c76a00640 1 -- 172.21.15.111:0/1546885607 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c031cb0 con 0x55f283910090 2024-09-15T23:07:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c76a00640 1 -- 172.21.15.111:0/1546885607 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c7c036040 con 0x55f283910090 2024-09-15T23:07:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c76a00640 1 -- 172.21.15.111:0/1546885607 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c004300 con 0x55f283910090 2024-09-15T23:07:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c82000640 1 --2- 172.21.15.111:0/3888810723 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8c6c0a32f0 0x7f8c6c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c82000640 1 -- 172.21.15.111:0/3888810723 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f8c700cfbe0 con 0x7f8c84104f50 2024-09-15T23:07:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c89a00640 1 --2- 172.21.15.111:0/3888810723 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8c6c0a32f0 0x7f8c6c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c8f2aa740 1 -- 172.21.15.111:0/1546885607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f283910090 msgr2=0x55f283910490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c8f2aa740 1 --2- 172.21.15.111:0/1546885607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f283910090 0x55f283910490 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c035000 tx=0x7f8c7c005dc0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c8f2aa740 1 -- 172.21.15.111:0/1546885607 shutdown_connections 2024-09-15T23:07:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c8f2aa740 1 --2- 172.21.15.111:0/1546885607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f283915a60 0x55f283917f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c8f2aa740 1 --2- 172.21.15.111:0/1546885607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2839109d0 0x55f283915490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c8f2aa740 1 --2- 172.21.15.111:0/1546885607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f283910090 0x55f283910490 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.230+0000 7f8c8f2aa740 1 -- 172.21.15.111:0/1546885607 >> 172.21.15.111:0/1546885607 conn(0x55f28390b700 msgr2=0x55f28390bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8f2aa740 1 -- 172.21.15.111:0/1546885607 shutdown_connections 2024-09-15T23:07:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8f2aa740 1 -- 172.21.15.111:0/1546885607 wait complete. 2024-09-15T23:07:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c89a00640 1 --2- 172.21.15.111:0/3888810723 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8c6c0a32f0 0x7f8c6c0a57b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55f28390c580 tx=0x7f8c78005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8f2aa740 1 -- start start 2024-09-15T23:07:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8f2aa740 1 Processor -- start 2024-09-15T23:07:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8f2aa740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f283910090 0x55f28391a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8f2aa740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2839109d0 0x55f28391a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8f2aa740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f283915a60 0x55f283921510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8f2aa740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f2839187f0 con 0x55f283915a60 2024-09-15T23:07:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8f2aa740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f2837fe090 con 0x55f283910090 2024-09-15T23:07:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8f2aa740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f283918970 con 0x55f2839109d0 2024-09-15T23:07:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8f2aa740 10 client.0 _ll_register_callbacks cb 0x55f283852f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:07:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8a400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f283915a60 0x55f283921510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8a400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f283915a60 0x55f283921510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56630/0 (socket says 172.21.15.111:56630) 2024-09-15T23:07:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c89a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f283910090 0x55f28391a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8a400640 1 -- 172.21.15.111:0/4171406712 learned_addr learned my addr 172.21.15.111:0/4171406712 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:07:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c89000640 1 --2- 172.21.15.111:0/4171406712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2839109d0 0x55f28391a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8a400640 1 -- 172.21.15.111:0/4171406712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2839109d0 msgr2=0x55f28391a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8a400640 1 --2- 172.21.15.111:0/4171406712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2839109d0 0x55f28391a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8a400640 1 -- 172.21.15.111:0/4171406712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f283910090 msgr2=0x55f28391a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8a400640 1 --2- 172.21.15.111:0/4171406712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f283910090 0x55f28391a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8a400640 1 -- 172.21.15.111:0/4171406712 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f283654620 con 0x55f283915a60 2024-09-15T23:07:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c89000640 1 --2- 172.21.15.111:0/4171406712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2839109d0 0x55f28391a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:07:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c89a00640 1 --2- 172.21.15.111:0/4171406712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f283910090 0x55f28391a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:07:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8a400640 1 --2- 172.21.15.111:0/4171406712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f283915a60 0x55f283921510 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c005e30 tx=0x7f8c7c031930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab00e58863b9ef8e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c6b400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:07:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c6b400640 1 -- 172.21.15.111:0/4171406712 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c05d070 con 0x55f283915a60 2024-09-15T23:07:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c6b400640 1 -- 172.21.15.111:0/4171406712 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f8c7c0042a0 con 0x55f283915a60 2024-09-15T23:07:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8f2aa740 1 -- 172.21.15.111:0/4171406712 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c7c036040 con 0x55f283915a60 2024-09-15T23:07:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c6b400640 1 -- 172.21.15.111:0/4171406712 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8c7c069050 con 0x55f283915a60 2024-09-15T23:07:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8f2aa740 10 client.16677 fetch_fsmap learned FSMap version 35 2024-09-15T23:07:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8f2aa740 1 -- 172.21.15.111:0/4171406712 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f2839187f0 con 0x55f283915a60 2024-09-15T23:07:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c6b400640 1 -- 172.21.15.111:0/4171406712 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c7c036040 con 0x55f283915a60 2024-09-15T23:07:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c6b400640 1 -- 172.21.15.111:0/4171406712 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f8c7c02f9f0 con 0x55f283915a60 2024-09-15T23:07:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8f2aa740 10 client.16677 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:07:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8f2aa740 10 client.16677 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:07:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8f2aa740 20 client.16677 populate_metadata read hostname 'smithi111' 2024-09-15T23:07:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8f2aa740 1 --2- 172.21.15.111:0/4171406712 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f2839257a0 0x55f283927be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8f2aa740 4 client.16677 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:07:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c8f2aa740 1 -- 172.21.15.111:0/4171406712 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f283882830 con 0x55f2839257a0 2024-09-15T23:07:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c89a00640 1 --2- 172.21.15.111:0/4171406712 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f2839257a0 0x55f283927be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c89a00640 1 --2- 172.21.15.111:0/4171406712 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f2839257a0 0x55f283927be0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2c21a66dfa1cfdc server_cookie=ec575cdc05c93028 in_seq=0 out_seq=0 2024-09-15T23:07:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.235+0000 7f8c6b400640 10 client.16677 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:07:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.240+0000 7f8c6b400640 1 -- 172.21.15.111:0/4171406712 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f283882830 con 0x55f2839257a0 2024-09-15T23:07:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.240+0000 7f8c6b400640 10 client.16677 handle_command_reply: tid=0 2024-09-15T23:07:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.240+0000 7f8c8f2aa740 1 client.16677 shutdown 2024-09-15T23:07:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.240+0000 7f8c8f2aa740 1 -- 172.21.15.111:0/4171406712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f283915a60 msgr2=0x55f283921510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.240+0000 7f8c8f2aa740 1 --2- 172.21.15.111:0/4171406712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f283915a60 0x55f283921510 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c005e30 tx=0x7f8c7c031930 comp rx=0 tx=0).stop 2024-09-15T23:07:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.240+0000 7f8c8f2aa740 1 -- 172.21.15.111:0/4171406712 shutdown_connections 2024-09-15T23:07:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.240+0000 7f8c8f2aa740 1 --2- 172.21.15.111:0/4171406712 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f2839257a0 0x55f283927be0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.240+0000 7f8c8f2aa740 1 --2- 172.21.15.111:0/4171406712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f283915a60 0x55f283921510 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.240+0000 7f8c8f2aa740 1 --2- 172.21.15.111:0/4171406712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f2839109d0 0x55f28391a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.240+0000 7f8c8f2aa740 1 --2- 172.21.15.111:0/4171406712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f283910090 0x55f28391a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.240+0000 7f8c8f2aa740 1 -- 172.21.15.111:0/4171406712 >> 172.21.15.111:0/4171406712 conn(0x55f28390b700 msgr2=0x55f28390cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.240+0000 7f8c6b400640 0 client.16677 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:07:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.240+0000 7f8c8f2aa740 1 -- 172.21.15.111:0/4171406712 shutdown_connections 2024-09-15T23:07:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.240+0000 7f8c8f2aa740 1 -- 172.21.15.111:0/4171406712 wait complete. 2024-09-15T23:07:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.240+0000 7f8c8f2aa740 20 client.16677 trim_cache size 0 max 16384 2024-09-15T23:07:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c8f2aa740 1 Processor -- start 2024-09-15T23:07:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c8f2aa740 1 -- start start 2024-09-15T23:07:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c8f2aa740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f28390b700 0x55f28390a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c8f2aa740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f283910090 0x55f283921430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c8f2aa740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2839109d0 0x55f28392c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c8f2aa740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841a94a0 con 0x55f2839109d0 2024-09-15T23:07:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c8f2aa740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841a6d70 con 0x55f28390b700 2024-09-15T23:07:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c8f2aa740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841a4be0 con 0x55f283910090 2024-09-15T23:07:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c8a400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2839109d0 0x55f28392c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c8a400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2839109d0 0x55f28392c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56636/0 (socket says 172.21.15.111:56636) 2024-09-15T23:07:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c8a400640 1 -- 172.21.15.111:0/3301968763 learned_addr learned my addr 172.21.15.111:0/3301968763 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:07:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c89000640 1 --2- 172.21.15.111:0/3301968763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f283910090 0x55f283921430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c89a00640 1 --2- 172.21.15.111:0/3301968763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f28390b700 0x55f28390a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c8a400640 1 -- 172.21.15.111:0/3301968763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f283910090 msgr2=0x55f283921430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c8a400640 1 --2- 172.21.15.111:0/3301968763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f283910090 0x55f283921430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c8a400640 1 -- 172.21.15.111:0/3301968763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f28390b700 msgr2=0x55f28390a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c8a400640 1 --2- 172.21.15.111:0/3301968763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f28390b700 0x55f28390a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c8a400640 1 -- 172.21.15.111:0/3301968763 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f283918ad0 con 0x55f2839109d0 2024-09-15T23:07:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c8a400640 1 --2- 172.21.15.111:0/3301968763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2839109d0 0x55f28392c4e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c02ade0 tx=0x7f8c7c005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e9554c2a19d526 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c76000640 1 -- 172.21.15.111:0/3301968763 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c05d070 con 0x55f2839109d0 2024-09-15T23:07:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c76000640 1 -- 172.21.15.111:0/3301968763 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c7c036070 con 0x55f2839109d0 2024-09-15T23:07:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c76000640 1 -- 172.21.15.111:0/3301968763 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c004290 con 0x55f2839109d0 2024-09-15T23:07:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c8f2aa740 1 -- 172.21.15.111:0/3301968763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2839109d0 msgr2=0x55f28392c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c8f2aa740 1 --2- 172.21.15.111:0/3301968763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2839109d0 0x55f28392c4e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c02ade0 tx=0x7f8c7c005d80 comp rx=0 tx=0).stop 2024-09-15T23:07:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c8f2aa740 1 -- 172.21.15.111:0/3301968763 shutdown_connections 2024-09-15T23:07:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c8f2aa740 1 --2- 172.21.15.111:0/3301968763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2839109d0 0x55f28392c4e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c8f2aa740 1 --2- 172.21.15.111:0/3301968763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f283910090 0x55f283921430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c8f2aa740 1 --2- 172.21.15.111:0/3301968763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f28390b700 0x55f28390a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c8f2aa740 1 -- 172.21.15.111:0/3301968763 >> 172.21.15.111:0/3301968763 conn(0x55f283915580 msgr2=0x55f28390a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c8f2aa740 1 -- 172.21.15.111:0/3301968763 shutdown_connections 2024-09-15T23:07:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.253+0000 7f8c8f2aa740 1 -- 172.21.15.111:0/3301968763 wait complete. 2024-09-15T23:07:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.258+0000 7f8c8f2aa740 1 -- start start 2024-09-15T23:07:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.258+0000 7f8c8f2aa740 1 Processor -- start 2024-09-15T23:07:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.258+0000 7f8c8f2aa740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f28390b700 0x55f28391e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.258+0000 7f8c8f2aa740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f283910090 0x55f28391ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.258+0000 7f8c8f2aa740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2839109d0 0x55f28391fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.258+0000 7f8c8f2aa740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841a5230 con 0x55f2839109d0 2024-09-15T23:07:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.258+0000 7f8c8f2aa740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841a66a0 con 0x55f28390b700 2024-09-15T23:07:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.258+0000 7f8c8f2aa740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841a5900 con 0x55f283910090 2024-09-15T23:07:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.258+0000 7f8c8f2aa740 10 client.0 _ll_register_callbacks cb 0x7f8c78009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:07:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.258+0000 7f8c8a400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2839109d0 0x55f28391fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.258+0000 7f8c8a400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2839109d0 0x55f28391fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56650/0 (socket says 172.21.15.111:56650) 2024-09-15T23:07:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.258+0000 7f8c8a400640 1 -- 172.21.15.111:0/4294339666 learned_addr learned my addr 172.21.15.111:0/4294339666 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:07:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.258+0000 7f8c89000640 1 --2- 172.21.15.111:0/4294339666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f283910090 0x55f28391ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.258+0000 7f8c89a00640 1 --2- 172.21.15.111:0/4294339666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f28390b700 0x55f28391e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.259+0000 7f8c8a400640 1 -- 172.21.15.111:0/4294339666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f283910090 msgr2=0x55f28391ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.259+0000 7f8c8a400640 1 --2- 172.21.15.111:0/4294339666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f283910090 0x55f28391ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.259+0000 7f8c8a400640 1 -- 172.21.15.111:0/4294339666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f28390b700 msgr2=0x55f28391e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.259+0000 7f8c8a400640 1 --2- 172.21.15.111:0/4294339666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f28390b700 0x55f28391e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.259+0000 7f8c8a400640 1 -- 172.21.15.111:0/4294339666 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f283920460 con 0x55f2839109d0 2024-09-15T23:07:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.259+0000 7f8c8a400640 1 --2- 172.21.15.111:0/4294339666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2839109d0 0x55f28391fe50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c031a40 tx=0x7f8c7c02f820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef995746ba3a8da server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.259+0000 7f8c6b400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:07:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.259+0000 7f8c6b400640 1 -- 172.21.15.111:0/4294339666 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c066030 con 0x55f2839109d0 2024-09-15T23:07:15.265 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.259+0000 7f8c6b400640 1 -- 172.21.15.111:0/4294339666 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f8c7c0307c0 con 0x55f2839109d0 2024-09-15T23:07:15.265 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.259+0000 7f8c8f2aa740 1 -- 172.21.15.111:0/4294339666 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c7c036070 con 0x55f2839109d0 2024-09-15T23:07:15.265 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.259+0000 7f8c6b400640 1 -- 172.21.15.111:0/4294339666 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8c7c05d070 con 0x55f2839109d0 2024-09-15T23:07:15.265 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.259+0000 7f8c8f2aa740 10 client.16683 fetch_fsmap learned FSMap version 35 2024-09-15T23:07:15.265 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.259+0000 7f8c8f2aa740 1 -- 172.21.15.111:0/4294339666 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f283921770 con 0x55f2839109d0 2024-09-15T23:07:15.265 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.259+0000 7f8c6b400640 1 -- 172.21.15.111:0/4294339666 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c7c036070 con 0x55f2839109d0 2024-09-15T23:07:15.265 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.259+0000 7f8c6b400640 1 -- 172.21.15.111:0/4294339666 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f8c7c062520 con 0x55f2839109d0 2024-09-15T23:07:15.265 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.259+0000 7f8c8f2aa740 10 client.16683 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:07:15.265 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.259+0000 7f8c8f2aa740 10 client.16683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:07:15.265 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.259+0000 7f8c8f2aa740 20 client.16683 populate_metadata read hostname 'smithi111' 2024-09-15T23:07:15.265 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.259+0000 7f8c8f2aa740 1 --2- 172.21.15.111:0/4294339666 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f283915a60 0x55f283992b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:15.265 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.259+0000 7f8c8f2aa740 4 client.16683 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:07:15.265 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.259+0000 7f8c8f2aa740 1 -- 172.21.15.111:0/4294339666 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f28387cb80 con 0x55f283915a60 2024-09-15T23:07:15.265 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.259+0000 7f8c89a00640 1 --2- 172.21.15.111:0/4294339666 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f283915a60 0x55f283992b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.259+0000 7f8c89a00640 1 --2- 172.21.15.111:0/4294339666 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f283915a60 0x55f283992b90 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7233f83a9b0d6dcc server_cookie=b03c66d75a18c2c2 in_seq=0 out_seq=0 2024-09-15T23:07:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.259+0000 7f8c6b400640 10 client.16683 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:07:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.294+0000 7f8c6b400640 1 -- 172.21.15.111:0/4294339666 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f28387cb80 con 0x55f283915a60 2024-09-15T23:07:15.300 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.294+0000 7f8c6b400640 10 client.16683 handle_command_reply: tid=0 2024-09-15T23:07:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.294+0000 7f8c8f2aa740 1 client.16683 shutdown 2024-09-15T23:07:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.294+0000 7f8c8f2aa740 1 -- 172.21.15.111:0/4294339666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2839109d0 msgr2=0x55f28391fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.294+0000 7f8c8f2aa740 1 --2- 172.21.15.111:0/4294339666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2839109d0 0x55f28391fe50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c031a40 tx=0x7f8c7c02f820 comp rx=0 tx=0).stop 2024-09-15T23:07:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.294+0000 7f8c8f2aa740 1 -- 172.21.15.111:0/4294339666 shutdown_connections 2024-09-15T23:07:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.294+0000 7f8c8f2aa740 1 --2- 172.21.15.111:0/4294339666 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f283915a60 0x55f283992b90 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.294+0000 7f8c8f2aa740 1 --2- 172.21.15.111:0/4294339666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f2839109d0 0x55f28391fe50 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.294+0000 7f8c8f2aa740 1 --2- 172.21.15.111:0/4294339666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f283910090 0x55f28391ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.294+0000 7f8c8f2aa740 1 --2- 172.21.15.111:0/4294339666 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f28390b700 0x55f28391e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.294+0000 7f8c8f2aa740 1 -- 172.21.15.111:0/4294339666 >> 172.21.15.111:0/4294339666 conn(0x55f283915580 msgr2=0x55f28390a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.294+0000 7f8c6b400640 0 client.16683 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:07:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.294+0000 7f8c8f2aa740 1 -- 172.21.15.111:0/4294339666 shutdown_connections 2024-09-15T23:07:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.294+0000 7f8c8f2aa740 1 -- 172.21.15.111:0/4294339666 wait complete. 2024-09-15T23:07:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.294+0000 7f8c8f2aa740 20 client.16683 trim_cache size 0 max 16384 2024-09-15T23:07:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:07:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:07:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:07:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:07:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.300+0000 7f8c8c400640 1 -- 172.21.15.111:0/3888810723 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8c6c0a32f0 msgr2=0x7f8c6c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.300+0000 7f8c8c400640 1 --2- 172.21.15.111:0/3888810723 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8c6c0a32f0 0x7f8c6c0a57b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55f28390c580 tx=0x7f8c78005f90 comp rx=0 tx=0).stop 2024-09-15T23:07:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.300+0000 7f8c8c400640 1 -- 172.21.15.111:0/3888810723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c84104f50 msgr2=0x7f8c8419a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.300+0000 7f8c8c400640 1 --2- 172.21.15.111:0/3888810723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c84104f50 0x7f8c8419a510 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000e9f0 tx=0x7f8c7000eeb0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.300+0000 7f8c8c400640 1 -- 172.21.15.111:0/3888810723 shutdown_connections 2024-09-15T23:07:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.300+0000 7f8c8c400640 1 --2- 172.21.15.111:0/3888810723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c84105910 0x7f8c841a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.300+0000 7f8c8c400640 1 --2- 172.21.15.111:0/3888810723 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8c6c0a32f0 0x7f8c6c0a57b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.300+0000 7f8c8c400640 1 --2- 172.21.15.111:0/3888810723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c84104f50 0x7f8c8419a510 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.300+0000 7f8c8c400640 1 --2- 172.21.15.111:0/3888810723 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c84103d60 0x7f8c84199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.300+0000 7f8c8c400640 1 -- 172.21.15.111:0/3888810723 >> 172.21.15.111:0/3888810723 conn(0x7f8c840ff4e0 msgr2=0x7f8c841081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.300+0000 7f8c8c400640 1 -- 172.21.15.111:0/3888810723 shutdown_connections 2024-09-15T23:07:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:15.300+0000 7f8c8c400640 1 -- 172.21.15.111:0/3888810723 wait complete. 2024-09-15T23:07:15.679 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149430 15.91 MB/sec execute 343 sec latency 13910.864 ms 2024-09-15T23:07:16.249 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:15 smithi111 ceph-mon[30629]: pgmap v991: 97 pgs: 97 active+clean; 535 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 3.8 MiB/s wr, 178 op/s 2024-09-15T23:07:16.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:15 smithi192 ceph-mon[35427]: pgmap v991: 97 pgs: 97 active+clean; 535 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 3.8 MiB/s wr, 178 op/s 2024-09-15T23:07:16.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:15 smithi121 ceph-mon[34815]: pgmap v991: 97 pgs: 97 active+clean; 535 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 3.8 MiB/s wr, 178 op/s 2024-09-15T23:07:16.679 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149466 15.86 MB/sec execute 344 sec latency 14676.839 ms 2024-09-15T23:07:17.679 INFO:tasks.workunit.client.0.smithi111.stdout: 10 149867 15.87 MB/sec execute 345 sec latency 11571.431 ms 2024-09-15T23:07:18.134 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:17 smithi111 ceph-mon[30629]: pgmap v992: 97 pgs: 97 active+clean; 535 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 2.9 MiB/s wr, 136 op/s 2024-09-15T23:07:18.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:17 smithi192 ceph-mon[35427]: pgmap v992: 97 pgs: 97 active+clean; 535 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 2.9 MiB/s wr, 136 op/s 2024-09-15T23:07:18.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:17 smithi121 ceph-mon[34815]: pgmap v992: 97 pgs: 97 active+clean; 535 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 2.9 MiB/s wr, 136 op/s 2024-09-15T23:07:18.679 INFO:tasks.workunit.client.0.smithi111.stdout: 10 150661 15.94 MB/sec execute 346 sec latency 61.145 ms 2024-09-15T23:07:19.680 INFO:tasks.workunit.client.0.smithi111.stdout: 10 151481 16.02 MB/sec execute 347 sec latency 61.217 ms 2024-09-15T23:07:20.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:19 smithi192 ceph-mon[35427]: pgmap v993: 97 pgs: 97 active+clean; 535 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 137 op/s 2024-09-15T23:07:20.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:19 smithi121 ceph-mon[34815]: pgmap v993: 97 pgs: 97 active+clean; 535 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 137 op/s 2024-09-15T23:07:20.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:19 smithi111 ceph-mon[30629]: pgmap v993: 97 pgs: 97 active+clean; 535 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 137 op/s 2024-09-15T23:07:20.680 INFO:tasks.workunit.client.0.smithi111.stdout: 10 152318 16.13 MB/sec execute 348 sec latency 74.608 ms 2024-09-15T23:07:21.680 INFO:tasks.workunit.client.0.smithi111.stdout: 10 152870 16.15 MB/sec execute 349 sec latency 105.486 ms 2024-09-15T23:07:22.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:22 smithi192 ceph-mon[35427]: pgmap v994: 97 pgs: 97 active+clean; 540 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 215 op/s 2024-09-15T23:07:22.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:22 smithi121 ceph-mon[34815]: pgmap v994: 97 pgs: 97 active+clean; 540 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 215 op/s 2024-09-15T23:07:22.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:22 smithi111 ceph-mon[30629]: pgmap v994: 97 pgs: 97 active+clean; 540 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 215 op/s 2024-09-15T23:07:22.680 INFO:tasks.workunit.client.0.smithi111.stdout: 10 153640 16.20 MB/sec execute 350 sec latency 62.974 ms 2024-09-15T23:07:23.680 INFO:tasks.workunit.client.0.smithi111.stdout: 10 154535 16.34 MB/sec execute 351 sec latency 37.271 ms 2024-09-15T23:07:24.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:24 smithi192 ceph-mon[35427]: pgmap v995: 97 pgs: 97 active+clean; 537 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.4 MiB/s wr, 211 op/s 2024-09-15T23:07:24.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:24 smithi121 ceph-mon[34815]: pgmap v995: 97 pgs: 97 active+clean; 537 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.4 MiB/s wr, 211 op/s 2024-09-15T23:07:24.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:24 smithi111 ceph-mon[30629]: pgmap v995: 97 pgs: 97 active+clean; 537 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.4 MiB/s wr, 211 op/s 2024-09-15T23:07:24.680 INFO:tasks.workunit.client.0.smithi111.stdout: 10 155165 16.37 MB/sec execute 352 sec latency 72.689 ms 2024-09-15T23:07:25.681 INFO:tasks.workunit.client.0.smithi111.stdout: 10 155936 16.43 MB/sec execute 353 sec latency 43.450 ms 2024-09-15T23:07:26.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:26 smithi192 ceph-mon[35427]: pgmap v996: 97 pgs: 97 active+clean; 536 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.7 MiB/s wr, 233 op/s 2024-09-15T23:07:26.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:26 smithi121 ceph-mon[34815]: pgmap v996: 97 pgs: 97 active+clean; 536 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.7 MiB/s wr, 233 op/s 2024-09-15T23:07:26.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:26 smithi111 ceph-mon[30629]: pgmap v996: 97 pgs: 97 active+clean; 536 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.7 MiB/s wr, 233 op/s 2024-09-15T23:07:26.681 INFO:tasks.workunit.client.0.smithi111.stdout: 10 156687 16.51 MB/sec execute 354 sec latency 75.503 ms 2024-09-15T23:07:27.681 INFO:tasks.workunit.client.0.smithi111.stdout: 10 157508 16.57 MB/sec execute 355 sec latency 140.037 ms 2024-09-15T23:07:28.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:28 smithi192 ceph-mon[35427]: pgmap v997: 97 pgs: 97 active+clean; 545 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.6 MiB/s wr, 343 op/s 2024-09-15T23:07:28.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:28 smithi121 ceph-mon[34815]: pgmap v997: 97 pgs: 97 active+clean; 545 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.6 MiB/s wr, 343 op/s 2024-09-15T23:07:28.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:28 smithi111 ceph-mon[30629]: pgmap v997: 97 pgs: 97 active+clean; 545 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.6 MiB/s wr, 343 op/s 2024-09-15T23:07:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.465+0000 7f9d38e00640 1 -- 172.21.15.111:0/2549850980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d3410e9b0 msgr2=0x7f9d34110da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.465+0000 7f9d38e00640 1 --2- 172.21.15.111:0/2549850980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d3410e9b0 0x7f9d34110da0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009900 tx=0x7f9d2402a3f0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.466+0000 7f9d38e00640 1 -- 172.21.15.111:0/2549850980 shutdown_connections 2024-09-15T23:07:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.466+0000 7f9d38e00640 1 --2- 172.21.15.111:0/2549850980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d3410e9b0 0x7f9d34110da0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.466+0000 7f9d38e00640 1 --2- 172.21.15.111:0/2549850980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d34109cb0 0x7f9d3410a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.466+0000 7f9d38e00640 1 --2- 172.21.15.111:0/2549850980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d34073e20 0x7f9d34109770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.466+0000 7f9d38e00640 1 -- 172.21.15.111:0/2549850980 >> 172.21.15.111:0/2549850980 conn(0x7f9d3406ece0 msgr2=0x7f9d34071120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.466+0000 7f9d38e00640 1 -- 172.21.15.111:0/2549850980 shutdown_connections 2024-09-15T23:07:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.466+0000 7f9d38e00640 1 -- 172.21.15.111:0/2549850980 wait complete. 2024-09-15T23:07:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.469+0000 7f9d38e00640 1 Processor -- start 2024-09-15T23:07:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.469+0000 7f9d38e00640 1 -- start start 2024-09-15T23:07:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.469+0000 7f9d38e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d34073e20 0x7f9d3419e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.470+0000 7f9d38e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d34109cb0 0x7f9d3419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.470+0000 7f9d38e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d3410e9b0 0x7f9d341a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.470+0000 7f9d38e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9d34116050 con 0x7f9d34073e20 2024-09-15T23:07:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.470+0000 7f9d38e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9d34115ed0 con 0x7f9d3410e9b0 2024-09-15T23:07:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.470+0000 7f9d38e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9d341161d0 con 0x7f9d34109cb0 2024-09-15T23:07:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.470+0000 7f9d31400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d34109cb0 0x7f9d3419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.470+0000 7f9d32800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d3410e9b0 0x7f9d341a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.470+0000 7f9d31e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d34073e20 0x7f9d3419e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.470+0000 7f9d32800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d3410e9b0 0x7f9d341a58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:37318/0 (socket says 172.21.15.111:37318) 2024-09-15T23:07:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.470+0000 7f9d32800640 1 -- 172.21.15.111:0/1532759407 learned_addr learned my addr 172.21.15.111:0/1532759407 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:07:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.471+0000 7f9d32800640 1 -- 172.21.15.111:0/1532759407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d34109cb0 msgr2=0x7f9d3419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.471+0000 7f9d32800640 1 --2- 172.21.15.111:0/1532759407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d34109cb0 0x7f9d3419e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.471+0000 7f9d32800640 1 -- 172.21.15.111:0/1532759407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d34073e20 msgr2=0x7f9d3419e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.471+0000 7f9d32800640 1 --2- 172.21.15.111:0/1532759407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d34073e20 0x7f9d3419e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.471+0000 7f9d32800640 1 -- 172.21.15.111:0/1532759407 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d341a5f30 con 0x7f9d3410e9b0 2024-09-15T23:07:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.471+0000 7f9d32800640 1 --2- 172.21.15.111:0/1532759407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d3410e9b0 0x7f9d341a58c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9d2402a910 tx=0x7f9d24032c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.471+0000 7f9d2aa00640 1 -- 172.21.15.111:0/1532759407 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2403f030 con 0x7f9d3410e9b0 2024-09-15T23:07:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.471+0000 7f9d2aa00640 1 -- 172.21.15.111:0/1532759407 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d24032e50 con 0x7f9d3410e9b0 2024-09-15T23:07:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.471+0000 7f9d2aa00640 1 -- 172.21.15.111:0/1532759407 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d24035070 con 0x7f9d3410e9b0 2024-09-15T23:07:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.471+0000 7f9d38e00640 1 -- 172.21.15.111:0/1532759407 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d341a61c0 con 0x7f9d3410e9b0 2024-09-15T23:07:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.471+0000 7f9d38e00640 1 -- 172.21.15.111:0/1532759407 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d341a6660 con 0x7f9d3410e9b0 2024-09-15T23:07:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.476+0000 7f9d2aa00640 1 -- 172.21.15.111:0/1532759407 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f9d24042030 con 0x7f9d3410e9b0 2024-09-15T23:07:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.477+0000 7f9d3bbf7740 1 Processor -- start 2024-09-15T23:07:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.477+0000 7f9d3bbf7740 1 -- start start 2024-09-15T23:07:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.477+0000 7f9d3bbf7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0fe9ba090 0x55a0fe9ba490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.477+0000 7f9d3bbf7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9ba9d0 0x55a0fe9bf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.477+0000 7f9d3bbf7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0fe9bfa60 0x55a0fe9c1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.477+0000 7f9d3bbf7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a0fe91f100 con 0x55a0fe9ba090 2024-09-15T23:07:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.477+0000 7f9d3bbf7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a0fe8c71b0 con 0x55a0fe9ba9d0 2024-09-15T23:07:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.477+0000 7f9d3bbf7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a0fe91b0c0 con 0x55a0fe9bfa60 2024-09-15T23:07:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.478+0000 7f9d31e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9ba9d0 0x55a0fe9bf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.478+0000 7f9d31e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9ba9d0 0x55a0fe9bf490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:37320/0 (socket says 172.21.15.111:37320) 2024-09-15T23:07:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.478+0000 7f9d31e00640 1 -- 172.21.15.111:0/413059794 learned_addr learned my addr 172.21.15.111:0/413059794 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:07:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.478+0000 7f9d31400640 1 --2- 172.21.15.111:0/413059794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0fe9bfa60 0x55a0fe9c1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.478+0000 7f9d31e00640 1 -- 172.21.15.111:0/413059794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0fe9bfa60 msgr2=0x55a0fe9c1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.478+0000 7f9d31e00640 1 --2- 172.21.15.111:0/413059794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0fe9bfa60 0x55a0fe9c1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.478+0000 7f9d31e00640 1 -- 172.21.15.111:0/413059794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0fe9ba090 msgr2=0x55a0fe9ba490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.478+0000 7f9d31e00640 1 --2- 172.21.15.111:0/413059794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0fe9ba090 0x55a0fe9ba490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.478+0000 7f9d31e00640 1 -- 172.21.15.111:0/413059794 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0fe632210 con 0x55a0fe9ba9d0 2024-09-15T23:07:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.478+0000 7f9d2aa00640 1 --2- 172.21.15.111:0/1532759407 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9d180a3370 0x7f9d180a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.478+0000 7f9d31e00640 1 --2- 172.21.15.111:0/413059794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9ba9d0 0x55a0fe9bf490 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9d20009ca0 tx=0x7f9d20011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e60d3a184490d10c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.479+0000 7f9d2aa00640 1 -- 172.21.15.111:0/1532759407 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f9d240efe80 con 0x7f9d3410e9b0 2024-09-15T23:07:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.479+0000 7f9d1f400640 1 -- 172.21.15.111:0/413059794 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d20004220 con 0x55a0fe9ba9d0 2024-09-15T23:07:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.479+0000 7f9d1f400640 1 -- 172.21.15.111:0/413059794 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d200043c0 con 0x55a0fe9ba9d0 2024-09-15T23:07:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.479+0000 7f9d1f400640 1 -- 172.21.15.111:0/413059794 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d20019930 con 0x55a0fe9ba9d0 2024-09-15T23:07:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.479+0000 7f9d31400640 1 --2- 172.21.15.111:0/1532759407 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9d180a3370 0x7f9d180a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.480+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/413059794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9ba9d0 msgr2=0x55a0fe9bf490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.480+0000 7f9d3bbf7740 1 --2- 172.21.15.111:0/413059794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9ba9d0 0x55a0fe9bf490 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9d20009ca0 tx=0x7f9d20011810 comp rx=0 tx=0).stop 2024-09-15T23:07:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.480+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/413059794 shutdown_connections 2024-09-15T23:07:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.480+0000 7f9d3bbf7740 1 --2- 172.21.15.111:0/413059794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0fe9bfa60 0x55a0fe9c1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.480+0000 7f9d3bbf7740 1 --2- 172.21.15.111:0/413059794 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9ba9d0 0x55a0fe9bf490 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.480+0000 7f9d3bbf7740 1 --2- 172.21.15.111:0/413059794 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0fe9ba090 0x55a0fe9ba490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.480+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/413059794 >> 172.21.15.111:0/413059794 conn(0x55a0fe9b5700 msgr2=0x55a0fe9b5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:28.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.481+0000 7f9d31400640 1 --2- 172.21.15.111:0/1532759407 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9d180a3370 0x7f9d180a5830 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9d3419f820 tx=0x7f9d2c007910 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:28.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.482+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/413059794 shutdown_connections 2024-09-15T23:07:28.483 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.482+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/413059794 wait complete. 2024-09-15T23:07:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.482+0000 7f9d3bbf7740 1 -- start start 2024-09-15T23:07:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.482+0000 7f9d3bbf7740 1 Processor -- start 2024-09-15T23:07:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.483+0000 7f9d3bbf7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0fe9ba090 0x55a0fe9c4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.483+0000 7f9d3bbf7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0fe9ba9d0 0x55a0fe9c4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.483+0000 7f9d3bbf7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9bfa60 0x55a0fe9cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.483+0000 7f9d3bbf7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a0fe9c2a30 con 0x55a0fe9ba090 2024-09-15T23:07:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.483+0000 7f9d3bbf7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a0fe8a8090 con 0x55a0fe9bfa60 2024-09-15T23:07:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.483+0000 7f9d3bbf7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a0fe9c2bb0 con 0x55a0fe9ba9d0 2024-09-15T23:07:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.483+0000 7f9d3bbf7740 10 client.0 _ll_register_callbacks cb 0x55a0fe8fcf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:07:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.483+0000 7f9d31e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0fe9ba090 0x55a0fe9c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.483+0000 7f9d32800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9bfa60 0x55a0fe9cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.483+0000 7f9d32800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9bfa60 0x55a0fe9cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:37328/0 (socket says 172.21.15.111:37328) 2024-09-15T23:07:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.483+0000 7f9d32800640 1 -- 172.21.15.111:0/298105712 learned_addr learned my addr 172.21.15.111:0/298105712 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:07:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.483+0000 7f9d31400640 1 --2- 172.21.15.111:0/298105712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0fe9ba9d0 0x55a0fe9c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.484+0000 7f9d32800640 1 -- 172.21.15.111:0/298105712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0fe9ba9d0 msgr2=0x55a0fe9c4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.484+0000 7f9d32800640 1 --2- 172.21.15.111:0/298105712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0fe9ba9d0 0x55a0fe9c4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.484+0000 7f9d32800640 1 -- 172.21.15.111:0/298105712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0fe9ba090 msgr2=0x55a0fe9c4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.484+0000 7f9d32800640 1 --2- 172.21.15.111:0/298105712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0fe9ba090 0x55a0fe9c4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.484+0000 7f9d32800640 1 -- 172.21.15.111:0/298105712 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0fe6fe620 con 0x55a0fe9bfa60 2024-09-15T23:07:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.484+0000 7f9d32800640 1 --2- 172.21.15.111:0/298105712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9bfa60 0x55a0fe9cb7c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9d240c4be0 tx=0x7f9d240bce40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55b5d28d514ea3eb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.484+0000 7f9d13e00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:07:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.484+0000 7f9d13e00640 1 -- 172.21.15.111:0/298105712 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d24049040 con 0x55a0fe9bfa60 2024-09-15T23:07:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.484+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/298105712 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d200043c0 con 0x55a0fe9bfa60 2024-09-15T23:07:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.484+0000 7f9d13e00640 1 -- 172.21.15.111:0/298105712 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f9d240b9b50 con 0x55a0fe9bfa60 2024-09-15T23:07:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.484+0000 7f9d13e00640 1 -- 172.21.15.111:0/298105712 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9d2403bc30 con 0x55a0fe9bfa60 2024-09-15T23:07:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.484+0000 7f9d3bbf7740 10 client.26630 fetch_fsmap learned FSMap version 35 2024-09-15T23:07:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.484+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/298105712 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a0fe9ce5b0 con 0x55a0fe9bfa60 2024-09-15T23:07:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.485+0000 7f9d13e00640 1 -- 172.21.15.111:0/298105712 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d2403bdd0 con 0x55a0fe9bfa60 2024-09-15T23:07:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.485+0000 7f9d13e00640 1 -- 172.21.15.111:0/298105712 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f9d240d5290 con 0x55a0fe9bfa60 2024-09-15T23:07:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.485+0000 7f9d3bbf7740 10 client.26630 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:07:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.485+0000 7f9d3bbf7740 10 client.26630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:07:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.485+0000 7f9d3bbf7740 20 client.26630 populate_metadata read hostname 'smithi111' 2024-09-15T23:07:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.485+0000 7f9d3bbf7740 1 --2- 172.21.15.111:0/298105712 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a0fe9cfce0 0x55a0fe9d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.485+0000 7f9d3bbf7740 4 client.26630 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:07:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.485+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/298105712 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9d200043c0 con 0x55a0fe9cfce0 2024-09-15T23:07:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.486+0000 7f9d31e00640 1 --2- 172.21.15.111:0/298105712 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a0fe9cfce0 0x55a0fe9d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.486+0000 7f9d31e00640 1 --2- 172.21.15.111:0/298105712 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a0fe9cfce0 0x55a0fe9d20f0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad2c9e2463d522e0 server_cookie=94615b8735fd2f6a in_seq=0 out_seq=0 2024-09-15T23:07:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.486+0000 7f9d13e00640 10 client.26630 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:07:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.488+0000 7f9d13e00640 1 -- 172.21.15.111:0/298105712 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f9d200043c0 con 0x55a0fe9cfce0 2024-09-15T23:07:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.488+0000 7f9d13e00640 10 client.26630 handle_command_reply: tid=0 2024-09-15T23:07:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.488+0000 7f9d3bbf7740 1 client.26630 shutdown 2024-09-15T23:07:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.488+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/298105712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9bfa60 msgr2=0x55a0fe9cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.488+0000 7f9d3bbf7740 1 --2- 172.21.15.111:0/298105712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9bfa60 0x55a0fe9cb7c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9d240c4be0 tx=0x7f9d240bce40 comp rx=0 tx=0).stop 2024-09-15T23:07:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.488+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/298105712 shutdown_connections 2024-09-15T23:07:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.488+0000 7f9d3bbf7740 1 --2- 172.21.15.111:0/298105712 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a0fe9cfce0 0x55a0fe9d20f0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.488+0000 7f9d3bbf7740 1 --2- 172.21.15.111:0/298105712 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9bfa60 0x55a0fe9cb7c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.488+0000 7f9d3bbf7740 1 --2- 172.21.15.111:0/298105712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0fe9ba9d0 0x55a0fe9c4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.488+0000 7f9d3bbf7740 1 --2- 172.21.15.111:0/298105712 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0fe9ba090 0x55a0fe9c4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.488+0000 7f9d13e00640 0 client.26630 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:07:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.488+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/298105712 >> 172.21.15.111:0/298105712 conn(0x55a0fe9b5700 msgr2=0x55a0fe9b6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.489+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/298105712 shutdown_connections 2024-09-15T23:07:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.489+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/298105712 wait complete. 2024-09-15T23:07:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.489+0000 7f9d3bbf7740 20 client.26630 trim_cache size 0 max 16384 2024-09-15T23:07:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.507+0000 7f9d3bbf7740 1 Processor -- start 2024-09-15T23:07:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.507+0000 7f9d3bbf7740 1 -- start start 2024-09-15T23:07:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.508+0000 7f9d3bbf7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0fe9ba090 0x55a0fe9ba470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.508+0000 7f9d3bbf7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9ba9d0 0x55a0fe9cb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.508+0000 7f9d3bbf7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0fe9bfa60 0x55a0fe9b65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.508+0000 7f9d3bbf7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9d341c2cd0 con 0x55a0fe9ba090 2024-09-15T23:07:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.508+0000 7f9d3bbf7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9d341c05a0 con 0x55a0fe9ba9d0 2024-09-15T23:07:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.508+0000 7f9d3bbf7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9d341be3f0 con 0x55a0fe9bfa60 2024-09-15T23:07:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.508+0000 7f9d31400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9ba9d0 0x55a0fe9cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.508+0000 7f9d31400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9ba9d0 0x55a0fe9cb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:37334/0 (socket says 172.21.15.111:37334) 2024-09-15T23:07:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.508+0000 7f9d31400640 1 -- 172.21.15.111:0/3097582642 learned_addr learned my addr 172.21.15.111:0/3097582642 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:07:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.508+0000 7f9d31e00640 1 --2- 172.21.15.111:0/3097582642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0fe9ba090 0x55a0fe9ba470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.509+0000 7f9d32800640 1 --2- 172.21.15.111:0/3097582642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0fe9bfa60 0x55a0fe9b65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.509+0000 7f9d31400640 1 -- 172.21.15.111:0/3097582642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0fe9bfa60 msgr2=0x55a0fe9b65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.509+0000 7f9d31400640 1 --2- 172.21.15.111:0/3097582642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0fe9bfa60 0x55a0fe9b65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.509+0000 7f9d31400640 1 -- 172.21.15.111:0/3097582642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0fe9ba090 msgr2=0x55a0fe9ba470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.509+0000 7f9d31400640 1 --2- 172.21.15.111:0/3097582642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0fe9ba090 0x55a0fe9ba470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.509+0000 7f9d31400640 1 -- 172.21.15.111:0/3097582642 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0fe9b5520 con 0x55a0fe9ba9d0 2024-09-15T23:07:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.509+0000 7f9d31400640 1 --2- 172.21.15.111:0/3097582642 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9ba9d0 0x55a0fe9cb780 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c012840 tx=0x7f9d2c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=433a70b2d69136a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.509+0000 7f9d1ea00640 1 -- 172.21.15.111:0/3097582642 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c020dc0 con 0x55a0fe9ba9d0 2024-09-15T23:07:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.509+0000 7f9d1ea00640 1 -- 172.21.15.111:0/3097582642 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d2c005020 con 0x55a0fe9ba9d0 2024-09-15T23:07:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.509+0000 7f9d1ea00640 1 -- 172.21.15.111:0/3097582642 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c00f600 con 0x55a0fe9ba9d0 2024-09-15T23:07:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.510+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/3097582642 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9ba9d0 msgr2=0x55a0fe9cb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.510+0000 7f9d3bbf7740 1 --2- 172.21.15.111:0/3097582642 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9ba9d0 0x55a0fe9cb780 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c012840 tx=0x7f9d2c014000 comp rx=0 tx=0).stop 2024-09-15T23:07:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.511+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/3097582642 shutdown_connections 2024-09-15T23:07:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.511+0000 7f9d3bbf7740 1 --2- 172.21.15.111:0/3097582642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0fe9bfa60 0x55a0fe9b65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.511+0000 7f9d3bbf7740 1 --2- 172.21.15.111:0/3097582642 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9ba9d0 0x55a0fe9cb780 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.511+0000 7f9d3bbf7740 1 --2- 172.21.15.111:0/3097582642 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0fe9ba090 0x55a0fe9ba470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.511+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/3097582642 >> 172.21.15.111:0/3097582642 conn(0x55a0fe9b5700 msgr2=0x55a0fe9c4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.511+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/3097582642 shutdown_connections 2024-09-15T23:07:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.511+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/3097582642 wait complete. 2024-09-15T23:07:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.511+0000 7f9d3bbf7740 1 -- start start 2024-09-15T23:07:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.512+0000 7f9d3bbf7740 1 Processor -- start 2024-09-15T23:07:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.513+0000 7f9d3bbf7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9ba090 0x55a0fe9c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.513+0000 7f9d3bbf7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0fe9ba9d0 0x55a0fe9c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.513+0000 7f9d3bbf7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0fe9bfa60 0x55a0fe9ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.513+0000 7f9d31e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9ba090 0x55a0fe9c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.513+0000 7f9d31e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9ba090 0x55a0fe9c8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:37348/0 (socket says 172.21.15.111:37348) 2024-09-15T23:07:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.514+0000 7f9d31400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0fe9ba9d0 0x55a0fe9c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.514+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/1504355958 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9d341bea60 con 0x55a0fe9bfa60 2024-09-15T23:07:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.514+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/1504355958 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9d341bfed0 con 0x55a0fe9ba090 2024-09-15T23:07:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.514+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/1504355958 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9d341bf130 con 0x55a0fe9ba9d0 2024-09-15T23:07:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.514+0000 7f9d3bbf7740 10 client.0 _ll_register_callbacks cb 0x7f9d24005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:07:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.514+0000 7f9d31e00640 1 -- 172.21.15.111:0/1504355958 learned_addr learned my addr 172.21.15.111:0/1504355958 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:07:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.514+0000 7f9d32800640 1 --2- 172.21.15.111:0/1504355958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0fe9bfa60 0x55a0fe9ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.514+0000 7f9d31e00640 1 -- 172.21.15.111:0/1504355958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0fe9ba9d0 msgr2=0x55a0fe9c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.514+0000 7f9d31e00640 1 --2- 172.21.15.111:0/1504355958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0fe9ba9d0 0x55a0fe9c91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.514+0000 7f9d31e00640 1 -- 172.21.15.111:0/1504355958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0fe9bfa60 msgr2=0x55a0fe9ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.514+0000 7f9d31e00640 1 --2- 172.21.15.111:0/1504355958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0fe9bfa60 0x55a0fe9ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.514+0000 7f9d31e00640 1 -- 172.21.15.111:0/1504355958 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0fe9ca800 con 0x55a0fe9ba090 2024-09-15T23:07:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.514+0000 7f9d32800640 1 --2- 172.21.15.111:0/1504355958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0fe9bfa60 0x55a0fe9ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:07:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.515+0000 7f9d13e00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:07:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.515+0000 7f9d31e00640 1 --2- 172.21.15.111:0/1504355958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9ba090 0x55a0fe9c8ca0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9d20003e80 tx=0x7f9d20009960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6aca9ab28bbbc4e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.515+0000 7f9d13e00640 1 -- 172.21.15.111:0/1504355958 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d20005250 con 0x55a0fe9ba090 2024-09-15T23:07:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.515+0000 7f9d13e00640 1 -- 172.21.15.111:0/1504355958 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f9d200053f0 con 0x55a0fe9ba090 2024-09-15T23:07:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.516+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/1504355958 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d2c005020 con 0x55a0fe9ba090 2024-09-15T23:07:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.516+0000 7f9d13e00640 1 -- 172.21.15.111:0/1504355958 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9d20005610 con 0x55a0fe9ba090 2024-09-15T23:07:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.516+0000 7f9d3bbf7740 10 client.26636 fetch_fsmap learned FSMap version 35 2024-09-15T23:07:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.516+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/1504355958 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a0fe9cbb10 con 0x55a0fe9ba090 2024-09-15T23:07:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.517+0000 7f9d13e00640 1 -- 172.21.15.111:0/1504355958 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d2c005020 con 0x55a0fe9ba090 2024-09-15T23:07:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.517+0000 7f9d13e00640 1 -- 172.21.15.111:0/1504355958 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f9d20019a80 con 0x55a0fe9ba090 2024-09-15T23:07:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.517+0000 7f9d3bbf7740 10 client.26636 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:07:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.517+0000 7f9d3bbf7740 10 client.26636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:07:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.517+0000 7f9d3bbf7740 20 client.26636 populate_metadata read hostname 'smithi111' 2024-09-15T23:07:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.517+0000 7f9d3bbf7740 1 --2- 172.21.15.111:0/1504355958 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a0fea3afc0 0x55a0fea3d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.517+0000 7f9d3bbf7740 4 client.26636 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:07:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.517+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/1504355958 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a0fe92c830 con 0x55a0fea3afc0 2024-09-15T23:07:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.518+0000 7f9d31400640 1 --2- 172.21.15.111:0/1504355958 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a0fea3afc0 0x55a0fea3d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.518+0000 7f9d31400640 1 --2- 172.21.15.111:0/1504355958 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a0fea3afc0 0x55a0fea3d3e0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a95244dc0b22e88 server_cookie=3082ed0a052557f3 in_seq=0 out_seq=0 2024-09-15T23:07:28.520 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.518+0000 7f9d13e00640 10 client.26636 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:07:28.681 INFO:tasks.workunit.client.0.smithi111.stdout: 10 158166 16.64 MB/sec execute 356 sec latency 149.198 ms 2024-09-15T23:07:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.785+0000 7f9d13e00640 1 -- 172.21.15.111:0/1504355958 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a0fe92c830 con 0x55a0fea3afc0 2024-09-15T23:07:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.786+0000 7f9d13e00640 10 client.26636 handle_command_reply: tid=0 2024-09-15T23:07:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.786+0000 7f9d3bbf7740 1 client.26636 shutdown 2024-09-15T23:07:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.786+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/1504355958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9ba090 msgr2=0x55a0fe9c8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.786+0000 7f9d3bbf7740 1 --2- 172.21.15.111:0/1504355958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9ba090 0x55a0fe9c8ca0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9d20003e80 tx=0x7f9d20009960 comp rx=0 tx=0).stop 2024-09-15T23:07:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.786+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/1504355958 shutdown_connections 2024-09-15T23:07:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.786+0000 7f9d3bbf7740 1 --2- 172.21.15.111:0/1504355958 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a0fea3afc0 0x55a0fea3d3e0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.786+0000 7f9d3bbf7740 1 --2- 172.21.15.111:0/1504355958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a0fe9bfa60 0x55a0fe9ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.786+0000 7f9d3bbf7740 1 --2- 172.21.15.111:0/1504355958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a0fe9ba9d0 0x55a0fe9c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.786+0000 7f9d3bbf7740 1 --2- 172.21.15.111:0/1504355958 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a0fe9ba090 0x55a0fe9c8ca0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.786+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/1504355958 >> 172.21.15.111:0/1504355958 conn(0x55a0fe9b5700 msgr2=0x55a0fe9b4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.786+0000 7f9d13e00640 0 client.26636 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:07:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.787+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/1504355958 shutdown_connections 2024-09-15T23:07:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.787+0000 7f9d3bbf7740 1 -- 172.21.15.111:0/1504355958 wait complete. 2024-09-15T23:07:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.787+0000 7f9d3bbf7740 20 client.26636 trim_cache size 0 max 16384 2024-09-15T23:07:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:07:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:07:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:07:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:07:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.788+0000 7f9d38e00640 1 -- 172.21.15.111:0/1532759407 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9d180a3370 msgr2=0x7f9d180a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.788+0000 7f9d38e00640 1 --2- 172.21.15.111:0/1532759407 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9d180a3370 0x7f9d180a5830 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9d3419f820 tx=0x7f9d2c007910 comp rx=0 tx=0).stop 2024-09-15T23:07:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.788+0000 7f9d38e00640 1 -- 172.21.15.111:0/1532759407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d3410e9b0 msgr2=0x7f9d341a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.788+0000 7f9d38e00640 1 --2- 172.21.15.111:0/1532759407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d3410e9b0 0x7f9d341a58c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9d2402a910 tx=0x7f9d24032c60 comp rx=0 tx=0).stop 2024-09-15T23:07:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.789+0000 7f9d38e00640 1 -- 172.21.15.111:0/1532759407 shutdown_connections 2024-09-15T23:07:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.789+0000 7f9d38e00640 1 --2- 172.21.15.111:0/1532759407 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9d180a3370 0x7f9d180a5830 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.789+0000 7f9d38e00640 1 --2- 172.21.15.111:0/1532759407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9d3410e9b0 0x7f9d341a58c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.789+0000 7f9d38e00640 1 --2- 172.21.15.111:0/1532759407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d34109cb0 0x7f9d3419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.789+0000 7f9d38e00640 1 --2- 172.21.15.111:0/1532759407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d34073e20 0x7f9d3419e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.789+0000 7f9d38e00640 1 -- 172.21.15.111:0/1532759407 >> 172.21.15.111:0/1532759407 conn(0x7f9d3406ece0 msgr2=0x7f9d34113540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.789+0000 7f9d38e00640 1 -- 172.21.15.111:0/1532759407 shutdown_connections 2024-09-15T23:07:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:28.789+0000 7f9d38e00640 1 -- 172.21.15.111:0/1532759407 wait complete. 2024-09-15T23:07:29.681 INFO:tasks.workunit.client.0.smithi111.stdout: 10 158807 16.69 MB/sec execute 357 sec latency 159.332 ms 2024-09-15T23:07:30.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:30 smithi192 ceph-mon[35427]: pgmap v998: 97 pgs: 97 active+clean; 550 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.6 MiB/s wr, 464 op/s 2024-09-15T23:07:30.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:30 smithi121 ceph-mon[34815]: pgmap v998: 97 pgs: 97 active+clean; 550 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.6 MiB/s wr, 464 op/s 2024-09-15T23:07:30.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:30 smithi111 ceph-mon[30629]: pgmap v998: 97 pgs: 97 active+clean; 550 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.6 MiB/s wr, 464 op/s 2024-09-15T23:07:30.681 INFO:tasks.workunit.client.0.smithi111.stdout: 10 159440 16.73 MB/sec execute 358 sec latency 113.913 ms 2024-09-15T23:07:31.681 INFO:tasks.workunit.client.0.smithi111.stdout: 10 160113 16.78 MB/sec execute 359 sec latency 107.084 ms 2024-09-15T23:07:32.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:32 smithi111 ceph-mon[30629]: pgmap v999: 97 pgs: 97 active+clean; 546 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.2 MiB/s wr, 560 op/s 2024-09-15T23:07:32.317 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:32 smithi192 ceph-mon[35427]: pgmap v999: 97 pgs: 97 active+clean; 546 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.2 MiB/s wr, 560 op/s 2024-09-15T23:07:32.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:32 smithi121 ceph-mon[34815]: pgmap v999: 97 pgs: 97 active+clean; 546 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 4.2 MiB/s wr, 560 op/s 2024-09-15T23:07:32.681 INFO:tasks.workunit.client.0.smithi111.stdout: 10 161013 16.88 MB/sec execute 360 sec latency 47.435 ms 2024-09-15T23:07:33.681 INFO:tasks.workunit.client.0.smithi111.stdout: 10 161763 16.95 MB/sec execute 361 sec latency 51.435 ms 2024-09-15T23:07:34.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:34 smithi192 ceph-mon[35427]: pgmap v1000: 97 pgs: 97 active+clean; 551 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.8 MiB/s wr, 508 op/s 2024-09-15T23:07:34.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:34 smithi121 ceph-mon[34815]: pgmap v1000: 97 pgs: 97 active+clean; 551 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.8 MiB/s wr, 508 op/s 2024-09-15T23:07:34.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:34 smithi111 ceph-mon[30629]: pgmap v1000: 97 pgs: 97 active+clean; 551 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.8 MiB/s wr, 508 op/s 2024-09-15T23:07:34.681 INFO:tasks.workunit.client.0.smithi111.stdout: 10 162317 16.98 MB/sec execute 362 sec latency 58.798 ms 2024-09-15T23:07:35.681 INFO:tasks.workunit.client.0.smithi111.stdout: 10 162968 17.03 MB/sec execute 363 sec latency 87.153 ms 2024-09-15T23:07:36.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:36 smithi192 ceph-mon[35427]: pgmap v1001: 97 pgs: 97 active+clean; 544 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.5 MiB/s wr, 618 op/s 2024-09-15T23:07:36.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:36 smithi121 ceph-mon[34815]: pgmap v1001: 97 pgs: 97 active+clean; 544 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.5 MiB/s wr, 618 op/s 2024-09-15T23:07:36.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:36 smithi111 ceph-mon[30629]: pgmap v1001: 97 pgs: 97 active+clean; 544 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.5 MiB/s wr, 618 op/s 2024-09-15T23:07:36.681 INFO:tasks.workunit.client.0.smithi111.stdout: 10 163582 17.08 MB/sec execute 364 sec latency 88.073 ms 2024-09-15T23:07:37.681 INFO:tasks.workunit.client.0.smithi111.stdout: 10 164201 17.12 MB/sec execute 365 sec latency 114.376 ms 2024-09-15T23:07:38.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:38 smithi111 ceph-mon[30629]: pgmap v1002: 97 pgs: 97 active+clean; 546 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.4 MiB/s wr, 611 op/s 2024-09-15T23:07:38.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:38 smithi192 ceph-mon[35427]: pgmap v1002: 97 pgs: 97 active+clean; 546 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.4 MiB/s wr, 611 op/s 2024-09-15T23:07:38.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:38 smithi121 ceph-mon[34815]: pgmap v1002: 97 pgs: 97 active+clean; 546 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.4 MiB/s wr, 611 op/s 2024-09-15T23:07:38.681 INFO:tasks.workunit.client.0.smithi111.stdout: 10 164870 17.16 MB/sec execute 366 sec latency 80.583 ms 2024-09-15T23:07:39.681 INFO:tasks.workunit.client.0.smithi111.stdout: 10 165780 17.25 MB/sec execute 367 sec latency 45.334 ms 2024-09-15T23:07:40.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:40 smithi192 ceph-mon[35427]: pgmap v1003: 97 pgs: 97 active+clean; 552 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.3 MiB/s wr, 576 op/s 2024-09-15T23:07:40.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:40 smithi121 ceph-mon[34815]: pgmap v1003: 97 pgs: 97 active+clean; 552 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.3 MiB/s wr, 576 op/s 2024-09-15T23:07:40.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:40 smithi111 ceph-mon[30629]: pgmap v1003: 97 pgs: 97 active+clean; 552 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.3 MiB/s wr, 576 op/s 2024-09-15T23:07:40.681 INFO:tasks.workunit.client.0.smithi111.stdout: 10 166508 17.31 MB/sec execute 368 sec latency 67.388 ms 2024-09-15T23:07:41.681 INFO:tasks.workunit.client.0.smithi111.stdout: 10 167249 17.38 MB/sec execute 369 sec latency 86.285 ms 2024-09-15T23:07:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.925+0000 7f0ebea00640 1 -- 172.21.15.111:0/2477441840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eb8101f40 msgr2=0x7f0eb8109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.925+0000 7f0ebea00640 1 --2- 172.21.15.111:0/2477441840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eb8101f40 0x7f0eb8109db0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009ca0 tx=0x7f0eb002a8c0 comp rx=0 tx=0).stop 2024-09-15T23:07:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.925+0000 7f0ebea00640 1 -- 172.21.15.111:0/2477441840 shutdown_connections 2024-09-15T23:07:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.925+0000 7f0ebea00640 1 --2- 172.21.15.111:0/2477441840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eb8101f40 0x7f0eb8109db0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.925+0000 7f0ebea00640 1 --2- 172.21.15.111:0/2477441840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0eb8101580 0x7f0eb8101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.925+0000 7f0ebea00640 1 --2- 172.21.15.111:0/2477441840 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0eb8103050 0x7f0eb8103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.925+0000 7f0ebea00640 1 -- 172.21.15.111:0/2477441840 >> 172.21.15.111:0/2477441840 conn(0x7f0eb80fd0d0 msgr2=0x7f0eb80ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.925+0000 7f0ebea00640 1 -- 172.21.15.111:0/2477441840 shutdown_connections 2024-09-15T23:07:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.925+0000 7f0ebea00640 1 -- 172.21.15.111:0/2477441840 wait complete. 2024-09-15T23:07:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.926+0000 7f0ebea00640 1 Processor -- start 2024-09-15T23:07:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.926+0000 7f0ebea00640 1 -- start start 2024-09-15T23:07:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.927+0000 7f0ebea00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0eb8101580 0x7f0eb8075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.927+0000 7f0ebea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0eb8101f40 0x7f0eb80745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.927+0000 7f0ebea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eb8103050 0x7f0eb8074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.927+0000 7f0ebea00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb810d720 con 0x7f0eb8101f40 2024-09-15T23:07:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.927+0000 7f0ebea00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb810d5a0 con 0x7f0eb8103050 2024-09-15T23:07:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.927+0000 7f0ebea00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb810d8a0 con 0x7f0eb8101580 2024-09-15T23:07:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.927+0000 7f0ebd600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0eb8101580 0x7f0eb8075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.927+0000 7f0ebd600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0eb8101580 0x7f0eb8075fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:57670/0 (socket says 172.21.15.111:57670) 2024-09-15T23:07:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.927+0000 7f0ebd600640 1 -- 172.21.15.111:0/427400851 learned_addr learned my addr 172.21.15.111:0/427400851 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:07:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.927+0000 7f0ebcc00640 1 --2- 172.21.15.111:0/427400851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0eb8101f40 0x7f0eb80745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.927+0000 7f0ebe000640 1 --2- 172.21.15.111:0/427400851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eb8103050 0x7f0eb8074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.927+0000 7f0ebd600640 1 -- 172.21.15.111:0/427400851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eb8103050 msgr2=0x7f0eb8074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.927+0000 7f0ebd600640 1 --2- 172.21.15.111:0/427400851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eb8103050 0x7f0eb8074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.927+0000 7f0ebd600640 1 -- 172.21.15.111:0/427400851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0eb8101f40 msgr2=0x7f0eb80745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.927+0000 7f0ebd600640 1 --2- 172.21.15.111:0/427400851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0eb8101f40 0x7f0eb80745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.927+0000 7f0ebd600640 1 -- 172.21.15.111:0/427400851 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb80753b0 con 0x7f0eb8101580 2024-09-15T23:07:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.928+0000 7f0ebd600640 1 --2- 172.21.15.111:0/427400851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0eb8101580 0x7f0eb8075fa0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00e9c0 tx=0x7f0eac00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.928+0000 7f0eb6000640 1 -- 172.21.15.111:0/427400851 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac00cd80 con 0x7f0eb8101580 2024-09-15T23:07:41.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.928+0000 7f0eb6000640 1 -- 172.21.15.111:0/427400851 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eac0035f0 con 0x7f0eb8101580 2024-09-15T23:07:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.928+0000 7f0ebea00640 1 -- 172.21.15.111:0/427400851 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb81a1e70 con 0x7f0eb8101580 2024-09-15T23:07:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.928+0000 7f0eb6000640 1 -- 172.21.15.111:0/427400851 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac010690 con 0x7f0eb8101580 2024-09-15T23:07:41.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.928+0000 7f0ebea00640 1 -- 172.21.15.111:0/427400851 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb81a2280 con 0x7f0eb8101580 2024-09-15T23:07:41.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.932+0000 7f0eb6000640 1 -- 172.21.15.111:0/427400851 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f0eac0037a0 con 0x7f0eb8101580 2024-09-15T23:07:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.932+0000 7f0ec2e72740 1 Processor -- start 2024-09-15T23:07:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.932+0000 7f0ec2e72740 1 -- start start 2024-09-15T23:07:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.933+0000 7f0ec2e72740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e57ca8260 0x558e57ca8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.933+0000 7f0ec2e72740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e57ca8ba0 0x558e57cad660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.933+0000 7f0ec2e72740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e57cadc30 0x558e57cb00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.933+0000 7f0ec2e72740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x558e57c0d100 con 0x558e57ca8ba0 2024-09-15T23:07:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.933+0000 7f0ec2e72740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558e57bb51b0 con 0x558e57ca8260 2024-09-15T23:07:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.933+0000 7f0ec2e72740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558e57c090c0 con 0x558e57cadc30 2024-09-15T23:07:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.933+0000 7f0ebe000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e57ca8260 0x558e57ca8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.933+0000 7f0ebe000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e57ca8260 0x558e57ca8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:42708/0 (socket says 172.21.15.111:42708) 2024-09-15T23:07:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.933+0000 7f0ebe000640 1 -- 172.21.15.111:0/1578072992 learned_addr learned my addr 172.21.15.111:0/1578072992 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:07:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.933+0000 7f0ebcc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e57cadc30 0x558e57cb00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.933+0000 7f0ebd600640 1 --2- 172.21.15.111:0/1578072992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e57ca8ba0 0x558e57cad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.933+0000 7f0ebe000640 1 -- 172.21.15.111:0/1578072992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e57cadc30 msgr2=0x558e57cb00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.933+0000 7f0ebe000640 1 --2- 172.21.15.111:0/1578072992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e57cadc30 0x558e57cb00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.933+0000 7f0ebe000640 1 -- 172.21.15.111:0/1578072992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e57ca8ba0 msgr2=0x558e57cad660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.934+0000 7f0ebe000640 1 --2- 172.21.15.111:0/1578072992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e57ca8ba0 0x558e57cad660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.934+0000 7f0ebe000640 1 -- 172.21.15.111:0/1578072992 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e57920210 con 0x558e57ca8260 2024-09-15T23:07:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.934+0000 7f0eb6000640 1 --2- 172.21.15.111:0/427400851 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0ea00a3240 0x7f0ea00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.934+0000 7f0ebe000640 1 --2- 172.21.15.111:0/1578072992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e57ca8260 0x558e57ca8660 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009ca0 tx=0x7f0eb00093c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4ce5e253dd658b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.934+0000 7f0eb6000640 1 -- 172.21.15.111:0/427400851 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f0eac0cfbc0 con 0x7f0eb8101580 2024-09-15T23:07:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.934+0000 7f0ebd600640 1 --2- 172.21.15.111:0/427400851 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0ea00a3240 0x7f0ea00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.934+0000 7f0eaa600640 1 -- 172.21.15.111:0/1578072992 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0031650 con 0x558e57ca8260 2024-09-15T23:07:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.934+0000 7f0eaa600640 1 -- 172.21.15.111:0/1578072992 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eb0036030 con 0x558e57ca8260 2024-09-15T23:07:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.934+0000 7f0eaa600640 1 -- 172.21.15.111:0/1578072992 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0004720 con 0x558e57ca8260 2024-09-15T23:07:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.935+0000 7f0ec2e72740 1 -- 172.21.15.111:0/1578072992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e57ca8260 msgr2=0x558e57ca8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.935+0000 7f0ebd600640 1 --2- 172.21.15.111:0/427400851 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0ea00a3240 0x7f0ea00a5700 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0eac098c30 tx=0x7f0eac0d36b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.935+0000 7f0ec2e72740 1 --2- 172.21.15.111:0/1578072992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e57ca8260 0x558e57ca8660 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009ca0 tx=0x7f0eb00093c0 comp rx=0 tx=0).stop 2024-09-15T23:07:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.935+0000 7f0ec2e72740 1 -- 172.21.15.111:0/1578072992 shutdown_connections 2024-09-15T23:07:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.935+0000 7f0ec2e72740 1 --2- 172.21.15.111:0/1578072992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e57cadc30 0x558e57cb00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.935+0000 7f0ec2e72740 1 --2- 172.21.15.111:0/1578072992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e57ca8ba0 0x558e57cad660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.935+0000 7f0ec2e72740 1 --2- 172.21.15.111:0/1578072992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e57ca8260 0x558e57ca8660 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.935+0000 7f0ec2e72740 1 -- 172.21.15.111:0/1578072992 >> 172.21.15.111:0/1578072992 conn(0x558e57ca38d0 msgr2=0x558e57ca3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.936+0000 7f0ec2e72740 1 -- 172.21.15.111:0/1578072992 shutdown_connections 2024-09-15T23:07:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.936+0000 7f0ec2e72740 1 -- 172.21.15.111:0/1578072992 wait complete. 2024-09-15T23:07:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.936+0000 7f0ec2e72740 1 -- start start 2024-09-15T23:07:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.936+0000 7f0ec2e72740 1 Processor -- start 2024-09-15T23:07:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.937+0000 7f0ec2e72740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e57ca8260 0x558e57cb2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.937+0000 7f0ec2e72740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e57ca8ba0 0x558e57cb2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.937+0000 7f0ec2e72740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e57cadc30 0x558e57cb97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.937+0000 7f0ec2e72740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x558e57cb0a20 con 0x558e57ca8ba0 2024-09-15T23:07:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.937+0000 7f0ec2e72740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558e57b96090 con 0x558e57cadc30 2024-09-15T23:07:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.937+0000 7f0ec2e72740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558e57cb0ba0 con 0x558e57ca8260 2024-09-15T23:07:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.937+0000 7f0ec2e72740 10 client.0 _ll_register_callbacks cb 0x558e57beaf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:07:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.937+0000 7f0ebe000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e57ca8260 0x558e57cb2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.937+0000 7f0ebe000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e57ca8260 0x558e57cb2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:57690/0 (socket says 172.21.15.111:57690) 2024-09-15T23:07:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.937+0000 7f0ebe000640 1 -- 172.21.15.111:0/2123787428 learned_addr learned my addr 172.21.15.111:0/2123787428 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:07:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.938+0000 7f0ebe000640 1 -- 172.21.15.111:0/2123787428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e57cadc30 msgr2=0x558e57cb97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.938+0000 7f0ebe000640 1 --2- 172.21.15.111:0/2123787428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e57cadc30 0x558e57cb97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.938+0000 7f0ebe000640 1 -- 172.21.15.111:0/2123787428 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e57ca8ba0 msgr2=0x558e57cb2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.938+0000 7f0ebe000640 1 --2- 172.21.15.111:0/2123787428 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e57ca8ba0 0x558e57cb2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.938+0000 7f0ebe000640 1 -- 172.21.15.111:0/2123787428 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e579ec620 con 0x558e57ca8260 2024-09-15T23:07:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.938+0000 7f0ebe000640 1 --2- 172.21.15.111:0/2123787428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e57ca8260 0x558e57cb2230 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009bf0 tx=0x7f0eb0009940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b366d96dfd00e902 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.938+0000 7f0e9ea00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:07:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.938+0000 7f0e9ea00640 1 -- 172.21.15.111:0/2123787428 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0056060 con 0x558e57ca8260 2024-09-15T23:07:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.938+0000 7f0ec2e72740 1 -- 172.21.15.111:0/2123787428 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0eb0036030 con 0x558e57ca8260 2024-09-15T23:07:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.938+0000 7f0e9ea00640 1 -- 172.21.15.111:0/2123787428 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f0eb0003970 con 0x558e57ca8260 2024-09-15T23:07:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.939+0000 7f0e9ea00640 1 -- 172.21.15.111:0/2123787428 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0eb0003b70 con 0x558e57ca8260 2024-09-15T23:07:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.939+0000 7f0ec2e72740 10 client.26635 fetch_fsmap learned FSMap version 35 2024-09-15T23:07:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.939+0000 7f0ec2e72740 1 -- 172.21.15.111:0/2123787428 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558e57cb0a20 con 0x558e57ca8260 2024-09-15T23:07:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.939+0000 7f0e9ea00640 1 -- 172.21.15.111:0/2123787428 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eb0036030 con 0x558e57ca8260 2024-09-15T23:07:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.939+0000 7f0e9ea00640 1 -- 172.21.15.111:0/2123787428 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f0eb0043070 con 0x558e57ca8260 2024-09-15T23:07:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.940+0000 7f0ec2e72740 10 client.26635 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:07:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.940+0000 7f0ec2e72740 10 client.26635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:07:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.940+0000 7f0ec2e72740 20 client.26635 populate_metadata read hostname 'smithi111' 2024-09-15T23:07:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.940+0000 7f0ec2e72740 1 --2- 172.21.15.111:0/2123787428 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x558e57cbd9a0 0x558e57cbfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.940+0000 7f0ec2e72740 4 client.26635 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:07:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.940+0000 7f0ec2e72740 1 -- 172.21.15.111:0/2123787428 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e57c1a830 con 0x558e57cbd9a0 2024-09-15T23:07:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.940+0000 7f0ebcc00640 1 --2- 172.21.15.111:0/2123787428 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x558e57cbd9a0 0x558e57cbfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.941+0000 7f0ebcc00640 1 --2- 172.21.15.111:0/2123787428 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x558e57cbd9a0 0x558e57cbfd90 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4d087fef9e1935f server_cookie=cdc320c5d4302ec9 in_seq=0 out_seq=0 2024-09-15T23:07:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.942+0000 7f0e9ea00640 10 client.26635 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:07:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.943+0000 7f0e9ea00640 1 -- 172.21.15.111:0/2123787428 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e57c1a830 con 0x558e57cbd9a0 2024-09-15T23:07:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.943+0000 7f0e9ea00640 10 client.26635 handle_command_reply: tid=0 2024-09-15T23:07:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.943+0000 7f0ec2e72740 1 client.26635 shutdown 2024-09-15T23:07:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.943+0000 7f0ec2e72740 1 -- 172.21.15.111:0/2123787428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e57ca8260 msgr2=0x558e57cb2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.943+0000 7f0ec2e72740 1 --2- 172.21.15.111:0/2123787428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e57ca8260 0x558e57cb2230 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009bf0 tx=0x7f0eb0009940 comp rx=0 tx=0).stop 2024-09-15T23:07:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.943+0000 7f0ec2e72740 1 -- 172.21.15.111:0/2123787428 shutdown_connections 2024-09-15T23:07:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.943+0000 7f0ec2e72740 1 --2- 172.21.15.111:0/2123787428 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x558e57cbd9a0 0x558e57cbfd90 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.943+0000 7f0ec2e72740 1 --2- 172.21.15.111:0/2123787428 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e57cadc30 0x558e57cb97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.943+0000 7f0ec2e72740 1 --2- 172.21.15.111:0/2123787428 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e57ca8ba0 0x558e57cb2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.943+0000 7f0ec2e72740 1 --2- 172.21.15.111:0/2123787428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e57ca8260 0x558e57cb2230 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.943+0000 7f0ec2e72740 1 -- 172.21.15.111:0/2123787428 >> 172.21.15.111:0/2123787428 conn(0x558e57ca38d0 msgr2=0x558e57ca50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.943+0000 7f0e9ea00640 0 client.26635 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:07:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.944+0000 7f0ec2e72740 1 -- 172.21.15.111:0/2123787428 shutdown_connections 2024-09-15T23:07:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.944+0000 7f0ec2e72740 1 -- 172.21.15.111:0/2123787428 wait complete. 2024-09-15T23:07:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.944+0000 7f0ec2e72740 20 client.26635 trim_cache size 0 max 16384 2024-09-15T23:07:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.976+0000 7f0ec2e72740 1 Processor -- start 2024-09-15T23:07:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.976+0000 7f0ec2e72740 1 -- start start 2024-09-15T23:07:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.977+0000 7f0ec2e72740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e57ca38d0 0x558e57cb26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.977+0000 7f0ec2e72740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e57ca8260 0x558e57cb96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.977+0000 7f0ec2e72740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e57ca8ba0 0x558e57ca21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.977+0000 7f0ec2e72740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81ad6d0 con 0x558e57ca38d0 2024-09-15T23:07:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.977+0000 7f0ec2e72740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81aafa0 con 0x558e57ca8ba0 2024-09-15T23:07:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.977+0000 7f0ec2e72740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81a8df0 con 0x558e57ca8260 2024-09-15T23:07:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.977+0000 7f0ebcc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e57ca8260 0x558e57cb96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.977+0000 7f0ebcc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e57ca8260 0x558e57cb96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:57706/0 (socket says 172.21.15.111:57706) 2024-09-15T23:07:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.977+0000 7f0ebcc00640 1 -- 172.21.15.111:0/492259992 learned_addr learned my addr 172.21.15.111:0/492259992 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:07:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.978+0000 7f0ebcc00640 1 -- 172.21.15.111:0/492259992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e57ca8ba0 msgr2=0x558e57ca21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.978+0000 7f0ebcc00640 1 --2- 172.21.15.111:0/492259992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e57ca8ba0 0x558e57ca21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.978+0000 7f0ebcc00640 1 -- 172.21.15.111:0/492259992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e57ca38d0 msgr2=0x558e57cb26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.978+0000 7f0ebcc00640 1 --2- 172.21.15.111:0/492259992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e57ca38d0 0x558e57cb26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.978+0000 7f0ebcc00640 1 -- 172.21.15.111:0/492259992 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e57ca36f0 con 0x558e57ca8260 2024-09-15T23:07:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.979+0000 7f0ebcc00640 1 --2- 172.21.15.111:0/492259992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e57ca8260 0x558e57cb96d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8075d10 tx=0x7f0ea4004410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42661ceaf251a5fd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.979+0000 7f0ea9c00640 1 -- 172.21.15.111:0/492259992 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4021070 con 0x558e57ca8260 2024-09-15T23:07:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.979+0000 7f0ea9c00640 1 -- 172.21.15.111:0/492259992 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea4003d20 con 0x558e57ca8260 2024-09-15T23:07:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.979+0000 7f0ea9c00640 1 -- 172.21.15.111:0/492259992 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea40107b0 con 0x558e57ca8260 2024-09-15T23:07:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.980+0000 7f0ec2e72740 1 -- 172.21.15.111:0/492259992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e57ca8260 msgr2=0x558e57cb96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.980+0000 7f0ec2e72740 1 --2- 172.21.15.111:0/492259992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e57ca8260 0x558e57cb96d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8075d10 tx=0x7f0ea4004410 comp rx=0 tx=0).stop 2024-09-15T23:07:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.981+0000 7f0ec2e72740 1 -- 172.21.15.111:0/492259992 shutdown_connections 2024-09-15T23:07:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.981+0000 7f0ec2e72740 1 --2- 172.21.15.111:0/492259992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e57ca8ba0 0x558e57ca21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.981+0000 7f0ec2e72740 1 --2- 172.21.15.111:0/492259992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e57ca8260 0x558e57cb96d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.981+0000 7f0ec2e72740 1 --2- 172.21.15.111:0/492259992 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e57ca38d0 0x558e57cb26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.981+0000 7f0ec2e72740 1 -- 172.21.15.111:0/492259992 >> 172.21.15.111:0/492259992 conn(0x558e57cad780 msgr2=0x558e57cb1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.982+0000 7f0ec2e72740 1 -- 172.21.15.111:0/492259992 shutdown_connections 2024-09-15T23:07:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.982+0000 7f0ec2e72740 1 -- 172.21.15.111:0/492259992 wait complete. 2024-09-15T23:07:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.982+0000 7f0ec2e72740 1 -- start start 2024-09-15T23:07:41.983 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.982+0000 7f0ec2e72740 1 Processor -- start 2024-09-15T23:07:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.983+0000 7f0ec2e72740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e57ca38d0 0x558e57cb7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:41.984 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.983+0000 7f0ec2e72740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e57ca8260 0x558e57cb76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.984+0000 7f0ebe000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e57ca38d0 0x558e57cb7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.984+0000 7f0ebe000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e57ca38d0 0x558e57cb7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:42748/0 (socket says 172.21.15.111:42748) 2024-09-15T23:07:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.984+0000 7f0ec2e72740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e57ca8ba0 0x558e57cb86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.984+0000 7f0ec2e72740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81a9460 con 0x558e57ca8260 2024-09-15T23:07:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.984+0000 7f0ec2e72740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81aa8d0 con 0x558e57ca38d0 2024-09-15T23:07:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.984+0000 7f0ec2e72740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb81a9b30 con 0x558e57ca8ba0 2024-09-15T23:07:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.984+0000 7f0ec2e72740 10 client.0 _ll_register_callbacks cb 0x7f0eac0d3580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:07:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.984+0000 7f0ebcc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e57ca8260 0x558e57cb76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.984+0000 7f0ebe000640 1 -- 172.21.15.111:0/2369273950 learned_addr learned my addr 172.21.15.111:0/2369273950 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:07:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.985+0000 7f0ebe000640 1 --2- 172.21.15.111:0/2369273950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e57ca8ba0 0x558e57cb86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.985+0000 7f0ebcc00640 1 -- 172.21.15.111:0/2369273950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e57ca8ba0 msgr2=0x558e57cb86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.985+0000 7f0ebcc00640 1 --2- 172.21.15.111:0/2369273950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e57ca8ba0 0x558e57cb86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.985+0000 7f0ebcc00640 1 -- 172.21.15.111:0/2369273950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e57ca38d0 msgr2=0x558e57cb7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.985+0000 7f0ebcc00640 1 --2- 172.21.15.111:0/2369273950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e57ca38d0 0x558e57cb7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.985+0000 7f0ebcc00640 1 -- 172.21.15.111:0/2369273950 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e57cb8cd0 con 0x558e57ca8260 2024-09-15T23:07:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.985+0000 7f0ebcc00640 1 --2- 172.21.15.111:0/2369273950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e57ca8260 0x558e57cb76b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x558e57c9eac0 tx=0x7f0ea4003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f6ee738966c497 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.985+0000 7f0e9ea00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:07:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.986+0000 7f0e9ea00640 1 -- 172.21.15.111:0/2369273950 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea402c020 con 0x558e57ca8260 2024-09-15T23:07:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.986+0000 7f0e9ea00640 1 -- 172.21.15.111:0/2369273950 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f0ea401c3c0 con 0x558e57ca8260 2024-09-15T23:07:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.986+0000 7f0ec2e72740 1 -- 172.21.15.111:0/2369273950 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ea4003d20 con 0x558e57ca8260 2024-09-15T23:07:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.986+0000 7f0e9ea00640 1 -- 172.21.15.111:0/2369273950 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0ea4021030 con 0x558e57ca8260 2024-09-15T23:07:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.987+0000 7f0ec2e72740 10 client.16707 fetch_fsmap learned FSMap version 35 2024-09-15T23:07:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.987+0000 7f0ec2e72740 1 -- 172.21.15.111:0/2369273950 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558e57d31070 con 0x558e57ca8260 2024-09-15T23:07:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.988+0000 7f0e9ea00640 1 -- 172.21.15.111:0/2369273950 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ea4003d20 con 0x558e57ca8260 2024-09-15T23:07:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.988+0000 7f0e9ea00640 1 -- 172.21.15.111:0/2369273950 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f0ea4027070 con 0x558e57ca8260 2024-09-15T23:07:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.988+0000 7f0ec2e72740 10 client.16707 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:07:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.988+0000 7f0ec2e72740 10 client.16707 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:07:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.988+0000 7f0ec2e72740 20 client.16707 populate_metadata read hostname 'smithi111' 2024-09-15T23:07:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.988+0000 7f0ec2e72740 1 --2- 172.21.15.111:0/2369273950 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x558e57cadc30 0x558e57d33910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.988+0000 7f0ec2e72740 4 client.16707 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:07:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.988+0000 7f0ec2e72740 1 -- 172.21.15.111:0/2369273950 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558e57c14b80 con 0x558e57cadc30 2024-09-15T23:07:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.988+0000 7f0ebe000640 1 --2- 172.21.15.111:0/2369273950 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x558e57cadc30 0x558e57d33910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.989+0000 7f0e9ea00640 10 client.16707 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:07:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:41.989+0000 7f0ebe000640 1 --2- 172.21.15.111:0/2369273950 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x558e57cadc30 0x558e57d33910 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4537bfeb898cf68 server_cookie=b105cb7b22cf8553 in_seq=0 out_seq=0 2024-09-15T23:07:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:42.285+0000 7f0e9ea00640 1 -- 172.21.15.111:0/2369273950 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558e57c14b80 con 0x558e57cadc30 2024-09-15T23:07:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:42.285+0000 7f0e9ea00640 10 client.16707 handle_command_reply: tid=0 2024-09-15T23:07:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:42.285+0000 7f0ec2e72740 1 client.16707 shutdown 2024-09-15T23:07:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:42.285+0000 7f0ec2e72740 1 -- 172.21.15.111:0/2369273950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e57ca8260 msgr2=0x558e57cb76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:42.285+0000 7f0ec2e72740 1 --2- 172.21.15.111:0/2369273950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e57ca8260 0x558e57cb76b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x558e57c9eac0 tx=0x7f0ea4003ed0 comp rx=0 tx=0).stop 2024-09-15T23:07:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:42.285+0000 7f0ec2e72740 1 -- 172.21.15.111:0/2369273950 shutdown_connections 2024-09-15T23:07:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:42.285+0000 7f0ec2e72740 1 --2- 172.21.15.111:0/2369273950 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x558e57cadc30 0x558e57d33910 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:42.285+0000 7f0ec2e72740 1 --2- 172.21.15.111:0/2369273950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558e57ca8ba0 0x558e57cb86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:42.285+0000 7f0ec2e72740 1 --2- 172.21.15.111:0/2369273950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558e57ca8260 0x558e57cb76b0 secure :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0x558e57c9eac0 tx=0x7f0ea4003ed0 comp rx=0 tx=0).stop 2024-09-15T23:07:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:42.285+0000 7f0ec2e72740 1 --2- 172.21.15.111:0/2369273950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558e57ca38d0 0x558e57cb7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:42.285+0000 7f0ec2e72740 1 -- 172.21.15.111:0/2369273950 >> 172.21.15.111:0/2369273950 conn(0x558e57cad780 msgr2=0x558e57ca4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:42.285+0000 7f0e9ea00640 0 client.16707 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:07:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:42.286+0000 7f0ec2e72740 1 -- 172.21.15.111:0/2369273950 shutdown_connections 2024-09-15T23:07:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:42.286+0000 7f0ec2e72740 1 -- 172.21.15.111:0/2369273950 wait complete. 2024-09-15T23:07:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:42.286+0000 7f0ec2e72740 20 client.16707 trim_cache size 0 max 16384 2024-09-15T23:07:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:07:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:07:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:07:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:07:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:42.287+0000 7f0ebea00640 1 -- 172.21.15.111:0/427400851 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0ea00a3240 msgr2=0x7f0ea00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:42.287+0000 7f0ebea00640 1 --2- 172.21.15.111:0/427400851 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0ea00a3240 0x7f0ea00a5700 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0eac098c30 tx=0x7f0eac0d36b0 comp rx=0 tx=0).stop 2024-09-15T23:07:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:42.287+0000 7f0ebea00640 1 -- 172.21.15.111:0/427400851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0eb8101580 msgr2=0x7f0eb8075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:42.287+0000 7f0ebea00640 1 --2- 172.21.15.111:0/427400851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0eb8101580 0x7f0eb8075fa0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00e9c0 tx=0x7f0eac00ee80 comp rx=0 tx=0).stop 2024-09-15T23:07:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:42.287+0000 7f0ebea00640 1 -- 172.21.15.111:0/427400851 shutdown_connections 2024-09-15T23:07:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:42.287+0000 7f0ebea00640 1 --2- 172.21.15.111:0/427400851 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f0ea00a3240 0x7f0ea00a5700 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:42.287+0000 7f0ebea00640 1 --2- 172.21.15.111:0/427400851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0eb8103050 0x7f0eb8074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:42.287+0000 7f0ebea00640 1 --2- 172.21.15.111:0/427400851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0eb8101f40 0x7f0eb80745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:42.287+0000 7f0ebea00640 1 --2- 172.21.15.111:0/427400851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0eb8101580 0x7f0eb8075fa0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:42.287+0000 7f0ebea00640 1 -- 172.21.15.111:0/427400851 >> 172.21.15.111:0/427400851 conn(0x7f0eb80fd0d0 msgr2=0x7f0eb8105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:42.287+0000 7f0ebea00640 1 -- 172.21.15.111:0/427400851 shutdown_connections 2024-09-15T23:07:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:42.287+0000 7f0ebea00640 1 -- 172.21.15.111:0/427400851 wait complete. 2024-09-15T23:07:42.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:42 smithi192 ceph-mon[35427]: pgmap v1004: 97 pgs: 97 active+clean; 562 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 4.3 MiB/s wr, 568 op/s 2024-09-15T23:07:42.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:42 smithi121 ceph-mon[34815]: pgmap v1004: 97 pgs: 97 active+clean; 562 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 4.3 MiB/s wr, 568 op/s 2024-09-15T23:07:42.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:42 smithi111 ceph-mon[30629]: pgmap v1004: 97 pgs: 97 active+clean; 562 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 4.3 MiB/s wr, 568 op/s 2024-09-15T23:07:42.681 INFO:tasks.workunit.client.0.smithi111.stdout: 10 167929 17.44 MB/sec execute 370 sec latency 174.922 ms 2024-09-15T23:07:43.681 INFO:tasks.workunit.client.0.smithi111.stdout: 10 168459 17.47 MB/sec execute 371 sec latency 74.718 ms 2024-09-15T23:07:44.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:44 smithi192 ceph-mon[35427]: pgmap v1005: 97 pgs: 97 active+clean; 566 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.9 MiB/s wr, 500 op/s 2024-09-15T23:07:44.335 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:44 smithi111 ceph-mon[30629]: pgmap v1005: 97 pgs: 97 active+clean; 566 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.9 MiB/s wr, 500 op/s 2024-09-15T23:07:44.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:44 smithi121 ceph-mon[34815]: pgmap v1005: 97 pgs: 97 active+clean; 566 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.9 MiB/s wr, 500 op/s 2024-09-15T23:07:44.681 INFO:tasks.workunit.client.0.smithi111.stdout: 10 169203 17.50 MB/sec execute 372 sec latency 58.097 ms 2024-09-15T23:07:45.682 INFO:tasks.workunit.client.0.smithi111.stdout: 10 169994 17.58 MB/sec execute 373 sec latency 80.489 ms 2024-09-15T23:07:46.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:46 smithi192 ceph-mon[35427]: pgmap v1006: 97 pgs: 97 active+clean; 565 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.1 MiB/s wr, 563 op/s 2024-09-15T23:07:46.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:46 smithi121 ceph-mon[34815]: pgmap v1006: 97 pgs: 97 active+clean; 565 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.1 MiB/s wr, 563 op/s 2024-09-15T23:07:46.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:46 smithi111 ceph-mon[30629]: pgmap v1006: 97 pgs: 97 active+clean; 565 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.1 MiB/s wr, 563 op/s 2024-09-15T23:07:46.682 INFO:tasks.workunit.client.0.smithi111.stdout: 10 170935 17.67 MB/sec execute 374 sec latency 54.733 ms 2024-09-15T23:07:47.682 INFO:tasks.workunit.client.0.smithi111.stdout: 10 171647 17.75 MB/sec execute 375 sec latency 55.453 ms 2024-09-15T23:07:48.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:48 smithi111 ceph-mon[30629]: pgmap v1007: 97 pgs: 97 active+clean; 570 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.7 MiB/s wr, 513 op/s 2024-09-15T23:07:48.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:48 smithi192 ceph-mon[35427]: pgmap v1007: 97 pgs: 97 active+clean; 570 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.7 MiB/s wr, 513 op/s 2024-09-15T23:07:48.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:48 smithi121 ceph-mon[34815]: pgmap v1007: 97 pgs: 97 active+clean; 570 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.7 MiB/s wr, 513 op/s 2024-09-15T23:07:48.682 INFO:tasks.workunit.client.0.smithi111.stdout: 10 171767 17.72 MB/sec execute 376 sec latency 893.537 ms 2024-09-15T23:07:49.682 INFO:tasks.workunit.client.0.smithi111.stdout: 10 172130 17.71 MB/sec execute 377 sec latency 1268.162 ms 2024-09-15T23:07:50.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:50 smithi192 ceph-mon[35427]: pgmap v1008: 97 pgs: 97 active+clean; 578 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.3 MiB/s wr, 588 op/s 2024-09-15T23:07:50.362 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:50 smithi111 ceph-mon[30629]: pgmap v1008: 97 pgs: 97 active+clean; 578 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.3 MiB/s wr, 588 op/s 2024-09-15T23:07:50.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:50 smithi121 ceph-mon[34815]: pgmap v1008: 97 pgs: 97 active+clean; 578 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.3 MiB/s wr, 588 op/s 2024-09-15T23:07:50.682 INFO:tasks.workunit.client.0.smithi111.stdout: 10 172616 17.73 MB/sec execute 378 sec latency 97.011 ms 2024-09-15T23:07:51.682 INFO:tasks.workunit.client.0.smithi111.stdout: 10 173238 17.79 MB/sec execute 379 sec latency 94.193 ms 2024-09-15T23:07:52.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:52 smithi192 ceph-mon[35427]: pgmap v1009: 97 pgs: 97 active+clean; 586 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.6 MiB/s wr, 622 op/s 2024-09-15T23:07:52.377 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:52 smithi111 ceph-mon[30629]: pgmap v1009: 97 pgs: 97 active+clean; 586 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.6 MiB/s wr, 622 op/s 2024-09-15T23:07:52.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:52 smithi121 ceph-mon[34815]: pgmap v1009: 97 pgs: 97 active+clean; 586 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.6 MiB/s wr, 622 op/s 2024-09-15T23:07:52.682 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174024 17.81 MB/sec execute 380 sec latency 56.285 ms 2024-09-15T23:07:53.217 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:53 smithi111 ceph-mon[30629]: pgmap v1010: 97 pgs: 97 active+clean; 586 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.9 MiB/s wr, 519 op/s 2024-09-15T23:07:53.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:53 smithi121 ceph-mon[34815]: pgmap v1010: 97 pgs: 97 active+clean; 586 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.9 MiB/s wr, 519 op/s 2024-09-15T23:07:53.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:53 smithi192 ceph-mon[35427]: pgmap v1010: 97 pgs: 97 active+clean; 586 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.9 MiB/s wr, 519 op/s 2024-09-15T23:07:53.682 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174256 17.79 MB/sec execute 381 sec latency 744.775 ms 2024-09-15T23:07:54.682 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174256 17.74 MB/sec execute 382 sec latency 1744.832 ms 2024-09-15T23:07:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfbe00640 1 -- 172.21.15.111:0/2248860654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2dfc103050 msgr2=0x7f2dfc103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfbe00640 1 --2- 172.21.15.111:0/2248860654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2dfc103050 0x7f2dfc103450 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009900 tx=0x7f2dec02a3f0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfbe00640 1 -- 172.21.15.111:0/2248860654 shutdown_connections 2024-09-15T23:07:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfbe00640 1 --2- 172.21.15.111:0/2248860654 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2dfc101f40 0x7f2dfc109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfbe00640 1 --2- 172.21.15.111:0/2248860654 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2dfc101580 0x7f2dfc101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfbe00640 1 --2- 172.21.15.111:0/2248860654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2dfc103050 0x7f2dfc103450 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfbe00640 1 -- 172.21.15.111:0/2248860654 >> 172.21.15.111:0/2248860654 conn(0x7f2dfc0fd0d0 msgr2=0x7f2dfc0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfbe00640 1 -- 172.21.15.111:0/2248860654 shutdown_connections 2024-09-15T23:07:55.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfbe00640 1 -- 172.21.15.111:0/2248860654 wait complete. 2024-09-15T23:07:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfbe00640 1 Processor -- start 2024-09-15T23:07:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfbe00640 1 -- start start 2024-09-15T23:07:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2dfc101580 0x7f2dfc19a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:55.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfbe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2dfc101f40 0x7f2dfc19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfbe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2dfc103050 0x7f2dfc1a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfbe00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc10d720 con 0x7f2dfc101f40 2024-09-15T23:07:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfbe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc10d5a0 con 0x7f2dfc101580 2024-09-15T23:07:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfbe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc10d8a0 con 0x7f2dfc103050 2024-09-15T23:07:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfa000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2dfc101f40 0x7f2dfc19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:55.411 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfaa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2dfc101580 0x7f2dfc19a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfa000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2dfc101f40 0x7f2dfc19a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59528/0 (socket says 172.21.15.111:59528) 2024-09-15T23:07:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfaa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2dfc101580 0x7f2dfc19a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:58514/0 (socket says 172.21.15.111:58514) 2024-09-15T23:07:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfa000640 1 -- 172.21.15.111:0/32173288 learned_addr learned my addr 172.21.15.111:0/32173288 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:07:55.416 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfb400640 1 --2- 172.21.15.111:0/32173288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2dfc103050 0x7f2dfc1a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfa000640 1 -- 172.21.15.111:0/32173288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2dfc103050 msgr2=0x7f2dfc1a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfa000640 1 --2- 172.21.15.111:0/32173288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2dfc103050 0x7f2dfc1a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfa000640 1 -- 172.21.15.111:0/32173288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2dfc101580 msgr2=0x7f2dfc19a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfa000640 1 --2- 172.21.15.111:0/32173288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2dfc101580 0x7f2dfc19a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfa000640 1 -- 172.21.15.111:0/32173288 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dfc1a1c30 con 0x7f2dfc101f40 2024-09-15T23:07:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.404+0000 7f2dfa000640 1 --2- 172.21.15.111:0/32173288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2dfc101f40 0x7f2dfc19a550 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2de800ea60 tx=0x7f2de800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.410+0000 7f2df3400640 1 -- 172.21.15.111:0/32173288 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de800cde0 con 0x7f2dfc101f40 2024-09-15T23:07:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.410+0000 7f2df3400640 1 -- 172.21.15.111:0/32173288 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de80035f0 con 0x7f2dfc101f40 2024-09-15T23:07:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.410+0000 7f2df3400640 1 -- 172.21.15.111:0/32173288 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8010670 con 0x7f2dfc101f40 2024-09-15T23:07:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.410+0000 7f2dfbe00640 1 -- 172.21.15.111:0/32173288 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dfc1a1f20 con 0x7f2dfc101f40 2024-09-15T23:07:55.417 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.410+0000 7f2dfbe00640 1 -- 172.21.15.111:0/32173288 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dfc1a2390 con 0x7f2dfc101f40 2024-09-15T23:07:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2e049e0740 1 Processor -- start 2024-09-15T23:07:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2e049e0740 1 -- start start 2024-09-15T23:07:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2e049e0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2b260 0x55c0f3b2b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2e049e0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0f3b2bba0 0x55c0f3b30660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2e049e0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f3b30c30 0x55c0f3b330f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2e049e0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c0f3a90100 con 0x55c0f3b2b260 2024-09-15T23:07:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2e049e0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c0f3a381b0 con 0x55c0f3b2bba0 2024-09-15T23:07:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2e049e0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c0f3a8c0c0 con 0x55c0f3b30c30 2024-09-15T23:07:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2dfb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2b260 0x55c0f3b2b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2dfb400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2b260 0x55c0f3b2b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59530/0 (socket says 172.21.15.111:59530) 2024-09-15T23:07:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2dfb400640 1 -- 172.21.15.111:0/3992728349 learned_addr learned my addr 172.21.15.111:0/3992728349 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:07:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2dfaa00640 1 --2- 172.21.15.111:0/3992728349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0f3b2bba0 0x55c0f3b30660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:55.418 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2df3400640 1 -- 172.21.15.111:0/32173288 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f2de80037a0 con 0x7f2dfc101f40 2024-09-15T23:07:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2dfaa00640 1 -- 172.21.15.111:0/3992728349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f3b30c30 msgr2=0x55c0f3b330f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:07:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2dfaa00640 1 --2- 172.21.15.111:0/3992728349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f3b30c30 0x55c0f3b330f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2dfaa00640 1 -- 172.21.15.111:0/3992728349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2b260 msgr2=0x55c0f3b2b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2dfaa00640 1 --2- 172.21.15.111:0/3992728349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2b260 0x55c0f3b2b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2dfaa00640 1 -- 172.21.15.111:0/3992728349 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0f37a3210 con 0x55c0f3b2bba0 2024-09-15T23:07:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2dfb400640 1 --2- 172.21.15.111:0/3992728349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2b260 0x55c0f3b2b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:07:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2dfaa00640 1 --2- 172.21.15.111:0/3992728349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0f3b2bba0 0x55c0f3b30660 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009230 tx=0x7f2dec004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a444bcda2884dab4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2de7e00640 1 -- 172.21.15.111:0/3992728349 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec035040 con 0x55c0f3b2bba0 2024-09-15T23:07:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2de7e00640 1 -- 172.21.15.111:0/3992728349 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dec005520 con 0x55c0f3b2bba0 2024-09-15T23:07:55.419 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2de7e00640 1 -- 172.21.15.111:0/3992728349 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec0316b0 con 0x55c0f3b2bba0 2024-09-15T23:07:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2df3400640 1 --2- 172.21.15.111:0/32173288 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2de00a3240 0x7f2de00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2e049e0740 1 -- 172.21.15.111:0/3992728349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0f3b2bba0 msgr2=0x55c0f3b30660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2e049e0740 1 --2- 172.21.15.111:0/3992728349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0f3b2bba0 0x55c0f3b30660 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009230 tx=0x7f2dec004040 comp rx=0 tx=0).stop 2024-09-15T23:07:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2df3400640 1 -- 172.21.15.111:0/32173288 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f2de80cfbe0 con 0x7f2dfc101f40 2024-09-15T23:07:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2e049e0740 1 -- 172.21.15.111:0/3992728349 shutdown_connections 2024-09-15T23:07:55.423 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2e049e0740 1 --2- 172.21.15.111:0/3992728349 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f3b30c30 0x55c0f3b330f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2e049e0740 1 --2- 172.21.15.111:0/3992728349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0f3b2bba0 0x55c0f3b30660 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2e049e0740 1 --2- 172.21.15.111:0/3992728349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2b260 0x55c0f3b2b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2e049e0740 1 -- 172.21.15.111:0/3992728349 >> 172.21.15.111:0/3992728349 conn(0x55c0f3b268d0 msgr2=0x55c0f3b26cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2dfb400640 1 --2- 172.21.15.111:0/32173288 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2de00a3240 0x7f2de00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2e049e0740 1 -- 172.21.15.111:0/3992728349 shutdown_connections 2024-09-15T23:07:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2e049e0740 1 -- 172.21.15.111:0/3992728349 wait complete. 2024-09-15T23:07:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2dfb400640 1 --2- 172.21.15.111:0/32173288 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2de00a3240 0x7f2de00a5700 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2df4005fd0 tx=0x7f2df4009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2e049e0740 1 -- start start 2024-09-15T23:07:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2e049e0740 1 Processor -- start 2024-09-15T23:07:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2e049e0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2b260 0x55c0f3b35450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2e049e0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0f3b2bba0 0x55c0f3b35990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:55.424 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2e049e0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f3b30c30 0x55c0f3b3c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2e049e0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c0f3b33ba0 con 0x55c0f3b2b260 2024-09-15T23:07:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2e049e0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c0f3a19090 con 0x55c0f3b2bba0 2024-09-15T23:07:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2e049e0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c0f3b33d20 con 0x55c0f3b30c30 2024-09-15T23:07:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2e049e0740 10 client.0 _ll_register_callbacks cb 0x55c0f3a6df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:07:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2dfaa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2b260 0x55c0f3b35450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2dfaa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2b260 0x55c0f3b35450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59534/0 (socket says 172.21.15.111:59534) 2024-09-15T23:07:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2dfaa00640 1 -- 172.21.15.111:0/3512595481 learned_addr learned my addr 172.21.15.111:0/3512595481 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:07:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2dfa000640 1 --2- 172.21.15.111:0/3512595481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0f3b2bba0 0x55c0f3b35990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2dfaa00640 1 -- 172.21.15.111:0/3512595481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f3b30c30 msgr2=0x55c0f3b3c9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:07:55.425 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2dfaa00640 1 --2- 172.21.15.111:0/3512595481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f3b30c30 0x55c0f3b3c9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2dfaa00640 1 -- 172.21.15.111:0/3512595481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0f3b2bba0 msgr2=0x55c0f3b35990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2dfaa00640 1 --2- 172.21.15.111:0/3512595481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0f3b2bba0 0x55c0f3b35990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2dfaa00640 1 -- 172.21.15.111:0/3512595481 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0f386f620 con 0x55c0f3b2b260 2024-09-15T23:07:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2dfaa00640 1 --2- 172.21.15.111:0/3512595481 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2b260 0x55c0f3b35450 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2dec02a9e0 tx=0x7f2dec004f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b46ebdeed3f95c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2de4c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:07:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2de4c00640 1 -- 172.21.15.111:0/3512595481 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec035070 con 0x55c0f3b2b260 2024-09-15T23:07:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2de4c00640 1 -- 172.21.15.111:0/3512595481 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f2dec005000 con 0x55c0f3b2b260 2024-09-15T23:07:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.414+0000 7f2e049e0740 1 -- 172.21.15.111:0/3512595481 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dec005520 con 0x55c0f3b2b260 2024-09-15T23:07:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2de4c00640 1 -- 172.21.15.111:0/3512595481 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2dec009520 con 0x55c0f3b2b260 2024-09-15T23:07:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2e049e0740 10 client.16719 fetch_fsmap learned FSMap version 35 2024-09-15T23:07:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2e049e0740 1 -- 172.21.15.111:0/3512595481 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c0f3b3f7c0 con 0x55c0f3b2b260 2024-09-15T23:07:55.426 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2de4c00640 1 -- 172.21.15.111:0/3512595481 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dec005520 con 0x55c0f3b2b260 2024-09-15T23:07:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2de4c00640 1 -- 172.21.15.111:0/3512595481 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f2dec0035f0 con 0x55c0f3b2b260 2024-09-15T23:07:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2e049e0740 10 client.16719 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:07:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2e049e0740 10 client.16719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:07:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2e049e0740 20 client.16719 populate_metadata read hostname 'smithi111' 2024-09-15T23:07:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2e049e0740 1 --2- 172.21.15.111:0/3512595481 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c0f3b40b10 0x55c0f3b42f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2e049e0740 4 client.16719 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:07:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2e049e0740 1 -- 172.21.15.111:0/3512595481 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0f3a9d830 con 0x55c0f3b40b10 2024-09-15T23:07:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2dfa000640 1 --2- 172.21.15.111:0/3512595481 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c0f3b40b10 0x55c0f3b42f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2dfa000640 1 --2- 172.21.15.111:0/3512595481 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c0f3b40b10 0x55c0f3b42f00 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=324429d1e91b4390 server_cookie=902e30975039f1bb in_seq=0 out_seq=0 2024-09-15T23:07:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2de4c00640 10 client.16719 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:07:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2de4c00640 1 -- 172.21.15.111:0/3512595481 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c0f3a9d830 con 0x55c0f3b40b10 2024-09-15T23:07:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2de4c00640 10 client.16719 handle_command_reply: tid=0 2024-09-15T23:07:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2e049e0740 1 client.16719 shutdown 2024-09-15T23:07:55.427 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2e049e0740 1 -- 172.21.15.111:0/3512595481 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2b260 msgr2=0x55c0f3b35450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2e049e0740 1 --2- 172.21.15.111:0/3512595481 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2b260 0x55c0f3b35450 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2dec02a9e0 tx=0x7f2dec004f20 comp rx=0 tx=0).stop 2024-09-15T23:07:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2e049e0740 1 -- 172.21.15.111:0/3512595481 shutdown_connections 2024-09-15T23:07:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2e049e0740 1 --2- 172.21.15.111:0/3512595481 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c0f3b40b10 0x55c0f3b42f00 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2e049e0740 1 --2- 172.21.15.111:0/3512595481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f3b30c30 0x55c0f3b3c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2e049e0740 1 --2- 172.21.15.111:0/3512595481 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0f3b2bba0 0x55c0f3b35990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2e049e0740 1 --2- 172.21.15.111:0/3512595481 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2b260 0x55c0f3b35450 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2e049e0740 1 -- 172.21.15.111:0/3512595481 >> 172.21.15.111:0/3512595481 conn(0x55c0f3b268d0 msgr2=0x55c0f3b280e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2de4c00640 0 client.16719 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:07:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2e049e0740 1 -- 172.21.15.111:0/3512595481 shutdown_connections 2024-09-15T23:07:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2e049e0740 1 -- 172.21.15.111:0/3512595481 wait complete. 2024-09-15T23:07:55.428 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.419+0000 7f2e049e0740 20 client.16719 trim_cache size 0 max 16384 2024-09-15T23:07:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.436+0000 7f2e049e0740 1 Processor -- start 2024-09-15T23:07:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.436+0000 7f2e049e0740 1 -- start start 2024-09-15T23:07:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.436+0000 7f2e049e0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f3b2b260 0x55c0f3b218d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.436+0000 7f2e049e0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2bba0 0x55c0f3b3c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.436+0000 7f2e049e0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0f3b30c30 0x55c0f3b27f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.436+0000 7f2e049e0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc1ad7a0 con 0x55c0f3b2bba0 2024-09-15T23:07:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.436+0000 7f2e049e0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc1ab070 con 0x55c0f3b30c30 2024-09-15T23:07:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.436+0000 7f2e049e0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc1a8ec0 con 0x55c0f3b2b260 2024-09-15T23:07:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.436+0000 7f2dfa000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2bba0 0x55c0f3b3c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.436+0000 7f2dfa000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2bba0 0x55c0f3b3c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59548/0 (socket says 172.21.15.111:59548) 2024-09-15T23:07:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.436+0000 7f2dfa000640 1 -- 172.21.15.111:0/299362284 learned_addr learned my addr 172.21.15.111:0/299362284 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:07:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.436+0000 7f2dfaa00640 1 --2- 172.21.15.111:0/299362284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f3b2b260 0x55c0f3b218d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.436+0000 7f2dfb400640 1 --2- 172.21.15.111:0/299362284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0f3b30c30 0x55c0f3b27f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.436+0000 7f2dfa000640 1 -- 172.21.15.111:0/299362284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f3b2b260 msgr2=0x55c0f3b218d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.436+0000 7f2dfa000640 1 --2- 172.21.15.111:0/299362284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f3b2b260 0x55c0f3b218d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.436+0000 7f2dfa000640 1 -- 172.21.15.111:0/299362284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0f3b30c30 msgr2=0x55c0f3b27f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.436+0000 7f2dfa000640 1 --2- 172.21.15.111:0/299362284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0f3b30c30 0x55c0f3b27f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.436+0000 7f2dfa000640 1 -- 172.21.15.111:0/299362284 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0f3b266f0 con 0x55c0f3b2bba0 2024-09-15T23:07:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2dfaa00640 1 --2- 172.21.15.111:0/299362284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f3b2b260 0x55c0f3b218d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:07:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2dfa000640 1 --2- 172.21.15.111:0/299362284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2bba0 0x55c0f3b3c990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2de809bf00 tx=0x7f2de80cd5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead1f513d11d9523 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2de7400640 1 -- 172.21.15.111:0/299362284 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de80b9020 con 0x55c0f3b2bba0 2024-09-15T23:07:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2de7400640 1 -- 172.21.15.111:0/299362284 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de809c200 con 0x55c0f3b2bba0 2024-09-15T23:07:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2de7400640 1 -- 172.21.15.111:0/299362284 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8003b30 con 0x55c0f3b2bba0 2024-09-15T23:07:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2e049e0740 1 -- 172.21.15.111:0/299362284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2bba0 msgr2=0x55c0f3b3c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2e049e0740 1 --2- 172.21.15.111:0/299362284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2bba0 0x55c0f3b3c990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2de809bf00 tx=0x7f2de80cd5b0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2e049e0740 1 -- 172.21.15.111:0/299362284 shutdown_connections 2024-09-15T23:07:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2e049e0740 1 --2- 172.21.15.111:0/299362284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0f3b30c30 0x55c0f3b27f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2e049e0740 1 --2- 172.21.15.111:0/299362284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2bba0 0x55c0f3b3c990 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2e049e0740 1 --2- 172.21.15.111:0/299362284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f3b2b260 0x55c0f3b218d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2e049e0740 1 -- 172.21.15.111:0/299362284 >> 172.21.15.111:0/299362284 conn(0x55c0f3b268d0 msgr2=0x55c0f3b27840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2e049e0740 1 -- 172.21.15.111:0/299362284 shutdown_connections 2024-09-15T23:07:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2e049e0740 1 -- 172.21.15.111:0/299362284 wait complete. 2024-09-15T23:07:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2e049e0740 1 -- start start 2024-09-15T23:07:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2e049e0740 1 Processor -- start 2024-09-15T23:07:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2e049e0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0f3b2b260 0x55c0f3b396d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2e049e0740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2bba0 0x55c0f3b39c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2e049e0740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f3b30c30 0x55c0f3ba96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2e049e0740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc1a9530 con 0x55c0f3b2bba0 2024-09-15T23:07:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2e049e0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc1aa9a0 con 0x55c0f3b2b260 2024-09-15T23:07:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2e049e0740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc1a9c00 con 0x55c0f3b30c30 2024-09-15T23:07:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2e049e0740 10 client.0 _ll_register_callbacks cb 0x7f2df4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:07:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2dfa000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2bba0 0x55c0f3b39c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2dfa000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2bba0 0x55c0f3b39c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59550/0 (socket says 172.21.15.111:59550) 2024-09-15T23:07:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2dfa000640 1 -- 172.21.15.111:0/3314963993 learned_addr learned my addr 172.21.15.111:0/3314963993 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:07:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2dfaa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0f3b2b260 0x55c0f3b396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2dfb400640 1 --2- 172.21.15.111:0/3314963993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f3b30c30 0x55c0f3ba96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2dfa000640 1 -- 172.21.15.111:0/3314963993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f3b30c30 msgr2=0x55c0f3ba96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2dfa000640 1 --2- 172.21.15.111:0/3314963993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f3b30c30 0x55c0f3ba96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2dfa000640 1 -- 172.21.15.111:0/3314963993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0f3b2b260 msgr2=0x55c0f3b396d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:07:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2dfa000640 1 --2- 172.21.15.111:0/3314963993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0f3b2b260 0x55c0f3b396d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2dfa000640 1 -- 172.21.15.111:0/3314963993 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0f3b3a220 con 0x55c0f3b2bba0 2024-09-15T23:07:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2dfa000640 1 --2- 172.21.15.111:0/3314963993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2bba0 0x55c0f3b39c10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2de8003ee0 tx=0x7f2de800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acbf4654504a5050 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:07:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2de4c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:07:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2de4c00640 1 -- 172.21.15.111:0/3314963993 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de80b9030 con 0x55c0f3b2bba0 2024-09-15T23:07:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2de4c00640 1 -- 172.21.15.111:0/3314963993 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f2de8003b30 con 0x55c0f3b2bba0 2024-09-15T23:07:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2e049e0740 1 -- 172.21.15.111:0/3314963993 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2de809c200 con 0x55c0f3b2bba0 2024-09-15T23:07:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2de4c00640 1 -- 172.21.15.111:0/3314963993 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2de8098870 con 0x55c0f3b2bba0 2024-09-15T23:07:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2e049e0740 10 client.16725 fetch_fsmap learned FSMap version 35 2024-09-15T23:07:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2e049e0740 1 -- 172.21.15.111:0/3314963993 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c0f3baac80 con 0x55c0f3b2bba0 2024-09-15T23:07:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2de4c00640 1 -- 172.21.15.111:0/3314963993 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de809c200 con 0x55c0f3b2bba0 2024-09-15T23:07:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2de4c00640 1 -- 172.21.15.111:0/3314963993 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f2de80a49e0 con 0x55c0f3b2bba0 2024-09-15T23:07:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2e049e0740 10 client.16725 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:07:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2e049e0740 10 client.16725 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:07:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2e049e0740 20 client.16725 populate_metadata read hostname 'smithi111' 2024-09-15T23:07:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2e049e0740 1 --2- 172.21.15.111:0/3314963993 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c0f3bab5e0 0x55c0f3bada00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:07:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2e049e0740 4 client.16725 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:07:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2e049e0740 1 -- 172.21.15.111:0/3314963993 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c0f3a97b80 con 0x55c0f3bab5e0 2024-09-15T23:07:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2dfaa00640 1 --2- 172.21.15.111:0/3314963993 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c0f3bab5e0 0x55c0f3bada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:07:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2de4c00640 10 client.16725 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:07:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.438+0000 7f2dfaa00640 1 --2- 172.21.15.111:0/3314963993 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c0f3bab5e0 0x55c0f3bada00 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1daa113799f90245 server_cookie=4b8de09170eeb2a8 in_seq=0 out_seq=0 2024-09-15T23:07:55.571 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:55 smithi111 ceph-mon[30629]: pgmap v1011: 97 pgs: 97 active+clean; 580 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 4.7 MiB/s wr, 509 op/s 2024-09-15T23:07:55.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:55 smithi121 ceph-mon[34815]: pgmap v1011: 97 pgs: 97 active+clean; 580 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 4.7 MiB/s wr, 509 op/s 2024-09-15T23:07:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.649+0000 7f2de4c00640 1 -- 172.21.15.111:0/3314963993 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c0f3a97b80 con 0x55c0f3bab5e0 2024-09-15T23:07:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.649+0000 7f2de4c00640 10 client.16725 handle_command_reply: tid=0 2024-09-15T23:07:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.649+0000 7f2e049e0740 1 client.16725 shutdown 2024-09-15T23:07:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.650+0000 7f2e049e0740 1 -- 172.21.15.111:0/3314963993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2bba0 msgr2=0x55c0f3b39c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.650+0000 7f2e049e0740 1 --2- 172.21.15.111:0/3314963993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2bba0 0x55c0f3b39c10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2de8003ee0 tx=0x7f2de800c890 comp rx=0 tx=0).stop 2024-09-15T23:07:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.650+0000 7f2e049e0740 1 -- 172.21.15.111:0/3314963993 shutdown_connections 2024-09-15T23:07:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.650+0000 7f2e049e0740 1 --2- 172.21.15.111:0/3314963993 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c0f3bab5e0 0x55c0f3bada00 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.650+0000 7f2e049e0740 1 --2- 172.21.15.111:0/3314963993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c0f3b30c30 0x55c0f3ba96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.650+0000 7f2e049e0740 1 --2- 172.21.15.111:0/3314963993 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c0f3b2bba0 0x55c0f3b39c10 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.650+0000 7f2e049e0740 1 --2- 172.21.15.111:0/3314963993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0f3b2b260 0x55c0f3b396d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.650+0000 7f2e049e0740 1 -- 172.21.15.111:0/3314963993 >> 172.21.15.111:0/3314963993 conn(0x55c0f3b268d0 msgr2=0x55c0f3b34a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.650+0000 7f2de4c00640 0 client.16725 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:07:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.650+0000 7f2e049e0740 1 -- 172.21.15.111:0/3314963993 shutdown_connections 2024-09-15T23:07:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.650+0000 7f2e049e0740 1 -- 172.21.15.111:0/3314963993 wait complete. 2024-09-15T23:07:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.650+0000 7f2e049e0740 20 client.16725 trim_cache size 0 max 16384 2024-09-15T23:07:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:07:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:07:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:07:55.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:07:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.652+0000 7f2dfbe00640 1 -- 172.21.15.111:0/32173288 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2de00a3240 msgr2=0x7f2de00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.652+0000 7f2dfbe00640 1 --2- 172.21.15.111:0/32173288 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2de00a3240 0x7f2de00a5700 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2df4005fd0 tx=0x7f2df4009500 comp rx=0 tx=0).stop 2024-09-15T23:07:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.652+0000 7f2dfbe00640 1 -- 172.21.15.111:0/32173288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2dfc101f40 msgr2=0x7f2dfc19a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:07:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.652+0000 7f2dfbe00640 1 --2- 172.21.15.111:0/32173288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2dfc101f40 0x7f2dfc19a550 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2de800ea60 tx=0x7f2de800ef20 comp rx=0 tx=0).stop 2024-09-15T23:07:55.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.652+0000 7f2dfbe00640 1 -- 172.21.15.111:0/32173288 shutdown_connections 2024-09-15T23:07:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.652+0000 7f2dfbe00640 1 --2- 172.21.15.111:0/32173288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2dfc103050 0x7f2dfc1a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.652+0000 7f2dfbe00640 1 --2- 172.21.15.111:0/32173288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2dfc101f40 0x7f2dfc19a550 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.652+0000 7f2dfbe00640 1 --2- 172.21.15.111:0/32173288 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2de00a3240 0x7f2de00a5700 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.652+0000 7f2dfbe00640 1 --2- 172.21.15.111:0/32173288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2dfc101580 0x7f2dfc19a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:07:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.652+0000 7f2dfbe00640 1 -- 172.21.15.111:0/32173288 >> 172.21.15.111:0/32173288 conn(0x7f2dfc0fd0d0 msgr2=0x7f2dfc108300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:07:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.652+0000 7f2dfbe00640 1 -- 172.21.15.111:0/32173288 shutdown_connections 2024-09-15T23:07:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:07:55.652+0000 7f2dfbe00640 1 -- 172.21.15.111:0/32173288 wait complete. 2024-09-15T23:07:55.713 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174256 17.70 MB/sec execute 383 sec latency 2744.870 ms 2024-09-15T23:07:55.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:55 smithi192 ceph-mon[35427]: pgmap v1011: 97 pgs: 97 active+clean; 580 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 4.7 MiB/s wr, 509 op/s 2024-09-15T23:07:56.682 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174258 17.65 MB/sec execute 384 sec latency 3649.143 ms 2024-09-15T23:07:57.682 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174259 17.61 MB/sec execute 385 sec latency 4649.212 ms 2024-09-15T23:07:58.162 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:57 smithi111 ceph-mon[30629]: pgmap v1012: 97 pgs: 97 active+clean; 575 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.6 MiB/s wr, 494 op/s 2024-09-15T23:07:58.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:57 smithi192 ceph-mon[35427]: pgmap v1012: 97 pgs: 97 active+clean; 575 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.6 MiB/s wr, 494 op/s 2024-09-15T23:07:58.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:57 smithi121 ceph-mon[34815]: pgmap v1012: 97 pgs: 97 active+clean; 575 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.6 MiB/s wr, 494 op/s 2024-09-15T23:07:58.682 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174260 17.56 MB/sec execute 386 sec latency 5649.254 ms 2024-09-15T23:07:59.682 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174260 17.52 MB/sec execute 387 sec latency 6649.298 ms 2024-09-15T23:08:00.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:07:59 smithi192 ceph-mon[35427]: pgmap v1013: 97 pgs: 97 active+clean; 574 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.1 MiB/s wr, 524 op/s 2024-09-15T23:08:00.379 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:07:59 smithi111 ceph-mon[30629]: pgmap v1013: 97 pgs: 97 active+clean; 574 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.1 MiB/s wr, 524 op/s 2024-09-15T23:08:00.389 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:07:59 smithi121 ceph-mon[34815]: pgmap v1013: 97 pgs: 97 active+clean; 574 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 7.1 MiB/s wr, 524 op/s 2024-09-15T23:08:00.682 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174260 17.47 MB/sec execute 388 sec latency 7649.339 ms 2024-09-15T23:08:01.682 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174268 17.43 MB/sec execute 389 sec latency 8649.408 ms 2024-09-15T23:08:02.073 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:02 smithi111 ceph-mon[30629]: pgmap v1014: 97 pgs: 97 active+clean; 580 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.3 MiB/s wr, 462 op/s 2024-09-15T23:08:02.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:02 smithi192 ceph-mon[35427]: pgmap v1014: 97 pgs: 97 active+clean; 580 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.3 MiB/s wr, 462 op/s 2024-09-15T23:08:02.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:02 smithi121 ceph-mon[34815]: pgmap v1014: 97 pgs: 97 active+clean; 580 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.3 MiB/s wr, 462 op/s 2024-09-15T23:08:02.682 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174274 17.38 MB/sec execute 390 sec latency 8660.609 ms 2024-09-15T23:08:03.682 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174276 17.34 MB/sec execute 391 sec latency 4871.258 ms 2024-09-15T23:08:04.046 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:04 smithi111 ceph-mon[30629]: pgmap v1015: 97 pgs: 97 active+clean; 586 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.1 MiB/s wr, 369 op/s 2024-09-15T23:08:04.046 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:04 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:08:04.046 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:04 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:04.047 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:04 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:04.047 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:04 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:04.047 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:04 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:04.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:04 smithi192 ceph-mon[35427]: pgmap v1015: 97 pgs: 97 active+clean; 586 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.1 MiB/s wr, 369 op/s 2024-09-15T23:08:04.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:04 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:08:04.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:04 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:04.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:04 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:04.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:04 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:04.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:04 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:04.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:04 smithi121 ceph-mon[34815]: pgmap v1015: 97 pgs: 97 active+clean; 586 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 7.1 MiB/s wr, 369 op/s 2024-09-15T23:08:04.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:04 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:08:04.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:04 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:04.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:04 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:04.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:04 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:04.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:04 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:04.682 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174278 17.29 MB/sec execute 392 sec latency 4916.206 ms 2024-09-15T23:08:05.200 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:05 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:05.200 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:05 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:05.200 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:05 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:05.200 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:05 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:05.200 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:05 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:05.201 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:05 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:05.201 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:05 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:08:05.201 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:05 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:08:05.201 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:05 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:05.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:05 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:05.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:05 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:05.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:05 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:05.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:05 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:05.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:05 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:05.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:05 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:05.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:05 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:08:05.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:05 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:08:05.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:05 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:05.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:05 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:05.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:05 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:05.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:05 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:05.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:05 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:05.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:05 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:05.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:05 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:05.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:05 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:08:05.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:05 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:08:05.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:05 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:08:05.683 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174278 17.25 MB/sec execute 393 sec latency 4724.314 ms 2024-09-15T23:08:06.296 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:06 smithi111 ceph-mon[30629]: pgmap v1016: 97 pgs: 97 active+clean; 580 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.7 MiB/s wr, 391 op/s 2024-09-15T23:08:06.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:06 smithi192 ceph-mon[35427]: pgmap v1016: 97 pgs: 97 active+clean; 580 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.7 MiB/s wr, 391 op/s 2024-09-15T23:08:06.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:06 smithi121 ceph-mon[34815]: pgmap v1016: 97 pgs: 97 active+clean; 580 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.7 MiB/s wr, 391 op/s 2024-09-15T23:08:06.683 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174278 17.21 MB/sec execute 394 sec latency 5724.399 ms 2024-09-15T23:08:07.683 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174343 17.17 MB/sec execute 395 sec latency 6724.441 ms 2024-09-15T23:08:08.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:08 smithi192 ceph-mon[35427]: pgmap v1017: 97 pgs: 97 active+clean; 573 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.2 MiB/s wr, 373 op/s 2024-09-15T23:08:08.335 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:08 smithi111 ceph-mon[30629]: pgmap v1017: 97 pgs: 97 active+clean; 573 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.2 MiB/s wr, 373 op/s 2024-09-15T23:08:08.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:08 smithi121 ceph-mon[34815]: pgmap v1017: 97 pgs: 97 active+clean; 573 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.2 MiB/s wr, 373 op/s 2024-09-15T23:08:08.683 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174497 17.16 MB/sec execute 396 sec latency 7724.502 ms 2024-09-15T23:08:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.778+0000 7f4db7e00640 1 -- 172.21.15.111:0/2200901934 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4db8074680 msgr2=0x7f4db8074a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.778+0000 7f4db7e00640 1 --2- 172.21.15.111:0/2200901934 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4db8074680 0x7f4db8074a80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4da8009900 tx=0x7f4da802a3e0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.778+0000 7f4db7e00640 1 -- 172.21.15.111:0/2200901934 shutdown_connections 2024-09-15T23:08:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.778+0000 7f4db7e00640 1 --2- 172.21.15.111:0/2200901934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4db8109c90 0x7f4db810c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.778+0000 7f4db7e00640 1 --2- 172.21.15.111:0/2200901934 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4db8074fc0 0x7f4db8109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.778+0000 7f4db7e00640 1 --2- 172.21.15.111:0/2200901934 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4db8074680 0x7f4db8074a80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.778+0000 7f4db7e00640 1 -- 172.21.15.111:0/2200901934 >> 172.21.15.111:0/2200901934 conn(0x7f4db80fd0d0 msgr2=0x7f4db80ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:08:08.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.778+0000 7f4db7e00640 1 -- 172.21.15.111:0/2200901934 shutdown_connections 2024-09-15T23:08:08.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.778+0000 7f4db7e00640 1 -- 172.21.15.111:0/2200901934 wait complete. 2024-09-15T23:08:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.779+0000 7f4db7e00640 1 Processor -- start 2024-09-15T23:08:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.779+0000 7f4db7e00640 1 -- start start 2024-09-15T23:08:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.779+0000 7f4db7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4db8074680 0x7f4db8199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.780+0000 7f4db7e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4db8074fc0 0x7f4db819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.780+0000 7f4db7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4db8109c90 0x7f4db81a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.780+0000 7f4db7e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4db810f8f0 con 0x7f4db8109c90 2024-09-15T23:08:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.780+0000 7f4db7e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4db810f770 con 0x7f4db8074680 2024-09-15T23:08:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.780+0000 7f4db7e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4db810fa70 con 0x7f4db8074fc0 2024-09-15T23:08:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.780+0000 7f4db6a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4db8074680 0x7f4db8199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.780+0000 7f4db6a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4db8074680 0x7f4db8199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:47826/0 (socket says 172.21.15.111:47826) 2024-09-15T23:08:08.782 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.780+0000 7f4db6a00640 1 -- 172.21.15.111:0/3014398177 learned_addr learned my addr 172.21.15.111:0/3014398177 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:08:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.780+0000 7f4db6000640 1 --2- 172.21.15.111:0/3014398177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4db8074fc0 0x7f4db819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.780+0000 7f4db7400640 1 --2- 172.21.15.111:0/3014398177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4db8109c90 0x7f4db81a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.780+0000 7f4db6a00640 1 -- 172.21.15.111:0/3014398177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4db8074fc0 msgr2=0x7f4db819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.780+0000 7f4db6a00640 1 --2- 172.21.15.111:0/3014398177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4db8074fc0 0x7f4db819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.780+0000 7f4db6a00640 1 -- 172.21.15.111:0/3014398177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4db8109c90 msgr2=0x7f4db81a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.780+0000 7f4db6a00640 1 --2- 172.21.15.111:0/3014398177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4db8109c90 0x7f4db81a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.780+0000 7f4db6a00640 1 -- 172.21.15.111:0/3014398177 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db81a1bb0 con 0x7f4db8074680 2024-09-15T23:08:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.781+0000 7f4db6a00640 1 --2- 172.21.15.111:0/3014398177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4db8074680 0x7f4db8199f90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4da8009230 tx=0x7f4da80042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:08:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.781+0000 7f4daf400640 1 -- 172.21.15.111:0/3014398177 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da803f030 con 0x7f4db8074680 2024-09-15T23:08:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.781+0000 7f4daf400640 1 -- 172.21.15.111:0/3014398177 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4da8003ce0 con 0x7f4db8074680 2024-09-15T23:08:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.781+0000 7f4daf400640 1 -- 172.21.15.111:0/3014398177 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8035070 con 0x7f4db8074680 2024-09-15T23:08:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.781+0000 7f4db7e00640 1 -- 172.21.15.111:0/3014398177 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db81a1e40 con 0x7f4db8074680 2024-09-15T23:08:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.781+0000 7f4db7e00640 1 -- 172.21.15.111:0/3014398177 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db81a22a0 con 0x7f4db8074680 2024-09-15T23:08:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.785+0000 7f4dc0a57740 1 Processor -- start 2024-09-15T23:08:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.785+0000 7f4daf400640 1 -- 172.21.15.111:0/3014398177 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f4da8042030 con 0x7f4db8074680 2024-09-15T23:08:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.785+0000 7f4dc0a57740 1 -- start start 2024-09-15T23:08:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.786+0000 7f4dc0a57740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b1620ca70 0x557b1620ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.786+0000 7f4dc0a57740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b1620d3b0 0x557b16211e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.786+0000 7f4dc0a57740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b16212440 0x557b16214900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.786+0000 7f4dc0a57740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557b16169100 con 0x557b1620ca70 2024-09-15T23:08:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.786+0000 7f4dc0a57740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x557b161111b0 con 0x557b16212440 2024-09-15T23:08:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.786+0000 7f4dc0a57740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557b161650c0 con 0x557b1620d3b0 2024-09-15T23:08:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.786+0000 7f4db7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b1620ca70 0x557b1620ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.786+0000 7f4db6a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b1620d3b0 0x557b16211e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.786+0000 7f4db6a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b1620d3b0 0x557b16211e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:47412/0 (socket says 172.21.15.111:47412) 2024-09-15T23:08:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.786+0000 7f4db6a00640 1 -- 172.21.15.111:0/2334507427 learned_addr learned my addr 172.21.15.111:0/2334507427 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:08:08.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.786+0000 7f4db6000640 1 --2- 172.21.15.111:0/2334507427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b16212440 0x557b16214900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:08.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.786+0000 7f4db6a00640 1 -- 172.21.15.111:0/2334507427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b16212440 msgr2=0x557b16214900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:08.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.786+0000 7f4db6a00640 1 --2- 172.21.15.111:0/2334507427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b16212440 0x557b16214900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.786+0000 7f4db6a00640 1 -- 172.21.15.111:0/2334507427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b1620ca70 msgr2=0x557b1620ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:08.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.786+0000 7f4db6a00640 1 --2- 172.21.15.111:0/2334507427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b1620ca70 0x557b1620ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.786+0000 7f4db6a00640 1 -- 172.21.15.111:0/2334507427 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b15e7c210 con 0x557b1620d3b0 2024-09-15T23:08:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.787+0000 7f4db6000640 1 --2- 172.21.15.111:0/2334507427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b16212440 0x557b16214900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:08:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.787+0000 7f4db6a00640 1 --2- 172.21.15.111:0/2334507427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b1620d3b0 0x557b16211e70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4da80f3e00 tx=0x7f4da80f3ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b94a71b1a7b5beb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:08:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.787+0000 7f4da3e00640 1 -- 172.21.15.111:0/2334507427 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8102470 con 0x557b1620d3b0 2024-09-15T23:08:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.787+0000 7f4daf400640 1 --2- 172.21.15.111:0/3014398177 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4d9c0a32c0 0x7f4d9c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.787+0000 7f4da3e00640 1 -- 172.21.15.111:0/2334507427 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4da80fd070 con 0x557b1620d3b0 2024-09-15T23:08:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.787+0000 7f4da3e00640 1 -- 172.21.15.111:0/2334507427 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da80b8640 con 0x557b1620d3b0 2024-09-15T23:08:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.788+0000 7f4db7400640 1 --2- 172.21.15.111:0/3014398177 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4d9c0a32c0 0x7f4d9c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.788+0000 7f4daf400640 1 -- 172.21.15.111:0/3014398177 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f4da80efc50 con 0x7f4db8074680 2024-09-15T23:08:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.788+0000 7f4dc0a57740 1 -- 172.21.15.111:0/2334507427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b1620d3b0 msgr2=0x557b16211e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.788+0000 7f4dc0a57740 1 --2- 172.21.15.111:0/2334507427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b1620d3b0 0x557b16211e70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4da80f3e00 tx=0x7f4da80f3ef0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.788+0000 7f4db7400640 1 --2- 172.21.15.111:0/3014398177 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4d9c0a32c0 0x7f4d9c0a5780 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4db0005fd0 tx=0x7f4db0009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:08:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.788+0000 7f4dc0a57740 1 -- 172.21.15.111:0/2334507427 shutdown_connections 2024-09-15T23:08:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.788+0000 7f4dc0a57740 1 --2- 172.21.15.111:0/2334507427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b16212440 0x557b16214900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.788+0000 7f4dc0a57740 1 --2- 172.21.15.111:0/2334507427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b1620d3b0 0x557b16211e70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.788+0000 7f4dc0a57740 1 --2- 172.21.15.111:0/2334507427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b1620ca70 0x557b1620ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.788+0000 7f4dc0a57740 1 -- 172.21.15.111:0/2334507427 >> 172.21.15.111:0/2334507427 conn(0x557b161ff8d0 msgr2=0x557b161ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:08:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.789+0000 7f4dc0a57740 1 -- 172.21.15.111:0/2334507427 shutdown_connections 2024-09-15T23:08:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.789+0000 7f4dc0a57740 1 -- 172.21.15.111:0/2334507427 wait complete. 2024-09-15T23:08:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.789+0000 7f4dc0a57740 1 -- start start 2024-09-15T23:08:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.789+0000 7f4dc0a57740 1 Processor -- start 2024-09-15T23:08:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.790+0000 7f4dc0a57740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b1620ca70 0x557b162058d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.790+0000 7f4dc0a57740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b1620d3b0 0x557b16205e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.790+0000 7f4dc0a57740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b16212440 0x557b1620ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.790+0000 7f4dc0a57740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557b162040c0 con 0x557b1620ca70 2024-09-15T23:08:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.790+0000 7f4dc0a57740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x557b160f2090 con 0x557b1620d3b0 2024-09-15T23:08:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.790+0000 7f4dc0a57740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557b16204240 con 0x557b16212440 2024-09-15T23:08:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.790+0000 7f4dc0a57740 10 client.0 _ll_register_callbacks cb 0x557b16146f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:08:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.790+0000 7f4db6000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b1620d3b0 0x557b16205e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.790+0000 7f4db6000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b1620d3b0 0x557b16205e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:47836/0 (socket says 172.21.15.111:47836) 2024-09-15T23:08:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.790+0000 7f4db6000640 1 -- 172.21.15.111:0/549317205 learned_addr learned my addr 172.21.15.111:0/549317205 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:08:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.790+0000 7f4db6a00640 1 --2- 172.21.15.111:0/549317205 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b1620ca70 0x557b162058d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.790+0000 7f4db7400640 1 --2- 172.21.15.111:0/549317205 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b16212440 0x557b1620ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.791+0000 7f4db6000640 1 -- 172.21.15.111:0/549317205 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b16212440 msgr2=0x557b1620ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.791+0000 7f4db6000640 1 --2- 172.21.15.111:0/549317205 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b16212440 0x557b1620ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.791+0000 7f4db6000640 1 -- 172.21.15.111:0/549317205 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b1620ca70 msgr2=0x557b162058d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.791+0000 7f4db6000640 1 --2- 172.21.15.111:0/549317205 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b1620ca70 0x557b162058d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.791+0000 7f4db6000640 1 -- 172.21.15.111:0/549317205 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b15f48620 con 0x557b1620d3b0 2024-09-15T23:08:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.791+0000 7f4db6000640 1 --2- 172.21.15.111:0/549317205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b1620d3b0 0x557b16205e10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4da40136e0 tx=0x7f4da4013ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=906d3fb8a7323485 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:08:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.791+0000 7f4da0c00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:08:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.791+0000 7f4da0c00640 1 -- 172.21.15.111:0/549317205 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4004440 con 0x557b1620d3b0 2024-09-15T23:08:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.791+0000 7f4dc0a57740 1 -- 172.21.15.111:0/549317205 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4da80fd070 con 0x557b1620d3b0 2024-09-15T23:08:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.791+0000 7f4da0c00640 1 -- 172.21.15.111:0/549317205 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f4da40119c0 con 0x557b1620d3b0 2024-09-15T23:08:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.791+0000 7f4da0c00640 1 -- 172.21.15.111:0/549317205 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4da4011bc0 con 0x557b1620d3b0 2024-09-15T23:08:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.791+0000 7f4dc0a57740 10 client.26675 fetch_fsmap learned FSMap version 35 2024-09-15T23:08:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.791+0000 7f4dc0a57740 1 -- 172.21.15.111:0/549317205 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557b162296e0 con 0x557b1620d3b0 2024-09-15T23:08:08.796 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.792+0000 7f4da0c00640 1 -- 172.21.15.111:0/549317205 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4da80fd070 con 0x557b1620d3b0 2024-09-15T23:08:08.796 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.792+0000 7f4da0c00640 1 -- 172.21.15.111:0/549317205 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f4da401a070 con 0x557b1620d3b0 2024-09-15T23:08:08.796 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.792+0000 7f4dc0a57740 10 client.26675 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:08:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.792+0000 7f4dc0a57740 10 client.26675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:08:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.792+0000 7f4dc0a57740 20 client.26675 populate_metadata read hostname 'smithi111' 2024-09-15T23:08:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.792+0000 7f4dc0a57740 1 --2- 172.21.15.111:0/549317205 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557b1622ac20 0x557b1622d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.792+0000 7f4dc0a57740 4 client.26675 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:08:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.792+0000 7f4dc0a57740 1 -- 172.21.15.111:0/549317205 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b16176830 con 0x557b1622ac20 2024-09-15T23:08:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.792+0000 7f4db6a00640 1 --2- 172.21.15.111:0/549317205 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557b1622ac20 0x557b1622d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.793+0000 7f4db6a00640 1 --2- 172.21.15.111:0/549317205 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557b1622ac20 0x557b1622d090 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=699c63310df873bd server_cookie=30adca5300c5aa39 in_seq=0 out_seq=0 2024-09-15T23:08:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.793+0000 7f4da0c00640 10 client.26675 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:08:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.794+0000 7f4da0c00640 1 -- 172.21.15.111:0/549317205 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557b16176830 con 0x557b1622ac20 2024-09-15T23:08:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.794+0000 7f4da0c00640 10 client.26675 handle_command_reply: tid=0 2024-09-15T23:08:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.794+0000 7f4dc0a57740 1 client.26675 shutdown 2024-09-15T23:08:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.794+0000 7f4dc0a57740 1 -- 172.21.15.111:0/549317205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b1620d3b0 msgr2=0x557b16205e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:08.797 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.794+0000 7f4dc0a57740 1 --2- 172.21.15.111:0/549317205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b1620d3b0 0x557b16205e10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4da40136e0 tx=0x7f4da4013ba0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.794+0000 7f4dc0a57740 1 -- 172.21.15.111:0/549317205 shutdown_connections 2024-09-15T23:08:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.794+0000 7f4dc0a57740 1 --2- 172.21.15.111:0/549317205 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557b1622ac20 0x557b1622d090 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.794+0000 7f4dc0a57740 1 --2- 172.21.15.111:0/549317205 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b16212440 0x557b1620ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.794+0000 7f4dc0a57740 1 --2- 172.21.15.111:0/549317205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b1620d3b0 0x557b16205e10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.794+0000 7f4dc0a57740 1 --2- 172.21.15.111:0/549317205 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b1620ca70 0x557b162058d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.794+0000 7f4dc0a57740 1 -- 172.21.15.111:0/549317205 >> 172.21.15.111:0/549317205 conn(0x557b161ff8d0 msgr2=0x557b162010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:08:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.794+0000 7f4da0c00640 0 client.26675 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:08:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.794+0000 7f4dc0a57740 1 -- 172.21.15.111:0/549317205 shutdown_connections 2024-09-15T23:08:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.794+0000 7f4dc0a57740 1 -- 172.21.15.111:0/549317205 wait complete. 2024-09-15T23:08:08.798 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.795+0000 7f4dc0a57740 20 client.26675 trim_cache size 0 max 16384 2024-09-15T23:08:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.809+0000 7f4dc0a57740 1 Processor -- start 2024-09-15T23:08:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.809+0000 7f4dc0a57740 1 -- start start 2024-09-15T23:08:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.809+0000 7f4dc0a57740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b1620ca70 0x557b1622cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.810+0000 7f4dc0a57740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b1620d3b0 0x557b161fe1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.810+0000 7f4dc0a57740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b16212440 0x557b161fe730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.810+0000 7f4dc0a57740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4db81a9330 con 0x557b1620ca70 2024-09-15T23:08:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.810+0000 7f4dc0a57740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4db81a6c00 con 0x557b16212440 2024-09-15T23:08:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.810+0000 7f4dc0a57740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4db81a4a50 con 0x557b1620d3b0 2024-09-15T23:08:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.810+0000 7f4db7400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b16212440 0x557b161fe730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.810+0000 7f4db6000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b1620d3b0 0x557b161fe1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.810+0000 7f4db7400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b16212440 0x557b161fe730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:47852/0 (socket says 172.21.15.111:47852) 2024-09-15T23:08:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.810+0000 7f4db6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b1620ca70 0x557b1622cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.810+0000 7f4db7400640 1 -- 172.21.15.111:0/3662081873 learned_addr learned my addr 172.21.15.111:0/3662081873 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:08:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.810+0000 7f4db7400640 1 -- 172.21.15.111:0/3662081873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b1620d3b0 msgr2=0x557b161fe1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.810+0000 7f4db7400640 1 --2- 172.21.15.111:0/3662081873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b1620d3b0 0x557b161fe1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.810+0000 7f4db7400640 1 -- 172.21.15.111:0/3662081873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b1620ca70 msgr2=0x557b1622cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.810+0000 7f4db7400640 1 --2- 172.21.15.111:0/3662081873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b1620ca70 0x557b1622cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.810+0000 7f4db7400640 1 -- 172.21.15.111:0/3662081873 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b1622d400 con 0x557b16212440 2024-09-15T23:08:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.811+0000 7f4db7400640 1 --2- 172.21.15.111:0/3662081873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b16212440 0x557b161fe730 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4db0003f10 tx=0x7f4db0016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=481b4b994f9475c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:08:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.811+0000 7f4da3400640 1 -- 172.21.15.111:0/3662081873 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db000f4e0 con 0x557b16212440 2024-09-15T23:08:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.811+0000 7f4da3400640 1 -- 172.21.15.111:0/3662081873 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4db000fb60 con 0x557b16212440 2024-09-15T23:08:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.811+0000 7f4da3400640 1 -- 172.21.15.111:0/3662081873 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db001dbc0 con 0x557b16212440 2024-09-15T23:08:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.812+0000 7f4dc0a57740 1 -- 172.21.15.111:0/3662081873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b16212440 msgr2=0x557b161fe730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.812+0000 7f4dc0a57740 1 --2- 172.21.15.111:0/3662081873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b16212440 0x557b161fe730 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4db0003f10 tx=0x7f4db0016000 comp rx=0 tx=0).stop 2024-09-15T23:08:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.812+0000 7f4dc0a57740 1 -- 172.21.15.111:0/3662081873 shutdown_connections 2024-09-15T23:08:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.812+0000 7f4dc0a57740 1 --2- 172.21.15.111:0/3662081873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b16212440 0x557b161fe730 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.812+0000 7f4dc0a57740 1 --2- 172.21.15.111:0/3662081873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b1620d3b0 0x557b161fe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.812+0000 7f4dc0a57740 1 --2- 172.21.15.111:0/3662081873 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b1620ca70 0x557b1622cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.812+0000 7f4dc0a57740 1 -- 172.21.15.111:0/3662081873 >> 172.21.15.111:0/3662081873 conn(0x557b161ff8d0 msgr2=0x557b162058d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:08:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.812+0000 7f4dc0a57740 1 -- 172.21.15.111:0/3662081873 shutdown_connections 2024-09-15T23:08:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.812+0000 7f4dc0a57740 1 -- 172.21.15.111:0/3662081873 wait complete. 2024-09-15T23:08:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.812+0000 7f4dc0a57740 1 -- start start 2024-09-15T23:08:08.814 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.813+0000 7f4dc0a57740 1 Processor -- start 2024-09-15T23:08:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.813+0000 7f4dc0a57740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b1620ca70 0x557b1623c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:08.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.813+0000 7f4dc0a57740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b1620d3b0 0x557b1623cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.813+0000 7f4dc0a57740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b16212440 0x557b1623db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.813+0000 7f4dc0a57740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4db81a50c0 con 0x557b16212440 2024-09-15T23:08:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.813+0000 7f4dc0a57740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4db81a6530 con 0x557b1620d3b0 2024-09-15T23:08:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.813+0000 7f4dc0a57740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4db81a5790 con 0x557b1620ca70 2024-09-15T23:08:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.813+0000 7f4dc0a57740 10 client.0 _ll_register_callbacks cb 0x7f4da40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:08:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.814+0000 7f4db6000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b1620d3b0 0x557b1623cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.814+0000 7f4db6a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b1620ca70 0x557b1623c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.814+0000 7f4db6000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b1620d3b0 0x557b1623cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:47860/0 (socket says 172.21.15.111:47860) 2024-09-15T23:08:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.814+0000 7f4db6000640 1 -- 172.21.15.111:0/1042585886 learned_addr learned my addr 172.21.15.111:0/1042585886 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:08:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.814+0000 7f4db7400640 1 --2- 172.21.15.111:0/1042585886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b16212440 0x557b1623db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.814+0000 7f4db6000640 1 -- 172.21.15.111:0/1042585886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b1620ca70 msgr2=0x557b1623c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.814+0000 7f4db6000640 1 --2- 172.21.15.111:0/1042585886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b1620ca70 0x557b1623c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.814+0000 7f4db6000640 1 -- 172.21.15.111:0/1042585886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b16212440 msgr2=0x557b1623db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.814+0000 7f4db6000640 1 --2- 172.21.15.111:0/1042585886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b16212440 0x557b1623db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.814+0000 7f4db6000640 1 -- 172.21.15.111:0/1042585886 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b1623e150 con 0x557b1620d3b0 2024-09-15T23:08:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.814+0000 7f4db6000640 1 --2- 172.21.15.111:0/1042585886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b1620d3b0 0x557b1623cb30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x557b16206db0 tx=0x7f4da4009d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4dafdbab727844b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:08:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.814+0000 7f4da0c00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:08:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.815+0000 7f4da0c00640 1 -- 172.21.15.111:0/1042585886 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da401c8c0 con 0x557b1620d3b0 2024-09-15T23:08:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.815+0000 7f4dc0a57740 1 -- 172.21.15.111:0/1042585886 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4db000fb60 con 0x557b1620d3b0 2024-09-15T23:08:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.815+0000 7f4da0c00640 1 -- 172.21.15.111:0/1042585886 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f4da401ca60 con 0x557b1620d3b0 2024-09-15T23:08:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.815+0000 7f4da0c00640 1 -- 172.21.15.111:0/1042585886 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4da401cca0 con 0x557b1620d3b0 2024-09-15T23:08:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.815+0000 7f4dc0a57740 10 client.26681 fetch_fsmap learned FSMap version 35 2024-09-15T23:08:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.815+0000 7f4dc0a57740 1 -- 172.21.15.111:0/1042585886 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557b162957f0 con 0x557b1620d3b0 2024-09-15T23:08:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.815+0000 7f4da0c00640 1 -- 172.21.15.111:0/1042585886 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4db000fb60 con 0x557b1620d3b0 2024-09-15T23:08:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.815+0000 7f4da0c00640 1 -- 172.21.15.111:0/1042585886 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f4da401a040 con 0x557b1620d3b0 2024-09-15T23:08:08.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.815+0000 7f4dc0a57740 10 client.26681 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:08:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.816+0000 7f4dc0a57740 10 client.26681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:08:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.816+0000 7f4dc0a57740 20 client.26681 populate_metadata read hostname 'smithi111' 2024-09-15T23:08:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.816+0000 7f4dc0a57740 1 --2- 172.21.15.111:0/1042585886 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557b16296100 0x557b16298520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.816+0000 7f4dc0a57740 4 client.26681 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:08:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.816+0000 7f4dc0a57740 1 -- 172.21.15.111:0/1042585886 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b16170b80 con 0x557b16296100 2024-09-15T23:08:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.816+0000 7f4db6a00640 1 --2- 172.21.15.111:0/1042585886 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557b16296100 0x557b16298520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.816+0000 7f4db6a00640 1 --2- 172.21.15.111:0/1042585886 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557b16296100 0x557b16298520 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98c4a37fc8842b3f server_cookie=8e152321e318eb1d in_seq=0 out_seq=0 2024-09-15T23:08:08.819 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.816+0000 7f4da0c00640 10 client.26681 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:08:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.910+0000 7f4da0c00640 1 -- 172.21.15.111:0/1042585886 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557b16170b80 con 0x557b16296100 2024-09-15T23:08:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.910+0000 7f4da0c00640 10 client.26681 handle_command_reply: tid=0 2024-09-15T23:08:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.911+0000 7f4dc0a57740 1 client.26681 shutdown 2024-09-15T23:08:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.911+0000 7f4dc0a57740 1 -- 172.21.15.111:0/1042585886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b1620d3b0 msgr2=0x557b1623cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.911+0000 7f4dc0a57740 1 --2- 172.21.15.111:0/1042585886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b1620d3b0 0x557b1623cb30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x557b16206db0 tx=0x7f4da4009d80 comp rx=0 tx=0).stop 2024-09-15T23:08:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.911+0000 7f4dc0a57740 1 -- 172.21.15.111:0/1042585886 shutdown_connections 2024-09-15T23:08:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.911+0000 7f4dc0a57740 1 --2- 172.21.15.111:0/1042585886 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557b16296100 0x557b16298520 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.911+0000 7f4dc0a57740 1 --2- 172.21.15.111:0/1042585886 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557b16212440 0x557b1623db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.911+0000 7f4dc0a57740 1 --2- 172.21.15.111:0/1042585886 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557b1620d3b0 0x557b1623cb30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.911+0000 7f4dc0a57740 1 --2- 172.21.15.111:0/1042585886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557b1620ca70 0x557b1623c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.913 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.911+0000 7f4da0c00640 0 client.26681 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:08:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.911+0000 7f4dc0a57740 1 -- 172.21.15.111:0/1042585886 >> 172.21.15.111:0/1042585886 conn(0x557b161ff8d0 msgr2=0x557b162007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:08:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.911+0000 7f4dc0a57740 1 -- 172.21.15.111:0/1042585886 shutdown_connections 2024-09-15T23:08:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.911+0000 7f4dc0a57740 1 -- 172.21.15.111:0/1042585886 wait complete. 2024-09-15T23:08:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.911+0000 7f4dc0a57740 20 client.26681 trim_cache size 0 max 16384 2024-09-15T23:08:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:08:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:08:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:08:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:08:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.912+0000 7f4db7e00640 1 -- 172.21.15.111:0/3014398177 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4d9c0a32c0 msgr2=0x7f4d9c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.912+0000 7f4db7e00640 1 --2- 172.21.15.111:0/3014398177 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4d9c0a32c0 0x7f4d9c0a5780 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4db0005fd0 tx=0x7f4db0009500 comp rx=0 tx=0).stop 2024-09-15T23:08:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.912+0000 7f4db7e00640 1 -- 172.21.15.111:0/3014398177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4db8074680 msgr2=0x7f4db8199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.912+0000 7f4db7e00640 1 --2- 172.21.15.111:0/3014398177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4db8074680 0x7f4db8199f90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4da8009230 tx=0x7f4da80042f0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.912+0000 7f4db7e00640 1 -- 172.21.15.111:0/3014398177 shutdown_connections 2024-09-15T23:08:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.912+0000 7f4db7e00640 1 --2- 172.21.15.111:0/3014398177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4db8109c90 0x7f4db81a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.912+0000 7f4db7e00640 1 --2- 172.21.15.111:0/3014398177 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4db8074fc0 0x7f4db819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.912+0000 7f4db7e00640 1 --2- 172.21.15.111:0/3014398177 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4d9c0a32c0 0x7f4d9c0a5780 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.912+0000 7f4db7e00640 1 --2- 172.21.15.111:0/3014398177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4db8074680 0x7f4db8199f90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.912+0000 7f4db7e00640 1 -- 172.21.15.111:0/3014398177 >> 172.21.15.111:0/3014398177 conn(0x7f4db80fd0d0 msgr2=0x7f4db8105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:08:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.912+0000 7f4db7e00640 1 -- 172.21.15.111:0/3014398177 shutdown_connections 2024-09-15T23:08:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:08.912+0000 7f4db7e00640 1 -- 172.21.15.111:0/3014398177 wait complete. 2024-09-15T23:08:09.683 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174498 17.12 MB/sec execute 397 sec latency 7826.182 ms 2024-09-15T23:08:10.070 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:10 smithi111 ceph-mon[30629]: pgmap v1018: 97 pgs: 97 active+clean; 562 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 6.8 MiB/s wr, 309 op/s 2024-09-15T23:08:10.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:10 smithi192 ceph-mon[35427]: pgmap v1018: 97 pgs: 97 active+clean; 562 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 6.8 MiB/s wr, 309 op/s 2024-09-15T23:08:10.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:10 smithi121 ceph-mon[34815]: pgmap v1018: 97 pgs: 97 active+clean; 562 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 6.8 MiB/s wr, 309 op/s 2024-09-15T23:08:10.683 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174498 17.08 MB/sec execute 398 sec latency 3090.847 ms 2024-09-15T23:08:11.683 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174498 17.03 MB/sec execute 399 sec latency 4090.897 ms 2024-09-15T23:08:12.073 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:12 smithi111 ceph-mon[30629]: pgmap v1019: 97 pgs: 97 active+clean; 556 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 5.9 MiB/s wr, 252 op/s 2024-09-15T23:08:12.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:12 smithi192 ceph-mon[35427]: pgmap v1019: 97 pgs: 97 active+clean; 556 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 5.9 MiB/s wr, 252 op/s 2024-09-15T23:08:12.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:12 smithi121 ceph-mon[34815]: pgmap v1019: 97 pgs: 97 active+clean; 556 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 5.9 MiB/s wr, 252 op/s 2024-09-15T23:08:12.683 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174499 16.99 MB/sec execute 400 sec latency 5090.949 ms 2024-09-15T23:08:13.683 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174572 16.97 MB/sec execute 401 sec latency 6091.010 ms 2024-09-15T23:08:14.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:14 smithi192 ceph-mon[35427]: pgmap v1020: 97 pgs: 97 active+clean; 554 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 5.5 MiB/s wr, 240 op/s 2024-09-15T23:08:14.316 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:14 smithi111 ceph-mon[30629]: pgmap v1020: 97 pgs: 97 active+clean; 554 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 5.5 MiB/s wr, 240 op/s 2024-09-15T23:08:14.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:14 smithi121 ceph-mon[34815]: pgmap v1020: 97 pgs: 97 active+clean; 554 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 5.5 MiB/s wr, 240 op/s 2024-09-15T23:08:14.683 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174573 16.93 MB/sec execute 402 sec latency 7091.067 ms 2024-09-15T23:08:15.684 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174573 16.88 MB/sec execute 403 sec latency 8091.126 ms 2024-09-15T23:08:16.307 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:16 smithi111 ceph-mon[30629]: pgmap v1021: 97 pgs: 97 active+clean; 558 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 5.9 MiB/s wr, 265 op/s 2024-09-15T23:08:16.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:16 smithi192 ceph-mon[35427]: pgmap v1021: 97 pgs: 97 active+clean; 558 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 5.9 MiB/s wr, 265 op/s 2024-09-15T23:08:16.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:16 smithi121 ceph-mon[34815]: pgmap v1021: 97 pgs: 97 active+clean; 558 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 5.9 MiB/s wr, 265 op/s 2024-09-15T23:08:16.683 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174573 16.84 MB/sec execute 404 sec latency 9091.180 ms 2024-09-15T23:08:17.683 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174575 16.80 MB/sec execute 405 sec latency 10091.245 ms 2024-09-15T23:08:18.299 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:18 smithi111 ceph-mon[30629]: pgmap v1022: 97 pgs: 97 active+clean; 563 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 5.7 MiB/s wr, 247 op/s 2024-09-15T23:08:18.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:18 smithi192 ceph-mon[35427]: pgmap v1022: 97 pgs: 97 active+clean; 563 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 5.7 MiB/s wr, 247 op/s 2024-09-15T23:08:18.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:18 smithi121 ceph-mon[34815]: pgmap v1022: 97 pgs: 97 active+clean; 563 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 5.7 MiB/s wr, 247 op/s 2024-09-15T23:08:18.683 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174575 16.76 MB/sec execute 406 sec latency 11091.299 ms 2024-09-15T23:08:19.684 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174589 16.72 MB/sec execute 407 sec latency 11391.339 ms 2024-09-15T23:08:20.269 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:20 smithi111 ceph-mon[30629]: pgmap v1023: 97 pgs: 97 active+clean; 564 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 6.5 MiB/s wr, 251 op/s 2024-09-15T23:08:20.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:20 smithi192 ceph-mon[35427]: pgmap v1023: 97 pgs: 97 active+clean; 564 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 6.5 MiB/s wr, 251 op/s 2024-09-15T23:08:20.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:20 smithi121 ceph-mon[34815]: pgmap v1023: 97 pgs: 97 active+clean; 564 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 6.5 MiB/s wr, 251 op/s 2024-09-15T23:08:20.683 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174589 16.68 MB/sec execute 408 sec latency 6108.581 ms 2024-09-15T23:08:21.684 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174589 16.64 MB/sec execute 409 sec latency 7107.037 ms 2024-09-15T23:08:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.034+0000 7fc6f4e00640 1 -- 172.21.15.111:0/1031019097 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6f0103050 msgr2=0x7fc6f0103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.034+0000 7fc6f4e00640 1 --2- 172.21.15.111:0/1031019097 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6f0103050 0x7fc6f0103450 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009900 tx=0x7fc6e002a3f0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.034+0000 7fc6f4e00640 1 -- 172.21.15.111:0/1031019097 shutdown_connections 2024-09-15T23:08:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.034+0000 7fc6f4e00640 1 --2- 172.21.15.111:0/1031019097 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6f0101f40 0x7fc6f0109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.034+0000 7fc6f4e00640 1 --2- 172.21.15.111:0/1031019097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6f0101580 0x7fc6f0101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.034+0000 7fc6f4e00640 1 --2- 172.21.15.111:0/1031019097 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6f0103050 0x7fc6f0103450 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.036 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.034+0000 7fc6f4e00640 1 -- 172.21.15.111:0/1031019097 >> 172.21.15.111:0/1031019097 conn(0x7fc6f00fd0d0 msgr2=0x7fc6f00ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:08:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.034+0000 7fc6f4e00640 1 -- 172.21.15.111:0/1031019097 shutdown_connections 2024-09-15T23:08:22.037 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.034+0000 7fc6f4e00640 1 -- 172.21.15.111:0/1031019097 wait complete. 2024-09-15T23:08:22.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.035+0000 7fc6f4e00640 1 Processor -- start 2024-09-15T23:08:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.035+0000 7fc6f4e00640 1 -- start start 2024-09-15T23:08:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.036+0000 7fc6f4e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6f0101580 0x7fc6f0199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.036+0000 7fc6f4e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6f0101f40 0x7fc6f019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.036+0000 7fc6f4e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6f0103050 0x7fc6f01a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.036+0000 7fc6f4e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f010d6f0 con 0x7fc6f0101f40 2024-09-15T23:08:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.036+0000 7fc6f4e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f010d570 con 0x7fc6f0103050 2024-09-15T23:08:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.036+0000 7fc6f4e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f010d870 con 0x7fc6f0101580 2024-09-15T23:08:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.036+0000 7fc6eea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6f0101f40 0x7fc6f019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.036+0000 7fc6ef400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6f0101580 0x7fc6f0199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.036+0000 7fc6ef400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6f0101580 0x7fc6f0199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:51780/0 (socket says 172.21.15.111:51780) 2024-09-15T23:08:22.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.036+0000 7fc6ef400640 1 -- 172.21.15.111:0/3149728392 learned_addr learned my addr 172.21.15.111:0/3149728392 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:08:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.036+0000 7fc6efe00640 1 --2- 172.21.15.111:0/3149728392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6f0103050 0x7fc6f01a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.036+0000 7fc6eea00640 1 -- 172.21.15.111:0/3149728392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6f0101580 msgr2=0x7fc6f0199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.036+0000 7fc6eea00640 1 --2- 172.21.15.111:0/3149728392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6f0101580 0x7fc6f0199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.036+0000 7fc6eea00640 1 -- 172.21.15.111:0/3149728392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6f0103050 msgr2=0x7fc6f01a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.036+0000 7fc6eea00640 1 --2- 172.21.15.111:0/3149728392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6f0103050 0x7fc6f01a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.036+0000 7fc6eea00640 1 -- 172.21.15.111:0/3149728392 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f01a1c10 con 0x7fc6f0101f40 2024-09-15T23:08:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.037+0000 7fc6ef400640 1 --2- 172.21.15.111:0/3149728392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6f0101580 0x7fc6f0199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:08:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.037+0000 7fc6eea00640 1 --2- 172.21.15.111:0/3149728392 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6f0101f40 0x7fc6f019a530 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc00e9f0 tx=0x7fc6dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:08:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.037+0000 7fc6e7e00640 1 -- 172.21.15.111:0/3149728392 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc00ccc0 con 0x7fc6f0101f40 2024-09-15T23:08:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.037+0000 7fc6e7e00640 1 -- 172.21.15.111:0/3149728392 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6dc005020 con 0x7fc6f0101f40 2024-09-15T23:08:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.037+0000 7fc6f4e00640 1 -- 172.21.15.111:0/3149728392 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f01a1f00 con 0x7fc6f0101f40 2024-09-15T23:08:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.037+0000 7fc6e7e00640 1 -- 172.21.15.111:0/3149728392 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc004a00 con 0x7fc6f0101f40 2024-09-15T23:08:22.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.037+0000 7fc6f4e00640 1 -- 172.21.15.111:0/3149728392 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6f01a2370 con 0x7fc6f0101f40 2024-09-15T23:08:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.041+0000 7fc6f93db740 1 Processor -- start 2024-09-15T23:08:22.042 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.041+0000 7fc6f93db740 1 -- start start 2024-09-15T23:08:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.042+0000 7fc6f93db740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559441ec7260 0x559441ec7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:22.043 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.042+0000 7fc6f93db740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559441ec7ba0 0x559441ecc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.042+0000 7fc6f93db740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559441eccc30 0x559441ecf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.042+0000 7fc6f93db740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559441e2c100 con 0x559441eccc30 2024-09-15T23:08:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.042+0000 7fc6f93db740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559441dd41b0 con 0x559441ec7260 2024-09-15T23:08:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.042+0000 7fc6f93db740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x559441e280c0 con 0x559441ec7ba0 2024-09-15T23:08:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.042+0000 7fc6efe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559441ec7260 0x559441ec7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.042+0000 7fc6efe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559441ec7260 0x559441ec7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:54346/0 (socket says 172.21.15.111:54346) 2024-09-15T23:08:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.042+0000 7fc6efe00640 1 -- 172.21.15.111:0/2697509382 learned_addr learned my addr 172.21.15.111:0/2697509382 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:08:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.042+0000 7fc6e7e00640 1 -- 172.21.15.111:0/3149728392 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fc6dc0051d0 con 0x7fc6f0101f40 2024-09-15T23:08:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.042+0000 7fc6ef400640 1 --2- 172.21.15.111:0/2697509382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559441ec7ba0 0x559441ecc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.042+0000 7fc6efe00640 1 -- 172.21.15.111:0/2697509382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559441ec7ba0 msgr2=0x559441ecc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.042+0000 7fc6efe00640 1 --2- 172.21.15.111:0/2697509382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559441ec7ba0 0x559441ecc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.042+0000 7fc6efe00640 1 -- 172.21.15.111:0/2697509382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559441eccc30 msgr2=0x559441ecf0f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T23:08:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.042+0000 7fc6efe00640 1 --2- 172.21.15.111:0/2697509382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559441eccc30 0x559441ecf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.042+0000 7fc6efe00640 1 -- 172.21.15.111:0/2697509382 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559441b3f210 con 0x559441ec7260 2024-09-15T23:08:22.045 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.042+0000 7fc6ef400640 1 --2- 172.21.15.111:0/2697509382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559441ec7ba0 0x559441ecc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:08:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.043+0000 7fc6efe00640 1 --2- 172.21.15.111:0/2697509382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559441ec7260 0x559441ec7660 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8011440 tx=0x7fc6e8011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b2dfdb2041f21a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:08:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.043+0000 7fc6e4c00640 1 -- 172.21.15.111:0/2697509382 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8003c30 con 0x559441ec7260 2024-09-15T23:08:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.043+0000 7fc6e4c00640 1 -- 172.21.15.111:0/2697509382 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6e8003dd0 con 0x559441ec7260 2024-09-15T23:08:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.043+0000 7fc6e4c00640 1 -- 172.21.15.111:0/2697509382 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e800fac0 con 0x559441ec7260 2024-09-15T23:08:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.043+0000 7fc6e7e00640 1 --2- 172.21.15.111:0/3149728392 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc6d40a3240 0x7fc6d40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.043+0000 7fc6e7e00640 1 -- 172.21.15.111:0/3149728392 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fc6dc0cf820 con 0x7fc6f0101f40 2024-09-15T23:08:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.044+0000 7fc6f93db740 1 -- 172.21.15.111:0/2697509382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559441ec7260 msgr2=0x559441ec7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.044+0000 7fc6f93db740 1 --2- 172.21.15.111:0/2697509382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559441ec7260 0x559441ec7660 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8011440 tx=0x7fc6e8011900 comp rx=0 tx=0).stop 2024-09-15T23:08:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.044+0000 7fc6ef400640 1 --2- 172.21.15.111:0/3149728392 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc6d40a3240 0x7fc6d40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.044+0000 7fc6f93db740 1 -- 172.21.15.111:0/2697509382 shutdown_connections 2024-09-15T23:08:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.044+0000 7fc6f93db740 1 --2- 172.21.15.111:0/2697509382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559441eccc30 0x559441ecf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.044+0000 7fc6f93db740 1 --2- 172.21.15.111:0/2697509382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559441ec7ba0 0x559441ecc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.044+0000 7fc6f93db740 1 --2- 172.21.15.111:0/2697509382 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559441ec7260 0x559441ec7660 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.044+0000 7fc6f93db740 1 -- 172.21.15.111:0/2697509382 >> 172.21.15.111:0/2697509382 conn(0x559441ec28d0 msgr2=0x559441ec2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:08:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.044+0000 7fc6f93db740 1 -- 172.21.15.111:0/2697509382 shutdown_connections 2024-09-15T23:08:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.044+0000 7fc6f93db740 1 -- 172.21.15.111:0/2697509382 wait complete. 2024-09-15T23:08:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.044+0000 7fc6f93db740 1 -- start start 2024-09-15T23:08:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.044+0000 7fc6ef400640 1 --2- 172.21.15.111:0/3149728392 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc6d40a3240 0x7fc6d40a5700 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0000c00 tx=0x7fc6e00046c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:08:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.045+0000 7fc6f93db740 1 Processor -- start 2024-09-15T23:08:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.045+0000 7fc6f93db740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559441ec7260 0x559441ed1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.045+0000 7fc6f93db740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559441ec7ba0 0x559441ed19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.045+0000 7fc6f93db740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559441eccc30 0x559441ed8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.045+0000 7fc6f93db740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559441ecfc00 con 0x559441ec7260 2024-09-15T23:08:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.045+0000 7fc6f93db740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559441db5090 con 0x559441eccc30 2024-09-15T23:08:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.045+0000 7fc6f93db740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x559441ecfd80 con 0x559441ec7ba0 2024-09-15T23:08:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.045+0000 7fc6f93db740 10 client.0 _ll_register_callbacks cb 0x559441e09f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:08:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.045+0000 7fc6ef400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559441ec7260 0x559441ed1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.045+0000 7fc6eea00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559441ec7ba0 0x559441ed19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.045+0000 7fc6ef400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559441ec7260 0x559441ed1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56604/0 (socket says 172.21.15.111:56604) 2024-09-15T23:08:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.045+0000 7fc6ef400640 1 -- 172.21.15.111:0/3003420001 learned_addr learned my addr 172.21.15.111:0/3003420001 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:08:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.045+0000 7fc6efe00640 1 --2- 172.21.15.111:0/3003420001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559441eccc30 0x559441ed8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.046+0000 7fc6eea00640 1 -- 172.21.15.111:0/3003420001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559441eccc30 msgr2=0x559441ed8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.046+0000 7fc6eea00640 1 --2- 172.21.15.111:0/3003420001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559441eccc30 0x559441ed8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.046+0000 7fc6eea00640 1 -- 172.21.15.111:0/3003420001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559441ec7260 msgr2=0x559441ed1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.046+0000 7fc6eea00640 1 --2- 172.21.15.111:0/3003420001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559441ec7260 0x559441ed1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.046+0000 7fc6eea00640 1 -- 172.21.15.111:0/3003420001 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559441c0b620 con 0x559441ec7ba0 2024-09-15T23:08:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.046+0000 7fc6eea00640 1 --2- 172.21.15.111:0/3003420001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559441ec7ba0 0x559441ed19c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc0cd320 tx=0x7fc6dc09c600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8172243dd38c4299 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:08:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.046+0000 7fc6d9600640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:08:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.046+0000 7fc6d9600640 1 -- 172.21.15.111:0/3003420001 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc09e070 con 0x559441ec7ba0 2024-09-15T23:08:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.046+0000 7fc6d9600640 1 -- 172.21.15.111:0/3003420001 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fc6dc0a4620 con 0x559441ec7ba0 2024-09-15T23:08:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.046+0000 7fc6f93db740 1 -- 172.21.15.111:0/3003420001 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6e8003dd0 con 0x559441ec7ba0 2024-09-15T23:08:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.046+0000 7fc6d9600640 1 -- 172.21.15.111:0/3003420001 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc6dc01c040 con 0x559441ec7ba0 2024-09-15T23:08:22.051 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.046+0000 7fc6f93db740 10 client.26683 fetch_fsmap learned FSMap version 35 2024-09-15T23:08:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.046+0000 7fc6f93db740 1 -- 172.21.15.111:0/3003420001 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559441edb850 con 0x559441ec7ba0 2024-09-15T23:08:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.046+0000 7fc6d9600640 1 -- 172.21.15.111:0/3003420001 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6e8003dd0 con 0x559441ec7ba0 2024-09-15T23:08:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.047+0000 7fc6d9600640 1 -- 172.21.15.111:0/3003420001 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc6dc09b290 con 0x559441ec7ba0 2024-09-15T23:08:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.047+0000 7fc6f93db740 10 client.26683 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:08:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.047+0000 7fc6f93db740 10 client.26683 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:08:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.047+0000 7fc6f93db740 20 client.26683 populate_metadata read hostname 'smithi111' 2024-09-15T23:08:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.047+0000 7fc6f93db740 1 --2- 172.21.15.111:0/3003420001 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559441edce80 0x559441edf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.047+0000 7fc6f93db740 4 client.26683 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:08:22.052 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.047+0000 7fc6f93db740 1 -- 172.21.15.111:0/3003420001 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559441e39830 con 0x559441edce80 2024-09-15T23:08:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.047+0000 7fc6ef400640 1 --2- 172.21.15.111:0/3003420001 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559441edce80 0x559441edf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.048+0000 7fc6d9600640 10 client.26683 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:08:22.053 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.048+0000 7fc6ef400640 1 --2- 172.21.15.111:0/3003420001 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559441edce80 0x559441edf270 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96b3818a6b2fb7a9 server_cookie=7eb2393852ce3e09 in_seq=0 out_seq=0 2024-09-15T23:08:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.048+0000 7fc6d9600640 1 -- 172.21.15.111:0/3003420001 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559441e39830 con 0x559441edce80 2024-09-15T23:08:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.048+0000 7fc6d9600640 10 client.26683 handle_command_reply: tid=0 2024-09-15T23:08:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.048+0000 7fc6f93db740 1 client.26683 shutdown 2024-09-15T23:08:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.049+0000 7fc6f93db740 1 -- 172.21.15.111:0/3003420001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559441ec7ba0 msgr2=0x559441ed19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.049+0000 7fc6f93db740 1 --2- 172.21.15.111:0/3003420001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559441ec7ba0 0x559441ed19c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc0cd320 tx=0x7fc6dc09c600 comp rx=0 tx=0).stop 2024-09-15T23:08:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.049+0000 7fc6f93db740 1 -- 172.21.15.111:0/3003420001 shutdown_connections 2024-09-15T23:08:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.049+0000 7fc6f93db740 1 --2- 172.21.15.111:0/3003420001 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559441edce80 0x559441edf270 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.049+0000 7fc6f93db740 1 --2- 172.21.15.111:0/3003420001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559441eccc30 0x559441ed8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.049+0000 7fc6f93db740 1 --2- 172.21.15.111:0/3003420001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559441ec7ba0 0x559441ed19c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.049+0000 7fc6f93db740 1 --2- 172.21.15.111:0/3003420001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559441ec7260 0x559441ed1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.049+0000 7fc6d9600640 0 client.26683 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:08:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.049+0000 7fc6f93db740 1 -- 172.21.15.111:0/3003420001 >> 172.21.15.111:0/3003420001 conn(0x559441ec28d0 msgr2=0x559441ec40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:08:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.049+0000 7fc6f93db740 1 -- 172.21.15.111:0/3003420001 shutdown_connections 2024-09-15T23:08:22.055 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.049+0000 7fc6f93db740 1 -- 172.21.15.111:0/3003420001 wait complete. 2024-09-15T23:08:22.055 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.049+0000 7fc6f93db740 20 client.26683 trim_cache size 0 max 16384 2024-09-15T23:08:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.064+0000 7fc6f93db740 1 Processor -- start 2024-09-15T23:08:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.064+0000 7fc6f93db740 1 -- start start 2024-09-15T23:08:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.064+0000 7fc6f93db740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559441ec7260 0x559441ed1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.065+0000 7fc6f93db740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559441ec7ba0 0x559441ed8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.065+0000 7fc6f93db740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559441eccc30 0x559441ed19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.065+0000 7fc6f93db740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f01ad820 con 0x559441eccc30 2024-09-15T23:08:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.065+0000 7fc6f93db740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f01ab0f0 con 0x559441ec7ba0 2024-09-15T23:08:22.066 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.065+0000 7fc6f93db740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f01a8f40 con 0x559441ec7260 2024-09-15T23:08:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.065+0000 7fc6ef400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559441ec7260 0x559441ed1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:22.067 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.065+0000 7fc6efe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559441eccc30 0x559441ed19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.065+0000 7fc6ef400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559441ec7260 0x559441ed1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:51792/0 (socket says 172.21.15.111:51792) 2024-09-15T23:08:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.065+0000 7fc6ef400640 1 -- 172.21.15.111:0/4266102543 learned_addr learned my addr 172.21.15.111:0/4266102543 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:08:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.065+0000 7fc6eea00640 1 --2- 172.21.15.111:0/4266102543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559441ec7ba0 0x559441ed8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.065+0000 7fc6ef400640 1 -- 172.21.15.111:0/4266102543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559441ec7ba0 msgr2=0x559441ed8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.065+0000 7fc6ef400640 1 --2- 172.21.15.111:0/4266102543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559441ec7ba0 0x559441ed8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.065+0000 7fc6ef400640 1 -- 172.21.15.111:0/4266102543 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559441eccc30 msgr2=0x559441ed19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.065+0000 7fc6ef400640 1 --2- 172.21.15.111:0/4266102543 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559441eccc30 0x559441ed19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.065+0000 7fc6ef400640 1 -- 172.21.15.111:0/4266102543 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559441edf8a0 con 0x559441ec7260 2024-09-15T23:08:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.066+0000 7fc6ef400640 1 --2- 172.21.15.111:0/4266102543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559441ec7260 0x559441ed1480 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0005ab0 tx=0x7fc6e003e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cae379400cd02d1a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:08:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.066+0000 7fc6dbe00640 1 -- 172.21.15.111:0/4266102543 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0042070 con 0x559441ec7260 2024-09-15T23:08:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.066+0000 7fc6dbe00640 1 -- 172.21.15.111:0/4266102543 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6e0057070 con 0x559441ec7260 2024-09-15T23:08:22.068 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.066+0000 7fc6dbe00640 1 -- 172.21.15.111:0/4266102543 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0035040 con 0x559441ec7260 2024-09-15T23:08:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.067+0000 7fc6f93db740 1 -- 172.21.15.111:0/4266102543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559441ec7260 msgr2=0x559441ed1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.067+0000 7fc6f93db740 1 --2- 172.21.15.111:0/4266102543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559441ec7260 0x559441ed1480 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0005ab0 tx=0x7fc6e003e000 comp rx=0 tx=0).stop 2024-09-15T23:08:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.067+0000 7fc6f93db740 1 -- 172.21.15.111:0/4266102543 shutdown_connections 2024-09-15T23:08:22.071 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.067+0000 7fc6f93db740 1 --2- 172.21.15.111:0/4266102543 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559441eccc30 0x559441ed19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.067+0000 7fc6f93db740 1 --2- 172.21.15.111:0/4266102543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559441ec7ba0 0x559441ed8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.067+0000 7fc6f93db740 1 --2- 172.21.15.111:0/4266102543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559441ec7260 0x559441ed1480 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.067+0000 7fc6f93db740 1 -- 172.21.15.111:0/4266102543 >> 172.21.15.111:0/4266102543 conn(0x559441ec28d0 msgr2=0x559441ec8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:08:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.067+0000 7fc6f93db740 1 -- 172.21.15.111:0/4266102543 shutdown_connections 2024-09-15T23:08:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.067+0000 7fc6f93db740 1 -- 172.21.15.111:0/4266102543 wait complete. 2024-09-15T23:08:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.067+0000 7fc6f93db740 1 -- start start 2024-09-15T23:08:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.067+0000 7fc6f93db740 1 Processor -- start 2024-09-15T23:08:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.068+0000 7fc6f93db740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559441ec7260 0x559441ed5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.068+0000 7fc6f93db740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559441ec7ba0 0x559441ed5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.068+0000 7fc6f93db740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559441eccc30 0x559441f45b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.068+0000 7fc6f93db740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f01a95b0 con 0x559441ec7ba0 2024-09-15T23:08:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.068+0000 7fc6f93db740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f01aaa20 con 0x559441eccc30 2024-09-15T23:08:22.072 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.068+0000 7fc6f93db740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f01a9c80 con 0x559441ec7260 2024-09-15T23:08:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.068+0000 7fc6f93db740 10 client.0 _ll_register_callbacks cb 0x7fc6dc00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:08:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.068+0000 7fc6eea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559441ec7ba0 0x559441ed5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.068+0000 7fc6ef400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559441ec7260 0x559441ed5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.068+0000 7fc6eea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559441ec7ba0 0x559441ed5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56632/0 (socket says 172.21.15.111:56632) 2024-09-15T23:08:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.068+0000 7fc6eea00640 1 -- 172.21.15.111:0/925308703 learned_addr learned my addr 172.21.15.111:0/925308703 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:08:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.068+0000 7fc6efe00640 1 --2- 172.21.15.111:0/925308703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559441eccc30 0x559441f45b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.068+0000 7fc6ef400640 1 -- 172.21.15.111:0/925308703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559441eccc30 msgr2=0x559441f45b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.068+0000 7fc6ef400640 1 --2- 172.21.15.111:0/925308703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559441eccc30 0x559441f45b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.068+0000 7fc6ef400640 1 -- 172.21.15.111:0/925308703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559441ec7ba0 msgr2=0x559441ed5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.068+0000 7fc6ef400640 1 --2- 172.21.15.111:0/925308703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559441ec7ba0 0x559441ed5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.068+0000 7fc6ef400640 1 -- 172.21.15.111:0/925308703 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559441f46080 con 0x559441ec7260 2024-09-15T23:08:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.068+0000 7fc6eea00640 1 --2- 172.21.15.111:0/925308703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559441ec7ba0 0x559441ed5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:08:22.073 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.069+0000 7fc6ef400640 1 --2- 172.21.15.111:0/925308703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559441ec7260 0x559441ed5970 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0031440 tx=0x7fc6e00044a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47061fdf2a15a2e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:08:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.069+0000 7fc6d9600640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:08:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.069+0000 7fc6d9600640 1 -- 172.21.15.111:0/925308703 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0061070 con 0x559441ec7260 2024-09-15T23:08:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.069+0000 7fc6f93db740 1 -- 172.21.15.111:0/925308703 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6e0057070 con 0x559441ec7260 2024-09-15T23:08:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.069+0000 7fc6d9600640 1 -- 172.21.15.111:0/925308703 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fc6e0003c40 con 0x559441ec7260 2024-09-15T23:08:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.069+0000 7fc6d9600640 1 -- 172.21.15.111:0/925308703 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc6e006d050 con 0x559441ec7260 2024-09-15T23:08:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.069+0000 7fc6f93db740 10 client.26689 fetch_fsmap learned FSMap version 35 2024-09-15T23:08:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.069+0000 7fc6f93db740 1 -- 172.21.15.111:0/925308703 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559441f472b0 con 0x559441ec7260 2024-09-15T23:08:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.070+0000 7fc6d9600640 1 -- 172.21.15.111:0/925308703 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6e0057070 con 0x559441ec7260 2024-09-15T23:08:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.070+0000 7fc6d9600640 1 -- 172.21.15.111:0/925308703 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fc6e003c840 con 0x559441ec7260 2024-09-15T23:08:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.070+0000 7fc6f93db740 10 client.26689 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:08:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.070+0000 7fc6f93db740 10 client.26689 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:08:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.070+0000 7fc6f93db740 20 client.26689 populate_metadata read hostname 'smithi111' 2024-09-15T23:08:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.070+0000 7fc6f93db740 1 --2- 172.21.15.111:0/925308703 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559441f47be0 0x559441f4a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:22.074 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.070+0000 7fc6f93db740 4 client.26689 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:08:22.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.070+0000 7fc6f93db740 1 -- 172.21.15.111:0/925308703 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559441e33b80 con 0x559441f47be0 2024-09-15T23:08:22.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.070+0000 7fc6eea00640 1 --2- 172.21.15.111:0/925308703 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559441f47be0 0x559441f4a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:22.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.071+0000 7fc6eea00640 1 --2- 172.21.15.111:0/925308703 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559441f47be0 0x559441f4a000 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d290ad88e6ba44c server_cookie=e1352134495cee2c in_seq=0 out_seq=0 2024-09-15T23:08:22.075 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.071+0000 7fc6d9600640 10 client.26689 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:08:22.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.129+0000 7fc6d9600640 1 -- 172.21.15.111:0/925308703 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559441e33b80 con 0x559441f47be0 2024-09-15T23:08:22.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.129+0000 7fc6d9600640 10 client.26689 handle_command_reply: tid=0 2024-09-15T23:08:22.131 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.129+0000 7fc6f93db740 1 client.26689 shutdown 2024-09-15T23:08:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.130+0000 7fc6f93db740 1 -- 172.21.15.111:0/925308703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559441ec7260 msgr2=0x559441ed5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.130+0000 7fc6f93db740 1 --2- 172.21.15.111:0/925308703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559441ec7260 0x559441ed5970 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0031440 tx=0x7fc6e00044a0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.130+0000 7fc6f93db740 1 -- 172.21.15.111:0/925308703 shutdown_connections 2024-09-15T23:08:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.130+0000 7fc6f93db740 1 --2- 172.21.15.111:0/925308703 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559441f47be0 0x559441f4a000 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.130+0000 7fc6f93db740 1 --2- 172.21.15.111:0/925308703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559441eccc30 0x559441f45b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.130+0000 7fc6f93db740 1 --2- 172.21.15.111:0/925308703 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559441ec7ba0 0x559441ed5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.130+0000 7fc6d9600640 0 client.26689 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:08:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.130+0000 7fc6f93db740 1 --2- 172.21.15.111:0/925308703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559441ec7260 0x559441ed5970 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.130+0000 7fc6f93db740 1 -- 172.21.15.111:0/925308703 >> 172.21.15.111:0/925308703 conn(0x559441ec28d0 msgr2=0x559441ed1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:08:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.130+0000 7fc6f93db740 1 -- 172.21.15.111:0/925308703 shutdown_connections 2024-09-15T23:08:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.130+0000 7fc6f93db740 1 -- 172.21.15.111:0/925308703 wait complete. 2024-09-15T23:08:22.132 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.130+0000 7fc6f93db740 20 client.26689 trim_cache size 0 max 16384 2024-09-15T23:08:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:08:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:08:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:08:22.133 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:08:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.131+0000 7fc6f4e00640 1 -- 172.21.15.111:0/3149728392 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc6d40a3240 msgr2=0x7fc6d40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.131+0000 7fc6f4e00640 1 --2- 172.21.15.111:0/3149728392 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc6d40a3240 0x7fc6d40a5700 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0000c00 tx=0x7fc6e00046c0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.131+0000 7fc6f4e00640 1 -- 172.21.15.111:0/3149728392 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6f0101f40 msgr2=0x7fc6f019a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.131+0000 7fc6f4e00640 1 --2- 172.21.15.111:0/3149728392 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6f0101f40 0x7fc6f019a530 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc00e9f0 tx=0x7fc6dc00edf0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.131+0000 7fc6f4e00640 1 -- 172.21.15.111:0/3149728392 shutdown_connections 2024-09-15T23:08:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.131+0000 7fc6f4e00640 1 --2- 172.21.15.111:0/3149728392 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fc6d40a3240 0x7fc6d40a5700 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.131+0000 7fc6f4e00640 1 --2- 172.21.15.111:0/3149728392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc6f0103050 0x7fc6f01a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.131+0000 7fc6f4e00640 1 --2- 172.21.15.111:0/3149728392 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc6f0101f40 0x7fc6f019a530 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.131+0000 7fc6f4e00640 1 --2- 172.21.15.111:0/3149728392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6f0101580 0x7fc6f0199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.131+0000 7fc6f4e00640 1 -- 172.21.15.111:0/3149728392 >> 172.21.15.111:0/3149728392 conn(0x7fc6f00fd0d0 msgr2=0x7fc6f0108380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:08:22.134 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.131+0000 7fc6f4e00640 1 -- 172.21.15.111:0/3149728392 shutdown_connections 2024-09-15T23:08:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:22.131+0000 7fc6f4e00640 1 -- 172.21.15.111:0/3149728392 wait complete. 2024-09-15T23:08:22.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:22 smithi192 ceph-mon[35427]: pgmap v1024: 97 pgs: 97 active+clean; 556 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 6.0 MiB/s wr, 238 op/s 2024-09-15T23:08:22.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:22 smithi121 ceph-mon[34815]: pgmap v1024: 97 pgs: 97 active+clean; 556 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 6.0 MiB/s wr, 238 op/s 2024-09-15T23:08:22.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:22 smithi111 ceph-mon[30629]: pgmap v1024: 97 pgs: 97 active+clean; 556 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 6.0 MiB/s wr, 238 op/s 2024-09-15T23:08:22.684 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174671 16.61 MB/sec execute 410 sec latency 3787.403 ms 2024-09-15T23:08:23.684 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174671 16.57 MB/sec execute 411 sec latency 4787.458 ms 2024-09-15T23:08:24.202 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:24 smithi111 ceph-mon[30629]: pgmap v1025: 97 pgs: 97 active+clean; 549 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 5.6 MiB/s wr, 224 op/s 2024-09-15T23:08:24.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:24 smithi192 ceph-mon[35427]: pgmap v1025: 97 pgs: 97 active+clean; 549 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 5.6 MiB/s wr, 224 op/s 2024-09-15T23:08:24.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:24 smithi121 ceph-mon[34815]: pgmap v1025: 97 pgs: 97 active+clean; 549 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 5.6 MiB/s wr, 224 op/s 2024-09-15T23:08:24.684 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174671 16.53 MB/sec execute 412 sec latency 5787.551 ms 2024-09-15T23:08:25.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:25 smithi121 ceph-mon[34815]: pgmap v1026: 97 pgs: 97 active+clean; 545 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 5.8 MiB/s wr, 241 op/s 2024-09-15T23:08:25.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:25 smithi111 ceph-mon[30629]: pgmap v1026: 97 pgs: 97 active+clean; 545 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 5.8 MiB/s wr, 241 op/s 2024-09-15T23:08:25.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:25 smithi192 ceph-mon[35427]: pgmap v1026: 97 pgs: 97 active+clean; 545 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 5.8 MiB/s wr, 241 op/s 2024-09-15T23:08:25.684 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174671 16.49 MB/sec execute 413 sec latency 6787.629 ms 2024-09-15T23:08:26.684 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174671 16.45 MB/sec execute 414 sec latency 7787.685 ms 2024-09-15T23:08:27.684 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174671 16.41 MB/sec execute 415 sec latency 8787.726 ms 2024-09-15T23:08:28.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:27 smithi192 ceph-mon[35427]: pgmap v1027: 97 pgs: 97 active+clean; 546 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 5.2 MiB/s wr, 229 op/s 2024-09-15T23:08:28.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:27 smithi121 ceph-mon[34815]: pgmap v1027: 97 pgs: 97 active+clean; 546 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 5.2 MiB/s wr, 229 op/s 2024-09-15T23:08:28.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:27 smithi111 ceph-mon[30629]: pgmap v1027: 97 pgs: 97 active+clean; 546 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 5.2 MiB/s wr, 229 op/s 2024-09-15T23:08:28.684 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174671 16.37 MB/sec execute 416 sec latency 9787.771 ms 2024-09-15T23:08:29.684 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174671 16.33 MB/sec execute 417 sec latency 10787.813 ms 2024-09-15T23:08:30.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:29 smithi192 ceph-mon[35427]: pgmap v1028: 97 pgs: 97 active+clean; 544 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 4.8 MiB/s wr, 225 op/s 2024-09-15T23:08:30.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:29 smithi121 ceph-mon[34815]: pgmap v1028: 97 pgs: 97 active+clean; 544 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 4.8 MiB/s wr, 225 op/s 2024-09-15T23:08:30.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:29 smithi111 ceph-mon[30629]: pgmap v1028: 97 pgs: 97 active+clean; 544 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 4.8 MiB/s wr, 225 op/s 2024-09-15T23:08:30.684 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174671 16.29 MB/sec execute 418 sec latency 11787.872 ms 2024-09-15T23:08:31.685 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174671 16.25 MB/sec execute 419 sec latency 12787.914 ms 2024-09-15T23:08:32.242 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:31 smithi111 ceph-mon[30629]: pgmap v1029: 97 pgs: 97 active+clean; 543 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 3.2 MiB/s wr, 181 op/s 2024-09-15T23:08:32.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:31 smithi192 ceph-mon[35427]: pgmap v1029: 97 pgs: 97 active+clean; 543 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 3.2 MiB/s wr, 181 op/s 2024-09-15T23:08:32.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:31 smithi121 ceph-mon[34815]: pgmap v1029: 97 pgs: 97 active+clean; 543 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 3.2 MiB/s wr, 181 op/s 2024-09-15T23:08:32.684 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174675 16.22 MB/sec execute 420 sec latency 13700.556 ms 2024-09-15T23:08:33.684 INFO:tasks.workunit.client.0.smithi111.stdout: 10 174724 16.18 MB/sec execute 421 sec latency 14623.533 ms 2024-09-15T23:08:34.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:34 smithi192 ceph-mon[35427]: pgmap v1030: 97 pgs: 97 active+clean; 543 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 2.3 MiB/s wr, 136 op/s 2024-09-15T23:08:34.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:34 smithi121 ceph-mon[34815]: pgmap v1030: 97 pgs: 97 active+clean; 543 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 2.3 MiB/s wr, 136 op/s 2024-09-15T23:08:34.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:34 smithi111 ceph-mon[30629]: pgmap v1030: 97 pgs: 97 active+clean; 543 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 2.3 MiB/s wr, 136 op/s 2024-09-15T23:08:34.684 INFO:tasks.workunit.client.0.smithi111.stdout: 10 175564 16.26 MB/sec execute 422 sec latency 1675.488 ms 2024-09-15T23:08:35.266 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.264+0000 7f8676200640 1 -- 172.21.15.111:0/3995148491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8670104d90 msgr2=0x7f8670107180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.264+0000 7f8676200640 1 --2- 172.21.15.111:0/3995148491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8670104d90 0x7f8670107180 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f865c009900 tx=0x7f865c02a410 comp rx=0 tx=0).stop 2024-09-15T23:08:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.265+0000 7f8676200640 1 -- 172.21.15.111:0/3995148491 shutdown_connections 2024-09-15T23:08:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.265+0000 7f8676200640 1 --2- 172.21.15.111:0/3995148491 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86701076c0 0x7f8670109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.265+0000 7f8676200640 1 --2- 172.21.15.111:0/3995148491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8670104d90 0x7f8670107180 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.265+0000 7f8676200640 1 --2- 172.21.15.111:0/3995148491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8670102460 0x7f8670104850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.265+0000 7f8676200640 1 -- 172.21.15.111:0/3995148491 >> 172.21.15.111:0/3995148491 conn(0x7f86700fbfa0 msgr2=0x7f86700fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:08:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.265+0000 7f8676200640 1 -- 172.21.15.111:0/3995148491 shutdown_connections 2024-09-15T23:08:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.265+0000 7f8676200640 1 -- 172.21.15.111:0/3995148491 wait complete. 2024-09-15T23:08:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.266+0000 7f8676200640 1 Processor -- start 2024-09-15T23:08:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.266+0000 7f8676200640 1 -- start start 2024-09-15T23:08:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.266+0000 7f8676200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8670102460 0x7f8670195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.266+0000 7f8676200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8670104d90 0x7f8670196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.266+0000 7f8676200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86701076c0 0x7f86701a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.266+0000 7f8676200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f867010d600 con 0x7f8670102460 2024-09-15T23:08:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.266+0000 7f8676200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f867010d480 con 0x7f8670104d90 2024-09-15T23:08:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.266+0000 7f8676200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f867010d780 con 0x7f86701076c0 2024-09-15T23:08:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.266+0000 7f866fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8670104d90 0x7f8670196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.267+0000 7f8674e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8670102460 0x7f8670195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.267+0000 7f866fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8670104d90 0x7f8670196090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:44690/0 (socket says 172.21.15.111:44690) 2024-09-15T23:08:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.267+0000 7f8675800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86701076c0 0x7f86701a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.267+0000 7f866fe00640 1 -- 172.21.15.111:0/1485611890 learned_addr learned my addr 172.21.15.111:0/1485611890 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.267+0000 7f8675800640 1 -- 172.21.15.111:0/1485611890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8670104d90 msgr2=0x7f8670196090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.267+0000 7f8675800640 1 --2- 172.21.15.111:0/1485611890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8670104d90 0x7f8670196090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.267+0000 7f8675800640 1 -- 172.21.15.111:0/1485611890 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8670102460 msgr2=0x7f8670195b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.267+0000 7f8675800640 1 --2- 172.21.15.111:0/1485611890 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8670102460 0x7f8670195b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.267+0000 7f8675800640 1 -- 172.21.15.111:0/1485611890 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86701a5fa0 con 0x7f86701076c0 2024-09-15T23:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.267+0000 7f8675800640 1 --2- 172.21.15.111:0/1485611890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86701076c0 0x7f86701a5930 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8668005fd0 tx=0x7f866800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.268+0000 7f866d600640 1 -- 172.21.15.111:0/1485611890 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866800cdf0 con 0x7f86701076c0 2024-09-15T23:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.268+0000 7f866d600640 1 -- 172.21.15.111:0/1485611890 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8668005020 con 0x7f86701076c0 2024-09-15T23:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.268+0000 7f866d600640 1 -- 172.21.15.111:0/1485611890 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86680106d0 con 0x7f86701076c0 2024-09-15T23:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.268+0000 7f8676200640 1 -- 172.21.15.111:0/1485611890 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86701a6290 con 0x7f86701076c0 2024-09-15T23:08:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.269+0000 7f8676200640 1 -- 172.21.15.111:0/1485611890 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86701a6760 con 0x7f86701076c0 2024-09-15T23:08:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.272+0000 7f867a801740 1 Processor -- start 2024-09-15T23:08:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.272+0000 7f866d600640 1 -- 172.21.15.111:0/1485611890 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f86680054d0 con 0x7f86701076c0 2024-09-15T23:08:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.272+0000 7f867a801740 1 -- start start 2024-09-15T23:08:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.273+0000 7f867a801740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642c62d0260 0x5642c62d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.273+0000 7f867a801740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642c62d0ba0 0x5642c62d5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.273+0000 7f867a801740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642c62d5c30 0x5642c62d80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.273+0000 7f867a801740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5642c6235100 con 0x5642c62d5c30 2024-09-15T23:08:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.273+0000 7f867a801740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5642c61dd1b0 con 0x5642c62d0260 2024-09-15T23:08:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.273+0000 7f867a801740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5642c62310c0 con 0x5642c62d0ba0 2024-09-15T23:08:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.273+0000 7f8674e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642c62d0ba0 0x5642c62d5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.273+0000 7f8674e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642c62d0ba0 0x5642c62d5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:33104/0 (socket says 172.21.15.111:33104) 2024-09-15T23:08:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.273+0000 7f8674e00640 1 -- 172.21.15.111:0/182603587 learned_addr learned my addr 172.21.15.111:0/182603587 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:08:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.273+0000 7f866fe00640 1 --2- 172.21.15.111:0/182603587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642c62d0260 0x5642c62d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.274+0000 7f866fe00640 1 --2- 172.21.15.111:0/182603587 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642c62d5c30 0x5642c62d80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.274+0000 7f8674e00640 1 -- 172.21.15.111:0/182603587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642c62d0260 msgr2=0x5642c62d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.274+0000 7f8674e00640 1 --2- 172.21.15.111:0/182603587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642c62d0260 0x5642c62d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.274+0000 7f8674e00640 1 -- 172.21.15.111:0/182603587 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642c62d5c30 msgr2=0x5642c62d80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.274+0000 7f8674e00640 1 --2- 172.21.15.111:0/182603587 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642c62d5c30 0x5642c62d80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.274+0000 7f8674e00640 1 -- 172.21.15.111:0/182603587 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642c5f48210 con 0x5642c62d0ba0 2024-09-15T23:08:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.274+0000 7f866d600640 1 --2- 172.21.15.111:0/1485611890 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f86580a3240 0x7f86580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.274+0000 7f8674e00640 1 --2- 172.21.15.111:0/182603587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642c62d0ba0 0x5642c62d5660 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8664009c90 tx=0x7f8664011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60a4b0fd362e4b82 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:08:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.274+0000 7f866d600640 1 -- 172.21.15.111:0/1485611890 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f86680d0340 con 0x7f86701076c0 2024-09-15T23:08:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.274+0000 7f8662000640 1 -- 172.21.15.111:0/182603587 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664005220 con 0x5642c62d0ba0 2024-09-15T23:08:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.275+0000 7f8662000640 1 -- 172.21.15.111:0/182603587 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8664004c80 con 0x5642c62d0ba0 2024-09-15T23:08:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.275+0000 7f8662000640 1 -- 172.21.15.111:0/182603587 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664019710 con 0x5642c62d0ba0 2024-09-15T23:08:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.275+0000 7f866fe00640 1 --2- 172.21.15.111:0/1485611890 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f86580a3240 0x7f86580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.276+0000 7f867a801740 1 -- 172.21.15.111:0/182603587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642c62d0ba0 msgr2=0x5642c62d5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.276+0000 7f867a801740 1 --2- 172.21.15.111:0/182603587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642c62d0ba0 0x5642c62d5660 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8664009c90 tx=0x7f8664011810 comp rx=0 tx=0).stop 2024-09-15T23:08:35.279 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.276+0000 7f866fe00640 1 --2- 172.21.15.111:0/1485611890 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f86580a3240 0x7f86580a5700 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f865c000c00 tx=0x7f865c0315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:08:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.276+0000 7f867a801740 1 -- 172.21.15.111:0/182603587 shutdown_connections 2024-09-15T23:08:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.276+0000 7f867a801740 1 --2- 172.21.15.111:0/182603587 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642c62d5c30 0x5642c62d80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.276+0000 7f867a801740 1 --2- 172.21.15.111:0/182603587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642c62d0ba0 0x5642c62d5660 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.276+0000 7f867a801740 1 --2- 172.21.15.111:0/182603587 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642c62d0260 0x5642c62d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.276+0000 7f867a801740 1 -- 172.21.15.111:0/182603587 >> 172.21.15.111:0/182603587 conn(0x5642c62cb8d0 msgr2=0x5642c62cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:08:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.276+0000 7f867a801740 1 -- 172.21.15.111:0/182603587 shutdown_connections 2024-09-15T23:08:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.276+0000 7f867a801740 1 -- 172.21.15.111:0/182603587 wait complete. 2024-09-15T23:08:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.277+0000 7f867a801740 1 -- start start 2024-09-15T23:08:35.281 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.277+0000 7f867a801740 1 Processor -- start 2024-09-15T23:08:35.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.278+0000 7f867a801740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642c62d0260 0x5642c62da3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:35.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.278+0000 7f867a801740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642c62d0ba0 0x5642c62da920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:35.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.278+0000 7f867a801740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642c62d5c30 0x5642c62e1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:35.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.278+0000 7f867a801740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5642c62d8c00 con 0x5642c62d0ba0 2024-09-15T23:08:35.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.278+0000 7f867a801740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5642c61be090 con 0x5642c62d0260 2024-09-15T23:08:35.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.278+0000 7f867a801740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5642c62d8d80 con 0x5642c62d5c30 2024-09-15T23:08:35.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.278+0000 7f867a801740 10 client.0 _ll_register_callbacks cb 0x5642c6212f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:08:35.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.278+0000 7f8674e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642c62d0260 0x5642c62da3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:35.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.278+0000 7f8674e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642c62d0260 0x5642c62da3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:44704/0 (socket says 172.21.15.111:44704) 2024-09-15T23:08:35.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.278+0000 7f8674e00640 1 -- 172.21.15.111:0/1359465729 learned_addr learned my addr 172.21.15.111:0/1359465729 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:08:35.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.278+0000 7f866fe00640 1 --2- 172.21.15.111:0/1359465729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642c62d0ba0 0x5642c62da920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:35.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.278+0000 7f8675800640 1 --2- 172.21.15.111:0/1359465729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642c62d5c30 0x5642c62e1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:35.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.279+0000 7f8674e00640 1 -- 172.21.15.111:0/1359465729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642c62d5c30 msgr2=0x5642c62e1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.279+0000 7f8674e00640 1 --2- 172.21.15.111:0/1359465729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642c62d5c30 0x5642c62e1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.279+0000 7f8674e00640 1 -- 172.21.15.111:0/1359465729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642c62d0ba0 msgr2=0x5642c62da920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.279+0000 7f8674e00640 1 --2- 172.21.15.111:0/1359465729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642c62d0ba0 0x5642c62da920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.279+0000 7f8674e00640 1 -- 172.21.15.111:0/1359465729 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642c6014620 con 0x5642c62d0260 2024-09-15T23:08:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.279+0000 7f8674e00640 1 --2- 172.21.15.111:0/1359465729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642c62d0260 0x5642c62da3e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8664000c00 tx=0x7f8664004970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c8a0e3a8530bd65 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:08:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.279+0000 7f8656a00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:08:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.280+0000 7f8656a00640 1 -- 172.21.15.111:0/1359465729 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664029070 con 0x5642c62d0260 2024-09-15T23:08:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.280+0000 7f8656a00640 1 -- 172.21.15.111:0/1359465729 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f8664023840 con 0x5642c62d0260 2024-09-15T23:08:35.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.280+0000 7f867a801740 1 -- 172.21.15.111:0/1359465729 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8664004c80 con 0x5642c62d0260 2024-09-15T23:08:35.285 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.280+0000 7f8656a00640 1 -- 172.21.15.111:0/1359465729 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8664023a40 con 0x5642c62d0260 2024-09-15T23:08:35.286 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.280+0000 7f867a801740 10 client.26711 fetch_fsmap learned FSMap version 35 2024-09-15T23:08:35.286 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.280+0000 7f867a801740 1 -- 172.21.15.111:0/1359465729 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5642c62e4750 con 0x5642c62d0260 2024-09-15T23:08:35.286 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.280+0000 7f8656a00640 1 -- 172.21.15.111:0/1359465729 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8664004c80 con 0x5642c62d0260 2024-09-15T23:08:35.286 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.281+0000 7f8656a00640 1 -- 172.21.15.111:0/1359465729 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f86640199a0 con 0x5642c62d0260 2024-09-15T23:08:35.286 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.281+0000 7f867a801740 10 client.26711 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:08:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.281+0000 7f867a801740 10 client.26711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:08:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.281+0000 7f867a801740 20 client.26711 populate_metadata read hostname 'smithi111' 2024-09-15T23:08:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.281+0000 7f867a801740 1 --2- 172.21.15.111:0/1359465729 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5642c62e5bc0 0x5642c62e7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.281+0000 7f867a801740 4 client.26711 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:08:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.281+0000 7f867a801740 1 -- 172.21.15.111:0/1359465729 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642c6242830 con 0x5642c62e5bc0 2024-09-15T23:08:35.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.281+0000 7f866fe00640 1 --2- 172.21.15.111:0/1359465729 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5642c62e5bc0 0x5642c62e7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.282+0000 7f866fe00640 1 --2- 172.21.15.111:0/1359465729 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5642c62e5bc0 0x5642c62e7fb0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ae0d555d22b1745 server_cookie=12a14ebbedeb4b1b in_seq=0 out_seq=0 2024-09-15T23:08:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.282+0000 7f8656a00640 10 client.26711 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:08:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.282+0000 7f8656a00640 1 -- 172.21.15.111:0/1359465729 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5642c6242830 con 0x5642c62e5bc0 2024-09-15T23:08:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.282+0000 7f8656a00640 10 client.26711 handle_command_reply: tid=0 2024-09-15T23:08:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.282+0000 7f867a801740 1 client.26711 shutdown 2024-09-15T23:08:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.283+0000 7f867a801740 1 -- 172.21.15.111:0/1359465729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642c62d0260 msgr2=0x5642c62da3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.283+0000 7f867a801740 1 --2- 172.21.15.111:0/1359465729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642c62d0260 0x5642c62da3e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8664000c00 tx=0x7f8664004970 comp rx=0 tx=0).stop 2024-09-15T23:08:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.283+0000 7f867a801740 1 -- 172.21.15.111:0/1359465729 shutdown_connections 2024-09-15T23:08:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.283+0000 7f867a801740 1 --2- 172.21.15.111:0/1359465729 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5642c62e5bc0 0x5642c62e7fb0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.283+0000 7f867a801740 1 --2- 172.21.15.111:0/1359465729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642c62d5c30 0x5642c62e1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.283+0000 7f867a801740 1 --2- 172.21.15.111:0/1359465729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642c62d0ba0 0x5642c62da920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.283+0000 7f867a801740 1 --2- 172.21.15.111:0/1359465729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642c62d0260 0x5642c62da3e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.288 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.283+0000 7f8656a00640 0 client.26711 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:08:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.283+0000 7f867a801740 1 -- 172.21.15.111:0/1359465729 >> 172.21.15.111:0/1359465729 conn(0x5642c62cb8d0 msgr2=0x5642c62cd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:08:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.283+0000 7f867a801740 1 -- 172.21.15.111:0/1359465729 shutdown_connections 2024-09-15T23:08:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.283+0000 7f867a801740 1 -- 172.21.15.111:0/1359465729 wait complete. 2024-09-15T23:08:35.289 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.284+0000 7f867a801740 20 client.26711 trim_cache size 0 max 16384 2024-09-15T23:08:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.301+0000 7f867a801740 1 Processor -- start 2024-09-15T23:08:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.301+0000 7f867a801740 1 -- start start 2024-09-15T23:08:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.302+0000 7f867a801740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642c62d0260 0x5642c62d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:35.303 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.302+0000 7f867a801740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642c62d0ba0 0x5642c62da9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.302+0000 7f867a801740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642c62d5c30 0x5642c62e1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.302+0000 7f867a801740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f86701b1c20 con 0x5642c62d0260 2024-09-15T23:08:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.302+0000 7f867a801740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f86701af4f0 con 0x5642c62d0ba0 2024-09-15T23:08:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.302+0000 7f867a801740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f86701ad340 con 0x5642c62d5c30 2024-09-15T23:08:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.302+0000 7f866fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642c62d0ba0 0x5642c62da9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.302+0000 7f866fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642c62d0ba0 0x5642c62da9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:44714/0 (socket says 172.21.15.111:44714) 2024-09-15T23:08:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.302+0000 7f866fe00640 1 -- 172.21.15.111:0/3434357388 learned_addr learned my addr 172.21.15.111:0/3434357388 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:08:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.302+0000 7f8675800640 1 --2- 172.21.15.111:0/3434357388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642c62d5c30 0x5642c62e1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:35.304 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.302+0000 7f8674e00640 1 --2- 172.21.15.111:0/3434357388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642c62d0260 0x5642c62d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.303+0000 7f8675800640 1 -- 172.21.15.111:0/3434357388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642c62d0ba0 msgr2=0x5642c62da9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.303+0000 7f8675800640 1 --2- 172.21.15.111:0/3434357388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642c62d0ba0 0x5642c62da9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.303+0000 7f8675800640 1 -- 172.21.15.111:0/3434357388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642c62d0260 msgr2=0x5642c62d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.303+0000 7f8675800640 1 --2- 172.21.15.111:0/3434357388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642c62d0260 0x5642c62d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.303+0000 7f8675800640 1 -- 172.21.15.111:0/3434357388 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642c62e8320 con 0x5642c62d5c30 2024-09-15T23:08:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.303+0000 7f866fe00640 1 --2- 172.21.15.111:0/3434357388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642c62d0ba0 0x5642c62da9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:08:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.303+0000 7f8674e00640 1 --2- 172.21.15.111:0/3434357388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642c62d0260 0x5642c62d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:08:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.303+0000 7f8675800640 1 --2- 172.21.15.111:0/3434357388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642c62d5c30 0x5642c62e1920 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f86680cdf10 tx=0x7f86680cdd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9422fdc8ba30a2b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:08:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.303+0000 7f8661600640 1 -- 172.21.15.111:0/3434357388 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86680ba070 con 0x5642c62d5c30 2024-09-15T23:08:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.303+0000 7f8661600640 1 -- 172.21.15.111:0/3434357388 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86680a5650 con 0x5642c62d5c30 2024-09-15T23:08:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.304+0000 7f8661600640 1 -- 172.21.15.111:0/3434357388 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86680b4030 con 0x5642c62d5c30 2024-09-15T23:08:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.305+0000 7f867a801740 1 -- 172.21.15.111:0/3434357388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642c62d5c30 msgr2=0x5642c62e1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.305+0000 7f867a801740 1 --2- 172.21.15.111:0/3434357388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642c62d5c30 0x5642c62e1920 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f86680cdf10 tx=0x7f86680cdd90 comp rx=0 tx=0).stop 2024-09-15T23:08:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.305+0000 7f867a801740 1 -- 172.21.15.111:0/3434357388 shutdown_connections 2024-09-15T23:08:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.305+0000 7f867a801740 1 --2- 172.21.15.111:0/3434357388 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642c62d5c30 0x5642c62e1920 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.305+0000 7f867a801740 1 --2- 172.21.15.111:0/3434357388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642c62d0ba0 0x5642c62da9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.305+0000 7f867a801740 1 --2- 172.21.15.111:0/3434357388 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642c62d0260 0x5642c62d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.305+0000 7f867a801740 1 -- 172.21.15.111:0/3434357388 >> 172.21.15.111:0/3434357388 conn(0x5642c62cb8d0 msgr2=0x5642c62d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:08:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.305+0000 7f867a801740 1 -- 172.21.15.111:0/3434357388 shutdown_connections 2024-09-15T23:08:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.306+0000 7f867a801740 1 -- 172.21.15.111:0/3434357388 wait complete. 2024-09-15T23:08:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.306+0000 7f867a801740 1 -- start start 2024-09-15T23:08:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.307+0000 7f867a801740 1 Processor -- start 2024-09-15T23:08:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.307+0000 7f867a801740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642c62d0260 0x5642c62de920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.307+0000 7f867a801740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642c62d0ba0 0x5642c62dee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.307+0000 7f867a801740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642c62d5c30 0x5642c62dfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.307+0000 7f867a801740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f86701ad9b0 con 0x5642c62d5c30 2024-09-15T23:08:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.307+0000 7f867a801740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f86701aee20 con 0x5642c62d0260 2024-09-15T23:08:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.307+0000 7f867a801740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f86701ae080 con 0x5642c62d0ba0 2024-09-15T23:08:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.307+0000 7f867a801740 10 client.0 _ll_register_callbacks cb 0x7f8664009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:08:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.308+0000 7f866fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642c62d0ba0 0x5642c62dee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.308+0000 7f866fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642c62d0ba0 0x5642c62dee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:33142/0 (socket says 172.21.15.111:33142) 2024-09-15T23:08:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.308+0000 7f8674e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642c62d0260 0x5642c62de920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.308+0000 7f866fe00640 1 -- 172.21.15.111:0/2440785933 learned_addr learned my addr 172.21.15.111:0/2440785933 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:08:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.308+0000 7f866fe00640 1 -- 172.21.15.111:0/2440785933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642c62d0260 msgr2=0x5642c62de920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.308+0000 7f866fe00640 1 --2- 172.21.15.111:0/2440785933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642c62d0260 0x5642c62de920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.308+0000 7f866fe00640 1 -- 172.21.15.111:0/2440785933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642c62d5c30 msgr2=0x5642c62dfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.308+0000 7f866fe00640 1 --2- 172.21.15.111:0/2440785933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642c62d5c30 0x5642c62dfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.308+0000 7f866fe00640 1 -- 172.21.15.111:0/2440785933 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642c62e0480 con 0x5642c62d0ba0 2024-09-15T23:08:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.309+0000 7f866fe00640 1 --2- 172.21.15.111:0/2440785933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642c62d0ba0 0x5642c62dee60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f865c02aa00 tx=0x7f865c005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e2d1445aa69c9f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:08:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.309+0000 7f8656a00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:08:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.309+0000 7f8656a00640 1 -- 172.21.15.111:0/2440785933 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865c031df0 con 0x5642c62d0ba0 2024-09-15T23:08:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.309+0000 7f8656a00640 1 -- 172.21.15.111:0/2440785933 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f865c054480 con 0x5642c62d0ba0 2024-09-15T23:08:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.309+0000 7f867a801740 1 -- 172.21.15.111:0/2440785933 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86680a5650 con 0x5642c62d0ba0 2024-09-15T23:08:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.309+0000 7f8656a00640 1 -- 172.21.15.111:0/2440785933 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f865c054680 con 0x5642c62d0ba0 2024-09-15T23:08:35.314 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.310+0000 7f867a801740 10 client.26707 fetch_fsmap learned FSMap version 35 2024-09-15T23:08:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.310+0000 7f867a801740 1 -- 172.21.15.111:0/2440785933 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5642c62e1860 con 0x5642c62d0ba0 2024-09-15T23:08:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.310+0000 7f8656a00640 1 -- 172.21.15.111:0/2440785933 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86680a5650 con 0x5642c62d0ba0 2024-09-15T23:08:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.310+0000 7f8656a00640 1 -- 172.21.15.111:0/2440785933 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f865c044070 con 0x5642c62d0ba0 2024-09-15T23:08:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.310+0000 7f867a801740 10 client.26707 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:08:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.310+0000 7f867a801740 10 client.26707 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:08:35.315 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.310+0000 7f867a801740 20 client.26707 populate_metadata read hostname 'smithi111' 2024-09-15T23:08:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.311+0000 7f867a801740 1 --2- 172.21.15.111:0/2440785933 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5642c6350bb0 0x5642c6352fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.311+0000 7f867a801740 4 client.26707 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:08:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.311+0000 7f867a801740 1 -- 172.21.15.111:0/2440785933 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5642c623cb80 con 0x5642c6350bb0 2024-09-15T23:08:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.311+0000 7f8674e00640 1 --2- 172.21.15.111:0/2440785933 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5642c6350bb0 0x5642c6352fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.311+0000 7f8674e00640 1 --2- 172.21.15.111:0/2440785933 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5642c6350bb0 0x5642c6352fd0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e7a862702132a60 server_cookie=f2759a95441cfa8e in_seq=0 out_seq=0 2024-09-15T23:08:35.316 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.311+0000 7f8656a00640 10 client.26707 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:08:35.441 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.439+0000 7f8656a00640 1 -- 172.21.15.111:0/2440785933 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5642c623cb80 con 0x5642c6350bb0 2024-09-15T23:08:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.439+0000 7f8656a00640 10 client.26707 handle_command_reply: tid=0 2024-09-15T23:08:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.439+0000 7f867a801740 1 client.26707 shutdown 2024-09-15T23:08:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.440+0000 7f867a801740 1 -- 172.21.15.111:0/2440785933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642c62d0ba0 msgr2=0x5642c62dee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.440+0000 7f867a801740 1 --2- 172.21.15.111:0/2440785933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642c62d0ba0 0x5642c62dee60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f865c02aa00 tx=0x7f865c005e50 comp rx=0 tx=0).stop 2024-09-15T23:08:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.440+0000 7f867a801740 1 -- 172.21.15.111:0/2440785933 shutdown_connections 2024-09-15T23:08:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.440+0000 7f867a801740 1 --2- 172.21.15.111:0/2440785933 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5642c6350bb0 0x5642c6352fd0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.440+0000 7f867a801740 1 --2- 172.21.15.111:0/2440785933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5642c62d5c30 0x5642c62dfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.440+0000 7f867a801740 1 --2- 172.21.15.111:0/2440785933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5642c62d0ba0 0x5642c62dee60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.440+0000 7f8656a00640 0 client.26707 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:08:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.440+0000 7f867a801740 1 --2- 172.21.15.111:0/2440785933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5642c62d0260 0x5642c62de920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.440+0000 7f867a801740 1 -- 172.21.15.111:0/2440785933 >> 172.21.15.111:0/2440785933 conn(0x5642c62cb8d0 msgr2=0x5642c62ca1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:08:35.442 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.440+0000 7f867a801740 1 -- 172.21.15.111:0/2440785933 shutdown_connections 2024-09-15T23:08:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.440+0000 7f867a801740 1 -- 172.21.15.111:0/2440785933 wait complete. 2024-09-15T23:08:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.440+0000 7f867a801740 20 client.26707 trim_cache size 0 max 16384 2024-09-15T23:08:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:08:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:08:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:08:35.443 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:08:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.441+0000 7f8676200640 1 -- 172.21.15.111:0/1485611890 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f86580a3240 msgr2=0x7f86580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.441+0000 7f8676200640 1 --2- 172.21.15.111:0/1485611890 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f86580a3240 0x7f86580a5700 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f865c000c00 tx=0x7f865c0315b0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.441+0000 7f8676200640 1 -- 172.21.15.111:0/1485611890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86701076c0 msgr2=0x7f86701a5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.441+0000 7f8676200640 1 --2- 172.21.15.111:0/1485611890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86701076c0 0x7f86701a5930 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8668005fd0 tx=0x7f866800ef10 comp rx=0 tx=0).stop 2024-09-15T23:08:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.441+0000 7f8676200640 1 -- 172.21.15.111:0/1485611890 shutdown_connections 2024-09-15T23:08:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.441+0000 7f8676200640 1 --2- 172.21.15.111:0/1485611890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86701076c0 0x7f86701a5930 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.441+0000 7f8676200640 1 --2- 172.21.15.111:0/1485611890 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f86580a3240 0x7f86580a5700 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.441+0000 7f8676200640 1 --2- 172.21.15.111:0/1485611890 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8670104d90 0x7f8670196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.444 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.441+0000 7f8676200640 1 --2- 172.21.15.111:0/1485611890 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8670102460 0x7f8670195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.441+0000 7f8676200640 1 -- 172.21.15.111:0/1485611890 >> 172.21.15.111:0/1485611890 conn(0x7f86700fbfa0 msgr2=0x7f8670101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:08:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.441+0000 7f8676200640 1 -- 172.21.15.111:0/1485611890 shutdown_connections 2024-09-15T23:08:35.445 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:35.441+0000 7f8676200640 1 -- 172.21.15.111:0/1485611890 wait complete. 2024-09-15T23:08:35.684 INFO:tasks.workunit.client.0.smithi111.stdout: 10 176075 16.29 MB/sec execute 423 sec latency 90.823 ms 2024-09-15T23:08:36.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:36 smithi192 ceph-mon[35427]: pgmap v1031: 97 pgs: 97 active+clean; 541 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.8 MiB/s wr, 104 op/s 2024-09-15T23:08:36.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:36 smithi121 ceph-mon[34815]: pgmap v1031: 97 pgs: 97 active+clean; 541 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.8 MiB/s wr, 104 op/s 2024-09-15T23:08:36.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:36 smithi111 ceph-mon[30629]: pgmap v1031: 97 pgs: 97 active+clean; 541 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.8 MiB/s wr, 104 op/s 2024-09-15T23:08:36.684 INFO:tasks.workunit.client.0.smithi111.stdout: 10 176719 16.32 MB/sec execute 424 sec latency 72.177 ms 2024-09-15T23:08:37.684 INFO:tasks.workunit.client.0.smithi111.stdout: 10 177517 16.39 MB/sec execute 425 sec latency 61.550 ms 2024-09-15T23:08:38.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:38 smithi192 ceph-mon[35427]: pgmap v1032: 97 pgs: 97 active+clean; 542 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-09-15T23:08:38.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:38 smithi121 ceph-mon[34815]: pgmap v1032: 97 pgs: 97 active+clean; 542 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-09-15T23:08:38.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:38 smithi111 ceph-mon[30629]: pgmap v1032: 97 pgs: 97 active+clean; 542 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-09-15T23:08:38.685 INFO:tasks.workunit.client.0.smithi111.stdout: 10 178217 16.43 MB/sec execute 426 sec latency 58.633 ms 2024-09-15T23:08:39.685 INFO:tasks.workunit.client.0.smithi111.stdout: 10 178813 16.47 MB/sec execute 427 sec latency 210.762 ms 2024-09-15T23:08:40.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:40 smithi192 ceph-mon[35427]: pgmap v1033: 97 pgs: 97 active+clean; 541 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.4 MiB/s wr, 210 op/s 2024-09-15T23:08:40.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:40 smithi121 ceph-mon[34815]: pgmap v1033: 97 pgs: 97 active+clean; 541 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.4 MiB/s wr, 210 op/s 2024-09-15T23:08:40.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:40 smithi111 ceph-mon[30629]: pgmap v1033: 97 pgs: 97 active+clean; 541 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.4 MiB/s wr, 210 op/s 2024-09-15T23:08:40.685 INFO:tasks.workunit.client.0.smithi111.stdout: 10 179714 16.56 MB/sec execute 428 sec latency 63.834 ms 2024-09-15T23:08:41.685 INFO:tasks.workunit.client.0.smithi111.stdout: 10 180392 16.61 MB/sec execute 429 sec latency 46.086 ms 2024-09-15T23:08:42.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:42 smithi192 ceph-mon[35427]: pgmap v1034: 97 pgs: 97 active+clean; 550 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.2 MiB/s wr, 315 op/s 2024-09-15T23:08:42.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:42 smithi121 ceph-mon[34815]: pgmap v1034: 97 pgs: 97 active+clean; 550 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.2 MiB/s wr, 315 op/s 2024-09-15T23:08:42.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:42 smithi111 ceph-mon[30629]: pgmap v1034: 97 pgs: 97 active+clean; 550 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.2 MiB/s wr, 315 op/s 2024-09-15T23:08:42.686 INFO:tasks.workunit.client.0.smithi111.stdout: 10 181040 16.63 MB/sec execute 430 sec latency 74.590 ms 2024-09-15T23:08:43.687 INFO:tasks.workunit.client.0.smithi111.stdout: 10 181904 16.72 MB/sec execute 431 sec latency 56.790 ms 2024-09-15T23:08:44.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:44 smithi192 ceph-mon[35427]: pgmap v1035: 97 pgs: 97 active+clean; 551 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.6 MiB/s wr, 329 op/s 2024-09-15T23:08:44.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:44 smithi121 ceph-mon[34815]: pgmap v1035: 97 pgs: 97 active+clean; 551 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.6 MiB/s wr, 329 op/s 2024-09-15T23:08:44.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:44 smithi111 ceph-mon[30629]: pgmap v1035: 97 pgs: 97 active+clean; 551 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.6 MiB/s wr, 329 op/s 2024-09-15T23:08:44.687 INFO:tasks.workunit.client.0.smithi111.stdout: 10 182516 16.74 MB/sec execute 432 sec latency 69.936 ms 2024-09-15T23:08:45.687 INFO:tasks.workunit.client.0.smithi111.stdout: 10 183198 16.79 MB/sec execute 433 sec latency 76.144 ms 2024-09-15T23:08:46.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:46 smithi192 ceph-mon[35427]: pgmap v1036: 97 pgs: 97 active+clean; 556 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.3 MiB/s wr, 403 op/s 2024-09-15T23:08:46.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:46 smithi121 ceph-mon[34815]: pgmap v1036: 97 pgs: 97 active+clean; 556 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.3 MiB/s wr, 403 op/s 2024-09-15T23:08:46.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:46 smithi111 ceph-mon[30629]: pgmap v1036: 97 pgs: 97 active+clean; 556 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.3 MiB/s wr, 403 op/s 2024-09-15T23:08:46.687 INFO:tasks.workunit.client.0.smithi111.stdout: 10 184011 16.86 MB/sec execute 434 sec latency 54.832 ms 2024-09-15T23:08:47.687 INFO:tasks.workunit.client.0.smithi111.stdout: 10 184693 16.90 MB/sec execute 435 sec latency 56.362 ms 2024-09-15T23:08:48.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:48 smithi192 ceph-mon[35427]: pgmap v1037: 97 pgs: 97 active+clean; 566 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 4.4 MiB/s wr, 526 op/s 2024-09-15T23:08:48.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:48 smithi121 ceph-mon[34815]: pgmap v1037: 97 pgs: 97 active+clean; 566 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 4.4 MiB/s wr, 526 op/s 2024-09-15T23:08:48.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:48 smithi111 ceph-mon[30629]: pgmap v1037: 97 pgs: 97 active+clean; 566 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 4.4 MiB/s wr, 526 op/s 2024-09-15T23:08:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.587+0000 7f8beae00640 1 -- 172.21.15.111:0/1303772513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8be4077df0 msgr2=0x7f8be4078250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.587+0000 7f8beae00640 1 --2- 172.21.15.111:0/1303772513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8be4077df0 0x7f8be4078250 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009900 tx=0x7f8bd002a3f0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.587+0000 7f8beae00640 1 -- 172.21.15.111:0/1303772513 shutdown_connections 2024-09-15T23:08:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.587+0000 7f8beae00640 1 --2- 172.21.15.111:0/1303772513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8be4078790 0x7f8be4079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.587+0000 7f8beae00640 1 --2- 172.21.15.111:0/1303772513 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8be4077df0 0x7f8be4078250 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.587+0000 7f8beae00640 1 --2- 172.21.15.111:0/1303772513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8be4076bb0 0x7f8be4076fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.587+0000 7f8beae00640 1 -- 172.21.15.111:0/1303772513 >> 172.21.15.111:0/1303772513 conn(0x7f8be40ff4e0 msgr2=0x7f8be4101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:08:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.587+0000 7f8beae00640 1 -- 172.21.15.111:0/1303772513 shutdown_connections 2024-09-15T23:08:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.587+0000 7f8beae00640 1 -- 172.21.15.111:0/1303772513 wait complete. 2024-09-15T23:08:48.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.588+0000 7f8beae00640 1 Processor -- start 2024-09-15T23:08:48.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.588+0000 7f8beae00640 1 -- start start 2024-09-15T23:08:48.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.589+0000 7f8beae00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8be4076bb0 0x7f8be419e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:48.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.589+0000 7f8beae00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8be4077df0 0x7f8be419e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:48.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.589+0000 7f8beae00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8be4078790 0x7f8be41a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.589+0000 7f8beae00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8be4113e50 con 0x7f8be4077df0 2024-09-15T23:08:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.589+0000 7f8beae00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8be4113cd0 con 0x7f8be4078790 2024-09-15T23:08:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.589+0000 7f8beae00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8be4113fd0 con 0x7f8be4076bb0 2024-09-15T23:08:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.589+0000 7f8be3e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8be4076bb0 0x7f8be419e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.589+0000 7f8be3e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8be4076bb0 0x7f8be419e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:57978/0 (socket says 172.21.15.111:57978) 2024-09-15T23:08:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.589+0000 7f8be3e00640 1 -- 172.21.15.111:0/2000432477 learned_addr learned my addr 172.21.15.111:0/2000432477 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:08:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.589+0000 7f8be3400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8be4077df0 0x7f8be419e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.589+0000 7f8be8e00640 1 --2- 172.21.15.111:0/2000432477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8be4078790 0x7f8be41a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.589+0000 7f8be3e00640 1 -- 172.21.15.111:0/2000432477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8be4078790 msgr2=0x7f8be41a5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.589+0000 7f8be3e00640 1 --2- 172.21.15.111:0/2000432477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8be4078790 0x7f8be41a5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.589+0000 7f8be3e00640 1 -- 172.21.15.111:0/2000432477 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8be4077df0 msgr2=0x7f8be419e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.589+0000 7f8be3e00640 1 --2- 172.21.15.111:0/2000432477 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8be4077df0 0x7f8be419e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.589+0000 7f8be3e00640 1 -- 172.21.15.111:0/2000432477 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be41a60a0 con 0x7f8be4076bb0 2024-09-15T23:08:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.590+0000 7f8be3e00640 1 --2- 172.21.15.111:0/2000432477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8be4076bb0 0x7f8be419e480 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400e9f0 tx=0x7f8bd400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:08:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.590+0000 7f8be0c00640 1 -- 172.21.15.111:0/2000432477 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd400cd80 con 0x7f8be4076bb0 2024-09-15T23:08:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.590+0000 7f8be0c00640 1 -- 172.21.15.111:0/2000432477 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bd4010910 con 0x7f8be4076bb0 2024-09-15T23:08:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.590+0000 7f8beae00640 1 -- 172.21.15.111:0/2000432477 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be41a6390 con 0x7f8be4076bb0 2024-09-15T23:08:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.590+0000 7f8be0c00640 1 -- 172.21.15.111:0/2000432477 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4018990 con 0x7f8be4076bb0 2024-09-15T23:08:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.590+0000 7f8beae00640 1 -- 172.21.15.111:0/2000432477 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be41a6930 con 0x7f8be4076bb0 2024-09-15T23:08:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.594+0000 7f8beddd7740 1 Processor -- start 2024-09-15T23:08:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.594+0000 7f8be0c00640 1 -- 172.21.15.111:0/2000432477 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f8bd4010ac0 con 0x7f8be4076bb0 2024-09-15T23:08:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.594+0000 7f8beddd7740 1 -- start start 2024-09-15T23:08:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.594+0000 7f8beddd7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3aa99f090 0x55a3aa99f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.594+0000 7f8beddd7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3aa99f9d0 0x55a3aa9a4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.594+0000 7f8beddd7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3aa9a4a60 0x55a3aa9a6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.594+0000 7f8beddd7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a3aa904100 con 0x55a3aa99f9d0 2024-09-15T23:08:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.594+0000 7f8beddd7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a3aa8ac1b0 con 0x55a3aa99f090 2024-09-15T23:08:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.594+0000 7f8beddd7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a3aa9000c0 con 0x55a3aa9a4a60 2024-09-15T23:08:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.594+0000 7f8be8e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3aa99f090 0x55a3aa99f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.594+0000 7f8be3400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3aa9a4a60 0x55a3aa9a6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.594+0000 7f8be8e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3aa99f090 0x55a3aa99f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:42444/0 (socket says 172.21.15.111:42444) 2024-09-15T23:08:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.594+0000 7f8be8e00640 1 -- 172.21.15.111:0/3599221366 learned_addr learned my addr 172.21.15.111:0/3599221366 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:08:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.595+0000 7f8be3e00640 1 --2- 172.21.15.111:0/3599221366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3aa99f9d0 0x55a3aa9a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.595+0000 7f8be3400640 1 -- 172.21.15.111:0/3599221366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3aa99f090 msgr2=0x55a3aa99f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.595+0000 7f8be3400640 1 --2- 172.21.15.111:0/3599221366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3aa99f090 0x55a3aa99f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.595+0000 7f8be3400640 1 -- 172.21.15.111:0/3599221366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3aa99f9d0 msgr2=0x55a3aa9a4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.595+0000 7f8be3400640 1 --2- 172.21.15.111:0/3599221366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3aa99f9d0 0x55a3aa9a4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.595+0000 7f8be3400640 1 -- 172.21.15.111:0/3599221366 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3aa617210 con 0x55a3aa9a4a60 2024-09-15T23:08:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.595+0000 7f8be3e00640 1 --2- 172.21.15.111:0/3599221366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3aa99f9d0 0x55a3aa9a4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:08:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.595+0000 7f8be3400640 1 --2- 172.21.15.111:0/3599221366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3aa9a4a60 0x55a3aa9a6f20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0031440 tx=0x7f8bd0005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fe66c55220d80c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:08:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.595+0000 7f8bd9600640 1 -- 172.21.15.111:0/3599221366 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0040070 con 0x55a3aa9a4a60 2024-09-15T23:08:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.595+0000 7f8be0c00640 1 --2- 172.21.15.111:0/2000432477 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8bc80a32f0 0x7f8bc80a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.595+0000 7f8bd9600640 1 -- 172.21.15.111:0/3599221366 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bd00318b0 con 0x55a3aa9a4a60 2024-09-15T23:08:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.595+0000 7f8bd9600640 1 -- 172.21.15.111:0/3599221366 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0035040 con 0x55a3aa9a4a60 2024-09-15T23:08:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.596+0000 7f8be0c00640 1 -- 172.21.15.111:0/2000432477 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f8bd40cfc20 con 0x7f8be4076bb0 2024-09-15T23:08:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.596+0000 7f8be8e00640 1 --2- 172.21.15.111:0/2000432477 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8bc80a32f0 0x7f8bc80a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.596+0000 7f8beddd7740 1 -- 172.21.15.111:0/3599221366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3aa9a4a60 msgr2=0x55a3aa9a6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.596+0000 7f8beddd7740 1 --2- 172.21.15.111:0/3599221366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3aa9a4a60 0x55a3aa9a6f20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0031440 tx=0x7f8bd0005fd0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.596+0000 7f8beddd7740 1 -- 172.21.15.111:0/3599221366 shutdown_connections 2024-09-15T23:08:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.596+0000 7f8beddd7740 1 --2- 172.21.15.111:0/3599221366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3aa9a4a60 0x55a3aa9a6f20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.596+0000 7f8beddd7740 1 --2- 172.21.15.111:0/3599221366 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3aa99f9d0 0x55a3aa9a4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.596+0000 7f8beddd7740 1 --2- 172.21.15.111:0/3599221366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3aa99f090 0x55a3aa99f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.596+0000 7f8beddd7740 1 -- 172.21.15.111:0/3599221366 >> 172.21.15.111:0/3599221366 conn(0x55a3aa99a700 msgr2=0x55a3aa99ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:08:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.596+0000 7f8be8e00640 1 --2- 172.21.15.111:0/2000432477 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8bc80a32f0 0x7f8bc80a57b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00a510 tx=0x7f8bdc00d040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:08:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.596+0000 7f8beddd7740 1 -- 172.21.15.111:0/3599221366 shutdown_connections 2024-09-15T23:08:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.596+0000 7f8beddd7740 1 -- 172.21.15.111:0/3599221366 wait complete. 2024-09-15T23:08:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.597+0000 7f8beddd7740 1 -- start start 2024-09-15T23:08:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.597+0000 7f8beddd7740 1 Processor -- start 2024-09-15T23:08:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.597+0000 7f8beddd7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3aa99f090 0x55a3aa9a9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.597+0000 7f8beddd7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3aa99f9d0 0x55a3aa9a9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.597+0000 7f8beddd7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3aa9a4a60 0x55a3aa9b0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.597+0000 7f8beddd7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55a3aa9a79d0 con 0x55a3aa99f9d0 2024-09-15T23:08:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.597+0000 7f8beddd7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a3aa88d090 con 0x55a3aa99f090 2024-09-15T23:08:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.597+0000 7f8beddd7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55a3aa9a7b50 con 0x55a3aa9a4a60 2024-09-15T23:08:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.597+0000 7f8beddd7740 10 client.0 _ll_register_callbacks cb 0x55a3aa8e1f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:08:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.598+0000 7f8be8e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3aa9a4a60 0x55a3aa9b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.598+0000 7f8be3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3aa99f090 0x55a3aa9a9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.598+0000 7f8be8e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3aa9a4a60 0x55a3aa9b0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:58010/0 (socket says 172.21.15.111:58010) 2024-09-15T23:08:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.598+0000 7f8be3400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3aa99f9d0 0x55a3aa9a9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.598+0000 7f8be3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3aa99f090 0x55a3aa9a9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:42446/0 (socket says 172.21.15.111:42446) 2024-09-15T23:08:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.598+0000 7f8be8e00640 1 -- 172.21.15.111:0/276551370 learned_addr learned my addr 172.21.15.111:0/276551370 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:08:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.598+0000 7f8be3e00640 1 -- 172.21.15.111:0/276551370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3aa9a4a60 msgr2=0x55a3aa9b0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.598+0000 7f8be3e00640 1 --2- 172.21.15.111:0/276551370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3aa9a4a60 0x55a3aa9b0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.598+0000 7f8be3e00640 1 -- 172.21.15.111:0/276551370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3aa99f9d0 msgr2=0x55a3aa9a9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.598+0000 7f8be3e00640 1 --2- 172.21.15.111:0/276551370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3aa99f9d0 0x55a3aa9a9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.598+0000 7f8be3e00640 1 -- 172.21.15.111:0/276551370 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3aa6e3620 con 0x55a3aa99f090 2024-09-15T23:08:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.598+0000 7f8be8e00640 1 --2- 172.21.15.111:0/276551370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3aa9a4a60 0x55a3aa9b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:08:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.598+0000 7f8be3400640 1 --2- 172.21.15.111:0/276551370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3aa99f9d0 0x55a3aa9a9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:08:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.598+0000 7f8be3e00640 1 --2- 172.21.15.111:0/276551370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3aa99f090 0x55a3aa9a9210 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x55a3aa9a0400 tx=0x7f8bd409c1b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d42a107a5a9add07 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:08:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.598+0000 7f8bce000640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:08:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.598+0000 7f8bce000640 1 -- 172.21.15.111:0/276551370 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4096e60 con 0x55a3aa99f090 2024-09-15T23:08:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.599+0000 7f8beddd7740 1 -- 172.21.15.111:0/276551370 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bd00318b0 con 0x55a3aa99f090 2024-09-15T23:08:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.599+0000 7f8bce000640 1 -- 172.21.15.111:0/276551370 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f8bd40bc2b0 con 0x55a3aa99f090 2024-09-15T23:08:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.599+0000 7f8bce000640 1 -- 172.21.15.111:0/276551370 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8bd409c2f0 con 0x55a3aa99f090 2024-09-15T23:08:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.599+0000 7f8beddd7740 10 client.26726 fetch_fsmap learned FSMap version 35 2024-09-15T23:08:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.599+0000 7f8beddd7740 1 -- 172.21.15.111:0/276551370 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a3aa9b3640 con 0x55a3aa99f090 2024-09-15T23:08:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.599+0000 7f8bce000640 1 -- 172.21.15.111:0/276551370 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bd00318b0 con 0x55a3aa99f090 2024-09-15T23:08:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.599+0000 7f8bce000640 1 -- 172.21.15.111:0/276551370 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f8bd40a0130 con 0x55a3aa99f090 2024-09-15T23:08:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.599+0000 7f8beddd7740 10 client.26726 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:08:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.599+0000 7f8beddd7740 10 client.26726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:08:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.599+0000 7f8beddd7740 20 client.26726 populate_metadata read hostname 'smithi111' 2024-09-15T23:08:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.599+0000 7f8beddd7740 1 --2- 172.21.15.111:0/276551370 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a3aa9b4cc0 0x55a3aa9b70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.599+0000 7f8beddd7740 4 client.26726 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:08:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.599+0000 7f8beddd7740 1 -- 172.21.15.111:0/276551370 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3aa911830 con 0x55a3aa9b4cc0 2024-09-15T23:08:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.600+0000 7f8be3400640 1 --2- 172.21.15.111:0/276551370 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a3aa9b4cc0 0x55a3aa9b70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.600+0000 7f8be3400640 1 --2- 172.21.15.111:0/276551370 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a3aa9b4cc0 0x55a3aa9b70b0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f7918187056d8b3 server_cookie=f614f6715c2f1fdd in_seq=0 out_seq=0 2024-09-15T23:08:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.600+0000 7f8bce000640 10 client.26726 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:08:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.601+0000 7f8bce000640 1 -- 172.21.15.111:0/276551370 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a3aa911830 con 0x55a3aa9b4cc0 2024-09-15T23:08:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.601+0000 7f8bce000640 10 client.26726 handle_command_reply: tid=0 2024-09-15T23:08:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.601+0000 7f8beddd7740 1 client.26726 shutdown 2024-09-15T23:08:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.601+0000 7f8beddd7740 1 -- 172.21.15.111:0/276551370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3aa99f090 msgr2=0x55a3aa9a9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.601+0000 7f8beddd7740 1 --2- 172.21.15.111:0/276551370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3aa99f090 0x55a3aa9a9210 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x55a3aa9a0400 tx=0x7f8bd409c1b0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.601+0000 7f8beddd7740 1 -- 172.21.15.111:0/276551370 shutdown_connections 2024-09-15T23:08:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.601+0000 7f8beddd7740 1 --2- 172.21.15.111:0/276551370 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a3aa9b4cc0 0x55a3aa9b70b0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.601+0000 7f8beddd7740 1 --2- 172.21.15.111:0/276551370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3aa9a4a60 0x55a3aa9b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.601+0000 7f8beddd7740 1 --2- 172.21.15.111:0/276551370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3aa99f9d0 0x55a3aa9a9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.601+0000 7f8bce000640 0 client.26726 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:08:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.601+0000 7f8beddd7740 1 --2- 172.21.15.111:0/276551370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3aa99f090 0x55a3aa9a9210 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.601+0000 7f8beddd7740 1 -- 172.21.15.111:0/276551370 >> 172.21.15.111:0/276551370 conn(0x55a3aa99a700 msgr2=0x55a3aa99bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:08:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.602+0000 7f8beddd7740 1 -- 172.21.15.111:0/276551370 shutdown_connections 2024-09-15T23:08:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.602+0000 7f8beddd7740 1 -- 172.21.15.111:0/276551370 wait complete. 2024-09-15T23:08:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.602+0000 7f8beddd7740 20 client.26726 trim_cache size 0 max 16384 2024-09-15T23:08:48.620 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.619+0000 7f8beddd7740 1 Processor -- start 2024-09-15T23:08:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.619+0000 7f8beddd7740 1 -- start start 2024-09-15T23:08:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.619+0000 7f8beddd7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3aa99f090 0x55a3aa9a97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.619+0000 7f8beddd7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3aa99f9d0 0x55a3aa9b0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.619+0000 7f8beddd7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3aa9a4a60 0x55a3aa999770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.619+0000 7f8beddd7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8be41b1dc0 con 0x55a3aa99f9d0 2024-09-15T23:08:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.619+0000 7f8beddd7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8be41af690 con 0x55a3aa9a4a60 2024-09-15T23:08:48.621 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.619+0000 7f8beddd7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8be41ad4e0 con 0x55a3aa99f090 2024-09-15T23:08:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.619+0000 7f8be3e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3aa99f090 0x55a3aa9a97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.620+0000 7f8be3e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3aa99f090 0x55a3aa9a97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:58022/0 (socket says 172.21.15.111:58022) 2024-09-15T23:08:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.620+0000 7f8be8e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3aa9a4a60 0x55a3aa999770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.620+0000 7f8be3e00640 1 -- 172.21.15.111:0/1499476201 learned_addr learned my addr 172.21.15.111:0/1499476201 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:08:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.620+0000 7f8be3400640 1 --2- 172.21.15.111:0/1499476201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3aa99f9d0 0x55a3aa9b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.620+0000 7f8be3e00640 1 -- 172.21.15.111:0/1499476201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3aa9a4a60 msgr2=0x55a3aa999770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.620+0000 7f8be3e00640 1 --2- 172.21.15.111:0/1499476201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3aa9a4a60 0x55a3aa999770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.620+0000 7f8be3e00640 1 -- 172.21.15.111:0/1499476201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3aa99f9d0 msgr2=0x55a3aa9b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.620+0000 7f8be3e00640 1 --2- 172.21.15.111:0/1499476201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3aa99f9d0 0x55a3aa9b0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.620+0000 7f8be3e00640 1 -- 172.21.15.111:0/1499476201 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3aa9b7420 con 0x55a3aa99f090 2024-09-15T23:08:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.620+0000 7f8be3e00640 1 --2- 172.21.15.111:0/1499476201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3aa99f090 0x55a3aa9a97e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8bd409bff0 tx=0x7f8bd400e940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=165b06d38ace8c38 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:08:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.620+0000 7f8bd8c00640 1 -- 172.21.15.111:0/1499476201 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd409bc50 con 0x55a3aa99f090 2024-09-15T23:08:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.620+0000 7f8bd8c00640 1 -- 172.21.15.111:0/1499476201 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bd40a4eb0 con 0x55a3aa99f090 2024-09-15T23:08:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.620+0000 7f8bd8c00640 1 -- 172.21.15.111:0/1499476201 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4018b30 con 0x55a3aa99f090 2024-09-15T23:08:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.621+0000 7f8beddd7740 1 -- 172.21.15.111:0/1499476201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3aa99f090 msgr2=0x55a3aa9a97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.621+0000 7f8beddd7740 1 --2- 172.21.15.111:0/1499476201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3aa99f090 0x55a3aa9a97e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8bd409bff0 tx=0x7f8bd400e940 comp rx=0 tx=0).stop 2024-09-15T23:08:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.621+0000 7f8beddd7740 1 -- 172.21.15.111:0/1499476201 shutdown_connections 2024-09-15T23:08:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.622+0000 7f8beddd7740 1 --2- 172.21.15.111:0/1499476201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3aa9a4a60 0x55a3aa999770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.622+0000 7f8beddd7740 1 --2- 172.21.15.111:0/1499476201 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3aa99f9d0 0x55a3aa9b0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.622+0000 7f8beddd7740 1 --2- 172.21.15.111:0/1499476201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3aa99f090 0x55a3aa9a97e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.622+0000 7f8beddd7740 1 -- 172.21.15.111:0/1499476201 >> 172.21.15.111:0/1499476201 conn(0x55a3aa99a700 msgr2=0x55a3aa999060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:08:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.622+0000 7f8beddd7740 1 -- 172.21.15.111:0/1499476201 shutdown_connections 2024-09-15T23:08:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.622+0000 7f8beddd7740 1 -- 172.21.15.111:0/1499476201 wait complete. 2024-09-15T23:08:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.622+0000 7f8beddd7740 1 -- start start 2024-09-15T23:08:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.622+0000 7f8beddd7740 1 Processor -- start 2024-09-15T23:08:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.623+0000 7f8beddd7740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3aa99f090 0x55a3aa9adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.623+0000 7f8beddd7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3aa99f9d0 0x55a3aa9ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.623+0000 7f8beddd7740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3aa9a4a60 0x55a3aa9af100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.623+0000 7f8beddd7740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f8be41adb50 con 0x55a3aa99f090 2024-09-15T23:08:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.623+0000 7f8beddd7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8be41aefc0 con 0x55a3aa99f9d0 2024-09-15T23:08:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.623+0000 7f8beddd7740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8be41ae220 con 0x55a3aa9a4a60 2024-09-15T23:08:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.623+0000 7f8beddd7740 10 client.0 _ll_register_callbacks cb 0x7f8bd400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:08:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.623+0000 7f8be8e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3aa9a4a60 0x55a3aa9af100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.623+0000 7f8be3e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3aa99f090 0x55a3aa9adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.623+0000 7f8be8e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3aa9a4a60 0x55a3aa9af100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:58032/0 (socket says 172.21.15.111:58032) 2024-09-15T23:08:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.623+0000 7f8be8e00640 1 -- 172.21.15.111:0/1436088597 learned_addr learned my addr 172.21.15.111:0/1436088597 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:08:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.623+0000 7f8be3400640 1 --2- 172.21.15.111:0/1436088597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3aa99f9d0 0x55a3aa9ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.623+0000 7f8be8e00640 1 -- 172.21.15.111:0/1436088597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3aa99f9d0 msgr2=0x55a3aa9ae0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.623+0000 7f8be8e00640 1 --2- 172.21.15.111:0/1436088597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3aa99f9d0 0x55a3aa9ae0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.627 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.623+0000 7f8be8e00640 1 -- 172.21.15.111:0/1436088597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3aa99f090 msgr2=0x55a3aa9adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:08:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.623+0000 7f8be8e00640 1 --2- 172.21.15.111:0/1436088597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3aa99f090 0x55a3aa9adbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.623+0000 7f8be8e00640 1 -- 172.21.15.111:0/1436088597 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3aa9af710 con 0x55a3aa9a4a60 2024-09-15T23:08:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.624+0000 7f8be3400640 1 --2- 172.21.15.111:0/1436088597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3aa99f9d0 0x55a3aa9ae0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:08:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.624+0000 7f8be8e00640 1 --2- 172.21.15.111:0/1436088597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3aa9a4a60 0x55a3aa9af100 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc009c90 tx=0x7f8bdc004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc8f4248d05752b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:08:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.624+0000 7f8bce000640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:08:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.624+0000 7f8bce000640 1 -- 172.21.15.111:0/1436088597 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc028070 con 0x55a3aa9a4a60 2024-09-15T23:08:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.624+0000 7f8bce000640 1 -- 172.21.15.111:0/1436088597 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f8bdc017890 con 0x55a3aa9a4a60 2024-09-15T23:08:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.624+0000 7f8beddd7740 1 -- 172.21.15.111:0/1436088597 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bd40a4eb0 con 0x55a3aa9a4a60 2024-09-15T23:08:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.624+0000 7f8bce000640 1 -- 172.21.15.111:0/1436088597 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8bdc034050 con 0x55a3aa9a4a60 2024-09-15T23:08:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.625+0000 7f8beddd7740 10 client.26725 fetch_fsmap learned FSMap version 35 2024-09-15T23:08:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.625+0000 7f8beddd7740 1 -- 172.21.15.111:0/1436088597 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a3aa9b0af0 con 0x55a3aa9a4a60 2024-09-15T23:08:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.625+0000 7f8bce000640 1 -- 172.21.15.111:0/1436088597 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bd40a4eb0 con 0x55a3aa9a4a60 2024-09-15T23:08:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.625+0000 7f8bce000640 1 -- 172.21.15.111:0/1436088597 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f8bdc02c3b0 con 0x55a3aa9a4a60 2024-09-15T23:08:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.625+0000 7f8beddd7740 10 client.26725 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:08:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.625+0000 7f8beddd7740 10 client.26725 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:08:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.625+0000 7f8beddd7740 20 client.26725 populate_metadata read hostname 'smithi111' 2024-09-15T23:08:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.625+0000 7f8beddd7740 1 --2- 172.21.15.111:0/1436088597 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a3aaa201a0 0x55a3aaa225c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:08:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.625+0000 7f8beddd7740 4 client.26725 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:08:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.625+0000 7f8beddd7740 1 -- 172.21.15.111:0/1436088597 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a3aa9a47e0 con 0x55a3aaa201a0 2024-09-15T23:08:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.625+0000 7f8be3e00640 1 --2- 172.21.15.111:0/1436088597 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a3aaa201a0 0x55a3aaa225c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:08:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.626+0000 7f8be3e00640 1 --2- 172.21.15.111:0/1436088597 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a3aaa201a0 0x55a3aaa225c0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58caf99608654909 server_cookie=365eb5fdae82507e in_seq=0 out_seq=0 2024-09-15T23:08:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.626+0000 7f8bce000640 10 client.26725 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:08:48.688 INFO:tasks.workunit.client.0.smithi111.stdout: 10 185351 16.94 MB/sec execute 436 sec latency 107.117 ms 2024-09-15T23:08:48.962 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.960+0000 7f8bce000640 1 -- 172.21.15.111:0/1436088597 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a3aa9a47e0 con 0x55a3aaa201a0 2024-09-15T23:08:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.960+0000 7f8bce000640 10 client.26725 handle_command_reply: tid=0 2024-09-15T23:08:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.961+0000 7f8beddd7740 1 client.26725 shutdown 2024-09-15T23:08:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.961+0000 7f8beddd7740 1 -- 172.21.15.111:0/1436088597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3aa9a4a60 msgr2=0x55a3aa9af100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.961+0000 7f8beddd7740 1 --2- 172.21.15.111:0/1436088597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3aa9a4a60 0x55a3aa9af100 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc009c90 tx=0x7f8bdc004a40 comp rx=0 tx=0).stop 2024-09-15T23:08:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.961+0000 7f8beddd7740 1 -- 172.21.15.111:0/1436088597 shutdown_connections 2024-09-15T23:08:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.961+0000 7f8beddd7740 1 --2- 172.21.15.111:0/1436088597 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55a3aaa201a0 0x55a3aaa225c0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.961+0000 7f8beddd7740 1 --2- 172.21.15.111:0/1436088597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55a3aa9a4a60 0x55a3aa9af100 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.961+0000 7f8beddd7740 1 --2- 172.21.15.111:0/1436088597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a3aa99f9d0 0x55a3aa9ae0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.961+0000 7f8beddd7740 1 --2- 172.21.15.111:0/1436088597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55a3aa99f090 0x55a3aa9adbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.961+0000 7f8beddd7740 1 -- 172.21.15.111:0/1436088597 >> 172.21.15.111:0/1436088597 conn(0x55a3aa99a700 msgr2=0x55a3aa99bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:08:48.963 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.961+0000 7f8bce000640 0 client.26725 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:08:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.961+0000 7f8beddd7740 1 -- 172.21.15.111:0/1436088597 shutdown_connections 2024-09-15T23:08:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.961+0000 7f8beddd7740 1 -- 172.21.15.111:0/1436088597 wait complete. 2024-09-15T23:08:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.961+0000 7f8beddd7740 20 client.26725 trim_cache size 0 max 16384 2024-09-15T23:08:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:08:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:08:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:08:48.964 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:08:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.962+0000 7f8beae00640 1 -- 172.21.15.111:0/2000432477 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8bc80a32f0 msgr2=0x7f8bc80a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.962+0000 7f8beae00640 1 --2- 172.21.15.111:0/2000432477 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8bc80a32f0 0x7f8bc80a57b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00a510 tx=0x7f8bdc00d040 comp rx=0 tx=0).stop 2024-09-15T23:08:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.962+0000 7f8beae00640 1 -- 172.21.15.111:0/2000432477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8be4076bb0 msgr2=0x7f8be419e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:08:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.962+0000 7f8beae00640 1 --2- 172.21.15.111:0/2000432477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8be4076bb0 0x7f8be419e480 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400e9f0 tx=0x7f8bd400eeb0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.962+0000 7f8beae00640 1 -- 172.21.15.111:0/2000432477 shutdown_connections 2024-09-15T23:08:48.965 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.962+0000 7f8beae00640 1 --2- 172.21.15.111:0/2000432477 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f8bc80a32f0 0x7f8bc80a57b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.962+0000 7f8beae00640 1 --2- 172.21.15.111:0/2000432477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8be4078790 0x7f8be41a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.962+0000 7f8beae00640 1 --2- 172.21.15.111:0/2000432477 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8be4077df0 0x7f8be419e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.963+0000 7f8beae00640 1 --2- 172.21.15.111:0/2000432477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8be4076bb0 0x7f8be419e480 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:08:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.963+0000 7f8beae00640 1 -- 172.21.15.111:0/2000432477 >> 172.21.15.111:0/2000432477 conn(0x7f8be40ff4e0 msgr2=0x7f8be4100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:08:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.963+0000 7f8beae00640 1 -- 172.21.15.111:0/2000432477 shutdown_connections 2024-09-15T23:08:48.966 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:08:48.963+0000 7f8beae00640 1 -- 172.21.15.111:0/2000432477 wait complete. 2024-09-15T23:08:49.688 INFO:tasks.workunit.client.0.smithi111.stdout: 10 185896 16.98 MB/sec execute 437 sec latency 208.394 ms 2024-09-15T23:08:50.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:50 smithi111 ceph-mon[30629]: pgmap v1038: 97 pgs: 97 active+clean; 574 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 4.9 MiB/s wr, 580 op/s 2024-09-15T23:08:50.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:50 smithi192 ceph-mon[35427]: pgmap v1038: 97 pgs: 97 active+clean; 574 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 4.9 MiB/s wr, 580 op/s 2024-09-15T23:08:50.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:50 smithi121 ceph-mon[34815]: pgmap v1038: 97 pgs: 97 active+clean; 574 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 4.9 MiB/s wr, 580 op/s 2024-09-15T23:08:50.688 INFO:tasks.workunit.client.0.smithi111.stdout: 10 186516 17.02 MB/sec execute 438 sec latency 82.522 ms 2024-09-15T23:08:51.688 INFO:tasks.workunit.client.0.smithi111.stdout: 10 187191 17.05 MB/sec execute 439 sec latency 95.330 ms 2024-09-15T23:08:52.302 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:52 smithi111 ceph-mon[30629]: pgmap v1039: 97 pgs: 97 active+clean; 571 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.7 MiB/s wr, 578 op/s 2024-09-15T23:08:52.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:52 smithi192 ceph-mon[35427]: pgmap v1039: 97 pgs: 97 active+clean; 571 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.7 MiB/s wr, 578 op/s 2024-09-15T23:08:52.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:52 smithi121 ceph-mon[34815]: pgmap v1039: 97 pgs: 97 active+clean; 571 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.7 MiB/s wr, 578 op/s 2024-09-15T23:08:52.688 INFO:tasks.workunit.client.0.smithi111.stdout: 10 187934 17.10 MB/sec execute 440 sec latency 42.046 ms 2024-09-15T23:08:53.688 INFO:tasks.workunit.client.0.smithi111.stdout: 10 188782 17.17 MB/sec execute 441 sec latency 49.797 ms 2024-09-15T23:08:54.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:54 smithi192 ceph-mon[35427]: pgmap v1040: 97 pgs: 97 active+clean; 565 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.8 MiB/s wr, 501 op/s 2024-09-15T23:08:54.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:54 smithi121 ceph-mon[34815]: pgmap v1040: 97 pgs: 97 active+clean; 565 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.8 MiB/s wr, 501 op/s 2024-09-15T23:08:54.399 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:54 smithi111 ceph-mon[30629]: pgmap v1040: 97 pgs: 97 active+clean; 565 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.8 MiB/s wr, 501 op/s 2024-09-15T23:08:54.688 INFO:tasks.workunit.client.0.smithi111.stdout: 10 189411 17.21 MB/sec execute 442 sec latency 148.163 ms 2024-09-15T23:08:55.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:55 smithi121 ceph-mon[34815]: pgmap v1041: 97 pgs: 97 active+clean; 559 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 4.6 MiB/s wr, 626 op/s 2024-09-15T23:08:55.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:55 smithi111 ceph-mon[30629]: pgmap v1041: 97 pgs: 97 active+clean; 559 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 4.6 MiB/s wr, 626 op/s 2024-09-15T23:08:55.688 INFO:tasks.workunit.client.0.smithi111.stdout: 10 189933 17.24 MB/sec execute 443 sec latency 172.033 ms 2024-09-15T23:08:55.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:55 smithi192 ceph-mon[35427]: pgmap v1041: 97 pgs: 97 active+clean; 559 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 4.6 MiB/s wr, 626 op/s 2024-09-15T23:08:56.688 INFO:tasks.workunit.client.0.smithi111.stdout: 10 190647 17.29 MB/sec execute 444 sec latency 87.495 ms 2024-09-15T23:08:57.688 INFO:tasks.workunit.client.0.smithi111.stdout: 10 191511 17.35 MB/sec execute 445 sec latency 71.209 ms 2024-09-15T23:08:58.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:08:58 smithi192 ceph-mon[35427]: pgmap v1042: 97 pgs: 97 active+clean; 563 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 4.4 MiB/s wr, 606 op/s 2024-09-15T23:08:58.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:08:58 smithi121 ceph-mon[34815]: pgmap v1042: 97 pgs: 97 active+clean; 563 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 4.4 MiB/s wr, 606 op/s 2024-09-15T23:08:58.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:08:58 smithi111 ceph-mon[30629]: pgmap v1042: 97 pgs: 97 active+clean; 563 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 4.4 MiB/s wr, 606 op/s 2024-09-15T23:08:58.689 INFO:tasks.workunit.client.0.smithi111.stdout: 10 192179 17.39 MB/sec execute 446 sec latency 60.789 ms 2024-09-15T23:08:59.689 INFO:tasks.workunit.client.0.smithi111.stdout: 10 192858 17.43 MB/sec execute 447 sec latency 126.849 ms 2024-09-15T23:09:00.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:00 smithi111 ceph-mon[30629]: pgmap v1043: 97 pgs: 97 active+clean; 572 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.1 MiB/s wr, 570 op/s 2024-09-15T23:09:00.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:00 smithi192 ceph-mon[35427]: pgmap v1043: 97 pgs: 97 active+clean; 572 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.1 MiB/s wr, 570 op/s 2024-09-15T23:09:00.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:00 smithi121 ceph-mon[34815]: pgmap v1043: 97 pgs: 97 active+clean; 572 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.1 MiB/s wr, 570 op/s 2024-09-15T23:09:00.689 INFO:tasks.workunit.client.0.smithi111.stdout: 10 193599 17.49 MB/sec execute 448 sec latency 77.645 ms 2024-09-15T23:09:01.689 INFO:tasks.workunit.client.0.smithi111.stdout: 10 194185 17.51 MB/sec execute 449 sec latency 57.680 ms 2024-09-15T23:09:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.114+0000 7f0113e00640 1 -- 172.21.15.111:0/307573064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0114101580 msgr2=0x7f0114101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.114+0000 7f0113e00640 1 --2- 172.21.15.111:0/307573064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0114101580 0x7f0114101a00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0100009860 tx=0x7f010002a340 comp rx=0 tx=0).stop 2024-09-15T23:09:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.114+0000 7f0113e00640 1 -- 172.21.15.111:0/307573064 shutdown_connections 2024-09-15T23:09:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.114+0000 7f0113e00640 1 --2- 172.21.15.111:0/307573064 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0114101f40 0x7f0114109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.114+0000 7f0113e00640 1 --2- 172.21.15.111:0/307573064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0114101580 0x7f0114101a00 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.114+0000 7f0113e00640 1 --2- 172.21.15.111:0/307573064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0114103050 0x7f0114103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.114+0000 7f0113e00640 1 -- 172.21.15.111:0/307573064 >> 172.21.15.111:0/307573064 conn(0x7f01140fd0d0 msgr2=0x7f01140ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.114+0000 7f0113e00640 1 -- 172.21.15.111:0/307573064 shutdown_connections 2024-09-15T23:09:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.114+0000 7f0113e00640 1 -- 172.21.15.111:0/307573064 wait complete. 2024-09-15T23:09:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.115+0000 7f0113e00640 1 Processor -- start 2024-09-15T23:09:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.116+0000 7f0113e00640 1 -- start start 2024-09-15T23:09:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.116+0000 7f0113e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0114101580 0x7f011419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.116+0000 7f0113e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0114101f40 0x7f011419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.116+0000 7f0113e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0114103050 0x7f01141a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.116+0000 7f0113e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f011410d6d0 con 0x7f0114101580 2024-09-15T23:09:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.116+0000 7f0113e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f011410d550 con 0x7f0114103050 2024-09-15T23:09:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.116+0000 7f0113e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f011410d850 con 0x7f0114101f40 2024-09-15T23:09:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.116+0000 7f0112a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0114101580 0x7f011419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.116+0000 7f0112000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0114101f40 0x7f011419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.116+0000 7f0113400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0114103050 0x7f01141a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.117+0000 7f0113400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0114103050 0x7f01141a15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:59710/0 (socket says 172.21.15.111:59710) 2024-09-15T23:09:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.117+0000 7f0113400640 1 -- 172.21.15.111:0/3459645414 learned_addr learned my addr 172.21.15.111:0/3459645414 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:09:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.116+0000 7f0112000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0114101f40 0x7f011419a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:40454/0 (socket says 172.21.15.111:40454) 2024-09-15T23:09:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.117+0000 7f0113400640 1 -- 172.21.15.111:0/3459645414 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0114101f40 msgr2=0x7f011419a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.117+0000 7f0113400640 1 --2- 172.21.15.111:0/3459645414 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0114101f40 0x7f011419a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.117+0000 7f0113400640 1 -- 172.21.15.111:0/3459645414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0114101580 msgr2=0x7f011419a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.117+0000 7f0113400640 1 --2- 172.21.15.111:0/3459645414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0114101580 0x7f011419a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.117+0000 7f0113400640 1 -- 172.21.15.111:0/3459645414 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01141a1c30 con 0x7f0114103050 2024-09-15T23:09:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.117+0000 7f0112a00640 1 --2- 172.21.15.111:0/3459645414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0114101580 0x7f011419a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:09:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.118+0000 7f0113400640 1 --2- 172.21.15.111:0/3459645414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0114103050 0x7f01141a15c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f010c00efc0 tx=0x7f010c00c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.118+0000 7f010b400640 1 -- 172.21.15.111:0/3459645414 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c003970 con 0x7f0114103050 2024-09-15T23:09:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.118+0000 7f010b400640 1 -- 172.21.15.111:0/3459645414 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f010c004020 con 0x7f0114103050 2024-09-15T23:09:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.118+0000 7f0113e00640 1 -- 172.21.15.111:0/3459645414 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01141a1f20 con 0x7f0114103050 2024-09-15T23:09:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.118+0000 7f010b400640 1 -- 172.21.15.111:0/3459645414 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010c00b9f0 con 0x7f0114103050 2024-09-15T23:09:02.126 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.118+0000 7f0113e00640 1 -- 172.21.15.111:0/3459645414 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01141a2390 con 0x7f0114103050 2024-09-15T23:09:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.122+0000 7f011c828740 1 Processor -- start 2024-09-15T23:09:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.122+0000 7f010b400640 1 -- 172.21.15.111:0/3459645414 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f010c01d020 con 0x7f0114103050 2024-09-15T23:09:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.123+0000 7f011c828740 1 -- start start 2024-09-15T23:09:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.123+0000 7f011c828740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557c53d7e260 0x557c53d7e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.123+0000 7f011c828740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557c53d7eba0 0x557c53d83660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.123+0000 7f011c828740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c53d83c30 0x557c53d860f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.123+0000 7f011c828740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557c53ce3100 con 0x557c53d7eba0 2024-09-15T23:09:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.123+0000 7f011c828740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x557c53c8b1b0 con 0x557c53d7e260 2024-09-15T23:09:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.123+0000 7f011c828740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557c53cdf0c0 con 0x557c53d83c30 2024-09-15T23:09:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.124+0000 7f0112000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557c53d7e260 0x557c53d7e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.124+0000 7f0112a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557c53d7eba0 0x557c53d83660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.124+0000 7f0112000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557c53d7e260 0x557c53d7e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:59714/0 (socket says 172.21.15.111:59714) 2024-09-15T23:09:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.124+0000 7f0112000640 1 -- 172.21.15.111:0/3470620496 learned_addr learned my addr 172.21.15.111:0/3470620496 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:09:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.124+0000 7f0112000640 1 --2- 172.21.15.111:0/3470620496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c53d83c30 0x557c53d860f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.124+0000 7f0112a00640 1 -- 172.21.15.111:0/3470620496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c53d83c30 msgr2=0x557c53d860f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.124+0000 7f0112a00640 1 --2- 172.21.15.111:0/3470620496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c53d83c30 0x557c53d860f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.124+0000 7f0112a00640 1 -- 172.21.15.111:0/3470620496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557c53d7e260 msgr2=0x557c53d7e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:02.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.124+0000 7f0112a00640 1 --2- 172.21.15.111:0/3470620496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557c53d7e260 0x557c53d7e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.124+0000 7f0112a00640 1 -- 172.21.15.111:0/3470620496 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c539f6210 con 0x557c53d7eba0 2024-09-15T23:09:02.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.124+0000 7f0112000640 1 --2- 172.21.15.111:0/3470620496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557c53d7e260 0x557c53d7e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:09:02.138 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.124+0000 7f0112a00640 1 --2- 172.21.15.111:0/3470620496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557c53d7eba0 0x557c53d83660 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0104009fa0 tx=0x7f01040118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2817e493d33f38 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.124+0000 7f010b400640 1 --2- 172.21.15.111:0/3459645414 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f00f80a3240 0x7f00f80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.125+0000 7f00ffe00640 1 -- 172.21.15.111:0/3470620496 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104004310 con 0x557c53d7eba0 2024-09-15T23:09:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.125+0000 7f010b400640 1 -- 172.21.15.111:0/3459645414 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f010c0cf6b0 con 0x7f0114103050 2024-09-15T23:09:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.125+0000 7f00ffe00640 1 -- 172.21.15.111:0/3470620496 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01040044b0 con 0x557c53d7eba0 2024-09-15T23:09:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.125+0000 7f00ffe00640 1 -- 172.21.15.111:0/3470620496 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01040035f0 con 0x557c53d7eba0 2024-09-15T23:09:02.139 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.126+0000 7f0112000640 1 --2- 172.21.15.111:0/3459645414 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f00f80a3240 0x7f00f80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:02.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.126+0000 7f0112000640 1 --2- 172.21.15.111:0/3459645414 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f00f80a3240 0x7f00f80a5700 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f010002a930 tx=0x7f01000056d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:02.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.126+0000 7f011c828740 1 -- 172.21.15.111:0/3470620496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557c53d7eba0 msgr2=0x557c53d83660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:02.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.126+0000 7f011c828740 1 --2- 172.21.15.111:0/3470620496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557c53d7eba0 0x557c53d83660 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0104009fa0 tx=0x7f01040118d0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.140 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.127+0000 7f011c828740 1 -- 172.21.15.111:0/3470620496 shutdown_connections 2024-09-15T23:09:02.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.127+0000 7f011c828740 1 --2- 172.21.15.111:0/3470620496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c53d83c30 0x557c53d860f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.127+0000 7f011c828740 1 --2- 172.21.15.111:0/3470620496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557c53d7eba0 0x557c53d83660 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.127+0000 7f011c828740 1 --2- 172.21.15.111:0/3470620496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557c53d7e260 0x557c53d7e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.127+0000 7f011c828740 1 -- 172.21.15.111:0/3470620496 >> 172.21.15.111:0/3470620496 conn(0x557c53d798d0 msgr2=0x557c53d79cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:02.141 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.127+0000 7f011c828740 1 -- 172.21.15.111:0/3470620496 shutdown_connections 2024-09-15T23:09:02.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.127+0000 7f011c828740 1 -- 172.21.15.111:0/3470620496 wait complete. 2024-09-15T23:09:02.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.127+0000 7f011c828740 1 -- start start 2024-09-15T23:09:02.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.128+0000 7f011c828740 1 Processor -- start 2024-09-15T23:09:02.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.128+0000 7f011c828740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557c53d7e260 0x557c53d88410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:02.142 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.128+0000 7f011c828740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c53d7eba0 0x557c53d88950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:02.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.128+0000 7f011c828740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557c53d83c30 0x557c53d8f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:02.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.128+0000 7f011c828740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x557c53d86c00 con 0x557c53d7e260 2024-09-15T23:09:02.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.128+0000 7f011c828740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x557c53c6c090 con 0x557c53d83c30 2024-09-15T23:09:02.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.128+0000 7f011c828740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557c53d86d80 con 0x557c53d7eba0 2024-09-15T23:09:02.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.129+0000 7f011c828740 10 client.0 _ll_register_callbacks cb 0x557c53cc0f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:09:02.143 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.129+0000 7f0112000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c53d7eba0 0x557c53d88950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.129+0000 7f0112000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c53d7eba0 0x557c53d88950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:40470/0 (socket says 172.21.15.111:40470) 2024-09-15T23:09:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.129+0000 7f0112000640 1 -- 172.21.15.111:0/3785339782 learned_addr learned my addr 172.21.15.111:0/3785339782 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:09:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.129+0000 7f0112a00640 1 --2- 172.21.15.111:0/3785339782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557c53d7e260 0x557c53d88410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.129+0000 7f0113400640 1 --2- 172.21.15.111:0/3785339782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557c53d83c30 0x557c53d8f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.129+0000 7f0112000640 1 -- 172.21.15.111:0/3785339782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557c53d83c30 msgr2=0x557c53d8f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.129+0000 7f0112000640 1 --2- 172.21.15.111:0/3785339782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557c53d83c30 0x557c53d8f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.129+0000 7f0112000640 1 -- 172.21.15.111:0/3785339782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557c53d7e260 msgr2=0x557c53d88410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:02.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.129+0000 7f0112000640 1 --2- 172.21.15.111:0/3785339782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557c53d7e260 0x557c53d88410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.129+0000 7f0112000640 1 -- 172.21.15.111:0/3785339782 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c53ac2620 con 0x557c53d7eba0 2024-09-15T23:09:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.129+0000 7f0113400640 1 --2- 172.21.15.111:0/3785339782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557c53d83c30 0x557c53d8f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:09:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.129+0000 7f0112a00640 1 --2- 172.21.15.111:0/3785339782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557c53d7e260 0x557c53d88410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:09:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.130+0000 7f0112000640 1 --2- 172.21.15.111:0/3785339782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c53d7eba0 0x557c53d88950 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0100035040 tx=0x7f010002a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5b99eae63dd9f7a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.130+0000 7f00fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:09:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.130+0000 7f00fcc00640 1 -- 172.21.15.111:0/3785339782 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010004f060 con 0x557c53d7eba0 2024-09-15T23:09:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.130+0000 7f00fcc00640 1 -- 172.21.15.111:0/3785339782 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f01000037f0 con 0x557c53d7eba0 2024-09-15T23:09:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.130+0000 7f011c828740 1 -- 172.21.15.111:0/3785339782 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01040044b0 con 0x557c53d7eba0 2024-09-15T23:09:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.130+0000 7f00fcc00640 1 -- 172.21.15.111:0/3785339782 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0100046040 con 0x557c53d7eba0 2024-09-15T23:09:02.145 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.131+0000 7f011c828740 10 client.26737 fetch_fsmap learned FSMap version 35 2024-09-15T23:09:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.131+0000 7f011c828740 1 -- 172.21.15.111:0/3785339782 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557c53d92820 con 0x557c53d7eba0 2024-09-15T23:09:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.131+0000 7f00fcc00640 1 -- 172.21.15.111:0/3785339782 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01040044b0 con 0x557c53d7eba0 2024-09-15T23:09:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.131+0000 7f00fcc00640 1 -- 172.21.15.111:0/3785339782 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f010004aa70 con 0x557c53d7eba0 2024-09-15T23:09:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.131+0000 7f011c828740 10 client.26737 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:09:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.131+0000 7f011c828740 10 client.26737 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:09:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.131+0000 7f011c828740 20 client.26737 populate_metadata read hostname 'smithi111' 2024-09-15T23:09:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.131+0000 7f011c828740 1 --2- 172.21.15.111:0/3785339782 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557c53d93ef0 0x557c53d96360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.131+0000 7f011c828740 4 client.26737 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:09:02.146 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.131+0000 7f011c828740 1 -- 172.21.15.111:0/3785339782 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c53cf0830 con 0x557c53d93ef0 2024-09-15T23:09:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.132+0000 7f0112a00640 1 --2- 172.21.15.111:0/3785339782 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557c53d93ef0 0x557c53d96360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.132+0000 7f0112a00640 1 --2- 172.21.15.111:0/3785339782 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557c53d93ef0 0x557c53d96360 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=275f6b133560582d server_cookie=fef8235a2dd55f55 in_seq=0 out_seq=0 2024-09-15T23:09:02.147 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.132+0000 7f00fcc00640 10 client.26737 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:09:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.134+0000 7f00fcc00640 1 -- 172.21.15.111:0/3785339782 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557c53cf0830 con 0x557c53d93ef0 2024-09-15T23:09:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.134+0000 7f00fcc00640 10 client.26737 handle_command_reply: tid=0 2024-09-15T23:09:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.134+0000 7f011c828740 1 client.26737 shutdown 2024-09-15T23:09:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.134+0000 7f011c828740 1 -- 172.21.15.111:0/3785339782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c53d7eba0 msgr2=0x557c53d88950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.135+0000 7f011c828740 1 --2- 172.21.15.111:0/3785339782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c53d7eba0 0x557c53d88950 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0100035040 tx=0x7f010002a340 comp rx=0 tx=0).stop 2024-09-15T23:09:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.135+0000 7f011c828740 1 -- 172.21.15.111:0/3785339782 shutdown_connections 2024-09-15T23:09:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.135+0000 7f011c828740 1 --2- 172.21.15.111:0/3785339782 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557c53d93ef0 0x557c53d96360 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.135+0000 7f00fcc00640 0 client.26737 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:09:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.135+0000 7f011c828740 1 --2- 172.21.15.111:0/3785339782 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557c53d83c30 0x557c53d8f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.135+0000 7f011c828740 1 --2- 172.21.15.111:0/3785339782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c53d7eba0 0x557c53d88950 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.135+0000 7f011c828740 1 --2- 172.21.15.111:0/3785339782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557c53d7e260 0x557c53d88410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.135+0000 7f011c828740 1 -- 172.21.15.111:0/3785339782 >> 172.21.15.111:0/3785339782 conn(0x557c53d798d0 msgr2=0x557c53d7b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.135+0000 7f011c828740 1 -- 172.21.15.111:0/3785339782 shutdown_connections 2024-09-15T23:09:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.138+0000 7f011c828740 1 -- 172.21.15.111:0/3785339782 wait complete. 2024-09-15T23:09:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.138+0000 7f011c828740 20 client.26737 trim_cache size 0 max 16384 2024-09-15T23:09:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.160+0000 7f011c828740 1 Processor -- start 2024-09-15T23:09:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.160+0000 7f011c828740 1 -- start start 2024-09-15T23:09:02.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.160+0000 7f011c828740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c53d7e260 0x557c53d7a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.160+0000 7f011c828740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557c53d83c30 0x557c53d8f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.160+0000 7f011c828740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557c53d88b20 0x557c53d7acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.160+0000 7f011c828740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f00f4010bc0 con 0x557c53d83c30 2024-09-15T23:09:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.160+0000 7f011c828740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f00f400e490 con 0x557c53d88b20 2024-09-15T23:09:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.160+0000 7f011c828740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f00f400c2e0 con 0x557c53d7e260 2024-09-15T23:09:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.160+0000 7f0112a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c53d7e260 0x557c53d7a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.161+0000 7f0112000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557c53d83c30 0x557c53d8f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.161+0000 7f0113400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557c53d88b20 0x557c53d7acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.161+0000 7f0112a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c53d7e260 0x557c53d7a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:40478/0 (socket says 172.21.15.111:40478) 2024-09-15T23:09:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.161+0000 7f0112a00640 1 -- 172.21.15.111:0/2558627850 learned_addr learned my addr 172.21.15.111:0/2558627850 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:09:02.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.161+0000 7f0112a00640 1 -- 172.21.15.111:0/2558627850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557c53d88b20 msgr2=0x557c53d7acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.161+0000 7f0112a00640 1 --2- 172.21.15.111:0/2558627850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557c53d88b20 0x557c53d7acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.161+0000 7f0112a00640 1 -- 172.21.15.111:0/2558627850 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557c53d83c30 msgr2=0x557c53d8f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.161+0000 7f0112a00640 1 --2- 172.21.15.111:0/2558627850 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557c53d83c30 0x557c53d8f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.161+0000 7f0112a00640 1 -- 172.21.15.111:0/2558627850 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c53d796f0 con 0x557c53d7e260 2024-09-15T23:09:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.161+0000 7f0112a00640 1 --2- 172.21.15.111:0/2558627850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c53d7e260 0x557c53d7a780 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f010400a010 tx=0x7f0104005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89eaa7d471b899f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.162+0000 7f00ff400640 1 -- 172.21.15.111:0/2558627850 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010401d040 con 0x557c53d7e260 2024-09-15T23:09:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.162+0000 7f00ff400640 1 -- 172.21.15.111:0/2558627850 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0104009b50 con 0x557c53d7e260 2024-09-15T23:09:02.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.162+0000 7f00ff400640 1 -- 172.21.15.111:0/2558627850 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010400f680 con 0x557c53d7e260 2024-09-15T23:09:02.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.163+0000 7f011c828740 1 -- 172.21.15.111:0/2558627850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c53d7e260 msgr2=0x557c53d7a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:02.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.163+0000 7f011c828740 1 --2- 172.21.15.111:0/2558627850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c53d7e260 0x557c53d7a780 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f010400a010 tx=0x7f0104005f90 comp rx=0 tx=0).stop 2024-09-15T23:09:02.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.163+0000 7f011c828740 1 -- 172.21.15.111:0/2558627850 shutdown_connections 2024-09-15T23:09:02.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.163+0000 7f011c828740 1 --2- 172.21.15.111:0/2558627850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557c53d88b20 0x557c53d7acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.167 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.163+0000 7f011c828740 1 --2- 172.21.15.111:0/2558627850 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557c53d83c30 0x557c53d8f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.163+0000 7f011c828740 1 --2- 172.21.15.111:0/2558627850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c53d7e260 0x557c53d7a780 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.163+0000 7f011c828740 1 -- 172.21.15.111:0/2558627850 >> 172.21.15.111:0/2558627850 conn(0x557c53d798d0 msgr2=0x557c53d88410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.163+0000 7f011c828740 1 -- 172.21.15.111:0/2558627850 shutdown_connections 2024-09-15T23:09:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.163+0000 7f011c828740 1 -- 172.21.15.111:0/2558627850 wait complete. 2024-09-15T23:09:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.164+0000 7f011c828740 1 -- start start 2024-09-15T23:09:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.164+0000 7f011c828740 1 Processor -- start 2024-09-15T23:09:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.164+0000 7f011c828740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557c53d7e260 0x557c53d8cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.164+0000 7f011c828740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557c53d83c30 0x557c53d8d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.164+0000 7f011c828740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c53d88b20 0x557c53d8e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.165+0000 7f011c828740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f00f400c950 con 0x557c53d83c30 2024-09-15T23:09:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.165+0000 7f011c828740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f00f400ddc0 con 0x557c53d7e260 2024-09-15T23:09:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.165+0000 7f011c828740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f00f400d020 con 0x557c53d88b20 2024-09-15T23:09:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.165+0000 7f011c828740 10 client.0 _ll_register_callbacks cb 0x7f0100043040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:09:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.165+0000 7f0112a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557c53d7e260 0x557c53d8cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.165+0000 7f0112a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557c53d7e260 0x557c53d8cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:59748/0 (socket says 172.21.15.111:59748) 2024-09-15T23:09:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.165+0000 7f0112a00640 1 -- 172.21.15.111:0/1379447539 learned_addr learned my addr 172.21.15.111:0/1379447539 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:09:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.165+0000 7f0112000640 1 --2- 172.21.15.111:0/1379447539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557c53d83c30 0x557c53d8d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.165+0000 7f0113400640 1 --2- 172.21.15.111:0/1379447539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c53d88b20 0x557c53d8e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.165+0000 7f0112a00640 1 -- 172.21.15.111:0/1379447539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c53d88b20 msgr2=0x557c53d8e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.165+0000 7f0112a00640 1 --2- 172.21.15.111:0/1379447539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c53d88b20 0x557c53d8e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.165+0000 7f0112a00640 1 -- 172.21.15.111:0/1379447539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557c53d83c30 msgr2=0x557c53d8d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.165+0000 7f0112a00640 1 --2- 172.21.15.111:0/1379447539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557c53d83c30 0x557c53d8d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.165+0000 7f0112a00640 1 -- 172.21.15.111:0/1379447539 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c53d8e950 con 0x557c53d7e260 2024-09-15T23:09:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.165+0000 7f0112000640 1 --2- 172.21.15.111:0/1379447539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557c53d83c30 0x557c53d8d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:09:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.166+0000 7f0112a00640 1 --2- 172.21.15.111:0/1379447539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557c53d7e260 0x557c53d8cdf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0104009310 tx=0x7f0104009340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f83c82ecb89b5a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:02.172 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.166+0000 7f00fcc00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:09:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.166+0000 7f00fcc00640 1 -- 172.21.15.111:0/1379447539 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104039030 con 0x557c53d7e260 2024-09-15T23:09:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.166+0000 7f00fcc00640 1 -- 172.21.15.111:0/1379447539 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f0104030040 con 0x557c53d7e260 2024-09-15T23:09:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.166+0000 7f011c828740 1 -- 172.21.15.111:0/1379447539 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0104009b50 con 0x557c53d7e260 2024-09-15T23:09:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.166+0000 7f00fcc00640 1 -- 172.21.15.111:0/1379447539 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f010401d070 con 0x557c53d7e260 2024-09-15T23:09:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.166+0000 7f011c828740 10 client.26750 fetch_fsmap learned FSMap version 35 2024-09-15T23:09:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.166+0000 7f011c828740 1 -- 172.21.15.111:0/1379447539 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557c53d8fc60 con 0x557c53d7e260 2024-09-15T23:09:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.167+0000 7f00fcc00640 1 -- 172.21.15.111:0/1379447539 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0104009b50 con 0x557c53d7e260 2024-09-15T23:09:02.174 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.167+0000 7f00fcc00640 1 -- 172.21.15.111:0/1379447539 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f010400faf0 con 0x557c53d7e260 2024-09-15T23:09:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.167+0000 7f011c828740 10 client.26750 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:09:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.167+0000 7f011c828740 10 client.26750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:09:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.167+0000 7f011c828740 20 client.26750 populate_metadata read hostname 'smithi111' 2024-09-15T23:09:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.167+0000 7f011c828740 1 --2- 172.21.15.111:0/1379447539 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557c53e079e0 0x557c53e09e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.167+0000 7f011c828740 4 client.26750 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:09:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.167+0000 7f011c828740 1 -- 172.21.15.111:0/1379447539 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557c53ceab80 con 0x557c53e079e0 2024-09-15T23:09:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.168+0000 7f0112000640 1 --2- 172.21.15.111:0/1379447539 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557c53e079e0 0x557c53e09e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.169+0000 7f0112000640 1 --2- 172.21.15.111:0/1379447539 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557c53e079e0 0x557c53e09e00 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e9d09bab560e8b4 server_cookie=e882720bf614dc1a in_seq=0 out_seq=0 2024-09-15T23:09:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.169+0000 7f00fcc00640 10 client.26750 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:09:02.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:02 smithi192 ceph-mon[35427]: pgmap v1044: 97 pgs: 97 active+clean; 580 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.2 MiB/s wr, 565 op/s 2024-09-15T23:09:02.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:02 smithi121 ceph-mon[34815]: pgmap v1044: 97 pgs: 97 active+clean; 580 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.2 MiB/s wr, 565 op/s 2024-09-15T23:09:02.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:02 smithi111 ceph-mon[30629]: pgmap v1044: 97 pgs: 97 active+clean; 580 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.2 MiB/s wr, 565 op/s 2024-09-15T23:09:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.427+0000 7f00fcc00640 1 -- 172.21.15.111:0/1379447539 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557c53ceab80 con 0x557c53e079e0 2024-09-15T23:09:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.427+0000 7f00fcc00640 10 client.26750 handle_command_reply: tid=0 2024-09-15T23:09:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.427+0000 7f011c828740 1 client.26750 shutdown 2024-09-15T23:09:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.428+0000 7f011c828740 1 -- 172.21.15.111:0/1379447539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557c53d7e260 msgr2=0x557c53d8cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:02.429 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.428+0000 7f011c828740 1 --2- 172.21.15.111:0/1379447539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557c53d7e260 0x557c53d8cdf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0104009310 tx=0x7f0104009340 comp rx=0 tx=0).stop 2024-09-15T23:09:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.428+0000 7f011c828740 1 -- 172.21.15.111:0/1379447539 shutdown_connections 2024-09-15T23:09:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.428+0000 7f011c828740 1 --2- 172.21.15.111:0/1379447539 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x557c53e079e0 0x557c53e09e00 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.428+0000 7f011c828740 1 --2- 172.21.15.111:0/1379447539 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557c53d88b20 0x557c53d8e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.428+0000 7f011c828740 1 --2- 172.21.15.111:0/1379447539 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557c53d83c30 0x557c53d8d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.428+0000 7f011c828740 1 --2- 172.21.15.111:0/1379447539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557c53d7e260 0x557c53d8cdf0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.428+0000 7f011c828740 1 -- 172.21.15.111:0/1379447539 >> 172.21.15.111:0/1379447539 conn(0x557c53d798d0 msgr2=0x557c53d781f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.428+0000 7f00fcc00640 0 client.26750 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:09:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.428+0000 7f011c828740 1 -- 172.21.15.111:0/1379447539 shutdown_connections 2024-09-15T23:09:02.430 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.428+0000 7f011c828740 1 -- 172.21.15.111:0/1379447539 wait complete. 2024-09-15T23:09:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.429+0000 7f011c828740 20 client.26750 trim_cache size 0 max 16384 2024-09-15T23:09:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:09:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:09:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:09:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:09:02.431 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.429+0000 7f0113e00640 1 -- 172.21.15.111:0/3459645414 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f00f80a3240 msgr2=0x7f00f80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.429+0000 7f0113e00640 1 --2- 172.21.15.111:0/3459645414 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f00f80a3240 0x7f00f80a5700 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f010002a930 tx=0x7f01000056d0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.429+0000 7f0113e00640 1 -- 172.21.15.111:0/3459645414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0114103050 msgr2=0x7f01141a15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.429+0000 7f0113e00640 1 --2- 172.21.15.111:0/3459645414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0114103050 0x7f01141a15c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f010c00efc0 tx=0x7f010c00c800 comp rx=0 tx=0).stop 2024-09-15T23:09:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.430+0000 7f0113e00640 1 -- 172.21.15.111:0/3459645414 shutdown_connections 2024-09-15T23:09:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.430+0000 7f0113e00640 1 --2- 172.21.15.111:0/3459645414 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f00f80a3240 0x7f00f80a5700 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.432 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.430+0000 7f0113e00640 1 --2- 172.21.15.111:0/3459645414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0114103050 0x7f01141a15c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.430+0000 7f0113e00640 1 --2- 172.21.15.111:0/3459645414 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0114101f40 0x7f011419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.430+0000 7f0113e00640 1 --2- 172.21.15.111:0/3459645414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0114101580 0x7f011419a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.430+0000 7f0113e00640 1 -- 172.21.15.111:0/3459645414 >> 172.21.15.111:0/3459645414 conn(0x7f01140fd0d0 msgr2=0x7f0114108330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.430+0000 7f0113e00640 1 -- 172.21.15.111:0/3459645414 shutdown_connections 2024-09-15T23:09:02.433 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:02.431+0000 7f0113e00640 1 -- 172.21.15.111:0/3459645414 wait complete. 2024-09-15T23:09:02.690 INFO:tasks.workunit.client.0.smithi111.stdout: 10 194836 17.54 MB/sec execute 450 sec latency 175.547 ms 2024-09-15T23:09:03.689 INFO:tasks.workunit.client.0.smithi111.stdout: 10 195728 17.64 MB/sec execute 451 sec latency 59.881 ms 2024-09-15T23:09:04.261 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:04 smithi111 ceph-mon[30629]: pgmap v1045: 97 pgs: 97 active+clean; 583 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.0 MiB/s wr, 490 op/s 2024-09-15T23:09:04.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:04 smithi192 ceph-mon[35427]: pgmap v1045: 97 pgs: 97 active+clean; 583 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.0 MiB/s wr, 490 op/s 2024-09-15T23:09:04.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:04 smithi121 ceph-mon[34815]: pgmap v1045: 97 pgs: 97 active+clean; 583 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 4.0 MiB/s wr, 490 op/s 2024-09-15T23:09:04.689 INFO:tasks.workunit.client.0.smithi111.stdout: 10 196300 17.66 MB/sec execute 452 sec latency 99.144 ms 2024-09-15T23:09:05.689 INFO:tasks.workunit.client.0.smithi111.stdout: 10 196310 17.62 MB/sec execute 453 sec latency 1099.206 ms 2024-09-15T23:09:06.196 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:06 smithi111 ceph-mon[30629]: pgmap v1046: 97 pgs: 97 active+clean; 581 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.4 MiB/s wr, 557 op/s 2024-09-15T23:09:06.196 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:06 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:09:06.196 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:06 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:09:06.196 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:06 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:09:06.197 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:06 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:09:06.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:06 smithi192 ceph-mon[35427]: pgmap v1046: 97 pgs: 97 active+clean; 581 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.4 MiB/s wr, 557 op/s 2024-09-15T23:09:06.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:06 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:09:06.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:06 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:09:06.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:06 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:09:06.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:06 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:09:06.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:06 smithi121 ceph-mon[34815]: pgmap v1046: 97 pgs: 97 active+clean; 581 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.4 MiB/s wr, 557 op/s 2024-09-15T23:09:06.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:06 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:09:06.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:06 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:09:06.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:06 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:09:06.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:06 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:09:06.689 INFO:tasks.workunit.client.0.smithi111.stdout: 10 196533 17.59 MB/sec execute 454 sec latency 1672.180 ms 2024-09-15T23:09:07.689 INFO:tasks.workunit.client.0.smithi111.stdout: 10 197400 17.68 MB/sec execute 455 sec latency 38.632 ms 2024-09-15T23:09:08.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:08 smithi192 ceph-mon[35427]: pgmap v1047: 97 pgs: 97 active+clean; 586 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.3 MiB/s wr, 507 op/s 2024-09-15T23:09:08.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:08 smithi121 ceph-mon[34815]: pgmap v1047: 97 pgs: 97 active+clean; 586 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.3 MiB/s wr, 507 op/s 2024-09-15T23:09:08.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:08 smithi111 ceph-mon[30629]: pgmap v1047: 97 pgs: 97 active+clean; 586 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.3 MiB/s wr, 507 op/s 2024-09-15T23:09:08.689 INFO:tasks.workunit.client.0.smithi111.stdout: 10 198240 17.72 MB/sec execute 456 sec latency 55.122 ms 2024-09-15T23:09:09.689 INFO:tasks.workunit.client.0.smithi111.stdout: 10 198918 17.76 MB/sec execute 457 sec latency 73.733 ms 2024-09-15T23:09:10.313 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:10 smithi111 ceph-mon[30629]: pgmap v1048: 97 pgs: 97 active+clean; 595 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.3 MiB/s wr, 584 op/s 2024-09-15T23:09:10.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:10 smithi192 ceph-mon[35427]: pgmap v1048: 97 pgs: 97 active+clean; 595 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.3 MiB/s wr, 584 op/s 2024-09-15T23:09:10.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:10 smithi121 ceph-mon[34815]: pgmap v1048: 97 pgs: 97 active+clean; 595 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.3 MiB/s wr, 584 op/s 2024-09-15T23:09:10.689 INFO:tasks.workunit.client.0.smithi111.stdout: 10 198981 17.74 MB/sec execute 458 sec latency 950.333 ms 2024-09-15T23:09:11.689 INFO:tasks.workunit.client.0.smithi111.stdout: 10 198981 17.70 MB/sec execute 459 sec latency 1950.382 ms 2024-09-15T23:09:12.302 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:12 smithi111 ceph-mon[30629]: pgmap v1049: 97 pgs: 97 active+clean; 601 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.5 MiB/s wr, 560 op/s 2024-09-15T23:09:12.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:12 smithi192 ceph-mon[35427]: pgmap v1049: 97 pgs: 97 active+clean; 601 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.5 MiB/s wr, 560 op/s 2024-09-15T23:09:12.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:12 smithi121 ceph-mon[34815]: pgmap v1049: 97 pgs: 97 active+clean; 601 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 5.5 MiB/s wr, 560 op/s 2024-09-15T23:09:12.689 INFO:tasks.workunit.client.0.smithi111.stdout: 10 198981 17.66 MB/sec execute 460 sec latency 2950.460 ms 2024-09-15T23:09:13.690 INFO:tasks.workunit.client.0.smithi111.stdout: 10 198982 17.62 MB/sec execute 461 sec latency 3950.534 ms 2024-09-15T23:09:14.270 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:14 smithi111 ceph-mon[30629]: pgmap v1050: 97 pgs: 97 active+clean; 606 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.4 MiB/s wr, 481 op/s 2024-09-15T23:09:14.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:14 smithi192 ceph-mon[35427]: pgmap v1050: 97 pgs: 97 active+clean; 606 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.4 MiB/s wr, 481 op/s 2024-09-15T23:09:14.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:14 smithi121 ceph-mon[34815]: pgmap v1050: 97 pgs: 97 active+clean; 606 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.4 MiB/s wr, 481 op/s 2024-09-15T23:09:14.690 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199019 17.59 MB/sec execute 462 sec latency 4950.604 ms 2024-09-15T23:09:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.572+0000 7f9e35a00640 1 -- 172.21.15.111:0/3016358728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e301001b0 msgr2=0x7f9e301025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.572+0000 7f9e35a00640 1 --2- 172.21.15.111:0/3016358728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e301001b0 0x7f9e301025c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9e18009900 tx=0x7f9e1802a3f0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.572+0000 7f9e35a00640 1 -- 172.21.15.111:0/3016358728 shutdown_connections 2024-09-15T23:09:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.572+0000 7f9e35a00640 1 --2- 172.21.15.111:0/3016358728 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e301054b0 0x7f9e30105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.572+0000 7f9e35a00640 1 --2- 172.21.15.111:0/3016358728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e30102b00 0x7f9e30104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.572+0000 7f9e35a00640 1 --2- 172.21.15.111:0/3016358728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e301001b0 0x7f9e301025c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.572+0000 7f9e35a00640 1 -- 172.21.15.111:0/3016358728 >> 172.21.15.111:0/3016358728 conn(0x7f9e300fbfa0 msgr2=0x7f9e300fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.572+0000 7f9e35a00640 1 -- 172.21.15.111:0/3016358728 shutdown_connections 2024-09-15T23:09:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.572+0000 7f9e35a00640 1 -- 172.21.15.111:0/3016358728 wait complete. 2024-09-15T23:09:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.577+0000 7f9e35a00640 1 Processor -- start 2024-09-15T23:09:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.577+0000 7f9e35a00640 1 -- start start 2024-09-15T23:09:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.577+0000 7f9e35a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e301001b0 0x7f9e3019e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.577+0000 7f9e35a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e30102b00 0x7f9e3019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.577+0000 7f9e35a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e301054b0 0x7f9e301a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.577+0000 7f9e35a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9e30113c00 con 0x7f9e301001b0 2024-09-15T23:09:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.577+0000 7f9e35a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9e30113a80 con 0x7f9e301054b0 2024-09-15T23:09:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.577+0000 7f9e35a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9e30113d80 con 0x7f9e30102b00 2024-09-15T23:09:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.577+0000 7f9e35000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e301054b0 0x7f9e301a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.577+0000 7f9e35000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e301054b0 0x7f9e301a5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:55044/0 (socket says 172.21.15.111:55044) 2024-09-15T23:09:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.577+0000 7f9e35000640 1 -- 172.21.15.111:0/1503679120 learned_addr learned my addr 172.21.15.111:0/1503679120 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:09:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.577+0000 7f9e2f400640 1 --2- 172.21.15.111:0/1503679120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e30102b00 0x7f9e3019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.577+0000 7f9e35000640 1 -- 172.21.15.111:0/1503679120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e30102b00 msgr2=0x7f9e3019e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.577+0000 7f9e35000640 1 --2- 172.21.15.111:0/1503679120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e30102b00 0x7f9e3019e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.577+0000 7f9e35000640 1 -- 172.21.15.111:0/1503679120 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e301001b0 msgr2=0x7f9e3019e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.577+0000 7f9e35000640 1 --2- 172.21.15.111:0/1503679120 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e301001b0 0x7f9e3019e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.577+0000 7f9e35000640 1 -- 172.21.15.111:0/1503679120 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e301a5df0 con 0x7f9e301054b0 2024-09-15T23:09:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.577+0000 7f9e35000640 1 --2- 172.21.15.111:0/1503679120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e301054b0 0x7f9e301a5780 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9e2800eb10 tx=0x7f9e2800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.582+0000 7f9e2cc00640 1 -- 172.21.15.111:0/1503679120 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e28004b20 con 0x7f9e301054b0 2024-09-15T23:09:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.582+0000 7f9e35a00640 1 -- 172.21.15.111:0/1503679120 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e301a60e0 con 0x7f9e301054b0 2024-09-15T23:09:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.582+0000 7f9e35a00640 1 -- 172.21.15.111:0/1503679120 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e301a6680 con 0x7f9e301054b0 2024-09-15T23:09:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.582+0000 7f9e2cc00640 1 -- 172.21.15.111:0/1503679120 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e28004cc0 con 0x7f9e301054b0 2024-09-15T23:09:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.582+0000 7f9e2cc00640 1 -- 172.21.15.111:0/1503679120 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e28004b20 con 0x7f9e301054b0 2024-09-15T23:09:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.583+0000 7f9e39df2740 1 Processor -- start 2024-09-15T23:09:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.589+0000 7f9e2cc00640 1 -- 172.21.15.111:0/1503679120 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f9e28010590 con 0x7f9e301054b0 2024-09-15T23:09:15.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.589+0000 7f9e2cc00640 1 --2- 172.21.15.111:0/1503679120 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9e140a3580 0x7f9e140a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.589+0000 7f9e2cc00640 1 -- 172.21.15.111:0/1503679120 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f9e280cfbc0 con 0x7f9e301054b0 2024-09-15T23:09:15.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.589+0000 7f9e2f400640 1 --2- 172.21.15.111:0/1503679120 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9e140a3580 0x7f9e140a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.589+0000 7f9e39df2740 1 -- start start 2024-09-15T23:09:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.589+0000 7f9e39df2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563636e89260 0x563636e89660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.589+0000 7f9e39df2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563636e89ba0 0x563636e8e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:15.594 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.589+0000 7f9e39df2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563636e8ec30 0x563636e910f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.589+0000 7f9e39df2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563636dee100 con 0x563636e8ec30 2024-09-15T23:09:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.589+0000 7f9e39df2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563636d961b0 con 0x563636e89ba0 2024-09-15T23:09:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.589+0000 7f9e39df2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563636dea0c0 con 0x563636e89260 2024-09-15T23:09:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.589+0000 7f9e2fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563636e89260 0x563636e89660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.589+0000 7f9e2fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563636e89260 0x563636e89660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:34864/0 (socket says 172.21.15.111:34864) 2024-09-15T23:09:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.589+0000 7f9e2fe00640 1 -- 172.21.15.111:0/1333666053 learned_addr learned my addr 172.21.15.111:0/1333666053 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:09:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.589+0000 7f9e35000640 1 --2- 172.21.15.111:0/1333666053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563636e8ec30 0x563636e910f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:15.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.594+0000 7f9e2fe00640 1 -- 172.21.15.111:0/1333666053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563636e89ba0 msgr2=0x563636e8e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.594+0000 7f9e2fe00640 1 --2- 172.21.15.111:0/1333666053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563636e89ba0 0x563636e8e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.594+0000 7f9e2fe00640 1 -- 172.21.15.111:0/1333666053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563636e8ec30 msgr2=0x563636e910f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.594+0000 7f9e2fe00640 1 --2- 172.21.15.111:0/1333666053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563636e8ec30 0x563636e910f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.594+0000 7f9e2fe00640 1 -- 172.21.15.111:0/1333666053 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563636b01210 con 0x563636e89260 2024-09-15T23:09:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.594+0000 7f9e2fe00640 1 --2- 172.21.15.111:0/1333666053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563636e89260 0x563636e89660 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9e18031610 tx=0x7f9e18000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdb1430baa8fa74d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:15.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.594+0000 7f9e2f400640 1 --2- 172.21.15.111:0/1503679120 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9e140a3580 0x7f9e140a5a40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9e20005e10 tx=0x7f9e2000af90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.595+0000 7f9e25600640 1 -- 172.21.15.111:0/1333666053 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1803e030 con 0x563636e89260 2024-09-15T23:09:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.595+0000 7f9e25600640 1 -- 172.21.15.111:0/1333666053 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e18032d60 con 0x563636e89260 2024-09-15T23:09:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.595+0000 7f9e39df2740 1 -- 172.21.15.111:0/1333666053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563636e89260 msgr2=0x563636e89660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:15.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.595+0000 7f9e25600640 1 -- 172.21.15.111:0/1333666053 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1803e030 con 0x563636e89260 2024-09-15T23:09:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.595+0000 7f9e39df2740 1 --2- 172.21.15.111:0/1333666053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563636e89260 0x563636e89660 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9e18031610 tx=0x7f9e18000bc0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.595+0000 7f9e39df2740 1 -- 172.21.15.111:0/1333666053 shutdown_connections 2024-09-15T23:09:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.595+0000 7f9e39df2740 1 --2- 172.21.15.111:0/1333666053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563636e8ec30 0x563636e910f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.595+0000 7f9e39df2740 1 --2- 172.21.15.111:0/1333666053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563636e89ba0 0x563636e8e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.595+0000 7f9e39df2740 1 --2- 172.21.15.111:0/1333666053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563636e89260 0x563636e89660 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.595+0000 7f9e39df2740 1 -- 172.21.15.111:0/1333666053 >> 172.21.15.111:0/1333666053 conn(0x563636e848d0 msgr2=0x563636e84cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.595+0000 7f9e39df2740 1 -- 172.21.15.111:0/1333666053 shutdown_connections 2024-09-15T23:09:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.595+0000 7f9e39df2740 1 -- 172.21.15.111:0/1333666053 wait complete. 2024-09-15T23:09:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.595+0000 7f9e39df2740 1 -- start start 2024-09-15T23:09:15.599 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.595+0000 7f9e39df2740 1 Processor -- start 2024-09-15T23:09:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.598+0000 7f9e39df2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563636e89260 0x563636e933a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.598+0000 7f9e2fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563636e89260 0x563636e933a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.598+0000 7f9e2fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563636e89260 0x563636e933a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:34868/0 (socket says 172.21.15.111:34868) 2024-09-15T23:09:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.598+0000 7f9e2fe00640 1 -- 172.21.15.111:0/2360446921 learned_addr learned my addr 172.21.15.111:0/2360446921 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:09:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.598+0000 7f9e39df2740 1 --2- 172.21.15.111:0/2360446921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563636e89ba0 0x563636e938e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.598+0000 7f9e39df2740 1 --2- 172.21.15.111:0/2360446921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563636e8ec30 0x563636e9a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.598+0000 7f9e2f400640 1 --2- 172.21.15.111:0/2360446921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563636e89ba0 0x563636e938e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.598+0000 7f9e39df2740 1 -- 172.21.15.111:0/2360446921 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x563636e91bf0 con 0x563636e8ec30 2024-09-15T23:09:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.598+0000 7f9e39df2740 1 -- 172.21.15.111:0/2360446921 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563636d77090 con 0x563636e89ba0 2024-09-15T23:09:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.598+0000 7f9e39df2740 1 -- 172.21.15.111:0/2360446921 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563636e91d70 con 0x563636e89260 2024-09-15T23:09:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.598+0000 7f9e35000640 1 --2- 172.21.15.111:0/2360446921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563636e8ec30 0x563636e9a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.598+0000 7f9e2f400640 1 -- 172.21.15.111:0/2360446921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563636e89260 msgr2=0x563636e933a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.598+0000 7f9e2f400640 1 --2- 172.21.15.111:0/2360446921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563636e89260 0x563636e933a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.598+0000 7f9e2f400640 1 -- 172.21.15.111:0/2360446921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563636e8ec30 msgr2=0x563636e9a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.598+0000 7f9e2f400640 1 --2- 172.21.15.111:0/2360446921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563636e8ec30 0x563636e9a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.598+0000 7f9e2f400640 1 -- 172.21.15.111:0/2360446921 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563636bcd620 con 0x563636e89ba0 2024-09-15T23:09:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.598+0000 7f9e35000640 1 --2- 172.21.15.111:0/2360446921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563636e8ec30 0x563636e9a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:09:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.598+0000 7f9e39df2740 10 client.0 _ll_register_callbacks cb 0x563636dcbf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:09:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.598+0000 7f9e39df2740 1 -- 172.21.15.111:0/2360446921 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e18032d60 con 0x563636e89ba0 2024-09-15T23:09:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.598+0000 7f9e2f400640 1 --2- 172.21.15.111:0/2360446921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563636e89ba0 0x563636e938e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x563636e8a5d0 tx=0x7f9e200046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d2f1fc36ce18ec8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.598+0000 7f9e1e000640 10 client.26759 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:09:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.603+0000 7f9e1e000640 1 -- 172.21.15.111:0/2360446921 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e20033020 con 0x563636e89ba0 2024-09-15T23:09:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.603+0000 7f9e1e000640 1 -- 172.21.15.111:0/2360446921 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f9e20010b80 con 0x563636e89ba0 2024-09-15T23:09:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.603+0000 7f9e1e000640 1 -- 172.21.15.111:0/2360446921 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9e20010dc0 con 0x563636e89ba0 2024-09-15T23:09:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.603+0000 7f9e39df2740 10 client.26759 fetch_fsmap learned FSMap version 35 2024-09-15T23:09:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.603+0000 7f9e39df2740 1 -- 172.21.15.111:0/2360446921 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({fsmap=35}) -- 0x563636e9d690 con 0x563636e89ba0 2024-09-15T23:09:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.603+0000 7f9e1e000640 1 -- 172.21.15.111:0/2360446921 <== mon.2 v2:172.21.15.121:3300/0 4 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f9e2002f3f0 con 0x563636e89ba0 2024-09-15T23:09:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.603+0000 7f9e39df2740 10 client.26759 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:09:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.603+0000 7f9e39df2740 10 client.26759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:09:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.603+0000 7f9e39df2740 20 client.26759 populate_metadata read hostname 'smithi111' 2024-09-15T23:09:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.603+0000 7f9e39df2740 1 --2- 172.21.15.111:0/2360446921 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563636e9ece0 0x563636ea11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.603+0000 7f9e39df2740 4 client.26759 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:09:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.603+0000 7f9e39df2740 1 -- 172.21.15.111:0/2360446921 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563636dfb830 con 0x563636e9ece0 2024-09-15T23:09:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.603+0000 7f9e2fe00640 1 --2- 172.21.15.111:0/2360446921 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563636e9ece0 0x563636ea11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.605+0000 7f9e2fe00640 1 --2- 172.21.15.111:0/2360446921 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563636e9ece0 0x563636ea11a0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5a1c71aa9a8d43f server_cookie=5686cdbffe452cb in_seq=0 out_seq=0 2024-09-15T23:09:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.605+0000 7f9e1e000640 10 client.26759 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:09:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.605+0000 7f9e1e000640 1 -- 172.21.15.111:0/2360446921 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563636dfb830 con 0x563636e9ece0 2024-09-15T23:09:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.605+0000 7f9e1e000640 10 client.26759 handle_command_reply: tid=0 2024-09-15T23:09:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.605+0000 7f9e39df2740 1 client.26759 shutdown 2024-09-15T23:09:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.606+0000 7f9e39df2740 1 -- 172.21.15.111:0/2360446921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563636e89ba0 msgr2=0x563636e938e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.606+0000 7f9e39df2740 1 --2- 172.21.15.111:0/2360446921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563636e89ba0 0x563636e938e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x563636e8a5d0 tx=0x7f9e200046c0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.606+0000 7f9e39df2740 1 -- 172.21.15.111:0/2360446921 shutdown_connections 2024-09-15T23:09:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.606+0000 7f9e39df2740 1 --2- 172.21.15.111:0/2360446921 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563636e9ece0 0x563636ea11a0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.606+0000 7f9e39df2740 1 --2- 172.21.15.111:0/2360446921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563636e8ec30 0x563636e9a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.606+0000 7f9e39df2740 1 --2- 172.21.15.111:0/2360446921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563636e89ba0 0x563636e938e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.606+0000 7f9e39df2740 1 --2- 172.21.15.111:0/2360446921 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563636e89260 0x563636e933a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.606+0000 7f9e39df2740 1 -- 172.21.15.111:0/2360446921 >> 172.21.15.111:0/2360446921 conn(0x563636e848d0 msgr2=0x563636e860e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.606+0000 7f9e1e000640 0 client.26759 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:09:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.606+0000 7f9e39df2740 1 -- 172.21.15.111:0/2360446921 shutdown_connections 2024-09-15T23:09:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.606+0000 7f9e39df2740 1 -- 172.21.15.111:0/2360446921 wait complete. 2024-09-15T23:09:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.607+0000 7f9e39df2740 20 client.26759 trim_cache size 0 max 16384 2024-09-15T23:09:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e39df2740 1 Processor -- start 2024-09-15T23:09:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e39df2740 1 -- start start 2024-09-15T23:09:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e39df2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563636e89260 0x563636e85e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e39df2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563636e89ba0 0x563636e86370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e39df2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563636e8ec30 0x563636e9a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e39df2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c010ba0 con 0x563636e8ec30 2024-09-15T23:09:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e39df2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c00e470 con 0x563636e89ba0 2024-09-15T23:09:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e39df2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c00c2c0 con 0x563636e89260 2024-09-15T23:09:15.628 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e2fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563636e89260 0x563636e85e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e2fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563636e89260 0x563636e85e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:34884/0 (socket says 172.21.15.111:34884) 2024-09-15T23:09:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e2fe00640 1 -- 172.21.15.111:0/1573777408 learned_addr learned my addr 172.21.15.111:0/1573777408 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:09:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e2f400640 1 --2- 172.21.15.111:0/1573777408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563636e89ba0 0x563636e86370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e35000640 1 --2- 172.21.15.111:0/1573777408 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563636e8ec30 0x563636e9a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e2fe00640 1 -- 172.21.15.111:0/1573777408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563636e89ba0 msgr2=0x563636e86370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e2fe00640 1 --2- 172.21.15.111:0/1573777408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563636e89ba0 0x563636e86370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e2fe00640 1 -- 172.21.15.111:0/1573777408 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563636e8ec30 msgr2=0x563636e9a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e2fe00640 1 --2- 172.21.15.111:0/1573777408 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563636e8ec30 0x563636e9a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e2fe00640 1 -- 172.21.15.111:0/1573777408 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563636e92420 con 0x563636e89260 2024-09-15T23:09:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e2fe00640 1 --2- 172.21.15.111:0/1573777408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563636e89260 0x563636e85e30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9e180314e0 tx=0x7f9e1802a8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4eeecb1002f13a30 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e24c00640 1 -- 172.21.15.111:0/1573777408 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e18051070 con 0x563636e89260 2024-09-15T23:09:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e24c00640 1 -- 172.21.15.111:0/1573777408 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e180055f0 con 0x563636e89260 2024-09-15T23:09:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e24c00640 1 -- 172.21.15.111:0/1573777408 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1803e040 con 0x563636e89260 2024-09-15T23:09:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e39df2740 1 -- 172.21.15.111:0/1573777408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563636e89260 msgr2=0x563636e85e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e39df2740 1 --2- 172.21.15.111:0/1573777408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563636e89260 0x563636e85e30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9e180314e0 tx=0x7f9e1802a8f0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e39df2740 1 -- 172.21.15.111:0/1573777408 shutdown_connections 2024-09-15T23:09:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e39df2740 1 --2- 172.21.15.111:0/1573777408 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563636e8ec30 0x563636e9a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e39df2740 1 --2- 172.21.15.111:0/1573777408 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563636e89ba0 0x563636e86370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e39df2740 1 --2- 172.21.15.111:0/1573777408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563636e89260 0x563636e85e30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e39df2740 1 -- 172.21.15.111:0/1573777408 >> 172.21.15.111:0/1573777408 conn(0x563636e848d0 msgr2=0x563636e85720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e39df2740 1 -- 172.21.15.111:0/1573777408 shutdown_connections 2024-09-15T23:09:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e39df2740 1 -- 172.21.15.111:0/1573777408 wait complete. 2024-09-15T23:09:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e39df2740 1 -- start start 2024-09-15T23:09:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.625+0000 7f9e39df2740 1 Processor -- start 2024-09-15T23:09:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e39df2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563636e89260 0x563636e978a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e39df2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563636e89ba0 0x563636e97de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e39df2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563636e8ec30 0x563636f07a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e39df2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c00c930 con 0x563636e8ec30 2024-09-15T23:09:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e39df2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c00dda0 con 0x563636e89260 2024-09-15T23:09:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e39df2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9e0c00d000 con 0x563636e89ba0 2024-09-15T23:09:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e39df2740 10 client.0 _ll_register_callbacks cb 0x7f9e20016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:09:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e2f400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563636e89ba0 0x563636e97de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e2fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563636e89260 0x563636e978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e2f400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563636e89ba0 0x563636e97de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:34896/0 (socket says 172.21.15.111:34896) 2024-09-15T23:09:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e2f400640 1 -- 172.21.15.111:0/1887747737 learned_addr learned my addr 172.21.15.111:0/1887747737 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:09:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e35000640 1 --2- 172.21.15.111:0/1887747737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563636e8ec30 0x563636f07a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e2fe00640 1 -- 172.21.15.111:0/1887747737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563636e89ba0 msgr2=0x563636e97de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e2fe00640 1 --2- 172.21.15.111:0/1887747737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563636e89ba0 0x563636e97de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e2fe00640 1 -- 172.21.15.111:0/1887747737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563636e8ec30 msgr2=0x563636f07a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e2fe00640 1 --2- 172.21.15.111:0/1887747737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563636e8ec30 0x563636f07a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e2fe00640 1 -- 172.21.15.111:0/1887747737 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563636f07fb0 con 0x563636e89260 2024-09-15T23:09:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e35000640 1 --2- 172.21.15.111:0/1887747737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563636e8ec30 0x563636f07a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:09:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e2fe00640 1 --2- 172.21.15.111:0/1887747737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563636e89260 0x563636e978a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9e180057c0 tx=0x7f9e18031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f3070952d9e21e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e1e000640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:09:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e1e000640 1 -- 172.21.15.111:0/1887747737 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e18051040 con 0x563636e89260 2024-09-15T23:09:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e1e000640 1 -- 172.21.15.111:0/1887747737 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f9e18004bf0 con 0x563636e89260 2024-09-15T23:09:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e39df2740 1 -- 172.21.15.111:0/1887747737 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e180055f0 con 0x563636e89260 2024-09-15T23:09:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e1e000640 1 -- 172.21.15.111:0/1887747737 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f9e1803e040 con 0x563636e89260 2024-09-15T23:09:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e39df2740 10 client.26765 fetch_fsmap learned FSMap version 35 2024-09-15T23:09:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e39df2740 1 -- 172.21.15.111:0/1887747737 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563636f09270 con 0x563636e89260 2024-09-15T23:09:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e1e000640 1 -- 172.21.15.111:0/1887747737 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e180055f0 con 0x563636e89260 2024-09-15T23:09:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e1e000640 1 -- 172.21.15.111:0/1887747737 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f9e18003680 con 0x563636e89260 2024-09-15T23:09:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e39df2740 10 client.26765 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:09:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e39df2740 10 client.26765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:09:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e39df2740 20 client.26765 populate_metadata read hostname 'smithi111' 2024-09-15T23:09:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e39df2740 1 --2- 172.21.15.111:0/1887747737 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563636f09770 0x563636f0bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e39df2740 4 client.26765 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:09:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e39df2740 1 -- 172.21.15.111:0/1887747737 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563636df5b80 con 0x563636f09770 2024-09-15T23:09:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e2f400640 1 --2- 172.21.15.111:0/1887747737 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563636f09770 0x563636f0bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e2f400640 1 --2- 172.21.15.111:0/1887747737 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563636f09770 0x563636f0bb60 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5766725eb0250cb6 server_cookie=45e0126ddc34ec2 in_seq=0 out_seq=0 2024-09-15T23:09:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.630+0000 7f9e1e000640 10 client.26765 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:09:15.690 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199035 17.55 MB/sec execute 463 sec latency 5950.819 ms 2024-09-15T23:09:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.855+0000 7f9e1e000640 1 -- 172.21.15.111:0/1887747737 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563636df5b80 con 0x563636f09770 2024-09-15T23:09:15.857 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.855+0000 7f9e1e000640 10 client.26765 handle_command_reply: tid=0 2024-09-15T23:09:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.855+0000 7f9e39df2740 1 client.26765 shutdown 2024-09-15T23:09:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.855+0000 7f9e39df2740 1 -- 172.21.15.111:0/1887747737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563636e89260 msgr2=0x563636e978a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.855+0000 7f9e39df2740 1 --2- 172.21.15.111:0/1887747737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563636e89260 0x563636e978a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9e180057c0 tx=0x7f9e18031400 comp rx=0 tx=0).stop 2024-09-15T23:09:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.855+0000 7f9e39df2740 1 -- 172.21.15.111:0/1887747737 shutdown_connections 2024-09-15T23:09:15.858 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.855+0000 7f9e39df2740 1 --2- 172.21.15.111:0/1887747737 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x563636f09770 0x563636f0bb60 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.855+0000 7f9e39df2740 1 --2- 172.21.15.111:0/1887747737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x563636e8ec30 0x563636f07a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.855+0000 7f9e39df2740 1 --2- 172.21.15.111:0/1887747737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563636e89ba0 0x563636e97de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.855+0000 7f9e39df2740 1 --2- 172.21.15.111:0/1887747737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563636e89260 0x563636e978a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.855+0000 7f9e39df2740 1 -- 172.21.15.111:0/1887747737 >> 172.21.15.111:0/1887747737 conn(0x563636e848d0 msgr2=0x563636e85f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.855+0000 7f9e1e000640 0 client.26765 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:09:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.855+0000 7f9e39df2740 1 -- 172.21.15.111:0/1887747737 shutdown_connections 2024-09-15T23:09:15.859 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.855+0000 7f9e39df2740 1 -- 172.21.15.111:0/1887747737 wait complete. 2024-09-15T23:09:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.855+0000 7f9e39df2740 20 client.26765 trim_cache size 0 max 16384 2024-09-15T23:09:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:09:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:09:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:09:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:09:15.860 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.855+0000 7f9e35a00640 1 -- 172.21.15.111:0/1503679120 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9e140a3580 msgr2=0x7f9e140a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.855+0000 7f9e35a00640 1 --2- 172.21.15.111:0/1503679120 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9e140a3580 0x7f9e140a5a40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9e20005e10 tx=0x7f9e2000af90 comp rx=0 tx=0).stop 2024-09-15T23:09:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.855+0000 7f9e35a00640 1 -- 172.21.15.111:0/1503679120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e301054b0 msgr2=0x7f9e301a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.855+0000 7f9e35a00640 1 --2- 172.21.15.111:0/1503679120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e301054b0 0x7f9e301a5780 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9e2800eb10 tx=0x7f9e2800c2d0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.855+0000 7f9e35a00640 1 -- 172.21.15.111:0/1503679120 shutdown_connections 2024-09-15T23:09:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.855+0000 7f9e35a00640 1 --2- 172.21.15.111:0/1503679120 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f9e140a3580 0x7f9e140a5a40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.855+0000 7f9e35a00640 1 --2- 172.21.15.111:0/1503679120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e301054b0 0x7f9e301a5780 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.855+0000 7f9e35a00640 1 --2- 172.21.15.111:0/1503679120 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e30102b00 0x7f9e3019e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.855+0000 7f9e35a00640 1 --2- 172.21.15.111:0/1503679120 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e301001b0 0x7f9e3019e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.855+0000 7f9e35a00640 1 -- 172.21.15.111:0/1503679120 >> 172.21.15.111:0/1503679120 conn(0x7f9e300fbfa0 msgr2=0x7f9e301034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.855+0000 7f9e35a00640 1 -- 172.21.15.111:0/1503679120 shutdown_connections 2024-09-15T23:09:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:15.855+0000 7f9e35a00640 1 -- 172.21.15.111:0/1503679120 wait complete. 2024-09-15T23:09:16.210 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:16 smithi111 ceph-mon[30629]: pgmap v1051: 97 pgs: 97 active+clean; 610 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.1 MiB/s wr, 490 op/s 2024-09-15T23:09:16.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:16 smithi192 ceph-mon[35427]: pgmap v1051: 97 pgs: 97 active+clean; 610 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.1 MiB/s wr, 490 op/s 2024-09-15T23:09:16.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:16 smithi121 ceph-mon[34815]: pgmap v1051: 97 pgs: 97 active+clean; 610 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.1 MiB/s wr, 490 op/s 2024-09-15T23:09:16.690 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199035 17.52 MB/sec execute 464 sec latency 6950.875 ms 2024-09-15T23:09:17.690 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199046 17.48 MB/sec execute 465 sec latency 7646.449 ms 2024-09-15T23:09:18.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:18 smithi192 ceph-mon[35427]: pgmap v1052: 97 pgs: 97 active+clean; 615 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.9 MiB/s wr, 434 op/s 2024-09-15T23:09:18.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:18 smithi121 ceph-mon[34815]: pgmap v1052: 97 pgs: 97 active+clean; 615 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.9 MiB/s wr, 434 op/s 2024-09-15T23:09:18.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:18 smithi111 ceph-mon[30629]: pgmap v1052: 97 pgs: 97 active+clean; 615 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.9 MiB/s wr, 434 op/s 2024-09-15T23:09:18.690 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199047 17.44 MB/sec execute 466 sec latency 8646.491 ms 2024-09-15T23:09:19.690 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199049 17.40 MB/sec execute 467 sec latency 9646.534 ms 2024-09-15T23:09:20.239 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:20 smithi111 ceph-mon[30629]: pgmap v1053: 97 pgs: 97 active+clean; 612 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.0 MiB/s wr, 410 op/s 2024-09-15T23:09:20.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:20 smithi121 ceph-mon[34815]: pgmap v1053: 97 pgs: 97 active+clean; 612 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.0 MiB/s wr, 410 op/s 2024-09-15T23:09:20.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:20 smithi192 ceph-mon[35427]: pgmap v1053: 97 pgs: 97 active+clean; 612 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.0 MiB/s wr, 410 op/s 2024-09-15T23:09:20.690 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199049 17.37 MB/sec execute 468 sec latency 10646.598 ms 2024-09-15T23:09:21.690 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199049 17.33 MB/sec execute 469 sec latency 11646.649 ms 2024-09-15T23:09:22.221 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:22 smithi111 ceph-mon[30629]: pgmap v1054: 97 pgs: 97 active+clean; 611 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.5 MiB/s wr, 335 op/s 2024-09-15T23:09:22.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:22 smithi121 ceph-mon[34815]: pgmap v1054: 97 pgs: 97 active+clean; 611 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.5 MiB/s wr, 335 op/s 2024-09-15T23:09:22.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:22 smithi192 ceph-mon[35427]: pgmap v1054: 97 pgs: 97 active+clean; 611 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.5 MiB/s wr, 335 op/s 2024-09-15T23:09:22.690 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199049 17.29 MB/sec execute 470 sec latency 12646.690 ms 2024-09-15T23:09:23.690 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199054 17.26 MB/sec execute 471 sec latency 13449.529 ms 2024-09-15T23:09:24.237 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:24 smithi111 ceph-mon[30629]: pgmap v1055: 97 pgs: 97 active+clean; 609 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.9 MiB/s wr, 285 op/s 2024-09-15T23:09:24.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:24 smithi121 ceph-mon[34815]: pgmap v1055: 97 pgs: 97 active+clean; 609 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.9 MiB/s wr, 285 op/s 2024-09-15T23:09:24.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:24 smithi192 ceph-mon[35427]: pgmap v1055: 97 pgs: 97 active+clean; 609 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.9 MiB/s wr, 285 op/s 2024-09-15T23:09:24.690 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199061 17.22 MB/sec execute 472 sec latency 7655.641 ms 2024-09-15T23:09:25.338 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:25 smithi111 ceph-mon[30629]: pgmap v1056: 97 pgs: 97 active+clean; 608 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 6.3 MiB/s wr, 290 op/s 2024-09-15T23:09:25.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:25 smithi192 ceph-mon[35427]: pgmap v1056: 97 pgs: 97 active+clean; 608 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 6.3 MiB/s wr, 290 op/s 2024-09-15T23:09:25.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:25 smithi121 ceph-mon[34815]: pgmap v1056: 97 pgs: 97 active+clean; 608 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 6.3 MiB/s wr, 290 op/s 2024-09-15T23:09:25.690 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199066 17.19 MB/sec execute 473 sec latency 8655.725 ms 2024-09-15T23:09:26.690 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199069 17.15 MB/sec execute 474 sec latency 8741.778 ms 2024-09-15T23:09:27.690 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199070 17.11 MB/sec execute 475 sec latency 2350.469 ms 2024-09-15T23:09:28.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:28 smithi192 ceph-mon[35427]: pgmap v1057: 97 pgs: 97 active+clean; 606 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 6.0 MiB/s wr, 278 op/s 2024-09-15T23:09:28.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:28 smithi121 ceph-mon[34815]: pgmap v1057: 97 pgs: 97 active+clean; 606 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 6.0 MiB/s wr, 278 op/s 2024-09-15T23:09:28.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:28 smithi111 ceph-mon[30629]: pgmap v1057: 97 pgs: 97 active+clean; 606 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 6.0 MiB/s wr, 278 op/s 2024-09-15T23:09:28.691 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199070 17.08 MB/sec execute 476 sec latency 3350.550 ms 2024-09-15T23:09:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.988+0000 7f5cdd200640 1 -- 172.21.15.111:0/4069302772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5cd8103050 msgr2=0x7f5cd8103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.988+0000 7f5cdd200640 1 --2- 172.21.15.111:0/4069302772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5cd8103050 0x7f5cd8103450 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8009860 tx=0x7f5cc802a340 comp rx=0 tx=0).stop 2024-09-15T23:09:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.988+0000 7f5cdd200640 1 -- 172.21.15.111:0/4069302772 shutdown_connections 2024-09-15T23:09:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.988+0000 7f5cdd200640 1 --2- 172.21.15.111:0/4069302772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cd8101f40 0x7f5cd8109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.988+0000 7f5cdd200640 1 --2- 172.21.15.111:0/4069302772 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cd8101580 0x7f5cd8101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.988+0000 7f5cdd200640 1 --2- 172.21.15.111:0/4069302772 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5cd8103050 0x7f5cd8103450 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.988+0000 7f5cdd200640 1 -- 172.21.15.111:0/4069302772 >> 172.21.15.111:0/4069302772 conn(0x7f5cd80fd0d0 msgr2=0x7f5cd80ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.988+0000 7f5cdd200640 1 -- 172.21.15.111:0/4069302772 shutdown_connections 2024-09-15T23:09:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.988+0000 7f5cdd200640 1 -- 172.21.15.111:0/4069302772 wait complete. 2024-09-15T23:09:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.989+0000 7f5cdd200640 1 Processor -- start 2024-09-15T23:09:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.989+0000 7f5cdd200640 1 -- start start 2024-09-15T23:09:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.989+0000 7f5cdd200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cd8101580 0x7f5cd8199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.989+0000 7f5cdd200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5cd8101f40 0x7f5cd819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.989+0000 7f5cdd200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cd8103050 0x7f5cd81a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.990+0000 7f5cdd200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd810d6f0 con 0x7f5cd8101580 2024-09-15T23:09:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.990+0000 7f5cdd200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd810d570 con 0x7f5cd8101f40 2024-09-15T23:09:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.990+0000 7f5cdd200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd810d870 con 0x7f5cd8103050 2024-09-15T23:09:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.990+0000 7f5cd7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cd8101580 0x7f5cd8199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.990+0000 7f5cd7400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cd8101580 0x7f5cd8199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59684/0 (socket says 172.21.15.111:59684) 2024-09-15T23:09:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.990+0000 7f5cd7400640 1 -- 172.21.15.111:0/169518001 learned_addr learned my addr 172.21.15.111:0/169518001 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:09:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.990+0000 7f5cd6a00640 1 --2- 172.21.15.111:0/169518001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5cd8101f40 0x7f5cd819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.990+0000 7f5cd7e00640 1 --2- 172.21.15.111:0/169518001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cd8103050 0x7f5cd81a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.990+0000 7f5cd7400640 1 -- 172.21.15.111:0/169518001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cd8103050 msgr2=0x7f5cd81a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.990+0000 7f5cd7400640 1 --2- 172.21.15.111:0/169518001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cd8103050 0x7f5cd81a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:28.993 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.990+0000 7f5cd7400640 1 -- 172.21.15.111:0/169518001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5cd8101f40 msgr2=0x7f5cd819a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.990+0000 7f5cd7400640 1 --2- 172.21.15.111:0/169518001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5cd8101f40 0x7f5cd819a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.990+0000 7f5cd7400640 1 -- 172.21.15.111:0/169518001 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd81a1c10 con 0x7f5cd8101580 2024-09-15T23:09:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.990+0000 7f5cd7400640 1 --2- 172.21.15.111:0/169518001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cd8101580 0x7f5cd8199ff0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8035040 tx=0x7f5cc8033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.991+0000 7f5ccfe00640 1 -- 172.21.15.111:0/169518001 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8037040 con 0x7f5cd8101580 2024-09-15T23:09:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.991+0000 7f5ccfe00640 1 -- 172.21.15.111:0/169518001 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cc8030850 con 0x7f5cd8101580 2024-09-15T23:09:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.991+0000 7f5cdd200640 1 -- 172.21.15.111:0/169518001 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cd81a1ea0 con 0x7f5cd8101580 2024-09-15T23:09:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.991+0000 7f5ccfe00640 1 -- 172.21.15.111:0/169518001 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8004920 con 0x7f5cd8101580 2024-09-15T23:09:28.994 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.991+0000 7f5cdd200640 1 -- 172.21.15.111:0/169518001 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cd81a22b0 con 0x7f5cd8101580 2024-09-15T23:09:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.994+0000 7f5ce168e740 1 Processor -- start 2024-09-15T23:09:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.994+0000 7f5ce168e740 1 -- start start 2024-09-15T23:09:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.994+0000 7f5ccfe00640 1 -- 172.21.15.111:0/169518001 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5cc8004050 con 0x7f5cd8101580 2024-09-15T23:09:28.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.994+0000 7f5ce168e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5562ee5a9260 0x5562ee5a9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:28.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.994+0000 7f5ce168e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5562ee5a9ba0 0x5562ee5ae660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:28.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.994+0000 7f5ce168e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5562ee5aec30 0x5562ee5b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:28.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.994+0000 7f5ce168e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5562ee50e100 con 0x5562ee5a9260 2024-09-15T23:09:28.998 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.994+0000 7f5ce168e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5562ee4b61b0 con 0x5562ee5aec30 2024-09-15T23:09:28.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.994+0000 7f5ce168e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5562ee50a0c0 con 0x5562ee5a9ba0 2024-09-15T23:09:28.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.994+0000 7f5cd7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5562ee5a9260 0x5562ee5a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:28.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.994+0000 7f5cd6a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5562ee5aec30 0x5562ee5b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:28.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.994+0000 7f5cd7e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5562ee5a9260 0x5562ee5a9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59686/0 (socket says 172.21.15.111:59686) 2024-09-15T23:09:28.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.994+0000 7f5cd7e00640 1 -- 172.21.15.111:0/3002119511 learned_addr learned my addr 172.21.15.111:0/3002119511 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:09:28.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.995+0000 7f5cd7400640 1 --2- 172.21.15.111:0/3002119511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5562ee5a9ba0 0x5562ee5ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:28.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.995+0000 7f5cd6a00640 1 -- 172.21.15.111:0/3002119511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5562ee5a9ba0 msgr2=0x5562ee5ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:28.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.995+0000 7f5cd6a00640 1 --2- 172.21.15.111:0/3002119511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5562ee5a9ba0 0x5562ee5ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:28.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.995+0000 7f5cd6a00640 1 -- 172.21.15.111:0/3002119511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5562ee5a9260 msgr2=0x5562ee5a9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:28.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.995+0000 7f5cd6a00640 1 --2- 172.21.15.111:0/3002119511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5562ee5a9260 0x5562ee5a9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:28.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.995+0000 7f5cd6a00640 1 -- 172.21.15.111:0/3002119511 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562ee221210 con 0x5562ee5aec30 2024-09-15T23:09:28.999 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.995+0000 7f5cd7e00640 1 --2- 172.21.15.111:0/3002119511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5562ee5a9260 0x5562ee5a9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:09:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.995+0000 7f5cd6a00640 1 --2- 172.21.15.111:0/3002119511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5562ee5aec30 0x5562ee5b10f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4009bd0 tx=0x7f5cc4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=796daf7dba83f939 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.995+0000 7f5ccfe00640 1 --2- 172.21.15.111:0/169518001 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5cbc0a3240 0x7f5cbc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.995+0000 7f5cccc00640 1 -- 172.21.15.111:0/3002119511 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4004230 con 0x5562ee5aec30 2024-09-15T23:09:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.996+0000 7f5cccc00640 1 -- 172.21.15.111:0/3002119511 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cc4009e40 con 0x5562ee5aec30 2024-09-15T23:09:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.996+0000 7f5cccc00640 1 -- 172.21.15.111:0/3002119511 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4004d70 con 0x5562ee5aec30 2024-09-15T23:09:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.996+0000 7f5ccfe00640 1 -- 172.21.15.111:0/169518001 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f5cc80ef760 con 0x7f5cd8101580 2024-09-15T23:09:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.996+0000 7f5cd7e00640 1 --2- 172.21.15.111:0/169518001 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5cbc0a3240 0x7f5cbc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.996+0000 7f5ce168e740 1 -- 172.21.15.111:0/3002119511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5562ee5aec30 msgr2=0x5562ee5b10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.996+0000 7f5ce168e740 1 --2- 172.21.15.111:0/3002119511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5562ee5aec30 0x5562ee5b10f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4009bd0 tx=0x7f5cc4011810 comp rx=0 tx=0).stop 2024-09-15T23:09:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.996+0000 7f5cd7e00640 1 --2- 172.21.15.111:0/169518001 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5cbc0a3240 0x7f5cbc0a5700 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0005c40 tx=0x7f5cd0010040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.996+0000 7f5ce168e740 1 -- 172.21.15.111:0/3002119511 shutdown_connections 2024-09-15T23:09:29.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.996+0000 7f5ce168e740 1 --2- 172.21.15.111:0/3002119511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5562ee5aec30 0x5562ee5b10f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:29.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.996+0000 7f5ce168e740 1 --2- 172.21.15.111:0/3002119511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5562ee5a9ba0 0x5562ee5ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:29.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.996+0000 7f5ce168e740 1 --2- 172.21.15.111:0/3002119511 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5562ee5a9260 0x5562ee5a9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:29.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.996+0000 7f5ce168e740 1 -- 172.21.15.111:0/3002119511 >> 172.21.15.111:0/3002119511 conn(0x5562ee5a48d0 msgr2=0x5562ee5a4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:29.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.996+0000 7f5ce168e740 1 -- 172.21.15.111:0/3002119511 shutdown_connections 2024-09-15T23:09:29.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.996+0000 7f5ce168e740 1 -- 172.21.15.111:0/3002119511 wait complete. 2024-09-15T23:09:29.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.997+0000 7f5ce168e740 1 -- start start 2024-09-15T23:09:29.001 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.997+0000 7f5ce168e740 1 Processor -- start 2024-09-15T23:09:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.997+0000 7f5ce168e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5562ee5a9260 0x5562ee5b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:29.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.997+0000 7f5ce168e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5562ee5a9ba0 0x5562ee5b3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:29.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.997+0000 7f5ce168e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5562ee5aec30 0x5562ee5ba8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:29.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.997+0000 7f5ce168e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5562ee5b1c00 con 0x5562ee5a9ba0 2024-09-15T23:09:29.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.997+0000 7f5ce168e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5562ee497090 con 0x5562ee5a9260 2024-09-15T23:09:29.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.998+0000 7f5ce168e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5562ee5b1d80 con 0x5562ee5aec30 2024-09-15T23:09:29.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.998+0000 7f5ce168e740 10 client.0 _ll_register_callbacks cb 0x5562ee4ebf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:09:29.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.998+0000 7f5cd6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5562ee5a9ba0 0x5562ee5b3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:29.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.998+0000 7f5cd6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5562ee5a9ba0 0x5562ee5b3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59700/0 (socket says 172.21.15.111:59700) 2024-09-15T23:09:29.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.998+0000 7f5cd6a00640 1 -- 172.21.15.111:0/1763293195 learned_addr learned my addr 172.21.15.111:0/1763293195 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:09:29.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.998+0000 7f5cd7400640 1 --2- 172.21.15.111:0/1763293195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5562ee5a9260 0x5562ee5b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:29.004 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.998+0000 7f5cd6a00640 1 -- 172.21.15.111:0/1763293195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5562ee5aec30 msgr2=0x5562ee5ba8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:09:29.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.998+0000 7f5cd6a00640 1 --2- 172.21.15.111:0/1763293195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5562ee5aec30 0x5562ee5ba8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:29.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.998+0000 7f5cd6a00640 1 -- 172.21.15.111:0/1763293195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5562ee5a9260 msgr2=0x5562ee5b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:29.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.998+0000 7f5cd6a00640 1 --2- 172.21.15.111:0/1763293195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5562ee5a9260 0x5562ee5b33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:29.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.998+0000 7f5cd6a00640 1 -- 172.21.15.111:0/1763293195 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562ee2ed620 con 0x5562ee5a9ba0 2024-09-15T23:09:29.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.998+0000 7f5cd7400640 1 --2- 172.21.15.111:0/1763293195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5562ee5a9260 0x5562ee5b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:09:29.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.999+0000 7f5cd6a00640 1 --2- 172.21.15.111:0/1763293195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5562ee5a9ba0 0x5562ee5b3920 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5cc40096e0 tx=0x7f5cc40038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ddcda4505a1137 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:29.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.999+0000 7f5cc1600640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:09:29.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.999+0000 7f5cc1600640 1 -- 172.21.15.111:0/1763293195 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4027020 con 0x5562ee5a9ba0 2024-09-15T23:09:29.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.999+0000 7f5ce168e740 1 -- 172.21.15.111:0/1763293195 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5562ee51b830 con 0x5562ee5a9ba0 2024-09-15T23:09:29.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.999+0000 7f5cc1600640 1 -- 172.21.15.111:0/1763293195 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f5cc4003cc0 con 0x5562ee5a9ba0 2024-09-15T23:09:29.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.999+0000 7f5cc1600640 1 -- 172.21.15.111:0/1763293195 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5cc401e040 con 0x5562ee5a9ba0 2024-09-15T23:09:29.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.999+0000 7f5ce168e740 10 client.16824 fetch_fsmap learned FSMap version 35 2024-09-15T23:09:29.005 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.999+0000 7f5ce168e740 1 -- 172.21.15.111:0/1763293195 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5562ee5bd6e0 con 0x5562ee5a9ba0 2024-09-15T23:09:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:28.999+0000 7f5cc1600640 1 -- 172.21.15.111:0/1763293195 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5562ee51b830 con 0x5562ee5a9ba0 2024-09-15T23:09:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.000+0000 7f5cc1600640 1 -- 172.21.15.111:0/1763293195 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f5cc4018070 con 0x5562ee5a9ba0 2024-09-15T23:09:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.000+0000 7f5ce168e740 10 client.16824 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:09:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.000+0000 7f5ce168e740 10 client.16824 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:09:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.000+0000 7f5ce168e740 20 client.16824 populate_metadata read hostname 'smithi111' 2024-09-15T23:09:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.000+0000 7f5ce168e740 1 --2- 172.21.15.111:0/1763293195 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5562ee5bebb0 0x5562ee5c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.000+0000 7f5ce168e740 4 client.16824 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:09:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.000+0000 7f5ce168e740 1 -- 172.21.15.111:0/1763293195 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562ee515b80 con 0x5562ee5bebb0 2024-09-15T23:09:29.043 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.042+0000 7f5cd7400640 1 --2- 172.21.15.111:0/1763293195 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5562ee5bebb0 0x5562ee5c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.066+0000 7f5cd7400640 1 --2- 172.21.15.111:0/1763293195 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5562ee5bebb0 0x5562ee5c0fa0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea87aef7b9c4fcbf server_cookie=52a2f334c58fc7af in_seq=0 out_seq=0 2024-09-15T23:09:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.066+0000 7f5cc1600640 10 client.16824 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:09:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.066+0000 7f5cc1600640 1 -- 172.21.15.111:0/1763293195 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5562ee515b80 con 0x5562ee5bebb0 2024-09-15T23:09:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.066+0000 7f5cc1600640 10 client.16824 handle_command_reply: tid=0 2024-09-15T23:09:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.067+0000 7f5ce168e740 1 client.16824 shutdown 2024-09-15T23:09:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.067+0000 7f5ce168e740 1 -- 172.21.15.111:0/1763293195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5562ee5a9ba0 msgr2=0x5562ee5b3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.067+0000 7f5ce168e740 1 --2- 172.21.15.111:0/1763293195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5562ee5a9ba0 0x5562ee5b3920 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5cc40096e0 tx=0x7f5cc40038a0 comp rx=0 tx=0).stop 2024-09-15T23:09:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.067+0000 7f5ce168e740 1 -- 172.21.15.111:0/1763293195 shutdown_connections 2024-09-15T23:09:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.067+0000 7f5ce168e740 1 --2- 172.21.15.111:0/1763293195 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5562ee5bebb0 0x5562ee5c0fa0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.067+0000 7f5ce168e740 1 --2- 172.21.15.111:0/1763293195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5562ee5aec30 0x5562ee5ba8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.067+0000 7f5ce168e740 1 --2- 172.21.15.111:0/1763293195 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5562ee5a9ba0 0x5562ee5b3920 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.067+0000 7f5cc1600640 0 client.16824 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:09:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.067+0000 7f5ce168e740 1 --2- 172.21.15.111:0/1763293195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5562ee5a9260 0x5562ee5b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.067+0000 7f5ce168e740 1 -- 172.21.15.111:0/1763293195 >> 172.21.15.111:0/1763293195 conn(0x5562ee5a48d0 msgr2=0x5562ee5a60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.067+0000 7f5ce168e740 1 -- 172.21.15.111:0/1763293195 shutdown_connections 2024-09-15T23:09:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.067+0000 7f5ce168e740 1 -- 172.21.15.111:0/1763293195 wait complete. 2024-09-15T23:09:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.067+0000 7f5ce168e740 20 client.16824 trim_cache size 0 max 16384 2024-09-15T23:09:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.082+0000 7f5ce168e740 1 Processor -- start 2024-09-15T23:09:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.082+0000 7f5ce168e740 1 -- start start 2024-09-15T23:09:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.082+0000 7f5ce168e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5562ee5a9260 0x5562ee5a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.082+0000 7f5ce168e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5562ee5a9ba0 0x5562ee5ba8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.082+0000 7f5ce168e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5562ee5aec30 0x5562ee5a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.082+0000 7f5ce168e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd81ad7b0 con 0x5562ee5a9ba0 2024-09-15T23:09:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.082+0000 7f5ce168e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd81ab080 con 0x5562ee5a9260 2024-09-15T23:09:29.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.082+0000 7f5ce168e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd81a8ed0 con 0x5562ee5aec30 2024-09-15T23:09:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.082+0000 7f5cd6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5562ee5a9ba0 0x5562ee5ba8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.082+0000 7f5cd7400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5562ee5a9260 0x5562ee5a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.082+0000 7f5cd6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5562ee5a9ba0 0x5562ee5ba8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59702/0 (socket says 172.21.15.111:59702) 2024-09-15T23:09:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.082+0000 7f5cd6a00640 1 -- 172.21.15.111:0/2574086905 learned_addr learned my addr 172.21.15.111:0/2574086905 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:09:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.082+0000 7f5cd7e00640 1 --2- 172.21.15.111:0/2574086905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5562ee5aec30 0x5562ee5a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.083+0000 7f5cd7400640 1 -- 172.21.15.111:0/2574086905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5562ee5aec30 msgr2=0x5562ee5a5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.083+0000 7f5cd7400640 1 --2- 172.21.15.111:0/2574086905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5562ee5aec30 0x5562ee5a5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.083+0000 7f5cd7400640 1 -- 172.21.15.111:0/2574086905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5562ee5a9ba0 msgr2=0x5562ee5ba8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.083+0000 7f5cd7400640 1 --2- 172.21.15.111:0/2574086905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5562ee5a9ba0 0x5562ee5ba8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.083+0000 7f5cd7400640 1 -- 172.21.15.111:0/2574086905 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562ee5a46f0 con 0x5562ee5a9260 2024-09-15T23:09:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.083+0000 7f5cd7e00640 1 --2- 172.21.15.111:0/2574086905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5562ee5aec30 0x5562ee5a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:09:29.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.083+0000 7f5cd7400640 1 --2- 172.21.15.111:0/2574086905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5562ee5a9260 0x5562ee5a3a10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x5562ee5aa5d0 tx=0x7f5cc80ed130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d4e56a60efc9f58 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.083+0000 7f5cc3e00640 1 -- 172.21.15.111:0/2574086905 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc80bc480 con 0x5562ee5a9260 2024-09-15T23:09:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.083+0000 7f5cc3e00640 1 -- 172.21.15.111:0/2574086905 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cc80bbd80 con 0x5562ee5a9260 2024-09-15T23:09:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.083+0000 7f5cc3e00640 1 -- 172.21.15.111:0/2574086905 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc80c4560 con 0x5562ee5a9260 2024-09-15T23:09:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.084+0000 7f5ce168e740 1 -- 172.21.15.111:0/2574086905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5562ee5a9260 msgr2=0x5562ee5a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.084+0000 7f5ce168e740 1 --2- 172.21.15.111:0/2574086905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5562ee5a9260 0x5562ee5a3a10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x5562ee5aa5d0 tx=0x7f5cc80ed130 comp rx=0 tx=0).stop 2024-09-15T23:09:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.084+0000 7f5ce168e740 1 -- 172.21.15.111:0/2574086905 shutdown_connections 2024-09-15T23:09:29.089 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.084+0000 7f5ce168e740 1 --2- 172.21.15.111:0/2574086905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5562ee5aec30 0x5562ee5a5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.084+0000 7f5ce168e740 1 --2- 172.21.15.111:0/2574086905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5562ee5a9ba0 0x5562ee5ba8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.084+0000 7f5ce168e740 1 --2- 172.21.15.111:0/2574086905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5562ee5a9260 0x5562ee5a3a10 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.084+0000 7f5ce168e740 1 -- 172.21.15.111:0/2574086905 >> 172.21.15.111:0/2574086905 conn(0x5562ee5a48d0 msgr2=0x5562ee5b27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.084+0000 7f5ce168e740 1 -- 172.21.15.111:0/2574086905 shutdown_connections 2024-09-15T23:09:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.084+0000 7f5ce168e740 1 -- 172.21.15.111:0/2574086905 wait complete. 2024-09-15T23:09:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.084+0000 7f5ce168e740 1 -- start start 2024-09-15T23:09:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.084+0000 7f5ce168e740 1 Processor -- start 2024-09-15T23:09:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.085+0000 7f5ce168e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5562ee5a9260 0x5562ee5b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.085+0000 7f5ce168e740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5562ee5a9ba0 0x5562ee5b8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.085+0000 7f5ce168e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5562ee5aec30 0x5562ee5b9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:29.090 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.085+0000 7f5ce168e740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd81a9540 con 0x5562ee5a9ba0 2024-09-15T23:09:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.085+0000 7f5ce168e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd81aa9b0 con 0x5562ee5aec30 2024-09-15T23:09:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.085+0000 7f5ce168e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd81a9c10 con 0x5562ee5a9260 2024-09-15T23:09:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.085+0000 7f5ce168e740 10 client.0 _ll_register_callbacks cb 0x7f5cd0009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:09:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.085+0000 7f5cd6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5562ee5a9ba0 0x5562ee5b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.085+0000 7f5cd6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5562ee5a9ba0 0x5562ee5b8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59706/0 (socket says 172.21.15.111:59706) 2024-09-15T23:09:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.085+0000 7f5cd7400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5562ee5a9260 0x5562ee5b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.085+0000 7f5cd6a00640 1 -- 172.21.15.111:0/1978484491 learned_addr learned my addr 172.21.15.111:0/1978484491 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:09:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.085+0000 7f5cd7e00640 1 --2- 172.21.15.111:0/1978484491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5562ee5aec30 0x5562ee5b9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.085+0000 7f5cd6a00640 1 -- 172.21.15.111:0/1978484491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5562ee5a9260 msgr2=0x5562ee5b7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.085+0000 7f5cd6a00640 1 --2- 172.21.15.111:0/1978484491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5562ee5a9260 0x5562ee5b7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.085+0000 7f5cd6a00640 1 -- 172.21.15.111:0/1978484491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5562ee5aec30 msgr2=0x5562ee5b9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.085+0000 7f5cd6a00640 1 --2- 172.21.15.111:0/1978484491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5562ee5aec30 0x5562ee5b9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.085+0000 7f5cd6a00640 1 -- 172.21.15.111:0/1978484491 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562ee5b9830 con 0x5562ee5a9ba0 2024-09-15T23:09:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.086+0000 7f5cd6a00640 1 --2- 172.21.15.111:0/1978484491 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5562ee5a9ba0 0x5562ee5b8260 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4023dc0 tx=0x7f5cc4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1616260bb7df7bc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.086+0000 7f5cc1600640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:09:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.086+0000 7f5cc1600640 1 -- 172.21.15.111:0/1978484491 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4015030 con 0x5562ee5a9ba0 2024-09-15T23:09:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.086+0000 7f5cc1600640 1 -- 172.21.15.111:0/1978484491 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f5cc4005240 con 0x5562ee5a9ba0 2024-09-15T23:09:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.086+0000 7f5ce168e740 1 -- 172.21.15.111:0/1978484491 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cc80bbd80 con 0x5562ee5a9ba0 2024-09-15T23:09:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.086+0000 7f5cc1600640 1 -- 172.21.15.111:0/1978484491 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5cc403d050 con 0x5562ee5a9ba0 2024-09-15T23:09:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.086+0000 7f5ce168e740 10 client.16830 fetch_fsmap learned FSMap version 35 2024-09-15T23:09:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.086+0000 7f5ce168e740 1 -- 172.21.15.111:0/1978484491 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5562ee5bab40 con 0x5562ee5a9ba0 2024-09-15T23:09:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.086+0000 7f5cc1600640 1 -- 172.21.15.111:0/1978484491 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cc80bbd80 con 0x5562ee5a9ba0 2024-09-15T23:09:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.087+0000 7f5cc1600640 1 -- 172.21.15.111:0/1978484491 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f5cc4035020 con 0x5562ee5a9ba0 2024-09-15T23:09:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.087+0000 7f5ce168e740 10 client.16830 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:09:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.087+0000 7f5ce168e740 10 client.16830 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:09:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.087+0000 7f5ce168e740 20 client.16830 populate_metadata read hostname 'smithi111' 2024-09-15T23:09:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.087+0000 7f5ce168e740 1 --2- 172.21.15.111:0/1978484491 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5562ee629ef0 0x5562ee62c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.087+0000 7f5ce168e740 4 client.16830 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:09:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.087+0000 7f5ce168e740 1 -- 172.21.15.111:0/1978484491 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5562ee4927c0 con 0x5562ee629ef0 2024-09-15T23:09:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.087+0000 7f5cd7400640 1 --2- 172.21.15.111:0/1978484491 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5562ee629ef0 0x5562ee62c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.087+0000 7f5cd7400640 1 --2- 172.21.15.111:0/1978484491 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5562ee629ef0 0x5562ee62c310 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a34d61d2cd6fb30b server_cookie=24ae896feacc4a7 in_seq=0 out_seq=0 2024-09-15T23:09:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.088+0000 7f5cc1600640 10 client.16830 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:09:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.160+0000 7f5cc1600640 1 -- 172.21.15.111:0/1978484491 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5562ee4927c0 con 0x5562ee629ef0 2024-09-15T23:09:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.160+0000 7f5cc1600640 10 client.16830 handle_command_reply: tid=0 2024-09-15T23:09:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.160+0000 7f5ce168e740 1 client.16830 shutdown 2024-09-15T23:09:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.161+0000 7f5ce168e740 1 -- 172.21.15.111:0/1978484491 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5562ee5a9ba0 msgr2=0x5562ee5b8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.161+0000 7f5ce168e740 1 --2- 172.21.15.111:0/1978484491 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5562ee5a9ba0 0x5562ee5b8260 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4023dc0 tx=0x7f5cc4005fd0 comp rx=0 tx=0).stop 2024-09-15T23:09:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.161+0000 7f5ce168e740 1 -- 172.21.15.111:0/1978484491 shutdown_connections 2024-09-15T23:09:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.161+0000 7f5ce168e740 1 --2- 172.21.15.111:0/1978484491 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5562ee629ef0 0x5562ee62c310 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.161+0000 7f5ce168e740 1 --2- 172.21.15.111:0/1978484491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5562ee5aec30 0x5562ee5b9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:29.163 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.161+0000 7f5ce168e740 1 --2- 172.21.15.111:0/1978484491 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5562ee5a9ba0 0x5562ee5b8260 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.161+0000 7f5ce168e740 1 --2- 172.21.15.111:0/1978484491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5562ee5a9260 0x5562ee5b7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.161+0000 7f5cc1600640 0 client.16830 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:09:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.161+0000 7f5ce168e740 1 -- 172.21.15.111:0/1978484491 >> 172.21.15.111:0/1978484491 conn(0x5562ee5a48d0 msgr2=0x5562ee5a3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.161+0000 7f5ce168e740 1 -- 172.21.15.111:0/1978484491 shutdown_connections 2024-09-15T23:09:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.161+0000 7f5ce168e740 1 -- 172.21.15.111:0/1978484491 wait complete. 2024-09-15T23:09:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.161+0000 7f5ce168e740 20 client.16830 trim_cache size 0 max 16384 2024-09-15T23:09:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:09:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:09:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:09:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:09:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.162+0000 7f5cdd200640 1 -- 172.21.15.111:0/169518001 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5cbc0a3240 msgr2=0x7f5cbc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.162+0000 7f5cdd200640 1 --2- 172.21.15.111:0/169518001 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5cbc0a3240 0x7f5cbc0a5700 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0005c40 tx=0x7f5cd0010040 comp rx=0 tx=0).stop 2024-09-15T23:09:29.165 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.162+0000 7f5cdd200640 1 -- 172.21.15.111:0/169518001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cd8101580 msgr2=0x7f5cd8199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.162+0000 7f5cdd200640 1 --2- 172.21.15.111:0/169518001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cd8101580 0x7f5cd8199ff0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8035040 tx=0x7f5cc8033d90 comp rx=0 tx=0).stop 2024-09-15T23:09:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.162+0000 7f5cdd200640 1 -- 172.21.15.111:0/169518001 shutdown_connections 2024-09-15T23:09:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.162+0000 7f5cdd200640 1 --2- 172.21.15.111:0/169518001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cd8103050 0x7f5cd81a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.162+0000 7f5cdd200640 1 --2- 172.21.15.111:0/169518001 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5cbc0a3240 0x7f5cbc0a5700 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.162+0000 7f5cdd200640 1 --2- 172.21.15.111:0/169518001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5cd8101f40 0x7f5cd819a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.162+0000 7f5cdd200640 1 --2- 172.21.15.111:0/169518001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cd8101580 0x7f5cd8199ff0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.162+0000 7f5cdd200640 1 -- 172.21.15.111:0/169518001 >> 172.21.15.111:0/169518001 conn(0x7f5cd80fd0d0 msgr2=0x7f5cd81083d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.162+0000 7f5cdd200640 1 -- 172.21.15.111:0/169518001 shutdown_connections 2024-09-15T23:09:29.166 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:29.162+0000 7f5cdd200640 1 -- 172.21.15.111:0/169518001 wait complete. 2024-09-15T23:09:29.691 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199070 17.04 MB/sec execute 477 sec latency 4350.632 ms 2024-09-15T23:09:30.308 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:30 smithi111 ceph-mon[30629]: pgmap v1058: 97 pgs: 97 active+clean; 596 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 5.9 MiB/s wr, 275 op/s 2024-09-15T23:09:30.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:30 smithi192 ceph-mon[35427]: pgmap v1058: 97 pgs: 97 active+clean; 596 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 5.9 MiB/s wr, 275 op/s 2024-09-15T23:09:30.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:30 smithi121 ceph-mon[34815]: pgmap v1058: 97 pgs: 97 active+clean; 596 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 5.9 MiB/s wr, 275 op/s 2024-09-15T23:09:30.691 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199071 17.01 MB/sec execute 478 sec latency 5350.694 ms 2024-09-15T23:09:31.691 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199078 16.97 MB/sec execute 479 sec latency 6320.127 ms 2024-09-15T23:09:32.253 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:32 smithi111 ceph-mon[30629]: pgmap v1059: 97 pgs: 97 active+clean; 590 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 5.9 MiB/s wr, 254 op/s 2024-09-15T23:09:32.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:32 smithi192 ceph-mon[35427]: pgmap v1059: 97 pgs: 97 active+clean; 590 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 5.9 MiB/s wr, 254 op/s 2024-09-15T23:09:32.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:32 smithi121 ceph-mon[34815]: pgmap v1059: 97 pgs: 97 active+clean; 590 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 5.9 MiB/s wr, 254 op/s 2024-09-15T23:09:32.691 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199080 16.94 MB/sec execute 480 sec latency 6431.131 ms 2024-09-15T23:09:33.691 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199080 16.90 MB/sec execute 481 sec latency 7431.184 ms 2024-09-15T23:09:34.222 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:34 smithi111 ceph-mon[30629]: pgmap v1060: 97 pgs: 97 active+clean; 591 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 5.6 MiB/s wr, 219 op/s 2024-09-15T23:09:34.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:34 smithi192 ceph-mon[35427]: pgmap v1060: 97 pgs: 97 active+clean; 591 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 5.6 MiB/s wr, 219 op/s 2024-09-15T23:09:34.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:34 smithi121 ceph-mon[34815]: pgmap v1060: 97 pgs: 97 active+clean; 591 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 5.6 MiB/s wr, 219 op/s 2024-09-15T23:09:34.691 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199090 16.87 MB/sec execute 482 sec latency 8107.209 ms 2024-09-15T23:09:35.691 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199112 16.84 MB/sec execute 483 sec latency 8337.910 ms 2024-09-15T23:09:36.222 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:36 smithi111 ceph-mon[30629]: pgmap v1061: 97 pgs: 97 active+clean; 571 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 6.2 MiB/s wr, 241 op/s 2024-09-15T23:09:36.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:36 smithi121 ceph-mon[34815]: pgmap v1061: 97 pgs: 97 active+clean; 571 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 6.2 MiB/s wr, 241 op/s 2024-09-15T23:09:36.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:36 smithi192 ceph-mon[35427]: pgmap v1061: 97 pgs: 97 active+clean; 571 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 6.2 MiB/s wr, 241 op/s 2024-09-15T23:09:36.691 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199117 16.80 MB/sec execute 484 sec latency 4284.543 ms 2024-09-15T23:09:37.691 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199125 16.77 MB/sec execute 485 sec latency 5284.589 ms 2024-09-15T23:09:38.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:38 smithi121 ceph-mon[34815]: pgmap v1062: 97 pgs: 97 active+clean; 566 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 5.9 MiB/s wr, 245 op/s 2024-09-15T23:09:38.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:38 smithi111 ceph-mon[30629]: pgmap v1062: 97 pgs: 97 active+clean; 566 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 5.9 MiB/s wr, 245 op/s 2024-09-15T23:09:38.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:38 smithi192 ceph-mon[35427]: pgmap v1062: 97 pgs: 97 active+clean; 566 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 5.9 MiB/s wr, 245 op/s 2024-09-15T23:09:38.691 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199126 16.73 MB/sec execute 486 sec latency 6284.659 ms 2024-09-15T23:09:39.691 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199126 16.70 MB/sec execute 487 sec latency 7284.729 ms 2024-09-15T23:09:40.367 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:40 smithi192 ceph-mon[35427]: pgmap v1063: 97 pgs: 97 active+clean; 558 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 5.9 MiB/s wr, 248 op/s 2024-09-15T23:09:40.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:40 smithi121 ceph-mon[34815]: pgmap v1063: 97 pgs: 97 active+clean; 558 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 5.9 MiB/s wr, 248 op/s 2024-09-15T23:09:40.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:40 smithi111 ceph-mon[30629]: pgmap v1063: 97 pgs: 97 active+clean; 558 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 5.9 MiB/s wr, 248 op/s 2024-09-15T23:09:40.691 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199126 16.66 MB/sec execute 488 sec latency 8284.774 ms 2024-09-15T23:09:41.691 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199126 16.63 MB/sec execute 489 sec latency 9284.860 ms 2024-09-15T23:09:42.157 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:42 smithi111 ceph-mon[30629]: pgmap v1064: 97 pgs: 97 active+clean; 555 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 5.9 MiB/s wr, 235 op/s 2024-09-15T23:09:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.278+0000 7f2e35000640 1 -- 172.21.15.111:0/3538969836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e30104cf0 msgr2=0x7f2e301070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:42.280 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.278+0000 7f2e35000640 1 --2- 172.21.15.111:0/3538969836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e30104cf0 0x7f2e301070e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c009860 tx=0x7f2e1c02a360 comp rx=0 tx=0).stop 2024-09-15T23:09:42.280 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.278+0000 7f2e35000640 1 -- 172.21.15.111:0/3538969836 shutdown_connections 2024-09-15T23:09:42.280 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.278+0000 7f2e35000640 1 --2- 172.21.15.111:0/3538969836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e30107620 0x7f2e30109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.280 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.278+0000 7f2e35000640 1 --2- 172.21.15.111:0/3538969836 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e30104cf0 0x7f2e301070e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.280 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.278+0000 7f2e35000640 1 --2- 172.21.15.111:0/3538969836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e301023c0 0x7f2e301047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.280 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.278+0000 7f2e35000640 1 -- 172.21.15.111:0/3538969836 >> 172.21.15.111:0/3538969836 conn(0x7f2e300fbfa0 msgr2=0x7f2e300fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:42.280 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.278+0000 7f2e35000640 1 -- 172.21.15.111:0/3538969836 shutdown_connections 2024-09-15T23:09:42.280 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.278+0000 7f2e35000640 1 -- 172.21.15.111:0/3538969836 wait complete. 2024-09-15T23:09:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.279+0000 7f2e35000640 1 Processor -- start 2024-09-15T23:09:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.279+0000 7f2e35000640 1 -- start start 2024-09-15T23:09:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.279+0000 7f2e35000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e301023c0 0x7f2e30197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.279+0000 7f2e35000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e30104cf0 0x7f2e30198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.279+0000 7f2e35000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e30107620 0x7f2e3019f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.279+0000 7f2e35000640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3010d650 con 0x7f2e30107620 2024-09-15T23:09:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.279+0000 7f2e35000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3010d4d0 con 0x7f2e301023c0 2024-09-15T23:09:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.279+0000 7f2e35000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3010d7d0 con 0x7f2e30104cf0 2024-09-15T23:09:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.279+0000 7f2e2ea00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e30104cf0 0x7f2e30198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.279+0000 7f2e2fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e30107620 0x7f2e3019f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.279+0000 7f2e2ea00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e30104cf0 0x7f2e30198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:40088/0 (socket says 172.21.15.111:40088) 2024-09-15T23:09:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.279+0000 7f2e2ea00640 1 -- 172.21.15.111:0/1938035136 learned_addr learned my addr 172.21.15.111:0/1938035136 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:09:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.280+0000 7f2e2f400640 1 --2- 172.21.15.111:0/1938035136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e301023c0 0x7f2e30197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.280+0000 7f2e2ea00640 1 -- 172.21.15.111:0/1938035136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e301023c0 msgr2=0x7f2e30197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.280+0000 7f2e2ea00640 1 --2- 172.21.15.111:0/1938035136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e301023c0 0x7f2e30197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.280+0000 7f2e2ea00640 1 -- 172.21.15.111:0/1938035136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e30107620 msgr2=0x7f2e3019f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.280+0000 7f2e2ea00640 1 --2- 172.21.15.111:0/1938035136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e30107620 0x7f2e3019f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.280+0000 7f2e2ea00640 1 -- 172.21.15.111:0/1938035136 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3019f9e0 con 0x7f2e30104cf0 2024-09-15T23:09:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.280+0000 7f2e2ea00640 1 --2- 172.21.15.111:0/1938035136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e30104cf0 0x7f2e30198300 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0304b0 tx=0x7f2e1c032830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.280+0000 7f2e27e00640 1 -- 172.21.15.111:0/1938035136 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c037040 con 0x7f2e30104cf0 2024-09-15T23:09:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.281+0000 7f2e35000640 1 -- 172.21.15.111:0/1938035136 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e3019fc70 con 0x7f2e30104cf0 2024-09-15T23:09:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.281+0000 7f2e27e00640 1 -- 172.21.15.111:0/1938035136 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e1c033de0 con 0x7f2e30104cf0 2024-09-15T23:09:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.281+0000 7f2e27e00640 1 -- 172.21.15.111:0/1938035136 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c005530 con 0x7f2e30104cf0 2024-09-15T23:09:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.281+0000 7f2e35000640 1 -- 172.21.15.111:0/1938035136 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e301a01b0 con 0x7f2e30104cf0 2024-09-15T23:09:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.283+0000 7f2e3965d740 1 Processor -- start 2024-09-15T23:09:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.284+0000 7f2e3965d740 1 -- start start 2024-09-15T23:09:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.284+0000 7f2e27e00640 1 -- 172.21.15.111:0/1938035136 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f2e1c030680 con 0x7f2e30104cf0 2024-09-15T23:09:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.284+0000 7f2e3965d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fbe0257260 0x55fbe0257660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.284+0000 7f2e3965d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fbe0257ba0 0x55fbe025c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.284+0000 7f2e3965d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbe025cc30 0x55fbe025f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.284+0000 7f2e3965d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fbe01bc100 con 0x55fbe0257ba0 2024-09-15T23:09:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.284+0000 7f2e3965d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55fbe01641b0 con 0x55fbe025cc30 2024-09-15T23:09:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.284+0000 7f2e3965d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55fbe01b80c0 con 0x55fbe0257260 2024-09-15T23:09:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.284+0000 7f2e2f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fbe0257ba0 0x55fbe025c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.284+0000 7f2e2ea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbe025cc30 0x55fbe025f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.284+0000 7f2e2f400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fbe0257ba0 0x55fbe025c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43540/0 (socket says 172.21.15.111:43540) 2024-09-15T23:09:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.284+0000 7f2e2f400640 1 -- 172.21.15.111:0/3550883835 learned_addr learned my addr 172.21.15.111:0/3550883835 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:09:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.285+0000 7f2e2ea00640 1 -- 172.21.15.111:0/3550883835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fbe0257260 msgr2=0x55fbe0257660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:09:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.285+0000 7f2e2ea00640 1 --2- 172.21.15.111:0/3550883835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fbe0257260 0x55fbe0257660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.285+0000 7f2e2ea00640 1 -- 172.21.15.111:0/3550883835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fbe0257ba0 msgr2=0x55fbe025c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.285+0000 7f2e2ea00640 1 --2- 172.21.15.111:0/3550883835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fbe0257ba0 0x55fbe025c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.285+0000 7f2e2ea00640 1 -- 172.21.15.111:0/3550883835 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbdfecf210 con 0x55fbe025cc30 2024-09-15T23:09:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.285+0000 7f2e2ea00640 1 --2- 172.21.15.111:0/3550883835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbe025cc30 0x55fbe025f0f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0ecfd0 tx=0x7f2e1c0f5510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6907a4d54590588c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:42.288 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.286+0000 7f2e27e00640 1 --2- 172.21.15.111:0/1938035136 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2e140a3240 0x7f2e140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.286+0000 7f2e27e00640 1 -- 172.21.15.111:0/1938035136 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f2e1c0ef500 con 0x7f2e30104cf0 2024-09-15T23:09:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.286+0000 7f2e24c00640 1 -- 172.21.15.111:0/3550883835 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c0b7dc0 con 0x55fbe025cc30 2024-09-15T23:09:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.286+0000 7f2e2fe00640 1 --2- 172.21.15.111:0/1938035136 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2e140a3240 0x7f2e140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.286+0000 7f2e24c00640 1 -- 172.21.15.111:0/3550883835 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e1c0fc030 con 0x55fbe025cc30 2024-09-15T23:09:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.286+0000 7f2e24c00640 1 -- 172.21.15.111:0/3550883835 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c0bb480 con 0x55fbe025cc30 2024-09-15T23:09:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.287+0000 7f2e3965d740 1 -- 172.21.15.111:0/3550883835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbe025cc30 msgr2=0x55fbe025f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.287+0000 7f2e3965d740 1 --2- 172.21.15.111:0/3550883835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbe025cc30 0x55fbe025f0f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0ecfd0 tx=0x7f2e1c0f5510 comp rx=0 tx=0).stop 2024-09-15T23:09:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.287+0000 7f2e2fe00640 1 --2- 172.21.15.111:0/1938035136 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2e140a3240 0x7f2e140a5700 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2e3006b1a0 tx=0x7f2e2800d040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.287+0000 7f2e3965d740 1 -- 172.21.15.111:0/3550883835 shutdown_connections 2024-09-15T23:09:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.287+0000 7f2e3965d740 1 --2- 172.21.15.111:0/3550883835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbe025cc30 0x55fbe025f0f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.287+0000 7f2e3965d740 1 --2- 172.21.15.111:0/3550883835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fbe0257ba0 0x55fbe025c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.287+0000 7f2e3965d740 1 --2- 172.21.15.111:0/3550883835 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fbe0257260 0x55fbe0257660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.287+0000 7f2e3965d740 1 -- 172.21.15.111:0/3550883835 >> 172.21.15.111:0/3550883835 conn(0x55fbe02528d0 msgr2=0x55fbe0252cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.287+0000 7f2e3965d740 1 -- 172.21.15.111:0/3550883835 shutdown_connections 2024-09-15T23:09:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.287+0000 7f2e3965d740 1 -- 172.21.15.111:0/3550883835 wait complete. 2024-09-15T23:09:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.288+0000 7f2e3965d740 1 -- start start 2024-09-15T23:09:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.288+0000 7f2e3965d740 1 Processor -- start 2024-09-15T23:09:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.288+0000 7f2e3965d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbe0257260 0x55fbe02655b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.288+0000 7f2e3965d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fbe0257ba0 0x55fbe0265af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.288+0000 7f2e3965d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fbe025cc30 0x55fbe026cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.288+0000 7f2e3965d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55fbe025f9c0 con 0x55fbe0257ba0 2024-09-15T23:09:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.288+0000 7f2e3965d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55fbe0145090 con 0x55fbe0257260 2024-09-15T23:09:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.288+0000 7f2e3965d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55fbe025fb40 con 0x55fbe025cc30 2024-09-15T23:09:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.288+0000 7f2e3965d740 10 client.0 _ll_register_callbacks cb 0x55fbe0199f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:09:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.289+0000 7f2e2f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbe0257260 0x55fbe02655b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.289+0000 7f2e2ea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fbe0257ba0 0x55fbe0265af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.289+0000 7f2e2f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbe0257260 0x55fbe02655b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:55854/0 (socket says 172.21.15.111:55854) 2024-09-15T23:09:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.289+0000 7f2e2f400640 1 -- 172.21.15.111:0/2003758912 learned_addr learned my addr 172.21.15.111:0/2003758912 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:09:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.289+0000 7f2e2fe00640 1 --2- 172.21.15.111:0/2003758912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fbe025cc30 0x55fbe026cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.289+0000 7f2e2f400640 1 -- 172.21.15.111:0/2003758912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fbe025cc30 msgr2=0x55fbe026cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.289+0000 7f2e2f400640 1 --2- 172.21.15.111:0/2003758912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fbe025cc30 0x55fbe026cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.289+0000 7f2e2f400640 1 -- 172.21.15.111:0/2003758912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fbe0257ba0 msgr2=0x55fbe0265af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.289+0000 7f2e2f400640 1 --2- 172.21.15.111:0/2003758912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fbe0257ba0 0x55fbe0265af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.289+0000 7f2e2f400640 1 -- 172.21.15.111:0/2003758912 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbdff9b620 con 0x55fbe0257260 2024-09-15T23:09:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.289+0000 7f2e2f400640 1 --2- 172.21.15.111:0/2003758912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbe0257260 0x55fbe02655b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2e20005c50 tx=0x7f2e20012bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d809af6867929bcc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.289+0000 7f2e19600640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:09:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.290+0000 7f2e19600640 1 -- 172.21.15.111:0/2003758912 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20004050 con 0x55fbe0257260 2024-09-15T23:09:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.290+0000 7f2e3965d740 1 -- 172.21.15.111:0/2003758912 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e1c0fc030 con 0x55fbe0257260 2024-09-15T23:09:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.290+0000 7f2e19600640 1 -- 172.21.15.111:0/2003758912 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f2e200041f0 con 0x55fbe0257260 2024-09-15T23:09:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.290+0000 7f2e19600640 1 -- 172.21.15.111:0/2003758912 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2e200043f0 con 0x55fbe0257260 2024-09-15T23:09:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.290+0000 7f2e3965d740 10 client.26792 fetch_fsmap learned FSMap version 35 2024-09-15T23:09:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.290+0000 7f2e3965d740 1 -- 172.21.15.111:0/2003758912 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55fbe026fa40 con 0x55fbe0257260 2024-09-15T23:09:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.290+0000 7f2e19600640 1 -- 172.21.15.111:0/2003758912 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e1c0fc030 con 0x55fbe0257260 2024-09-15T23:09:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.291+0000 7f2e19600640 1 -- 172.21.15.111:0/2003758912 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f2e20003c40 con 0x55fbe0257260 2024-09-15T23:09:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.291+0000 7f2e3965d740 10 client.26792 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:09:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.291+0000 7f2e3965d740 10 client.26792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:09:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.291+0000 7f2e3965d740 20 client.26792 populate_metadata read hostname 'smithi111' 2024-09-15T23:09:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.291+0000 7f2e3965d740 1 --2- 172.21.15.111:0/2003758912 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55fbe0271070 0x55fbe0273480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.291+0000 7f2e3965d740 4 client.26792 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:09:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.291+0000 7f2e3965d740 1 -- 172.21.15.111:0/2003758912 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbe01c9830 con 0x55fbe0271070 2024-09-15T23:09:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.291+0000 7f2e2ea00640 1 --2- 172.21.15.111:0/2003758912 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55fbe0271070 0x55fbe0273480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.291+0000 7f2e2ea00640 1 --2- 172.21.15.111:0/2003758912 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55fbe0271070 0x55fbe0273480 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56649749727fb6f6 server_cookie=1bcceb0d65843892 in_seq=0 out_seq=0 2024-09-15T23:09:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.291+0000 7f2e19600640 10 client.26792 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:09:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.292+0000 7f2e19600640 1 -- 172.21.15.111:0/2003758912 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fbe01c9830 con 0x55fbe0271070 2024-09-15T23:09:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.292+0000 7f2e19600640 10 client.26792 handle_command_reply: tid=0 2024-09-15T23:09:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.292+0000 7f2e3965d740 1 client.26792 shutdown 2024-09-15T23:09:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.292+0000 7f2e3965d740 1 -- 172.21.15.111:0/2003758912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbe0257260 msgr2=0x55fbe02655b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.292+0000 7f2e3965d740 1 --2- 172.21.15.111:0/2003758912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbe0257260 0x55fbe02655b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2e20005c50 tx=0x7f2e20012bd0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.293+0000 7f2e3965d740 1 -- 172.21.15.111:0/2003758912 shutdown_connections 2024-09-15T23:09:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.293+0000 7f2e3965d740 1 --2- 172.21.15.111:0/2003758912 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55fbe0271070 0x55fbe0273480 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.293+0000 7f2e3965d740 1 --2- 172.21.15.111:0/2003758912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fbe025cc30 0x55fbe026cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.293+0000 7f2e19600640 0 client.26792 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:09:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.293+0000 7f2e3965d740 1 --2- 172.21.15.111:0/2003758912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fbe0257ba0 0x55fbe0265af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.293+0000 7f2e3965d740 1 --2- 172.21.15.111:0/2003758912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbe0257260 0x55fbe02655b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.293+0000 7f2e3965d740 1 -- 172.21.15.111:0/2003758912 >> 172.21.15.111:0/2003758912 conn(0x55fbe02528d0 msgr2=0x55fbe02540e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:42.297 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.293+0000 7f2e3965d740 1 -- 172.21.15.111:0/2003758912 shutdown_connections 2024-09-15T23:09:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.293+0000 7f2e3965d740 1 -- 172.21.15.111:0/2003758912 wait complete. 2024-09-15T23:09:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.293+0000 7f2e3965d740 20 client.26792 trim_cache size 0 max 16384 2024-09-15T23:09:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.314+0000 7f2e3965d740 1 Processor -- start 2024-09-15T23:09:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.314+0000 7f2e3965d740 1 -- start start 2024-09-15T23:09:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.315+0000 7f2e3965d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fbe0257260 0x55fbe0251900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.315+0000 7f2e3965d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fbe0257ba0 0x55fbe0251e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.315+0000 7f2e3965d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbe025cc30 0x55fbe026cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.315+0000 7f2e3965d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2e301a9440 con 0x55fbe0257ba0 2024-09-15T23:09:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.315+0000 7f2e3965d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2e301a6d10 con 0x55fbe025cc30 2024-09-15T23:09:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.315+0000 7f2e3965d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2e301a4ba0 con 0x55fbe0257260 2024-09-15T23:09:42.317 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.315+0000 7f2e2f400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fbe0257260 0x55fbe0251900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.315+0000 7f2e2f400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fbe0257260 0x55fbe0251900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:40116/0 (socket says 172.21.15.111:40116) 2024-09-15T23:09:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.315+0000 7f2e2f400640 1 -- 172.21.15.111:0/3703238318 learned_addr learned my addr 172.21.15.111:0/3703238318 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:09:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.315+0000 7f2e2ea00640 1 --2- 172.21.15.111:0/3703238318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fbe0257ba0 0x55fbe0251e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.315+0000 7f2e2fe00640 1 --2- 172.21.15.111:0/3703238318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbe025cc30 0x55fbe026cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.315+0000 7f2e2f400640 1 -- 172.21.15.111:0/3703238318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbe025cc30 msgr2=0x55fbe026cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.315+0000 7f2e2f400640 1 --2- 172.21.15.111:0/3703238318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbe025cc30 0x55fbe026cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.315+0000 7f2e2f400640 1 -- 172.21.15.111:0/3703238318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fbe0257ba0 msgr2=0x55fbe0251e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.315+0000 7f2e2f400640 1 --2- 172.21.15.111:0/3703238318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fbe0257ba0 0x55fbe0251e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.315+0000 7f2e2f400640 1 -- 172.21.15.111:0/3703238318 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbe02526f0 con 0x55fbe0257260 2024-09-15T23:09:42.318 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.316+0000 7f2e2f400640 1 --2- 172.21.15.111:0/3703238318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fbe0257260 0x55fbe0251900 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2e20000c00 tx=0x7f2e20012320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9637c2d7f50ff99 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.316+0000 7f2e1be00640 1 -- 172.21.15.111:0/3703238318 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20035070 con 0x55fbe0257260 2024-09-15T23:09:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.316+0000 7f2e1be00640 1 -- 172.21.15.111:0/3703238318 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e200106f0 con 0x55fbe0257260 2024-09-15T23:09:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.316+0000 7f2e1be00640 1 -- 172.21.15.111:0/3703238318 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2002c070 con 0x55fbe0257260 2024-09-15T23:09:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.316+0000 7f2e3965d740 1 -- 172.21.15.111:0/3703238318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fbe0257260 msgr2=0x55fbe0251900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.316+0000 7f2e3965d740 1 --2- 172.21.15.111:0/3703238318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fbe0257260 0x55fbe0251900 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2e20000c00 tx=0x7f2e20012320 comp rx=0 tx=0).stop 2024-09-15T23:09:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.316+0000 7f2e3965d740 1 -- 172.21.15.111:0/3703238318 shutdown_connections 2024-09-15T23:09:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.316+0000 7f2e3965d740 1 --2- 172.21.15.111:0/3703238318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbe025cc30 0x55fbe026cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.316+0000 7f2e3965d740 1 --2- 172.21.15.111:0/3703238318 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fbe0257ba0 0x55fbe0251e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.316+0000 7f2e3965d740 1 --2- 172.21.15.111:0/3703238318 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fbe0257260 0x55fbe0251900 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.317+0000 7f2e3965d740 1 -- 172.21.15.111:0/3703238318 >> 172.21.15.111:0/3703238318 conn(0x55fbe02528d0 msgr2=0x55fbe02511f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.317+0000 7f2e3965d740 1 -- 172.21.15.111:0/3703238318 shutdown_connections 2024-09-15T23:09:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.317+0000 7f2e3965d740 1 -- 172.21.15.111:0/3703238318 wait complete. 2024-09-15T23:09:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.317+0000 7f2e3965d740 1 -- start start 2024-09-15T23:09:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.317+0000 7f2e3965d740 1 Processor -- start 2024-09-15T23:09:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.317+0000 7f2e3965d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbe0257260 0x55fbe026a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.317+0000 7f2e3965d740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fbe0257ba0 0x55fbe026a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.318+0000 7f2e3965d740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fbe025cc30 0x55fbe026b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.318+0000 7f2e3965d740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2e301a51f0 con 0x55fbe0257ba0 2024-09-15T23:09:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.318+0000 7f2e3965d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2e301a6640 con 0x55fbe0257260 2024-09-15T23:09:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.318+0000 7f2e3965d740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2e301a58a0 con 0x55fbe025cc30 2024-09-15T23:09:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.318+0000 7f2e3965d740 10 client.0 _ll_register_callbacks cb 0x7f2e20009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:09:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.318+0000 7f2e2ea00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fbe0257ba0 0x55fbe026a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.318+0000 7f2e2f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbe0257260 0x55fbe026a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.318+0000 7f2e2f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbe0257260 0x55fbe026a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:55880/0 (socket says 172.21.15.111:55880) 2024-09-15T23:09:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.318+0000 7f2e2f400640 1 -- 172.21.15.111:0/1940790468 learned_addr learned my addr 172.21.15.111:0/1940790468 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:09:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.318+0000 7f2e2fe00640 1 --2- 172.21.15.111:0/1940790468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fbe025cc30 0x55fbe026b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.318+0000 7f2e2f400640 1 -- 172.21.15.111:0/1940790468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fbe025cc30 msgr2=0x55fbe026b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.318+0000 7f2e2f400640 1 --2- 172.21.15.111:0/1940790468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fbe025cc30 0x55fbe026b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.318+0000 7f2e2f400640 1 -- 172.21.15.111:0/1940790468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fbe0257ba0 msgr2=0x55fbe026a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.318+0000 7f2e2f400640 1 --2- 172.21.15.111:0/1940790468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fbe0257ba0 0x55fbe026a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.318+0000 7f2e2f400640 1 -- 172.21.15.111:0/1940790468 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbe026bd90 con 0x55fbe0257260 2024-09-15T23:09:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.318+0000 7f2e2f400640 1 --2- 172.21.15.111:0/1940790468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbe0257260 0x55fbe026a230 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2e200105f0 tx=0x7f2e20009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14ff1c811d54977a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.318+0000 7f2e19600640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:09:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.319+0000 7f2e19600640 1 -- 172.21.15.111:0/1940790468 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e20035070 con 0x55fbe0257260 2024-09-15T23:09:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.319+0000 7f2e19600640 1 -- 172.21.15.111:0/1940790468 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f2e2000f920 con 0x55fbe0257260 2024-09-15T23:09:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.319+0000 7f2e3965d740 1 -- 172.21.15.111:0/1940790468 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e200106f0 con 0x55fbe0257260 2024-09-15T23:09:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.319+0000 7f2e19600640 1 -- 172.21.15.111:0/1940790468 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2e2002c070 con 0x55fbe0257260 2024-09-15T23:09:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.319+0000 7f2e3965d740 10 client.26798 fetch_fsmap learned FSMap version 35 2024-09-15T23:09:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.319+0000 7f2e3965d740 1 -- 172.21.15.111:0/1940790468 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55fbe02dbdb0 con 0x55fbe0257260 2024-09-15T23:09:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.319+0000 7f2e19600640 1 -- 172.21.15.111:0/1940790468 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e200106f0 con 0x55fbe0257260 2024-09-15T23:09:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.319+0000 7f2e19600640 1 -- 172.21.15.111:0/1940790468 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f2e20004c90 con 0x55fbe0257260 2024-09-15T23:09:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.319+0000 7f2e3965d740 10 client.26798 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:09:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.319+0000 7f2e3965d740 10 client.26798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:09:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.319+0000 7f2e3965d740 20 client.26798 populate_metadata read hostname 'smithi111' 2024-09-15T23:09:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.319+0000 7f2e3965d740 1 --2- 172.21.15.111:0/1940790468 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55fbe02dc4f0 0x55fbe02de8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.320+0000 7f2e3965d740 4 client.26798 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:09:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.320+0000 7f2e3965d740 1 -- 172.21.15.111:0/1940790468 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fbe01c3b80 con 0x55fbe02dc4f0 2024-09-15T23:09:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.320+0000 7f2e2ea00640 1 --2- 172.21.15.111:0/1940790468 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55fbe02dc4f0 0x55fbe02de8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.320+0000 7f2e2ea00640 1 --2- 172.21.15.111:0/1940790468 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55fbe02dc4f0 0x55fbe02de8e0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9aad6b0273e5a06d server_cookie=5e94084597079dd9 in_seq=0 out_seq=0 2024-09-15T23:09:42.326 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.320+0000 7f2e19600640 10 client.26798 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:09:42.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:42 smithi121 ceph-mon[34815]: pgmap v1064: 97 pgs: 97 active+clean; 555 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 5.9 MiB/s wr, 235 op/s 2024-09-15T23:09:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.398+0000 7f2e19600640 1 -- 172.21.15.111:0/1940790468 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fbe01c3b80 con 0x55fbe02dc4f0 2024-09-15T23:09:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.398+0000 7f2e19600640 10 client.26798 handle_command_reply: tid=0 2024-09-15T23:09:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.398+0000 7f2e3965d740 1 client.26798 shutdown 2024-09-15T23:09:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.398+0000 7f2e3965d740 1 -- 172.21.15.111:0/1940790468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbe0257260 msgr2=0x55fbe026a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.398+0000 7f2e3965d740 1 --2- 172.21.15.111:0/1940790468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbe0257260 0x55fbe026a230 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2e200105f0 tx=0x7f2e20009260 comp rx=0 tx=0).stop 2024-09-15T23:09:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.398+0000 7f2e3965d740 1 -- 172.21.15.111:0/1940790468 shutdown_connections 2024-09-15T23:09:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.398+0000 7f2e3965d740 1 --2- 172.21.15.111:0/1940790468 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55fbe02dc4f0 0x55fbe02de8e0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.398+0000 7f2e3965d740 1 --2- 172.21.15.111:0/1940790468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fbe025cc30 0x55fbe026b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.398+0000 7f2e3965d740 1 --2- 172.21.15.111:0/1940790468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fbe0257ba0 0x55fbe026a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.398+0000 7f2e19600640 0 client.26798 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:09:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.398+0000 7f2e3965d740 1 --2- 172.21.15.111:0/1940790468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbe0257260 0x55fbe026a230 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.398+0000 7f2e3965d740 1 -- 172.21.15.111:0/1940790468 >> 172.21.15.111:0/1940790468 conn(0x55fbe02528d0 msgr2=0x55fbe0251a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:42.403 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.398+0000 7f2e3965d740 1 -- 172.21.15.111:0/1940790468 shutdown_connections 2024-09-15T23:09:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.398+0000 7f2e3965d740 1 -- 172.21.15.111:0/1940790468 wait complete. 2024-09-15T23:09:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.398+0000 7f2e3965d740 20 client.26798 trim_cache size 0 max 16384 2024-09-15T23:09:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:09:42.404 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:09:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:09:42.405 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:09:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.399+0000 7f2e35000640 1 -- 172.21.15.111:0/1938035136 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2e140a3240 msgr2=0x7f2e140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:42.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.399+0000 7f2e35000640 1 --2- 172.21.15.111:0/1938035136 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2e140a3240 0x7f2e140a5700 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2e3006b1a0 tx=0x7f2e2800d040 comp rx=0 tx=0).stop 2024-09-15T23:09:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.399+0000 7f2e35000640 1 -- 172.21.15.111:0/1938035136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e30104cf0 msgr2=0x7f2e30198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.399+0000 7f2e35000640 1 --2- 172.21.15.111:0/1938035136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e30104cf0 0x7f2e30198300 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c0304b0 tx=0x7f2e1c032830 comp rx=0 tx=0).stop 2024-09-15T23:09:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.399+0000 7f2e35000640 1 -- 172.21.15.111:0/1938035136 shutdown_connections 2024-09-15T23:09:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.399+0000 7f2e35000640 1 --2- 172.21.15.111:0/1938035136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2e30107620 0x7f2e3019f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.399+0000 7f2e35000640 1 --2- 172.21.15.111:0/1938035136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2e30104cf0 0x7f2e30198300 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.399+0000 7f2e35000640 1 --2- 172.21.15.111:0/1938035136 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2e140a3240 0x7f2e140a5700 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.399+0000 7f2e35000640 1 --2- 172.21.15.111:0/1938035136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2e301023c0 0x7f2e30197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.399+0000 7f2e35000640 1 -- 172.21.15.111:0/1938035136 >> 172.21.15.111:0/1938035136 conn(0x7f2e300fbfa0 msgr2=0x7f2e30102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.399+0000 7f2e35000640 1 -- 172.21.15.111:0/1938035136 shutdown_connections 2024-09-15T23:09:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:42.399+0000 7f2e35000640 1 -- 172.21.15.111:0/1938035136 wait complete. 2024-09-15T23:09:42.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:42 smithi192 ceph-mon[35427]: pgmap v1064: 97 pgs: 97 active+clean; 555 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 5.9 MiB/s wr, 235 op/s 2024-09-15T23:09:42.691 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199126 16.60 MB/sec execute 490 sec latency 10284.945 ms 2024-09-15T23:09:43.300 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:43 smithi111 ceph-mon[30629]: pgmap v1065: 97 pgs: 97 active+clean; 555 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 5.3 MiB/s wr, 205 op/s 2024-09-15T23:09:43.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:43 smithi121 ceph-mon[34815]: pgmap v1065: 97 pgs: 97 active+clean; 555 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 5.3 MiB/s wr, 205 op/s 2024-09-15T23:09:43.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:43 smithi192 ceph-mon[35427]: pgmap v1065: 97 pgs: 97 active+clean; 555 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 5.3 MiB/s wr, 205 op/s 2024-09-15T23:09:43.692 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199127 16.56 MB/sec execute 491 sec latency 11285.013 ms 2024-09-15T23:09:44.692 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199127 16.53 MB/sec execute 492 sec latency 12285.093 ms 2024-09-15T23:09:45.692 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199127 16.49 MB/sec execute 493 sec latency 13285.181 ms 2024-09-15T23:09:46.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:46 smithi192 ceph-mon[35427]: pgmap v1066: 97 pgs: 97 active+clean; 562 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 5.3 MiB/s wr, 204 op/s 2024-09-15T23:09:46.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:46 smithi121 ceph-mon[34815]: pgmap v1066: 97 pgs: 97 active+clean; 562 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 5.3 MiB/s wr, 204 op/s 2024-09-15T23:09:46.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:46 smithi111 ceph-mon[30629]: pgmap v1066: 97 pgs: 97 active+clean; 562 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 5.3 MiB/s wr, 204 op/s 2024-09-15T23:09:46.692 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199127 16.46 MB/sec execute 494 sec latency 14285.247 ms 2024-09-15T23:09:47.692 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199130 16.43 MB/sec execute 495 sec latency 14415.817 ms 2024-09-15T23:09:48.256 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:48 smithi111 ceph-mon[30629]: pgmap v1067: 97 pgs: 97 active+clean; 561 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 4.3 MiB/s wr, 171 op/s 2024-09-15T23:09:48.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:48 smithi192 ceph-mon[35427]: pgmap v1067: 97 pgs: 97 active+clean; 561 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 4.3 MiB/s wr, 171 op/s 2024-09-15T23:09:48.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:48 smithi121 ceph-mon[34815]: pgmap v1067: 97 pgs: 97 active+clean; 561 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 4.3 MiB/s wr, 171 op/s 2024-09-15T23:09:48.692 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199134 16.39 MB/sec execute 496 sec latency 13163.803 ms 2024-09-15T23:09:49.692 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199329 16.38 MB/sec execute 497 sec latency 13683.967 ms 2024-09-15T23:09:50.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:50 smithi192 ceph-mon[35427]: pgmap v1068: 97 pgs: 97 active+clean; 560 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 3.5 MiB/s wr, 139 op/s 2024-09-15T23:09:50.317 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:50 smithi111 ceph-mon[30629]: pgmap v1068: 97 pgs: 97 active+clean; 560 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 3.5 MiB/s wr, 139 op/s 2024-09-15T23:09:50.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:50 smithi121 ceph-mon[34815]: pgmap v1068: 97 pgs: 97 active+clean; 560 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 3.5 MiB/s wr, 139 op/s 2024-09-15T23:09:50.692 INFO:tasks.workunit.client.0.smithi111.stdout: 10 199783 16.41 MB/sec execute 498 sec latency 12521.677 ms 2024-09-15T23:09:51.692 INFO:tasks.workunit.client.0.smithi111.stdout: 10 200434 16.43 MB/sec execute 499 sec latency 100.782 ms 2024-09-15T23:09:52.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:52 smithi192 ceph-mon[35427]: pgmap v1069: 97 pgs: 97 active+clean; 557 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 2.6 MiB/s wr, 108 op/s 2024-09-15T23:09:52.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:52 smithi121 ceph-mon[34815]: pgmap v1069: 97 pgs: 97 active+clean; 557 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 2.6 MiB/s wr, 108 op/s 2024-09-15T23:09:52.423 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:52 smithi111 ceph-mon[30629]: pgmap v1069: 97 pgs: 97 active+clean; 557 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 2.6 MiB/s wr, 108 op/s 2024-09-15T23:09:52.692 INFO:tasks.workunit.client.0.smithi111.stdout: 10 201062 16.47 MB/sec execute 500 sec latency 87.113 ms 2024-09-15T23:09:53.692 INFO:tasks.workunit.client.0.smithi111.stdout: 10 201737 16.51 MB/sec execute 501 sec latency 72.324 ms 2024-09-15T23:09:54.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:54 smithi192 ceph-mon[35427]: pgmap v1070: 97 pgs: 97 active+clean; 553 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 90 op/s 2024-09-15T23:09:54.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:54 smithi121 ceph-mon[34815]: pgmap v1070: 97 pgs: 97 active+clean; 553 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 90 op/s 2024-09-15T23:09:54.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:54 smithi111 ceph-mon[30629]: pgmap v1070: 97 pgs: 97 active+clean; 553 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 90 op/s 2024-09-15T23:09:54.692 INFO:tasks.workunit.client.0.smithi111.stdout: 10 202505 16.56 MB/sec execute 502 sec latency 41.910 ms 2024-09-15T23:09:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.544+0000 7f4987e00640 1 -- 172.21.15.111:0/1657479709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4988078790 msgr2=0x7f4988079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:55.545 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.544+0000 7f4987e00640 1 --2- 172.21.15.111:0/1657479709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4988078790 0x7f4988079030 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4980009be0 tx=0x7f498002a6c0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.544+0000 7f4987e00640 1 -- 172.21.15.111:0/1657479709 shutdown_connections 2024-09-15T23:09:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.544+0000 7f4987e00640 1 --2- 172.21.15.111:0/1657479709 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4988078790 0x7f4988079030 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.544+0000 7f4987e00640 1 --2- 172.21.15.111:0/1657479709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4988077df0 0x7f4988078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.544+0000 7f4987e00640 1 --2- 172.21.15.111:0/1657479709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4988076bb0 0x7f4988076fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.544+0000 7f4987e00640 1 -- 172.21.15.111:0/1657479709 >> 172.21.15.111:0/1657479709 conn(0x7f49880ff2b0 msgr2=0x7f49881016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.544+0000 7f4987e00640 1 -- 172.21.15.111:0/1657479709 shutdown_connections 2024-09-15T23:09:55.546 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.544+0000 7f4987e00640 1 -- 172.21.15.111:0/1657479709 wait complete. 2024-09-15T23:09:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.545+0000 7f4987e00640 1 Processor -- start 2024-09-15T23:09:55.547 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.545+0000 7f4987e00640 1 -- start start 2024-09-15T23:09:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.546+0000 7f4987e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4988076bb0 0x7f498819e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.546+0000 7f4987e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4988077df0 0x7f498819e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.546+0000 7f4987e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4988078790 0x7f49881a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.546+0000 7f4987e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4988113c20 con 0x7f4988076bb0 2024-09-15T23:09:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.546+0000 7f4987e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4988113aa0 con 0x7f4988077df0 2024-09-15T23:09:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.546+0000 7f4987e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4988113da0 con 0x7f4988078790 2024-09-15T23:09:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.546+0000 7f4986000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4988077df0 0x7f498819e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.546+0000 7f4987400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4988078790 0x7f49881a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.546+0000 7f4986000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4988077df0 0x7f498819e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:50504/0 (socket says 172.21.15.111:50504) 2024-09-15T23:09:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.546+0000 7f4986000640 1 -- 172.21.15.111:0/2232232525 learned_addr learned my addr 172.21.15.111:0/2232232525 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:09:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.546+0000 7f4986a00640 1 --2- 172.21.15.111:0/2232232525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4988076bb0 0x7f498819e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.546+0000 7f4986000640 1 -- 172.21.15.111:0/2232232525 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4988078790 msgr2=0x7f49881a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:55.548 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.546+0000 7f4986000640 1 --2- 172.21.15.111:0/2232232525 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4988078790 0x7f49881a57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.546+0000 7f4986000640 1 -- 172.21.15.111:0/2232232525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4988076bb0 msgr2=0x7f498819e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.546+0000 7f4986000640 1 --2- 172.21.15.111:0/2232232525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4988076bb0 0x7f498819e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.546+0000 7f4986000640 1 -- 172.21.15.111:0/2232232525 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49881a5e40 con 0x7f4988077df0 2024-09-15T23:09:55.549 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.546+0000 7f4986a00640 1 --2- 172.21.15.111:0/2232232525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4988076bb0 0x7f498819e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:09:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.547+0000 7f4986000640 1 --2- 172.21.15.111:0/2232232525 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4988077df0 0x7f498819e760 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f497400e9f0 tx=0x7f497400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.547+0000 7f498ca00640 1 -- 172.21.15.111:0/2232232525 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497400cd80 con 0x7f4988077df0 2024-09-15T23:09:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.547+0000 7f498ca00640 1 -- 172.21.15.111:0/2232232525 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4974010910 con 0x7f4988077df0 2024-09-15T23:09:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.547+0000 7f4987e00640 1 -- 172.21.15.111:0/2232232525 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49881a6130 con 0x7f4988077df0 2024-09-15T23:09:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.547+0000 7f498ca00640 1 -- 172.21.15.111:0/2232232525 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974018990 con 0x7f4988077df0 2024-09-15T23:09:55.550 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.547+0000 7f4987e00640 1 -- 172.21.15.111:0/2232232525 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49881a6680 con 0x7f4988077df0 2024-09-15T23:09:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.551+0000 7f4990bcb740 1 Processor -- start 2024-09-15T23:09:55.553 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.551+0000 7f4990bcb740 1 -- start start 2024-09-15T23:09:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.551+0000 7f4990bcb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba37ce2270 0x55ba37ce2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.551+0000 7f4990bcb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba37ce2bb0 0x55ba37ce7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.551+0000 7f4990bcb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba37ce7c40 0x55ba37cea100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.551+0000 7f4990bcb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ba37c36100 con 0x55ba37ce7c40 2024-09-15T23:09:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.551+0000 7f4990bcb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ba37bde1b0 con 0x55ba37ce2bb0 2024-09-15T23:09:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.551+0000 7f4990bcb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ba37c320c0 con 0x55ba37ce2270 2024-09-15T23:09:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.552+0000 7f4987400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba37ce2270 0x55ba37ce2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.552+0000 7f4987400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba37ce2270 0x55ba37ce2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:53532/0 (socket says 172.21.15.111:53532) 2024-09-15T23:09:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.552+0000 7f4987400640 1 -- 172.21.15.111:0/3873590390 learned_addr learned my addr 172.21.15.111:0/3873590390 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:09:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.552+0000 7f4986a00640 1 --2- 172.21.15.111:0/3873590390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba37ce2bb0 0x55ba37ce7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.552+0000 7f4987400640 1 -- 172.21.15.111:0/3873590390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba37ce2bb0 msgr2=0x55ba37ce7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.552+0000 7f4987400640 1 --2- 172.21.15.111:0/3873590390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba37ce2bb0 0x55ba37ce7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.554 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.552+0000 7f4987400640 1 -- 172.21.15.111:0/3873590390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba37ce7c40 msgr2=0x55ba37cea100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T23:09:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.552+0000 7f4987400640 1 --2- 172.21.15.111:0/3873590390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba37ce7c40 0x55ba37cea100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.555 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.552+0000 7f4987400640 1 -- 172.21.15.111:0/3873590390 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba37949210 con 0x55ba37ce2270 2024-09-15T23:09:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.553+0000 7f4987400640 1 --2- 172.21.15.111:0/3873590390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba37ce2270 0x55ba37ce2670 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4980009be0 tx=0x7f4980005eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d5ccc579964899f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.553+0000 7f498ca00640 1 -- 172.21.15.111:0/2232232525 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f4974010ac0 con 0x7f4988077df0 2024-09-15T23:09:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.553+0000 7f4973e00640 1 -- 172.21.15.111:0/3873590390 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4980063030 con 0x55ba37ce2270 2024-09-15T23:09:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.553+0000 7f4973e00640 1 -- 172.21.15.111:0/3873590390 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4980030c70 con 0x55ba37ce2270 2024-09-15T23:09:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.553+0000 7f4973e00640 1 -- 172.21.15.111:0/3873590390 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4980036040 con 0x55ba37ce2270 2024-09-15T23:09:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.554+0000 7f4990bcb740 1 -- 172.21.15.111:0/3873590390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba37ce2270 msgr2=0x55ba37ce2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.554+0000 7f4990bcb740 1 --2- 172.21.15.111:0/3873590390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba37ce2270 0x55ba37ce2670 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4980009be0 tx=0x7f4980005eb0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.556 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.554+0000 7f4990bcb740 1 -- 172.21.15.111:0/3873590390 shutdown_connections 2024-09-15T23:09:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.554+0000 7f4990bcb740 1 --2- 172.21.15.111:0/3873590390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba37ce7c40 0x55ba37cea100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.554+0000 7f4990bcb740 1 --2- 172.21.15.111:0/3873590390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba37ce2bb0 0x55ba37ce7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.554+0000 7f4990bcb740 1 --2- 172.21.15.111:0/3873590390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba37ce2270 0x55ba37ce2670 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.554+0000 7f4990bcb740 1 -- 172.21.15.111:0/3873590390 >> 172.21.15.111:0/3873590390 conn(0x55ba37ccc8d0 msgr2=0x55ba37ccccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.554+0000 7f4990bcb740 1 -- 172.21.15.111:0/3873590390 shutdown_connections 2024-09-15T23:09:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.554+0000 7f4990bcb740 1 -- 172.21.15.111:0/3873590390 wait complete. 2024-09-15T23:09:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.554+0000 7f498ca00640 1 --2- 172.21.15.111:0/2232232525 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f496c0a32a0 0x7f496c0a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.554+0000 7f4990bcb740 1 -- start start 2024-09-15T23:09:55.557 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.555+0000 7f4990bcb740 1 Processor -- start 2024-09-15T23:09:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.555+0000 7f498ca00640 1 -- 172.21.15.111:0/2232232525 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f49740cfb00 con 0x7f4988077df0 2024-09-15T23:09:55.559 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.555+0000 7f4987400640 1 --2- 172.21.15.111:0/2232232525 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f496c0a32a0 0x7f496c0a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.555+0000 7f4990bcb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba37ce2270 0x55ba37cd2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.555+0000 7f4990bcb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba37ce2bb0 0x55ba37cd2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.555+0000 7f4990bcb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba37ce7c40 0x55ba37cd9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.555+0000 7f4990bcb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55ba37cd10e0 con 0x55ba37ce2270 2024-09-15T23:09:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.555+0000 7f4990bcb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ba37bbf090 con 0x55ba37ce7c40 2024-09-15T23:09:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.555+0000 7f4990bcb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55ba37cd1260 con 0x55ba37ce2bb0 2024-09-15T23:09:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.555+0000 7f4990bcb740 10 client.0 _ll_register_callbacks cb 0x55ba37c13f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:09:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.555+0000 7f4986000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba37ce2bb0 0x55ba37cd2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.555+0000 7f4986000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba37ce2bb0 0x55ba37cd2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:53546/0 (socket says 172.21.15.111:53546) 2024-09-15T23:09:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.555+0000 7f4986000640 1 -- 172.21.15.111:0/3050969648 learned_addr learned my addr 172.21.15.111:0/3050969648 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:09:55.560 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.555+0000 7f4986a00640 1 --2- 172.21.15.111:0/3050969648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba37ce2270 0x55ba37cd2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.555+0000 7f4986000640 1 -- 172.21.15.111:0/3050969648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba37ce7c40 msgr2=0x55ba37cd9ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T23:09:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.556+0000 7f4986000640 1 --2- 172.21.15.111:0/3050969648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba37ce7c40 0x55ba37cd9ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.556+0000 7f4986000640 1 -- 172.21.15.111:0/3050969648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba37ce2270 msgr2=0x55ba37cd2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.556+0000 7f4986000640 1 --2- 172.21.15.111:0/3050969648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba37ce2270 0x55ba37cd2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.556+0000 7f4986000640 1 -- 172.21.15.111:0/3050969648 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba37a15620 con 0x55ba37ce2bb0 2024-09-15T23:09:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.556+0000 7f4986000640 1 --2- 172.21.15.111:0/3050969648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba37ce2bb0 0x55ba37cd2fb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f49740a4900 tx=0x7f497409ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=802b165b18e75651 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.556+0000 7f4970c00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:09:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.556+0000 7f4987400640 1 --2- 172.21.15.111:0/2232232525 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f496c0a32a0 0x7f496c0a5760 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4980003a90 tx=0x7f4980009560 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.556+0000 7f4970c00640 1 -- 172.21.15.111:0/3050969648 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497409a4c0 con 0x55ba37ce2bb0 2024-09-15T23:09:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.556+0000 7f4970c00640 1 -- 172.21.15.111:0/3050969648 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f49740bc0b0 con 0x55ba37ce2bb0 2024-09-15T23:09:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.556+0000 7f4990bcb740 1 -- 172.21.15.111:0/3050969648 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4980030c70 con 0x55ba37ce2bb0 2024-09-15T23:09:55.561 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.556+0000 7f4970c00640 1 -- 172.21.15.111:0/3050969648 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f49740a0120 con 0x55ba37ce2bb0 2024-09-15T23:09:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.557+0000 7f4990bcb740 10 client.26803 fetch_fsmap learned FSMap version 35 2024-09-15T23:09:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.557+0000 7f4990bcb740 1 -- 172.21.15.111:0/3050969648 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ba37cdcf00 con 0x55ba37ce2bb0 2024-09-15T23:09:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.557+0000 7f4970c00640 1 -- 172.21.15.111:0/3050969648 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4980030c70 con 0x55ba37ce2bb0 2024-09-15T23:09:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.557+0000 7f4970c00640 1 -- 172.21.15.111:0/3050969648 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f497409b610 con 0x55ba37ce2bb0 2024-09-15T23:09:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.557+0000 7f4990bcb740 10 client.26803 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:09:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.557+0000 7f4990bcb740 10 client.26803 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:09:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.557+0000 7f4990bcb740 20 client.26803 populate_metadata read hostname 'smithi111' 2024-09-15T23:09:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.557+0000 7f4990bcb740 1 --2- 172.21.15.111:0/3050969648 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ba37cde5d0 0x55ba37ce09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.557+0000 7f4990bcb740 4 client.26803 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:09:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.557+0000 7f4990bcb740 1 -- 172.21.15.111:0/3050969648 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba37c43830 con 0x55ba37cde5d0 2024-09-15T23:09:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.557+0000 7f4986a00640 1 --2- 172.21.15.111:0/3050969648 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ba37cde5d0 0x55ba37ce09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.558+0000 7f4986a00640 1 --2- 172.21.15.111:0/3050969648 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ba37cde5d0 0x55ba37ce09f0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64d23840378eb229 server_cookie=13fa2298c602a15f in_seq=0 out_seq=0 2024-09-15T23:09:55.562 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.558+0000 7f4970c00640 10 client.26803 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:09:55.563 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.559+0000 7f4970c00640 1 -- 172.21.15.111:0/3050969648 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ba37c43830 con 0x55ba37cde5d0 2024-09-15T23:09:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.559+0000 7f4970c00640 10 client.26803 handle_command_reply: tid=0 2024-09-15T23:09:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.560+0000 7f4990bcb740 1 client.26803 shutdown 2024-09-15T23:09:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.560+0000 7f4990bcb740 1 -- 172.21.15.111:0/3050969648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba37ce2bb0 msgr2=0x55ba37cd2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.560+0000 7f4990bcb740 1 --2- 172.21.15.111:0/3050969648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba37ce2bb0 0x55ba37cd2fb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f49740a4900 tx=0x7f497409ca00 comp rx=0 tx=0).stop 2024-09-15T23:09:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.560+0000 7f4990bcb740 1 -- 172.21.15.111:0/3050969648 shutdown_connections 2024-09-15T23:09:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.560+0000 7f4990bcb740 1 --2- 172.21.15.111:0/3050969648 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ba37cde5d0 0x55ba37ce09f0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.560+0000 7f4970c00640 0 client.26803 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:09:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.560+0000 7f4990bcb740 1 --2- 172.21.15.111:0/3050969648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba37ce7c40 0x55ba37cd9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.560+0000 7f4990bcb740 1 --2- 172.21.15.111:0/3050969648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba37ce2bb0 0x55ba37cd2fb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.560+0000 7f4990bcb740 1 --2- 172.21.15.111:0/3050969648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba37ce2270 0x55ba37cd2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.560+0000 7f4990bcb740 1 -- 172.21.15.111:0/3050969648 >> 172.21.15.111:0/3050969648 conn(0x55ba37ccc8d0 msgr2=0x55ba37cce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.560+0000 7f4990bcb740 1 -- 172.21.15.111:0/3050969648 shutdown_connections 2024-09-15T23:09:55.564 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.560+0000 7f4990bcb740 1 -- 172.21.15.111:0/3050969648 wait complete. 2024-09-15T23:09:55.565 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.560+0000 7f4990bcb740 20 client.26803 trim_cache size 0 max 16384 2024-09-15T23:09:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.588+0000 7f4990bcb740 1 Processor -- start 2024-09-15T23:09:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.588+0000 7f4990bcb740 1 -- start start 2024-09-15T23:09:55.589 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.588+0000 7f4990bcb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba37cde5d0 0x55ba37ce1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.588+0000 7f4990bcb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba37ce2270 0x55ba37cd9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.588+0000 7f4990bcb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba37ce2bb0 0x55ba37ccb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.588+0000 7f4990bcb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4988105e50 con 0x55ba37ce2bb0 2024-09-15T23:09:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.588+0000 7f4990bcb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4988103720 con 0x55ba37cde5d0 2024-09-15T23:09:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.588+0000 7f4990bcb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4988072410 con 0x55ba37ce2270 2024-09-15T23:09:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.588+0000 7f4986000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba37ce2270 0x55ba37cd9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.588+0000 7f4986a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba37cde5d0 0x55ba37ce1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.588+0000 7f4986000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba37ce2270 0x55ba37cd9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:53552/0 (socket says 172.21.15.111:53552) 2024-09-15T23:09:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.588+0000 7f4986a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba37cde5d0 0x55ba37ce1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:50540/0 (socket says 172.21.15.111:50540) 2024-09-15T23:09:55.590 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.588+0000 7f4986000640 1 -- 172.21.15.111:0/2804254168 learned_addr learned my addr 172.21.15.111:0/2804254168 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:09:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.589+0000 7f4986000640 1 -- 172.21.15.111:0/2804254168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba37cde5d0 msgr2=0x55ba37ce1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.589+0000 7f4986000640 1 --2- 172.21.15.111:0/2804254168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba37cde5d0 0x55ba37ce1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.589+0000 7f4986000640 1 -- 172.21.15.111:0/2804254168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba37ce2bb0 msgr2=0x55ba37ccb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.589+0000 7f4986000640 1 --2- 172.21.15.111:0/2804254168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba37ce2bb0 0x55ba37ccb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.589+0000 7f4986000640 1 -- 172.21.15.111:0/2804254168 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba37ce3980 con 0x55ba37ce2270 2024-09-15T23:09:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.589+0000 7f4986000640 1 --2- 172.21.15.111:0/2804254168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba37ce2270 0x55ba37cd9fb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f497409a470 tx=0x7f4974018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afd3ccf4351cdc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:55.591 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.589+0000 7f4973400640 1 -- 172.21.15.111:0/2804254168 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49740a02c0 con 0x55ba37ce2270 2024-09-15T23:09:55.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.589+0000 7f4973400640 1 -- 172.21.15.111:0/2804254168 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49740ba070 con 0x55ba37ce2270 2024-09-15T23:09:55.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.589+0000 7f4973400640 1 -- 172.21.15.111:0/2804254168 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497409b610 con 0x55ba37ce2270 2024-09-15T23:09:55.592 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.590+0000 7f4990bcb740 1 -- 172.21.15.111:0/2804254168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba37ce2270 msgr2=0x55ba37cd9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:55.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.590+0000 7f4990bcb740 1 --2- 172.21.15.111:0/2804254168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba37ce2270 0x55ba37cd9fb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f497409a470 tx=0x7f4974018420 comp rx=0 tx=0).stop 2024-09-15T23:09:55.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.590+0000 7f4990bcb740 1 -- 172.21.15.111:0/2804254168 shutdown_connections 2024-09-15T23:09:55.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.590+0000 7f4990bcb740 1 --2- 172.21.15.111:0/2804254168 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba37ce2bb0 0x55ba37ccb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.590+0000 7f4990bcb740 1 --2- 172.21.15.111:0/2804254168 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba37ce2270 0x55ba37cd9fb0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.590+0000 7f4990bcb740 1 --2- 172.21.15.111:0/2804254168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba37cde5d0 0x55ba37ce1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.590+0000 7f4990bcb740 1 -- 172.21.15.111:0/2804254168 >> 172.21.15.111:0/2804254168 conn(0x55ba37ccc8d0 msgr2=0x55ba37ce1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:55.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.591+0000 7f4990bcb740 1 -- 172.21.15.111:0/2804254168 shutdown_connections 2024-09-15T23:09:55.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.591+0000 7f4990bcb740 1 -- 172.21.15.111:0/2804254168 wait complete. 2024-09-15T23:09:55.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.591+0000 7f4990bcb740 1 -- start start 2024-09-15T23:09:55.593 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.591+0000 7f4990bcb740 1 Processor -- start 2024-09-15T23:09:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.592+0000 7f4990bcb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba37cde5d0 0x55ba37cd9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.592+0000 7f4990bcb740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba37ce2270 0x55ba37cd6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:55.595 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.592+0000 7f4990bcb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba37ce2bb0 0x55ba37cd6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.592+0000 7f4990bcb740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4988072a80 con 0x55ba37ce2270 2024-09-15T23:09:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.592+0000 7f4990bcb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4988073ef0 con 0x55ba37cde5d0 2024-09-15T23:09:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.592+0000 7f4990bcb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4988073150 con 0x55ba37ce2bb0 2024-09-15T23:09:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.592+0000 7f4990bcb740 10 client.0 _ll_register_callbacks cb 0x7f497400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:09:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.592+0000 7f4986000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba37ce2270 0x55ba37cd6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.592+0000 7f4986a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba37cde5d0 0x55ba37cd9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.592+0000 7f4987400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba37ce2bb0 0x55ba37cd6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.592+0000 7f4986a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba37cde5d0 0x55ba37cd9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:50550/0 (socket says 172.21.15.111:50550) 2024-09-15T23:09:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.592+0000 7f4986a00640 1 -- 172.21.15.111:0/3376599385 learned_addr learned my addr 172.21.15.111:0/3376599385 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:09:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.592+0000 7f4986a00640 1 -- 172.21.15.111:0/3376599385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba37ce2bb0 msgr2=0x55ba37cd6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.592+0000 7f4986a00640 1 --2- 172.21.15.111:0/3376599385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba37ce2bb0 0x55ba37cd6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.596 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.592+0000 7f4986a00640 1 -- 172.21.15.111:0/3376599385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba37ce2270 msgr2=0x55ba37cd6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:09:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.592+0000 7f4986a00640 1 --2- 172.21.15.111:0/3376599385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba37ce2270 0x55ba37cd6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.592+0000 7f4986a00640 1 -- 172.21.15.111:0/3376599385 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba37cda150 con 0x55ba37cde5d0 2024-09-15T23:09:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.593+0000 7f4986000640 1 --2- 172.21.15.111:0/3376599385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba37ce2270 0x55ba37cd6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:09:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.593+0000 7f4986a00640 1 --2- 172.21.15.111:0/3376599385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba37cde5d0 0x55ba37cd9af0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4978000c00 tx=0x7f4978009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdf538ffe5104a0a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:09:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.593+0000 7f4970c00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:09:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.593+0000 7f4970c00640 1 -- 172.21.15.111:0/3376599385 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497801d410 con 0x55ba37cde5d0 2024-09-15T23:09:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.593+0000 7f4990bcb740 1 -- 172.21.15.111:0/3376599385 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49740ba070 con 0x55ba37cde5d0 2024-09-15T23:09:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.593+0000 7f4970c00640 1 -- 172.21.15.111:0/3376599385 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f497801d5b0 con 0x55ba37cde5d0 2024-09-15T23:09:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.593+0000 7f4970c00640 1 -- 172.21.15.111:0/3376599385 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f497801d7b0 con 0x55ba37cde5d0 2024-09-15T23:09:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.593+0000 7f4990bcb740 10 client.26813 fetch_fsmap learned FSMap version 35 2024-09-15T23:09:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.593+0000 7f4990bcb740 1 -- 172.21.15.111:0/3376599385 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55ba37d62970 con 0x55ba37cde5d0 2024-09-15T23:09:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.594+0000 7f4970c00640 1 -- 172.21.15.111:0/3376599385 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49740ba070 con 0x55ba37cde5d0 2024-09-15T23:09:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.594+0000 7f4970c00640 1 -- 172.21.15.111:0/3376599385 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f4978024030 con 0x55ba37cde5d0 2024-09-15T23:09:55.597 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.594+0000 7f4990bcb740 10 client.26813 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:09:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.594+0000 7f4990bcb740 10 client.26813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:09:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.594+0000 7f4990bcb740 20 client.26813 populate_metadata read hostname 'smithi111' 2024-09-15T23:09:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.594+0000 7f4990bcb740 1 --2- 172.21.15.111:0/3376599385 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ba37ce7c40 0x55ba37d65340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:09:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.594+0000 7f4990bcb740 4 client.26813 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:09:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.594+0000 7f4990bcb740 1 -- 172.21.15.111:0/3376599385 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ba37c3db80 con 0x55ba37ce7c40 2024-09-15T23:09:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.594+0000 7f4986000640 1 --2- 172.21.15.111:0/3376599385 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ba37ce7c40 0x55ba37d65340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:09:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.595+0000 7f4970c00640 10 client.26813 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:09:55.598 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.595+0000 7f4986000640 1 --2- 172.21.15.111:0/3376599385 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ba37ce7c40 0x55ba37d65340 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6262862cdb25bee server_cookie=19193aa72a4f0651 in_seq=0 out_seq=0 2024-09-15T23:09:55.699 INFO:tasks.workunit.client.0.smithi111.stdout: 10 203139 16.59 MB/sec execute 503 sec latency 80.863 ms 2024-09-15T23:09:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.752+0000 7f4970c00640 1 -- 172.21.15.111:0/3376599385 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ba37c3db80 con 0x55ba37ce7c40 2024-09-15T23:09:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.752+0000 7f4970c00640 10 client.26813 handle_command_reply: tid=0 2024-09-15T23:09:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.753+0000 7f4990bcb740 1 client.26813 shutdown 2024-09-15T23:09:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.753+0000 7f4990bcb740 1 -- 172.21.15.111:0/3376599385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba37cde5d0 msgr2=0x55ba37cd9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.753+0000 7f4990bcb740 1 --2- 172.21.15.111:0/3376599385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba37cde5d0 0x55ba37cd9af0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4978000c00 tx=0x7f4978009ed0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.753+0000 7f4990bcb740 1 -- 172.21.15.111:0/3376599385 shutdown_connections 2024-09-15T23:09:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.753+0000 7f4990bcb740 1 --2- 172.21.15.111:0/3376599385 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55ba37ce7c40 0x55ba37d65340 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.753+0000 7f4990bcb740 1 --2- 172.21.15.111:0/3376599385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55ba37ce2bb0 0x55ba37cd6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.753+0000 7f4990bcb740 1 --2- 172.21.15.111:0/3376599385 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba37ce2270 0x55ba37cd6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.753+0000 7f4970c00640 0 client.26813 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:09:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.753+0000 7f4990bcb740 1 --2- 172.21.15.111:0/3376599385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba37cde5d0 0x55ba37cd9af0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.753+0000 7f4990bcb740 1 -- 172.21.15.111:0/3376599385 >> 172.21.15.111:0/3376599385 conn(0x55ba37ccc8d0 msgr2=0x55ba37ccdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.753+0000 7f4990bcb740 1 -- 172.21.15.111:0/3376599385 shutdown_connections 2024-09-15T23:09:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.753+0000 7f4990bcb740 1 -- 172.21.15.111:0/3376599385 wait complete. 2024-09-15T23:09:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.754+0000 7f4990bcb740 20 client.26813 trim_cache size 0 max 16384 2024-09-15T23:09:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:09:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:09:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:09:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:09:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.754+0000 7f4987e00640 1 -- 172.21.15.111:0/2232232525 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f496c0a32a0 msgr2=0x7f496c0a5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.754+0000 7f4987e00640 1 --2- 172.21.15.111:0/2232232525 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f496c0a32a0 0x7f496c0a5760 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4980003a90 tx=0x7f4980009560 comp rx=0 tx=0).stop 2024-09-15T23:09:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.754+0000 7f4987e00640 1 -- 172.21.15.111:0/2232232525 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4988077df0 msgr2=0x7f498819e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:09:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.754+0000 7f4987e00640 1 --2- 172.21.15.111:0/2232232525 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4988077df0 0x7f498819e760 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f497400e9f0 tx=0x7f497400eeb0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.755+0000 7f4987e00640 1 -- 172.21.15.111:0/2232232525 shutdown_connections 2024-09-15T23:09:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.755+0000 7f4987e00640 1 --2- 172.21.15.111:0/2232232525 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4988078790 0x7f49881a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.755+0000 7f4987e00640 1 --2- 172.21.15.111:0/2232232525 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f496c0a32a0 0x7f496c0a5760 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.755+0000 7f4987e00640 1 --2- 172.21.15.111:0/2232232525 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4988077df0 0x7f498819e760 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.755+0000 7f4987e00640 1 --2- 172.21.15.111:0/2232232525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4988076bb0 0x7f498819e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:09:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.755+0000 7f4987e00640 1 -- 172.21.15.111:0/2232232525 >> 172.21.15.111:0/2232232525 conn(0x7f49880ff2b0 msgr2=0x7f4988100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:09:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.755+0000 7f4987e00640 1 -- 172.21.15.111:0/2232232525 shutdown_connections 2024-09-15T23:09:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:09:55.755+0000 7f4987e00640 1 -- 172.21.15.111:0/2232232525 wait complete. 2024-09-15T23:09:56.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:56 smithi111 ceph-mon[30629]: pgmap v1071: 97 pgs: 97 active+clean; 559 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 221 op/s 2024-09-15T23:09:56.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:56 smithi192 ceph-mon[35427]: pgmap v1071: 97 pgs: 97 active+clean; 559 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 221 op/s 2024-09-15T23:09:56.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:56 smithi121 ceph-mon[34815]: pgmap v1071: 97 pgs: 97 active+clean; 559 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 221 op/s 2024-09-15T23:09:56.699 INFO:tasks.workunit.client.0.smithi111.stdout: 10 203743 16.62 MB/sec execute 504 sec latency 118.182 ms 2024-09-15T23:09:57.699 INFO:tasks.workunit.client.0.smithi111.stdout: 10 204614 16.68 MB/sec execute 505 sec latency 63.421 ms 2024-09-15T23:09:58.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:09:58 smithi192 ceph-mon[35427]: pgmap v1072: 97 pgs: 97 active+clean; 555 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.6 MiB/s wr, 250 op/s 2024-09-15T23:09:58.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:09:58 smithi121 ceph-mon[34815]: pgmap v1072: 97 pgs: 97 active+clean; 555 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.6 MiB/s wr, 250 op/s 2024-09-15T23:09:58.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:09:58 smithi111 ceph-mon[30629]: pgmap v1072: 97 pgs: 97 active+clean; 555 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.6 MiB/s wr, 250 op/s 2024-09-15T23:09:58.699 INFO:tasks.workunit.client.0.smithi111.stdout: 10 205343 16.74 MB/sec execute 506 sec latency 59.591 ms 2024-09-15T23:09:59.700 INFO:tasks.workunit.client.0.smithi111.stdout: 10 205927 16.76 MB/sec execute 507 sec latency 77.099 ms 2024-09-15T23:10:00.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:00 smithi192 ceph-mon[35427]: pgmap v1073: 97 pgs: 97 active+clean; 563 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.6 MiB/s wr, 385 op/s 2024-09-15T23:10:00.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:00 smithi192 ceph-mon[35427]: overall HEALTH_OK 2024-09-15T23:10:00.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:00 smithi121 ceph-mon[34815]: pgmap v1073: 97 pgs: 97 active+clean; 563 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.6 MiB/s wr, 385 op/s 2024-09-15T23:10:00.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:00 smithi121 ceph-mon[34815]: overall HEALTH_OK 2024-09-15T23:10:00.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:00 smithi111 ceph-mon[30629]: pgmap v1073: 97 pgs: 97 active+clean; 563 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.6 MiB/s wr, 385 op/s 2024-09-15T23:10:00.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:00 smithi111 ceph-mon[30629]: overall HEALTH_OK 2024-09-15T23:10:00.700 INFO:tasks.workunit.client.0.smithi111.stdout: 10 206420 16.79 MB/sec execute 508 sec latency 234.822 ms 2024-09-15T23:10:01.700 INFO:tasks.workunit.client.0.smithi111.stdout: 10 207134 16.81 MB/sec execute 509 sec latency 67.040 ms 2024-09-15T23:10:02.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:02 smithi192 ceph-mon[35427]: pgmap v1074: 97 pgs: 97 active+clean; 565 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.2 MiB/s wr, 474 op/s 2024-09-15T23:10:02.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:02 smithi121 ceph-mon[34815]: pgmap v1074: 97 pgs: 97 active+clean; 565 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.2 MiB/s wr, 474 op/s 2024-09-15T23:10:02.400 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:02 smithi111 ceph-mon[30629]: pgmap v1074: 97 pgs: 97 active+clean; 565 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.2 MiB/s wr, 474 op/s 2024-09-15T23:10:02.700 INFO:tasks.workunit.client.0.smithi111.stdout: 10 207794 16.85 MB/sec execute 510 sec latency 74.233 ms 2024-09-15T23:10:03.700 INFO:tasks.workunit.client.0.smithi111.stdout: 10 208618 16.91 MB/sec execute 511 sec latency 90.761 ms 2024-09-15T23:10:04.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:04 smithi111 ceph-mon[30629]: pgmap v1075: 97 pgs: 97 active+clean; 568 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.6 MiB/s wr, 512 op/s 2024-09-15T23:10:04.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:04 smithi192 ceph-mon[35427]: pgmap v1075: 97 pgs: 97 active+clean; 568 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.6 MiB/s wr, 512 op/s 2024-09-15T23:10:04.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:04 smithi121 ceph-mon[34815]: pgmap v1075: 97 pgs: 97 active+clean; 568 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.6 MiB/s wr, 512 op/s 2024-09-15T23:10:04.700 INFO:tasks.workunit.client.0.smithi111.stdout: 10 209390 16.97 MB/sec execute 512 sec latency 125.546 ms 2024-09-15T23:10:05.700 INFO:tasks.workunit.client.0.smithi111.stdout: 10 209890 16.99 MB/sec execute 513 sec latency 136.980 ms 2024-09-15T23:10:06.316 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:06 smithi192 ceph-mon[35427]: pgmap v1076: 97 pgs: 97 active+clean; 573 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.0 MiB/s wr, 555 op/s 2024-09-15T23:10:06.316 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:06 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:10:06.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:06 smithi121 ceph-mon[34815]: pgmap v1076: 97 pgs: 97 active+clean; 573 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.0 MiB/s wr, 555 op/s 2024-09-15T23:10:06.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:06 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:10:06.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:06 smithi111 ceph-mon[30629]: pgmap v1076: 97 pgs: 97 active+clean; 573 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.0 MiB/s wr, 555 op/s 2024-09-15T23:10:06.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:06 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:10:06.700 INFO:tasks.workunit.client.0.smithi111.stdout: 10 210305 17.00 MB/sec execute 514 sec latency 113.744 ms 2024-09-15T23:10:07.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:07 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:10:07.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:07 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:10:07.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:07 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:10:07.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:07 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:10:07.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:07 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:10:07.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:07 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:10:07.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:07 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:10:07.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:07 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:10:07.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:07 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:10:07.701 INFO:tasks.workunit.client.0.smithi111.stdout: 10 210907 17.01 MB/sec execute 515 sec latency 89.963 ms 2024-09-15T23:10:08.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:08 smithi192 ceph-mon[35427]: pgmap v1077: 97 pgs: 97 active+clean; 578 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.5 MiB/s wr, 460 op/s 2024-09-15T23:10:08.346 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:08 smithi111 ceph-mon[30629]: pgmap v1077: 97 pgs: 97 active+clean; 578 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.5 MiB/s wr, 460 op/s 2024-09-15T23:10:08.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:08 smithi121 ceph-mon[34815]: pgmap v1077: 97 pgs: 97 active+clean; 578 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.5 MiB/s wr, 460 op/s 2024-09-15T23:10:08.701 INFO:tasks.workunit.client.0.smithi111.stdout: 10 211572 17.05 MB/sec execute 516 sec latency 142.551 ms 2024-09-15T23:10:08.900 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.899+0000 7fa6e0600640 1 -- 172.21.15.111:0/3360900149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6d8103d60 msgr2=0x7fa6d8104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.899+0000 7fa6e0600640 1 --2- 172.21.15.111:0/3360900149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6d8103d60 0x7fa6d8104160 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009860 tx=0x7fa6c402a340 comp rx=0 tx=0).stop 2024-09-15T23:10:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.899+0000 7fa6e0600640 1 -- 172.21.15.111:0/3360900149 shutdown_connections 2024-09-15T23:10:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.899+0000 7fa6e0600640 1 --2- 172.21.15.111:0/3360900149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6d8105910 0x7fa6d810c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.899+0000 7fa6e0600640 1 --2- 172.21.15.111:0/3360900149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6d8104f50 0x7fa6d81053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.899+0000 7fa6e0600640 1 --2- 172.21.15.111:0/3360900149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6d8103d60 0x7fa6d8104160 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.899+0000 7fa6e0600640 1 -- 172.21.15.111:0/3360900149 >> 172.21.15.111:0/3360900149 conn(0x7fa6d80ff4e0 msgr2=0x7fa6d8101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:10:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.899+0000 7fa6e0600640 1 -- 172.21.15.111:0/3360900149 shutdown_connections 2024-09-15T23:10:08.901 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.899+0000 7fa6e0600640 1 -- 172.21.15.111:0/3360900149 wait complete. 2024-09-15T23:10:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.900+0000 7fa6e0600640 1 Processor -- start 2024-09-15T23:10:08.902 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.900+0000 7fa6e0600640 1 -- start start 2024-09-15T23:10:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.900+0000 7fa6e0600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6d8103d60 0x7fa6d8199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.900+0000 7fa6e0600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6d8104f50 0x7fa6d819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.900+0000 7fa6e0600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6d8105910 0x7fa6d81a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.900+0000 7fa6e0600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d810f9c0 con 0x7fa6d8104f50 2024-09-15T23:10:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.900+0000 7fa6e0600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d810f840 con 0x7fa6d8103d60 2024-09-15T23:10:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.900+0000 7fa6e0600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d810fb40 con 0x7fa6d8105910 2024-09-15T23:10:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.901+0000 7fa6dd200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6d8104f50 0x7fa6d819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.901+0000 7fa6ddc00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6d8103d60 0x7fa6d8199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.901+0000 7fa6ddc00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6d8103d60 0x7fa6d8199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:43660/0 (socket says 172.21.15.111:43660) 2024-09-15T23:10:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.901+0000 7fa6ddc00640 1 -- 172.21.15.111:0/4140474951 learned_addr learned my addr 172.21.15.111:0/4140474951 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:10:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.901+0000 7fa6de600640 1 --2- 172.21.15.111:0/4140474951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6d8105910 0x7fa6d81a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.901+0000 7fa6dd200640 1 -- 172.21.15.111:0/4140474951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6d8105910 msgr2=0x7fa6d81a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:08.903 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.901+0000 7fa6dd200640 1 --2- 172.21.15.111:0/4140474951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6d8105910 0x7fa6d81a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.901+0000 7fa6dd200640 1 -- 172.21.15.111:0/4140474951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6d8103d60 msgr2=0x7fa6d8199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.901+0000 7fa6dd200640 1 --2- 172.21.15.111:0/4140474951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6d8103d60 0x7fa6d8199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.901+0000 7fa6dd200640 1 -- 172.21.15.111:0/4140474951 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d81a1bb0 con 0x7fa6d8104f50 2024-09-15T23:10:08.904 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.902+0000 7fa6dd200640 1 --2- 172.21.15.111:0/4140474951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6d8104f50 0x7fa6d819a4d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc005fd0 tx=0x7fa6cc00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:10:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.902+0000 7fa6d6000640 1 -- 172.21.15.111:0/4140474951 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc00cd00 con 0x7fa6d8104f50 2024-09-15T23:10:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.902+0000 7fa6d6000640 1 -- 172.21.15.111:0/4140474951 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6cc0035f0 con 0x7fa6d8104f50 2024-09-15T23:10:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.902+0000 7fa6e0600640 1 -- 172.21.15.111:0/4140474951 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d81a1ea0 con 0x7fa6d8104f50 2024-09-15T23:10:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.902+0000 7fa6d6000640 1 -- 172.21.15.111:0/4140474951 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc010670 con 0x7fa6d8104f50 2024-09-15T23:10:08.905 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.902+0000 7fa6e0600640 1 -- 172.21.15.111:0/4140474951 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d81a2360 con 0x7fa6d8104f50 2024-09-15T23:10:08.906 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.905+0000 7fa6d6000640 1 -- 172.21.15.111:0/4140474951 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa6cc0037a0 con 0x7fa6d8104f50 2024-09-15T23:10:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.906+0000 7fa6e364b740 1 Processor -- start 2024-09-15T23:10:08.907 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.906+0000 7fa6e364b740 1 -- start start 2024-09-15T23:10:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.906+0000 7fa6e364b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583780f8090 0x5583780f8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.906+0000 7fa6e364b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583780f89d0 0x5583780fd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.906+0000 7fa6e364b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5583780fda60 0x5583780fff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.906+0000 7fa6e364b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55837805d100 con 0x5583780f8090 2024-09-15T23:10:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.906+0000 7fa6e364b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5583780051b0 con 0x5583780fda60 2024-09-15T23:10:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.906+0000 7fa6e364b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5583780590c0 con 0x5583780f89d0 2024-09-15T23:10:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.906+0000 7fa6ddc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583780f89d0 0x5583780fd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.906+0000 7fa6de600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583780f8090 0x5583780f8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.906+0000 7fa6ddc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583780f89d0 0x5583780fd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:59672/0 (socket says 172.21.15.111:59672) 2024-09-15T23:10:08.908 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.906+0000 7fa6ddc00640 1 -- 172.21.15.111:0/1732508973 learned_addr learned my addr 172.21.15.111:0/1732508973 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:10:08.909 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.907+0000 7fa6dd200640 1 --2- 172.21.15.111:0/1732508973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5583780fda60 0x5583780fff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:08.910 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.907+0000 7fa6ddc00640 1 -- 172.21.15.111:0/1732508973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5583780fda60 msgr2=0x5583780fff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:10:08.910 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.907+0000 7fa6ddc00640 1 --2- 172.21.15.111:0/1732508973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5583780fda60 0x5583780fff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.907+0000 7fa6ddc00640 1 -- 172.21.15.111:0/1732508973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583780f8090 msgr2=0x5583780f8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:10:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.907+0000 7fa6ddc00640 1 --2- 172.21.15.111:0/1732508973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583780f8090 0x5583780f8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.907+0000 7fa6ddc00640 1 -- 172.21.15.111:0/1732508973 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558377d70210 con 0x5583780f89d0 2024-09-15T23:10:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.907+0000 7fa6d6000640 1 --2- 172.21.15.111:0/4140474951 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa6c00a32f0 0x7fa6c00a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.907+0000 7fa6ddc00640 1 --2- 172.21.15.111:0/1732508973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583780f89d0 0x5583780fd490 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009780 tx=0x7fa6c4009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8533f9cc19dbed25 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:10:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.907+0000 7fa6d6000640 1 -- 172.21.15.111:0/4140474951 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fa6cc0cfb90 con 0x7fa6d8104f50 2024-09-15T23:10:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.907+0000 7fa6de600640 1 --2- 172.21.15.111:0/4140474951 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa6c00a32f0 0x7fa6c00a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.908+0000 7fa6caa00640 1 -- 172.21.15.111:0/1732508973 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4037040 con 0x5583780f89d0 2024-09-15T23:10:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.908+0000 7fa6caa00640 1 -- 172.21.15.111:0/1732508973 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6c4004040 con 0x5583780f89d0 2024-09-15T23:10:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.908+0000 7fa6caa00640 1 -- 172.21.15.111:0/1732508973 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c40046c0 con 0x5583780f89d0 2024-09-15T23:10:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.908+0000 7fa6e364b740 1 -- 172.21.15.111:0/1732508973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583780f89d0 msgr2=0x5583780fd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.908+0000 7fa6de600640 1 --2- 172.21.15.111:0/4140474951 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa6c00a32f0 0x7fa6c00a57b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0005fd0 tx=0x7fa6d0009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:10:08.911 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.908+0000 7fa6e364b740 1 --2- 172.21.15.111:0/1732508973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583780f89d0 0x5583780fd490 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009780 tx=0x7fa6c4009510 comp rx=0 tx=0).stop 2024-09-15T23:10:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.908+0000 7fa6e364b740 1 -- 172.21.15.111:0/1732508973 shutdown_connections 2024-09-15T23:10:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.908+0000 7fa6e364b740 1 --2- 172.21.15.111:0/1732508973 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5583780fda60 0x5583780fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.908+0000 7fa6e364b740 1 --2- 172.21.15.111:0/1732508973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583780f89d0 0x5583780fd490 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.908+0000 7fa6e364b740 1 --2- 172.21.15.111:0/1732508973 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583780f8090 0x5583780f8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.909+0000 7fa6e364b740 1 -- 172.21.15.111:0/1732508973 >> 172.21.15.111:0/1732508973 conn(0x5583780f3700 msgr2=0x5583780f3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:10:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.909+0000 7fa6e364b740 1 -- 172.21.15.111:0/1732508973 shutdown_connections 2024-09-15T23:10:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.909+0000 7fa6e364b740 1 -- 172.21.15.111:0/1732508973 wait complete. 2024-09-15T23:10:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.909+0000 7fa6e364b740 1 -- start start 2024-09-15T23:10:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.909+0000 7fa6e364b740 1 Processor -- start 2024-09-15T23:10:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.910+0000 7fa6e364b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583780f8090 0x5583781021b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.910+0000 7fa6e364b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5583780f89d0 0x5583781026f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.910+0000 7fa6e364b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583780fda60 0x558378109730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.910+0000 7fa6e364b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5583781009d0 con 0x5583780f8090 2024-09-15T23:10:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.910+0000 7fa6e364b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558377fe6090 con 0x5583780f89d0 2024-09-15T23:10:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.910+0000 7fa6e364b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558378100b50 con 0x5583780fda60 2024-09-15T23:10:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.910+0000 7fa6e364b740 10 client.0 _ll_register_callbacks cb 0x55837803af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:10:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.910+0000 7fa6ddc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583780f8090 0x5583781021b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.910+0000 7fa6dd200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5583780f89d0 0x5583781026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:08.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.910+0000 7fa6ddc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583780f8090 0x5583781021b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39828/0 (socket says 172.21.15.111:39828) 2024-09-15T23:10:08.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.910+0000 7fa6ddc00640 1 -- 172.21.15.111:0/463468184 learned_addr learned my addr 172.21.15.111:0/463468184 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:10:08.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.910+0000 7fa6de600640 1 --2- 172.21.15.111:0/463468184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583780fda60 0x558378109730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:08.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.910+0000 7fa6ddc00640 1 -- 172.21.15.111:0/463468184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583780fda60 msgr2=0x558378109730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:10:08.917 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.910+0000 7fa6ddc00640 1 --2- 172.21.15.111:0/463468184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583780fda60 0x558378109730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.910+0000 7fa6ddc00640 1 -- 172.21.15.111:0/463468184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5583780f89d0 msgr2=0x5583781026f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:10:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.910+0000 7fa6ddc00640 1 --2- 172.21.15.111:0/463468184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5583780f89d0 0x5583781026f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.910+0000 7fa6ddc00640 1 -- 172.21.15.111:0/463468184 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558377e3c620 con 0x5583780f8090 2024-09-15T23:10:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.911+0000 7fa6ddc00640 1 --2- 172.21.15.111:0/463468184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583780f8090 0x5583781021b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009830 tx=0x7fa6c4005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5d5ddc8405b97b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:10:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.911+0000 7fa6bf400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:10:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.911+0000 7fa6bf400640 1 -- 172.21.15.111:0/463468184 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c404a070 con 0x5583780f8090 2024-09-15T23:10:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.911+0000 7fa6bf400640 1 -- 172.21.15.111:0/463468184 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fa6c4030650 con 0x5583780f8090 2024-09-15T23:10:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.911+0000 7fa6e364b740 1 -- 172.21.15.111:0/463468184 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6c4004040 con 0x5583780f8090 2024-09-15T23:10:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.911+0000 7fa6bf400640 1 -- 172.21.15.111:0/463468184 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa6c4037040 con 0x5583780f8090 2024-09-15T23:10:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.911+0000 7fa6e364b740 10 client.16872 fetch_fsmap learned FSMap version 35 2024-09-15T23:10:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.911+0000 7fa6e364b740 1 -- 172.21.15.111:0/463468184 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55837810c520 con 0x5583780f8090 2024-09-15T23:10:08.918 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.911+0000 7fa6bf400640 1 -- 172.21.15.111:0/463468184 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6c4004040 con 0x5583780f8090 2024-09-15T23:10:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.912+0000 7fa6bf400640 1 -- 172.21.15.111:0/463468184 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fa6c40046c0 con 0x5583780f8090 2024-09-15T23:10:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.912+0000 7fa6e364b740 10 client.16872 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:10:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.912+0000 7fa6e364b740 10 client.16872 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:10:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.912+0000 7fa6e364b740 20 client.16872 populate_metadata read hostname 'smithi111' 2024-09-15T23:10:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.912+0000 7fa6e364b740 1 --2- 172.21.15.111:0/463468184 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55837810d870 0x55837810fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.912+0000 7fa6e364b740 4 client.16872 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:10:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.912+0000 7fa6e364b740 1 -- 172.21.15.111:0/463468184 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55837806a830 con 0x55837810d870 2024-09-15T23:10:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.912+0000 7fa6dd200640 1 --2- 172.21.15.111:0/463468184 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55837810d870 0x55837810fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.913+0000 7fa6bf400640 10 client.16872 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:10:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.913+0000 7fa6dd200640 1 --2- 172.21.15.111:0/463468184 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55837810d870 0x55837810fc60 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5530ae6c89554ed8 server_cookie=bada8856d6f2bb01 in_seq=0 out_seq=0 2024-09-15T23:10:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.913+0000 7fa6bf400640 1 -- 172.21.15.111:0/463468184 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55837806a830 con 0x55837810d870 2024-09-15T23:10:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.913+0000 7fa6bf400640 10 client.16872 handle_command_reply: tid=0 2024-09-15T23:10:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.914+0000 7fa6e364b740 1 client.16872 shutdown 2024-09-15T23:10:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.914+0000 7fa6e364b740 1 -- 172.21.15.111:0/463468184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583780f8090 msgr2=0x5583781021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.914+0000 7fa6e364b740 1 --2- 172.21.15.111:0/463468184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583780f8090 0x5583781021b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009830 tx=0x7fa6c4005f10 comp rx=0 tx=0).stop 2024-09-15T23:10:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.914+0000 7fa6e364b740 1 -- 172.21.15.111:0/463468184 shutdown_connections 2024-09-15T23:10:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.914+0000 7fa6e364b740 1 --2- 172.21.15.111:0/463468184 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55837810d870 0x55837810fc60 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.914+0000 7fa6e364b740 1 --2- 172.21.15.111:0/463468184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583780fda60 0x558378109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.914+0000 7fa6e364b740 1 --2- 172.21.15.111:0/463468184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5583780f89d0 0x5583781026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.914+0000 7fa6bf400640 0 client.16872 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:10:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.914+0000 7fa6e364b740 1 --2- 172.21.15.111:0/463468184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583780f8090 0x5583781021b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.914+0000 7fa6e364b740 1 -- 172.21.15.111:0/463468184 >> 172.21.15.111:0/463468184 conn(0x5583780f3700 msgr2=0x5583780f4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:10:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.914+0000 7fa6e364b740 1 -- 172.21.15.111:0/463468184 shutdown_connections 2024-09-15T23:10:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.914+0000 7fa6e364b740 1 -- 172.21.15.111:0/463468184 wait complete. 2024-09-15T23:10:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.914+0000 7fa6e364b740 20 client.16872 trim_cache size 0 max 16384 2024-09-15T23:10:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.930+0000 7fa6e364b740 1 Processor -- start 2024-09-15T23:10:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.930+0000 7fa6e364b740 1 -- start start 2024-09-15T23:10:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.930+0000 7fa6e364b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583780f8090 0x558378100f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.930+0000 7fa6e364b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5583780f89d0 0x5583781096f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.930+0000 7fa6e364b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583780fda60 0x5583780f2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.930+0000 7fa6e364b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81ad840 con 0x5583780f8090 2024-09-15T23:10:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.930+0000 7fa6e364b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81ab110 con 0x5583780f89d0 2024-09-15T23:10:08.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.930+0000 7fa6e364b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81a8fd0 con 0x5583780fda60 2024-09-15T23:10:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.930+0000 7fa6ddc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583780f8090 0x558378100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.930+0000 7fa6de600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583780fda60 0x5583780f2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.930+0000 7fa6dd200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5583780f89d0 0x5583781096f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.930+0000 7fa6de600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583780fda60 0x5583780f2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:59692/0 (socket says 172.21.15.111:59692) 2024-09-15T23:10:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.930+0000 7fa6de600640 1 -- 172.21.15.111:0/1547133848 learned_addr learned my addr 172.21.15.111:0/1547133848 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:10:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.931+0000 7fa6de600640 1 -- 172.21.15.111:0/1547133848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5583780f89d0 msgr2=0x5583781096f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:10:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.931+0000 7fa6de600640 1 --2- 172.21.15.111:0/1547133848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5583780f89d0 0x5583781096f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.931+0000 7fa6de600640 1 -- 172.21.15.111:0/1547133848 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583780f8090 msgr2=0x558378100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:10:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.931+0000 7fa6de600640 1 --2- 172.21.15.111:0/1547133848 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583780f8090 0x558378100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.931+0000 7fa6de600640 1 -- 172.21.15.111:0/1547133848 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583780f3520 con 0x5583780fda60 2024-09-15T23:10:08.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.931+0000 7fa6ddc00640 1 --2- 172.21.15.111:0/1547133848 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583780f8090 0x558378100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:10:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.931+0000 7fa6de600640 1 --2- 172.21.15.111:0/1547133848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583780fda60 0x5583780f2770 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa6d00049d0 tx=0x7fa6d0019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e8c6db6d0be158a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:10:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.931+0000 7fa6ca000640 1 -- 172.21.15.111:0/1547133848 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0029480 con 0x5583780fda60 2024-09-15T23:10:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.931+0000 7fa6ca000640 1 -- 172.21.15.111:0/1547133848 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6d0011920 con 0x5583780fda60 2024-09-15T23:10:08.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.931+0000 7fa6ca000640 1 -- 172.21.15.111:0/1547133848 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d001ca60 con 0x5583780fda60 2024-09-15T23:10:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.932+0000 7fa6e364b740 1 -- 172.21.15.111:0/1547133848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583780fda60 msgr2=0x5583780f2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.932+0000 7fa6e364b740 1 --2- 172.21.15.111:0/1547133848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583780fda60 0x5583780f2770 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa6d00049d0 tx=0x7fa6d0019000 comp rx=0 tx=0).stop 2024-09-15T23:10:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.932+0000 7fa6e364b740 1 -- 172.21.15.111:0/1547133848 shutdown_connections 2024-09-15T23:10:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.932+0000 7fa6e364b740 1 --2- 172.21.15.111:0/1547133848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583780fda60 0x5583780f2770 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:08.936 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.932+0000 7fa6e364b740 1 --2- 172.21.15.111:0/1547133848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5583780f89d0 0x5583781096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.932+0000 7fa6e364b740 1 --2- 172.21.15.111:0/1547133848 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583780f8090 0x558378100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.932+0000 7fa6e364b740 1 -- 172.21.15.111:0/1547133848 >> 172.21.15.111:0/1547133848 conn(0x5583780f3700 msgr2=0x5583780f2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:10:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.932+0000 7fa6e364b740 1 -- 172.21.15.111:0/1547133848 shutdown_connections 2024-09-15T23:10:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.932+0000 7fa6e364b740 1 -- 172.21.15.111:0/1547133848 wait complete. 2024-09-15T23:10:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.933+0000 7fa6e364b740 1 -- start start 2024-09-15T23:10:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.933+0000 7fa6e364b740 1 Processor -- start 2024-09-15T23:10:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.933+0000 7fa6e364b740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583780f8090 0x558378106e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.933+0000 7fa6e364b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583780f89d0 0x5583781073c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.933+0000 7fa6e364b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5583780fda60 0x5583781082f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.933+0000 7fa6e364b740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81a95d0 con 0x5583780f8090 2024-09-15T23:10:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.933+0000 7fa6e364b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81aaa40 con 0x5583780fda60 2024-09-15T23:10:08.937 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.933+0000 7fa6e364b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81a9ca0 con 0x5583780f89d0 2024-09-15T23:10:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.933+0000 7fa6e364b740 10 client.0 _ll_register_callbacks cb 0x7fa6d00099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:10:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.933+0000 7fa6dd200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583780f89d0 0x5583781073c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.933+0000 7fa6de600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5583780fda60 0x5583781082f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.933+0000 7fa6dd200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583780f89d0 0x5583781073c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:59706/0 (socket says 172.21.15.111:59706) 2024-09-15T23:10:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.933+0000 7fa6dd200640 1 -- 172.21.15.111:0/779110793 learned_addr learned my addr 172.21.15.111:0/779110793 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:10:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.933+0000 7fa6ddc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583780f8090 0x558378106e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.934+0000 7fa6dd200640 1 -- 172.21.15.111:0/779110793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5583780fda60 msgr2=0x5583781082f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:10:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.934+0000 7fa6dd200640 1 --2- 172.21.15.111:0/779110793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5583780fda60 0x5583781082f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.934+0000 7fa6dd200640 1 -- 172.21.15.111:0/779110793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583780f8090 msgr2=0x558378106e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:10:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.934+0000 7fa6dd200640 1 --2- 172.21.15.111:0/779110793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583780f8090 0x558378106e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.934+0000 7fa6dd200640 1 -- 172.21.15.111:0/779110793 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558378108900 con 0x5583780f89d0 2024-09-15T23:10:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.934+0000 7fa6dd200640 1 --2- 172.21.15.111:0/779110793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583780f89d0 0x5583781073c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc098610 tx=0x7fa6cc0bc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4123da4d2d13caac server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:10:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.934+0000 7fa6bf400640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:10:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.934+0000 7fa6bf400640 1 -- 172.21.15.111:0/779110793 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc0d5070 con 0x5583780f89d0 2024-09-15T23:10:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.934+0000 7fa6bf400640 1 -- 172.21.15.111:0/779110793 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fa6cc0cd7a0 con 0x5583780f89d0 2024-09-15T23:10:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.934+0000 7fa6e364b740 1 -- 172.21.15.111:0/779110793 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6d0011920 con 0x5583780f89d0 2024-09-15T23:10:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.934+0000 7fa6bf400640 1 -- 172.21.15.111:0/779110793 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa6cc0d3420 con 0x5583780f89d0 2024-09-15T23:10:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.934+0000 7fa6e364b740 10 client.26827 fetch_fsmap learned FSMap version 35 2024-09-15T23:10:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.934+0000 7fa6e364b740 1 -- 172.21.15.111:0/779110793 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558378180d10 con 0x5583780f89d0 2024-09-15T23:10:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.935+0000 7fa6bf400640 1 -- 172.21.15.111:0/779110793 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6d0011920 con 0x5583780f89d0 2024-09-15T23:10:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.935+0000 7fa6bf400640 1 -- 172.21.15.111:0/779110793 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fa6cc098820 con 0x5583780f89d0 2024-09-15T23:10:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.935+0000 7fa6e364b740 10 client.26827 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:10:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.935+0000 7fa6e364b740 10 client.26827 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:10:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.935+0000 7fa6e364b740 20 client.26827 populate_metadata read hostname 'smithi111' 2024-09-15T23:10:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.935+0000 7fa6e364b740 1 --2- 172.21.15.111:0/779110793 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x558378181620 0x558378183a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.935+0000 7fa6e364b740 4 client.26827 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:10:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.935+0000 7fa6e364b740 1 -- 172.21.15.111:0/779110793 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5583780eed80 con 0x558378181620 2024-09-15T23:10:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.935+0000 7fa6ddc00640 1 --2- 172.21.15.111:0/779110793 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x558378181620 0x558378183a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.936+0000 7fa6ddc00640 1 --2- 172.21.15.111:0/779110793 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x558378181620 0x558378183a40 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8352690ea1971bf0 server_cookie=dd17c0229032e665 in_seq=0 out_seq=0 2024-09-15T23:10:08.940 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:08.936+0000 7fa6bf400640 10 client.26827 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:10:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:09.232+0000 7fa6bf400640 1 -- 172.21.15.111:0/779110793 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5583780eed80 con 0x558378181620 2024-09-15T23:10:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:09.232+0000 7fa6bf400640 10 client.26827 handle_command_reply: tid=0 2024-09-15T23:10:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:09.232+0000 7fa6e364b740 1 client.26827 shutdown 2024-09-15T23:10:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:09.232+0000 7fa6e364b740 1 -- 172.21.15.111:0/779110793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583780f89d0 msgr2=0x5583781073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:09.232+0000 7fa6e364b740 1 --2- 172.21.15.111:0/779110793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583780f89d0 0x5583781073c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc098610 tx=0x7fa6cc0bc040 comp rx=0 tx=0).stop 2024-09-15T23:10:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:09.232+0000 7fa6e364b740 1 -- 172.21.15.111:0/779110793 shutdown_connections 2024-09-15T23:10:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:09.232+0000 7fa6e364b740 1 --2- 172.21.15.111:0/779110793 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x558378181620 0x558378183a40 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:09.232+0000 7fa6e364b740 1 --2- 172.21.15.111:0/779110793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5583780fda60 0x5583781082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:09.232+0000 7fa6e364b740 1 --2- 172.21.15.111:0/779110793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5583780f89d0 0x5583781073c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:09.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:09.232+0000 7fa6e364b740 1 --2- 172.21.15.111:0/779110793 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5583780f8090 0x558378106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:09.232+0000 7fa6e364b740 1 -- 172.21.15.111:0/779110793 >> 172.21.15.111:0/779110793 conn(0x5583780f3700 msgr2=0x5583780f2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:10:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:09.232+0000 7fa6bf400640 0 client.26827 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:10:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:09.232+0000 7fa6e364b740 1 -- 172.21.15.111:0/779110793 shutdown_connections 2024-09-15T23:10:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:09.232+0000 7fa6e364b740 1 -- 172.21.15.111:0/779110793 wait complete. 2024-09-15T23:10:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:09.232+0000 7fa6e364b740 20 client.26827 trim_cache size 0 max 16384 2024-09-15T23:10:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:10:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:10:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:10:09.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:10:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:09.233+0000 7fa6e0600640 1 -- 172.21.15.111:0/4140474951 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa6c00a32f0 msgr2=0x7fa6c00a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:09.233+0000 7fa6e0600640 1 --2- 172.21.15.111:0/4140474951 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa6c00a32f0 0x7fa6c00a57b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0005fd0 tx=0x7fa6d0009500 comp rx=0 tx=0).stop 2024-09-15T23:10:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:09.233+0000 7fa6e0600640 1 -- 172.21.15.111:0/4140474951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6d8104f50 msgr2=0x7fa6d819a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:09.233+0000 7fa6e0600640 1 --2- 172.21.15.111:0/4140474951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6d8104f50 0x7fa6d819a4d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc005fd0 tx=0x7fa6cc00ee50 comp rx=0 tx=0).stop 2024-09-15T23:10:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:09.233+0000 7fa6e0600640 1 -- 172.21.15.111:0/4140474951 shutdown_connections 2024-09-15T23:10:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:09.233+0000 7fa6e0600640 1 --2- 172.21.15.111:0/4140474951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa6d8105910 0x7fa6d81a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:09.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:09.233+0000 7fa6e0600640 1 --2- 172.21.15.111:0/4140474951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa6d8104f50 0x7fa6d819a4d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:09.233+0000 7fa6e0600640 1 --2- 172.21.15.111:0/4140474951 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa6c00a32f0 0x7fa6c00a57b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:09.233+0000 7fa6e0600640 1 --2- 172.21.15.111:0/4140474951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa6d8103d60 0x7fa6d8199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:09.233+0000 7fa6e0600640 1 -- 172.21.15.111:0/4140474951 >> 172.21.15.111:0/4140474951 conn(0x7fa6d80ff4e0 msgr2=0x7fa6d8108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:10:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:09.233+0000 7fa6e0600640 1 -- 172.21.15.111:0/4140474951 shutdown_connections 2024-09-15T23:10:09.237 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:09.233+0000 7fa6e0600640 1 -- 172.21.15.111:0/4140474951 wait complete. 2024-09-15T23:10:09.701 INFO:tasks.workunit.client.0.smithi111.stdout: 10 212015 17.05 MB/sec execute 517 sec latency 243.619 ms 2024-09-15T23:10:10.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:10 smithi192 ceph-mon[35427]: pgmap v1078: 97 pgs: 97 active+clean; 588 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.3 MiB/s wr, 539 op/s 2024-09-15T23:10:10.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:10 smithi121 ceph-mon[34815]: pgmap v1078: 97 pgs: 97 active+clean; 588 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.3 MiB/s wr, 539 op/s 2024-09-15T23:10:10.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:10 smithi111 ceph-mon[30629]: pgmap v1078: 97 pgs: 97 active+clean; 588 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.3 MiB/s wr, 539 op/s 2024-09-15T23:10:10.701 INFO:tasks.workunit.client.0.smithi111.stdout: 10 212682 17.10 MB/sec execute 518 sec latency 62.839 ms 2024-09-15T23:10:11.701 INFO:tasks.workunit.client.0.smithi111.stdout: 10 213375 17.16 MB/sec execute 519 sec latency 117.996 ms 2024-09-15T23:10:12.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:12 smithi192 ceph-mon[35427]: pgmap v1079: 97 pgs: 97 active+clean; 588 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.7 MiB/s wr, 478 op/s 2024-09-15T23:10:12.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:12 smithi121 ceph-mon[34815]: pgmap v1079: 97 pgs: 97 active+clean; 588 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.7 MiB/s wr, 478 op/s 2024-09-15T23:10:12.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:12 smithi111 ceph-mon[30629]: pgmap v1079: 97 pgs: 97 active+clean; 588 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.7 MiB/s wr, 478 op/s 2024-09-15T23:10:12.701 INFO:tasks.workunit.client.0.smithi111.stdout: 10 213871 17.18 MB/sec execute 520 sec latency 160.900 ms 2024-09-15T23:10:13.701 INFO:tasks.workunit.client.0.smithi111.stdout: 10 214384 17.19 MB/sec execute 521 sec latency 89.198 ms 2024-09-15T23:10:14.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:14 smithi192 ceph-mon[35427]: pgmap v1080: 97 pgs: 97 active+clean; 581 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 3.2 MiB/s wr, 430 op/s 2024-09-15T23:10:14.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:14 smithi121 ceph-mon[34815]: pgmap v1080: 97 pgs: 97 active+clean; 581 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 3.2 MiB/s wr, 430 op/s 2024-09-15T23:10:14.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:14 smithi111 ceph-mon[30629]: pgmap v1080: 97 pgs: 97 active+clean; 581 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 3.2 MiB/s wr, 430 op/s 2024-09-15T23:10:14.701 INFO:tasks.workunit.client.0.smithi111.stdout: 10 214994 17.22 MB/sec execute 522 sec latency 66.584 ms 2024-09-15T23:10:15.701 INFO:tasks.workunit.client.0.smithi111.stdout: 10 215720 17.24 MB/sec execute 523 sec latency 99.863 ms 2024-09-15T23:10:16.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:16 smithi121 ceph-mon[34815]: pgmap v1081: 97 pgs: 97 active+clean; 580 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.8 MiB/s wr, 528 op/s 2024-09-15T23:10:16.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:16 smithi111 ceph-mon[30629]: pgmap v1081: 97 pgs: 97 active+clean; 580 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.8 MiB/s wr, 528 op/s 2024-09-15T23:10:16.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:16 smithi192 ceph-mon[35427]: pgmap v1081: 97 pgs: 97 active+clean; 580 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.8 MiB/s wr, 528 op/s 2024-09-15T23:10:16.701 INFO:tasks.workunit.client.0.smithi111.stdout: 10 216523 17.30 MB/sec execute 524 sec latency 62.736 ms 2024-09-15T23:10:17.701 INFO:tasks.workunit.client.0.smithi111.stdout: 10 217241 17.34 MB/sec execute 525 sec latency 86.179 ms 2024-09-15T23:10:18.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:18 smithi121 ceph-mon[34815]: pgmap v1082: 97 pgs: 97 active+clean; 574 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.6 MiB/s wr, 495 op/s 2024-09-15T23:10:18.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:18 smithi111 ceph-mon[30629]: pgmap v1082: 97 pgs: 97 active+clean; 574 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.6 MiB/s wr, 495 op/s 2024-09-15T23:10:18.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:18 smithi192 ceph-mon[35427]: pgmap v1082: 97 pgs: 97 active+clean; 574 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.6 MiB/s wr, 495 op/s 2024-09-15T23:10:18.701 INFO:tasks.workunit.client.0.smithi111.stdout: 10 217930 17.40 MB/sec execute 526 sec latency 59.857 ms 2024-09-15T23:10:19.701 INFO:tasks.workunit.client.0.smithi111.stdout: 10 218497 17.42 MB/sec execute 527 sec latency 98.084 ms 2024-09-15T23:10:20.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:20 smithi121 ceph-mon[34815]: pgmap v1083: 97 pgs: 97 active+clean; 580 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.7 MiB/s wr, 522 op/s 2024-09-15T23:10:20.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:20 smithi111 ceph-mon[30629]: pgmap v1083: 97 pgs: 97 active+clean; 580 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.7 MiB/s wr, 522 op/s 2024-09-15T23:10:20.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:20 smithi192 ceph-mon[35427]: pgmap v1083: 97 pgs: 97 active+clean; 580 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.7 MiB/s wr, 522 op/s 2024-09-15T23:10:20.701 INFO:tasks.workunit.client.0.smithi111.stdout: 10 219195 17.44 MB/sec execute 528 sec latency 45.142 ms 2024-09-15T23:10:21.701 INFO:tasks.workunit.client.0.smithi111.stdout: 10 219804 17.47 MB/sec execute 529 sec latency 283.655 ms 2024-09-15T23:10:22.291 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:22 smithi111 ceph-mon[30629]: pgmap v1084: 97 pgs: 97 active+clean; 587 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.4 MiB/s wr, 474 op/s 2024-09-15T23:10:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.354+0000 7fa153e00640 1 -- 172.21.15.111:0/1482354743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa14c073250 msgr2=0x7fa14c0736d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.354+0000 7fa153e00640 1 --2- 172.21.15.111:0/1482354743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa14c073250 0x7fa14c0736d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa138009900 tx=0x7fa13802a400 comp rx=0 tx=0).stop 2024-09-15T23:10:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.354+0000 7fa153e00640 1 -- 172.21.15.111:0/1482354743 shutdown_connections 2024-09-15T23:10:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.354+0000 7fa153e00640 1 --2- 172.21.15.111:0/1482354743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa14c073c10 0x7fa14c10a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.354+0000 7fa153e00640 1 --2- 172.21.15.111:0/1482354743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa14c073250 0x7fa14c0736d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.354+0000 7fa153e00640 1 --2- 172.21.15.111:0/1482354743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa14c109560 0x7fa14c109960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.354+0000 7fa153e00640 1 -- 172.21.15.111:0/1482354743 >> 172.21.15.111:0/1482354743 conn(0x7fa14c06ece0 msgr2=0x7fa14c071120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:10:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.354+0000 7fa153e00640 1 -- 172.21.15.111:0/1482354743 shutdown_connections 2024-09-15T23:10:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.354+0000 7fa153e00640 1 -- 172.21.15.111:0/1482354743 wait complete. 2024-09-15T23:10:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.357+0000 7fa153e00640 1 Processor -- start 2024-09-15T23:10:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.357+0000 7fa153e00640 1 -- start start 2024-09-15T23:10:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.359+0000 7fa153e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa14c073c10 0x7fa14c1198f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.359+0000 7fa153e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa14c109560 0x7fa14c119e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.359+0000 7fa153e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa14c1a3e70 0x7fa14c1a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.359+0000 7fa153e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c10de30 con 0x7fa14c073c10 2024-09-15T23:10:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.359+0000 7fa153e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c10dcb0 con 0x7fa14c109560 2024-09-15T23:10:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.359+0000 7fa153e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c10dfb0 con 0x7fa14c1a3e70 2024-09-15T23:10:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.359+0000 7fa151e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa14c1a3e70 0x7fa14c1a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.359+0000 7fa151e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa14c1a3e70 0x7fa14c1a6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:51112/0 (socket says 172.21.15.111:51112) 2024-09-15T23:10:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.359+0000 7fa151e00640 1 -- 172.21.15.111:0/3989532512 learned_addr learned my addr 172.21.15.111:0/3989532512 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:10:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.359+0000 7fa150a00640 1 --2- 172.21.15.111:0/3989532512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa14c109560 0x7fa14c119e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.360+0000 7fa151400640 1 --2- 172.21.15.111:0/3989532512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa14c073c10 0x7fa14c1198f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.360+0000 7fa151e00640 1 -- 172.21.15.111:0/3989532512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa14c109560 msgr2=0x7fa14c119e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.360+0000 7fa151e00640 1 --2- 172.21.15.111:0/3989532512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa14c109560 0x7fa14c119e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.360+0000 7fa151e00640 1 -- 172.21.15.111:0/3989532512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa14c073c10 msgr2=0x7fa14c1198f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.360+0000 7fa151e00640 1 --2- 172.21.15.111:0/3989532512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa14c073c10 0x7fa14c1198f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.360+0000 7fa151e00640 1 -- 172.21.15.111:0/3989532512 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14c1a6870 con 0x7fa14c1a3e70 2024-09-15T23:10:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.360+0000 7fa151400640 1 --2- 172.21.15.111:0/3989532512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa14c073c10 0x7fa14c1198f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T23:10:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.361+0000 7fa151e00640 1 --2- 172.21.15.111:0/3989532512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa14c1a3e70 0x7fa14c1a6330 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa14400b970 tx=0x7fa14400be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:10:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.361+0000 7fa14a000640 1 -- 172.21.15.111:0/3989532512 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144019070 con 0x7fa14c1a3e70 2024-09-15T23:10:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.361+0000 7fa14a000640 1 -- 172.21.15.111:0/3989532512 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa14400cb40 con 0x7fa14c1a3e70 2024-09-15T23:10:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.361+0000 7fa14a000640 1 -- 172.21.15.111:0/3989532512 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144007650 con 0x7fa14c1a3e70 2024-09-15T23:10:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.361+0000 7fa153e00640 1 -- 172.21.15.111:0/3989532512 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa14c1a6b00 con 0x7fa14c1a3e70 2024-09-15T23:10:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.361+0000 7fa153e00640 1 -- 172.21.15.111:0/3989532512 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa14c1a7050 con 0x7fa14c1a3e70 2024-09-15T23:10:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa14a000640 1 -- 172.21.15.111:0/3989532512 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fa14401c070 con 0x7fa14c1a3e70 2024-09-15T23:10:22.369 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa156e30740 1 Processor -- start 2024-09-15T23:10:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa156e30740 1 -- start start 2024-09-15T23:10:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa156e30740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603b32ac090 0x5603b32ac490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa156e30740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603b32ac9d0 0x5603b32b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa156e30740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603b32b1a60 0x5603b32b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa156e30740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5603b3211100 con 0x5603b32ac090 2024-09-15T23:10:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa156e30740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5603b31b91b0 con 0x5603b32b1a60 2024-09-15T23:10:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa156e30740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5603b320d0c0 con 0x5603b32ac9d0 2024-09-15T23:10:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa150a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603b32ac090 0x5603b32ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa150a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603b32ac090 0x5603b32ac490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:35864/0 (socket says 172.21.15.111:35864) 2024-09-15T23:10:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa150a00640 1 -- 172.21.15.111:0/1773418975 learned_addr learned my addr 172.21.15.111:0/1773418975 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:10:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa151400640 1 --2- 172.21.15.111:0/1773418975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603b32ac9d0 0x5603b32b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa150a00640 1 --2- 172.21.15.111:0/1773418975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603b32b1a60 0x5603b32b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:22.372 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa14a000640 1 --2- 172.21.15.111:0/3989532512 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa1340a33c0 0x7fa1340a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa150a00640 1 -- 172.21.15.111:0/1773418975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603b32ac9d0 msgr2=0x5603b32b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:10:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa150a00640 1 --2- 172.21.15.111:0/1773418975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603b32ac9d0 0x5603b32b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa150a00640 1 -- 172.21.15.111:0/1773418975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603b32b1a60 msgr2=0x5603b32b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:10:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa150a00640 1 --2- 172.21.15.111:0/1773418975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603b32b1a60 0x5603b32b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa150a00640 1 -- 172.21.15.111:0/1773418975 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603b2f24210 con 0x5603b32ac090 2024-09-15T23:10:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa14a000640 1 -- 172.21.15.111:0/3989532512 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fa1440cedf0 con 0x7fa14c1a3e70 2024-09-15T23:10:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa151e00640 1 --2- 172.21.15.111:0/3989532512 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa1340a33c0 0x7fa1340a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa150a00640 1 --2- 172.21.15.111:0/1773418975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603b32ac090 0x5603b32ac490 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa138009850 tx=0x7fa138005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3d730fcfa7328b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:10:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa13e600640 1 -- 172.21.15.111:0/1773418975 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138035030 con 0x5603b32ac090 2024-09-15T23:10:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa13e600640 1 -- 172.21.15.111:0/1773418975 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa138005510 con 0x5603b32ac090 2024-09-15T23:10:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa13e600640 1 -- 172.21.15.111:0/1773418975 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138004920 con 0x5603b32ac090 2024-09-15T23:10:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa156e30740 1 -- 172.21.15.111:0/1773418975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603b32ac090 msgr2=0x5603b32ac490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa156e30740 1 --2- 172.21.15.111:0/1773418975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603b32ac090 0x5603b32ac490 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa138009850 tx=0x7fa138005fd0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.367+0000 7fa151e00640 1 --2- 172.21.15.111:0/3989532512 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa1340a33c0 0x7fa1340a5880 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa1440a3bf0 tx=0x7fa144004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:10:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.372+0000 7fa156e30740 1 -- 172.21.15.111:0/1773418975 shutdown_connections 2024-09-15T23:10:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.372+0000 7fa156e30740 1 --2- 172.21.15.111:0/1773418975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603b32b1a60 0x5603b32b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.372+0000 7fa156e30740 1 --2- 172.21.15.111:0/1773418975 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603b32ac9d0 0x5603b32b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.372+0000 7fa156e30740 1 --2- 172.21.15.111:0/1773418975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603b32ac090 0x5603b32ac490 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.372+0000 7fa156e30740 1 -- 172.21.15.111:0/1773418975 >> 172.21.15.111:0/1773418975 conn(0x5603b32a7700 msgr2=0x5603b32a7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:10:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.372+0000 7fa156e30740 1 -- 172.21.15.111:0/1773418975 shutdown_connections 2024-09-15T23:10:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.372+0000 7fa156e30740 1 -- 172.21.15.111:0/1773418975 wait complete. 2024-09-15T23:10:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.372+0000 7fa156e30740 1 -- start start 2024-09-15T23:10:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.372+0000 7fa156e30740 1 Processor -- start 2024-09-15T23:10:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa156e30740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603b32ac090 0x5603b32b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa156e30740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603b32ac9d0 0x5603b32b6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa156e30740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603b32b1a60 0x5603b32bd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa156e30740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5603b32b49d0 con 0x5603b32ac9d0 2024-09-15T23:10:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa156e30740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5603b319a090 con 0x5603b32b1a60 2024-09-15T23:10:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa156e30740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5603b32b4b50 con 0x5603b32ac090 2024-09-15T23:10:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa156e30740 10 client.0 _ll_register_callbacks cb 0x5603b31eef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:10:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa151400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603b32ac9d0 0x5603b32b6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa151400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603b32ac9d0 0x5603b32b6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:35876/0 (socket says 172.21.15.111:35876) 2024-09-15T23:10:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa151400640 1 -- 172.21.15.111:0/983609033 learned_addr learned my addr 172.21.15.111:0/983609033 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:10:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa150a00640 1 --2- 172.21.15.111:0/983609033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603b32ac090 0x5603b32b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa150a00640 1 --2- 172.21.15.111:0/983609033 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603b32b1a60 0x5603b32bd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa151400640 1 -- 172.21.15.111:0/983609033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603b32ac090 msgr2=0x5603b32b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:10:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa151400640 1 --2- 172.21.15.111:0/983609033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603b32ac090 0x5603b32b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa151400640 1 -- 172.21.15.111:0/983609033 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603b32b1a60 msgr2=0x5603b32bd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:10:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa151400640 1 --2- 172.21.15.111:0/983609033 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603b32b1a60 0x5603b32bd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa151400640 1 -- 172.21.15.111:0/983609033 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603b2ff0620 con 0x5603b32ac9d0 2024-09-15T23:10:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa150a00640 1 --2- 172.21.15.111:0/983609033 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603b32b1a60 0x5603b32bd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:10:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa151400640 1 --2- 172.21.15.111:0/983609033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603b32ac9d0 0x5603b32b6780 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa1400136e0 tx=0x7fa140013ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e175f7908c2867 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:10:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa132a00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:10:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa132a00640 1 -- 172.21.15.111:0/983609033 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa140011dd0 con 0x5603b32ac9d0 2024-09-15T23:10:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa132a00640 1 -- 172.21.15.111:0/983609033 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fa140004020 con 0x5603b32ac9d0 2024-09-15T23:10:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa156e30740 1 -- 172.21.15.111:0/983609033 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa138005510 con 0x5603b32ac9d0 2024-09-15T23:10:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa132a00640 1 -- 172.21.15.111:0/983609033 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa140004220 con 0x5603b32ac9d0 2024-09-15T23:10:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa156e30740 10 client.16884 fetch_fsmap learned FSMap version 35 2024-09-15T23:10:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa156e30740 1 -- 172.21.15.111:0/983609033 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5603b32c06d0 con 0x5603b32ac9d0 2024-09-15T23:10:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa132a00640 1 -- 172.21.15.111:0/983609033 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa138005510 con 0x5603b32ac9d0 2024-09-15T23:10:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa132a00640 1 -- 172.21.15.111:0/983609033 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fa14001a070 con 0x5603b32ac9d0 2024-09-15T23:10:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa156e30740 10 client.16884 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:10:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa156e30740 10 client.16884 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:10:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa156e30740 20 client.16884 populate_metadata read hostname 'smithi111' 2024-09-15T23:10:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa156e30740 1 --2- 172.21.15.111:0/983609033 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5603b32c1dc0 0x5603b32c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa156e30740 4 client.16884 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:10:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa156e30740 1 -- 172.21.15.111:0/983609033 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603b321e830 con 0x5603b32c1dc0 2024-09-15T23:10:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.373+0000 7fa150a00640 1 --2- 172.21.15.111:0/983609033 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5603b32c1dc0 0x5603b32c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.379+0000 7fa132a00640 10 client.16884 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:10:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.379+0000 7fa150a00640 1 --2- 172.21.15.111:0/983609033 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5603b32c1dc0 0x5603b32c41b0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6930f2b0f4586e4f server_cookie=21e3d489e19e724e in_seq=0 out_seq=0 2024-09-15T23:10:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.379+0000 7fa132a00640 1 -- 172.21.15.111:0/983609033 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5603b321e830 con 0x5603b32c1dc0 2024-09-15T23:10:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.379+0000 7fa132a00640 10 client.16884 handle_command_reply: tid=0 2024-09-15T23:10:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.379+0000 7fa156e30740 1 client.16884 shutdown 2024-09-15T23:10:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.384+0000 7fa156e30740 1 -- 172.21.15.111:0/983609033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603b32ac9d0 msgr2=0x5603b32b6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.384+0000 7fa156e30740 1 --2- 172.21.15.111:0/983609033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603b32ac9d0 0x5603b32b6780 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa1400136e0 tx=0x7fa140013ba0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.384+0000 7fa156e30740 1 -- 172.21.15.111:0/983609033 shutdown_connections 2024-09-15T23:10:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.384+0000 7fa156e30740 1 --2- 172.21.15.111:0/983609033 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5603b32c1dc0 0x5603b32c41b0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.384+0000 7fa156e30740 1 --2- 172.21.15.111:0/983609033 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603b32b1a60 0x5603b32bd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.384+0000 7fa156e30740 1 --2- 172.21.15.111:0/983609033 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603b32ac9d0 0x5603b32b6780 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.384+0000 7fa156e30740 1 --2- 172.21.15.111:0/983609033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603b32ac090 0x5603b32b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.384+0000 7fa156e30740 1 -- 172.21.15.111:0/983609033 >> 172.21.15.111:0/983609033 conn(0x5603b32a7700 msgr2=0x5603b32a8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:10:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.384+0000 7fa132a00640 0 client.16884 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:10:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.384+0000 7fa156e30740 1 -- 172.21.15.111:0/983609033 shutdown_connections 2024-09-15T23:10:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.384+0000 7fa156e30740 1 -- 172.21.15.111:0/983609033 wait complete. 2024-09-15T23:10:22.386 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.384+0000 7fa156e30740 20 client.16884 trim_cache size 0 max 16384 2024-09-15T23:10:22.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:22 smithi121 ceph-mon[34815]: pgmap v1084: 97 pgs: 97 active+clean; 587 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.4 MiB/s wr, 474 op/s 2024-09-15T23:10:22.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.404+0000 7fa156e30740 1 Processor -- start 2024-09-15T23:10:22.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.404+0000 7fa156e30740 1 -- start start 2024-09-15T23:10:22.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.405+0000 7fa156e30740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603b32ac090 0x5603b32c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:22.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.405+0000 7fa156e30740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603b32ac9d0 0x5603b32c40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:22.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.405+0000 7fa156e30740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603b32b1a60 0x5603b32bd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:22.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.405+0000 7fa156e30740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c1b23c0 con 0x5603b32ac9d0 2024-09-15T23:10:22.406 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.405+0000 7fa156e30740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c1afc90 con 0x5603b32b1a60 2024-09-15T23:10:22.407 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.405+0000 7fa156e30740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c1adae0 con 0x5603b32ac090 2024-09-15T23:10:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.405+0000 7fa150a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603b32ac090 0x5603b32c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:22.408 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.405+0000 7fa150a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603b32ac090 0x5603b32c3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:51136/0 (socket says 172.21.15.111:51136) 2024-09-15T23:10:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.405+0000 7fa150a00640 1 -- 172.21.15.111:0/1011552601 learned_addr learned my addr 172.21.15.111:0/1011552601 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:10:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.405+0000 7fa150a00640 1 --2- 172.21.15.111:0/1011552601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603b32b1a60 0x5603b32bd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.405+0000 7fa151400640 1 --2- 172.21.15.111:0/1011552601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603b32ac9d0 0x5603b32c40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.405+0000 7fa150a00640 1 -- 172.21.15.111:0/1011552601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603b32b1a60 msgr2=0x5603b32bd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:10:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.405+0000 7fa150a00640 1 --2- 172.21.15.111:0/1011552601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603b32b1a60 0x5603b32bd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.405+0000 7fa150a00640 1 -- 172.21.15.111:0/1011552601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603b32ac9d0 msgr2=0x5603b32c40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:10:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.405+0000 7fa150a00640 1 --2- 172.21.15.111:0/1011552601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603b32ac9d0 0x5603b32c40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.405+0000 7fa150a00640 1 -- 172.21.15.111:0/1011552601 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603b32a8390 con 0x5603b32ac090 2024-09-15T23:10:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.405+0000 7fa151400640 1 --2- 172.21.15.111:0/1011552601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603b32ac9d0 0x5603b32c40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:10:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa150a00640 1 --2- 172.21.15.111:0/1011552601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603b32ac090 0x5603b32c3ba0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa13802a920 tx=0x7fa138041040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3935b64e05d15a87 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:10:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa13dc00640 1 -- 172.21.15.111:0/1011552601 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa138035040 con 0x5603b32ac090 2024-09-15T23:10:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa13dc00640 1 -- 172.21.15.111:0/1011552601 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa138031b90 con 0x5603b32ac090 2024-09-15T23:10:22.409 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa13dc00640 1 -- 172.21.15.111:0/1011552601 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1380049f0 con 0x5603b32ac090 2024-09-15T23:10:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa156e30740 1 -- 172.21.15.111:0/1011552601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603b32ac090 msgr2=0x5603b32c3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa156e30740 1 --2- 172.21.15.111:0/1011552601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603b32ac090 0x5603b32c3ba0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa13802a920 tx=0x7fa138041040 comp rx=0 tx=0).stop 2024-09-15T23:10:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa156e30740 1 -- 172.21.15.111:0/1011552601 shutdown_connections 2024-09-15T23:10:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa156e30740 1 --2- 172.21.15.111:0/1011552601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603b32b1a60 0x5603b32bd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa156e30740 1 --2- 172.21.15.111:0/1011552601 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603b32ac9d0 0x5603b32c40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa156e30740 1 --2- 172.21.15.111:0/1011552601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603b32ac090 0x5603b32c3ba0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa156e30740 1 -- 172.21.15.111:0/1011552601 >> 172.21.15.111:0/1011552601 conn(0x5603b32a7700 msgr2=0x5603b32adc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:10:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa156e30740 1 -- 172.21.15.111:0/1011552601 shutdown_connections 2024-09-15T23:10:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa156e30740 1 -- 172.21.15.111:0/1011552601 wait complete. 2024-09-15T23:10:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa156e30740 1 -- start start 2024-09-15T23:10:22.410 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa156e30740 1 Processor -- start 2024-09-15T23:10:22.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa156e30740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603b32ac090 0x5603b32ba790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:22.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa156e30740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603b32ac9d0 0x5603b32bacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:22.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa156e30740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603b32b1a60 0x5603b332aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:22.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa156e30740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c1ae150 con 0x5603b32ac9d0 2024-09-15T23:10:22.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa156e30740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c1af5c0 con 0x5603b32b1a60 2024-09-15T23:10:22.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa156e30740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa14c1ae820 con 0x5603b32ac090 2024-09-15T23:10:22.413 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa156e30740 10 client.0 _ll_register_callbacks cb 0x7fa140005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:10:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa151400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603b32ac9d0 0x5603b32bacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa151400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603b32ac9d0 0x5603b32bacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:35888/0 (socket says 172.21.15.111:35888) 2024-09-15T23:10:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa151400640 1 -- 172.21.15.111:0/3616728285 learned_addr learned my addr 172.21.15.111:0/3616728285 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:10:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa150a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603b32ac090 0x5603b32ba790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa150a00640 1 --2- 172.21.15.111:0/3616728285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603b32b1a60 0x5603b332aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa151400640 1 -- 172.21.15.111:0/3616728285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603b32ac090 msgr2=0x5603b32ba790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:10:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa151400640 1 --2- 172.21.15.111:0/3616728285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603b32ac090 0x5603b32ba790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa151400640 1 -- 172.21.15.111:0/3616728285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603b32b1a60 msgr2=0x5603b332aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:10:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa151400640 1 --2- 172.21.15.111:0/3616728285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603b32b1a60 0x5603b332aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa151400640 1 -- 172.21.15.111:0/3616728285 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603b332aff0 con 0x5603b32ac9d0 2024-09-15T23:10:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.406+0000 7fa150a00640 1 --2- 172.21.15.111:0/3616728285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603b32b1a60 0x5603b332aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:10:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.409+0000 7fa151400640 1 --2- 172.21.15.111:0/3616728285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603b32ac9d0 0x5603b32bacd0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa140000c00 tx=0x7fa140005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f76bc2e9ac4661 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:10:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.409+0000 7fa132a00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:10:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.409+0000 7fa132a00640 1 -- 172.21.15.111:0/3616728285 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa140011dd0 con 0x5603b32ac9d0 2024-09-15T23:10:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.409+0000 7fa156e30740 1 -- 172.21.15.111:0/3616728285 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa138031b90 con 0x5603b32ac9d0 2024-09-15T23:10:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.409+0000 7fa132a00640 1 -- 172.21.15.111:0/3616728285 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fa14002f440 con 0x5603b32ac9d0 2024-09-15T23:10:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.409+0000 7fa132a00640 1 -- 172.21.15.111:0/3616728285 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa14002f640 con 0x5603b32ac9d0 2024-09-15T23:10:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.409+0000 7fa156e30740 10 client.16890 fetch_fsmap learned FSMap version 35 2024-09-15T23:10:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.409+0000 7fa156e30740 1 -- 172.21.15.111:0/3616728285 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5603b332c330 con 0x5603b32ac9d0 2024-09-15T23:10:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.409+0000 7fa132a00640 1 -- 172.21.15.111:0/3616728285 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa138031b90 con 0x5603b32ac9d0 2024-09-15T23:10:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.409+0000 7fa132a00640 1 -- 172.21.15.111:0/3616728285 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fa14002d020 con 0x5603b32ac9d0 2024-09-15T23:10:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.409+0000 7fa156e30740 10 client.16890 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:10:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.409+0000 7fa156e30740 10 client.16890 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:10:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.409+0000 7fa156e30740 20 client.16890 populate_metadata read hostname 'smithi111' 2024-09-15T23:10:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.409+0000 7fa156e30740 1 --2- 172.21.15.111:0/3616728285 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5603b332cd10 0x5603b332f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.409+0000 7fa156e30740 4 client.16890 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:10:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.409+0000 7fa156e30740 1 -- 172.21.15.111:0/3616728285 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5603b32a2d80 con 0x5603b332cd10 2024-09-15T23:10:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.409+0000 7fa150a00640 1 --2- 172.21.15.111:0/3616728285 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5603b332cd10 0x5603b332f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.409+0000 7fa150a00640 1 --2- 172.21.15.111:0/3616728285 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5603b332cd10 0x5603b332f130 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e59683dfe3013abd server_cookie=85893847d1792faf in_seq=0 out_seq=0 2024-09-15T23:10:22.416 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.409+0000 7fa132a00640 10 client.16890 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:10:22.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:22 smithi192 ceph-mon[35427]: pgmap v1084: 97 pgs: 97 active+clean; 587 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.4 MiB/s wr, 474 op/s 2024-09-15T23:10:22.634 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.632+0000 7fa132a00640 1 -- 172.21.15.111:0/3616728285 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5603b32a2d80 con 0x5603b332cd10 2024-09-15T23:10:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.632+0000 7fa132a00640 10 client.16890 handle_command_reply: tid=0 2024-09-15T23:10:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.632+0000 7fa156e30740 1 client.16890 shutdown 2024-09-15T23:10:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.633+0000 7fa156e30740 1 -- 172.21.15.111:0/3616728285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603b32ac9d0 msgr2=0x5603b32bacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.633+0000 7fa156e30740 1 --2- 172.21.15.111:0/3616728285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603b32ac9d0 0x5603b32bacd0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa140000c00 tx=0x7fa140005cc0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.633+0000 7fa156e30740 1 -- 172.21.15.111:0/3616728285 shutdown_connections 2024-09-15T23:10:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.633+0000 7fa156e30740 1 --2- 172.21.15.111:0/3616728285 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5603b332cd10 0x5603b332f130 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.633+0000 7fa156e30740 1 --2- 172.21.15.111:0/3616728285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603b32b1a60 0x5603b332aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.633+0000 7fa156e30740 1 --2- 172.21.15.111:0/3616728285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5603b32ac9d0 0x5603b32bacd0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.633+0000 7fa132a00640 0 client.16890 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:10:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.633+0000 7fa156e30740 1 --2- 172.21.15.111:0/3616728285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5603b32ac090 0x5603b32ba790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.633+0000 7fa156e30740 1 -- 172.21.15.111:0/3616728285 >> 172.21.15.111:0/3616728285 conn(0x5603b32a7700 msgr2=0x5603b32adc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:10:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.633+0000 7fa156e30740 1 -- 172.21.15.111:0/3616728285 shutdown_connections 2024-09-15T23:10:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.633+0000 7fa156e30740 1 -- 172.21.15.111:0/3616728285 wait complete. 2024-09-15T23:10:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.633+0000 7fa156e30740 20 client.16890 trim_cache size 0 max 16384 2024-09-15T23:10:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:10:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:10:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:10:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:10:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.633+0000 7fa153e00640 1 -- 172.21.15.111:0/3989532512 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa1340a33c0 msgr2=0x7fa1340a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.633+0000 7fa153e00640 1 --2- 172.21.15.111:0/3989532512 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa1340a33c0 0x7fa1340a5880 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa1440a3bf0 tx=0x7fa144004a40 comp rx=0 tx=0).stop 2024-09-15T23:10:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.633+0000 7fa153e00640 1 -- 172.21.15.111:0/3989532512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa14c1a3e70 msgr2=0x7fa14c1a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.633+0000 7fa153e00640 1 --2- 172.21.15.111:0/3989532512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa14c1a3e70 0x7fa14c1a6330 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa14400b970 tx=0x7fa14400be30 comp rx=0 tx=0).stop 2024-09-15T23:10:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.634+0000 7fa153e00640 1 -- 172.21.15.111:0/3989532512 shutdown_connections 2024-09-15T23:10:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.634+0000 7fa153e00640 1 --2- 172.21.15.111:0/3989532512 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa14c1a3e70 0x7fa14c1a6330 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.634+0000 7fa153e00640 1 --2- 172.21.15.111:0/3989532512 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fa1340a33c0 0x7fa1340a5880 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.638 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.634+0000 7fa153e00640 1 --2- 172.21.15.111:0/3989532512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa14c109560 0x7fa14c119e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.638 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.634+0000 7fa153e00640 1 --2- 172.21.15.111:0/3989532512 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa14c073c10 0x7fa14c1198f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:22.638 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.634+0000 7fa153e00640 1 -- 172.21.15.111:0/3989532512 >> 172.21.15.111:0/3989532512 conn(0x7fa14c06ece0 msgr2=0x7fa14c0717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:10:22.638 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.634+0000 7fa153e00640 1 -- 172.21.15.111:0/3989532512 shutdown_connections 2024-09-15T23:10:22.638 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:22.634+0000 7fa153e00640 1 -- 172.21.15.111:0/3989532512 wait complete. 2024-09-15T23:10:22.701 INFO:tasks.workunit.client.0.smithi111.stdout: 10 219808 17.44 MB/sec execute 530 sec latency 1278.264 ms 2024-09-15T23:10:23.701 INFO:tasks.workunit.client.0.smithi111.stdout: 10 219957 17.43 MB/sec execute 531 sec latency 2278.352 ms 2024-09-15T23:10:24.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:24 smithi121 ceph-mon[34815]: pgmap v1085: 97 pgs: 97 active+clean; 597 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 4.3 MiB/s wr, 527 op/s 2024-09-15T23:10:24.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:24 smithi111 ceph-mon[30629]: pgmap v1085: 97 pgs: 97 active+clean; 597 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 4.3 MiB/s wr, 527 op/s 2024-09-15T23:10:24.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:24 smithi192 ceph-mon[35427]: pgmap v1085: 97 pgs: 97 active+clean; 597 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 4.3 MiB/s wr, 527 op/s 2024-09-15T23:10:24.701 INFO:tasks.workunit.client.0.smithi111.stdout: 10 220579 17.46 MB/sec execute 532 sec latency 2342.688 ms 2024-09-15T23:10:25.702 INFO:tasks.workunit.client.0.smithi111.stdout: 10 221366 17.50 MB/sec execute 533 sec latency 60.119 ms 2024-09-15T23:10:26.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:26 smithi111 ceph-mon[30629]: pgmap v1086: 97 pgs: 97 active+clean; 594 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.6 MiB/s wr, 537 op/s 2024-09-15T23:10:26.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:26 smithi121 ceph-mon[34815]: pgmap v1086: 97 pgs: 97 active+clean; 594 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.6 MiB/s wr, 537 op/s 2024-09-15T23:10:26.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:26 smithi192 ceph-mon[35427]: pgmap v1086: 97 pgs: 97 active+clean; 594 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 4.6 MiB/s wr, 537 op/s 2024-09-15T23:10:26.707 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222032 17.54 MB/sec execute 534 sec latency 181.872 ms 2024-09-15T23:10:27.707 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222035 17.50 MB/sec execute 535 sec latency 1173.152 ms 2024-09-15T23:10:28.230 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:28 smithi111 ceph-mon[30629]: pgmap v1087: 97 pgs: 97 active+clean; 598 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.7 MiB/s wr, 459 op/s 2024-09-15T23:10:28.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:28 smithi121 ceph-mon[34815]: pgmap v1087: 97 pgs: 97 active+clean; 598 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.7 MiB/s wr, 459 op/s 2024-09-15T23:10:28.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:28 smithi192 ceph-mon[35427]: pgmap v1087: 97 pgs: 97 active+clean; 598 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.7 MiB/s wr, 459 op/s 2024-09-15T23:10:28.707 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222035 17.47 MB/sec execute 536 sec latency 2173.220 ms 2024-09-15T23:10:29.707 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222064 17.44 MB/sec execute 537 sec latency 3173.271 ms 2024-09-15T23:10:30.342 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:30 smithi111 ceph-mon[30629]: pgmap v1088: 97 pgs: 97 active+clean; 614 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.4 MiB/s wr, 576 op/s 2024-09-15T23:10:30.513 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:30 smithi121 ceph-mon[34815]: pgmap v1088: 97 pgs: 97 active+clean; 614 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.4 MiB/s wr, 576 op/s 2024-09-15T23:10:30.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:30 smithi192 ceph-mon[35427]: pgmap v1088: 97 pgs: 97 active+clean; 614 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.4 MiB/s wr, 576 op/s 2024-09-15T23:10:30.707 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222070 17.41 MB/sec execute 538 sec latency 4173.323 ms 2024-09-15T23:10:31.708 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222070 17.38 MB/sec execute 539 sec latency 5173.394 ms 2024-09-15T23:10:32.308 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:32 smithi111 ceph-mon[30629]: pgmap v1089: 97 pgs: 97 active+clean; 612 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.9 MiB/s wr, 543 op/s 2024-09-15T23:10:32.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:32 smithi121 ceph-mon[34815]: pgmap v1089: 97 pgs: 97 active+clean; 612 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.9 MiB/s wr, 543 op/s 2024-09-15T23:10:32.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:32 smithi192 ceph-mon[35427]: pgmap v1089: 97 pgs: 97 active+clean; 612 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.9 MiB/s wr, 543 op/s 2024-09-15T23:10:32.708 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222072 17.34 MB/sec execute 540 sec latency 5819.809 ms 2024-09-15T23:10:33.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:33 smithi111 ceph-mon[30629]: pgmap v1090: 97 pgs: 97 active+clean; 602 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.6 MiB/s wr, 488 op/s 2024-09-15T23:10:33.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:33 smithi192 ceph-mon[35427]: pgmap v1090: 97 pgs: 97 active+clean; 602 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.6 MiB/s wr, 488 op/s 2024-09-15T23:10:33.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:33 smithi121 ceph-mon[34815]: pgmap v1090: 97 pgs: 97 active+clean; 602 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.6 MiB/s wr, 488 op/s 2024-09-15T23:10:33.708 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222073 17.31 MB/sec execute 541 sec latency 5406.807 ms 2024-09-15T23:10:34.708 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222073 17.28 MB/sec execute 542 sec latency 6406.880 ms 2024-09-15T23:10:35.708 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222073 17.25 MB/sec execute 543 sec latency 7406.928 ms 2024-09-15T23:10:35.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.765+0000 7f2ef6a00640 1 -- 172.21.15.111:0/2958602840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ef0101f40 msgr2=0x7f2ef0109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:35.766 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.765+0000 7f2ef6a00640 1 --2- 172.21.15.111:0/2958602840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ef0101f40 0x7f2ef0109db0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009be0 tx=0x7f2ee802a6f0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.767+0000 7f2ef6a00640 1 -- 172.21.15.111:0/2958602840 shutdown_connections 2024-09-15T23:10:35.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.767+0000 7f2ef6a00640 1 --2- 172.21.15.111:0/2958602840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ef0101f40 0x7f2ef0109db0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.767+0000 7f2ef6a00640 1 --2- 172.21.15.111:0/2958602840 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ef0101580 0x7f2ef0101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.767+0000 7f2ef6a00640 1 --2- 172.21.15.111:0/2958602840 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ef0103050 0x7f2ef0103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.768 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.767+0000 7f2ef6a00640 1 -- 172.21.15.111:0/2958602840 >> 172.21.15.111:0/2958602840 conn(0x7f2ef00fd0d0 msgr2=0x7f2ef00ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:10:35.769 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.767+0000 7f2ef6a00640 1 -- 172.21.15.111:0/2958602840 shutdown_connections 2024-09-15T23:10:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.767+0000 7f2ef6a00640 1 -- 172.21.15.111:0/2958602840 wait complete. 2024-09-15T23:10:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.768+0000 7f2ef6a00640 1 Processor -- start 2024-09-15T23:10:35.770 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.769+0000 7f2ef6a00640 1 -- start start 2024-09-15T23:10:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.769+0000 7f2ef6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ef0101580 0x7f2ef01a6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.769+0000 7f2ef6a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ef0101f40 0x7f2ef01a71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.769+0000 7f2ef6a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ef0103050 0x7f2ef01ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.769+0000 7f2ef6a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef010b560 con 0x7f2ef0101580 2024-09-15T23:10:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.769+0000 7f2ef6a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef010b3e0 con 0x7f2ef0101f40 2024-09-15T23:10:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.769+0000 7f2ef6a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef010b6e0 con 0x7f2ef0103050 2024-09-15T23:10:35.771 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.770+0000 7f2ef4c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ef0101f40 0x7f2ef01a71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.770+0000 7f2ef4c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ef0101f40 0x7f2ef01a71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:34292/0 (socket says 172.21.15.111:34292) 2024-09-15T23:10:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.770+0000 7f2ef4c00640 1 -- 172.21.15.111:0/3407768430 learned_addr learned my addr 172.21.15.111:0/3407768430 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:10:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.770+0000 7f2ef6000640 1 --2- 172.21.15.111:0/3407768430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ef0103050 0x7f2ef01ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.770+0000 7f2ef5600640 1 --2- 172.21.15.111:0/3407768430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ef0101580 0x7f2ef01a6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.771+0000 7f2ef6000640 1 -- 172.21.15.111:0/3407768430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ef0101f40 msgr2=0x7f2ef01a71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.771+0000 7f2ef6000640 1 --2- 172.21.15.111:0/3407768430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ef0101f40 0x7f2ef01a71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.772 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.771+0000 7f2ef6000640 1 -- 172.21.15.111:0/3407768430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ef0101580 msgr2=0x7f2ef01a6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:35.773 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.771+0000 7f2ef6000640 1 --2- 172.21.15.111:0/3407768430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ef0101580 0x7f2ef01a6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.773 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.771+0000 7f2ef6000640 1 -- 172.21.15.111:0/3407768430 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef01ae880 con 0x7f2ef0103050 2024-09-15T23:10:35.773 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.771+0000 7f2ef6000640 1 --2- 172.21.15.111:0/3407768430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ef0103050 0x7f2ef01ae1e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8034040 tx=0x7f2ee80315d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:10:35.773 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.771+0000 7f2eee000640 1 -- 172.21.15.111:0/3407768430 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8041020 con 0x7f2ef0103050 2024-09-15T23:10:35.773 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.771+0000 7f2eee000640 1 -- 172.21.15.111:0/3407768430 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ee8003e50 con 0x7f2ef0103050 2024-09-15T23:10:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.772+0000 7f2eee000640 1 -- 172.21.15.111:0/3407768430 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8036040 con 0x7f2ef0103050 2024-09-15T23:10:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.773+0000 7f2ef6a00640 1 -- 172.21.15.111:0/3407768430 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef01aeb10 con 0x7f2ef0103050 2024-09-15T23:10:35.774 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.773+0000 7f2ef6a00640 1 -- 172.21.15.111:0/3407768430 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef01aef80 con 0x7f2ef0103050 2024-09-15T23:10:35.779 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.778+0000 7f2efaec8740 1 Processor -- start 2024-09-15T23:10:35.780 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.779+0000 7f2eee000640 1 -- 172.21.15.111:0/3407768430 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f2ee8044020 con 0x7f2ef0103050 2024-09-15T23:10:35.782 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.781+0000 7f2eee000640 1 --2- 172.21.15.111:0/3407768430 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2ed80a3300 0x7f2ed80a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.781+0000 7f2eee000640 1 -- 172.21.15.111:0/3407768430 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f2ee80f0170 con 0x7f2ef0103050 2024-09-15T23:10:35.783 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.782+0000 7f2efaec8740 1 -- start start 2024-09-15T23:10:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.782+0000 7f2ef4c00640 1 --2- 172.21.15.111:0/3407768430 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2ed80a3300 0x7f2ed80a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.782+0000 7f2efaec8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5a1d50260 0x55f5a1d50660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.782+0000 7f2efaec8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5a1d50ba0 0x55f5a1d55660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:35.784 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.782+0000 7f2efaec8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f5a1d55c30 0x55f5a1d580f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.782+0000 7f2efaec8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f5a1cb5100 con 0x55f5a1d55c30 2024-09-15T23:10:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.782+0000 7f2efaec8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f5a1c5d1b0 con 0x55f5a1d50260 2024-09-15T23:10:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.782+0000 7f2efaec8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f5a1cb10c0 con 0x55f5a1d50ba0 2024-09-15T23:10:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.782+0000 7f2ef5600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5a1d50260 0x55f5a1d50660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.782+0000 7f2ef5600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5a1d50260 0x55f5a1d50660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:34308/0 (socket says 172.21.15.111:34308) 2024-09-15T23:10:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.782+0000 7f2ef5600640 1 -- 172.21.15.111:0/545160157 learned_addr learned my addr 172.21.15.111:0/545160157 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:10:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.783+0000 7f2ef6000640 1 --2- 172.21.15.111:0/545160157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f5a1d55c30 0x55f5a1d580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.783+0000 7f2ef5600640 1 -- 172.21.15.111:0/545160157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5a1d50ba0 msgr2=0x55f5a1d55660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:10:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.783+0000 7f2ef5600640 1 --2- 172.21.15.111:0/545160157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5a1d50ba0 0x55f5a1d55660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.783+0000 7f2ef5600640 1 -- 172.21.15.111:0/545160157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f5a1d55c30 msgr2=0x55f5a1d580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:10:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.783+0000 7f2ef5600640 1 --2- 172.21.15.111:0/545160157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f5a1d55c30 0x55f5a1d580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.783+0000 7f2ef5600640 1 -- 172.21.15.111:0/545160157 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5a19c8210 con 0x55f5a1d50260 2024-09-15T23:10:35.785 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.783+0000 7f2ef4c00640 1 --2- 172.21.15.111:0/3407768430 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2ed80a3300 0x7f2ed80a57c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0058c0 tx=0x7f2edc013040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:10:35.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.783+0000 7f2ef5600640 1 --2- 172.21.15.111:0/545160157 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5a1d50260 0x55f5a1d50660 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4006a70 tx=0x7f2ee400bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b876e8922b762ac server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:10:35.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.783+0000 7f2ee2600640 1 -- 172.21.15.111:0/545160157 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee400d950 con 0x55f5a1d50260 2024-09-15T23:10:35.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.783+0000 7f2ee2600640 1 -- 172.21.15.111:0/545160157 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ee4018070 con 0x55f5a1d50260 2024-09-15T23:10:35.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.783+0000 7f2ee2600640 1 -- 172.21.15.111:0/545160157 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee40123e0 con 0x55f5a1d50260 2024-09-15T23:10:35.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.784+0000 7f2efaec8740 1 -- 172.21.15.111:0/545160157 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5a1d50260 msgr2=0x55f5a1d50660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:35.786 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.784+0000 7f2efaec8740 1 --2- 172.21.15.111:0/545160157 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5a1d50260 0x55f5a1d50660 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4006a70 tx=0x7f2ee400bc80 comp rx=0 tx=0).stop 2024-09-15T23:10:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.786+0000 7f2efaec8740 1 -- 172.21.15.111:0/545160157 shutdown_connections 2024-09-15T23:10:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.786+0000 7f2efaec8740 1 --2- 172.21.15.111:0/545160157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f5a1d55c30 0x55f5a1d580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.786+0000 7f2efaec8740 1 --2- 172.21.15.111:0/545160157 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5a1d50ba0 0x55f5a1d55660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.786+0000 7f2efaec8740 1 --2- 172.21.15.111:0/545160157 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5a1d50260 0x55f5a1d50660 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.786+0000 7f2efaec8740 1 -- 172.21.15.111:0/545160157 >> 172.21.15.111:0/545160157 conn(0x55f5a1d4b8d0 msgr2=0x55f5a1d4bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:10:35.787 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.786+0000 7f2efaec8740 1 -- 172.21.15.111:0/545160157 shutdown_connections 2024-09-15T23:10:35.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.786+0000 7f2efaec8740 1 -- 172.21.15.111:0/545160157 wait complete. 2024-09-15T23:10:35.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.787+0000 7f2efaec8740 1 -- start start 2024-09-15T23:10:35.788 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.787+0000 7f2efaec8740 1 Processor -- start 2024-09-15T23:10:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.788+0000 7f2efaec8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f5a1d50260 0x55f5a1d5a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.788+0000 7f2efaec8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5a1d50ba0 0x55f5a1d5a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.788+0000 7f2efaec8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5a1d55c30 0x55f5a1d619c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.788+0000 7f2efaec8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f5a1d58c00 con 0x55f5a1d50260 2024-09-15T23:10:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.788+0000 7f2efaec8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f5a1c3e090 con 0x55f5a1d50ba0 2024-09-15T23:10:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.788+0000 7f2efaec8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f5a1d58d80 con 0x55f5a1d55c30 2024-09-15T23:10:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.788+0000 7f2ef5600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f5a1d50260 0x55f5a1d5a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.788+0000 7f2ef5600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f5a1d50260 0x55f5a1d5a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43266/0 (socket says 172.21.15.111:43266) 2024-09-15T23:10:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.788+0000 7f2ef5600640 1 -- 172.21.15.111:0/3889230182 learned_addr learned my addr 172.21.15.111:0/3889230182 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:10:35.790 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.788+0000 7f2ef6000640 1 --2- 172.21.15.111:0/3889230182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5a1d55c30 0x55f5a1d619c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.788+0000 7f2efaec8740 10 client.0 _ll_register_callbacks cb 0x55f5a1c92f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:10:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.789+0000 7f2ef5600640 1 -- 172.21.15.111:0/3889230182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5a1d55c30 msgr2=0x55f5a1d619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:10:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.789+0000 7f2ef5600640 1 --2- 172.21.15.111:0/3889230182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5a1d55c30 0x55f5a1d619c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.789+0000 7f2ef5600640 1 -- 172.21.15.111:0/3889230182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5a1d50ba0 msgr2=0x55f5a1d5a980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:10:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.789+0000 7f2ef5600640 1 --2- 172.21.15.111:0/3889230182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5a1d50ba0 0x55f5a1d5a980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.789+0000 7f2ef5600640 1 -- 172.21.15.111:0/3889230182 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5a1a94620 con 0x55f5a1d50260 2024-09-15T23:10:35.791 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.789+0000 7f2ef6000640 1 --2- 172.21.15.111:0/3889230182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5a1d55c30 0x55f5a1d619c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:10:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.789+0000 7f2ef5600640 1 --2- 172.21.15.111:0/3889230182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f5a1d50260 0x55f5a1d5a440 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4000c00 tx=0x7f2ee40136b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1bc6630c8c46743 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:10:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.789+0000 7f2ed6a00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:10:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.789+0000 7f2ed6a00640 1 -- 172.21.15.111:0/3889230182 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4014ed0 con 0x55f5a1d50260 2024-09-15T23:10:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.789+0000 7f2ed6a00640 1 -- 172.21.15.111:0/3889230182 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f2ee400db50 con 0x55f5a1d50260 2024-09-15T23:10:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.789+0000 7f2efaec8740 1 -- 172.21.15.111:0/3889230182 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ee4018070 con 0x55f5a1d50260 2024-09-15T23:10:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.790+0000 7f2ed6a00640 1 -- 172.21.15.111:0/3889230182 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2ee400dd90 con 0x55f5a1d50260 2024-09-15T23:10:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.790+0000 7f2efaec8740 10 client.16902 fetch_fsmap learned FSMap version 35 2024-09-15T23:10:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.790+0000 7f2efaec8740 1 -- 172.21.15.111:0/3889230182 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f5a1d64700 con 0x55f5a1d50260 2024-09-15T23:10:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.790+0000 7f2ed6a00640 1 -- 172.21.15.111:0/3889230182 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ee4018070 con 0x55f5a1d50260 2024-09-15T23:10:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.790+0000 7f2ed6a00640 1 -- 172.21.15.111:0/3889230182 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f2ee401c580 con 0x55f5a1d50260 2024-09-15T23:10:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.790+0000 7f2efaec8740 10 client.16902 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:10:35.793 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.790+0000 7f2efaec8740 10 client.16902 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:10:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.790+0000 7f2efaec8740 20 client.16902 populate_metadata read hostname 'smithi111' 2024-09-15T23:10:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.790+0000 7f2efaec8740 1 --2- 172.21.15.111:0/3889230182 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f5a1d65bc0 0x55f5a1d67fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.790+0000 7f2efaec8740 4 client.16902 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:10:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.790+0000 7f2efaec8740 1 -- 172.21.15.111:0/3889230182 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5a1cc2830 con 0x55f5a1d65bc0 2024-09-15T23:10:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.791+0000 7f2ef4c00640 1 --2- 172.21.15.111:0/3889230182 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f5a1d65bc0 0x55f5a1d67fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.791+0000 7f2ef4c00640 1 --2- 172.21.15.111:0/3889230182 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f5a1d65bc0 0x55f5a1d67fb0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60b01ce791b47cd7 server_cookie=a5a6a448283f0beb in_seq=0 out_seq=0 2024-09-15T23:10:35.794 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.791+0000 7f2ed6a00640 10 client.16902 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:10:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.792+0000 7f2ed6a00640 1 -- 172.21.15.111:0/3889230182 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f5a1cc2830 con 0x55f5a1d65bc0 2024-09-15T23:10:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.792+0000 7f2ed6a00640 10 client.16902 handle_command_reply: tid=0 2024-09-15T23:10:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.792+0000 7f2efaec8740 1 client.16902 shutdown 2024-09-15T23:10:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.792+0000 7f2efaec8740 1 -- 172.21.15.111:0/3889230182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f5a1d50260 msgr2=0x55f5a1d5a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.792+0000 7f2efaec8740 1 --2- 172.21.15.111:0/3889230182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f5a1d50260 0x55f5a1d5a440 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4000c00 tx=0x7f2ee40136b0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.792+0000 7f2efaec8740 1 -- 172.21.15.111:0/3889230182 shutdown_connections 2024-09-15T23:10:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.792+0000 7f2efaec8740 1 --2- 172.21.15.111:0/3889230182 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f5a1d65bc0 0x55f5a1d67fb0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.795 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.792+0000 7f2efaec8740 1 --2- 172.21.15.111:0/3889230182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5a1d55c30 0x55f5a1d619c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.792+0000 7f2efaec8740 1 --2- 172.21.15.111:0/3889230182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5a1d50ba0 0x55f5a1d5a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.792+0000 7f2efaec8740 1 --2- 172.21.15.111:0/3889230182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f5a1d50260 0x55f5a1d5a440 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.792+0000 7f2efaec8740 1 -- 172.21.15.111:0/3889230182 >> 172.21.15.111:0/3889230182 conn(0x55f5a1d4b8d0 msgr2=0x55f5a1d4d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:10:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.792+0000 7f2ed6a00640 0 client.16902 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:10:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.793+0000 7f2efaec8740 1 -- 172.21.15.111:0/3889230182 shutdown_connections 2024-09-15T23:10:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.793+0000 7f2efaec8740 1 -- 172.21.15.111:0/3889230182 wait complete. 2024-09-15T23:10:35.796 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.793+0000 7f2efaec8740 20 client.16902 trim_cache size 0 max 16384 2024-09-15T23:10:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.808+0000 7f2efaec8740 1 Processor -- start 2024-09-15T23:10:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.808+0000 7f2efaec8740 1 -- start start 2024-09-15T23:10:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.808+0000 7f2efaec8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5a1d50260 0x55f5a1d50640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.808+0000 7f2efaec8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5a1d50ba0 0x55f5a1d618e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:35.810 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.808+0000 7f2efaec8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f5a1d55c30 0x55f5a1d5a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.808+0000 7f2efaec8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef01ba420 con 0x55f5a1d55c30 2024-09-15T23:10:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.808+0000 7f2efaec8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef01b7cf0 con 0x55f5a1d50ba0 2024-09-15T23:10:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.808+0000 7f2efaec8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef01b5b40 con 0x55f5a1d50260 2024-09-15T23:10:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.808+0000 7f2ef6000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f5a1d55c30 0x55f5a1d5a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.808+0000 7f2ef6000640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f5a1d55c30 0x55f5a1d5a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43274/0 (socket says 172.21.15.111:43274) 2024-09-15T23:10:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.808+0000 7f2ef6000640 1 -- 172.21.15.111:0/201450219 learned_addr learned my addr 172.21.15.111:0/201450219 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:10:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.808+0000 7f2ef4c00640 1 --2- 172.21.15.111:0/201450219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5a1d50ba0 0x55f5a1d618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.809+0000 7f2ef5600640 1 --2- 172.21.15.111:0/201450219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5a1d50260 0x55f5a1d50640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.809+0000 7f2ef6000640 1 -- 172.21.15.111:0/201450219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5a1d50260 msgr2=0x55f5a1d50640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:10:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.809+0000 7f2ef6000640 1 --2- 172.21.15.111:0/201450219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5a1d50260 0x55f5a1d50640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.809+0000 7f2ef6000640 1 -- 172.21.15.111:0/201450219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5a1d50ba0 msgr2=0x55f5a1d618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:10:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.809+0000 7f2ef6000640 1 --2- 172.21.15.111:0/201450219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5a1d50ba0 0x55f5a1d618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.811 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.809+0000 7f2ef6000640 1 -- 172.21.15.111:0/201450219 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5a1d58020 con 0x55f5a1d55c30 2024-09-15T23:10:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.809+0000 7f2ef5600640 1 --2- 172.21.15.111:0/201450219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5a1d50260 0x55f5a1d50640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:10:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.809+0000 7f2ef6000640 1 --2- 172.21.15.111:0/201450219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f5a1d55c30 0x55f5a1d5a1f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2ee80b9170 tx=0x7f2ee802abf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494122ab819319d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:10:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.809+0000 7f2ee1c00640 1 -- 172.21.15.111:0/201450219 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee80ba970 con 0x55f5a1d55c30 2024-09-15T23:10:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.809+0000 7f2ee1c00640 1 -- 172.21.15.111:0/201450219 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ee80bcf30 con 0x55f5a1d55c30 2024-09-15T23:10:35.812 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.809+0000 7f2ee1c00640 1 -- 172.21.15.111:0/201450219 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee80babc0 con 0x55f5a1d55c30 2024-09-15T23:10:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.810+0000 7f2efaec8740 1 -- 172.21.15.111:0/201450219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f5a1d55c30 msgr2=0x55f5a1d5a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.810+0000 7f2efaec8740 1 --2- 172.21.15.111:0/201450219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f5a1d55c30 0x55f5a1d5a1f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2ee80b9170 tx=0x7f2ee802abf0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.810+0000 7f2efaec8740 1 -- 172.21.15.111:0/201450219 shutdown_connections 2024-09-15T23:10:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.810+0000 7f2efaec8740 1 --2- 172.21.15.111:0/201450219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f5a1d55c30 0x55f5a1d5a1f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.810+0000 7f2efaec8740 1 --2- 172.21.15.111:0/201450219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5a1d50ba0 0x55f5a1d618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.810+0000 7f2efaec8740 1 --2- 172.21.15.111:0/201450219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5a1d50260 0x55f5a1d50640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.810+0000 7f2efaec8740 1 -- 172.21.15.111:0/201450219 >> 172.21.15.111:0/201450219 conn(0x55f5a1d4b8d0 msgr2=0x55f5a1d51de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:10:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.810+0000 7f2efaec8740 1 -- 172.21.15.111:0/201450219 shutdown_connections 2024-09-15T23:10:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.810+0000 7f2efaec8740 1 -- 172.21.15.111:0/201450219 wait complete. 2024-09-15T23:10:35.815 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.810+0000 7f2efaec8740 1 -- start start 2024-09-15T23:10:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.811+0000 7f2efaec8740 1 Processor -- start 2024-09-15T23:10:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.811+0000 7f2efaec8740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f5a1d50260 0x55f5a1d5e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.811+0000 7f2efaec8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5a1d50ba0 0x55f5a1d5eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.811+0000 7f2efaec8740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5a1d55c30 0x55f5a1dce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.811+0000 7f2efaec8740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef01b61b0 con 0x55f5a1d50260 2024-09-15T23:10:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.811+0000 7f2efaec8740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef01b7620 con 0x55f5a1d55c30 2024-09-15T23:10:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.811+0000 7f2efaec8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef01b6880 con 0x55f5a1d50ba0 2024-09-15T23:10:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.811+0000 7f2efaec8740 10 client.0 _ll_register_callbacks cb 0x7f2ee400b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:10:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.811+0000 7f2ef5600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f5a1d50260 0x55f5a1d5e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.811+0000 7f2ef5600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f5a1d50260 0x55f5a1d5e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:43276/0 (socket says 172.21.15.111:43276) 2024-09-15T23:10:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.811+0000 7f2ef5600640 1 -- 172.21.15.111:0/118713957 learned_addr learned my addr 172.21.15.111:0/118713957 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:10:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.811+0000 7f2ef4c00640 1 --2- 172.21.15.111:0/118713957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5a1d50ba0 0x55f5a1d5eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:35.816 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.812+0000 7f2ef5600640 1 -- 172.21.15.111:0/118713957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5a1d50ba0 msgr2=0x55f5a1d5eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:10:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.812+0000 7f2ef5600640 1 --2- 172.21.15.111:0/118713957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5a1d50ba0 0x55f5a1d5eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.812+0000 7f2ef5600640 1 -- 172.21.15.111:0/118713957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5a1d55c30 msgr2=0x55f5a1dce860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:10:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.812+0000 7f2ef5600640 1 --2- 172.21.15.111:0/118713957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5a1d55c30 0x55f5a1dce860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.812+0000 7f2ef5600640 1 -- 172.21.15.111:0/118713957 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5a1d61c70 con 0x55f5a1d50260 2024-09-15T23:10:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.812+0000 7f2ef5600640 1 --2- 172.21.15.111:0/118713957 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f5a1d50260 0x55f5a1d5e640 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400bf50 tx=0x7f2ee400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a60f72dbe845a02 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:10:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.812+0000 7f2ed6a00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:10:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.812+0000 7f2ed6a00640 1 -- 172.21.15.111:0/118713957 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4033070 con 0x55f5a1d50260 2024-09-15T23:10:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.812+0000 7f2ed6a00640 1 -- 172.21.15.111:0/118713957 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f2ee4004040 con 0x55f5a1d50260 2024-09-15T23:10:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.812+0000 7f2efaec8740 1 -- 172.21.15.111:0/118713957 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ee80bcf30 con 0x55f5a1d50260 2024-09-15T23:10:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.812+0000 7f2ed6a00640 1 -- 172.21.15.111:0/118713957 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2ee4004240 con 0x55f5a1d50260 2024-09-15T23:10:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.812+0000 7f2efaec8740 10 client.16908 fetch_fsmap learned FSMap version 35 2024-09-15T23:10:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.812+0000 7f2efaec8740 1 -- 172.21.15.111:0/118713957 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f5a1dcfe20 con 0x55f5a1d50260 2024-09-15T23:10:35.817 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.813+0000 7f2ed6a00640 1 -- 172.21.15.111:0/118713957 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ee80bcf30 con 0x55f5a1d50260 2024-09-15T23:10:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.813+0000 7f2ed6a00640 1 -- 172.21.15.111:0/118713957 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f2ee4003690 con 0x55f5a1d50260 2024-09-15T23:10:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.813+0000 7f2efaec8740 10 client.16908 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:10:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.813+0000 7f2efaec8740 10 client.16908 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:10:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.813+0000 7f2efaec8740 20 client.16908 populate_metadata read hostname 'smithi111' 2024-09-15T23:10:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.813+0000 7f2efaec8740 1 --2- 172.21.15.111:0/118713957 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f5a1dd0780 0x55f5a1dd2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.813+0000 7f2efaec8740 4 client.16908 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:10:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.813+0000 7f2efaec8740 1 -- 172.21.15.111:0/118713957 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f5a1d559f0 con 0x55f5a1dd0780 2024-09-15T23:10:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.813+0000 7f2ef4c00640 1 --2- 172.21.15.111:0/118713957 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f5a1dd0780 0x55f5a1dd2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.814+0000 7f2ef4c00640 1 --2- 172.21.15.111:0/118713957 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f5a1dd0780 0x55f5a1dd2ba0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3328cb0aa9dc185 server_cookie=9023039114f49a4e in_seq=0 out_seq=0 2024-09-15T23:10:35.818 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.814+0000 7f2ed6a00640 10 client.16908 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:10:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.927+0000 7f2ed6a00640 1 -- 172.21.15.111:0/118713957 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f5a1d559f0 con 0x55f5a1dd0780 2024-09-15T23:10:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.927+0000 7f2ed6a00640 10 client.16908 handle_command_reply: tid=0 2024-09-15T23:10:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.927+0000 7f2efaec8740 1 client.16908 shutdown 2024-09-15T23:10:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.927+0000 7f2efaec8740 1 -- 172.21.15.111:0/118713957 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f5a1d50260 msgr2=0x55f5a1d5e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.927+0000 7f2efaec8740 1 --2- 172.21.15.111:0/118713957 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f5a1d50260 0x55f5a1d5e640 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400bf50 tx=0x7f2ee400dc00 comp rx=0 tx=0).stop 2024-09-15T23:10:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.927+0000 7f2efaec8740 1 -- 172.21.15.111:0/118713957 shutdown_connections 2024-09-15T23:10:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.927+0000 7f2efaec8740 1 --2- 172.21.15.111:0/118713957 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f5a1dd0780 0x55f5a1dd2ba0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.927+0000 7f2efaec8740 1 --2- 172.21.15.111:0/118713957 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f5a1d55c30 0x55f5a1dce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.927+0000 7f2efaec8740 1 --2- 172.21.15.111:0/118713957 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f5a1d50ba0 0x55f5a1d5eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.927+0000 7f2efaec8740 1 --2- 172.21.15.111:0/118713957 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f5a1d50260 0x55f5a1d5e640 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.927+0000 7f2efaec8740 1 -- 172.21.15.111:0/118713957 >> 172.21.15.111:0/118713957 conn(0x55f5a1d4b8d0 msgr2=0x55f5a1d4a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:10:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.927+0000 7f2ed6a00640 0 client.16908 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:10:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.927+0000 7f2efaec8740 1 -- 172.21.15.111:0/118713957 shutdown_connections 2024-09-15T23:10:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.927+0000 7f2efaec8740 1 -- 172.21.15.111:0/118713957 wait complete. 2024-09-15T23:10:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.927+0000 7f2efaec8740 20 client.16908 trim_cache size 0 max 16384 2024-09-15T23:10:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:10:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:10:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:10:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:10:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.928+0000 7f2ef6a00640 1 -- 172.21.15.111:0/3407768430 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2ed80a3300 msgr2=0x7f2ed80a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.928+0000 7f2ef6a00640 1 --2- 172.21.15.111:0/3407768430 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2ed80a3300 0x7f2ed80a57c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0058c0 tx=0x7f2edc013040 comp rx=0 tx=0).stop 2024-09-15T23:10:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.928+0000 7f2ef6a00640 1 -- 172.21.15.111:0/3407768430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ef0103050 msgr2=0x7f2ef01ae1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.928+0000 7f2ef6a00640 1 --2- 172.21.15.111:0/3407768430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ef0103050 0x7f2ef01ae1e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8034040 tx=0x7f2ee80315d0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.928+0000 7f2ef6a00640 1 -- 172.21.15.111:0/3407768430 shutdown_connections 2024-09-15T23:10:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.928+0000 7f2ef6a00640 1 --2- 172.21.15.111:0/3407768430 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2ef0103050 0x7f2ef01ae1e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.928+0000 7f2ef6a00640 1 --2- 172.21.15.111:0/3407768430 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2ed80a3300 0x7f2ed80a57c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.928+0000 7f2ef6a00640 1 --2- 172.21.15.111:0/3407768430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2ef0101f40 0x7f2ef01a71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.928+0000 7f2ef6a00640 1 --2- 172.21.15.111:0/3407768430 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ef0101580 0x7f2ef01a6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.928+0000 7f2ef6a00640 1 -- 172.21.15.111:0/3407768430 >> 172.21.15.111:0/3407768430 conn(0x7f2ef00fd0d0 msgr2=0x7f2ef0105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:10:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.928+0000 7f2ef6a00640 1 -- 172.21.15.111:0/3407768430 shutdown_connections 2024-09-15T23:10:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:35.928+0000 7f2ef6a00640 1 -- 172.21.15.111:0/3407768430 wait complete. 2024-09-15T23:10:36.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:36 smithi192 ceph-mon[35427]: pgmap v1091: 97 pgs: 97 active+clean; 609 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.9 MiB/s wr, 372 op/s 2024-09-15T23:10:36.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:36 smithi121 ceph-mon[34815]: pgmap v1091: 97 pgs: 97 active+clean; 609 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.9 MiB/s wr, 372 op/s 2024-09-15T23:10:36.399 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:36 smithi111 ceph-mon[30629]: pgmap v1091: 97 pgs: 97 active+clean; 609 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.9 MiB/s wr, 372 op/s 2024-09-15T23:10:36.708 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222073 17.22 MB/sec execute 544 sec latency 8406.971 ms 2024-09-15T23:10:37.708 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222095 17.18 MB/sec execute 545 sec latency 9097.257 ms 2024-09-15T23:10:38.053 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:38 smithi111 ceph-mon[30629]: pgmap v1092: 97 pgs: 97 active+clean; 615 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.3 MiB/s wr, 352 op/s 2024-09-15T23:10:38.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:38 smithi192 ceph-mon[35427]: pgmap v1092: 97 pgs: 97 active+clean; 615 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.3 MiB/s wr, 352 op/s 2024-09-15T23:10:38.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:38 smithi121 ceph-mon[34815]: pgmap v1092: 97 pgs: 97 active+clean; 615 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.3 MiB/s wr, 352 op/s 2024-09-15T23:10:38.708 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222095 17.15 MB/sec execute 546 sec latency 6353.681 ms 2024-09-15T23:10:39.708 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222123 17.12 MB/sec execute 547 sec latency 7188.824 ms 2024-09-15T23:10:40.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:40 smithi192 ceph-mon[35427]: pgmap v1093: 97 pgs: 97 active+clean; 611 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.2 MiB/s wr, 344 op/s 2024-09-15T23:10:40.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:40 smithi121 ceph-mon[34815]: pgmap v1093: 97 pgs: 97 active+clean; 611 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.2 MiB/s wr, 344 op/s 2024-09-15T23:10:40.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:40 smithi111 ceph-mon[30629]: pgmap v1093: 97 pgs: 97 active+clean; 611 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.2 MiB/s wr, 344 op/s 2024-09-15T23:10:40.708 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222165 17.10 MB/sec execute 548 sec latency 8188.863 ms 2024-09-15T23:10:41.708 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222165 17.07 MB/sec execute 549 sec latency 9188.912 ms 2024-09-15T23:10:42.072 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:42 smithi111 ceph-mon[30629]: pgmap v1094: 97 pgs: 97 active+clean; 601 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 5.4 MiB/s wr, 258 op/s 2024-09-15T23:10:42.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:42 smithi192 ceph-mon[35427]: pgmap v1094: 97 pgs: 97 active+clean; 601 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 5.4 MiB/s wr, 258 op/s 2024-09-15T23:10:42.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:42 smithi121 ceph-mon[34815]: pgmap v1094: 97 pgs: 97 active+clean; 601 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 5.4 MiB/s wr, 258 op/s 2024-09-15T23:10:42.716 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222172 17.04 MB/sec execute 550 sec latency 9988.825 ms 2024-09-15T23:10:43.716 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222185 17.01 MB/sec execute 551 sec latency 4416.014 ms 2024-09-15T23:10:44.307 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:44 smithi111 ceph-mon[30629]: pgmap v1095: 97 pgs: 97 active+clean; 596 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 5.2 MiB/s wr, 249 op/s 2024-09-15T23:10:44.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:44 smithi192 ceph-mon[35427]: pgmap v1095: 97 pgs: 97 active+clean; 596 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 5.2 MiB/s wr, 249 op/s 2024-09-15T23:10:44.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:44 smithi121 ceph-mon[34815]: pgmap v1095: 97 pgs: 97 active+clean; 596 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 5.2 MiB/s wr, 249 op/s 2024-09-15T23:10:44.716 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222190 16.98 MB/sec execute 552 sec latency 5264.532 ms 2024-09-15T23:10:45.665 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:45 smithi111 ceph-mon[30629]: pgmap v1096: 97 pgs: 97 active+clean; 598 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 5.3 MiB/s wr, 246 op/s 2024-09-15T23:10:45.716 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222190 16.95 MB/sec execute 553 sec latency 6264.589 ms 2024-09-15T23:10:45.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:45 smithi192 ceph-mon[35427]: pgmap v1096: 97 pgs: 97 active+clean; 598 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 5.3 MiB/s wr, 246 op/s 2024-09-15T23:10:45.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:45 smithi121 ceph-mon[34815]: pgmap v1096: 97 pgs: 97 active+clean; 598 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 5.3 MiB/s wr, 246 op/s 2024-09-15T23:10:46.717 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222190 16.92 MB/sec execute 554 sec latency 7264.660 ms 2024-09-15T23:10:47.717 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222190 16.89 MB/sec execute 555 sec latency 8264.760 ms 2024-09-15T23:10:48.251 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:48 smithi111 ceph-mon[30629]: pgmap v1097: 97 pgs: 97 active+clean; 599 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 5.7 MiB/s wr, 261 op/s 2024-09-15T23:10:48.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:48 smithi192 ceph-mon[35427]: pgmap v1097: 97 pgs: 97 active+clean; 599 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 5.7 MiB/s wr, 261 op/s 2024-09-15T23:10:48.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:48 smithi121 ceph-mon[34815]: pgmap v1097: 97 pgs: 97 active+clean; 599 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 5.7 MiB/s wr, 261 op/s 2024-09-15T23:10:48.717 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222190 16.86 MB/sec execute 556 sec latency 9264.860 ms 2024-09-15T23:10:49.080 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.074+0000 7f7fb7200640 1 -- 172.21.15.111:0/2220353752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fb8104cf0 msgr2=0x7f7fb81070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:49.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.074+0000 7f7fb7200640 1 --2- 172.21.15.111:0/2220353752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fb8104cf0 0x7f7fb81070e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4009900 tx=0x7f7fa402a3f0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.074+0000 7f7fb7200640 1 -- 172.21.15.111:0/2220353752 shutdown_connections 2024-09-15T23:10:49.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.074+0000 7f7fb7200640 1 --2- 172.21.15.111:0/2220353752 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fb8107620 0x7f7fb8109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.081 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.074+0000 7f7fb7200640 1 --2- 172.21.15.111:0/2220353752 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fb8104cf0 0x7f7fb81070e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.074+0000 7f7fb7200640 1 --2- 172.21.15.111:0/2220353752 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fb81023c0 0x7f7fb81047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.074+0000 7f7fb7200640 1 -- 172.21.15.111:0/2220353752 >> 172.21.15.111:0/2220353752 conn(0x7f7fb80fbfa0 msgr2=0x7f7fb80fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:10:49.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.074+0000 7f7fb7200640 1 -- 172.21.15.111:0/2220353752 shutdown_connections 2024-09-15T23:10:49.082 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.074+0000 7f7fb7200640 1 -- 172.21.15.111:0/2220353752 wait complete. 2024-09-15T23:10:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.075+0000 7f7fb7200640 1 Processor -- start 2024-09-15T23:10:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.075+0000 7f7fb7200640 1 -- start start 2024-09-15T23:10:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.076+0000 7f7fb7200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fb81023c0 0x7f7fb8197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.076+0000 7f7fb7200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fb8104cf0 0x7f7fb8198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:49.083 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.076+0000 7f7fb7200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fb8107620 0x7f7fb819f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.076+0000 7f7fb7200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb810d650 con 0x7f7fb8104cf0 2024-09-15T23:10:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.076+0000 7f7fb7200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb810d4d0 con 0x7f7fb8107620 2024-09-15T23:10:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.076+0000 7f7fb7200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb810d7d0 con 0x7f7fb81023c0 2024-09-15T23:10:49.084 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.076+0000 7f7fb5400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fb8104cf0 0x7f7fb8198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:49.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.076+0000 7f7fb5400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fb8104cf0 0x7f7fb8198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39654/0 (socket says 172.21.15.111:39654) 2024-09-15T23:10:49.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.076+0000 7f7fb5e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fb81023c0 0x7f7fb8197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:49.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.076+0000 7f7fb5400640 1 -- 172.21.15.111:0/4080422816 learned_addr learned my addr 172.21.15.111:0/4080422816 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:10:49.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.076+0000 7f7fb6800640 1 --2- 172.21.15.111:0/4080422816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fb8107620 0x7f7fb819f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:49.085 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.076+0000 7f7fb5e00640 1 -- 172.21.15.111:0/4080422816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fb8107620 msgr2=0x7f7fb819f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.076+0000 7f7fb5e00640 1 --2- 172.21.15.111:0/4080422816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fb8107620 0x7f7fb819f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.076+0000 7f7fb5e00640 1 -- 172.21.15.111:0/4080422816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fb8104cf0 msgr2=0x7f7fb8198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.076+0000 7f7fb5e00640 1 --2- 172.21.15.111:0/4080422816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fb8104cf0 0x7f7fb8198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.076+0000 7f7fb5e00640 1 -- 172.21.15.111:0/4080422816 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb819f960 con 0x7f7fb81023c0 2024-09-15T23:10:49.086 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.076+0000 7f7fb6800640 1 --2- 172.21.15.111:0/4080422816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fb8107620 0x7f7fb819f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:10:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.077+0000 7f7fb5e00640 1 --2- 172.21.15.111:0/4080422816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fb81023c0 0x7f7fb8197d50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800e9f0 tx=0x7f7fa800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:10:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.077+0000 7f7faea00640 1 -- 172.21.15.111:0/4080422816 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa800ccc0 con 0x7f7fb81023c0 2024-09-15T23:10:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.077+0000 7f7fb7200640 1 -- 172.21.15.111:0/4080422816 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fb819fc50 con 0x7f7fb81023c0 2024-09-15T23:10:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.077+0000 7f7faea00640 1 -- 172.21.15.111:0/4080422816 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fa8010910 con 0x7f7fb81023c0 2024-09-15T23:10:49.087 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.077+0000 7f7faea00640 1 -- 172.21.15.111:0/4080422816 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa8018960 con 0x7f7fb81023c0 2024-09-15T23:10:49.088 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.077+0000 7f7fb7200640 1 -- 172.21.15.111:0/4080422816 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fb81a01a0 con 0x7f7fb81023c0 2024-09-15T23:10:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.080+0000 7f7faea00640 1 -- 172.21.15.111:0/4080422816 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f7fa8010ac0 con 0x7f7fb81023c0 2024-09-15T23:10:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.081+0000 7f7fbfe2c740 1 Processor -- start 2024-09-15T23:10:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.081+0000 7f7fbfe2c740 1 -- start start 2024-09-15T23:10:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.081+0000 7f7faea00640 1 --2- 172.21.15.111:0/4080422816 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7f9c0a7650 0x7f7f9c0a9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.082+0000 7f7faea00640 1 -- 172.21.15.111:0/4080422816 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f7fa80cfde0 con 0x7f7fb81023c0 2024-09-15T23:10:49.103 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.082+0000 7f7fb6800640 1 --2- 172.21.15.111:0/4080422816 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7f9c0a7650 0x7f7f9c0a9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.082+0000 7f7fbfe2c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de953af260 0x55de953af660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.082+0000 7f7fbfe2c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de953afba0 0x55de953b4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.082+0000 7f7fbfe2c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de953b4c30 0x55de953b70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.082+0000 7f7fbfe2c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55de95314100 con 0x55de953af260 2024-09-15T23:10:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.082+0000 7f7fbfe2c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55de952bc1b0 con 0x55de953b4c30 2024-09-15T23:10:49.104 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.082+0000 7f7fbfe2c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55de953100c0 con 0x55de953afba0 2024-09-15T23:10:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.082+0000 7f7fb5e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de953af260 0x55de953af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.082+0000 7f7fb5e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de953af260 0x55de953af660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39662/0 (socket says 172.21.15.111:39662) 2024-09-15T23:10:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.082+0000 7f7fb5e00640 1 -- 172.21.15.111:0/3059187390 learned_addr learned my addr 172.21.15.111:0/3059187390 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:10:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.082+0000 7f7fb6800640 1 --2- 172.21.15.111:0/3059187390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de953b4c30 0x55de953b70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:49.105 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.083+0000 7f7fb5e00640 1 -- 172.21.15.111:0/3059187390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de953afba0 msgr2=0x55de953b4660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:10:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.083+0000 7f7fb5e00640 1 --2- 172.21.15.111:0/3059187390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de953afba0 0x55de953b4660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.083+0000 7f7fb5e00640 1 -- 172.21.15.111:0/3059187390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de953b4c30 msgr2=0x55de953b70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:10:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.083+0000 7f7fb5e00640 1 --2- 172.21.15.111:0/3059187390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de953b4c30 0x55de953b70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.083+0000 7f7fb5e00640 1 -- 172.21.15.111:0/3059187390 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de95027210 con 0x55de953af260 2024-09-15T23:10:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.083+0000 7f7fb6800640 1 --2- 172.21.15.111:0/4080422816 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7f9c0a7650 0x7f7f9c0a9b10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7fb806a980 tx=0x7f7fb00107a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:10:49.106 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.083+0000 7f7fb5e00640 1 --2- 172.21.15.111:0/3059187390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de953af260 0x55de953af660 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8098d40 tx=0x7f7fa800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=521a1be497d3585d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:10:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.083+0000 7f7fa3400640 1 -- 172.21.15.111:0/3059187390 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa80b5070 con 0x55de953af260 2024-09-15T23:10:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.083+0000 7f7fa3400640 1 -- 172.21.15.111:0/3059187390 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fa809c400 con 0x55de953af260 2024-09-15T23:10:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.083+0000 7f7fa3400640 1 -- 172.21.15.111:0/3059187390 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa809b850 con 0x55de953af260 2024-09-15T23:10:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.084+0000 7f7fbfe2c740 1 -- 172.21.15.111:0/3059187390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de953af260 msgr2=0x55de953af660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.084+0000 7f7fbfe2c740 1 --2- 172.21.15.111:0/3059187390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de953af260 0x55de953af660 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8098d40 tx=0x7f7fa800cf60 comp rx=0 tx=0).stop 2024-09-15T23:10:49.107 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.084+0000 7f7fbfe2c740 1 -- 172.21.15.111:0/3059187390 shutdown_connections 2024-09-15T23:10:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.084+0000 7f7fbfe2c740 1 --2- 172.21.15.111:0/3059187390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de953b4c30 0x55de953b70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.084+0000 7f7fbfe2c740 1 --2- 172.21.15.111:0/3059187390 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de953afba0 0x55de953b4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.084+0000 7f7fbfe2c740 1 --2- 172.21.15.111:0/3059187390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de953af260 0x55de953af660 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.084+0000 7f7fbfe2c740 1 -- 172.21.15.111:0/3059187390 >> 172.21.15.111:0/3059187390 conn(0x55de953aa8d0 msgr2=0x55de953aacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:10:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.084+0000 7f7fbfe2c740 1 -- 172.21.15.111:0/3059187390 shutdown_connections 2024-09-15T23:10:49.108 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.084+0000 7f7fbfe2c740 1 -- 172.21.15.111:0/3059187390 wait complete. 2024-09-15T23:10:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.084+0000 7f7fbfe2c740 1 -- start start 2024-09-15T23:10:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.085+0000 7f7fbfe2c740 1 Processor -- start 2024-09-15T23:10:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.085+0000 7f7fbfe2c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de953af260 0x55de953b93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.085+0000 7f7fbfe2c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de953afba0 0x55de953b9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.085+0000 7f7fbfe2c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de953b4c30 0x55de953c0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:49.109 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.085+0000 7f7fbfe2c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55de953b7ba0 con 0x55de953af260 2024-09-15T23:10:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.085+0000 7f7fbfe2c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55de9529d090 con 0x55de953afba0 2024-09-15T23:10:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.085+0000 7f7fbfe2c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55de953b7d20 con 0x55de953b4c30 2024-09-15T23:10:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.085+0000 7f7fbfe2c740 10 client.0 _ll_register_callbacks cb 0x55de952f1f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:10:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.085+0000 7f7fb5400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de953afba0 0x55de953b9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.085+0000 7f7fb5e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de953af260 0x55de953b93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.085+0000 7f7fb5400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de953afba0 0x55de953b9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:38556/0 (socket says 172.21.15.111:38556) 2024-09-15T23:10:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.085+0000 7f7fb5400640 1 -- 172.21.15.111:0/400925564 learned_addr learned my addr 172.21.15.111:0/400925564 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:10:49.110 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.086+0000 7f7fb5400640 1 -- 172.21.15.111:0/400925564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de953b4c30 msgr2=0x55de953c0960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:10:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.086+0000 7f7fb5400640 1 --2- 172.21.15.111:0/400925564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de953b4c30 0x55de953c0960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.086+0000 7f7fb5400640 1 -- 172.21.15.111:0/400925564 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de953af260 msgr2=0x55de953b93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:10:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.086+0000 7f7fb5400640 1 --2- 172.21.15.111:0/400925564 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de953af260 0x55de953b93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.086+0000 7f7fb5400640 1 -- 172.21.15.111:0/400925564 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de950f3620 con 0x55de953afba0 2024-09-15T23:10:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.086+0000 7f7fb5e00640 1 --2- 172.21.15.111:0/400925564 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de953af260 0x55de953b93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:10:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.086+0000 7f7fb5400640 1 --2- 172.21.15.111:0/400925564 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de953afba0 0x55de953b9920 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7fa402a910 tx=0x7f7fa4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23aa05684fea6ff1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:10:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.086+0000 7f7f97e00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:10:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.086+0000 7f7f97e00640 1 -- 172.21.15.111:0/400925564 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4040070 con 0x55de953afba0 2024-09-15T23:10:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.086+0000 7f7fbfe2c740 1 -- 172.21.15.111:0/400925564 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fa809c400 con 0x55de953afba0 2024-09-15T23:10:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.086+0000 7f7f97e00640 1 -- 172.21.15.111:0/400925564 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f7fa4031ac0 con 0x55de953afba0 2024-09-15T23:10:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.087+0000 7f7f97e00640 1 -- 172.21.15.111:0/400925564 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7fa4035040 con 0x55de953afba0 2024-09-15T23:10:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.087+0000 7f7fbfe2c740 10 client.26873 fetch_fsmap learned FSMap version 35 2024-09-15T23:10:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.087+0000 7f7fbfe2c740 1 -- 172.21.15.111:0/400925564 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55de953c37c0 con 0x55de953afba0 2024-09-15T23:10:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.087+0000 7f7f97e00640 1 -- 172.21.15.111:0/400925564 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fa809c400 con 0x55de953afba0 2024-09-15T23:10:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.087+0000 7f7f97e00640 1 -- 172.21.15.111:0/400925564 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f7fa4004940 con 0x55de953afba0 2024-09-15T23:10:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.087+0000 7f7fbfe2c740 10 client.26873 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:10:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.087+0000 7f7fbfe2c740 10 client.26873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:10:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.088+0000 7f7fbfe2c740 20 client.26873 populate_metadata read hostname 'smithi111' 2024-09-15T23:10:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.088+0000 7f7fbfe2c740 1 --2- 172.21.15.111:0/400925564 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55de953c4e90 0x55de953c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.088+0000 7f7fbfe2c740 4 client.26873 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:10:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.088+0000 7f7fbfe2c740 1 -- 172.21.15.111:0/400925564 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de95321830 con 0x55de953c4e90 2024-09-15T23:10:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.088+0000 7f7fb5e00640 1 --2- 172.21.15.111:0/400925564 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55de953c4e90 0x55de953c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.088+0000 7f7fb5e00640 1 --2- 172.21.15.111:0/400925564 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55de953c4e90 0x55de953c72c0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec31787416f3f1f1 server_cookie=9d35caaa78444196 in_seq=0 out_seq=0 2024-09-15T23:10:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.088+0000 7f7f97e00640 10 client.26873 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:10:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.089+0000 7f7f97e00640 1 -- 172.21.15.111:0/400925564 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55de95321830 con 0x55de953c4e90 2024-09-15T23:10:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.089+0000 7f7f97e00640 10 client.26873 handle_command_reply: tid=0 2024-09-15T23:10:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.089+0000 7f7fbfe2c740 1 client.26873 shutdown 2024-09-15T23:10:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.089+0000 7f7fbfe2c740 1 -- 172.21.15.111:0/400925564 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de953afba0 msgr2=0x55de953b9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.089+0000 7f7fbfe2c740 1 --2- 172.21.15.111:0/400925564 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de953afba0 0x55de953b9920 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7fa402a910 tx=0x7f7fa4004270 comp rx=0 tx=0).stop 2024-09-15T23:10:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.090+0000 7f7fbfe2c740 1 -- 172.21.15.111:0/400925564 shutdown_connections 2024-09-15T23:10:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.090+0000 7f7fbfe2c740 1 --2- 172.21.15.111:0/400925564 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55de953c4e90 0x55de953c72c0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.090+0000 7f7fbfe2c740 1 --2- 172.21.15.111:0/400925564 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de953b4c30 0x55de953c0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.090+0000 7f7fbfe2c740 1 --2- 172.21.15.111:0/400925564 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de953afba0 0x55de953b9920 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.090+0000 7f7f97e00640 0 client.26873 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:10:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.090+0000 7f7fbfe2c740 1 --2- 172.21.15.111:0/400925564 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de953af260 0x55de953b93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.090+0000 7f7fbfe2c740 1 -- 172.21.15.111:0/400925564 >> 172.21.15.111:0/400925564 conn(0x55de953aa8d0 msgr2=0x55de953ac0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:10:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.090+0000 7f7fbfe2c740 1 -- 172.21.15.111:0/400925564 shutdown_connections 2024-09-15T23:10:49.113 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.090+0000 7f7fbfe2c740 1 -- 172.21.15.111:0/400925564 wait complete. 2024-09-15T23:10:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.090+0000 7f7fbfe2c740 20 client.26873 trim_cache size 0 max 16384 2024-09-15T23:10:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.104+0000 7f7fbfe2c740 1 Processor -- start 2024-09-15T23:10:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.105+0000 7f7fbfe2c740 1 -- start start 2024-09-15T23:10:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.105+0000 7f7fbfe2c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de953aa8d0 0x55de953a91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.105+0000 7f7fbfe2c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de953af260 0x55de953c0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.105+0000 7f7fbfe2c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de953afba0 0x55de953a9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.105+0000 7f7fbfe2c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb81a9580 con 0x55de953afba0 2024-09-15T23:10:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.105+0000 7f7fbfe2c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb81a6e50 con 0x55de953af260 2024-09-15T23:10:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.105+0000 7f7fbfe2c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb81a4ca0 con 0x55de953aa8d0 2024-09-15T23:10:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.105+0000 7f7fb6800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de953afba0 0x55de953a9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.105+0000 7f7fb6800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de953afba0 0x55de953a9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39676/0 (socket says 172.21.15.111:39676) 2024-09-15T23:10:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.105+0000 7f7fb6800640 1 -- 172.21.15.111:0/2591669370 learned_addr learned my addr 172.21.15.111:0/2591669370 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:10:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.105+0000 7f7fb5400640 1 --2- 172.21.15.111:0/2591669370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de953af260 0x55de953c0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.105+0000 7f7fb5e00640 1 --2- 172.21.15.111:0/2591669370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de953aa8d0 0x55de953a91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.106+0000 7f7fb6800640 1 -- 172.21.15.111:0/2591669370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de953aa8d0 msgr2=0x55de953a91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:10:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.106+0000 7f7fb6800640 1 --2- 172.21.15.111:0/2591669370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de953aa8d0 0x55de953a91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.106+0000 7f7fb6800640 1 -- 172.21.15.111:0/2591669370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de953af260 msgr2=0x55de953c0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:10:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.106+0000 7f7fb6800640 1 --2- 172.21.15.111:0/2591669370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de953af260 0x55de953c0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.106+0000 7f7fb6800640 1 -- 172.21.15.111:0/2591669370 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de953b7020 con 0x55de953afba0 2024-09-15T23:10:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.106+0000 7f7fb6800640 1 --2- 172.21.15.111:0/2591669370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de953afba0 0x55de953a9730 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7fb00137a0 tx=0x7f7fb0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e0f2e49d53d5c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:10:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.106+0000 7f7fa2a00640 1 -- 172.21.15.111:0/2591669370 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb00130d0 con 0x55de953afba0 2024-09-15T23:10:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.106+0000 7f7fa2a00640 1 -- 172.21.15.111:0/2591669370 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fb0013270 con 0x55de953afba0 2024-09-15T23:10:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.106+0000 7f7fa2a00640 1 -- 172.21.15.111:0/2591669370 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb0011d70 con 0x55de953afba0 2024-09-15T23:10:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.107+0000 7f7fbfe2c740 1 -- 172.21.15.111:0/2591669370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de953afba0 msgr2=0x55de953a9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.107+0000 7f7fbfe2c740 1 --2- 172.21.15.111:0/2591669370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de953afba0 0x55de953a9730 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7fb00137a0 tx=0x7f7fb0015000 comp rx=0 tx=0).stop 2024-09-15T23:10:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.107+0000 7f7fbfe2c740 1 -- 172.21.15.111:0/2591669370 shutdown_connections 2024-09-15T23:10:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.107+0000 7f7fbfe2c740 1 --2- 172.21.15.111:0/2591669370 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de953afba0 0x55de953a9730 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.107+0000 7f7fbfe2c740 1 --2- 172.21.15.111:0/2591669370 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de953af260 0x55de953c0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.107+0000 7f7fbfe2c740 1 --2- 172.21.15.111:0/2591669370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de953aa8d0 0x55de953a91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.107+0000 7f7fbfe2c740 1 -- 172.21.15.111:0/2591669370 >> 172.21.15.111:0/2591669370 conn(0x55de953b47c0 msgr2=0x55de953a56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:10:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.107+0000 7f7fbfe2c740 1 -- 172.21.15.111:0/2591669370 shutdown_connections 2024-09-15T23:10:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.107+0000 7f7fbfe2c740 1 -- 172.21.15.111:0/2591669370 wait complete. 2024-09-15T23:10:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.107+0000 7f7fbfe2c740 1 -- start start 2024-09-15T23:10:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.108+0000 7f7fbfe2c740 1 Processor -- start 2024-09-15T23:10:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.108+0000 7f7fbfe2c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de953aa8d0 0x55de953be1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.108+0000 7f7fbfe2c740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de953af260 0x55de953be6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.108+0000 7f7fbfe2c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de953afba0 0x55de953bf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.108+0000 7f7fbfe2c740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb81a5310 con 0x55de953af260 2024-09-15T23:10:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.108+0000 7f7fbfe2c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb81a6780 con 0x55de953aa8d0 2024-09-15T23:10:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.108+0000 7f7fbfe2c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb81a59e0 con 0x55de953afba0 2024-09-15T23:10:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.108+0000 7f7fbfe2c740 10 client.0 _ll_register_callbacks cb 0x7f7fb00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:10:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.108+0000 7f7fb5400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de953af260 0x55de953be6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.108+0000 7f7fb5e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de953aa8d0 0x55de953be1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.108+0000 7f7fb5e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de953aa8d0 0x55de953be1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:38568/0 (socket says 172.21.15.111:38568) 2024-09-15T23:10:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.108+0000 7f7fb5e00640 1 -- 172.21.15.111:0/2260466053 learned_addr learned my addr 172.21.15.111:0/2260466053 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:10:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.108+0000 7f7fb5400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de953af260 0x55de953be6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:39682/0 (socket says 172.21.15.111:39682) 2024-09-15T23:10:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.108+0000 7f7fb6800640 1 --2- 172.21.15.111:0/2260466053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de953afba0 0x55de953bf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.109+0000 7f7fb5e00640 1 -- 172.21.15.111:0/2260466053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de953afba0 msgr2=0x55de953bf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:10:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.109+0000 7f7fb5e00640 1 --2- 172.21.15.111:0/2260466053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de953afba0 0x55de953bf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.109+0000 7f7fb5e00640 1 -- 172.21.15.111:0/2260466053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de953af260 msgr2=0x55de953be6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:10:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.109+0000 7f7fb5e00640 1 --2- 172.21.15.111:0/2260466053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de953af260 0x55de953be6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.109+0000 7f7fb5e00640 1 -- 172.21.15.111:0/2260466053 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de953bfcb0 con 0x55de953aa8d0 2024-09-15T23:10:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.109+0000 7f7fb5e00640 1 --2- 172.21.15.111:0/2260466053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de953aa8d0 0x55de953be1a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800e9c0 tx=0x7f7fa809a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fceea35b3a96e088 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:10:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.109+0000 7f7f97e00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:10:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.109+0000 7f7f97e00640 1 -- 172.21.15.111:0/2260466053 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa80b5040 con 0x55de953aa8d0 2024-09-15T23:10:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.109+0000 7f7f97e00640 1 -- 172.21.15.111:0/2260466053 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f7fa80b51e0 con 0x55de953aa8d0 2024-09-15T23:10:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.109+0000 7f7fbfe2c740 1 -- 172.21.15.111:0/2260466053 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fb0013270 con 0x55de953aa8d0 2024-09-15T23:10:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.109+0000 7f7f97e00640 1 -- 172.21.15.111:0/2260466053 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7fa809bb60 con 0x55de953aa8d0 2024-09-15T23:10:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.109+0000 7f7fbfe2c740 10 client.26879 fetch_fsmap learned FSMap version 35 2024-09-15T23:10:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.109+0000 7f7fbfe2c740 1 -- 172.21.15.111:0/2260466053 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55de9542fcd0 con 0x55de953aa8d0 2024-09-15T23:10:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.110+0000 7f7f97e00640 1 -- 172.21.15.111:0/2260466053 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fb0013270 con 0x55de953aa8d0 2024-09-15T23:10:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.110+0000 7f7f97e00640 1 -- 172.21.15.111:0/2260466053 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f7fa80980c0 con 0x55de953aa8d0 2024-09-15T23:10:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.110+0000 7f7fbfe2c740 10 client.26879 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:10:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.110+0000 7f7fbfe2c740 10 client.26879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:10:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.110+0000 7f7fbfe2c740 20 client.26879 populate_metadata read hostname 'smithi111' 2024-09-15T23:10:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.110+0000 7f7fbfe2c740 1 --2- 172.21.15.111:0/2260466053 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55de953b4c30 0x55de95432250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:10:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.110+0000 7f7fbfe2c740 4 client.26879 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:10:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.110+0000 7f7fbfe2c740 1 -- 172.21.15.111:0/2260466053 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55de9531bb80 con 0x55de953b4c30 2024-09-15T23:10:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.110+0000 7f7fb5400640 1 --2- 172.21.15.111:0/2260466053 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55de953b4c30 0x55de95432250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:10:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.111+0000 7f7fb5400640 1 --2- 172.21.15.111:0/2260466053 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55de953b4c30 0x55de95432250 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d62af0c5e1b5517 server_cookie=7ff1a4b02321b801 in_seq=0 out_seq=0 2024-09-15T23:10:49.123 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.111+0000 7f7f97e00640 10 client.26879 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:10:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.188+0000 7f7f97e00640 1 -- 172.21.15.111:0/2260466053 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55de9531bb80 con 0x55de953b4c30 2024-09-15T23:10:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.188+0000 7f7f97e00640 10 client.26879 handle_command_reply: tid=0 2024-09-15T23:10:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.188+0000 7f7fbfe2c740 1 client.26879 shutdown 2024-09-15T23:10:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.188+0000 7f7fbfe2c740 1 -- 172.21.15.111:0/2260466053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de953aa8d0 msgr2=0x55de953be1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.188+0000 7f7fbfe2c740 1 --2- 172.21.15.111:0/2260466053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de953aa8d0 0x55de953be1a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800e9c0 tx=0x7f7fa809a410 comp rx=0 tx=0).stop 2024-09-15T23:10:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.188+0000 7f7fbfe2c740 1 -- 172.21.15.111:0/2260466053 shutdown_connections 2024-09-15T23:10:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.188+0000 7f7fbfe2c740 1 --2- 172.21.15.111:0/2260466053 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55de953b4c30 0x55de95432250 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.188+0000 7f7fbfe2c740 1 --2- 172.21.15.111:0/2260466053 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55de953afba0 0x55de953bf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.188+0000 7f7fbfe2c740 1 --2- 172.21.15.111:0/2260466053 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55de953af260 0x55de953be6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.188+0000 7f7f97e00640 0 client.26879 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:10:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.188+0000 7f7fbfe2c740 1 --2- 172.21.15.111:0/2260466053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de953aa8d0 0x55de953be1a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.188+0000 7f7fbfe2c740 1 -- 172.21.15.111:0/2260466053 >> 172.21.15.111:0/2260466053 conn(0x55de953b47c0 msgr2=0x55de953ab7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:10:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.189+0000 7f7fbfe2c740 1 -- 172.21.15.111:0/2260466053 shutdown_connections 2024-09-15T23:10:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.189+0000 7f7fbfe2c740 1 -- 172.21.15.111:0/2260466053 wait complete. 2024-09-15T23:10:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.189+0000 7f7fbfe2c740 20 client.26879 trim_cache size 0 max 16384 2024-09-15T23:10:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:10:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:10:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:10:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:10:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.189+0000 7f7fb7200640 1 -- 172.21.15.111:0/4080422816 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7f9c0a7650 msgr2=0x7f7f9c0a9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.189+0000 7f7fb7200640 1 --2- 172.21.15.111:0/4080422816 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7f9c0a7650 0x7f7f9c0a9b10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7fb806a980 tx=0x7f7fb00107a0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.189+0000 7f7fb7200640 1 -- 172.21.15.111:0/4080422816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fb81023c0 msgr2=0x7f7fb8197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:10:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.189+0000 7f7fb7200640 1 --2- 172.21.15.111:0/4080422816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fb81023c0 0x7f7fb8197d50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800e9f0 tx=0x7f7fa800eeb0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.189+0000 7f7fb7200640 1 -- 172.21.15.111:0/4080422816 shutdown_connections 2024-09-15T23:10:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.189+0000 7f7fb7200640 1 --2- 172.21.15.111:0/4080422816 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7f9c0a7650 0x7f7f9c0a9b10 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.189+0000 7f7fb7200640 1 --2- 172.21.15.111:0/4080422816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fb8107620 0x7f7fb819f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.189+0000 7f7fb7200640 1 --2- 172.21.15.111:0/4080422816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7fb8104cf0 0x7f7fb8198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.189+0000 7f7fb7200640 1 --2- 172.21.15.111:0/4080422816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fb81023c0 0x7f7fb8197d50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:10:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.189+0000 7f7fb7200640 1 -- 172.21.15.111:0/4080422816 >> 172.21.15.111:0/4080422816 conn(0x7f7fb80fbfa0 msgr2=0x7f7fb8102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:10:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.190+0000 7f7fb7200640 1 -- 172.21.15.111:0/4080422816 shutdown_connections 2024-09-15T23:10:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:10:49.190+0000 7f7fb7200640 1 -- 172.21.15.111:0/4080422816 wait complete. 2024-09-15T23:10:49.717 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222190 16.83 MB/sec execute 557 sec latency 10264.906 ms 2024-09-15T23:10:50.289 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:50 smithi111 ceph-mon[30629]: pgmap v1098: 97 pgs: 97 active+clean; 594 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 5.6 MiB/s wr, 261 op/s 2024-09-15T23:10:50.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:50 smithi121 ceph-mon[34815]: pgmap v1098: 97 pgs: 97 active+clean; 594 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 5.6 MiB/s wr, 261 op/s 2024-09-15T23:10:50.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:50 smithi192 ceph-mon[35427]: pgmap v1098: 97 pgs: 97 active+clean; 594 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 5.6 MiB/s wr, 261 op/s 2024-09-15T23:10:50.717 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222195 16.80 MB/sec execute 558 sec latency 10402.584 ms 2024-09-15T23:10:51.717 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222195 16.77 MB/sec execute 559 sec latency 8741.308 ms 2024-09-15T23:10:52.312 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:52 smithi111 ceph-mon[30629]: pgmap v1099: 97 pgs: 97 active+clean; 601 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 5.6 MiB/s wr, 233 op/s 2024-09-15T23:10:52.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:52 smithi121 ceph-mon[34815]: pgmap v1099: 97 pgs: 97 active+clean; 601 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 5.6 MiB/s wr, 233 op/s 2024-09-15T23:10:52.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:52 smithi192 ceph-mon[35427]: pgmap v1099: 97 pgs: 97 active+clean; 601 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 5.6 MiB/s wr, 233 op/s 2024-09-15T23:10:52.717 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222195 16.74 MB/sec execute 560 sec latency 9741.359 ms 2024-09-15T23:10:53.717 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222195 16.71 MB/sec execute 561 sec latency 10388.028 ms 2024-09-15T23:10:54.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:54 smithi121 ceph-mon[34815]: pgmap v1100: 97 pgs: 97 active+clean; 600 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 5.0 MiB/s wr, 198 op/s 2024-09-15T23:10:54.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:54 smithi111 ceph-mon[30629]: pgmap v1100: 97 pgs: 97 active+clean; 600 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 5.0 MiB/s wr, 198 op/s 2024-09-15T23:10:54.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:54 smithi192 ceph-mon[35427]: pgmap v1100: 97 pgs: 97 active+clean; 600 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 5.0 MiB/s wr, 198 op/s 2024-09-15T23:10:54.717 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222195 16.68 MB/sec execute 562 sec latency 11388.076 ms 2024-09-15T23:10:55.717 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222195 16.65 MB/sec execute 563 sec latency 12388.142 ms 2024-09-15T23:10:56.300 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:56 smithi111 ceph-mon[30629]: pgmap v1101: 97 pgs: 97 active+clean; 598 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 4.7 MiB/s wr, 189 op/s 2024-09-15T23:10:56.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:56 smithi121 ceph-mon[34815]: pgmap v1101: 97 pgs: 97 active+clean; 598 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 4.7 MiB/s wr, 189 op/s 2024-09-15T23:10:56.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:56 smithi192 ceph-mon[35427]: pgmap v1101: 97 pgs: 97 active+clean; 598 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 4.7 MiB/s wr, 189 op/s 2024-09-15T23:10:56.720 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222195 16.62 MB/sec execute 564 sec latency 13388.217 ms 2024-09-15T23:10:57.717 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222201 16.59 MB/sec execute 565 sec latency 13502.001 ms 2024-09-15T23:10:58.236 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:10:58 smithi111 ceph-mon[30629]: pgmap v1102: 97 pgs: 97 active+clean; 595 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 4.9 MiB/s wr, 197 op/s 2024-09-15T23:10:58.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:10:58 smithi121 ceph-mon[34815]: pgmap v1102: 97 pgs: 97 active+clean; 595 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 4.9 MiB/s wr, 197 op/s 2024-09-15T23:10:58.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:10:58 smithi192 ceph-mon[35427]: pgmap v1102: 97 pgs: 97 active+clean; 595 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 4.9 MiB/s wr, 197 op/s 2024-09-15T23:10:58.717 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222201 16.56 MB/sec execute 566 sec latency 8552.752 ms 2024-09-15T23:10:59.717 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222205 16.53 MB/sec execute 567 sec latency 9238.601 ms 2024-09-15T23:11:00.132 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:00 smithi111 ceph-mon[30629]: pgmap v1103: 97 pgs: 97 active+clean; 595 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 4.8 MiB/s wr, 190 op/s 2024-09-15T23:11:00.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:00 smithi121 ceph-mon[34815]: pgmap v1103: 97 pgs: 97 active+clean; 595 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 4.8 MiB/s wr, 190 op/s 2024-09-15T23:11:00.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:00 smithi192 ceph-mon[35427]: pgmap v1103: 97 pgs: 97 active+clean; 595 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 4.8 MiB/s wr, 190 op/s 2024-09-15T23:11:00.717 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222206 16.50 MB/sec execute 568 sec latency 3886.472 ms 2024-09-15T23:11:01.718 INFO:tasks.workunit.client.0.smithi111.stdout: 10 222736 16.53 MB/sec execute 569 sec latency 4211.800 ms 2024-09-15T23:11:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.334+0000 7f413be00640 1 -- 172.21.15.111:0/2670941815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f413c109cf0 msgr2=0x7f413c10c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.334+0000 7f413be00640 1 --2- 172.21.15.111:0/2670941815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f413c109cf0 0x7f413c10c0e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4134009be0 tx=0x7f413402a6c0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.335+0000 7f413be00640 1 -- 172.21.15.111:0/2670941815 shutdown_connections 2024-09-15T23:11:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.335+0000 7f413be00640 1 --2- 172.21.15.111:0/2670941815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f413c109cf0 0x7f413c10c0e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.335+0000 7f413be00640 1 --2- 172.21.15.111:0/2670941815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f413c074f60 0x7f413c1097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.335+0000 7f413be00640 1 --2- 172.21.15.111:0/2670941815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f413c074620 0x7f413c074a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.335+0000 7f413be00640 1 -- 172.21.15.111:0/2670941815 >> 172.21.15.111:0/2670941815 conn(0x7f413c0fd0d0 msgr2=0x7f413c0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.335+0000 7f413be00640 1 -- 172.21.15.111:0/2670941815 shutdown_connections 2024-09-15T23:11:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.335+0000 7f413be00640 1 -- 172.21.15.111:0/2670941815 wait complete. 2024-09-15T23:11:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.336+0000 7f413be00640 1 Processor -- start 2024-09-15T23:11:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.336+0000 7f413be00640 1 -- start start 2024-09-15T23:11:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.336+0000 7f413be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f413c074620 0x7f413c199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.336+0000 7f413be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f413c074f60 0x7f413c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.336+0000 7f413be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f413c109cf0 0x7f413c1a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.336+0000 7f413be00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f413c10f8d0 con 0x7f413c074620 2024-09-15T23:11:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.336+0000 7f413be00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f413c10f750 con 0x7f413c074f60 2024-09-15T23:11:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.336+0000 7f413be00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f413c10fa50 con 0x7f413c109cf0 2024-09-15T23:11:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.336+0000 7f413a000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f413c074f60 0x7f413c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.336+0000 7f413aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f413c074620 0x7f413c199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.336+0000 7f413b400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f413c109cf0 0x7f413c1a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.336+0000 7f413a000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f413c074f60 0x7f413c19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:42836/0 (socket says 172.21.15.111:42836) 2024-09-15T23:11:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.336+0000 7f413aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f413c074620 0x7f413c199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56520/0 (socket says 172.21.15.111:56520) 2024-09-15T23:11:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.336+0000 7f413a000640 1 -- 172.21.15.111:0/2008168249 learned_addr learned my addr 172.21.15.111:0/2008168249 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:11:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.337+0000 7f413a000640 1 -- 172.21.15.111:0/2008168249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f413c109cf0 msgr2=0x7f413c1a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.337+0000 7f413a000640 1 --2- 172.21.15.111:0/2008168249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f413c109cf0 0x7f413c1a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.337+0000 7f413a000640 1 -- 172.21.15.111:0/2008168249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f413c074620 msgr2=0x7f413c199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.337+0000 7f413a000640 1 --2- 172.21.15.111:0/2008168249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f413c074620 0x7f413c199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.337+0000 7f413a000640 1 -- 172.21.15.111:0/2008168249 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413c1a1b70 con 0x7f413c074f60 2024-09-15T23:11:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.337+0000 7f413a000640 1 --2- 172.21.15.111:0/2008168249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f413c074f60 0x7f413c19a490 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f412800e9c0 tx=0x7f412800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.337+0000 7f4133400640 1 -- 172.21.15.111:0/2008168249 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412800cde0 con 0x7f413c074f60 2024-09-15T23:11:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.338+0000 7f4133400640 1 -- 172.21.15.111:0/2008168249 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41280035f0 con 0x7f413c074f60 2024-09-15T23:11:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.338+0000 7f4133400640 1 -- 172.21.15.111:0/2008168249 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128010690 con 0x7f413c074f60 2024-09-15T23:11:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.338+0000 7f413be00640 1 -- 172.21.15.111:0/2008168249 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f413c1a1e60 con 0x7f413c074f60 2024-09-15T23:11:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.338+0000 7f413be00640 1 -- 172.21.15.111:0/2008168249 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f413c1a22d0 con 0x7f413c074f60 2024-09-15T23:11:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.342+0000 7f41447e3740 1 Processor -- start 2024-09-15T23:11:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.342+0000 7f41447e3740 1 -- start start 2024-09-15T23:11:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.342+0000 7f41447e3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c9590aa70 0x559c9590ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.342+0000 7f41447e3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c9590b3b0 0x559c9590fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:02.346 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.342+0000 7f41447e3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c95910440 0x559c95912900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.342+0000 7f4133400640 1 -- 172.21.15.111:0/2008168249 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f41280037a0 con 0x7f413c074f60 2024-09-15T23:11:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.342+0000 7f41447e3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559c95867100 con 0x559c9590aa70 2024-09-15T23:11:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.342+0000 7f41447e3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559c9580f1b0 con 0x559c95910440 2024-09-15T23:11:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.342+0000 7f41447e3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x559c958630c0 con 0x559c9590b3b0 2024-09-15T23:11:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.343+0000 7f413aa00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c9590b3b0 0x559c9590fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.343+0000 7f413aa00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c9590b3b0 0x559c9590fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:46330/0 (socket says 172.21.15.111:46330) 2024-09-15T23:11:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.343+0000 7f413aa00640 1 -- 172.21.15.111:0/4201254384 learned_addr learned my addr 172.21.15.111:0/4201254384 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:11:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.343+0000 7f413b400640 1 --2- 172.21.15.111:0/4201254384 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c9590aa70 0x559c9590ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.343+0000 7f413a000640 1 --2- 172.21.15.111:0/4201254384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c95910440 0x559c95912900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.343+0000 7f413aa00640 1 -- 172.21.15.111:0/4201254384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c95910440 msgr2=0x559c95912900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.343+0000 7f413aa00640 1 --2- 172.21.15.111:0/4201254384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c95910440 0x559c95912900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.343+0000 7f413aa00640 1 -- 172.21.15.111:0/4201254384 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c9590aa70 msgr2=0x559c9590ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.343+0000 7f413aa00640 1 --2- 172.21.15.111:0/4201254384 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c9590aa70 0x559c9590ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.343+0000 7f413aa00640 1 -- 172.21.15.111:0/4201254384 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c9557a210 con 0x559c9590b3b0 2024-09-15T23:11:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.343+0000 7f413b400640 1 --2- 172.21.15.111:0/4201254384 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c9590aa70 0x559c9590ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:11:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.343+0000 7f413aa00640 1 --2- 172.21.15.111:0/4201254384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c9590b3b0 0x559c9590fe70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f412c009cf0 tx=0x7f412c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ede6cc15df327cf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.344+0000 7f4127e00640 1 -- 172.21.15.111:0/4201254384 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c004230 con 0x559c9590b3b0 2024-09-15T23:11:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.344+0000 7f4127e00640 1 -- 172.21.15.111:0/4201254384 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f412c0043d0 con 0x559c9590b3b0 2024-09-15T23:11:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.344+0000 7f4127e00640 1 -- 172.21.15.111:0/4201254384 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c019960 con 0x559c9590b3b0 2024-09-15T23:11:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.344+0000 7f4133400640 1 --2- 172.21.15.111:0/2008168249 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f41200a3240 0x7f41200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.344+0000 7f41447e3740 1 -- 172.21.15.111:0/4201254384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c9590b3b0 msgr2=0x559c9590fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.344+0000 7f41447e3740 1 --2- 172.21.15.111:0/4201254384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c9590b3b0 0x559c9590fe70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f412c009cf0 tx=0x7f412c011810 comp rx=0 tx=0).stop 2024-09-15T23:11:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.345+0000 7f4133400640 1 -- 172.21.15.111:0/2008168249 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f41280cfc00 con 0x7f413c074f60 2024-09-15T23:11:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.345+0000 7f41447e3740 1 -- 172.21.15.111:0/4201254384 shutdown_connections 2024-09-15T23:11:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.345+0000 7f41447e3740 1 --2- 172.21.15.111:0/4201254384 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c95910440 0x559c95912900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.345+0000 7f41447e3740 1 --2- 172.21.15.111:0/4201254384 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c9590b3b0 0x559c9590fe70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.345+0000 7f41447e3740 1 --2- 172.21.15.111:0/4201254384 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c9590aa70 0x559c9590ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.345+0000 7f41447e3740 1 -- 172.21.15.111:0/4201254384 >> 172.21.15.111:0/4201254384 conn(0x559c958fd8d0 msgr2=0x559c958fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.345+0000 7f413b400640 1 --2- 172.21.15.111:0/2008168249 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f41200a3240 0x7f41200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:02.349 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.345+0000 7f41447e3740 1 -- 172.21.15.111:0/4201254384 shutdown_connections 2024-09-15T23:11:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.345+0000 7f41447e3740 1 -- 172.21.15.111:0/4201254384 wait complete. 2024-09-15T23:11:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.346+0000 7f41447e3740 1 -- start start 2024-09-15T23:11:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.346+0000 7f413b400640 1 --2- 172.21.15.111:0/2008168249 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f41200a3240 0x7f41200a5700 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f413402abe0 tx=0x7f4134005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.346+0000 7f41447e3740 1 Processor -- start 2024-09-15T23:11:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.347+0000 7f41447e3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c9590aa70 0x559c95914c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.347+0000 7f41447e3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c9590b3b0 0x559c95915140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.347+0000 7f41447e3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c95910440 0x559c9591c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.347+0000 7f41447e3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x559c95913410 con 0x559c9590b3b0 2024-09-15T23:11:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.347+0000 7f41447e3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559c957f0090 con 0x559c9590aa70 2024-09-15T23:11:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.347+0000 7f41447e3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x559c95913590 con 0x559c95910440 2024-09-15T23:11:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.347+0000 7f41447e3740 10 client.0 _ll_register_callbacks cb 0x559c95844f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:11:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.347+0000 7f413aa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c9590aa70 0x559c95914c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.347+0000 7f413aa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c9590aa70 0x559c95914c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:42868/0 (socket says 172.21.15.111:42868) 2024-09-15T23:11:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.347+0000 7f413b400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c95910440 0x559c9591c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.347+0000 7f413aa00640 1 -- 172.21.15.111:0/3834106584 learned_addr learned my addr 172.21.15.111:0/3834106584 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:11:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.347+0000 7f413a000640 1 --2- 172.21.15.111:0/3834106584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c9590b3b0 0x559c95915140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.347+0000 7f413b400640 1 -- 172.21.15.111:0/3834106584 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c9590aa70 msgr2=0x559c95914c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.348+0000 7f413b400640 1 --2- 172.21.15.111:0/3834106584 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c9590aa70 0x559c95914c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.348+0000 7f413b400640 1 -- 172.21.15.111:0/3834106584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c9590b3b0 msgr2=0x559c95915140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.348+0000 7f413b400640 1 --2- 172.21.15.111:0/3834106584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c9590b3b0 0x559c95915140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.348+0000 7f413b400640 1 -- 172.21.15.111:0/3834106584 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c95646620 con 0x559c95910440 2024-09-15T23:11:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.348+0000 7f413b400640 1 --2- 172.21.15.111:0/3834106584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c95910440 0x559c9591c180 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4134034040 tx=0x7f413402a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8423200a2bcc814 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.348+0000 7f4124c00640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:11:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.348+0000 7f4124c00640 1 -- 172.21.15.111:0/3834106584 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413404f020 con 0x559c95910440 2024-09-15T23:11:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.348+0000 7f41447e3740 1 -- 172.21.15.111:0/3834106584 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f412c0043d0 con 0x559c95910440 2024-09-15T23:11:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.348+0000 7f4124c00640 1 -- 172.21.15.111:0/3834106584 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f4134003b70 con 0x559c95910440 2024-09-15T23:11:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.348+0000 7f4124c00640 1 -- 172.21.15.111:0/3834106584 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4134036040 con 0x559c95910440 2024-09-15T23:11:02.354 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.349+0000 7f41447e3740 10 client.26881 fetch_fsmap learned FSMap version 35 2024-09-15T23:11:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.349+0000 7f41447e3740 1 -- 172.21.15.111:0/3834106584 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559c9591f030 con 0x559c95910440 2024-09-15T23:11:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.349+0000 7f4124c00640 1 -- 172.21.15.111:0/3834106584 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f412c0043d0 con 0x559c95910440 2024-09-15T23:11:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.349+0000 7f4124c00640 1 -- 172.21.15.111:0/3834106584 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f413404b480 con 0x559c95910440 2024-09-15T23:11:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.349+0000 7f41447e3740 10 client.26881 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:11:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.349+0000 7f41447e3740 10 client.26881 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:11:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.349+0000 7f41447e3740 20 client.26881 populate_metadata read hostname 'smithi111' 2024-09-15T23:11:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.349+0000 7f41447e3740 1 --2- 172.21.15.111:0/3834106584 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559c959206d0 0x559c95922b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.349+0000 7f41447e3740 4 client.26881 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:11:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.349+0000 7f41447e3740 1 -- 172.21.15.111:0/3834106584 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c95874830 con 0x559c959206d0 2024-09-15T23:11:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.349+0000 7f413aa00640 1 --2- 172.21.15.111:0/3834106584 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559c959206d0 0x559c95922b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.350+0000 7f413aa00640 1 --2- 172.21.15.111:0/3834106584 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559c959206d0 0x559c95922b30 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a334c342ad2e2d52 server_cookie=8b1eb37c6cbaf1bc in_seq=0 out_seq=0 2024-09-15T23:11:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.350+0000 7f4124c00640 10 client.26881 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:11:02.355 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.351+0000 7f4124c00640 1 -- 172.21.15.111:0/3834106584 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559c95874830 con 0x559c959206d0 2024-09-15T23:11:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.351+0000 7f4124c00640 10 client.26881 handle_command_reply: tid=0 2024-09-15T23:11:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.351+0000 7f41447e3740 1 client.26881 shutdown 2024-09-15T23:11:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.351+0000 7f41447e3740 1 -- 172.21.15.111:0/3834106584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c95910440 msgr2=0x559c9591c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.351+0000 7f41447e3740 1 --2- 172.21.15.111:0/3834106584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c95910440 0x559c9591c180 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4134034040 tx=0x7f413402a6c0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.356 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.352+0000 7f41447e3740 1 -- 172.21.15.111:0/3834106584 shutdown_connections 2024-09-15T23:11:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.352+0000 7f41447e3740 1 --2- 172.21.15.111:0/3834106584 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559c959206d0 0x559c95922b30 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.352+0000 7f41447e3740 1 --2- 172.21.15.111:0/3834106584 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c95910440 0x559c9591c180 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.352+0000 7f41447e3740 1 --2- 172.21.15.111:0/3834106584 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c9590b3b0 0x559c95915140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.352+0000 7f4124c00640 0 client.26881 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:11:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.352+0000 7f41447e3740 1 --2- 172.21.15.111:0/3834106584 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c9590aa70 0x559c95914c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.352+0000 7f41447e3740 1 -- 172.21.15.111:0/3834106584 >> 172.21.15.111:0/3834106584 conn(0x559c958fd8d0 msgr2=0x559c958ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.352+0000 7f41447e3740 1 -- 172.21.15.111:0/3834106584 shutdown_connections 2024-09-15T23:11:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.352+0000 7f41447e3740 1 -- 172.21.15.111:0/3834106584 wait complete. 2024-09-15T23:11:02.357 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.352+0000 7f41447e3740 20 client.26881 trim_cache size 0 max 16384 2024-09-15T23:11:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.372+0000 7f41447e3740 1 Processor -- start 2024-09-15T23:11:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.372+0000 7f41447e3740 1 -- start start 2024-09-15T23:11:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.372+0000 7f41447e3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c9590aa70 0x559c958fdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.372+0000 7f41447e3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c9590b3b0 0x559c9591c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.372+0000 7f41447e3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c95910440 0x559c958fe1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.372+0000 7f41447e3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1a9350 con 0x559c9590aa70 2024-09-15T23:11:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.372+0000 7f41447e3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1a6c20 con 0x559c95910440 2024-09-15T23:11:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.372+0000 7f41447e3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1a4a90 con 0x559c9590b3b0 2024-09-15T23:11:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.373+0000 7f413b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c95910440 0x559c958fe1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.373+0000 7f413aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c9590aa70 0x559c958fdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.373+0000 7f413b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c95910440 0x559c958fe1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:42880/0 (socket says 172.21.15.111:42880) 2024-09-15T23:11:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.373+0000 7f413b400640 1 -- 172.21.15.111:0/2624670803 learned_addr learned my addr 172.21.15.111:0/2624670803 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:11:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.373+0000 7f413a000640 1 --2- 172.21.15.111:0/2624670803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c9590b3b0 0x559c9591c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.373+0000 7f413b400640 1 -- 172.21.15.111:0/2624670803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c9590b3b0 msgr2=0x559c9591c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.373+0000 7f413b400640 1 --2- 172.21.15.111:0/2624670803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c9590b3b0 0x559c9591c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.373+0000 7f413b400640 1 -- 172.21.15.111:0/2624670803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c9590aa70 msgr2=0x559c958fdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.373+0000 7f413b400640 1 --2- 172.21.15.111:0/2624670803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c9590aa70 0x559c958fdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.373+0000 7f413b400640 1 -- 172.21.15.111:0/2624670803 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c958fd6f0 con 0x559c95910440 2024-09-15T23:11:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.373+0000 7f413aa00640 1 --2- 172.21.15.111:0/2624670803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c9590aa70 0x559c958fdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:11:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.374+0000 7f413b400640 1 --2- 172.21.15.111:0/2624670803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c95910440 0x559c958fe1f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4134030960 tx=0x7f413404bd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcc91059f57f96da server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.374+0000 7f4127400640 1 -- 172.21.15.111:0/2624670803 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413404f070 con 0x559c95910440 2024-09-15T23:11:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.374+0000 7f4127400640 1 -- 172.21.15.111:0/2624670803 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4134003e50 con 0x559c95910440 2024-09-15T23:11:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.374+0000 7f4127400640 1 -- 172.21.15.111:0/2624670803 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134004660 con 0x559c95910440 2024-09-15T23:11:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.375+0000 7f41447e3740 1 -- 172.21.15.111:0/2624670803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c95910440 msgr2=0x559c958fe1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.375+0000 7f41447e3740 1 --2- 172.21.15.111:0/2624670803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c95910440 0x559c958fe1f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4134030960 tx=0x7f413404bd20 comp rx=0 tx=0).stop 2024-09-15T23:11:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.375+0000 7f41447e3740 1 -- 172.21.15.111:0/2624670803 shutdown_connections 2024-09-15T23:11:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.375+0000 7f41447e3740 1 --2- 172.21.15.111:0/2624670803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c95910440 0x559c958fe1f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.375+0000 7f41447e3740 1 --2- 172.21.15.111:0/2624670803 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c9590b3b0 0x559c9591c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.375+0000 7f41447e3740 1 --2- 172.21.15.111:0/2624670803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c9590aa70 0x559c958fdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.375+0000 7f41447e3740 1 -- 172.21.15.111:0/2624670803 >> 172.21.15.111:0/2624670803 conn(0x559c958fd8d0 msgr2=0x559c9590fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.375+0000 7f41447e3740 1 -- 172.21.15.111:0/2624670803 shutdown_connections 2024-09-15T23:11:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.375+0000 7f41447e3740 1 -- 172.21.15.111:0/2624670803 wait complete. 2024-09-15T23:11:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.376+0000 7f41447e3740 1 -- start start 2024-09-15T23:11:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.376+0000 7f41447e3740 1 Processor -- start 2024-09-15T23:11:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.377+0000 7f41447e3740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c9590aa70 0x559c95919710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.377+0000 7f41447e3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c9590b3b0 0x559c95919c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.377+0000 7f41447e3740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c95910440 0x559c9591ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.377+0000 7f41447e3740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1a50e0 con 0x559c9590aa70 2024-09-15T23:11:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.377+0000 7f41447e3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1a6550 con 0x559c9590b3b0 2024-09-15T23:11:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.377+0000 7f41447e3740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f413c1a57b0 con 0x559c95910440 2024-09-15T23:11:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.377+0000 7f41447e3740 10 client.0 _ll_register_callbacks cb 0x7f4134042070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:11:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.377+0000 7f413aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c9590aa70 0x559c95919710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.377+0000 7f413aa00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c9590aa70 0x559c95919710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56552/0 (socket says 172.21.15.111:56552) 2024-09-15T23:11:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.377+0000 7f413aa00640 1 -- 172.21.15.111:0/3199526474 learned_addr learned my addr 172.21.15.111:0/3199526474 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:11:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.377+0000 7f413b400640 1 --2- 172.21.15.111:0/3199526474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c95910440 0x559c9591ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.377+0000 7f413aa00640 1 -- 172.21.15.111:0/3199526474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c95910440 msgr2=0x559c9591ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.377+0000 7f413aa00640 1 --2- 172.21.15.111:0/3199526474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c95910440 0x559c9591ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.377+0000 7f413aa00640 1 -- 172.21.15.111:0/3199526474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c9590b3b0 msgr2=0x559c95919c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T23:11:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.377+0000 7f413aa00640 1 --2- 172.21.15.111:0/3199526474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c9590b3b0 0x559c95919c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.377+0000 7f413aa00640 1 -- 172.21.15.111:0/3199526474 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c9591b270 con 0x559c9590aa70 2024-09-15T23:11:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.378+0000 7f413aa00640 1 --2- 172.21.15.111:0/3199526474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c9590aa70 0x559c95919710 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f412c014040 tx=0x7f412c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60721292407bbf60 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.378+0000 7f4124c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:11:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.378+0000 7f4124c00640 1 -- 172.21.15.111:0/3199526474 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c005450 con 0x559c9590aa70 2024-09-15T23:11:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.378+0000 7f4124c00640 1 -- 172.21.15.111:0/3199526474 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f412c0055f0 con 0x559c9590aa70 2024-09-15T23:11:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.378+0000 7f41447e3740 1 -- 172.21.15.111:0/3199526474 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4134003e50 con 0x559c9590aa70 2024-09-15T23:11:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.378+0000 7f4124c00640 1 -- 172.21.15.111:0/3199526474 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f412c005d30 con 0x559c9590aa70 2024-09-15T23:11:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.379+0000 7f41447e3740 10 client.16944 fetch_fsmap learned FSMap version 35 2024-09-15T23:11:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.379+0000 7f41447e3740 1 -- 172.21.15.111:0/3199526474 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559c9598b410 con 0x559c9590aa70 2024-09-15T23:11:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.379+0000 7f4124c00640 1 -- 172.21.15.111:0/3199526474 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4134003e50 con 0x559c9590aa70 2024-09-15T23:11:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.379+0000 7f4124c00640 1 -- 172.21.15.111:0/3199526474 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7f412c019a10 con 0x559c9590aa70 2024-09-15T23:11:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.379+0000 7f41447e3740 10 client.16944 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:11:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.379+0000 7f41447e3740 10 client.16944 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:11:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.379+0000 7f41447e3740 20 client.16944 populate_metadata read hostname 'smithi111' 2024-09-15T23:11:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.379+0000 7f41447e3740 1 --2- 172.21.15.111:0/3199526474 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559c9598bb80 0x559c9598df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.379+0000 7f41447e3740 4 client.16944 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:11:02.383 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.379+0000 7f41447e3740 1 -- 172.21.15.111:0/3199526474 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559c9586eb80 con 0x559c9598bb80 2024-09-15T23:11:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.380+0000 7f413a000640 1 --2- 172.21.15.111:0/3199526474 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559c9598bb80 0x559c9598df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:02.384 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:02 smithi111 ceph-mon[30629]: pgmap v1104: 97 pgs: 97 active+clean; 589 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 4.8 MiB/s wr, 185 op/s 2024-09-15T23:11:02.384 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.380+0000 7f413a000640 1 --2- 172.21.15.111:0/3199526474 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559c9598bb80 0x559c9598df70 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c3ad6a044d897b1 server_cookie=73ecfe23c890669 in_seq=0 out_seq=0 2024-09-15T23:11:02.385 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.380+0000 7f4124c00640 10 client.16944 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:11:02.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:02 smithi121 ceph-mon[34815]: pgmap v1104: 97 pgs: 97 active+clean; 589 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 4.8 MiB/s wr, 185 op/s 2024-09-15T23:11:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.529+0000 7f4124c00640 1 -- 172.21.15.111:0/3199526474 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559c9586eb80 con 0x559c9598bb80 2024-09-15T23:11:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.529+0000 7f4124c00640 10 client.16944 handle_command_reply: tid=0 2024-09-15T23:11:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.529+0000 7f41447e3740 1 client.16944 shutdown 2024-09-15T23:11:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.529+0000 7f41447e3740 1 -- 172.21.15.111:0/3199526474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c9590aa70 msgr2=0x559c95919710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.529+0000 7f41447e3740 1 --2- 172.21.15.111:0/3199526474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c9590aa70 0x559c95919710 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f412c014040 tx=0x7f412c009310 comp rx=0 tx=0).stop 2024-09-15T23:11:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.529+0000 7f41447e3740 1 -- 172.21.15.111:0/3199526474 shutdown_connections 2024-09-15T23:11:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.529+0000 7f41447e3740 1 --2- 172.21.15.111:0/3199526474 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x559c9598bb80 0x559c9598df70 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.529+0000 7f41447e3740 1 --2- 172.21.15.111:0/3199526474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x559c95910440 0x559c9591ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.529+0000 7f41447e3740 1 --2- 172.21.15.111:0/3199526474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559c9590b3b0 0x559c95919c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.529+0000 7f41447e3740 1 --2- 172.21.15.111:0/3199526474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x559c9590aa70 0x559c95919710 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.529+0000 7f41447e3740 1 -- 172.21.15.111:0/3199526474 >> 172.21.15.111:0/3199526474 conn(0x559c958fd8d0 msgr2=0x559c958fc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.529+0000 7f4124c00640 0 client.16944 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:11:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.530+0000 7f41447e3740 1 -- 172.21.15.111:0/3199526474 shutdown_connections 2024-09-15T23:11:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.530+0000 7f41447e3740 1 -- 172.21.15.111:0/3199526474 wait complete. 2024-09-15T23:11:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.530+0000 7f41447e3740 20 client.16944 trim_cache size 0 max 16384 2024-09-15T23:11:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:11:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:11:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:11:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:11:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.532+0000 7f413be00640 1 -- 172.21.15.111:0/2008168249 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f41200a3240 msgr2=0x7f41200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.532+0000 7f413be00640 1 --2- 172.21.15.111:0/2008168249 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f41200a3240 0x7f41200a5700 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f413402abe0 tx=0x7f4134005fd0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.532+0000 7f413be00640 1 -- 172.21.15.111:0/2008168249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f413c074f60 msgr2=0x7f413c19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.532+0000 7f413be00640 1 --2- 172.21.15.111:0/2008168249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f413c074f60 0x7f413c19a490 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f412800e9c0 tx=0x7f412800ee80 comp rx=0 tx=0).stop 2024-09-15T23:11:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.532+0000 7f413be00640 1 -- 172.21.15.111:0/2008168249 shutdown_connections 2024-09-15T23:11:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.532+0000 7f413be00640 1 --2- 172.21.15.111:0/2008168249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f413c109cf0 0x7f413c1a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.532+0000 7f413be00640 1 --2- 172.21.15.111:0/2008168249 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f41200a3240 0x7f41200a5700 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.532+0000 7f413be00640 1 --2- 172.21.15.111:0/2008168249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f413c074f60 0x7f413c19a490 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.532+0000 7f413be00640 1 --2- 172.21.15.111:0/2008168249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f413c074620 0x7f413c199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.532+0000 7f413be00640 1 -- 172.21.15.111:0/2008168249 >> 172.21.15.111:0/2008168249 conn(0x7f413c0fd0d0 msgr2=0x7f413c105790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.532+0000 7f413be00640 1 -- 172.21.15.111:0/2008168249 shutdown_connections 2024-09-15T23:11:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:02.532+0000 7f413be00640 1 -- 172.21.15.111:0/2008168249 wait complete. 2024-09-15T23:11:02.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:02 smithi192 ceph-mon[35427]: pgmap v1104: 97 pgs: 97 active+clean; 589 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 4.8 MiB/s wr, 185 op/s 2024-09-15T23:11:02.718 INFO:tasks.workunit.client.0.smithi111.stdout: 10 223343 16.57 MB/sec execute 570 sec latency 108.380 ms 2024-09-15T23:11:03.718 INFO:tasks.workunit.client.0.smithi111.stdout: 10 223855 16.57 MB/sec execute 571 sec latency 66.630 ms 2024-09-15T23:11:04.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:04 smithi121 ceph-mon[34815]: pgmap v1105: 97 pgs: 97 active+clean; 589 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 4.4 MiB/s wr, 174 op/s 2024-09-15T23:11:04.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:04 smithi111 ceph-mon[30629]: pgmap v1105: 97 pgs: 97 active+clean; 589 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 4.4 MiB/s wr, 174 op/s 2024-09-15T23:11:04.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:04 smithi192 ceph-mon[35427]: pgmap v1105: 97 pgs: 97 active+clean; 589 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 4.4 MiB/s wr, 174 op/s 2024-09-15T23:11:04.718 INFO:tasks.workunit.client.0.smithi111.stdout: 10 224423 16.59 MB/sec execute 572 sec latency 70.220 ms 2024-09-15T23:11:05.718 INFO:tasks.workunit.client.0.smithi111.stdout: 10 225277 16.66 MB/sec execute 573 sec latency 68.475 ms 2024-09-15T23:11:06.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:06 smithi121 ceph-mon[34815]: pgmap v1106: 97 pgs: 97 active+clean; 582 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 227 op/s 2024-09-15T23:11:06.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:06 smithi111 ceph-mon[30629]: pgmap v1106: 97 pgs: 97 active+clean; 582 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 227 op/s 2024-09-15T23:11:06.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:06 smithi192 ceph-mon[35427]: pgmap v1106: 97 pgs: 97 active+clean; 582 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 227 op/s 2024-09-15T23:11:06.718 INFO:tasks.workunit.client.0.smithi111.stdout: 10 225976 16.67 MB/sec execute 574 sec latency 53.236 ms 2024-09-15T23:11:07.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:07 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:11:07.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:07 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:11:07.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:07 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:11:07.718 INFO:tasks.workunit.client.0.smithi111.stdout: 10 226638 16.72 MB/sec execute 575 sec latency 100.661 ms 2024-09-15T23:11:08.313 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:08 smithi111 ceph-mon[30629]: pgmap v1107: 97 pgs: 97 active+clean; 583 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 306 op/s 2024-09-15T23:11:08.313 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:08 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:11:08.314 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:08 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:11:08.314 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:08 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:11:08.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:08 smithi121 ceph-mon[34815]: pgmap v1107: 97 pgs: 97 active+clean; 583 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 306 op/s 2024-09-15T23:11:08.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:08 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:11:08.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:08 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:11:08.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:08 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:11:08.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:08 smithi192 ceph-mon[35427]: pgmap v1107: 97 pgs: 97 active+clean; 583 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.4 MiB/s wr, 306 op/s 2024-09-15T23:11:08.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:08 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:11:08.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:08 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:11:08.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:08 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:11:08.718 INFO:tasks.workunit.client.0.smithi111.stdout: 10 227297 16.77 MB/sec execute 576 sec latency 108.904 ms 2024-09-15T23:11:09.718 INFO:tasks.workunit.client.0.smithi111.stdout: 10 227718 16.78 MB/sec execute 577 sec latency 80.009 ms 2024-09-15T23:11:10.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:10 smithi121 ceph-mon[34815]: pgmap v1108: 97 pgs: 97 active+clean; 560 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.2 MiB/s wr, 340 op/s 2024-09-15T23:11:10.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:10 smithi111 ceph-mon[30629]: pgmap v1108: 97 pgs: 97 active+clean; 560 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.2 MiB/s wr, 340 op/s 2024-09-15T23:11:10.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:10 smithi192 ceph-mon[35427]: pgmap v1108: 97 pgs: 97 active+clean; 560 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.2 MiB/s wr, 340 op/s 2024-09-15T23:11:10.718 INFO:tasks.workunit.client.0.smithi111.stdout: 10 228198 16.77 MB/sec execute 578 sec latency 115.262 ms 2024-09-15T23:11:11.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:11 smithi121 ceph-mon[34815]: pgmap v1109: 97 pgs: 97 active+clean; 569 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.3 MiB/s wr, 438 op/s 2024-09-15T23:11:11.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:11 smithi111 ceph-mon[30629]: pgmap v1109: 97 pgs: 97 active+clean; 569 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.3 MiB/s wr, 438 op/s 2024-09-15T23:11:11.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:11 smithi192 ceph-mon[35427]: pgmap v1109: 97 pgs: 97 active+clean; 569 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.3 MiB/s wr, 438 op/s 2024-09-15T23:11:11.718 INFO:tasks.workunit.client.0.smithi111.stdout: 10 228759 16.79 MB/sec execute 579 sec latency 103.488 ms 2024-09-15T23:11:12.718 INFO:tasks.workunit.client.0.smithi111.stdout: 10 229418 16.83 MB/sec execute 580 sec latency 90.686 ms 2024-09-15T23:11:13.718 INFO:tasks.workunit.client.0.smithi111.stdout: 10 230166 16.86 MB/sec execute 581 sec latency 148.361 ms 2024-09-15T23:11:14.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:14 smithi192 ceph-mon[35427]: pgmap v1110: 97 pgs: 97 active+clean; 570 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.6 MiB/s wr, 419 op/s 2024-09-15T23:11:14.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:14 smithi121 ceph-mon[34815]: pgmap v1110: 97 pgs: 97 active+clean; 570 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.6 MiB/s wr, 419 op/s 2024-09-15T23:11:14.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:14 smithi111 ceph-mon[30629]: pgmap v1110: 97 pgs: 97 active+clean; 570 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.6 MiB/s wr, 419 op/s 2024-09-15T23:11:14.718 INFO:tasks.workunit.client.0.smithi111.stdout: 10 230790 16.91 MB/sec execute 582 sec latency 64.559 ms 2024-09-15T23:11:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.691+0000 7fea9d200640 1 -- 172.21.15.111:0/2588561856 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea98103b70 msgr2=0x7fea98103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.691+0000 7fea9d200640 1 --2- 172.21.15.111:0/2588561856 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea98103b70 0x7fea98103f70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fea88009900 tx=0x7fea8802a3f0 comp rx=0 tx=0).stop 2024-09-15T23:11:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.691+0000 7fea9d200640 1 -- 172.21.15.111:0/2588561856 shutdown_connections 2024-09-15T23:11:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.691+0000 7fea9d200640 1 --2- 172.21.15.111:0/2588561856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea98105720 0x7fea9810bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.691+0000 7fea9d200640 1 --2- 172.21.15.111:0/2588561856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea98104d60 0x7fea981051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.691+0000 7fea9d200640 1 --2- 172.21.15.111:0/2588561856 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea98103b70 0x7fea98103f70 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.691+0000 7fea9d200640 1 -- 172.21.15.111:0/2588561856 >> 172.21.15.111:0/2588561856 conn(0x7fea980ff2f0 msgr2=0x7fea98101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.691+0000 7fea9d200640 1 -- 172.21.15.111:0/2588561856 shutdown_connections 2024-09-15T23:11:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.691+0000 7fea9d200640 1 -- 172.21.15.111:0/2588561856 wait complete. 2024-09-15T23:11:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.692+0000 7fea9d200640 1 Processor -- start 2024-09-15T23:11:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.692+0000 7fea9d200640 1 -- start start 2024-09-15T23:11:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.692+0000 7fea9d200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea98103b70 0x7fea98199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.692+0000 7fea9d200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea98104d60 0x7fea9819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.692+0000 7fea9d200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea98105720 0x7fea981a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.693+0000 7fea9d200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fea9810f7f0 con 0x7fea98105720 2024-09-15T23:11:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.693+0000 7fea9d200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fea9810f670 con 0x7fea98104d60 2024-09-15T23:11:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.693+0000 7fea9d200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fea9810f970 con 0x7fea98103b70 2024-09-15T23:11:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.693+0000 7fea97400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea98103b70 0x7fea98199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.693+0000 7fea97400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea98103b70 0x7fea98199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:52490/0 (socket says 172.21.15.111:52490) 2024-09-15T23:11:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.693+0000 7fea97400640 1 -- 172.21.15.111:0/4059975411 learned_addr learned my addr 172.21.15.111:0/4059975411 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:11:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.693+0000 7fea96a00640 1 --2- 172.21.15.111:0/4059975411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea98104d60 0x7fea9819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.693+0000 7fea97e00640 1 --2- 172.21.15.111:0/4059975411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea98105720 0x7fea981a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.693+0000 7fea97400640 1 -- 172.21.15.111:0/4059975411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea98104d60 msgr2=0x7fea9819a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.693+0000 7fea97400640 1 --2- 172.21.15.111:0/4059975411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea98104d60 0x7fea9819a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.693+0000 7fea97400640 1 -- 172.21.15.111:0/4059975411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea98105720 msgr2=0x7fea981a1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.693+0000 7fea97400640 1 --2- 172.21.15.111:0/4059975411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea98105720 0x7fea981a1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.693+0000 7fea97400640 1 -- 172.21.15.111:0/4059975411 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea981a1ac0 con 0x7fea98103b70 2024-09-15T23:11:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.694+0000 7fea97400640 1 --2- 172.21.15.111:0/4059975411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea98103b70 0x7fea98199ea0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fea88031610 tx=0x7fea88004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.694+0000 7fea8fe00640 1 -- 172.21.15.111:0/4059975411 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8803f030 con 0x7fea98103b70 2024-09-15T23:11:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.694+0000 7fea8fe00640 1 -- 172.21.15.111:0/4059975411 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea88003ad0 con 0x7fea98103b70 2024-09-15T23:11:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.694+0000 7fea8fe00640 1 -- 172.21.15.111:0/4059975411 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea88035070 con 0x7fea98103b70 2024-09-15T23:11:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.694+0000 7fea9d200640 1 -- 172.21.15.111:0/4059975411 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea981a1d50 con 0x7fea98103b70 2024-09-15T23:11:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.694+0000 7fea9d200640 1 -- 172.21.15.111:0/4059975411 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea981a2190 con 0x7fea98103b70 2024-09-15T23:11:15.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.697+0000 7feaa16d6740 1 Processor -- start 2024-09-15T23:11:15.699 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.698+0000 7feaa16d6740 1 -- start start 2024-09-15T23:11:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.698+0000 7fea8fe00640 1 -- 172.21.15.111:0/4059975411 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fea88042030 con 0x7fea98103b70 2024-09-15T23:11:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.698+0000 7feaa16d6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640676df260 0x5640676df660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.698+0000 7feaa16d6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640676dfba0 0x5640676e4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.698+0000 7feaa16d6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640676e4c30 0x5640676e70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.698+0000 7feaa16d6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x564067644100 con 0x5640676dfba0 2024-09-15T23:11:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.698+0000 7feaa16d6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5640675ec1b0 con 0x5640676e4c30 2024-09-15T23:11:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.698+0000 7feaa16d6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5640676400c0 con 0x5640676df260 2024-09-15T23:11:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.698+0000 7fea97e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640676df260 0x5640676df660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.698+0000 7fea96a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640676e4c30 0x5640676e70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.698+0000 7fea97e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640676df260 0x5640676df660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:52506/0 (socket says 172.21.15.111:52506) 2024-09-15T23:11:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.698+0000 7fea97e00640 1 -- 172.21.15.111:0/2680473351 learned_addr learned my addr 172.21.15.111:0/2680473351 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:11:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.698+0000 7fea97400640 1 --2- 172.21.15.111:0/2680473351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640676dfba0 0x5640676e4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.699+0000 7fea97e00640 1 -- 172.21.15.111:0/2680473351 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640676e4c30 msgr2=0x5640676e70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.699+0000 7fea97e00640 1 --2- 172.21.15.111:0/2680473351 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640676e4c30 0x5640676e70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.699+0000 7fea97e00640 1 -- 172.21.15.111:0/2680473351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640676dfba0 msgr2=0x5640676e4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.699+0000 7fea97e00640 1 --2- 172.21.15.111:0/2680473351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640676dfba0 0x5640676e4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.699+0000 7fea97e00640 1 -- 172.21.15.111:0/2680473351 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564067357210 con 0x5640676df260 2024-09-15T23:11:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.699+0000 7fea97e00640 1 --2- 172.21.15.111:0/2680473351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640676df260 0x5640676df660 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fea90009fd0 tx=0x7fea900118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1f5d19bfb54d2f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.700+0000 7fea8cc00640 1 -- 172.21.15.111:0/2680473351 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90004d20 con 0x5640676df260 2024-09-15T23:11:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.700+0000 7fea8cc00640 1 -- 172.21.15.111:0/2680473351 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea90004050 con 0x5640676df260 2024-09-15T23:11:15.702 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.700+0000 7fea8cc00640 1 -- 172.21.15.111:0/2680473351 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90005280 con 0x5640676df260 2024-09-15T23:11:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.700+0000 7fea8fe00640 1 --2- 172.21.15.111:0/4059975411 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fea7c0a3240 0x7fea7c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.700+0000 7fea97400640 1 --2- 172.21.15.111:0/4059975411 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fea7c0a3240 0x7fea7c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.700+0000 7fea8fe00640 1 -- 172.21.15.111:0/4059975411 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fea880efe20 con 0x7fea98103b70 2024-09-15T23:11:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.701+0000 7feaa16d6740 1 -- 172.21.15.111:0/2680473351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640676df260 msgr2=0x5640676df660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:15.703 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.701+0000 7feaa16d6740 1 --2- 172.21.15.111:0/2680473351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640676df260 0x5640676df660 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fea90009fd0 tx=0x7fea900118d0 comp rx=0 tx=0).stop 2024-09-15T23:11:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.701+0000 7feaa16d6740 1 -- 172.21.15.111:0/2680473351 shutdown_connections 2024-09-15T23:11:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.701+0000 7feaa16d6740 1 --2- 172.21.15.111:0/2680473351 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640676e4c30 0x5640676e70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.701+0000 7feaa16d6740 1 --2- 172.21.15.111:0/2680473351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640676dfba0 0x5640676e4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.701+0000 7feaa16d6740 1 --2- 172.21.15.111:0/2680473351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640676df260 0x5640676df660 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.701+0000 7feaa16d6740 1 -- 172.21.15.111:0/2680473351 >> 172.21.15.111:0/2680473351 conn(0x5640676da8d0 msgr2=0x5640676dacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.701+0000 7feaa16d6740 1 -- 172.21.15.111:0/2680473351 shutdown_connections 2024-09-15T23:11:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.702+0000 7feaa16d6740 1 -- 172.21.15.111:0/2680473351 wait complete. 2024-09-15T23:11:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.702+0000 7fea97400640 1 --2- 172.21.15.111:0/4059975411 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fea7c0a3240 0x7fea7c0a5700 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fea880edd10 tx=0x7fea880f3920 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.702+0000 7feaa16d6740 1 -- start start 2024-09-15T23:11:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.702+0000 7feaa16d6740 1 Processor -- start 2024-09-15T23:11:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.703+0000 7feaa16d6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640676df260 0x5640676e9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.703+0000 7feaa16d6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640676dfba0 0x5640676e9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.703+0000 7feaa16d6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640676e4c30 0x5640676f0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.703+0000 7feaa16d6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x5640676e7c00 con 0x5640676df260 2024-09-15T23:11:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.703+0000 7feaa16d6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5640675cd090 con 0x5640676e4c30 2024-09-15T23:11:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.703+0000 7feaa16d6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5640676e7d80 con 0x5640676dfba0 2024-09-15T23:11:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.703+0000 7feaa16d6740 10 client.0 _ll_register_callbacks cb 0x564067621f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:11:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.703+0000 7fea96a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640676dfba0 0x5640676e9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.703+0000 7fea96a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640676dfba0 0x5640676e9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:52520/0 (socket says 172.21.15.111:52520) 2024-09-15T23:11:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.703+0000 7fea96a00640 1 -- 172.21.15.111:0/3186701359 learned_addr learned my addr 172.21.15.111:0/3186701359 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:11:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.703+0000 7fea97e00640 1 --2- 172.21.15.111:0/3186701359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640676df260 0x5640676e9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.703+0000 7fea97e00640 1 --2- 172.21.15.111:0/3186701359 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640676e4c30 0x5640676f0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.703+0000 7fea96a00640 1 -- 172.21.15.111:0/3186701359 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640676e4c30 msgr2=0x5640676f0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.703+0000 7fea96a00640 1 --2- 172.21.15.111:0/3186701359 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640676e4c30 0x5640676f0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.703+0000 7fea96a00640 1 -- 172.21.15.111:0/3186701359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640676df260 msgr2=0x5640676e9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.703+0000 7fea96a00640 1 --2- 172.21.15.111:0/3186701359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640676df260 0x5640676e9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.703+0000 7fea96a00640 1 -- 172.21.15.111:0/3186701359 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564067423620 con 0x5640676dfba0 2024-09-15T23:11:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.704+0000 7fea96a00640 1 --2- 172.21.15.111:0/3186701359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640676dfba0 0x5640676e9950 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fea840126e0 tx=0x7fea84012ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8232d95b27eeb666 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.704+0000 7fea81600640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:11:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.704+0000 7fea81600640 1 -- 172.21.15.111:0/3186701359 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8400f920 con 0x5640676dfba0 2024-09-15T23:11:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.704+0000 7feaa16d6740 1 -- 172.21.15.111:0/3186701359 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea90004050 con 0x5640676dfba0 2024-09-15T23:11:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.704+0000 7fea81600640 1 -- 172.21.15.111:0/3186701359 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fea8400fac0 con 0x5640676dfba0 2024-09-15T23:11:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.704+0000 7fea81600640 1 -- 172.21.15.111:0/3186701359 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fea8400fcc0 con 0x5640676dfba0 2024-09-15T23:11:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.704+0000 7feaa16d6740 10 client.26899 fetch_fsmap learned FSMap version 35 2024-09-15T23:11:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.704+0000 7feaa16d6740 1 -- 172.21.15.111:0/3186701359 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5640676f38a0 con 0x5640676dfba0 2024-09-15T23:11:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.705+0000 7fea81600640 1 -- 172.21.15.111:0/3186701359 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea90004050 con 0x5640676dfba0 2024-09-15T23:11:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.705+0000 7fea81600640 1 -- 172.21.15.111:0/3186701359 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fea8401c440 con 0x5640676dfba0 2024-09-15T23:11:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.705+0000 7feaa16d6740 10 client.26899 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:11:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.705+0000 7feaa16d6740 10 client.26899 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:11:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.705+0000 7feaa16d6740 20 client.26899 populate_metadata read hostname 'smithi111' 2024-09-15T23:11:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.705+0000 7feaa16d6740 1 --2- 172.21.15.111:0/3186701359 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5640676f4f20 0x5640676f7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.705+0000 7feaa16d6740 4 client.26899 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:11:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.705+0000 7feaa16d6740 1 -- 172.21.15.111:0/3186701359 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564067651830 con 0x5640676f4f20 2024-09-15T23:11:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.705+0000 7fea97e00640 1 --2- 172.21.15.111:0/3186701359 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5640676f4f20 0x5640676f7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.706+0000 7fea97e00640 1 --2- 172.21.15.111:0/3186701359 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5640676f4f20 0x5640676f7330 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d04de40192341317 server_cookie=ccb7295d67203001 in_seq=0 out_seq=0 2024-09-15T23:11:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.706+0000 7fea81600640 10 client.26899 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:11:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.708+0000 7fea81600640 1 -- 172.21.15.111:0/3186701359 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564067651830 con 0x5640676f4f20 2024-09-15T23:11:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.708+0000 7fea81600640 10 client.26899 handle_command_reply: tid=0 2024-09-15T23:11:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.708+0000 7feaa16d6740 1 client.26899 shutdown 2024-09-15T23:11:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.708+0000 7feaa16d6740 1 -- 172.21.15.111:0/3186701359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640676dfba0 msgr2=0x5640676e9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.708+0000 7feaa16d6740 1 --2- 172.21.15.111:0/3186701359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640676dfba0 0x5640676e9950 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fea840126e0 tx=0x7fea84012ba0 comp rx=0 tx=0).stop 2024-09-15T23:11:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.709+0000 7feaa16d6740 1 -- 172.21.15.111:0/3186701359 shutdown_connections 2024-09-15T23:11:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.709+0000 7feaa16d6740 1 --2- 172.21.15.111:0/3186701359 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x5640676f4f20 0x5640676f7330 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.709+0000 7feaa16d6740 1 --2- 172.21.15.111:0/3186701359 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640676e4c30 0x5640676f0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.709+0000 7feaa16d6740 1 --2- 172.21.15.111:0/3186701359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640676dfba0 0x5640676e9950 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.709+0000 7fea81600640 0 client.26899 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:11:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.709+0000 7feaa16d6740 1 --2- 172.21.15.111:0/3186701359 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640676df260 0x5640676e9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.709+0000 7feaa16d6740 1 -- 172.21.15.111:0/3186701359 >> 172.21.15.111:0/3186701359 conn(0x5640676da8d0 msgr2=0x5640676dc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.709+0000 7feaa16d6740 1 -- 172.21.15.111:0/3186701359 shutdown_connections 2024-09-15T23:11:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.709+0000 7feaa16d6740 1 -- 172.21.15.111:0/3186701359 wait complete. 2024-09-15T23:11:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.709+0000 7feaa16d6740 20 client.26899 trim_cache size 0 max 16384 2024-09-15T23:11:15.718 INFO:tasks.workunit.client.0.smithi111.stdout: 10 231360 16.94 MB/sec execute 583 sec latency 76.989 ms 2024-09-15T23:11:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.726+0000 7feaa16d6740 1 Processor -- start 2024-09-15T23:11:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.726+0000 7feaa16d6740 1 -- start start 2024-09-15T23:11:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.726+0000 7feaa16d6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640676df260 0x5640676df640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.726+0000 7feaa16d6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640676dfba0 0x5640676f0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.726+0000 7feaa16d6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640676e4c30 0x5640676e0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.726+0000 7feaa16d6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fea981ad7d0 con 0x5640676df260 2024-09-15T23:11:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.726+0000 7feaa16d6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fea981ab0a0 con 0x5640676dfba0 2024-09-15T23:11:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.726+0000 7feaa16d6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fea981a8ef0 con 0x5640676e4c30 2024-09-15T23:11:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.726+0000 7fea96a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640676dfba0 0x5640676f0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.726+0000 7fea96a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640676dfba0 0x5640676f0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:34722/0 (socket says 172.21.15.111:34722) 2024-09-15T23:11:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.726+0000 7fea96a00640 1 -- 172.21.15.111:0/3425954980 learned_addr learned my addr 172.21.15.111:0/3425954980 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:11:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.726+0000 7fea97e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640676df260 0x5640676df640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.727+0000 7fea97e00640 1 --2- 172.21.15.111:0/3425954980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640676e4c30 0x5640676e0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.727+0000 7fea96a00640 1 -- 172.21.15.111:0/3425954980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640676e4c30 msgr2=0x5640676e0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.727+0000 7fea96a00640 1 --2- 172.21.15.111:0/3425954980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640676e4c30 0x5640676e0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.727+0000 7fea96a00640 1 -- 172.21.15.111:0/3425954980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640676df260 msgr2=0x5640676df640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.727+0000 7fea96a00640 1 --2- 172.21.15.111:0/3425954980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640676df260 0x5640676df640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.727+0000 7fea96a00640 1 -- 172.21.15.111:0/3425954980 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640676e8480 con 0x5640676dfba0 2024-09-15T23:11:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.727+0000 7fea97e00640 1 --2- 172.21.15.111:0/3425954980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640676df260 0x5640676df640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:11:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.727+0000 7fea96a00640 1 --2- 172.21.15.111:0/3425954980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640676dfba0 0x5640676f0950 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fea8401c830 tx=0x7fea8400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf8d634658af2b44 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.727+0000 7fea83e00640 1 -- 172.21.15.111:0/3425954980 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8402b030 con 0x5640676dfba0 2024-09-15T23:11:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.727+0000 7fea83e00640 1 -- 172.21.15.111:0/3425954980 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea84005510 con 0x5640676dfba0 2024-09-15T23:11:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.727+0000 7fea83e00640 1 -- 172.21.15.111:0/3425954980 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8401c640 con 0x5640676dfba0 2024-09-15T23:11:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.728+0000 7feaa16d6740 1 -- 172.21.15.111:0/3425954980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640676dfba0 msgr2=0x5640676f0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.728+0000 7feaa16d6740 1 --2- 172.21.15.111:0/3425954980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640676dfba0 0x5640676f0950 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fea8401c830 tx=0x7fea8400f790 comp rx=0 tx=0).stop 2024-09-15T23:11:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.728+0000 7feaa16d6740 1 -- 172.21.15.111:0/3425954980 shutdown_connections 2024-09-15T23:11:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.728+0000 7feaa16d6740 1 --2- 172.21.15.111:0/3425954980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640676e4c30 0x5640676e0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.728+0000 7feaa16d6740 1 --2- 172.21.15.111:0/3425954980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640676dfba0 0x5640676f0950 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.728+0000 7feaa16d6740 1 --2- 172.21.15.111:0/3425954980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640676df260 0x5640676df640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.728+0000 7feaa16d6740 1 -- 172.21.15.111:0/3425954980 >> 172.21.15.111:0/3425954980 conn(0x5640676da8d0 msgr2=0x5640676e43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.728+0000 7feaa16d6740 1 -- 172.21.15.111:0/3425954980 shutdown_connections 2024-09-15T23:11:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.729+0000 7feaa16d6740 1 -- 172.21.15.111:0/3425954980 wait complete. 2024-09-15T23:11:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.729+0000 7feaa16d6740 1 -- start start 2024-09-15T23:11:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.729+0000 7feaa16d6740 1 Processor -- start 2024-09-15T23:11:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.730+0000 7feaa16d6740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640676df260 0x5640676ed960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.730+0000 7feaa16d6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640676dfba0 0x5640676edea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.730+0000 7feaa16d6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640676e4c30 0x56406775dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.730+0000 7feaa16d6740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fea981a9560 con 0x5640676df260 2024-09-15T23:11:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.730+0000 7feaa16d6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fea981aa9d0 con 0x5640676e4c30 2024-09-15T23:11:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.730+0000 7feaa16d6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fea981a9c30 con 0x5640676dfba0 2024-09-15T23:11:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.730+0000 7feaa16d6740 10 client.0 _ll_register_callbacks cb 0x7fea84009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:11:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.730+0000 7fea96a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640676dfba0 0x5640676edea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.730+0000 7fea96a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640676dfba0 0x5640676edea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:52530/0 (socket says 172.21.15.111:52530) 2024-09-15T23:11:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.730+0000 7fea96a00640 1 -- 172.21.15.111:0/2575565970 learned_addr learned my addr 172.21.15.111:0/2575565970 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:11:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.730+0000 7fea97e00640 1 --2- 172.21.15.111:0/2575565970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640676df260 0x5640676ed960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.730+0000 7fea96a00640 1 -- 172.21.15.111:0/2575565970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640676e4c30 msgr2=0x56406775dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.730+0000 7fea96a00640 1 --2- 172.21.15.111:0/2575565970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640676e4c30 0x56406775dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.730+0000 7fea96a00640 1 -- 172.21.15.111:0/2575565970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640676df260 msgr2=0x5640676ed960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.730+0000 7fea96a00640 1 --2- 172.21.15.111:0/2575565970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640676df260 0x5640676ed960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.730+0000 7fea96a00640 1 -- 172.21.15.111:0/2575565970 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56406775e170 con 0x5640676dfba0 2024-09-15T23:11:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.731+0000 7fea96a00640 1 --2- 172.21.15.111:0/2575565970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640676dfba0 0x5640676edea0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fea84012420 tx=0x7fea84005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23360e4756ef1cf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.731+0000 7fea81600640 10 client.0 ms_handle_connect on v2:172.21.15.192:3300/0 2024-09-15T23:11:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.731+0000 7fea81600640 1 -- 172.21.15.111:0/2575565970 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea84035060 con 0x5640676dfba0 2024-09-15T23:11:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.731+0000 7feaa16d6740 1 -- 172.21.15.111:0/2575565970 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea84005510 con 0x5640676dfba0 2024-09-15T23:11:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.731+0000 7fea81600640 1 -- 172.21.15.111:0/2575565970 <== mon.1 v2:172.21.15.192:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fea840052d0 con 0x5640676dfba0 2024-09-15T23:11:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.731+0000 7fea81600640 1 -- 172.21.15.111:0/2575565970 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fea8402b070 con 0x5640676dfba0 2024-09-15T23:11:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.731+0000 7feaa16d6740 10 client.26905 fetch_fsmap learned FSMap version 35 2024-09-15T23:11:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.731+0000 7feaa16d6740 1 -- 172.21.15.111:0/2575565970 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56406775f480 con 0x5640676dfba0 2024-09-15T23:11:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.732+0000 7fea81600640 1 -- 172.21.15.111:0/2575565970 <== mon.1 v2:172.21.15.192:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea84005510 con 0x5640676dfba0 2024-09-15T23:11:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.732+0000 7fea81600640 1 -- 172.21.15.111:0/2575565970 <== mon.1 v2:172.21.15.192:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fea8403a070 con 0x5640676dfba0 2024-09-15T23:11:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.732+0000 7feaa16d6740 10 client.26905 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:11:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.732+0000 7feaa16d6740 10 client.26905 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:11:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.732+0000 7feaa16d6740 20 client.26905 populate_metadata read hostname 'smithi111' 2024-09-15T23:11:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.732+0000 7feaa16d6740 1 --2- 172.21.15.111:0/2575565970 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56406775fdd0 0x5640677621f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.732+0000 7feaa16d6740 4 client.26905 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:11:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.732+0000 7feaa16d6740 1 -- 172.21.15.111:0/2575565970 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56406764bb80 con 0x56406775fdd0 2024-09-15T23:11:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.732+0000 7fea97e00640 1 --2- 172.21.15.111:0/2575565970 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56406775fdd0 0x5640677621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.733+0000 7fea97e00640 1 --2- 172.21.15.111:0/2575565970 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56406775fdd0 0x5640677621f0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccadfa60316e6c2 server_cookie=7728e3d45e695b66 in_seq=0 out_seq=0 2024-09-15T23:11:15.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:15.733+0000 7fea81600640 10 client.26905 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:11:16.058 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:16.056+0000 7fea81600640 1 -- 172.21.15.111:0/2575565970 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56406764bb80 con 0x56406775fdd0 2024-09-15T23:11:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:16.057+0000 7fea81600640 10 client.26905 handle_command_reply: tid=0 2024-09-15T23:11:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:16.057+0000 7feaa16d6740 1 client.26905 shutdown 2024-09-15T23:11:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:16.057+0000 7feaa16d6740 1 -- 172.21.15.111:0/2575565970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640676dfba0 msgr2=0x5640676edea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:16.057+0000 7feaa16d6740 1 --2- 172.21.15.111:0/2575565970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640676dfba0 0x5640676edea0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fea84012420 tx=0x7fea84005f80 comp rx=0 tx=0).stop 2024-09-15T23:11:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:16.057+0000 7feaa16d6740 1 -- 172.21.15.111:0/2575565970 shutdown_connections 2024-09-15T23:11:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:16.057+0000 7feaa16d6740 1 --2- 172.21.15.111:0/2575565970 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x56406775fdd0 0x5640677621f0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:16.057+0000 7feaa16d6740 1 --2- 172.21.15.111:0/2575565970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5640676e4c30 0x56406775dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:16.057+0000 7feaa16d6740 1 --2- 172.21.15.111:0/2575565970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5640676dfba0 0x5640676edea0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:16.057+0000 7fea81600640 0 client.26905 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:11:16.059 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:16.057+0000 7feaa16d6740 1 --2- 172.21.15.111:0/2575565970 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5640676df260 0x5640676ed960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:16.057+0000 7feaa16d6740 1 -- 172.21.15.111:0/2575565970 >> 172.21.15.111:0/2575565970 conn(0x5640676da8d0 msgr2=0x5640676e93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:16.057+0000 7feaa16d6740 1 -- 172.21.15.111:0/2575565970 shutdown_connections 2024-09-15T23:11:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:16.057+0000 7feaa16d6740 1 -- 172.21.15.111:0/2575565970 wait complete. 2024-09-15T23:11:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:16.057+0000 7feaa16d6740 20 client.26905 trim_cache size 0 max 16384 2024-09-15T23:11:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:11:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:11:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:11:16.060 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:11:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:16.058+0000 7fea9d200640 1 -- 172.21.15.111:0/4059975411 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fea7c0a3240 msgr2=0x7fea7c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:16.058+0000 7fea9d200640 1 --2- 172.21.15.111:0/4059975411 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fea7c0a3240 0x7fea7c0a5700 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fea880edd10 tx=0x7fea880f3920 comp rx=0 tx=0).stop 2024-09-15T23:11:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:16.058+0000 7fea9d200640 1 -- 172.21.15.111:0/4059975411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea98103b70 msgr2=0x7fea98199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:16.058+0000 7fea9d200640 1 --2- 172.21.15.111:0/4059975411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea98103b70 0x7fea98199ea0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fea88031610 tx=0x7fea88004270 comp rx=0 tx=0).stop 2024-09-15T23:11:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:16.058+0000 7fea9d200640 1 -- 172.21.15.111:0/4059975411 shutdown_connections 2024-09-15T23:11:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:16.058+0000 7fea9d200640 1 --2- 172.21.15.111:0/4059975411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea98105720 0x7fea981a1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:16.061 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:16.058+0000 7fea9d200640 1 --2- 172.21.15.111:0/4059975411 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fea7c0a3240 0x7fea7c0a5700 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:16.058+0000 7fea9d200640 1 --2- 172.21.15.111:0/4059975411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea98104d60 0x7fea9819a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:16.058+0000 7fea9d200640 1 --2- 172.21.15.111:0/4059975411 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea98103b70 0x7fea98199ea0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:16.058+0000 7fea9d200640 1 -- 172.21.15.111:0/4059975411 >> 172.21.15.111:0/4059975411 conn(0x7fea980ff2f0 msgr2=0x7fea98108180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:16.058+0000 7fea9d200640 1 -- 172.21.15.111:0/4059975411 shutdown_connections 2024-09-15T23:11:16.062 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:16.058+0000 7fea9d200640 1 -- 172.21.15.111:0/4059975411 wait complete. 2024-09-15T23:11:16.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:16 smithi192 ceph-mon[35427]: pgmap v1111: 97 pgs: 97 active+clean; 575 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.4 MiB/s wr, 457 op/s 2024-09-15T23:11:16.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:16 smithi121 ceph-mon[34815]: pgmap v1111: 97 pgs: 97 active+clean; 575 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.4 MiB/s wr, 457 op/s 2024-09-15T23:11:16.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:16 smithi111 ceph-mon[30629]: pgmap v1111: 97 pgs: 97 active+clean; 575 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.4 MiB/s wr, 457 op/s 2024-09-15T23:11:16.718 INFO:tasks.workunit.client.0.smithi111.stdout: 10 231786 16.95 MB/sec execute 584 sec latency 194.619 ms 2024-09-15T23:11:17.718 INFO:tasks.workunit.client.0.smithi111.stdout: 10 232291 16.96 MB/sec execute 585 sec latency 101.217 ms 2024-09-15T23:11:18.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:18 smithi192 ceph-mon[35427]: pgmap v1112: 97 pgs: 97 active+clean; 582 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.7 MiB/s wr, 487 op/s 2024-09-15T23:11:18.357 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:18 smithi111 ceph-mon[30629]: pgmap v1112: 97 pgs: 97 active+clean; 582 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.7 MiB/s wr, 487 op/s 2024-09-15T23:11:18.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:18 smithi121 ceph-mon[34815]: pgmap v1112: 97 pgs: 97 active+clean; 582 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.7 MiB/s wr, 487 op/s 2024-09-15T23:11:18.718 INFO:tasks.workunit.client.0.smithi111.stdout: 10 232787 16.96 MB/sec execute 586 sec latency 88.167 ms 2024-09-15T23:11:19.718 INFO:tasks.workunit.client.0.smithi111.stdout: 10 233574 17.00 MB/sec execute 587 sec latency 74.886 ms 2024-09-15T23:11:20.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:20 smithi192 ceph-mon[35427]: pgmap v1113: 97 pgs: 97 active+clean; 586 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.5 MiB/s wr, 469 op/s 2024-09-15T23:11:20.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:20 smithi121 ceph-mon[34815]: pgmap v1113: 97 pgs: 97 active+clean; 586 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.5 MiB/s wr, 469 op/s 2024-09-15T23:11:20.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:20 smithi111 ceph-mon[30629]: pgmap v1113: 97 pgs: 97 active+clean; 586 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.5 MiB/s wr, 469 op/s 2024-09-15T23:11:20.719 INFO:tasks.workunit.client.0.smithi111.stdout: 10 234526 17.07 MB/sec execute 588 sec latency 42.944 ms 2024-09-15T23:11:21.719 INFO:tasks.workunit.client.0.smithi111.stdout: 10 235114 17.10 MB/sec execute 589 sec latency 82.457 ms 2024-09-15T23:11:22.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:22 smithi192 ceph-mon[35427]: pgmap v1114: 97 pgs: 97 active+clean; 591 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.8 MiB/s wr, 546 op/s 2024-09-15T23:11:22.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:22 smithi121 ceph-mon[34815]: pgmap v1114: 97 pgs: 97 active+clean; 591 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.8 MiB/s wr, 546 op/s 2024-09-15T23:11:22.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:22 smithi111 ceph-mon[30629]: pgmap v1114: 97 pgs: 97 active+clean; 591 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.8 MiB/s wr, 546 op/s 2024-09-15T23:11:22.719 INFO:tasks.workunit.client.0.smithi111.stdout: 10 235745 17.13 MB/sec execute 590 sec latency 89.561 ms 2024-09-15T23:11:23.719 INFO:tasks.workunit.client.0.smithi111.stdout: 10 236374 17.16 MB/sec execute 591 sec latency 80.103 ms 2024-09-15T23:11:24.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:24 smithi192 ceph-mon[35427]: pgmap v1115: 97 pgs: 97 active+clean; 596 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.5 MiB/s wr, 489 op/s 2024-09-15T23:11:24.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:24 smithi121 ceph-mon[34815]: pgmap v1115: 97 pgs: 97 active+clean; 596 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.5 MiB/s wr, 489 op/s 2024-09-15T23:11:24.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:24 smithi111 ceph-mon[30629]: pgmap v1115: 97 pgs: 97 active+clean; 596 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.5 MiB/s wr, 489 op/s 2024-09-15T23:11:24.719 INFO:tasks.workunit.client.0.smithi111.stdout: 10 236995 17.18 MB/sec execute 592 sec latency 65.627 ms 2024-09-15T23:11:25.720 INFO:tasks.workunit.client.0.smithi111.stdout: 10 237685 17.24 MB/sec execute 593 sec latency 54.903 ms 2024-09-15T23:11:26.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:26 smithi192 ceph-mon[35427]: pgmap v1116: 97 pgs: 97 active+clean; 600 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.6 MiB/s wr, 511 op/s 2024-09-15T23:11:26.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:26 smithi121 ceph-mon[34815]: pgmap v1116: 97 pgs: 97 active+clean; 600 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.6 MiB/s wr, 511 op/s 2024-09-15T23:11:26.403 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:26 smithi111 ceph-mon[30629]: pgmap v1116: 97 pgs: 97 active+clean; 600 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.6 MiB/s wr, 511 op/s 2024-09-15T23:11:26.720 INFO:tasks.workunit.client.0.smithi111.stdout: 10 238354 17.28 MB/sec execute 594 sec latency 42.653 ms 2024-09-15T23:11:27.720 INFO:tasks.workunit.client.0.smithi111.stdout: 10 239043 17.30 MB/sec execute 595 sec latency 78.855 ms 2024-09-15T23:11:28.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:28 smithi111 ceph-mon[30629]: pgmap v1117: 97 pgs: 97 active+clean; 604 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.7 MiB/s wr, 536 op/s 2024-09-15T23:11:28.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:28 smithi192 ceph-mon[35427]: pgmap v1117: 97 pgs: 97 active+clean; 604 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.7 MiB/s wr, 536 op/s 2024-09-15T23:11:28.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:28 smithi121 ceph-mon[34815]: pgmap v1117: 97 pgs: 97 active+clean; 604 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.7 MiB/s wr, 536 op/s 2024-09-15T23:11:28.720 INFO:tasks.workunit.client.0.smithi111.stdout: 10 239738 17.35 MB/sec execute 596 sec latency 73.030 ms 2024-09-15T23:11:29.193 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.192+0000 7fe43ac00640 1 -- 172.21.15.111:0/797633040 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe434104cf0 msgr2=0x7fe4341070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.192+0000 7fe43ac00640 1 --2- 172.21.15.111:0/797633040 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe434104cf0 0x7fe4341070e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe420009900 tx=0x7fe42002a410 comp rx=0 tx=0).stop 2024-09-15T23:11:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.192+0000 7fe43ac00640 1 -- 172.21.15.111:0/797633040 shutdown_connections 2024-09-15T23:11:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.192+0000 7fe43ac00640 1 --2- 172.21.15.111:0/797633040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe434107620 0x7fe434109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.192+0000 7fe43ac00640 1 --2- 172.21.15.111:0/797633040 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe434104cf0 0x7fe4341070e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.192+0000 7fe43ac00640 1 --2- 172.21.15.111:0/797633040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4341023c0 0x7fe4341047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.192+0000 7fe43ac00640 1 -- 172.21.15.111:0/797633040 >> 172.21.15.111:0/797633040 conn(0x7fe4340fbfa0 msgr2=0x7fe4340fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.192+0000 7fe43ac00640 1 -- 172.21.15.111:0/797633040 shutdown_connections 2024-09-15T23:11:29.194 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.192+0000 7fe43ac00640 1 -- 172.21.15.111:0/797633040 wait complete. 2024-09-15T23:11:29.196 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.193+0000 7fe43ac00640 1 Processor -- start 2024-09-15T23:11:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.193+0000 7fe43ac00640 1 -- start start 2024-09-15T23:11:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.193+0000 7fe43ac00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4341023c0 0x7fe434199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.193+0000 7fe43ac00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe434104cf0 0x7fe43419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.193+0000 7fe43ac00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe434107620 0x7fe4341a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.193+0000 7fe43ac00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe43410d5d0 con 0x7fe4341023c0 2024-09-15T23:11:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.193+0000 7fe43ac00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe43410d450 con 0x7fe434104cf0 2024-09-15T23:11:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.193+0000 7fe43ac00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe43410d750 con 0x7fe434107620 2024-09-15T23:11:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.193+0000 7fe439800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4341023c0 0x7fe434199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.193+0000 7fe43a200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe434107620 0x7fe4341a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.193+0000 7fe438e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe434104cf0 0x7fe43419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.194+0000 7fe43a200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe434107620 0x7fe4341a14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:49760/0 (socket says 172.21.15.111:49760) 2024-09-15T23:11:29.197 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.194+0000 7fe43a200640 1 -- 172.21.15.111:0/3557742674 learned_addr learned my addr 172.21.15.111:0/3557742674 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:11:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.194+0000 7fe438e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe434104cf0 0x7fe43419a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:52916/0 (socket says 172.21.15.111:52916) 2024-09-15T23:11:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.194+0000 7fe43a200640 1 -- 172.21.15.111:0/3557742674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe434104cf0 msgr2=0x7fe43419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.194+0000 7fe43a200640 1 --2- 172.21.15.111:0/3557742674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe434104cf0 0x7fe43419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.194+0000 7fe43a200640 1 -- 172.21.15.111:0/3557742674 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4341023c0 msgr2=0x7fe434199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.194+0000 7fe43a200640 1 --2- 172.21.15.111:0/3557742674 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4341023c0 0x7fe434199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.194+0000 7fe43a200640 1 -- 172.21.15.111:0/3557742674 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4341a1b10 con 0x7fe434107620 2024-09-15T23:11:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.194+0000 7fe43a200640 1 --2- 172.21.15.111:0/3557742674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe434107620 0x7fe4341a14a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe42c00eb40 tx=0x7fe42c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.194+0000 7fe432000640 1 -- 172.21.15.111:0/3557742674 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c004b20 con 0x7fe434107620 2024-09-15T23:11:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.194+0000 7fe43ac00640 1 -- 172.21.15.111:0/3557742674 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4341a1e00 con 0x7fe434107620 2024-09-15T23:11:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.194+0000 7fe432000640 1 -- 172.21.15.111:0/3557742674 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe42c004cc0 con 0x7fe434107620 2024-09-15T23:11:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.194+0000 7fe432000640 1 -- 172.21.15.111:0/3557742674 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c010850 con 0x7fe434107620 2024-09-15T23:11:29.198 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.194+0000 7fe43ac00640 1 -- 172.21.15.111:0/3557742674 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4341a2270 con 0x7fe434107620 2024-09-15T23:11:29.199 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.198+0000 7fe43efd2740 1 Processor -- start 2024-09-15T23:11:29.200 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.199+0000 7fe432000640 1 -- 172.21.15.111:0/3557742674 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fe42c005020 con 0x7fe434107620 2024-09-15T23:11:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.199+0000 7fe43efd2740 1 -- start start 2024-09-15T23:11:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.200+0000 7fe43efd2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f314cc6a70 0x55f314cc6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.200+0000 7fe43efd2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f314cc73b0 0x55f314ccbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.200+0000 7fe43efd2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f314ccc440 0x55f314cce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.200+0000 7fe43efd2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f314c23100 con 0x55f314cc6a70 2024-09-15T23:11:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.200+0000 7fe43efd2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f314bcb1b0 con 0x55f314ccc440 2024-09-15T23:11:29.202 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.200+0000 7fe43efd2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f314c1f0c0 con 0x55f314cc73b0 2024-09-15T23:11:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.201+0000 7fe439800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f314cc73b0 0x55f314ccbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.201+0000 7fe439800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f314cc73b0 0x55f314ccbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:49762/0 (socket says 172.21.15.111:49762) 2024-09-15T23:11:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.201+0000 7fe439800640 1 -- 172.21.15.111:0/314383541 learned_addr learned my addr 172.21.15.111:0/314383541 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:11:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.201+0000 7fe432000640 1 --2- 172.21.15.111:0/3557742674 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fe41c0a3240 0x7fe41c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.201+0000 7fe439800640 1 -- 172.21.15.111:0/314383541 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f314ccc440 msgr2=0x55f314cce900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T23:11:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.201+0000 7fe439800640 1 --2- 172.21.15.111:0/314383541 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f314ccc440 0x55f314cce900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.201+0000 7fe439800640 1 -- 172.21.15.111:0/314383541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f314cc6a70 msgr2=0x55f314cc6e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T23:11:29.203 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.201+0000 7fe432000640 1 -- 172.21.15.111:0/3557742674 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fe42c0d0110 con 0x7fe434107620 2024-09-15T23:11:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.202+0000 7fe43a200640 1 --2- 172.21.15.111:0/3557742674 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fe41c0a3240 0x7fe41c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.202+0000 7fe439800640 1 --2- 172.21.15.111:0/314383541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f314cc6a70 0x55f314cc6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.204 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.202+0000 7fe439800640 1 -- 172.21.15.111:0/314383541 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f314936210 con 0x55f314cc73b0 2024-09-15T23:11:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.202+0000 7fe439800640 1 --2- 172.21.15.111:0/314383541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f314cc73b0 0x55f314ccbe70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe428009cf0 tx=0x7fe428011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9385bb8c13dae31 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.202+0000 7fe426a00640 1 -- 172.21.15.111:0/314383541 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42801e070 con 0x55f314cc73b0 2024-09-15T23:11:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.202+0000 7fe43a200640 1 --2- 172.21.15.111:0/3557742674 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fe41c0a3240 0x7fe41c0a5700 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe42c099100 tx=0x7fe42c005920 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.203+0000 7fe426a00640 1 -- 172.21.15.111:0/314383541 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe428009de0 con 0x55f314cc73b0 2024-09-15T23:11:29.205 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.203+0000 7fe426a00640 1 -- 172.21.15.111:0/314383541 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4280196b0 con 0x55f314cc73b0 2024-09-15T23:11:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.203+0000 7fe43efd2740 1 -- 172.21.15.111:0/314383541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f314cc73b0 msgr2=0x55f314ccbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.203+0000 7fe43efd2740 1 --2- 172.21.15.111:0/314383541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f314cc73b0 0x55f314ccbe70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe428009cf0 tx=0x7fe428011810 comp rx=0 tx=0).stop 2024-09-15T23:11:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.204+0000 7fe43efd2740 1 -- 172.21.15.111:0/314383541 shutdown_connections 2024-09-15T23:11:29.206 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.204+0000 7fe43efd2740 1 --2- 172.21.15.111:0/314383541 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f314ccc440 0x55f314cce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.204+0000 7fe43efd2740 1 --2- 172.21.15.111:0/314383541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f314cc73b0 0x55f314ccbe70 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.204+0000 7fe43efd2740 1 --2- 172.21.15.111:0/314383541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f314cc6a70 0x55f314cc6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.204+0000 7fe43efd2740 1 -- 172.21.15.111:0/314383541 >> 172.21.15.111:0/314383541 conn(0x55f314cb98d0 msgr2=0x55f314cb9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.204+0000 7fe43efd2740 1 -- 172.21.15.111:0/314383541 shutdown_connections 2024-09-15T23:11:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.204+0000 7fe43efd2740 1 -- 172.21.15.111:0/314383541 wait complete. 2024-09-15T23:11:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.204+0000 7fe43efd2740 1 -- start start 2024-09-15T23:11:29.207 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.205+0000 7fe43efd2740 1 Processor -- start 2024-09-15T23:11:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.205+0000 7fe43efd2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f314cc6a70 0x55f314cbfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.205+0000 7fe43efd2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f314cc73b0 0x55f314cc01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.205+0000 7fe43efd2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f314ccc440 0x55f314cc51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.205+0000 7fe43efd2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55f314cbe300 con 0x55f314cc73b0 2024-09-15T23:11:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.205+0000 7fe43efd2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f314bac090 con 0x55f314cc6a70 2024-09-15T23:11:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.205+0000 7fe43efd2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55f314cbe480 con 0x55f314ccc440 2024-09-15T23:11:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.205+0000 7fe43efd2740 10 client.0 _ll_register_callbacks cb 0x55f314c00f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:11:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.206+0000 7fe438e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f314cc6a70 0x55f314cbfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.206+0000 7fe438e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f314cc6a70 0x55f314cbfc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:52926/0 (socket says 172.21.15.111:52926) 2024-09-15T23:11:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.206+0000 7fe438e00640 1 -- 172.21.15.111:0/502655534 learned_addr learned my addr 172.21.15.111:0/502655534 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:11:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.206+0000 7fe439800640 1 --2- 172.21.15.111:0/502655534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f314cc73b0 0x55f314cc01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.206+0000 7fe438e00640 1 --2- 172.21.15.111:0/502655534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f314ccc440 0x55f314cc51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.206+0000 7fe438e00640 1 -- 172.21.15.111:0/502655534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f314ccc440 msgr2=0x55f314cc51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.206+0000 7fe438e00640 1 --2- 172.21.15.111:0/502655534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f314ccc440 0x55f314cc51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.206+0000 7fe438e00640 1 -- 172.21.15.111:0/502655534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f314cc73b0 msgr2=0x55f314cc01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.206+0000 7fe438e00640 1 --2- 172.21.15.111:0/502655534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f314cc73b0 0x55f314cc01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.206+0000 7fe438e00640 1 -- 172.21.15.111:0/502655534 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f314a02620 con 0x55f314cc6a70 2024-09-15T23:11:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.206+0000 7fe438e00640 1 --2- 172.21.15.111:0/502655534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f314cc6a70 0x55f314cbfc60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe420031610 tx=0x7fe420005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92e147cfbce09455 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.206+0000 7fe41b400640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:11:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.207+0000 7fe41b400640 1 -- 172.21.15.111:0/502655534 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe420035070 con 0x55f314cc6a70 2024-09-15T23:11:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.207+0000 7fe43efd2740 1 -- 172.21.15.111:0/502655534 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe428009de0 con 0x55f314cc6a70 2024-09-15T23:11:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.207+0000 7fe41b400640 1 -- 172.21.15.111:0/502655534 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fe420004240 con 0x55f314cc6a70 2024-09-15T23:11:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.207+0000 7fe41b400640 1 -- 172.21.15.111:0/502655534 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe420005030 con 0x55f314cc6a70 2024-09-15T23:11:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.207+0000 7fe43efd2740 10 client.26915 fetch_fsmap learned FSMap version 35 2024-09-15T23:11:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.207+0000 7fe43efd2740 1 -- 172.21.15.111:0/502655534 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f314ce3930 con 0x55f314cc6a70 2024-09-15T23:11:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.207+0000 7fe41b400640 1 -- 172.21.15.111:0/502655534 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe428009de0 con 0x55f314cc6a70 2024-09-15T23:11:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.208+0000 7fe41b400640 1 -- 172.21.15.111:0/502655534 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fe4200035f0 con 0x55f314cc6a70 2024-09-15T23:11:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.208+0000 7fe43efd2740 10 client.26915 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:11:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.208+0000 7fe43efd2740 10 client.26915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:11:29.212 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.208+0000 7fe43efd2740 20 client.26915 populate_metadata read hostname 'smithi111' 2024-09-15T23:11:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.208+0000 7fe43efd2740 1 --2- 172.21.15.111:0/502655534 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f314ce4f40 0x55f314ce7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.208+0000 7fe43efd2740 4 client.26915 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:11:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.208+0000 7fe43efd2740 1 -- 172.21.15.111:0/502655534 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f314c30830 con 0x55f314ce4f40 2024-09-15T23:11:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.208+0000 7fe439800640 1 --2- 172.21.15.111:0/502655534 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f314ce4f40 0x55f314ce7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.209+0000 7fe439800640 1 --2- 172.21.15.111:0/502655534 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f314ce4f40 0x55f314ce7350 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=646de32ec3062633 server_cookie=d3783d79283a1961 in_seq=0 out_seq=0 2024-09-15T23:11:29.213 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.209+0000 7fe41b400640 10 client.26915 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:11:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.212+0000 7fe41b400640 1 -- 172.21.15.111:0/502655534 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f314c30830 con 0x55f314ce4f40 2024-09-15T23:11:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.212+0000 7fe41b400640 10 client.26915 handle_command_reply: tid=0 2024-09-15T23:11:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.212+0000 7fe43efd2740 1 client.26915 shutdown 2024-09-15T23:11:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.212+0000 7fe43efd2740 1 -- 172.21.15.111:0/502655534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f314cc6a70 msgr2=0x55f314cbfc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.212+0000 7fe43efd2740 1 --2- 172.21.15.111:0/502655534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f314cc6a70 0x55f314cbfc60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe420031610 tx=0x7fe420005fd0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.213+0000 7fe43efd2740 1 -- 172.21.15.111:0/502655534 shutdown_connections 2024-09-15T23:11:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.213+0000 7fe43efd2740 1 --2- 172.21.15.111:0/502655534 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f314ce4f40 0x55f314ce7350 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.213+0000 7fe43efd2740 1 --2- 172.21.15.111:0/502655534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f314ccc440 0x55f314cc51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.213+0000 7fe43efd2740 1 --2- 172.21.15.111:0/502655534 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f314cc73b0 0x55f314cc01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.213+0000 7fe43efd2740 1 --2- 172.21.15.111:0/502655534 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f314cc6a70 0x55f314cbfc60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.213+0000 7fe41b400640 0 client.26915 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:11:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.213+0000 7fe43efd2740 1 -- 172.21.15.111:0/502655534 >> 172.21.15.111:0/502655534 conn(0x55f314cb98d0 msgr2=0x55f314cbb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.213+0000 7fe43efd2740 1 -- 172.21.15.111:0/502655534 shutdown_connections 2024-09-15T23:11:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.213+0000 7fe43efd2740 1 -- 172.21.15.111:0/502655534 wait complete. 2024-09-15T23:11:29.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.213+0000 7fe43efd2740 20 client.26915 trim_cache size 0 max 16384 2024-09-15T23:11:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.229+0000 7fe43efd2740 1 Processor -- start 2024-09-15T23:11:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.230+0000 7fe43efd2740 1 -- start start 2024-09-15T23:11:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.230+0000 7fe43efd2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f314cc6a70 0x55f314cba780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.230+0000 7fe43efd2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f314cc73b0 0x55f314cbacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.230+0000 7fe43efd2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f314ccc440 0x55f314cbb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.230+0000 7fe43efd2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe4341a9470 con 0x55f314cc6a70 2024-09-15T23:11:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.230+0000 7fe43efd2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe4341a6d40 con 0x55f314ccc440 2024-09-15T23:11:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.230+0000 7fe43efd2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe4341a4bb0 con 0x55f314cc73b0 2024-09-15T23:11:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.230+0000 7fe438e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f314cc6a70 0x55f314cba780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.230+0000 7fe439800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f314cc73b0 0x55f314cbacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:29.232 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.230+0000 7fe438e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f314cc6a70 0x55f314cba780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:56810/0 (socket says 172.21.15.111:56810) 2024-09-15T23:11:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.230+0000 7fe438e00640 1 -- 172.21.15.111:0/2453955275 learned_addr learned my addr 172.21.15.111:0/2453955275 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:11:29.233 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.230+0000 7fe438e00640 1 --2- 172.21.15.111:0/2453955275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f314ccc440 0x55f314cbb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.231+0000 7fe438e00640 1 -- 172.21.15.111:0/2453955275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f314cc73b0 msgr2=0x55f314cbacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.231+0000 7fe438e00640 1 --2- 172.21.15.111:0/2453955275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f314cc73b0 0x55f314cbacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.234 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.231+0000 7fe438e00640 1 -- 172.21.15.111:0/2453955275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f314ccc440 msgr2=0x55f314cbb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.231+0000 7fe438e00640 1 --2- 172.21.15.111:0/2453955275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f314ccc440 0x55f314cbb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.231+0000 7fe438e00640 1 -- 172.21.15.111:0/2453955275 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f314cb96f0 con 0x55f314cc6a70 2024-09-15T23:11:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.231+0000 7fe439800640 1 --2- 172.21.15.111:0/2453955275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f314cc73b0 0x55f314cbacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:11:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.231+0000 7fe438e00640 1 --2- 172.21.15.111:0/2453955275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f314cc6a70 0x55f314cba780 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe420033000 tx=0x7fe420000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88485d8a6ccb40df server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.231+0000 7fe426000640 1 -- 172.21.15.111:0/2453955275 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe420035040 con 0x55f314cc6a70 2024-09-15T23:11:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.231+0000 7fe426000640 1 -- 172.21.15.111:0/2453955275 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4200056b0 con 0x55f314cc6a70 2024-09-15T23:11:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.231+0000 7fe426000640 1 -- 172.21.15.111:0/2453955275 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4200040a0 con 0x55f314cc6a70 2024-09-15T23:11:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.232+0000 7fe43efd2740 1 -- 172.21.15.111:0/2453955275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f314cc6a70 msgr2=0x55f314cba780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.232+0000 7fe43efd2740 1 --2- 172.21.15.111:0/2453955275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f314cc6a70 0x55f314cba780 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe420033000 tx=0x7fe420000bc0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.232+0000 7fe43efd2740 1 -- 172.21.15.111:0/2453955275 shutdown_connections 2024-09-15T23:11:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.232+0000 7fe43efd2740 1 --2- 172.21.15.111:0/2453955275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f314ccc440 0x55f314cbb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.232+0000 7fe43efd2740 1 --2- 172.21.15.111:0/2453955275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f314cc73b0 0x55f314cbacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.232+0000 7fe43efd2740 1 --2- 172.21.15.111:0/2453955275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f314cc6a70 0x55f314cba780 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.232+0000 7fe43efd2740 1 -- 172.21.15.111:0/2453955275 >> 172.21.15.111:0/2453955275 conn(0x55f314cb98d0 msgr2=0x55f314cbfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.232+0000 7fe43efd2740 1 -- 172.21.15.111:0/2453955275 shutdown_connections 2024-09-15T23:11:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.232+0000 7fe43efd2740 1 -- 172.21.15.111:0/2453955275 wait complete. 2024-09-15T23:11:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.233+0000 7fe43efd2740 1 -- start start 2024-09-15T23:11:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.233+0000 7fe43efd2740 1 Processor -- start 2024-09-15T23:11:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.233+0000 7fe43efd2740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f314cc6a70 0x55f314cf6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.233+0000 7fe43efd2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f314cc73b0 0x55f314cf7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.233+0000 7fe43efd2740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f314ccc440 0x55f314cf80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.233+0000 7fe43efd2740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fe4341a5200 con 0x55f314ccc440 2024-09-15T23:11:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.233+0000 7fe43efd2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe4341a6670 con 0x55f314cc73b0 2024-09-15T23:11:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.233+0000 7fe43efd2740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe4341a58d0 con 0x55f314cc6a70 2024-09-15T23:11:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.233+0000 7fe43efd2740 10 client.0 _ll_register_callbacks cb 0x7fe42c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:11:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.234+0000 7fe438e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f314cc6a70 0x55f314cf6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.234+0000 7fe439800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f314cc73b0 0x55f314cf7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.234+0000 7fe438e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f314ccc440 0x55f314cf80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.234+0000 7fe439800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f314cc73b0 0x55f314cf7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:52948/0 (socket says 172.21.15.111:52948) 2024-09-15T23:11:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.234+0000 7fe439800640 1 -- 172.21.15.111:0/3570241273 learned_addr learned my addr 172.21.15.111:0/3570241273 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:11:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.234+0000 7fe438e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f314cc6a70 0x55f314cf6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:49794/0 (socket says 172.21.15.111:49794) 2024-09-15T23:11:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.234+0000 7fe439800640 1 -- 172.21.15.111:0/3570241273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f314cc6a70 msgr2=0x55f314cf6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.234+0000 7fe439800640 1 --2- 172.21.15.111:0/3570241273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f314cc6a70 0x55f314cf6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.234+0000 7fe439800640 1 -- 172.21.15.111:0/3570241273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f314ccc440 msgr2=0x55f314cf80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.234+0000 7fe439800640 1 --2- 172.21.15.111:0/3570241273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f314ccc440 0x55f314cf80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.234+0000 7fe439800640 1 -- 172.21.15.111:0/3570241273 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f314cf86e0 con 0x55f314cc73b0 2024-09-15T23:11:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.234+0000 7fe439800640 1 --2- 172.21.15.111:0/3570241273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f314cc73b0 0x55f314cf7110 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe428005fd0 tx=0x7fe428009930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adddc8a8bfccf9bd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.234+0000 7fe41b400640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:11:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.235+0000 7fe41b400640 1 -- 172.21.15.111:0/3570241273 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42801e040 con 0x55f314cc73b0 2024-09-15T23:11:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.235+0000 7fe43efd2740 1 -- 172.21.15.111:0/3570241273 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4200056b0 con 0x55f314cc73b0 2024-09-15T23:11:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.235+0000 7fe41b400640 1 -- 172.21.15.111:0/3570241273 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fe428003850 con 0x55f314cc73b0 2024-09-15T23:11:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.235+0000 7fe41b400640 1 -- 172.21.15.111:0/3570241273 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe428035050 con 0x55f314cc73b0 2024-09-15T23:11:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.235+0000 7fe43efd2740 10 client.26921 fetch_fsmap learned FSMap version 35 2024-09-15T23:11:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.235+0000 7fe43efd2740 1 -- 172.21.15.111:0/3570241273 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f314d4fd30 con 0x55f314cc73b0 2024-09-15T23:11:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.235+0000 7fe41b400640 1 -- 172.21.15.111:0/3570241273 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4200056b0 con 0x55f314cc73b0 2024-09-15T23:11:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.235+0000 7fe41b400640 1 -- 172.21.15.111:0/3570241273 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 35) ==== 8346+0+0 (secure 0 0 0) 0x7fe428019570 con 0x55f314cc73b0 2024-09-15T23:11:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.236+0000 7fe43efd2740 10 client.26921 fetch_fsmap finished waiting for FSMap version 35 2024-09-15T23:11:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.236+0000 7fe43efd2740 10 client.26921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:11:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.236+0000 7fe43efd2740 20 client.26921 populate_metadata read hostname 'smithi111' 2024-09-15T23:11:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.236+0000 7fe43efd2740 1 --2- 172.21.15.111:0/3570241273 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f314d50580 0x55f314d529a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.236+0000 7fe43efd2740 4 client.26921 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:11:29.241 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.236+0000 7fe43efd2740 1 -- 172.21.15.111:0/3570241273 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f314c2ab80 con 0x55f314d50580 2024-09-15T23:11:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.236+0000 7fe438e00640 1 --2- 172.21.15.111:0/3570241273 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f314d50580 0x55f314d529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.236+0000 7fe438e00640 1 --2- 172.21.15.111:0/3570241273 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f314d50580 0x55f314d529a0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=127b18b8d53c2516 server_cookie=10f897e21d94e063 in_seq=0 out_seq=0 2024-09-15T23:11:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.236+0000 7fe41b400640 10 client.26921 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:11:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.599+0000 7fe41b400640 1 -- 172.21.15.111:0/3570241273 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f314c2ab80 con 0x55f314d50580 2024-09-15T23:11:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.599+0000 7fe41b400640 10 client.26921 handle_command_reply: tid=0 2024-09-15T23:11:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.599+0000 7fe43efd2740 1 client.26921 shutdown 2024-09-15T23:11:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.599+0000 7fe43efd2740 1 -- 172.21.15.111:0/3570241273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f314cc73b0 msgr2=0x55f314cf7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.599+0000 7fe43efd2740 1 --2- 172.21.15.111:0/3570241273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f314cc73b0 0x55f314cf7110 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe428005fd0 tx=0x7fe428009930 comp rx=0 tx=0).stop 2024-09-15T23:11:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.600+0000 7fe43efd2740 1 -- 172.21.15.111:0/3570241273 shutdown_connections 2024-09-15T23:11:29.602 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.600+0000 7fe43efd2740 1 --2- 172.21.15.111:0/3570241273 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55f314d50580 0x55f314d529a0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.600+0000 7fe43efd2740 1 --2- 172.21.15.111:0/3570241273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f314ccc440 0x55f314cf80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.600+0000 7fe43efd2740 1 --2- 172.21.15.111:0/3570241273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f314cc73b0 0x55f314cf7110 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.600+0000 7fe43efd2740 1 --2- 172.21.15.111:0/3570241273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55f314cc6a70 0x55f314cf6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.600+0000 7fe43efd2740 1 -- 172.21.15.111:0/3570241273 >> 172.21.15.111:0/3570241273 conn(0x55f314cb98d0 msgr2=0x55f314cb81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.600+0000 7fe41b400640 0 client.26921 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:11:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.600+0000 7fe43efd2740 1 -- 172.21.15.111:0/3570241273 shutdown_connections 2024-09-15T23:11:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.600+0000 7fe43efd2740 1 -- 172.21.15.111:0/3570241273 wait complete. 2024-09-15T23:11:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.600+0000 7fe43efd2740 20 client.26921 trim_cache size 0 max 16384 2024-09-15T23:11:29.603 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:11:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:11:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:11:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:11:29.604 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.601+0000 7fe43ac00640 1 -- 172.21.15.111:0/3557742674 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fe41c0a3240 msgr2=0x7fe41c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.601+0000 7fe43ac00640 1 --2- 172.21.15.111:0/3557742674 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fe41c0a3240 0x7fe41c0a5700 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe42c099100 tx=0x7fe42c005920 comp rx=0 tx=0).stop 2024-09-15T23:11:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.601+0000 7fe43ac00640 1 -- 172.21.15.111:0/3557742674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe434107620 msgr2=0x7fe4341a14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.601+0000 7fe43ac00640 1 --2- 172.21.15.111:0/3557742674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe434107620 0x7fe4341a14a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe42c00eb40 tx=0x7fe42c00c2d0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.601+0000 7fe43ac00640 1 -- 172.21.15.111:0/3557742674 shutdown_connections 2024-09-15T23:11:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.601+0000 7fe43ac00640 1 --2- 172.21.15.111:0/3557742674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe434107620 0x7fe4341a14a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.601+0000 7fe43ac00640 1 --2- 172.21.15.111:0/3557742674 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fe41c0a3240 0x7fe41c0a5700 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.601+0000 7fe43ac00640 1 --2- 172.21.15.111:0/3557742674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe434104cf0 0x7fe43419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.601+0000 7fe43ac00640 1 --2- 172.21.15.111:0/3557742674 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4341023c0 0x7fe434199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.601+0000 7fe43ac00640 1 -- 172.21.15.111:0/3557742674 >> 172.21.15.111:0/3557742674 conn(0x7fe4340fbfa0 msgr2=0x7fe434101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:29.605 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.601+0000 7fe43ac00640 1 -- 172.21.15.111:0/3557742674 shutdown_connections 2024-09-15T23:11:29.606 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:29.601+0000 7fe43ac00640 1 -- 172.21.15.111:0/3557742674 wait complete. 2024-09-15T23:11:29.720 INFO:tasks.workunit.client.0.smithi111.stdout: 10 240306 17.37 MB/sec execute 597 sec latency 260.712 ms 2024-09-15T23:11:30.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:30 smithi121 ceph-mon[34815]: pgmap v1118: 97 pgs: 97 active+clean; 592 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.9 MiB/s wr, 532 op/s 2024-09-15T23:11:30.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:30 smithi111 ceph-mon[30629]: pgmap v1118: 97 pgs: 97 active+clean; 592 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.9 MiB/s wr, 532 op/s 2024-09-15T23:11:30.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:30 smithi192 ceph-mon[35427]: pgmap v1118: 97 pgs: 97 active+clean; 592 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.9 MiB/s wr, 532 op/s 2024-09-15T23:11:30.720 INFO:tasks.workunit.client.0.smithi111.stdout: 10 241013 17.40 MB/sec execute 598 sec latency 93.542 ms 2024-09-15T23:11:31.720 INFO:tasks.workunit.client.0.smithi111.stdout: 10 241628 17.42 MB/sec execute 599 sec latency 64.677 ms 2024-09-15T23:11:32.332 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:32 smithi111 ceph-mon[30629]: pgmap v1119: 97 pgs: 97 active+clean; 573 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.9 MiB/s wr, 529 op/s 2024-09-15T23:11:32.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:32 smithi121 ceph-mon[34815]: pgmap v1119: 97 pgs: 97 active+clean; 573 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.9 MiB/s wr, 529 op/s 2024-09-15T23:11:32.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:32 smithi192 ceph-mon[35427]: pgmap v1119: 97 pgs: 97 active+clean; 573 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.9 MiB/s wr, 529 op/s 2024-09-15T23:11:32.720 INFO:tasks.workunit.client.0.smithi111.stdout: 10 cleanup 600 sec 2024-09-15T23:11:33.726 INFO:tasks.workunit.client.0.smithi111.stdout: 6 cleanup 601 sec 2024-09-15T23:11:34.210 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:34 smithi111 ceph-mon[30629]: pgmap v1120: 97 pgs: 97 active+clean; 578 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.5 MiB/s wr, 474 op/s 2024-09-15T23:11:34.392 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:34 smithi121 ceph-mon[34815]: pgmap v1120: 97 pgs: 97 active+clean; 578 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.5 MiB/s wr, 474 op/s 2024-09-15T23:11:34.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:34 smithi192 ceph-mon[35427]: pgmap v1120: 97 pgs: 97 active+clean; 578 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.5 MiB/s wr, 474 op/s 2024-09-15T23:11:34.726 INFO:tasks.workunit.client.0.smithi111.stdout: 6 cleanup 602 sec 2024-09-15T23:11:35.726 INFO:tasks.workunit.client.0.smithi111.stdout: 6 cleanup 603 sec 2024-09-15T23:11:36.230 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:36 smithi111 ceph-mon[30629]: pgmap v1121: 97 pgs: 97 active+clean; 588 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.2 MiB/s wr, 510 op/s 2024-09-15T23:11:36.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:36 smithi121 ceph-mon[34815]: pgmap v1121: 97 pgs: 97 active+clean; 588 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.2 MiB/s wr, 510 op/s 2024-09-15T23:11:36.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:36 smithi192 ceph-mon[35427]: pgmap v1121: 97 pgs: 97 active+clean; 588 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.2 MiB/s wr, 510 op/s 2024-09-15T23:11:36.726 INFO:tasks.workunit.client.0.smithi111.stdout: 6 cleanup 604 sec 2024-09-15T23:11:37.047 INFO:tasks.workunit.client.0.smithi111.stdout: 0 cleanup 604 sec 2024-09-15T23:11:37.047 INFO:tasks.workunit.client.0.smithi111.stdout: 2024-09-15T23:11:37.048 INFO:tasks.workunit.client.0.smithi111.stdout: Operation Count AvgLat MaxLat 2024-09-15T23:11:37.048 INFO:tasks.workunit.client.0.smithi111.stdout: ---------------------------------------- 2024-09-15T23:11:37.048 INFO:tasks.workunit.client.0.smithi111.stdout: NTCreateX 335780 7.295 34519.637 2024-09-15T23:11:37.048 INFO:tasks.workunit.client.0.smithi111.stdout: Close 246463 0.005 1.745 2024-09-15T23:11:37.048 INFO:tasks.workunit.client.0.smithi111.stdout: Rename 14221 30.576 19078.417 2024-09-15T23:11:37.049 INFO:tasks.workunit.client.0.smithi111.stdout: Unlink 67936 16.114 27334.929 2024-09-15T23:11:37.049 INFO:tasks.workunit.client.0.smithi111.stdout: Deltree 2 134.761 243.988 2024-09-15T23:11:37.049 INFO:tasks.workunit.client.0.smithi111.stdout: Mkdir 1 0.015 0.015 2024-09-15T23:11:37.049 INFO:tasks.workunit.client.0.smithi111.stdout: Qpathinfo 304472 5.267 27336.839 2024-09-15T23:11:37.049 INFO:tasks.workunit.client.0.smithi111.stdout: Qfileinfo 53099 0.002 1.278 2024-09-15T23:11:37.049 INFO:tasks.workunit.client.0.smithi111.stdout: Qfsinfo 55818 0.252 42.941 2024-09-15T23:11:37.050 INFO:tasks.workunit.client.0.smithi111.stdout: Sfileinfo 27351 0.418 7069.157 2024-09-15T23:11:37.050 INFO:tasks.workunit.client.0.smithi111.stdout: Find 117636 0.854 11179.601 2024-09-15T23:11:37.050 INFO:tasks.workunit.client.0.smithi111.stdout: WriteX 166165 0.060 935.908 2024-09-15T23:11:37.050 INFO:tasks.workunit.client.0.smithi111.stdout: ReadX 526088 0.040 3765.779 2024-09-15T23:11:37.050 INFO:tasks.workunit.client.0.smithi111.stdout: LockX 1092 29.678 13031.969 2024-09-15T23:11:37.050 INFO:tasks.workunit.client.0.smithi111.stdout: UnlockX 1092 20.191 6402.329 2024-09-15T23:11:37.050 INFO:tasks.workunit.client.0.smithi111.stdout: Flush 23542 9.906 10052.941 2024-09-15T23:11:37.050 INFO:tasks.workunit.client.0.smithi111.stdout: 2024-09-15T23:11:37.051 INFO:tasks.workunit.client.0.smithi111.stdout:Throughput 17.4214 MB/sec 10 clients 10 procs max_latency=34519.642 ms 2024-09-15T23:11:37.052 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-15T23:11:37.053 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-15T23:11:37.943 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-09-15T23:11:37.943 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-15T23:11:38.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:38 smithi121 ceph-mon[34815]: pgmap v1122: 97 pgs: 97 active+clean; 550 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.2 MiB/s wr, 528 op/s 2024-09-15T23:11:38.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:38 smithi121 ceph-mon[34815]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T23:11:38.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:38 smithi111 ceph-mon[30629]: pgmap v1122: 97 pgs: 97 active+clean; 550 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.2 MiB/s wr, 528 op/s 2024-09-15T23:11:38.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:38 smithi111 ceph-mon[30629]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T23:11:38.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:38 smithi192 ceph-mon[35427]: pgmap v1122: 97 pgs: 97 active+clean; 550 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.2 MiB/s wr, 528 op/s 2024-09-15T23:11:38.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:38 smithi192 ceph-mon[35427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T23:11:39.265 DEBUG:teuthology.parallel:result is None 2024-09-15T23:11:39.265 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-15T23:11:40.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:40 smithi111 ceph-mon[30629]: pgmap v1123: 97 pgs: 97 active+clean; 489 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.2 MiB/s wr, 491 op/s 2024-09-15T23:11:40.366 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:40 smithi192 ceph-mon[35427]: pgmap v1123: 97 pgs: 97 active+clean; 489 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.2 MiB/s wr, 491 op/s 2024-09-15T23:11:40.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:40 smithi121 ceph-mon[34815]: pgmap v1123: 97 pgs: 97 active+clean; 489 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.2 MiB/s wr, 491 op/s 2024-09-15T23:11:41.289 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:41 smithi111 ceph-mon[30629]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T23:11:41.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:41 smithi121 ceph-mon[34815]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T23:11:41.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:41 smithi192 ceph-mon[35427]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T23:11:42.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:42 smithi121 ceph-mon[34815]: pgmap v1124: 97 pgs: 97 active+clean; 437 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.1 MiB/s wr, 439 op/s 2024-09-15T23:11:42.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:42 smithi111 ceph-mon[30629]: pgmap v1124: 97 pgs: 97 active+clean; 437 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.1 MiB/s wr, 439 op/s 2024-09-15T23:11:42.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:42 smithi192 ceph-mon[35427]: pgmap v1124: 97 pgs: 97 active+clean; 437 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.1 MiB/s wr, 439 op/s 2024-09-15T23:11:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.723+0000 7f6428a00640 1 -- 172.21.15.111:0/3494619561 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64241023c0 msgr2=0x7f64241047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.723+0000 7f6428a00640 1 --2- 172.21.15.111:0/3494619561 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64241023c0 0x7f64241047b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6414009860 tx=0x7f641402a360 comp rx=0 tx=0).stop 2024-09-15T23:11:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.723+0000 7f6428a00640 1 -- 172.21.15.111:0/3494619561 shutdown_connections 2024-09-15T23:11:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.723+0000 7f6428a00640 1 --2- 172.21.15.111:0/3494619561 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6424107620 0x7f6424109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.723+0000 7f6428a00640 1 --2- 172.21.15.111:0/3494619561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6424104cf0 0x7f64241070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.723+0000 7f6428a00640 1 --2- 172.21.15.111:0/3494619561 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64241023c0 0x7f64241047b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.723+0000 7f6428a00640 1 -- 172.21.15.111:0/3494619561 >> 172.21.15.111:0/3494619561 conn(0x7f64240fbfa0 msgr2=0x7f64240fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.723+0000 7f6428a00640 1 -- 172.21.15.111:0/3494619561 shutdown_connections 2024-09-15T23:11:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.723+0000 7f6428a00640 1 -- 172.21.15.111:0/3494619561 wait complete. 2024-09-15T23:11:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.724+0000 7f6428a00640 1 Processor -- start 2024-09-15T23:11:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.724+0000 7f6428a00640 1 -- start start 2024-09-15T23:11:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.724+0000 7f6428a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64241023c0 0x7f6424199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.724+0000 7f6428a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6424104cf0 0x7f642419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.724+0000 7f6428a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6424107620 0x7f64241a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.724+0000 7f6428a00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f642410d680 con 0x7f6424107620 2024-09-15T23:11:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.724+0000 7f6428a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f642410d500 con 0x7f6424104cf0 2024-09-15T23:11:42.726 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.724+0000 7f6428a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f642410d800 con 0x7f64241023c0 2024-09-15T23:11:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.724+0000 7f641be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6424104cf0 0x7f642419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.724+0000 7f6423e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6424107620 0x7f64241a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:42.729 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.724+0000 7f641be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6424104cf0 0x7f642419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:44128/0 (socket says 172.21.15.111:44128) 2024-09-15T23:11:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.724+0000 7f641be00640 1 -- 172.21.15.111:0/2695012743 learned_addr learned my addr 172.21.15.111:0/2695012743 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:11:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.725+0000 7f6423400640 1 --2- 172.21.15.111:0/2695012743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64241023c0 0x7f6424199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.725+0000 7f641be00640 1 -- 172.21.15.111:0/2695012743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64241023c0 msgr2=0x7f6424199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.725+0000 7f641be00640 1 --2- 172.21.15.111:0/2695012743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64241023c0 0x7f6424199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.725+0000 7f641be00640 1 -- 172.21.15.111:0/2695012743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6424107620 msgr2=0x7f64241a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.725+0000 7f641be00640 1 --2- 172.21.15.111:0/2695012743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6424107620 0x7f64241a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.725+0000 7f641be00640 1 -- 172.21.15.111:0/2695012743 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64241a1ba0 con 0x7f6424104cf0 2024-09-15T23:11:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.725+0000 7f641be00640 1 --2- 172.21.15.111:0/2695012743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6424104cf0 0x7f642419a4c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f640c00e9a0 tx=0x7f640c00ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.725+0000 7f6420c00640 1 -- 172.21.15.111:0/2695012743 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c00cd80 con 0x7f6424104cf0 2024-09-15T23:11:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.725+0000 7f6420c00640 1 -- 172.21.15.111:0/2695012743 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f640c0035f0 con 0x7f6424104cf0 2024-09-15T23:11:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.725+0000 7f6428a00640 1 -- 172.21.15.111:0/2695012743 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64241a1e90 con 0x7f6424104cf0 2024-09-15T23:11:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.725+0000 7f6420c00640 1 -- 172.21.15.111:0/2695012743 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c010670 con 0x7f6424104cf0 2024-09-15T23:11:42.730 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.725+0000 7f6428a00640 1 -- 172.21.15.111:0/2695012743 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64241a22f0 con 0x7f6424104cf0 2024-09-15T23:11:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.728+0000 7f642cfe1740 1 Processor -- start 2024-09-15T23:11:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.728+0000 7f642cfe1740 1 -- start start 2024-09-15T23:11:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.728+0000 7f6420c00640 1 -- 172.21.15.111:0/2695012743 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f640c0037a0 con 0x7f6424104cf0 2024-09-15T23:11:42.734 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.729+0000 7f642cfe1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d880e3c260 0x55d880e3c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.729+0000 7f642cfe1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d880e3cba0 0x55d880e41660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.729+0000 7f642cfe1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e41c30 0x55d880e440f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.729+0000 7f642cfe1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d880da1100 con 0x55d880e41c30 2024-09-15T23:11:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.729+0000 7f642cfe1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d880d491b0 con 0x55d880e3c260 2024-09-15T23:11:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.729+0000 7f642cfe1740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d880d9d0c0 con 0x55d880e3cba0 2024-09-15T23:11:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.729+0000 7f641be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e41c30 0x55d880e440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.729+0000 7f641be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e41c30 0x55d880e440f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48596/0 (socket says 172.21.15.111:48596) 2024-09-15T23:11:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.729+0000 7f641be00640 1 -- 172.21.15.111:0/663379741 learned_addr learned my addr 172.21.15.111:0/663379741 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:11:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.729+0000 7f6423400640 1 --2- 172.21.15.111:0/663379741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d880e3cba0 0x55d880e41660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.729+0000 7f641be00640 1 -- 172.21.15.111:0/663379741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d880e3cba0 msgr2=0x55d880e41660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.729+0000 7f641be00640 1 --2- 172.21.15.111:0/663379741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d880e3cba0 0x55d880e41660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.735 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.729+0000 7f641be00640 1 -- 172.21.15.111:0/663379741 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d880e3c260 msgr2=0x55d880e3c660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:11:42.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.729+0000 7f641be00640 1 --2- 172.21.15.111:0/663379741 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d880e3c260 0x55d880e3c660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.729+0000 7f641be00640 1 -- 172.21.15.111:0/663379741 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d880ab4210 con 0x55d880e41c30 2024-09-15T23:11:42.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.730+0000 7f641be00640 1 --2- 172.21.15.111:0/663379741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e41c30 0x55d880e440f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f640c0d3b20 tx=0x7f640c0d8200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c06c5eec13e98a16 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:42.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.730+0000 7f6420c00640 1 --2- 172.21.15.111:0/2695012743 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f64080a3240 0x7f64080a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:42.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.730+0000 7f6413e00640 1 -- 172.21.15.111:0/663379741 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c0d8c30 con 0x55d880e41c30 2024-09-15T23:11:42.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.730+0000 7f6413e00640 1 -- 172.21.15.111:0/663379741 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f640c0e1070 con 0x55d880e41c30 2024-09-15T23:11:42.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.730+0000 7f6413e00640 1 -- 172.21.15.111:0/663379741 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c0b7cf0 con 0x55d880e41c30 2024-09-15T23:11:42.736 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.730+0000 7f6420c00640 1 -- 172.21.15.111:0/2695012743 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f640c0cfb80 con 0x7f6424104cf0 2024-09-15T23:11:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.730+0000 7f6423e00640 1 --2- 172.21.15.111:0/2695012743 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f64080a3240 0x7f64080a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.731+0000 7f642cfe1740 1 -- 172.21.15.111:0/663379741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e41c30 msgr2=0x55d880e440f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.731+0000 7f642cfe1740 1 --2- 172.21.15.111:0/663379741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e41c30 0x55d880e440f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f640c0d3b20 tx=0x7f640c0d8200 comp rx=0 tx=0).stop 2024-09-15T23:11:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.731+0000 7f642cfe1740 1 -- 172.21.15.111:0/663379741 shutdown_connections 2024-09-15T23:11:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.731+0000 7f642cfe1740 1 --2- 172.21.15.111:0/663379741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e41c30 0x55d880e440f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.731+0000 7f642cfe1740 1 --2- 172.21.15.111:0/663379741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d880e3cba0 0x55d880e41660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.731+0000 7f642cfe1740 1 --2- 172.21.15.111:0/663379741 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d880e3c260 0x55d880e3c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.737 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.731+0000 7f642cfe1740 1 -- 172.21.15.111:0/663379741 >> 172.21.15.111:0/663379741 conn(0x55d880e378d0 msgr2=0x55d880e37cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.731+0000 7f6423e00640 1 --2- 172.21.15.111:0/2695012743 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f64080a3240 0x7f64080a5700 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f642406a980 tx=0x7f641c00d040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.731+0000 7f642cfe1740 1 -- 172.21.15.111:0/663379741 shutdown_connections 2024-09-15T23:11:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.731+0000 7f642cfe1740 1 -- 172.21.15.111:0/663379741 wait complete. 2024-09-15T23:11:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.732+0000 7f642cfe1740 1 -- start start 2024-09-15T23:11:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.732+0000 7f642cfe1740 1 Processor -- start 2024-09-15T23:11:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.732+0000 7f642cfe1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e3c260 0x55d880e462c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.732+0000 7f642cfe1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d880e3cba0 0x55d880e46800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.732+0000 7f642cfe1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d880e41c30 0x55d880e4d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.732+0000 7f642cfe1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55d880e44ba0 con 0x55d880e3c260 2024-09-15T23:11:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.732+0000 7f642cfe1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d880d2a090 con 0x55d880e41c30 2024-09-15T23:11:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.732+0000 7f642cfe1740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55d880e44d20 con 0x55d880e3cba0 2024-09-15T23:11:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.732+0000 7f642cfe1740 10 client.0 _ll_register_callbacks cb 0x55d880d7ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:11:42.738 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.732+0000 7f6423400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e3c260 0x55d880e462c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.733+0000 7f6423400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e3c260 0x55d880e462c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48608/0 (socket says 172.21.15.111:48608) 2024-09-15T23:11:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.733+0000 7f6423400640 1 -- 172.21.15.111:0/1374599895 learned_addr learned my addr 172.21.15.111:0/1374599895 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:11:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.733+0000 7f6423e00640 1 --2- 172.21.15.111:0/1374599895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d880e41c30 0x55d880e4d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.733+0000 7f6423400640 1 -- 172.21.15.111:0/1374599895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d880e3cba0 msgr2=0x55d880e46800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T23:11:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.733+0000 7f6423400640 1 --2- 172.21.15.111:0/1374599895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d880e3cba0 0x55d880e46800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.733+0000 7f6423400640 1 -- 172.21.15.111:0/1374599895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d880e41c30 msgr2=0x55d880e4d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.733+0000 7f6423400640 1 --2- 172.21.15.111:0/1374599895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d880e41c30 0x55d880e4d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.739 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.733+0000 7f6423400640 1 -- 172.21.15.111:0/1374599895 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d880b80620 con 0x55d880e3c260 2024-09-15T23:11:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.733+0000 7f6423400640 1 --2- 172.21.15.111:0/1374599895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e3c260 0x55d880e462c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6414030410 tx=0x7f641402ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b3e67112eec76e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.733+0000 7f6410c00640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:11:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.733+0000 7f6410c00640 1 -- 172.21.15.111:0/1374599895 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6414037040 con 0x55d880e3c260 2024-09-15T23:11:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.734+0000 7f6410c00640 1 -- 172.21.15.111:0/1374599895 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f6414004040 con 0x55d880e3c260 2024-09-15T23:11:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.734+0000 7f642cfe1740 1 -- 172.21.15.111:0/1374599895 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f640c0e1070 con 0x55d880e3c260 2024-09-15T23:11:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.734+0000 7f6410c00640 1 -- 172.21.15.111:0/1374599895 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6414004240 con 0x55d880e3c260 2024-09-15T23:11:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.734+0000 7f642cfe1740 10 client.16989 fetch_fsmap learned FSMap version 37 2024-09-15T23:11:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.734+0000 7f642cfe1740 1 -- 172.21.15.111:0/1374599895 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d880e506a0 con 0x55d880e3c260 2024-09-15T23:11:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.734+0000 7f6410c00640 1 -- 172.21.15.111:0/1374599895 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f640c0e1070 con 0x55d880e3c260 2024-09-15T23:11:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.734+0000 7f6410c00640 1 -- 172.21.15.111:0/1374599895 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f6414004c70 con 0x55d880e3c260 2024-09-15T23:11:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.734+0000 7f642cfe1740 10 client.16989 fetch_fsmap finished waiting for FSMap version 37 2024-09-15T23:11:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.734+0000 7f642cfe1740 10 client.16989 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:11:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.734+0000 7f642cfe1740 20 client.16989 populate_metadata read hostname 'smithi111' 2024-09-15T23:11:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.734+0000 7f642cfe1740 1 --2- 172.21.15.111:0/1374599895 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d880e51e30 0x55d880e542b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.734+0000 7f642cfe1740 4 client.16989 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:11:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.734+0000 7f642cfe1740 1 -- 172.21.15.111:0/1374599895 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d880dae830 con 0x55d880e51e30 2024-09-15T23:11:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.735+0000 7f641be00640 1 --2- 172.21.15.111:0/1374599895 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d880e51e30 0x55d880e542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.735+0000 7f641be00640 1 --2- 172.21.15.111:0/1374599895 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d880e51e30 0x55d880e542b0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81dc38bfbab4b69a server_cookie=f8813934f5a4795c in_seq=0 out_seq=0 2024-09-15T23:11:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.735+0000 7f6410c00640 10 client.16989 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:11:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.736+0000 7f6410c00640 1 -- 172.21.15.111:0/1374599895 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d880dae830 con 0x55d880e51e30 2024-09-15T23:11:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.736+0000 7f6410c00640 10 client.16989 handle_command_reply: tid=0 2024-09-15T23:11:42.743 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.736+0000 7f642cfe1740 1 client.16989 shutdown 2024-09-15T23:11:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.736+0000 7f642cfe1740 1 -- 172.21.15.111:0/1374599895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e3c260 msgr2=0x55d880e462c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.736+0000 7f642cfe1740 1 --2- 172.21.15.111:0/1374599895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e3c260 0x55d880e462c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6414030410 tx=0x7f641402ade0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.736+0000 7f642cfe1740 1 -- 172.21.15.111:0/1374599895 shutdown_connections 2024-09-15T23:11:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.736+0000 7f642cfe1740 1 --2- 172.21.15.111:0/1374599895 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d880e51e30 0x55d880e542b0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.736+0000 7f642cfe1740 1 --2- 172.21.15.111:0/1374599895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d880e41c30 0x55d880e4d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.736+0000 7f642cfe1740 1 --2- 172.21.15.111:0/1374599895 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d880e3cba0 0x55d880e46800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.736+0000 7f642cfe1740 1 --2- 172.21.15.111:0/1374599895 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e3c260 0x55d880e462c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.736+0000 7f6410c00640 0 client.16989 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:11:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.736+0000 7f642cfe1740 1 -- 172.21.15.111:0/1374599895 >> 172.21.15.111:0/1374599895 conn(0x55d880e378d0 msgr2=0x55d880e390e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.736+0000 7f642cfe1740 1 -- 172.21.15.111:0/1374599895 shutdown_connections 2024-09-15T23:11:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.736+0000 7f642cfe1740 1 -- 172.21.15.111:0/1374599895 wait complete. 2024-09-15T23:11:42.744 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.737+0000 7f642cfe1740 20 client.16989 trim_cache size 0 max 16384 2024-09-15T23:11:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.752+0000 7f642cfe1740 1 Processor -- start 2024-09-15T23:11:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.752+0000 7f642cfe1740 1 -- start start 2024-09-15T23:11:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.752+0000 7f642cfe1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d880e3c260 0x55d880e466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.752+0000 7f642cfe1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e3cba0 0x55d880e4d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.752+0000 7f642cfe1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d880e41c30 0x55d880e38cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:42.754 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.752+0000 7f642cfe1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f64241ad930 con 0x55d880e3cba0 2024-09-15T23:11:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.752+0000 7f642cfe1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f64241ab1a0 con 0x55d880e41c30 2024-09-15T23:11:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.752+0000 7f642cfe1740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f64241a8ff0 con 0x55d880e3c260 2024-09-15T23:11:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.752+0000 7f641be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e3cba0 0x55d880e4d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.752+0000 7f641be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e3cba0 0x55d880e4d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48612/0 (socket says 172.21.15.111:48612) 2024-09-15T23:11:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.752+0000 7f641be00640 1 -- 172.21.15.111:0/561559648 learned_addr learned my addr 172.21.15.111:0/561559648 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:11:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.752+0000 7f6423400640 1 --2- 172.21.15.111:0/561559648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d880e3c260 0x55d880e466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.752+0000 7f6423e00640 1 --2- 172.21.15.111:0/561559648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d880e41c30 0x55d880e38cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.752+0000 7f641be00640 1 -- 172.21.15.111:0/561559648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d880e3c260 msgr2=0x55d880e466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.752+0000 7f641be00640 1 --2- 172.21.15.111:0/561559648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d880e3c260 0x55d880e466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.752+0000 7f641be00640 1 -- 172.21.15.111:0/561559648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d880e41c30 msgr2=0x55d880e38cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.752+0000 7f641be00640 1 --2- 172.21.15.111:0/561559648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d880e41c30 0x55d880e38cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.752+0000 7f641be00640 1 -- 172.21.15.111:0/561559648 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d880e376f0 con 0x55d880e3cba0 2024-09-15T23:11:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.753+0000 7f641be00640 1 --2- 172.21.15.111:0/561559648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e3cba0 0x55d880e4d800 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f640c0b8f60 tx=0x7f640c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4163870954dfed45 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.753+0000 7f6413400640 1 -- 172.21.15.111:0/561559648 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c0bd900 con 0x55d880e3cba0 2024-09-15T23:11:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.753+0000 7f6413400640 1 -- 172.21.15.111:0/561559648 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f640c0e1070 con 0x55d880e3cba0 2024-09-15T23:11:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.753+0000 7f6413400640 1 -- 172.21.15.111:0/561559648 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c0bdaf0 con 0x55d880e3cba0 2024-09-15T23:11:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.754+0000 7f642cfe1740 1 -- 172.21.15.111:0/561559648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e3cba0 msgr2=0x55d880e4d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.754+0000 7f642cfe1740 1 --2- 172.21.15.111:0/561559648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e3cba0 0x55d880e4d800 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f640c0b8f60 tx=0x7f640c003f10 comp rx=0 tx=0).stop 2024-09-15T23:11:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.754+0000 7f642cfe1740 1 -- 172.21.15.111:0/561559648 shutdown_connections 2024-09-15T23:11:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.754+0000 7f642cfe1740 1 --2- 172.21.15.111:0/561559648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d880e41c30 0x55d880e38cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.754+0000 7f642cfe1740 1 --2- 172.21.15.111:0/561559648 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e3cba0 0x55d880e4d800 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.754+0000 7f642cfe1740 1 --2- 172.21.15.111:0/561559648 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d880e3c260 0x55d880e466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.754+0000 7f642cfe1740 1 -- 172.21.15.111:0/561559648 >> 172.21.15.111:0/561559648 conn(0x55d880e378d0 msgr2=0x55d880e36a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.754+0000 7f642cfe1740 1 -- 172.21.15.111:0/561559648 shutdown_connections 2024-09-15T23:11:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.754+0000 7f642cfe1740 1 -- 172.21.15.111:0/561559648 wait complete. 2024-09-15T23:11:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.754+0000 7f642cfe1740 1 -- start start 2024-09-15T23:11:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.755+0000 7f642cfe1740 1 Processor -- start 2024-09-15T23:11:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.755+0000 7f642cfe1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d880e3c260 0x55d880e4b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.755+0000 7f642cfe1740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e3cba0 0x55d880e4b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.755+0000 7f642cfe1740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d880e41c30 0x55d880e4c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.755+0000 7f642cfe1740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f64241a9660 con 0x55d880e3cba0 2024-09-15T23:11:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.755+0000 7f642cfe1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f64241aaad0 con 0x55d880e3c260 2024-09-15T23:11:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.755+0000 7f642cfe1740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f64241a9d30 con 0x55d880e41c30 2024-09-15T23:11:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.755+0000 7f642cfe1740 10 client.0 _ll_register_callbacks cb 0x7f64140096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:11:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.755+0000 7f641be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e3cba0 0x55d880e4b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.755+0000 7f641be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e3cba0 0x55d880e4b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:48622/0 (socket says 172.21.15.111:48622) 2024-09-15T23:11:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.755+0000 7f641be00640 1 -- 172.21.15.111:0/6794906 learned_addr learned my addr 172.21.15.111:0/6794906 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:11:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.755+0000 7f6423400640 1 --2- 172.21.15.111:0/6794906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d880e3c260 0x55d880e4b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.756+0000 7f6423e00640 1 --2- 172.21.15.111:0/6794906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d880e41c30 0x55d880e4c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.756+0000 7f6423400640 1 -- 172.21.15.111:0/6794906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d880e41c30 msgr2=0x55d880e4c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.756+0000 7f6423400640 1 --2- 172.21.15.111:0/6794906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d880e41c30 0x55d880e4c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.756+0000 7f6423400640 1 -- 172.21.15.111:0/6794906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e3cba0 msgr2=0x55d880e4b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.756+0000 7f6423400640 1 --2- 172.21.15.111:0/6794906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e3cba0 0x55d880e4b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.756+0000 7f6423400640 1 -- 172.21.15.111:0/6794906 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d880e4cbe0 con 0x55d880e3c260 2024-09-15T23:11:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.756+0000 7f641be00640 1 --2- 172.21.15.111:0/6794906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e3cba0 0x55d880e4b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:11:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.756+0000 7f6423400640 1 --2- 172.21.15.111:0/6794906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d880e3c260 0x55d880e4b0f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6414003730 tx=0x7f6414004150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8174ec8040d42d55 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.756+0000 7f6410c00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-15T23:11:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.756+0000 7f6410c00640 1 -- 172.21.15.111:0/6794906 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6414056060 con 0x55d880e3c260 2024-09-15T23:11:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.756+0000 7f6410c00640 1 -- 172.21.15.111:0/6794906 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f6414005230 con 0x55d880e3c260 2024-09-15T23:11:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.756+0000 7f642cfe1740 1 -- 172.21.15.111:0/6794906 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f640c0e1070 con 0x55d880e3c260 2024-09-15T23:11:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.757+0000 7f6410c00640 1 -- 172.21.15.111:0/6794906 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6414037070 con 0x55d880e3c260 2024-09-15T23:11:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.757+0000 7f642cfe1740 10 client.26936 fetch_fsmap learned FSMap version 37 2024-09-15T23:11:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.757+0000 7f642cfe1740 1 -- 172.21.15.111:0/6794906 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d880ebccf0 con 0x55d880e3c260 2024-09-15T23:11:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.757+0000 7f6410c00640 1 -- 172.21.15.111:0/6794906 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f640c0e1070 con 0x55d880e3c260 2024-09-15T23:11:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.757+0000 7f6410c00640 1 -- 172.21.15.111:0/6794906 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f641405b070 con 0x55d880e3c260 2024-09-15T23:11:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.757+0000 7f642cfe1740 10 client.26936 fetch_fsmap finished waiting for FSMap version 37 2024-09-15T23:11:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.757+0000 7f642cfe1740 10 client.26936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:11:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.757+0000 7f642cfe1740 20 client.26936 populate_metadata read hostname 'smithi111' 2024-09-15T23:11:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.757+0000 7f642cfe1740 1 --2- 172.21.15.111:0/6794906 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d880ebd6d0 0x55d880ebfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.757+0000 7f642cfe1740 4 client.26936 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:11:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.757+0000 7f642cfe1740 1 -- 172.21.15.111:0/6794906 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d880da8b80 con 0x55d880ebd6d0 2024-09-15T23:11:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.758+0000 7f641be00640 1 --2- 172.21.15.111:0/6794906 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d880ebd6d0 0x55d880ebfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.758+0000 7f641be00640 1 --2- 172.21.15.111:0/6794906 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d880ebd6d0 0x55d880ebfaf0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6361b8c9bc04854 server_cookie=4c93d3d448220c6d in_seq=0 out_seq=0 2024-09-15T23:11:42.776 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.758+0000 7f6410c00640 10 client.26936 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:11:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.890+0000 7f6410c00640 1 -- 172.21.15.111:0/6794906 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d880da8b80 con 0x55d880ebd6d0 2024-09-15T23:11:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.890+0000 7f6410c00640 10 client.26936 handle_command_reply: tid=0 2024-09-15T23:11:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.890+0000 7f642cfe1740 1 client.26936 shutdown 2024-09-15T23:11:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.891+0000 7f642cfe1740 1 -- 172.21.15.111:0/6794906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d880e3c260 msgr2=0x55d880e4b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.891+0000 7f642cfe1740 1 --2- 172.21.15.111:0/6794906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d880e3c260 0x55d880e4b0f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6414003730 tx=0x7f6414004150 comp rx=0 tx=0).stop 2024-09-15T23:11:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.891+0000 7f642cfe1740 1 -- 172.21.15.111:0/6794906 shutdown_connections 2024-09-15T23:11:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.891+0000 7f642cfe1740 1 --2- 172.21.15.111:0/6794906 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55d880ebd6d0 0x55d880ebfaf0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.891+0000 7f642cfe1740 1 --2- 172.21.15.111:0/6794906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d880e41c30 0x55d880e4c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.891+0000 7f642cfe1740 1 --2- 172.21.15.111:0/6794906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55d880e3cba0 0x55d880e4b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.891+0000 7f642cfe1740 1 --2- 172.21.15.111:0/6794906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d880e3c260 0x55d880e4b0f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.891+0000 7f6410c00640 0 client.26936 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:11:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.891+0000 7f642cfe1740 1 -- 172.21.15.111:0/6794906 >> 172.21.15.111:0/6794906 conn(0x55d880e378d0 msgr2=0x55d880e37cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.891+0000 7f642cfe1740 1 -- 172.21.15.111:0/6794906 shutdown_connections 2024-09-15T23:11:42.894 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.891+0000 7f642cfe1740 1 -- 172.21.15.111:0/6794906 wait complete. 2024-09-15T23:11:42.894 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.891+0000 7f642cfe1740 20 client.26936 trim_cache size 0 max 16384 2024-09-15T23:11:42.894 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:11:42.894 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:11:42.894 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:11:42.894 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:11:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.892+0000 7f6428a00640 1 -- 172.21.15.111:0/2695012743 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f64080a3240 msgr2=0x7f64080a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.892+0000 7f6428a00640 1 --2- 172.21.15.111:0/2695012743 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f64080a3240 0x7f64080a5700 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f642406a980 tx=0x7f641c00d040 comp rx=0 tx=0).stop 2024-09-15T23:11:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.892+0000 7f6428a00640 1 -- 172.21.15.111:0/2695012743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6424104cf0 msgr2=0x7f642419a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.892+0000 7f6428a00640 1 --2- 172.21.15.111:0/2695012743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6424104cf0 0x7f642419a4c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f640c00e9a0 tx=0x7f640c00ee60 comp rx=0 tx=0).stop 2024-09-15T23:11:42.895 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.892+0000 7f6428a00640 1 -- 172.21.15.111:0/2695012743 shutdown_connections 2024-09-15T23:11:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.892+0000 7f6428a00640 1 --2- 172.21.15.111:0/2695012743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6424107620 0x7f64241a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.892+0000 7f6428a00640 1 --2- 172.21.15.111:0/2695012743 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f64080a3240 0x7f64080a5700 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.892+0000 7f6428a00640 1 --2- 172.21.15.111:0/2695012743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6424104cf0 0x7f642419a4c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.892+0000 7f6428a00640 1 --2- 172.21.15.111:0/2695012743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64241023c0 0x7f6424199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.892+0000 7f6428a00640 1 -- 172.21.15.111:0/2695012743 >> 172.21.15.111:0/2695012743 conn(0x7f64240fbfa0 msgr2=0x7f6424101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.892+0000 7f6428a00640 1 -- 172.21.15.111:0/2695012743 shutdown_connections 2024-09-15T23:11:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:42.892+0000 7f6428a00640 1 -- 172.21.15.111:0/2695012743 wait complete. 2024-09-15T23:11:43.392 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:43 smithi111 ceph-mon[30629]: pgmap v1125: 97 pgs: 97 active+clean; 428 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.9 MiB/s wr, 387 op/s 2024-09-15T23:11:43.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:43 smithi192 ceph-mon[35427]: pgmap v1125: 97 pgs: 97 active+clean; 428 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.9 MiB/s wr, 387 op/s 2024-09-15T23:11:43.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:43 smithi121 ceph-mon[34815]: pgmap v1125: 97 pgs: 97 active+clean; 428 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.9 MiB/s wr, 387 op/s 2024-09-15T23:11:46.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:46 smithi192 ceph-mon[35427]: pgmap v1126: 97 pgs: 97 active+clean; 419 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.0 MiB/s wr, 345 op/s 2024-09-15T23:11:46.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:46 smithi121 ceph-mon[34815]: pgmap v1126: 97 pgs: 97 active+clean; 419 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.0 MiB/s wr, 345 op/s 2024-09-15T23:11:46.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:46 smithi111 ceph-mon[30629]: pgmap v1126: 97 pgs: 97 active+clean; 419 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.0 MiB/s wr, 345 op/s 2024-09-15T23:11:48.144 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:48 smithi111 ceph-mon[30629]: pgmap v1127: 97 pgs: 97 active+clean; 418 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 5.1 MiB/s wr, 276 op/s 2024-09-15T23:11:48.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:48 smithi121 ceph-mon[34815]: pgmap v1127: 97 pgs: 97 active+clean; 418 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 5.1 MiB/s wr, 276 op/s 2024-09-15T23:11:48.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:48 smithi192 ceph-mon[35427]: pgmap v1127: 97 pgs: 97 active+clean; 418 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 5.1 MiB/s wr, 276 op/s 2024-09-15T23:11:50.154 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:50 smithi111 ceph-mon[30629]: pgmap v1128: 97 pgs: 97 active+clean; 417 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 4.4 MiB/s wr, 255 op/s 2024-09-15T23:11:50.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:50 smithi121 ceph-mon[34815]: pgmap v1128: 97 pgs: 97 active+clean; 417 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 4.4 MiB/s wr, 255 op/s 2024-09-15T23:11:50.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:50 smithi192 ceph-mon[35427]: pgmap v1128: 97 pgs: 97 active+clean; 417 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 4.4 MiB/s wr, 255 op/s 2024-09-15T23:11:52.241 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:52 smithi111 ceph-mon[30629]: pgmap v1129: 97 pgs: 97 active+clean; 417 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 4.8 MiB/s wr, 237 op/s 2024-09-15T23:11:52.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:52 smithi121 ceph-mon[34815]: pgmap v1129: 97 pgs: 97 active+clean; 417 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 4.8 MiB/s wr, 237 op/s 2024-09-15T23:11:52.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:52 smithi192 ceph-mon[35427]: pgmap v1129: 97 pgs: 97 active+clean; 417 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 4.8 MiB/s wr, 237 op/s 2024-09-15T23:11:54.208 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:54 smithi111 ceph-mon[30629]: pgmap v1130: 97 pgs: 97 active+clean; 412 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 4.7 MiB/s wr, 210 op/s 2024-09-15T23:11:54.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:54 smithi121 ceph-mon[34815]: pgmap v1130: 97 pgs: 97 active+clean; 412 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 4.7 MiB/s wr, 210 op/s 2024-09-15T23:11:54.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:54 smithi192 ceph-mon[35427]: pgmap v1130: 97 pgs: 97 active+clean; 412 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 4.7 MiB/s wr, 210 op/s 2024-09-15T23:11:55.614 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:55 smithi111 ceph-mon[30629]: pgmap v1131: 97 pgs: 97 active+clean; 413 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 4.1 MiB/s wr, 191 op/s 2024-09-15T23:11:55.814 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:55 smithi192 ceph-mon[35427]: pgmap v1131: 97 pgs: 97 active+clean; 413 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 4.1 MiB/s wr, 191 op/s 2024-09-15T23:11:55.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:55 smithi121 ceph-mon[34815]: pgmap v1131: 97 pgs: 97 active+clean; 413 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 4.1 MiB/s wr, 191 op/s 2024-09-15T23:11:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.005+0000 7f6a6f200640 1 -- 172.21.15.111:0/2363867500 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a70107620 msgr2=0x7f6a70109a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.005+0000 7f6a6f200640 1 --2- 172.21.15.111:0/2363867500 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a70107620 0x7f6a70109a90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6a68009ca0 tx=0x7f6a6802a8c0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.005+0000 7f6a6f200640 1 -- 172.21.15.111:0/2363867500 shutdown_connections 2024-09-15T23:11:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.005+0000 7f6a6f200640 1 --2- 172.21.15.111:0/2363867500 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a70107620 0x7f6a70109a90 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.005+0000 7f6a6f200640 1 --2- 172.21.15.111:0/2363867500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a70104cf0 0x7f6a701070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.005+0000 7f6a6f200640 1 --2- 172.21.15.111:0/2363867500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a701023c0 0x7f6a701047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.005+0000 7f6a6f200640 1 -- 172.21.15.111:0/2363867500 >> 172.21.15.111:0/2363867500 conn(0x7f6a700fbfa0 msgr2=0x7f6a700fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.005+0000 7f6a6f200640 1 -- 172.21.15.111:0/2363867500 shutdown_connections 2024-09-15T23:11:56.009 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.005+0000 7f6a6f200640 1 -- 172.21.15.111:0/2363867500 wait complete. 2024-09-15T23:11:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.005+0000 7f6a6f200640 1 Processor -- start 2024-09-15T23:11:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.005+0000 7f6a6f200640 1 -- start start 2024-09-15T23:11:56.011 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.005+0000 7f6a6f200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a701023c0 0x7f6a70101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.005+0000 7f6a6f200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a70104cf0 0x7f6a70100370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.005+0000 7f6a6f200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a70107620 0x7f6a701008b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.005+0000 7f6a6f200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7010d5e0 con 0x7f6a70104cf0 2024-09-15T23:11:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.005+0000 7f6a6f200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7010d460 con 0x7f6a701023c0 2024-09-15T23:11:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.005+0000 7f6a6f200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7010d760 con 0x7f6a70107620 2024-09-15T23:11:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.005+0000 7f6a67e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a70104cf0 0x7f6a70100370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.005+0000 7f6a67e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a70104cf0 0x7f6a70100370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60368/0 (socket says 172.21.15.111:60368) 2024-09-15T23:11:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.005+0000 7f6a67e00640 1 -- 172.21.15.111:0/1496907228 learned_addr learned my addr 172.21.15.111:0/1496907228 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:11:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.005+0000 7f6a6de00640 1 --2- 172.21.15.111:0/1496907228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a701023c0 0x7f6a70101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.005+0000 7f6a67e00640 1 -- 172.21.15.111:0/1496907228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a70107620 msgr2=0x7f6a701008b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T23:11:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.005+0000 7f6a67e00640 1 --2- 172.21.15.111:0/1496907228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a70107620 0x7f6a701008b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.012 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.005+0000 7f6a67e00640 1 -- 172.21.15.111:0/1496907228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a701023c0 msgr2=0x7f6a70101d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.005+0000 7f6a67e00640 1 --2- 172.21.15.111:0/1496907228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a701023c0 0x7f6a70101d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.005+0000 7f6a67e00640 1 -- 172.21.15.111:0/1496907228 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a70101140 con 0x7f6a70104cf0 2024-09-15T23:11:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.010+0000 7f6a67e00640 1 --2- 172.21.15.111:0/1496907228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a70104cf0 0x7f6a70100370 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6a5800e9c0 tx=0x7f6a5800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.010+0000 7f6a66a00640 1 -- 172.21.15.111:0/1496907228 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5800cd80 con 0x7f6a70104cf0 2024-09-15T23:11:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.010+0000 7f6a66a00640 1 -- 172.21.15.111:0/1496907228 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a58005500 con 0x7f6a70104cf0 2024-09-15T23:11:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.010+0000 7f6a6f200640 1 -- 172.21.15.111:0/1496907228 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a701a1de0 con 0x7f6a70104cf0 2024-09-15T23:11:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.010+0000 7f6a66a00640 1 -- 172.21.15.111:0/1496907228 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a58010690 con 0x7f6a70104cf0 2024-09-15T23:11:56.013 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.010+0000 7f6a6f200640 1 -- 172.21.15.111:0/1496907228 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a701a22a0 con 0x7f6a70104cf0 2024-09-15T23:11:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 1 Processor -- start 2024-09-15T23:11:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a66a00640 1 -- 172.21.15.111:0/1496907228 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f6a580046c0 con 0x7f6a70104cf0 2024-09-15T23:11:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 1 -- start start 2024-09-15T23:11:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292260 0x55c3a7292660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c3a7292ba0 0x55c3a7297660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c3a7297c30 0x55c3a729a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c3a71f7100 con 0x55c3a7292260 2024-09-15T23:11:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c3a719f1b0 con 0x55c3a7297c30 2024-09-15T23:11:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c3a71f30c0 con 0x55c3a7292ba0 2024-09-15T23:11:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a6e800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292260 0x55c3a7292660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a6e800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292260 0x55c3a7292660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60376/0 (socket says 172.21.15.111:60376) 2024-09-15T23:11:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a6e800640 1 -- 172.21.15.111:0/2174640281 learned_addr learned my addr 172.21.15.111:0/2174640281 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:11:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a6de00640 1 --2- 172.21.15.111:0/2174640281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c3a7292ba0 0x55c3a7297660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a67e00640 1 --2- 172.21.15.111:0/2174640281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c3a7297c30 0x55c3a729a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a6de00640 1 -- 172.21.15.111:0/2174640281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c3a7297c30 msgr2=0x55c3a729a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a6de00640 1 --2- 172.21.15.111:0/2174640281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c3a7297c30 0x55c3a729a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a6de00640 1 -- 172.21.15.111:0/2174640281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292260 msgr2=0x55c3a7292660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a6de00640 1 --2- 172.21.15.111:0/2174640281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292260 0x55c3a7292660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a6de00640 1 -- 172.21.15.111:0/2174640281 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3a6f0a210 con 0x55c3a7292ba0 2024-09-15T23:11:56.018 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a6e800640 1 --2- 172.21.15.111:0/2174640281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292260 0x55c3a7292660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:11:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a67e00640 1 --2- 172.21.15.111:0/2174640281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c3a7297c30 0x55c3a729a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:11:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a6de00640 1 --2- 172.21.15.111:0/2174640281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c3a7292ba0 0x55c3a7297660 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6a60009c20 tx=0x7f6a60011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8dba6e755ea86d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a66a00640 1 --2- 172.21.15.111:0/1496907228 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6a540a32c0 0x7f6a540a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a66a00640 1 -- 172.21.15.111:0/1496907228 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f6a580cfcb0 con 0x7f6a70104cf0 2024-09-15T23:11:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a5f000640 1 -- 172.21.15.111:0/2174640281 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6001e070 con 0x55c3a7292ba0 2024-09-15T23:11:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a5f000640 1 -- 172.21.15.111:0/2174640281 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a60009dd0 con 0x55c3a7292ba0 2024-09-15T23:11:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a5f000640 1 -- 172.21.15.111:0/2174640281 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a60018830 con 0x55c3a7292ba0 2024-09-15T23:11:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a6e800640 1 --2- 172.21.15.111:0/1496907228 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6a540a32c0 0x7f6a540a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 1 -- 172.21.15.111:0/2174640281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c3a7292ba0 msgr2=0x55c3a7297660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a6e800640 1 --2- 172.21.15.111:0/1496907228 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6a540a32c0 0x7f6a540a5780 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6a68004af0 tx=0x7f6a68004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 1 --2- 172.21.15.111:0/2174640281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c3a7292ba0 0x55c3a7297660 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6a60009c20 tx=0x7f6a60011810 comp rx=0 tx=0).stop 2024-09-15T23:11:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 1 -- 172.21.15.111:0/2174640281 shutdown_connections 2024-09-15T23:11:56.019 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 1 --2- 172.21.15.111:0/2174640281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c3a7297c30 0x55c3a729a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 1 --2- 172.21.15.111:0/2174640281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c3a7292ba0 0x55c3a7297660 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 1 --2- 172.21.15.111:0/2174640281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292260 0x55c3a7292660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 1 -- 172.21.15.111:0/2174640281 >> 172.21.15.111:0/2174640281 conn(0x55c3a728d8d0 msgr2=0x55c3a728dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 1 -- 172.21.15.111:0/2174640281 shutdown_connections 2024-09-15T23:11:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 1 -- 172.21.15.111:0/2174640281 wait complete. 2024-09-15T23:11:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 1 -- start start 2024-09-15T23:11:56.020 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 1 Processor -- start 2024-09-15T23:11:56.023 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c3a7292260 0x55c3a729c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:56.023 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292ba0 0x55c3a729c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:56.023 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c3a7297c30 0x55c3a72a3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:56.023 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x55c3a729ac00 con 0x55c3a7292ba0 2024-09-15T23:11:56.023 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c3a7180090 con 0x55c3a7292260 2024-09-15T23:11:56.023 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c3a729ad80 con 0x55c3a7297c30 2024-09-15T23:11:56.023 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 10 client.0 _ll_register_callbacks cb 0x55c3a71d4f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:11:56.023 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a67e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292ba0 0x55c3a729c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:56.023 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a6e800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c3a7297c30 0x55c3a72a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:56.024 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a67e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292ba0 0x55c3a729c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60378/0 (socket says 172.21.15.111:60378) 2024-09-15T23:11:56.024 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a67e00640 1 -- 172.21.15.111:0/2175731178 learned_addr learned my addr 172.21.15.111:0/2175731178 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:11:56.024 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a6de00640 1 --2- 172.21.15.111:0/2175731178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c3a7292260 0x55c3a729c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:56.024 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a67e00640 1 -- 172.21.15.111:0/2175731178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c3a7297c30 msgr2=0x55c3a72a3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:56.024 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a67e00640 1 --2- 172.21.15.111:0/2175731178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c3a7297c30 0x55c3a72a3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.024 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a67e00640 1 -- 172.21.15.111:0/2175731178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c3a7292260 msgr2=0x55c3a729c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:56.024 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a67e00640 1 --2- 172.21.15.111:0/2175731178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c3a7292260 0x55c3a729c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.024 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a67e00640 1 -- 172.21.15.111:0/2175731178 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3a6fd6620 con 0x55c3a7292ba0 2024-09-15T23:11:56.024 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a67e00640 1 --2- 172.21.15.111:0/2175731178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292ba0 0x55c3a729c950 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6a580cd7b0 tx=0x7f6a5809c0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46837b29aa09256c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:56.024 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a53400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:11:56.024 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a53400640 1 -- 172.21.15.111:0/2175731178 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a580d3020 con 0x55c3a7292ba0 2024-09-15T23:11:56.024 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a53400640 1 -- 172.21.15.111:0/2175731178 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f6a580cddd0 con 0x55c3a7292ba0 2024-09-15T23:11:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 1 -- 172.21.15.111:0/2175731178 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a60009dd0 con 0x55c3a7292ba0 2024-09-15T23:11:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a53400640 1 -- 172.21.15.111:0/2175731178 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6a5809c9d0 con 0x55c3a7292ba0 2024-09-15T23:11:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 10 client.17007 fetch_fsmap learned FSMap version 37 2024-09-15T23:11:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.014+0000 7f6a77808740 1 -- 172.21.15.111:0/2175731178 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c3a72a67f0 con 0x55c3a7292ba0 2024-09-15T23:11:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.019+0000 7f6a53400640 1 -- 172.21.15.111:0/2175731178 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a60009dd0 con 0x55c3a7292ba0 2024-09-15T23:11:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.019+0000 7f6a53400640 1 -- 172.21.15.111:0/2175731178 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f6a580a4ab0 con 0x55c3a7292ba0 2024-09-15T23:11:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.019+0000 7f6a77808740 10 client.17007 fetch_fsmap finished waiting for FSMap version 37 2024-09-15T23:11:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.019+0000 7f6a77808740 10 client.17007 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:11:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.019+0000 7f6a77808740 20 client.17007 populate_metadata read hostname 'smithi111' 2024-09-15T23:11:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.019+0000 7f6a77808740 1 --2- 172.21.15.111:0/2175731178 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c3a72a7f10 0x55c3a72aa320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.019+0000 7f6a77808740 4 client.17007 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T23:11:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.019+0000 7f6a77808740 1 -- 172.21.15.111:0/2175731178 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3a7204830 con 0x55c3a72a7f10 2024-09-15T23:11:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.019+0000 7f6a6de00640 1 --2- 172.21.15.111:0/2175731178 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c3a72a7f10 0x55c3a72aa320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.019+0000 7f6a6de00640 1 --2- 172.21.15.111:0/2175731178 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c3a72a7f10 0x55c3a72aa320 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43cb9b50c95662e8 server_cookie=63791a053953c2b9 in_seq=0 out_seq=0 2024-09-15T23:11:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.019+0000 7f6a53400640 10 client.17007 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:11:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.019+0000 7f6a53400640 1 -- 172.21.15.111:0/2175731178 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c3a7204830 con 0x55c3a72a7f10 2024-09-15T23:11:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.019+0000 7f6a53400640 10 client.17007 handle_command_reply: tid=0 2024-09-15T23:11:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.019+0000 7f6a77808740 1 client.17007 shutdown 2024-09-15T23:11:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.019+0000 7f6a77808740 1 -- 172.21.15.111:0/2175731178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292ba0 msgr2=0x55c3a729c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.019+0000 7f6a77808740 1 --2- 172.21.15.111:0/2175731178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292ba0 0x55c3a729c950 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6a580cd7b0 tx=0x7f6a5809c0a0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.019+0000 7f6a77808740 1 -- 172.21.15.111:0/2175731178 shutdown_connections 2024-09-15T23:11:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.019+0000 7f6a77808740 1 --2- 172.21.15.111:0/2175731178 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c3a72a7f10 0x55c3a72aa320 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.019+0000 7f6a77808740 1 --2- 172.21.15.111:0/2175731178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c3a7297c30 0x55c3a72a3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.019+0000 7f6a77808740 1 --2- 172.21.15.111:0/2175731178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292ba0 0x55c3a729c950 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.019+0000 7f6a53400640 0 client.17007 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:11:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.019+0000 7f6a77808740 1 --2- 172.21.15.111:0/2175731178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c3a7292260 0x55c3a729c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.019+0000 7f6a77808740 1 -- 172.21.15.111:0/2175731178 >> 172.21.15.111:0/2175731178 conn(0x55c3a728d8d0 msgr2=0x55c3a728f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.019+0000 7f6a77808740 1 -- 172.21.15.111:0/2175731178 shutdown_connections 2024-09-15T23:11:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.019+0000 7f6a77808740 1 -- 172.21.15.111:0/2175731178 wait complete. 2024-09-15T23:11:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.019+0000 7f6a77808740 20 client.17007 trim_cache size 0 max 16384 2024-09-15T23:11:56.038 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.035+0000 7f6a77808740 1 Processor -- start 2024-09-15T23:11:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.035+0000 7f6a77808740 1 -- start start 2024-09-15T23:11:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.035+0000 7f6a77808740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292260 0x55c3a728e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.035+0000 7f6a77808740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c3a7292ba0 0x55c3a72a3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.035+0000 7f6a77808740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c3a7297c30 0x55c3a728ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6a701ad920 con 0x55c3a7292260 2024-09-15T23:11:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6a701ab190 con 0x55c3a7297c30 2024-09-15T23:11:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6a701a8fe0 con 0x55c3a7292ba0 2024-09-15T23:11:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a6de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292260 0x55c3a728e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a6de00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292260 0x55c3a728e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60386/0 (socket says 172.21.15.111:60386) 2024-09-15T23:11:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a6de00640 1 -- 172.21.15.111:0/2542609221 learned_addr learned my addr 172.21.15.111:0/2542609221 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:11:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a67e00640 1 --2- 172.21.15.111:0/2542609221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c3a7292ba0 0x55c3a72a3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:56.039 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a6e800640 1 --2- 172.21.15.111:0/2542609221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c3a7297c30 0x55c3a728ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a6de00640 1 -- 172.21.15.111:0/2542609221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c3a7292ba0 msgr2=0x55c3a72a3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a6de00640 1 --2- 172.21.15.111:0/2542609221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c3a7292ba0 0x55c3a72a3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a6de00640 1 -- 172.21.15.111:0/2542609221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c3a7297c30 msgr2=0x55c3a728ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a6de00640 1 --2- 172.21.15.111:0/2542609221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c3a7297c30 0x55c3a728ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a6de00640 1 -- 172.21.15.111:0/2542609221 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3a728d6f0 con 0x55c3a7292260 2024-09-15T23:11:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a6de00640 1 --2- 172.21.15.111:0/2542609221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292260 0x55c3a728e780 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x55c3a72935d0 tx=0x7f6a60005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824c9e415f2352fc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a5e600640 1 -- 172.21.15.111:0/2542609221 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6001e040 con 0x55c3a7292260 2024-09-15T23:11:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a5e600640 1 -- 172.21.15.111:0/2542609221 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a6002a070 con 0x55c3a7292260 2024-09-15T23:11:56.040 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a5e600640 1 -- 172.21.15.111:0/2542609221 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a60026690 con 0x55c3a7292260 2024-09-15T23:11:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 1 -- 172.21.15.111:0/2542609221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292260 msgr2=0x55c3a728e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 1 --2- 172.21.15.111:0/2542609221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292260 0x55c3a728e780 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x55c3a72935d0 tx=0x7f6a60005d30 comp rx=0 tx=0).stop 2024-09-15T23:11:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 1 -- 172.21.15.111:0/2542609221 shutdown_connections 2024-09-15T23:11:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 1 --2- 172.21.15.111:0/2542609221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c3a7297c30 0x55c3a728ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 1 --2- 172.21.15.111:0/2542609221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c3a7292ba0 0x55c3a72a3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 1 --2- 172.21.15.111:0/2542609221 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292260 0x55c3a728e780 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 1 -- 172.21.15.111:0/2542609221 >> 172.21.15.111:0/2542609221 conn(0x55c3a728d8d0 msgr2=0x55c3a7293d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 1 -- 172.21.15.111:0/2542609221 shutdown_connections 2024-09-15T23:11:56.043 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 1 -- 172.21.15.111:0/2542609221 wait complete. 2024-09-15T23:11:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 1 -- start start 2024-09-15T23:11:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 1 Processor -- start 2024-09-15T23:11:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c3a7292260 0x55c3a72a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292ba0 0x55c3a72a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c3a7297c30 0x55c3a72a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f6a701a9650 con 0x55c3a7292ba0 2024-09-15T23:11:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6a701aaac0 con 0x55c3a7292260 2024-09-15T23:11:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6a701a9d20 con 0x55c3a7297c30 2024-09-15T23:11:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 10 client.0 _ll_register_callbacks cb 0x7f6a58009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T23:11:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a67e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292ba0 0x55c3a72a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a67e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292ba0 0x55c3a72a0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:60402/0 (socket says 172.21.15.111:60402) 2024-09-15T23:11:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a67e00640 1 -- 172.21.15.111:0/726083336 learned_addr learned my addr 172.21.15.111:0/726083336 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:11:56.044 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a6de00640 1 --2- 172.21.15.111:0/726083336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c3a7292260 0x55c3a72a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a6e800640 1 --2- 172.21.15.111:0/726083336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c3a7297c30 0x55c3a72a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a67e00640 1 -- 172.21.15.111:0/726083336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c3a7297c30 msgr2=0x55c3a72a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a67e00640 1 --2- 172.21.15.111:0/726083336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c3a7297c30 0x55c3a72a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a67e00640 1 -- 172.21.15.111:0/726083336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c3a7292260 msgr2=0x55c3a72a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T23:11:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a67e00640 1 --2- 172.21.15.111:0/726083336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c3a7292260 0x55c3a72a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a67e00640 1 -- 172.21.15.111:0/726083336 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3a72a2500 con 0x55c3a7292ba0 2024-09-15T23:11:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a67e00640 1 --2- 172.21.15.111:0/726083336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292ba0 0x55c3a72a0ee0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6a58000c00 tx=0x7f6a58009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4210ce9b3296542b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:11:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a53400640 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2024-09-15T23:11:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a53400640 1 -- 172.21.15.111:0/726083336 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a580bc1e0 con 0x55c3a7292ba0 2024-09-15T23:11:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a53400640 1 -- 172.21.15.111:0/726083336 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f6a5809b720 con 0x55c3a7292ba0 2024-09-15T23:11:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 1 -- 172.21.15.111:0/726083336 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a6002a070 con 0x55c3a7292ba0 2024-09-15T23:11:56.045 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a53400640 1 -- 172.21.15.111:0/726083336 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6a5809c2d0 con 0x55c3a7292ba0 2024-09-15T23:11:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 10 client.17013 fetch_fsmap learned FSMap version 37 2024-09-15T23:11:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 1 -- 172.21.15.111:0/726083336 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c3a72a3880 con 0x55c3a7292ba0 2024-09-15T23:11:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a53400640 1 -- 172.21.15.111:0/726083336 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a6002a070 con 0x55c3a7292ba0 2024-09-15T23:11:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a53400640 1 -- 172.21.15.111:0/726083336 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 37) ==== 8358+0+0 (secure 0 0 0) 0x7f6a58098020 con 0x55c3a7292ba0 2024-09-15T23:11:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 10 client.17013 fetch_fsmap finished waiting for FSMap version 37 2024-09-15T23:11:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 10 client.17013 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-15T23:11:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 20 client.17013 populate_metadata read hostname 'smithi111' 2024-09-15T23:11:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 1 --2- 172.21.15.111:0/726083336 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c3a7312fc0 0x55c3a73153e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:11:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 4 client.17013 mds_command: new command op to 24545 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-15T23:11:56.046 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.037+0000 7f6a77808740 1 -- 172.21.15.111:0/726083336 --> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c3a71feb80 con 0x55c3a7312fc0 2024-09-15T23:11:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.073+0000 7f6a6de00640 1 --2- 172.21.15.111:0/726083336 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c3a7312fc0 0x55c3a73153e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:11:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.142+0000 7f6a6de00640 1 --2- 172.21.15.111:0/726083336 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c3a7312fc0 0x55c3a73153e0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dac4b769e6bbb388 server_cookie=a5aa2b52f75eb61d in_seq=0 out_seq=0 2024-09-15T23:11:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.142+0000 7f6a53400640 10 client.17013 ms_handle_connect on v2:172.21.15.121:6838/1797352097 2024-09-15T23:11:56.215 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.214+0000 7f6a53400640 1 -- 172.21.15.111:0/726083336 <== mds.0 v2:172.21.15.121:6838/1797352097 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c3a71feb80 con 0x55c3a7312fc0 2024-09-15T23:11:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.214+0000 7f6a53400640 10 client.17013 handle_command_reply: tid=0 2024-09-15T23:11:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.214+0000 7f6a77808740 1 client.17013 shutdown 2024-09-15T23:11:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.214+0000 7f6a77808740 1 -- 172.21.15.111:0/726083336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292ba0 msgr2=0x55c3a72a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.214+0000 7f6a77808740 1 --2- 172.21.15.111:0/726083336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292ba0 0x55c3a72a0ee0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f6a58000c00 tx=0x7f6a58009700 comp rx=0 tx=0).stop 2024-09-15T23:11:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.214+0000 7f6a77808740 1 -- 172.21.15.111:0/726083336 shutdown_connections 2024-09-15T23:11:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.214+0000 7f6a77808740 1 --2- 172.21.15.111:0/726083336 >> [v2:172.21.15.121:6838/1797352097,v1:172.21.15.121:6839/1797352097] conn(0x55c3a7312fc0 0x55c3a73153e0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.214+0000 7f6a77808740 1 --2- 172.21.15.111:0/726083336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c3a7297c30 0x55c3a72a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.214+0000 7f6a53400640 0 client.17013 ms_handle_reset on v2:172.21.15.121:6838/1797352097 2024-09-15T23:11:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.214+0000 7f6a77808740 1 --2- 172.21.15.111:0/726083336 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55c3a7292ba0 0x55c3a72a0ee0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.214+0000 7f6a77808740 1 --2- 172.21.15.111:0/726083336 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c3a7292260 0x55c3a72a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.214+0000 7f6a77808740 1 -- 172.21.15.111:0/726083336 >> 172.21.15.111:0/726083336 conn(0x55c3a728d8d0 msgr2=0x55c3a728c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.214+0000 7f6a77808740 1 -- 172.21.15.111:0/726083336 shutdown_connections 2024-09-15T23:11:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.214+0000 7f6a77808740 1 -- 172.21.15.111:0/726083336 wait complete. 2024-09-15T23:11:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.214+0000 7f6a77808740 20 client.17013 trim_cache size 0 max 16384 2024-09-15T23:11:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:{ 2024-09-15T23:11:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "message": "", 2024-09-15T23:11:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout: "return_code": 0 2024-09-15T23:11:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stdout:} 2024-09-15T23:11:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.214+0000 7f6a6f200640 1 -- 172.21.15.111:0/1496907228 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6a540a32c0 msgr2=0x7f6a540a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.214+0000 7f6a6f200640 1 --2- 172.21.15.111:0/1496907228 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6a540a32c0 0x7f6a540a5780 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6a68004af0 tx=0x7f6a68004a40 comp rx=0 tx=0).stop 2024-09-15T23:11:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.214+0000 7f6a6f200640 1 -- 172.21.15.111:0/1496907228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a70104cf0 msgr2=0x7f6a70100370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:11:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.214+0000 7f6a6f200640 1 --2- 172.21.15.111:0/1496907228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a70104cf0 0x7f6a70100370 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6a5800e9c0 tx=0x7f6a5800ee80 comp rx=0 tx=0).stop 2024-09-15T23:11:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.214+0000 7f6a6f200640 1 -- 172.21.15.111:0/1496907228 shutdown_connections 2024-09-15T23:11:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.214+0000 7f6a6f200640 1 --2- 172.21.15.111:0/1496907228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6a70107620 0x7f6a701008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.214+0000 7f6a6f200640 1 --2- 172.21.15.111:0/1496907228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6a70104cf0 0x7f6a70100370 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.214+0000 7f6a6f200640 1 --2- 172.21.15.111:0/1496907228 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f6a540a32c0 0x7f6a540a5780 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.214+0000 7f6a6f200640 1 --2- 172.21.15.111:0/1496907228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6a701023c0 0x7f6a70101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:11:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.214+0000 7f6a6f200640 1 -- 172.21.15.111:0/1496907228 >> 172.21.15.111:0/1496907228 conn(0x7f6a700fbfa0 msgr2=0x7f6a70102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:11:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.214+0000 7f6a6f200640 1 -- 172.21.15.111:0/1496907228 shutdown_connections 2024-09-15T23:11:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi111.front.sepia.ceph.com.smithi111.stderr:2024-09-15T23:11:56.214+0000 7f6a6f200640 1 -- 172.21.15.111:0/1496907228 wait complete. 2024-09-15T23:11:58.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:11:58 smithi121 ceph-mon[34815]: pgmap v1132: 97 pgs: 97 active+clean; 409 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 4.6 MiB/s wr, 196 op/s 2024-09-15T23:11:58.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:11:58 smithi111 ceph-mon[30629]: pgmap v1132: 97 pgs: 97 active+clean; 409 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 4.6 MiB/s wr, 196 op/s 2024-09-15T23:11:58.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:11:58 smithi192 ceph-mon[35427]: pgmap v1132: 97 pgs: 97 active+clean; 409 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 4.6 MiB/s wr, 196 op/s 2024-09-15T23:12:00.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:00 smithi111 ceph-mon[30629]: pgmap v1133: 97 pgs: 97 active+clean; 397 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 5.3 MiB/s wr, 223 op/s 2024-09-15T23:12:00.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:00 smithi121 ceph-mon[34815]: pgmap v1133: 97 pgs: 97 active+clean; 397 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 5.3 MiB/s wr, 223 op/s 2024-09-15T23:12:00.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:00 smithi192 ceph-mon[35427]: pgmap v1133: 97 pgs: 97 active+clean; 397 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 5.3 MiB/s wr, 223 op/s 2024-09-15T23:12:02.183 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:02 smithi111 ceph-mon[30629]: pgmap v1134: 97 pgs: 97 active+clean; 406 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 6.6 MiB/s wr, 227 op/s 2024-09-15T23:12:02.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:02 smithi121 ceph-mon[34815]: pgmap v1134: 97 pgs: 97 active+clean; 406 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 6.6 MiB/s wr, 227 op/s 2024-09-15T23:12:02.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:02 smithi192 ceph-mon[35427]: pgmap v1134: 97 pgs: 97 active+clean; 406 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 6.6 MiB/s wr, 227 op/s 2024-09-15T23:12:04.183 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:04 smithi111 ceph-mon[30629]: pgmap v1135: 97 pgs: 97 active+clean; 420 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 7.1 MiB/s wr, 219 op/s 2024-09-15T23:12:04.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:04 smithi121 ceph-mon[34815]: pgmap v1135: 97 pgs: 97 active+clean; 420 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 7.1 MiB/s wr, 219 op/s 2024-09-15T23:12:04.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:04 smithi192 ceph-mon[35427]: pgmap v1135: 97 pgs: 97 active+clean; 420 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 7.1 MiB/s wr, 219 op/s 2024-09-15T23:12:06.203 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:06 smithi111 ceph-mon[30629]: pgmap v1136: 97 pgs: 97 active+clean; 417 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 7.0 MiB/s wr, 209 op/s 2024-09-15T23:12:06.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:06 smithi121 ceph-mon[34815]: pgmap v1136: 97 pgs: 97 active+clean; 417 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 7.0 MiB/s wr, 209 op/s 2024-09-15T23:12:06.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:06 smithi192 ceph-mon[35427]: pgmap v1136: 97 pgs: 97 active+clean; 417 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 7.0 MiB/s wr, 209 op/s 2024-09-15T23:12:08.289 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:08 smithi111 ceph-mon[30629]: pgmap v1137: 97 pgs: 97 active+clean; 416 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 6.9 MiB/s wr, 213 op/s 2024-09-15T23:12:08.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:08 smithi121 ceph-mon[34815]: pgmap v1137: 97 pgs: 97 active+clean; 416 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 6.9 MiB/s wr, 213 op/s 2024-09-15T23:12:08.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:08 smithi192 ceph-mon[35427]: pgmap v1137: 97 pgs: 97 active+clean; 416 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 6.9 MiB/s wr, 213 op/s 2024-09-15T23:12:08.807 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-15T23:12:08.808 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-15T23:12:08.819 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi111.front.sepia.ceph.com 2024-09-15T23:12:08.862 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-15T23:12:08.941 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T23:12:09.105 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.104+0000 7f79dca00640 1 -- 172.21.15.111:0/2264875833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d8104d10 msgr2=0x7f79d8107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:09.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.104+0000 7f79dca00640 1 --2- 172.21.15.111:0/2264875833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d8104d10 0x7f79d8107100 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009900 tx=0x7f79c402a3f0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.104+0000 7f79dca00640 1 -- 172.21.15.111:0/2264875833 shutdown_connections 2024-09-15T23:12:09.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.104+0000 7f79dca00640 1 --2- 172.21.15.111:0/2264875833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79d8107640 0x7f79d8109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.104+0000 7f79dca00640 1 --2- 172.21.15.111:0/2264875833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d8104d10 0x7f79d8107100 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.104+0000 7f79dca00640 1 --2- 172.21.15.111:0/2264875833 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79d81023e0 0x7f79d81047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.104+0000 7f79dca00640 1 -- 172.21.15.111:0/2264875833 >> 172.21.15.111:0/2264875833 conn(0x7f79d80fc000 msgr2=0x7f79d80fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:12:09.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.104+0000 7f79dca00640 1 -- 172.21.15.111:0/2264875833 shutdown_connections 2024-09-15T23:12:09.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.104+0000 7f79dca00640 1 -- 172.21.15.111:0/2264875833 wait complete. 2024-09-15T23:12:09.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.105+0000 7f79dca00640 1 Processor -- start 2024-09-15T23:12:09.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.105+0000 7f79dca00640 1 -- start start 2024-09-15T23:12:09.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.105+0000 7f79dca00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d81023e0 0x7f79d8199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:09.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.105+0000 7f79dca00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79d8104d10 0x7f79d819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:09.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.105+0000 7f79dca00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79d8107640 0x7f79d81a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:09.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.105+0000 7f79dca00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f79d810d670 con 0x7f79d8107640 2024-09-15T23:12:09.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.105+0000 7f79dca00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f79d810d4f0 con 0x7f79d8104d10 2024-09-15T23:12:09.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.105+0000 7f79dca00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f79d810d7f0 con 0x7f79d81023e0 2024-09-15T23:12:09.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.106+0000 7f79d6800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d81023e0 0x7f79d8199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:09.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.106+0000 7f79d7200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79d8107640 0x7f79d81a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:09.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.106+0000 7f79d6800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d81023e0 0x7f79d8199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:34200/0 (socket says 172.21.15.111:34200) 2024-09-15T23:12:09.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.106+0000 7f79d6800640 1 -- 172.21.15.111:0/411521090 learned_addr learned my addr 172.21.15.111:0/411521090 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:12:09.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.106+0000 7f79d5e00640 1 --2- 172.21.15.111:0/411521090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79d8104d10 0x7f79d819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:09.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.106+0000 7f79d6800640 1 -- 172.21.15.111:0/411521090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79d8104d10 msgr2=0x7f79d819a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:09.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.106+0000 7f79d6800640 1 --2- 172.21.15.111:0/411521090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79d8104d10 0x7f79d819a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.106+0000 7f79d6800640 1 -- 172.21.15.111:0/411521090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79d8107640 msgr2=0x7f79d81a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:09.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.106+0000 7f79d6800640 1 --2- 172.21.15.111:0/411521090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79d8107640 0x7f79d81a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.106+0000 7f79d6800640 1 -- 172.21.15.111:0/411521090 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d81a1b50 con 0x7f79d81023e0 2024-09-15T23:12:09.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.106+0000 7f79d5e00640 1 --2- 172.21.15.111:0/411521090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79d8104d10 0x7f79d819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:12:09.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.106+0000 7f79d6800640 1 --2- 172.21.15.111:0/411521090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d81023e0 0x7f79d8199f30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f79c800e9f0 tx=0x7f79c800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:12:09.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.107+0000 7f79cf400640 1 -- 172.21.15.111:0/411521090 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c800cd80 con 0x7f79d81023e0 2024-09-15T23:12:09.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.107+0000 7f79cf400640 1 -- 172.21.15.111:0/411521090 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79c8010910 con 0x7f79d81023e0 2024-09-15T23:12:09.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.107+0000 7f79dca00640 1 -- 172.21.15.111:0/411521090 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d81a1e40 con 0x7f79d81023e0 2024-09-15T23:12:09.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.107+0000 7f79cf400640 1 -- 172.21.15.111:0/411521090 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c8018990 con 0x7f79d81023e0 2024-09-15T23:12:09.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.107+0000 7f79dca00640 1 -- 172.21.15.111:0/411521090 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d81a22a0 con 0x7f79d81023e0 2024-09-15T23:12:09.110 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.108+0000 7f79dca00640 1 -- 172.21.15.111:0/411521090 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79d81091b0 con 0x7f79d81023e0 2024-09-15T23:12:09.111 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.110+0000 7f79cf400640 1 -- 172.21.15.111:0/411521090 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f79c8010ac0 con 0x7f79d81023e0 2024-09-15T23:12:09.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.112+0000 7f79cf400640 1 --2- 172.21.15.111:0/411521090 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f79bc0a3250 0x7f79bc0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:09.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.112+0000 7f79cf400640 1 -- 172.21.15.111:0/411521090 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f79c80cfcb0 con 0x7f79d81023e0 2024-09-15T23:12:09.113 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.112+0000 7f79d5e00640 1 --2- 172.21.15.111:0/411521090 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f79bc0a3250 0x7f79bc0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:09.114 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.113+0000 7f79d5e00640 1 --2- 172.21.15.111:0/411521090 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f79bc0a3250 0x7f79bc0a5710 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f79c402a9e0 tx=0x7f79c4000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:12:09.117 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.115+0000 7f79cf400640 1 -- 172.21.15.111:0/411521090 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f79c809c9d0 con 0x7f79d81023e0 2024-09-15T23:12:09.181 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:09 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:12:09.181 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:09 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:12:09.181 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:09 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:12:09.182 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:09 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:12:09.326 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.324+0000 7f79dca00640 1 -- 172.21.15.111:0/411521090 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79d8062ef0 con 0x7f79d81023e0 2024-09-15T23:12:09.327 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.326+0000 7f79cf400640 1 -- 172.21.15.111:0/411521090 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22701 (secure 0 0 0) 0x7f79c809c2d0 con 0x7f79d81023e0 2024-09-15T23:12:09.328 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T23:12:09.328 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":89,"fsid":"b7614d3c-73b2-11ef-bceb-c7b262605968","created":"2024-09-15T22:35:52.571473+0000","modified":"2024-09-15T23:01:53.289289+0000","last_up_change":"2024-09-15T22:43:19.567274+0000","last_in_change":"2024-09-15T22:42:56.994246+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:39:18.954207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:44:12.846418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:44:15.685946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04f39604-be0e-4261-b316-67fa087c6666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6803","nonce":1386823446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6805","nonce":1386823446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6809","nonce":1386823446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6807","nonce":1386823446}]},"public_addr":"172.21.15.111:6803/1386823446","cluster_addr":"172.21.15.111:6805/1386823446","heartbeat_back_addr":"172.21.15.111:6809/1386823446","heartbeat_front_addr":"172.21.15.111:6807/1386823446","state":["exists","up"]},{"osd":1,"uuid":"1bdbef6e-23fa-4618-a788-66e57741a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6801","nonce":2051512291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6803","nonce":2051512291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6807","nonce":2051512291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6805","nonce":2051512291}]},"public_addr":"172.21.15.121:6801/2051512291","cluster_addr":"172.21.15.121:6803/2051512291","heartbeat_back_addr":"172.21.15.121:6807/2051512291","heartbeat_front_addr":"172.21.15.121:6805/2051512291","state":["exists","up"]},{"osd":2,"uuid":"46735ccf-6d0e-4154-bb55-422929819bfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6801","nonce":93085639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6803","nonce":93085639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6807","nonce":93085639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6805","nonce":93085639}]},"public_addr":"172.21.15.192:6801/93085639","cluster_addr":"172.21.15.192:6803/93085639","heartbeat_back_addr":"172.21.15.192:6807/93085639","heartbeat_front_addr":"172.21.15.192:6805/93085639","state":["exists","up"]},{"osd":3,"uuid":"4b243af8-c958-40b1-ba24-9c7ace00c4a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6811","nonce":2578259041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6813","nonce":2578259041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6817","nonce":2578259041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6815","nonce":2578259041}]},"public_addr":"172.21.15.111:6811/2578259041","cluster_addr":"172.21.15.111:6813/2578259041","heartbeat_back_addr":"172.21.15.111:6817/2578259041","heartbeat_front_addr":"172.21.15.111:6815/2578259041","state":["exists","up"]},{"osd":4,"uuid":"49f27541-9cbf-48e0-a466-6bb0fa5410b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6809","nonce":2667595915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6811","nonce":2667595915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6815","nonce":2667595915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6813","nonce":2667595915}]},"public_addr":"172.21.15.121:6809/2667595915","cluster_addr":"172.21.15.121:6811/2667595915","heartbeat_back_addr":"172.21.15.121:6815/2667595915","heartbeat_front_addr":"172.21.15.121:6813/2667595915","state":["exists","up"]},{"osd":5,"uuid":"9290e769-5470-44d3-8d24-25093af34335","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6809","nonce":1958639856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6811","nonce":1958639856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6815","nonce":1958639856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6813","nonce":1958639856}]},"public_addr":"172.21.15.192:6809/1958639856","cluster_addr":"172.21.15.192:6811/1958639856","heartbeat_back_addr":"172.21.15.192:6815/1958639856","heartbeat_front_addr":"172.21.15.192:6813/1958639856","state":["exists","up"]},{"osd":6,"uuid":"18911b97-1c68-4e8f-92ec-35d2ff0b5f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6819","nonce":1371543706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6821","nonce":1371543706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6825","nonce":1371543706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6823","nonce":1371543706}]},"public_addr":"172.21.15.111:6819/1371543706","cluster_addr":"172.21.15.111:6821/1371543706","heartbeat_back_addr":"172.21.15.111:6825/1371543706","heartbeat_front_addr":"172.21.15.111:6823/1371543706","state":["exists","up"]},{"osd":7,"uuid":"470647c5-c00d-426b-a134-cca235902307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6817","nonce":3501842998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6819","nonce":3501842998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6823","nonce":3501842998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6821","nonce":3501842998}]},"public_addr":"172.21.15.121:6817/3501842998","cluster_addr":"172.21.15.121:6819/3501842998","heartbeat_back_addr":"172.21.15.121:6823/3501842998","heartbeat_front_addr":"172.21.15.121:6821/3501842998","state":["exists","up"]},{"osd":8,"uuid":"fa2d3dd5-a768-4ada-a036-4c596b76d654","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6817","nonce":2417771374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6819","nonce":2417771374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6823","nonce":2417771374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6821","nonce":2417771374}]},"public_addr":"172.21.15.192:6817/2417771374","cluster_addr":"172.21.15.192:6819/2417771374","heartbeat_back_addr":"172.21.15.192:6823/2417771374","heartbeat_front_addr":"172.21.15.192:6821/2417771374","state":["exists","up"]},{"osd":9,"uuid":"71b4f5fb-1c02-461b-a257-c6e48381d768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6827","nonce":3979119431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6829","nonce":3979119431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6833","nonce":3979119431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6831","nonce":3979119431}]},"public_addr":"172.21.15.111:6827/3979119431","cluster_addr":"172.21.15.111:6829/3979119431","heartbeat_back_addr":"172.21.15.111:6833/3979119431","heartbeat_front_addr":"172.21.15.111:6831/3979119431","state":["exists","up"]},{"osd":10,"uuid":"e6a39e77-fce3-4e22-bf66-a677c9f8b6bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6825","nonce":2897902637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6827","nonce":2897902637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6831","nonce":2897902637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6829","nonce":2897902637}]},"public_addr":"172.21.15.121:6825/2897902637","cluster_addr":"172.21.15.121:6827/2897902637","heartbeat_back_addr":"172.21.15.121:6831/2897902637","heartbeat_front_addr":"172.21.15.121:6829/2897902637","state":["exists","up"]},{"osd":11,"uuid":"411a7921-6717-4cef-b8df-d9e252edcc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6825","nonce":2431464812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6827","nonce":2431464812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6831","nonce":2431464812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6829","nonce":2431464812}]},"public_addr":"172.21.15.192:6825/2431464812","cluster_addr":"172.21.15.192:6827/2431464812","heartbeat_back_addr":"172.21.15.192:6831/2431464812","heartbeat_front_addr":"172.21.15.192:6829/2431464812","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:38:23.442715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:38:49.501124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:39:14.759077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:39:45.121397+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:40:11.137612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:40:36.278567+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:07.027190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:32.439215+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:57.058702+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:42:28.741846+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:42:53.395442+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:43:18.265067+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:6801/2107775445":"2024-09-16T22:36:50.357551+0000","172.21.15.111:6800/2107775445":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/2394746540":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/4140020843":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/2114354263":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/1054173642":"2024-09-16T22:36:22.652875+0000","172.21.15.111:0/3408238004":"2024-09-16T22:36:22.652875+0000","172.21.15.111:0/2879044772":"2024-09-16T22:36:22.652875+0000","172.21.15.111:6801/3350425104":"2024-09-16T22:36:22.652875+0000","172.21.15.111:6800/3350425104":"2024-09-16T22:36:22.652875+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:12:09.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.328+0000 7f79dca00640 1 -- 172.21.15.111:0/411521090 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f79bc0a3250 msgr2=0x7f79bc0a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:09.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.329+0000 7f79dca00640 1 --2- 172.21.15.111:0/411521090 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f79bc0a3250 0x7f79bc0a5710 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f79c402a9e0 tx=0x7f79c4000bc0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.329+0000 7f79dca00640 1 -- 172.21.15.111:0/411521090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d81023e0 msgr2=0x7f79d8199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:09.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.329+0000 7f79dca00640 1 --2- 172.21.15.111:0/411521090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d81023e0 0x7f79d8199f30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f79c800e9f0 tx=0x7f79c800eeb0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.329+0000 7f79dca00640 1 -- 172.21.15.111:0/411521090 shutdown_connections 2024-09-15T23:12:09.330 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.329+0000 7f79dca00640 1 --2- 172.21.15.111:0/411521090 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f79d8107640 0x7f79d81a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.329+0000 7f79dca00640 1 --2- 172.21.15.111:0/411521090 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f79bc0a3250 0x7f79bc0a5710 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.329+0000 7f79dca00640 1 --2- 172.21.15.111:0/411521090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f79d8104d10 0x7f79d819a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.329+0000 7f79dca00640 1 --2- 172.21.15.111:0/411521090 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d81023e0 0x7f79d8199f30 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.329+0000 7f79dca00640 1 -- 172.21.15.111:0/411521090 >> 172.21.15.111:0/411521090 conn(0x7f79d80fc000 msgr2=0x7f79d8101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:12:09.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.329+0000 7f79dca00640 1 -- 172.21.15.111:0/411521090 shutdown_connections 2024-09-15T23:12:09.331 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.329+0000 7f79dca00640 1 -- 172.21.15.111:0/411521090 wait complete. 2024-09-15T23:12:09.347 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T22:39:18.954207+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-15T22:44:12.846418+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-15T22:44:15.685946+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-15T23:12:09.347 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-15T23:12:09.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:09 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:12:09.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:09 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:12:09.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:09 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:12:09.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:09 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:12:09.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.506+0000 7efcc3e00640 1 -- 172.21.15.111:0/592498729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcbc102cf0 msgr2=0x7efcbc103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:09.508 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.506+0000 7efcc3e00640 1 --2- 172.21.15.111:0/592498729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcbc102cf0 0x7efcbc103170 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7efca8009900 tx=0x7efca802a3f0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.507+0000 7efcc3e00640 1 -- 172.21.15.111:0/592498729 shutdown_connections 2024-09-15T23:12:09.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.507+0000 7efcc3e00640 1 --2- 172.21.15.111:0/592498729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcbc1036b0 0x7efcbc109f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.507+0000 7efcc3e00640 1 --2- 172.21.15.111:0/592498729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcbc102cf0 0x7efcbc103170 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.507+0000 7efcc3e00640 1 --2- 172.21.15.111:0/592498729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcbc101b00 0x7efcbc101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.507+0000 7efcc3e00640 1 -- 172.21.15.111:0/592498729 >> 172.21.15.111:0/592498729 conn(0x7efcbc0fd2a0 msgr2=0x7efcbc0ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:12:09.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.507+0000 7efcc3e00640 1 -- 172.21.15.111:0/592498729 shutdown_connections 2024-09-15T23:12:09.509 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.507+0000 7efcc3e00640 1 -- 172.21.15.111:0/592498729 wait complete. 2024-09-15T23:12:09.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.508+0000 7efcc3e00640 1 Processor -- start 2024-09-15T23:12:09.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.508+0000 7efcc3e00640 1 -- start start 2024-09-15T23:12:09.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.508+0000 7efcc3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcbc101b00 0x7efcbc075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:09.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.508+0000 7efcc3e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcbc102cf0 0x7efcbc0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:09.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.508+0000 7efcc3e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcbc1036b0 0x7efcbc074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:09.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.508+0000 7efcc3e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7efcbc10d870 con 0x7efcbc1036b0 2024-09-15T23:12:09.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.508+0000 7efcc3e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efcbc10d6f0 con 0x7efcbc101b00 2024-09-15T23:12:09.511 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.508+0000 7efcc3e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efcbc10d9f0 con 0x7efcbc102cf0 2024-09-15T23:12:09.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.509+0000 7efcc1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcbc1036b0 0x7efcbc074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:09.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.509+0000 7efcc0a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcbc102cf0 0x7efcbc0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:09.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.509+0000 7efcc1400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcbc101b00 0x7efcbc075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:09.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.509+0000 7efcc1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcbc1036b0 0x7efcbc074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53866/0 (socket says 172.21.15.111:53866) 2024-09-15T23:12:09.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.509+0000 7efcc1e00640 1 -- 172.21.15.111:0/961467451 learned_addr learned my addr 172.21.15.111:0/961467451 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:12:09.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.509+0000 7efcc1e00640 1 -- 172.21.15.111:0/961467451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcbc102cf0 msgr2=0x7efcbc0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:09.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.509+0000 7efcc1e00640 1 --2- 172.21.15.111:0/961467451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcbc102cf0 0x7efcbc0745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.509+0000 7efcc1e00640 1 -- 172.21.15.111:0/961467451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcbc101b00 msgr2=0x7efcbc075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:09.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.509+0000 7efcc1e00640 1 --2- 172.21.15.111:0/961467451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcbc101b00 0x7efcbc075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.509+0000 7efcc1e00640 1 -- 172.21.15.111:0/961467451 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcbc0753b0 con 0x7efcbc1036b0 2024-09-15T23:12:09.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.509+0000 7efcc0a00640 1 --2- 172.21.15.111:0/961467451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcbc102cf0 0x7efcbc0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:12:09.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.509+0000 7efcc1e00640 1 --2- 172.21.15.111:0/961467451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcbc1036b0 0x7efcbc074b20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7efcb400eb10 tx=0x7efcb400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:12:09.512 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.510+0000 7efcba000640 1 -- 172.21.15.111:0/961467451 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb400ce20 con 0x7efcbc1036b0 2024-09-15T23:12:09.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.510+0000 7efcba000640 1 -- 172.21.15.111:0/961467451 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb4005020 con 0x7efcbc1036b0 2024-09-15T23:12:09.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.510+0000 7efcc3e00640 1 -- 172.21.15.111:0/961467451 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcbc1a1fe0 con 0x7efcbc1036b0 2024-09-15T23:12:09.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.510+0000 7efcba000640 1 -- 172.21.15.111:0/961467451 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb4010690 con 0x7efcbc1036b0 2024-09-15T23:12:09.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.510+0000 7efcc3e00640 1 -- 172.21.15.111:0/961467451 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcbc1a2570 con 0x7efcbc1036b0 2024-09-15T23:12:09.513 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.511+0000 7efcc3e00640 1 -- 172.21.15.111:0/961467451 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcbc108f40 con 0x7efcbc1036b0 2024-09-15T23:12:09.514 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.513+0000 7efcba000640 1 -- 172.21.15.111:0/961467451 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7efcb4005540 con 0x7efcbc1036b0 2024-09-15T23:12:09.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.514+0000 7efcba000640 1 --2- 172.21.15.111:0/961467451 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7efca40a32d0 0x7efca40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:09.515 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.514+0000 7efcba000640 1 -- 172.21.15.111:0/961467451 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7efcb40cffe0 con 0x7efcbc1036b0 2024-09-15T23:12:09.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.514+0000 7efcc1400640 1 --2- 172.21.15.111:0/961467451 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7efca40a32d0 0x7efca40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:09.516 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.515+0000 7efcc1400640 1 --2- 172.21.15.111:0/961467451 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7efca40a32d0 0x7efca40a5790 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7efcbc102b50 tx=0x7efcb0009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:12:09.519 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.518+0000 7efcba000640 1 -- 172.21.15.111:0/961467451 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efcb409cd00 con 0x7efcbc1036b0 2024-09-15T23:12:09.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:09 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T23:12:09.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:09 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T23:12:09.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:09 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T23:12:09.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:09 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' 2024-09-15T23:12:09.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.705+0000 7efcc3e00640 1 -- 172.21.15.111:0/961467451 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efcbc062df0 con 0x7efcbc1036b0 2024-09-15T23:12:09.707 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.706+0000 7efcba000640 1 -- 172.21.15.111:0/961467451 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7efcb409c600 con 0x7efcbc1036b0 2024-09-15T23:12:09.708 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2024-09-15T23:12:09.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.708+0000 7efcc3e00640 1 -- 172.21.15.111:0/961467451 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7efca40a32d0 msgr2=0x7efca40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:09.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.708+0000 7efcc3e00640 1 --2- 172.21.15.111:0/961467451 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7efca40a32d0 0x7efca40a5790 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7efcbc102b50 tx=0x7efcb0009110 comp rx=0 tx=0).stop 2024-09-15T23:12:09.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.708+0000 7efcc3e00640 1 -- 172.21.15.111:0/961467451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcbc1036b0 msgr2=0x7efcbc074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:09.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.708+0000 7efcc3e00640 1 --2- 172.21.15.111:0/961467451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcbc1036b0 0x7efcbc074b20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7efcb400eb10 tx=0x7efcb400ef10 comp rx=0 tx=0).stop 2024-09-15T23:12:09.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.708+0000 7efcc3e00640 1 -- 172.21.15.111:0/961467451 shutdown_connections 2024-09-15T23:12:09.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.708+0000 7efcc3e00640 1 --2- 172.21.15.111:0/961467451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efcbc1036b0 0x7efcbc074b20 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.708+0000 7efcc3e00640 1 --2- 172.21.15.111:0/961467451 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcbc102cf0 0x7efcbc0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.709+0000 7efcc3e00640 1 --2- 172.21.15.111:0/961467451 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7efca40a32d0 0x7efca40a5790 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.709+0000 7efcc3e00640 1 --2- 172.21.15.111:0/961467451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcbc101b00 0x7efcbc075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.709+0000 7efcc3e00640 1 -- 172.21.15.111:0/961467451 >> 172.21.15.111:0/961467451 conn(0x7efcbc0fd2a0 msgr2=0x7efcbc106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:12:09.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.709+0000 7efcc3e00640 1 -- 172.21.15.111:0/961467451 shutdown_connections 2024-09-15T23:12:09.711 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.709+0000 7efcc3e00640 1 -- 172.21.15.111:0/961467451 wait complete. 2024-09-15T23:12:09.733 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-15T23:12:09.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.891+0000 7f5e53200640 1 -- 172.21.15.111:0/2994571715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e541035b0 msgr2=0x7f5e54109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:09.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.891+0000 7f5e53200640 1 --2- 172.21.15.111:0/2994571715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e541035b0 0x7f5e54109e40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c009ca0 tx=0x7f5e4c02a8c0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.891+0000 7f5e53200640 1 -- 172.21.15.111:0/2994571715 shutdown_connections 2024-09-15T23:12:09.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.891+0000 7f5e53200640 1 --2- 172.21.15.111:0/2994571715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e541035b0 0x7f5e54109e40 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.891+0000 7f5e53200640 1 --2- 172.21.15.111:0/2994571715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e54102bf0 0x7f5e54103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.891+0000 7f5e53200640 1 --2- 172.21.15.111:0/2994571715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e54101a00 0x7f5e54101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.891+0000 7f5e53200640 1 -- 172.21.15.111:0/2994571715 >> 172.21.15.111:0/2994571715 conn(0x7f5e540fd180 msgr2=0x7f5e540ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:12:09.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.891+0000 7f5e53200640 1 -- 172.21.15.111:0/2994571715 shutdown_connections 2024-09-15T23:12:09.894 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.891+0000 7f5e53200640 1 -- 172.21.15.111:0/2994571715 wait complete. 2024-09-15T23:12:09.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.892+0000 7f5e53200640 1 Processor -- start 2024-09-15T23:12:09.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.892+0000 7f5e53200640 1 -- start start 2024-09-15T23:12:09.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.892+0000 7f5e53200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e54101a00 0x7f5e54199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:09.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.892+0000 7f5e53200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e54102bf0 0x7f5e5419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:09.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.892+0000 7f5e53200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e541035b0 0x7f5e541a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:09.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.892+0000 7f5e53200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5410d6d0 con 0x7f5e54102bf0 2024-09-15T23:12:09.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.893+0000 7f5e53200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5410d550 con 0x7f5e541035b0 2024-09-15T23:12:09.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.893+0000 7f5e53200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5410d850 con 0x7f5e54101a00 2024-09-15T23:12:09.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.893+0000 7f5e51400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e54102bf0 0x7f5e5419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:09.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.893+0000 7f5e51e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e54101a00 0x7f5e54199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:09.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.893+0000 7f5e51e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e54101a00 0x7f5e54199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.111:34230/0 (socket says 172.21.15.111:34230) 2024-09-15T23:12:09.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.893+0000 7f5e51e00640 1 -- 172.21.15.111:0/2656543955 learned_addr learned my addr 172.21.15.111:0/2656543955 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:12:09.896 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.893+0000 7f5e52800640 1 --2- 172.21.15.111:0/2656543955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e541035b0 0x7f5e541a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:09.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.893+0000 7f5e51e00640 1 -- 172.21.15.111:0/2656543955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e541035b0 msgr2=0x7f5e541a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:09.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.893+0000 7f5e51e00640 1 --2- 172.21.15.111:0/2656543955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e541035b0 0x7f5e541a1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.893+0000 7f5e51e00640 1 -- 172.21.15.111:0/2656543955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e54102bf0 msgr2=0x7f5e5419a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:09.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.893+0000 7f5e51e00640 1 --2- 172.21.15.111:0/2656543955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e54102bf0 0x7f5e5419a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:09.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.893+0000 7f5e51e00640 1 -- 172.21.15.111:0/2656543955 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e541a1ae0 con 0x7f5e54101a00 2024-09-15T23:12:09.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.893+0000 7f5e51400640 1 --2- 172.21.15.111:0/2656543955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e54102bf0 0x7f5e5419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:12:09.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.893+0000 7f5e51e00640 1 --2- 172.21.15.111:0/2656543955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e54101a00 0x7f5e54199ec0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5e4400e9c0 tx=0x7f5e4400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:12:09.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.894+0000 7f5e4aa00640 1 -- 172.21.15.111:0/2656543955 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4400cde0 con 0x7f5e54101a00 2024-09-15T23:12:09.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.894+0000 7f5e4aa00640 1 -- 172.21.15.111:0/2656543955 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e44010910 con 0x7f5e54101a00 2024-09-15T23:12:09.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.894+0000 7f5e53200640 1 -- 172.21.15.111:0/2656543955 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e541a1dd0 con 0x7f5e54101a00 2024-09-15T23:12:09.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.894+0000 7f5e4aa00640 1 -- 172.21.15.111:0/2656543955 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e44018a00 con 0x7f5e54101a00 2024-09-15T23:12:09.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.894+0000 7f5e53200640 1 -- 172.21.15.111:0/2656543955 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e541a2230 con 0x7f5e54101a00 2024-09-15T23:12:09.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.895+0000 7f5e53200640 1 -- 172.21.15.111:0/2656543955 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e541a2490 con 0x7f5e54101a00 2024-09-15T23:12:09.898 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.897+0000 7f5e4aa00640 1 -- 172.21.15.111:0/2656543955 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5e4401f020 con 0x7f5e54101a00 2024-09-15T23:12:09.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.898+0000 7f5e4aa00640 1 --2- 172.21.15.111:0/2656543955 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5e380a3250 0x7f5e380a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:09.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.898+0000 7f5e4aa00640 1 -- 172.21.15.111:0/2656543955 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f5e440cfea0 con 0x7f5e54101a00 2024-09-15T23:12:09.899 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.898+0000 7f5e51400640 1 --2- 172.21.15.111:0/2656543955 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5e380a3250 0x7f5e380a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:09.900 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.899+0000 7f5e51400640 1 --2- 172.21.15.111:0/2656543955 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5e380a3250 0x7f5e380a5710 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5e5419b3d0 tx=0x7f5e40005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:12:09.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:09.900+0000 7f5e4aa00640 1 -- 172.21.15.111:0/2656543955 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5e4409cbc0 con 0x7f5e54101a00 2024-09-15T23:12:10.101 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.099+0000 7f5e53200640 1 -- 172.21.15.111:0/2656543955 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5e541a2780 con 0x7f5e54101a00 2024-09-15T23:12:10.102 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.100+0000 7f5e4aa00640 1 -- 172.21.15.111:0/2656543955 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f5e4409c4c0 con 0x7f5e54101a00 2024-09-15T23:12:10.102 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2024-09-15T23:12:10.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.105+0000 7f5e53200640 1 -- 172.21.15.111:0/2656543955 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5e380a3250 msgr2=0x7f5e380a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:10.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.105+0000 7f5e53200640 1 --2- 172.21.15.111:0/2656543955 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5e380a3250 0x7f5e380a5710 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5e5419b3d0 tx=0x7f5e40005f90 comp rx=0 tx=0).stop 2024-09-15T23:12:10.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.105+0000 7f5e53200640 1 -- 172.21.15.111:0/2656543955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e54101a00 msgr2=0x7f5e54199ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:10.106 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.105+0000 7f5e53200640 1 --2- 172.21.15.111:0/2656543955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e54101a00 0x7f5e54199ec0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5e4400e9c0 tx=0x7f5e4400ee80 comp rx=0 tx=0).stop 2024-09-15T23:12:10.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.106+0000 7f5e53200640 1 -- 172.21.15.111:0/2656543955 shutdown_connections 2024-09-15T23:12:10.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.107+0000 7f5e53200640 1 --2- 172.21.15.111:0/2656543955 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5e380a3250 0x7f5e380a5710 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:10.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.107+0000 7f5e53200640 1 --2- 172.21.15.111:0/2656543955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e541035b0 0x7f5e541a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:10.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.107+0000 7f5e53200640 1 --2- 172.21.15.111:0/2656543955 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e54102bf0 0x7f5e5419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:10.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.107+0000 7f5e53200640 1 --2- 172.21.15.111:0/2656543955 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5e54101a00 0x7f5e54199ec0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:10.108 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.107+0000 7f5e53200640 1 -- 172.21.15.111:0/2656543955 >> 172.21.15.111:0/2656543955 conn(0x7f5e540fd180 msgr2=0x7f5e54074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:12:10.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.107+0000 7f5e53200640 1 -- 172.21.15.111:0/2656543955 shutdown_connections 2024-09-15T23:12:10.109 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.107+0000 7f5e53200640 1 -- 172.21.15.111:0/2656543955 wait complete. 2024-09-15T23:12:10.135 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-15T23:12:10.297 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.296+0000 7f1391400640 1 -- 172.21.15.111:0/585151060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f138c101a00 msgr2=0x7f138c101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:10.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.296+0000 7f1391400640 1 --2- 172.21.15.111:0/585151060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f138c101a00 0x7f138c101e00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f137c009900 tx=0x7f137c02a410 comp rx=0 tx=0).stop 2024-09-15T23:12:10.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.296+0000 7f1391400640 1 -- 172.21.15.111:0/585151060 shutdown_connections 2024-09-15T23:12:10.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.296+0000 7f1391400640 1 --2- 172.21.15.111:0/585151060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f138c1035b0 0x7f138c109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:10.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.296+0000 7f1391400640 1 --2- 172.21.15.111:0/585151060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f138c102bf0 0x7f138c103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:10.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.296+0000 7f1391400640 1 --2- 172.21.15.111:0/585151060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f138c101a00 0x7f138c101e00 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:10.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.296+0000 7f1391400640 1 -- 172.21.15.111:0/585151060 >> 172.21.15.111:0/585151060 conn(0x7f138c0fd180 msgr2=0x7f138c0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:12:10.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.296+0000 7f1391400640 1 -- 172.21.15.111:0/585151060 shutdown_connections 2024-09-15T23:12:10.298 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.296+0000 7f1391400640 1 -- 172.21.15.111:0/585151060 wait complete. 2024-09-15T23:12:10.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.297+0000 7f1391400640 1 Processor -- start 2024-09-15T23:12:10.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.297+0000 7f1391400640 1 -- start start 2024-09-15T23:12:10.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.297+0000 7f1391400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f138c101a00 0x7f138c199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:10.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.297+0000 7f1391400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f138c102bf0 0x7f138c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:10.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.297+0000 7f1391400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f138c1035b0 0x7f138c1a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:10.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.297+0000 7f1391400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f138c10d6d0 con 0x7f138c1035b0 2024-09-15T23:12:10.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.297+0000 7f1391400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f138c10d550 con 0x7f138c102bf0 2024-09-15T23:12:10.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.297+0000 7f1391400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f138c10d850 con 0x7f138c101a00 2024-09-15T23:12:10.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.297+0000 7f1390a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f138c1035b0 0x7f138c1a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:10.300 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.297+0000 7f138b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f138c102bf0 0x7f138c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:10.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.297+0000 7f138b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f138c102bf0 0x7f138c19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:40072/0 (socket says 172.21.15.111:40072) 2024-09-15T23:12:10.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.297+0000 7f138b400640 1 -- 172.21.15.111:0/3009045566 learned_addr learned my addr 172.21.15.111:0/3009045566 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:12:10.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.297+0000 7f138be00640 1 --2- 172.21.15.111:0/3009045566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f138c101a00 0x7f138c199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:10.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.298+0000 7f138b400640 1 -- 172.21.15.111:0/3009045566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f138c101a00 msgr2=0x7f138c199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:10.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.298+0000 7f138b400640 1 --2- 172.21.15.111:0/3009045566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f138c101a00 0x7f138c199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:10.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.298+0000 7f138b400640 1 -- 172.21.15.111:0/3009045566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f138c1035b0 msgr2=0x7f138c1a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:10.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.298+0000 7f138b400640 1 --2- 172.21.15.111:0/3009045566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f138c1035b0 0x7f138c1a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:10.301 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.298+0000 7f138b400640 1 -- 172.21.15.111:0/3009045566 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138c1a1b70 con 0x7f138c102bf0 2024-09-15T23:12:10.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.298+0000 7f138b400640 1 --2- 172.21.15.111:0/3009045566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f138c102bf0 0x7f138c19a490 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f137800e9f0 tx=0x7f137800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:12:10.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.298+0000 7f1388c00640 1 -- 172.21.15.111:0/3009045566 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137800ccc0 con 0x7f138c102bf0 2024-09-15T23:12:10.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.298+0000 7f1388c00640 1 -- 172.21.15.111:0/3009045566 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f137800ce60 con 0x7f138c102bf0 2024-09-15T23:12:10.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.298+0000 7f1391400640 1 -- 172.21.15.111:0/3009045566 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f138c1a1e60 con 0x7f138c102bf0 2024-09-15T23:12:10.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.298+0000 7f1388c00640 1 -- 172.21.15.111:0/3009045566 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1378010690 con 0x7f138c102bf0 2024-09-15T23:12:10.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.298+0000 7f1391400640 1 -- 172.21.15.111:0/3009045566 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f138c1a2320 con 0x7f138c102bf0 2024-09-15T23:12:10.302 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.299+0000 7f1391400640 1 -- 172.21.15.111:0/3009045566 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f138c1a25b0 con 0x7f138c102bf0 2024-09-15T23:12:10.303 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.302+0000 7f1388c00640 1 -- 172.21.15.111:0/3009045566 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f13780046c0 con 0x7f138c102bf0 2024-09-15T23:12:10.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.303+0000 7f1388c00640 1 --2- 172.21.15.111:0/3009045566 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f13700a32d0 0x7f13700a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:10.304 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.303+0000 7f1388c00640 1 -- 172.21.15.111:0/3009045566 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f13780cfe90 con 0x7f138c102bf0 2024-09-15T23:12:10.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.303+0000 7f138be00640 1 --2- 172.21.15.111:0/3009045566 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f13700a32d0 0x7f13700a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:10.305 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.304+0000 7f138be00640 1 --2- 172.21.15.111:0/3009045566 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f13700a32d0 0x7f13700a5790 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f137c02aa00 tx=0x7f137c0315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:12:10.306 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.305+0000 7f1388c00640 1 -- 172.21.15.111:0/3009045566 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f137809cbb0 con 0x7f138c102bf0 2024-09-15T23:12:10.317 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:10 smithi111 ceph-mon[30629]: pgmap v1138: 97 pgs: 97 active+clean; 420 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 6.4 MiB/s wr, 183 op/s 2024-09-15T23:12:10.318 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:10 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/411521090' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:12:10.318 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:10 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/961467451' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T23:12:10.318 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:10 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/2656543955' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-15T23:12:10.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:10 smithi121 ceph-mon[34815]: pgmap v1138: 97 pgs: 97 active+clean; 420 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 6.4 MiB/s wr, 183 op/s 2024-09-15T23:12:10.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:10 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/411521090' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:12:10.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:10 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/961467451' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T23:12:10.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:10 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/2656543955' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-15T23:12:10.466 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:10 smithi192 ceph-mon[35427]: pgmap v1138: 97 pgs: 97 active+clean; 420 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 6.4 MiB/s wr, 183 op/s 2024-09-15T23:12:10.466 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:10 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/411521090' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:12:10.466 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:10 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/961467451' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T23:12:10.466 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:10 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/2656543955' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-15T23:12:10.488 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.486+0000 7f1391400640 1 -- 172.21.15.111:0/3009045566 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f138c1a2780 con 0x7f138c102bf0 2024-09-15T23:12:10.500 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.498+0000 7f1388c00640 1 -- 172.21.15.111:0/3009045566 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f137809c4b0 con 0x7f138c102bf0 2024-09-15T23:12:10.500 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2024-09-15T23:12:10.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.500+0000 7f1391400640 1 -- 172.21.15.111:0/3009045566 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f13700a32d0 msgr2=0x7f13700a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:10.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.500+0000 7f1391400640 1 --2- 172.21.15.111:0/3009045566 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f13700a32d0 0x7f13700a5790 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f137c02aa00 tx=0x7f137c0315b0 comp rx=0 tx=0).stop 2024-09-15T23:12:10.502 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.500+0000 7f1391400640 1 -- 172.21.15.111:0/3009045566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f138c102bf0 msgr2=0x7f138c19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:10.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.500+0000 7f1391400640 1 --2- 172.21.15.111:0/3009045566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f138c102bf0 0x7f138c19a490 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f137800e9f0 tx=0x7f137800edf0 comp rx=0 tx=0).stop 2024-09-15T23:12:10.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.500+0000 7f1391400640 1 -- 172.21.15.111:0/3009045566 shutdown_connections 2024-09-15T23:12:10.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.501+0000 7f1391400640 1 --2- 172.21.15.111:0/3009045566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f138c1035b0 0x7f138c1a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:10.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.501+0000 7f1391400640 1 --2- 172.21.15.111:0/3009045566 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f13700a32d0 0x7f13700a5790 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:10.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.501+0000 7f1391400640 1 --2- 172.21.15.111:0/3009045566 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f138c102bf0 0x7f138c19a490 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:10.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.501+0000 7f1391400640 1 --2- 172.21.15.111:0/3009045566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f138c101a00 0x7f138c199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:10.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.501+0000 7f1391400640 1 -- 172.21.15.111:0/3009045566 >> 172.21.15.111:0/3009045566 conn(0x7f138c0fd180 msgr2=0x7f138c074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:12:10.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.501+0000 7f1391400640 1 -- 172.21.15.111:0/3009045566 shutdown_connections 2024-09-15T23:12:10.503 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.501+0000 7f1391400640 1 -- 172.21.15.111:0/3009045566 wait complete. 2024-09-15T23:12:10.518 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-15T23:12:10.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.695+0000 7faf19e00640 1 -- 172.21.15.111:0/3286171129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf14109c90 msgr2=0x7faf1410c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:10.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.695+0000 7faf19e00640 1 --2- 172.21.15.111:0/3286171129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf14109c90 0x7faf1410c080 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7faf0c009be0 tx=0x7faf0c02a6b0 comp rx=0 tx=0).stop 2024-09-15T23:12:10.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.695+0000 7faf19e00640 1 -- 172.21.15.111:0/3286171129 shutdown_connections 2024-09-15T23:12:10.696 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.695+0000 7faf19e00640 1 --2- 172.21.15.111:0/3286171129 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf14109c90 0x7faf1410c080 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:10.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.695+0000 7faf19e00640 1 --2- 172.21.15.111:0/3286171129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf14074fc0 0x7faf14109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:10.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.695+0000 7faf19e00640 1 --2- 172.21.15.111:0/3286171129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf14074680 0x7faf14074a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:10.697 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.695+0000 7faf19e00640 1 -- 172.21.15.111:0/3286171129 >> 172.21.15.111:0/3286171129 conn(0x7faf140fd110 msgr2=0x7faf140ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:12:10.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.695+0000 7faf19e00640 1 -- 172.21.15.111:0/3286171129 shutdown_connections 2024-09-15T23:12:10.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.695+0000 7faf19e00640 1 -- 172.21.15.111:0/3286171129 wait complete. 2024-09-15T23:12:10.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.696+0000 7faf19e00640 1 Processor -- start 2024-09-15T23:12:10.698 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.696+0000 7faf19e00640 1 -- start start 2024-09-15T23:12:10.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.696+0000 7faf19e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf14074680 0x7faf14199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:10.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.696+0000 7faf19e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf14074fc0 0x7faf1419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:10.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.696+0000 7faf19e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf14109c90 0x7faf141a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:10.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.696+0000 7faf19e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7faf1410f8d0 con 0x7faf14074680 2024-09-15T23:12:10.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.696+0000 7faf19e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faf1410f750 con 0x7faf14109c90 2024-09-15T23:12:10.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.696+0000 7faf19e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faf1410fa50 con 0x7faf14074fc0 2024-09-15T23:12:10.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.696+0000 7faf18a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf14074680 0x7faf14199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:10.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.696+0000 7faf18a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf14074680 0x7faf14199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53916/0 (socket says 172.21.15.111:53916) 2024-09-15T23:12:10.699 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.696+0000 7faf18a00640 1 -- 172.21.15.111:0/1441430800 learned_addr learned my addr 172.21.15.111:0/1441430800 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:12:10.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.696+0000 7faf19400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf14109c90 0x7faf141a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:10.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.696+0000 7faf13e00640 1 --2- 172.21.15.111:0/1441430800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf14074fc0 0x7faf1419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:10.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.697+0000 7faf18a00640 1 -- 172.21.15.111:0/1441430800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf14074fc0 msgr2=0x7faf1419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:10.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.697+0000 7faf18a00640 1 --2- 172.21.15.111:0/1441430800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf14074fc0 0x7faf1419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:10.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.697+0000 7faf18a00640 1 -- 172.21.15.111:0/1441430800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf14109c90 msgr2=0x7faf141a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:10.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.697+0000 7faf18a00640 1 --2- 172.21.15.111:0/1441430800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf14109c90 0x7faf141a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:10.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.697+0000 7faf18a00640 1 -- 172.21.15.111:0/1441430800 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf141a1b50 con 0x7faf14074680 2024-09-15T23:12:10.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.697+0000 7faf19400640 1 --2- 172.21.15.111:0/1441430800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf14109c90 0x7faf141a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:12:10.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.697+0000 7faf18a00640 1 --2- 172.21.15.111:0/1441430800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf14074680 0x7faf14199f30 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7faf0800e9c0 tx=0x7faf0800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:12:10.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.697+0000 7faf11600640 1 -- 172.21.15.111:0/1441430800 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0800ccc0 con 0x7faf14074680 2024-09-15T23:12:10.709 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.697+0000 7faf11600640 1 -- 172.21.15.111:0/1441430800 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf08005500 con 0x7faf14074680 2024-09-15T23:12:10.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.697+0000 7faf19e00640 1 -- 172.21.15.111:0/1441430800 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf141a1e40 con 0x7faf14074680 2024-09-15T23:12:10.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.697+0000 7faf11600640 1 -- 172.21.15.111:0/1441430800 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf08010430 con 0x7faf14074680 2024-09-15T23:12:10.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.697+0000 7faf19e00640 1 -- 172.21.15.111:0/1441430800 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf141a22a0 con 0x7faf14074680 2024-09-15T23:12:10.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.698+0000 7faf19e00640 1 -- 172.21.15.111:0/1441430800 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf14074a80 con 0x7faf14074680 2024-09-15T23:12:10.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.701+0000 7faf11600640 1 -- 172.21.15.111:0/1441430800 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7faf080046c0 con 0x7faf14074680 2024-09-15T23:12:10.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.702+0000 7faf11600640 1 --2- 172.21.15.111:0/1441430800 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faefc0a3250 0x7faefc0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:10.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.702+0000 7faf11600640 1 -- 172.21.15.111:0/1441430800 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7faf080cfe00 con 0x7faf14074680 2024-09-15T23:12:10.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.702+0000 7faf13e00640 1 --2- 172.21.15.111:0/1441430800 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faefc0a3250 0x7faefc0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:10.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.703+0000 7faf13e00640 1 --2- 172.21.15.111:0/1441430800 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faefc0a3250 0x7faefc0a5710 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7faf1419b440 tx=0x7faf00005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:12:10.710 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.706+0000 7faf11600640 1 -- 172.21.15.111:0/1441430800 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faf080541b0 con 0x7faf14074680 2024-09-15T23:12:10.940 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.939+0000 7faf19e00640 1 -- 172.21.15.111:0/1441430800 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7faf14108560 con 0x7faf14074680 2024-09-15T23:12:10.954 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.953+0000 7faf11600640 1 -- 172.21.15.111:0/1441430800 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+742348 (secure 0 0 0) 0x7faf0809cb20 con 0x7faf14074680 2024-09-15T23:12:10.956 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T23:12:10.956 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-15T23:12:10.956 INFO:teuthology.orchestra.run.smithi111.stdout: "epoch": 20, 2024-09-15T23:12:10.956 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:10.956 INFO:teuthology.orchestra.run.smithi111.stdout: "active_gid": 14148, 2024-09-15T23:12:10.957 INFO:teuthology.orchestra.run.smithi111.stdout: "active_name": "x", 2024-09-15T23:12:10.957 INFO:teuthology.orchestra.run.smithi111.stdout: "active_addrs": { 2024-09-15T23:12:10.957 INFO:teuthology.orchestra.run.smithi111.stdout: "addrvec": [ 2024-09-15T23:12:10.957 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:10.957 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v2", 2024-09-15T23:12:10.957 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "172.21.15.111:6800", 2024-09-15T23:12:10.957 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 3125866964 2024-09-15T23:12:10.957 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.957 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:10.957 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2024-09-15T23:12:10.957 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "172.21.15.111:6801", 2024-09-15T23:12:10.957 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 3125866964 2024-09-15T23:12:10.957 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:10.958 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-15T23:12:10.958 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.958 INFO:teuthology.orchestra.run.smithi111.stdout: "active_addr": "172.21.15.111:6801/3125866964", 2024-09-15T23:12:10.958 INFO:teuthology.orchestra.run.smithi111.stdout: "active_change": "2024-09-15T22:36:50.357676+0000", 2024-09-15T23:12:10.958 INFO:teuthology.orchestra.run.smithi111.stdout: "active_mgr_features": 4540701547738038271, 2024-09-15T23:12:10.958 INFO:teuthology.orchestra.run.smithi111.stdout: "available": true, 2024-09-15T23:12:10.958 INFO:teuthology.orchestra.run.smithi111.stdout: "standbys": [ 2024-09-15T23:12:10.958 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:10.958 INFO:teuthology.orchestra.run.smithi111.stdout: "gid": 24106, 2024-09-15T23:12:10.958 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "z", 2024-09-15T23:12:10.958 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr_features": 4540701547738038271, 2024-09-15T23:12:10.958 INFO:teuthology.orchestra.run.smithi111.stdout: "available_modules": [ 2024-09-15T23:12:10.958 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:10.959 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "alerts", 2024-09-15T23:12:10.959 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:10.959 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:10.959 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:10.959 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-09-15T23:12:10.959 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-09-15T23:12:10.959 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:10.959 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.959 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.959 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-09-15T23:12:10.959 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:10.959 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:10.959 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:10.960 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "How frequently to reexamine health status", 2024-09-15T23:12:10.960 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:10.960 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.960 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.960 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.960 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:10.960 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:10.960 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:10.960 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.960 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.960 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:10.960 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:10.960 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:10.961 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:10.961 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:10.961 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:10.961 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:10.961 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:10.961 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:10.961 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:10.961 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:10.961 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:10.961 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:10.961 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.961 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.961 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.962 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:10.962 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:10.962 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:10.962 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.962 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.962 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:10.962 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:10.962 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:10.962 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:10.962 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:10.962 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:10.962 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.962 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.963 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.963 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:10.963 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:10.963 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:10.963 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.963 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.963 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:10.963 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:10.963 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:10.963 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:10.963 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:10.963 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:10.963 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:10.964 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:10.964 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:10.964 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:10.964 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:10.964 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:10.964 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:10.964 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.964 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.964 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.964 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:10.964 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:10.964 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:10.964 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.965 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.965 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:10.965 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:10.965 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:10.965 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:10.965 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:10.965 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:10.965 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.965 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.965 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.965 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_destination": { 2024-09-15T23:12:10.965 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_destination", 2024-09-15T23:12:10.965 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:10.966 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.966 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.966 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:10.966 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:10.966 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:10.966 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:10.966 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Email address to send alerts to", 2024-09-15T23:12:10.966 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:10.966 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.966 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.966 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.966 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_from_name": { 2024-09-15T23:12:10.966 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_from_name", 2024-09-15T23:12:10.966 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:10.966 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.967 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.967 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "Ceph", 2024-09-15T23:12:10.967 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:10.967 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:10.967 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:10.967 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Email From: name", 2024-09-15T23:12:10.967 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:10.967 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.967 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.967 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.967 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_host": { 2024-09-15T23:12:10.967 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_host", 2024-09-15T23:12:10.967 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:10.967 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.968 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.968 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:10.968 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:10.968 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:10.968 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:10.968 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "SMTP server", 2024-09-15T23:12:10.968 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:10.968 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.968 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.968 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.968 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_password": { 2024-09-15T23:12:10.968 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_password", 2024-09-15T23:12:10.968 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:10.968 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.969 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.969 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:10.969 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:10.969 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:10.969 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:10.969 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Password to authenticate with", 2024-09-15T23:12:10.969 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:10.969 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.969 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.969 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.969 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_port": { 2024-09-15T23:12:10.969 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_port", 2024-09-15T23:12:10.969 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:10.970 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.970 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.970 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "465", 2024-09-15T23:12:10.970 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:10.970 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:10.970 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:10.970 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "SMTP port", 2024-09-15T23:12:10.970 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:10.970 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.970 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.970 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.970 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_sender": { 2024-09-15T23:12:10.970 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_sender", 2024-09-15T23:12:10.970 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:10.971 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.971 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.971 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:10.971 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:10.971 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:10.971 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:10.971 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "SMTP envelope sender", 2024-09-15T23:12:10.971 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:10.971 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.971 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.971 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.971 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_ssl": { 2024-09-15T23:12:10.971 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_ssl", 2024-09-15T23:12:10.972 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:10.972 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.972 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.972 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:10.972 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:10.972 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:10.972 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:10.972 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-15T23:12:10.972 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:10.972 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.972 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.972 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.972 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_user": { 2024-09-15T23:12:10.972 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_user", 2024-09-15T23:12:10.973 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:10.973 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.973 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.973 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:10.973 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:10.973 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:10.973 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:10.973 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "User to authenticate as", 2024-09-15T23:12:10.973 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:10.973 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.973 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.973 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:10.973 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:10.973 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.974 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:10.974 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "balancer", 2024-09-15T23:12:10.974 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:10.974 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:10.974 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:10.974 INFO:teuthology.orchestra.run.smithi111.stdout: "active": { 2024-09-15T23:12:10.974 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "active", 2024-09-15T23:12:10.974 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:10.974 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.974 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.974 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:10.974 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:10.974 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:10.975 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:10.975 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "automatically balance PGs across cluster", 2024-09-15T23:12:10.975 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:10.975 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.975 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.975 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.975 INFO:teuthology.orchestra.run.smithi111.stdout: "begin_time": { 2024-09-15T23:12:10.975 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "begin_time", 2024-09-15T23:12:10.975 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:10.975 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.975 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.975 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0000", 2024-09-15T23:12:10.975 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:10.976 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:10.976 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:10.976 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "beginning time of day to automatically balance", 2024-09-15T23:12:10.976 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-15T23:12:10.976 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.976 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.976 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.976 INFO:teuthology.orchestra.run.smithi111.stdout: "begin_weekday": { 2024-09-15T23:12:10.976 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "begin_weekday", 2024-09-15T23:12:10.976 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-09-15T23:12:10.976 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.976 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.976 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-09-15T23:12:10.977 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "0", 2024-09-15T23:12:10.977 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "6", 2024-09-15T23:12:10.977 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:10.977 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-15T23:12:10.977 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-15T23:12:10.977 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.977 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.977 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.977 INFO:teuthology.orchestra.run.smithi111.stdout: "crush_compat_max_iterations": { 2024-09-15T23:12:10.977 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crush_compat_max_iterations", 2024-09-15T23:12:10.977 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-09-15T23:12:10.977 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.978 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.978 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "25", 2024-09-15T23:12:10.978 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1", 2024-09-15T23:12:10.978 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "250", 2024-09-15T23:12:10.978 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:10.978 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-15T23:12:10.978 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:10.978 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.978 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.978 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.978 INFO:teuthology.orchestra.run.smithi111.stdout: "crush_compat_metrics": { 2024-09-15T23:12:10.978 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crush_compat_metrics", 2024-09-15T23:12:10.979 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:10.979 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.979 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.979 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "pgs,objects,bytes", 2024-09-15T23:12:10.979 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:10.979 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:10.979 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:10.979 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-15T23:12:10.979 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-15T23:12:10.979 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.979 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.979 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.979 INFO:teuthology.orchestra.run.smithi111.stdout: "crush_compat_step": { 2024-09-15T23:12:10.980 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crush_compat_step", 2024-09-15T23:12:10.980 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-09-15T23:12:10.980 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.980 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.980 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0.5", 2024-09-15T23:12:10.980 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "0.001", 2024-09-15T23:12:10.980 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "0.999", 2024-09-15T23:12:10.980 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:10.980 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "aggressiveness of optimization", 2024-09-15T23:12:10.980 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-15T23:12:10.980 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.980 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.980 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.981 INFO:teuthology.orchestra.run.smithi111.stdout: "end_time": { 2024-09-15T23:12:10.981 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "end_time", 2024-09-15T23:12:10.981 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:10.981 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.981 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.981 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "2359", 2024-09-15T23:12:10.981 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:10.981 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:10.981 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:10.981 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "ending time of day to automatically balance", 2024-09-15T23:12:10.981 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-15T23:12:10.981 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.981 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.981 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.982 INFO:teuthology.orchestra.run.smithi111.stdout: "end_weekday": { 2024-09-15T23:12:10.982 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "end_weekday", 2024-09-15T23:12:10.982 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-09-15T23:12:10.982 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.982 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.982 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-09-15T23:12:10.982 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "0", 2024-09-15T23:12:10.982 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "6", 2024-09-15T23:12:10.982 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:10.982 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-15T23:12:10.982 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-15T23:12:10.982 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.982 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.983 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.983 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:10.983 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:10.983 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:10.983 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.983 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.983 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:10.983 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:10.983 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:10.983 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:10.983 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:10.983 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:10.983 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:10.983 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:10.984 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:10.984 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:10.984 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:10.984 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:10.984 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:10.984 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.984 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.984 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.984 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:10.984 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:10.984 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:10.984 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.984 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.989 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:10.989 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:10.989 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:10.989 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:10.989 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:10.989 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:10.989 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.989 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.989 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.989 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:10.990 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:10.990 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:10.990 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.990 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.990 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:10.990 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:10.990 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:10.990 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:10.990 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:10.990 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:10.990 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:10.990 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:10.991 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:10.991 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:10.991 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:10.991 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:10.991 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:10.991 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.991 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.991 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.991 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:10.991 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:10.991 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:10.991 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.991 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.992 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:10.992 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:10.992 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:10.992 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:10.992 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:10.992 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:10.992 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.992 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.992 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.992 INFO:teuthology.orchestra.run.smithi111.stdout: "min_score": { 2024-09-15T23:12:10.992 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "min_score", 2024-09-15T23:12:10.992 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-09-15T23:12:10.993 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.993 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.993 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-09-15T23:12:10.993 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:10.993 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:10.993 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:10.993 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-15T23:12:10.993 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:10.993 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.993 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.993 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.993 INFO:teuthology.orchestra.run.smithi111.stdout: "mode": { 2024-09-15T23:12:10.993 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mode", 2024-09-15T23:12:10.994 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:10.994 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.994 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.994 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "upmap", 2024-09-15T23:12:10.994 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:10.994 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:10.994 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:10.994 INFO:teuthology.orchestra.run.smithi111.stdout: "crush-compat", 2024-09-15T23:12:10.994 INFO:teuthology.orchestra.run.smithi111.stdout: "none", 2024-09-15T23:12:10.994 INFO:teuthology.orchestra.run.smithi111.stdout: "read", 2024-09-15T23:12:10.994 INFO:teuthology.orchestra.run.smithi111.stdout: "upmap", 2024-09-15T23:12:10.994 INFO:teuthology.orchestra.run.smithi111.stdout: "upmap-read" 2024-09-15T23:12:10.994 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:10.995 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Balancer mode", 2024-09-15T23:12:10.995 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:10.995 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.995 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.995 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.995 INFO:teuthology.orchestra.run.smithi111.stdout: "pool_ids": { 2024-09-15T23:12:10.995 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "pool_ids", 2024-09-15T23:12:10.995 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:10.995 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.995 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.995 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:10.995 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:10.995 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:10.995 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:10.996 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-15T23:12:10.996 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:10.996 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.996 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.996 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.996 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-09-15T23:12:10.996 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-09-15T23:12:10.996 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:10.996 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.996 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.996 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-09-15T23:12:10.996 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:10.996 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:10.997 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:10.997 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-15T23:12:10.997 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:10.997 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.997 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.997 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.997 INFO:teuthology.orchestra.run.smithi111.stdout: "upmap_max_deviation": { 2024-09-15T23:12:10.997 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "upmap_max_deviation", 2024-09-15T23:12:10.997 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:10.997 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.997 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.997 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5", 2024-09-15T23:12:10.997 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1", 2024-09-15T23:12:10.998 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:10.998 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:10.998 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-15T23:12:10.998 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-15T23:12:10.998 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.998 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.998 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.998 INFO:teuthology.orchestra.run.smithi111.stdout: "upmap_max_optimizations": { 2024-09-15T23:12:10.998 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "upmap_max_optimizations", 2024-09-15T23:12:10.998 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-09-15T23:12:10.998 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:10.998 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:10.998 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-09-15T23:12:10.998 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:10.999 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:10.999 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:10.999 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-15T23:12:10.999 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:10.999 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:10.999 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:10.999 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:10.999 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:10.999 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:10.999 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:10.999 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cephadm", 2024-09-15T23:12:10.999 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:10.999 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:10.999 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.000 INFO:teuthology.orchestra.run.smithi111.stdout: "agent_down_multiplier": { 2024-09-15T23:12:11.000 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "agent_down_multiplier", 2024-09-15T23:12:11.000 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-09-15T23:12:11.000 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.000 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.000 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "3.0", 2024-09-15T23:12:11.000 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.000 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.000 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.000 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-15T23:12:11.000 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.000 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.001 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.001 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.001 INFO:teuthology.orchestra.run.smithi111.stdout: "agent_refresh_rate": { 2024-09-15T23:12:11.001 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "agent_refresh_rate", 2024-09-15T23:12:11.001 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.001 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.001 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.001 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "20", 2024-09-15T23:12:11.001 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.001 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.001 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.001 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-15T23:12:11.001 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.002 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.002 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.002 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.002 INFO:teuthology.orchestra.run.smithi111.stdout: "agent_starting_port": { 2024-09-15T23:12:11.002 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "agent_starting_port", 2024-09-15T23:12:11.002 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.002 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.002 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.002 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "4721", 2024-09-15T23:12:11.002 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.002 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.002 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.002 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-15T23:12:11.003 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.003 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.003 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.003 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.003 INFO:teuthology.orchestra.run.smithi111.stdout: "allow_ptrace": { 2024-09-15T23:12:11.003 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "allow_ptrace", 2024-09-15T23:12:11.003 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.003 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.003 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.003 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.003 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.003 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.003 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.004 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-15T23:12:11.004 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-15T23:12:11.004 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.004 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.004 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.004 INFO:teuthology.orchestra.run.smithi111.stdout: "autotune_interval": { 2024-09-15T23:12:11.004 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "autotune_interval", 2024-09-15T23:12:11.004 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.004 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.004 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.004 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-09-15T23:12:11.004 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.004 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.005 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.005 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-15T23:12:11.005 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.005 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.005 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.005 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.005 INFO:teuthology.orchestra.run.smithi111.stdout: "autotune_memory_target_ratio": { 2024-09-15T23:12:11.005 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "autotune_memory_target_ratio", 2024-09-15T23:12:11.005 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-09-15T23:12:11.005 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.005 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.005 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0.7", 2024-09-15T23:12:11.005 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.006 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.006 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.006 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-15T23:12:11.006 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.006 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.006 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.006 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.006 INFO:teuthology.orchestra.run.smithi111.stdout: "cephadm_log_destination": { 2024-09-15T23:12:11.006 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cephadm_log_destination", 2024-09-15T23:12:11.006 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.006 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.006 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.006 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.007 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.007 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.007 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.007 INFO:teuthology.orchestra.run.smithi111.stdout: "file", 2024-09-15T23:12:11.007 INFO:teuthology.orchestra.run.smithi111.stdout: "file,syslog", 2024-09-15T23:12:11.007 INFO:teuthology.orchestra.run.smithi111.stdout: "syslog" 2024-09-15T23:12:11.007 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.007 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-15T23:12:11.007 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.007 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.007 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.007 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.007 INFO:teuthology.orchestra.run.smithi111.stdout: "cgroups_split": { 2024-09-15T23:12:11.008 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cgroups_split", 2024-09-15T23:12:11.008 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.008 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.008 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.008 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.008 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.008 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.008 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.008 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-15T23:12:11.008 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.008 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.008 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.008 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.009 INFO:teuthology.orchestra.run.smithi111.stdout: "config_checks_enabled": { 2024-09-15T23:12:11.009 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "config_checks_enabled", 2024-09-15T23:12:11.009 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.009 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.009 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.009 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.009 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.009 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.009 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.009 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-15T23:12:11.009 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.009 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.010 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.010 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.010 INFO:teuthology.orchestra.run.smithi111.stdout: "config_dashboard": { 2024-09-15T23:12:11.010 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "config_dashboard", 2024-09-15T23:12:11.010 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.010 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.010 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.010 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.010 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.010 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.010 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.010 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-15T23:12:11.010 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.011 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.011 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.011 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.011 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_alertmanager": { 2024-09-15T23:12:11.011 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_alertmanager", 2024-09-15T23:12:11.011 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.011 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.011 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.011 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-15T23:12:11.011 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.011 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.011 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.011 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Prometheus container image", 2024-09-15T23:12:11.011 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.012 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.012 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.012 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.012 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_base": { 2024-09-15T23:12:11.012 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_base", 2024-09-15T23:12:11.012 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.012 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.012 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.012 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-15T23:12:11.012 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.012 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.012 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.012 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Container image name, without the tag", 2024-09-15T23:12:11.012 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.013 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.013 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.013 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.013 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_elasticsearch": { 2024-09-15T23:12:11.013 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_elasticsearch", 2024-09-15T23:12:11.013 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.013 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.013 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.013 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-15T23:12:11.013 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.013 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.013 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.013 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "elasticsearch container image", 2024-09-15T23:12:11.013 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.014 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.014 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.014 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.014 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_grafana": { 2024-09-15T23:12:11.014 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_grafana", 2024-09-15T23:12:11.014 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.014 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.014 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.014 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-15T23:12:11.014 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.014 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.014 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.014 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Prometheus container image", 2024-09-15T23:12:11.014 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.015 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.015 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.015 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.015 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_haproxy": { 2024-09-15T23:12:11.015 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_haproxy", 2024-09-15T23:12:11.015 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.015 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.015 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.015 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-15T23:12:11.015 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.015 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.015 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.016 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "HAproxy container image", 2024-09-15T23:12:11.016 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.016 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.016 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.016 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.016 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_jaeger_agent": { 2024-09-15T23:12:11.016 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_jaeger_agent", 2024-09-15T23:12:11.016 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.016 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.016 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.016 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-15T23:12:11.016 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.016 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.016 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.017 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Jaeger agent container image", 2024-09-15T23:12:11.017 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.017 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.017 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.017 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.017 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_jaeger_collector": { 2024-09-15T23:12:11.017 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_jaeger_collector", 2024-09-15T23:12:11.017 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.017 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.017 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.017 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-15T23:12:11.017 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.017 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.018 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.018 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Jaeger collector container image", 2024-09-15T23:12:11.018 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.018 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.018 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.018 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.018 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_jaeger_query": { 2024-09-15T23:12:11.018 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_jaeger_query", 2024-09-15T23:12:11.018 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.018 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.018 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.018 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-15T23:12:11.018 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.019 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.019 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.019 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Jaeger query container image", 2024-09-15T23:12:11.019 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.019 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.019 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.019 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.019 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_keepalived": { 2024-09-15T23:12:11.019 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_keepalived", 2024-09-15T23:12:11.019 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.019 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.019 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.019 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-15T23:12:11.019 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.020 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.020 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.020 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Keepalived container image", 2024-09-15T23:12:11.020 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.020 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.020 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.020 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.020 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_loki": { 2024-09-15T23:12:11.020 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_loki", 2024-09-15T23:12:11.020 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.020 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.020 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.020 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-15T23:12:11.021 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.021 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.021 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.021 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Loki container image", 2024-09-15T23:12:11.021 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.021 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.021 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.021 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.021 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_node_exporter": { 2024-09-15T23:12:11.021 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_node_exporter", 2024-09-15T23:12:11.021 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.021 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.021 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.021 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-15T23:12:11.022 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.022 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.022 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.022 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Prometheus container image", 2024-09-15T23:12:11.022 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.022 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.022 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.022 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.022 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_nvmeof": { 2024-09-15T23:12:11.022 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_nvmeof", 2024-09-15T23:12:11.022 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.022 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.022 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.023 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-15T23:12:11.023 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.023 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.023 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.023 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Nvme-of container image", 2024-09-15T23:12:11.023 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.023 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.023 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.023 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.023 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_prometheus": { 2024-09-15T23:12:11.023 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_prometheus", 2024-09-15T23:12:11.023 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.023 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.024 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.024 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-15T23:12:11.024 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.024 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.024 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.024 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Prometheus container image", 2024-09-15T23:12:11.024 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.024 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.024 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.024 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.024 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_promtail": { 2024-09-15T23:12:11.024 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_promtail", 2024-09-15T23:12:11.024 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.025 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.025 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.025 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-15T23:12:11.025 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.025 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.025 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.025 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Promtail container image", 2024-09-15T23:12:11.025 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.025 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.025 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.025 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.025 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_samba": { 2024-09-15T23:12:11.025 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_samba", 2024-09-15T23:12:11.025 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.026 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.026 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.026 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-15T23:12:11.026 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.026 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.026 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.026 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Samba/SMB container image", 2024-09-15T23:12:11.026 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.026 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.026 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.026 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.026 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_snmp_gateway": { 2024-09-15T23:12:11.026 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_snmp_gateway", 2024-09-15T23:12:11.027 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.027 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.027 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.027 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-15T23:12:11.027 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.027 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.027 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.027 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "SNMP Gateway container image", 2024-09-15T23:12:11.027 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.027 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.027 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.027 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.027 INFO:teuthology.orchestra.run.smithi111.stdout: "container_init": { 2024-09-15T23:12:11.027 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_init", 2024-09-15T23:12:11.028 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.028 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.028 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.028 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.028 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.028 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.028 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.028 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Run podman/docker with `--init`", 2024-09-15T23:12:11.028 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.028 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.028 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.028 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.028 INFO:teuthology.orchestra.run.smithi111.stdout: "daemon_cache_timeout": { 2024-09-15T23:12:11.029 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "daemon_cache_timeout", 2024-09-15T23:12:11.029 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.029 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.029 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.029 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-09-15T23:12:11.029 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.029 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.029 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.029 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-15T23:12:11.029 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.029 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.029 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.029 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.030 INFO:teuthology.orchestra.run.smithi111.stdout: "default_cephadm_command_timeout": { 2024-09-15T23:12:11.030 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "default_cephadm_command_timeout", 2024-09-15T23:12:11.030 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.030 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.030 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.030 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "900", 2024-09-15T23:12:11.030 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.030 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.030 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.030 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-15T23:12:11.030 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.030 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.031 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.031 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.031 INFO:teuthology.orchestra.run.smithi111.stdout: "default_registry": { 2024-09-15T23:12:11.031 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "default_registry", 2024-09-15T23:12:11.031 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.031 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.031 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.031 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "docker.io", 2024-09-15T23:12:11.031 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.031 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.031 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.031 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-15T23:12:11.031 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.032 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.032 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.032 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.032 INFO:teuthology.orchestra.run.smithi111.stdout: "device_cache_timeout": { 2024-09-15T23:12:11.032 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "device_cache_timeout", 2024-09-15T23:12:11.032 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.032 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.032 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.032 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "1800", 2024-09-15T23:12:11.032 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.032 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.032 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.032 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "seconds to cache device inventory", 2024-09-15T23:12:11.032 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.033 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.033 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.033 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.033 INFO:teuthology.orchestra.run.smithi111.stdout: "device_enhanced_scan": { 2024-09-15T23:12:11.033 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "device_enhanced_scan", 2024-09-15T23:12:11.033 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.033 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.033 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.033 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.033 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.033 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.033 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.033 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-15T23:12:11.034 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.034 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.034 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.034 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.034 INFO:teuthology.orchestra.run.smithi111.stdout: "facts_cache_timeout": { 2024-09-15T23:12:11.034 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "facts_cache_timeout", 2024-09-15T23:12:11.034 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.034 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.034 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.034 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-09-15T23:12:11.034 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.034 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.034 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.035 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "seconds to cache host facts data", 2024-09-15T23:12:11.035 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.035 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.035 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.035 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.035 INFO:teuthology.orchestra.run.smithi111.stdout: "grafana_dashboards_path": { 2024-09-15T23:12:11.035 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "grafana_dashboards_path", 2024-09-15T23:12:11.035 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.035 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.035 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.035 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-15T23:12:11.035 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.035 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.035 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.036 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-15T23:12:11.036 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.036 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.036 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.036 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.036 INFO:teuthology.orchestra.run.smithi111.stdout: "host_check_interval": { 2024-09-15T23:12:11.036 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "host_check_interval", 2024-09-15T23:12:11.036 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.036 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.036 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.036 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-09-15T23:12:11.036 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.036 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.037 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.037 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to perform a host check", 2024-09-15T23:12:11.037 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.037 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.037 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.037 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.037 INFO:teuthology.orchestra.run.smithi111.stdout: "hw_monitoring": { 2024-09-15T23:12:11.037 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "hw_monitoring", 2024-09-15T23:12:11.037 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.037 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.037 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.037 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.037 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.038 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.038 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.038 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-15T23:12:11.038 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.038 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.038 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.038 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.038 INFO:teuthology.orchestra.run.smithi111.stdout: "inventory_list_all": { 2024-09-15T23:12:11.038 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "inventory_list_all", 2024-09-15T23:12:11.038 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.038 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.038 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.038 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.038 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.039 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.039 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.039 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-15T23:12:11.039 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.039 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.039 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.039 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.039 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.039 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.039 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.039 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.039 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.040 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.040 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.040 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.040 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.040 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.040 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.040 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.040 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.040 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.040 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.040 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.040 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.040 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.041 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.041 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.041 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.041 INFO:teuthology.orchestra.run.smithi111.stdout: "log_refresh_metadata": { 2024-09-15T23:12:11.041 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_refresh_metadata", 2024-09-15T23:12:11.041 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.041 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.041 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.041 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.041 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.041 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.041 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.041 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-15T23:12:11.042 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.042 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.042 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.042 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.042 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.042 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.042 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.042 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.042 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.042 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.042 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.042 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.042 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.043 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-15T23:12:11.043 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.043 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.043 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.043 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.043 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.043 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.043 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.043 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.043 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.043 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.043 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.043 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.044 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.044 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.044 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.044 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.044 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.044 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.044 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.044 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.044 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.044 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.044 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.044 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.044 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.045 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.045 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.045 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.045 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.045 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.045 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.045 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.045 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.045 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.045 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.045 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.045 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.045 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.046 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.046 INFO:teuthology.orchestra.run.smithi111.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-15T23:12:11.046 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-15T23:12:11.046 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.046 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.046 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.046 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.046 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.046 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.046 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.046 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-15T23:12:11.046 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.046 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.046 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.047 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.047 INFO:teuthology.orchestra.run.smithi111.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-15T23:12:11.047 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-15T23:12:11.047 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.047 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.047 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.047 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "*", 2024-09-15T23:12:11.047 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.047 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.047 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.047 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-15T23:12:11.047 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.047 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.048 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.048 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.048 INFO:teuthology.orchestra.run.smithi111.stdout: "max_count_per_host": { 2024-09-15T23:12:11.048 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_count_per_host", 2024-09-15T23:12:11.048 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.048 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.048 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.048 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-09-15T23:12:11.048 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.048 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.048 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.048 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "max number of daemons per service per host", 2024-09-15T23:12:11.048 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.048 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.049 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.049 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.049 INFO:teuthology.orchestra.run.smithi111.stdout: "max_osd_draining_count": { 2024-09-15T23:12:11.049 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_osd_draining_count", 2024-09-15T23:12:11.049 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.049 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.049 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.049 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-09-15T23:12:11.049 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.049 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.049 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.049 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-15T23:12:11.049 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.050 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.050 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.050 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.050 INFO:teuthology.orchestra.run.smithi111.stdout: "migration_current": { 2024-09-15T23:12:11.050 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "migration_current", 2024-09-15T23:12:11.050 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.050 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.050 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.050 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.050 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.050 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.050 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.050 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "internal - do not modify", 2024-09-15T23:12:11.050 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.051 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.051 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.051 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.051 INFO:teuthology.orchestra.run.smithi111.stdout: "mode": { 2024-09-15T23:12:11.051 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mode", 2024-09-15T23:12:11.051 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.051 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.051 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.051 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "root", 2024-09-15T23:12:11.051 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.051 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.051 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.051 INFO:teuthology.orchestra.run.smithi111.stdout: "cephadm-package", 2024-09-15T23:12:11.052 INFO:teuthology.orchestra.run.smithi111.stdout: "root" 2024-09-15T23:12:11.052 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.052 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "mode for remote execution of cephadm", 2024-09-15T23:12:11.052 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.052 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.052 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.052 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.052 INFO:teuthology.orchestra.run.smithi111.stdout: "oob_default_addr": { 2024-09-15T23:12:11.052 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "oob_default_addr", 2024-09-15T23:12:11.052 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.052 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.052 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.052 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "169.254.1.1", 2024-09-15T23:12:11.052 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.052 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.053 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.053 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-15T23:12:11.053 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.053 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.053 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.053 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.053 INFO:teuthology.orchestra.run.smithi111.stdout: "prometheus_alerts_path": { 2024-09-15T23:12:11.053 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "prometheus_alerts_path", 2024-09-15T23:12:11.053 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.053 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.053 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.053 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-15T23:12:11.053 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.054 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.054 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.054 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-15T23:12:11.054 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.054 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.054 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.054 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.054 INFO:teuthology.orchestra.run.smithi111.stdout: "registry_insecure": { 2024-09-15T23:12:11.054 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "registry_insecure", 2024-09-15T23:12:11.054 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.054 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.054 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.054 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.054 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.055 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.055 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.055 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-15T23:12:11.055 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.055 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.055 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.055 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.055 INFO:teuthology.orchestra.run.smithi111.stdout: "registry_password": { 2024-09-15T23:12:11.055 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "registry_password", 2024-09-15T23:12:11.055 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.055 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.055 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.055 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.055 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.056 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.056 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.056 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-15T23:12:11.056 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.056 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.056 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.056 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.056 INFO:teuthology.orchestra.run.smithi111.stdout: "registry_url": { 2024-09-15T23:12:11.056 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "registry_url", 2024-09-15T23:12:11.056 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.056 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.056 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.056 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.057 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.057 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.057 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.057 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-15T23:12:11.057 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.057 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.057 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.057 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.057 INFO:teuthology.orchestra.run.smithi111.stdout: "registry_username": { 2024-09-15T23:12:11.057 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "registry_username", 2024-09-15T23:12:11.057 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.057 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.057 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.058 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.058 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.058 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.058 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.058 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-15T23:12:11.058 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.058 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.058 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.058 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.058 INFO:teuthology.orchestra.run.smithi111.stdout: "secure_monitoring_stack": { 2024-09-15T23:12:11.058 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "secure_monitoring_stack", 2024-09-15T23:12:11.058 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.058 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.058 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.059 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.059 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.059 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.059 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.059 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-15T23:12:11.059 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.059 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.059 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.059 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.059 INFO:teuthology.orchestra.run.smithi111.stdout: "service_discovery_port": { 2024-09-15T23:12:11.059 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "service_discovery_port", 2024-09-15T23:12:11.059 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.059 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.059 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.060 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8765", 2024-09-15T23:12:11.060 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.060 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.060 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.060 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "cephadm service discovery port", 2024-09-15T23:12:11.060 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.060 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.060 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.060 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.060 INFO:teuthology.orchestra.run.smithi111.stdout: "ssh_config_file": { 2024-09-15T23:12:11.060 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ssh_config_file", 2024-09-15T23:12:11.060 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.060 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.060 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.061 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.061 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.061 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.061 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.061 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-15T23:12:11.061 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.061 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.061 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.061 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.061 INFO:teuthology.orchestra.run.smithi111.stdout: "use_agent": { 2024-09-15T23:12:11.061 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "use_agent", 2024-09-15T23:12:11.061 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.061 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.062 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.062 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.062 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.062 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.062 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.062 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-15T23:12:11.062 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.062 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.062 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.062 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.062 INFO:teuthology.orchestra.run.smithi111.stdout: "use_repo_digest": { 2024-09-15T23:12:11.062 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "use_repo_digest", 2024-09-15T23:12:11.062 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.062 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.063 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.063 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.063 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.063 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.063 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.063 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-15T23:12:11.063 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.063 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.063 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.063 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.063 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_on_failed_host_check": { 2024-09-15T23:12:11.063 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_on_failed_host_check", 2024-09-15T23:12:11.063 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.064 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.064 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.064 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.064 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.064 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.064 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.064 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "raise a health warning if the host check fails", 2024-09-15T23:12:11.064 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.064 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.064 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.064 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.064 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_on_stray_daemons": { 2024-09-15T23:12:11.064 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_on_stray_daemons", 2024-09-15T23:12:11.064 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.065 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.065 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.065 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.065 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.065 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.065 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.065 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-15T23:12:11.065 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.065 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.065 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.065 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.065 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_on_stray_hosts": { 2024-09-15T23:12:11.065 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_on_stray_hosts", 2024-09-15T23:12:11.065 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.066 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.066 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.066 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.066 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.066 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.066 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.066 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-15T23:12:11.066 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.066 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.066 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.066 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.066 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.066 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.066 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.067 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crash", 2024-09-15T23:12:11.067 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.067 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.067 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.067 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.067 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.067 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.067 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.067 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.067 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.067 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.067 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.067 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.067 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.068 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.068 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.068 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.068 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.068 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.068 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.068 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.068 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.068 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.068 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.068 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.068 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.068 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.069 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.069 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.069 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.069 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.069 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.069 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.069 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.069 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.069 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.069 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.069 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.069 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.069 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.070 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.070 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.070 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.070 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.070 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.070 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.070 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.070 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.070 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.070 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.070 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.070 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.070 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.070 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.071 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.071 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.071 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.071 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.071 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.071 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.071 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.071 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.071 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.071 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.071 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.071 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.071 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.072 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.072 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.072 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.072 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.072 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.072 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.072 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.072 INFO:teuthology.orchestra.run.smithi111.stdout: "retain_interval": { 2024-09-15T23:12:11.072 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "retain_interval", 2024-09-15T23:12:11.072 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.072 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.072 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.072 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "31536000", 2024-09-15T23:12:11.072 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.073 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.073 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.073 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-15T23:12:11.073 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.073 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.073 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.073 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.073 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_recent_interval": { 2024-09-15T23:12:11.073 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_recent_interval", 2024-09-15T23:12:11.073 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.073 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.073 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.073 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "1209600", 2024-09-15T23:12:11.074 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.074 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.074 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.074 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-15T23:12:11.074 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.074 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.074 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.074 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.074 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.074 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.074 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.074 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "dashboard", 2024-09-15T23:12:11.074 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.075 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.075 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.075 INFO:teuthology.orchestra.run.smithi111.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-15T23:12:11.075 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-15T23:12:11.075 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.075 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.075 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.075 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-09-15T23:12:11.075 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.075 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.075 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.075 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.075 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.076 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.076 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.076 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.076 INFO:teuthology.orchestra.run.smithi111.stdout: "ALERTMANAGER_API_HOST": { 2024-09-15T23:12:11.076 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-15T23:12:11.076 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.076 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.076 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.076 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.076 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.076 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.076 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.076 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.076 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.076 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.077 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.077 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.077 INFO:teuthology.orchestra.run.smithi111.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-15T23:12:11.077 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-15T23:12:11.077 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.077 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.077 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.077 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.077 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.077 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.078 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.078 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.078 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.078 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.078 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.078 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.078 INFO:teuthology.orchestra.run.smithi111.stdout: "AUDIT_API_ENABLED": { 2024-09-15T23:12:11.078 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "AUDIT_API_ENABLED", 2024-09-15T23:12:11.078 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.078 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.078 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.078 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.079 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.079 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.079 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.079 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.079 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.079 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.079 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.079 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.079 INFO:teuthology.orchestra.run.smithi111.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-15T23:12:11.079 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-15T23:12:11.079 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.079 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.079 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.079 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.080 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.080 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.080 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.080 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.080 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.080 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.080 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.080 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.080 INFO:teuthology.orchestra.run.smithi111.stdout: "ENABLE_BROWSABLE_API": { 2024-09-15T23:12:11.080 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-15T23:12:11.080 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.081 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.081 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.081 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.081 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.081 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.081 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.081 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.081 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.081 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.081 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.082 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.082 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-15T23:12:11.082 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-15T23:12:11.082 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.082 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.082 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.082 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.082 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.082 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.082 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.082 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.082 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.083 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.083 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.083 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.083 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-15T23:12:11.083 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-15T23:12:11.083 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.083 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.083 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.083 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.083 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.083 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.084 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.084 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.084 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.084 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.084 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.084 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.084 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-15T23:12:11.084 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-15T23:12:11.084 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.084 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.084 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.084 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.085 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.085 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.085 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.085 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.085 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.085 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.085 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.085 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.085 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-15T23:12:11.085 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-15T23:12:11.085 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.086 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.086 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.086 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.086 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.086 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.086 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.086 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.086 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.086 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.086 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.086 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.087 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-15T23:12:11.087 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-15T23:12:11.087 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.087 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.087 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.087 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.087 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.087 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.087 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.087 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.087 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.088 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.088 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.088 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.088 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-15T23:12:11.088 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-15T23:12:11.088 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.088 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.088 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.088 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.088 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.088 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.089 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.089 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.089 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.089 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.089 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.089 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.089 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-15T23:12:11.089 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-15T23:12:11.089 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.089 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.089 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.090 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.090 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.090 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.090 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.090 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.090 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.090 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.090 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.090 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.090 INFO:teuthology.orchestra.run.smithi111.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-15T23:12:11.090 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-15T23:12:11.091 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.091 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.091 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.091 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.091 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.091 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.091 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.091 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.091 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.091 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.091 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.092 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.092 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_API_PASSWORD": { 2024-09-15T23:12:11.092 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-15T23:12:11.092 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.092 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.092 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.092 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "admin", 2024-09-15T23:12:11.092 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.092 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.092 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.092 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.093 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.093 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.093 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.093 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.093 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-15T23:12:11.093 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-15T23:12:11.093 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.093 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.093 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.093 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.093 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.094 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.094 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.094 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.094 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.094 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.094 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.094 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.094 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_API_URL": { 2024-09-15T23:12:11.094 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_API_URL", 2024-09-15T23:12:11.094 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.094 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.094 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.095 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.095 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.095 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.095 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.095 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.095 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.095 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.095 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.095 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.095 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_API_USERNAME": { 2024-09-15T23:12:11.095 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-15T23:12:11.095 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.095 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.096 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.096 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "admin", 2024-09-15T23:12:11.096 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.096 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.096 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.096 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.096 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.096 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.096 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.096 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.096 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-15T23:12:11.097 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-15T23:12:11.097 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.097 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.097 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.097 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.097 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.097 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.097 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.097 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.097 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.097 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.097 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.098 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.098 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-15T23:12:11.098 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-15T23:12:11.098 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.098 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.098 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.098 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.098 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.098 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.098 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.099 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.099 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.099 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.099 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.099 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.099 INFO:teuthology.orchestra.run.smithi111.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-15T23:12:11.099 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-15T23:12:11.099 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.099 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.099 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.099 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.099 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.100 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.100 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.100 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.100 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.100 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.100 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.100 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.100 INFO:teuthology.orchestra.run.smithi111.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-15T23:12:11.100 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-15T23:12:11.100 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.100 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.101 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.101 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.101 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.101 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.101 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.101 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.101 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.101 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.101 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.101 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.101 INFO:teuthology.orchestra.run.smithi111.stdout: "PROMETHEUS_API_HOST": { 2024-09-15T23:12:11.101 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-15T23:12:11.102 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.102 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.102 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.102 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.102 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.102 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.102 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.102 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.102 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.102 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.102 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.103 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.103 INFO:teuthology.orchestra.run.smithi111.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-15T23:12:11.103 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-15T23:12:11.103 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.103 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.103 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.103 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.103 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.103 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.103 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.103 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.104 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.104 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.104 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.104 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.104 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-15T23:12:11.104 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-15T23:12:11.104 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.104 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.104 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.104 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.104 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.105 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.105 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.105 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.105 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.105 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.105 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.105 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.105 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-15T23:12:11.105 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-15T23:12:11.105 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.105 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.105 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.106 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.106 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.106 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.106 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.106 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.106 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.106 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.106 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.106 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.106 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-15T23:12:11.106 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-15T23:12:11.106 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.107 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.107 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.107 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.107 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.107 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.107 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.107 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.107 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.107 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.107 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.107 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.108 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-15T23:12:11.108 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-15T23:12:11.108 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.108 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.108 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.108 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.108 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.108 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.108 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.108 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.108 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.108 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.109 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.109 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.109 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-15T23:12:11.109 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-15T23:12:11.109 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.109 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.109 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.109 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.109 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.109 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.109 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.110 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.110 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.110 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.110 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.110 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.110 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-15T23:12:11.110 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-15T23:12:11.110 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.110 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.110 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.110 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.111 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.111 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.111 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.111 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.111 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.111 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.111 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.111 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.111 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-15T23:12:11.111 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-15T23:12:11.111 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.111 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.112 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.112 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.112 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.112 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.112 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.112 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.112 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.112 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.112 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.112 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.112 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_ENABLED": { 2024-09-15T23:12:11.112 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-15T23:12:11.113 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.113 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.113 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.113 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.113 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.113 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.113 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.113 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.113 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.113 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.113 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.113 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.114 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-15T23:12:11.114 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-15T23:12:11.114 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.114 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.114 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.114 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-15T23:12:11.114 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.114 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.114 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.114 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.114 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.114 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.115 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.115 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.115 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-15T23:12:11.115 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-15T23:12:11.115 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.115 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.115 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.115 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-09-15T23:12:11.115 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.115 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.115 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.116 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.116 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.116 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.116 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.116 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.116 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-15T23:12:11.116 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-15T23:12:11.116 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.116 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.116 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.116 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8", 2024-09-15T23:12:11.117 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.117 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.117 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.117 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.117 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.117 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.117 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.117 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.117 INFO:teuthology.orchestra.run.smithi111.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-15T23:12:11.117 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-15T23:12:11.117 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.117 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.118 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.118 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "45", 2024-09-15T23:12:11.118 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.118 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.118 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.118 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.118 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.118 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.118 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.118 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.118 INFO:teuthology.orchestra.run.smithi111.stdout: "RGW_API_ACCESS_KEY": { 2024-09-15T23:12:11.118 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-15T23:12:11.118 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.119 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.119 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.119 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.119 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.119 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.119 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.119 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.119 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.119 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.119 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.119 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.119 INFO:teuthology.orchestra.run.smithi111.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-15T23:12:11.120 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-15T23:12:11.120 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.120 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.120 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.120 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "admin", 2024-09-15T23:12:11.120 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.120 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.120 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.120 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.120 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.120 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.121 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.121 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.121 INFO:teuthology.orchestra.run.smithi111.stdout: "RGW_API_SECRET_KEY": { 2024-09-15T23:12:11.121 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-15T23:12:11.121 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.121 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.121 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.121 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.121 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.121 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.121 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.121 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.122 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.122 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.122 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.122 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.122 INFO:teuthology.orchestra.run.smithi111.stdout: "RGW_API_SSL_VERIFY": { 2024-09-15T23:12:11.122 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-15T23:12:11.122 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.122 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.122 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.122 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.122 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.122 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.123 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.123 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.123 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.123 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.123 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.123 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.123 INFO:teuthology.orchestra.run.smithi111.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-15T23:12:11.123 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-15T23:12:11.123 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.123 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.123 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.124 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-09-15T23:12:11.124 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.124 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.124 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.124 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.124 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.124 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.124 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.124 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.124 INFO:teuthology.orchestra.run.smithi111.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-15T23:12:11.124 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-15T23:12:11.124 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.125 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.125 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.125 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-09-15T23:12:11.125 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.125 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.125 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.125 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.125 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.125 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.125 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.125 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.126 INFO:teuthology.orchestra.run.smithi111.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-15T23:12:11.126 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-15T23:12:11.126 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.126 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.126 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.126 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5", 2024-09-15T23:12:11.126 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.126 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.126 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.126 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.126 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.126 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.126 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.127 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.127 INFO:teuthology.orchestra.run.smithi111.stdout: "cross_origin_url": { 2024-09-15T23:12:11.127 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cross_origin_url", 2024-09-15T23:12:11.127 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.127 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.127 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.127 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.127 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.127 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.127 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.127 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.128 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.128 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.128 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.128 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.128 INFO:teuthology.orchestra.run.smithi111.stdout: "crt_file": { 2024-09-15T23:12:11.128 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crt_file", 2024-09-15T23:12:11.128 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.128 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.128 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.128 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.128 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.128 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.129 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.129 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.129 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.129 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.129 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.129 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.129 INFO:teuthology.orchestra.run.smithi111.stdout: "debug": { 2024-09-15T23:12:11.129 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "debug", 2024-09-15T23:12:11.129 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.129 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.129 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.129 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.130 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.130 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.130 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.130 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Enable/disable debug options", 2024-09-15T23:12:11.130 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.130 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.130 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.130 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.130 INFO:teuthology.orchestra.run.smithi111.stdout: "jwt_token_ttl": { 2024-09-15T23:12:11.130 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "jwt_token_ttl", 2024-09-15T23:12:11.130 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.131 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.131 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.131 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "28800", 2024-09-15T23:12:11.131 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.131 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.131 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.131 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.131 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.131 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.131 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.131 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.131 INFO:teuthology.orchestra.run.smithi111.stdout: "key_file": { 2024-09-15T23:12:11.132 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "key_file", 2024-09-15T23:12:11.132 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.132 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.132 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.132 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.132 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.132 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.132 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.132 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.132 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.132 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.132 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.133 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.133 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.133 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.133 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.133 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.133 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.133 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.133 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.133 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.133 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.133 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.133 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.134 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.134 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.134 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.134 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.134 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.134 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.134 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.134 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.134 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.134 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.134 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.134 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.135 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.135 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.135 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.135 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.135 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.135 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.135 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.135 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.135 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.135 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.135 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.135 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.136 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.136 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.136 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.136 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.136 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.136 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.136 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.136 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.136 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.136 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.137 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.137 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.137 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.137 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.137 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.137 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.137 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.137 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.137 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.137 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.137 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.138 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.138 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.138 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.138 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.138 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.138 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.138 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.138 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.138 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.138 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.138 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.138 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.139 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.139 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.139 INFO:teuthology.orchestra.run.smithi111.stdout: "motd": { 2024-09-15T23:12:11.139 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "motd", 2024-09-15T23:12:11.139 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.139 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.139 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.139 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.139 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.139 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.139 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.140 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "The message of the day", 2024-09-15T23:12:11.140 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.140 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.140 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.140 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.140 INFO:teuthology.orchestra.run.smithi111.stdout: "redirect_resolve_ip_addr": { 2024-09-15T23:12:11.140 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "redirect_resolve_ip_addr", 2024-09-15T23:12:11.140 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.140 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.140 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.140 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.141 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.141 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.141 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.141 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.141 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.141 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.141 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.141 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.141 INFO:teuthology.orchestra.run.smithi111.stdout: "server_addr": { 2024-09-15T23:12:11.141 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_addr", 2024-09-15T23:12:11.142 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.142 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.142 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.142 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "::", 2024-09-15T23:12:11.142 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.142 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.142 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.142 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.142 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.142 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.142 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.143 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.143 INFO:teuthology.orchestra.run.smithi111.stdout: "server_port": { 2024-09-15T23:12:11.143 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_port", 2024-09-15T23:12:11.143 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.143 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.143 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.143 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8080", 2024-09-15T23:12:11.143 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.143 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.143 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.143 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.143 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.144 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.144 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.144 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.144 INFO:teuthology.orchestra.run.smithi111.stdout: "ssl": { 2024-09-15T23:12:11.144 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ssl", 2024-09-15T23:12:11.144 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.144 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.144 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.144 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.144 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.144 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.145 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.145 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.145 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.145 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.145 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.145 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.145 INFO:teuthology.orchestra.run.smithi111.stdout: "ssl_server_port": { 2024-09-15T23:12:11.145 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ssl_server_port", 2024-09-15T23:12:11.145 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.145 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.145 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.145 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8443", 2024-09-15T23:12:11.146 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.146 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.146 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.146 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.146 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.146 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.146 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.146 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.146 INFO:teuthology.orchestra.run.smithi111.stdout: "standby_behaviour": { 2024-09-15T23:12:11.146 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "standby_behaviour", 2024-09-15T23:12:11.146 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.147 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.147 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.147 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "redirect", 2024-09-15T23:12:11.147 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.147 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.147 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.147 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.147 INFO:teuthology.orchestra.run.smithi111.stdout: "redirect" 2024-09-15T23:12:11.147 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.147 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.147 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.148 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.148 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.148 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.148 INFO:teuthology.orchestra.run.smithi111.stdout: "standby_error_status_code": { 2024-09-15T23:12:11.148 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "standby_error_status_code", 2024-09-15T23:12:11.148 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.148 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.148 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.148 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "500", 2024-09-15T23:12:11.148 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "400", 2024-09-15T23:12:11.148 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "599", 2024-09-15T23:12:11.149 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.149 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.149 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.149 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.149 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.149 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.149 INFO:teuthology.orchestra.run.smithi111.stdout: "url_prefix": { 2024-09-15T23:12:11.149 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "url_prefix", 2024-09-15T23:12:11.149 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.149 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.149 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.149 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.150 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.150 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.150 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.150 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.150 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.150 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.150 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.150 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.150 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.150 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.150 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.150 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "devicehealth", 2024-09-15T23:12:11.151 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.151 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.151 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.151 INFO:teuthology.orchestra.run.smithi111.stdout: "enable_monitoring": { 2024-09-15T23:12:11.151 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "enable_monitoring", 2024-09-15T23:12:11.151 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.151 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.151 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.151 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.151 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.151 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.151 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.152 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "monitor device health metrics", 2024-09-15T23:12:11.152 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.152 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.152 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.152 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.152 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.152 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.152 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.152 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.152 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.152 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.153 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.153 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.153 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.153 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.153 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.153 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.153 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.153 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.153 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.153 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.153 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.153 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.154 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.154 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.154 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.154 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.154 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.154 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.154 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.154 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.154 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.154 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.154 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.154 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.155 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.155 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.155 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.155 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.155 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.155 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.155 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.155 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.155 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.155 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.156 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.156 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.156 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.156 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.156 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.156 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.156 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.156 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.156 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.156 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.156 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.157 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.157 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.157 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.157 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.157 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.157 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.157 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.157 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.157 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.157 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.157 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.158 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.158 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.158 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.158 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.158 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.158 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.158 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.158 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.158 INFO:teuthology.orchestra.run.smithi111.stdout: "mark_out_threshold": { 2024-09-15T23:12:11.158 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mark_out_threshold", 2024-09-15T23:12:11.158 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.158 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.159 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.159 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "2419200", 2024-09-15T23:12:11.159 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.159 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.159 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.159 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-15T23:12:11.159 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.159 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.159 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.159 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.159 INFO:teuthology.orchestra.run.smithi111.stdout: "pool_name": { 2024-09-15T23:12:11.159 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "pool_name", 2024-09-15T23:12:11.159 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.160 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.160 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.160 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "device_health_metrics", 2024-09-15T23:12:11.160 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.160 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.160 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.160 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-15T23:12:11.160 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.160 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.160 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.160 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.160 INFO:teuthology.orchestra.run.smithi111.stdout: "retention_period": { 2024-09-15T23:12:11.160 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "retention_period", 2024-09-15T23:12:11.161 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.161 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.161 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.161 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "15552000", 2024-09-15T23:12:11.161 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.161 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.161 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.161 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how long to retain device health metrics", 2024-09-15T23:12:11.161 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.161 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.161 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.161 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.162 INFO:teuthology.orchestra.run.smithi111.stdout: "scrape_frequency": { 2024-09-15T23:12:11.162 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "scrape_frequency", 2024-09-15T23:12:11.162 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.162 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.162 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.162 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "86400", 2024-09-15T23:12:11.162 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.162 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.162 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.162 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-15T23:12:11.162 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.162 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.163 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.163 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.163 INFO:teuthology.orchestra.run.smithi111.stdout: "self_heal": { 2024-09-15T23:12:11.163 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "self_heal", 2024-09-15T23:12:11.163 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.163 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.163 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.163 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.163 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.163 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.163 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.164 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-15T23:12:11.164 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.164 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.164 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.164 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.164 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-09-15T23:12:11.164 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-09-15T23:12:11.164 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.164 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.164 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.164 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-09-15T23:12:11.165 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.165 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.165 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.165 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to wake up and check device health", 2024-09-15T23:12:11.165 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.165 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.165 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.165 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.165 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_threshold": { 2024-09-15T23:12:11.165 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_threshold", 2024-09-15T23:12:11.165 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.166 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.166 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.166 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "7257600", 2024-09-15T23:12:11.166 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.166 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.166 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.166 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-15T23:12:11.166 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.166 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.166 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.167 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.167 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.167 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.167 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.167 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "diskprediction_local", 2024-09-15T23:12:11.167 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.167 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.167 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.167 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.167 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.167 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.167 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.168 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.168 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.168 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.168 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.168 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.168 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.168 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.168 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.168 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.168 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.168 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.169 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.169 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.169 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.169 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.169 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.169 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.169 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.169 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.169 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.169 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.169 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.169 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.170 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.170 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.170 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.170 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.170 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.170 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.170 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.170 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.170 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.170 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.170 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.170 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.171 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.171 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.171 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.171 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.171 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.171 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.171 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.171 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.171 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.171 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.172 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.172 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.172 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.172 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.172 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.172 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.172 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.172 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.172 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.172 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.172 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.172 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.173 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.173 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.173 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.173 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.173 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.173 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.173 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.173 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.173 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.173 INFO:teuthology.orchestra.run.smithi111.stdout: "predict_interval": { 2024-09-15T23:12:11.173 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "predict_interval", 2024-09-15T23:12:11.173 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.174 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.174 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.174 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "86400", 2024-09-15T23:12:11.174 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.174 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.174 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.174 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.174 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.174 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.174 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.174 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.175 INFO:teuthology.orchestra.run.smithi111.stdout: "predictor_model": { 2024-09-15T23:12:11.175 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "predictor_model", 2024-09-15T23:12:11.175 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.175 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.175 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.175 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "prophetstor", 2024-09-15T23:12:11.175 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.175 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.175 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.175 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.175 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.175 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.176 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.176 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.176 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-09-15T23:12:11.176 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-09-15T23:12:11.176 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.176 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.176 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.176 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-09-15T23:12:11.176 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.176 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.176 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.177 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.177 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.177 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.177 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.177 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.177 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.177 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.177 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.177 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "influx", 2024-09-15T23:12:11.177 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": false, 2024-09-15T23:12:11.177 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "influxdb python module not found", 2024-09-15T23:12:11.177 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.178 INFO:teuthology.orchestra.run.smithi111.stdout: "batch_size": { 2024-09-15T23:12:11.178 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "batch_size", 2024-09-15T23:12:11.178 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.178 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.178 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.178 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5000", 2024-09-15T23:12:11.178 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.178 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.178 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.178 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-15T23:12:11.178 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.179 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.179 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.179 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.179 INFO:teuthology.orchestra.run.smithi111.stdout: "database": { 2024-09-15T23:12:11.179 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "database", 2024-09-15T23:12:11.179 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.179 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.179 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.179 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "ceph", 2024-09-15T23:12:11.179 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.179 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.180 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.180 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-15T23:12:11.180 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.180 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.180 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.180 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.180 INFO:teuthology.orchestra.run.smithi111.stdout: "hostname": { 2024-09-15T23:12:11.180 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "hostname", 2024-09-15T23:12:11.180 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.180 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.180 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.180 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.181 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.181 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.181 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.181 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "InfluxDB server hostname", 2024-09-15T23:12:11.181 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.181 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.181 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.181 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.181 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-09-15T23:12:11.181 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-09-15T23:12:11.181 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.181 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.182 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.182 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "30", 2024-09-15T23:12:11.182 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "5", 2024-09-15T23:12:11.182 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.182 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.182 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-15T23:12:11.182 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.182 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.182 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.182 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.183 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.183 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.183 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.183 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.183 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.183 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.183 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.183 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.183 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.183 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.183 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.183 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.184 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.184 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.184 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.184 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.184 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.184 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.184 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.184 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.184 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.184 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.184 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.184 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.185 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.185 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.185 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.185 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.185 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.185 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.185 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.185 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.185 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.185 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.185 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.186 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.186 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.186 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.186 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.186 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.186 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.186 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.186 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.186 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.186 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.186 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.187 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.187 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.187 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.187 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.187 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.187 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.187 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.187 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.187 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.187 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.187 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.187 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.188 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.188 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.188 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.188 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.188 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.188 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.188 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.188 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.188 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.188 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.188 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.188 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.188 INFO:teuthology.orchestra.run.smithi111.stdout: "password": { 2024-09-15T23:12:11.189 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "password", 2024-09-15T23:12:11.189 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.189 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.189 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.189 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.189 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.189 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.189 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.189 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "password of InfluxDB server user", 2024-09-15T23:12:11.189 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.189 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.189 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.190 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.190 INFO:teuthology.orchestra.run.smithi111.stdout: "port": { 2024-09-15T23:12:11.190 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "port", 2024-09-15T23:12:11.190 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.190 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.190 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.190 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8086", 2024-09-15T23:12:11.190 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.190 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.190 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.190 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "InfluxDB server port", 2024-09-15T23:12:11.190 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.190 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.191 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.191 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.191 INFO:teuthology.orchestra.run.smithi111.stdout: "ssl": { 2024-09-15T23:12:11.191 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ssl", 2024-09-15T23:12:11.191 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.191 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.191 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.191 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "false", 2024-09-15T23:12:11.191 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.191 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.191 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.192 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-15T23:12:11.192 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.192 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.192 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.192 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.192 INFO:teuthology.orchestra.run.smithi111.stdout: "threads": { 2024-09-15T23:12:11.192 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "threads", 2024-09-15T23:12:11.192 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.192 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.192 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.192 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5", 2024-09-15T23:12:11.192 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1", 2024-09-15T23:12:11.193 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "32", 2024-09-15T23:12:11.193 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.193 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-15T23:12:11.193 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.193 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.193 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.193 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.193 INFO:teuthology.orchestra.run.smithi111.stdout: "username": { 2024-09-15T23:12:11.193 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "username", 2024-09-15T23:12:11.193 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.193 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.194 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.194 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.194 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.194 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.194 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.194 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "username of InfluxDB server user", 2024-09-15T23:12:11.194 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.194 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.194 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.194 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.194 INFO:teuthology.orchestra.run.smithi111.stdout: "verify_ssl": { 2024-09-15T23:12:11.194 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "verify_ssl", 2024-09-15T23:12:11.194 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.195 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.195 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.195 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "true", 2024-09-15T23:12:11.195 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.195 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.195 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.195 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-15T23:12:11.195 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.195 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.195 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.195 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.196 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.196 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.196 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.196 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "insights", 2024-09-15T23:12:11.196 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.196 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.196 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.196 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.196 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.196 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.196 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.197 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.197 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.197 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.197 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.197 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.197 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.197 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.197 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.197 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.197 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.197 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.198 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.198 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.198 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.198 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.198 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.198 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.198 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.198 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.198 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.198 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.198 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.198 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.199 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.199 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.199 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.199 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.199 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.199 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.199 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.199 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.199 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.199 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.199 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.199 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.200 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.200 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.200 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.200 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.200 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.200 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.200 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.200 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.200 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.200 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.200 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.201 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.201 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.201 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.201 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.201 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.201 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.201 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.201 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.201 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.201 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.201 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.201 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.202 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.202 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.202 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.202 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.202 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.202 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.202 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.202 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.202 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.202 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.202 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.202 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "iostat", 2024-09-15T23:12:11.203 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.203 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.203 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.203 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.203 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.203 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.203 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.203 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.203 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.203 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.203 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.203 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.203 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.204 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.204 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.204 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.204 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.204 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.204 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.204 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.204 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.204 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.204 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.204 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.205 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.205 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.205 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.205 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.205 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.205 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.205 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.205 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.205 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.205 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.205 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.206 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.206 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.206 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.206 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.206 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.206 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.206 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.206 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.206 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.206 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.206 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.206 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.207 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.207 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.207 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.207 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.207 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.207 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.207 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.207 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.207 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.207 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.207 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.207 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.208 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.208 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.208 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.208 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.208 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.208 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.208 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.208 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.208 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.208 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.208 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.208 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.209 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.209 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.209 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.209 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.209 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.209 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "k8sevents", 2024-09-15T23:12:11.209 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.209 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.209 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.209 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_event_retention_days": { 2024-09-15T23:12:11.209 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ceph_event_retention_days", 2024-09-15T23:12:11.210 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.210 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.210 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.210 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "7", 2024-09-15T23:12:11.210 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.210 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.210 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.210 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-15T23:12:11.210 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.210 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.210 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.210 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.211 INFO:teuthology.orchestra.run.smithi111.stdout: "config_check_secs": { 2024-09-15T23:12:11.211 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "config_check_secs", 2024-09-15T23:12:11.211 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.211 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.211 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.211 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-09-15T23:12:11.211 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "10", 2024-09-15T23:12:11.211 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.211 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.211 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-15T23:12:11.211 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.212 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.212 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.212 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.212 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.212 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.212 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.212 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.212 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.212 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.212 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.212 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.213 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.213 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.213 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.213 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.213 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.213 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.213 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.213 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.213 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.213 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.213 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.213 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.214 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.214 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.214 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.214 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.214 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.214 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.214 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.214 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.214 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.214 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.214 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.215 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.215 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.215 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.215 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.215 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.215 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.215 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.215 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.215 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.215 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.215 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.216 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.216 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.216 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.216 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.216 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.216 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.216 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.216 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.216 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.216 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.216 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.217 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.217 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.217 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.217 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.217 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.217 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.217 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.217 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.217 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.217 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.217 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.217 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.218 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.218 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.218 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.218 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.218 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.218 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.218 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.218 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.218 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "localpool", 2024-09-15T23:12:11.218 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.218 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.218 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.219 INFO:teuthology.orchestra.run.smithi111.stdout: "failure_domain": { 2024-09-15T23:12:11.219 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "failure_domain", 2024-09-15T23:12:11.219 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.219 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.219 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.219 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "host", 2024-09-15T23:12:11.219 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.219 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.219 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.219 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "failure domain for any created local pool", 2024-09-15T23:12:11.219 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-15T23:12:11.220 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.220 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.220 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.220 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.220 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.220 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.220 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.220 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.220 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.220 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.221 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.221 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.221 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.221 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.221 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.221 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.221 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.221 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.221 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.221 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.221 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.221 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.222 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.222 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.222 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.222 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.222 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.222 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.222 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.222 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.222 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.222 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.222 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.223 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.223 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.223 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.223 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.223 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.223 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.223 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.223 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.223 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.223 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.223 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.223 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.224 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.224 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.224 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.224 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.224 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.224 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.224 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.224 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.224 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.224 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.224 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.224 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.225 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.225 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.225 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.225 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.225 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.225 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.225 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.225 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.225 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.225 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.225 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.226 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.226 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.226 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.226 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.226 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.226 INFO:teuthology.orchestra.run.smithi111.stdout: "min_size": { 2024-09-15T23:12:11.226 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "min_size", 2024-09-15T23:12:11.226 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.226 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.226 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.226 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.226 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.227 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.227 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.227 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "default min_size for any created local pool", 2024-09-15T23:12:11.227 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-15T23:12:11.227 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.227 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.227 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.227 INFO:teuthology.orchestra.run.smithi111.stdout: "num_rep": { 2024-09-15T23:12:11.227 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "num_rep", 2024-09-15T23:12:11.227 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.227 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.228 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.228 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "3", 2024-09-15T23:12:11.228 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.228 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.228 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.228 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "default replica count for any created local pool", 2024-09-15T23:12:11.228 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.228 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.228 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.228 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.228 INFO:teuthology.orchestra.run.smithi111.stdout: "pg_num": { 2024-09-15T23:12:11.229 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "pg_num", 2024-09-15T23:12:11.229 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.229 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.229 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.229 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "128", 2024-09-15T23:12:11.229 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.229 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.229 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.229 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "default pg_num for any created local pool", 2024-09-15T23:12:11.229 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.229 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.230 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.230 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.230 INFO:teuthology.orchestra.run.smithi111.stdout: "prefix": { 2024-09-15T23:12:11.230 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "prefix", 2024-09-15T23:12:11.230 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.230 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.230 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.230 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.230 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.230 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.230 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.231 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "name prefix for any created local pool", 2024-09-15T23:12:11.231 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.231 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.231 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.231 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.231 INFO:teuthology.orchestra.run.smithi111.stdout: "subtree": { 2024-09-15T23:12:11.231 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "subtree", 2024-09-15T23:12:11.231 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.231 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.231 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.231 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "rack", 2024-09-15T23:12:11.232 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.232 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.232 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.232 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-15T23:12:11.232 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-15T23:12:11.232 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.232 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.232 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.232 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.232 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.232 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.233 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mds_autoscaler", 2024-09-15T23:12:11.233 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.233 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.233 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.233 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.233 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.233 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.233 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.233 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.233 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.233 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.234 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.234 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.234 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.234 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.234 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.234 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.234 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.234 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.234 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.234 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.234 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.235 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.235 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.235 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.235 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.235 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.235 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.235 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.235 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.235 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.235 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.235 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.235 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.235 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.236 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.236 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.236 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.236 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.236 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.236 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.236 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.236 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.236 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.236 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.236 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.237 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.237 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.237 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.237 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.237 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.237 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.237 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.237 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.237 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.237 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.237 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.238 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.238 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.238 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.238 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.238 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.238 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.238 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.238 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.238 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.238 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.238 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.239 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.239 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.239 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.239 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.239 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.239 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.239 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.239 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.239 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.239 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mirroring", 2024-09-15T23:12:11.239 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.239 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.239 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.239 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.240 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.240 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.240 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.240 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.240 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.240 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.240 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.240 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.240 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.240 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.240 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.240 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.241 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.241 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.241 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.241 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.241 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.241 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.241 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.241 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.241 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.241 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.241 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.242 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.242 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.242 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.242 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.242 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.242 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.242 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.242 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.242 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.242 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.242 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.242 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.242 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.243 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.243 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.243 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.243 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.243 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.243 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.243 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.243 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.243 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.243 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.243 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.243 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.244 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.244 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.244 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.244 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.244 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.244 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.244 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.244 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.244 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.244 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.244 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.244 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.245 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.245 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.245 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.245 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.245 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.245 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.245 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.245 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.245 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.245 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.245 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.245 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.246 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "nfs", 2024-09-15T23:12:11.246 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.246 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.246 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.246 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.246 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.246 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.246 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.246 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.246 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.246 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.247 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.247 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.247 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.247 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.247 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.247 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.247 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.247 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.247 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.247 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.247 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.247 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.248 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.248 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.248 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.248 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.248 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.248 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.248 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.248 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.248 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.248 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.248 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.249 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.249 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.249 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.249 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.249 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.249 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.249 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.249 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.249 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.249 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.249 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.250 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.250 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.250 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.250 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.250 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.250 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.250 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.250 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.250 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.250 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.250 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.250 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.251 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.251 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.251 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.251 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.251 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.251 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.251 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.251 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.251 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.251 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.251 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.251 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.251 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.251 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.252 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.252 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.252 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.252 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.252 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.252 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.252 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "orchestrator", 2024-09-15T23:12:11.252 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.252 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.252 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.252 INFO:teuthology.orchestra.run.smithi111.stdout: "fail_fs": { 2024-09-15T23:12:11.252 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "fail_fs", 2024-09-15T23:12:11.253 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.253 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.253 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.253 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.253 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.253 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.253 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.253 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-15T23:12:11.253 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.253 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.253 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.253 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.254 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.254 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.254 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.254 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.254 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.254 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.254 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.254 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.254 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.254 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.254 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.254 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.255 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.255 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.255 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.255 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.255 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.255 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.255 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.255 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.255 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.255 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.255 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.255 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.256 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.256 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.256 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.256 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.256 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.256 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.256 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.256 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.256 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.256 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.256 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.256 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.256 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.257 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.257 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.257 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.257 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.257 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.257 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.257 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.257 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.257 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.257 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.257 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.257 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.258 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.258 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.258 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.258 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.258 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.258 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.258 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.258 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.258 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.258 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.258 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.258 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.258 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.259 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.259 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.259 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.259 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.259 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.259 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.259 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.259 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.259 INFO:teuthology.orchestra.run.smithi111.stdout: "orchestrator": { 2024-09-15T23:12:11.259 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "orchestrator", 2024-09-15T23:12:11.259 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.259 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.260 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.260 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.260 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.260 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.260 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.260 INFO:teuthology.orchestra.run.smithi111.stdout: "cephadm", 2024-09-15T23:12:11.260 INFO:teuthology.orchestra.run.smithi111.stdout: "rook", 2024-09-15T23:12:11.260 INFO:teuthology.orchestra.run.smithi111.stdout: "test_orchestrator" 2024-09-15T23:12:11.260 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.260 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Orchestrator backend", 2024-09-15T23:12:11.260 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.260 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.261 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.261 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.261 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.261 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.261 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.261 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "osd_perf_query", 2024-09-15T23:12:11.261 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.261 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.261 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.261 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.261 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.262 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.262 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.262 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.262 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.262 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.262 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.262 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.262 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.262 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.262 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.262 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.262 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.263 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.263 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.263 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.263 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.263 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.263 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.263 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.263 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.263 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.263 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.263 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.263 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.264 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.264 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.264 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.264 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.264 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.264 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.264 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.264 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.264 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.264 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.264 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.265 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.265 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.265 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.265 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.265 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.265 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.265 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.265 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.265 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.265 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.265 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.265 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.266 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.266 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.266 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.266 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.266 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.266 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.266 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.266 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.266 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.266 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.266 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.266 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.267 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.267 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.267 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.267 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.267 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.267 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.267 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.267 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.267 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.267 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.267 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.267 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.268 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "osd_support", 2024-09-15T23:12:11.268 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.268 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.268 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.268 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.268 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.268 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.268 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.268 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.268 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.268 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.268 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.269 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.269 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.269 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.269 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.269 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.269 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.269 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.269 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.269 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.269 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.269 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.269 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.270 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.270 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.270 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.270 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.270 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.270 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.270 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.270 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.270 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.270 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.270 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.271 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.271 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.271 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.271 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.271 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.271 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.271 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.271 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.271 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.271 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.271 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.271 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.272 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.272 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.272 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.272 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.272 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.272 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.272 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.272 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.272 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.272 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.272 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.273 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.273 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.273 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.273 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.273 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.273 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.273 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.273 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.273 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.273 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.273 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.274 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.274 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.274 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.274 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.274 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.274 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.274 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.274 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.274 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "pg_autoscaler", 2024-09-15T23:12:11.274 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.274 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.274 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.275 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.275 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.275 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.275 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.275 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.275 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.275 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.275 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.275 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.275 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.275 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.275 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.276 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.276 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.276 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.276 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.276 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.276 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.276 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.276 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.276 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.276 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.276 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.277 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.277 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.277 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.277 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.277 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.277 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.277 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.277 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.277 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.277 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.277 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.278 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.278 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.278 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.278 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.278 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.278 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.278 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.278 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.278 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.278 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.279 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.279 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.279 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.279 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.279 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.279 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.279 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.279 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.279 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.279 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.279 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.279 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.280 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.280 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.280 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.280 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.280 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.280 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.280 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.280 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.280 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.280 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.281 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.281 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.281 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.281 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.281 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-09-15T23:12:11.281 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-09-15T23:12:11.281 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.281 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.281 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.281 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-09-15T23:12:11.281 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.281 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.282 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.282 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.282 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.282 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.282 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.282 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.282 INFO:teuthology.orchestra.run.smithi111.stdout: "threshold": { 2024-09-15T23:12:11.282 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "threshold", 2024-09-15T23:12:11.282 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-09-15T23:12:11.282 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.282 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.282 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "3.0", 2024-09-15T23:12:11.283 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1.0", 2024-09-15T23:12:11.283 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.283 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.283 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "scaling threshold", 2024-09-15T23:12:11.283 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-15T23:12:11.283 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.283 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.283 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.283 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.283 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.283 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.284 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "progress", 2024-09-15T23:12:11.284 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.284 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.284 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.284 INFO:teuthology.orchestra.run.smithi111.stdout: "allow_pg_recovery_event": { 2024-09-15T23:12:11.284 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "allow_pg_recovery_event", 2024-09-15T23:12:11.284 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.284 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.284 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.284 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.284 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.284 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.285 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.285 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-15T23:12:11.285 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.285 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.285 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.285 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.285 INFO:teuthology.orchestra.run.smithi111.stdout: "enabled": { 2024-09-15T23:12:11.285 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "enabled", 2024-09-15T23:12:11.285 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.285 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.285 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.286 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.286 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.286 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.286 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.286 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.286 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.286 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.286 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.286 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.286 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.286 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.286 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.287 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.287 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.287 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.287 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.287 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.287 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.287 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.287 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.287 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.287 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.287 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.287 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.288 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.288 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.288 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.288 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.288 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.288 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.288 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.288 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.288 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.288 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.288 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.288 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.289 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.289 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.289 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.289 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.289 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.289 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.289 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.289 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.289 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.289 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.289 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.290 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.290 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.290 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.290 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.290 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.290 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.290 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.290 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.290 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.290 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.290 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.291 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.291 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.291 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.291 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.291 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.291 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.291 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.291 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.291 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.291 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.291 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.292 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.292 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.292 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.292 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.292 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.292 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.292 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.292 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.292 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.292 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.292 INFO:teuthology.orchestra.run.smithi111.stdout: "max_completed_events": { 2024-09-15T23:12:11.293 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_completed_events", 2024-09-15T23:12:11.293 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.293 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.293 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.293 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "50", 2024-09-15T23:12:11.293 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.293 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.293 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.293 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "number of past completed events to remember", 2024-09-15T23:12:11.293 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.293 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.293 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.294 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.294 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-09-15T23:12:11.294 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-09-15T23:12:11.294 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.294 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.294 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.294 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5", 2024-09-15T23:12:11.294 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.294 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.294 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.294 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how long the module is going to sleep", 2024-09-15T23:12:11.295 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.295 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.295 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.295 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.295 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.295 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.295 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.295 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "prometheus", 2024-09-15T23:12:11.295 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.295 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.295 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.295 INFO:teuthology.orchestra.run.smithi111.stdout: "cache": { 2024-09-15T23:12:11.296 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cache", 2024-09-15T23:12:11.296 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.296 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.296 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.296 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.296 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.296 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.296 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.296 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.296 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.296 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.297 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.297 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.297 INFO:teuthology.orchestra.run.smithi111.stdout: "exclude_perf_counters": { 2024-09-15T23:12:11.297 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "exclude_perf_counters", 2024-09-15T23:12:11.297 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.297 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.297 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.297 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.297 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.297 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.297 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.297 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-15T23:12:11.298 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-15T23:12:11.298 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.298 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.298 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.298 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.298 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.298 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.298 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.298 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.298 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.298 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.298 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.299 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.299 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.299 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.299 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.299 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.299 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.299 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.299 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.299 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.299 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.299 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.299 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.300 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.300 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.300 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.300 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.300 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.300 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.300 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.300 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.300 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.300 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.300 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.301 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.301 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.301 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.301 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.301 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.301 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.301 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.301 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.301 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.301 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.301 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.301 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.302 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.302 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.302 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.302 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.302 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.302 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.302 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.302 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.302 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.302 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.302 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.302 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.303 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.303 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.303 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.303 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.303 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.303 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.303 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.303 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.303 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.303 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.303 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.303 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.304 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.304 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.304 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.304 INFO:teuthology.orchestra.run.smithi111.stdout: "rbd_stats_pools": { 2024-09-15T23:12:11.304 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rbd_stats_pools", 2024-09-15T23:12:11.304 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.304 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.304 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.304 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.304 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.304 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.304 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.305 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.305 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.305 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.305 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.305 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.305 INFO:teuthology.orchestra.run.smithi111.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-15T23:12:11.305 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-15T23:12:11.305 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.305 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.305 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.305 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "300", 2024-09-15T23:12:11.306 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.306 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.306 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.306 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.306 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.306 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.306 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.306 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.306 INFO:teuthology.orchestra.run.smithi111.stdout: "scrape_interval": { 2024-09-15T23:12:11.306 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "scrape_interval", 2024-09-15T23:12:11.306 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-09-15T23:12:11.306 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.307 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.307 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "15.0", 2024-09-15T23:12:11.307 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.307 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.307 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.307 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.307 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.307 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.307 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.307 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.307 INFO:teuthology.orchestra.run.smithi111.stdout: "server_addr": { 2024-09-15T23:12:11.307 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_addr", 2024-09-15T23:12:11.308 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.308 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.308 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.308 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "::", 2024-09-15T23:12:11.308 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.308 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.308 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.308 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-15T23:12:11.308 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.308 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.308 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.309 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.309 INFO:teuthology.orchestra.run.smithi111.stdout: "server_port": { 2024-09-15T23:12:11.309 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_port", 2024-09-15T23:12:11.309 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.309 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.309 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.309 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "9283", 2024-09-15T23:12:11.309 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.309 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.309 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.309 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-15T23:12:11.309 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.309 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.310 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.310 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.310 INFO:teuthology.orchestra.run.smithi111.stdout: "stale_cache_strategy": { 2024-09-15T23:12:11.310 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "stale_cache_strategy", 2024-09-15T23:12:11.310 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.310 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.310 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.310 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "log", 2024-09-15T23:12:11.310 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.310 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.310 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.310 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.311 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.311 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.311 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.311 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.311 INFO:teuthology.orchestra.run.smithi111.stdout: "standby_behaviour": { 2024-09-15T23:12:11.311 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "standby_behaviour", 2024-09-15T23:12:11.311 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.311 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.311 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.311 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "default", 2024-09-15T23:12:11.311 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.312 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.312 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.312 INFO:teuthology.orchestra.run.smithi111.stdout: "default", 2024-09-15T23:12:11.312 INFO:teuthology.orchestra.run.smithi111.stdout: "error" 2024-09-15T23:12:11.312 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.312 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.312 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.312 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.312 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.312 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.312 INFO:teuthology.orchestra.run.smithi111.stdout: "standby_error_status_code": { 2024-09-15T23:12:11.313 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "standby_error_status_code", 2024-09-15T23:12:11.313 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.313 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.313 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.313 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "500", 2024-09-15T23:12:11.313 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "400", 2024-09-15T23:12:11.313 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "599", 2024-09-15T23:12:11.313 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.313 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.313 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.313 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.313 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.314 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.314 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.314 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.314 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.314 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rbd_support", 2024-09-15T23:12:11.314 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.314 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.314 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.314 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.314 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.314 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.314 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.315 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.315 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.315 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.315 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.315 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.315 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.315 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.315 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.315 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.315 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.315 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.315 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.316 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.316 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.316 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.316 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.316 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.316 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.316 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.316 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.316 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.316 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.316 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.317 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.317 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.317 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.317 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.317 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.317 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.317 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.317 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.317 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.317 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.317 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.318 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.318 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.318 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.318 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.318 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.318 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.318 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.318 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.318 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.318 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.318 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.318 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.319 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.319 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.319 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.319 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.319 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.319 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.319 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.319 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.319 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.319 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.319 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.319 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.320 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.320 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.320 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.320 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.320 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.320 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.320 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.320 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.320 INFO:teuthology.orchestra.run.smithi111.stdout: "max_concurrent_snap_create": { 2024-09-15T23:12:11.320 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_concurrent_snap_create", 2024-09-15T23:12:11.320 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.321 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.321 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.321 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-09-15T23:12:11.321 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.321 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.321 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.321 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.321 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.321 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.321 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.321 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.321 INFO:teuthology.orchestra.run.smithi111.stdout: "mirror_snapshot_schedule": { 2024-09-15T23:12:11.322 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mirror_snapshot_schedule", 2024-09-15T23:12:11.322 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.322 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.322 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.322 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.322 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.322 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.322 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.322 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.322 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.322 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.322 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.323 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.323 INFO:teuthology.orchestra.run.smithi111.stdout: "trash_purge_schedule": { 2024-09-15T23:12:11.323 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "trash_purge_schedule", 2024-09-15T23:12:11.323 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.323 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.323 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.323 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.323 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.323 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.323 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.323 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.324 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.324 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.324 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.324 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.324 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.324 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.324 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.324 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "restful", 2024-09-15T23:12:11.324 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.324 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.324 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.325 INFO:teuthology.orchestra.run.smithi111.stdout: "enable_auth": { 2024-09-15T23:12:11.325 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "enable_auth", 2024-09-15T23:12:11.325 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.325 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.325 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.325 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.325 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.325 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.325 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.325 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.325 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.326 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.326 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.326 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.326 INFO:teuthology.orchestra.run.smithi111.stdout: "key_file": { 2024-09-15T23:12:11.326 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "key_file", 2024-09-15T23:12:11.326 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.326 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.326 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.326 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.326 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.326 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.326 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.327 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.327 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.327 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.327 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.327 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.327 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.327 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.327 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.327 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.328 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.328 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.328 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.328 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.328 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.328 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.328 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.328 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.328 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.328 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.328 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.329 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.329 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.329 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.329 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.329 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.329 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.329 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.329 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.329 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.329 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.329 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.330 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.330 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.330 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.330 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.330 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.330 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.330 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.330 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.330 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.330 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.330 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.331 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.331 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.331 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.331 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.331 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.331 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.331 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.331 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.331 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.331 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.331 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.332 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.332 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.332 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.332 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.332 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.332 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.332 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.332 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.332 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.332 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.333 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.333 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.333 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.333 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.333 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.333 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.333 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.333 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.333 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.333 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.333 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.334 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.334 INFO:teuthology.orchestra.run.smithi111.stdout: "server_addr": { 2024-09-15T23:12:11.334 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_addr", 2024-09-15T23:12:11.334 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.334 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.334 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.334 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.334 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.334 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.334 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.334 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.335 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.335 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.335 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.335 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.335 INFO:teuthology.orchestra.run.smithi111.stdout: "server_port": { 2024-09-15T23:12:11.335 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_port", 2024-09-15T23:12:11.335 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.335 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.335 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.335 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.335 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.336 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.336 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.336 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.336 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.336 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.336 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.336 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.336 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.336 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.336 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.336 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rgw", 2024-09-15T23:12:11.337 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.337 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.337 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.337 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.337 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.337 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.337 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.337 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.337 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.337 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.337 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.337 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.338 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.338 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.338 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.338 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.338 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.338 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.338 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.338 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.338 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.338 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.338 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.339 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.339 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.339 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.339 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.339 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.339 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.339 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.339 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.339 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.339 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.339 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.339 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.340 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.340 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.340 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.340 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.340 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.340 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.340 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.340 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.340 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.340 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.340 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.340 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.341 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.341 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.341 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.341 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.341 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.341 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.341 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.341 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.341 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.341 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.341 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.341 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.341 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.342 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.342 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.342 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.342 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.342 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.342 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.342 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.342 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.342 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.342 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.342 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.342 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.343 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.343 INFO:teuthology.orchestra.run.smithi111.stdout: "secondary_zone_period_retry_limit": { 2024-09-15T23:12:11.343 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-15T23:12:11.343 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.343 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.343 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.343 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5", 2024-09-15T23:12:11.343 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.343 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.343 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.343 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-15T23:12:11.343 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.343 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.344 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.344 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.344 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.344 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.344 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.344 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rook", 2024-09-15T23:12:11.344 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.344 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.344 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.344 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.344 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.345 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.345 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.345 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.345 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.345 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.345 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.345 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.345 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.345 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.345 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.345 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.346 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.346 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.346 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.346 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.346 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.346 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.346 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.346 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.346 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.346 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.346 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.347 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.347 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.347 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.347 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.347 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.347 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.347 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.347 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.347 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.347 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.347 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.348 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.348 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.348 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.348 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.348 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.348 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.348 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.348 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.348 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.348 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.348 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.348 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.349 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.349 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.349 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.349 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.349 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.349 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.349 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.349 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.349 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.349 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.349 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.350 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.350 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.350 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.350 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.350 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.350 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.350 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.350 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.350 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.350 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.350 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.350 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.350 INFO:teuthology.orchestra.run.smithi111.stdout: "storage_class": { 2024-09-15T23:12:11.351 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "storage_class", 2024-09-15T23:12:11.351 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.351 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.351 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.351 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "local", 2024-09-15T23:12:11.351 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.351 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.351 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.351 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-15T23:12:11.351 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.351 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.351 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.352 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.352 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.352 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.352 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.352 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "selftest", 2024-09-15T23:12:11.352 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.352 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.352 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.352 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.352 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.352 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.352 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.352 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.353 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.353 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.353 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.353 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.353 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.353 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.353 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.353 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.353 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.353 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.353 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.353 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.354 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.354 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.354 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.354 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.354 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.354 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.354 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.354 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.354 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.354 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.354 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.354 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.355 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.355 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.355 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.355 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.355 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.355 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.355 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.355 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.355 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.355 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.355 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.355 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.355 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.356 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.356 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.356 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.356 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.356 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.356 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.356 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.356 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.356 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.356 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.356 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.356 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.356 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.357 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.357 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.357 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.357 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.357 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.357 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.357 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.357 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.357 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.357 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.357 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.357 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.357 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.358 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.358 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.358 INFO:teuthology.orchestra.run.smithi111.stdout: "roption1": { 2024-09-15T23:12:11.358 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "roption1", 2024-09-15T23:12:11.358 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.358 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.358 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.358 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.358 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.358 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.358 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.358 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.359 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.359 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.359 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.359 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.359 INFO:teuthology.orchestra.run.smithi111.stdout: "roption2": { 2024-09-15T23:12:11.359 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "roption2", 2024-09-15T23:12:11.359 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.359 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.359 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.359 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "xyz", 2024-09-15T23:12:11.359 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.359 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.360 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.360 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.360 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.360 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.360 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.360 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.360 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption1": { 2024-09-15T23:12:11.360 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption1", 2024-09-15T23:12:11.360 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.360 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.360 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.361 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.361 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.361 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.361 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.361 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.361 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.361 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.361 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.361 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.361 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption2": { 2024-09-15T23:12:11.362 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption2", 2024-09-15T23:12:11.362 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.362 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.362 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.362 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.362 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.362 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.362 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.362 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.362 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.362 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.363 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.363 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.363 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption3": { 2024-09-15T23:12:11.363 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption3", 2024-09-15T23:12:11.363 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-09-15T23:12:11.363 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.363 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.363 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.363 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.363 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.363 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.364 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.364 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.364 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.364 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.364 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.364 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption4": { 2024-09-15T23:12:11.364 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption4", 2024-09-15T23:12:11.364 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.364 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.364 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.364 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.365 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.365 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.365 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.365 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.365 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.365 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.365 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.365 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.365 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption5": { 2024-09-15T23:12:11.365 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption5", 2024-09-15T23:12:11.365 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.366 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.366 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.366 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.366 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.366 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.366 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.366 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.366 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.366 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.366 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.366 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.367 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption6": { 2024-09-15T23:12:11.367 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption6", 2024-09-15T23:12:11.367 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.367 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.367 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.367 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.367 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.367 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.367 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.367 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.367 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.368 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.368 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.368 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.368 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption7": { 2024-09-15T23:12:11.368 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption7", 2024-09-15T23:12:11.368 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.368 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.368 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.368 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.368 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1", 2024-09-15T23:12:11.369 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "42", 2024-09-15T23:12:11.369 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.369 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.369 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.369 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.369 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.369 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.369 INFO:teuthology.orchestra.run.smithi111.stdout: "testkey": { 2024-09-15T23:12:11.369 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "testkey", 2024-09-15T23:12:11.369 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.369 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.369 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.370 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.370 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.370 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.370 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.370 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.370 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.370 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.370 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.370 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.370 INFO:teuthology.orchestra.run.smithi111.stdout: "testlkey": { 2024-09-15T23:12:11.370 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "testlkey", 2024-09-15T23:12:11.370 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.371 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.371 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.371 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.371 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.371 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.371 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.371 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.371 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.371 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.371 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.371 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.372 INFO:teuthology.orchestra.run.smithi111.stdout: "testnewline": { 2024-09-15T23:12:11.372 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "testnewline", 2024-09-15T23:12:11.372 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.372 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.372 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.372 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.372 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.372 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.372 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.372 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.372 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.373 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.373 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.373 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.373 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.373 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.373 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.373 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "snap_schedule", 2024-09-15T23:12:11.373 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.373 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.373 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.373 INFO:teuthology.orchestra.run.smithi111.stdout: "allow_m_granularity": { 2024-09-15T23:12:11.374 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "allow_m_granularity", 2024-09-15T23:12:11.374 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.374 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.374 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.374 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.374 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.374 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.374 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.374 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "allow minute scheduled snapshots", 2024-09-15T23:12:11.374 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.374 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.375 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.375 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.375 INFO:teuthology.orchestra.run.smithi111.stdout: "dump_on_update": { 2024-09-15T23:12:11.375 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "dump_on_update", 2024-09-15T23:12:11.375 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.375 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.375 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.375 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.375 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.375 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.375 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.376 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "dump database to debug log on update", 2024-09-15T23:12:11.376 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.376 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.376 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.376 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.376 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.376 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.376 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.376 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.376 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.376 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.376 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.376 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.377 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.377 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.377 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.377 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.377 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.377 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.377 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.377 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.377 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.377 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.377 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.377 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.378 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.378 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.378 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.378 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.378 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.378 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.378 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.378 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.378 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.378 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.378 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.379 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.379 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.379 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.379 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.379 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.379 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.379 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.379 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.379 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.379 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.379 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.380 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.380 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.380 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.380 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.380 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.380 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.380 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.380 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.380 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.380 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.380 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.380 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.381 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.381 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.381 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.381 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.381 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.381 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.381 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.381 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.381 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.381 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.381 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.382 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.382 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.382 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.382 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.382 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.382 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.382 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.382 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.382 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "stats", 2024-09-15T23:12:11.382 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.382 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.383 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.383 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.383 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.383 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.383 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.383 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.383 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.383 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.383 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.383 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.383 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.383 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.384 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.384 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.384 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.384 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.384 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.384 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.384 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.384 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.384 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.384 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.384 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.384 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.385 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.385 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.385 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.385 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.385 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.385 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.385 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.385 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.385 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.385 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.385 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.386 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.386 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.386 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.386 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.386 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.386 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.386 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.386 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.386 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.386 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.386 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.387 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.387 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.387 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.387 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.387 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.387 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.387 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.387 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.387 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.387 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.387 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.387 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.388 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.388 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.388 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.388 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.388 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.388 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.388 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.388 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.388 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.388 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.388 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.389 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.389 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.389 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.389 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.389 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.389 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "status", 2024-09-15T23:12:11.389 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.389 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.389 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.389 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.389 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.390 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.390 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.390 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.390 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.390 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.390 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.390 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.390 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.390 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.390 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.390 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.391 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.391 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.391 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.391 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.391 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.391 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.391 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.391 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.391 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.391 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.391 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.391 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.391 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.392 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.392 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.392 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.392 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.392 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.392 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.392 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.392 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.392 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.392 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.392 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.392 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.392 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.393 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.393 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.393 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.393 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.393 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.393 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.393 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.393 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.393 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.393 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.394 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.394 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.394 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.394 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.394 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.394 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.394 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.394 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.394 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.394 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.394 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.394 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.394 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.395 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.395 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.395 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.395 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.395 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.395 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.395 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.395 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.395 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.395 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.395 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.395 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "telegraf", 2024-09-15T23:12:11.396 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.396 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.396 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.396 INFO:teuthology.orchestra.run.smithi111.stdout: "address": { 2024-09-15T23:12:11.396 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "address", 2024-09-15T23:12:11.396 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.396 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.396 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.396 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-15T23:12:11.396 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.396 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.397 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.397 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.397 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.397 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.397 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.397 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.397 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-09-15T23:12:11.397 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-09-15T23:12:11.397 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.397 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.397 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.397 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "15", 2024-09-15T23:12:11.398 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.398 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.398 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.398 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.398 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.398 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.398 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.398 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.398 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.398 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.399 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.399 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.399 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.399 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.399 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.399 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.399 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.399 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.399 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.399 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.399 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.399 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.400 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.400 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.400 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.400 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.400 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.400 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.400 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.400 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.400 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.400 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.400 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.401 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.401 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.401 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.401 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.401 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.401 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.401 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.401 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.401 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.401 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.401 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.402 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.402 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.402 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.402 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.402 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.402 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.402 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.402 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.402 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.402 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.402 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.402 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.403 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.403 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.403 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.403 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.403 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.403 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.403 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.403 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.403 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.403 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.403 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.403 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.404 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.404 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.404 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.404 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.404 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.404 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.404 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.404 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.404 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.404 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.404 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.404 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.405 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.405 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "telemetry", 2024-09-15T23:12:11.405 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.405 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.405 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.405 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_basic": { 2024-09-15T23:12:11.405 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_basic", 2024-09-15T23:12:11.405 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.405 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.405 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.405 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.405 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.406 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.406 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.406 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-15T23:12:11.406 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.406 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.406 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.406 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.406 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_crash": { 2024-09-15T23:12:11.406 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_crash", 2024-09-15T23:12:11.406 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.406 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.406 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.407 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.407 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.407 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.407 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.407 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-15T23:12:11.407 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.407 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.407 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.407 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.407 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_device": { 2024-09-15T23:12:11.407 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_device", 2024-09-15T23:12:11.407 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.408 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.408 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.408 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.408 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.408 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.408 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.408 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-15T23:12:11.408 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.408 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.408 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.408 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.409 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_ident": { 2024-09-15T23:12:11.409 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_ident", 2024-09-15T23:12:11.409 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.409 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.409 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.409 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.409 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.409 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.409 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.409 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-15T23:12:11.409 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.409 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.410 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.410 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.410 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_perf": { 2024-09-15T23:12:11.410 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_perf", 2024-09-15T23:12:11.410 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.410 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.410 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.410 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.410 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.410 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.410 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.410 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-15T23:12:11.411 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.411 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.411 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.411 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.411 INFO:teuthology.orchestra.run.smithi111.stdout: "contact": { 2024-09-15T23:12:11.411 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "contact", 2024-09-15T23:12:11.411 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.411 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.411 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.411 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.411 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.411 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.412 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.412 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.412 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.412 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.412 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.412 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.412 INFO:teuthology.orchestra.run.smithi111.stdout: "description": { 2024-09-15T23:12:11.412 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "description", 2024-09-15T23:12:11.412 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.412 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.413 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.413 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.413 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.413 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.413 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.413 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.413 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.413 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.413 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.413 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.413 INFO:teuthology.orchestra.run.smithi111.stdout: "device_url": { 2024-09-15T23:12:11.413 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "device_url", 2024-09-15T23:12:11.414 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.414 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.414 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.414 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-15T23:12:11.414 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.414 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.414 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.414 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.414 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.414 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.414 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.415 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.415 INFO:teuthology.orchestra.run.smithi111.stdout: "enabled": { 2024-09-15T23:12:11.415 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "enabled", 2024-09-15T23:12:11.415 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.415 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.415 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.415 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.415 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.415 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.415 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.415 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.416 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.416 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.416 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.416 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.416 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-09-15T23:12:11.416 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-09-15T23:12:11.416 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.416 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.416 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.416 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "24", 2024-09-15T23:12:11.416 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "8", 2024-09-15T23:12:11.416 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.417 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.417 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.417 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.417 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.417 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.417 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.417 INFO:teuthology.orchestra.run.smithi111.stdout: "last_opt_revision": { 2024-09-15T23:12:11.417 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "last_opt_revision", 2024-09-15T23:12:11.417 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.417 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.417 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.418 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "1", 2024-09-15T23:12:11.418 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.418 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.418 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.418 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.418 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.418 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.418 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.418 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.418 INFO:teuthology.orchestra.run.smithi111.stdout: "leaderboard": { 2024-09-15T23:12:11.418 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "leaderboard", 2024-09-15T23:12:11.419 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.419 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.419 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.419 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.419 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.419 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.419 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.419 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.419 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.419 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.419 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.419 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.420 INFO:teuthology.orchestra.run.smithi111.stdout: "leaderboard_description": { 2024-09-15T23:12:11.420 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "leaderboard_description", 2024-09-15T23:12:11.420 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.420 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.420 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.420 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.420 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.420 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.420 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.420 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.421 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.421 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.421 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.421 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.421 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.421 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.421 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.421 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.421 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.421 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.421 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.422 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.422 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.422 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.422 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.422 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.422 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.422 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.422 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.422 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.422 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.422 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.423 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.423 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.423 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.423 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.423 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.423 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.423 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.423 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.423 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.423 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.423 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.423 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.424 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.424 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.424 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.424 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.424 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.424 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.424 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.424 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.424 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.424 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.424 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.424 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.425 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.425 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.425 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.425 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.425 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.425 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.425 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.425 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.425 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.425 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.426 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.426 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.426 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.426 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.426 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.426 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.426 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.426 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.426 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.426 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.426 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.427 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.427 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.427 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.427 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.427 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.427 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.427 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.427 INFO:teuthology.orchestra.run.smithi111.stdout: "organization": { 2024-09-15T23:12:11.427 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "organization", 2024-09-15T23:12:11.427 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.427 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.428 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.428 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.428 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.428 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.428 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.428 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.428 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.428 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.428 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.428 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.428 INFO:teuthology.orchestra.run.smithi111.stdout: "proxy": { 2024-09-15T23:12:11.428 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "proxy", 2024-09-15T23:12:11.429 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.429 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.429 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.429 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.429 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.429 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.429 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.429 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.429 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.429 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.430 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.430 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.430 INFO:teuthology.orchestra.run.smithi111.stdout: "url": { 2024-09-15T23:12:11.430 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "url", 2024-09-15T23:12:11.430 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.430 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.430 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.430 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-15T23:12:11.430 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.430 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.430 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.431 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.431 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.431 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.431 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.431 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.431 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.431 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.431 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.431 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "test_orchestrator", 2024-09-15T23:12:11.431 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.431 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.431 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.432 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.432 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.432 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.432 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.432 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.432 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.432 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.432 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.432 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.432 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.432 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.433 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.433 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.433 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.433 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.433 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.433 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.433 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.433 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.433 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.433 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.433 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.434 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.434 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.434 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.434 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.434 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.434 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.434 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.434 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.434 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.434 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.434 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.435 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.435 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.435 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.435 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.435 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.435 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.435 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.435 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.435 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.435 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.435 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.436 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.436 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.436 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.436 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.436 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.436 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.436 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.436 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.436 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.436 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.436 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.437 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.437 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.437 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.437 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.437 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.437 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.437 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.437 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.437 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.437 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.437 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.438 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.438 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.438 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.438 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.438 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.438 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.438 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.438 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "volumes", 2024-09-15T23:12:11.438 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.438 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.438 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.438 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.439 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.439 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.439 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.439 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.439 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.439 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.439 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.439 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.439 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.439 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.439 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.440 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.440 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.440 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.440 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.440 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.440 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.440 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.440 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.440 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.440 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.440 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.440 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.441 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.441 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.441 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.441 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.441 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.441 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.441 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.441 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.441 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.441 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.441 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.442 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.442 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.442 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.442 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.442 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.442 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.442 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.442 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.442 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.442 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.442 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.442 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.443 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.443 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.443 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.443 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.443 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.443 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.443 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.443 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.443 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.443 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.443 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.443 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.444 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.444 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.444 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.444 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.444 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.444 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.444 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.444 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.444 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.444 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.444 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.444 INFO:teuthology.orchestra.run.smithi111.stdout: "max_concurrent_clones": { 2024-09-15T23:12:11.445 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_concurrent_clones", 2024-09-15T23:12:11.445 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.445 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.445 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.445 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "4", 2024-09-15T23:12:11.445 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.445 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.445 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.445 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-15T23:12:11.445 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.445 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.445 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.446 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.446 INFO:teuthology.orchestra.run.smithi111.stdout: "periodic_async_work": { 2024-09-15T23:12:11.446 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "periodic_async_work", 2024-09-15T23:12:11.446 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.446 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.446 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.446 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.446 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.446 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.446 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.446 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Periodically check for async work", 2024-09-15T23:12:11.446 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.447 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.447 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.447 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.447 INFO:teuthology.orchestra.run.smithi111.stdout: "snapshot_clone_delay": { 2024-09-15T23:12:11.447 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "snapshot_clone_delay", 2024-09-15T23:12:11.447 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.447 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.447 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.447 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-09-15T23:12:11.447 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.447 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.447 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.448 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-15T23:12:11.448 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.448 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.448 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.448 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.448 INFO:teuthology.orchestra.run.smithi111.stdout: "snapshot_clone_no_wait": { 2024-09-15T23:12:11.448 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "snapshot_clone_no_wait", 2024-09-15T23:12:11.448 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.448 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.448 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.448 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.449 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.449 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.449 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.449 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-15T23:12:11.449 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.449 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.449 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.449 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.449 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.449 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.449 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.450 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "zabbix", 2024-09-15T23:12:11.450 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.450 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.450 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.450 INFO:teuthology.orchestra.run.smithi111.stdout: "discovery_interval": { 2024-09-15T23:12:11.450 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "discovery_interval", 2024-09-15T23:12:11.450 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-09-15T23:12:11.450 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.450 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.450 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "100", 2024-09-15T23:12:11.450 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.451 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.451 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.451 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.451 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.451 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.451 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.451 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.451 INFO:teuthology.orchestra.run.smithi111.stdout: "identifier": { 2024-09-15T23:12:11.451 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "identifier", 2024-09-15T23:12:11.451 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.451 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.452 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.452 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.452 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.452 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.452 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.452 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.452 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.452 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.452 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.452 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.452 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-09-15T23:12:11.453 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-09-15T23:12:11.453 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.453 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.453 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.453 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-09-15T23:12:11.453 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.453 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.453 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.453 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.453 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.453 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.453 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.454 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.454 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.454 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.454 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.454 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.454 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.454 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.454 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.454 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.454 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.454 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.455 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.455 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.455 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.455 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.455 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.455 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.455 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.455 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.455 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.455 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.455 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.456 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.456 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.456 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.456 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.456 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.456 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.456 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.456 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.456 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.456 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.456 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.456 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.457 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.457 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.457 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.457 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.457 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.457 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.457 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.457 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.457 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.457 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.457 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.458 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.458 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.458 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.458 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.458 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.458 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.458 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.458 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.458 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.458 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.458 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.458 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.459 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.459 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.459 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.459 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.459 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.459 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.459 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.459 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.459 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.459 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.459 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.460 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.460 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.460 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.460 INFO:teuthology.orchestra.run.smithi111.stdout: "zabbix_host": { 2024-09-15T23:12:11.460 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "zabbix_host", 2024-09-15T23:12:11.460 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.460 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.460 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.460 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.460 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.460 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.460 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.460 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.461 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.461 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.461 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.461 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.461 INFO:teuthology.orchestra.run.smithi111.stdout: "zabbix_port": { 2024-09-15T23:12:11.461 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "zabbix_port", 2024-09-15T23:12:11.461 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.461 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.461 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.461 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10051", 2024-09-15T23:12:11.461 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.461 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.461 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.462 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.462 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.462 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.462 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.462 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.462 INFO:teuthology.orchestra.run.smithi111.stdout: "zabbix_sender": { 2024-09-15T23:12:11.462 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "zabbix_sender", 2024-09-15T23:12:11.462 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.462 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.462 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.462 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-15T23:12:11.463 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.463 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.463 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.463 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.463 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.463 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.463 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.463 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.463 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.463 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.463 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-15T23:12:11.463 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.463 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.464 INFO:teuthology.orchestra.run.smithi111.stdout: "gid": 24116, 2024-09-15T23:12:11.464 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "y", 2024-09-15T23:12:11.464 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr_features": 4540701547738038271, 2024-09-15T23:12:11.464 INFO:teuthology.orchestra.run.smithi111.stdout: "available_modules": [ 2024-09-15T23:12:11.464 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.464 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "alerts", 2024-09-15T23:12:11.464 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.464 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.464 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.464 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-09-15T23:12:11.464 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-09-15T23:12:11.464 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.465 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.465 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.465 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-09-15T23:12:11.465 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.465 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.465 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.465 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "How frequently to reexamine health status", 2024-09-15T23:12:11.465 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.465 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.465 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.465 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.465 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.466 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.466 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.466 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.466 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.466 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.466 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.466 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.466 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.466 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.466 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.466 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.466 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.466 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.467 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.467 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.467 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.467 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.467 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.467 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.467 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.467 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.467 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.467 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.467 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.467 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.467 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.468 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.468 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.468 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.468 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.468 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.468 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.468 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.468 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.468 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.468 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.468 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.468 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.469 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.469 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.469 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.469 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.469 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.469 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.469 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.469 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.469 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.469 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.469 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.470 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.470 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.470 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.470 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.470 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.470 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.470 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.470 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.470 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.470 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.470 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.470 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.471 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.471 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.471 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.471 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.471 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.471 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.471 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.471 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.471 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_destination": { 2024-09-15T23:12:11.471 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_destination", 2024-09-15T23:12:11.471 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.471 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.472 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.472 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.472 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.472 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.472 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.472 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Email address to send alerts to", 2024-09-15T23:12:11.472 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.472 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.472 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.472 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.472 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_from_name": { 2024-09-15T23:12:11.473 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_from_name", 2024-09-15T23:12:11.473 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.473 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.473 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.473 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "Ceph", 2024-09-15T23:12:11.473 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.473 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.473 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.473 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Email From: name", 2024-09-15T23:12:11.473 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.473 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.473 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.474 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.474 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_host": { 2024-09-15T23:12:11.474 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_host", 2024-09-15T23:12:11.474 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.474 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.474 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.474 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.474 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.474 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.474 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.474 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "SMTP server", 2024-09-15T23:12:11.474 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.475 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.475 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.475 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.475 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_password": { 2024-09-15T23:12:11.475 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_password", 2024-09-15T23:12:11.475 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.475 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.475 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.475 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.475 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.475 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.475 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.476 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Password to authenticate with", 2024-09-15T23:12:11.476 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.476 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.476 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.476 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.476 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_port": { 2024-09-15T23:12:11.476 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_port", 2024-09-15T23:12:11.476 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.476 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.476 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.476 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "465", 2024-09-15T23:12:11.477 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.477 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.477 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.477 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "SMTP port", 2024-09-15T23:12:11.477 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.477 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.477 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.477 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.477 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_sender": { 2024-09-15T23:12:11.477 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_sender", 2024-09-15T23:12:11.477 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.477 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.478 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.478 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.478 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.478 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.478 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.478 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "SMTP envelope sender", 2024-09-15T23:12:11.478 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.478 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.478 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.478 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.478 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_ssl": { 2024-09-15T23:12:11.478 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_ssl", 2024-09-15T23:12:11.479 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.479 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.479 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.479 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.479 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.479 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.479 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.479 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-15T23:12:11.479 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.479 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.479 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.480 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.480 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_user": { 2024-09-15T23:12:11.480 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_user", 2024-09-15T23:12:11.480 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.480 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.480 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.480 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.480 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.480 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.480 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.480 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "User to authenticate as", 2024-09-15T23:12:11.480 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.481 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.481 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.481 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.481 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.481 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.481 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.481 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "balancer", 2024-09-15T23:12:11.481 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.481 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.481 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.481 INFO:teuthology.orchestra.run.smithi111.stdout: "active": { 2024-09-15T23:12:11.482 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "active", 2024-09-15T23:12:11.482 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.482 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.482 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.482 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.482 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.482 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.482 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.482 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "automatically balance PGs across cluster", 2024-09-15T23:12:11.482 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.482 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.482 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.483 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.483 INFO:teuthology.orchestra.run.smithi111.stdout: "begin_time": { 2024-09-15T23:12:11.483 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "begin_time", 2024-09-15T23:12:11.483 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.483 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.483 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.483 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0000", 2024-09-15T23:12:11.483 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.483 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.483 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.483 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "beginning time of day to automatically balance", 2024-09-15T23:12:11.483 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-15T23:12:11.484 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.484 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.484 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.484 INFO:teuthology.orchestra.run.smithi111.stdout: "begin_weekday": { 2024-09-15T23:12:11.484 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "begin_weekday", 2024-09-15T23:12:11.484 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-09-15T23:12:11.484 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.484 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.484 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-09-15T23:12:11.484 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "0", 2024-09-15T23:12:11.485 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "6", 2024-09-15T23:12:11.485 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.485 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-15T23:12:11.485 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-15T23:12:11.485 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.485 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.485 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.485 INFO:teuthology.orchestra.run.smithi111.stdout: "crush_compat_max_iterations": { 2024-09-15T23:12:11.485 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crush_compat_max_iterations", 2024-09-15T23:12:11.485 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-09-15T23:12:11.485 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.486 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.486 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "25", 2024-09-15T23:12:11.486 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1", 2024-09-15T23:12:11.486 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "250", 2024-09-15T23:12:11.486 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.486 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-15T23:12:11.486 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.486 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.486 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.486 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.486 INFO:teuthology.orchestra.run.smithi111.stdout: "crush_compat_metrics": { 2024-09-15T23:12:11.487 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crush_compat_metrics", 2024-09-15T23:12:11.487 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.487 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.487 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.487 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "pgs,objects,bytes", 2024-09-15T23:12:11.487 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.487 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.487 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.487 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-15T23:12:11.487 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-15T23:12:11.487 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.487 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.488 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.488 INFO:teuthology.orchestra.run.smithi111.stdout: "crush_compat_step": { 2024-09-15T23:12:11.488 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crush_compat_step", 2024-09-15T23:12:11.488 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-09-15T23:12:11.488 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.488 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.488 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0.5", 2024-09-15T23:12:11.488 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "0.001", 2024-09-15T23:12:11.488 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "0.999", 2024-09-15T23:12:11.488 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.488 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "aggressiveness of optimization", 2024-09-15T23:12:11.489 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-15T23:12:11.489 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.489 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.489 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.489 INFO:teuthology.orchestra.run.smithi111.stdout: "end_time": { 2024-09-15T23:12:11.489 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "end_time", 2024-09-15T23:12:11.489 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.489 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.489 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.489 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "2359", 2024-09-15T23:12:11.489 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.490 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.490 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.490 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "ending time of day to automatically balance", 2024-09-15T23:12:11.490 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-15T23:12:11.490 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.490 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.490 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.490 INFO:teuthology.orchestra.run.smithi111.stdout: "end_weekday": { 2024-09-15T23:12:11.490 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "end_weekday", 2024-09-15T23:12:11.490 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-09-15T23:12:11.490 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.490 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.491 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-09-15T23:12:11.491 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "0", 2024-09-15T23:12:11.491 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "6", 2024-09-15T23:12:11.491 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.491 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-15T23:12:11.491 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-15T23:12:11.491 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.491 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.491 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.491 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.491 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.491 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.491 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.492 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.492 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.492 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.492 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.492 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.492 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.492 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.492 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.492 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.492 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.492 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.492 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.493 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.493 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.493 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.493 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.493 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.493 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.493 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.493 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.493 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.493 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.493 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.494 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.494 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.494 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.494 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.494 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.494 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.494 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.494 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.494 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.494 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.494 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.494 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.495 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.495 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.495 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.495 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.495 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.495 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.495 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.495 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.495 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.495 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.495 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.496 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.496 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.496 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.496 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.496 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.496 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.496 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.496 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.496 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.496 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.496 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.496 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.496 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.497 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.497 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.497 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.497 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.497 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.497 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.497 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.497 INFO:teuthology.orchestra.run.smithi111.stdout: "min_score": { 2024-09-15T23:12:11.497 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "min_score", 2024-09-15T23:12:11.497 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-09-15T23:12:11.497 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.497 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.497 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-09-15T23:12:11.498 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.498 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.498 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.498 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-15T23:12:11.498 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.498 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.498 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.498 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.498 INFO:teuthology.orchestra.run.smithi111.stdout: "mode": { 2024-09-15T23:12:11.498 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mode", 2024-09-15T23:12:11.498 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.498 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.499 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.499 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "upmap", 2024-09-15T23:12:11.499 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.499 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.499 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.499 INFO:teuthology.orchestra.run.smithi111.stdout: "crush-compat", 2024-09-15T23:12:11.499 INFO:teuthology.orchestra.run.smithi111.stdout: "none", 2024-09-15T23:12:11.499 INFO:teuthology.orchestra.run.smithi111.stdout: "read", 2024-09-15T23:12:11.499 INFO:teuthology.orchestra.run.smithi111.stdout: "upmap", 2024-09-15T23:12:11.499 INFO:teuthology.orchestra.run.smithi111.stdout: "upmap-read" 2024-09-15T23:12:11.500 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.500 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Balancer mode", 2024-09-15T23:12:11.500 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.500 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.500 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.500 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.500 INFO:teuthology.orchestra.run.smithi111.stdout: "pool_ids": { 2024-09-15T23:12:11.500 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "pool_ids", 2024-09-15T23:12:11.500 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.500 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.500 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.500 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.501 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.501 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.501 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.501 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-15T23:12:11.501 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.501 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.501 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.501 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.501 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-09-15T23:12:11.501 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-09-15T23:12:11.501 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.502 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.502 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.502 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-09-15T23:12:11.502 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.502 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.502 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.502 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-15T23:12:11.502 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.502 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.502 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.502 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.502 INFO:teuthology.orchestra.run.smithi111.stdout: "upmap_max_deviation": { 2024-09-15T23:12:11.502 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "upmap_max_deviation", 2024-09-15T23:12:11.503 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.503 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.503 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.503 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5", 2024-09-15T23:12:11.503 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1", 2024-09-15T23:12:11.503 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.503 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.503 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-15T23:12:11.503 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-15T23:12:11.503 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.503 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.504 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.504 INFO:teuthology.orchestra.run.smithi111.stdout: "upmap_max_optimizations": { 2024-09-15T23:12:11.504 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "upmap_max_optimizations", 2024-09-15T23:12:11.504 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-09-15T23:12:11.504 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.504 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.504 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-09-15T23:12:11.504 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.504 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.504 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.504 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-15T23:12:11.505 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.505 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.505 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.505 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.505 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.505 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.505 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.505 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cephadm", 2024-09-15T23:12:11.505 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.505 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.505 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.505 INFO:teuthology.orchestra.run.smithi111.stdout: "agent_down_multiplier": { 2024-09-15T23:12:11.506 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "agent_down_multiplier", 2024-09-15T23:12:11.506 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-09-15T23:12:11.506 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.506 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.506 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "3.0", 2024-09-15T23:12:11.506 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.506 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.506 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.506 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-15T23:12:11.506 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.506 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.506 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.507 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.507 INFO:teuthology.orchestra.run.smithi111.stdout: "agent_refresh_rate": { 2024-09-15T23:12:11.507 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "agent_refresh_rate", 2024-09-15T23:12:11.507 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.507 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.507 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.507 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "20", 2024-09-15T23:12:11.507 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.507 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.507 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.507 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-15T23:12:11.507 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.508 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.508 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.508 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.508 INFO:teuthology.orchestra.run.smithi111.stdout: "agent_starting_port": { 2024-09-15T23:12:11.508 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "agent_starting_port", 2024-09-15T23:12:11.508 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.508 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.508 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.508 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "4721", 2024-09-15T23:12:11.508 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.508 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.508 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.509 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-15T23:12:11.509 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.509 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.509 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.509 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.509 INFO:teuthology.orchestra.run.smithi111.stdout: "allow_ptrace": { 2024-09-15T23:12:11.509 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "allow_ptrace", 2024-09-15T23:12:11.509 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.509 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.509 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.510 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.510 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.510 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.510 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.510 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-15T23:12:11.510 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-15T23:12:11.510 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.510 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.510 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.510 INFO:teuthology.orchestra.run.smithi111.stdout: "autotune_interval": { 2024-09-15T23:12:11.510 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "autotune_interval", 2024-09-15T23:12:11.510 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.511 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.511 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.511 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-09-15T23:12:11.511 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.511 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.511 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.511 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-15T23:12:11.511 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.511 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.511 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.511 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.511 INFO:teuthology.orchestra.run.smithi111.stdout: "autotune_memory_target_ratio": { 2024-09-15T23:12:11.512 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "autotune_memory_target_ratio", 2024-09-15T23:12:11.512 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-09-15T23:12:11.512 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.512 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.512 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0.7", 2024-09-15T23:12:11.512 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.512 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.512 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.512 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-15T23:12:11.512 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.512 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.513 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.513 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.513 INFO:teuthology.orchestra.run.smithi111.stdout: "cephadm_log_destination": { 2024-09-15T23:12:11.513 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cephadm_log_destination", 2024-09-15T23:12:11.513 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.513 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.513 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.513 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.513 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.513 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.513 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.513 INFO:teuthology.orchestra.run.smithi111.stdout: "file", 2024-09-15T23:12:11.514 INFO:teuthology.orchestra.run.smithi111.stdout: "file,syslog", 2024-09-15T23:12:11.514 INFO:teuthology.orchestra.run.smithi111.stdout: "syslog" 2024-09-15T23:12:11.514 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.514 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-15T23:12:11.514 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.514 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.514 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.514 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.514 INFO:teuthology.orchestra.run.smithi111.stdout: "cgroups_split": { 2024-09-15T23:12:11.514 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cgroups_split", 2024-09-15T23:12:11.514 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.515 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.515 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.515 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.515 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.515 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.515 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.515 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-15T23:12:11.515 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.515 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.515 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.515 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.515 INFO:teuthology.orchestra.run.smithi111.stdout: "config_checks_enabled": { 2024-09-15T23:12:11.516 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "config_checks_enabled", 2024-09-15T23:12:11.516 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.516 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.516 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.516 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.516 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.516 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.516 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.516 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-15T23:12:11.516 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.517 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.517 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.517 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.517 INFO:teuthology.orchestra.run.smithi111.stdout: "config_dashboard": { 2024-09-15T23:12:11.517 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "config_dashboard", 2024-09-15T23:12:11.517 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.517 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.517 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.517 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.517 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.517 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.518 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.518 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-15T23:12:11.518 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.518 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.518 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.518 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.518 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_alertmanager": { 2024-09-15T23:12:11.518 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_alertmanager", 2024-09-15T23:12:11.518 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.518 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.518 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.518 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-15T23:12:11.519 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.519 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.519 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.519 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Prometheus container image", 2024-09-15T23:12:11.519 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.519 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.519 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.519 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.519 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_base": { 2024-09-15T23:12:11.519 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_base", 2024-09-15T23:12:11.519 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.520 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.520 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.520 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-15T23:12:11.520 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.520 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.520 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.520 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Container image name, without the tag", 2024-09-15T23:12:11.520 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.520 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.520 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.520 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.520 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_elasticsearch": { 2024-09-15T23:12:11.521 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_elasticsearch", 2024-09-15T23:12:11.521 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.521 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.521 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.521 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-15T23:12:11.521 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.521 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.521 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.521 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "elasticsearch container image", 2024-09-15T23:12:11.521 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.521 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.522 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.522 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.522 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_grafana": { 2024-09-15T23:12:11.522 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_grafana", 2024-09-15T23:12:11.522 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.522 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.522 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.522 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-15T23:12:11.522 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.522 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.522 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.522 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Prometheus container image", 2024-09-15T23:12:11.522 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.523 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.523 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.523 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.523 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_haproxy": { 2024-09-15T23:12:11.523 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_haproxy", 2024-09-15T23:12:11.523 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.965+0000 7faf19e00640 1 -- 172.21.15.111:0/1441430800 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faefc0a3250 msgr2=0x7faefc0a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:11.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.965+0000 7faf19e00640 1 --2- 172.21.15.111:0/1441430800 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faefc0a3250 0x7faefc0a5710 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7faf1419b440 tx=0x7faf00005f90 comp rx=0 tx=0).stop 2024-09-15T23:12:11.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.965+0000 7faf19e00640 1 -- 172.21.15.111:0/1441430800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf14074680 msgr2=0x7faf14199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:11.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.965+0000 7faf19e00640 1 --2- 172.21.15.111:0/1441430800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf14074680 0x7faf14199f30 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7faf0800e9c0 tx=0x7faf0800ee80 comp rx=0 tx=0).stop 2024-09-15T23:12:11.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.965+0000 7faf19e00640 1 -- 172.21.15.111:0/1441430800 shutdown_connections 2024-09-15T23:12:11.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.965+0000 7faf19e00640 1 --2- 172.21.15.111:0/1441430800 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7faefc0a3250 0x7faefc0a5710 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:11.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.965+0000 7faf19e00640 1 --2- 172.21.15.111:0/1441430800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf14109c90 0x7faf141a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:11.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.965+0000 7faf19e00640 1 --2- 172.21.15.111:0/1441430800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf14074fc0 0x7faf1419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:11.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.965+0000 7faf19e00640 1 --2- 172.21.15.111:0/1441430800 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf14074680 0x7faf14199f30 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:11.524 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.965+0000 7faf19e00640 1 -- 172.21.15.111:0/1441430800 >> 172.21.15.111:0/1441430800 conn(0x7faf140fd110 msgr2=0x7faf14105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:12:11.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.966+0000 7faf19e00640 1 -- 172.21.15.111:0/1441430800 shutdown_connections 2024-09-15T23:12:11.525 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:10.966+0000 7faf19e00640 1 -- 172.21.15.111:0/1441430800 wait complete. 2024-09-15T23:12:11.528 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:11 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/3009045566' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-15T23:12:11.528 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:11 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/1441430800' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-15T23:12:11.530 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.530 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.530 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-15T23:12:11.530 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.530 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.531 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.531 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "HAproxy container image", 2024-09-15T23:12:11.531 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.531 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.531 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.531 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.531 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_jaeger_agent": { 2024-09-15T23:12:11.531 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_jaeger_agent", 2024-09-15T23:12:11.531 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.531 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.531 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.532 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-15T23:12:11.532 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.532 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.532 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.532 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Jaeger agent container image", 2024-09-15T23:12:11.532 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.532 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.532 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.532 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.532 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_jaeger_collector": { 2024-09-15T23:12:11.532 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_jaeger_collector", 2024-09-15T23:12:11.533 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.533 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.533 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.533 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-15T23:12:11.533 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.533 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.533 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.533 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Jaeger collector container image", 2024-09-15T23:12:11.533 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.533 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.533 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.533 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.533 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_jaeger_query": { 2024-09-15T23:12:11.534 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_jaeger_query", 2024-09-15T23:12:11.534 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.534 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.534 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.534 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-15T23:12:11.534 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.534 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.534 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.534 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Jaeger query container image", 2024-09-15T23:12:11.534 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.534 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.535 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.535 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.535 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_keepalived": { 2024-09-15T23:12:11.535 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_keepalived", 2024-09-15T23:12:11.535 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.535 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.535 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.535 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-15T23:12:11.535 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.535 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.535 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.535 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Keepalived container image", 2024-09-15T23:12:11.536 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.536 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.536 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.536 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.536 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_loki": { 2024-09-15T23:12:11.536 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_loki", 2024-09-15T23:12:11.536 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.536 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.536 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.536 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-15T23:12:11.536 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.537 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.537 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.537 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Loki container image", 2024-09-15T23:12:11.537 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.537 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.537 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.537 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.537 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_node_exporter": { 2024-09-15T23:12:11.537 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_node_exporter", 2024-09-15T23:12:11.537 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.537 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.537 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.537 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-15T23:12:11.538 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.538 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.538 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.538 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Prometheus container image", 2024-09-15T23:12:11.538 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.538 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.538 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.538 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.538 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_nvmeof": { 2024-09-15T23:12:11.538 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_nvmeof", 2024-09-15T23:12:11.538 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.538 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.539 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.539 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-15T23:12:11.539 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.539 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.539 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.539 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Nvme-of container image", 2024-09-15T23:12:11.539 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.539 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.539 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.539 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.539 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_prometheus": { 2024-09-15T23:12:11.540 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_prometheus", 2024-09-15T23:12:11.540 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.540 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.540 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.540 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-15T23:12:11.540 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.540 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.540 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.540 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Prometheus container image", 2024-09-15T23:12:11.540 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.540 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.541 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.541 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.541 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_promtail": { 2024-09-15T23:12:11.541 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_promtail", 2024-09-15T23:12:11.541 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.541 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.541 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.541 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-15T23:12:11.541 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.541 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.541 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.541 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Promtail container image", 2024-09-15T23:12:11.542 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.542 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.542 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.542 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.542 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_samba": { 2024-09-15T23:12:11.542 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_samba", 2024-09-15T23:12:11.542 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.542 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.542 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.542 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-15T23:12:11.542 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.542 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.543 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.543 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Samba/SMB container image", 2024-09-15T23:12:11.543 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.543 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.543 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.543 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.543 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_snmp_gateway": { 2024-09-15T23:12:11.543 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_snmp_gateway", 2024-09-15T23:12:11.543 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.543 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.543 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.543 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-15T23:12:11.544 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.544 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.544 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.544 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "SNMP Gateway container image", 2024-09-15T23:12:11.544 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.544 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.544 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.544 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.544 INFO:teuthology.orchestra.run.smithi111.stdout: "container_init": { 2024-09-15T23:12:11.544 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_init", 2024-09-15T23:12:11.544 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.545 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.545 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.545 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.545 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.545 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.545 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.545 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Run podman/docker with `--init`", 2024-09-15T23:12:11.545 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.545 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.545 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.545 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.546 INFO:teuthology.orchestra.run.smithi111.stdout: "daemon_cache_timeout": { 2024-09-15T23:12:11.546 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "daemon_cache_timeout", 2024-09-15T23:12:11.546 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.546 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.546 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.546 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-09-15T23:12:11.546 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.546 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.546 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.546 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-15T23:12:11.546 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.546 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.547 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.547 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.547 INFO:teuthology.orchestra.run.smithi111.stdout: "default_cephadm_command_timeout": { 2024-09-15T23:12:11.547 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "default_cephadm_command_timeout", 2024-09-15T23:12:11.547 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.547 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.547 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.547 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "900", 2024-09-15T23:12:11.547 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.547 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.547 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.547 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-15T23:12:11.548 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.548 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.548 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.548 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.548 INFO:teuthology.orchestra.run.smithi111.stdout: "default_registry": { 2024-09-15T23:12:11.548 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "default_registry", 2024-09-15T23:12:11.548 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.548 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.548 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.548 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "docker.io", 2024-09-15T23:12:11.548 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.549 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.549 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.549 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-15T23:12:11.549 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.549 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.549 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.549 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.549 INFO:teuthology.orchestra.run.smithi111.stdout: "device_cache_timeout": { 2024-09-15T23:12:11.549 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "device_cache_timeout", 2024-09-15T23:12:11.549 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.549 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.550 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.550 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "1800", 2024-09-15T23:12:11.550 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.550 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.550 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.550 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "seconds to cache device inventory", 2024-09-15T23:12:11.550 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.550 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.550 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.550 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.550 INFO:teuthology.orchestra.run.smithi111.stdout: "device_enhanced_scan": { 2024-09-15T23:12:11.550 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "device_enhanced_scan", 2024-09-15T23:12:11.551 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.551 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.551 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.551 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.551 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.551 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.551 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.551 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-15T23:12:11.551 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.551 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.551 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.552 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.552 INFO:teuthology.orchestra.run.smithi111.stdout: "facts_cache_timeout": { 2024-09-15T23:12:11.552 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "facts_cache_timeout", 2024-09-15T23:12:11.552 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.552 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.552 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.552 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-09-15T23:12:11.552 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.552 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.552 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.552 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "seconds to cache host facts data", 2024-09-15T23:12:11.552 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.553 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.553 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.553 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.553 INFO:teuthology.orchestra.run.smithi111.stdout: "grafana_dashboards_path": { 2024-09-15T23:12:11.553 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "grafana_dashboards_path", 2024-09-15T23:12:11.553 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.553 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.553 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.553 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-15T23:12:11.553 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.553 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.554 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.554 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-15T23:12:11.554 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.554 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.554 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.554 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.554 INFO:teuthology.orchestra.run.smithi111.stdout: "host_check_interval": { 2024-09-15T23:12:11.554 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "host_check_interval", 2024-09-15T23:12:11.554 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.554 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.554 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.554 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-09-15T23:12:11.555 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.555 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.555 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.555 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to perform a host check", 2024-09-15T23:12:11.555 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.555 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.555 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.555 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.555 INFO:teuthology.orchestra.run.smithi111.stdout: "hw_monitoring": { 2024-09-15T23:12:11.555 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "hw_monitoring", 2024-09-15T23:12:11.555 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.555 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.556 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.556 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.556 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.556 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.556 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.556 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-15T23:12:11.556 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.556 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.556 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.556 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.556 INFO:teuthology.orchestra.run.smithi111.stdout: "inventory_list_all": { 2024-09-15T23:12:11.557 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "inventory_list_all", 2024-09-15T23:12:11.557 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.557 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.557 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.557 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.557 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.557 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.557 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.557 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-15T23:12:11.557 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.557 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.558 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.558 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.558 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.558 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.558 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.558 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.558 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.558 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.558 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.558 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.558 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.558 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.559 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.559 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.559 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.559 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.559 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.559 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.559 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.559 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.559 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.559 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.559 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.559 INFO:teuthology.orchestra.run.smithi111.stdout: "log_refresh_metadata": { 2024-09-15T23:12:11.560 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_refresh_metadata", 2024-09-15T23:12:11.560 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.560 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.560 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.560 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.560 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.560 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.560 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.560 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-15T23:12:11.560 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.560 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.560 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.561 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.561 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.561 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.561 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.561 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.561 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.561 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.561 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.561 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.561 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.561 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-15T23:12:11.561 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.562 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.562 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.562 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.562 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.562 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.562 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.562 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.562 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.562 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.562 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.562 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.563 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.563 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.563 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.563 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.563 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.563 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.563 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.563 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.563 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.563 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.563 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.563 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.564 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.564 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.564 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.564 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.564 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.564 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.564 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.564 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.564 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.564 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.564 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.565 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.565 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.565 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.565 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.565 INFO:teuthology.orchestra.run.smithi111.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-15T23:12:11.565 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-15T23:12:11.565 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.565 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.565 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.565 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.565 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.565 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.566 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.566 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-15T23:12:11.566 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.566 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.566 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.566 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.566 INFO:teuthology.orchestra.run.smithi111.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-15T23:12:11.566 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-15T23:12:11.566 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.566 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.566 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.566 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "*", 2024-09-15T23:12:11.566 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.567 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.567 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.567 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-15T23:12:11.567 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.567 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.567 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.567 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.567 INFO:teuthology.orchestra.run.smithi111.stdout: "max_count_per_host": { 2024-09-15T23:12:11.567 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_count_per_host", 2024-09-15T23:12:11.567 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.567 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.567 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.568 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-09-15T23:12:11.568 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.568 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.568 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.568 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "max number of daemons per service per host", 2024-09-15T23:12:11.568 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.568 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.568 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.568 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.568 INFO:teuthology.orchestra.run.smithi111.stdout: "max_osd_draining_count": { 2024-09-15T23:12:11.568 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_osd_draining_count", 2024-09-15T23:12:11.568 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.568 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.569 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.569 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-09-15T23:12:11.569 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.569 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.569 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.569 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-15T23:12:11.569 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.569 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.569 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.569 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.570 INFO:teuthology.orchestra.run.smithi111.stdout: "migration_current": { 2024-09-15T23:12:11.570 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "migration_current", 2024-09-15T23:12:11.570 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.570 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.570 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.570 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.570 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.570 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.570 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.570 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "internal - do not modify", 2024-09-15T23:12:11.570 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.570 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.571 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.571 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.571 INFO:teuthology.orchestra.run.smithi111.stdout: "mode": { 2024-09-15T23:12:11.571 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mode", 2024-09-15T23:12:11.571 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.571 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.571 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.571 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "root", 2024-09-15T23:12:11.571 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.571 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.571 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.572 INFO:teuthology.orchestra.run.smithi111.stdout: "cephadm-package", 2024-09-15T23:12:11.572 INFO:teuthology.orchestra.run.smithi111.stdout: "root" 2024-09-15T23:12:11.572 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.572 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "mode for remote execution of cephadm", 2024-09-15T23:12:11.572 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.572 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.572 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.572 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.572 INFO:teuthology.orchestra.run.smithi111.stdout: "oob_default_addr": { 2024-09-15T23:12:11.572 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "oob_default_addr", 2024-09-15T23:12:11.572 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.572 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.573 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.573 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "169.254.1.1", 2024-09-15T23:12:11.573 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.573 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.573 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.573 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-15T23:12:11.573 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.573 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.573 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.573 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.573 INFO:teuthology.orchestra.run.smithi111.stdout: "prometheus_alerts_path": { 2024-09-15T23:12:11.574 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "prometheus_alerts_path", 2024-09-15T23:12:11.574 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.574 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.574 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.574 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-15T23:12:11.574 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.574 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.574 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.574 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-15T23:12:11.574 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.574 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.575 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.575 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.575 INFO:teuthology.orchestra.run.smithi111.stdout: "registry_insecure": { 2024-09-15T23:12:11.575 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "registry_insecure", 2024-09-15T23:12:11.575 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.575 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.575 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.575 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.575 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.575 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.575 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.576 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-15T23:12:11.576 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.576 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.576 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.576 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.576 INFO:teuthology.orchestra.run.smithi111.stdout: "registry_password": { 2024-09-15T23:12:11.576 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "registry_password", 2024-09-15T23:12:11.576 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.576 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.576 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.576 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.577 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.577 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.577 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.577 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-15T23:12:11.577 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.577 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.577 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.577 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.577 INFO:teuthology.orchestra.run.smithi111.stdout: "registry_url": { 2024-09-15T23:12:11.577 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "registry_url", 2024-09-15T23:12:11.577 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.577 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.578 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.578 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.578 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.578 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.578 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.578 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-15T23:12:11.578 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.578 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.578 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.578 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.578 INFO:teuthology.orchestra.run.smithi111.stdout: "registry_username": { 2024-09-15T23:12:11.579 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "registry_username", 2024-09-15T23:12:11.579 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.579 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.579 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.579 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.579 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.579 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.579 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.579 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-15T23:12:11.579 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.579 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.579 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.580 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.580 INFO:teuthology.orchestra.run.smithi111.stdout: "secure_monitoring_stack": { 2024-09-15T23:12:11.580 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "secure_monitoring_stack", 2024-09-15T23:12:11.580 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.580 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.580 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.580 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.580 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.580 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.580 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.580 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-15T23:12:11.581 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.581 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.581 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.581 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.581 INFO:teuthology.orchestra.run.smithi111.stdout: "service_discovery_port": { 2024-09-15T23:12:11.581 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "service_discovery_port", 2024-09-15T23:12:11.581 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.581 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.581 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.581 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8765", 2024-09-15T23:12:11.581 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.582 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.582 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.582 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "cephadm service discovery port", 2024-09-15T23:12:11.582 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.582 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.582 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.582 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.582 INFO:teuthology.orchestra.run.smithi111.stdout: "ssh_config_file": { 2024-09-15T23:12:11.582 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ssh_config_file", 2024-09-15T23:12:11.582 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.582 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.582 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.583 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.583 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.583 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.583 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.583 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-15T23:12:11.583 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.583 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.583 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.583 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.583 INFO:teuthology.orchestra.run.smithi111.stdout: "use_agent": { 2024-09-15T23:12:11.583 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "use_agent", 2024-09-15T23:12:11.583 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.584 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.584 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.584 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.584 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.584 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.584 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.584 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-15T23:12:11.584 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.584 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.584 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.585 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.585 INFO:teuthology.orchestra.run.smithi111.stdout: "use_repo_digest": { 2024-09-15T23:12:11.585 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "use_repo_digest", 2024-09-15T23:12:11.585 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.585 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.585 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.585 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.585 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.585 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.585 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.585 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-15T23:12:11.586 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.586 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.586 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.586 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.586 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_on_failed_host_check": { 2024-09-15T23:12:11.586 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_on_failed_host_check", 2024-09-15T23:12:11.586 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.586 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.586 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.586 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.587 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.587 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.587 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.587 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "raise a health warning if the host check fails", 2024-09-15T23:12:11.587 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.587 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.587 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.587 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.587 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_on_stray_daemons": { 2024-09-15T23:12:11.587 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_on_stray_daemons", 2024-09-15T23:12:11.587 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.587 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.588 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.588 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.588 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.588 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.588 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.588 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-15T23:12:11.588 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.588 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.588 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.588 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.588 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_on_stray_hosts": { 2024-09-15T23:12:11.589 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_on_stray_hosts", 2024-09-15T23:12:11.589 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.589 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.589 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.589 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.589 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.589 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.589 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.589 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-15T23:12:11.589 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.589 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.589 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.590 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.590 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.590 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.590 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.590 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crash", 2024-09-15T23:12:11.590 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.590 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.590 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.590 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.590 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.590 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.591 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.591 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.591 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.591 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.591 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.591 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.591 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.591 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.591 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.591 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.591 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.591 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.592 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.592 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.592 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.592 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.592 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.592 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.592 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.592 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.592 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.592 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.592 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.593 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.593 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.593 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.593 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.593 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.593 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.593 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.593 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.593 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.593 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.593 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.593 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.594 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.594 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.594 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.594 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.594 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.594 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.594 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.594 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.594 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.594 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.594 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.594 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.595 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.595 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.595 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.595 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.595 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.595 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.595 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.595 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.595 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.595 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.595 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.595 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.596 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.596 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.596 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.596 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.596 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.596 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.596 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.596 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.596 INFO:teuthology.orchestra.run.smithi111.stdout: "retain_interval": { 2024-09-15T23:12:11.596 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "retain_interval", 2024-09-15T23:12:11.596 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.596 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.597 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.597 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "31536000", 2024-09-15T23:12:11.597 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.597 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.597 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.597 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-15T23:12:11.597 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.597 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.597 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.597 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.597 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_recent_interval": { 2024-09-15T23:12:11.597 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_recent_interval", 2024-09-15T23:12:11.597 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.598 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.598 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.598 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "1209600", 2024-09-15T23:12:11.598 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.598 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.598 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.598 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-15T23:12:11.598 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.598 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.598 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.598 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.598 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.599 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.599 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.599 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "dashboard", 2024-09-15T23:12:11.599 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.599 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.599 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.599 INFO:teuthology.orchestra.run.smithi111.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-15T23:12:11.599 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-15T23:12:11.599 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.599 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.599 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.600 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-09-15T23:12:11.600 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.600 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.600 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.600 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.600 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.600 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.600 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.600 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.600 INFO:teuthology.orchestra.run.smithi111.stdout: "ALERTMANAGER_API_HOST": { 2024-09-15T23:12:11.601 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-15T23:12:11.601 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.601 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.601 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.601 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.601 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.601 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.601 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.602 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.602 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.602 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.602 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.602 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.602 INFO:teuthology.orchestra.run.smithi111.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-15T23:12:11.602 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-15T23:12:11.602 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.602 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.602 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.603 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.603 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.603 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.603 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.603 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.603 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.603 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.603 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.603 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.603 INFO:teuthology.orchestra.run.smithi111.stdout: "AUDIT_API_ENABLED": { 2024-09-15T23:12:11.603 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "AUDIT_API_ENABLED", 2024-09-15T23:12:11.603 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.604 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.604 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.604 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.604 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.604 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.604 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.604 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.604 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.604 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.604 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.604 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.604 INFO:teuthology.orchestra.run.smithi111.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-15T23:12:11.605 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-15T23:12:11.605 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.605 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.605 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.605 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.605 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.605 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.605 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.605 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.605 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.605 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.605 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.606 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.606 INFO:teuthology.orchestra.run.smithi111.stdout: "ENABLE_BROWSABLE_API": { 2024-09-15T23:12:11.606 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-15T23:12:11.606 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.606 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.606 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.606 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.606 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.606 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.606 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.606 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.607 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.607 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.607 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.607 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.607 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-15T23:12:11.607 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-15T23:12:11.607 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.607 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.607 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.607 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.607 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.607 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.608 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.608 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.608 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.608 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.608 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.608 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.608 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-15T23:12:11.608 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-15T23:12:11.608 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.608 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.609 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.609 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.609 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.609 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.609 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.609 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.609 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.609 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.609 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.609 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.609 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-15T23:12:11.609 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-15T23:12:11.610 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.610 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.610 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.610 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.610 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.610 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.610 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.610 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.610 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.610 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.610 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.611 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.611 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-15T23:12:11.611 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-15T23:12:11.611 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.611 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.611 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.611 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.611 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.611 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.611 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.611 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.611 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.612 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.612 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.612 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.612 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-15T23:12:11.612 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-15T23:12:11.612 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.612 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.612 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.612 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.612 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.612 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.613 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.613 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.613 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.613 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.613 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.613 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.613 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-15T23:12:11.613 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-15T23:12:11.613 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.613 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.613 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.614 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.614 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.614 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.614 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.614 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.614 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.614 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.614 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.614 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.614 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-15T23:12:11.614 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-15T23:12:11.614 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.615 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.615 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.615 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.615 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.615 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.615 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.615 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.615 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.615 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.616 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.616 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.616 INFO:teuthology.orchestra.run.smithi111.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-15T23:12:11.616 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-15T23:12:11.616 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.616 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.616 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.616 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.616 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.616 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.616 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.617 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.617 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.617 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.617 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.617 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.617 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_API_PASSWORD": { 2024-09-15T23:12:11.617 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-15T23:12:11.617 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.617 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.617 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.617 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "admin", 2024-09-15T23:12:11.618 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.618 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.618 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.618 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.618 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.618 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.618 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.618 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.618 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-15T23:12:11.618 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-15T23:12:11.618 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.619 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.619 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.619 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.619 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.619 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.619 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.619 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.619 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.619 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.619 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.619 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.620 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_API_URL": { 2024-09-15T23:12:11.620 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_API_URL", 2024-09-15T23:12:11.620 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.620 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.620 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.620 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.620 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.620 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.620 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.620 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.620 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.621 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.621 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.621 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.621 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_API_USERNAME": { 2024-09-15T23:12:11.621 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-15T23:12:11.621 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.621 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.621 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.621 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "admin", 2024-09-15T23:12:11.621 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.621 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.621 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.622 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.622 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.622 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.622 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.622 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.622 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-15T23:12:11.622 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-15T23:12:11.622 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.622 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.622 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.622 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.622 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.623 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.623 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.623 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.623 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.623 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.623 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.623 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.623 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-15T23:12:11.623 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-15T23:12:11.623 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.623 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.624 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.624 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.624 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.624 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.624 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.624 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.624 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.624 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.624 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.624 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.624 INFO:teuthology.orchestra.run.smithi111.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-15T23:12:11.625 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-15T23:12:11.625 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.625 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.625 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.625 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.625 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.625 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.625 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.625 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.625 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.625 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.625 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.626 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.626 INFO:teuthology.orchestra.run.smithi111.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-15T23:12:11.626 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-15T23:12:11.626 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.626 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.626 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.626 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.626 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.626 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.626 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.626 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.626 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.627 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.627 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.627 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.627 INFO:teuthology.orchestra.run.smithi111.stdout: "PROMETHEUS_API_HOST": { 2024-09-15T23:12:11.627 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-15T23:12:11.627 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.627 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.627 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.627 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.627 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.627 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.628 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.628 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.628 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.628 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.628 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.628 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.628 INFO:teuthology.orchestra.run.smithi111.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-15T23:12:11.628 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-15T23:12:11.628 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.628 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.628 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.628 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.629 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.629 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.629 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.629 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.629 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.629 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.629 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.629 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.629 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-15T23:12:11.629 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-15T23:12:11.629 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.630 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.630 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.630 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.630 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.630 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.630 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.630 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.630 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.630 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.630 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.630 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.631 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-15T23:12:11.631 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-15T23:12:11.631 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.631 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.631 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.631 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.631 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.631 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.631 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.631 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.631 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.632 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.632 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.632 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.632 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-15T23:12:11.632 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-15T23:12:11.632 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.632 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.632 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.632 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.632 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.633 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.633 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.633 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.633 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.633 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.633 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.633 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.633 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-15T23:12:11.633 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-15T23:12:11.633 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.633 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.634 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.634 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.634 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.634 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.634 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.634 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.634 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.634 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.634 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.634 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.635 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-15T23:12:11.635 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-15T23:12:11.635 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.635 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.635 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.635 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.635 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.635 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.635 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.635 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.635 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.635 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.636 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.636 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.636 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-15T23:12:11.636 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-15T23:12:11.636 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.636 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.636 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.636 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.636 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.636 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.636 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.637 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.637 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.637 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.637 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.637 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.637 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-15T23:12:11.637 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-15T23:12:11.637 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.637 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.637 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.637 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.637 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.638 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.638 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.638 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.638 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.638 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.638 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.638 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.638 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_ENABLED": { 2024-09-15T23:12:11.638 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-15T23:12:11.638 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.638 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.638 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.639 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.639 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.639 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.639 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.639 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.639 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.639 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.639 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.639 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.639 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-15T23:12:11.640 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-15T23:12:11.640 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.640 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.640 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.640 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-15T23:12:11.640 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.640 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.640 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.640 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.640 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.640 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.641 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.641 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.641 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-15T23:12:11.641 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-15T23:12:11.641 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.641 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.641 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.641 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-09-15T23:12:11.641 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.641 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.641 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.642 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.642 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.642 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.642 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.642 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.642 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-15T23:12:11.642 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-15T23:12:11.642 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.642 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.642 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.642 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8", 2024-09-15T23:12:11.642 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.643 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.643 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.643 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.643 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.643 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.643 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.643 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.643 INFO:teuthology.orchestra.run.smithi111.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-15T23:12:11.643 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-15T23:12:11.643 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.643 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.643 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.643 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "45", 2024-09-15T23:12:11.644 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.644 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.644 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.644 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.644 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.644 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.644 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.644 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.644 INFO:teuthology.orchestra.run.smithi111.stdout: "RGW_API_ACCESS_KEY": { 2024-09-15T23:12:11.644 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-15T23:12:11.644 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.644 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.645 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.645 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.645 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.645 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.645 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.645 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.645 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.645 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.645 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.645 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.645 INFO:teuthology.orchestra.run.smithi111.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-15T23:12:11.646 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-15T23:12:11.646 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.646 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.646 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.646 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "admin", 2024-09-15T23:12:11.646 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.646 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.646 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.646 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.646 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.646 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.646 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.647 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.647 INFO:teuthology.orchestra.run.smithi111.stdout: "RGW_API_SECRET_KEY": { 2024-09-15T23:12:11.647 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-15T23:12:11.647 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.647 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.647 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.647 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.647 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.647 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.647 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.647 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.647 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.647 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.648 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.648 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.648 INFO:teuthology.orchestra.run.smithi111.stdout: "RGW_API_SSL_VERIFY": { 2024-09-15T23:12:11.648 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-15T23:12:11.648 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.648 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.648 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.648 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.648 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.648 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.648 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.649 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.649 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.649 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.649 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.649 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.649 INFO:teuthology.orchestra.run.smithi111.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-15T23:12:11.649 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-15T23:12:11.649 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.649 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.649 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.649 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-09-15T23:12:11.649 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.650 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.650 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.650 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.650 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.650 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.650 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.650 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.650 INFO:teuthology.orchestra.run.smithi111.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-15T23:12:11.650 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-15T23:12:11.650 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.650 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.651 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.651 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-09-15T23:12:11.651 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.651 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.651 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.651 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.651 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.651 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.651 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.651 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.652 INFO:teuthology.orchestra.run.smithi111.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-15T23:12:11.652 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-15T23:12:11.652 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.652 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.652 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.652 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5", 2024-09-15T23:12:11.652 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.652 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.652 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.652 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.652 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.653 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.653 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.653 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.653 INFO:teuthology.orchestra.run.smithi111.stdout: "cross_origin_url": { 2024-09-15T23:12:11.653 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cross_origin_url", 2024-09-15T23:12:11.653 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.653 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.653 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.653 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.653 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.653 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.654 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.654 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.654 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.654 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.654 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.654 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.654 INFO:teuthology.orchestra.run.smithi111.stdout: "crt_file": { 2024-09-15T23:12:11.654 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crt_file", 2024-09-15T23:12:11.654 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.654 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.655 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.655 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.655 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.655 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.655 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.655 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.655 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.655 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.655 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.655 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.656 INFO:teuthology.orchestra.run.smithi111.stdout: "debug": { 2024-09-15T23:12:11.656 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "debug", 2024-09-15T23:12:11.656 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.656 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.656 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.656 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.656 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.656 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.656 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.656 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Enable/disable debug options", 2024-09-15T23:12:11.656 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.657 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.657 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.657 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.657 INFO:teuthology.orchestra.run.smithi111.stdout: "jwt_token_ttl": { 2024-09-15T23:12:11.657 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "jwt_token_ttl", 2024-09-15T23:12:11.657 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.657 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.657 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.657 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "28800", 2024-09-15T23:12:11.657 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.657 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.657 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.657 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.658 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.658 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.658 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.658 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.658 INFO:teuthology.orchestra.run.smithi111.stdout: "key_file": { 2024-09-15T23:12:11.658 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "key_file", 2024-09-15T23:12:11.658 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.658 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.658 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.658 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.659 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.659 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.659 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.659 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.659 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.659 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.659 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.659 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.659 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.659 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.659 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.659 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.660 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.660 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.660 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.660 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.660 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.660 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.660 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.660 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.660 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.660 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.660 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.661 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.661 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.661 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.661 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.661 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.661 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.661 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.661 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.661 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.662 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.662 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.662 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.662 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.662 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.662 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.662 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.662 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.662 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.662 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.662 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.662 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.662 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.663 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.663 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.663 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.663 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.663 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.663 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.663 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.663 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.663 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.663 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.663 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.663 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.663 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.664 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.664 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.664 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.664 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.664 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.664 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.664 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.664 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.664 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.664 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.664 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.664 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.665 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.665 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.665 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.665 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.665 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.665 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.665 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.665 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.665 INFO:teuthology.orchestra.run.smithi111.stdout: "motd": { 2024-09-15T23:12:11.665 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "motd", 2024-09-15T23:12:11.666 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.666 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.666 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.666 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.666 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.666 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.666 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.666 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "The message of the day", 2024-09-15T23:12:11.666 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.666 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.666 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.666 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.666 INFO:teuthology.orchestra.run.smithi111.stdout: "redirect_resolve_ip_addr": { 2024-09-15T23:12:11.667 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "redirect_resolve_ip_addr", 2024-09-15T23:12:11.667 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.667 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.667 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.667 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.667 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.667 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.667 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.667 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.667 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.667 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.667 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.668 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.668 INFO:teuthology.orchestra.run.smithi111.stdout: "server_addr": { 2024-09-15T23:12:11.668 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_addr", 2024-09-15T23:12:11.668 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.668 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.668 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.668 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "::", 2024-09-15T23:12:11.668 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.668 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.668 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.668 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.668 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.668 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.669 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.669 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.669 INFO:teuthology.orchestra.run.smithi111.stdout: "server_port": { 2024-09-15T23:12:11.669 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_port", 2024-09-15T23:12:11.669 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.669 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.669 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.669 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8080", 2024-09-15T23:12:11.669 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.669 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.669 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.669 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.669 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.670 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.670 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.670 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.670 INFO:teuthology.orchestra.run.smithi111.stdout: "ssl": { 2024-09-15T23:12:11.670 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ssl", 2024-09-15T23:12:11.670 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.670 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.670 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.670 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.670 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.670 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.670 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.671 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.671 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.671 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.671 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.671 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.671 INFO:teuthology.orchestra.run.smithi111.stdout: "ssl_server_port": { 2024-09-15T23:12:11.671 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ssl_server_port", 2024-09-15T23:12:11.671 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.671 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.671 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.671 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8443", 2024-09-15T23:12:11.671 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.672 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.672 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.672 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.672 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.672 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.672 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.672 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.672 INFO:teuthology.orchestra.run.smithi111.stdout: "standby_behaviour": { 2024-09-15T23:12:11.672 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "standby_behaviour", 2024-09-15T23:12:11.672 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.673 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.673 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.673 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "redirect", 2024-09-15T23:12:11.673 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.673 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.673 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.673 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.673 INFO:teuthology.orchestra.run.smithi111.stdout: "redirect" 2024-09-15T23:12:11.673 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.673 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.673 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.673 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.673 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.674 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.674 INFO:teuthology.orchestra.run.smithi111.stdout: "standby_error_status_code": { 2024-09-15T23:12:11.674 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "standby_error_status_code", 2024-09-15T23:12:11.674 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.674 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.674 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.674 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "500", 2024-09-15T23:12:11.674 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "400", 2024-09-15T23:12:11.674 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "599", 2024-09-15T23:12:11.674 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.674 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.675 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.675 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.675 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.675 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.675 INFO:teuthology.orchestra.run.smithi111.stdout: "url_prefix": { 2024-09-15T23:12:11.675 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "url_prefix", 2024-09-15T23:12:11.675 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.675 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.675 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.675 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.675 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.675 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.676 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.676 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.676 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.676 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.676 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.676 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.676 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.676 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.676 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.676 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "devicehealth", 2024-09-15T23:12:11.676 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.676 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.677 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.677 INFO:teuthology.orchestra.run.smithi111.stdout: "enable_monitoring": { 2024-09-15T23:12:11.677 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "enable_monitoring", 2024-09-15T23:12:11.677 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.677 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.677 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.677 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.677 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.677 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.677 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.677 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "monitor device health metrics", 2024-09-15T23:12:11.678 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.678 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.678 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.678 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.678 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.678 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.678 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.678 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.678 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.678 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.679 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.679 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.679 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.679 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.679 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.679 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.679 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.679 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.679 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.679 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.679 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.680 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.680 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.680 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.680 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.680 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.680 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.680 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.680 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.680 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.680 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.680 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.680 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.681 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.681 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.681 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.681 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.681 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.681 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.681 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.681 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.681 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.681 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.681 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.682 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.682 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.682 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.682 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.682 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.682 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.682 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.682 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.682 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.682 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.682 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.682 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.682 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.683 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.683 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.683 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.683 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.683 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.683 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.683 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.683 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.683 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.683 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.684 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.684 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.684 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.684 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.684 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.684 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.684 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.684 INFO:teuthology.orchestra.run.smithi111.stdout: "mark_out_threshold": { 2024-09-15T23:12:11.684 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mark_out_threshold", 2024-09-15T23:12:11.684 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.684 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.685 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.685 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "2419200", 2024-09-15T23:12:11.685 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.685 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.685 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.685 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-15T23:12:11.685 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.685 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.685 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.685 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.686 INFO:teuthology.orchestra.run.smithi111.stdout: "pool_name": { 2024-09-15T23:12:11.686 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "pool_name", 2024-09-15T23:12:11.686 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.686 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.686 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.686 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "device_health_metrics", 2024-09-15T23:12:11.686 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.686 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.686 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.686 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-15T23:12:11.687 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.687 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.687 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.687 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.687 INFO:teuthology.orchestra.run.smithi111.stdout: "retention_period": { 2024-09-15T23:12:11.687 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "retention_period", 2024-09-15T23:12:11.687 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.687 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.687 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.687 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "15552000", 2024-09-15T23:12:11.687 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.687 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.688 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.688 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how long to retain device health metrics", 2024-09-15T23:12:11.688 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.688 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.688 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.688 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.688 INFO:teuthology.orchestra.run.smithi111.stdout: "scrape_frequency": { 2024-09-15T23:12:11.688 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "scrape_frequency", 2024-09-15T23:12:11.688 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.688 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.688 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.689 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "86400", 2024-09-15T23:12:11.689 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.689 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.689 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.689 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-15T23:12:11.689 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.689 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.689 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.689 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.689 INFO:teuthology.orchestra.run.smithi111.stdout: "self_heal": { 2024-09-15T23:12:11.689 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "self_heal", 2024-09-15T23:12:11.690 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.690 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.690 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.690 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.690 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.690 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.690 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.690 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-15T23:12:11.690 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.690 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.690 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.691 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.691 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-09-15T23:12:11.691 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-09-15T23:12:11.691 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.691 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.691 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.691 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-09-15T23:12:11.691 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.691 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.691 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.691 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to wake up and check device health", 2024-09-15T23:12:11.691 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.692 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.692 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.692 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.692 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_threshold": { 2024-09-15T23:12:11.692 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_threshold", 2024-09-15T23:12:11.692 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.692 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.692 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.692 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "7257600", 2024-09-15T23:12:11.692 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.692 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.693 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.693 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-15T23:12:11.693 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.693 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.693 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.693 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.693 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.693 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.693 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.693 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "diskprediction_local", 2024-09-15T23:12:11.694 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.694 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.694 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.694 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.694 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.694 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.694 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.694 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.694 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.694 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.694 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.694 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.695 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.695 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.695 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.695 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.695 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.695 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.695 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.695 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.695 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.695 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.695 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.696 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.696 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.696 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.696 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.696 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.696 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.696 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.696 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.696 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.696 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.696 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.697 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.697 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.697 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.697 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.697 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.697 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.697 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.697 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.697 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.697 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.697 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.698 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.698 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.698 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.698 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.698 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.698 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.698 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.698 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.698 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.698 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.698 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.698 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.699 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.699 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.699 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.699 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.699 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.699 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.699 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.699 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.699 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.699 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.699 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.700 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.700 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.700 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.700 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.700 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.700 INFO:teuthology.orchestra.run.smithi111.stdout: "predict_interval": { 2024-09-15T23:12:11.700 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "predict_interval", 2024-09-15T23:12:11.700 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.700 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.700 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.701 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "86400", 2024-09-15T23:12:11.701 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.701 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.701 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.701 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.701 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.701 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.701 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.701 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.701 INFO:teuthology.orchestra.run.smithi111.stdout: "predictor_model": { 2024-09-15T23:12:11.701 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "predictor_model", 2024-09-15T23:12:11.702 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.702 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.702 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.702 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "prophetstor", 2024-09-15T23:12:11.702 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.702 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.702 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.702 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.702 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.702 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.702 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.702 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.703 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-09-15T23:12:11.703 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-09-15T23:12:11.703 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.703 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.703 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.703 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-09-15T23:12:11.703 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.703 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.703 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.703 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.703 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.703 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.703 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.704 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.704 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.704 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.704 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.704 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "influx", 2024-09-15T23:12:11.704 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": false, 2024-09-15T23:12:11.704 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "influxdb python module not found", 2024-09-15T23:12:11.704 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.704 INFO:teuthology.orchestra.run.smithi111.stdout: "batch_size": { 2024-09-15T23:12:11.704 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "batch_size", 2024-09-15T23:12:11.704 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.704 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.705 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.705 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5000", 2024-09-15T23:12:11.705 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.705 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.705 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.705 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-15T23:12:11.705 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.705 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.705 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.705 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.706 INFO:teuthology.orchestra.run.smithi111.stdout: "database": { 2024-09-15T23:12:11.706 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "database", 2024-09-15T23:12:11.706 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.706 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.706 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.706 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "ceph", 2024-09-15T23:12:11.706 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.706 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.706 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.706 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-15T23:12:11.707 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.707 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.707 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.707 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.707 INFO:teuthology.orchestra.run.smithi111.stdout: "hostname": { 2024-09-15T23:12:11.707 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "hostname", 2024-09-15T23:12:11.707 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.707 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.707 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.707 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.707 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.707 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.707 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.708 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "InfluxDB server hostname", 2024-09-15T23:12:11.708 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.708 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.708 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.708 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.708 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-09-15T23:12:11.708 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-09-15T23:12:11.708 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.708 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.708 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.709 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "30", 2024-09-15T23:12:11.709 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "5", 2024-09-15T23:12:11.709 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.709 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.709 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-15T23:12:11.709 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.709 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.709 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.709 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.709 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.709 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.710 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.710 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.710 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.710 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.710 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.710 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.710 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.710 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.710 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.710 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.710 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.710 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.711 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.711 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.711 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.711 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.711 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.711 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.711 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.711 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.711 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.711 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.711 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.711 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.712 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.712 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.712 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.712 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.712 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.712 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.712 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.712 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.712 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.712 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.712 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.713 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.713 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.713 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.713 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.713 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.713 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.713 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.713 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.713 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.713 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.713 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.713 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.714 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.714 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.714 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.714 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.714 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.714 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.714 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.714 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.714 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.714 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.714 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.714 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.715 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.715 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.715 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.715 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.715 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.715 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.715 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.715 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.715 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.715 INFO:teuthology.orchestra.run.smithi111.stdout: "password": { 2024-09-15T23:12:11.715 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "password", 2024-09-15T23:12:11.716 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.716 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.716 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.716 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.716 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.716 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.716 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.716 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "password of InfluxDB server user", 2024-09-15T23:12:11.716 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.716 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.716 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.716 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.717 INFO:teuthology.orchestra.run.smithi111.stdout: "port": { 2024-09-15T23:12:11.717 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "port", 2024-09-15T23:12:11.717 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.717 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.717 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.717 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8086", 2024-09-15T23:12:11.717 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.717 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.717 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.717 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "InfluxDB server port", 2024-09-15T23:12:11.717 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.718 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.718 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.718 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.718 INFO:teuthology.orchestra.run.smithi111.stdout: "ssl": { 2024-09-15T23:12:11.718 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ssl", 2024-09-15T23:12:11.718 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.718 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.718 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.718 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "false", 2024-09-15T23:12:11.718 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.718 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.719 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.719 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-15T23:12:11.719 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.719 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.719 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.719 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.719 INFO:teuthology.orchestra.run.smithi111.stdout: "threads": { 2024-09-15T23:12:11.719 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "threads", 2024-09-15T23:12:11.719 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.719 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.719 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.719 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5", 2024-09-15T23:12:11.719 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1", 2024-09-15T23:12:11.720 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "32", 2024-09-15T23:12:11.720 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.720 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-15T23:12:11.720 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.720 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.720 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.720 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.720 INFO:teuthology.orchestra.run.smithi111.stdout: "username": { 2024-09-15T23:12:11.720 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "username", 2024-09-15T23:12:11.720 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.720 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.720 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.720 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.721 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.721 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.721 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.721 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "username of InfluxDB server user", 2024-09-15T23:12:11.721 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.721 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.721 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.721 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.721 INFO:teuthology.orchestra.run.smithi111.stdout: "verify_ssl": { 2024-09-15T23:12:11.721 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "verify_ssl", 2024-09-15T23:12:11.721 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.722 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.722 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.722 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "true", 2024-09-15T23:12:11.722 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.722 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.722 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.722 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-15T23:12:11.722 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.722 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.722 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.722 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.723 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.723 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.723 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.723 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "insights", 2024-09-15T23:12:11.723 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.723 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.723 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.723 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.723 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.723 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.723 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.724 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.724 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.724 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.724 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.724 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.724 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.724 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.724 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.724 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.724 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.724 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.725 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.725 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.725 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.725 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.725 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.725 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.725 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.725 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.725 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.725 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.725 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.725 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.726 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.726 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.726 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.726 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.726 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.726 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.726 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.726 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.726 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.727 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.727 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.727 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.727 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.727 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.727 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.727 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.727 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.727 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.727 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.728 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.728 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.728 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.728 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.728 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.728 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.728 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.728 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.728 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.728 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.728 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.728 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.729 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.729 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.729 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.729 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.729 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.729 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.729 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.729 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.729 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.729 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.729 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.730 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.730 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.730 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.730 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.730 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "iostat", 2024-09-15T23:12:11.730 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.730 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.730 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.730 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.730 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.730 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.730 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.731 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.731 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.731 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.731 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.731 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.731 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.731 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.731 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.731 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.731 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.731 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.731 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.731 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.732 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.732 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.732 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.732 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.732 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.732 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.732 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.732 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.732 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.732 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.732 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.733 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.733 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.733 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.733 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.733 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.733 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.733 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.733 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.733 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.733 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.733 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.733 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.734 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.734 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.734 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.734 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.734 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.734 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.734 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.734 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.734 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.734 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.734 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.734 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.735 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.735 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.735 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.735 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.735 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.735 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.735 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.735 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.735 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.735 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.735 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.735 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.736 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.736 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.736 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.736 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.736 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.736 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.736 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.736 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.736 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.736 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "k8sevents", 2024-09-15T23:12:11.736 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.736 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.736 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.737 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_event_retention_days": { 2024-09-15T23:12:11.737 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ceph_event_retention_days", 2024-09-15T23:12:11.737 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.737 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.737 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.737 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "7", 2024-09-15T23:12:11.737 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.737 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.737 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.737 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-15T23:12:11.737 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.738 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.738 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.738 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.738 INFO:teuthology.orchestra.run.smithi111.stdout: "config_check_secs": { 2024-09-15T23:12:11.738 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "config_check_secs", 2024-09-15T23:12:11.738 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.738 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.738 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.738 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-09-15T23:12:11.738 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "10", 2024-09-15T23:12:11.738 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.738 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.739 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-15T23:12:11.739 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.739 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.739 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.739 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.739 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.739 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.739 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.739 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.739 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.739 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.740 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.740 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.740 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.740 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.740 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.740 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.740 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.740 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.740 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.741 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.741 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.741 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.741 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.741 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.741 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.741 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.741 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.741 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.741 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.741 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.742 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.742 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.742 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.742 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.742 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.742 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.742 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.742 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.742 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.742 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.742 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.742 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.744 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.744 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.744 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.744 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.744 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.744 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.744 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.744 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.745 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.745 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.745 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.745 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.745 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.745 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.745 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.745 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.745 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.745 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.745 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.746 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.746 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.746 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.746 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.746 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.746 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.746 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.746 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.746 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.746 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.746 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.747 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.747 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.747 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.747 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.747 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.747 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "localpool", 2024-09-15T23:12:11.747 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.747 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.747 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.747 INFO:teuthology.orchestra.run.smithi111.stdout: "failure_domain": { 2024-09-15T23:12:11.747 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "failure_domain", 2024-09-15T23:12:11.747 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.747 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.748 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.748 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "host", 2024-09-15T23:12:11.748 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.748 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.748 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.748 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "failure domain for any created local pool", 2024-09-15T23:12:11.748 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-15T23:12:11.748 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.748 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.748 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.748 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.749 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.749 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.749 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.749 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.749 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.749 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.749 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.749 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.749 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.749 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.749 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.749 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.750 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.750 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.750 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.750 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.750 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.750 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.750 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.750 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.750 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.750 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.750 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.751 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.751 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.751 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.751 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.751 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.751 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.751 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.751 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.751 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.751 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.751 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.751 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.752 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.752 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.752 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.752 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.752 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.752 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.752 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.752 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.752 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.752 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.752 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.752 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.753 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.753 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.753 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.753 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.753 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.753 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.753 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.753 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.753 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.753 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.753 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.754 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.754 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.754 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.754 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.754 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.754 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.754 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.754 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.754 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.754 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.754 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.755 INFO:teuthology.orchestra.run.smithi111.stdout: "min_size": { 2024-09-15T23:12:11.755 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "min_size", 2024-09-15T23:12:11.755 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.755 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.755 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.755 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.755 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.755 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.755 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.755 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "default min_size for any created local pool", 2024-09-15T23:12:11.755 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-15T23:12:11.755 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.756 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.756 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.756 INFO:teuthology.orchestra.run.smithi111.stdout: "num_rep": { 2024-09-15T23:12:11.756 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "num_rep", 2024-09-15T23:12:11.756 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.756 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.756 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.756 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "3", 2024-09-15T23:12:11.756 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.756 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.756 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.756 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "default replica count for any created local pool", 2024-09-15T23:12:11.757 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.757 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.757 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.757 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.757 INFO:teuthology.orchestra.run.smithi111.stdout: "pg_num": { 2024-09-15T23:12:11.757 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "pg_num", 2024-09-15T23:12:11.757 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.757 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.757 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.757 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "128", 2024-09-15T23:12:11.757 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.757 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.758 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.758 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "default pg_num for any created local pool", 2024-09-15T23:12:11.758 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.758 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.758 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.758 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.758 INFO:teuthology.orchestra.run.smithi111.stdout: "prefix": { 2024-09-15T23:12:11.758 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "prefix", 2024-09-15T23:12:11.758 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.758 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.758 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.758 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.759 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.759 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.759 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.759 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "name prefix for any created local pool", 2024-09-15T23:12:11.759 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.759 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.759 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.759 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.759 INFO:teuthology.orchestra.run.smithi111.stdout: "subtree": { 2024-09-15T23:12:11.759 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "subtree", 2024-09-15T23:12:11.759 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.760 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.760 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.760 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "rack", 2024-09-15T23:12:11.760 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.760 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.760 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.760 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-15T23:12:11.760 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-15T23:12:11.760 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.760 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.760 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.761 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.761 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.761 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.761 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mds_autoscaler", 2024-09-15T23:12:11.761 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.761 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.761 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.761 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.761 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.761 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.761 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.762 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.762 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.762 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.762 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.762 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.762 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.762 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.762 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.762 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.762 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.762 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.762 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.763 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.763 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.763 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.763 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.763 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.763 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.763 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.763 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.763 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.763 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.763 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.763 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.764 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.764 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.764 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.764 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.764 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.764 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.764 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.764 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.764 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.764 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.764 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.765 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.765 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.765 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.765 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.765 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.765 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.765 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.765 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.765 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.765 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.765 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.766 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.766 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.766 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.766 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.766 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.766 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.766 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.766 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.766 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.766 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.766 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.766 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.766 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.767 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.767 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.767 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.767 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.767 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.767 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.767 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.767 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.767 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.767 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.767 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mirroring", 2024-09-15T23:12:11.767 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.767 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.768 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.768 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.768 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.768 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.768 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.768 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.768 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.768 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.768 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.768 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.768 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.768 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.769 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.769 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.769 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.769 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.769 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.769 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.769 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.769 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.769 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.769 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.769 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.769 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.770 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.770 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.770 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.770 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.770 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.770 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.770 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.770 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.770 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.770 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.770 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.770 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.770 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.771 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.771 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.771 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.771 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.771 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.771 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.771 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.771 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.771 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.771 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.771 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.771 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.771 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.772 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.772 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.772 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.772 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.772 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.772 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.772 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.772 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.772 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.772 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.772 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.773 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.773 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.773 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.773 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.773 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.773 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.773 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.773 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.773 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.773 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.773 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.774 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.774 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.774 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "nfs", 2024-09-15T23:12:11.774 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.774 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.774 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.774 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.774 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.774 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.774 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.774 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.775 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.775 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.775 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.775 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.775 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.775 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.775 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.775 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.775 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.775 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.775 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.775 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.776 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.776 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.776 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.776 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.776 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.776 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.776 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.776 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.776 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.776 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.776 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.777 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.777 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.777 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.777 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.777 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.777 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.777 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.777 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.777 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.777 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.777 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.778 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.778 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.778 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.778 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.778 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.778 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.778 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.778 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.778 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.778 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.778 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.778 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.779 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.779 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.779 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.779 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.779 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.779 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.779 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.779 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.779 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.779 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.779 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.779 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.780 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.780 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.780 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.780 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.780 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.780 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.780 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.780 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.780 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.780 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.780 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "orchestrator", 2024-09-15T23:12:11.781 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.781 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.781 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.781 INFO:teuthology.orchestra.run.smithi111.stdout: "fail_fs": { 2024-09-15T23:12:11.781 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "fail_fs", 2024-09-15T23:12:11.781 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.781 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.781 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.781 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.781 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.781 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.781 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.782 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-15T23:12:11.782 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.782 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.782 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.782 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.782 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.782 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.782 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.782 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.782 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.782 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.783 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.783 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.783 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.783 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.783 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.783 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.783 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.783 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.783 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.783 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.783 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.783 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.784 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.784 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.784 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.784 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.784 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.784 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.784 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.784 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.784 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.784 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.784 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.785 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.785 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.785 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.785 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.785 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.785 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.785 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.785 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.785 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.785 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.785 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.786 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.786 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.786 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.786 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.786 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.786 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.786 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.786 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.786 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.786 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.786 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.786 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.787 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.787 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.787 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.787 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.787 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.787 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.787 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.787 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.787 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.787 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.787 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.787 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.788 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.788 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.788 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.788 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.788 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.788 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.788 INFO:teuthology.orchestra.run.smithi111.stdout: "orchestrator": { 2024-09-15T23:12:11.788 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "orchestrator", 2024-09-15T23:12:11.788 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.788 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.788 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.788 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.789 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.789 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.789 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.789 INFO:teuthology.orchestra.run.smithi111.stdout: "cephadm", 2024-09-15T23:12:11.789 INFO:teuthology.orchestra.run.smithi111.stdout: "rook", 2024-09-15T23:12:11.789 INFO:teuthology.orchestra.run.smithi111.stdout: "test_orchestrator" 2024-09-15T23:12:11.789 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.789 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Orchestrator backend", 2024-09-15T23:12:11.789 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.790 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.790 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.790 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.790 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.790 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.790 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.790 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "osd_perf_query", 2024-09-15T23:12:11.790 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.790 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.790 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.790 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.790 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.791 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.791 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.791 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.791 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.791 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.791 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.791 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.791 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.791 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.791 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.791 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.791 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.792 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.792 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.792 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.792 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.792 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.792 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.792 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.792 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.792 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.792 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.792 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.793 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.793 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.793 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.793 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.793 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.793 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.793 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.793 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.793 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.793 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.793 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.793 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.794 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.794 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.794 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.794 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.794 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.794 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.794 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.794 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.794 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.794 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.794 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.795 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.795 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.795 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.795 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.795 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.795 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.795 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.795 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.795 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.795 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.795 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.796 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.796 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.796 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.796 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.796 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.796 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.796 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.796 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.796 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.796 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.796 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.797 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.797 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.797 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.797 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "osd_support", 2024-09-15T23:12:11.797 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.797 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.797 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.797 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.797 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.797 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.797 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.797 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.798 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.798 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.798 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.798 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.798 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.798 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.798 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.798 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.798 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.798 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.798 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.799 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.799 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.799 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.799 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.799 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.799 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.799 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.799 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.799 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.799 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.799 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.800 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.800 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.800 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.800 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.800 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.800 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.800 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.800 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.800 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.800 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.800 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.801 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.801 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.801 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.801 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.801 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.801 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.801 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.801 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.801 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.801 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.801 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.801 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.802 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.802 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.802 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.802 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.802 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.802 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.802 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.802 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.802 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.802 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.802 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.802 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.803 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.803 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.803 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.803 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.803 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.803 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.803 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.803 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.803 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.803 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.803 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.804 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "pg_autoscaler", 2024-09-15T23:12:11.804 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.804 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.804 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.804 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.804 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.804 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.804 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.804 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.804 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.804 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.805 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.805 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.805 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.805 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.805 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.805 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.805 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.805 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.805 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.805 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.805 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.805 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.806 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.806 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.806 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.806 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.806 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.806 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.806 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.806 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.806 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.806 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.806 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.806 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.807 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.807 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.807 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.807 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.807 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.807 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.807 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.807 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.807 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.807 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.808 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.808 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.808 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.808 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.808 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.808 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.808 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.808 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.808 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.808 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.808 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.808 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.809 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.809 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.809 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.809 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.809 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.809 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.809 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.809 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.809 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.809 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.809 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.810 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.810 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.810 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.810 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.810 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.810 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.810 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-09-15T23:12:11.810 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-09-15T23:12:11.810 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.810 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.810 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.811 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-09-15T23:12:11.811 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.811 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.811 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.811 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.811 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.811 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.811 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.811 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.811 INFO:teuthology.orchestra.run.smithi111.stdout: "threshold": { 2024-09-15T23:12:11.811 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "threshold", 2024-09-15T23:12:11.812 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-09-15T23:12:11.812 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.812 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.812 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "3.0", 2024-09-15T23:12:11.812 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1.0", 2024-09-15T23:12:11.812 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.812 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.812 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "scaling threshold", 2024-09-15T23:12:11.812 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-15T23:12:11.812 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.812 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.813 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.813 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.813 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.813 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.813 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "progress", 2024-09-15T23:12:11.813 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.813 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.813 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.813 INFO:teuthology.orchestra.run.smithi111.stdout: "allow_pg_recovery_event": { 2024-09-15T23:12:11.813 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "allow_pg_recovery_event", 2024-09-15T23:12:11.813 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.814 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.814 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.814 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.814 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.814 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.814 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.814 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-15T23:12:11.814 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.814 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.814 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.814 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.815 INFO:teuthology.orchestra.run.smithi111.stdout: "enabled": { 2024-09-15T23:12:11.815 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "enabled", 2024-09-15T23:12:11.815 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.815 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.815 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.815 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.815 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.815 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.815 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.815 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.815 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.816 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.816 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.816 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.816 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.816 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.816 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.816 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.816 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.816 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.816 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.816 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.817 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.817 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.817 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.817 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.817 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.817 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.817 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.817 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.817 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.817 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.817 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.817 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.818 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.818 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.818 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.818 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.818 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.818 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.818 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.818 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.818 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.818 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.818 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.819 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.819 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.819 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.819 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.819 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.819 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.819 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.819 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.819 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.819 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.819 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.820 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.820 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.820 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.820 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.820 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.820 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.820 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.820 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.820 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.820 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.820 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.820 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.821 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.821 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.821 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.821 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.821 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.821 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.821 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.821 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.821 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.821 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.821 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.822 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.822 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.822 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.822 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.822 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.822 INFO:teuthology.orchestra.run.smithi111.stdout: "max_completed_events": { 2024-09-15T23:12:11.822 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_completed_events", 2024-09-15T23:12:11.822 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.822 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.822 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.822 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "50", 2024-09-15T23:12:11.822 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.823 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.823 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.823 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "number of past completed events to remember", 2024-09-15T23:12:11.823 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.823 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.823 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.823 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.823 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-09-15T23:12:11.823 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-09-15T23:12:11.823 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.823 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.823 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.824 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5", 2024-09-15T23:12:11.824 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.824 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.824 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.824 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how long the module is going to sleep", 2024-09-15T23:12:11.824 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.824 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.824 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.824 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.824 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.825 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.825 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.825 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "prometheus", 2024-09-15T23:12:11.825 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.825 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.825 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.825 INFO:teuthology.orchestra.run.smithi111.stdout: "cache": { 2024-09-15T23:12:11.825 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cache", 2024-09-15T23:12:11.825 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.825 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.825 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.825 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.826 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.826 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.826 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.826 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.826 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.826 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.826 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.826 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.826 INFO:teuthology.orchestra.run.smithi111.stdout: "exclude_perf_counters": { 2024-09-15T23:12:11.826 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "exclude_perf_counters", 2024-09-15T23:12:11.826 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.826 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.827 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.827 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.827 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.827 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.827 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.827 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-15T23:12:11.827 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-15T23:12:11.827 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.827 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.827 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.827 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.827 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.828 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.828 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.828 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.828 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.828 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.828 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.828 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.828 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.828 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.828 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.828 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.829 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.829 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.829 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.829 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.829 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.829 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.829 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.829 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.829 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.829 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.829 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.830 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.830 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.830 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.830 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.830 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.830 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.830 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.830 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.830 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.830 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.830 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.830 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.831 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.831 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.831 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.831 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.831 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.831 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.831 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.831 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.831 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.831 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.831 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.831 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.832 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.832 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.832 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.832 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.832 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.832 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.832 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.832 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.832 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.832 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.832 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.833 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.833 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.833 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.833 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.833 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.833 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.833 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.833 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.833 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.833 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.833 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.834 INFO:teuthology.orchestra.run.smithi111.stdout: "rbd_stats_pools": { 2024-09-15T23:12:11.834 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rbd_stats_pools", 2024-09-15T23:12:11.834 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.834 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.834 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.834 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.834 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.834 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.834 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.834 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.834 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.834 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.835 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.835 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.835 INFO:teuthology.orchestra.run.smithi111.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-15T23:12:11.835 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-15T23:12:11.835 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.835 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.835 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.835 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "300", 2024-09-15T23:12:11.835 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.835 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.835 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.836 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.836 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.836 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.836 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.836 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.836 INFO:teuthology.orchestra.run.smithi111.stdout: "scrape_interval": { 2024-09-15T23:12:11.836 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "scrape_interval", 2024-09-15T23:12:11.836 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-09-15T23:12:11.836 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.836 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.836 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "15.0", 2024-09-15T23:12:11.836 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.837 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.837 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.837 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.837 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.837 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.837 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.837 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.837 INFO:teuthology.orchestra.run.smithi111.stdout: "server_addr": { 2024-09-15T23:12:11.837 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_addr", 2024-09-15T23:12:11.837 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.837 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.838 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.838 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "::", 2024-09-15T23:12:11.838 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.838 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.838 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.838 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-15T23:12:11.838 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.838 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.838 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.838 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.838 INFO:teuthology.orchestra.run.smithi111.stdout: "server_port": { 2024-09-15T23:12:11.838 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_port", 2024-09-15T23:12:11.839 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.839 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.839 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.839 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "9283", 2024-09-15T23:12:11.839 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.839 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.839 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.839 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-15T23:12:11.839 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.839 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.839 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.840 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.840 INFO:teuthology.orchestra.run.smithi111.stdout: "stale_cache_strategy": { 2024-09-15T23:12:11.840 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "stale_cache_strategy", 2024-09-15T23:12:11.840 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.840 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.840 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.840 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "log", 2024-09-15T23:12:11.840 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.840 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.840 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.840 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.841 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.841 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.841 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.841 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.841 INFO:teuthology.orchestra.run.smithi111.stdout: "standby_behaviour": { 2024-09-15T23:12:11.841 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "standby_behaviour", 2024-09-15T23:12:11.841 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.841 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.841 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.841 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "default", 2024-09-15T23:12:11.841 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.842 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.842 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.842 INFO:teuthology.orchestra.run.smithi111.stdout: "default", 2024-09-15T23:12:11.842 INFO:teuthology.orchestra.run.smithi111.stdout: "error" 2024-09-15T23:12:11.842 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.842 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.842 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.842 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.842 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.842 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.842 INFO:teuthology.orchestra.run.smithi111.stdout: "standby_error_status_code": { 2024-09-15T23:12:11.842 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "standby_error_status_code", 2024-09-15T23:12:11.843 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.843 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.843 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.843 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "500", 2024-09-15T23:12:11.843 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "400", 2024-09-15T23:12:11.843 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "599", 2024-09-15T23:12:11.843 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.843 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.843 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.843 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.844 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.844 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.844 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.844 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.844 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.844 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rbd_support", 2024-09-15T23:12:11.844 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.844 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.844 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.844 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.844 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.845 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.845 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.845 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.845 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.845 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.845 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.845 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.845 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.845 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.845 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.846 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.846 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.846 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.846 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.846 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.846 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.846 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.846 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.846 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.846 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.847 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.847 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.847 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.847 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.847 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.847 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.847 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.847 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.847 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.847 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.847 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.847 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.848 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.848 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.848 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.848 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.848 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.848 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.848 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.848 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.848 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.848 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.848 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.848 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.849 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.849 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.849 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.849 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.849 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.849 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.849 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.849 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.849 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.849 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.849 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.850 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.850 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.850 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.850 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.850 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.850 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.850 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.850 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.850 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.850 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.850 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.851 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.851 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.851 INFO:teuthology.orchestra.run.smithi111.stdout: "max_concurrent_snap_create": { 2024-09-15T23:12:11.851 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_concurrent_snap_create", 2024-09-15T23:12:11.851 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.851 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.851 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.851 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-09-15T23:12:11.851 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.851 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.851 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.852 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.852 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.852 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.852 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.852 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.852 INFO:teuthology.orchestra.run.smithi111.stdout: "mirror_snapshot_schedule": { 2024-09-15T23:12:11.852 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mirror_snapshot_schedule", 2024-09-15T23:12:11.852 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.852 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.852 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.852 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.853 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.853 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.853 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.853 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.853 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.853 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.853 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.853 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.853 INFO:teuthology.orchestra.run.smithi111.stdout: "trash_purge_schedule": { 2024-09-15T23:12:11.853 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "trash_purge_schedule", 2024-09-15T23:12:11.853 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.854 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.854 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.854 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.854 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.854 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.854 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.854 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.854 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.854 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.854 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.854 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.854 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.855 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.855 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.855 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "restful", 2024-09-15T23:12:11.855 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.855 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.855 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.855 INFO:teuthology.orchestra.run.smithi111.stdout: "enable_auth": { 2024-09-15T23:12:11.855 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "enable_auth", 2024-09-15T23:12:11.855 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.855 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.855 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.856 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.856 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.856 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.856 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.856 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.856 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.856 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.856 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.856 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.856 INFO:teuthology.orchestra.run.smithi111.stdout: "key_file": { 2024-09-15T23:12:11.856 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "key_file", 2024-09-15T23:12:11.857 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.857 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.857 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.857 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.857 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.857 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.857 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.857 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.857 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.857 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.857 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.858 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.858 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.858 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.858 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.858 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.858 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.858 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.858 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.858 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.858 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.859 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.859 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.859 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.859 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.859 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.859 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.859 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.859 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.859 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.859 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.859 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.859 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.860 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.860 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.860 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.860 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.860 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.860 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.860 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.860 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.860 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.860 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.860 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.861 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.861 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.861 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.861 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.861 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.861 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.861 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.861 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.861 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.861 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.862 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.862 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.862 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.862 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.862 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.862 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.862 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.862 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.862 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.862 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.862 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.862 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.863 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.863 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.863 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.863 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.863 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.863 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.863 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.863 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.863 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.863 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.863 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.864 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.864 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.864 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.864 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.864 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.864 INFO:teuthology.orchestra.run.smithi111.stdout: "server_addr": { 2024-09-15T23:12:11.864 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_addr", 2024-09-15T23:12:11.864 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.864 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.864 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.864 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.865 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.865 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.865 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.865 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.865 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.865 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.865 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.865 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.865 INFO:teuthology.orchestra.run.smithi111.stdout: "server_port": { 2024-09-15T23:12:11.865 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_port", 2024-09-15T23:12:11.865 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.865 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.866 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.866 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.866 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.866 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.866 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.866 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.866 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.866 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.866 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.866 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.866 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.867 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.867 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.867 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rgw", 2024-09-15T23:12:11.867 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.867 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.867 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.867 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.867 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.867 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.867 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.867 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.867 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.868 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.868 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.868 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.868 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.868 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.868 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.868 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.868 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.868 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.868 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.868 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.868 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.869 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.869 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.869 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.869 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.869 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.869 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.869 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.869 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.869 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.869 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.869 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.870 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.870 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.870 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.870 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.870 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.870 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.870 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.870 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.870 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.870 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.870 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.870 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.871 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.871 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.871 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.871 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.871 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.871 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.871 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.871 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.871 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.871 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.871 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.871 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.872 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.872 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.872 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.872 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.872 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.872 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.872 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.872 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.872 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.872 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.872 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.872 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.873 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.873 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.873 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.873 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.873 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.873 INFO:teuthology.orchestra.run.smithi111.stdout: "secondary_zone_period_retry_limit": { 2024-09-15T23:12:11.873 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-15T23:12:11.873 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.873 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.873 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.873 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5", 2024-09-15T23:12:11.873 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.874 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.874 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.874 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-15T23:12:11.874 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.874 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.874 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.874 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.874 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.874 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.874 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.875 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rook", 2024-09-15T23:12:11.875 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.875 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.875 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.875 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.875 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.875 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.875 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.875 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.875 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.875 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.875 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.876 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.876 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.876 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.876 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.876 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.876 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.876 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.876 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.876 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.876 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.876 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.877 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.877 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.877 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.877 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.877 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.877 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.877 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.877 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.877 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.877 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.877 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.878 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.878 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.878 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.878 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.878 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.878 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.878 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.878 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.878 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.878 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.878 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.879 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.879 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.879 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.879 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.879 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.879 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.879 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.879 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.879 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.879 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.879 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.880 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.880 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.880 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.880 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.880 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.880 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.880 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.880 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.880 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.880 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.880 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.881 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.881 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.881 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.881 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.881 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.881 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.881 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.881 INFO:teuthology.orchestra.run.smithi111.stdout: "storage_class": { 2024-09-15T23:12:11.881 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "storage_class", 2024-09-15T23:12:11.881 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.881 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.882 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.882 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "local", 2024-09-15T23:12:11.882 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.882 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.882 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.882 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-15T23:12:11.882 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.882 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.882 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.882 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.882 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.882 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.883 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.883 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "selftest", 2024-09-15T23:12:11.883 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.883 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.883 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.883 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.883 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.883 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.883 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.883 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.884 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.884 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.884 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.884 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.884 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.884 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.884 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.884 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.884 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.884 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.884 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.885 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.885 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.885 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.885 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.885 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.885 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.885 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.885 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.885 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.885 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.885 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.886 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.886 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.886 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.886 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.886 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.886 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.886 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.886 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.886 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.886 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.886 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.887 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.887 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.887 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.887 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.887 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.887 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.887 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.887 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.887 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.887 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.887 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.887 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.888 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.888 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.888 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.888 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.888 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.888 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.888 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.888 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.888 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.888 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.888 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.889 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.889 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.889 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.889 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.889 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.889 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.889 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.889 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.889 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.889 INFO:teuthology.orchestra.run.smithi111.stdout: "roption1": { 2024-09-15T23:12:11.889 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "roption1", 2024-09-15T23:12:11.890 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.890 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.890 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.890 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.890 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.890 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.890 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.890 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.890 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.890 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.890 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.890 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.891 INFO:teuthology.orchestra.run.smithi111.stdout: "roption2": { 2024-09-15T23:12:11.891 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "roption2", 2024-09-15T23:12:11.891 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.891 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.891 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.891 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "xyz", 2024-09-15T23:12:11.891 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.891 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.891 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.891 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.891 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.891 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.892 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.892 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.892 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption1": { 2024-09-15T23:12:11.892 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption1", 2024-09-15T23:12:11.892 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.892 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.892 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.892 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.892 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.892 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.892 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.893 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.893 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.893 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.893 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.893 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.893 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption2": { 2024-09-15T23:12:11.893 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption2", 2024-09-15T23:12:11.893 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.893 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.893 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.893 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.894 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.894 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.894 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.894 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.894 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.894 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.894 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.894 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.894 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption3": { 2024-09-15T23:12:11.894 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption3", 2024-09-15T23:12:11.894 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-09-15T23:12:11.894 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.895 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.895 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.895 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.895 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.895 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.895 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.895 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.895 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.895 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.895 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.895 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption4": { 2024-09-15T23:12:11.896 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption4", 2024-09-15T23:12:11.896 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.896 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.896 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.896 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.896 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.896 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.896 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.896 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.896 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.896 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.897 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.897 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.897 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption5": { 2024-09-15T23:12:11.897 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption5", 2024-09-15T23:12:11.897 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.897 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.897 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.897 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.897 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.897 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.897 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.898 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.898 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.898 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.898 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.898 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.898 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption6": { 2024-09-15T23:12:11.898 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption6", 2024-09-15T23:12:11.898 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.898 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.898 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.898 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.898 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.899 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.899 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.899 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.899 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.899 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.899 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.899 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.899 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption7": { 2024-09-15T23:12:11.899 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption7", 2024-09-15T23:12:11.899 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.899 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.900 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.900 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.900 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1", 2024-09-15T23:12:11.900 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "42", 2024-09-15T23:12:11.900 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.900 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.900 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.900 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.900 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.900 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.900 INFO:teuthology.orchestra.run.smithi111.stdout: "testkey": { 2024-09-15T23:12:11.901 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "testkey", 2024-09-15T23:12:11.901 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.901 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.901 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.901 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.901 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.901 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.901 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.901 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.901 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.901 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.901 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.902 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.902 INFO:teuthology.orchestra.run.smithi111.stdout: "testlkey": { 2024-09-15T23:12:11.902 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "testlkey", 2024-09-15T23:12:11.902 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.902 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.902 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.902 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.902 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.902 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.902 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.902 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.903 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.903 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.903 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.903 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.903 INFO:teuthology.orchestra.run.smithi111.stdout: "testnewline": { 2024-09-15T23:12:11.903 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "testnewline", 2024-09-15T23:12:11.903 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.903 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.903 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.903 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.903 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.903 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.903 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.904 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.904 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.904 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.904 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.904 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.904 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.904 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.904 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.904 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "snap_schedule", 2024-09-15T23:12:11.904 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.904 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.905 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.905 INFO:teuthology.orchestra.run.smithi111.stdout: "allow_m_granularity": { 2024-09-15T23:12:11.905 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "allow_m_granularity", 2024-09-15T23:12:11.905 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.905 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.905 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.905 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.905 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.905 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.905 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.905 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "allow minute scheduled snapshots", 2024-09-15T23:12:11.906 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.906 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.906 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.906 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.906 INFO:teuthology.orchestra.run.smithi111.stdout: "dump_on_update": { 2024-09-15T23:12:11.906 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "dump_on_update", 2024-09-15T23:12:11.906 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.906 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.906 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.906 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.906 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.906 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.907 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.907 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "dump database to debug log on update", 2024-09-15T23:12:11.907 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.907 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.907 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.907 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.907 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.907 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.907 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.907 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.907 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.908 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.908 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.908 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.908 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.908 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.908 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.908 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.908 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.908 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.908 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.908 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.908 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.909 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.909 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.909 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.909 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.909 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.909 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.909 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.909 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.909 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.909 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.909 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.909 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.910 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.910 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.910 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.910 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.910 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.910 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.910 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.910 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.910 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.910 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.910 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.911 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.911 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.911 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.911 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.911 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.911 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.911 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.911 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.911 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.911 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.911 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.912 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.912 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.912 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.912 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.912 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.912 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.912 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.912 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.912 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.912 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.912 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.912 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.913 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.913 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.913 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.913 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.913 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.913 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.913 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.913 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.913 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.913 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.913 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "stats", 2024-09-15T23:12:11.913 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.914 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.914 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.914 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.914 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.914 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.914 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.914 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.914 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.914 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.914 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.914 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.915 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.915 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.915 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.915 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.915 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.915 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.915 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.915 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.915 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.915 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.915 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.916 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.916 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.916 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.916 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.916 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.916 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.916 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.916 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.916 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.916 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.916 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.916 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.917 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.917 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.917 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.917 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.917 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.917 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.917 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.917 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.917 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.917 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.917 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.918 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.918 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.918 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.918 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.918 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.918 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.918 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.918 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.918 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.918 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.918 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.919 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.919 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.919 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.919 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.919 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.919 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.919 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.919 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.919 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.919 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.919 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.919 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.920 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.920 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.920 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.920 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.920 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.920 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.920 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.920 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "status", 2024-09-15T23:12:11.920 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.920 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.920 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.920 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.921 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.921 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.921 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.921 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.921 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.921 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.921 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.921 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.921 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.921 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.921 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.922 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.922 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.922 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.922 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.922 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.922 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.922 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.922 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.922 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.922 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.922 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.923 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.923 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.923 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.923 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.923 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.923 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.923 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.923 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.923 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.923 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.923 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.923 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.924 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.924 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.924 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.924 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.924 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.924 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.924 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.924 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.924 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.924 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.924 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.924 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.925 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.925 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.925 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.925 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.925 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.925 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.925 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.925 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.925 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.925 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.925 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.926 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.926 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.926 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.926 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.926 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.926 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.926 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.926 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.926 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.926 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.926 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.927 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.927 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.927 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.927 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.927 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "telegraf", 2024-09-15T23:12:11.927 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.927 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.927 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.927 INFO:teuthology.orchestra.run.smithi111.stdout: "address": { 2024-09-15T23:12:11.927 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "address", 2024-09-15T23:12:11.927 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.927 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.928 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.928 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-15T23:12:11.928 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.928 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.928 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.928 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.928 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.928 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.928 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.928 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.928 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-09-15T23:12:11.929 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-09-15T23:12:11.929 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.929 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.929 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.929 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "15", 2024-09-15T23:12:11.929 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.929 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.929 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.929 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.929 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.929 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.930 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.930 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.930 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.930 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.930 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.930 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.930 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.930 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.930 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.930 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.930 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.931 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.931 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.931 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.931 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.931 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.931 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.931 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.931 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.931 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.931 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.931 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.931 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.932 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.932 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.932 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.932 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.932 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.932 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.932 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.932 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.932 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.932 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.932 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.933 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.933 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.933 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.933 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.933 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.933 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.933 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.933 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.933 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.933 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.933 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.933 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.934 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.934 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.934 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.934 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.934 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.934 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.934 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.934 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.934 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.934 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.934 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.935 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.935 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.935 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.935 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.935 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.935 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.935 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.935 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.935 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.935 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.935 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.936 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.936 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.936 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.936 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.936 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.936 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.936 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.936 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "telemetry", 2024-09-15T23:12:11.936 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.936 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.936 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.937 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_basic": { 2024-09-15T23:12:11.937 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_basic", 2024-09-15T23:12:11.937 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.937 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.937 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.937 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.937 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.937 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.937 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.937 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-15T23:12:11.937 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.938 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.938 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.938 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.938 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_crash": { 2024-09-15T23:12:11.938 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_crash", 2024-09-15T23:12:11.938 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.938 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.938 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.938 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.938 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.938 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.939 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.939 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-15T23:12:11.939 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.939 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.939 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.939 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.939 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_device": { 2024-09-15T23:12:11.939 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_device", 2024-09-15T23:12:11.939 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.939 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.939 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.940 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.940 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.940 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.940 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.940 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-15T23:12:11.940 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.940 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.940 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.940 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.940 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_ident": { 2024-09-15T23:12:11.940 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_ident", 2024-09-15T23:12:11.941 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.941 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.941 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.941 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.941 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.941 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.941 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.941 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-15T23:12:11.941 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.941 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.941 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.942 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.942 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_perf": { 2024-09-15T23:12:11.942 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_perf", 2024-09-15T23:12:11.942 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.942 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.942 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.942 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.942 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.942 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.942 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.942 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-15T23:12:11.942 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.942 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.943 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.943 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.943 INFO:teuthology.orchestra.run.smithi111.stdout: "contact": { 2024-09-15T23:12:11.943 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "contact", 2024-09-15T23:12:11.943 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.943 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.943 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.943 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.943 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.943 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.943 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.943 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.943 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.944 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.944 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.944 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.944 INFO:teuthology.orchestra.run.smithi111.stdout: "description": { 2024-09-15T23:12:11.944 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "description", 2024-09-15T23:12:11.944 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.944 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.944 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.944 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.944 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.944 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.944 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.945 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.945 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.945 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.945 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.945 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.945 INFO:teuthology.orchestra.run.smithi111.stdout: "device_url": { 2024-09-15T23:12:11.945 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "device_url", 2024-09-15T23:12:11.945 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.945 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.945 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.945 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-15T23:12:11.946 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.946 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.946 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.946 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.946 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.946 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.946 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.946 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.946 INFO:teuthology.orchestra.run.smithi111.stdout: "enabled": { 2024-09-15T23:12:11.946 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "enabled", 2024-09-15T23:12:11.946 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.947 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.947 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.947 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.947 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.947 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.947 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.947 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.947 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.947 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.947 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.947 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.948 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-09-15T23:12:11.948 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-09-15T23:12:11.948 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.948 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.948 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.948 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "24", 2024-09-15T23:12:11.948 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "8", 2024-09-15T23:12:11.948 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.948 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.948 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.948 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.949 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.949 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.949 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.949 INFO:teuthology.orchestra.run.smithi111.stdout: "last_opt_revision": { 2024-09-15T23:12:11.949 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "last_opt_revision", 2024-09-15T23:12:11.949 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.949 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.949 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.949 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "1", 2024-09-15T23:12:11.949 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.950 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.950 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.950 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.950 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.950 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.950 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.950 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.950 INFO:teuthology.orchestra.run.smithi111.stdout: "leaderboard": { 2024-09-15T23:12:11.950 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "leaderboard", 2024-09-15T23:12:11.950 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.950 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.950 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.951 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.951 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.951 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.951 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.951 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.951 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.951 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.951 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.951 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.951 INFO:teuthology.orchestra.run.smithi111.stdout: "leaderboard_description": { 2024-09-15T23:12:11.951 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "leaderboard_description", 2024-09-15T23:12:11.952 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.952 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.952 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.952 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.952 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.952 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.952 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.952 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.952 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.952 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.952 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.953 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.953 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.953 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.953 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.953 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.953 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.953 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.953 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.953 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.953 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.953 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.953 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.954 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.954 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.954 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.954 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.954 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.954 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.954 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.954 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.954 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.954 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.954 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.954 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.955 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.955 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.955 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.955 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.955 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.955 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.955 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.955 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.955 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.955 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.955 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.956 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.956 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.956 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.956 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.956 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.956 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.956 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.956 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.956 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.956 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.956 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.956 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.957 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.957 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.957 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.957 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.957 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.957 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.957 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.957 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.957 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.957 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.957 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.957 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.957 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.958 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.958 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.958 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.958 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.958 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.958 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.958 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.958 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.958 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.958 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.958 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.958 INFO:teuthology.orchestra.run.smithi111.stdout: "organization": { 2024-09-15T23:12:11.958 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "organization", 2024-09-15T23:12:11.958 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.959 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.959 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.959 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.959 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.959 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.959 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.959 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.959 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.959 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.959 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.960 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.960 INFO:teuthology.orchestra.run.smithi111.stdout: "proxy": { 2024-09-15T23:12:11.960 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "proxy", 2024-09-15T23:12:11.960 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.960 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.960 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.960 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.960 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.960 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.960 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.960 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.961 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.961 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.961 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.961 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.961 INFO:teuthology.orchestra.run.smithi111.stdout: "url": { 2024-09-15T23:12:11.961 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "url", 2024-09-15T23:12:11.961 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.961 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.961 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.961 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-15T23:12:11.961 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.962 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.962 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.962 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.962 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.962 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.962 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.962 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.962 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.962 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.962 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.962 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "test_orchestrator", 2024-09-15T23:12:11.963 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.963 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.963 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.963 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.963 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.963 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.963 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.963 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.963 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.963 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.963 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.964 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.964 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.964 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.964 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.964 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.964 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.964 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.964 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.964 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.964 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.964 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.965 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.965 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.965 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.965 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.965 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.965 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.965 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.965 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.965 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.965 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.965 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.966 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.966 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.966 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.966 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.966 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.966 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.966 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.966 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.966 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.966 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.966 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.967 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.967 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.967 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.967 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.967 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.967 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.967 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.967 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.967 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.967 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.967 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.967 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.968 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.968 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.968 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.968 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.968 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.968 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.968 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.968 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.968 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.968 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.968 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.968 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.968 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.969 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.969 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.969 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.969 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.969 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.969 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.969 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.969 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "volumes", 2024-09-15T23:12:11.969 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.969 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.969 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.969 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.970 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.970 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.970 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.970 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.970 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.970 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.970 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.970 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.970 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.970 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.970 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.970 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.970 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.971 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.971 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.971 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.971 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.971 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.971 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.971 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.971 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.971 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.971 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.971 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.971 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.972 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.972 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.972 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.972 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.972 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.972 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.972 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.972 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.972 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.972 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.973 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.973 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.973 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.973 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.973 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.973 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.973 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.973 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.973 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.973 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.973 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.974 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.974 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.974 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.974 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.974 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.974 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.974 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.974 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.974 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.974 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.974 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.975 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.975 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.975 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.975 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.975 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.975 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.975 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.975 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.975 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.975 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.975 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.976 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.976 INFO:teuthology.orchestra.run.smithi111.stdout: "max_concurrent_clones": { 2024-09-15T23:12:11.976 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_concurrent_clones", 2024-09-15T23:12:11.976 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.976 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.976 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.976 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "4", 2024-09-15T23:12:11.976 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.976 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.976 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.977 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-15T23:12:11.977 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.977 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.977 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.977 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.977 INFO:teuthology.orchestra.run.smithi111.stdout: "periodic_async_work": { 2024-09-15T23:12:11.977 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "periodic_async_work", 2024-09-15T23:12:11.977 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.977 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.977 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.977 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.978 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.978 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.978 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.978 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Periodically check for async work", 2024-09-15T23:12:11.978 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.978 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.978 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.978 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.978 INFO:teuthology.orchestra.run.smithi111.stdout: "snapshot_clone_delay": { 2024-09-15T23:12:11.978 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "snapshot_clone_delay", 2024-09-15T23:12:11.978 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.978 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.979 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.979 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-09-15T23:12:11.979 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.979 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.979 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.979 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-15T23:12:11.979 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.979 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.979 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.979 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.979 INFO:teuthology.orchestra.run.smithi111.stdout: "snapshot_clone_no_wait": { 2024-09-15T23:12:11.979 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "snapshot_clone_no_wait", 2024-09-15T23:12:11.980 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.980 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.980 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.980 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:11.980 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.980 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.980 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.980 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-15T23:12:11.980 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.980 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.980 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.981 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.981 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.981 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.981 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.981 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "zabbix", 2024-09-15T23:12:11.981 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.981 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.981 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.981 INFO:teuthology.orchestra.run.smithi111.stdout: "discovery_interval": { 2024-09-15T23:12:11.981 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "discovery_interval", 2024-09-15T23:12:11.981 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-09-15T23:12:11.981 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.982 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.982 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "100", 2024-09-15T23:12:11.982 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.982 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.982 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.982 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.982 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.982 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.982 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.982 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.982 INFO:teuthology.orchestra.run.smithi111.stdout: "identifier": { 2024-09-15T23:12:11.982 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "identifier", 2024-09-15T23:12:11.983 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.983 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.983 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.983 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.983 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.983 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.983 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.983 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.983 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.983 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.983 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.984 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.984 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-09-15T23:12:11.984 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-09-15T23:12:11.984 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.984 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.984 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.984 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-09-15T23:12:11.984 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.984 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.984 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.984 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.984 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.985 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.985 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.985 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.985 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.985 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.985 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.985 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.985 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.985 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.985 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.985 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.986 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.986 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.986 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.986 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.986 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.986 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.986 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.986 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.986 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.986 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.986 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.986 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.987 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.987 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.987 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.987 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.987 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.987 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.987 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.987 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.987 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.987 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.987 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.988 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.988 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.988 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.988 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.988 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:11.988 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:11.988 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.988 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.988 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.988 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:11.988 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.988 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.989 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.989 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.989 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.989 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.989 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.989 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.989 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.989 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.989 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.989 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.989 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.990 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.990 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.990 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:11.990 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:11.990 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.990 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.990 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.990 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:11.990 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.990 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.990 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.991 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.991 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.991 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.991 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.991 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.991 INFO:teuthology.orchestra.run.smithi111.stdout: "zabbix_host": { 2024-09-15T23:12:11.991 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "zabbix_host", 2024-09-15T23:12:11.991 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.991 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.991 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.991 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.992 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.992 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.992 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.992 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.992 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.992 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.992 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.992 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.992 INFO:teuthology.orchestra.run.smithi111.stdout: "zabbix_port": { 2024-09-15T23:12:11.992 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "zabbix_port", 2024-09-15T23:12:11.992 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:11.992 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.992 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.993 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10051", 2024-09-15T23:12:11.993 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.993 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.993 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.993 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.993 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.993 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.993 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.993 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.993 INFO:teuthology.orchestra.run.smithi111.stdout: "zabbix_sender": { 2024-09-15T23:12:11.993 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "zabbix_sender", 2024-09-15T23:12:11.993 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.994 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.994 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:11.994 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-15T23:12:11.994 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.994 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.994 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.994 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.994 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.994 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.994 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.994 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.994 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.995 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.995 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-15T23:12:11.995 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:11.995 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.995 INFO:teuthology.orchestra.run.smithi111.stdout: "modules": [ 2024-09-15T23:12:11.995 INFO:teuthology.orchestra.run.smithi111.stdout: "cephadm", 2024-09-15T23:12:11.995 INFO:teuthology.orchestra.run.smithi111.stdout: "dashboard", 2024-09-15T23:12:11.995 INFO:teuthology.orchestra.run.smithi111.stdout: "iostat", 2024-09-15T23:12:11.995 INFO:teuthology.orchestra.run.smithi111.stdout: "nfs", 2024-09-15T23:12:11.995 INFO:teuthology.orchestra.run.smithi111.stdout: "restful" 2024-09-15T23:12:11.996 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.996 INFO:teuthology.orchestra.run.smithi111.stdout: "available_modules": [ 2024-09-15T23:12:11.996 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:11.996 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "alerts", 2024-09-15T23:12:11.996 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:11.996 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:11.996 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:11.996 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-09-15T23:12:11.996 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-09-15T23:12:11.996 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:11.996 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.996 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.997 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-09-15T23:12:11.997 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.997 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.997 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:11.997 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "How frequently to reexamine health status", 2024-09-15T23:12:11.997 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.997 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.997 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.997 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.997 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:11.997 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:11.998 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:11.998 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.998 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:11.998 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:11.998 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:11.998 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:11.998 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:11.998 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:11.998 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:11.998 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:11.998 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:11.998 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:11.999 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:11.999 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:11.999 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:11.999 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:11.999 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:11.999 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:11.999 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:11.999 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:11.999 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:11.999 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:11.999 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:11.999 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.000 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.000 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.000 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.000 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.000 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.000 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.000 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.000 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.000 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.000 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.000 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.001 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.001 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.001 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.001 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.001 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.001 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.001 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.001 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.001 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.001 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.001 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.001 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.002 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.002 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.002 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.002 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.002 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.002 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.002 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.002 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.002 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.002 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.002 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.002 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.003 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.003 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.003 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.003 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.003 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.003 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.003 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.003 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.003 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.003 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_destination": { 2024-09-15T23:12:12.003 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_destination", 2024-09-15T23:12:12.003 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.003 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.004 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.004 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.004 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.004 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.004 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.004 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Email address to send alerts to", 2024-09-15T23:12:12.004 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.004 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.004 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.004 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.004 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_from_name": { 2024-09-15T23:12:12.005 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_from_name", 2024-09-15T23:12:12.005 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.005 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.005 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.005 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "Ceph", 2024-09-15T23:12:12.005 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.005 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.005 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.005 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Email From: name", 2024-09-15T23:12:12.005 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.005 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.005 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.006 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.006 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_host": { 2024-09-15T23:12:12.006 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_host", 2024-09-15T23:12:12.006 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.006 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.006 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.006 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.006 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.006 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.006 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.006 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "SMTP server", 2024-09-15T23:12:12.006 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.006 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.006 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.007 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.007 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_password": { 2024-09-15T23:12:12.007 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_password", 2024-09-15T23:12:12.007 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.007 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.007 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.007 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.007 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.007 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.007 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.007 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Password to authenticate with", 2024-09-15T23:12:12.007 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.008 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.008 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.008 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.008 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_port": { 2024-09-15T23:12:12.008 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_port", 2024-09-15T23:12:12.008 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.008 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.008 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.008 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "465", 2024-09-15T23:12:12.008 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.008 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.008 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.009 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "SMTP port", 2024-09-15T23:12:12.009 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.009 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.009 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.009 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.009 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_sender": { 2024-09-15T23:12:12.009 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_sender", 2024-09-15T23:12:12.009 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.009 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.009 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.009 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.010 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.010 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.010 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.010 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "SMTP envelope sender", 2024-09-15T23:12:12.010 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.010 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.010 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.010 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.010 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_ssl": { 2024-09-15T23:12:12.010 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_ssl", 2024-09-15T23:12:12.010 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.010 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.011 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.011 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.011 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.011 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.011 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.011 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-15T23:12:12.011 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.011 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.011 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.011 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.011 INFO:teuthology.orchestra.run.smithi111.stdout: "smtp_user": { 2024-09-15T23:12:12.011 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "smtp_user", 2024-09-15T23:12:12.011 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.012 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.012 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.012 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.012 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.012 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.012 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.012 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "User to authenticate as", 2024-09-15T23:12:12.012 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.012 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.012 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.012 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.012 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.013 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.013 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.013 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "balancer", 2024-09-15T23:12:12.013 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.013 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.013 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.013 INFO:teuthology.orchestra.run.smithi111.stdout: "active": { 2024-09-15T23:12:12.013 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "active", 2024-09-15T23:12:12.013 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.013 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.013 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.014 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.014 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.014 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.014 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.014 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "automatically balance PGs across cluster", 2024-09-15T23:12:12.014 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.014 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.014 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.014 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.014 INFO:teuthology.orchestra.run.smithi111.stdout: "begin_time": { 2024-09-15T23:12:12.015 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "begin_time", 2024-09-15T23:12:12.015 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.015 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.015 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.015 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0000", 2024-09-15T23:12:12.015 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.015 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.015 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.015 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "beginning time of day to automatically balance", 2024-09-15T23:12:12.015 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-15T23:12:12.015 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.016 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.016 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.016 INFO:teuthology.orchestra.run.smithi111.stdout: "begin_weekday": { 2024-09-15T23:12:12.016 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "begin_weekday", 2024-09-15T23:12:12.016 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-09-15T23:12:12.016 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.016 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.016 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-09-15T23:12:12.016 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "0", 2024-09-15T23:12:12.016 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "6", 2024-09-15T23:12:12.016 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.016 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-15T23:12:12.017 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-15T23:12:12.017 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.017 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.017 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.017 INFO:teuthology.orchestra.run.smithi111.stdout: "crush_compat_max_iterations": { 2024-09-15T23:12:12.017 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crush_compat_max_iterations", 2024-09-15T23:12:12.017 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-09-15T23:12:12.017 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.017 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.017 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "25", 2024-09-15T23:12:12.017 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1", 2024-09-15T23:12:12.018 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "250", 2024-09-15T23:12:12.018 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.018 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-15T23:12:12.018 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.018 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.018 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.018 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.018 INFO:teuthology.orchestra.run.smithi111.stdout: "crush_compat_metrics": { 2024-09-15T23:12:12.018 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crush_compat_metrics", 2024-09-15T23:12:12.018 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.018 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.019 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.019 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "pgs,objects,bytes", 2024-09-15T23:12:12.019 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.019 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.019 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.019 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-15T23:12:12.019 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-15T23:12:12.019 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.019 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.019 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.019 INFO:teuthology.orchestra.run.smithi111.stdout: "crush_compat_step": { 2024-09-15T23:12:12.019 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crush_compat_step", 2024-09-15T23:12:12.020 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-09-15T23:12:12.020 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.020 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.020 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0.5", 2024-09-15T23:12:12.020 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "0.001", 2024-09-15T23:12:12.020 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "0.999", 2024-09-15T23:12:12.020 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.020 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "aggressiveness of optimization", 2024-09-15T23:12:12.020 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-15T23:12:12.020 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.020 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.021 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.021 INFO:teuthology.orchestra.run.smithi111.stdout: "end_time": { 2024-09-15T23:12:12.021 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "end_time", 2024-09-15T23:12:12.021 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.021 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.021 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.021 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "2359", 2024-09-15T23:12:12.021 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.021 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.021 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.021 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "ending time of day to automatically balance", 2024-09-15T23:12:12.022 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-15T23:12:12.022 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.022 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.022 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.022 INFO:teuthology.orchestra.run.smithi111.stdout: "end_weekday": { 2024-09-15T23:12:12.022 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "end_weekday", 2024-09-15T23:12:12.022 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-09-15T23:12:12.022 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.022 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.022 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-09-15T23:12:12.022 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "0", 2024-09-15T23:12:12.023 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "6", 2024-09-15T23:12:12.023 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.023 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-15T23:12:12.023 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-15T23:12:12.023 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.023 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.023 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.023 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.023 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.023 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.023 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.024 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.024 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.024 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.024 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.024 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.024 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.024 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.024 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.024 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.024 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.024 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.024 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.025 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.025 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.025 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.025 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.025 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.025 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.025 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.025 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.025 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.025 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.025 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.025 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.026 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.026 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.026 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.026 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.026 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.026 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.026 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.026 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.026 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.026 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.026 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.027 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.027 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.027 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.027 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.027 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.027 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.027 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.027 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.027 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.027 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.027 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.027 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.028 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.028 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.028 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.028 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.028 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.028 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.028 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.028 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.028 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.028 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.028 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.028 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.029 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.029 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.029 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.029 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.029 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.029 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.029 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.029 INFO:teuthology.orchestra.run.smithi111.stdout: "min_score": { 2024-09-15T23:12:12.029 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "min_score", 2024-09-15T23:12:12.029 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-09-15T23:12:12.029 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.029 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.030 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-09-15T23:12:12.030 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.030 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.030 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.030 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-15T23:12:12.030 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.030 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.030 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.030 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.030 INFO:teuthology.orchestra.run.smithi111.stdout: "mode": { 2024-09-15T23:12:12.030 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mode", 2024-09-15T23:12:12.030 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.031 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.031 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.031 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "upmap", 2024-09-15T23:12:12.031 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.031 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.031 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.031 INFO:teuthology.orchestra.run.smithi111.stdout: "crush-compat", 2024-09-15T23:12:12.031 INFO:teuthology.orchestra.run.smithi111.stdout: "none", 2024-09-15T23:12:12.031 INFO:teuthology.orchestra.run.smithi111.stdout: "read", 2024-09-15T23:12:12.031 INFO:teuthology.orchestra.run.smithi111.stdout: "upmap", 2024-09-15T23:12:12.031 INFO:teuthology.orchestra.run.smithi111.stdout: "upmap-read" 2024-09-15T23:12:12.032 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.032 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Balancer mode", 2024-09-15T23:12:12.032 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.032 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.032 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.032 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.032 INFO:teuthology.orchestra.run.smithi111.stdout: "pool_ids": { 2024-09-15T23:12:12.032 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "pool_ids", 2024-09-15T23:12:12.032 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.032 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.032 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.033 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.033 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.033 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.033 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.033 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-15T23:12:12.033 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.033 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.033 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.033 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.033 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-09-15T23:12:12.033 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-09-15T23:12:12.033 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:12.034 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.034 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.034 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-09-15T23:12:12.034 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.034 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.034 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.034 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-15T23:12:12.034 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.034 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.034 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.034 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.035 INFO:teuthology.orchestra.run.smithi111.stdout: "upmap_max_deviation": { 2024-09-15T23:12:12.035 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "upmap_max_deviation", 2024-09-15T23:12:12.035 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.035 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.035 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.035 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5", 2024-09-15T23:12:12.035 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1", 2024-09-15T23:12:12.035 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.035 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.035 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-15T23:12:12.035 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-15T23:12:12.036 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.036 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.036 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.036 INFO:teuthology.orchestra.run.smithi111.stdout: "upmap_max_optimizations": { 2024-09-15T23:12:12.036 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "upmap_max_optimizations", 2024-09-15T23:12:12.036 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-09-15T23:12:12.036 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.036 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.036 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-09-15T23:12:12.036 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.036 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.037 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.037 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-15T23:12:12.037 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.037 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.037 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.037 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.037 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.037 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.037 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.037 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cephadm", 2024-09-15T23:12:12.038 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.038 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.038 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.038 INFO:teuthology.orchestra.run.smithi111.stdout: "agent_down_multiplier": { 2024-09-15T23:12:12.038 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "agent_down_multiplier", 2024-09-15T23:12:12.038 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-09-15T23:12:12.038 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.038 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.038 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "3.0", 2024-09-15T23:12:12.038 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.038 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.038 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.039 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-15T23:12:12.039 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.039 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.039 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.039 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.039 INFO:teuthology.orchestra.run.smithi111.stdout: "agent_refresh_rate": { 2024-09-15T23:12:12.039 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "agent_refresh_rate", 2024-09-15T23:12:12.039 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:12.039 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.039 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.039 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "20", 2024-09-15T23:12:12.040 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.040 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.040 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.040 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-15T23:12:12.040 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.040 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.040 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.040 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.040 INFO:teuthology.orchestra.run.smithi111.stdout: "agent_starting_port": { 2024-09-15T23:12:12.040 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "agent_starting_port", 2024-09-15T23:12:12.040 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.041 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.041 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.041 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "4721", 2024-09-15T23:12:12.041 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.041 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.041 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.041 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-15T23:12:12.041 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.041 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.041 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.041 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.041 INFO:teuthology.orchestra.run.smithi111.stdout: "allow_ptrace": { 2024-09-15T23:12:12.042 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "allow_ptrace", 2024-09-15T23:12:12.042 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.042 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.042 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.042 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.042 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.042 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.042 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.042 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-15T23:12:12.042 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-15T23:12:12.042 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.042 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.043 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.043 INFO:teuthology.orchestra.run.smithi111.stdout: "autotune_interval": { 2024-09-15T23:12:12.043 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "autotune_interval", 2024-09-15T23:12:12.043 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:12.043 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.043 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.043 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-09-15T23:12:12.043 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.043 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.043 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.043 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-15T23:12:12.044 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.044 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.044 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.044 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.044 INFO:teuthology.orchestra.run.smithi111.stdout: "autotune_memory_target_ratio": { 2024-09-15T23:12:12.044 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "autotune_memory_target_ratio", 2024-09-15T23:12:12.044 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-09-15T23:12:12.044 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.044 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.044 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0.7", 2024-09-15T23:12:12.044 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.044 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.045 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.045 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-15T23:12:12.045 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.045 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.045 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.045 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.045 INFO:teuthology.orchestra.run.smithi111.stdout: "cephadm_log_destination": { 2024-09-15T23:12:12.045 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cephadm_log_destination", 2024-09-15T23:12:12.045 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.045 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.045 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.046 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.046 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.046 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.046 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.046 INFO:teuthology.orchestra.run.smithi111.stdout: "file", 2024-09-15T23:12:12.046 INFO:teuthology.orchestra.run.smithi111.stdout: "file,syslog", 2024-09-15T23:12:12.046 INFO:teuthology.orchestra.run.smithi111.stdout: "syslog" 2024-09-15T23:12:12.046 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.046 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-15T23:12:12.046 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.046 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.046 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.047 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.047 INFO:teuthology.orchestra.run.smithi111.stdout: "cgroups_split": { 2024-09-15T23:12:12.047 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cgroups_split", 2024-09-15T23:12:12.047 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.047 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.047 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.047 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.047 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.047 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.047 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.047 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-15T23:12:12.047 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.048 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.048 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.048 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.048 INFO:teuthology.orchestra.run.smithi111.stdout: "config_checks_enabled": { 2024-09-15T23:12:12.048 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "config_checks_enabled", 2024-09-15T23:12:12.048 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.048 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.048 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.048 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.048 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.048 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.049 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.049 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-15T23:12:12.049 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.049 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.049 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.049 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.049 INFO:teuthology.orchestra.run.smithi111.stdout: "config_dashboard": { 2024-09-15T23:12:12.049 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "config_dashboard", 2024-09-15T23:12:12.049 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.049 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.049 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.049 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.050 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.050 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.050 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.050 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-15T23:12:12.050 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.050 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.050 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.050 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.050 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_alertmanager": { 2024-09-15T23:12:12.050 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_alertmanager", 2024-09-15T23:12:12.050 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.050 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.051 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.051 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-15T23:12:12.051 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.051 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.051 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.051 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Prometheus container image", 2024-09-15T23:12:12.051 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.051 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.051 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.051 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.051 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_base": { 2024-09-15T23:12:12.052 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_base", 2024-09-15T23:12:12.052 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.052 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.052 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.052 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-15T23:12:12.052 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.052 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.052 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.052 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Container image name, without the tag", 2024-09-15T23:12:12.052 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.052 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.053 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.053 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.053 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_elasticsearch": { 2024-09-15T23:12:12.053 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_elasticsearch", 2024-09-15T23:12:12.053 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.053 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.053 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.053 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-15T23:12:12.053 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.053 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.053 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.054 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "elasticsearch container image", 2024-09-15T23:12:12.054 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.054 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.054 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.054 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.054 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_grafana": { 2024-09-15T23:12:12.054 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_grafana", 2024-09-15T23:12:12.054 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.054 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.054 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.054 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-15T23:12:12.055 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.055 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.055 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.055 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Prometheus container image", 2024-09-15T23:12:12.055 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.055 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.055 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.055 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.055 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_haproxy": { 2024-09-15T23:12:12.055 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_haproxy", 2024-09-15T23:12:12.055 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.056 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.056 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.056 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-15T23:12:12.056 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.056 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.056 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.056 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "HAproxy container image", 2024-09-15T23:12:12.056 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.056 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.056 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.056 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.056 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_jaeger_agent": { 2024-09-15T23:12:12.057 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_jaeger_agent", 2024-09-15T23:12:12.057 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.057 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.057 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.057 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-15T23:12:12.057 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.057 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.057 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.057 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Jaeger agent container image", 2024-09-15T23:12:12.057 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.057 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.058 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.058 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.058 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_jaeger_collector": { 2024-09-15T23:12:12.058 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_jaeger_collector", 2024-09-15T23:12:12.058 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.058 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.058 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.058 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-15T23:12:12.058 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.058 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.058 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.058 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Jaeger collector container image", 2024-09-15T23:12:12.059 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.059 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.059 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.059 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.059 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_jaeger_query": { 2024-09-15T23:12:12.059 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_jaeger_query", 2024-09-15T23:12:12.059 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.059 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.059 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.059 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-15T23:12:12.060 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.060 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.060 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.060 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Jaeger query container image", 2024-09-15T23:12:12.060 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.060 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.060 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.060 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.060 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_keepalived": { 2024-09-15T23:12:12.060 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_keepalived", 2024-09-15T23:12:12.060 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.061 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.061 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.061 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-15T23:12:12.061 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.061 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.061 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.061 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Keepalived container image", 2024-09-15T23:12:12.061 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.061 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.061 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.062 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.062 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_loki": { 2024-09-15T23:12:12.062 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_loki", 2024-09-15T23:12:12.062 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.062 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.062 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.062 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-15T23:12:12.062 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.062 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.062 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.062 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Loki container image", 2024-09-15T23:12:12.063 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.063 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.063 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.063 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.063 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_node_exporter": { 2024-09-15T23:12:12.063 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_node_exporter", 2024-09-15T23:12:12.063 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.063 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.063 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.063 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-15T23:12:12.063 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.063 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.064 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.064 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Prometheus container image", 2024-09-15T23:12:12.064 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.064 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.064 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.064 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.064 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_nvmeof": { 2024-09-15T23:12:12.064 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_nvmeof", 2024-09-15T23:12:12.064 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.064 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.064 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.065 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-15T23:12:12.065 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.065 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.065 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.065 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Nvme-of container image", 2024-09-15T23:12:12.065 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.065 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.065 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.065 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.065 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_prometheus": { 2024-09-15T23:12:12.065 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_prometheus", 2024-09-15T23:12:12.065 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.065 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.066 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.066 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-15T23:12:12.066 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.066 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.066 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.066 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Prometheus container image", 2024-09-15T23:12:12.066 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.066 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.066 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.066 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.066 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_promtail": { 2024-09-15T23:12:12.067 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_promtail", 2024-09-15T23:12:12.067 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.067 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.067 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.067 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-15T23:12:12.067 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.067 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.067 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.067 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Promtail container image", 2024-09-15T23:12:12.067 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.067 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.068 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.068 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.068 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_samba": { 2024-09-15T23:12:12.068 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_samba", 2024-09-15T23:12:12.068 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.068 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.068 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.068 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-15T23:12:12.068 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.068 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.068 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.068 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Samba/SMB container image", 2024-09-15T23:12:12.069 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.069 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.069 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.069 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.069 INFO:teuthology.orchestra.run.smithi111.stdout: "container_image_snmp_gateway": { 2024-09-15T23:12:12.069 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_image_snmp_gateway", 2024-09-15T23:12:12.069 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.069 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.069 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.069 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-15T23:12:12.069 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.070 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.070 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.070 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "SNMP Gateway container image", 2024-09-15T23:12:12.070 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.070 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.070 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.070 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.070 INFO:teuthology.orchestra.run.smithi111.stdout: "container_init": { 2024-09-15T23:12:12.070 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "container_init", 2024-09-15T23:12:12.070 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.070 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.070 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.071 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.071 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.071 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.071 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.071 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Run podman/docker with `--init`", 2024-09-15T23:12:12.071 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.071 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.071 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.071 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.071 INFO:teuthology.orchestra.run.smithi111.stdout: "daemon_cache_timeout": { 2024-09-15T23:12:12.071 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "daemon_cache_timeout", 2024-09-15T23:12:12.071 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:12.072 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.072 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.072 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-09-15T23:12:12.072 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.072 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.072 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.072 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-15T23:12:12.072 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.072 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.072 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.072 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.072 INFO:teuthology.orchestra.run.smithi111.stdout: "default_cephadm_command_timeout": { 2024-09-15T23:12:12.073 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "default_cephadm_command_timeout", 2024-09-15T23:12:12.073 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.073 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.073 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.073 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "900", 2024-09-15T23:12:12.073 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.073 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.073 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.073 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-15T23:12:12.074 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.074 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.074 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.074 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.074 INFO:teuthology.orchestra.run.smithi111.stdout: "default_registry": { 2024-09-15T23:12:12.074 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "default_registry", 2024-09-15T23:12:12.074 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.074 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.074 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.074 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "docker.io", 2024-09-15T23:12:12.074 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.074 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.075 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.075 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-15T23:12:12.075 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.075 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.075 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.075 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.075 INFO:teuthology.orchestra.run.smithi111.stdout: "device_cache_timeout": { 2024-09-15T23:12:12.075 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "device_cache_timeout", 2024-09-15T23:12:12.075 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:12.075 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.075 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.076 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "1800", 2024-09-15T23:12:12.076 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.076 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.076 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.076 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "seconds to cache device inventory", 2024-09-15T23:12:12.076 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.076 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.076 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.076 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.076 INFO:teuthology.orchestra.run.smithi111.stdout: "device_enhanced_scan": { 2024-09-15T23:12:12.076 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "device_enhanced_scan", 2024-09-15T23:12:12.076 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.076 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.077 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.077 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.077 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.077 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.077 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.077 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-15T23:12:12.077 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.077 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.077 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.077 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.077 INFO:teuthology.orchestra.run.smithi111.stdout: "facts_cache_timeout": { 2024-09-15T23:12:12.077 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "facts_cache_timeout", 2024-09-15T23:12:12.077 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:12.078 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.078 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.078 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-09-15T23:12:12.078 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.078 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.078 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.078 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "seconds to cache host facts data", 2024-09-15T23:12:12.078 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.078 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.078 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.078 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.078 INFO:teuthology.orchestra.run.smithi111.stdout: "grafana_dashboards_path": { 2024-09-15T23:12:12.079 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "grafana_dashboards_path", 2024-09-15T23:12:12.079 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.079 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.079 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.079 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-15T23:12:12.079 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.079 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.079 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.079 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-15T23:12:12.079 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.079 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.080 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.080 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.080 INFO:teuthology.orchestra.run.smithi111.stdout: "host_check_interval": { 2024-09-15T23:12:12.080 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "host_check_interval", 2024-09-15T23:12:12.080 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:12.080 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.080 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.080 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-09-15T23:12:12.080 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.080 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.080 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.080 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to perform a host check", 2024-09-15T23:12:12.080 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.081 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.081 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.081 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.081 INFO:teuthology.orchestra.run.smithi111.stdout: "hw_monitoring": { 2024-09-15T23:12:12.081 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "hw_monitoring", 2024-09-15T23:12:12.081 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.081 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.081 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.081 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.081 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.081 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.082 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.082 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-15T23:12:12.082 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.082 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.082 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.082 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.082 INFO:teuthology.orchestra.run.smithi111.stdout: "inventory_list_all": { 2024-09-15T23:12:12.082 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "inventory_list_all", 2024-09-15T23:12:12.082 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.082 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.082 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.082 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.082 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.083 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.083 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.083 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-15T23:12:12.083 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.083 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.083 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.083 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.083 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.083 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.083 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.083 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.083 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.084 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.084 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.084 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.084 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.084 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.084 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.084 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.084 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.084 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.084 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.084 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.084 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.085 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.085 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.085 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.085 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.085 INFO:teuthology.orchestra.run.smithi111.stdout: "log_refresh_metadata": { 2024-09-15T23:12:12.085 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_refresh_metadata", 2024-09-15T23:12:12.085 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.085 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.085 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.085 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.086 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.086 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.086 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.086 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-15T23:12:12.086 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.086 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.086 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.086 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.086 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.086 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.087 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.087 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.087 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.087 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.087 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.087 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.087 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.087 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-15T23:12:12.087 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.087 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.087 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.087 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.088 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.088 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.088 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.088 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.088 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.088 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.088 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.088 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.088 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.088 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.088 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.089 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.089 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.089 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.089 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.089 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.089 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.089 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.089 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.089 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.089 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.089 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.089 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.090 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.090 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.090 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.090 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.090 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.090 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.090 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.090 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.090 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.090 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.090 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.091 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.091 INFO:teuthology.orchestra.run.smithi111.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-15T23:12:12.091 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-15T23:12:12.091 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.091 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.091 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.091 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.091 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.091 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.091 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.091 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-15T23:12:12.092 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.092 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.092 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.092 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.092 INFO:teuthology.orchestra.run.smithi111.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-15T23:12:12.092 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-15T23:12:12.092 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.092 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.092 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.092 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "*", 2024-09-15T23:12:12.092 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.092 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.093 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.093 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-15T23:12:12.093 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.093 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.093 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.093 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.093 INFO:teuthology.orchestra.run.smithi111.stdout: "max_count_per_host": { 2024-09-15T23:12:12.093 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_count_per_host", 2024-09-15T23:12:12.093 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.093 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.093 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.094 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-09-15T23:12:12.094 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.094 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.094 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.094 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "max number of daemons per service per host", 2024-09-15T23:12:12.094 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.094 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.094 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.094 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.094 INFO:teuthology.orchestra.run.smithi111.stdout: "max_osd_draining_count": { 2024-09-15T23:12:12.094 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_osd_draining_count", 2024-09-15T23:12:12.095 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.095 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.095 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.095 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-09-15T23:12:12.095 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.095 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.095 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.095 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-15T23:12:12.095 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.095 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.095 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.096 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.096 INFO:teuthology.orchestra.run.smithi111.stdout: "migration_current": { 2024-09-15T23:12:12.096 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "migration_current", 2024-09-15T23:12:12.096 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.096 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.096 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.096 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.096 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.096 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.096 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.096 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "internal - do not modify", 2024-09-15T23:12:12.097 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.097 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.097 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.097 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.097 INFO:teuthology.orchestra.run.smithi111.stdout: "mode": { 2024-09-15T23:12:12.097 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mode", 2024-09-15T23:12:12.097 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.097 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.097 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.097 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "root", 2024-09-15T23:12:12.097 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.097 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.098 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.098 INFO:teuthology.orchestra.run.smithi111.stdout: "cephadm-package", 2024-09-15T23:12:12.098 INFO:teuthology.orchestra.run.smithi111.stdout: "root" 2024-09-15T23:12:12.098 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.098 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "mode for remote execution of cephadm", 2024-09-15T23:12:12.098 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.098 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.098 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.098 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.098 INFO:teuthology.orchestra.run.smithi111.stdout: "oob_default_addr": { 2024-09-15T23:12:12.098 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "oob_default_addr", 2024-09-15T23:12:12.099 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.099 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.099 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.099 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "169.254.1.1", 2024-09-15T23:12:12.099 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.099 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.099 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.099 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-15T23:12:12.099 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.099 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.099 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.099 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.100 INFO:teuthology.orchestra.run.smithi111.stdout: "prometheus_alerts_path": { 2024-09-15T23:12:12.100 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "prometheus_alerts_path", 2024-09-15T23:12:12.100 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.100 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.100 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.100 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-15T23:12:12.100 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.100 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.100 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.100 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-15T23:12:12.100 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.101 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.101 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.101 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.101 INFO:teuthology.orchestra.run.smithi111.stdout: "registry_insecure": { 2024-09-15T23:12:12.101 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "registry_insecure", 2024-09-15T23:12:12.101 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.101 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.101 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.101 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.101 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.101 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.102 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.102 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-15T23:12:12.102 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.102 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.102 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.102 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.102 INFO:teuthology.orchestra.run.smithi111.stdout: "registry_password": { 2024-09-15T23:12:12.102 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "registry_password", 2024-09-15T23:12:12.102 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.102 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.102 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.102 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.102 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.103 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.103 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.103 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-15T23:12:12.103 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.103 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.103 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.103 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.103 INFO:teuthology.orchestra.run.smithi111.stdout: "registry_url": { 2024-09-15T23:12:12.103 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "registry_url", 2024-09-15T23:12:12.103 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.103 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.104 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.104 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.104 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.104 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.104 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.104 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-15T23:12:12.104 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.104 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.104 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.104 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.104 INFO:teuthology.orchestra.run.smithi111.stdout: "registry_username": { 2024-09-15T23:12:12.105 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "registry_username", 2024-09-15T23:12:12.105 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.105 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.105 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.105 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.105 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.105 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.105 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.105 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-15T23:12:12.105 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.105 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.105 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.106 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.106 INFO:teuthology.orchestra.run.smithi111.stdout: "secure_monitoring_stack": { 2024-09-15T23:12:12.106 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "secure_monitoring_stack", 2024-09-15T23:12:12.106 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.106 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.106 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.106 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.106 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.106 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.106 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.106 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-15T23:12:12.106 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.106 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.107 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.107 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.107 INFO:teuthology.orchestra.run.smithi111.stdout: "service_discovery_port": { 2024-09-15T23:12:12.107 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "service_discovery_port", 2024-09-15T23:12:12.107 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.107 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.107 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.107 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8765", 2024-09-15T23:12:12.107 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.107 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.107 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.107 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "cephadm service discovery port", 2024-09-15T23:12:12.107 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.108 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.108 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.108 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.108 INFO:teuthology.orchestra.run.smithi111.stdout: "ssh_config_file": { 2024-09-15T23:12:12.108 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ssh_config_file", 2024-09-15T23:12:12.108 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.108 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.108 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.108 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.108 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.108 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.108 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.109 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-15T23:12:12.109 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.109 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.109 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.109 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.109 INFO:teuthology.orchestra.run.smithi111.stdout: "use_agent": { 2024-09-15T23:12:12.109 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "use_agent", 2024-09-15T23:12:12.109 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.109 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.109 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.109 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.109 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.109 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.110 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.110 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-15T23:12:12.110 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.110 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.110 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.110 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.110 INFO:teuthology.orchestra.run.smithi111.stdout: "use_repo_digest": { 2024-09-15T23:12:12.110 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "use_repo_digest", 2024-09-15T23:12:12.110 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.110 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.110 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.110 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.111 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.111 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.111 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.111 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-15T23:12:12.111 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.111 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.111 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.111 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.111 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_on_failed_host_check": { 2024-09-15T23:12:12.111 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_on_failed_host_check", 2024-09-15T23:12:12.111 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.111 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.111 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.112 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.112 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.112 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.112 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.112 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "raise a health warning if the host check fails", 2024-09-15T23:12:12.112 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.112 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.112 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.112 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.112 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_on_stray_daemons": { 2024-09-15T23:12:12.112 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_on_stray_daemons", 2024-09-15T23:12:12.112 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.113 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.113 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.113 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.113 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.113 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.113 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.113 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-15T23:12:12.113 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.113 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.113 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.113 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.114 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_on_stray_hosts": { 2024-09-15T23:12:12.114 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_on_stray_hosts", 2024-09-15T23:12:12.114 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.114 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.114 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.114 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.114 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.114 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.114 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.114 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-15T23:12:12.114 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.115 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.115 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.115 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.115 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.115 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.115 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.115 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crash", 2024-09-15T23:12:12.115 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.115 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.115 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.115 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.116 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.116 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.116 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.116 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.116 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.116 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.116 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.116 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.116 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.116 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.116 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.116 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.117 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.117 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.117 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.117 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.117 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.117 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.117 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.117 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.117 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.117 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.117 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.118 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.118 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.118 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.118 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.118 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.118 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.118 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.118 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.118 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.118 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.118 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.118 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.119 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.119 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.119 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.119 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.119 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.119 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.119 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.119 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.119 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.119 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.119 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.120 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.120 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.120 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.120 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.120 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.120 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.120 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.120 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.120 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.120 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.120 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.120 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.120 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.121 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.121 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.121 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.121 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.121 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.121 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.121 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.121 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.121 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.121 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.121 INFO:teuthology.orchestra.run.smithi111.stdout: "retain_interval": { 2024-09-15T23:12:12.121 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "retain_interval", 2024-09-15T23:12:12.122 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:12.122 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.122 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.122 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "31536000", 2024-09-15T23:12:12.122 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.122 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.122 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.122 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-15T23:12:12.122 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.122 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.122 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.122 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.123 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_recent_interval": { 2024-09-15T23:12:12.123 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_recent_interval", 2024-09-15T23:12:12.123 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:12.123 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.123 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.123 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "1209600", 2024-09-15T23:12:12.123 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.123 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.123 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.123 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-15T23:12:12.123 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.123 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.124 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.124 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.124 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.124 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.124 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.124 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "dashboard", 2024-09-15T23:12:12.124 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.124 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.124 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.124 INFO:teuthology.orchestra.run.smithi111.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-15T23:12:12.124 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-15T23:12:12.125 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.125 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.125 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.125 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-09-15T23:12:12.125 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.125 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.125 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.125 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.125 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.125 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.125 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.125 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.126 INFO:teuthology.orchestra.run.smithi111.stdout: "ALERTMANAGER_API_HOST": { 2024-09-15T23:12:12.126 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-15T23:12:12.126 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.126 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.126 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.126 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.126 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.126 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.126 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.126 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.126 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.126 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.127 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.127 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.127 INFO:teuthology.orchestra.run.smithi111.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-15T23:12:12.127 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-15T23:12:12.127 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.127 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.127 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.127 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.127 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.127 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.127 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.128 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.128 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.128 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.128 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.128 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.128 INFO:teuthology.orchestra.run.smithi111.stdout: "AUDIT_API_ENABLED": { 2024-09-15T23:12:12.128 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "AUDIT_API_ENABLED", 2024-09-15T23:12:12.128 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.128 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.128 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.129 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.129 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.129 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.129 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.129 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.129 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.129 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.129 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.129 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.129 INFO:teuthology.orchestra.run.smithi111.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-15T23:12:12.130 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-15T23:12:12.130 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.130 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.130 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.130 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.130 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.130 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.130 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.130 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.130 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.130 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.131 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.131 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.131 INFO:teuthology.orchestra.run.smithi111.stdout: "ENABLE_BROWSABLE_API": { 2024-09-15T23:12:12.131 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-15T23:12:12.131 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.131 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.131 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.131 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.131 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.131 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.132 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.132 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.132 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.132 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.132 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.132 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.132 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-15T23:12:12.132 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-15T23:12:12.132 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.132 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.132 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.132 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.133 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.133 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.133 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.133 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.133 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.133 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.133 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.133 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.133 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-15T23:12:12.133 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-15T23:12:12.133 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.134 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.134 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.134 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.134 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.134 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.134 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.134 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.134 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.134 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.134 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.134 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.134 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-15T23:12:12.135 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-15T23:12:12.135 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.135 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.135 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.135 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.135 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.135 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.135 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.135 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.135 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.135 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.135 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.136 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.136 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-15T23:12:12.136 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-15T23:12:12.136 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.136 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.136 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.136 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.136 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.136 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.136 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.136 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.137 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.137 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.137 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.137 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.137 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-15T23:12:12.137 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-15T23:12:12.137 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.137 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.137 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.137 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.137 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.138 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.138 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.138 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.138 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.138 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.138 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.138 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.138 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-15T23:12:12.138 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-15T23:12:12.138 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.138 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.138 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.139 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.139 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.139 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.139 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.139 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.139 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.139 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.139 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.139 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.139 INFO:teuthology.orchestra.run.smithi111.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-15T23:12:12.139 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-15T23:12:12.139 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.139 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.140 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.140 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.140 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.140 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.140 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.140 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.140 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.140 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.140 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.140 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.140 INFO:teuthology.orchestra.run.smithi111.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-15T23:12:12.140 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-15T23:12:12.141 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.141 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.141 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.141 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.141 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.141 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.141 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.141 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.141 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.141 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.141 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.142 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.142 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_API_PASSWORD": { 2024-09-15T23:12:12.142 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-15T23:12:12.142 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.142 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.142 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.142 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "admin", 2024-09-15T23:12:12.142 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.142 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.142 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.142 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.142 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.143 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.143 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.143 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.143 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-15T23:12:12.143 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-15T23:12:12.143 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.143 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.143 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.143 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.143 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.143 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.143 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.144 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.144 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.144 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.144 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.144 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.144 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_API_URL": { 2024-09-15T23:12:12.144 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_API_URL", 2024-09-15T23:12:12.144 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.144 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.144 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.144 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.144 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.145 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.145 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.145 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.145 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.145 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.145 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.145 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.145 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_API_USERNAME": { 2024-09-15T23:12:12.145 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-15T23:12:12.145 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.145 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.145 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.145 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "admin", 2024-09-15T23:12:12.146 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.146 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.146 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.146 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.146 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.146 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.146 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.146 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.146 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-15T23:12:12.146 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-15T23:12:12.146 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.146 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.147 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.147 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.147 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.147 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.147 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.147 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.147 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.147 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.147 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.147 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.147 INFO:teuthology.orchestra.run.smithi111.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-15T23:12:12.148 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-15T23:12:12.148 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.148 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.148 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.148 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.148 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.148 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.148 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.148 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.148 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.148 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.148 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.149 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.149 INFO:teuthology.orchestra.run.smithi111.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-15T23:12:12.149 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-15T23:12:12.149 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.149 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.149 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.149 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.149 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.149 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.149 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.150 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.150 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.150 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.150 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.150 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.150 INFO:teuthology.orchestra.run.smithi111.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-15T23:12:12.150 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-15T23:12:12.150 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.150 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.150 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.150 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.150 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.150 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.151 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.151 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.151 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.151 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.151 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.151 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.151 INFO:teuthology.orchestra.run.smithi111.stdout: "PROMETHEUS_API_HOST": { 2024-09-15T23:12:12.151 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-15T23:12:12.151 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.151 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.151 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.152 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.152 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.152 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.152 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.152 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.152 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.152 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.152 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.152 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.152 INFO:teuthology.orchestra.run.smithi111.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-15T23:12:12.152 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-15T23:12:12.153 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.153 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.153 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.153 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.153 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.153 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.153 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.153 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.153 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.153 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.153 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.154 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.154 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-15T23:12:12.154 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-15T23:12:12.154 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.154 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.154 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.154 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.154 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.154 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.154 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.154 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.155 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.155 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.155 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.155 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.155 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-15T23:12:12.155 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-15T23:12:12.155 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.155 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.155 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.155 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.156 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.156 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.156 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.156 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.156 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.156 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.156 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.156 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.156 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-15T23:12:12.156 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-15T23:12:12.156 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.157 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.157 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.157 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.157 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.157 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.157 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.157 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.157 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.157 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.157 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.157 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.157 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-15T23:12:12.158 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-15T23:12:12.158 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.158 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.158 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.158 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.158 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.158 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.158 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.158 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.158 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.158 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.158 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.159 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.159 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-15T23:12:12.159 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-15T23:12:12.159 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.159 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.159 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.159 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.159 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.159 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.159 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.159 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.159 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.159 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.160 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.160 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.160 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-15T23:12:12.160 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-15T23:12:12.160 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.160 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.160 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.160 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.160 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.160 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.160 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.161 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.161 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.161 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.161 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.161 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.161 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-15T23:12:12.161 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-15T23:12:12.161 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.161 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.161 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.161 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.161 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.161 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.162 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.162 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.162 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.162 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.162 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.162 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.162 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_ENABLED": { 2024-09-15T23:12:12.162 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-15T23:12:12.162 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.162 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.162 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.163 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.163 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.163 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.163 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.163 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.163 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.163 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.163 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.163 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.163 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-15T23:12:12.163 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-15T23:12:12.163 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.164 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.164 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.164 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-15T23:12:12.164 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.164 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.164 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.164 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.164 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.164 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.164 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.164 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.165 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-15T23:12:12.165 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-15T23:12:12.165 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.165 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.165 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.165 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-09-15T23:12:12.165 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.165 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.165 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.165 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.165 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.166 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.166 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.166 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.166 INFO:teuthology.orchestra.run.smithi111.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-15T23:12:12.166 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-15T23:12:12.166 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.166 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.166 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.166 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8", 2024-09-15T23:12:12.166 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.166 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.166 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.167 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.167 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.167 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.167 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.167 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.167 INFO:teuthology.orchestra.run.smithi111.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-15T23:12:12.167 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-15T23:12:12.167 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.167 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.167 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.167 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "45", 2024-09-15T23:12:12.167 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.168 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.168 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.168 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.168 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.168 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.168 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.168 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.168 INFO:teuthology.orchestra.run.smithi111.stdout: "RGW_API_ACCESS_KEY": { 2024-09-15T23:12:12.168 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-15T23:12:12.168 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.168 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.168 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.169 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.169 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.169 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.169 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.169 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.169 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.169 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.169 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.169 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.169 INFO:teuthology.orchestra.run.smithi111.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-15T23:12:12.169 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-15T23:12:12.170 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.170 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.170 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.170 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "admin", 2024-09-15T23:12:12.170 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.170 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.170 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.170 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.170 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.170 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.170 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.170 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.171 INFO:teuthology.orchestra.run.smithi111.stdout: "RGW_API_SECRET_KEY": { 2024-09-15T23:12:12.171 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-15T23:12:12.171 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.171 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.171 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.171 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.171 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.171 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.171 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.171 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.171 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.172 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.172 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.172 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.172 INFO:teuthology.orchestra.run.smithi111.stdout: "RGW_API_SSL_VERIFY": { 2024-09-15T23:12:12.172 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-15T23:12:12.172 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.172 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.172 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.172 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.172 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.172 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.173 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.173 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.173 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.173 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.173 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.173 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.173 INFO:teuthology.orchestra.run.smithi111.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-15T23:12:12.173 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-15T23:12:12.173 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.173 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.173 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.174 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-09-15T23:12:12.174 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.174 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.174 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.174 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.174 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.174 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.174 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.174 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.174 INFO:teuthology.orchestra.run.smithi111.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-15T23:12:12.174 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-15T23:12:12.174 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.175 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.175 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.175 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-09-15T23:12:12.175 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.175 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.175 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.175 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.175 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.175 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.175 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.175 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.176 INFO:teuthology.orchestra.run.smithi111.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-15T23:12:12.176 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-15T23:12:12.176 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.176 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.176 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.176 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5", 2024-09-15T23:12:12.176 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.176 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.176 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.176 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.176 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.176 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.177 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.177 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.177 INFO:teuthology.orchestra.run.smithi111.stdout: "cross_origin_url": { 2024-09-15T23:12:12.177 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cross_origin_url", 2024-09-15T23:12:12.177 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.177 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.177 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.177 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.177 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.177 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.177 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.178 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.178 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.178 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.178 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.178 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.178 INFO:teuthology.orchestra.run.smithi111.stdout: "crt_file": { 2024-09-15T23:12:12.178 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "crt_file", 2024-09-15T23:12:12.178 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.178 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.178 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.178 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.178 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.179 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.179 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.179 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.179 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.179 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.179 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.179 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.179 INFO:teuthology.orchestra.run.smithi111.stdout: "debug": { 2024-09-15T23:12:12.179 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "debug", 2024-09-15T23:12:12.179 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.179 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.180 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.180 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.180 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.180 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.180 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.180 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Enable/disable debug options", 2024-09-15T23:12:12.180 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.180 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.180 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.180 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.180 INFO:teuthology.orchestra.run.smithi111.stdout: "jwt_token_ttl": { 2024-09-15T23:12:12.181 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "jwt_token_ttl", 2024-09-15T23:12:12.181 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.181 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.181 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.181 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "28800", 2024-09-15T23:12:12.181 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.181 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.181 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.181 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.181 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.181 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.182 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.182 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.182 INFO:teuthology.orchestra.run.smithi111.stdout: "key_file": { 2024-09-15T23:12:12.182 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "key_file", 2024-09-15T23:12:12.182 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.182 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.182 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.182 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.182 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.182 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.182 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.182 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.183 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.183 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.183 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.183 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.183 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.183 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.183 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.183 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.183 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.183 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.183 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.184 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.184 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.184 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.184 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.184 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.184 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.184 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.184 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.184 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.184 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.184 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.185 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.185 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.185 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.185 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.185 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.185 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.185 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.185 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.185 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.185 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.185 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.186 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.186 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.186 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.186 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.186 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.186 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.186 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.186 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.186 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.186 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.186 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.186 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.187 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.187 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.187 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.187 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.187 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.187 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.187 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.187 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.187 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.187 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.187 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.188 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.188 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.188 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.188 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.188 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.188 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.188 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.188 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.188 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.188 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.188 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.189 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.189 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.189 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.189 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.189 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.189 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.189 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.189 INFO:teuthology.orchestra.run.smithi111.stdout: "motd": { 2024-09-15T23:12:12.189 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "motd", 2024-09-15T23:12:12.189 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.189 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.190 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.190 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.190 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.190 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.190 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.190 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "The message of the day", 2024-09-15T23:12:12.190 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.190 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.190 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.190 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.190 INFO:teuthology.orchestra.run.smithi111.stdout: "redirect_resolve_ip_addr": { 2024-09-15T23:12:12.190 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "redirect_resolve_ip_addr", 2024-09-15T23:12:12.191 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.191 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.191 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.191 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.191 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.191 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.191 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.191 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.191 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.191 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.191 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.192 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.192 INFO:teuthology.orchestra.run.smithi111.stdout: "server_addr": { 2024-09-15T23:12:12.192 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_addr", 2024-09-15T23:12:12.192 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.192 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.192 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.192 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "::", 2024-09-15T23:12:12.192 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.192 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.192 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.192 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.192 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.193 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.193 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.193 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.193 INFO:teuthology.orchestra.run.smithi111.stdout: "server_port": { 2024-09-15T23:12:12.193 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_port", 2024-09-15T23:12:12.193 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.193 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.193 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.193 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8080", 2024-09-15T23:12:12.193 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.193 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.194 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.194 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.194 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.194 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.194 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.194 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.194 INFO:teuthology.orchestra.run.smithi111.stdout: "ssl": { 2024-09-15T23:12:12.194 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ssl", 2024-09-15T23:12:12.194 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.194 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.194 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.194 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.195 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.195 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.195 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.195 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.195 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.195 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.195 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.195 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.195 INFO:teuthology.orchestra.run.smithi111.stdout: "ssl_server_port": { 2024-09-15T23:12:12.195 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ssl_server_port", 2024-09-15T23:12:12.195 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.195 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.196 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.196 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8443", 2024-09-15T23:12:12.196 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.196 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.196 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.196 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.196 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.196 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.196 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.196 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.196 INFO:teuthology.orchestra.run.smithi111.stdout: "standby_behaviour": { 2024-09-15T23:12:12.196 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "standby_behaviour", 2024-09-15T23:12:12.197 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.197 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.197 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.197 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "redirect", 2024-09-15T23:12:12.197 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.197 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.197 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.197 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.197 INFO:teuthology.orchestra.run.smithi111.stdout: "redirect" 2024-09-15T23:12:12.197 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.197 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.197 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.198 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.198 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.198 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.198 INFO:teuthology.orchestra.run.smithi111.stdout: "standby_error_status_code": { 2024-09-15T23:12:12.198 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "standby_error_status_code", 2024-09-15T23:12:12.198 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.198 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.198 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.198 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "500", 2024-09-15T23:12:12.198 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "400", 2024-09-15T23:12:12.198 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "599", 2024-09-15T23:12:12.199 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.199 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.199 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.199 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.199 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.199 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.199 INFO:teuthology.orchestra.run.smithi111.stdout: "url_prefix": { 2024-09-15T23:12:12.199 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "url_prefix", 2024-09-15T23:12:12.199 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.199 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.199 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.199 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.199 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.200 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.200 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.200 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.200 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.200 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.200 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.200 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.200 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.200 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.200 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.200 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "devicehealth", 2024-09-15T23:12:12.201 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.201 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.201 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.201 INFO:teuthology.orchestra.run.smithi111.stdout: "enable_monitoring": { 2024-09-15T23:12:12.201 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "enable_monitoring", 2024-09-15T23:12:12.201 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.201 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.201 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.201 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.201 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.201 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.202 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.202 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "monitor device health metrics", 2024-09-15T23:12:12.202 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.202 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.202 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.202 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.202 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.202 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.202 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.202 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.202 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.202 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.203 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.203 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.203 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.203 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.203 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.203 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.203 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.203 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.203 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.203 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.203 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.204 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.204 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.204 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.204 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.204 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.204 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.204 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.204 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.204 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.204 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.204 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.204 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.205 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.205 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.205 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.205 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.205 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.205 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.205 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.205 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.205 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.205 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.205 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.206 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.206 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.206 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.206 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.206 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.206 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.206 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.206 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.206 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.206 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.206 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.206 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.207 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.207 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.207 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.207 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.207 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.207 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.207 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.207 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.207 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.207 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.207 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.208 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.208 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.208 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.208 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.208 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.208 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.208 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.208 INFO:teuthology.orchestra.run.smithi111.stdout: "mark_out_threshold": { 2024-09-15T23:12:12.208 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mark_out_threshold", 2024-09-15T23:12:12.208 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:12.208 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.208 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.209 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "2419200", 2024-09-15T23:12:12.209 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.209 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.209 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.209 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-15T23:12:12.209 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.209 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.209 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.209 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.209 INFO:teuthology.orchestra.run.smithi111.stdout: "pool_name": { 2024-09-15T23:12:12.209 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "pool_name", 2024-09-15T23:12:12.210 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.210 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.210 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.210 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "device_health_metrics", 2024-09-15T23:12:12.210 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.210 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.210 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.210 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-15T23:12:12.210 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.210 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.210 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.210 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.211 INFO:teuthology.orchestra.run.smithi111.stdout: "retention_period": { 2024-09-15T23:12:12.211 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "retention_period", 2024-09-15T23:12:12.211 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:12.211 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.211 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.211 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "15552000", 2024-09-15T23:12:12.211 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.211 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.211 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.211 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how long to retain device health metrics", 2024-09-15T23:12:12.211 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.212 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.212 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.212 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.212 INFO:teuthology.orchestra.run.smithi111.stdout: "scrape_frequency": { 2024-09-15T23:12:12.212 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "scrape_frequency", 2024-09-15T23:12:12.212 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:12.212 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.212 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.212 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "86400", 2024-09-15T23:12:12.212 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.212 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.212 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.213 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-15T23:12:12.213 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.213 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.213 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.213 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.213 INFO:teuthology.orchestra.run.smithi111.stdout: "self_heal": { 2024-09-15T23:12:12.213 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "self_heal", 2024-09-15T23:12:12.213 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.213 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.213 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.213 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.214 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.214 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.214 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.214 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-15T23:12:12.214 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.214 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.214 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.214 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.214 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-09-15T23:12:12.214 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-09-15T23:12:12.214 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:12.214 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.214 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.215 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-09-15T23:12:12.215 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.215 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.215 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.215 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how frequently to wake up and check device health", 2024-09-15T23:12:12.215 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.215 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.215 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.215 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.215 INFO:teuthology.orchestra.run.smithi111.stdout: "warn_threshold": { 2024-09-15T23:12:12.215 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "warn_threshold", 2024-09-15T23:12:12.215 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:12.216 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.216 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.216 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "7257600", 2024-09-15T23:12:12.216 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.216 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.216 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.216 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-15T23:12:12.216 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.216 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.216 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.216 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.216 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.217 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.217 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.217 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "diskprediction_local", 2024-09-15T23:12:12.217 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.217 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.217 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.217 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.217 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.217 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.217 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.217 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.218 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.218 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.218 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.218 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.218 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.218 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.218 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.218 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.218 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.218 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.218 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.218 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.219 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.219 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.219 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.219 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.219 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.219 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.219 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.219 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.219 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.219 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.219 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.219 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.220 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.220 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.220 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.220 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.220 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.220 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.220 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.220 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.220 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.220 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.220 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.220 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.221 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.221 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.221 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.221 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.221 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.221 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.221 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.221 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.221 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.221 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.221 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.221 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.222 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.222 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.222 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.222 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.222 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.222 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.222 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.222 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.222 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.222 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.222 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.223 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.223 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.223 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.223 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.223 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.223 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.223 INFO:teuthology.orchestra.run.smithi111.stdout: "predict_interval": { 2024-09-15T23:12:12.223 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "predict_interval", 2024-09-15T23:12:12.223 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.223 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.223 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.224 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "86400", 2024-09-15T23:12:12.224 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.224 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.224 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.224 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.224 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.224 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.224 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.224 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.224 INFO:teuthology.orchestra.run.smithi111.stdout: "predictor_model": { 2024-09-15T23:12:12.224 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "predictor_model", 2024-09-15T23:12:12.224 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.225 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.225 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.225 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "prophetstor", 2024-09-15T23:12:12.225 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.225 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.225 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.225 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.225 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.225 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.225 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.225 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.226 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-09-15T23:12:12.226 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-09-15T23:12:12.226 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.226 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.226 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.226 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "600", 2024-09-15T23:12:12.226 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.226 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.226 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.226 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.226 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.226 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.227 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.227 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.227 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.227 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.227 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.227 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "influx", 2024-09-15T23:12:12.227 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": false, 2024-09-15T23:12:12.227 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "influxdb python module not found", 2024-09-15T23:12:12.227 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.227 INFO:teuthology.orchestra.run.smithi111.stdout: "batch_size": { 2024-09-15T23:12:12.227 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "batch_size", 2024-09-15T23:12:12.227 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.228 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.228 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.228 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5000", 2024-09-15T23:12:12.228 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.228 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.228 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.228 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-15T23:12:12.228 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.228 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.228 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.228 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.228 INFO:teuthology.orchestra.run.smithi111.stdout: "database": { 2024-09-15T23:12:12.229 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "database", 2024-09-15T23:12:12.229 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.229 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.229 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.229 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "ceph", 2024-09-15T23:12:12.229 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.229 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.229 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.229 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-15T23:12:12.229 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.229 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.230 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.230 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.230 INFO:teuthology.orchestra.run.smithi111.stdout: "hostname": { 2024-09-15T23:12:12.230 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "hostname", 2024-09-15T23:12:12.230 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.230 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.230 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.230 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.230 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.230 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.230 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.231 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "InfluxDB server hostname", 2024-09-15T23:12:12.231 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.231 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.231 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.231 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.231 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-09-15T23:12:12.231 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-09-15T23:12:12.231 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:12.231 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.231 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.231 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "30", 2024-09-15T23:12:12.232 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "5", 2024-09-15T23:12:12.232 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.232 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.232 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-15T23:12:12.232 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.232 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.232 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.232 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.232 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.232 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.232 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.233 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.233 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.233 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.233 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.233 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.233 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.233 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.233 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.233 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.233 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.233 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.233 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.234 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.234 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.234 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.234 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.234 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.234 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.234 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.234 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.234 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.234 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.234 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.234 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.235 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.235 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.235 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.235 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.235 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.235 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.235 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.235 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.235 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.235 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.235 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.235 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.236 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.236 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.236 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.236 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.236 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.236 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.236 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.236 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.236 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.236 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.236 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.236 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.237 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.237 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.237 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.237 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.237 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.237 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.237 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.237 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.237 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.237 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.237 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.238 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.238 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.238 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.238 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.238 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.238 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.238 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.238 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.238 INFO:teuthology.orchestra.run.smithi111.stdout: "password": { 2024-09-15T23:12:12.238 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "password", 2024-09-15T23:12:12.238 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.238 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.239 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.239 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.239 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.239 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.239 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.239 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "password of InfluxDB server user", 2024-09-15T23:12:12.239 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.239 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.239 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.239 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.239 INFO:teuthology.orchestra.run.smithi111.stdout: "port": { 2024-09-15T23:12:12.239 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "port", 2024-09-15T23:12:12.240 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.240 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.240 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.240 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "8086", 2024-09-15T23:12:12.240 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.240 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.240 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.240 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "InfluxDB server port", 2024-09-15T23:12:12.240 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.240 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.240 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.240 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.241 INFO:teuthology.orchestra.run.smithi111.stdout: "ssl": { 2024-09-15T23:12:12.241 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ssl", 2024-09-15T23:12:12.241 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.241 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.241 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.241 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "false", 2024-09-15T23:12:12.241 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.241 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.241 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.241 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-15T23:12:12.241 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.242 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.242 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.242 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.242 INFO:teuthology.orchestra.run.smithi111.stdout: "threads": { 2024-09-15T23:12:12.242 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "threads", 2024-09-15T23:12:12.242 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.242 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.242 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.242 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5", 2024-09-15T23:12:12.242 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1", 2024-09-15T23:12:12.242 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "32", 2024-09-15T23:12:12.242 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.243 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-15T23:12:12.243 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.243 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.243 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.243 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.243 INFO:teuthology.orchestra.run.smithi111.stdout: "username": { 2024-09-15T23:12:12.243 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "username", 2024-09-15T23:12:12.243 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.243 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.243 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.243 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.243 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.244 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.244 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.244 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "username of InfluxDB server user", 2024-09-15T23:12:12.244 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.244 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.244 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.244 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.244 INFO:teuthology.orchestra.run.smithi111.stdout: "verify_ssl": { 2024-09-15T23:12:12.244 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "verify_ssl", 2024-09-15T23:12:12.244 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.244 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.244 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.245 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "true", 2024-09-15T23:12:12.245 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.245 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.245 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.245 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-15T23:12:12.245 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.245 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.245 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.245 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.245 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.245 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.245 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.246 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "insights", 2024-09-15T23:12:12.246 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.246 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.246 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.246 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.246 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.246 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.246 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.246 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.246 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.246 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.246 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.246 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.247 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.247 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.247 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.247 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.247 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.247 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.247 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.247 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.247 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.247 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.247 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.247 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.248 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.248 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.248 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.248 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.248 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.248 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.248 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.248 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.248 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.248 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.248 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.249 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.249 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.249 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.249 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.249 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.249 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.249 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.249 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.249 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.249 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.249 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.250 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.250 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.250 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.250 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.250 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.250 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.250 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.250 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.250 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.250 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.250 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.250 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.250 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.251 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.251 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.251 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.251 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.251 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.251 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.251 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.251 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.251 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.251 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.251 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.251 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.251 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.252 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.252 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.252 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.252 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.252 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "iostat", 2024-09-15T23:12:12.252 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.252 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.252 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.252 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.252 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.252 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.253 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.253 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.253 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.253 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.253 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.253 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.253 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.253 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.253 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.253 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.253 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.253 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.254 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.254 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.254 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.254 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.254 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.254 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.254 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.254 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.254 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.254 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.254 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.255 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.255 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.255 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.255 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.255 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.255 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.255 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.255 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.255 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.255 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.255 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.256 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.256 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.256 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.256 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.256 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.256 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.256 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.256 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.256 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.256 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.256 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.257 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.257 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.257 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.257 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.257 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.257 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.257 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.257 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.257 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.257 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.257 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.257 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.258 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.258 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.258 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.258 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.258 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.258 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.258 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.258 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.258 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.258 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.258 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.259 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.259 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.259 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "k8sevents", 2024-09-15T23:12:12.259 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.259 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.259 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.259 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_event_retention_days": { 2024-09-15T23:12:12.259 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "ceph_event_retention_days", 2024-09-15T23:12:12.259 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.259 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.259 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.259 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "7", 2024-09-15T23:12:12.260 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.260 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.260 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.260 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-15T23:12:12.260 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.260 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.260 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.260 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.260 INFO:teuthology.orchestra.run.smithi111.stdout: "config_check_secs": { 2024-09-15T23:12:12.260 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "config_check_secs", 2024-09-15T23:12:12.260 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.261 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.261 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.261 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-09-15T23:12:12.261 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "10", 2024-09-15T23:12:12.261 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.261 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.261 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-15T23:12:12.261 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.261 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.261 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.262 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.262 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.262 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.262 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.262 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.262 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.262 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.262 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.262 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.262 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.262 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.262 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.262 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.263 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.263 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.263 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.263 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.263 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.263 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.263 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.263 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.263 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.263 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.263 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.264 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.264 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.264 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.264 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.264 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.264 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.264 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.264 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.264 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.264 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.264 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.264 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.265 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.265 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.265 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.265 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.265 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.265 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.265 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.265 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.265 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.265 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.265 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.266 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.266 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.266 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.266 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.266 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.266 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.266 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.266 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.266 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.266 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.266 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.267 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.267 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.267 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.267 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.267 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.267 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.267 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.267 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.267 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.267 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.267 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.267 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.268 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.268 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.268 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.268 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.268 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "localpool", 2024-09-15T23:12:12.268 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.268 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.268 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.268 INFO:teuthology.orchestra.run.smithi111.stdout: "failure_domain": { 2024-09-15T23:12:12.268 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "failure_domain", 2024-09-15T23:12:12.268 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.268 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.269 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.269 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "host", 2024-09-15T23:12:12.269 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.269 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.269 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.269 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "failure domain for any created local pool", 2024-09-15T23:12:12.269 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-15T23:12:12.269 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.269 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.269 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.269 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.270 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.270 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.270 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.270 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.270 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.270 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.270 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.270 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.270 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.270 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.270 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.270 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.271 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.271 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.271 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.271 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.271 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.271 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.271 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.271 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.271 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.271 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.272 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.272 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.272 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.272 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.272 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.272 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.272 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.272 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.272 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.272 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.272 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.272 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.273 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.273 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.273 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.273 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.273 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.273 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.273 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.273 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.273 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.273 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.273 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.274 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.274 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.274 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.274 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.274 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.274 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.274 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.274 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.274 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.274 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.274 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.274 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.275 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.275 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.275 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.275 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.275 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.275 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.275 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.275 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.275 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.275 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.275 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.275 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.276 INFO:teuthology.orchestra.run.smithi111.stdout: "min_size": { 2024-09-15T23:12:12.276 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "min_size", 2024-09-15T23:12:12.276 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.276 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.276 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.276 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.276 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.276 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.276 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.276 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "default min_size for any created local pool", 2024-09-15T23:12:12.277 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-15T23:12:12.277 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.277 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.277 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.277 INFO:teuthology.orchestra.run.smithi111.stdout: "num_rep": { 2024-09-15T23:12:12.277 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "num_rep", 2024-09-15T23:12:12.277 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.277 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.277 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.277 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "3", 2024-09-15T23:12:12.277 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.277 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.278 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.278 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "default replica count for any created local pool", 2024-09-15T23:12:12.278 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.278 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.278 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.278 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.278 INFO:teuthology.orchestra.run.smithi111.stdout: "pg_num": { 2024-09-15T23:12:12.278 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "pg_num", 2024-09-15T23:12:12.278 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.278 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.278 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.279 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "128", 2024-09-15T23:12:12.279 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.279 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.279 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.279 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "default pg_num for any created local pool", 2024-09-15T23:12:12.279 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.279 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.279 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.279 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.279 INFO:teuthology.orchestra.run.smithi111.stdout: "prefix": { 2024-09-15T23:12:12.279 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "prefix", 2024-09-15T23:12:12.280 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.280 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.280 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.280 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.280 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.280 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.280 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.280 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "name prefix for any created local pool", 2024-09-15T23:12:12.280 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.280 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.280 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.281 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.281 INFO:teuthology.orchestra.run.smithi111.stdout: "subtree": { 2024-09-15T23:12:12.281 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "subtree", 2024-09-15T23:12:12.281 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.281 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.281 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.281 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "rack", 2024-09-15T23:12:12.281 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.281 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.281 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.281 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-15T23:12:12.282 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-15T23:12:12.282 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.282 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.282 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.282 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.282 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.282 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.282 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mds_autoscaler", 2024-09-15T23:12:12.282 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.282 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.282 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.283 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.283 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.283 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.283 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.283 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.283 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.283 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.283 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.283 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.283 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.283 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.283 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.284 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.284 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.284 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.284 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.284 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.284 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.284 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.284 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.284 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.284 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.284 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.285 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.285 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.285 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.285 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.285 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.285 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.285 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.285 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.285 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.285 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.285 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.285 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.286 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.286 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.286 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.286 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.286 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.286 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.286 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.286 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.286 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.286 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.286 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.286 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.287 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.287 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.287 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.287 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.287 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.287 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.287 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.287 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.287 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.287 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.287 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.287 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.288 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.288 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.288 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.288 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.288 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.288 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.288 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.288 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.288 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.288 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.288 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.289 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.289 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.289 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.289 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mirroring", 2024-09-15T23:12:12.289 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.289 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.289 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.289 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.289 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.289 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.289 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.289 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.290 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.290 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.290 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.290 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.290 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.290 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.290 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.290 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.290 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.290 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.290 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.290 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.291 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.291 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.291 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.291 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.291 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.291 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.291 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.291 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.291 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.291 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.291 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.291 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.292 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.292 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.292 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.292 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.292 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.292 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.292 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.292 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.292 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.292 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.292 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.293 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.293 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.293 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.293 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.293 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.293 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.293 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.293 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.293 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.293 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.293 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.294 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.294 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.294 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.294 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.294 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.294 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.294 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.294 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.294 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.294 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.294 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.294 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.295 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.295 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.295 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.295 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.295 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.295 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.295 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.295 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.295 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.295 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.295 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "nfs", 2024-09-15T23:12:12.296 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.296 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.296 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.296 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.296 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.296 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.296 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.296 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.296 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.296 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.296 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.296 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.296 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.297 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.297 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.297 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.297 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.297 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.297 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.297 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.297 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.297 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.297 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.297 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.298 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.298 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.298 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.298 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.298 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.298 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.298 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.298 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.298 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.298 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.298 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.298 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.299 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.299 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.299 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.299 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.299 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.299 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.299 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.299 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.299 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.299 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.299 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.299 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.300 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.300 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.300 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.300 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.300 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.300 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.300 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.300 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.300 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.300 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.300 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.300 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.301 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.301 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.301 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.301 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.301 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.301 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.301 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.301 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.301 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.301 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.301 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.302 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.302 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.302 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.302 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.302 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.302 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "orchestrator", 2024-09-15T23:12:12.302 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.302 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.302 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.302 INFO:teuthology.orchestra.run.smithi111.stdout: "fail_fs": { 2024-09-15T23:12:12.302 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "fail_fs", 2024-09-15T23:12:12.303 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.303 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.303 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.303 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.303 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.303 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.303 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.303 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-15T23:12:12.303 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.303 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.303 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.304 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.304 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.304 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.304 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.304 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.304 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.304 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.304 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.304 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.304 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.304 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.305 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.305 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.305 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.305 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.305 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.305 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.305 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.305 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.305 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.305 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.305 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.305 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.306 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.306 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.306 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.306 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.306 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.306 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.306 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.306 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.306 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.306 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.306 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.307 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.307 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.307 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.307 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.307 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.307 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.307 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.307 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.307 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.307 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.307 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.307 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.308 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.308 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.308 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.308 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.308 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.308 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.308 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.308 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.308 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.308 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.308 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.309 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.309 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.309 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.309 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.309 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.309 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.309 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.309 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.309 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.309 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.309 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.309 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.310 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.310 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.310 INFO:teuthology.orchestra.run.smithi111.stdout: "orchestrator": { 2024-09-15T23:12:12.310 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "orchestrator", 2024-09-15T23:12:12.310 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.310 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.310 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.310 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.310 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.310 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.310 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.310 INFO:teuthology.orchestra.run.smithi111.stdout: "cephadm", 2024-09-15T23:12:12.311 INFO:teuthology.orchestra.run.smithi111.stdout: "rook", 2024-09-15T23:12:12.311 INFO:teuthology.orchestra.run.smithi111.stdout: "test_orchestrator" 2024-09-15T23:12:12.311 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.311 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Orchestrator backend", 2024-09-15T23:12:12.311 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.311 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.311 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.311 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.311 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.311 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.311 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.312 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "osd_perf_query", 2024-09-15T23:12:12.312 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.312 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.312 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.312 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.312 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.312 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.312 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.312 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.312 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.312 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.312 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.313 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.313 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.313 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.313 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.313 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.313 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.313 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.313 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.313 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.313 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.313 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.314 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.314 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.314 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.314 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.314 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.314 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.314 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.314 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.314 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.314 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.314 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.315 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.315 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.315 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.315 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.315 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.315 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.315 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.315 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.315 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.315 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.315 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.316 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.316 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.316 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.316 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.316 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.316 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.316 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.316 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.316 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.316 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.316 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.316 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.316 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.317 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.317 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.317 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.317 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.317 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.317 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.317 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.317 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.317 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.317 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.317 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.318 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.318 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.318 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.318 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.318 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.318 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.318 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.318 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.318 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "osd_support", 2024-09-15T23:12:12.318 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.318 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.318 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.319 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.319 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.319 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.319 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.319 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.319 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.319 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.319 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.319 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.319 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.319 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.320 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.320 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.320 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.320 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.320 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.320 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.320 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.320 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.320 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.320 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.320 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.320 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.321 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.321 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.321 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.321 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.321 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.321 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.321 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.321 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.321 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.321 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.321 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.322 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.322 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.322 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.322 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.322 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.322 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.322 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.322 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.322 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.322 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.322 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.322 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.323 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.323 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.323 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.323 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.323 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.323 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.323 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.323 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.323 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.323 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.324 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.324 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.324 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.324 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.324 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.324 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.324 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.324 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.324 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.324 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.324 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.325 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.325 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.325 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.325 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.325 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.325 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.325 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "pg_autoscaler", 2024-09-15T23:12:12.325 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.325 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.325 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.325 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.326 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.326 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.326 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.326 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.326 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.326 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.326 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.326 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.326 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.326 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.326 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.326 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.327 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.327 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.327 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.327 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.327 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.327 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.327 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.327 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.327 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.327 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.327 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.327 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.328 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.328 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.328 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.328 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.328 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.328 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.328 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.328 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.328 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.328 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.328 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.329 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.329 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.329 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.329 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.329 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.329 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.329 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.329 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.329 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.329 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.329 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.329 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.330 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.330 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.330 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.330 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.330 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.330 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.330 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.330 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.330 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.330 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.330 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.330 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.331 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.331 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.331 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.331 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.331 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.331 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.331 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.331 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.331 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.331 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.331 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-09-15T23:12:12.331 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-09-15T23:12:12.332 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:12.332 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.332 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.332 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-09-15T23:12:12.332 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.332 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.332 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.332 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.332 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.332 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.332 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.333 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.333 INFO:teuthology.orchestra.run.smithi111.stdout: "threshold": { 2024-09-15T23:12:12.333 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "threshold", 2024-09-15T23:12:12.333 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-09-15T23:12:12.333 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.333 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.333 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "3.0", 2024-09-15T23:12:12.333 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1.0", 2024-09-15T23:12:12.333 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.333 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.333 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "scaling threshold", 2024-09-15T23:12:12.334 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-15T23:12:12.334 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.334 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.334 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.334 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.334 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.334 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.334 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "progress", 2024-09-15T23:12:12.334 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.334 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.334 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.335 INFO:teuthology.orchestra.run.smithi111.stdout: "allow_pg_recovery_event": { 2024-09-15T23:12:12.335 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "allow_pg_recovery_event", 2024-09-15T23:12:12.335 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.335 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.335 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.335 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.335 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.335 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.335 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.335 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-15T23:12:12.335 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.336 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.336 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.336 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.336 INFO:teuthology.orchestra.run.smithi111.stdout: "enabled": { 2024-09-15T23:12:12.336 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "enabled", 2024-09-15T23:12:12.336 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.336 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.336 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.336 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.336 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.336 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.337 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.337 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.337 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.337 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.337 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.337 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.337 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.337 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.337 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.337 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.337 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.337 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.338 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.338 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.338 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.338 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.338 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.338 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.338 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.338 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.338 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.338 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.338 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.338 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.339 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.339 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.339 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.339 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.339 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.339 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.339 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.339 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.339 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.339 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.339 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.340 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.340 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.340 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.340 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.340 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.340 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.340 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.340 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.340 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.340 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.340 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.341 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.341 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.341 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.341 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.341 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.341 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.341 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.341 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.341 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.341 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.341 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.342 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.342 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.342 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.342 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.342 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.342 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.342 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.342 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.342 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.342 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.342 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.342 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.343 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.343 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.343 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.343 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.343 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.343 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.343 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.343 INFO:teuthology.orchestra.run.smithi111.stdout: "max_completed_events": { 2024-09-15T23:12:12.343 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_completed_events", 2024-09-15T23:12:12.343 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.343 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.343 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.344 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "50", 2024-09-15T23:12:12.344 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.344 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.344 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.344 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "number of past completed events to remember", 2024-09-15T23:12:12.344 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.344 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.344 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.344 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.344 INFO:teuthology.orchestra.run.smithi111.stdout: "sleep_interval": { 2024-09-15T23:12:12.344 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "sleep_interval", 2024-09-15T23:12:12.345 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:12.345 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.345 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.345 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5", 2024-09-15T23:12:12.345 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.345 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.345 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.345 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "how long the module is going to sleep", 2024-09-15T23:12:12.345 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.345 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.345 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.345 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.346 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.346 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.346 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.346 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "prometheus", 2024-09-15T23:12:12.346 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.346 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.346 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.346 INFO:teuthology.orchestra.run.smithi111.stdout: "cache": { 2024-09-15T23:12:12.346 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "cache", 2024-09-15T23:12:12.346 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.346 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.346 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.347 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.347 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.347 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.347 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.347 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.347 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.347 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.347 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.347 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.347 INFO:teuthology.orchestra.run.smithi111.stdout: "exclude_perf_counters": { 2024-09-15T23:12:12.347 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "exclude_perf_counters", 2024-09-15T23:12:12.348 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.348 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.348 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.348 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.348 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.348 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.348 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.348 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-15T23:12:12.348 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-15T23:12:12.348 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.348 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.349 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.349 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.349 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.349 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.349 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.349 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.349 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.349 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.349 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.349 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.349 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.349 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.350 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.350 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.350 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.350 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.350 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.350 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.350 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.350 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.350 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.350 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.350 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.351 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.351 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.351 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.351 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.351 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.351 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.351 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.351 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.351 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.351 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.351 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.352 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.352 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.352 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.352 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.352 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.352 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.352 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.352 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.352 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.352 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.352 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.352 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.353 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.353 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.353 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.353 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.353 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.353 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.353 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.353 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.353 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.353 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.353 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.353 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.354 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.354 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.354 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.354 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.354 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.354 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.354 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.354 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.354 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.354 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.354 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.355 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.355 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.355 INFO:teuthology.orchestra.run.smithi111.stdout: "rbd_stats_pools": { 2024-09-15T23:12:12.355 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rbd_stats_pools", 2024-09-15T23:12:12.355 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.355 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.355 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.355 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.355 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.355 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.355 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.355 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.356 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.356 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.356 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.356 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.356 INFO:teuthology.orchestra.run.smithi111.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-15T23:12:12.356 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-15T23:12:12.356 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.356 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.356 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.356 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "300", 2024-09-15T23:12:12.356 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.356 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.357 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.357 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.357 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.357 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.357 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.357 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.357 INFO:teuthology.orchestra.run.smithi111.stdout: "scrape_interval": { 2024-09-15T23:12:12.357 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "scrape_interval", 2024-09-15T23:12:12.357 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-09-15T23:12:12.357 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.357 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.358 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "15.0", 2024-09-15T23:12:12.358 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.358 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.358 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.358 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.358 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.358 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.358 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.358 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.358 INFO:teuthology.orchestra.run.smithi111.stdout: "server_addr": { 2024-09-15T23:12:12.358 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_addr", 2024-09-15T23:12:12.358 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.359 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.359 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.359 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "::", 2024-09-15T23:12:12.359 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.359 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.359 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.359 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-15T23:12:12.359 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.359 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.359 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.359 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.360 INFO:teuthology.orchestra.run.smithi111.stdout: "server_port": { 2024-09-15T23:12:12.360 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_port", 2024-09-15T23:12:12.360 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.360 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.360 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.360 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "9283", 2024-09-15T23:12:12.360 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.360 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.360 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.360 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-15T23:12:12.360 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.361 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.361 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.361 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.361 INFO:teuthology.orchestra.run.smithi111.stdout: "stale_cache_strategy": { 2024-09-15T23:12:12.361 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "stale_cache_strategy", 2024-09-15T23:12:12.361 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.361 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.361 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.361 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "log", 2024-09-15T23:12:12.361 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.361 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.362 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.362 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.362 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.362 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.362 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.362 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.362 INFO:teuthology.orchestra.run.smithi111.stdout: "standby_behaviour": { 2024-09-15T23:12:12.362 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "standby_behaviour", 2024-09-15T23:12:12.362 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.362 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.362 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.362 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "default", 2024-09-15T23:12:12.363 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.363 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.363 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.363 INFO:teuthology.orchestra.run.smithi111.stdout: "default", 2024-09-15T23:12:12.363 INFO:teuthology.orchestra.run.smithi111.stdout: "error" 2024-09-15T23:12:12.363 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.363 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.363 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.363 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.363 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.363 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.363 INFO:teuthology.orchestra.run.smithi111.stdout: "standby_error_status_code": { 2024-09-15T23:12:12.364 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "standby_error_status_code", 2024-09-15T23:12:12.364 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.364 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.364 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.364 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "500", 2024-09-15T23:12:12.364 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "400", 2024-09-15T23:12:12.364 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "599", 2024-09-15T23:12:12.364 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.364 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.364 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.364 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.365 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.365 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.365 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.365 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.365 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.365 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rbd_support", 2024-09-15T23:12:12.365 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.365 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.365 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.365 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.365 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.366 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.366 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.366 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.366 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.366 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.366 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.366 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.366 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.366 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.366 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.366 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.367 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.367 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.367 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.367 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.367 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.367 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.367 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.367 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.367 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.367 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.367 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.368 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.368 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.368 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.368 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.368 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.368 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.368 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.368 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.368 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.368 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.368 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.369 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.369 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.369 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.369 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.369 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.369 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.369 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.369 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.369 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.369 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.369 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.369 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.370 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.370 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.370 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.370 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.370 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.370 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.370 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.370 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.370 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.370 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.370 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.370 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.371 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.371 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.371 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.371 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.371 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.371 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.371 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.371 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.371 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.371 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.371 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.371 INFO:teuthology.orchestra.run.smithi111.stdout: "max_concurrent_snap_create": { 2024-09-15T23:12:12.372 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_concurrent_snap_create", 2024-09-15T23:12:12.372 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.372 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.372 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.372 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10", 2024-09-15T23:12:12.372 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.372 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.372 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.372 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.372 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.372 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.373 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.373 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.373 INFO:teuthology.orchestra.run.smithi111.stdout: "mirror_snapshot_schedule": { 2024-09-15T23:12:12.373 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "mirror_snapshot_schedule", 2024-09-15T23:12:12.373 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.373 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.373 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.373 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.373 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.373 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.373 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.374 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.374 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.374 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.374 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.374 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.374 INFO:teuthology.orchestra.run.smithi111.stdout: "trash_purge_schedule": { 2024-09-15T23:12:12.374 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "trash_purge_schedule", 2024-09-15T23:12:12.374 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.374 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.374 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.374 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.374 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.375 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.375 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.375 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.375 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.375 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.375 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.375 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.375 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.375 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.375 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.375 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "restful", 2024-09-15T23:12:12.375 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.375 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.376 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.376 INFO:teuthology.orchestra.run.smithi111.stdout: "enable_auth": { 2024-09-15T23:12:12.376 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "enable_auth", 2024-09-15T23:12:12.376 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.376 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.376 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.376 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.376 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.376 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.376 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.376 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.376 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.377 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.377 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.377 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.377 INFO:teuthology.orchestra.run.smithi111.stdout: "key_file": { 2024-09-15T23:12:12.377 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "key_file", 2024-09-15T23:12:12.377 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.377 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.377 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.377 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.377 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.377 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.377 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.378 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.378 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.378 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.378 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.378 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.378 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.378 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.378 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.378 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.378 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.378 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.378 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.379 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.379 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.379 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.379 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.379 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.379 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.379 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.379 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.379 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.379 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.379 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.379 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.380 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.380 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.380 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.380 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.380 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.380 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.380 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.380 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.380 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.380 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.380 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.380 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.380 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.381 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.381 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.381 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.381 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.381 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.381 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.381 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.381 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.381 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.381 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.381 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.382 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.382 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.382 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.382 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.382 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.382 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.382 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.382 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.382 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.382 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.382 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.382 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.382 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.383 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.383 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.383 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.383 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.383 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.383 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.383 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.383 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.383 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.383 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.383 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.383 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.383 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.384 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.384 INFO:teuthology.orchestra.run.smithi111.stdout: "server_addr": { 2024-09-15T23:12:12.384 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_addr", 2024-09-15T23:12:12.384 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.384 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.384 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.384 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.384 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.384 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.384 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.384 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.384 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.385 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.385 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.385 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.385 INFO:teuthology.orchestra.run.smithi111.stdout: "server_port": { 2024-09-15T23:12:12.385 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "server_port", 2024-09-15T23:12:12.385 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.385 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.385 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.385 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.385 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.385 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.386 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.386 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.386 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.386 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.386 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.386 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.386 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.386 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.386 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.386 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rgw", 2024-09-15T23:12:12.386 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.386 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.387 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.387 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.387 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.387 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.387 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.387 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.387 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.387 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.387 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.387 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.387 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.387 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.388 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.388 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.388 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.388 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.388 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.388 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.388 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.388 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.388 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.388 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.388 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.389 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.389 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.389 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.389 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.389 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.389 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.389 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.389 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.389 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.389 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.389 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.390 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.390 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.390 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.390 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.390 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.390 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.390 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.390 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.390 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.390 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.390 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.390 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.391 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.391 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.391 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.391 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.391 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.391 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.391 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.391 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.391 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.391 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.392 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.392 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.392 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.392 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.392 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.392 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.392 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.392 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.392 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.392 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.392 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.392 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.393 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.393 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.393 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.393 INFO:teuthology.orchestra.run.smithi111.stdout: "secondary_zone_period_retry_limit": { 2024-09-15T23:12:12.393 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-15T23:12:12.393 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.393 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.393 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.393 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "5", 2024-09-15T23:12:12.393 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.394 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.394 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.394 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-15T23:12:12.394 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.394 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.394 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.394 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.394 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.394 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.394 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.394 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rook", 2024-09-15T23:12:12.395 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.395 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.395 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.395 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.395 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.395 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.395 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.395 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.395 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.395 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.395 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.396 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.396 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.396 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.396 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.396 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.396 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.396 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.396 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.396 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.396 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.396 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.397 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.397 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.397 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.397 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.397 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.397 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.397 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.397 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.397 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.397 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.397 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.398 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.398 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.398 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.398 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.398 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.398 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.398 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.398 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.398 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.398 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.399 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.399 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.399 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.399 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.399 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.399 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.399 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.399 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.399 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.399 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.399 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.399 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.400 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.400 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.400 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.400 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.400 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.400 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.400 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.400 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.400 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.400 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.400 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.400 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.401 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.401 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.401 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.401 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.401 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.401 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.401 INFO:teuthology.orchestra.run.smithi111.stdout: "storage_class": { 2024-09-15T23:12:12.401 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "storage_class", 2024-09-15T23:12:12.401 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.401 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.401 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.402 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "local", 2024-09-15T23:12:12.402 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.402 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.402 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.402 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-15T23:12:12.402 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.402 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.402 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.402 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.402 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.402 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.402 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.403 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "selftest", 2024-09-15T23:12:12.403 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.403 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.403 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.403 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.403 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.403 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.403 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.403 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.403 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.403 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.403 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.403 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.404 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.404 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.404 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.404 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.404 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.404 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.404 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.404 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.404 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.404 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.404 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.404 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.405 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.405 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.405 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.405 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.405 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.405 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.405 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.405 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.405 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.405 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.406 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.406 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.406 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.406 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.406 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.406 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.406 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.406 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.406 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.406 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.406 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.406 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.407 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.407 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.407 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.407 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.407 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.407 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.407 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.407 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.407 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.407 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.407 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.408 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.408 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.408 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.408 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.408 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.408 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.408 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.408 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.408 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.408 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.408 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.409 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.409 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.409 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.409 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.409 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.409 INFO:teuthology.orchestra.run.smithi111.stdout: "roption1": { 2024-09-15T23:12:12.409 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "roption1", 2024-09-15T23:12:12.409 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.409 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.409 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.410 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.410 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.410 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.410 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.410 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.410 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.410 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.410 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.410 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.410 INFO:teuthology.orchestra.run.smithi111.stdout: "roption2": { 2024-09-15T23:12:12.410 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "roption2", 2024-09-15T23:12:12.411 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.411 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.411 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.411 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "xyz", 2024-09-15T23:12:12.411 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.411 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.411 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.411 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.411 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.411 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.411 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.411 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.412 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption1": { 2024-09-15T23:12:12.412 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption1", 2024-09-15T23:12:12.412 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.412 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.412 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.412 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.412 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.412 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.412 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.412 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.412 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.413 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.413 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.413 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.413 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption2": { 2024-09-15T23:12:12.413 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption2", 2024-09-15T23:12:12.413 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.413 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.413 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.413 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.413 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.413 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.414 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.414 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.414 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.414 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.414 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.414 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.414 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption3": { 2024-09-15T23:12:12.414 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption3", 2024-09-15T23:12:12.414 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "float", 2024-09-15T23:12:12.414 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.414 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.414 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.415 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.415 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.415 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.415 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.415 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.415 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.415 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.415 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.415 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption4": { 2024-09-15T23:12:12.415 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption4", 2024-09-15T23:12:12.416 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.416 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.416 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.416 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.416 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.416 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.416 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.416 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.416 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.416 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.416 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.417 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.417 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption5": { 2024-09-15T23:12:12.417 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption5", 2024-09-15T23:12:12.417 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.417 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.417 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.417 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.417 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.417 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.417 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.417 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.418 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.418 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.418 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.418 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.418 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption6": { 2024-09-15T23:12:12.418 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption6", 2024-09-15T23:12:12.418 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.418 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.418 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.418 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.418 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.418 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.419 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.419 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.419 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.419 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.419 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.419 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.419 INFO:teuthology.orchestra.run.smithi111.stdout: "rwoption7": { 2024-09-15T23:12:12.419 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rwoption7", 2024-09-15T23:12:12.419 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.419 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.419 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.420 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.420 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "1", 2024-09-15T23:12:12.420 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "42", 2024-09-15T23:12:12.420 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.420 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.420 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.420 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.420 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.420 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.420 INFO:teuthology.orchestra.run.smithi111.stdout: "testkey": { 2024-09-15T23:12:12.420 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "testkey", 2024-09-15T23:12:12.420 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.421 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.421 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.421 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.421 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.421 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.421 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.421 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.421 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.421 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.421 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.421 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.422 INFO:teuthology.orchestra.run.smithi111.stdout: "testlkey": { 2024-09-15T23:12:12.422 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "testlkey", 2024-09-15T23:12:12.422 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.422 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.422 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.422 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.422 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.422 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.422 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.422 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.422 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.423 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.423 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.423 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.423 INFO:teuthology.orchestra.run.smithi111.stdout: "testnewline": { 2024-09-15T23:12:12.423 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "testnewline", 2024-09-15T23:12:12.423 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.423 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.423 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.423 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.423 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.424 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.424 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.424 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.424 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.424 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.424 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.424 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.424 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.424 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.424 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.424 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "snap_schedule", 2024-09-15T23:12:12.424 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.425 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.425 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.425 INFO:teuthology.orchestra.run.smithi111.stdout: "allow_m_granularity": { 2024-09-15T23:12:12.425 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "allow_m_granularity", 2024-09-15T23:12:12.425 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.425 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.425 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.425 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.425 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.425 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.425 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.425 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "allow minute scheduled snapshots", 2024-09-15T23:12:12.426 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.426 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.426 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.426 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.426 INFO:teuthology.orchestra.run.smithi111.stdout: "dump_on_update": { 2024-09-15T23:12:12.426 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "dump_on_update", 2024-09-15T23:12:12.426 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.426 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.426 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.426 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.426 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.427 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.427 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.427 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "dump database to debug log on update", 2024-09-15T23:12:12.427 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.427 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.427 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.427 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.427 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.427 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.427 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.427 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.427 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.428 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.428 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.428 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.428 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.428 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.428 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.428 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.428 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.428 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.428 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.428 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.428 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.429 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.429 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.429 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.429 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.429 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.429 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.429 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.429 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.429 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.429 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.429 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.430 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.430 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.430 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.430 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.430 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.430 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.430 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.430 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.430 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.430 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.430 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.431 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.431 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.431 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.431 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.431 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.431 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.431 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.431 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.431 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.431 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.431 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.431 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.432 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.432 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.432 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.432 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.432 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.432 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.432 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.432 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.432 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.432 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.432 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.432 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.433 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.433 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.433 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.433 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.433 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.433 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.433 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.433 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.433 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.433 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.433 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "stats", 2024-09-15T23:12:12.433 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.434 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.434 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.434 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.434 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.434 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.434 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.434 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.434 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.434 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.434 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.434 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.435 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.435 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.435 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.435 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.435 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.435 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.435 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.435 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.435 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.435 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.435 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.435 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.436 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.436 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.436 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.436 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.436 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.436 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.436 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.436 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.436 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.436 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.437 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.437 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.437 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.437 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.437 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.437 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.437 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.437 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.437 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.437 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.437 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.438 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.438 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.438 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.438 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.438 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.438 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.438 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.438 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.438 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.438 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.438 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.439 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.439 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.439 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.439 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.439 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.439 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.439 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.439 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.439 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.439 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.439 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.439 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.440 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.440 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.440 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.440 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.440 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.440 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.440 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.440 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.440 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "status", 2024-09-15T23:12:12.440 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.440 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.441 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.441 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.441 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.441 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.441 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.441 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.441 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.441 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.441 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.441 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.441 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.441 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.442 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.442 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.442 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.442 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.442 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.442 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.442 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.442 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.442 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.442 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.442 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.442 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.443 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.443 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.443 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.443 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.443 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.443 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.443 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.443 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.443 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.443 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.443 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.444 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.444 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.444 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.444 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.444 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.444 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.444 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.444 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.444 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.444 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.444 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.445 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.445 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.445 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.445 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.445 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.445 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.445 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.445 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.445 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.445 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.445 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.445 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.446 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.446 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.446 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.446 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.446 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.446 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.446 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.446 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.446 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.446 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.446 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.446 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.447 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.447 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.447 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.447 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.447 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "telegraf", 2024-09-15T23:12:12.447 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.447 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.447 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.447 INFO:teuthology.orchestra.run.smithi111.stdout: "address": { 2024-09-15T23:12:12.447 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "address", 2024-09-15T23:12:12.448 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.448 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.448 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.448 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-15T23:12:12.448 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.448 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.448 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.448 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.448 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.448 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.448 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.449 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.449 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-09-15T23:12:12.449 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-09-15T23:12:12.449 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:12.449 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.449 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.449 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "15", 2024-09-15T23:12:12.449 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.449 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.449 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.449 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.449 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.450 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.450 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.450 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.450 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.450 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.450 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.450 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.450 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.450 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.450 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.450 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.450 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.451 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.451 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.451 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.451 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.451 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.451 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.451 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.451 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.451 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.451 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.451 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.452 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.452 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.452 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.452 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.452 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.452 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.452 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.452 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.452 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.452 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.452 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.453 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.453 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.453 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.453 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.453 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.453 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.453 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.453 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.453 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.453 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.453 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.454 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.454 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.454 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.454 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.454 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.454 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.454 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.454 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.454 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.454 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.454 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.455 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.455 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.455 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.455 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.455 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.455 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.455 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.455 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.455 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.455 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.455 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.456 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.456 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.456 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.456 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.456 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.456 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.456 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.456 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.456 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.456 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "telemetry", 2024-09-15T23:12:12.456 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.457 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.457 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.457 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_basic": { 2024-09-15T23:12:12.457 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_basic", 2024-09-15T23:12:12.457 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.457 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.457 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.457 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.457 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.457 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.457 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.458 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-15T23:12:12.458 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.458 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.458 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.458 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.458 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_crash": { 2024-09-15T23:12:12.458 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_crash", 2024-09-15T23:12:12.458 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.458 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.458 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.458 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.459 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.459 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.459 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.459 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-15T23:12:12.459 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.459 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.459 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.459 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.459 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_device": { 2024-09-15T23:12:12.459 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_device", 2024-09-15T23:12:12.459 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.459 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.460 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.460 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.460 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.460 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.460 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.460 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-15T23:12:12.460 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.460 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.460 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.460 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.460 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_ident": { 2024-09-15T23:12:12.460 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_ident", 2024-09-15T23:12:12.461 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.461 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.461 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.461 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.461 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.461 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.461 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.461 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-15T23:12:12.461 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.461 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.461 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.462 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.462 INFO:teuthology.orchestra.run.smithi111.stdout: "channel_perf": { 2024-09-15T23:12:12.462 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "channel_perf", 2024-09-15T23:12:12.462 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.462 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.462 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.462 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.462 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.462 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.462 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.462 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-15T23:12:12.462 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.463 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.463 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.463 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.463 INFO:teuthology.orchestra.run.smithi111.stdout: "contact": { 2024-09-15T23:12:12.463 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "contact", 2024-09-15T23:12:12.463 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.463 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.463 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.463 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.463 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.463 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.464 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.464 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.464 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.464 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.464 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.464 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.464 INFO:teuthology.orchestra.run.smithi111.stdout: "description": { 2024-09-15T23:12:12.464 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "description", 2024-09-15T23:12:12.464 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.464 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.464 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.465 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.465 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.465 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.465 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.465 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.465 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.465 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.465 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.465 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.465 INFO:teuthology.orchestra.run.smithi111.stdout: "device_url": { 2024-09-15T23:12:12.465 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "device_url", 2024-09-15T23:12:12.465 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.466 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.466 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.466 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-15T23:12:12.466 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.466 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.466 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.466 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.466 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.466 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.466 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.466 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.466 INFO:teuthology.orchestra.run.smithi111.stdout: "enabled": { 2024-09-15T23:12:12.466 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "enabled", 2024-09-15T23:12:12.467 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.467 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.467 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.467 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.467 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.467 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.467 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.467 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.467 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.467 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.467 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.467 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.468 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-09-15T23:12:12.468 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-09-15T23:12:12.468 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.468 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.468 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.468 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "24", 2024-09-15T23:12:12.468 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "8", 2024-09-15T23:12:12.468 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.468 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.468 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.468 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.468 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.469 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.469 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.469 INFO:teuthology.orchestra.run.smithi111.stdout: "last_opt_revision": { 2024-09-15T23:12:12.469 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "last_opt_revision", 2024-09-15T23:12:12.469 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.469 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.469 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.469 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "1", 2024-09-15T23:12:12.469 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.469 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.469 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.470 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.470 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.470 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.470 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.470 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.470 INFO:teuthology.orchestra.run.smithi111.stdout: "leaderboard": { 2024-09-15T23:12:12.470 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "leaderboard", 2024-09-15T23:12:12.470 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.470 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.470 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.470 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.470 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.471 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.471 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.471 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.471 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.471 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.471 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.471 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.471 INFO:teuthology.orchestra.run.smithi111.stdout: "leaderboard_description": { 2024-09-15T23:12:12.471 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "leaderboard_description", 2024-09-15T23:12:12.471 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.471 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.471 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.472 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.472 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.472 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.472 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.472 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.472 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.472 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.472 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.472 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.472 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.472 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.472 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.473 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.473 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.473 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.473 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.473 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.473 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.473 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.473 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.473 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.473 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.473 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.473 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.474 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.474 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.474 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.474 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.474 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.474 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.474 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.474 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.474 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.474 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.474 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.474 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.475 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.475 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.475 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.475 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.475 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.475 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.475 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.475 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.475 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.475 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.475 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.475 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.476 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.476 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.476 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.476 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.476 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.476 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.476 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.476 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.476 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.476 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.476 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.477 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.477 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.477 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.477 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.477 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.477 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.477 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.477 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.477 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.477 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.477 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.478 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.478 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.478 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.478 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.478 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.478 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.478 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.478 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.478 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.478 INFO:teuthology.orchestra.run.smithi111.stdout: "organization": { 2024-09-15T23:12:12.478 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "organization", 2024-09-15T23:12:12.479 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.479 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.479 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.479 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.479 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.479 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.479 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.479 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.479 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.479 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.479 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.479 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.480 INFO:teuthology.orchestra.run.smithi111.stdout: "proxy": { 2024-09-15T23:12:12.480 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "proxy", 2024-09-15T23:12:12.480 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.480 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.480 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.480 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.480 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.480 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.480 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.480 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.480 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.480 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.481 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.481 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.481 INFO:teuthology.orchestra.run.smithi111.stdout: "url": { 2024-09-15T23:12:12.481 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "url", 2024-09-15T23:12:12.481 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.481 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.481 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.481 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-15T23:12:12.481 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.481 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.481 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.482 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.482 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.482 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.482 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.482 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.482 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.482 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.482 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.482 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "test_orchestrator", 2024-09-15T23:12:12.482 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.482 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.482 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.483 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.483 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.483 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.483 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.483 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.483 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.483 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.483 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.483 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.483 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.483 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.483 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.484 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.484 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.484 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.484 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.484 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.484 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.484 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.484 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.484 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.484 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.484 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.485 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.485 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.485 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.485 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.485 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.485 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.485 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.485 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.485 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.485 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.485 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.485 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.486 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.486 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.486 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.486 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.486 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.486 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.486 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.486 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.486 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.486 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.486 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.487 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.487 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.487 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.487 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.487 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.487 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.487 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.487 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.487 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.487 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.487 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.487 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.488 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.488 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.488 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.488 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.488 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.488 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.488 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.488 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.488 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.488 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.488 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.489 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.489 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.489 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.489 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.489 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "volumes", 2024-09-15T23:12:12.489 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.489 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.489 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.489 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.489 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.489 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.489 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.490 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.490 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.490 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.490 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.490 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.490 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.490 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.490 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.490 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.490 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.490 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.491 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.491 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.491 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.491 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.491 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.491 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.491 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.491 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.491 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.491 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.491 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.491 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.492 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.492 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.492 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.492 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.492 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.492 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.492 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.492 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.492 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.492 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.492 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.493 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.493 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.493 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.493 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.493 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.493 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.493 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.493 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.493 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.493 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.493 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.493 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.494 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.494 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.494 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.494 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.494 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.494 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.494 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.494 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.494 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.494 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.494 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.494 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.495 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.495 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.495 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.495 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.495 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.495 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.495 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.495 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.495 INFO:teuthology.orchestra.run.smithi111.stdout: "max_concurrent_clones": { 2024-09-15T23:12:12.495 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "max_concurrent_clones", 2024-09-15T23:12:12.495 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.496 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.496 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.496 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "4", 2024-09-15T23:12:12.496 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.496 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.496 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.496 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-15T23:12:12.496 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.496 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.496 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.496 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.496 INFO:teuthology.orchestra.run.smithi111.stdout: "periodic_async_work": { 2024-09-15T23:12:12.497 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "periodic_async_work", 2024-09-15T23:12:12.497 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.497 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.497 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.497 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.497 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.497 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.497 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.497 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Periodically check for async work", 2024-09-15T23:12:12.497 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.497 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.498 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.498 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.498 INFO:teuthology.orchestra.run.smithi111.stdout: "snapshot_clone_delay": { 2024-09-15T23:12:12.498 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "snapshot_clone_delay", 2024-09-15T23:12:12.498 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.498 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.498 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.498 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "0", 2024-09-15T23:12:12.498 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.498 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.498 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.499 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-15T23:12:12.499 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.499 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.499 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.499 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.499 INFO:teuthology.orchestra.run.smithi111.stdout: "snapshot_clone_no_wait": { 2024-09-15T23:12:12.499 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "snapshot_clone_no_wait", 2024-09-15T23:12:12.499 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.499 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.499 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.499 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "True", 2024-09-15T23:12:12.499 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.500 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.500 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.500 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-15T23:12:12.500 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.500 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.500 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.500 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.500 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.500 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.500 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.500 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "zabbix", 2024-09-15T23:12:12.501 INFO:teuthology.orchestra.run.smithi111.stdout: "can_run": true, 2024-09-15T23:12:12.501 INFO:teuthology.orchestra.run.smithi111.stdout: "error_string": "", 2024-09-15T23:12:12.501 INFO:teuthology.orchestra.run.smithi111.stdout: "module_options": { 2024-09-15T23:12:12.501 INFO:teuthology.orchestra.run.smithi111.stdout: "discovery_interval": { 2024-09-15T23:12:12.501 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "discovery_interval", 2024-09-15T23:12:12.501 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "uint", 2024-09-15T23:12:12.501 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.501 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.501 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "100", 2024-09-15T23:12:12.501 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.501 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.502 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.502 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.502 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.502 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.502 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.502 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.502 INFO:teuthology.orchestra.run.smithi111.stdout: "identifier": { 2024-09-15T23:12:12.502 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "identifier", 2024-09-15T23:12:12.502 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.502 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.502 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.503 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.503 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.503 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.503 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.503 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.503 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.503 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.503 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.503 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.503 INFO:teuthology.orchestra.run.smithi111.stdout: "interval": { 2024-09-15T23:12:12.503 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "interval", 2024-09-15T23:12:12.503 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "secs", 2024-09-15T23:12:12.504 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.504 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.504 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "60", 2024-09-15T23:12:12.504 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.504 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.504 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.504 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.504 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.504 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.504 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.504 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.504 INFO:teuthology.orchestra.run.smithi111.stdout: "log_level": { 2024-09-15T23:12:12.505 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_level", 2024-09-15T23:12:12.505 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.505 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.505 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.505 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.505 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.505 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.505 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.505 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.505 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.505 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.505 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.506 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.506 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.506 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.506 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.506 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.506 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.506 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.506 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.506 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster": { 2024-09-15T23:12:12.506 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster", 2024-09-15T23:12:12.506 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.506 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.507 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.507 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.507 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.507 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.507 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.507 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.507 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.507 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.507 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.507 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.507 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_cluster_level": { 2024-09-15T23:12:12.507 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_cluster_level", 2024-09-15T23:12:12.508 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.508 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.508 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.508 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "info", 2024-09-15T23:12:12.508 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.508 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.508 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [ 2024-09-15T23:12:12.508 INFO:teuthology.orchestra.run.smithi111.stdout: "", 2024-09-15T23:12:12.508 INFO:teuthology.orchestra.run.smithi111.stdout: "critical", 2024-09-15T23:12:12.508 INFO:teuthology.orchestra.run.smithi111.stdout: "debug", 2024-09-15T23:12:12.508 INFO:teuthology.orchestra.run.smithi111.stdout: "error", 2024-09-15T23:12:12.509 INFO:teuthology.orchestra.run.smithi111.stdout: "info", 2024-09-15T23:12:12.509 INFO:teuthology.orchestra.run.smithi111.stdout: "warning" 2024-09-15T23:12:12.509 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.509 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.509 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.509 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.509 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.509 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.509 INFO:teuthology.orchestra.run.smithi111.stdout: "log_to_file": { 2024-09-15T23:12:12.509 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "log_to_file", 2024-09-15T23:12:12.509 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "bool", 2024-09-15T23:12:12.510 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.510 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 1, 2024-09-15T23:12:12.510 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "False", 2024-09-15T23:12:12.510 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.510 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.510 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.510 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.510 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.510 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.510 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.510 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.511 INFO:teuthology.orchestra.run.smithi111.stdout: "zabbix_host": { 2024-09-15T23:12:12.511 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "zabbix_host", 2024-09-15T23:12:12.511 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.511 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.511 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.511 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "", 2024-09-15T23:12:12.511 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.511 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.511 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.511 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.511 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.511 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.511 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.512 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.512 INFO:teuthology.orchestra.run.smithi111.stdout: "zabbix_port": { 2024-09-15T23:12:12.512 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "zabbix_port", 2024-09-15T23:12:12.512 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "int", 2024-09-15T23:12:12.512 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.512 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.512 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "10051", 2024-09-15T23:12:12.512 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.512 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.512 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.512 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.513 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.513 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.513 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.513 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.513 INFO:teuthology.orchestra.run.smithi111.stdout: "zabbix_sender": { 2024-09-15T23:12:12.513 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "zabbix_sender", 2024-09-15T23:12:12.513 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "str", 2024-09-15T23:12:12.513 INFO:teuthology.orchestra.run.smithi111.stdout: "level": "advanced", 2024-09-15T23:12:12.513 INFO:teuthology.orchestra.run.smithi111.stdout: "flags": 0, 2024-09-15T23:12:12.513 INFO:teuthology.orchestra.run.smithi111.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-15T23:12:12.513 INFO:teuthology.orchestra.run.smithi111.stdout: "min": "", 2024-09-15T23:12:12.513 INFO:teuthology.orchestra.run.smithi111.stdout: "max": "", 2024-09-15T23:12:12.513 INFO:teuthology.orchestra.run.smithi111.stdout: "enum_allowed": [], 2024-09-15T23:12:12.514 INFO:teuthology.orchestra.run.smithi111.stdout: "desc": "", 2024-09-15T23:12:12.514 INFO:teuthology.orchestra.run.smithi111.stdout: "long_desc": "", 2024-09-15T23:12:12.514 INFO:teuthology.orchestra.run.smithi111.stdout: "tags": [], 2024-09-15T23:12:12.514 INFO:teuthology.orchestra.run.smithi111.stdout: "see_also": [] 2024-09-15T23:12:12.514 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.514 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.514 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.514 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.514 INFO:teuthology.orchestra.run.smithi111.stdout: "services": { 2024-09-15T23:12:12.514 INFO:teuthology.orchestra.run.smithi111.stdout: "dashboard": "https://172.21.15.111:8443/" 2024-09-15T23:12:12.514 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.514 INFO:teuthology.orchestra.run.smithi111.stdout: "always_on_modules": { 2024-09-15T23:12:12.514 INFO:teuthology.orchestra.run.smithi111.stdout: "octopus": [ 2024-09-15T23:12:12.515 INFO:teuthology.orchestra.run.smithi111.stdout: "balancer", 2024-09-15T23:12:12.515 INFO:teuthology.orchestra.run.smithi111.stdout: "crash", 2024-09-15T23:12:12.515 INFO:teuthology.orchestra.run.smithi111.stdout: "devicehealth", 2024-09-15T23:12:12.515 INFO:teuthology.orchestra.run.smithi111.stdout: "orchestrator", 2024-09-15T23:12:12.515 INFO:teuthology.orchestra.run.smithi111.stdout: "pg_autoscaler", 2024-09-15T23:12:12.515 INFO:teuthology.orchestra.run.smithi111.stdout: "progress", 2024-09-15T23:12:12.515 INFO:teuthology.orchestra.run.smithi111.stdout: "rbd_support", 2024-09-15T23:12:12.515 INFO:teuthology.orchestra.run.smithi111.stdout: "status", 2024-09-15T23:12:12.515 INFO:teuthology.orchestra.run.smithi111.stdout: "telemetry", 2024-09-15T23:12:12.515 INFO:teuthology.orchestra.run.smithi111.stdout: "volumes" 2024-09-15T23:12:12.515 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.516 INFO:teuthology.orchestra.run.smithi111.stdout: "pacific": [ 2024-09-15T23:12:12.516 INFO:teuthology.orchestra.run.smithi111.stdout: "balancer", 2024-09-15T23:12:12.516 INFO:teuthology.orchestra.run.smithi111.stdout: "crash", 2024-09-15T23:12:12.516 INFO:teuthology.orchestra.run.smithi111.stdout: "devicehealth", 2024-09-15T23:12:12.516 INFO:teuthology.orchestra.run.smithi111.stdout: "orchestrator", 2024-09-15T23:12:12.516 INFO:teuthology.orchestra.run.smithi111.stdout: "pg_autoscaler", 2024-09-15T23:12:12.516 INFO:teuthology.orchestra.run.smithi111.stdout: "progress", 2024-09-15T23:12:12.516 INFO:teuthology.orchestra.run.smithi111.stdout: "rbd_support", 2024-09-15T23:12:12.516 INFO:teuthology.orchestra.run.smithi111.stdout: "status", 2024-09-15T23:12:12.516 INFO:teuthology.orchestra.run.smithi111.stdout: "telemetry", 2024-09-15T23:12:12.516 INFO:teuthology.orchestra.run.smithi111.stdout: "volumes" 2024-09-15T23:12:12.516 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.516 INFO:teuthology.orchestra.run.smithi111.stdout: "quincy": [ 2024-09-15T23:12:12.517 INFO:teuthology.orchestra.run.smithi111.stdout: "balancer", 2024-09-15T23:12:12.517 INFO:teuthology.orchestra.run.smithi111.stdout: "crash", 2024-09-15T23:12:12.517 INFO:teuthology.orchestra.run.smithi111.stdout: "devicehealth", 2024-09-15T23:12:12.517 INFO:teuthology.orchestra.run.smithi111.stdout: "orchestrator", 2024-09-15T23:12:12.517 INFO:teuthology.orchestra.run.smithi111.stdout: "pg_autoscaler", 2024-09-15T23:12:12.517 INFO:teuthology.orchestra.run.smithi111.stdout: "progress", 2024-09-15T23:12:12.517 INFO:teuthology.orchestra.run.smithi111.stdout: "rbd_support", 2024-09-15T23:12:12.517 INFO:teuthology.orchestra.run.smithi111.stdout: "status", 2024-09-15T23:12:12.517 INFO:teuthology.orchestra.run.smithi111.stdout: "telemetry", 2024-09-15T23:12:12.517 INFO:teuthology.orchestra.run.smithi111.stdout: "volumes" 2024-09-15T23:12:12.517 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.517 INFO:teuthology.orchestra.run.smithi111.stdout: "reef": [ 2024-09-15T23:12:12.518 INFO:teuthology.orchestra.run.smithi111.stdout: "balancer", 2024-09-15T23:12:12.518 INFO:teuthology.orchestra.run.smithi111.stdout: "crash", 2024-09-15T23:12:12.518 INFO:teuthology.orchestra.run.smithi111.stdout: "devicehealth", 2024-09-15T23:12:12.518 INFO:teuthology.orchestra.run.smithi111.stdout: "orchestrator", 2024-09-15T23:12:12.518 INFO:teuthology.orchestra.run.smithi111.stdout: "pg_autoscaler", 2024-09-15T23:12:12.518 INFO:teuthology.orchestra.run.smithi111.stdout: "progress", 2024-09-15T23:12:12.518 INFO:teuthology.orchestra.run.smithi111.stdout: "rbd_support", 2024-09-15T23:12:12.518 INFO:teuthology.orchestra.run.smithi111.stdout: "status", 2024-09-15T23:12:12.518 INFO:teuthology.orchestra.run.smithi111.stdout: "telemetry", 2024-09-15T23:12:12.518 INFO:teuthology.orchestra.run.smithi111.stdout: "volumes" 2024-09-15T23:12:12.518 INFO:teuthology.orchestra.run.smithi111.stdout: ], 2024-09-15T23:12:12.518 INFO:teuthology.orchestra.run.smithi111.stdout: "squid": [ 2024-09-15T23:12:12.519 INFO:teuthology.orchestra.run.smithi111.stdout: "balancer", 2024-09-15T23:12:12.519 INFO:teuthology.orchestra.run.smithi111.stdout: "crash", 2024-09-15T23:12:12.519 INFO:teuthology.orchestra.run.smithi111.stdout: "devicehealth", 2024-09-15T23:12:12.519 INFO:teuthology.orchestra.run.smithi111.stdout: "orchestrator", 2024-09-15T23:12:12.519 INFO:teuthology.orchestra.run.smithi111.stdout: "pg_autoscaler", 2024-09-15T23:12:12.519 INFO:teuthology.orchestra.run.smithi111.stdout: "progress", 2024-09-15T23:12:12.519 INFO:teuthology.orchestra.run.smithi111.stdout: "rbd_support", 2024-09-15T23:12:12.519 INFO:teuthology.orchestra.run.smithi111.stdout: "status", 2024-09-15T23:12:12.519 INFO:teuthology.orchestra.run.smithi111.stdout: "telemetry", 2024-09-15T23:12:12.519 INFO:teuthology.orchestra.run.smithi111.stdout: "volumes" 2024-09-15T23:12:12.519 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-15T23:12:12.520 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.520 INFO:teuthology.orchestra.run.smithi111.stdout: "last_failure_osd_epoch": 3, 2024-09-15T23:12:12.520 INFO:teuthology.orchestra.run.smithi111.stdout: "active_clients": [ 2024-09-15T23:12:12.520 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.520 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "devicehealth", 2024-09-15T23:12:12.520 INFO:teuthology.orchestra.run.smithi111.stdout: "addrvec": [ 2024-09-15T23:12:12.520 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.520 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v2", 2024-09-15T23:12:12.520 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "172.21.15.111:0", 2024-09-15T23:12:12.520 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 1734278339 2024-09-15T23:12:12.520 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.520 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-15T23:12:12.521 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.521 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.521 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "libcephsqlite", 2024-09-15T23:12:12.521 INFO:teuthology.orchestra.run.smithi111.stdout: "addrvec": [ 2024-09-15T23:12:12.521 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.521 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v2", 2024-09-15T23:12:12.521 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "172.21.15.111:0", 2024-09-15T23:12:12.521 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 1342823489 2024-09-15T23:12:12.521 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.521 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-15T23:12:12.521 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.521 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.522 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "rbd_support", 2024-09-15T23:12:12.522 INFO:teuthology.orchestra.run.smithi111.stdout: "addrvec": [ 2024-09-15T23:12:12.522 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.522 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v2", 2024-09-15T23:12:12.522 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "172.21.15.111:0", 2024-09-15T23:12:12.522 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 738939510 2024-09-15T23:12:12.522 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.522 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-15T23:12:12.522 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:12.522 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.522 INFO:teuthology.orchestra.run.smithi111.stdout: "name": "volumes", 2024-09-15T23:12:12.522 INFO:teuthology.orchestra.run.smithi111.stdout: "addrvec": [ 2024-09-15T23:12:12.523 INFO:teuthology.orchestra.run.smithi111.stdout: { 2024-09-15T23:12:12.523 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v2", 2024-09-15T23:12:12.523 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "172.21.15.111:0", 2024-09-15T23:12:12.523 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 3843671844 2024-09-15T23:12:12.523 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.523 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-15T23:12:12.523 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:12.523 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2024-09-15T23:12:12.523 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-15T23:12:12.530 DEBUG:tasks.check_counter:Getting stats from a 2024-09-15T23:12:12.531 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-15T23:12:12.531 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:11 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/3009045566' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-15T23:12:12.531 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:11 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/1441430800' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-15T23:12:12.531 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:12 smithi111 ceph-mon[30629]: pgmap v1139: 97 pgs: 97 active+clean; 417 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 5.3 MiB/s wr, 127 op/s 2024-09-15T23:12:12.532 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:11 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/3009045566' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-15T23:12:12.533 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:11 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/1441430800' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-15T23:12:12.533 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:12 smithi121 ceph-mon[34815]: pgmap v1139: 97 pgs: 97 active+clean; 417 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 5.3 MiB/s wr, 127 op/s 2024-09-15T23:12:12.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:12 smithi192 ceph-mon[35427]: pgmap v1139: 97 pgs: 97 active+clean; 417 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 5.3 MiB/s wr, 127 op/s 2024-09-15T23:12:13.011 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:12:13.930 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-15T23:12:13.931 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T23:12:13.931 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 4, 2024-09-15T23:12:13.931 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 4, 2024-09-15T23:12:13.931 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 3024, 2024-09-15T23:12:13.931 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 448, 2024-09-15T23:12:13.931 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 4, 2024-09-15T23:12:13.931 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 0, 2024-09-15T23:12:13.931 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.006084922, 2024-09-15T23:12:13.931 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.000073859, 2024-09-15T23:12:13.931 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.003157619, 2024-09-15T23:12:13.932 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T23:12:13.932 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:13.932 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 3, 2024-09-15T23:12:13.932 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.005539062, 2024-09-15T23:12:13.932 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001846354 2024-09-15T23:12:13.932 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:13.932 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:13.932 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-15T23:12:13.932 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-15T23:12:13.932 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-15T23:12:13.932 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:13.932 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-15T23:12:13.933 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 448 2024-09-15T23:12:13.933 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:13.933 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T23:12:13.933 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 452, 2024-09-15T23:12:13.933 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 447, 2024-09-15T23:12:13.933 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 823168, 2024-09-15T23:12:13.933 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 277232, 2024-09-15T23:12:13.933 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 2, 2024-09-15T23:12:13.933 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 1, 2024-09-15T23:12:13.933 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.076022986, 2024-09-15T23:12:13.933 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.042071289, 2024-09-15T23:12:13.933 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.041729532, 2024-09-15T23:12:13.933 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T23:12:13.934 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:13.934 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 446, 2024-09-15T23:12:13.934 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.026311336, 2024-09-15T23:12:13.934 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000058994 2024-09-15T23:12:13.934 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:13.934 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:13.934 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-15T23:12:13.934 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-15T23:12:13.934 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-15T23:12:13.934 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:13.934 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 823168, 2024-09-15T23:12:13.934 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 277232 2024-09-15T23:12:13.934 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:13.935 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T23:12:13.935 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 2, 2024-09-15T23:12:13.935 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 343, 2024-09-15T23:12:13.935 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 320, 2024-09-15T23:12:13.935 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 217808, 2024-09-15T23:12:13.935 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 4, 2024-09-15T23:12:13.935 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 2, 2024-09-15T23:12:13.935 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.031345870, 2024-09-15T23:12:13.935 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.025416926, 2024-09-15T23:12:13.935 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.002226355, 2024-09-15T23:12:13.935 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T23:12:13.935 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:13.935 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 341, 2024-09-15T23:12:13.936 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.020350679, 2024-09-15T23:12:13.936 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000059679 2024-09-15T23:12:13.936 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:13.936 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:13.936 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-15T23:12:13.936 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-15T23:12:13.936 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-15T23:12:13.936 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:13.936 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-15T23:12:13.936 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 217808 2024-09-15T23:12:13.936 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:13.936 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-09-15T23:12:13.936 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 0, 2024-09-15T23:12:13.937 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-09-15T23:12:13.937 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:13.937 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-09-15T23:12:13.937 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-09-15T23:12:13.937 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-09-15T23:12:13.937 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T23:12:13.937 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-09-15T23:12:13.937 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T23:12:13.937 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-09-15T23:12:13.937 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T23:12:13.937 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T23:12:13.937 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T23:12:13.937 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T23:12:13.938 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T23:12:13.938 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-09-15T23:12:13.938 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T23:12:13.938 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T23:12:13.938 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-09-15T23:12:13.938 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-09-15T23:12:13.938 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-09-15T23:12:13.938 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-09-15T23:12:13.938 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T23:12:13.938 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T23:12:13.938 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T23:12:13.938 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T23:12:13.938 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T23:12:13.938 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-09-15T23:12:13.938 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-09-15T23:12:13.939 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-09-15T23:12:13.939 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T23:12:13.939 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T23:12:13.939 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-09-15T23:12:13.939 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-09-15T23:12:13.939 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-09-15T23:12:13.939 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-09-15T23:12:13.939 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T23:12:13.939 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-09-15T23:12:13.939 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T23:12:13.939 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-09-15T23:12:13.939 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 148704, 2024-09-15T23:12:13.939 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 29, 2024-09-15T23:12:13.940 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2024-09-15T23:12:13.940 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2024-09-15T23:12:13.940 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-09-15T23:12:13.940 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-09-15T23:12:13.940 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-09-15T23:12:13.940 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-09-15T23:12:13.940 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-09-15T23:12:13.940 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-09-15T23:12:13.940 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 0, 2024-09-15T23:12:13.940 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 0, 2024-09-15T23:12:13.940 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T23:12:13.940 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-09-15T23:12:13.940 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-09-15T23:12:13.941 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-09-15T23:12:13.941 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 928, 2024-09-15T23:12:13.941 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 2, 2024-09-15T23:12:13.941 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-09-15T23:12:13.941 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-09-15T23:12:13.941 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-09-15T23:12:13.941 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-09-15T23:12:13.941 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:13.941 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T23:12:13.941 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-15T23:12:13.941 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-09-15T23:12:13.941 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-15T23:12:13.941 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 454, 2024-09-15T23:12:13.941 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 759243, 2024-09-15T23:12:13.942 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:13.942 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 454, 2024-09-15T23:12:13.942 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-15T23:12:13.942 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-15T23:12:13.942 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 454, 2024-09-15T23:12:13.942 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 759243, 2024-09-15T23:12:13.942 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-15T23:12:13.942 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-15T23:12:13.942 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-15T23:12:13.942 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-15T23:12:13.942 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:13.942 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:13.942 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-15T23:12:14.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:14 smithi121 ceph-mon[34815]: pgmap v1140: 97 pgs: 97 active+clean; 405 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 3.4 MiB/s wr, 102 op/s 2024-09-15T23:12:14.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:14 smithi111 ceph-mon[30629]: pgmap v1140: 97 pgs: 97 active+clean; 405 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 3.4 MiB/s wr, 102 op/s 2024-09-15T23:12:14.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:14 smithi192 ceph-mon[35427]: pgmap v1140: 97 pgs: 97 active+clean; 405 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 3.4 MiB/s wr, 102 op/s 2024-09-15T23:12:14.990 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-15T23:12:14.990 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-15T23:12:14.990 DEBUG:tasks.check_counter:Getting stats from d 2024-09-15T23:12:14.990 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-15T23:12:15.358 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:12:16.050 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-15T23:12:16.051 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T23:12:16.051 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 4, 2024-09-15T23:12:16.051 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 4, 2024-09-15T23:12:16.051 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 3024, 2024-09-15T23:12:16.051 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 448, 2024-09-15T23:12:16.051 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 4, 2024-09-15T23:12:16.051 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 0, 2024-09-15T23:12:16.051 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.005787506, 2024-09-15T23:12:16.051 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.000044411, 2024-09-15T23:12:16.051 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.003149662, 2024-09-15T23:12:16.051 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T23:12:16.051 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:16.052 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 3, 2024-09-15T23:12:16.052 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.006307664, 2024-09-15T23:12:16.052 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.002102554 2024-09-15T23:12:16.052 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:16.052 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:16.052 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-15T23:12:16.052 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-15T23:12:16.052 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-15T23:12:16.052 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:16.052 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-15T23:12:16.052 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 448 2024-09-15T23:12:16.052 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:16.053 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T23:12:16.053 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 2, 2024-09-15T23:12:16.053 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 347, 2024-09-15T23:12:16.053 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 320, 2024-09-15T23:12:16.053 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 218512, 2024-09-15T23:12:16.053 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 4, 2024-09-15T23:12:16.053 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 2, 2024-09-15T23:12:16.053 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.031809152, 2024-09-15T23:12:16.053 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.025390401, 2024-09-15T23:12:16.053 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.002585660, 2024-09-15T23:12:16.053 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T23:12:16.053 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:16.053 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 345, 2024-09-15T23:12:16.054 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.020333590, 2024-09-15T23:12:16.054 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000058937 2024-09-15T23:12:16.054 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:16.054 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:16.054 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-15T23:12:16.054 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-15T23:12:16.054 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-15T23:12:16.054 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:16.054 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-15T23:12:16.054 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 218512 2024-09-15T23:12:16.054 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:16.054 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T23:12:16.054 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 459, 2024-09-15T23:12:16.055 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 454, 2024-09-15T23:12:16.055 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 825136, 2024-09-15T23:12:16.055 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 281232, 2024-09-15T23:12:16.055 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 2, 2024-09-15T23:12:16.055 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 1, 2024-09-15T23:12:16.055 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.071243779, 2024-09-15T23:12:16.055 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.038627319, 2024-09-15T23:12:16.055 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.038514597, 2024-09-15T23:12:16.055 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T23:12:16.055 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:16.055 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 453, 2024-09-15T23:12:16.055 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.026999873, 2024-09-15T23:12:16.055 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000059602 2024-09-15T23:12:16.055 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:16.056 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:16.056 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-15T23:12:16.056 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-15T23:12:16.056 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-15T23:12:16.056 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:16.056 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 825136, 2024-09-15T23:12:16.056 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 281232 2024-09-15T23:12:16.056 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:16.056 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-09-15T23:12:16.056 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 0, 2024-09-15T23:12:16.056 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-09-15T23:12:16.056 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:16.056 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-09-15T23:12:16.057 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-09-15T23:12:16.057 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-09-15T23:12:16.057 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T23:12:16.057 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-09-15T23:12:16.057 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T23:12:16.057 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-09-15T23:12:16.057 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T23:12:16.057 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T23:12:16.057 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T23:12:16.057 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T23:12:16.057 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T23:12:16.057 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-09-15T23:12:16.057 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T23:12:16.057 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T23:12:16.058 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-09-15T23:12:16.058 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-09-15T23:12:16.058 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-09-15T23:12:16.058 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-09-15T23:12:16.058 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T23:12:16.058 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T23:12:16.058 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T23:12:16.058 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T23:12:16.058 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T23:12:16.058 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-09-15T23:12:16.058 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-09-15T23:12:16.058 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-09-15T23:12:16.058 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T23:12:16.059 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T23:12:16.059 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-09-15T23:12:16.059 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-09-15T23:12:16.059 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-09-15T23:12:16.059 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-09-15T23:12:16.059 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T23:12:16.059 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-09-15T23:12:16.059 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T23:12:16.059 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-09-15T23:12:16.059 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 148682, 2024-09-15T23:12:16.059 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 29, 2024-09-15T23:12:16.059 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2024-09-15T23:12:16.059 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2024-09-15T23:12:16.060 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-09-15T23:12:16.060 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-09-15T23:12:16.060 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-09-15T23:12:16.060 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-09-15T23:12:16.060 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-09-15T23:12:16.060 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-09-15T23:12:16.060 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 0, 2024-09-15T23:12:16.060 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 0, 2024-09-15T23:12:16.060 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T23:12:16.060 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-09-15T23:12:16.060 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-09-15T23:12:16.060 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-09-15T23:12:16.060 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 928, 2024-09-15T23:12:16.060 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 2, 2024-09-15T23:12:16.061 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-09-15T23:12:16.061 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-09-15T23:12:16.061 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-09-15T23:12:16.061 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-09-15T23:12:16.061 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:16.061 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T23:12:16.061 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-15T23:12:16.061 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-09-15T23:12:16.061 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-15T23:12:16.061 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 461, 2024-09-15T23:12:16.061 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 760217, 2024-09-15T23:12:16.061 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:16.061 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 461, 2024-09-15T23:12:16.062 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-15T23:12:16.062 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-15T23:12:16.062 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 461, 2024-09-15T23:12:16.062 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 760217, 2024-09-15T23:12:16.062 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-15T23:12:16.062 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-15T23:12:16.062 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-15T23:12:16.062 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-15T23:12:16.062 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:16.062 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:16.062 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-15T23:12:16.408 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:16 smithi111 ceph-mon[30629]: pgmap v1141: 97 pgs: 97 active+clean; 393 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.8 MiB/s wr, 94 op/s 2024-09-15T23:12:16.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:16 smithi192 ceph-mon[35427]: pgmap v1141: 97 pgs: 97 active+clean; 393 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.8 MiB/s wr, 94 op/s 2024-09-15T23:12:16.639 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:16 smithi121 ceph-mon[34815]: pgmap v1141: 97 pgs: 97 active+clean; 393 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.8 MiB/s wr, 94 op/s 2024-09-15T23:12:16.809 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-15T23:12:16.809 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-15T23:12:16.809 DEBUG:tasks.check_counter:Getting stats from g 2024-09-15T23:12:16.809 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-15T23:12:17.178 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:12:17.817 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-15T23:12:17.818 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T23:12:17.818 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 454, 2024-09-15T23:12:17.818 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 449, 2024-09-15T23:12:17.818 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 823712, 2024-09-15T23:12:17.818 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 278512, 2024-09-15T23:12:17.818 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 4, 2024-09-15T23:12:17.818 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 0, 2024-09-15T23:12:17.818 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.073461060, 2024-09-15T23:12:17.818 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.039160195, 2024-09-15T23:12:17.818 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.038036047, 2024-09-15T23:12:17.818 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T23:12:17.818 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:17.819 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 448, 2024-09-15T23:12:17.819 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.026925411, 2024-09-15T23:12:17.819 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000060101 2024-09-15T23:12:17.819 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:17.819 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:17.819 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-15T23:12:17.819 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-15T23:12:17.819 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-15T23:12:17.819 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:17.819 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 823712, 2024-09-15T23:12:17.819 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 278512 2024-09-15T23:12:17.819 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:17.819 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T23:12:17.820 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 1, 2024-09-15T23:12:17.820 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 343, 2024-09-15T23:12:17.820 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 160, 2024-09-15T23:12:17.820 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 138912, 2024-09-15T23:12:17.820 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 3, 2024-09-15T23:12:17.820 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 2, 2024-09-15T23:12:17.820 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.030877580, 2024-09-15T23:12:17.820 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.025775576, 2024-09-15T23:12:17.820 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.001372222, 2024-09-15T23:12:17.820 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T23:12:17.820 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:17.820 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 341, 2024-09-15T23:12:17.820 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.022160936, 2024-09-15T23:12:17.821 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000064988 2024-09-15T23:12:17.821 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:17.821 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:17.821 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-15T23:12:17.821 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-15T23:12:17.821 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-15T23:12:17.821 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:17.821 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-15T23:12:17.821 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 138912 2024-09-15T23:12:17.821 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:17.821 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T23:12:17.821 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 4, 2024-09-15T23:12:17.821 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 4, 2024-09-15T23:12:17.822 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 3024, 2024-09-15T23:12:17.822 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 448, 2024-09-15T23:12:17.822 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 2, 2024-09-15T23:12:17.822 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 1, 2024-09-15T23:12:17.822 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.003348020, 2024-09-15T23:12:17.822 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.000071206, 2024-09-15T23:12:17.822 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.003205124, 2024-09-15T23:12:17.822 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T23:12:17.822 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:17.822 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 3, 2024-09-15T23:12:17.822 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.005833708, 2024-09-15T23:12:17.822 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001944569 2024-09-15T23:12:17.822 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:17.822 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:17.823 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-15T23:12:17.823 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-15T23:12:17.823 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-15T23:12:17.823 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:17.823 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-15T23:12:17.823 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 448 2024-09-15T23:12:17.823 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:17.823 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-09-15T23:12:17.823 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 0, 2024-09-15T23:12:17.823 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-09-15T23:12:17.823 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:17.823 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-09-15T23:12:17.823 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-09-15T23:12:17.823 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-09-15T23:12:17.824 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T23:12:17.824 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-09-15T23:12:17.824 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T23:12:17.824 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-09-15T23:12:17.824 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T23:12:17.824 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T23:12:17.824 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T23:12:17.824 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T23:12:17.824 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T23:12:17.824 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-09-15T23:12:17.824 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T23:12:17.824 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T23:12:17.824 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-09-15T23:12:17.825 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-09-15T23:12:17.825 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-09-15T23:12:17.825 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-09-15T23:12:17.825 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T23:12:17.825 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T23:12:17.825 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T23:12:17.825 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T23:12:17.825 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T23:12:17.825 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-09-15T23:12:17.825 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-09-15T23:12:17.825 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-09-15T23:12:17.825 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T23:12:17.825 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T23:12:17.825 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-09-15T23:12:17.825 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-09-15T23:12:17.826 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-09-15T23:12:17.826 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-09-15T23:12:17.826 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T23:12:17.826 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-09-15T23:12:17.826 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T23:12:17.826 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-09-15T23:12:17.826 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 148984, 2024-09-15T23:12:17.826 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 30, 2024-09-15T23:12:17.826 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2024-09-15T23:12:17.826 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2024-09-15T23:12:17.826 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-09-15T23:12:17.826 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-09-15T23:12:17.826 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-09-15T23:12:17.827 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-09-15T23:12:17.827 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-09-15T23:12:17.827 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-09-15T23:12:17.827 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 0, 2024-09-15T23:12:17.827 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 0, 2024-09-15T23:12:17.827 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T23:12:17.827 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-09-15T23:12:17.827 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-09-15T23:12:17.827 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-09-15T23:12:17.827 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 928, 2024-09-15T23:12:17.827 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 2, 2024-09-15T23:12:17.827 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-09-15T23:12:17.827 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-09-15T23:12:17.827 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-09-15T23:12:17.827 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-09-15T23:12:17.828 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:17.828 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T23:12:17.828 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-15T23:12:17.828 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-09-15T23:12:17.828 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-15T23:12:17.828 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 455, 2024-09-15T23:12:17.828 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 759476, 2024-09-15T23:12:17.828 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:17.828 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 455, 2024-09-15T23:12:17.828 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-15T23:12:17.828 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-15T23:12:17.828 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 455, 2024-09-15T23:12:17.828 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 759476, 2024-09-15T23:12:17.828 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-15T23:12:17.829 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-15T23:12:17.829 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-15T23:12:17.829 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-15T23:12:17.829 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:17.829 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:17.829 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-15T23:12:18.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:18 smithi111 ceph-mon[30629]: pgmap v1142: 97 pgs: 97 active+clean; 381 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 1.3 MiB/s wr, 97 op/s 2024-09-15T23:12:18.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:18 smithi192 ceph-mon[35427]: pgmap v1142: 97 pgs: 97 active+clean; 381 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 1.3 MiB/s wr, 97 op/s 2024-09-15T23:12:18.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:18 smithi121 ceph-mon[34815]: pgmap v1142: 97 pgs: 97 active+clean; 381 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 1.3 MiB/s wr, 97 op/s 2024-09-15T23:12:18.738 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-15T23:12:18.738 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-15T23:12:18.738 DEBUG:tasks.check_counter:Getting stats from j 2024-09-15T23:12:18.739 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-15T23:12:19.103 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:12:19.522 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:19 smithi111 ceph-mon[30629]: pgmap v1143: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 866 KiB/s wr, 84 op/s 2024-09-15T23:12:19.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:19 smithi192 ceph-mon[35427]: pgmap v1143: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 866 KiB/s wr, 84 op/s 2024-09-15T23:12:19.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:19 smithi121 ceph-mon[34815]: pgmap v1143: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 866 KiB/s wr, 84 op/s 2024-09-15T23:12:19.829 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2024-09-15T23:12:19.829 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T23:12:19.829 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 0, 2024-09-15T23:12:19.830 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 2, 2024-09-15T23:12:19.830 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 0, 2024-09-15T23:12:19.830 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 0, 2024-09-15T23:12:19.830 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 4, 2024-09-15T23:12:19.830 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 0, 2024-09-15T23:12:19.830 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.003196711, 2024-09-15T23:12:19.830 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.000000000, 2024-09-15T23:12:19.830 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.000491688, 2024-09-15T23:12:19.830 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T23:12:19.830 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:19.830 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-15T23:12:19.830 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-15T23:12:19.830 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-15T23:12:19.831 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:19.831 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:19.831 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-15T23:12:19.831 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-15T23:12:19.831 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-15T23:12:19.831 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:19.831 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-15T23:12:19.831 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 0 2024-09-15T23:12:19.831 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:19.831 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T23:12:19.832 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 465, 2024-09-15T23:12:19.832 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 458, 2024-09-15T23:12:19.832 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 828704, 2024-09-15T23:12:19.832 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 282960, 2024-09-15T23:12:19.832 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 2, 2024-09-15T23:12:19.832 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 1, 2024-09-15T23:12:19.832 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.075178629, 2024-09-15T23:12:19.832 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.038022162, 2024-09-15T23:12:19.832 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.044995292, 2024-09-15T23:12:19.832 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T23:12:19.832 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:19.832 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 458, 2024-09-15T23:12:19.833 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.034412892, 2024-09-15T23:12:19.833 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000075137 2024-09-15T23:12:19.833 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:19.833 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:19.833 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-15T23:12:19.833 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-15T23:12:19.833 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-15T23:12:19.833 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:19.833 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 828704, 2024-09-15T23:12:19.833 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 282960 2024-09-15T23:12:19.833 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:19.833 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T23:12:19.833 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 2, 2024-09-15T23:12:19.833 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 349, 2024-09-15T23:12:19.834 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 320, 2024-09-15T23:12:19.834 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 218864, 2024-09-15T23:12:19.834 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 4, 2024-09-15T23:12:19.834 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 2, 2024-09-15T23:12:19.834 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.032295123, 2024-09-15T23:12:19.834 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.025960130, 2024-09-15T23:12:19.834 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.002514172, 2024-09-15T23:12:19.834 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T23:12:19.834 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:19.834 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 347, 2024-09-15T23:12:19.834 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.020779924, 2024-09-15T23:12:19.834 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000059884 2024-09-15T23:12:19.834 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:19.834 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:19.835 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-15T23:12:19.835 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-15T23:12:19.835 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-15T23:12:19.835 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:19.835 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-15T23:12:19.835 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 218864 2024-09-15T23:12:19.835 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:19.835 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2024-09-15T23:12:19.835 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 0, 2024-09-15T23:12:19.835 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2024-09-15T23:12:19.835 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:19.835 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2024-09-15T23:12:19.835 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2024-09-15T23:12:19.835 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2024-09-15T23:12:19.836 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T23:12:19.836 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2024-09-15T23:12:19.836 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T23:12:19.836 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2024-09-15T23:12:19.836 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T23:12:19.836 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T23:12:19.836 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T23:12:19.836 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T23:12:19.836 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T23:12:19.836 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2024-09-15T23:12:19.836 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T23:12:19.836 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T23:12:19.836 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2024-09-15T23:12:19.836 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2024-09-15T23:12:19.837 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2024-09-15T23:12:19.837 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2024-09-15T23:12:19.837 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T23:12:19.837 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T23:12:19.837 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T23:12:19.837 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T23:12:19.837 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T23:12:19.837 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2024-09-15T23:12:19.837 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2024-09-15T23:12:19.837 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2024-09-15T23:12:19.837 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T23:12:19.837 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T23:12:19.837 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2024-09-15T23:12:19.837 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2024-09-15T23:12:19.838 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2024-09-15T23:12:19.838 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2024-09-15T23:12:19.838 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T23:12:19.838 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2024-09-15T23:12:19.838 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T23:12:19.838 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2024-09-15T23:12:19.838 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 149735, 2024-09-15T23:12:19.838 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 30, 2024-09-15T23:12:19.838 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2024-09-15T23:12:19.838 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2024-09-15T23:12:19.838 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2024-09-15T23:12:19.838 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2024-09-15T23:12:19.838 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2024-09-15T23:12:19.838 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2024-09-15T23:12:19.839 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2024-09-15T23:12:19.839 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2024-09-15T23:12:19.839 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 0, 2024-09-15T23:12:19.839 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 0, 2024-09-15T23:12:19.839 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T23:12:19.839 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2024-09-15T23:12:19.839 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2024-09-15T23:12:19.839 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2024-09-15T23:12:19.839 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 928, 2024-09-15T23:12:19.839 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 2, 2024-09-15T23:12:19.839 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2024-09-15T23:12:19.839 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2024-09-15T23:12:19.839 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2024-09-15T23:12:19.839 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2024-09-15T23:12:19.840 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2024-09-15T23:12:19.840 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T23:12:19.840 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2024-09-15T23:12:19.840 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2024-09-15T23:12:19.840 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2024-09-15T23:12:19.840 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 463, 2024-09-15T23:12:19.840 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 760477, 2024-09-15T23:12:19.840 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:19.840 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 463, 2024-09-15T23:12:19.840 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2024-09-15T23:12:19.840 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2024-09-15T23:12:19.840 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 463, 2024-09-15T23:12:19.840 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 760477, 2024-09-15T23:12:19.840 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2024-09-15T23:12:19.840 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2024-09-15T23:12:19.841 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2024-09-15T23:12:19.841 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2024-09-15T23:12:19.841 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:19.841 INFO:teuthology.orchestra.run.smithi111.stdout: } 2024-09-15T23:12:19.841 INFO:teuthology.orchestra.run.smithi111.stdout:} 2024-09-15T23:12:20.627 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-15T23:12:20.627 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-15T23:12:20.627 DEBUG:tasks.check_counter:Getting stats from b 2024-09-15T23:12:20.628 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-15T23:12:20.948 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T23:12:21.572 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-15T23:12:21.572 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T23:12:21.572 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 257203, 2024-09-15T23:12:21.573 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 378199, 2024-09-15T23:12:21.573 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 80415295, 2024-09-15T23:12:21.573 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 388132596, 2024-09-15T23:12:21.573 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 233, 2024-09-15T23:12:21.573 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 9, 2024-09-15T23:12:21.573 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 13.892992108, 2024-09-15T23:12:21.573 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 6.564219369, 2024-09-15T23:12:21.573 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 7.913362967, 2024-09-15T23:12:21.573 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 1.220035775, 2024-09-15T23:12:21.573 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:21.573 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 378195, 2024-09-15T23:12:21.573 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 23.681374110, 2024-09-15T23:12:21.574 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000062616 2024-09-15T23:12:21.574 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.574 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:21.574 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 285129, 2024-09-15T23:12:21.574 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.079364159, 2024-09-15T23:12:21.574 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000278 2024-09-15T23:12:21.574 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.574 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 1006416, 2024-09-15T23:12:21.574 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 317376 2024-09-15T23:12:21.574 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.574 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T23:12:21.574 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 228834, 2024-09-15T23:12:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 157271, 2024-09-15T23:12:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 62867717, 2024-09-15T23:12:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 117250509, 2024-09-15T23:12:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 20, 2024-09-15T23:12:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 5, 2024-09-15T23:12:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 11.683234745, 2024-09-15T23:12:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 4.906634355, 2024-09-15T23:12:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 7.547915582, 2024-09-15T23:12:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.939758632, 2024-09-15T23:12:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 157272, 2024-09-15T23:12:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 14.044198389, 2024-09-15T23:12:21.575 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000089298 2024-09-15T23:12:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 325752, 2024-09-15T23:12:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.044513430, 2024-09-15T23:12:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000136 2024-09-15T23:12:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-15T23:12:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 698192 2024-09-15T23:12:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T23:12:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 42626, 2024-09-15T23:12:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 48574, 2024-09-15T23:12:21.576 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 10734561, 2024-09-15T23:12:21.577 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 102962577, 2024-09-15T23:12:21.577 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 8, 2024-09-15T23:12:21.577 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 6, 2024-09-15T23:12:21.577 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 3.676637228, 2024-09-15T23:12:21.577 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 1.431855943, 2024-09-15T23:12:21.577 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 1.093273004, 2024-09-15T23:12:21.577 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 1.082186365, 2024-09-15T23:12:21.577 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:21.577 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 48573, 2024-09-15T23:12:21.577 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 16.034413385, 2024-09-15T23:12:21.577 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000330109 2024-09-15T23:12:21.577 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.577 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:21.577 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 36016, 2024-09-15T23:12:21.578 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.006404506, 2024-09-15T23:12:21.578 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000177 2024-09-15T23:12:21.578 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.578 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-15T23:12:21.578 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 448 2024-09-15T23:12:21.578 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.578 INFO:teuthology.orchestra.run.smithi121.stdout: "cct": { 2024-09-15T23:12:21.578 INFO:teuthology.orchestra.run.smithi121.stdout: "total_workers": 1, 2024-09-15T23:12:21.578 INFO:teuthology.orchestra.run.smithi121.stdout: "unhealthy_workers": 0 2024-09-15T23:12:21.578 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.578 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-MDSRank": { 2024-09-15T23:12:21.578 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2024-09-15T23:12:21.578 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2024-09-15T23:12:21.579 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 75656, 2024-09-15T23:12:21.579 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 88.736306250, 2024-09-15T23:12:21.579 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.001172891 2024-09-15T23:12:21.579 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:21.579 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.579 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-PurgeQueue": { 2024-09-15T23:12:21.579 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2024-09-15T23:12:21.579 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2024-09-15T23:12:21.579 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 42212, 2024-09-15T23:12:21.579 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 3.476722439, 2024-09-15T23:12:21.579 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000082363 2024-09-15T23:12:21.579 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:21.580 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.580 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": { 2024-09-15T23:12:21.580 INFO:teuthology.orchestra.run.smithi121.stdout: "request": 163162, 2024-09-15T23:12:21.580 INFO:teuthology.orchestra.run.smithi121.stdout: "reply": 135412, 2024-09-15T23:12:21.580 INFO:teuthology.orchestra.run.smithi121.stdout: "reply_latency": { 2024-09-15T23:12:21.580 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 135412, 2024-09-15T23:12:21.580 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 604.976417163, 2024-09-15T23:12:21.580 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.004467672 2024-09-15T23:12:21.580 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.580 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_reply": 0, 2024-09-15T23:12:21.580 INFO:teuthology.orchestra.run.smithi121.stdout: "forward": 27750, 2024-09-15T23:12:21.581 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_complete": 0, 2024-09-15T23:12:21.581 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_keys": 0, 2024-09-15T23:12:21.581 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_commit": 3073, 2024-09-15T23:12:21.581 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_split": 22, 2024-09-15T23:12:21.581 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_merge": 22, 2024-09-15T23:12:21.581 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes": 422, 2024-09-15T23:12:21.581 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_top": 84, 2024-09-15T23:12:21.581 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_bottom": 42, 2024-09-15T23:12:21.581 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pin_tail": 296, 2024-09-15T23:12:21.581 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pinned": 301, 2024-09-15T23:12:21.581 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_expired": 237, 2024-09-15T23:12:21.582 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_with_caps": 2, 2024-09-15T23:12:21.582 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": 2, 2024-09-15T23:12:21.582 INFO:teuthology.orchestra.run.smithi121.stdout: "subtrees": 4, 2024-09-15T23:12:21.582 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse": 231261, 2024-09-15T23:12:21.582 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_hit": 125671, 2024-09-15T23:12:21.582 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_forward": 27748, 2024-09-15T23:12:21.582 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_discover": 18, 2024-09-15T23:12:21.582 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_dir_fetch": 0, 2024-09-15T23:12:21.582 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_remote_ino": 0, 2024-09-15T23:12:21.582 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_lock": 1646, 2024-09-15T23:12:21.582 INFO:teuthology.orchestra.run.smithi121.stdout: "load_cent": 0, 2024-09-15T23:12:21.582 INFO:teuthology.orchestra.run.smithi121.stdout: "q": 0, 2024-09-15T23:12:21.582 INFO:teuthology.orchestra.run.smithi121.stdout: "exported": 6, 2024-09-15T23:12:21.582 INFO:teuthology.orchestra.run.smithi121.stdout: "exported_inodes": 155, 2024-09-15T23:12:21.583 INFO:teuthology.orchestra.run.smithi121.stdout: "imported": 7, 2024-09-15T23:12:21.583 INFO:teuthology.orchestra.run.smithi121.stdout: "imported_inodes": 203, 2024-09-15T23:12:21.583 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_dir_fetch": 0, 2024-09-15T23:12:21.583 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_backtrace_fetch": 0, 2024-09-15T23:12:21.583 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_peer_discover": 0, 2024-09-15T23:12:21.583 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rfiles": 4, 2024-09-15T23:12:21.583 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rbytes": 234, 2024-09-15T23:12:21.583 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rsnaps": 0, 2024-09-15T23:12:21.583 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_fetch": 0, 2024-09-15T23:12:21.583 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_set_tag": 0, 2024-09-15T23:12:21.583 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T23:12:21.583 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_inotable_repaired": 0, 2024-09-15T23:12:21.583 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_inodes": 0, 2024-09-15T23:12:21.583 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_base_inodes": 0, 2024-09-15T23:12:21.584 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dirfrag_rstats": 0, 2024-09-15T23:12:21.584 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_file_inodes": 0, 2024-09-15T23:12:21.584 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_inode_file_caps": 8, 2024-09-15T23:12:21.584 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_revoke": 6644, 2024-09-15T23:12:21.584 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_grant": 99978, 2024-09-15T23:12:21.584 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T23:12:21.584 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T23:12:21.584 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T23:12:21.584 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps": 51019, 2024-09-15T23:12:21.584 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps_dirty": 21601, 2024-09-15T23:12:21.584 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_cap_release": 5732, 2024-09-15T23:12:21.584 INFO:teuthology.orchestra.run.smithi121.stdout: "process_request_cap_release": 86818 2024-09-15T23:12:21.584 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_cache": { 2024-09-15T23:12:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update": 6, 2024-09-15T23:12:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update_receipt": 35, 2024-09-15T23:12:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_try_discover": 24, 2024-09-15T23:12:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_send_discover": 32, 2024-09-15T23:12:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_handle_discover": 18, 2024-09-15T23:12:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays": 1, 2024-09-15T23:12:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_delayed": 0, 2024-09-15T23:12:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_enqueuing": 0, 2024-09-15T23:12:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_created": 24510, 2024-09-15T23:12:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_enqueued": 24509, 2024-09-15T23:12:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_reintegrated": 0, 2024-09-15T23:12:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_migrated": 0, 2024-09-15T23:12:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_processing": 0, 2024-09-15T23:12:21.586 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_enqueued": 0, 2024-09-15T23:12:21.586 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_prioritized": 0, 2024-09-15T23:12:21.586 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_started": 0, 2024-09-15T23:12:21.586 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_completed": 0, 2024-09-15T23:12:21.586 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_quiesce_path": 0, 2024-09-15T23:12:21.586 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_quiesce_inode": 0, 2024-09-15T23:12:21.586 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_enqueue_scrub": 0, 2024-09-15T23:12:21.586 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_exportdir": 6, 2024-09-15T23:12:21.586 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_flush": 0, 2024-09-15T23:12:21.586 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragmentdir": 44, 2024-09-15T23:12:21.586 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragstats": 0, 2024-09-15T23:12:21.586 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_inodestats": 0 2024-09-15T23:12:21.587 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.587 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_log": { 2024-09-15T23:12:21.587 INFO:teuthology.orchestra.run.smithi121.stdout: "evlrg": 0, 2024-09-15T23:12:21.587 INFO:teuthology.orchestra.run.smithi121.stdout: "evadd": 126005, 2024-09-15T23:12:21.587 INFO:teuthology.orchestra.run.smithi121.stdout: "evex": 125428, 2024-09-15T23:12:21.587 INFO:teuthology.orchestra.run.smithi121.stdout: "evtrm": 125428, 2024-09-15T23:12:21.587 INFO:teuthology.orchestra.run.smithi121.stdout: "ev": 577, 2024-09-15T23:12:21.587 INFO:teuthology.orchestra.run.smithi121.stdout: "evexg": 0, 2024-09-15T23:12:21.587 INFO:teuthology.orchestra.run.smithi121.stdout: "evexd": 811, 2024-09-15T23:12:21.587 INFO:teuthology.orchestra.run.smithi121.stdout: "segadd": 233, 2024-09-15T23:12:21.587 INFO:teuthology.orchestra.run.smithi121.stdout: "segex": 232, 2024-09-15T23:12:21.587 INFO:teuthology.orchestra.run.smithi121.stdout: "segtrm": 232, 2024-09-15T23:12:21.587 INFO:teuthology.orchestra.run.smithi121.stdout: "seg": 1, 2024-09-15T23:12:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: "segmjr": 1, 2024-09-15T23:12:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: "segexg": 0, 2024-09-15T23:12:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: "segexd": 1, 2024-09-15T23:12:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: "expos": 258726535, 2024-09-15T23:12:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: "wrpos": 259506318, 2024-09-15T23:12:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: "rdpos": 0, 2024-09-15T23:12:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: "jlat": { 2024-09-15T23:12:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 39373, 2024-09-15T23:12:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 208.910401774, 2024-09-15T23:12:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.005305930 2024-09-15T23:12:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: "replayed": 0 2024-09-15T23:12:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.588 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_mem": { 2024-09-15T23:12:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: "ino": 23, 2024-09-15T23:12:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: "ino+": 24668, 2024-09-15T23:12:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: "ino-": 24645, 2024-09-15T23:12:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: "dir": 61, 2024-09-15T23:12:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: "dir+": 296, 2024-09-15T23:12:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: "dir-": 235, 2024-09-15T23:12:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: "dn": 422, 2024-09-15T23:12:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: "dn+": 50150, 2024-09-15T23:12:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: "dn-": 49728, 2024-09-15T23:12:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: "cap": 2, 2024-09-15T23:12:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: "cap+": 27701, 2024-09-15T23:12:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: "cap-": 27699, 2024-09-15T23:12:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: "rss": 79356, 2024-09-15T23:12:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: "heap": 231712 2024-09-15T23:12:21.589 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_server": { 2024-09-15T23:12:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_client_request": 211399, 2024-09-15T23:12:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_server_request": 5286, 2024-09-15T23:12:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_request": 163162, 2024-09-15T23:12:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_session": 3764, 2024-09-15T23:12:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_peer_request": 14095, 2024-09-15T23:12:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: "req_create_latency": { 2024-09-15T23:12:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 24464, 2024-09-15T23:12:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 162.732324572, 2024-09-15T23:12:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.006651909 2024-09-15T23:12:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getattr_latency": { 2024-09-15T23:12:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 6289, 2024-09-15T23:12:21.590 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 88.412813249, 2024-09-15T23:12:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.014058326 2024-09-15T23:12:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getfilelock_latency": { 2024-09-15T23:12:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: "req_link_latency": { 2024-09-15T23:12:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookup_latency": { 2024-09-15T23:12:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 36582, 2024-09-15T23:12:21.591 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 34.649116623, 2024-09-15T23:12:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000947162 2024-09-15T23:12:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookuphash_latency": { 2024-09-15T23:12:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupino_latency": { 2024-09-15T23:12:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 1, 2024-09-15T23:12:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000114634, 2024-09-15T23:12:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000114634 2024-09-15T23:12:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupname_latency": { 2024-09-15T23:12:21.592 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:21.593 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:21.593 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:21.593 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.593 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupparent_latency": { 2024-09-15T23:12:21.593 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:21.593 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:21.593 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:21.593 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.593 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupsnap_latency": { 2024-09-15T23:12:21.593 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:21.593 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:21.594 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:21.594 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.594 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lssnap_latency": { 2024-09-15T23:12:21.594 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:21.594 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:21.594 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:21.594 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.594 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mkdir_latency": { 2024-09-15T23:12:21.594 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 20, 2024-09-15T23:12:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.019577121, 2024-09-15T23:12:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000978856 2024-09-15T23:12:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mknod_latency": { 2024-09-15T23:12:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mksnap_latency": { 2024-09-15T23:12:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.595 INFO:teuthology.orchestra.run.smithi121.stdout: "req_open_latency": { 2024-09-15T23:12:21.596 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 1, 2024-09-15T23:12:21.596 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000359701, 2024-09-15T23:12:21.596 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000359701 2024-09-15T23:12:21.596 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.596 INFO:teuthology.orchestra.run.smithi121.stdout: "req_readdir_latency": { 2024-09-15T23:12:21.596 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 37036, 2024-09-15T23:12:21.596 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 164.058287739, 2024-09-15T23:12:21.596 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.004429697 2024-09-15T23:12:21.596 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.596 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rename_latency": { 2024-09-15T23:12:21.596 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 5639, 2024-09-15T23:12:21.596 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 18.742105302, 2024-09-15T23:12:21.596 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.003323657 2024-09-15T23:12:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: "req_renamesnap_latency": { 2024-09-15T23:12:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: "req_snapdiff_latency": { 2024-09-15T23:12:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmdir_latency": { 2024-09-15T23:12:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 19, 2024-09-15T23:12:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.272509355, 2024-09-15T23:12:21.597 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.014342597 2024-09-15T23:12:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmsnap_latency": { 2024-09-15T23:12:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmxattr_latency": { 2024-09-15T23:12:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setattr_latency": { 2024-09-15T23:12:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 42, 2024-09-15T23:12:21.598 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.395045062, 2024-09-15T23:12:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.009405834 2024-09-15T23:12:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setdirlayout_latency": { 2024-09-15T23:12:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setfilelock_latency": { 2024-09-15T23:12:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 820, 2024-09-15T23:12:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.655219916, 2024-09-15T23:12:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000799048 2024-09-15T23:12:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setlayout_latency": { 2024-09-15T23:12:21.599 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setxattr_latency": { 2024-09-15T23:12:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 8, 2024-09-15T23:12:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.018787283, 2024-09-15T23:12:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.002348410 2024-09-15T23:12:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: "req_symlink_latency": { 2024-09-15T23:12:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: "req_unlink_latency": { 2024-09-15T23:12:21.601 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 24491, 2024-09-15T23:12:21.601 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 135.020156606, 2024-09-15T23:12:21.601 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.005513052 2024-09-15T23:12:21.601 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.601 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_revoke_eviction": 0, 2024-09-15T23:12:21.601 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_acquisition_throttle": 0, 2024-09-15T23:12:21.601 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getvxattr_latency": { 2024-09-15T23:12:21.601 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:21.601 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:21.601 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:21.601 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:21.601 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.601 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_sessions": { 2024-09-15T23:12:21.601 INFO:teuthology.orchestra.run.smithi121.stdout: "session_count": 1, 2024-09-15T23:12:21.602 INFO:teuthology.orchestra.run.smithi121.stdout: "session_add": 3, 2024-09-15T23:12:21.602 INFO:teuthology.orchestra.run.smithi121.stdout: "session_remove": 2, 2024-09-15T23:12:21.602 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_open": 1, 2024-09-15T23:12:21.602 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_stale": 0, 2024-09-15T23:12:21.602 INFO:teuthology.orchestra.run.smithi121.stdout: "total_load": 11211, 2024-09-15T23:12:21.602 INFO:teuthology.orchestra.run.smithi121.stdout: "average_load": 11211, 2024-09-15T23:12:21.602 INFO:teuthology.orchestra.run.smithi121.stdout: "avg_session_uptime": 1553, 2024-09-15T23:12:21.602 INFO:teuthology.orchestra.run.smithi121.stdout: "mdthresh_evicted": 0 2024-09-15T23:12:21.602 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.602 INFO:teuthology.orchestra.run.smithi121.stdout: "mempool": { 2024-09-15T23:12:21.602 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_bytes": 360, 2024-09-15T23:12:21.602 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_items": 360, 2024-09-15T23:12:21.602 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T23:12:21.602 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_items": 0, 2024-09-15T23:12:21.603 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T23:12:21.603 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_items": 0, 2024-09-15T23:12:21.603 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T23:12:21.603 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T23:12:21.603 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T23:12:21.603 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T23:12:21.603 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T23:12:21.603 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_items": 0, 2024-09-15T23:12:21.603 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T23:12:21.603 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T23:12:21.603 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_bytes": 0, 2024-09-15T23:12:21.603 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_items": 0, 2024-09-15T23:12:21.603 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_bytes": 0, 2024-09-15T23:12:21.603 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_items": 0, 2024-09-15T23:12:21.604 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T23:12:21.604 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T23:12:21.604 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T23:12:21.604 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T23:12:21.604 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T23:12:21.604 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_items": 0, 2024-09-15T23:12:21.604 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_bytes": 0, 2024-09-15T23:12:21.604 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_items": 0, 2024-09-15T23:12:21.604 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T23:12:21.604 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T23:12:21.604 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_bytes": 0, 2024-09-15T23:12:21.604 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_items": 0, 2024-09-15T23:12:21.604 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_bytes": 0, 2024-09-15T23:12:21.604 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_items": 0, 2024-09-15T23:12:21.605 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T23:12:21.605 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_items": 0, 2024-09-15T23:12:21.605 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T23:12:21.605 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_items": 0, 2024-09-15T23:12:21.605 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_bytes": 283941, 2024-09-15T23:12:21.605 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_items": 123, 2024-09-15T23:12:21.605 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_bytes": 616, 2024-09-15T23:12:21.605 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_items": 7, 2024-09-15T23:12:21.605 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_bytes": 0, 2024-09-15T23:12:21.605 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_items": 0, 2024-09-15T23:12:21.605 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_bytes": 0, 2024-09-15T23:12:21.605 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_items": 0, 2024-09-15T23:12:21.605 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_bytes": 0, 2024-09-15T23:12:21.605 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_items": 0, 2024-09-15T23:12:21.606 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_bytes": 7304, 2024-09-15T23:12:21.606 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_items": 150, 2024-09-15T23:12:21.606 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T23:12:21.606 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_items": 0, 2024-09-15T23:12:21.606 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_bytes": 0, 2024-09-15T23:12:21.606 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_items": 0, 2024-09-15T23:12:21.606 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_bytes": 452315, 2024-09-15T23:12:21.606 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_items": 4531, 2024-09-15T23:12:21.606 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_bytes": 0, 2024-09-15T23:12:21.606 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_items": 0, 2024-09-15T23:12:21.606 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_bytes": 0, 2024-09-15T23:12:21.606 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_items": 0 2024-09-15T23:12:21.606 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.606 INFO:teuthology.orchestra.run.smithi121.stdout: "objecter": { 2024-09-15T23:12:21.607 INFO:teuthology.orchestra.run.smithi121.stdout: "op_active": 0, 2024-09-15T23:12:21.607 INFO:teuthology.orchestra.run.smithi121.stdout: "op_laggy": 0, 2024-09-15T23:12:21.607 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send": 84959, 2024-09-15T23:12:21.607 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send_bytes": 280015833, 2024-09-15T23:12:21.607 INFO:teuthology.orchestra.run.smithi121.stdout: "op_resend": 37, 2024-09-15T23:12:21.607 INFO:teuthology.orchestra.run.smithi121.stdout: "op_reply": 84959, 2024-09-15T23:12:21.607 INFO:teuthology.orchestra.run.smithi121.stdout: "op_latency": { 2024-09-15T23:12:21.607 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 84959, 2024-09-15T23:12:21.607 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 1846.753749615, 2024-09-15T23:12:21.607 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.021736999 2024-09-15T23:12:21.607 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.607 INFO:teuthology.orchestra.run.smithi121.stdout: "op_inflight": 0, 2024-09-15T23:12:21.607 INFO:teuthology.orchestra.run.smithi121.stdout: "oplen_avg": { 2024-09-15T23:12:21.607 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 84959, 2024-09-15T23:12:21.608 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 120990 2024-09-15T23:12:21.608 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.608 INFO:teuthology.orchestra.run.smithi121.stdout: "op": 84959, 2024-09-15T23:12:21.608 INFO:teuthology.orchestra.run.smithi121.stdout: "op_r": 754, 2024-09-15T23:12:21.608 INFO:teuthology.orchestra.run.smithi121.stdout: "op_w": 84205, 2024-09-15T23:12:21.608 INFO:teuthology.orchestra.run.smithi121.stdout: "op_rmw": 0, 2024-09-15T23:12:21.608 INFO:teuthology.orchestra.run.smithi121.stdout: "op_pg": 0, 2024-09-15T23:12:21.608 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_stat": 2821, 2024-09-15T23:12:21.608 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_create": 14156, 2024-09-15T23:12:21.608 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_read": 754, 2024-09-15T23:12:21.608 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_write": 40127, 2024-09-15T23:12:21.608 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writefull": 829, 2024-09-15T23:12:21.608 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writesame": 0, 2024-09-15T23:12:21.608 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_append": 0, 2024-09-15T23:12:21.609 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_zero": 0, 2024-09-15T23:12:21.609 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_truncate": 22, 2024-09-15T23:12:21.609 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_delete": 25289, 2024-09-15T23:12:21.609 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_mapext": 0, 2024-09-15T23:12:21.609 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_sparse_read": 0, 2024-09-15T23:12:21.609 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_clonerange": 0, 2024-09-15T23:12:21.609 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_getxattr": 0, 2024-09-15T23:12:21.609 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_setxattr": 28312, 2024-09-15T23:12:21.609 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_cmpxattr": 0, 2024-09-15T23:12:21.609 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_rmxattr": 0, 2024-09-15T23:12:21.609 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_resetxattrs": 0, 2024-09-15T23:12:21.609 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_call": 0, 2024-09-15T23:12:21.609 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_watch": 0, 2024-09-15T23:12:21.609 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_notify": 0, 2024-09-15T23:12:21.610 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T23:12:21.610 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls": 0, 2024-09-15T23:12:21.610 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls_filter": 0, 2024-09-15T23:12:21.610 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_other": 0, 2024-09-15T23:12:21.610 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_active": 0, 2024-09-15T23:12:21.610 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_send": 0, 2024-09-15T23:12:21.610 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_resend": 0, 2024-09-15T23:12:21.610 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_ping": 0, 2024-09-15T23:12:21.610 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_active": 0, 2024-09-15T23:12:21.610 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_send": 0, 2024-09-15T23:12:21.610 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_resend": 0, 2024-09-15T23:12:21.610 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_active": 0, 2024-09-15T23:12:21.610 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_send": 0, 2024-09-15T23:12:21.610 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_resend": 0, 2024-09-15T23:12:21.611 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_active": 0, 2024-09-15T23:12:21.611 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_send": 0, 2024-09-15T23:12:21.611 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_resend": 0, 2024-09-15T23:12:21.611 INFO:teuthology.orchestra.run.smithi121.stdout: "command_active": 0, 2024-09-15T23:12:21.611 INFO:teuthology.orchestra.run.smithi121.stdout: "command_send": 0, 2024-09-15T23:12:21.611 INFO:teuthology.orchestra.run.smithi121.stdout: "command_resend": 0, 2024-09-15T23:12:21.611 INFO:teuthology.orchestra.run.smithi121.stdout: "map_epoch": 89, 2024-09-15T23:12:21.611 INFO:teuthology.orchestra.run.smithi121.stdout: "map_full": 0, 2024-09-15T23:12:21.611 INFO:teuthology.orchestra.run.smithi121.stdout: "map_inc": 3, 2024-09-15T23:12:21.611 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_sessions": 12, 2024-09-15T23:12:21.611 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_open": 12, 2024-09-15T23:12:21.611 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_close": 0, 2024-09-15T23:12:21.611 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_laggy": 0, 2024-09-15T23:12:21.611 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_wr": 6644, 2024-09-15T23:12:21.612 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_rd": 0, 2024-09-15T23:12:21.612 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_del": 2036 2024-09-15T23:12:21.612 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.612 INFO:teuthology.orchestra.run.smithi121.stdout: "oft": { 2024-09-15T23:12:21.612 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_objs": 1, 2024-09-15T23:12:21.612 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_kv_pairs": 0, 2024-09-15T23:12:21.612 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_updates": 0, 2024-09-15T23:12:21.612 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_removes": 0 2024-09-15T23:12:21.612 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.612 INFO:teuthology.orchestra.run.smithi121.stdout: "osd-slow-ops": { 2024-09-15T23:12:21.612 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_ops_count": 7 2024-09-15T23:12:21.612 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.612 INFO:teuthology.orchestra.run.smithi121.stdout: "purge_queue": { 2024-09-15T23:12:21.612 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops": 0, 2024-09-15T23:12:21.613 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops_high_water": 10, 2024-09-15T23:12:21.613 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing": 0, 2024-09-15T23:12:21.613 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_high_water": 10, 2024-09-15T23:12:21.613 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executed_ops": 24509, 2024-09-15T23:12:21.613 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executed": 24509, 2024-09-15T23:12:21.613 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_item_in_journal": 0 2024-09-15T23:12:21.613 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.613 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T23:12:21.613 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-15T23:12:21.613 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2024-09-15T23:12:21.613 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-15T23:12:21.613 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 528659, 2024-09-15T23:12:21.613 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 110046919, 2024-09-15T23:12:21.613 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:21.614 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 528659, 2024-09-15T23:12:21.614 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-15T23:12:21.614 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-15T23:12:21.614 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 528659, 2024-09-15T23:12:21.614 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 110046919, 2024-09-15T23:12:21.614 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-15T23:12:21.614 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:21.614 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:21.614 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:21.614 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:21.614 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.614 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_bytes": { 2024-09-15T23:12:21.614 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-15T23:12:21.614 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2024-09-15T23:12:21.615 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-15T23:12:21.615 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-15T23:12:21.615 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-15T23:12:21.615 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:21.615 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-15T23:12:21.615 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 84959, 2024-09-15T23:12:21.615 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 282491242, 2024-09-15T23:12:21.615 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 84959, 2024-09-15T23:12:21.615 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 282491242, 2024-09-15T23:12:21.615 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-15T23:12:21.615 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:21.615 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:21.615 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:21.615 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:21.615 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.616 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_ops": { 2024-09-15T23:12:21.616 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-15T23:12:21.616 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 1024, 2024-09-15T23:12:21.616 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-15T23:12:21.616 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-15T23:12:21.616 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-15T23:12:21.616 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:21.616 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-15T23:12:21.616 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 84959, 2024-09-15T23:12:21.616 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 84959, 2024-09-15T23:12:21.616 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 84959, 2024-09-15T23:12:21.616 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 84959, 2024-09-15T23:12:21.616 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-15T23:12:21.617 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:21.617 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:21.617 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:21.617 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:21.617 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.617 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle": { 2024-09-15T23:12:21.617 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-15T23:12:21.617 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2024-09-15T23:12:21.617 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-15T23:12:21.617 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 24509, 2024-09-15T23:12:21.617 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 2475409, 2024-09-15T23:12:21.617 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:21.617 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 24509, 2024-09-15T23:12:21.617 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-15T23:12:21.618 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-15T23:12:21.618 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 754, 2024-09-15T23:12:21.618 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 2475409, 2024-09-15T23:12:21.618 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-15T23:12:21.618 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:21.618 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:21.618 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:21.618 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:21.618 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:21.618 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle-0x56476d066960": { 2024-09-15T23:12:21.618 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-15T23:12:21.618 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2024-09-15T23:12:21.618 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-15T23:12:21.618 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 126005, 2024-09-15T23:12:21.618 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 255312014, 2024-09-15T23:12:21.619 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:21.619 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 126005, 2024-09-15T23:12:21.619 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-15T23:12:21.619 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-15T23:12:21.619 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 39373, 2024-09-15T23:12:21.619 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 255312014, 2024-09-15T23:12:21.619 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-15T23:12:21.619 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:21.619 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:21.619 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:21.619 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:21.619 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:21.619 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-15T23:12:22.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:22 smithi121 ceph-mon[34815]: pgmap v1144: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 66 op/s 2024-09-15T23:12:22.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:22 smithi111 ceph-mon[30629]: pgmap v1144: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 66 op/s 2024-09-15T23:12:22.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:22 smithi192 ceph-mon[35427]: pgmap v1144: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 147 KiB/s wr, 66 op/s 2024-09-15T23:12:22.648 INFO:tasks.check_counter:Daemon mds.b mds.exported=6 2024-09-15T23:12:22.648 INFO:tasks.check_counter:Daemon mds.b mds.imported=7 2024-09-15T23:12:22.648 DEBUG:tasks.check_counter:Getting stats from e 2024-09-15T23:12:22.648 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-15T23:12:22.908 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T23:12:23.525 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-15T23:12:23.526 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T23:12:23.526 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 2152, 2024-09-15T23:12:23.526 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 2154, 2024-09-15T23:12:23.526 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 361269463, 2024-09-15T23:12:23.526 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 684400, 2024-09-15T23:12:23.526 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 8, 2024-09-15T23:12:23.526 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 4, 2024-09-15T23:12:23.526 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.343002952, 2024-09-15T23:12:23.526 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.045262211, 2024-09-15T23:12:23.526 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.236695938, 2024-09-15T23:12:23.526 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.046994478, 2024-09-15T23:12:23.526 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:23.527 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 2152, 2024-09-15T23:12:23.527 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.057381230, 2024-09-15T23:12:23.527 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000026664 2024-09-15T23:12:23.527 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.527 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:23.527 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.527 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.527 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.527 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.527 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-15T23:12:23.527 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 0 2024-09-15T23:12:23.527 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.527 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T23:12:23.528 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 2404, 2024-09-15T23:12:23.528 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 2386, 2024-09-15T23:12:23.528 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 336936227, 2024-09-15T23:12:23.528 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 894751, 2024-09-15T23:12:23.528 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 6, 2024-09-15T23:12:23.528 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 5, 2024-09-15T23:12:23.528 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.389601667, 2024-09-15T23:12:23.528 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.061603034, 2024-09-15T23:12:23.528 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.280331615, 2024-09-15T23:12:23.528 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.043901610, 2024-09-15T23:12:23.528 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:23.528 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 2385, 2024-09-15T23:12:23.528 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.073342309, 2024-09-15T23:12:23.529 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000030751 2024-09-15T23:12:23.529 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.529 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:23.529 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.529 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.529 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.529 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.529 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 895824, 2024-09-15T23:12:23.529 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 283232 2024-09-15T23:12:23.529 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.529 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T23:12:23.529 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 2573, 2024-09-15T23:12:23.529 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 2917, 2024-09-15T23:12:23.530 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 187114847, 2024-09-15T23:12:23.530 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 1505712, 2024-09-15T23:12:23.530 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 6, 2024-09-15T23:12:23.530 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 5, 2024-09-15T23:12:23.530 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.329438205, 2024-09-15T23:12:23.530 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.074373485, 2024-09-15T23:12:23.530 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.164570226, 2024-09-15T23:12:23.530 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.069034695, 2024-09-15T23:12:23.530 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:23.530 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 2916, 2024-09-15T23:12:23.530 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.106526332, 2024-09-15T23:12:23.530 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000036531 2024-09-15T23:12:23.530 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.530 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:23.531 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.531 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.531 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.531 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.531 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-15T23:12:23.531 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 689088 2024-09-15T23:12:23.531 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.531 INFO:teuthology.orchestra.run.smithi121.stdout: "cct": { 2024-09-15T23:12:23.531 INFO:teuthology.orchestra.run.smithi121.stdout: "total_workers": 1, 2024-09-15T23:12:23.531 INFO:teuthology.orchestra.run.smithi121.stdout: "unhealthy_workers": 0 2024-09-15T23:12:23.531 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.531 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-MDSRank": { 2024-09-15T23:12:23.531 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2024-09-15T23:12:23.531 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2024-09-15T23:12:23.532 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 10935, 2024-09-15T23:12:23.532 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.365651472, 2024-09-15T23:12:23.532 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000033438 2024-09-15T23:12:23.532 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:23.532 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.532 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-PurgeQueue": { 2024-09-15T23:12:23.532 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2024-09-15T23:12:23.532 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2024-09-15T23:12:23.532 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.532 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.532 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.532 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:23.532 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.532 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": { 2024-09-15T23:12:23.533 INFO:teuthology.orchestra.run.smithi121.stdout: "request": 0, 2024-09-15T23:12:23.533 INFO:teuthology.orchestra.run.smithi121.stdout: "reply": 0, 2024-09-15T23:12:23.533 INFO:teuthology.orchestra.run.smithi121.stdout: "reply_latency": { 2024-09-15T23:12:23.533 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.533 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.533 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.533 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.533 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_reply": 0, 2024-09-15T23:12:23.533 INFO:teuthology.orchestra.run.smithi121.stdout: "forward": 0, 2024-09-15T23:12:23.533 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_complete": 0, 2024-09-15T23:12:23.533 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_keys": 0, 2024-09-15T23:12:23.533 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_commit": 0, 2024-09-15T23:12:23.533 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_split": 0, 2024-09-15T23:12:23.533 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_merge": 0, 2024-09-15T23:12:23.534 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes": 92958, 2024-09-15T23:12:23.534 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_top": 46708, 2024-09-15T23:12:23.534 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_bottom": 46250, 2024-09-15T23:12:23.534 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pin_tail": 0, 2024-09-15T23:12:23.534 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pinned": 26232, 2024-09-15T23:12:23.534 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_expired": 0, 2024-09-15T23:12:23.534 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_with_caps": 0, 2024-09-15T23:12:23.534 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": 0, 2024-09-15T23:12:23.534 INFO:teuthology.orchestra.run.smithi121.stdout: "subtrees": 1, 2024-09-15T23:12:23.534 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse": 0, 2024-09-15T23:12:23.534 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_hit": 0, 2024-09-15T23:12:23.534 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_forward": 0, 2024-09-15T23:12:23.534 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_discover": 0, 2024-09-15T23:12:23.534 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_dir_fetch": 0, 2024-09-15T23:12:23.535 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_remote_ino": 0, 2024-09-15T23:12:23.535 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_lock": 0, 2024-09-15T23:12:23.535 INFO:teuthology.orchestra.run.smithi121.stdout: "load_cent": 0, 2024-09-15T23:12:23.535 INFO:teuthology.orchestra.run.smithi121.stdout: "q": 0, 2024-09-15T23:12:23.535 INFO:teuthology.orchestra.run.smithi121.stdout: "exported": 0, 2024-09-15T23:12:23.535 INFO:teuthology.orchestra.run.smithi121.stdout: "exported_inodes": 0, 2024-09-15T23:12:23.535 INFO:teuthology.orchestra.run.smithi121.stdout: "imported": 0, 2024-09-15T23:12:23.535 INFO:teuthology.orchestra.run.smithi121.stdout: "imported_inodes": 0, 2024-09-15T23:12:23.535 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_dir_fetch": 0, 2024-09-15T23:12:23.535 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_backtrace_fetch": 0, 2024-09-15T23:12:23.535 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_peer_discover": 0, 2024-09-15T23:12:23.535 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rfiles": 0, 2024-09-15T23:12:23.535 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rbytes": 0, 2024-09-15T23:12:23.536 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rsnaps": 0, 2024-09-15T23:12:23.536 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_fetch": 0, 2024-09-15T23:12:23.536 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_set_tag": 0, 2024-09-15T23:12:23.536 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T23:12:23.536 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_inotable_repaired": 0, 2024-09-15T23:12:23.536 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_inodes": 0, 2024-09-15T23:12:23.536 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_base_inodes": 0, 2024-09-15T23:12:23.536 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dirfrag_rstats": 0, 2024-09-15T23:12:23.536 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_file_inodes": 0, 2024-09-15T23:12:23.536 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_inode_file_caps": 0, 2024-09-15T23:12:23.536 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_revoke": 0, 2024-09-15T23:12:23.536 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_grant": 0, 2024-09-15T23:12:23.536 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T23:12:23.537 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T23:12:23.537 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T23:12:23.537 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps": 0, 2024-09-15T23:12:23.537 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps_dirty": 0, 2024-09-15T23:12:23.537 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_cap_release": 0, 2024-09-15T23:12:23.537 INFO:teuthology.orchestra.run.smithi121.stdout: "process_request_cap_release": 0 2024-09-15T23:12:23.537 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.537 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_cache": { 2024-09-15T23:12:23.537 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update": 0, 2024-09-15T23:12:23.537 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update_receipt": 0, 2024-09-15T23:12:23.537 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_try_discover": 0, 2024-09-15T23:12:23.537 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_send_discover": 0, 2024-09-15T23:12:23.537 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_handle_discover": 0, 2024-09-15T23:12:23.538 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays": 0, 2024-09-15T23:12:23.538 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_delayed": 0, 2024-09-15T23:12:23.538 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_enqueuing": 0, 2024-09-15T23:12:23.538 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_created": 0, 2024-09-15T23:12:23.538 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_enqueued": 0, 2024-09-15T23:12:23.538 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_reintegrated": 0, 2024-09-15T23:12:23.538 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_migrated": 0, 2024-09-15T23:12:23.538 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_processing": 0, 2024-09-15T23:12:23.538 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_enqueued": 0, 2024-09-15T23:12:23.538 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_prioritized": 0, 2024-09-15T23:12:23.538 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_started": 0, 2024-09-15T23:12:23.538 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_completed": 0, 2024-09-15T23:12:23.538 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_quiesce_path": 0, 2024-09-15T23:12:23.538 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_quiesce_inode": 0, 2024-09-15T23:12:23.539 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_enqueue_scrub": 0, 2024-09-15T23:12:23.539 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_exportdir": 0, 2024-09-15T23:12:23.539 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_flush": 0, 2024-09-15T23:12:23.539 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragmentdir": 0, 2024-09-15T23:12:23.539 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragstats": 0, 2024-09-15T23:12:23.539 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_inodestats": 0 2024-09-15T23:12:23.539 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.539 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_log": { 2024-09-15T23:12:23.539 INFO:teuthology.orchestra.run.smithi121.stdout: "evlrg": 0, 2024-09-15T23:12:23.539 INFO:teuthology.orchestra.run.smithi121.stdout: "evadd": 0, 2024-09-15T23:12:23.539 INFO:teuthology.orchestra.run.smithi121.stdout: "evex": 0, 2024-09-15T23:12:23.539 INFO:teuthology.orchestra.run.smithi121.stdout: "evtrm": 0, 2024-09-15T23:12:23.539 INFO:teuthology.orchestra.run.smithi121.stdout: "ev": 422836, 2024-09-15T23:12:23.539 INFO:teuthology.orchestra.run.smithi121.stdout: "evexg": 0, 2024-09-15T23:12:23.539 INFO:teuthology.orchestra.run.smithi121.stdout: "evexd": 0, 2024-09-15T23:12:23.540 INFO:teuthology.orchestra.run.smithi121.stdout: "segadd": 0, 2024-09-15T23:12:23.540 INFO:teuthology.orchestra.run.smithi121.stdout: "segex": 0, 2024-09-15T23:12:23.540 INFO:teuthology.orchestra.run.smithi121.stdout: "segtrm": 0, 2024-09-15T23:12:23.540 INFO:teuthology.orchestra.run.smithi121.stdout: "seg": 136, 2024-09-15T23:12:23.540 INFO:teuthology.orchestra.run.smithi121.stdout: "segmjr": 36, 2024-09-15T23:12:23.540 INFO:teuthology.orchestra.run.smithi121.stdout: "segexg": 0, 2024-09-15T23:12:23.540 INFO:teuthology.orchestra.run.smithi121.stdout: "segexd": 0, 2024-09-15T23:12:23.540 INFO:teuthology.orchestra.run.smithi121.stdout: "expos": 649922448, 2024-09-15T23:12:23.540 INFO:teuthology.orchestra.run.smithi121.stdout: "wrpos": 886770670, 2024-09-15T23:12:23.540 INFO:teuthology.orchestra.run.smithi121.stdout: "rdpos": 886770481, 2024-09-15T23:12:23.540 INFO:teuthology.orchestra.run.smithi121.stdout: "jlat": { 2024-09-15T23:12:23.540 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.540 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.541 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.541 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.541 INFO:teuthology.orchestra.run.smithi121.stdout: "replayed": 422836 2024-09-15T23:12:23.541 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.541 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_mem": { 2024-09-15T23:12:23.541 INFO:teuthology.orchestra.run.smithi121.stdout: "ino": 29, 2024-09-15T23:12:23.541 INFO:teuthology.orchestra.run.smithi121.stdout: "ino+": 93070, 2024-09-15T23:12:23.541 INFO:teuthology.orchestra.run.smithi121.stdout: "ino-": 93041, 2024-09-15T23:12:23.541 INFO:teuthology.orchestra.run.smithi121.stdout: "dir": 83, 2024-09-15T23:12:23.541 INFO:teuthology.orchestra.run.smithi121.stdout: "dir+": 495, 2024-09-15T23:12:23.541 INFO:teuthology.orchestra.run.smithi121.stdout: "dir-": 412, 2024-09-15T23:12:23.541 INFO:teuthology.orchestra.run.smithi121.stdout: "dn": 92958, 2024-09-15T23:12:23.541 INFO:teuthology.orchestra.run.smithi121.stdout: "dn+": 94135, 2024-09-15T23:12:23.542 INFO:teuthology.orchestra.run.smithi121.stdout: "dn-": 1177, 2024-09-15T23:12:23.542 INFO:teuthology.orchestra.run.smithi121.stdout: "cap": 0, 2024-09-15T23:12:23.542 INFO:teuthology.orchestra.run.smithi121.stdout: "cap+": 0, 2024-09-15T23:12:23.542 INFO:teuthology.orchestra.run.smithi121.stdout: "cap-": 0, 2024-09-15T23:12:23.542 INFO:teuthology.orchestra.run.smithi121.stdout: "rss": 130708, 2024-09-15T23:12:23.542 INFO:teuthology.orchestra.run.smithi121.stdout: "heap": 215328 2024-09-15T23:12:23.542 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.542 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_server": { 2024-09-15T23:12:23.542 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_client_request": 0, 2024-09-15T23:12:23.542 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_server_request": 0, 2024-09-15T23:12:23.542 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_request": 0, 2024-09-15T23:12:23.542 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_session": 0, 2024-09-15T23:12:23.542 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_peer_request": 0, 2024-09-15T23:12:23.542 INFO:teuthology.orchestra.run.smithi121.stdout: "req_create_latency": { 2024-09-15T23:12:23.543 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.543 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.543 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.543 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.543 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getattr_latency": { 2024-09-15T23:12:23.543 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.543 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.543 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.543 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.543 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getfilelock_latency": { 2024-09-15T23:12:23.543 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.543 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.543 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.543 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.544 INFO:teuthology.orchestra.run.smithi121.stdout: "req_link_latency": { 2024-09-15T23:12:23.544 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.544 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.544 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.544 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.544 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookup_latency": { 2024-09-15T23:12:23.544 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.544 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.544 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.544 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.544 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookuphash_latency": { 2024-09-15T23:12:23.544 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.544 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.545 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.545 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.545 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupino_latency": { 2024-09-15T23:12:23.545 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.545 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.545 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.545 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.545 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupname_latency": { 2024-09-15T23:12:23.545 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.545 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.545 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.545 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.545 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupparent_latency": { 2024-09-15T23:12:23.545 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.546 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.546 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.546 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.546 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupsnap_latency": { 2024-09-15T23:12:23.546 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.546 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.546 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.546 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.546 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lssnap_latency": { 2024-09-15T23:12:23.546 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.546 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.546 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.546 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.546 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mkdir_latency": { 2024-09-15T23:12:23.546 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.547 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.547 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.547 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.547 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mknod_latency": { 2024-09-15T23:12:23.547 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.547 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.547 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.547 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.547 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mksnap_latency": { 2024-09-15T23:12:23.547 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.547 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.547 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.547 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.548 INFO:teuthology.orchestra.run.smithi121.stdout: "req_open_latency": { 2024-09-15T23:12:23.548 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.548 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.548 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.548 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.548 INFO:teuthology.orchestra.run.smithi121.stdout: "req_readdir_latency": { 2024-09-15T23:12:23.548 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.548 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.548 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.548 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.548 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rename_latency": { 2024-09-15T23:12:23.548 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.548 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.549 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.549 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.549 INFO:teuthology.orchestra.run.smithi121.stdout: "req_renamesnap_latency": { 2024-09-15T23:12:23.549 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.549 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.549 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.549 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.549 INFO:teuthology.orchestra.run.smithi121.stdout: "req_snapdiff_latency": { 2024-09-15T23:12:23.549 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.549 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.549 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.549 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.549 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmdir_latency": { 2024-09-15T23:12:23.549 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.550 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.550 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.550 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.550 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmsnap_latency": { 2024-09-15T23:12:23.550 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.550 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.550 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.550 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.550 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmxattr_latency": { 2024-09-15T23:12:23.550 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.550 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.550 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.550 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.550 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setattr_latency": { 2024-09-15T23:12:23.551 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.551 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.551 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.551 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.551 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setdirlayout_latency": { 2024-09-15T23:12:23.551 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.551 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.551 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.551 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.551 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setfilelock_latency": { 2024-09-15T23:12:23.551 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.551 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.551 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.551 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.551 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setlayout_latency": { 2024-09-15T23:12:23.552 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.552 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.552 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.552 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.552 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setxattr_latency": { 2024-09-15T23:12:23.552 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.552 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.552 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.552 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.552 INFO:teuthology.orchestra.run.smithi121.stdout: "req_symlink_latency": { 2024-09-15T23:12:23.552 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.552 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.552 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.553 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.553 INFO:teuthology.orchestra.run.smithi121.stdout: "req_unlink_latency": { 2024-09-15T23:12:23.553 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.553 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.553 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.553 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.553 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_revoke_eviction": 0, 2024-09-15T23:12:23.553 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_acquisition_throttle": 0, 2024-09-15T23:12:23.553 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getvxattr_latency": { 2024-09-15T23:12:23.553 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.553 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.553 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.553 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:23.553 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.554 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_sessions": { 2024-09-15T23:12:23.554 INFO:teuthology.orchestra.run.smithi121.stdout: "session_count": 1, 2024-09-15T23:12:23.554 INFO:teuthology.orchestra.run.smithi121.stdout: "session_add": 1, 2024-09-15T23:12:23.554 INFO:teuthology.orchestra.run.smithi121.stdout: "session_remove": 0, 2024-09-15T23:12:23.554 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_open": 1, 2024-09-15T23:12:23.554 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_stale": 0, 2024-09-15T23:12:23.554 INFO:teuthology.orchestra.run.smithi121.stdout: "total_load": 0, 2024-09-15T23:12:23.554 INFO:teuthology.orchestra.run.smithi121.stdout: "average_load": 0, 2024-09-15T23:12:23.554 INFO:teuthology.orchestra.run.smithi121.stdout: "avg_session_uptime": 2942, 2024-09-15T23:12:23.554 INFO:teuthology.orchestra.run.smithi121.stdout: "mdthresh_evicted": 0 2024-09-15T23:12:23.554 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.554 INFO:teuthology.orchestra.run.smithi121.stdout: "mempool": { 2024-09-15T23:12:23.554 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_bytes": 0, 2024-09-15T23:12:23.554 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_items": 0, 2024-09-15T23:12:23.555 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T23:12:23.555 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_items": 0, 2024-09-15T23:12:23.555 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T23:12:23.555 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_items": 0, 2024-09-15T23:12:23.555 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T23:12:23.555 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T23:12:23.555 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T23:12:23.555 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T23:12:23.555 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T23:12:23.555 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_items": 0, 2024-09-15T23:12:23.555 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T23:12:23.555 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T23:12:23.555 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_bytes": 0, 2024-09-15T23:12:23.555 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_items": 0, 2024-09-15T23:12:23.556 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_bytes": 0, 2024-09-15T23:12:23.556 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_items": 0, 2024-09-15T23:12:23.556 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T23:12:23.556 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T23:12:23.556 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T23:12:23.556 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T23:12:23.556 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T23:12:23.556 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_items": 0, 2024-09-15T23:12:23.556 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_bytes": 0, 2024-09-15T23:12:23.556 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_items": 0, 2024-09-15T23:12:23.556 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T23:12:23.556 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T23:12:23.556 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_bytes": 0, 2024-09-15T23:12:23.556 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_items": 0, 2024-09-15T23:12:23.557 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_bytes": 0, 2024-09-15T23:12:23.557 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_items": 0, 2024-09-15T23:12:23.557 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T23:12:23.557 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_items": 0, 2024-09-15T23:12:23.557 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T23:12:23.557 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_items": 0, 2024-09-15T23:12:23.557 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_bytes": 529834, 2024-09-15T23:12:23.557 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_items": 97, 2024-09-15T23:12:23.557 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_bytes": 88, 2024-09-15T23:12:23.557 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_items": 1, 2024-09-15T23:12:23.557 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_bytes": 0, 2024-09-15T23:12:23.557 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_items": 0, 2024-09-15T23:12:23.557 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_bytes": 0, 2024-09-15T23:12:23.557 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_items": 0, 2024-09-15T23:12:23.558 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_bytes": 0, 2024-09-15T23:12:23.558 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_items": 0, 2024-09-15T23:12:23.558 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_bytes": 7304, 2024-09-15T23:12:23.558 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_items": 150, 2024-09-15T23:12:23.558 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T23:12:23.558 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_items": 0, 2024-09-15T23:12:23.558 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_bytes": 0, 2024-09-15T23:12:23.558 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_items": 0, 2024-09-15T23:12:23.558 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_bytes": 65626105, 2024-09-15T23:12:23.558 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_items": 374097, 2024-09-15T23:12:23.558 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_bytes": 0, 2024-09-15T23:12:23.558 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_items": 0, 2024-09-15T23:12:23.558 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_bytes": 0, 2024-09-15T23:12:23.558 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_items": 0 2024-09-15T23:12:23.559 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.559 INFO:teuthology.orchestra.run.smithi121.stdout: "objecter": { 2024-09-15T23:12:23.559 INFO:teuthology.orchestra.run.smithi121.stdout: "op_active": 0, 2024-09-15T23:12:23.559 INFO:teuthology.orchestra.run.smithi121.stdout: "op_laggy": 0, 2024-09-15T23:12:23.559 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send": 6643, 2024-09-15T23:12:23.559 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send_bytes": 21, 2024-09-15T23:12:23.559 INFO:teuthology.orchestra.run.smithi121.stdout: "op_resend": 3, 2024-09-15T23:12:23.559 INFO:teuthology.orchestra.run.smithi121.stdout: "op_reply": 6643, 2024-09-15T23:12:23.559 INFO:teuthology.orchestra.run.smithi121.stdout: "op_latency": { 2024-09-15T23:12:23.559 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 6643, 2024-09-15T23:12:23.559 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 11.249856827, 2024-09-15T23:12:23.559 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.001693490 2024-09-15T23:12:23.559 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.559 INFO:teuthology.orchestra.run.smithi121.stdout: "op_inflight": 0, 2024-09-15T23:12:23.559 INFO:teuthology.orchestra.run.smithi121.stdout: "oplen_avg": { 2024-09-15T23:12:23.560 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 6643, 2024-09-15T23:12:23.560 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 6644 2024-09-15T23:12:23.560 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.560 INFO:teuthology.orchestra.run.smithi121.stdout: "op": 6643, 2024-09-15T23:12:23.560 INFO:teuthology.orchestra.run.smithi121.stdout: "op_r": 6643, 2024-09-15T23:12:23.560 INFO:teuthology.orchestra.run.smithi121.stdout: "op_w": 0, 2024-09-15T23:12:23.560 INFO:teuthology.orchestra.run.smithi121.stdout: "op_rmw": 0, 2024-09-15T23:12:23.560 INFO:teuthology.orchestra.run.smithi121.stdout: "op_pg": 0, 2024-09-15T23:12:23.560 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_stat": 3629, 2024-09-15T23:12:23.560 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_create": 0, 2024-09-15T23:12:23.560 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_read": 3012, 2024-09-15T23:12:23.560 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_write": 0, 2024-09-15T23:12:23.560 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writefull": 0, 2024-09-15T23:12:23.560 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writesame": 0, 2024-09-15T23:12:23.561 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_append": 0, 2024-09-15T23:12:23.561 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_zero": 0, 2024-09-15T23:12:23.561 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_truncate": 0, 2024-09-15T23:12:23.561 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_delete": 0, 2024-09-15T23:12:23.561 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_mapext": 0, 2024-09-15T23:12:23.561 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_sparse_read": 0, 2024-09-15T23:12:23.561 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_clonerange": 0, 2024-09-15T23:12:23.561 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_getxattr": 1, 2024-09-15T23:12:23.561 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_setxattr": 0, 2024-09-15T23:12:23.561 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_cmpxattr": 0, 2024-09-15T23:12:23.561 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_rmxattr": 0, 2024-09-15T23:12:23.561 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_resetxattrs": 0, 2024-09-15T23:12:23.561 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_call": 0, 2024-09-15T23:12:23.561 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_watch": 0, 2024-09-15T23:12:23.562 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_notify": 0, 2024-09-15T23:12:23.562 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T23:12:23.562 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls": 0, 2024-09-15T23:12:23.562 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls_filter": 0, 2024-09-15T23:12:23.562 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_other": 0, 2024-09-15T23:12:23.562 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_active": 0, 2024-09-15T23:12:23.562 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_send": 0, 2024-09-15T23:12:23.562 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_resend": 0, 2024-09-15T23:12:23.562 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_ping": 0, 2024-09-15T23:12:23.562 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_active": 0, 2024-09-15T23:12:23.562 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_send": 0, 2024-09-15T23:12:23.562 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_resend": 0, 2024-09-15T23:12:23.562 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_active": 0, 2024-09-15T23:12:23.562 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_send": 0, 2024-09-15T23:12:23.563 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_resend": 0, 2024-09-15T23:12:23.563 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_active": 0, 2024-09-15T23:12:23.563 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_send": 0, 2024-09-15T23:12:23.563 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_resend": 0, 2024-09-15T23:12:23.563 INFO:teuthology.orchestra.run.smithi121.stdout: "command_active": 0, 2024-09-15T23:12:23.563 INFO:teuthology.orchestra.run.smithi121.stdout: "command_send": 0, 2024-09-15T23:12:23.563 INFO:teuthology.orchestra.run.smithi121.stdout: "command_resend": 0, 2024-09-15T23:12:23.563 INFO:teuthology.orchestra.run.smithi121.stdout: "map_epoch": 89, 2024-09-15T23:12:23.563 INFO:teuthology.orchestra.run.smithi121.stdout: "map_full": 0, 2024-09-15T23:12:23.563 INFO:teuthology.orchestra.run.smithi121.stdout: "map_inc": 3, 2024-09-15T23:12:23.563 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_sessions": 11, 2024-09-15T23:12:23.563 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_open": 11, 2024-09-15T23:12:23.563 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_close": 0, 2024-09-15T23:12:23.563 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_laggy": 0, 2024-09-15T23:12:23.564 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_wr": 0, 2024-09-15T23:12:23.564 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_rd": 2, 2024-09-15T23:12:23.564 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_del": 0 2024-09-15T23:12:23.564 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.564 INFO:teuthology.orchestra.run.smithi121.stdout: "oft": { 2024-09-15T23:12:23.564 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_objs": 0, 2024-09-15T23:12:23.564 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_kv_pairs": 0, 2024-09-15T23:12:23.564 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_updates": 0, 2024-09-15T23:12:23.564 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_removes": 0 2024-09-15T23:12:23.564 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.564 INFO:teuthology.orchestra.run.smithi121.stdout: "osd-slow-ops": { 2024-09-15T23:12:23.564 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_ops_count": 0 2024-09-15T23:12:23.564 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.565 INFO:teuthology.orchestra.run.smithi121.stdout: "purge_queue": { 2024-09-15T23:12:23.565 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops": 0, 2024-09-15T23:12:23.565 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops_high_water": 0, 2024-09-15T23:12:23.565 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing": 0, 2024-09-15T23:12:23.565 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_high_water": 0, 2024-09-15T23:12:23.565 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executed_ops": 0, 2024-09-15T23:12:23.565 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executed": 0, 2024-09-15T23:12:23.565 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_item_in_journal": 0 2024-09-15T23:12:23.565 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.565 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T23:12:23.565 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-15T23:12:23.565 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2024-09-15T23:12:23.565 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-15T23:12:23.565 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 7125, 2024-09-15T23:12:23.566 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 884651513, 2024-09-15T23:12:23.566 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:23.566 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 7125, 2024-09-15T23:12:23.566 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-15T23:12:23.566 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-15T23:12:23.566 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 7125, 2024-09-15T23:12:23.566 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 884651513, 2024-09-15T23:12:23.566 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-15T23:12:23.566 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.566 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.566 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.566 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:23.566 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.567 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_bytes": { 2024-09-15T23:12:23.567 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-15T23:12:23.567 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2024-09-15T23:12:23.567 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-15T23:12:23.567 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-15T23:12:23.567 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-15T23:12:23.567 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:23.567 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-15T23:12:23.567 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 6643, 2024-09-15T23:12:23.567 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 882591407, 2024-09-15T23:12:23.567 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 6643, 2024-09-15T23:12:23.567 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 882591407, 2024-09-15T23:12:23.567 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-15T23:12:23.568 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.568 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.568 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.568 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:23.568 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.568 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_ops": { 2024-09-15T23:12:23.568 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-15T23:12:23.568 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 1024, 2024-09-15T23:12:23.568 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-15T23:12:23.568 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-15T23:12:23.568 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-15T23:12:23.568 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:23.568 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-15T23:12:23.568 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 6643, 2024-09-15T23:12:23.569 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 6643, 2024-09-15T23:12:23.569 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 6643, 2024-09-15T23:12:23.569 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 6643, 2024-09-15T23:12:23.569 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-15T23:12:23.569 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.569 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.569 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.569 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:23.569 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.569 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle": { 2024-09-15T23:12:23.569 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-15T23:12:23.569 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2024-09-15T23:12:23.569 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-15T23:12:23.570 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-15T23:12:23.570 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-15T23:12:23.570 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:23.570 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-15T23:12:23.570 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-15T23:12:23.570 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-15T23:12:23.570 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 0, 2024-09-15T23:12:23.570 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 0, 2024-09-15T23:12:23.570 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-15T23:12:23.570 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.570 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.570 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.570 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:23.570 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:23.570 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle-0x55db2d828b40": { 2024-09-15T23:12:23.571 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-15T23:12:23.571 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2024-09-15T23:12:23.571 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-15T23:12:23.571 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-15T23:12:23.571 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-15T23:12:23.571 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:23.571 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-15T23:12:23.571 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-15T23:12:23.571 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-15T23:12:23.571 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 0, 2024-09-15T23:12:23.571 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 0, 2024-09-15T23:12:23.571 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-15T23:12:23.571 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:23.571 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:23.572 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:23.572 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:23.572 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:23.572 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-15T23:12:24.354 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:24 smithi121 ceph-mon[34815]: pgmap v1145: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 125 KiB/s wr, 58 op/s 2024-09-15T23:12:24.382 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-15T23:12:24.382 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-15T23:12:24.382 DEBUG:tasks.check_counter:Getting stats from h 2024-09-15T23:12:24.383 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-15T23:12:24.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:24 smithi111 ceph-mon[30629]: pgmap v1145: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 125 KiB/s wr, 58 op/s 2024-09-15T23:12:24.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:24 smithi192 ceph-mon[35427]: pgmap v1145: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 125 KiB/s wr, 58 op/s 2024-09-15T23:12:24.637 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T23:12:25.196 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-15T23:12:25.196 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T23:12:25.196 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 773885, 2024-09-15T23:12:25.196 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 1271618, 2024-09-15T23:12:25.196 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 244347481, 2024-09-15T23:12:25.196 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 1418279972, 2024-09-15T23:12:25.197 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 14, 2024-09-15T23:12:25.197 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 5, 2024-09-15T23:12:25.197 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 38.749451880, 2024-09-15T23:12:25.197 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 19.165775259, 2024-09-15T23:12:25.197 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 24.440001806, 2024-09-15T23:12:25.197 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 2.915070603, 2024-09-15T23:12:25.197 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:25.197 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 1271617, 2024-09-15T23:12:25.197 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 41.800645009, 2024-09-15T23:12:25.197 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000032872 2024-09-15T23:12:25.197 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.197 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:25.198 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 945988, 2024-09-15T23:12:25.198 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.230917475, 2024-09-15T23:12:25.198 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000244 2024-09-15T23:12:25.198 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.198 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-15T23:12:25.198 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 448 2024-09-15T23:12:25.198 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.198 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T23:12:25.198 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 211483, 2024-09-15T23:12:25.198 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 266414, 2024-09-15T23:12:25.198 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 46893868, 2024-09-15T23:12:25.198 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 394482967, 2024-09-15T23:12:25.198 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 9, 2024-09-15T23:12:25.199 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 6, 2024-09-15T23:12:25.199 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 13.915021906, 2024-09-15T23:12:25.199 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 5.102654468, 2024-09-15T23:12:25.199 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 5.980173488, 2024-09-15T23:12:25.199 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 3.016900112, 2024-09-15T23:12:25.199 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:25.199 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 266412, 2024-09-15T23:12:25.199 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 10.273238936, 2024-09-15T23:12:25.199 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000038561 2024-09-15T23:12:25.199 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.199 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:25.199 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 313948, 2024-09-15T23:12:25.199 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.052012757, 2024-09-15T23:12:25.199 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000165 2024-09-15T23:12:25.200 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.200 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-15T23:12:25.200 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 778432 2024-09-15T23:12:25.200 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.200 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T23:12:25.200 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 190862, 2024-09-15T23:12:25.200 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 262749, 2024-09-15T23:12:25.200 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 47378106, 2024-09-15T23:12:25.200 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 244085975, 2024-09-15T23:12:25.200 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 7, 2024-09-15T23:12:25.200 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 6, 2024-09-15T23:12:25.200 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 16.780759118, 2024-09-15T23:12:25.200 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 8.344367820, 2024-09-15T23:12:25.201 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 6.800177400, 2024-09-15T23:12:25.201 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 2.172512033, 2024-09-15T23:12:25.201 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:25.201 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 262748, 2024-09-15T23:12:25.201 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 8.913483183, 2024-09-15T23:12:25.201 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000033924 2024-09-15T23:12:25.201 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.201 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:25.201 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 325030, 2024-09-15T23:12:25.201 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.056264433, 2024-09-15T23:12:25.201 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000173 2024-09-15T23:12:25.201 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.201 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 907360, 2024-09-15T23:12:25.201 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 287280 2024-09-15T23:12:25.202 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.202 INFO:teuthology.orchestra.run.smithi121.stdout: "cct": { 2024-09-15T23:12:25.202 INFO:teuthology.orchestra.run.smithi121.stdout: "total_workers": 1, 2024-09-15T23:12:25.202 INFO:teuthology.orchestra.run.smithi121.stdout: "unhealthy_workers": 0 2024-09-15T23:12:25.202 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.202 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-MDSRank": { 2024-09-15T23:12:25.202 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2024-09-15T23:12:25.202 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2024-09-15T23:12:25.202 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 212194, 2024-09-15T23:12:25.202 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 289.551680717, 2024-09-15T23:12:25.202 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.001364561 2024-09-15T23:12:25.202 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:25.202 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.203 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-PurgeQueue": { 2024-09-15T23:12:25.203 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2024-09-15T23:12:25.203 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2024-09-15T23:12:25.203 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 155400, 2024-09-15T23:12:25.203 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 27.088014985, 2024-09-15T23:12:25.203 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000174311 2024-09-15T23:12:25.203 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:25.203 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.203 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": { 2024-09-15T23:12:25.203 INFO:teuthology.orchestra.run.smithi121.stdout: "request": 510275, 2024-09-15T23:12:25.203 INFO:teuthology.orchestra.run.smithi121.stdout: "reply": 493994, 2024-09-15T23:12:25.203 INFO:teuthology.orchestra.run.smithi121.stdout: "reply_latency": { 2024-09-15T23:12:25.203 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 493994, 2024-09-15T23:12:25.203 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 2590.731014227, 2024-09-15T23:12:25.203 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.005244458 2024-09-15T23:12:25.204 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.204 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_reply": 0, 2024-09-15T23:12:25.204 INFO:teuthology.orchestra.run.smithi121.stdout: "forward": 16281, 2024-09-15T23:12:25.204 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_complete": 0, 2024-09-15T23:12:25.204 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_keys": 0, 2024-09-15T23:12:25.204 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_commit": 500, 2024-09-15T23:12:25.204 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_split": 40, 2024-09-15T23:12:25.204 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_merge": 46, 2024-09-15T23:12:25.204 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes": 5377, 2024-09-15T23:12:25.204 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_top": 2934, 2024-09-15T23:12:25.204 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_bottom": 1604, 2024-09-15T23:12:25.204 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pin_tail": 839, 2024-09-15T23:12:25.204 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pinned": 1165, 2024-09-15T23:12:25.205 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_expired": 436, 2024-09-15T23:12:25.205 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_with_caps": 0, 2024-09-15T23:12:25.205 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": 0, 2024-09-15T23:12:25.205 INFO:teuthology.orchestra.run.smithi121.stdout: "subtrees": 5, 2024-09-15T23:12:25.205 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse": 696698, 2024-09-15T23:12:25.205 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_hit": 419669, 2024-09-15T23:12:25.205 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_forward": 16279, 2024-09-15T23:12:25.205 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_discover": 2, 2024-09-15T23:12:25.205 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_dir_fetch": 0, 2024-09-15T23:12:25.205 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_remote_ino": 0, 2024-09-15T23:12:25.205 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_lock": 4244, 2024-09-15T23:12:25.205 INFO:teuthology.orchestra.run.smithi121.stdout: "load_cent": 0, 2024-09-15T23:12:25.205 INFO:teuthology.orchestra.run.smithi121.stdout: "q": 0, 2024-09-15T23:12:25.205 INFO:teuthology.orchestra.run.smithi121.stdout: "exported": 19, 2024-09-15T23:12:25.206 INFO:teuthology.orchestra.run.smithi121.stdout: "exported_inodes": 311, 2024-09-15T23:12:25.206 INFO:teuthology.orchestra.run.smithi121.stdout: "imported": 9, 2024-09-15T23:12:25.206 INFO:teuthology.orchestra.run.smithi121.stdout: "imported_inodes": 163, 2024-09-15T23:12:25.206 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_dir_fetch": 0, 2024-09-15T23:12:25.206 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_backtrace_fetch": 0, 2024-09-15T23:12:25.206 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_peer_discover": 1, 2024-09-15T23:12:25.206 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rfiles": 4, 2024-09-15T23:12:25.206 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rbytes": 234, 2024-09-15T23:12:25.206 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rsnaps": 0, 2024-09-15T23:12:25.206 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_fetch": 0, 2024-09-15T23:12:25.206 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_set_tag": 0, 2024-09-15T23:12:25.206 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T23:12:25.206 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_inotable_repaired": 0, 2024-09-15T23:12:25.206 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_inodes": 0, 2024-09-15T23:12:25.207 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_base_inodes": 0, 2024-09-15T23:12:25.207 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dirfrag_rstats": 0, 2024-09-15T23:12:25.207 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_file_inodes": 0, 2024-09-15T23:12:25.207 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_inode_file_caps": 20, 2024-09-15T23:12:25.207 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_revoke": 26376, 2024-09-15T23:12:25.207 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_grant": 377065, 2024-09-15T23:12:25.207 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T23:12:25.207 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T23:12:25.207 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T23:12:25.207 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps": 144330, 2024-09-15T23:12:25.207 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps_dirty": 69959, 2024-09-15T23:12:25.207 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_cap_release": 23919, 2024-09-15T23:12:25.207 INFO:teuthology.orchestra.run.smithi121.stdout: "process_request_cap_release": 305248 2024-09-15T23:12:25.207 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.207 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_cache": { 2024-09-15T23:12:25.208 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update": 18, 2024-09-15T23:12:25.208 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update_receipt": 6, 2024-09-15T23:12:25.208 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_try_discover": 4, 2024-09-15T23:12:25.208 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_send_discover": 5, 2024-09-15T23:12:25.208 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_handle_discover": 61, 2024-09-15T23:12:25.208 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays": 16, 2024-09-15T23:12:25.208 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_delayed": 0, 2024-09-15T23:12:25.208 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_enqueuing": 0, 2024-09-15T23:12:25.208 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_created": 92830, 2024-09-15T23:12:25.208 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_enqueued": 92814, 2024-09-15T23:12:25.208 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_reintegrated": 0, 2024-09-15T23:12:25.208 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_migrated": 0, 2024-09-15T23:12:25.208 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_processing": 0, 2024-09-15T23:12:25.208 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_enqueued": 0, 2024-09-15T23:12:25.209 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_prioritized": 0, 2024-09-15T23:12:25.209 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_started": 0, 2024-09-15T23:12:25.209 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_completed": 0, 2024-09-15T23:12:25.209 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_quiesce_path": 0, 2024-09-15T23:12:25.209 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_quiesce_inode": 0, 2024-09-15T23:12:25.209 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_enqueue_scrub": 0, 2024-09-15T23:12:25.209 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_exportdir": 19, 2024-09-15T23:12:25.209 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_flush": 0, 2024-09-15T23:12:25.209 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragmentdir": 86, 2024-09-15T23:12:25.209 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragstats": 0, 2024-09-15T23:12:25.209 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_inodestats": 0 2024-09-15T23:12:25.209 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.209 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_log": { 2024-09-15T23:12:25.209 INFO:teuthology.orchestra.run.smithi121.stdout: "evlrg": 0, 2024-09-15T23:12:25.209 INFO:teuthology.orchestra.run.smithi121.stdout: "evadd": 422836, 2024-09-15T23:12:25.210 INFO:teuthology.orchestra.run.smithi121.stdout: "evex": 313508, 2024-09-15T23:12:25.210 INFO:teuthology.orchestra.run.smithi121.stdout: "evtrm": 307251, 2024-09-15T23:12:25.210 INFO:teuthology.orchestra.run.smithi121.stdout: "ev": 115585, 2024-09-15T23:12:25.210 INFO:teuthology.orchestra.run.smithi121.stdout: "evexg": 0, 2024-09-15T23:12:25.210 INFO:teuthology.orchestra.run.smithi121.stdout: "evexd": 6257, 2024-09-15T23:12:25.210 INFO:teuthology.orchestra.run.smithi121.stdout: "segadd": 487, 2024-09-15T23:12:25.210 INFO:teuthology.orchestra.run.smithi121.stdout: "segex": 359, 2024-09-15T23:12:25.210 INFO:teuthology.orchestra.run.smithi121.stdout: "segtrm": 351, 2024-09-15T23:12:25.210 INFO:teuthology.orchestra.run.smithi121.stdout: "seg": 136, 2024-09-15T23:12:25.210 INFO:teuthology.orchestra.run.smithi121.stdout: "segmjr": 10, 2024-09-15T23:12:25.210 INFO:teuthology.orchestra.run.smithi121.stdout: "segexg": 0, 2024-09-15T23:12:25.210 INFO:teuthology.orchestra.run.smithi121.stdout: "segexd": 8, 2024-09-15T23:12:25.210 INFO:teuthology.orchestra.run.smithi121.stdout: "expos": 649922448, 2024-09-15T23:12:25.210 INFO:teuthology.orchestra.run.smithi121.stdout: "wrpos": 886770670, 2024-09-15T23:12:25.210 INFO:teuthology.orchestra.run.smithi121.stdout: "rdpos": 0, 2024-09-15T23:12:25.211 INFO:teuthology.orchestra.run.smithi121.stdout: "jlat": { 2024-09-15T23:12:25.211 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 120834, 2024-09-15T23:12:25.211 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 659.607278338, 2024-09-15T23:12:25.211 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.005458788 2024-09-15T23:12:25.211 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.211 INFO:teuthology.orchestra.run.smithi121.stdout: "replayed": 0 2024-09-15T23:12:25.211 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.211 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_mem": { 2024-09-15T23:12:25.211 INFO:teuthology.orchestra.run.smithi121.stdout: "ino": 39, 2024-09-15T23:12:25.211 INFO:teuthology.orchestra.run.smithi121.stdout: "ino+": 93079, 2024-09-15T23:12:25.211 INFO:teuthology.orchestra.run.smithi121.stdout: "ino-": 93040, 2024-09-15T23:12:25.211 INFO:teuthology.orchestra.run.smithi121.stdout: "dir": 96, 2024-09-15T23:12:25.211 INFO:teuthology.orchestra.run.smithi121.stdout: "dir+": 505, 2024-09-15T23:12:25.211 INFO:teuthology.orchestra.run.smithi121.stdout: "dir-": 409, 2024-09-15T23:12:25.211 INFO:teuthology.orchestra.run.smithi121.stdout: "dn": 5377, 2024-09-15T23:12:25.212 INFO:teuthology.orchestra.run.smithi121.stdout: "dn+": 182249, 2024-09-15T23:12:25.212 INFO:teuthology.orchestra.run.smithi121.stdout: "dn-": 176872, 2024-09-15T23:12:25.212 INFO:teuthology.orchestra.run.smithi121.stdout: "cap": 0, 2024-09-15T23:12:25.212 INFO:teuthology.orchestra.run.smithi121.stdout: "cap+": 96801, 2024-09-15T23:12:25.212 INFO:teuthology.orchestra.run.smithi121.stdout: "cap-": 96801, 2024-09-15T23:12:25.212 INFO:teuthology.orchestra.run.smithi121.stdout: "rss": 100336, 2024-09-15T23:12:25.212 INFO:teuthology.orchestra.run.smithi121.stdout: "heap": 223520 2024-09-15T23:12:25.212 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.212 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_server": { 2024-09-15T23:12:25.212 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_client_request": 643389, 2024-09-15T23:12:25.212 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_server_request": 6175, 2024-09-15T23:12:25.212 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_request": 510275, 2024-09-15T23:12:25.212 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_session": 12766, 2024-09-15T23:12:25.212 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_peer_request": 17655, 2024-09-15T23:12:25.213 INFO:teuthology.orchestra.run.smithi121.stdout: "req_create_latency": { 2024-09-15T23:12:25.213 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 92855, 2024-09-15T23:12:25.213 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 400.873060914, 2024-09-15T23:12:25.213 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.004317194 2024-09-15T23:12:25.213 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.213 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getattr_latency": { 2024-09-15T23:12:25.213 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 4865, 2024-09-15T23:12:25.213 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 113.192453401, 2024-09-15T23:12:25.213 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.023266691 2024-09-15T23:12:25.213 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.213 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getfilelock_latency": { 2024-09-15T23:12:25.213 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:25.213 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:25.213 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:25.213 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.214 INFO:teuthology.orchestra.run.smithi121.stdout: "req_link_latency": { 2024-09-15T23:12:25.214 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:25.214 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:25.214 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:25.214 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.214 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookup_latency": { 2024-09-15T23:12:25.214 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 140996, 2024-09-15T23:12:25.214 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 381.119626398, 2024-09-15T23:12:25.214 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.002703052 2024-09-15T23:12:25.214 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.214 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookuphash_latency": { 2024-09-15T23:12:25.214 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:25.214 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:25.214 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:25.214 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.215 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupino_latency": { 2024-09-15T23:12:25.215 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:25.215 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:25.215 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:25.215 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.215 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupname_latency": { 2024-09-15T23:12:25.215 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:25.215 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:25.215 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:25.215 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.215 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupparent_latency": { 2024-09-15T23:12:25.215 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:25.215 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:25.215 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:25.215 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.216 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupsnap_latency": { 2024-09-15T23:12:25.216 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:25.216 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:25.216 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:25.216 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.216 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lssnap_latency": { 2024-09-15T23:12:25.216 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:25.216 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:25.216 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:25.216 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.216 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mkdir_latency": { 2024-09-15T23:12:25.216 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 99, 2024-09-15T23:12:25.217 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.611936003, 2024-09-15T23:12:25.217 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.006181171 2024-09-15T23:12:25.217 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.217 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mknod_latency": { 2024-09-15T23:12:25.217 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:25.217 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:25.217 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:25.217 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.217 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mksnap_latency": { 2024-09-15T23:12:25.217 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:25.217 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:25.217 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:25.217 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.217 INFO:teuthology.orchestra.run.smithi121.stdout: "req_open_latency": { 2024-09-15T23:12:25.218 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 9, 2024-09-15T23:12:25.218 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.009166453, 2024-09-15T23:12:25.218 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.001018494 2024-09-15T23:12:25.218 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.218 INFO:teuthology.orchestra.run.smithi121.stdout: "req_readdir_latency": { 2024-09-15T23:12:25.218 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 140176, 2024-09-15T23:12:25.218 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 1019.337897429, 2024-09-15T23:12:25.218 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.007271843 2024-09-15T23:12:25.218 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.218 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rename_latency": { 2024-09-15T23:12:25.218 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 19043, 2024-09-15T23:12:25.218 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 78.994094074, 2024-09-15T23:12:25.218 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.004148195 2024-09-15T23:12:25.218 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.219 INFO:teuthology.orchestra.run.smithi121.stdout: "req_renamesnap_latency": { 2024-09-15T23:12:25.219 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:25.219 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:25.219 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:25.219 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.219 INFO:teuthology.orchestra.run.smithi121.stdout: "req_snapdiff_latency": { 2024-09-15T23:12:25.219 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:25.219 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:25.219 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:25.219 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.219 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmdir_latency": { 2024-09-15T23:12:25.219 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 96, 2024-09-15T23:12:25.219 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.655528606, 2024-09-15T23:12:25.219 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.006828422 2024-09-15T23:12:25.219 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.220 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmsnap_latency": { 2024-09-15T23:12:25.220 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:25.220 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:25.220 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:25.220 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.220 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmxattr_latency": { 2024-09-15T23:12:25.220 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:25.220 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:25.220 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:25.220 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.220 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setattr_latency": { 2024-09-15T23:12:25.220 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 134, 2024-09-15T23:12:25.220 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 2.843672125, 2024-09-15T23:12:25.220 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.021221433 2024-09-15T23:12:25.221 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.221 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setdirlayout_latency": { 2024-09-15T23:12:25.221 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:25.221 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:25.221 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:25.221 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.221 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setfilelock_latency": { 2024-09-15T23:12:25.221 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 2976, 2024-09-15T23:12:25.221 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 9.218762347, 2024-09-15T23:12:25.221 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.003097702 2024-09-15T23:12:25.221 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.221 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setlayout_latency": { 2024-09-15T23:12:25.221 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:25.221 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:25.222 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:25.222 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.222 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setxattr_latency": { 2024-09-15T23:12:25.222 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:25.222 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:25.222 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:25.222 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.222 INFO:teuthology.orchestra.run.smithi121.stdout: "req_symlink_latency": { 2024-09-15T23:12:25.222 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:25.222 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:25.222 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:25.222 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.222 INFO:teuthology.orchestra.run.smithi121.stdout: "req_unlink_latency": { 2024-09-15T23:12:25.222 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 92745, 2024-09-15T23:12:25.223 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 583.874816477, 2024-09-15T23:12:25.223 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.006295485 2024-09-15T23:12:25.223 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.223 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_revoke_eviction": 0, 2024-09-15T23:12:25.223 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_acquisition_throttle": 0, 2024-09-15T23:12:25.223 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getvxattr_latency": { 2024-09-15T23:12:25.223 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:25.223 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:25.223 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:25.223 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:25.223 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.223 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_sessions": { 2024-09-15T23:12:25.223 INFO:teuthology.orchestra.run.smithi121.stdout: "session_count": 1, 2024-09-15T23:12:25.223 INFO:teuthology.orchestra.run.smithi121.stdout: "session_add": 1, 2024-09-15T23:12:25.224 INFO:teuthology.orchestra.run.smithi121.stdout: "session_remove": 0, 2024-09-15T23:12:25.224 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_open": 1, 2024-09-15T23:12:25.224 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_stale": 0, 2024-09-15T23:12:25.224 INFO:teuthology.orchestra.run.smithi121.stdout: "total_load": 31600, 2024-09-15T23:12:25.224 INFO:teuthology.orchestra.run.smithi121.stdout: "average_load": 31600, 2024-09-15T23:12:25.224 INFO:teuthology.orchestra.run.smithi121.stdout: "avg_session_uptime": 1553, 2024-09-15T23:12:25.224 INFO:teuthology.orchestra.run.smithi121.stdout: "mdthresh_evicted": 0 2024-09-15T23:12:25.224 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.224 INFO:teuthology.orchestra.run.smithi121.stdout: "mempool": { 2024-09-15T23:12:25.224 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_bytes": 0, 2024-09-15T23:12:25.224 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_items": 0, 2024-09-15T23:12:25.224 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T23:12:25.224 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_items": 0, 2024-09-15T23:12:25.224 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T23:12:25.225 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_items": 0, 2024-09-15T23:12:25.225 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T23:12:25.225 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T23:12:25.225 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T23:12:25.225 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T23:12:25.225 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T23:12:25.225 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_items": 0, 2024-09-15T23:12:25.225 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T23:12:25.225 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T23:12:25.225 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_bytes": 0, 2024-09-15T23:12:25.225 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_items": 0, 2024-09-15T23:12:25.225 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_bytes": 0, 2024-09-15T23:12:25.225 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_items": 0, 2024-09-15T23:12:25.225 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T23:12:25.226 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T23:12:25.226 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T23:12:25.226 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T23:12:25.226 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T23:12:25.226 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_items": 0, 2024-09-15T23:12:25.226 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_bytes": 0, 2024-09-15T23:12:25.226 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_items": 0, 2024-09-15T23:12:25.226 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T23:12:25.226 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T23:12:25.226 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_bytes": 0, 2024-09-15T23:12:25.226 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_items": 0, 2024-09-15T23:12:25.226 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_bytes": 0, 2024-09-15T23:12:25.226 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_items": 0, 2024-09-15T23:12:25.226 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T23:12:25.227 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_items": 0, 2024-09-15T23:12:25.227 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T23:12:25.227 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_items": 0, 2024-09-15T23:12:25.227 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_bytes": 3059079, 2024-09-15T23:12:25.227 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_items": 828, 2024-09-15T23:12:25.227 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_bytes": 176, 2024-09-15T23:12:25.227 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_items": 2, 2024-09-15T23:12:25.227 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_bytes": 0, 2024-09-15T23:12:25.227 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_items": 0, 2024-09-15T23:12:25.227 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_bytes": 0, 2024-09-15T23:12:25.227 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_items": 0, 2024-09-15T23:12:25.227 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_bytes": 0, 2024-09-15T23:12:25.227 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_items": 0, 2024-09-15T23:12:25.227 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_bytes": 7304, 2024-09-15T23:12:25.228 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_items": 150, 2024-09-15T23:12:25.228 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T23:12:25.228 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_items": 0, 2024-09-15T23:12:25.228 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_bytes": 0, 2024-09-15T23:12:25.228 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_items": 0, 2024-09-15T23:12:25.228 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_bytes": 4313302, 2024-09-15T23:12:25.228 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_items": 61923, 2024-09-15T23:12:25.228 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_bytes": 0, 2024-09-15T23:12:25.228 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_items": 0, 2024-09-15T23:12:25.228 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_bytes": 0, 2024-09-15T23:12:25.228 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_items": 0 2024-09-15T23:12:25.228 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.228 INFO:teuthology.orchestra.run.smithi121.stdout: "objecter": { 2024-09-15T23:12:25.228 INFO:teuthology.orchestra.run.smithi121.stdout: "op_active": 0, 2024-09-15T23:12:25.228 INFO:teuthology.orchestra.run.smithi121.stdout: "op_laggy": 0, 2024-09-15T23:12:25.229 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send": 222441, 2024-09-15T23:12:25.229 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send_bytes": 905694773, 2024-09-15T23:12:25.229 INFO:teuthology.orchestra.run.smithi121.stdout: "op_resend": 28, 2024-09-15T23:12:25.229 INFO:teuthology.orchestra.run.smithi121.stdout: "op_reply": 222441, 2024-09-15T23:12:25.229 INFO:teuthology.orchestra.run.smithi121.stdout: "op_latency": { 2024-09-15T23:12:25.229 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 222441, 2024-09-15T23:12:25.229 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 1097.194100396, 2024-09-15T23:12:25.229 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.004932517 2024-09-15T23:12:25.229 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.229 INFO:teuthology.orchestra.run.smithi121.stdout: "op_inflight": 0, 2024-09-15T23:12:25.229 INFO:teuthology.orchestra.run.smithi121.stdout: "oplen_avg": { 2024-09-15T23:12:25.229 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 222441, 2024-09-15T23:12:25.229 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 226248 2024-09-15T23:12:25.229 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.230 INFO:teuthology.orchestra.run.smithi121.stdout: "op": 222441, 2024-09-15T23:12:25.230 INFO:teuthology.orchestra.run.smithi121.stdout: "op_r": 2400, 2024-09-15T23:12:25.230 INFO:teuthology.orchestra.run.smithi121.stdout: "op_w": 220041, 2024-09-15T23:12:25.230 INFO:teuthology.orchestra.run.smithi121.stdout: "op_rmw": 0, 2024-09-15T23:12:25.230 INFO:teuthology.orchestra.run.smithi121.stdout: "op_pg": 0, 2024-09-15T23:12:25.230 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_stat": 58, 2024-09-15T23:12:25.230 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_create": 478, 2024-09-15T23:12:25.230 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_read": 2400, 2024-09-15T23:12:25.230 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_write": 123234, 2024-09-15T23:12:25.230 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writefull": 1143, 2024-09-15T23:12:25.230 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writesame": 0, 2024-09-15T23:12:25.230 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_append": 0, 2024-09-15T23:12:25.230 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_zero": 0, 2024-09-15T23:12:25.230 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_truncate": 40, 2024-09-15T23:12:25.231 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_delete": 93463, 2024-09-15T23:12:25.231 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_mapext": 0, 2024-09-15T23:12:25.231 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_sparse_read": 0, 2024-09-15T23:12:25.231 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_clonerange": 0, 2024-09-15T23:12:25.231 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_getxattr": 0, 2024-09-15T23:12:25.231 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_setxattr": 956, 2024-09-15T23:12:25.231 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_cmpxattr": 0, 2024-09-15T23:12:25.231 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_rmxattr": 0, 2024-09-15T23:12:25.231 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_resetxattrs": 0, 2024-09-15T23:12:25.231 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_call": 0, 2024-09-15T23:12:25.231 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_watch": 0, 2024-09-15T23:12:25.231 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_notify": 0, 2024-09-15T23:12:25.231 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T23:12:25.231 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls": 0, 2024-09-15T23:12:25.231 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls_filter": 0, 2024-09-15T23:12:25.232 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_other": 0, 2024-09-15T23:12:25.232 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_active": 0, 2024-09-15T23:12:25.232 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_send": 0, 2024-09-15T23:12:25.232 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_resend": 0, 2024-09-15T23:12:25.232 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_ping": 0, 2024-09-15T23:12:25.232 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_active": 0, 2024-09-15T23:12:25.232 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_send": 0, 2024-09-15T23:12:25.232 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_resend": 0, 2024-09-15T23:12:25.232 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_active": 0, 2024-09-15T23:12:25.232 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_send": 0, 2024-09-15T23:12:25.232 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_resend": 0, 2024-09-15T23:12:25.232 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_active": 0, 2024-09-15T23:12:25.232 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_send": 0, 2024-09-15T23:12:25.232 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_resend": 0, 2024-09-15T23:12:25.232 INFO:teuthology.orchestra.run.smithi121.stdout: "command_active": 0, 2024-09-15T23:12:25.233 INFO:teuthology.orchestra.run.smithi121.stdout: "command_send": 0, 2024-09-15T23:12:25.233 INFO:teuthology.orchestra.run.smithi121.stdout: "command_resend": 0, 2024-09-15T23:12:25.233 INFO:teuthology.orchestra.run.smithi121.stdout: "map_epoch": 89, 2024-09-15T23:12:25.233 INFO:teuthology.orchestra.run.smithi121.stdout: "map_full": 0, 2024-09-15T23:12:25.233 INFO:teuthology.orchestra.run.smithi121.stdout: "map_inc": 3, 2024-09-15T23:12:25.233 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_sessions": 12, 2024-09-15T23:12:25.233 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_open": 12, 2024-09-15T23:12:25.233 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_close": 0, 2024-09-15T23:12:25.233 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_laggy": 0, 2024-09-15T23:12:25.233 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_wr": 3267, 2024-09-15T23:12:25.233 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_rd": 0, 2024-09-15T23:12:25.233 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_del": 1209 2024-09-15T23:12:25.233 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.233 INFO:teuthology.orchestra.run.smithi121.stdout: "oft": { 2024-09-15T23:12:25.233 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_objs": 1, 2024-09-15T23:12:25.234 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_kv_pairs": 0, 2024-09-15T23:12:25.234 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_updates": 0, 2024-09-15T23:12:25.234 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_removes": 0 2024-09-15T23:12:25.234 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.234 INFO:teuthology.orchestra.run.smithi121.stdout: "osd-slow-ops": { 2024-09-15T23:12:25.234 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_ops_count": 12 2024-09-15T23:12:25.234 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.234 INFO:teuthology.orchestra.run.smithi121.stdout: "purge_queue": { 2024-09-15T23:12:25.234 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops": 0, 2024-09-15T23:12:25.234 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops_high_water": 10, 2024-09-15T23:12:25.234 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing": 0, 2024-09-15T23:12:25.234 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_high_water": 10, 2024-09-15T23:12:25.234 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executed_ops": 92814, 2024-09-15T23:12:25.234 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executed": 92814, 2024-09-15T23:12:25.235 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_item_in_journal": 0 2024-09-15T23:12:25.235 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.235 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T23:12:25.235 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-15T23:12:25.235 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2024-09-15T23:12:25.235 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-15T23:12:25.235 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 1176226, 2024-09-15T23:12:25.235 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 243824404, 2024-09-15T23:12:25.235 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:25.235 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 1176226, 2024-09-15T23:12:25.235 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-15T23:12:25.235 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-15T23:12:25.235 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 1176226, 2024-09-15T23:12:25.235 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 243824404, 2024-09-15T23:12:25.235 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-15T23:12:25.236 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:25.236 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:25.236 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:25.236 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:25.236 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.236 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_bytes": { 2024-09-15T23:12:25.236 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-15T23:12:25.236 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2024-09-15T23:12:25.236 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-15T23:12:25.236 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-15T23:12:25.236 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-15T23:12:25.236 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:25.236 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-15T23:12:25.236 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 222441, 2024-09-15T23:12:25.237 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 915068987, 2024-09-15T23:12:25.237 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 222441, 2024-09-15T23:12:25.237 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 915068987, 2024-09-15T23:12:25.237 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-15T23:12:25.237 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:25.237 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:25.237 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:25.237 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:25.237 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.237 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_ops": { 2024-09-15T23:12:25.237 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-15T23:12:25.237 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 1024, 2024-09-15T23:12:25.237 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-15T23:12:25.238 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-15T23:12:25.238 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-15T23:12:25.238 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:25.238 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-15T23:12:25.238 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 222441, 2024-09-15T23:12:25.238 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 222441, 2024-09-15T23:12:25.238 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 222441, 2024-09-15T23:12:25.238 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 222441, 2024-09-15T23:12:25.238 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-15T23:12:25.238 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:25.238 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:25.238 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:25.238 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:25.238 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.238 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle": { 2024-09-15T23:12:25.238 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-15T23:12:25.239 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2024-09-15T23:12:25.239 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-15T23:12:25.239 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 92814, 2024-09-15T23:12:25.239 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 9374214, 2024-09-15T23:12:25.239 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:25.239 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 92814, 2024-09-15T23:12:25.239 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-15T23:12:25.239 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-15T23:12:25.239 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 2400, 2024-09-15T23:12:25.239 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 9374214, 2024-09-15T23:12:25.239 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-15T23:12:25.239 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:25.239 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:25.239 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:25.240 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:25.240 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:25.240 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle-0x563356734960": { 2024-09-15T23:12:25.240 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-15T23:12:25.240 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2024-09-15T23:12:25.240 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-15T23:12:25.240 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 422836, 2024-09-15T23:12:25.240 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 882576366, 2024-09-15T23:12:25.240 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:25.240 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 422836, 2024-09-15T23:12:25.240 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-15T23:12:25.240 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-15T23:12:25.240 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 120834, 2024-09-15T23:12:25.240 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 882576366, 2024-09-15T23:12:25.241 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-15T23:12:25.241 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:25.241 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:25.241 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:25.241 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:25.241 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:25.241 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-15T23:12:25.966 INFO:tasks.check_counter:Daemon mds.h mds.exported=19 2024-09-15T23:12:25.967 INFO:tasks.check_counter:Daemon mds.h mds.imported=9 2024-09-15T23:12:25.967 DEBUG:tasks.check_counter:Getting stats from k 2024-09-15T23:12:25.967 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-15T23:12:26.212 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T23:12:26.332 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:26 smithi121 ceph-mon[34815]: pgmap v1146: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 124 KiB/s wr, 47 op/s 2024-09-15T23:12:26.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:26 smithi111 ceph-mon[30629]: pgmap v1146: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 124 KiB/s wr, 47 op/s 2024-09-15T23:12:26.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:26 smithi192 ceph-mon[35427]: pgmap v1146: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 124 KiB/s wr, 47 op/s 2024-09-15T23:12:26.711 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-15T23:12:26.712 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T23:12:26.712 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 469, 2024-09-15T23:12:26.712 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 462, 2024-09-15T23:12:26.712 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 829856, 2024-09-15T23:12:26.712 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 285040, 2024-09-15T23:12:26.712 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 4, 2024-09-15T23:12:26.712 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 0, 2024-09-15T23:12:26.712 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.057194530, 2024-09-15T23:12:26.712 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.025730494, 2024-09-15T23:12:26.712 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.037155563, 2024-09-15T23:12:26.712 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T23:12:26.713 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:26.713 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 462, 2024-09-15T23:12:26.713 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.026067318, 2024-09-15T23:12:26.713 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000056422 2024-09-15T23:12:26.713 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:26.713 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:26.713 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:26.713 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:26.713 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:26.713 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:26.713 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 829856, 2024-09-15T23:12:26.713 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 285040 2024-09-15T23:12:26.713 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:26.714 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T23:12:26.714 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 2, 2024-09-15T23:12:26.714 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 351, 2024-09-15T23:12:26.714 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 320, 2024-09-15T23:12:26.714 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 219216, 2024-09-15T23:12:26.714 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 4, 2024-09-15T23:12:26.714 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 2, 2024-09-15T23:12:26.714 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.017326018, 2024-09-15T23:12:26.714 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.012846793, 2024-09-15T23:12:26.714 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.001525032, 2024-09-15T23:12:26.714 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T23:12:26.714 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:26.715 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 349, 2024-09-15T23:12:26.715 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.016811220, 2024-09-15T23:12:26.715 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000048169 2024-09-15T23:12:26.715 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:26.715 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:26.715 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:26.715 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:26.715 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:26.715 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:26.715 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-15T23:12:26.715 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 219216 2024-09-15T23:12:26.715 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:26.715 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T23:12:26.715 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 0, 2024-09-15T23:12:26.716 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 2, 2024-09-15T23:12:26.716 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 0, 2024-09-15T23:12:26.716 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 0, 2024-09-15T23:12:26.716 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 2, 2024-09-15T23:12:26.716 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 1, 2024-09-15T23:12:26.716 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.000301337, 2024-09-15T23:12:26.716 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.000000000, 2024-09-15T23:12:26.716 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.000209004, 2024-09-15T23:12:26.716 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T23:12:26.716 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:26.716 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:26.716 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:26.716 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:26.716 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:26.717 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:26.717 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:26.717 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:26.717 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:26.717 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:26.717 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-15T23:12:26.717 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 0 2024-09-15T23:12:26.717 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:26.717 INFO:teuthology.orchestra.run.smithi121.stdout: "cct": { 2024-09-15T23:12:26.717 INFO:teuthology.orchestra.run.smithi121.stdout: "total_workers": 0, 2024-09-15T23:12:26.717 INFO:teuthology.orchestra.run.smithi121.stdout: "unhealthy_workers": 0 2024-09-15T23:12:26.717 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:26.717 INFO:teuthology.orchestra.run.smithi121.stdout: "mempool": { 2024-09-15T23:12:26.718 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_bytes": 0, 2024-09-15T23:12:26.718 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_items": 0, 2024-09-15T23:12:26.718 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T23:12:26.718 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_items": 0, 2024-09-15T23:12:26.718 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T23:12:26.718 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_items": 0, 2024-09-15T23:12:26.718 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T23:12:26.718 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T23:12:26.718 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T23:12:26.718 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T23:12:26.718 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T23:12:26.718 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_items": 0, 2024-09-15T23:12:26.718 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T23:12:26.718 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T23:12:26.719 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_bytes": 0, 2024-09-15T23:12:26.719 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_items": 0, 2024-09-15T23:12:26.719 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_bytes": 0, 2024-09-15T23:12:26.719 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_items": 0, 2024-09-15T23:12:26.719 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T23:12:26.719 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T23:12:26.719 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T23:12:26.719 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T23:12:26.719 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T23:12:26.719 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_items": 0, 2024-09-15T23:12:26.719 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_bytes": 0, 2024-09-15T23:12:26.719 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_items": 0, 2024-09-15T23:12:26.719 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T23:12:26.719 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T23:12:26.720 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_bytes": 0, 2024-09-15T23:12:26.720 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_items": 0, 2024-09-15T23:12:26.720 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_bytes": 0, 2024-09-15T23:12:26.720 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_items": 0, 2024-09-15T23:12:26.720 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T23:12:26.720 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_items": 0, 2024-09-15T23:12:26.720 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T23:12:26.720 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_items": 0, 2024-09-15T23:12:26.720 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_bytes": 148744, 2024-09-15T23:12:26.720 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_items": 30, 2024-09-15T23:12:26.720 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_bytes": 0, 2024-09-15T23:12:26.721 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_items": 0, 2024-09-15T23:12:26.721 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_bytes": 0, 2024-09-15T23:12:26.721 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_items": 0, 2024-09-15T23:12:26.721 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_bytes": 0, 2024-09-15T23:12:26.721 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_items": 0, 2024-09-15T23:12:26.721 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_bytes": 0, 2024-09-15T23:12:26.721 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_items": 0, 2024-09-15T23:12:26.721 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_bytes": 0, 2024-09-15T23:12:26.721 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_items": 0, 2024-09-15T23:12:26.721 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T23:12:26.721 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_items": 0, 2024-09-15T23:12:26.721 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_bytes": 0, 2024-09-15T23:12:26.721 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_items": 0, 2024-09-15T23:12:26.722 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_bytes": 928, 2024-09-15T23:12:26.722 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_items": 2, 2024-09-15T23:12:26.722 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_bytes": 0, 2024-09-15T23:12:26.722 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_items": 0, 2024-09-15T23:12:26.722 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_bytes": 0, 2024-09-15T23:12:26.722 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_items": 0 2024-09-15T23:12:26.722 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-15T23:12:26.722 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T23:12:26.722 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-15T23:12:26.722 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2024-09-15T23:12:26.722 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-15T23:12:26.722 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 467, 2024-09-15T23:12:26.722 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 761061, 2024-09-15T23:12:26.722 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:26.722 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 467, 2024-09-15T23:12:26.723 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-15T23:12:26.723 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-15T23:12:26.723 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 467, 2024-09-15T23:12:26.723 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 761061, 2024-09-15T23:12:26.723 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-15T23:12:26.723 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-15T23:12:26.723 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-15T23:12:26.723 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-15T23:12:26.723 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:26.723 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-15T23:12:26.723 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-15T23:12:27.289 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-15T23:12:27.289 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-15T23:12:27.289 DEBUG:tasks.check_counter:Getting stats from c 2024-09-15T23:12:27.290 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-15T23:12:27.548 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T23:12:28.024 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T23:12:28.024 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T23:12:28.024 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 721, 2024-09-15T23:12:28.024 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 716, 2024-09-15T23:12:28.024 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 58736064, 2024-09-15T23:12:28.024 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 277254, 2024-09-15T23:12:28.024 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 7, 2024-09-15T23:12:28.024 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 3, 2024-09-15T23:12:28.024 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.090078789, 2024-09-15T23:12:28.024 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.023791053, 2024-09-15T23:12:28.024 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.050071332, 2024-09-15T23:12:28.025 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.012985067, 2024-09-15T23:12:28.025 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:28.025 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 715, 2024-09-15T23:12:28.025 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.025124889, 2024-09-15T23:12:28.025 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000035139 2024-09-15T23:12:28.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.025 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:28.025 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.025 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.025 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.025 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 220848, 2024-09-15T23:12:28.025 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 106160 2024-09-15T23:12:28.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.026 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T23:12:28.026 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 1301, 2024-09-15T23:12:28.026 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 1428, 2024-09-15T23:12:28.026 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 75342301, 2024-09-15T23:12:28.026 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 726957, 2024-09-15T23:12:28.026 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 7, 2024-09-15T23:12:28.026 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 6, 2024-09-15T23:12:28.026 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.156275374, 2024-09-15T23:12:28.026 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.035024354, 2024-09-15T23:12:28.026 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.075408883, 2024-09-15T23:12:28.026 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.035760000, 2024-09-15T23:12:28.026 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1427, 2024-09-15T23:12:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.055977158, 2024-09-15T23:12:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000039227 2024-09-15T23:12:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-15T23:12:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 314768 2024-09-15T23:12:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T23:12:28.027 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 567, 2024-09-15T23:12:28.028 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 569, 2024-09-15T23:12:28.028 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 67253091, 2024-09-15T23:12:28.028 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 180312, 2024-09-15T23:12:28.028 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 6, 2024-09-15T23:12:28.028 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 5, 2024-09-15T23:12:28.028 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.074244186, 2024-09-15T23:12:28.028 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.013502588, 2024-09-15T23:12:28.028 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.044399925, 2024-09-15T23:12:28.028 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.012667362, 2024-09-15T23:12:28.028 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:28.028 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 567, 2024-09-15T23:12:28.028 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.021590964, 2024-09-15T23:12:28.028 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000038079 2024-09-15T23:12:28.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.029 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:28.029 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.029 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.029 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.029 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-15T23:12:28.029 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 0 2024-09-15T23:12:28.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.029 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2024-09-15T23:12:28.029 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 1, 2024-09-15T23:12:28.029 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2024-09-15T23:12:28.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.029 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-MDSRank": { 2024-09-15T23:12:28.030 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-09-15T23:12:28.030 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-09-15T23:12:28.030 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 4008, 2024-09-15T23:12:28.030 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.253062739, 2024-09-15T23:12:28.030 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000063139 2024-09-15T23:12:28.030 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:28.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.030 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-PurgeQueue": { 2024-09-15T23:12:28.030 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-09-15T23:12:28.030 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-09-15T23:12:28.030 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.030 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.030 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.030 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:28.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.031 INFO:teuthology.orchestra.run.smithi192.stdout: "mds": { 2024-09-15T23:12:28.031 INFO:teuthology.orchestra.run.smithi192.stdout: "request": 0, 2024-09-15T23:12:28.031 INFO:teuthology.orchestra.run.smithi192.stdout: "reply": 0, 2024-09-15T23:12:28.031 INFO:teuthology.orchestra.run.smithi192.stdout: "reply_latency": { 2024-09-15T23:12:28.031 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.031 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.031 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.031 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_reply": 0, 2024-09-15T23:12:28.031 INFO:teuthology.orchestra.run.smithi192.stdout: "forward": 0, 2024-09-15T23:12:28.031 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_complete": 0, 2024-09-15T23:12:28.031 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_keys": 0, 2024-09-15T23:12:28.031 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_commit": 0, 2024-09-15T23:12:28.031 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_split": 0, 2024-09-15T23:12:28.032 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_merge": 0, 2024-09-15T23:12:28.032 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes": 19404, 2024-09-15T23:12:28.032 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_top": 13371, 2024-09-15T23:12:28.032 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_bottom": 6033, 2024-09-15T23:12:28.032 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pin_tail": 0, 2024-09-15T23:12:28.032 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pinned": 302, 2024-09-15T23:12:28.032 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_expired": 0, 2024-09-15T23:12:28.032 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_with_caps": 0, 2024-09-15T23:12:28.032 INFO:teuthology.orchestra.run.smithi192.stdout: "caps": 0, 2024-09-15T23:12:28.032 INFO:teuthology.orchestra.run.smithi192.stdout: "subtrees": 2, 2024-09-15T23:12:28.032 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse": 0, 2024-09-15T23:12:28.032 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_hit": 0, 2024-09-15T23:12:28.032 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_forward": 0, 2024-09-15T23:12:28.033 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_discover": 0, 2024-09-15T23:12:28.033 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_dir_fetch": 0, 2024-09-15T23:12:28.033 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_remote_ino": 0, 2024-09-15T23:12:28.033 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_lock": 0, 2024-09-15T23:12:28.033 INFO:teuthology.orchestra.run.smithi192.stdout: "load_cent": 0, 2024-09-15T23:12:28.033 INFO:teuthology.orchestra.run.smithi192.stdout: "q": 0, 2024-09-15T23:12:28.033 INFO:teuthology.orchestra.run.smithi192.stdout: "exported": 0, 2024-09-15T23:12:28.033 INFO:teuthology.orchestra.run.smithi192.stdout: "exported_inodes": 0, 2024-09-15T23:12:28.033 INFO:teuthology.orchestra.run.smithi192.stdout: "imported": 0, 2024-09-15T23:12:28.033 INFO:teuthology.orchestra.run.smithi192.stdout: "imported_inodes": 0, 2024-09-15T23:12:28.033 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_dir_fetch": 0, 2024-09-15T23:12:28.033 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_backtrace_fetch": 0, 2024-09-15T23:12:28.033 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_peer_discover": 0, 2024-09-15T23:12:28.033 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rfiles": 4, 2024-09-15T23:12:28.034 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rbytes": 234, 2024-09-15T23:12:28.034 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rsnaps": 0, 2024-09-15T23:12:28.034 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_fetch": 0, 2024-09-15T23:12:28.034 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_set_tag": 0, 2024-09-15T23:12:28.034 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T23:12:28.034 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_inotable_repaired": 0, 2024-09-15T23:12:28.034 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_inodes": 0, 2024-09-15T23:12:28.034 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_base_inodes": 0, 2024-09-15T23:12:28.034 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dirfrag_rstats": 0, 2024-09-15T23:12:28.034 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_file_inodes": 0, 2024-09-15T23:12:28.034 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_inode_file_caps": 0, 2024-09-15T23:12:28.034 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_revoke": 0, 2024-09-15T23:12:28.034 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_grant": 0, 2024-09-15T23:12:28.034 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T23:12:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T23:12:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T23:12:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps": 0, 2024-09-15T23:12:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps_dirty": 0, 2024-09-15T23:12:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_cap_release": 0, 2024-09-15T23:12:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: "process_request_cap_release": 0 2024-09-15T23:12:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_cache": { 2024-09-15T23:12:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update": 0, 2024-09-15T23:12:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update_receipt": 0, 2024-09-15T23:12:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_try_discover": 0, 2024-09-15T23:12:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_send_discover": 0, 2024-09-15T23:12:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_handle_discover": 0, 2024-09-15T23:12:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays": 0, 2024-09-15T23:12:28.035 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_delayed": 0, 2024-09-15T23:12:28.036 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_enqueuing": 0, 2024-09-15T23:12:28.036 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_created": 0, 2024-09-15T23:12:28.036 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_enqueued": 0, 2024-09-15T23:12:28.036 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_reintegrated": 0, 2024-09-15T23:12:28.036 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_migrated": 0, 2024-09-15T23:12:28.036 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_processing": 0, 2024-09-15T23:12:28.036 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_enqueued": 0, 2024-09-15T23:12:28.036 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_prioritized": 0, 2024-09-15T23:12:28.036 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_started": 0, 2024-09-15T23:12:28.036 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_completed": 0, 2024-09-15T23:12:28.036 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_quiesce_path": 0, 2024-09-15T23:12:28.036 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_quiesce_inode": 0, 2024-09-15T23:12:28.036 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_enqueue_scrub": 0, 2024-09-15T23:12:28.037 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_exportdir": 0, 2024-09-15T23:12:28.037 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_flush": 0, 2024-09-15T23:12:28.037 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragmentdir": 0, 2024-09-15T23:12:28.037 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragstats": 0, 2024-09-15T23:12:28.037 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_inodestats": 0 2024-09-15T23:12:28.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.037 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_log": { 2024-09-15T23:12:28.037 INFO:teuthology.orchestra.run.smithi192.stdout: "evlrg": 0, 2024-09-15T23:12:28.037 INFO:teuthology.orchestra.run.smithi192.stdout: "evadd": 0, 2024-09-15T23:12:28.037 INFO:teuthology.orchestra.run.smithi192.stdout: "evex": 0, 2024-09-15T23:12:28.037 INFO:teuthology.orchestra.run.smithi192.stdout: "evtrm": 0, 2024-09-15T23:12:28.037 INFO:teuthology.orchestra.run.smithi192.stdout: "ev": 99388, 2024-09-15T23:12:28.037 INFO:teuthology.orchestra.run.smithi192.stdout: "evexg": 0, 2024-09-15T23:12:28.037 INFO:teuthology.orchestra.run.smithi192.stdout: "evexd": 0, 2024-09-15T23:12:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: "segadd": 0, 2024-09-15T23:12:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: "segex": 0, 2024-09-15T23:12:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: "segtrm": 0, 2024-09-15T23:12:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: "seg": 2, 2024-09-15T23:12:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: "segmjr": 46, 2024-09-15T23:12:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: "segexg": 0, 2024-09-15T23:12:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: "segexd": 0, 2024-09-15T23:12:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: "expos": 258726535, 2024-09-15T23:12:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: "wrpos": 259506318, 2024-09-15T23:12:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: "rdpos": 259505180, 2024-09-15T23:12:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: "jlat": { 2024-09-15T23:12:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: "replayed": 99388 2024-09-15T23:12:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_mem": { 2024-09-15T23:12:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: "ino": 18, 2024-09-15T23:12:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: "ino+": 19223, 2024-09-15T23:12:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: "ino-": 19205, 2024-09-15T23:12:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: "dir": 45, 2024-09-15T23:12:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: "dir+": 224, 2024-09-15T23:12:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: "dir-": 179, 2024-09-15T23:12:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: "dn": 19404, 2024-09-15T23:12:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: "dn+": 19711, 2024-09-15T23:12:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: "dn-": 307, 2024-09-15T23:12:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: "cap": 0, 2024-09-15T23:12:28.039 INFO:teuthology.orchestra.run.smithi192.stdout: "cap+": 0, 2024-09-15T23:12:28.040 INFO:teuthology.orchestra.run.smithi192.stdout: "cap-": 0, 2024-09-15T23:12:28.040 INFO:teuthology.orchestra.run.smithi192.stdout: "rss": 65372, 2024-09-15T23:12:28.040 INFO:teuthology.orchestra.run.smithi192.stdout: "heap": 215328 2024-09-15T23:12:28.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.040 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_server": { 2024-09-15T23:12:28.040 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_client_request": 0, 2024-09-15T23:12:28.040 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_server_request": 0, 2024-09-15T23:12:28.040 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_request": 0, 2024-09-15T23:12:28.040 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_session": 0, 2024-09-15T23:12:28.040 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_peer_request": 0, 2024-09-15T23:12:28.040 INFO:teuthology.orchestra.run.smithi192.stdout: "req_create_latency": { 2024-09-15T23:12:28.040 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.040 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.040 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getattr_latency": { 2024-09-15T23:12:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getfilelock_latency": { 2024-09-15T23:12:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: "req_link_latency": { 2024-09-15T23:12:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.041 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.042 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookup_latency": { 2024-09-15T23:12:28.042 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.042 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.042 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.042 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookuphash_latency": { 2024-09-15T23:12:28.042 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.042 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.042 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.042 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupino_latency": { 2024-09-15T23:12:28.042 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.042 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupname_latency": { 2024-09-15T23:12:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupparent_latency": { 2024-09-15T23:12:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupsnap_latency": { 2024-09-15T23:12:28.043 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lssnap_latency": { 2024-09-15T23:12:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mkdir_latency": { 2024-09-15T23:12:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mknod_latency": { 2024-09-15T23:12:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mksnap_latency": { 2024-09-15T23:12:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: "req_open_latency": { 2024-09-15T23:12:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.045 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.046 INFO:teuthology.orchestra.run.smithi192.stdout: "req_readdir_latency": { 2024-09-15T23:12:28.046 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.046 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.046 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.046 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rename_latency": { 2024-09-15T23:12:28.046 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.046 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.046 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.046 INFO:teuthology.orchestra.run.smithi192.stdout: "req_renamesnap_latency": { 2024-09-15T23:12:28.046 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.047 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.047 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.047 INFO:teuthology.orchestra.run.smithi192.stdout: "req_snapdiff_latency": { 2024-09-15T23:12:28.047 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.047 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.047 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.047 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmdir_latency": { 2024-09-15T23:12:28.047 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.047 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.047 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.047 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmsnap_latency": { 2024-09-15T23:12:28.048 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.048 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.048 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.048 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmxattr_latency": { 2024-09-15T23:12:28.048 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.048 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.048 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.048 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setattr_latency": { 2024-09-15T23:12:28.048 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.048 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.048 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setdirlayout_latency": { 2024-09-15T23:12:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setfilelock_latency": { 2024-09-15T23:12:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setlayout_latency": { 2024-09-15T23:12:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.049 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setxattr_latency": { 2024-09-15T23:12:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: "req_symlink_latency": { 2024-09-15T23:12:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: "req_unlink_latency": { 2024-09-15T23:12:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.050 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.051 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.051 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_revoke_eviction": 0, 2024-09-15T23:12:28.051 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_acquisition_throttle": 0, 2024-09-15T23:12:28.051 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getvxattr_latency": { 2024-09-15T23:12:28.051 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.051 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.051 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.051 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:28.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.051 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_sessions": { 2024-09-15T23:12:28.051 INFO:teuthology.orchestra.run.smithi192.stdout: "session_count": 1, 2024-09-15T23:12:28.051 INFO:teuthology.orchestra.run.smithi192.stdout: "session_add": 1, 2024-09-15T23:12:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: "session_remove": 0, 2024-09-15T23:12:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_open": 0, 2024-09-15T23:12:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_stale": 0, 2024-09-15T23:12:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: "total_load": 0, 2024-09-15T23:12:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: "average_load": 0, 2024-09-15T23:12:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: "avg_session_uptime": 2941, 2024-09-15T23:12:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: "mdthresh_evicted": 0 2024-09-15T23:12:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2024-09-15T23:12:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2024-09-15T23:12:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2024-09-15T23:12:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T23:12:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2024-09-15T23:12:28.052 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T23:12:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2024-09-15T23:12:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T23:12:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T23:12:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T23:12:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T23:12:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T23:12:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2024-09-15T23:12:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T23:12:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T23:12:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2024-09-15T23:12:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2024-09-15T23:12:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2024-09-15T23:12:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2024-09-15T23:12:28.053 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T23:12:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T23:12:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T23:12:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T23:12:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T23:12:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2024-09-15T23:12:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2024-09-15T23:12:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2024-09-15T23:12:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T23:12:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T23:12:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2024-09-15T23:12:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2024-09-15T23:12:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2024-09-15T23:12:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2024-09-15T23:12:28.054 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T23:12:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2024-09-15T23:12:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T23:12:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2024-09-15T23:12:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 238684, 2024-09-15T23:12:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 77, 2024-09-15T23:12:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 0, 2024-09-15T23:12:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 0, 2024-09-15T23:12:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2024-09-15T23:12:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2024-09-15T23:12:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2024-09-15T23:12:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2024-09-15T23:12:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2024-09-15T23:12:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2024-09-15T23:12:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 7304, 2024-09-15T23:12:28.055 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 150, 2024-09-15T23:12:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T23:12:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2024-09-15T23:12:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2024-09-15T23:12:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2024-09-15T23:12:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 13761027, 2024-09-15T23:12:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 77855, 2024-09-15T23:12:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2024-09-15T23:12:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2024-09-15T23:12:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2024-09-15T23:12:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2024-09-15T23:12:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: "objecter": { 2024-09-15T23:12:28.056 INFO:teuthology.orchestra.run.smithi192.stdout: "op_active": 0, 2024-09-15T23:12:28.057 INFO:teuthology.orchestra.run.smithi192.stdout: "op_laggy": 0, 2024-09-15T23:12:28.057 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send": 2401, 2024-09-15T23:12:28.057 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send_bytes": 26, 2024-09-15T23:12:28.057 INFO:teuthology.orchestra.run.smithi192.stdout: "op_resend": 4, 2024-09-15T23:12:28.057 INFO:teuthology.orchestra.run.smithi192.stdout: "op_reply": 2401, 2024-09-15T23:12:28.057 INFO:teuthology.orchestra.run.smithi192.stdout: "op_latency": { 2024-09-15T23:12:28.057 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 2401, 2024-09-15T23:12:28.057 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 4.040920554, 2024-09-15T23:12:28.057 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.001683015 2024-09-15T23:12:28.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.057 INFO:teuthology.orchestra.run.smithi192.stdout: "op_inflight": 0, 2024-09-15T23:12:28.057 INFO:teuthology.orchestra.run.smithi192.stdout: "oplen_avg": { 2024-09-15T23:12:28.057 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 2401, 2024-09-15T23:12:28.057 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 2402 2024-09-15T23:12:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: "op": 2401, 2024-09-15T23:12:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: "op_r": 2401, 2024-09-15T23:12:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: "op_w": 0, 2024-09-15T23:12:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: "op_rmw": 0, 2024-09-15T23:12:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: "op_pg": 0, 2024-09-15T23:12:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_stat": 1304, 2024-09-15T23:12:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_create": 0, 2024-09-15T23:12:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_read": 1094, 2024-09-15T23:12:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_write": 0, 2024-09-15T23:12:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writefull": 0, 2024-09-15T23:12:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writesame": 0, 2024-09-15T23:12:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_append": 0, 2024-09-15T23:12:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_zero": 0, 2024-09-15T23:12:28.058 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_truncate": 0, 2024-09-15T23:12:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_delete": 0, 2024-09-15T23:12:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_mapext": 0, 2024-09-15T23:12:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_sparse_read": 0, 2024-09-15T23:12:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_clonerange": 0, 2024-09-15T23:12:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_getxattr": 2, 2024-09-15T23:12:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_setxattr": 0, 2024-09-15T23:12:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_cmpxattr": 0, 2024-09-15T23:12:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_rmxattr": 0, 2024-09-15T23:12:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_resetxattrs": 0, 2024-09-15T23:12:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_call": 0, 2024-09-15T23:12:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_watch": 0, 2024-09-15T23:12:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_notify": 0, 2024-09-15T23:12:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T23:12:28.059 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls": 0, 2024-09-15T23:12:28.060 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls_filter": 0, 2024-09-15T23:12:28.060 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_other": 0, 2024-09-15T23:12:28.060 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_active": 0, 2024-09-15T23:12:28.060 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_send": 0, 2024-09-15T23:12:28.060 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_resend": 0, 2024-09-15T23:12:28.060 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_ping": 0, 2024-09-15T23:12:28.060 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_active": 0, 2024-09-15T23:12:28.060 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_send": 0, 2024-09-15T23:12:28.060 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_resend": 0, 2024-09-15T23:12:28.060 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_active": 0, 2024-09-15T23:12:28.060 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_send": 0, 2024-09-15T23:12:28.060 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_resend": 0, 2024-09-15T23:12:28.060 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_active": 0, 2024-09-15T23:12:28.060 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_send": 0, 2024-09-15T23:12:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_resend": 0, 2024-09-15T23:12:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: "command_active": 0, 2024-09-15T23:12:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: "command_send": 0, 2024-09-15T23:12:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: "command_resend": 0, 2024-09-15T23:12:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: "map_epoch": 0, 2024-09-15T23:12:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: "map_full": 0, 2024-09-15T23:12:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: "map_inc": 0, 2024-09-15T23:12:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_sessions": 11, 2024-09-15T23:12:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_open": 11, 2024-09-15T23:12:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_close": 0, 2024-09-15T23:12:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_laggy": 0, 2024-09-15T23:12:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_wr": 0, 2024-09-15T23:12:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_rd": 2, 2024-09-15T23:12:28.061 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_del": 0 2024-09-15T23:12:28.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.062 INFO:teuthology.orchestra.run.smithi192.stdout: "oft": { 2024-09-15T23:12:28.062 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_objs": 0, 2024-09-15T23:12:28.062 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_kv_pairs": 0, 2024-09-15T23:12:28.062 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_updates": 0, 2024-09-15T23:12:28.062 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_removes": 0 2024-09-15T23:12:28.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.062 INFO:teuthology.orchestra.run.smithi192.stdout: "osd-slow-ops": { 2024-09-15T23:12:28.062 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_ops_count": 0 2024-09-15T23:12:28.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.062 INFO:teuthology.orchestra.run.smithi192.stdout: "purge_queue": { 2024-09-15T23:12:28.062 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops": 0, 2024-09-15T23:12:28.062 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops_high_water": 0, 2024-09-15T23:12:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing": 0, 2024-09-15T23:12:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_high_water": 0, 2024-09-15T23:12:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed_ops": 0, 2024-09-15T23:12:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed": 0, 2024-09-15T23:12:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_item_in_journal": 0 2024-09-15T23:12:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T23:12:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-15T23:12:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-09-15T23:12:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-15T23:12:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 2585, 2024-09-15T23:12:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 201086376, 2024-09-15T23:12:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:28.063 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 2585, 2024-09-15T23:12:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-09-15T23:12:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-09-15T23:12:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 2585, 2024-09-15T23:12:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 201086376, 2024-09-15T23:12:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-15T23:12:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_bytes": { 2024-09-15T23:12:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-15T23:12:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-09-15T23:12:28.064 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-15T23:12:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-09-15T23:12:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-09-15T23:12:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-09-15T23:12:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 2401, 2024-09-15T23:12:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 200443867, 2024-09-15T23:12:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 2401, 2024-09-15T23:12:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 200443867, 2024-09-15T23:12:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-15T23:12:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:28.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.066 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_ops": { 2024-09-15T23:12:28.066 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-15T23:12:28.066 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 1024, 2024-09-15T23:12:28.066 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-15T23:12:28.066 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-09-15T23:12:28.066 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-09-15T23:12:28.066 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:28.066 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-09-15T23:12:28.066 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 2401, 2024-09-15T23:12:28.066 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 2401, 2024-09-15T23:12:28.066 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 2401, 2024-09-15T23:12:28.066 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 2401, 2024-09-15T23:12:28.066 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-15T23:12:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle": { 2024-09-15T23:12:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-15T23:12:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-09-15T23:12:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-15T23:12:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-09-15T23:12:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-09-15T23:12:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-09-15T23:12:28.067 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-09-15T23:12:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-09-15T23:12:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 0, 2024-09-15T23:12:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 0, 2024-09-15T23:12:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-15T23:12:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle-0x55b2d8c706e0": { 2024-09-15T23:12:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-15T23:12:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-09-15T23:12:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-15T23:12:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-09-15T23:12:28.068 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-09-15T23:12:28.069 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:28.069 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-09-15T23:12:28.069 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-09-15T23:12:28.069 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-09-15T23:12:28.069 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 0, 2024-09-15T23:12:28.069 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 0, 2024-09-15T23:12:28.069 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-15T23:12:28.069 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:28.069 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:28.069 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:28.069 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:28.069 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:28.069 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T23:12:28.297 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:28 smithi192 ceph-mon[35427]: pgmap v1147: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 118 KiB/s wr, 34 op/s 2024-09-15T23:12:28.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:28 smithi121 ceph-mon[34815]: pgmap v1147: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 118 KiB/s wr, 34 op/s 2024-09-15T23:12:28.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:28 smithi111 ceph-mon[30629]: pgmap v1147: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 118 KiB/s wr, 34 op/s 2024-09-15T23:12:28.975 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-15T23:12:28.975 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-15T23:12:28.975 DEBUG:tasks.check_counter:Getting stats from f 2024-09-15T23:12:28.975 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-15T23:12:29.228 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T23:12:29.764 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T23:12:29.765 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T23:12:29.765 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 2463, 2024-09-15T23:12:29.765 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 2445, 2024-09-15T23:12:29.765 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 26707362, 2024-09-15T23:12:29.765 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 913672, 2024-09-15T23:12:29.765 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 7, 2024-09-15T23:12:29.765 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 3, 2024-09-15T23:12:29.765 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.255050322, 2024-09-15T23:12:29.765 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.082717181, 2024-09-15T23:12:29.765 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.100340483, 2024-09-15T23:12:29.765 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.061452778, 2024-09-15T23:12:29.766 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:29.766 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 2444, 2024-09-15T23:12:29.766 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.103281784, 2024-09-15T23:12:29.766 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000042259 2024-09-15T23:12:29.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.766 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:29.766 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.766 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.766 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.766 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 896432, 2024-09-15T23:12:29.766 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 284032 2024-09-15T23:12:29.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.767 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T23:12:29.767 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 883, 2024-09-15T23:12:29.767 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 1228, 2024-09-15T23:12:29.767 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 52872364, 2024-09-15T23:12:29.767 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 1059796, 2024-09-15T23:12:29.767 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 9, 2024-09-15T23:12:29.767 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 6, 2024-09-15T23:12:29.767 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.115234008, 2024-09-15T23:12:29.767 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.030294487, 2024-09-15T23:12:29.767 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.054900593, 2024-09-15T23:12:29.767 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.021688146, 2024-09-15T23:12:29.767 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:29.767 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1227, 2024-09-15T23:12:29.767 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.100832599, 2024-09-15T23:12:29.768 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000082178 2024-09-15T23:12:29.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.768 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:29.768 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.768 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.768 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.768 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-15T23:12:29.768 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 780848 2024-09-15T23:12:29.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.768 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T23:12:29.768 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 2591, 2024-09-15T23:12:29.769 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 2592, 2024-09-15T23:12:29.769 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 63574635, 2024-09-15T23:12:29.769 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 823627, 2024-09-15T23:12:29.769 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 6, 2024-09-15T23:12:29.769 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 5, 2024-09-15T23:12:29.769 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.192563585, 2024-09-15T23:12:29.769 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.050209914, 2024-09-15T23:12:29.769 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.075433021, 2024-09-15T23:12:29.769 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.055087723, 2024-09-15T23:12:29.769 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:29.769 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 2590, 2024-09-15T23:12:29.769 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.081065515, 2024-09-15T23:12:29.769 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000031299 2024-09-15T23:12:29.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.770 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:29.770 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.770 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.770 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.770 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-15T23:12:29.770 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 0 2024-09-15T23:12:29.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.770 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2024-09-15T23:12:29.770 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 1, 2024-09-15T23:12:29.770 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2024-09-15T23:12:29.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.770 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-MDSRank": { 2024-09-15T23:12:29.771 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-09-15T23:12:29.771 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-09-15T23:12:29.771 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 9165, 2024-09-15T23:12:29.771 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.209295394, 2024-09-15T23:12:29.771 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000022836 2024-09-15T23:12:29.771 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:29.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.771 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-PurgeQueue": { 2024-09-15T23:12:29.771 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-09-15T23:12:29.771 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-09-15T23:12:29.771 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.771 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.771 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.771 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:29.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.772 INFO:teuthology.orchestra.run.smithi192.stdout: "mds": { 2024-09-15T23:12:29.772 INFO:teuthology.orchestra.run.smithi192.stdout: "request": 0, 2024-09-15T23:12:29.772 INFO:teuthology.orchestra.run.smithi192.stdout: "reply": 0, 2024-09-15T23:12:29.772 INFO:teuthology.orchestra.run.smithi192.stdout: "reply_latency": { 2024-09-15T23:12:29.772 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.772 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.772 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.772 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_reply": 0, 2024-09-15T23:12:29.772 INFO:teuthology.orchestra.run.smithi192.stdout: "forward": 0, 2024-09-15T23:12:29.772 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_complete": 0, 2024-09-15T23:12:29.772 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_keys": 0, 2024-09-15T23:12:29.773 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_commit": 0, 2024-09-15T23:12:29.773 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_split": 0, 2024-09-15T23:12:29.773 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_merge": 0, 2024-09-15T23:12:29.773 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes": 14076, 2024-09-15T23:12:29.773 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_top": 0, 2024-09-15T23:12:29.773 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_bottom": 14076, 2024-09-15T23:12:29.773 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pin_tail": 0, 2024-09-15T23:12:29.773 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pinned": 14076, 2024-09-15T23:12:29.773 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_expired": 0, 2024-09-15T23:12:29.773 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_with_caps": 0, 2024-09-15T23:12:29.773 INFO:teuthology.orchestra.run.smithi192.stdout: "caps": 0, 2024-09-15T23:12:29.773 INFO:teuthology.orchestra.run.smithi192.stdout: "subtrees": 1, 2024-09-15T23:12:29.773 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse": 0, 2024-09-15T23:12:29.774 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_hit": 0, 2024-09-15T23:12:29.774 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_forward": 0, 2024-09-15T23:12:29.774 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_discover": 0, 2024-09-15T23:12:29.774 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_dir_fetch": 0, 2024-09-15T23:12:29.774 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_remote_ino": 0, 2024-09-15T23:12:29.774 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_lock": 0, 2024-09-15T23:12:29.774 INFO:teuthology.orchestra.run.smithi192.stdout: "load_cent": 0, 2024-09-15T23:12:29.774 INFO:teuthology.orchestra.run.smithi192.stdout: "q": 0, 2024-09-15T23:12:29.774 INFO:teuthology.orchestra.run.smithi192.stdout: "exported": 0, 2024-09-15T23:12:29.774 INFO:teuthology.orchestra.run.smithi192.stdout: "exported_inodes": 0, 2024-09-15T23:12:29.774 INFO:teuthology.orchestra.run.smithi192.stdout: "imported": 0, 2024-09-15T23:12:29.774 INFO:teuthology.orchestra.run.smithi192.stdout: "imported_inodes": 0, 2024-09-15T23:12:29.774 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_dir_fetch": 0, 2024-09-15T23:12:29.775 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_backtrace_fetch": 0, 2024-09-15T23:12:29.775 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_peer_discover": 0, 2024-09-15T23:12:29.775 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rfiles": 0, 2024-09-15T23:12:29.775 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rbytes": 0, 2024-09-15T23:12:29.775 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rsnaps": 0, 2024-09-15T23:12:29.775 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_fetch": 0, 2024-09-15T23:12:29.775 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_set_tag": 0, 2024-09-15T23:12:29.775 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T23:12:29.775 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_inotable_repaired": 0, 2024-09-15T23:12:29.775 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_inodes": 0, 2024-09-15T23:12:29.775 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_base_inodes": 0, 2024-09-15T23:12:29.775 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dirfrag_rstats": 0, 2024-09-15T23:12:29.775 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_file_inodes": 0, 2024-09-15T23:12:29.775 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_inode_file_caps": 0, 2024-09-15T23:12:29.776 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_revoke": 0, 2024-09-15T23:12:29.776 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_grant": 0, 2024-09-15T23:12:29.776 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T23:12:29.776 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T23:12:29.776 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T23:12:29.776 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps": 0, 2024-09-15T23:12:29.776 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps_dirty": 0, 2024-09-15T23:12:29.776 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_cap_release": 0, 2024-09-15T23:12:29.776 INFO:teuthology.orchestra.run.smithi192.stdout: "process_request_cap_release": 0 2024-09-15T23:12:29.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.776 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_cache": { 2024-09-15T23:12:29.776 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update": 0, 2024-09-15T23:12:29.776 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update_receipt": 0, 2024-09-15T23:12:29.777 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_try_discover": 0, 2024-09-15T23:12:29.777 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_send_discover": 0, 2024-09-15T23:12:29.777 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_handle_discover": 0, 2024-09-15T23:12:29.777 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays": 0, 2024-09-15T23:12:29.777 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_delayed": 0, 2024-09-15T23:12:29.777 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_enqueuing": 0, 2024-09-15T23:12:29.777 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_created": 0, 2024-09-15T23:12:29.777 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_enqueued": 0, 2024-09-15T23:12:29.777 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_reintegrated": 0, 2024-09-15T23:12:29.777 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_migrated": 0, 2024-09-15T23:12:29.777 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_processing": 0, 2024-09-15T23:12:29.777 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_enqueued": 0, 2024-09-15T23:12:29.777 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_prioritized": 0, 2024-09-15T23:12:29.777 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_started": 0, 2024-09-15T23:12:29.778 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_completed": 0, 2024-09-15T23:12:29.778 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_quiesce_path": 0, 2024-09-15T23:12:29.778 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_quiesce_inode": 0, 2024-09-15T23:12:29.778 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_enqueue_scrub": 0, 2024-09-15T23:12:29.778 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_exportdir": 0, 2024-09-15T23:12:29.778 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_flush": 0, 2024-09-15T23:12:29.778 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragmentdir": 0, 2024-09-15T23:12:29.778 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragstats": 0, 2024-09-15T23:12:29.778 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_inodestats": 0 2024-09-15T23:12:29.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.778 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_log": { 2024-09-15T23:12:29.778 INFO:teuthology.orchestra.run.smithi192.stdout: "evlrg": 0, 2024-09-15T23:12:29.778 INFO:teuthology.orchestra.run.smithi192.stdout: "evadd": 0, 2024-09-15T23:12:29.778 INFO:teuthology.orchestra.run.smithi192.stdout: "evex": 0, 2024-09-15T23:12:29.779 INFO:teuthology.orchestra.run.smithi192.stdout: "evtrm": 0, 2024-09-15T23:12:29.779 INFO:teuthology.orchestra.run.smithi192.stdout: "ev": 70558, 2024-09-15T23:12:29.779 INFO:teuthology.orchestra.run.smithi192.stdout: "evexg": 0, 2024-09-15T23:12:29.779 INFO:teuthology.orchestra.run.smithi192.stdout: "evexd": 0, 2024-09-15T23:12:29.779 INFO:teuthology.orchestra.run.smithi192.stdout: "segadd": 0, 2024-09-15T23:12:29.779 INFO:teuthology.orchestra.run.smithi192.stdout: "segex": 0, 2024-09-15T23:12:29.779 INFO:teuthology.orchestra.run.smithi192.stdout: "segtrm": 0, 2024-09-15T23:12:29.779 INFO:teuthology.orchestra.run.smithi192.stdout: "seg": 85, 2024-09-15T23:12:29.779 INFO:teuthology.orchestra.run.smithi192.stdout: "segmjr": 7, 2024-09-15T23:12:29.779 INFO:teuthology.orchestra.run.smithi192.stdout: "segexg": 0, 2024-09-15T23:12:29.779 INFO:teuthology.orchestra.run.smithi192.stdout: "segexd": 0, 2024-09-15T23:12:29.779 INFO:teuthology.orchestra.run.smithi192.stdout: "expos": 4194304, 2024-09-15T23:12:29.779 INFO:teuthology.orchestra.run.smithi192.stdout: "wrpos": 144919125, 2024-09-15T23:12:29.780 INFO:teuthology.orchestra.run.smithi192.stdout: "rdpos": 144918526, 2024-09-15T23:12:29.780 INFO:teuthology.orchestra.run.smithi192.stdout: "jlat": { 2024-09-15T23:12:29.780 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.780 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.780 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.780 INFO:teuthology.orchestra.run.smithi192.stdout: "replayed": 70558 2024-09-15T23:12:29.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.780 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_mem": { 2024-09-15T23:12:29.780 INFO:teuthology.orchestra.run.smithi192.stdout: "ino": 13, 2024-09-15T23:12:29.780 INFO:teuthology.orchestra.run.smithi192.stdout: "ino+": 14096, 2024-09-15T23:12:29.780 INFO:teuthology.orchestra.run.smithi192.stdout: "ino-": 14083, 2024-09-15T23:12:29.780 INFO:teuthology.orchestra.run.smithi192.stdout: "dir": 11, 2024-09-15T23:12:29.780 INFO:teuthology.orchestra.run.smithi192.stdout: "dir+": 39, 2024-09-15T23:12:29.781 INFO:teuthology.orchestra.run.smithi192.stdout: "dir-": 28, 2024-09-15T23:12:29.781 INFO:teuthology.orchestra.run.smithi192.stdout: "dn": 14076, 2024-09-15T23:12:29.781 INFO:teuthology.orchestra.run.smithi192.stdout: "dn+": 14281, 2024-09-15T23:12:29.781 INFO:teuthology.orchestra.run.smithi192.stdout: "dn-": 205, 2024-09-15T23:12:29.781 INFO:teuthology.orchestra.run.smithi192.stdout: "cap": 0, 2024-09-15T23:12:29.781 INFO:teuthology.orchestra.run.smithi192.stdout: "cap+": 0, 2024-09-15T23:12:29.781 INFO:teuthology.orchestra.run.smithi192.stdout: "cap-": 0, 2024-09-15T23:12:29.781 INFO:teuthology.orchestra.run.smithi192.stdout: "rss": 63964, 2024-09-15T23:12:29.781 INFO:teuthology.orchestra.run.smithi192.stdout: "heap": 215328 2024-09-15T23:12:29.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.781 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_server": { 2024-09-15T23:12:29.781 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_client_request": 0, 2024-09-15T23:12:29.781 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_server_request": 0, 2024-09-15T23:12:29.782 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_request": 0, 2024-09-15T23:12:29.782 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_session": 0, 2024-09-15T23:12:29.782 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_peer_request": 0, 2024-09-15T23:12:29.782 INFO:teuthology.orchestra.run.smithi192.stdout: "req_create_latency": { 2024-09-15T23:12:29.782 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.782 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.782 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.782 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getattr_latency": { 2024-09-15T23:12:29.782 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.782 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.782 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.782 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getfilelock_latency": { 2024-09-15T23:12:29.783 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.783 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.783 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.783 INFO:teuthology.orchestra.run.smithi192.stdout: "req_link_latency": { 2024-09-15T23:12:29.783 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.783 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.783 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.783 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookup_latency": { 2024-09-15T23:12:29.783 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.783 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.783 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.784 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookuphash_latency": { 2024-09-15T23:12:29.784 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.784 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.784 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.784 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupino_latency": { 2024-09-15T23:12:29.784 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.784 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.784 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.784 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupname_latency": { 2024-09-15T23:12:29.784 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.784 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.785 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.785 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupparent_latency": { 2024-09-15T23:12:29.785 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.785 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.785 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.785 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupsnap_latency": { 2024-09-15T23:12:29.785 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.785 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.785 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.785 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lssnap_latency": { 2024-09-15T23:12:29.786 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.786 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.786 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.786 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mkdir_latency": { 2024-09-15T23:12:29.786 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.787 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.787 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.787 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mknod_latency": { 2024-09-15T23:12:29.787 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.787 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.787 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.787 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mksnap_latency": { 2024-09-15T23:12:29.787 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.787 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.787 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.788 INFO:teuthology.orchestra.run.smithi192.stdout: "req_open_latency": { 2024-09-15T23:12:29.788 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.788 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.788 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.788 INFO:teuthology.orchestra.run.smithi192.stdout: "req_readdir_latency": { 2024-09-15T23:12:29.788 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.788 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.788 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.788 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rename_latency": { 2024-09-15T23:12:29.788 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.788 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.789 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.789 INFO:teuthology.orchestra.run.smithi192.stdout: "req_renamesnap_latency": { 2024-09-15T23:12:29.789 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.789 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.789 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.789 INFO:teuthology.orchestra.run.smithi192.stdout: "req_snapdiff_latency": { 2024-09-15T23:12:29.789 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.789 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.789 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.789 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmdir_latency": { 2024-09-15T23:12:29.790 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.790 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.790 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.790 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmsnap_latency": { 2024-09-15T23:12:29.790 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.790 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.790 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.790 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmxattr_latency": { 2024-09-15T23:12:29.790 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.790 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.790 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.791 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setattr_latency": { 2024-09-15T23:12:29.791 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.791 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.791 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.791 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setdirlayout_latency": { 2024-09-15T23:12:29.791 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.791 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.791 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.791 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setfilelock_latency": { 2024-09-15T23:12:29.791 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.791 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.792 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.792 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setlayout_latency": { 2024-09-15T23:12:29.792 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.792 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.792 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.792 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setxattr_latency": { 2024-09-15T23:12:29.792 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.792 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.792 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.792 INFO:teuthology.orchestra.run.smithi192.stdout: "req_symlink_latency": { 2024-09-15T23:12:29.793 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.793 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.793 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.793 INFO:teuthology.orchestra.run.smithi192.stdout: "req_unlink_latency": { 2024-09-15T23:12:29.793 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.793 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.793 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.793 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_revoke_eviction": 0, 2024-09-15T23:12:29.793 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_acquisition_throttle": 0, 2024-09-15T23:12:29.793 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getvxattr_latency": { 2024-09-15T23:12:29.793 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.793 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.794 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.794 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:29.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.794 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_sessions": { 2024-09-15T23:12:29.794 INFO:teuthology.orchestra.run.smithi192.stdout: "session_count": 1, 2024-09-15T23:12:29.794 INFO:teuthology.orchestra.run.smithi192.stdout: "session_add": 3, 2024-09-15T23:12:29.794 INFO:teuthology.orchestra.run.smithi192.stdout: "session_remove": 2, 2024-09-15T23:12:29.794 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_open": 2, 2024-09-15T23:12:29.794 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_stale": 0, 2024-09-15T23:12:29.794 INFO:teuthology.orchestra.run.smithi192.stdout: "total_load": 0, 2024-09-15T23:12:29.794 INFO:teuthology.orchestra.run.smithi192.stdout: "average_load": 0, 2024-09-15T23:12:29.794 INFO:teuthology.orchestra.run.smithi192.stdout: "avg_session_uptime": 4332, 2024-09-15T23:12:29.794 INFO:teuthology.orchestra.run.smithi192.stdout: "mdthresh_evicted": 0 2024-09-15T23:12:29.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.795 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2024-09-15T23:12:29.795 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2024-09-15T23:12:29.795 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2024-09-15T23:12:29.795 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T23:12:29.795 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2024-09-15T23:12:29.795 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T23:12:29.795 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2024-09-15T23:12:29.795 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T23:12:29.795 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T23:12:29.795 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T23:12:29.795 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T23:12:29.795 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T23:12:29.795 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2024-09-15T23:12:29.796 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T23:12:29.796 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T23:12:29.796 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2024-09-15T23:12:29.796 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2024-09-15T23:12:29.796 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2024-09-15T23:12:29.796 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2024-09-15T23:12:29.796 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T23:12:29.796 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T23:12:29.796 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T23:12:29.796 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T23:12:29.796 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T23:12:29.796 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2024-09-15T23:12:29.796 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2024-09-15T23:12:29.797 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2024-09-15T23:12:29.797 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T23:12:29.797 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T23:12:29.797 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2024-09-15T23:12:29.797 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2024-09-15T23:12:29.797 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2024-09-15T23:12:29.797 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2024-09-15T23:12:29.797 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T23:12:29.797 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2024-09-15T23:12:29.797 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T23:12:29.797 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2024-09-15T23:12:29.797 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 241017, 2024-09-15T23:12:29.797 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 83, 2024-09-15T23:12:29.797 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 0, 2024-09-15T23:12:29.798 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 0, 2024-09-15T23:12:29.798 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2024-09-15T23:12:29.798 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2024-09-15T23:12:29.798 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2024-09-15T23:12:29.798 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2024-09-15T23:12:29.798 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2024-09-15T23:12:29.798 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2024-09-15T23:12:29.798 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 7304, 2024-09-15T23:12:29.798 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 150, 2024-09-15T23:12:29.798 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T23:12:29.798 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2024-09-15T23:12:29.798 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2024-09-15T23:12:29.798 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2024-09-15T23:12:29.799 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 9953096, 2024-09-15T23:12:29.799 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 56432, 2024-09-15T23:12:29.799 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2024-09-15T23:12:29.799 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2024-09-15T23:12:29.799 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2024-09-15T23:12:29.799 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2024-09-15T23:12:29.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.799 INFO:teuthology.orchestra.run.smithi192.stdout: "objecter": { 2024-09-15T23:12:29.799 INFO:teuthology.orchestra.run.smithi192.stdout: "op_active": 0, 2024-09-15T23:12:29.799 INFO:teuthology.orchestra.run.smithi192.stdout: "op_laggy": 0, 2024-09-15T23:12:29.799 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send": 5447, 2024-09-15T23:12:29.799 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send_bytes": 21, 2024-09-15T23:12:29.799 INFO:teuthology.orchestra.run.smithi192.stdout: "op_resend": 3, 2024-09-15T23:12:29.799 INFO:teuthology.orchestra.run.smithi192.stdout: "op_reply": 5447, 2024-09-15T23:12:29.800 INFO:teuthology.orchestra.run.smithi192.stdout: "op_latency": { 2024-09-15T23:12:29.800 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 5447, 2024-09-15T23:12:29.800 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 5.127898967, 2024-09-15T23:12:29.800 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000941417 2024-09-15T23:12:29.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.800 INFO:teuthology.orchestra.run.smithi192.stdout: "op_inflight": 0, 2024-09-15T23:12:29.800 INFO:teuthology.orchestra.run.smithi192.stdout: "oplen_avg": { 2024-09-15T23:12:29.800 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 5447, 2024-09-15T23:12:29.800 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 5448 2024-09-15T23:12:29.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.800 INFO:teuthology.orchestra.run.smithi192.stdout: "op": 5447, 2024-09-15T23:12:29.800 INFO:teuthology.orchestra.run.smithi192.stdout: "op_r": 5447, 2024-09-15T23:12:29.800 INFO:teuthology.orchestra.run.smithi192.stdout: "op_w": 0, 2024-09-15T23:12:29.801 INFO:teuthology.orchestra.run.smithi192.stdout: "op_rmw": 0, 2024-09-15T23:12:29.801 INFO:teuthology.orchestra.run.smithi192.stdout: "op_pg": 0, 2024-09-15T23:12:29.801 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_stat": 3267, 2024-09-15T23:12:29.801 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_create": 0, 2024-09-15T23:12:29.801 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_read": 2178, 2024-09-15T23:12:29.801 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_write": 0, 2024-09-15T23:12:29.801 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writefull": 0, 2024-09-15T23:12:29.801 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writesame": 0, 2024-09-15T23:12:29.801 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_append": 0, 2024-09-15T23:12:29.801 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_zero": 0, 2024-09-15T23:12:29.801 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_truncate": 0, 2024-09-15T23:12:29.801 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_delete": 0, 2024-09-15T23:12:29.801 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_mapext": 0, 2024-09-15T23:12:29.801 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_sparse_read": 0, 2024-09-15T23:12:29.802 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_clonerange": 0, 2024-09-15T23:12:29.802 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_getxattr": 1, 2024-09-15T23:12:29.802 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_setxattr": 0, 2024-09-15T23:12:29.802 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_cmpxattr": 0, 2024-09-15T23:12:29.802 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_rmxattr": 0, 2024-09-15T23:12:29.802 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_resetxattrs": 0, 2024-09-15T23:12:29.802 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_call": 0, 2024-09-15T23:12:29.802 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_watch": 0, 2024-09-15T23:12:29.802 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_notify": 0, 2024-09-15T23:12:29.802 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T23:12:29.802 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls": 0, 2024-09-15T23:12:29.802 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls_filter": 0, 2024-09-15T23:12:29.802 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_other": 0, 2024-09-15T23:12:29.802 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_active": 0, 2024-09-15T23:12:29.802 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_send": 0, 2024-09-15T23:12:29.803 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_resend": 0, 2024-09-15T23:12:29.803 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_ping": 0, 2024-09-15T23:12:29.803 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_active": 0, 2024-09-15T23:12:29.803 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_send": 0, 2024-09-15T23:12:29.803 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_resend": 0, 2024-09-15T23:12:29.803 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_active": 0, 2024-09-15T23:12:29.803 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_send": 0, 2024-09-15T23:12:29.803 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_resend": 0, 2024-09-15T23:12:29.803 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_active": 0, 2024-09-15T23:12:29.803 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_send": 0, 2024-09-15T23:12:29.803 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_resend": 0, 2024-09-15T23:12:29.803 INFO:teuthology.orchestra.run.smithi192.stdout: "command_active": 0, 2024-09-15T23:12:29.803 INFO:teuthology.orchestra.run.smithi192.stdout: "command_send": 0, 2024-09-15T23:12:29.804 INFO:teuthology.orchestra.run.smithi192.stdout: "command_resend": 0, 2024-09-15T23:12:29.804 INFO:teuthology.orchestra.run.smithi192.stdout: "map_epoch": 89, 2024-09-15T23:12:29.804 INFO:teuthology.orchestra.run.smithi192.stdout: "map_full": 0, 2024-09-15T23:12:29.804 INFO:teuthology.orchestra.run.smithi192.stdout: "map_inc": 3, 2024-09-15T23:12:29.804 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_sessions": 11, 2024-09-15T23:12:29.804 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_open": 12, 2024-09-15T23:12:29.804 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_close": 1, 2024-09-15T23:12:29.804 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_laggy": 0, 2024-09-15T23:12:29.804 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_wr": 0, 2024-09-15T23:12:29.804 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_rd": 2, 2024-09-15T23:12:29.804 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_del": 0 2024-09-15T23:12:29.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.804 INFO:teuthology.orchestra.run.smithi192.stdout: "oft": { 2024-09-15T23:12:29.804 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_objs": 0, 2024-09-15T23:12:29.805 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_kv_pairs": 0, 2024-09-15T23:12:29.805 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_updates": 0, 2024-09-15T23:12:29.805 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_removes": 0 2024-09-15T23:12:29.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.805 INFO:teuthology.orchestra.run.smithi192.stdout: "osd-slow-ops": { 2024-09-15T23:12:29.805 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_ops_count": 0 2024-09-15T23:12:29.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.805 INFO:teuthology.orchestra.run.smithi192.stdout: "purge_queue": { 2024-09-15T23:12:29.805 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops": 0, 2024-09-15T23:12:29.805 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops_high_water": 0, 2024-09-15T23:12:29.805 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing": 0, 2024-09-15T23:12:29.805 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_high_water": 0, 2024-09-15T23:12:29.805 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed_ops": 0, 2024-09-15T23:12:29.806 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed": 0, 2024-09-15T23:12:29.806 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_item_in_journal": 0 2024-09-15T23:12:29.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.806 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T23:12:29.806 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-15T23:12:29.806 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-09-15T23:12:29.806 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-15T23:12:29.806 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 5933, 2024-09-15T23:12:29.806 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 142592470, 2024-09-15T23:12:29.806 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:29.806 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 5933, 2024-09-15T23:12:29.806 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-09-15T23:12:29.806 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-09-15T23:12:29.806 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 5933, 2024-09-15T23:12:29.807 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 142592470, 2024-09-15T23:12:29.807 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-15T23:12:29.807 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.807 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.807 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.807 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:29.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.807 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_bytes": { 2024-09-15T23:12:29.807 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-15T23:12:29.807 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-09-15T23:12:29.807 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-15T23:12:29.807 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-09-15T23:12:29.807 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-09-15T23:12:29.807 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:29.808 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-09-15T23:12:29.808 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 5447, 2024-09-15T23:12:29.808 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 140732101, 2024-09-15T23:12:29.808 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 5447, 2024-09-15T23:12:29.808 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 140732101, 2024-09-15T23:12:29.808 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-15T23:12:29.808 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.808 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.808 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.808 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:29.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.808 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_ops": { 2024-09-15T23:12:29.808 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-15T23:12:29.808 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 1024, 2024-09-15T23:12:29.809 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-15T23:12:29.809 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-09-15T23:12:29.809 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-09-15T23:12:29.809 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:29.809 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-09-15T23:12:29.809 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 5447, 2024-09-15T23:12:29.809 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 5447, 2024-09-15T23:12:29.809 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 5447, 2024-09-15T23:12:29.809 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 5447, 2024-09-15T23:12:29.809 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-15T23:12:29.809 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.809 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.809 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.809 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:29.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.810 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle": { 2024-09-15T23:12:29.810 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-15T23:12:29.810 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-09-15T23:12:29.810 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-15T23:12:29.810 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-09-15T23:12:29.810 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-09-15T23:12:29.810 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:29.810 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-09-15T23:12:29.810 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-09-15T23:12:29.810 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-09-15T23:12:29.810 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 0, 2024-09-15T23:12:29.810 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 0, 2024-09-15T23:12:29.811 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-15T23:12:29.811 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.811 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.811 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.811 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:29.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:29.811 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle-0x55d2b3340b40": { 2024-09-15T23:12:29.811 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-15T23:12:29.811 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-09-15T23:12:29.811 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-15T23:12:29.811 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-09-15T23:12:29.811 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-09-15T23:12:29.811 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:29.811 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-09-15T23:12:29.812 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-09-15T23:12:29.812 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-09-15T23:12:29.812 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 0, 2024-09-15T23:12:29.812 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 0, 2024-09-15T23:12:29.812 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-15T23:12:29.812 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:29.812 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:29.812 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:29.812 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:29.812 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:29.812 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T23:12:30.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:30 smithi111 ceph-mon[30629]: pgmap v1148: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-09-15T23:12:30.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:30 smithi121 ceph-mon[34815]: pgmap v1148: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-09-15T23:12:30.516 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:30 smithi192 ceph-mon[35427]: pgmap v1148: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 14 op/s 2024-09-15T23:12:30.543 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-15T23:12:30.543 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-15T23:12:30.543 DEBUG:tasks.check_counter:Getting stats from i 2024-09-15T23:12:30.544 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-15T23:12:30.788 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T23:12:31.398 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T23:12:31.398 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T23:12:31.398 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 219242, 2024-09-15T23:12:31.399 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 158711, 2024-09-15T23:12:31.399 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 56470049, 2024-09-15T23:12:31.399 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 75863289, 2024-09-15T23:12:31.399 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 9, 2024-09-15T23:12:31.399 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 4, 2024-09-15T23:12:31.399 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 10.481035291, 2024-09-15T23:12:31.400 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 3.984728367, 2024-09-15T23:12:31.400 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 8.063934297, 2024-09-15T23:12:31.400 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.468219119, 2024-09-15T23:12:31.400 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:31.400 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 158710, 2024-09-15T23:12:31.400 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 2.786093188, 2024-09-15T23:12:31.401 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000017554 2024-09-15T23:12:31.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.401 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:31.401 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 353475, 2024-09-15T23:12:31.401 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.046710118, 2024-09-15T23:12:31.401 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000132 2024-09-15T23:12:31.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.402 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-15T23:12:31.402 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 448 2024-09-15T23:12:31.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.402 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T23:12:31.403 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 16617, 2024-09-15T23:12:31.403 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 16595, 2024-09-15T23:12:31.403 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 4922252, 2024-09-15T23:12:31.403 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 66441851, 2024-09-15T23:12:31.403 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 15, 2024-09-15T23:12:31.403 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 6, 2024-09-15T23:12:31.404 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 1.251488225, 2024-09-15T23:12:31.404 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.448819206, 2024-09-15T23:12:31.404 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.308220482, 2024-09-15T23:12:31.404 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.420305370, 2024-09-15T23:12:31.404 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:31.405 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 16594, 2024-09-15T23:12:31.405 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.524863908, 2024-09-15T23:12:31.405 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000031629 2024-09-15T23:12:31.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.405 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:31.405 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 29, 2024-09-15T23:12:31.405 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000003758, 2024-09-15T23:12:31.406 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000129 2024-09-15T23:12:31.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.406 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 915360, 2024-09-15T23:12:31.406 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 292176 2024-09-15T23:12:31.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.407 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T23:12:31.407 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 160415, 2024-09-15T23:12:31.407 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 218646, 2024-09-15T23:12:31.407 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 50957132, 2024-09-15T23:12:31.407 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 200267791, 2024-09-15T23:12:31.407 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 10, 2024-09-15T23:12:31.407 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 6, 2024-09-15T23:12:31.408 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 8.159265547, 2024-09-15T23:12:31.408 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 3.892214096, 2024-09-15T23:12:31.408 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 5.561514186, 2024-09-15T23:12:31.408 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.334548744, 2024-09-15T23:12:31.408 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:31.408 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 218644, 2024-09-15T23:12:31.408 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 5.059932705, 2024-09-15T23:12:31.408 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000023142 2024-09-15T23:12:31.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.408 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:31.408 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 176434, 2024-09-15T23:12:31.409 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.051117676, 2024-09-15T23:12:31.409 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000289 2024-09-15T23:12:31.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.409 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-15T23:12:31.409 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 693488 2024-09-15T23:12:31.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.409 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2024-09-15T23:12:31.409 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 1, 2024-09-15T23:12:31.409 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2024-09-15T23:12:31.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.409 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-MDSRank": { 2024-09-15T23:12:31.410 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-09-15T23:12:31.410 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-09-15T23:12:31.410 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 44235, 2024-09-15T23:12:31.410 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 32.569168519, 2024-09-15T23:12:31.410 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000736275 2024-09-15T23:12:31.410 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:31.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.410 INFO:teuthology.orchestra.run.smithi192.stdout: "finisher-PurgeQueue": { 2024-09-15T23:12:31.410 INFO:teuthology.orchestra.run.smithi192.stdout: "queue_len": 0, 2024-09-15T23:12:31.410 INFO:teuthology.orchestra.run.smithi192.stdout: "complete_latency": { 2024-09-15T23:12:31.410 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 25745, 2024-09-15T23:12:31.411 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 1.257773460, 2024-09-15T23:12:31.411 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000048855 2024-09-15T23:12:31.411 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:31.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.411 INFO:teuthology.orchestra.run.smithi192.stdout: "mds": { 2024-09-15T23:12:31.411 INFO:teuthology.orchestra.run.smithi192.stdout: "request": 111615, 2024-09-15T23:12:31.411 INFO:teuthology.orchestra.run.smithi192.stdout: "reply": 84479, 2024-09-15T23:12:31.411 INFO:teuthology.orchestra.run.smithi192.stdout: "reply_latency": { 2024-09-15T23:12:31.411 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 84479, 2024-09-15T23:12:31.411 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 240.925846276, 2024-09-15T23:12:31.411 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.002851902 2024-09-15T23:12:31.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.412 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_reply": 0, 2024-09-15T23:12:31.412 INFO:teuthology.orchestra.run.smithi192.stdout: "forward": 27136, 2024-09-15T23:12:31.412 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_complete": 0, 2024-09-15T23:12:31.412 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_fetch_keys": 0, 2024-09-15T23:12:31.412 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_commit": 11, 2024-09-15T23:12:31.412 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_split": 0, 2024-09-15T23:12:31.412 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_merge": 0, 2024-09-15T23:12:31.413 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes": 11, 2024-09-15T23:12:31.413 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_top": 0, 2024-09-15T23:12:31.413 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_bottom": 1, 2024-09-15T23:12:31.413 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pin_tail": 10, 2024-09-15T23:12:31.413 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_pinned": 10, 2024-09-15T23:12:31.413 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_expired": 89, 2024-09-15T23:12:31.413 INFO:teuthology.orchestra.run.smithi192.stdout: "inodes_with_caps": 1, 2024-09-15T23:12:31.413 INFO:teuthology.orchestra.run.smithi192.stdout: "caps": 1, 2024-09-15T23:12:31.413 INFO:teuthology.orchestra.run.smithi192.stdout: "subtrees": 3, 2024-09-15T23:12:31.413 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse": 173829, 2024-09-15T23:12:31.413 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_hit": 78040, 2024-09-15T23:12:31.413 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_forward": 27134, 2024-09-15T23:12:31.414 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_discover": 30, 2024-09-15T23:12:31.414 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_dir_fetch": 0, 2024-09-15T23:12:31.414 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_remote_ino": 0, 2024-09-15T23:12:31.414 INFO:teuthology.orchestra.run.smithi192.stdout: "traverse_lock": 1148, 2024-09-15T23:12:31.414 INFO:teuthology.orchestra.run.smithi192.stdout: "load_cent": 0, 2024-09-15T23:12:31.414 INFO:teuthology.orchestra.run.smithi192.stdout: "q": 0, 2024-09-15T23:12:31.414 INFO:teuthology.orchestra.run.smithi192.stdout: "exported": 3, 2024-09-15T23:12:31.414 INFO:teuthology.orchestra.run.smithi192.stdout: "exported_inodes": 8, 2024-09-15T23:12:31.414 INFO:teuthology.orchestra.run.smithi192.stdout: "imported": 12, 2024-09-15T23:12:31.414 INFO:teuthology.orchestra.run.smithi192.stdout: "imported_inodes": 108, 2024-09-15T23:12:31.415 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_dir_fetch": 0, 2024-09-15T23:12:31.415 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_backtrace_fetch": 0, 2024-09-15T23:12:31.415 INFO:teuthology.orchestra.run.smithi192.stdout: "openino_peer_discover": 0, 2024-09-15T23:12:31.415 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rfiles": 4, 2024-09-15T23:12:31.415 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rbytes": 234, 2024-09-15T23:12:31.415 INFO:teuthology.orchestra.run.smithi192.stdout: "root_rsnaps": 0, 2024-09-15T23:12:31.415 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_fetch": 0, 2024-09-15T23:12:31.415 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_set_tag": 0, 2024-09-15T23:12:31.415 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T23:12:31.415 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_inotable_repaired": 0, 2024-09-15T23:12:31.415 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_inodes": 0, 2024-09-15T23:12:31.415 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dir_base_inodes": 0, 2024-09-15T23:12:31.415 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_dirfrag_rstats": 0, 2024-09-15T23:12:31.416 INFO:teuthology.orchestra.run.smithi192.stdout: "scrub_file_inodes": 0, 2024-09-15T23:12:31.416 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_inode_file_caps": 4, 2024-09-15T23:12:31.416 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_revoke": 3229, 2024-09-15T23:12:31.416 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_grant": 46910, 2024-09-15T23:12:31.416 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T23:12:31.416 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T23:12:31.417 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T23:12:31.417 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps": 30118, 2024-09-15T23:12:31.417 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_caps_dirty": 11938, 2024-09-15T23:12:31.417 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_cap_release": 3017, 2024-09-15T23:12:31.417 INFO:teuthology.orchestra.run.smithi192.stdout: "process_request_cap_release": 57096 2024-09-15T23:12:31.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.417 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_cache": { 2024-09-15T23:12:31.417 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update": 2, 2024-09-15T23:12:31.417 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_update_receipt": 43, 2024-09-15T23:12:31.418 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_try_discover": 30, 2024-09-15T23:12:31.418 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_send_discover": 46, 2024-09-15T23:12:31.418 INFO:teuthology.orchestra.run.smithi192.stdout: "dir_handle_discover": 4, 2024-09-15T23:12:31.418 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays": 0, 2024-09-15T23:12:31.418 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_delayed": 0, 2024-09-15T23:12:31.418 INFO:teuthology.orchestra.run.smithi192.stdout: "num_strays_enqueuing": 0, 2024-09-15T23:12:31.418 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_created": 14066, 2024-09-15T23:12:31.419 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_enqueued": 14066, 2024-09-15T23:12:31.419 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_reintegrated": 0, 2024-09-15T23:12:31.419 INFO:teuthology.orchestra.run.smithi192.stdout: "strays_migrated": 0, 2024-09-15T23:12:31.419 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_processing": 0, 2024-09-15T23:12:31.419 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_enqueued": 0, 2024-09-15T23:12:31.419 INFO:teuthology.orchestra.run.smithi192.stdout: "num_recovering_prioritized": 0, 2024-09-15T23:12:31.419 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_started": 0, 2024-09-15T23:12:31.419 INFO:teuthology.orchestra.run.smithi192.stdout: "recovery_completed": 0, 2024-09-15T23:12:31.419 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_quiesce_path": 0, 2024-09-15T23:12:31.419 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_quiesce_inode": 0, 2024-09-15T23:12:31.419 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_enqueue_scrub": 0, 2024-09-15T23:12:31.419 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_exportdir": 3, 2024-09-15T23:12:31.420 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_flush": 0, 2024-09-15T23:12:31.420 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragmentdir": 0, 2024-09-15T23:12:31.420 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_fragstats": 0, 2024-09-15T23:12:31.420 INFO:teuthology.orchestra.run.smithi192.stdout: "ireq_inodestats": 0 2024-09-15T23:12:31.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.420 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_log": { 2024-09-15T23:12:31.420 INFO:teuthology.orchestra.run.smithi192.stdout: "evlrg": 0, 2024-09-15T23:12:31.420 INFO:teuthology.orchestra.run.smithi192.stdout: "evadd": 70558, 2024-09-15T23:12:31.420 INFO:teuthology.orchestra.run.smithi192.stdout: "evex": 0, 2024-09-15T23:12:31.420 INFO:teuthology.orchestra.run.smithi192.stdout: "evtrm": 0, 2024-09-15T23:12:31.421 INFO:teuthology.orchestra.run.smithi192.stdout: "ev": 70558, 2024-09-15T23:12:31.421 INFO:teuthology.orchestra.run.smithi192.stdout: "evexg": 0, 2024-09-15T23:12:31.421 INFO:teuthology.orchestra.run.smithi192.stdout: "evexd": 0, 2024-09-15T23:12:31.421 INFO:teuthology.orchestra.run.smithi192.stdout: "segadd": 85, 2024-09-15T23:12:31.421 INFO:teuthology.orchestra.run.smithi192.stdout: "segex": 0, 2024-09-15T23:12:31.421 INFO:teuthology.orchestra.run.smithi192.stdout: "segtrm": 0, 2024-09-15T23:12:31.421 INFO:teuthology.orchestra.run.smithi192.stdout: "seg": 85, 2024-09-15T23:12:31.421 INFO:teuthology.orchestra.run.smithi192.stdout: "segmjr": 7, 2024-09-15T23:12:31.421 INFO:teuthology.orchestra.run.smithi192.stdout: "segexg": 0, 2024-09-15T23:12:31.422 INFO:teuthology.orchestra.run.smithi192.stdout: "segexd": 0, 2024-09-15T23:12:31.422 INFO:teuthology.orchestra.run.smithi192.stdout: "expos": 4194304, 2024-09-15T23:12:31.422 INFO:teuthology.orchestra.run.smithi192.stdout: "wrpos": 144919125, 2024-09-15T23:12:31.422 INFO:teuthology.orchestra.run.smithi192.stdout: "rdpos": 0, 2024-09-15T23:12:31.422 INFO:teuthology.orchestra.run.smithi192.stdout: "jlat": { 2024-09-15T23:12:31.422 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 22644, 2024-09-15T23:12:31.422 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 116.926695885, 2024-09-15T23:12:31.422 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.005163694 2024-09-15T23:12:31.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.422 INFO:teuthology.orchestra.run.smithi192.stdout: "replayed": 0 2024-09-15T23:12:31.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.423 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_mem": { 2024-09-15T23:12:31.423 INFO:teuthology.orchestra.run.smithi192.stdout: "ino": 16, 2024-09-15T23:12:31.423 INFO:teuthology.orchestra.run.smithi192.stdout: "ino+": 14126, 2024-09-15T23:12:31.423 INFO:teuthology.orchestra.run.smithi192.stdout: "ino-": 14110, 2024-09-15T23:12:31.423 INFO:teuthology.orchestra.run.smithi192.stdout: "dir": 21, 2024-09-15T23:12:31.423 INFO:teuthology.orchestra.run.smithi192.stdout: "dir+": 80, 2024-09-15T23:12:31.423 INFO:teuthology.orchestra.run.smithi192.stdout: "dir-": 59, 2024-09-15T23:12:31.423 INFO:teuthology.orchestra.run.smithi192.stdout: "dn": 11, 2024-09-15T23:12:31.424 INFO:teuthology.orchestra.run.smithi192.stdout: "dn+": 27606, 2024-09-15T23:12:31.424 INFO:teuthology.orchestra.run.smithi192.stdout: "dn-": 27595, 2024-09-15T23:12:31.424 INFO:teuthology.orchestra.run.smithi192.stdout: "cap": 1, 2024-09-15T23:12:31.424 INFO:teuthology.orchestra.run.smithi192.stdout: "cap+": 16324, 2024-09-15T23:12:31.424 INFO:teuthology.orchestra.run.smithi192.stdout: "cap-": 16323, 2024-09-15T23:12:31.424 INFO:teuthology.orchestra.run.smithi192.stdout: "rss": 69660, 2024-09-15T23:12:31.424 INFO:teuthology.orchestra.run.smithi192.stdout: "heap": 223520 2024-09-15T23:12:31.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.425 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_server": { 2024-09-15T23:12:31.425 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_client_request": 152376, 2024-09-15T23:12:31.425 INFO:teuthology.orchestra.run.smithi192.stdout: "dispatch_server_request": 7, 2024-09-15T23:12:31.425 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_request": 111615, 2024-09-15T23:12:31.425 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_client_session": 1957, 2024-09-15T23:12:31.425 INFO:teuthology.orchestra.run.smithi192.stdout: "handle_peer_request": 2668, 2024-09-15T23:12:31.426 INFO:teuthology.orchestra.run.smithi192.stdout: "req_create_latency": { 2024-09-15T23:12:31.426 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 13978, 2024-09-15T23:12:31.426 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 88.817964595, 2024-09-15T23:12:31.426 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.006354125 2024-09-15T23:12:31.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.426 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getattr_latency": { 2024-09-15T23:12:31.426 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 4304, 2024-09-15T23:12:31.426 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 20.000348532, 2024-09-15T23:12:31.426 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.004646921 2024-09-15T23:12:31.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.427 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getfilelock_latency": { 2024-09-15T23:12:31.427 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:31.427 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:31.427 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:31.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.427 INFO:teuthology.orchestra.run.smithi192.stdout: "req_link_latency": { 2024-09-15T23:12:31.427 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:31.427 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:31.427 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:31.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.427 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookup_latency": { 2024-09-15T23:12:31.427 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 28563, 2024-09-15T23:12:31.427 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 13.628592254, 2024-09-15T23:12:31.427 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000477141 2024-09-15T23:12:31.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.428 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookuphash_latency": { 2024-09-15T23:12:31.428 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:31.428 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:31.428 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:31.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.428 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupino_latency": { 2024-09-15T23:12:31.428 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 1, 2024-09-15T23:12:31.428 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000109886, 2024-09-15T23:12:31.428 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000109886 2024-09-15T23:12:31.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.428 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupname_latency": { 2024-09-15T23:12:31.428 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:31.429 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:31.429 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:31.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.429 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupparent_latency": { 2024-09-15T23:12:31.429 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:31.429 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:31.429 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:31.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.429 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lookupsnap_latency": { 2024-09-15T23:12:31.429 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:31.429 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:31.430 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:31.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.430 INFO:teuthology.orchestra.run.smithi192.stdout: "req_lssnap_latency": { 2024-09-15T23:12:31.430 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:31.431 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:31.431 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:31.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.431 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mkdir_latency": { 2024-09-15T23:12:31.431 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:31.431 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:31.431 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:31.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.432 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mknod_latency": { 2024-09-15T23:12:31.432 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:31.432 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:31.432 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:31.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.432 INFO:teuthology.orchestra.run.smithi192.stdout: "req_mksnap_latency": { 2024-09-15T23:12:31.432 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:31.433 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:31.433 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:31.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.433 INFO:teuthology.orchestra.run.smithi192.stdout: "req_open_latency": { 2024-09-15T23:12:31.433 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:31.433 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:31.434 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:31.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.434 INFO:teuthology.orchestra.run.smithi192.stdout: "req_readdir_latency": { 2024-09-15T23:12:31.434 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 20416, 2024-09-15T23:12:31.434 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 68.868593247, 2024-09-15T23:12:31.435 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.003373265 2024-09-15T23:12:31.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.435 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rename_latency": { 2024-09-15T23:12:31.435 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 2720, 2024-09-15T23:12:31.435 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 11.841367950, 2024-09-15T23:12:31.435 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.004353444 2024-09-15T23:12:31.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.436 INFO:teuthology.orchestra.run.smithi192.stdout: "req_renamesnap_latency": { 2024-09-15T23:12:31.436 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:31.436 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:31.436 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:31.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.437 INFO:teuthology.orchestra.run.smithi192.stdout: "req_snapdiff_latency": { 2024-09-15T23:12:31.437 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:31.437 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:31.437 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:31.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.437 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmdir_latency": { 2024-09-15T23:12:31.437 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 9, 2024-09-15T23:12:31.438 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.031652270, 2024-09-15T23:12:31.438 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.003516918 2024-09-15T23:12:31.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.438 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmsnap_latency": { 2024-09-15T23:12:31.438 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:31.438 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:31.438 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:31.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.439 INFO:teuthology.orchestra.run.smithi192.stdout: "req_rmxattr_latency": { 2024-09-15T23:12:31.439 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:31.439 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:31.439 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:31.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.439 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setattr_latency": { 2024-09-15T23:12:31.440 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 19, 2024-09-15T23:12:31.440 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.131902307, 2024-09-15T23:12:31.440 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.006942226 2024-09-15T23:12:31.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.440 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setdirlayout_latency": { 2024-09-15T23:12:31.440 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:31.441 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:31.441 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:31.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.441 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setfilelock_latency": { 2024-09-15T23:12:31.441 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 412, 2024-09-15T23:12:31.441 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.196687189, 2024-09-15T23:12:31.441 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000477396 2024-09-15T23:12:31.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.442 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setlayout_latency": { 2024-09-15T23:12:31.442 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:31.442 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:31.442 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:31.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.442 INFO:teuthology.orchestra.run.smithi192.stdout: "req_setxattr_latency": { 2024-09-15T23:12:31.442 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:31.442 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:31.442 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:31.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.443 INFO:teuthology.orchestra.run.smithi192.stdout: "req_symlink_latency": { 2024-09-15T23:12:31.443 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:31.443 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:31.443 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:31.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.443 INFO:teuthology.orchestra.run.smithi192.stdout: "req_unlink_latency": { 2024-09-15T23:12:31.443 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 14057, 2024-09-15T23:12:31.443 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 37.408628046, 2024-09-15T23:12:31.443 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.002661209 2024-09-15T23:12:31.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.444 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_revoke_eviction": 0, 2024-09-15T23:12:31.444 INFO:teuthology.orchestra.run.smithi192.stdout: "cap_acquisition_throttle": 0, 2024-09-15T23:12:31.444 INFO:teuthology.orchestra.run.smithi192.stdout: "req_getvxattr_latency": { 2024-09-15T23:12:31.444 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:31.444 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:31.444 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:31.444 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:31.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.444 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_sessions": { 2024-09-15T23:12:31.444 INFO:teuthology.orchestra.run.smithi192.stdout: "session_count": 1, 2024-09-15T23:12:31.444 INFO:teuthology.orchestra.run.smithi192.stdout: "session_add": 3, 2024-09-15T23:12:31.444 INFO:teuthology.orchestra.run.smithi192.stdout: "session_remove": 2, 2024-09-15T23:12:31.445 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_open": 1, 2024-09-15T23:12:31.445 INFO:teuthology.orchestra.run.smithi192.stdout: "sessions_stale": 0, 2024-09-15T23:12:31.445 INFO:teuthology.orchestra.run.smithi192.stdout: "total_load": 11278, 2024-09-15T23:12:31.445 INFO:teuthology.orchestra.run.smithi192.stdout: "average_load": 11278, 2024-09-15T23:12:31.445 INFO:teuthology.orchestra.run.smithi192.stdout: "avg_session_uptime": 1561, 2024-09-15T23:12:31.445 INFO:teuthology.orchestra.run.smithi192.stdout: "mdthresh_evicted": 0 2024-09-15T23:12:31.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.445 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2024-09-15T23:12:31.445 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2024-09-15T23:12:31.445 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2024-09-15T23:12:31.445 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T23:12:31.446 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2024-09-15T23:12:31.446 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T23:12:31.446 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2024-09-15T23:12:31.446 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T23:12:31.446 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T23:12:31.446 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T23:12:31.446 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T23:12:31.446 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T23:12:31.446 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2024-09-15T23:12:31.447 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T23:12:31.447 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T23:12:31.447 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2024-09-15T23:12:31.447 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2024-09-15T23:12:31.447 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2024-09-15T23:12:31.447 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2024-09-15T23:12:31.447 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T23:12:31.447 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T23:12:31.447 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T23:12:31.448 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T23:12:31.448 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T23:12:31.448 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2024-09-15T23:12:31.448 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2024-09-15T23:12:31.448 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2024-09-15T23:12:31.448 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T23:12:31.448 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T23:12:31.448 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2024-09-15T23:12:31.448 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2024-09-15T23:12:31.449 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2024-09-15T23:12:31.449 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2024-09-15T23:12:31.449 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T23:12:31.449 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2024-09-15T23:12:31.449 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T23:12:31.449 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2024-09-15T23:12:31.449 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 284674, 2024-09-15T23:12:31.449 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 118, 2024-09-15T23:12:31.449 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 440, 2024-09-15T23:12:31.449 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 5, 2024-09-15T23:12:31.450 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2024-09-15T23:12:31.450 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2024-09-15T23:12:31.450 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2024-09-15T23:12:31.450 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2024-09-15T23:12:31.450 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2024-09-15T23:12:31.450 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2024-09-15T23:12:31.450 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 7304, 2024-09-15T23:12:31.451 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 150, 2024-09-15T23:12:31.451 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T23:12:31.451 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2024-09-15T23:12:31.451 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2024-09-15T23:12:31.451 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2024-09-15T23:12:31.451 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 73296, 2024-09-15T23:12:31.451 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 336, 2024-09-15T23:12:31.451 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2024-09-15T23:12:31.451 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2024-09-15T23:12:31.452 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2024-09-15T23:12:31.452 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2024-09-15T23:12:31.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.452 INFO:teuthology.orchestra.run.smithi192.stdout: "objecter": { 2024-09-15T23:12:31.452 INFO:teuthology.orchestra.run.smithi192.stdout: "op_active": 0, 2024-09-15T23:12:31.452 INFO:teuthology.orchestra.run.smithi192.stdout: "op_laggy": 0, 2024-09-15T23:12:31.452 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send": 38665, 2024-09-15T23:12:31.453 INFO:teuthology.orchestra.run.smithi192.stdout: "op_send_bytes": 143701024, 2024-09-15T23:12:31.453 INFO:teuthology.orchestra.run.smithi192.stdout: "op_resend": 28, 2024-09-15T23:12:31.453 INFO:teuthology.orchestra.run.smithi192.stdout: "op_reply": 38665, 2024-09-15T23:12:31.453 INFO:teuthology.orchestra.run.smithi192.stdout: "op_latency": { 2024-09-15T23:12:31.453 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 38665, 2024-09-15T23:12:31.453 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 227.836519645, 2024-09-15T23:12:31.453 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.005892577 2024-09-15T23:12:31.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.453 INFO:teuthology.orchestra.run.smithi192.stdout: "op_inflight": 0, 2024-09-15T23:12:31.454 INFO:teuthology.orchestra.run.smithi192.stdout: "oplen_avg": { 2024-09-15T23:12:31.454 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 38665, 2024-09-15T23:12:31.454 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 39466 2024-09-15T23:12:31.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.454 INFO:teuthology.orchestra.run.smithi192.stdout: "op": 38665, 2024-09-15T23:12:31.454 INFO:teuthology.orchestra.run.smithi192.stdout: "op_r": 503, 2024-09-15T23:12:31.454 INFO:teuthology.orchestra.run.smithi192.stdout: "op_w": 38162, 2024-09-15T23:12:31.454 INFO:teuthology.orchestra.run.smithi192.stdout: "op_rmw": 0, 2024-09-15T23:12:31.454 INFO:teuthology.orchestra.run.smithi192.stdout: "op_pg": 0, 2024-09-15T23:12:31.455 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_stat": 0, 2024-09-15T23:12:31.455 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_create": 10, 2024-09-15T23:12:31.455 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_read": 503, 2024-09-15T23:12:31.455 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_write": 23147, 2024-09-15T23:12:31.455 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writefull": 331, 2024-09-15T23:12:31.455 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_writesame": 0, 2024-09-15T23:12:31.455 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_append": 0, 2024-09-15T23:12:31.455 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_zero": 0, 2024-09-15T23:12:31.455 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_truncate": 0, 2024-09-15T23:12:31.456 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_delete": 14111, 2024-09-15T23:12:31.456 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_mapext": 0, 2024-09-15T23:12:31.456 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_sparse_read": 0, 2024-09-15T23:12:31.456 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_clonerange": 0, 2024-09-15T23:12:31.456 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_getxattr": 0, 2024-09-15T23:12:31.456 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_setxattr": 20, 2024-09-15T23:12:31.456 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_cmpxattr": 0, 2024-09-15T23:12:31.456 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_rmxattr": 0, 2024-09-15T23:12:31.456 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_resetxattrs": 0, 2024-09-15T23:12:31.456 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_call": 0, 2024-09-15T23:12:31.456 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_watch": 0, 2024-09-15T23:12:31.456 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_notify": 0, 2024-09-15T23:12:31.456 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T23:12:31.457 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls": 0, 2024-09-15T23:12:31.457 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_pgls_filter": 0, 2024-09-15T23:12:31.457 INFO:teuthology.orchestra.run.smithi192.stdout: "osdop_other": 0, 2024-09-15T23:12:31.457 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_active": 0, 2024-09-15T23:12:31.457 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_send": 0, 2024-09-15T23:12:31.457 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_resend": 0, 2024-09-15T23:12:31.457 INFO:teuthology.orchestra.run.smithi192.stdout: "linger_ping": 0, 2024-09-15T23:12:31.457 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_active": 0, 2024-09-15T23:12:31.457 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_send": 0, 2024-09-15T23:12:31.457 INFO:teuthology.orchestra.run.smithi192.stdout: "poolop_resend": 0, 2024-09-15T23:12:31.457 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_active": 0, 2024-09-15T23:12:31.457 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_send": 0, 2024-09-15T23:12:31.457 INFO:teuthology.orchestra.run.smithi192.stdout: "poolstat_resend": 0, 2024-09-15T23:12:31.458 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_active": 0, 2024-09-15T23:12:31.458 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_send": 0, 2024-09-15T23:12:31.458 INFO:teuthology.orchestra.run.smithi192.stdout: "statfs_resend": 0, 2024-09-15T23:12:31.458 INFO:teuthology.orchestra.run.smithi192.stdout: "command_active": 0, 2024-09-15T23:12:31.458 INFO:teuthology.orchestra.run.smithi192.stdout: "command_send": 0, 2024-09-15T23:12:31.458 INFO:teuthology.orchestra.run.smithi192.stdout: "command_resend": 0, 2024-09-15T23:12:31.458 INFO:teuthology.orchestra.run.smithi192.stdout: "map_epoch": 89, 2024-09-15T23:12:31.458 INFO:teuthology.orchestra.run.smithi192.stdout: "map_full": 0, 2024-09-15T23:12:31.458 INFO:teuthology.orchestra.run.smithi192.stdout: "map_inc": 3, 2024-09-15T23:12:31.458 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_sessions": 12, 2024-09-15T23:12:31.458 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_open": 16, 2024-09-15T23:12:31.458 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_session_close": 4, 2024-09-15T23:12:31.458 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_laggy": 0, 2024-09-15T23:12:31.458 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_wr": 1002, 2024-09-15T23:12:31.459 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_rd": 0, 2024-09-15T23:12:31.459 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_del": 342 2024-09-15T23:12:31.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.459 INFO:teuthology.orchestra.run.smithi192.stdout: "oft": { 2024-09-15T23:12:31.459 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_objs": 1, 2024-09-15T23:12:31.459 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_kv_pairs": 0, 2024-09-15T23:12:31.459 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_updates": 0, 2024-09-15T23:12:31.459 INFO:teuthology.orchestra.run.smithi192.stdout: "omap_total_removes": 0 2024-09-15T23:12:31.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.459 INFO:teuthology.orchestra.run.smithi192.stdout: "osd-slow-ops": { 2024-09-15T23:12:31.459 INFO:teuthology.orchestra.run.smithi192.stdout: "slow_ops_count": 1 2024-09-15T23:12:31.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.459 INFO:teuthology.orchestra.run.smithi192.stdout: "purge_queue": { 2024-09-15T23:12:31.459 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops": 0, 2024-09-15T23:12:31.460 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_ops_high_water": 10, 2024-09-15T23:12:31.460 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing": 0, 2024-09-15T23:12:31.460 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executing_high_water": 10, 2024-09-15T23:12:31.460 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed_ops": 14066, 2024-09-15T23:12:31.460 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_executed": 14066, 2024-09-15T23:12:31.460 INFO:teuthology.orchestra.run.smithi192.stdout: "pq_item_in_journal": 0 2024-09-15T23:12:31.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.460 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T23:12:31.460 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-15T23:12:31.460 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-09-15T23:12:31.460 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-15T23:12:31.460 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 396270, 2024-09-15T23:12:31.460 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 79044605, 2024-09-15T23:12:31.460 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:31.461 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 396270, 2024-09-15T23:12:31.461 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-09-15T23:12:31.461 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-09-15T23:12:31.461 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 396270, 2024-09-15T23:12:31.461 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 79044605, 2024-09-15T23:12:31.461 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-15T23:12:31.461 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:31.461 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:31.462 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:31.462 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:31.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.462 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_bytes": { 2024-09-15T23:12:31.462 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-15T23:12:31.462 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-09-15T23:12:31.462 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-15T23:12:31.462 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-09-15T23:12:31.462 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-09-15T23:12:31.462 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:31.463 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-09-15T23:12:31.463 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 38665, 2024-09-15T23:12:31.463 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 145121690, 2024-09-15T23:12:31.463 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 38665, 2024-09-15T23:12:31.463 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 145121690, 2024-09-15T23:12:31.463 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-15T23:12:31.463 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:31.464 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:31.464 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:31.464 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:31.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.464 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-objecter_ops": { 2024-09-15T23:12:31.464 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-15T23:12:31.464 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 1024, 2024-09-15T23:12:31.465 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-15T23:12:31.465 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 0, 2024-09-15T23:12:31.465 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 0, 2024-09-15T23:12:31.465 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:31.465 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 0, 2024-09-15T23:12:31.465 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 38665, 2024-09-15T23:12:31.465 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 38665, 2024-09-15T23:12:31.465 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 38665, 2024-09-15T23:12:31.465 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 38665, 2024-09-15T23:12:31.465 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-15T23:12:31.465 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:31.465 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:31.466 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:31.466 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:31.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.466 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle": { 2024-09-15T23:12:31.466 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-15T23:12:31.466 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-09-15T23:12:31.466 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-15T23:12:31.466 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 14066, 2024-09-15T23:12:31.466 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 1420666, 2024-09-15T23:12:31.466 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:31.466 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 14066, 2024-09-15T23:12:31.466 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-09-15T23:12:31.467 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-09-15T23:12:31.467 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 503, 2024-09-15T23:12:31.467 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 1420666, 2024-09-15T23:12:31.467 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-15T23:12:31.467 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:31.467 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:31.467 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:31.467 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:31.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:31.467 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-write_buf_throttle-0x55fb007c0960": { 2024-09-15T23:12:31.467 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-15T23:12:31.467 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 3758096384, 2024-09-15T23:12:31.467 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-15T23:12:31.468 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 70558, 2024-09-15T23:12:31.468 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 140724821, 2024-09-15T23:12:31.468 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:31.468 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 70558, 2024-09-15T23:12:31.468 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-09-15T23:12:31.468 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-09-15T23:12:31.468 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 22644, 2024-09-15T23:12:31.468 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 140724821, 2024-09-15T23:12:31.468 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-15T23:12:31.468 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:31.468 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:31.468 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:31.468 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:31.469 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:31.469 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T23:12:32.209 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:32 smithi192 ceph-mon[35427]: pgmap v1149: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-15T23:12:32.211 INFO:tasks.check_counter:Daemon mds.i mds.exported=3 2024-09-15T23:12:32.211 INFO:tasks.check_counter:Daemon mds.i mds.imported=12 2024-09-15T23:12:32.211 DEBUG:tasks.check_counter:Getting stats from l 2024-09-15T23:12:32.211 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-15T23:12:32.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:32 smithi121 ceph-mon[34815]: pgmap v1149: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-15T23:12:32.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:32 smithi111 ceph-mon[30629]: pgmap v1149: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-09-15T23:12:32.453 INFO:teuthology.orchestra.run.smithi192.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T23:12:33.063 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T23:12:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T23:12:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 467, 2024-09-15T23:12:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 462, 2024-09-15T23:12:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 827376, 2024-09-15T23:12:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 285872, 2024-09-15T23:12:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 4, 2024-09-15T23:12:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 0, 2024-09-15T23:12:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.058444471, 2024-09-15T23:12:33.064 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.027410113, 2024-09-15T23:12:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.038107639, 2024-09-15T23:12:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T23:12:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 461, 2024-09-15T23:12:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.022767125, 2024-09-15T23:12:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000049386 2024-09-15T23:12:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:33.065 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 827376, 2024-09-15T23:12:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 285872 2024-09-15T23:12:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T23:12:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 6, 2024-09-15T23:12:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 355, 2024-09-15T23:12:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 3344, 2024-09-15T23:12:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 220016, 2024-09-15T23:12:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 4, 2024-09-15T23:12:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 2, 2024-09-15T23:12:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.020246167, 2024-09-15T23:12:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.014117795, 2024-09-15T23:12:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.002993466, 2024-09-15T23:12:33.066 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T23:12:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 354, 2024-09-15T23:12:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.022857923, 2024-09-15T23:12:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000064570 2024-09-15T23:12:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-15T23:12:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 220016 2024-09-15T23:12:33.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T23:12:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_messages": 0, 2024-09-15T23:12:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages": 2, 2024-09-15T23:12:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_bytes": 0, 2024-09-15T23:12:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_bytes": 0, 2024-09-15T23:12:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_created_connections": 2, 2024-09-15T23:12:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_active_connections": 1, 2024-09-15T23:12:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_total_time": 0.000258217, 2024-09-15T23:12:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_send_time": 0.000000000, 2024-09-15T23:12:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_recv_time": 0.000164845, 2024-09-15T23:12:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T23:12:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T23:12:33.068 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_handle_ack_lat": { 2024-09-15T23:12:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-15T23:12:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: "msgr_send_encrypted_bytes": 0 2024-09-15T23:12:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: "cct": { 2024-09-15T23:12:33.069 INFO:teuthology.orchestra.run.smithi192.stdout: "total_workers": 0, 2024-09-15T23:12:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "unhealthy_workers": 0 2024-09-15T23:12:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "mempool": { 2024-09-15T23:12:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_bytes": 0, 2024-09-15T23:12:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "bloom_filter_items": 0, 2024-09-15T23:12:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T23:12:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_alloc_items": 0, 2024-09-15T23:12:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T23:12:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_data_items": 0, 2024-09-15T23:12:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T23:12:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T23:12:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T23:12:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T23:12:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T23:12:33.070 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_other_items": 0, 2024-09-15T23:12:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T23:12:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T23:12:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_bytes": 0, 2024-09-15T23:12:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_extent_items": 0, 2024-09-15T23:12:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_bytes": 0, 2024-09-15T23:12:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_blob_items": 0, 2024-09-15T23:12:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T23:12:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T23:12:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T23:12:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T23:12:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T23:12:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_fsck_items": 0, 2024-09-15T23:12:33.071 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_bytes": 0, 2024-09-15T23:12:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_txc_items": 0, 2024-09-15T23:12:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T23:12:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T23:12:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_bytes": 0, 2024-09-15T23:12:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "bluestore_writing_items": 0, 2024-09-15T23:12:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_bytes": 0, 2024-09-15T23:12:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_items": 0, 2024-09-15T23:12:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T23:12:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_reader_items": 0, 2024-09-15T23:12:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T23:12:33.072 INFO:teuthology.orchestra.run.smithi192.stdout: "bluefs_file_writer_items": 0, 2024-09-15T23:12:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_bytes": 148744, 2024-09-15T23:12:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_anon_items": 30, 2024-09-15T23:12:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_bytes": 0, 2024-09-15T23:12:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "buffer_meta_items": 0, 2024-09-15T23:12:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_bytes": 0, 2024-09-15T23:12:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_items": 0, 2024-09-15T23:12:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_bytes": 0, 2024-09-15T23:12:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_mapbl_items": 0, 2024-09-15T23:12:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_bytes": 0, 2024-09-15T23:12:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_pglog_items": 0, 2024-09-15T23:12:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_bytes": 0, 2024-09-15T23:12:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_items": 0, 2024-09-15T23:12:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T23:12:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "osdmap_mapping_items": 0, 2024-09-15T23:12:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_bytes": 0, 2024-09-15T23:12:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "pgmap_items": 0, 2024-09-15T23:12:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_bytes": 928, 2024-09-15T23:12:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_co_items": 2, 2024-09-15T23:12:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_bytes": 0, 2024-09-15T23:12:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_1_items": 0, 2024-09-15T23:12:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_bytes": 0, 2024-09-15T23:12:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "unittest_2_items": 0 2024-09-15T23:12:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T23:12:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T23:12:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "val": 0, 2024-09-15T23:12:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "max": 104857600, 2024-09-15T23:12:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "get_started": 0, 2024-09-15T23:12:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "get": 469, 2024-09-15T23:12:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "get_sum": 761321, 2024-09-15T23:12:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_fail": 0, 2024-09-15T23:12:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "get_or_fail_success": 469, 2024-09-15T23:12:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "take": 0, 2024-09-15T23:12:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "take_sum": 0, 2024-09-15T23:12:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "put": 469, 2024-09-15T23:12:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "put_sum": 761321, 2024-09-15T23:12:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "wait": { 2024-09-15T23:12:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "avgcount": 0, 2024-09-15T23:12:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "sum": 0.000000000, 2024-09-15T23:12:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "avgtime": 0.000000000 2024-09-15T23:12:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T23:12:33.076 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T23:12:33.795 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-15T23:12:33.795 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-15T23:12:33.795 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-15T23:12:33.806 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-15T23:12:33.806 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-15T23:12:33.806 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:12:33.849 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2024-09-15T23:12:33.849 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T23:12:33.928 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-15T23:12:34.338 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:12:34.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:34 smithi121 ceph-mon[34815]: pgmap v1150: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T23:12:34.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:34 smithi111 ceph-mon[30629]: pgmap v1150: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T23:12:34.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:34 smithi192 ceph-mon[35427]: pgmap v1150: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T23:12:34.927 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.920+0000 7f4d59e00640 1 -- 172.21.15.111:0/33932795 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d541012a0 msgr2=0x7f4d54105770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:34.927 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.920+0000 7f4d59e00640 1 --2- 172.21.15.111:0/33932795 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d541012a0 0x7f4d54105770 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4d40009900 tx=0x7f4d4002a470 comp rx=0 tx=0).stop 2024-09-15T23:12:34.927 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.920+0000 7f4d59e00640 1 -- 172.21.15.111:0/33932795 shutdown_connections 2024-09-15T23:12:34.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.920+0000 7f4d59e00640 1 --2- 172.21.15.111:0/33932795 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d54105d90 0x7f4d54106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:34.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.920+0000 7f4d59e00640 1 --2- 172.21.15.111:0/33932795 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d541012a0 0x7f4d54105770 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:34.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.920+0000 7f4d59e00640 1 --2- 172.21.15.111:0/33932795 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d54100980 0x7f4d54100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:34.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.920+0000 7f4d59e00640 1 -- 172.21.15.111:0/33932795 >> 172.21.15.111:0/33932795 conn(0x7f4d540fc610 msgr2=0x7f4d540fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:12:34.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.920+0000 7f4d59e00640 1 -- 172.21.15.111:0/33932795 shutdown_connections 2024-09-15T23:12:34.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.920+0000 7f4d59e00640 1 -- 172.21.15.111:0/33932795 wait complete. 2024-09-15T23:12:34.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.921+0000 7f4d59e00640 1 Processor -- start 2024-09-15T23:12:34.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.921+0000 7f4d59e00640 1 -- start start 2024-09-15T23:12:34.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.922+0000 7f4d59e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d54100980 0x7f4d541a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:34.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.922+0000 7f4d59e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d541012a0 0x7f4d541a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:34.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.922+0000 7f4d59e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d54105d90 0x7f4d5419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:34.928 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.922+0000 7f4d59e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f4d541183b0 con 0x7f4d541012a0 2024-09-15T23:12:34.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.922+0000 7f4d59e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4d54118230 con 0x7f4d54100980 2024-09-15T23:12:34.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.922+0000 7f4d59e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4d54118530 con 0x7f4d54105d90 2024-09-15T23:12:34.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.922+0000 7f4d53e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d541012a0 0x7f4d541a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:34.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.922+0000 7f4d53e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d541012a0 0x7f4d541a4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50132/0 (socket says 172.21.15.111:50132) 2024-09-15T23:12:34.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.922+0000 7f4d53e00640 1 -- 172.21.15.111:0/4124196192 learned_addr learned my addr 172.21.15.111:0/4124196192 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:12:34.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.922+0000 7f4d59400640 1 --2- 172.21.15.111:0/4124196192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d54105d90 0x7f4d5419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:34.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.922+0000 7f4d58a00640 1 --2- 172.21.15.111:0/4124196192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d54100980 0x7f4d541a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:34.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.922+0000 7f4d53e00640 1 -- 172.21.15.111:0/4124196192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d54105d90 msgr2=0x7f4d5419e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:34.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.922+0000 7f4d53e00640 1 --2- 172.21.15.111:0/4124196192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d54105d90 0x7f4d5419e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:34.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.922+0000 7f4d53e00640 1 -- 172.21.15.111:0/4124196192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d54100980 msgr2=0x7f4d541a45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:34.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.922+0000 7f4d53e00640 1 --2- 172.21.15.111:0/4124196192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d54100980 0x7f4d541a45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:34.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.922+0000 7f4d53e00640 1 -- 172.21.15.111:0/4124196192 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d5419ef10 con 0x7f4d541012a0 2024-09-15T23:12:34.929 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.923+0000 7f4d53e00640 1 --2- 172.21.15.111:0/4124196192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d541012a0 0x7f4d541a4b10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4d40006010 tx=0x7f4d40031b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:12:34.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.923+0000 7f4d51600640 1 -- 172.21.15.111:0/4124196192 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4003f030 con 0x7f4d541012a0 2024-09-15T23:12:34.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.923+0000 7f4d59e00640 1 -- 172.21.15.111:0/4124196192 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d5419f1a0 con 0x7f4d541012a0 2024-09-15T23:12:34.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.923+0000 7f4d51600640 1 -- 172.21.15.111:0/4124196192 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d400043d0 con 0x7f4d541012a0 2024-09-15T23:12:34.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.923+0000 7f4d59e00640 1 -- 172.21.15.111:0/4124196192 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d541ab380 con 0x7f4d541012a0 2024-09-15T23:12:34.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.923+0000 7f4d51600640 1 -- 172.21.15.111:0/4124196192 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40049060 con 0x7f4d541012a0 2024-09-15T23:12:34.930 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.924+0000 7f4d59e00640 1 -- 172.21.15.111:0/4124196192 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d5406a850 con 0x7f4d541012a0 2024-09-15T23:12:34.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.927+0000 7f4d51600640 1 -- 172.21.15.111:0/4124196192 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f4d40042030 con 0x7f4d541012a0 2024-09-15T23:12:34.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.928+0000 7f4d51600640 1 --2- 172.21.15.111:0/4124196192 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4d3c0a3240 0x7f4d3c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:34.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.928+0000 7f4d51600640 1 -- 172.21.15.111:0/4124196192 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f4d400efe70 con 0x7f4d541012a0 2024-09-15T23:12:34.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.928+0000 7f4d58a00640 1 --2- 172.21.15.111:0/4124196192 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4d3c0a3240 0x7f4d3c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:34.931 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.929+0000 7f4d58a00640 1 --2- 172.21.15.111:0/4124196192 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4d3c0a3240 0x7f4d3c0a5700 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4d48009bd0 tx=0x7f4d48005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:12:34.933 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:34.932+0000 7f4d51600640 1 -- 172.21.15.111:0/4124196192 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4d400bc6a0 con 0x7f4d541012a0 2024-09-15T23:12:35.123 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:35.122+0000 7f4d59e00640 1 -- 172.21.15.111:0/4124196192 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4d541ab550 con 0x7f4d541012a0 2024-09-15T23:12:35.124 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:35.122+0000 7f4d51600640 1 -- 172.21.15.111:0/4124196192 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+602 (secure 0 0 0) 0x7f4d400c09a0 con 0x7f4d541012a0 2024-09-15T23:12:35.124 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:6801/2107775445 2024-09-16T22:36:50.357551+0000 2024-09-15T23:12:35.124 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:6800/2107775445 2024-09-16T22:36:50.357551+0000 2024-09-15T23:12:35.125 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:0/2394746540 2024-09-16T22:36:50.357551+0000 2024-09-15T23:12:35.125 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:0/4140020843 2024-09-16T22:36:50.357551+0000 2024-09-15T23:12:35.125 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:0/2114354263 2024-09-16T22:36:50.357551+0000 2024-09-15T23:12:35.125 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:0/1054173642 2024-09-16T22:36:22.652875+0000 2024-09-15T23:12:35.125 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:0/3408238004 2024-09-16T22:36:22.652875+0000 2024-09-15T23:12:35.125 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:0/2879044772 2024-09-16T22:36:22.652875+0000 2024-09-15T23:12:35.125 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:6801/3350425104 2024-09-16T22:36:22.652875+0000 2024-09-15T23:12:35.125 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.15.111:6800/3350425104 2024-09-16T22:36:22.652875+0000 2024-09-15T23:12:35.126 INFO:teuthology.orchestra.run.smithi111.stderr:listed 10 entries 2024-09-15T23:12:35.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:35.125+0000 7f4d59e00640 1 -- 172.21.15.111:0/4124196192 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4d3c0a3240 msgr2=0x7f4d3c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:35.126 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:35.125+0000 7f4d59e00640 1 --2- 172.21.15.111:0/4124196192 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4d3c0a3240 0x7f4d3c0a5700 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4d48009bd0 tx=0x7f4d48005f90 comp rx=0 tx=0).stop 2024-09-15T23:12:35.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:35.125+0000 7f4d59e00640 1 -- 172.21.15.111:0/4124196192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d541012a0 msgr2=0x7f4d541a4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:35.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:35.125+0000 7f4d59e00640 1 --2- 172.21.15.111:0/4124196192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d541012a0 0x7f4d541a4b10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4d40006010 tx=0x7f4d40031b20 comp rx=0 tx=0).stop 2024-09-15T23:12:35.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:35.125+0000 7f4d59e00640 1 -- 172.21.15.111:0/4124196192 shutdown_connections 2024-09-15T23:12:35.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:35.125+0000 7f4d59e00640 1 --2- 172.21.15.111:0/4124196192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d54105d90 0x7f4d5419e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:35.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:35.125+0000 7f4d59e00640 1 --2- 172.21.15.111:0/4124196192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4d541012a0 0x7f4d541a4b10 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:35.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:35.125+0000 7f4d59e00640 1 --2- 172.21.15.111:0/4124196192 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f4d3c0a3240 0x7f4d3c0a5700 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:35.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:35.125+0000 7f4d59e00640 1 --2- 172.21.15.111:0/4124196192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d54100980 0x7f4d541a45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:35.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:35.125+0000 7f4d59e00640 1 -- 172.21.15.111:0/4124196192 >> 172.21.15.111:0/4124196192 conn(0x7f4d540fc610 msgr2=0x7f4d54103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:12:35.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:35.125+0000 7f4d59e00640 1 -- 172.21.15.111:0/4124196192 shutdown_connections 2024-09-15T23:12:35.127 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:35.125+0000 7f4d59e00640 1 -- 172.21.15.111:0/4124196192 wait complete. 2024-09-15T23:12:35.951 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-15T23:12:35.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:12:35.952 DEBUG:teuthology.orchestra.run.smithi111:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-15T23:12:36.102 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2024-09-15T23:12:36.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:12:36.102 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-15T23:12:36.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:36 smithi111 ceph-mon[30629]: pgmap v1151: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T23:12:36.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:36 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/4124196192' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T23:12:36.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:12:36.195 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2024-09-15T23:12:36.278 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-15T23:12:36.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:12:36.279 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-15T23:12:36.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:36 smithi121 ceph-mon[34815]: pgmap v1151: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T23:12:36.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:36 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/4124196192' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T23:12:36.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T23:12:36.403 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link delete ceph-brx 2024-09-15T23:12:36.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:36 smithi192 ceph-mon[35427]: pgmap v1151: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T23:12:36.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:36 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/4124196192' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T23:12:36.714 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-15T23:12:36.726 INFO:tasks.cephadm:Teardown begin 2024-09-15T23:12:36.727 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-15T23:12:36.727 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T23:12:37.097 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:12:37.771 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.739+0000 7fd204600640 1 -- 172.21.15.111:0/3027504622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1fc1049a0 msgr2=0x7fd1fc1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:37.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.739+0000 7fd204600640 1 --2- 172.21.15.111:0/3027504622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1fc1049a0 0x7fd1fc1089f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8009900 tx=0x7fd1e802a410 comp rx=0 tx=0).stop 2024-09-15T23:12:37.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.739+0000 7fd204600640 1 -- 172.21.15.111:0/3027504622 shutdown_connections 2024-09-15T23:12:37.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.739+0000 7fd204600640 1 --2- 172.21.15.111:0/3027504622 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1fc109120 0x7fd1fc110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:37.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.739+0000 7fd204600640 1 --2- 172.21.15.111:0/3027504622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1fc1049a0 0x7fd1fc1089f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:37.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.739+0000 7fd204600640 1 --2- 172.21.15.111:0/3027504622 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1fc103ff0 0x7fd1fc1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:37.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.739+0000 7fd204600640 1 -- 172.21.15.111:0/3027504622 >> 172.21.15.111:0/3027504622 conn(0x7fd1fc0ffa90 msgr2=0x7fd1fc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:12:37.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.739+0000 7fd204600640 1 -- 172.21.15.111:0/3027504622 shutdown_connections 2024-09-15T23:12:37.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.739+0000 7fd204600640 1 -- 172.21.15.111:0/3027504622 wait complete. 2024-09-15T23:12:37.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.741+0000 7fd204600640 1 Processor -- start 2024-09-15T23:12:37.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.741+0000 7fd204600640 1 -- start start 2024-09-15T23:12:37.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.741+0000 7fd204600640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1fc103ff0 0x7fd1fc19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:37.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.741+0000 7fd204600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1fc1049a0 0x7fd1fc19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:37.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.741+0000 7fd204600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1fc109120 0x7fd1fc19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:37.772 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.741+0000 7fd204600640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc114170 con 0x7fd1fc103ff0 2024-09-15T23:12:37.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.741+0000 7fd204600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc113ff0 con 0x7fd1fc109120 2024-09-15T23:12:37.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.741+0000 7fd204600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc1142f0 con 0x7fd1fc1049a0 2024-09-15T23:12:37.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.741+0000 7fd201c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1fc103ff0 0x7fd1fc19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:37.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.742+0000 7fd201c00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1fc103ff0 0x7fd1fc19a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50144/0 (socket says 172.21.15.111:50144) 2024-09-15T23:12:37.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.742+0000 7fd201c00640 1 -- 172.21.15.111:0/1808292125 learned_addr learned my addr 172.21.15.111:0/1808292125 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:12:37.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.742+0000 7fd202600640 1 --2- 172.21.15.111:0/1808292125 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1fc109120 0x7fd1fc19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:37.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.742+0000 7fd201200640 1 --2- 172.21.15.111:0/1808292125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1fc1049a0 0x7fd1fc19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:37.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.742+0000 7fd201c00640 1 -- 172.21.15.111:0/1808292125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1fc1049a0 msgr2=0x7fd1fc19ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:37.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.742+0000 7fd201c00640 1 --2- 172.21.15.111:0/1808292125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1fc1049a0 0x7fd1fc19ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:37.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.742+0000 7fd201c00640 1 -- 172.21.15.111:0/1808292125 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1fc109120 msgr2=0x7fd1fc19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:37.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.742+0000 7fd201c00640 1 --2- 172.21.15.111:0/1808292125 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1fc109120 0x7fd1fc19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:37.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.742+0000 7fd201c00640 1 -- 172.21.15.111:0/1808292125 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1fc19f600 con 0x7fd1fc103ff0 2024-09-15T23:12:37.773 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.742+0000 7fd201200640 1 --2- 172.21.15.111:0/1808292125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1fc1049a0 0x7fd1fc19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:12:37.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.742+0000 7fd201c00640 1 --2- 172.21.15.111:0/1808292125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1fc103ff0 0x7fd1fc19a650 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd1f000e9f0 tx=0x7fd1f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:12:37.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.742+0000 7fd1fa000640 1 -- 172.21.15.111:0/1808292125 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f000cd80 con 0x7fd1fc103ff0 2024-09-15T23:12:37.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.742+0000 7fd1fa000640 1 -- 172.21.15.111:0/1808292125 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1f0010910 con 0x7fd1fc103ff0 2024-09-15T23:12:37.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.743+0000 7fd1fa000640 1 -- 172.21.15.111:0/1808292125 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0018990 con 0x7fd1fc103ff0 2024-09-15T23:12:37.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.743+0000 7fd204600640 1 -- 172.21.15.111:0/1808292125 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1fc19f8f0 con 0x7fd1fc103ff0 2024-09-15T23:12:37.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.743+0000 7fd204600640 1 -- 172.21.15.111:0/1808292125 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1fc1a71d0 con 0x7fd1fc103ff0 2024-09-15T23:12:37.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.744+0000 7fd204600640 1 -- 172.21.15.111:0/1808292125 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1fc1043d0 con 0x7fd1fc103ff0 2024-09-15T23:12:37.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.745+0000 7fd1fa000640 1 -- 172.21.15.111:0/1808292125 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd1f0010ac0 con 0x7fd1fc103ff0 2024-09-15T23:12:37.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.747+0000 7fd1fa000640 1 --2- 172.21.15.111:0/1808292125 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd1e40a32c0 0x7fd1e40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:37.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.747+0000 7fd1fa000640 1 -- 172.21.15.111:0/1808292125 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fd1f00cfc20 con 0x7fd1fc103ff0 2024-09-15T23:12:37.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.747+0000 7fd201200640 1 --2- 172.21.15.111:0/1808292125 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd1e40a32c0 0x7fd1e40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:37.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.748+0000 7fd201200640 1 --2- 172.21.15.111:0/1808292125 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd1e40a32c0 0x7fd1e40a5780 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8000c00 tx=0x7fd1e8005a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:12:37.774 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.750+0000 7fd1fa000640 1 -- 172.21.15.111:0/1808292125 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd1f009c940 con 0x7fd1fc103ff0 2024-09-15T23:12:37.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.959+0000 7fd204600640 1 -- 172.21.15.111:0/1808292125 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1fc10c360 con 0x7fd1fc103ff0 2024-09-15T23:12:37.961 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.960+0000 7fd1fa000640 1 -- 172.21.15.111:0/1808292125 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11934 (secure 0 0 0) 0x7fd1f009c240 con 0x7fd1fc103ff0 2024-09-15T23:12:37.962 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T23:12:37.962 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":37,"btime":"2024-09-15T23:11:40:107019+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/453336603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":453336603},{"type":"v1","addr":"172.21.15.111:6835","nonce":453336603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/3181832287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3181832287},{"type":"v1","addr":"172.21.15.111:6837","nonce":3181832287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/4190237053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":4190237053},{"type":"v1","addr":"172.21.15.111:6839","nonce":4190237053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14502,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6841/267728828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6840","nonce":267728828},{"type":"v1","addr":"172.21.15.111:6841","nonce":267728828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/4061441359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":4061441359},{"type":"v1","addr":"172.21.15.192:6833","nonce":4061441359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/4220531578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":4220531578},{"type":"v1","addr":"172.21.15.121:6833","nonce":4220531578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:44:18.436394+0000","modified":"2024-09-15T23:11:39.957406+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24514,"mds_2":24503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3371796806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3371796806},{"type":"v1","addr":"172.21.15.121:6835","nonce":3371796806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/4080914921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":4080914921},{"type":"v1","addr":"172.21.15.192:6835","nonce":4080914921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.121:6837/1443772791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1443772791},{"type":"v1","addr":"172.21.15.121:6837","nonce":1443772791}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6839/800136261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":800136261},{"type":"v1","addr":"172.21.15.192:6839","nonce":800136261}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.121:6839/1797352097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1797352097},{"type":"v1","addr":"172.21.15.121:6839","nonce":1797352097}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26245":{"gid":26245,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/1721868112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1721868112},{"type":"v1","addr":"172.21.15.192:6837","nonce":1721868112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545,24514,24503]},"id":1}]} 2024-09-15T23:12:37.962 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 37 2024-09-15T23:12:37.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.962+0000 7fd204600640 1 -- 172.21.15.111:0/1808292125 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd1e40a32c0 msgr2=0x7fd1e40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:37.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.962+0000 7fd204600640 1 --2- 172.21.15.111:0/1808292125 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd1e40a32c0 0x7fd1e40a5780 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8000c00 tx=0x7fd1e8005a40 comp rx=0 tx=0).stop 2024-09-15T23:12:37.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.962+0000 7fd204600640 1 -- 172.21.15.111:0/1808292125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1fc103ff0 msgr2=0x7fd1fc19a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:37.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.962+0000 7fd204600640 1 --2- 172.21.15.111:0/1808292125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1fc103ff0 0x7fd1fc19a650 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd1f000e9f0 tx=0x7fd1f000eeb0 comp rx=0 tx=0).stop 2024-09-15T23:12:37.964 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.963+0000 7fd204600640 1 -- 172.21.15.111:0/1808292125 shutdown_connections 2024-09-15T23:12:37.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.963+0000 7fd204600640 1 --2- 172.21.15.111:0/1808292125 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd1e40a32c0 0x7fd1e40a5780 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:37.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.963+0000 7fd204600640 1 --2- 172.21.15.111:0/1808292125 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd1fc109120 0x7fd1fc19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:37.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.963+0000 7fd204600640 1 --2- 172.21.15.111:0/1808292125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1fc1049a0 0x7fd1fc19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:37.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.963+0000 7fd204600640 1 --2- 172.21.15.111:0/1808292125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1fc103ff0 0x7fd1fc19a650 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:37.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.963+0000 7fd204600640 1 -- 172.21.15.111:0/1808292125 >> 172.21.15.111:0/1808292125 conn(0x7fd1fc0ffa90 msgr2=0x7fd1fc100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:12:37.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.963+0000 7fd204600640 1 -- 172.21.15.111:0/1808292125 shutdown_connections 2024-09-15T23:12:37.965 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:37.963+0000 7fd204600640 1 -- 172.21.15.111:0/1808292125 wait complete. 2024-09-15T23:12:38.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:38 smithi121 ceph-mon[34815]: pgmap v1152: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T23:12:38.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:38 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/1808292125' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:12:38.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:38 smithi111 ceph-mon[30629]: pgmap v1152: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T23:12:38.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:38 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/1808292125' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:12:38.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:38 smithi192 ceph-mon[35427]: pgmap v1152: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-15T23:12:38.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:38 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/1808292125' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:12:38.694 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T23:12:39.064 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:12:39.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.816+0000 7fd9cd400640 1 -- 172.21.15.111:0/2014233809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9c8101de0 msgr2=0x7fd9c81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:39.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.816+0000 7fd9cd400640 1 --2- 172.21.15.111:0/2014233809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9c8101de0 0x7fd9c81021c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009900 tx=0x7fd9b802a3f0 comp rx=0 tx=0).stop 2024-09-15T23:12:39.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.816+0000 7fd9cd400640 1 -- 172.21.15.111:0/2014233809 shutdown_connections 2024-09-15T23:12:39.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.816+0000 7fd9cd400640 1 --2- 172.21.15.111:0/2014233809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9c8106f10 0x7fd9c810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:39.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.816+0000 7fd9cd400640 1 --2- 172.21.15.111:0/2014233809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9c8102790 0x7fd9c81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:39.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.816+0000 7fd9cd400640 1 --2- 172.21.15.111:0/2014233809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9c8101de0 0x7fd9c81021c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:39.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.816+0000 7fd9cd400640 1 -- 172.21.15.111:0/2014233809 >> 172.21.15.111:0/2014233809 conn(0x7fd9c80fd8c0 msgr2=0x7fd9c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:12:39.818 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.816+0000 7fd9cd400640 1 -- 172.21.15.111:0/2014233809 shutdown_connections 2024-09-15T23:12:39.819 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.816+0000 7fd9cd400640 1 -- 172.21.15.111:0/2014233809 wait complete. 2024-09-15T23:12:39.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.817+0000 7fd9cd400640 1 Processor -- start 2024-09-15T23:12:39.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.817+0000 7fd9cd400640 1 -- start start 2024-09-15T23:12:39.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.818+0000 7fd9cd400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9c8101de0 0x7fd9c819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:39.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.818+0000 7fd9cd400640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9c8102790 0x7fd9c819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:39.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.818+0000 7fd9cd400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9c8106f10 0x7fd9c819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:39.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.818+0000 7fd9cd400640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c8111f70 con 0x7fd9c8102790 2024-09-15T23:12:39.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.818+0000 7fd9cd400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c8111df0 con 0x7fd9c8106f10 2024-09-15T23:12:39.820 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.818+0000 7fd9cd400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c81120f0 con 0x7fd9c8101de0 2024-09-15T23:12:39.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.818+0000 7fd9c5e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9c8102790 0x7fd9c819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:39.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.818+0000 7fd9c5e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9c8102790 0x7fd9c819ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50166/0 (socket says 172.21.15.111:50166) 2024-09-15T23:12:39.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.818+0000 7fd9c5e00640 1 -- 172.21.15.111:0/1389635624 learned_addr learned my addr 172.21.15.111:0/1389635624 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:12:39.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.818+0000 7fd9c6800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9c8101de0 0x7fd9c819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:39.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.818+0000 7fd9c7200640 1 --2- 172.21.15.111:0/1389635624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9c8106f10 0x7fd9c819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:39.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.818+0000 7fd9c6800640 1 -- 172.21.15.111:0/1389635624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9c8106f10 msgr2=0x7fd9c819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:39.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.818+0000 7fd9c6800640 1 --2- 172.21.15.111:0/1389635624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9c8106f10 0x7fd9c819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:39.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.818+0000 7fd9c6800640 1 -- 172.21.15.111:0/1389635624 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9c8102790 msgr2=0x7fd9c819ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:39.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.818+0000 7fd9c6800640 1 --2- 172.21.15.111:0/1389635624 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9c8102790 0x7fd9c819ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:39.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.818+0000 7fd9c6800640 1 -- 172.21.15.111:0/1389635624 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c819f600 con 0x7fd9c8101de0 2024-09-15T23:12:39.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.818+0000 7fd9c5e00640 1 --2- 172.21.15.111:0/1389635624 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9c8102790 0x7fd9c819ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:12:39.821 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.819+0000 7fd9c6800640 1 --2- 172.21.15.111:0/1389635624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9c8101de0 0x7fd9c819a6f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8005940 tx=0x7fd9b80040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:12:39.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.819+0000 7fd9bf400640 1 -- 172.21.15.111:0/1389635624 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b803f030 con 0x7fd9c8101de0 2024-09-15T23:12:39.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.819+0000 7fd9cd400640 1 -- 172.21.15.111:0/1389635624 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9c819f890 con 0x7fd9c8101de0 2024-09-15T23:12:39.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.819+0000 7fd9cd400640 1 -- 172.21.15.111:0/1389635624 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9c81a71d0 con 0x7fd9c8101de0 2024-09-15T23:12:39.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.819+0000 7fd9bf400640 1 -- 172.21.15.111:0/1389635624 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9b80044c0 con 0x7fd9c8101de0 2024-09-15T23:12:39.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.819+0000 7fd9bf400640 1 -- 172.21.15.111:0/1389635624 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8049060 con 0x7fd9c8101de0 2024-09-15T23:12:39.822 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.821+0000 7fd9cd400640 1 -- 172.21.15.111:0/1389635624 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9c81021c0 con 0x7fd9c8101de0 2024-09-15T23:12:39.824 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.822+0000 7fd9bf400640 1 -- 172.21.15.111:0/1389635624 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7fd9b8042030 con 0x7fd9c8101de0 2024-09-15T23:12:39.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.824+0000 7fd9bf400640 1 --2- 172.21.15.111:0/1389635624 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd9ac0a3240 0x7fd9ac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:39.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.824+0000 7fd9bf400640 1 -- 172.21.15.111:0/1389635624 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7fd9b80efaa0 con 0x7fd9c8101de0 2024-09-15T23:12:39.825 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.824+0000 7fd9c5e00640 1 --2- 172.21.15.111:0/1389635624 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd9ac0a3240 0x7fd9ac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:39.826 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.825+0000 7fd9c5e00640 1 --2- 172.21.15.111:0/1389635624 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd9ac0a3240 0x7fd9ac0a5700 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd9c810eee0 tx=0x7fd9b4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:12:39.828 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:39.826+0000 7fd9bf400640 1 -- 172.21.15.111:0/1389635624 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd9b80bc7c0 con 0x7fd9c8101de0 2024-09-15T23:12:40.038 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:40.037+0000 7fd9cd400640 1 -- 172.21.15.111:0/1389635624 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9c81a7430 con 0x7fd9c8101de0 2024-09-15T23:12:40.040 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:40.038+0000 7fd9bf400640 1 -- 172.21.15.111:0/1389635624 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11934 (secure 0 0 0) 0x7fd9b80bc0c0 con 0x7fd9c8101de0 2024-09-15T23:12:40.040 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T23:12:40.040 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":37,"btime":"2024-09-15T23:11:40:107019+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/453336603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":453336603},{"type":"v1","addr":"172.21.15.111:6835","nonce":453336603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14454,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/3181832287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3181832287},{"type":"v1","addr":"172.21.15.111:6837","nonce":3181832287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14496,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/4190237053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":4190237053},{"type":"v1","addr":"172.21.15.111:6839","nonce":4190237053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14502,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6841/267728828","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6840","nonce":267728828},{"type":"v1","addr":"172.21.15.111:6841","nonce":267728828}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24448,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/4061441359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":4061441359},{"type":"v1","addr":"172.21.15.192:6833","nonce":4061441359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/4220531578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":4220531578},{"type":"v1","addr":"172.21.15.121:6833","nonce":4220531578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:44:18.436394+0000","modified":"2024-09-15T23:11:39.957406+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24545,"mds_1":24514,"mds_2":24503},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24485":{"gid":24485,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3371796806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3371796806},{"type":"v1","addr":"172.21.15.121:6835","nonce":3371796806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6835/4080914921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":4080914921},{"type":"v1","addr":"172.21.15.192:6835","nonce":4080914921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.121:6837/1443772791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":1443772791},{"type":"v1","addr":"172.21.15.121:6837","nonce":1443772791}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":10,"addr":"172.21.15.192:6839/800136261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6838","nonce":800136261},{"type":"v1","addr":"172.21.15.192:6839","nonce":800136261}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24545":{"gid":24545,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.121:6839/1797352097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1797352097},{"type":"v1","addr":"172.21.15.121:6839","nonce":1797352097}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26245":{"gid":26245,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.192:6837/1721868112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1721868112},{"type":"v1","addr":"172.21.15.192:6837","nonce":1721868112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24545,"qdb_cluster":[24545,24514,24503]},"id":1}]} 2024-09-15T23:12:40.040 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 37 2024-09-15T23:12:40.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:40.040+0000 7fd9cd400640 1 -- 172.21.15.111:0/1389635624 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd9ac0a3240 msgr2=0x7fd9ac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:40.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:40.040+0000 7fd9cd400640 1 --2- 172.21.15.111:0/1389635624 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd9ac0a3240 0x7fd9ac0a5700 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd9c810eee0 tx=0x7fd9b4005f90 comp rx=0 tx=0).stop 2024-09-15T23:12:40.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:40.040+0000 7fd9cd400640 1 -- 172.21.15.111:0/1389635624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9c8101de0 msgr2=0x7fd9c819a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:40.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:40.040+0000 7fd9cd400640 1 --2- 172.21.15.111:0/1389635624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9c8101de0 0x7fd9c819a6f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8005940 tx=0x7fd9b80040c0 comp rx=0 tx=0).stop 2024-09-15T23:12:40.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:40.040+0000 7fd9cd400640 1 -- 172.21.15.111:0/1389635624 shutdown_connections 2024-09-15T23:12:40.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:40.040+0000 7fd9cd400640 1 --2- 172.21.15.111:0/1389635624 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7fd9ac0a3240 0x7fd9ac0a5700 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:40.042 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:40.040+0000 7fd9cd400640 1 --2- 172.21.15.111:0/1389635624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd9c8106f10 0x7fd9c819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:40.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:40.040+0000 7fd9cd400640 1 --2- 172.21.15.111:0/1389635624 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9c8102790 0x7fd9c819ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:40.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:40.040+0000 7fd9cd400640 1 --2- 172.21.15.111:0/1389635624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9c8101de0 0x7fd9c819a6f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:40.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:40.040+0000 7fd9cd400640 1 -- 172.21.15.111:0/1389635624 >> 172.21.15.111:0/1389635624 conn(0x7fd9c80fd8c0 msgr2=0x7fd9c80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:12:40.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:40.041+0000 7fd9cd400640 1 -- 172.21.15.111:0/1389635624 shutdown_connections 2024-09-15T23:12:40.043 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:40.041+0000 7fd9cd400640 1 -- 172.21.15.111:0/1389635624 wait complete. 2024-09-15T23:12:40.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:40 smithi121 ceph-mon[34815]: pgmap v1153: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-15T23:12:40.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:40 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/1389635624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:12:40.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:40 smithi111 ceph-mon[30629]: pgmap v1153: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-15T23:12:40.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:40 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/1389635624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:12:40.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:40 smithi192 ceph-mon[35427]: pgmap v1153: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-15T23:12:40.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:40 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/1389635624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T23:12:40.994 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T23:12:41.365 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:12:41.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.985+0000 7f5a87200640 1 -- 172.21.15.111:0/2137477943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a88103e30 msgr2=0x7f5a88104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:41.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.985+0000 7f5a87200640 1 --2- 172.21.15.111:0/2137477943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a88103e30 0x7f5a88104210 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5a78009900 tx=0x7f5a7802a410 comp rx=0 tx=0).stop 2024-09-15T23:12:41.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.985+0000 7f5a87200640 1 -- 172.21.15.111:0/2137477943 shutdown_connections 2024-09-15T23:12:41.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.985+0000 7f5a87200640 1 --2- 172.21.15.111:0/2137477943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a88108f60 0x7f5a88110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:41.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.985+0000 7f5a87200640 1 --2- 172.21.15.111:0/2137477943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a881047e0 0x7f5a88108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:41.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.985+0000 7f5a87200640 1 --2- 172.21.15.111:0/2137477943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a88103e30 0x7f5a88104210 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:41.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.985+0000 7f5a87200640 1 -- 172.21.15.111:0/2137477943 >> 172.21.15.111:0/2137477943 conn(0x7f5a880ff8d0 msgr2=0x7f5a88101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:12:41.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.985+0000 7f5a87200640 1 -- 172.21.15.111:0/2137477943 shutdown_connections 2024-09-15T23:12:41.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.985+0000 7f5a87200640 1 -- 172.21.15.111:0/2137477943 wait complete. 2024-09-15T23:12:41.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.986+0000 7f5a87200640 1 Processor -- start 2024-09-15T23:12:41.988 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.986+0000 7f5a87200640 1 -- start start 2024-09-15T23:12:41.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.987+0000 7f5a87200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a88103e30 0x7f5a8819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:41.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.987+0000 7f5a87200640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a881047e0 0x7f5a8819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:41.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.987+0000 7f5a87200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a88108f60 0x7f5a8819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:41.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.987+0000 7f5a87200640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f5a88113fc0 con 0x7f5a881047e0 2024-09-15T23:12:41.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.987+0000 7f5a87200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5a88113e40 con 0x7f5a88103e30 2024-09-15T23:12:41.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.987+0000 7f5a87200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5a88114140 con 0x7f5a88108f60 2024-09-15T23:12:41.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.987+0000 7f5a7fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a881047e0 0x7f5a8819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:41.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.987+0000 7f5a7fe00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a881047e0 0x7f5a8819a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50610/0 (socket says 172.21.15.111:50610) 2024-09-15T23:12:41.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.987+0000 7f5a7fe00640 1 -- 172.21.15.111:0/1011920139 learned_addr learned my addr 172.21.15.111:0/1011920139 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:12:41.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.987+0000 7f5a85e00640 1 --2- 172.21.15.111:0/1011920139 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a88103e30 0x7f5a8819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:41.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.987+0000 7f5a86800640 1 --2- 172.21.15.111:0/1011920139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a88108f60 0x7f5a8819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:41.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.987+0000 7f5a7fe00640 1 -- 172.21.15.111:0/1011920139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a88108f60 msgr2=0x7f5a8819ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:41.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.987+0000 7f5a7fe00640 1 --2- 172.21.15.111:0/1011920139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a88108f60 0x7f5a8819ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:41.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.987+0000 7f5a7fe00640 1 -- 172.21.15.111:0/1011920139 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a88103e30 msgr2=0x7f5a8819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:41.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.987+0000 7f5a7fe00640 1 --2- 172.21.15.111:0/1011920139 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a88103e30 0x7f5a8819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:41.990 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.987+0000 7f5a7fe00640 1 -- 172.21.15.111:0/1011920139 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a8819f3f0 con 0x7f5a881047e0 2024-09-15T23:12:41.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.987+0000 7f5a86800640 1 --2- 172.21.15.111:0/1011920139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a88108f60 0x7f5a8819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:12:41.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.987+0000 7f5a85e00640 1 --2- 172.21.15.111:0/1011920139 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a88103e30 0x7f5a8819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:12:41.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.988+0000 7f5a7fe00640 1 --2- 172.21.15.111:0/1011920139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a881047e0 0x7f5a8819a980 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000e9f0 tx=0x7f5a7000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:12:41.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.988+0000 7f5a7ea00640 1 -- 172.21.15.111:0/1011920139 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7000cd80 con 0x7f5a881047e0 2024-09-15T23:12:41.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.988+0000 7f5a7ea00640 1 -- 172.21.15.111:0/1011920139 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a70010910 con 0x7f5a881047e0 2024-09-15T23:12:41.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.988+0000 7f5a87200640 1 -- 172.21.15.111:0/1011920139 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a8819f6e0 con 0x7f5a881047e0 2024-09-15T23:12:41.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.988+0000 7f5a7ea00640 1 -- 172.21.15.111:0/1011920139 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70018990 con 0x7f5a881047e0 2024-09-15T23:12:41.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.988+0000 7f5a87200640 1 -- 172.21.15.111:0/1011920139 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a881a6fc0 con 0x7f5a881047e0 2024-09-15T23:12:41.991 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.989+0000 7f5a87200640 1 -- 172.21.15.111:0/1011920139 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a88104210 con 0x7f5a881047e0 2024-09-15T23:12:41.992 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.991+0000 7f5a7ea00640 1 -- 172.21.15.111:0/1011920139 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5a70010ac0 con 0x7f5a881047e0 2024-09-15T23:12:41.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.993+0000 7f5a7ea00640 1 --2- 172.21.15.111:0/1011920139 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5a6c0a3240 0x7f5a6c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:41.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.993+0000 7f5a7ea00640 1 -- 172.21.15.111:0/1011920139 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f5a700cfc40 con 0x7f5a881047e0 2024-09-15T23:12:41.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.993+0000 7f5a85e00640 1 --2- 172.21.15.111:0/1011920139 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5a6c0a3240 0x7f5a6c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:41.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.994+0000 7f5a85e00640 1 --2- 172.21.15.111:0/1011920139 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5a6c0a3240 0x7f5a6c0a5700 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5a78009230 tx=0x7f5a780058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:12:41.997 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:41.996+0000 7f5a7ea00640 1 -- 172.21.15.111:0/1011920139 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5a7009c310 con 0x7f5a881047e0 2024-09-15T23:12:42.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:42 smithi111 ceph-mon[30629]: pgmap v1154: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-15T23:12:42.179 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:42.178+0000 7f5a87200640 1 -- 172.21.15.111:0/1011920139 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a8810bbb0 con 0x7f5a881047e0 2024-09-15T23:12:42.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:42.180+0000 7f5a7ea00640 1 -- 172.21.15.111:0/1011920139 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22701 (secure 0 0 0) 0x7f5a700a0610 con 0x7f5a881047e0 2024-09-15T23:12:42.181 INFO:teuthology.orchestra.run.smithi111.stdout: 2024-09-15T23:12:42.182 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":89,"fsid":"b7614d3c-73b2-11ef-bceb-c7b262605968","created":"2024-09-15T22:35:52.571473+0000","modified":"2024-09-15T23:01:53.289289+0000","last_up_change":"2024-09-15T22:43:19.567274+0000","last_in_change":"2024-09-15T22:42:56.994246+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:39:18.954207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:44:12.846418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:44:15.685946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04f39604-be0e-4261-b316-67fa087c6666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6803","nonce":1386823446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6805","nonce":1386823446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6809","nonce":1386823446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":1386823446},{"type":"v1","addr":"172.21.15.111:6807","nonce":1386823446}]},"public_addr":"172.21.15.111:6803/1386823446","cluster_addr":"172.21.15.111:6805/1386823446","heartbeat_back_addr":"172.21.15.111:6809/1386823446","heartbeat_front_addr":"172.21.15.111:6807/1386823446","state":["exists","up"]},{"osd":1,"uuid":"1bdbef6e-23fa-4618-a788-66e57741a2a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6801","nonce":2051512291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6803","nonce":2051512291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6807","nonce":2051512291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":2051512291},{"type":"v1","addr":"172.21.15.121:6805","nonce":2051512291}]},"public_addr":"172.21.15.121:6801/2051512291","cluster_addr":"172.21.15.121:6803/2051512291","heartbeat_back_addr":"172.21.15.121:6807/2051512291","heartbeat_front_addr":"172.21.15.121:6805/2051512291","state":["exists","up"]},{"osd":2,"uuid":"46735ccf-6d0e-4154-bb55-422929819bfa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6801","nonce":93085639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6803","nonce":93085639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6807","nonce":93085639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":93085639},{"type":"v1","addr":"172.21.15.192:6805","nonce":93085639}]},"public_addr":"172.21.15.192:6801/93085639","cluster_addr":"172.21.15.192:6803/93085639","heartbeat_back_addr":"172.21.15.192:6807/93085639","heartbeat_front_addr":"172.21.15.192:6805/93085639","state":["exists","up"]},{"osd":3,"uuid":"4b243af8-c958-40b1-ba24-9c7ace00c4a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6811","nonce":2578259041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6813","nonce":2578259041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6817","nonce":2578259041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":2578259041},{"type":"v1","addr":"172.21.15.111:6815","nonce":2578259041}]},"public_addr":"172.21.15.111:6811/2578259041","cluster_addr":"172.21.15.111:6813/2578259041","heartbeat_back_addr":"172.21.15.111:6817/2578259041","heartbeat_front_addr":"172.21.15.111:6815/2578259041","state":["exists","up"]},{"osd":4,"uuid":"49f27541-9cbf-48e0-a466-6bb0fa5410b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6809","nonce":2667595915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6811","nonce":2667595915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6815","nonce":2667595915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":2667595915},{"type":"v1","addr":"172.21.15.121:6813","nonce":2667595915}]},"public_addr":"172.21.15.121:6809/2667595915","cluster_addr":"172.21.15.121:6811/2667595915","heartbeat_back_addr":"172.21.15.121:6815/2667595915","heartbeat_front_addr":"172.21.15.121:6813/2667595915","state":["exists","up"]},{"osd":5,"uuid":"9290e769-5470-44d3-8d24-25093af34335","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6809","nonce":1958639856}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6811","nonce":1958639856}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6815","nonce":1958639856}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":1958639856},{"type":"v1","addr":"172.21.15.192:6813","nonce":1958639856}]},"public_addr":"172.21.15.192:6809/1958639856","cluster_addr":"172.21.15.192:6811/1958639856","heartbeat_back_addr":"172.21.15.192:6815/1958639856","heartbeat_front_addr":"172.21.15.192:6813/1958639856","state":["exists","up"]},{"osd":6,"uuid":"18911b97-1c68-4e8f-92ec-35d2ff0b5f19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6819","nonce":1371543706}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6821","nonce":1371543706}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6825","nonce":1371543706}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1371543706},{"type":"v1","addr":"172.21.15.111:6823","nonce":1371543706}]},"public_addr":"172.21.15.111:6819/1371543706","cluster_addr":"172.21.15.111:6821/1371543706","heartbeat_back_addr":"172.21.15.111:6825/1371543706","heartbeat_front_addr":"172.21.15.111:6823/1371543706","state":["exists","up"]},{"osd":7,"uuid":"470647c5-c00d-426b-a134-cca235902307","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6817","nonce":3501842998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6819","nonce":3501842998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6823","nonce":3501842998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3501842998},{"type":"v1","addr":"172.21.15.121:6821","nonce":3501842998}]},"public_addr":"172.21.15.121:6817/3501842998","cluster_addr":"172.21.15.121:6819/3501842998","heartbeat_back_addr":"172.21.15.121:6823/3501842998","heartbeat_front_addr":"172.21.15.121:6821/3501842998","state":["exists","up"]},{"osd":8,"uuid":"fa2d3dd5-a768-4ada-a036-4c596b76d654","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6817","nonce":2417771374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6819","nonce":2417771374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6823","nonce":2417771374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":2417771374},{"type":"v1","addr":"172.21.15.192:6821","nonce":2417771374}]},"public_addr":"172.21.15.192:6817/2417771374","cluster_addr":"172.21.15.192:6819/2417771374","heartbeat_back_addr":"172.21.15.192:6823/2417771374","heartbeat_front_addr":"172.21.15.192:6821/2417771374","state":["exists","up"]},{"osd":9,"uuid":"71b4f5fb-1c02-461b-a257-c6e48381d768","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6827","nonce":3979119431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6829","nonce":3979119431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6833","nonce":3979119431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":3979119431},{"type":"v1","addr":"172.21.15.111:6831","nonce":3979119431}]},"public_addr":"172.21.15.111:6827/3979119431","cluster_addr":"172.21.15.111:6829/3979119431","heartbeat_back_addr":"172.21.15.111:6833/3979119431","heartbeat_front_addr":"172.21.15.111:6831/3979119431","state":["exists","up"]},{"osd":10,"uuid":"e6a39e77-fce3-4e22-bf66-a677c9f8b6bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6825","nonce":2897902637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6827","nonce":2897902637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6831","nonce":2897902637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2897902637},{"type":"v1","addr":"172.21.15.121:6829","nonce":2897902637}]},"public_addr":"172.21.15.121:6825/2897902637","cluster_addr":"172.21.15.121:6827/2897902637","heartbeat_back_addr":"172.21.15.121:6831/2897902637","heartbeat_front_addr":"172.21.15.121:6829/2897902637","state":["exists","up"]},{"osd":11,"uuid":"411a7921-6717-4cef-b8df-d9e252edcc84","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6825","nonce":2431464812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6827","nonce":2431464812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6831","nonce":2431464812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":2431464812},{"type":"v1","addr":"172.21.15.192:6829","nonce":2431464812}]},"public_addr":"172.21.15.192:6825/2431464812","cluster_addr":"172.21.15.192:6827/2431464812","heartbeat_back_addr":"172.21.15.192:6831/2431464812","heartbeat_front_addr":"172.21.15.192:6829/2431464812","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:38:23.442715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:38:49.501124+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:39:14.759077+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:39:45.121397+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:40:11.137612+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:40:36.278567+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:07.027190+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:32.439215+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:41:57.058702+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:42:28.741846+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:42:53.395442+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:43:18.265067+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.111:6801/2107775445":"2024-09-16T22:36:50.357551+0000","172.21.15.111:6800/2107775445":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/2394746540":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/4140020843":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/2114354263":"2024-09-16T22:36:50.357551+0000","172.21.15.111:0/1054173642":"2024-09-16T22:36:22.652875+0000","172.21.15.111:0/3408238004":"2024-09-16T22:36:22.652875+0000","172.21.15.111:0/2879044772":"2024-09-16T22:36:22.652875+0000","172.21.15.111:6801/3350425104":"2024-09-16T22:36:22.652875+0000","172.21.15.111:6800/3350425104":"2024-09-16T22:36:22.652875+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T23:12:42.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:42.182+0000 7f5a87200640 1 -- 172.21.15.111:0/1011920139 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5a6c0a3240 msgr2=0x7f5a6c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:42.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:42.182+0000 7f5a87200640 1 --2- 172.21.15.111:0/1011920139 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5a6c0a3240 0x7f5a6c0a5700 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5a78009230 tx=0x7f5a780058d0 comp rx=0 tx=0).stop 2024-09-15T23:12:42.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:42.182+0000 7f5a87200640 1 -- 172.21.15.111:0/1011920139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a881047e0 msgr2=0x7f5a8819a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:42.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:42.182+0000 7f5a87200640 1 --2- 172.21.15.111:0/1011920139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a881047e0 0x7f5a8819a980 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000e9f0 tx=0x7f5a7000eeb0 comp rx=0 tx=0).stop 2024-09-15T23:12:42.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:42.182+0000 7f5a87200640 1 -- 172.21.15.111:0/1011920139 shutdown_connections 2024-09-15T23:12:42.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:42.182+0000 7f5a87200640 1 --2- 172.21.15.111:0/1011920139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a88108f60 0x7f5a8819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:42.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:42.182+0000 7f5a87200640 1 --2- 172.21.15.111:0/1011920139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5a881047e0 0x7f5a8819a980 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:42.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:42.182+0000 7f5a87200640 1 --2- 172.21.15.111:0/1011920139 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f5a6c0a3240 0x7f5a6c0a5700 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:42.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:42.182+0000 7f5a87200640 1 --2- 172.21.15.111:0/1011920139 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a88103e30 0x7f5a8819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:42.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:42.182+0000 7f5a87200640 1 -- 172.21.15.111:0/1011920139 >> 172.21.15.111:0/1011920139 conn(0x7f5a880ff8d0 msgr2=0x7f5a88100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:12:42.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:42.183+0000 7f5a87200640 1 -- 172.21.15.111:0/1011920139 shutdown_connections 2024-09-15T23:12:42.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:42.183+0000 7f5a87200640 1 -- 172.21.15.111:0/1011920139 wait complete. 2024-09-15T23:12:42.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:42 smithi121 ceph-mon[34815]: pgmap v1154: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-15T23:12:42.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:42 smithi192 ceph-mon[35427]: pgmap v1154: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-15T23:12:42.985 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-15T23:12:43.355 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:12:43.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:43 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/1011920139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:12:43.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:43 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/1011920139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:12:43.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:43 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/1011920139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T23:12:43.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.980+0000 7f40f8e00640 1 -- 172.21.15.111:0/3965590611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40f410e3c0 msgr2=0x7f40f4110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:43.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.980+0000 7f40f8e00640 1 --2- 172.21.15.111:0/3965590611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40f410e3c0 0x7f40f4110810 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009ca0 tx=0x7f40ec02a950 comp rx=0 tx=0).stop 2024-09-15T23:12:43.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.980+0000 7f40f8e00640 1 -- 172.21.15.111:0/3965590611 shutdown_connections 2024-09-15T23:12:43.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.980+0000 7f40f8e00640 1 --2- 172.21.15.111:0/3965590611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40f410e3c0 0x7f40f4110810 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:43.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.980+0000 7f40f8e00640 1 --2- 172.21.15.111:0/3965590611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40f4074f40 0x7f40f410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:43.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.980+0000 7f40f8e00640 1 --2- 172.21.15.111:0/3965590611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40f4074620 0x7f40f4074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:43.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.980+0000 7f40f8e00640 1 -- 172.21.15.111:0/3965590611 >> 172.21.15.111:0/3965590611 conn(0x7f40f40fd8c0 msgr2=0x7f40f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:12:43.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.980+0000 7f40f8e00640 1 -- 172.21.15.111:0/3965590611 shutdown_connections 2024-09-15T23:12:43.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.980+0000 7f40f8e00640 1 -- 172.21.15.111:0/3965590611 wait complete. 2024-09-15T23:12:43.993 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.981+0000 7f40f8e00640 1 Processor -- start 2024-09-15T23:12:43.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.981+0000 7f40f8e00640 1 -- start start 2024-09-15T23:12:43.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.981+0000 7f40f8e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40f4074620 0x7f40f419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:43.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.981+0000 7f40f8e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40f4074f40 0x7f40f419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:43.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.981+0000 7f40f8e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40f410e3c0 0x7f40f419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:43.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.981+0000 7f40f8e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f40f4114150 con 0x7f40f4074620 2024-09-15T23:12:43.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.981+0000 7f40f8e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f40f4113fd0 con 0x7f40f410e3c0 2024-09-15T23:12:43.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.981+0000 7f40f8e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f40f41142d0 con 0x7f40f4074f40 2024-09-15T23:12:43.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.982+0000 7f40f1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40f4074620 0x7f40f419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:43.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.982+0000 7f40f2800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40f410e3c0 0x7f40f419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:43.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.982+0000 7f40f1e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40f4074620 0x7f40f419a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50624/0 (socket says 172.21.15.111:50624) 2024-09-15T23:12:43.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.982+0000 7f40f1e00640 1 -- 172.21.15.111:0/2257945295 learned_addr learned my addr 172.21.15.111:0/2257945295 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:12:43.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.982+0000 7f40f2800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40f410e3c0 0x7f40f419eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:48858/0 (socket says 172.21.15.111:48858) 2024-09-15T23:12:43.994 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.982+0000 7f40e9400640 1 --2- 172.21.15.111:0/2257945295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40f4074f40 0x7f40f419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:43.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.982+0000 7f40f2800640 1 -- 172.21.15.111:0/2257945295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40f4074f40 msgr2=0x7f40f419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:43.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.982+0000 7f40f2800640 1 --2- 172.21.15.111:0/2257945295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40f4074f40 0x7f40f419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:43.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.982+0000 7f40f2800640 1 -- 172.21.15.111:0/2257945295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40f4074620 msgr2=0x7f40f419a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:43.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.982+0000 7f40f2800640 1 --2- 172.21.15.111:0/2257945295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40f4074620 0x7f40f419a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:43.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.982+0000 7f40f2800640 1 -- 172.21.15.111:0/2257945295 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f419f5a0 con 0x7f40f410e3c0 2024-09-15T23:12:43.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.982+0000 7f40f1e00640 1 --2- 172.21.15.111:0/2257945295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40f4074620 0x7f40f419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:12:43.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.982+0000 7f40f2800640 1 --2- 172.21.15.111:0/2257945295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40f410e3c0 0x7f40f419eec0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f40ec005cc0 tx=0x7f40ec02f3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:12:43.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.982+0000 7f40eb400640 1 -- 172.21.15.111:0/2257945295 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec034040 con 0x7f40f410e3c0 2024-09-15T23:12:43.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.983+0000 7f40f8e00640 1 -- 172.21.15.111:0/2257945295 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f419f830 con 0x7f40f410e3c0 2024-09-15T23:12:43.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.983+0000 7f40eb400640 1 -- 172.21.15.111:0/2257945295 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40ec030920 con 0x7f40f410e3c0 2024-09-15T23:12:43.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.983+0000 7f40eb400640 1 -- 172.21.15.111:0/2257945295 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec0045b0 con 0x7f40f410e3c0 2024-09-15T23:12:43.995 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.983+0000 7f40f8e00640 1 -- 172.21.15.111:0/2257945295 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f41a7170 con 0x7f40f410e3c0 2024-09-15T23:12:43.996 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.984+0000 7f40f8e00640 1 -- 172.21.15.111:0/2257945295 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40f4074a00 con 0x7f40f410e3c0 2024-09-15T23:12:43.996 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.986+0000 7f40eb400640 1 -- 172.21.15.111:0/2257945295 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f40ec037020 con 0x7f40f410e3c0 2024-09-15T23:12:43.996 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.987+0000 7f40eb400640 1 --2- 172.21.15.111:0/2257945295 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f40d80a3240 0x7f40d80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:43.996 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.987+0000 7f40eb400640 1 -- 172.21.15.111:0/2257945295 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8153+0+0 (secure 0 0 0) 0x7f40ec0f0790 con 0x7f40f410e3c0 2024-09-15T23:12:43.996 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.987+0000 7f40f1e00640 1 --2- 172.21.15.111:0/2257945295 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f40d80a3240 0x7f40d80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:43.996 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.988+0000 7f40f1e00640 1 --2- 172.21.15.111:0/2257945295 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f40d80a3240 0x7f40d80a5700 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f40e4011630 tx=0x7f40e4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:12:43.996 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:43.990+0000 7f40eb400640 1 -- 172.21.15.111:0/2257945295 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f40ec0811b0 con 0x7f40f410e3c0 2024-09-15T23:12:44.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:44 smithi111 ceph-mon[30629]: pgmap v1155: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-15T23:12:44.202 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:44.200+0000 7f40f8e00640 1 -- 172.21.15.111:0/2257945295 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f40f4109540 con 0x7f40f410e3c0 2024-09-15T23:12:44.215 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:44.214+0000 7f40eb400640 1 -- 172.21.15.111:0/2257945295 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) ==== 161+0+0 (secure 0 0 0) 0x7f40ec0bd4b0 con 0x7f40f410e3c0 2024-09-15T23:12:44.215 INFO:teuthology.orchestra.run.smithi111.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-15T23:12:44.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:44.216+0000 7f40f8e00640 1 -- 172.21.15.111:0/2257945295 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f40d80a3240 msgr2=0x7f40d80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:44.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:44.216+0000 7f40f8e00640 1 --2- 172.21.15.111:0/2257945295 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f40d80a3240 0x7f40d80a5700 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f40e4011630 tx=0x7f40e4005f90 comp rx=0 tx=0).stop 2024-09-15T23:12:44.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:44.216+0000 7f40f8e00640 1 -- 172.21.15.111:0/2257945295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40f410e3c0 msgr2=0x7f40f419eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:44.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:44.216+0000 7f40f8e00640 1 --2- 172.21.15.111:0/2257945295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40f410e3c0 0x7f40f419eec0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f40ec005cc0 tx=0x7f40ec02f3a0 comp rx=0 tx=0).stop 2024-09-15T23:12:44.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:44.216+0000 7f40f8e00640 1 -- 172.21.15.111:0/2257945295 shutdown_connections 2024-09-15T23:12:44.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:44.216+0000 7f40f8e00640 1 --2- 172.21.15.111:0/2257945295 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f40d80a3240 0x7f40d80a5700 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:44.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:44.216+0000 7f40f8e00640 1 --2- 172.21.15.111:0/2257945295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40f410e3c0 0x7f40f419eec0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:44.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:44.216+0000 7f40f8e00640 1 --2- 172.21.15.111:0/2257945295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40f4074f40 0x7f40f419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:44.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:44.216+0000 7f40f8e00640 1 --2- 172.21.15.111:0/2257945295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40f4074620 0x7f40f419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:44.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:44.216+0000 7f40f8e00640 1 -- 172.21.15.111:0/2257945295 >> 172.21.15.111:0/2257945295 conn(0x7f40f40fd8c0 msgr2=0x7f40f40fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:12:44.218 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:44.216+0000 7f40f8e00640 1 -- 172.21.15.111:0/2257945295 shutdown_connections 2024-09-15T23:12:44.219 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:44.216+0000 7f40f8e00640 1 -- 172.21.15.111:0/2257945295 wait complete. 2024-09-15T23:12:44.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:44 smithi121 ceph-mon[34815]: pgmap v1155: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-15T23:12:44.391 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 15 23:12:44 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-b[65483]: ignoring --setuser ceph since I am not root 2024-09-15T23:12:44.391 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 15 23:12:44 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-b[65483]: ignoring --setgroup ceph since I am not root 2024-09-15T23:12:44.391 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 15 23:12:44 smithi121 ceph-mds[65487]: -- 172.21.15.121:0/2823459599 <== mon.2 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564f23b4f860 con 0x564f23b2d000 2024-09-15T23:12:44.391 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 15 23:12:44 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-b[65483]: starting mds.b at 2024-09-15T23:12:44.391 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 15 23:12:44 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-h[65181]: ignoring --setuser ceph since I am not root 2024-09-15T23:12:44.391 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 15 23:12:44 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-h[65181]: ignoring --setgroup ceph since I am not root 2024-09-15T23:12:44.391 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 15 23:12:44 smithi121 ceph-mds[65185]: -- 172.21.15.121:0/2655533276 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561996c73860 con 0x561996c51400 2024-09-15T23:12:44.392 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 15 23:12:44 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-h[65181]: starting mds.h at 2024-09-15T23:12:44.392 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 15 23:12:44 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-e[64881]: ignoring --setuser ceph since I am not root 2024-09-15T23:12:44.392 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 15 23:12:44 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-e[64881]: ignoring --setgroup ceph since I am not root 2024-09-15T23:12:44.392 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 15 23:12:44 smithi121 ceph-mds[64885]: -- 172.21.15.121:0/1278805231 <== mon.2 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561eb6a83860 con 0x561eb6a60800 2024-09-15T23:12:44.392 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 15 23:12:44 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-e[64881]: starting mds.e at 2024-09-15T23:12:44.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:44 smithi192 ceph-mon[35427]: pgmap v1155: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-09-15T23:12:44.565 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 15 23:12:44 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-c[65843]: ignoring --setuser ceph since I am not root 2024-09-15T23:12:44.565 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 15 23:12:44 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-c[65843]: ignoring --setgroup ceph since I am not root 2024-09-15T23:12:44.565 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 15 23:12:44 smithi192 ceph-mds[65847]: -- 172.21.15.192:0/3991660238 <== mon.1 v2:172.21.15.192:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dc569a9860 con 0x55dc56987400 2024-09-15T23:12:44.565 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 15 23:12:44 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-c[65843]: starting mds.c at 2024-09-15T23:12:44.565 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 15 23:12:44 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-f[65543]: ignoring --setuser ceph since I am not root 2024-09-15T23:12:44.565 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 15 23:12:44 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-f[65543]: ignoring --setgroup ceph since I am not root 2024-09-15T23:12:44.566 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 15 23:12:44 smithi192 ceph-mds[65547]: -- 172.21.15.192:0/1076276850 <== mon.2 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563f5691f860 con 0x563f568fd400 2024-09-15T23:12:44.566 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 15 23:12:44 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-f[65543]: starting mds.f at 2024-09-15T23:12:44.566 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 15 23:12:44 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-i[66145]: ignoring --setuser ceph since I am not root 2024-09-15T23:12:44.566 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 15 23:12:44 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-i[66145]: ignoring --setgroup ceph since I am not root 2024-09-15T23:12:44.566 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 15 23:12:44 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-i[66145]: starting mds.i at 2024-09-15T23:12:44.614 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:44 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-a[30625]: 2024-09-15T23:12:44.202+0000 7f64f1c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-15T23:12:45.025 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-15T23:12:45.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:45 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/2257945295' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-15T23:12:45.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:45 smithi121 ceph-mon[34815]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-15T23:12:45.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:45 smithi121 ceph-mon[34815]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-15T23:12:45.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:45 smithi121 ceph-mon[34815]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-15T23:12:45.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:45 smithi121 ceph-mon[34815]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-15T23:12:45.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:45 smithi121 ceph-mon[34815]: osdmap e90: 12 total, 12 up, 12 in 2024-09-15T23:12:45.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:45 smithi121 ceph-mon[34815]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-15T23:12:45.395 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:12:45.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:45 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/2257945295' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-15T23:12:45.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:45 smithi111 ceph-mon[30629]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-15T23:12:45.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:45 smithi111 ceph-mon[30629]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-15T23:12:45.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:45 smithi111 ceph-mon[30629]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-15T23:12:45.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:45 smithi111 ceph-mon[30629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-15T23:12:45.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:45 smithi111 ceph-mon[30629]: osdmap e90: 12 total, 12 up, 12 in 2024-09-15T23:12:45.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:45 smithi111 ceph-mon[30629]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-15T23:12:45.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:45 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/2257945295' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-15T23:12:45.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:45 smithi192 ceph-mon[35427]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-15T23:12:45.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:45 smithi192 ceph-mon[35427]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-15T23:12:45.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:45 smithi192 ceph-mon[35427]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-15T23:12:45.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:45 smithi192 ceph-mon[35427]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-15T23:12:45.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:45 smithi192 ceph-mon[35427]: osdmap e90: 12 total, 12 up, 12 in 2024-09-15T23:12:45.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:45 smithi192 ceph-mon[35427]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-15T23:12:45.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.977+0000 7f5231800640 1 -- 172.21.15.111:0/240916951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f522c1047e0 msgr2=0x7f522c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:45.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.977+0000 7f5231800640 1 --2- 172.21.15.111:0/240916951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f522c1047e0 0x7f522c108830 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5218009900 tx=0x7f521802a410 comp rx=0 tx=0).stop 2024-09-15T23:12:45.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.977+0000 7f5231800640 1 -- 172.21.15.111:0/240916951 shutdown_connections 2024-09-15T23:12:45.979 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.977+0000 7f5231800640 1 --2- 172.21.15.111:0/240916951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f522c108f60 0x7f522c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:45.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.977+0000 7f5231800640 1 --2- 172.21.15.111:0/240916951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f522c1047e0 0x7f522c108830 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:45.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.977+0000 7f5231800640 1 --2- 172.21.15.111:0/240916951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f522c103e30 0x7f522c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:45.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.977+0000 7f5231800640 1 -- 172.21.15.111:0/240916951 >> 172.21.15.111:0/240916951 conn(0x7f522c0ff8d0 msgr2=0x7f522c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:12:45.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.977+0000 7f5231800640 1 -- 172.21.15.111:0/240916951 shutdown_connections 2024-09-15T23:12:45.980 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.977+0000 7f5231800640 1 -- 172.21.15.111:0/240916951 wait complete. 2024-09-15T23:12:45.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.979+0000 7f5231800640 1 Processor -- start 2024-09-15T23:12:45.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.979+0000 7f5231800640 1 -- start start 2024-09-15T23:12:45.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.979+0000 7f5231800640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f522c103e30 0x7f522c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:45.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.979+0000 7f5231800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f522c1047e0 0x7f522c19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:45.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.979+0000 7f5231800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f522c108f60 0x7f522c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:45.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.980+0000 7f5231800640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f522c113fc0 con 0x7f522c103e30 2024-09-15T23:12:45.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.980+0000 7f5231800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f522c113e40 con 0x7f522c1047e0 2024-09-15T23:12:45.981 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.980+0000 7f5231800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f522c114140 con 0x7f522c108f60 2024-09-15T23:12:45.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.980+0000 7f522be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f522c103e30 0x7f522c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:45.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.980+0000 7f522be00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f522c103e30 0x7f522c19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50642/0 (socket says 172.21.15.111:50642) 2024-09-15T23:12:45.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.980+0000 7f522be00640 1 -- 172.21.15.111:0/3445795191 learned_addr learned my addr 172.21.15.111:0/3445795191 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:12:45.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.980+0000 7f5230e00640 1 --2- 172.21.15.111:0/3445795191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f522c108f60 0x7f522c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:45.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.980+0000 7f522b400640 1 --2- 172.21.15.111:0/3445795191 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f522c1047e0 0x7f522c19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:45.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.980+0000 7f5230e00640 1 -- 172.21.15.111:0/3445795191 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f522c1047e0 msgr2=0x7f522c19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:45.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.980+0000 7f5230e00640 1 --2- 172.21.15.111:0/3445795191 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f522c1047e0 0x7f522c19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:45.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.980+0000 7f5230e00640 1 -- 172.21.15.111:0/3445795191 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f522c103e30 msgr2=0x7f522c19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:45.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.980+0000 7f5230e00640 1 --2- 172.21.15.111:0/3445795191 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f522c103e30 0x7f522c19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:45.983 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.980+0000 7f5230e00640 1 -- 172.21.15.111:0/3445795191 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f522c19f430 con 0x7f522c108f60 2024-09-15T23:12:45.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.980+0000 7f522be00640 1 --2- 172.21.15.111:0/3445795191 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f522c103e30 0x7f522c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T23:12:45.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.981+0000 7f5230e00640 1 --2- 172.21.15.111:0/3445795191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f522c108f60 0x7f522c19ed50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5224009fd0 tx=0x7f522400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:12:45.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.981+0000 7f5228c00640 1 -- 172.21.15.111:0/3445795191 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522400ce20 con 0x7f522c108f60 2024-09-15T23:12:45.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.981+0000 7f5228c00640 1 -- 172.21.15.111:0/3445795191 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5224005020 con 0x7f522c108f60 2024-09-15T23:12:45.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.981+0000 7f5228c00640 1 -- 172.21.15.111:0/3445795191 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5224010720 con 0x7f522c108f60 2024-09-15T23:12:45.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.981+0000 7f5231800640 1 -- 172.21.15.111:0/3445795191 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f522c19f720 con 0x7f522c108f60 2024-09-15T23:12:45.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.981+0000 7f5231800640 1 -- 172.21.15.111:0/3445795191 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f522c1a7000 con 0x7f522c108f60 2024-09-15T23:12:45.984 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.982+0000 7f5231800640 1 -- 172.21.15.111:0/3445795191 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f522c104210 con 0x7f522c108f60 2024-09-15T23:12:45.985 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.984+0000 7f5228c00640 1 -- 172.21.15.111:0/3445795191 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f5224005540 con 0x7f522c108f60 2024-09-15T23:12:45.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.985+0000 7f5228c00640 1 --2- 172.21.15.111:0/3445795191 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f52100a32c0 0x7f52100a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:45.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.986+0000 7f5228c00640 1 -- 172.21.15.111:0/3445795191 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8411+0+0 (secure 0 0 0) 0x7f52240d0200 con 0x7f522c108f60 2024-09-15T23:12:45.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.986+0000 7f522be00640 1 --2- 172.21.15.111:0/3445795191 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f52100a32c0 0x7f52100a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:45.987 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.986+0000 7f522be00640 1 --2- 172.21.15.111:0/3445795191 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f52100a32c0 0x7f52100a5780 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f521c011550 tx=0x7f521c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:12:45.989 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:45.988+0000 7f5228c00640 1 -- 172.21.15.111:0/3445795191 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f522409ce10 con 0x7f522c108f60 2024-09-15T23:12:46.197 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:46.196+0000 7f5231800640 1 -- 172.21.15.111:0/3445795191 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f522c02d7c0 con 0x7f522c108f60 2024-09-15T23:12:46.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:46 smithi121 ceph-mon[34815]: pgmap v1157: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-15T23:12:46.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:46 smithi121 ceph-mon[34815]: mds.? [v2:172.21.15.192:6834/3341898909,v1:172.21.15.192:6837/3341898909] up:boot 2024-09-15T23:12:46.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:46 smithi121 ceph-mon[34815]: mds.? [v2:172.21.15.192:6835/1082535352,v1:172.21.15.192:6838/1082535352] up:boot 2024-09-15T23:12:46.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:46 smithi121 ceph-mon[34815]: mds.? [v2:172.21.15.192:6836/840703306,v1:172.21.15.192:6839/840703306] up:boot 2024-09-15T23:12:46.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:46 smithi121 ceph-mon[34815]: mds.? [v2:172.21.15.121:6834/2181788383,v1:172.21.15.121:6837/2181788383] up:boot 2024-09-15T23:12:46.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:46 smithi121 ceph-mon[34815]: mds.? [v2:172.21.15.121:6835/215610478,v1:172.21.15.121:6838/215610478] up:boot 2024-09-15T23:12:46.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:46 smithi121 ceph-mon[34815]: mds.? [v2:172.21.15.121:6836/4092673042,v1:172.21.15.121:6839/4092673042] up:boot 2024-09-15T23:12:46.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:46 smithi121 ceph-mon[34815]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-15T23:12:46.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T23:12:46.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T23:12:46.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T23:12:46.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T23:12:46.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T23:12:46.391 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:46 smithi121 ceph-mon[34815]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T23:12:46.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:46 smithi111 ceph-mon[30629]: pgmap v1157: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-15T23:12:46.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:46 smithi111 ceph-mon[30629]: mds.? [v2:172.21.15.192:6834/3341898909,v1:172.21.15.192:6837/3341898909] up:boot 2024-09-15T23:12:46.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:46 smithi111 ceph-mon[30629]: mds.? [v2:172.21.15.192:6835/1082535352,v1:172.21.15.192:6838/1082535352] up:boot 2024-09-15T23:12:46.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:46 smithi111 ceph-mon[30629]: mds.? [v2:172.21.15.192:6836/840703306,v1:172.21.15.192:6839/840703306] up:boot 2024-09-15T23:12:46.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:46 smithi111 ceph-mon[30629]: mds.? [v2:172.21.15.121:6834/2181788383,v1:172.21.15.121:6837/2181788383] up:boot 2024-09-15T23:12:46.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:46 smithi111 ceph-mon[30629]: mds.? [v2:172.21.15.121:6835/215610478,v1:172.21.15.121:6838/215610478] up:boot 2024-09-15T23:12:46.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:46 smithi111 ceph-mon[30629]: mds.? [v2:172.21.15.121:6836/4092673042,v1:172.21.15.121:6839/4092673042] up:boot 2024-09-15T23:12:46.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:46 smithi111 ceph-mon[30629]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-15T23:12:46.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T23:12:46.417 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T23:12:46.417 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T23:12:46.417 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T23:12:46.417 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T23:12:46.417 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:46 smithi111 ceph-mon[30629]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T23:12:46.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:46 smithi192 ceph-mon[35427]: pgmap v1157: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 921 B/s wr, 7 op/s 2024-09-15T23:12:46.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:46 smithi192 ceph-mon[35427]: mds.? [v2:172.21.15.192:6834/3341898909,v1:172.21.15.192:6837/3341898909] up:boot 2024-09-15T23:12:46.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:46 smithi192 ceph-mon[35427]: mds.? [v2:172.21.15.192:6835/1082535352,v1:172.21.15.192:6838/1082535352] up:boot 2024-09-15T23:12:46.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:46 smithi192 ceph-mon[35427]: mds.? [v2:172.21.15.192:6836/840703306,v1:172.21.15.192:6839/840703306] up:boot 2024-09-15T23:12:46.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:46 smithi192 ceph-mon[35427]: mds.? [v2:172.21.15.121:6834/2181788383,v1:172.21.15.121:6837/2181788383] up:boot 2024-09-15T23:12:46.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:46 smithi192 ceph-mon[35427]: mds.? [v2:172.21.15.121:6835/215610478,v1:172.21.15.121:6838/215610478] up:boot 2024-09-15T23:12:46.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:46 smithi192 ceph-mon[35427]: mds.? [v2:172.21.15.121:6836/4092673042,v1:172.21.15.121:6839/4092673042] up:boot 2024-09-15T23:12:46.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:46 smithi192 ceph-mon[35427]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-15T23:12:46.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T23:12:46.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T23:12:46.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T23:12:46.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T23:12:46.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T23:12:46.566 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:46 smithi192 ceph-mon[35427]: from='mgr.14148 172.21.15.111:0/1673667055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T23:12:47.131 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:47.129+0000 7f5228c00640 1 -- 172.21.15.111:0/3445795191 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v40) ==== 104+0+0 (secure 0 0 0) 0x7f522409c710 con 0x7f522c108f60 2024-09-15T23:12:47.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:47.132+0000 7f5231800640 1 -- 172.21.15.111:0/3445795191 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f52100a32c0 msgr2=0x7f52100a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:47.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:47.132+0000 7f5231800640 1 --2- 172.21.15.111:0/3445795191 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f52100a32c0 0x7f52100a5780 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f521c011550 tx=0x7f521c005f90 comp rx=0 tx=0).stop 2024-09-15T23:12:47.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:47.132+0000 7f5231800640 1 -- 172.21.15.111:0/3445795191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f522c108f60 msgr2=0x7f522c19ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:47.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:47.132+0000 7f5231800640 1 --2- 172.21.15.111:0/3445795191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f522c108f60 0x7f522c19ed50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5224009fd0 tx=0x7f522400ef10 comp rx=0 tx=0).stop 2024-09-15T23:12:47.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:47.132+0000 7f5231800640 1 -- 172.21.15.111:0/3445795191 shutdown_connections 2024-09-15T23:12:47.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:47.132+0000 7f5231800640 1 --2- 172.21.15.111:0/3445795191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f522c108f60 0x7f522c19ed50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:47.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:47.132+0000 7f5231800640 1 --2- 172.21.15.111:0/3445795191 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f52100a32c0 0x7f52100a5780 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:47.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:47.132+0000 7f5231800640 1 --2- 172.21.15.111:0/3445795191 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f522c1047e0 0x7f522c19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:47.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:47.132+0000 7f5231800640 1 --2- 172.21.15.111:0/3445795191 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f522c103e30 0x7f522c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:47.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:47.132+0000 7f5231800640 1 -- 172.21.15.111:0/3445795191 >> 172.21.15.111:0/3445795191 conn(0x7f522c0ff8d0 msgr2=0x7f522c100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:12:47.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:47.132+0000 7f5231800640 1 -- 172.21.15.111:0/3445795191 shutdown_connections 2024-09-15T23:12:47.134 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:47.132+0000 7f5231800640 1 -- 172.21.15.111:0/3445795191 wait complete. 2024-09-15T23:12:47.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:47 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/3445795191' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T23:12:47.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:47 smithi121 ceph-mon[34815]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T23:12:47.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:47 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/3445795191' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T23:12:47.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:47 smithi111 ceph-mon[30629]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T23:12:47.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:47 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/3445795191' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T23:12:47.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:47 smithi192 ceph-mon[35427]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T23:12:47.908 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-15T23:12:48.274 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:12:48.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:48 smithi121 ceph-mon[34815]: pgmap v1158: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 921 B/s wr, 6 op/s 2024-09-15T23:12:48.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:48 smithi121 ceph-mon[34815]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-15T23:12:48.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:48 smithi121 ceph-mon[34815]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-15T23:12:48.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:48 smithi121 ceph-mon[34815]: Cluster is now healthy 2024-09-15T23:12:48.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:48 smithi121 ceph-mon[34815]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-15T23:12:48.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:48 smithi121 ceph-mon[34815]: fsmap 12 up:standby 2024-09-15T23:12:48.413 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:48 smithi111 ceph-mon[30629]: pgmap v1158: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 921 B/s wr, 6 op/s 2024-09-15T23:12:48.413 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:48 smithi111 ceph-mon[30629]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-15T23:12:48.413 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:48 smithi111 ceph-mon[30629]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-15T23:12:48.413 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:48 smithi111 ceph-mon[30629]: Cluster is now healthy 2024-09-15T23:12:48.413 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:48 smithi111 ceph-mon[30629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-15T23:12:48.413 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:48 smithi111 ceph-mon[30629]: fsmap 12 up:standby 2024-09-15T23:12:48.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:48 smithi192 ceph-mon[35427]: pgmap v1158: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 921 B/s wr, 6 op/s 2024-09-15T23:12:48.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:48 smithi192 ceph-mon[35427]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-15T23:12:48.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:48 smithi192 ceph-mon[35427]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-15T23:12:48.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:48 smithi192 ceph-mon[35427]: Cluster is now healthy 2024-09-15T23:12:48.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:48 smithi192 ceph-mon[35427]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-15T23:12:48.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:48 smithi192 ceph-mon[35427]: fsmap 12 up:standby 2024-09-15T23:12:48.889 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.888+0000 7f7bddc00640 1 -- 172.21.15.111:0/646515728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bd8101de0 msgr2=0x7f7bd81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:48.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.888+0000 7f7bddc00640 1 --2- 172.21.15.111:0/646515728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bd8101de0 0x7f7bd81021c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8009900 tx=0x7f7bc802a410 comp rx=0 tx=0).stop 2024-09-15T23:12:48.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.888+0000 7f7bddc00640 1 -- 172.21.15.111:0/646515728 shutdown_connections 2024-09-15T23:12:48.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.888+0000 7f7bddc00640 1 --2- 172.21.15.111:0/646515728 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bd8106f10 0x7f7bd810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:48.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.888+0000 7f7bddc00640 1 --2- 172.21.15.111:0/646515728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bd8102790 0x7f7bd81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:48.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.888+0000 7f7bddc00640 1 --2- 172.21.15.111:0/646515728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bd8101de0 0x7f7bd81021c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:48.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.888+0000 7f7bddc00640 1 -- 172.21.15.111:0/646515728 >> 172.21.15.111:0/646515728 conn(0x7f7bd80fd8c0 msgr2=0x7f7bd80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:12:48.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.888+0000 7f7bddc00640 1 -- 172.21.15.111:0/646515728 shutdown_connections 2024-09-15T23:12:48.890 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.888+0000 7f7bddc00640 1 -- 172.21.15.111:0/646515728 wait complete. 2024-09-15T23:12:48.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.889+0000 7f7bddc00640 1 Processor -- start 2024-09-15T23:12:48.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.889+0000 7f7bddc00640 1 -- start start 2024-09-15T23:12:48.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.890+0000 7f7bddc00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bd8101de0 0x7f7bd80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:48.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.890+0000 7f7bddc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bd8102790 0x7f7bd8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:48.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.890+0000 7f7bddc00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bd8106f10 0x7f7bd8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:48.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.890+0000 7f7bddc00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd8111f70 con 0x7f7bd8101de0 2024-09-15T23:12:48.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.890+0000 7f7bddc00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd8111df0 con 0x7f7bd8106f10 2024-09-15T23:12:48.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.890+0000 7f7bddc00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd81120f0 con 0x7f7bd8102790 2024-09-15T23:12:48.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.890+0000 7f7bd6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bd8101de0 0x7f7bd80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:48.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.890+0000 7f7bce000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bd8102790 0x7f7bd8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:48.892 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.890+0000 7f7bd6a00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bd8101de0 0x7f7bd80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50660/0 (socket says 172.21.15.111:50660) 2024-09-15T23:12:48.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.890+0000 7f7bd6a00640 1 -- 172.21.15.111:0/1391436091 learned_addr learned my addr 172.21.15.111:0/1391436091 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:12:48.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.890+0000 7f7bd7400640 1 --2- 172.21.15.111:0/1391436091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bd8106f10 0x7f7bd8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:48.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.890+0000 7f7bd6a00640 1 -- 172.21.15.111:0/1391436091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bd8102790 msgr2=0x7f7bd8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:48.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.890+0000 7f7bd6a00640 1 --2- 172.21.15.111:0/1391436091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bd8102790 0x7f7bd8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:48.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.890+0000 7f7bd6a00640 1 -- 172.21.15.111:0/1391436091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bd8106f10 msgr2=0x7f7bd8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:48.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.890+0000 7f7bd6a00640 1 --2- 172.21.15.111:0/1391436091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bd8106f10 0x7f7bd8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:48.893 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.890+0000 7f7bd6a00640 1 -- 172.21.15.111:0/1391436091 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd81a6950 con 0x7f7bd8101de0 2024-09-15T23:12:48.894 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.891+0000 7f7bd6a00640 1 --2- 172.21.15.111:0/1391436091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bd8101de0 0x7f7bd80744f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8006010 tx=0x7f7bc80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:12:48.894 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.891+0000 7f7bcfe00640 1 -- 172.21.15.111:0/1391436091 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc803f030 con 0x7f7bd8101de0 2024-09-15T23:12:48.894 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.891+0000 7f7bcfe00640 1 -- 172.21.15.111:0/1391436091 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bc8003ce0 con 0x7f7bd8101de0 2024-09-15T23:12:48.894 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.891+0000 7f7bddc00640 1 -- 172.21.15.111:0/1391436091 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd81a6be0 con 0x7f7bd8101de0 2024-09-15T23:12:48.894 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.891+0000 7f7bcfe00640 1 -- 172.21.15.111:0/1391436091 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8035070 con 0x7f7bd8101de0 2024-09-15T23:12:48.894 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.891+0000 7f7bddc00640 1 -- 172.21.15.111:0/1391436091 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd81a7110 con 0x7f7bd8101de0 2024-09-15T23:12:48.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.892+0000 7f7bddc00640 1 -- 172.21.15.111:0/1391436091 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bd81021c0 con 0x7f7bd8101de0 2024-09-15T23:12:48.895 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.894+0000 7f7bcfe00640 1 -- 172.21.15.111:0/1391436091 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f7bc8042030 con 0x7f7bd8101de0 2024-09-15T23:12:48.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.895+0000 7f7bcfe00640 1 --2- 172.21.15.111:0/1391436091 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7bbc0a32c0 0x7f7bbc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:48.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.895+0000 7f7bcfe00640 1 -- 172.21.15.111:0/1391436091 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8411+0+0 (secure 0 0 0) 0x7f7bc80efeb0 con 0x7f7bd8101de0 2024-09-15T23:12:48.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.896+0000 7f7bce000640 1 --2- 172.21.15.111:0/1391436091 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7bbc0a32c0 0x7f7bbc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:48.897 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.896+0000 7f7bce000640 1 --2- 172.21.15.111:0/1391436091 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7bbc0a32c0 0x7f7bbc0a5780 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8075f80 tx=0x7f7bc000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:12:48.901 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:48.900+0000 7f7bcfe00640 1 -- 172.21.15.111:0/1391436091 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7bc80bc530 con 0x7f7bd8101de0 2024-09-15T23:12:49.076 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:49.074+0000 7f7bddc00640 1 -- 172.21.15.111:0/1391436091 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7bd810a0f0 con 0x7f7bd8101de0 2024-09-15T23:12:49.135 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:49.133+0000 7f7bcfe00640 1 -- 172.21.15.111:0/1391436091 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) ==== 181+0+0 (secure 0 0 0) 0x7f7bc80c0830 con 0x7f7bd8101de0 2024-09-15T23:12:49.136 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_metadata' removed 2024-09-15T23:12:49.139 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:49.137+0000 7f7bddc00640 1 -- 172.21.15.111:0/1391436091 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7bbc0a32c0 msgr2=0x7f7bbc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:49.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:49.137+0000 7f7bddc00640 1 --2- 172.21.15.111:0/1391436091 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7bbc0a32c0 0x7f7bbc0a5780 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8075f80 tx=0x7f7bc000b410 comp rx=0 tx=0).stop 2024-09-15T23:12:49.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:49.137+0000 7f7bddc00640 1 -- 172.21.15.111:0/1391436091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bd8101de0 msgr2=0x7f7bd80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:49.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:49.137+0000 7f7bddc00640 1 --2- 172.21.15.111:0/1391436091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bd8101de0 0x7f7bd80744f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8006010 tx=0x7f7bc80042f0 comp rx=0 tx=0).stop 2024-09-15T23:12:49.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:49.138+0000 7f7bddc00640 1 -- 172.21.15.111:0/1391436091 shutdown_connections 2024-09-15T23:12:49.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:49.138+0000 7f7bddc00640 1 --2- 172.21.15.111:0/1391436091 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f7bbc0a32c0 0x7f7bbc0a5780 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:49.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:49.138+0000 7f7bddc00640 1 --2- 172.21.15.111:0/1391436091 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bd8106f10 0x7f7bd8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:49.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:49.138+0000 7f7bddc00640 1 --2- 172.21.15.111:0/1391436091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bd8102790 0x7f7bd8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:49.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:49.138+0000 7f7bddc00640 1 --2- 172.21.15.111:0/1391436091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bd8101de0 0x7f7bd80744f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:49.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:49.138+0000 7f7bddc00640 1 -- 172.21.15.111:0/1391436091 >> 172.21.15.111:0/1391436091 conn(0x7f7bd80fd8c0 msgr2=0x7f7bd80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:12:49.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:49.138+0000 7f7bddc00640 1 -- 172.21.15.111:0/1391436091 shutdown_connections 2024-09-15T23:12:49.140 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:49.138+0000 7f7bddc00640 1 -- 172.21.15.111:0/1391436091 wait complete. 2024-09-15T23:12:49.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:49 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/1391436091' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T23:12:49.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:49 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/1391436091' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T23:12:49.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:49 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/1391436091' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T23:12:50.094 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-15T23:12:50.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:50 smithi121 ceph-mon[34815]: pgmap v1159: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 716 B/s wr, 5 op/s 2024-09-15T23:12:50.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:50 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/1391436091' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T23:12:50.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:50 smithi121 ceph-mon[34815]: osdmap e91: 12 total, 12 up, 12 in 2024-09-15T23:12:50.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:50 smithi111 ceph-mon[30629]: pgmap v1159: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 716 B/s wr, 5 op/s 2024-09-15T23:12:50.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:50 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/1391436091' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T23:12:50.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:50 smithi111 ceph-mon[30629]: osdmap e91: 12 total, 12 up, 12 in 2024-09-15T23:12:50.474 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T23:12:50.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:50 smithi192 ceph-mon[35427]: pgmap v1159: 97 pgs: 97 active+clean; 373 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 716 B/s wr, 5 op/s 2024-09-15T23:12:50.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:50 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/1391436091' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T23:12:50.564 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:50 smithi192 ceph-mon[35427]: osdmap e91: 12 total, 12 up, 12 in 2024-09-15T23:12:51.181 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.179+0000 7f2ba0e00640 1 -- 172.21.15.111:0/4003591900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b9c074620 msgr2=0x7f2b9c074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:51.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.179+0000 7f2ba0e00640 1 --2- 172.21.15.111:0/4003591900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b9c074620 0x7f2b9c074a00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c009ca0 tx=0x7f2b8c02a950 comp rx=0 tx=0).stop 2024-09-15T23:12:51.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.180+0000 7f2ba0e00640 1 -- 172.21.15.111:0/4003591900 shutdown_connections 2024-09-15T23:12:51.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.180+0000 7f2ba0e00640 1 --2- 172.21.15.111:0/4003591900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b9c10e3c0 0x7f2b9c110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:51.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.180+0000 7f2ba0e00640 1 --2- 172.21.15.111:0/4003591900 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b9c074f40 0x7f2b9c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:51.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.180+0000 7f2ba0e00640 1 --2- 172.21.15.111:0/4003591900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b9c074620 0x7f2b9c074a00 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:51.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.180+0000 7f2ba0e00640 1 -- 172.21.15.111:0/4003591900 >> 172.21.15.111:0/4003591900 conn(0x7f2b9c0fd8c0 msgr2=0x7f2b9c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:12:51.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.180+0000 7f2ba0e00640 1 -- 172.21.15.111:0/4003591900 shutdown_connections 2024-09-15T23:12:51.182 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.180+0000 7f2ba0e00640 1 -- 172.21.15.111:0/4003591900 wait complete. 2024-09-15T23:12:51.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.181+0000 7f2ba0e00640 1 Processor -- start 2024-09-15T23:12:51.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.181+0000 7f2ba0e00640 1 -- start start 2024-09-15T23:12:51.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.181+0000 7f2ba0e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b9c074620 0x7f2b9c19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:51.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.181+0000 7f2ba0e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b9c074f40 0x7f2b9c19acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:51.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.181+0000 7f2ba0e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b9c10e3c0 0x7f2b9c19df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:51.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.181+0000 7f2ba0e00640 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9c114170 con 0x7f2b9c074620 2024-09-15T23:12:51.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.181+0000 7f2ba0e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9c113ff0 con 0x7f2b9c074f40 2024-09-15T23:12:51.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.181+0000 7f2ba0e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9c1142f0 con 0x7f2b9c10e3c0 2024-09-15T23:12:51.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.181+0000 7f2b99400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b9c074f40 0x7f2b9c19acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:51.184 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.181+0000 7f2b99e00640 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b9c074620 0x7f2b9c19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:51.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.181+0000 7f2b99400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b9c074f40 0x7f2b9c19acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.111:54220/0 (socket says 172.21.15.111:54220) 2024-09-15T23:12:51.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.181+0000 7f2b99400640 1 -- 172.21.15.111:0/638294623 learned_addr learned my addr 172.21.15.111:0/638294623 (peer_addr_for_me v2:172.21.15.111:0/0) 2024-09-15T23:12:51.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.181+0000 7f2b9a800640 1 --2- 172.21.15.111:0/638294623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b9c10e3c0 0x7f2b9c19df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:51.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.182+0000 7f2b99400640 1 -- 172.21.15.111:0/638294623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b9c10e3c0 msgr2=0x7f2b9c19df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:51.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.182+0000 7f2b99400640 1 --2- 172.21.15.111:0/638294623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b9c10e3c0 0x7f2b9c19df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:51.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.182+0000 7f2b99400640 1 -- 172.21.15.111:0/638294623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b9c074620 msgr2=0x7f2b9c19a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:51.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.182+0000 7f2b99400640 1 --2- 172.21.15.111:0/638294623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b9c074620 0x7f2b9c19a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:51.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.182+0000 7f2b99400640 1 -- 172.21.15.111:0/638294623 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b9c19e6c0 con 0x7f2b9c074f40 2024-09-15T23:12:51.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.182+0000 7f2b9a800640 1 --2- 172.21.15.111:0/638294623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b9c10e3c0 0x7f2b9c19df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T23:12:51.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.182+0000 7f2b99400640 1 --2- 172.21.15.111:0/638294623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b9c074f40 0x7f2b9c19acb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2b8800e9f0 tx=0x7f2b8800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:12:51.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.182+0000 7f2b92a00640 1 -- 172.21.15.111:0/638294623 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8800cd80 con 0x7f2b9c074f40 2024-09-15T23:12:51.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.182+0000 7f2b92a00640 1 -- 172.21.15.111:0/638294623 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b88010910 con 0x7f2b9c074f40 2024-09-15T23:12:51.185 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.182+0000 7f2b92a00640 1 -- 172.21.15.111:0/638294623 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b88018990 con 0x7f2b9c074f40 2024-09-15T23:12:51.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.182+0000 7f2ba0e00640 1 -- 172.21.15.111:0/638294623 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b9c19e9b0 con 0x7f2b9c074f40 2024-09-15T23:12:51.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.182+0000 7f2ba0e00640 1 -- 172.21.15.111:0/638294623 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b9c1a7320 con 0x7f2b9c074f40 2024-09-15T23:12:51.186 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.184+0000 7f2ba0e00640 1 -- 172.21.15.111:0/638294623 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b9c074a00 con 0x7f2b9c074f40 2024-09-15T23:12:51.187 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.186+0000 7f2b92a00640 1 -- 172.21.15.111:0/638294623 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 20) ==== 138356+0+0 (secure 0 0 0) 0x7f2b88010ac0 con 0x7f2b9c074f40 2024-09-15T23:12:51.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.187+0000 7f2b92a00640 1 --2- 172.21.15.111:0/638294623 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2b800a32c0 0x7f2b800a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T23:12:51.188 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.187+0000 7f2b92a00640 1 -- 172.21.15.111:0/638294623 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 7886+0+0 (secure 0 0 0) 0x7f2b880cfb20 con 0x7f2b9c074f40 2024-09-15T23:12:51.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.187+0000 7f2b99e00640 1 --2- 172.21.15.111:0/638294623 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2b800a32c0 0x7f2b800a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T23:12:51.189 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.188+0000 7f2b99e00640 1 --2- 172.21.15.111:0/638294623 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2b800a32c0 0x7f2b800a5780 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c004ad0 tx=0x7f2b8c004a60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T23:12:51.192 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.191+0000 7f2b92a00640 1 -- 172.21.15.111:0/638294623 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2b8809c950 con 0x7f2b9c074f40 2024-09-15T23:12:51.368 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:51.367+0000 7f2ba0e00640 1 -- 172.21.15.111:0/638294623 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2b9c109540 con 0x7f2b9c074f40 2024-09-15T23:12:52.153 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:52.151+0000 7f2b92a00640 1 -- 172.21.15.111:0/638294623 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) ==== 169+0+0 (secure 0 0 0) 0x7f2b8809c250 con 0x7f2b9c074f40 2024-09-15T23:12:52.153 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_data' removed 2024-09-15T23:12:52.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:52.155+0000 7f2ba0e00640 1 -- 172.21.15.111:0/638294623 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2b800a32c0 msgr2=0x7f2b800a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:52.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:52.155+0000 7f2ba0e00640 1 --2- 172.21.15.111:0/638294623 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2b800a32c0 0x7f2b800a5780 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c004ad0 tx=0x7f2b8c004a60 comp rx=0 tx=0).stop 2024-09-15T23:12:52.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:52.155+0000 7f2ba0e00640 1 -- 172.21.15.111:0/638294623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b9c074f40 msgr2=0x7f2b9c19acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T23:12:52.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:52.155+0000 7f2ba0e00640 1 --2- 172.21.15.111:0/638294623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b9c074f40 0x7f2b9c19acb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2b8800e9f0 tx=0x7f2b8800eeb0 comp rx=0 tx=0).stop 2024-09-15T23:12:52.157 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:52.156+0000 7f2ba0e00640 1 -- 172.21.15.111:0/638294623 shutdown_connections 2024-09-15T23:12:52.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:52.156+0000 7f2ba0e00640 1 --2- 172.21.15.111:0/638294623 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b9c10e3c0 0x7f2b9c19df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:52.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:52.156+0000 7f2ba0e00640 1 --2- 172.21.15.111:0/638294623 >> [v2:172.21.15.111:6800/3125866964,v1:172.21.15.111:6801/3125866964] conn(0x7f2b800a32c0 0x7f2b800a5780 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:52.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:52.156+0000 7f2ba0e00640 1 --2- 172.21.15.111:0/638294623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2b9c074f40 0x7f2b9c19acb0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:52.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:52.156+0000 7f2ba0e00640 1 --2- 172.21.15.111:0/638294623 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2b9c074620 0x7f2b9c19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T23:12:52.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:52.156+0000 7f2ba0e00640 1 -- 172.21.15.111:0/638294623 >> 172.21.15.111:0/638294623 conn(0x7f2b9c0fd8c0 msgr2=0x7f2b9c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T23:12:52.158 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:52.157+0000 7f2ba0e00640 1 -- 172.21.15.111:0/638294623 shutdown_connections 2024-09-15T23:12:52.161 INFO:teuthology.orchestra.run.smithi111.stderr:2024-09-15T23:12:52.160+0000 7f2ba0e00640 1 -- 172.21.15.111:0/638294623 wait complete. 2024-09-15T23:12:52.415 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:52 smithi111 ceph-mon[30629]: pgmap v1161: 65 pgs: 65 active+clean; 577 KiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:12:52.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:52 smithi111 ceph-mon[30629]: from='client.? 172.21.15.111:0/638294623' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T23:12:52.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:52 smithi111 ceph-mon[30629]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T23:12:52.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:52 smithi192 ceph-mon[35427]: pgmap v1161: 65 pgs: 65 active+clean; 577 KiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:12:52.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:52 smithi192 ceph-mon[35427]: from='client.? 172.21.15.111:0/638294623' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T23:12:52.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:52 smithi192 ceph-mon[35427]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T23:12:52.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:52 smithi121 ceph-mon[34815]: pgmap v1161: 65 pgs: 65 active+clean; 577 KiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T23:12:52.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:52 smithi121 ceph-mon[34815]: from='client.? 172.21.15.111:0/638294623' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T23:12:52.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:52 smithi121 ceph-mon[34815]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T23:12:52.858 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-15T23:12:52.916 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-15T23:12:52.958 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-15T23:12:52.988 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-15T23:12:52.988 DEBUG:teuthology.orchestra.run.smithi111:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-15T23:12:53.024 DEBUG:teuthology.orchestra.run.smithi121:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-15T23:12:53.039 DEBUG:teuthology.orchestra.run.smithi192:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-15T23:12:53.055 INFO:tasks.cephadm:Stopping all daemons... 2024-09-15T23:12:53.055 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-15T23:12:53.055 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mon.a 2024-09-15T23:12:53.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:53 smithi192 ceph-mon[35427]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T23:12:53.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:53 smithi192 ceph-mon[35427]: osdmap e92: 12 total, 12 up, 12 in 2024-09-15T23:12:53.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:53 smithi121 ceph-mon[34815]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T23:12:53.390 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:53 smithi121 ceph-mon[34815]: osdmap e92: 12 total, 12 up, 12 in 2024-09-15T23:12:53.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:53 smithi111 systemd[1]: Stopping Ceph mon.a for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:12:53.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:53 smithi111 ceph-mon[30629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T23:12:53.416 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:53 smithi111 ceph-mon[30629]: osdmap e92: 12 total, 12 up, 12 in 2024-09-15T23:12:53.843 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:53 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-a[30625]: 2024-09-15T23:12:53.486+0000 7f64f8a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:12:53.843 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:53 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-a[30625]: 2024-09-15T23:12:53.486+0000 7f64f8a00640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-15T23:12:53.843 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:53 smithi111 podman[93722]: 2024-09-15 23:12:53.538358054 +0000 UTC m=+0.192231691 container died 200de39e07abb43550a97f99fafefa8bc7eb2f15408fdb45d95fab09f14d6c02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T23:12:54.165 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:53 smithi111 podman[93722]: 2024-09-15 23:12:53.964968481 +0000 UTC m=+0.618842114 container cleanup 200de39e07abb43550a97f99fafefa8bc7eb2f15408fdb45d95fab09f14d6c02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, ceph=True) 2024-09-15T23:12:54.166 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:53 smithi111 bash[93722]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-a 2024-09-15T23:12:54.523 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:54 smithi111 podman[93748]: 2024-09-15 23:12:54.200423246 +0000 UTC m=+0.650494925 container remove 200de39e07abb43550a97f99fafefa8bc7eb2f15408fdb45d95fab09f14d6c02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T23:12:54.915 INFO:journalctl@ceph.mon.a.smithi111.stdout:Sep 15 23:12:54 smithi111 podman[93841]: 2024-09-15 23:12:54.761726725 +0000 UTC m=+0.760026556 container died 734e8190a8588a64d6263be9c5f9244c5bc3e176135148380ca571ec40d42d2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=festive_torvalds, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-15T23:12:55.526 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mon.a.service' 2024-09-15T23:12:55.668 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:12:55.668 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-15T23:12:55.669 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-15T23:12:55.669 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mon.b 2024-09-15T23:12:55.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:55 smithi121 systemd[1]: Stopping Ceph mon.b for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:12:56.348 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:56 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-b[34811]: 2024-09-15T23:12:56.074+0000 7f5fb3946640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:12:56.348 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:56 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-b[34811]: 2024-09-15T23:12:56.074+0000 7f5fb3946640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-15T23:12:56.349 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:56 smithi121 podman[73476]: 2024-09-15 23:12:56.125531655 +0000 UTC m=+0.190091006 container died d9e180790a275da76614d717a65f703bcedb54720a6560540f4efd31123ed3bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T23:12:56.607 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:56 smithi121 podman[73476]: 2024-09-15 23:12:56.508059941 +0000 UTC m=+0.572619286 container cleanup d9e180790a275da76614d717a65f703bcedb54720a6560540f4efd31123ed3bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-15T23:12:56.607 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:56 smithi121 bash[73476]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-b 2024-09-15T23:12:57.640 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 15 23:12:57 smithi121 podman[73701]: 2024-09-15 23:12:57.388667047 +0000 UTC m=+1.257026744 container remove d9e180790a275da76614d717a65f703bcedb54720a6560540f4efd31123ed3bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-b, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-15T23:12:57.724 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mon.b.service' 2024-09-15T23:12:57.773 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:12:57.773 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-15T23:12:57.773 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-15T23:12:57.773 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mon.c 2024-09-15T23:12:58.064 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:57 smithi192 systemd[1]: Stopping Ceph mon.c for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:12:58.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:58 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-c[35420]: 2024-09-15T23:12:58.162+0000 7f7d2ea6f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:12:58.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:58 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-c[35420]: 2024-09-15T23:12:58.162+0000 7f7d2ea6f640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-15T23:12:58.565 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:58 smithi192 podman[73953]: 2024-09-15 23:12:58.38729265 +0000 UTC m=+0.416623840 container died 2e1cba46ba4ca1d933e3763cdea255491a0589209bb5cf96d46f9e7195784dbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-15T23:12:59.039 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:58 smithi192 podman[73953]: 2024-09-15 23:12:58.820811295 +0000 UTC m=+0.850142440 container cleanup 2e1cba46ba4ca1d933e3763cdea255491a0589209bb5cf96d46f9e7195784dbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-15T23:12:59.039 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:58 smithi192 bash[73953]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-c 2024-09-15T23:12:59.258 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mon.c.service' 2024-09-15T23:12:59.314 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:59 smithi192 podman[73965]: 2024-09-15 23:12:59.039115411 +0000 UTC m=+0.643920012 container remove 2e1cba46ba4ca1d933e3763cdea255491a0589209bb5cf96d46f9e7195784dbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mon-c, GIT_CLEAN=True, ceph=True, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-15T23:12:59.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:59 smithi192 systemd[1]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mon.c.service: Deactivated successfully. 2024-09-15T23:12:59.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:59 smithi192 systemd[1]: Stopped Ceph mon.c for b7614d3c-73b2-11ef-bceb-c7b262605968. 2024-09-15T23:12:59.315 INFO:journalctl@ceph.mon.c.smithi192.stdout:Sep 15 23:12:59 smithi192 systemd[1]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mon.c.service: Consumed 17.138s CPU time. 2024-09-15T23:12:59.342 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:12:59.342 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-15T23:12:59.342 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-15T23:12:59.342 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mgr.x 2024-09-15T23:12:59.666 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 23:12:59 smithi111 systemd[1]: Stopping Ceph mgr.x for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:13:00.146 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 23:12:59 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T23:12:59.786+0000 7f2a09a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:13:00.146 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 23:12:59 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x[30852]: 2024-09-15T23:12:59.786+0000 7f2a09a00640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-15T23:13:00.146 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 23:12:59 smithi111 podman[96254]: 2024-09-15 23:12:59.869114552 +0000 UTC m=+0.224283721 container died 4fb6ded66cc6e4cb5312f6d51dacc12b9f4467b257c66856930945b1c5601e37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T23:13:00.416 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 23:13:00 smithi111 podman[96254]: 2024-09-15 23:13:00.300559007 +0000 UTC m=+0.655728182 container cleanup 4fb6ded66cc6e4cb5312f6d51dacc12b9f4467b257c66856930945b1c5601e37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-15T23:13:00.416 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 23:13:00 smithi111 bash[96254]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x 2024-09-15T23:13:00.888 INFO:journalctl@ceph.mgr.x.smithi111.stdout:Sep 15 23:13:00 smithi111 podman[96265]: 2024-09-15 23:13:00.552276989 +0000 UTC m=+0.670074126 container remove 4fb6ded66cc6e4cb5312f6d51dacc12b9f4467b257c66856930945b1c5601e37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux ) 2024-09-15T23:13:00.914 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-15T23:13:00.992 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:13:00.992 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-15T23:13:00.992 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-15T23:13:00.993 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mgr.y 2024-09-15T23:13:01.313 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 23:13:01 smithi121 systemd[1]: Stopping Ceph mgr.y for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:13:01.585 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 23:13:01 smithi121 podman[76152]: 2024-09-15 23:13:01.312712823 +0000 UTC m=+0.121067118 container died cca4847b6242f87ced0a954c5a0ed88a0713fef0d7c0c039393a46ba0176ca05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T23:13:01.890 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 23:13:01 smithi121 podman[76152]: 2024-09-15 23:13:01.70019858 +0000 UTC m=+0.508552871 container cleanup cca4847b6242f87ced0a954c5a0ed88a0713fef0d7c0c039393a46ba0176ca05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-y, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-15T23:13:01.890 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 15 23:13:01 smithi121 bash[76152]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-y 2024-09-15T23:13:02.118 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-15T23:13:02.154 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:13:02.155 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-15T23:13:02.155 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-15T23:13:02.155 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mgr.z 2024-09-15T23:13:02.564 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 23:13:02 smithi192 systemd[1]: Stopping Ceph mgr.z for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:13:03.060 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 23:13:02 smithi192 podman[74078]: 2024-09-15 23:13:02.771628123 +0000 UTC m=+0.425390161 container died 983cb48dfec383b62c06bd24f21dcfac2803e854a16459fddcd21879470e4eb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, ceph=True) 2024-09-15T23:13:03.314 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 23:13:03 smithi192 podman[74078]: 2024-09-15 23:13:03.142765137 +0000 UTC m=+0.796527173 container cleanup 983cb48dfec383b62c06bd24f21dcfac2803e854a16459fddcd21879470e4eb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-15T23:13:03.314 INFO:journalctl@ceph.mgr.z.smithi192.stdout:Sep 15 23:13:03 smithi192 bash[74078]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mgr-z 2024-09-15T23:13:03.571 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-15T23:13:03.605 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:13:03.605 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-15T23:13:03.605 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-15T23:13:03.605 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.0 2024-09-15T23:13:03.916 INFO:journalctl@ceph.osd.0.smithi111.stdout:Sep 15 23:13:03 smithi111 systemd[1]: Stopping Ceph osd.0 for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:13:04.416 INFO:journalctl@ceph.osd.0.smithi111.stdout:Sep 15 23:13:04 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-0[44883]: 2024-09-15T23:13:04.024+0000 7fdd77e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:13:04.416 INFO:journalctl@ceph.osd.0.smithi111.stdout:Sep 15 23:13:04 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-0[44883]: 2024-09-15T23:13:04.024+0000 7fdd77e00640 -1 osd.0 92 *** Got signal Terminated *** 2024-09-15T23:13:04.416 INFO:journalctl@ceph.osd.0.smithi111.stdout:Sep 15 23:13:04 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-0[44883]: 2024-09-15T23:13:04.024+0000 7fdd77e00640 -1 osd.0 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T23:13:09.891 INFO:journalctl@ceph.osd.0.smithi111.stdout:Sep 15 23:13:09 smithi111 podman[96386]: 2024-09-15 23:13:09.592482933 +0000 UTC m=+5.687556294 container died 3834fe2a83b16ec858a93ef1a85a7ad38089d3615d40f708276354417d025e99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-15T23:13:10.165 INFO:journalctl@ceph.osd.0.smithi111.stdout:Sep 15 23:13:10 smithi111 podman[96386]: 2024-09-15 23:13:10.030027682 +0000 UTC m=+6.125101042 container cleanup 3834fe2a83b16ec858a93ef1a85a7ad38089d3615d40f708276354417d025e99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-15T23:13:10.166 INFO:journalctl@ceph.osd.0.smithi111.stdout:Sep 15 23:13:10 smithi111 bash[96386]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-0 2024-09-15T23:13:10.665 INFO:journalctl@ceph.osd.0.smithi111.stdout:Sep 15 23:13:10 smithi111 podman[96404]: 2024-09-15 23:13:10.2732979 +0000 UTC m=+0.676889930 container remove 3834fe2a83b16ec858a93ef1a85a7ad38089d3615d40f708276354417d025e99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-0, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T23:13:11.102 INFO:journalctl@ceph.osd.0.smithi111.stdout:Sep 15 23:13:10 smithi111 podman[96493]: 2024-09-15 23:13:10.706077168 +0000 UTC m=+0.026557584 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T23:13:11.102 INFO:journalctl@ceph.osd.0.smithi111.stdout:Sep 15 23:13:10 smithi111 podman[96493]: 2024-09-15 23:13:10.807521885 +0000 UTC m=+0.128002275 container create e79b745168db74520eeba448f8b0cb8f620d96337189ac7b0164f72cbbb1411c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-15T23:13:11.102 INFO:journalctl@ceph.osd.0.smithi111.stdout:Sep 15 23:13:11 smithi111 podman[96493]: 2024-09-15 23:13:11.010525295 +0000 UTC m=+0.331005685 container init e79b745168db74520eeba448f8b0cb8f620d96337189ac7b0164f72cbbb1411c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T23:13:11.102 INFO:journalctl@ceph.osd.0.smithi111.stdout:Sep 15 23:13:11 smithi111 podman[96493]: 2024-09-15 23:13:11.018102365 +0000 UTC m=+0.338582756 container start e79b745168db74520eeba448f8b0cb8f620d96337189ac7b0164f72cbbb1411c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-8921c8e, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T23:13:11.102 INFO:journalctl@ceph.osd.0.smithi111.stdout:Sep 15 23:13:11 smithi111 podman[96493]: 2024-09-15 23:13:11.101524102 +0000 UTC m=+0.422004504 container attach e79b745168db74520eeba448f8b0cb8f620d96337189ac7b0164f72cbbb1411c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-15T23:13:11.415 INFO:journalctl@ceph.osd.0.smithi111.stdout:Sep 15 23:13:11 smithi111 podman[96493]: 2024-09-15 23:13:11.272182249 +0000 UTC m=+0.592662641 container died e79b745168db74520eeba448f8b0cb8f620d96337189ac7b0164f72cbbb1411c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-15T23:13:11.980 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.0.service' 2024-09-15T23:13:12.057 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:13:12.057 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-15T23:13:12.058 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-15T23:13:12.058 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.1 2024-09-15T23:13:12.355 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 15 23:13:12 smithi121 systemd[1]: Stopping Ceph osd.1 for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:13:12.640 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 15 23:13:12 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-1[39310]: 2024-09-15T23:13:12.353+0000 7fa95e715640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:13:12.640 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 15 23:13:12 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-1[39310]: 2024-09-15T23:13:12.353+0000 7fa95e715640 -1 osd.1 92 *** Got signal Terminated *** 2024-09-15T23:13:12.640 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 15 23:13:12 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-1[39310]: 2024-09-15T23:13:12.353+0000 7fa95e715640 -1 osd.1 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T23:13:18.080 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 15 23:13:17 smithi121 podman[76291]: 2024-09-15 23:13:17.805794831 +0000 UTC m=+5.555302996 container died d6f2291033d83690ff2af94e0c129ee3cd27aa7cb6540253b41b8cc55bbfe82c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-1, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2) 2024-09-15T23:13:18.390 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 15 23:13:18 smithi121 podman[76291]: 2024-09-15 23:13:18.161783339 +0000 UTC m=+5.911291493 container cleanup d6f2291033d83690ff2af94e0c129ee3cd27aa7cb6540253b41b8cc55bbfe82c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T23:13:18.390 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 15 23:13:18 smithi121 bash[76291]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-1 2024-09-15T23:13:18.826 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 15 23:13:18 smithi121 podman[76306]: 2024-09-15 23:13:18.472229477 +0000 UTC m=+0.662694220 container remove d6f2291033d83690ff2af94e0c129ee3cd27aa7cb6540253b41b8cc55bbfe82c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-1, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-15T23:13:19.134 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 15 23:13:18 smithi121 podman[76397]: 2024-09-15 23:13:18.72843415 +0000 UTC m=+0.015442804 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T23:13:19.134 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 15 23:13:18 smithi121 podman[76397]: 2024-09-15 23:13:18.86470556 +0000 UTC m=+0.151714206 container create 27e2f7e357d5fee85817f55ada6f74e8d7229605407421f75dbcb4563a101b04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T23:13:19.135 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 15 23:13:19 smithi121 podman[76397]: 2024-09-15 23:13:19.133972807 +0000 UTC m=+0.420981454 container init 27e2f7e357d5fee85817f55ada6f74e8d7229605407421f75dbcb4563a101b04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD) 2024-09-15T23:13:19.390 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 15 23:13:19 smithi121 podman[76397]: 2024-09-15 23:13:19.137497691 +0000 UTC m=+0.424506350 container start 27e2f7e357d5fee85817f55ada6f74e8d7229605407421f75dbcb4563a101b04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-15T23:13:19.390 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 15 23:13:19 smithi121 podman[76397]: 2024-09-15 23:13:19.242259542 +0000 UTC m=+0.529268201 container attach 27e2f7e357d5fee85817f55ada6f74e8d7229605407421f75dbcb4563a101b04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-15T23:13:19.390 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 15 23:13:19 smithi121 podman[76397]: 2024-09-15 23:13:19.330954823 +0000 UTC m=+0.617963470 container died 27e2f7e357d5fee85817f55ada6f74e8d7229605407421f75dbcb4563a101b04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-15T23:13:19.971 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.1.service' 2024-09-15T23:13:20.012 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:13:20.012 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-15T23:13:20.013 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-15T23:13:20.013 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.2 2024-09-15T23:13:20.306 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 15 23:13:20 smithi192 systemd[1]: Stopping Ceph osd.2 for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:13:20.564 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 15 23:13:20 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-2[40429]: 2024-09-15T23:13:20.304+0000 7f5476e72640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:13:20.565 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 15 23:13:20 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-2[40429]: 2024-09-15T23:13:20.304+0000 7f5476e72640 -1 osd.2 92 *** Got signal Terminated *** 2024-09-15T23:13:20.565 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 15 23:13:20 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-2[40429]: 2024-09-15T23:13:20.304+0000 7f5476e72640 -1 osd.2 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T23:13:26.114 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 15 23:13:25 smithi192 podman[74212]: 2024-09-15 23:13:25.830948482 +0000 UTC m=+5.627690732 container died da1156454396881a4d4b09f4adb09b5bdcd28c32179733593559d9fcce395cae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-15T23:13:26.365 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 15 23:13:26 smithi192 podman[74212]: 2024-09-15 23:13:26.213397468 +0000 UTC m=+6.010139716 container cleanup da1156454396881a4d4b09f4adb09b5bdcd28c32179733593559d9fcce395cae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-15T23:13:26.365 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 15 23:13:26 smithi192 bash[74212]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-2 2024-09-15T23:13:26.757 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 15 23:13:26 smithi192 podman[74228]: 2024-09-15 23:13:26.364804375 +0000 UTC m=+0.533714942 container remove da1156454396881a4d4b09f4adb09b5bdcd28c32179733593559d9fcce395cae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T23:13:27.064 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 15 23:13:26 smithi192 podman[74316]: 2024-09-15 23:13:26.757203238 +0000 UTC m=+0.107859082 container create 123ee6777f0366bb6974babe6450d6e1de8247b3a46ec63c81b44953f505060f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-15T23:13:27.065 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 15 23:13:26 smithi192 podman[74316]: 2024-09-15 23:13:26.665034736 +0000 UTC m=+0.015690572 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T23:13:27.065 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 15 23:13:27 smithi192 podman[74316]: 2024-09-15 23:13:27.001807158 +0000 UTC m=+0.352462994 container init 123ee6777f0366bb6974babe6450d6e1de8247b3a46ec63c81b44953f505060f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True) 2024-09-15T23:13:27.065 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 15 23:13:27 smithi192 podman[74316]: 2024-09-15 23:13:27.005176958 +0000 UTC m=+0.355832799 container start 123ee6777f0366bb6974babe6450d6e1de8247b3a46ec63c81b44953f505060f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-15T23:13:27.471 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 15 23:13:27 smithi192 podman[74316]: 2024-09-15 23:13:27.110050686 +0000 UTC m=+0.460706526 container attach 123ee6777f0366bb6974babe6450d6e1de8247b3a46ec63c81b44953f505060f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T23:13:27.472 INFO:journalctl@ceph.osd.2.smithi192.stdout:Sep 15 23:13:27 smithi192 podman[74316]: 2024-09-15 23:13:27.194581545 +0000 UTC m=+0.545237382 container died 123ee6777f0366bb6974babe6450d6e1de8247b3a46ec63c81b44953f505060f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e) 2024-09-15T23:13:27.761 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.2.service' 2024-09-15T23:13:27.802 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:13:27.803 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-15T23:13:27.803 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-15T23:13:27.803 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.3 2024-09-15T23:13:28.166 INFO:journalctl@ceph.osd.3.smithi111.stdout:Sep 15 23:13:27 smithi111 systemd[1]: Stopping Ceph osd.3 for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:13:28.665 INFO:journalctl@ceph.osd.3.smithi111.stdout:Sep 15 23:13:28 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-3[51379]: 2024-09-15T23:13:28.209+0000 7fb97c600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:13:28.666 INFO:journalctl@ceph.osd.3.smithi111.stdout:Sep 15 23:13:28 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-3[51379]: 2024-09-15T23:13:28.209+0000 7fb97c600640 -1 osd.3 92 *** Got signal Terminated *** 2024-09-15T23:13:28.666 INFO:journalctl@ceph.osd.3.smithi111.stdout:Sep 15 23:13:28 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-3[51379]: 2024-09-15T23:13:28.209+0000 7fb97c600640 -1 osd.3 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T23:13:30.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:30 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:30.024+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:30.753 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:30 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:30.620+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:31.415 INFO:journalctl@ceph.osd.3.smithi111.stdout:Sep 15 23:13:31 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-3[51379]: 2024-09-15T23:13:31.183+0000 7fb978000640 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:06.700315+0000 front 2024-09-15T23:13:06.700348+0000 (oldest deadline 2024-09-15T23:13:30.800586+0000) 2024-09-15T23:13:31.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:31 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:31.066+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:31.564 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:31 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:31.284+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:06.350849+0000 front 2024-09-15T23:13:06.351031+0000 (oldest deadline 2024-09-15T23:13:30.450780+0000) 2024-09-15T23:13:31.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:31 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:31.264+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:31.914 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:31 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:31.651+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:32.165 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:32 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:32.087+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:32.564 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:32 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:32.288+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:06.350849+0000 front 2024-09-15T23:13:06.351031+0000 (oldest deadline 2024-09-15T23:13:30.450780+0000) 2024-09-15T23:13:32.617 INFO:journalctl@ceph.osd.3.smithi111.stdout:Sep 15 23:13:32 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-3[51379]: 2024-09-15T23:13:32.200+0000 7fb978000640 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:06.700315+0000 front 2024-09-15T23:13:06.700348+0000 (oldest deadline 2024-09-15T23:13:30.800586+0000) 2024-09-15T23:13:32.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:32 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:32.306+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:32.915 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:32 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:32.616+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:33.295 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:32 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:32.999+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:13:33.415 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:33 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:33.110+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:33.564 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:33 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:33.294+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:06.350849+0000 front 2024-09-15T23:13:06.351031+0000 (oldest deadline 2024-09-15T23:13:30.450780+0000) 2024-09-15T23:13:33.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:33 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:33.293+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:33.915 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:33 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:33.643+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:34.300 INFO:journalctl@ceph.osd.3.smithi111.stdout:Sep 15 23:13:34 smithi111 podman[96599]: 2024-09-15 23:13:34.018044561 +0000 UTC m=+5.926153826 container died c1ea6241f8c3ad29eb4817912d47259da26b172989ee6798fbdd012ea9a901d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-15T23:13:34.300 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:34 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:34.084+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:34.314 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:33 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:33.997+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:13:34.600 INFO:journalctl@ceph.osd.3.smithi111.stdout:Sep 15 23:13:34 smithi111 podman[96599]: 2024-09-15 23:13:34.41448245 +0000 UTC m=+6.322591722 container cleanup c1ea6241f8c3ad29eb4817912d47259da26b172989ee6798fbdd012ea9a901d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-15T23:13:34.600 INFO:journalctl@ceph.osd.3.smithi111.stdout:Sep 15 23:13:34 smithi111 bash[96599]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-3 2024-09-15T23:13:34.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:34 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:34.291+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:34.814 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:34 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:34.319+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:06.350849+0000 front 2024-09-15T23:13:06.351031+0000 (oldest deadline 2024-09-15T23:13:30.450780+0000) 2024-09-15T23:13:34.915 INFO:journalctl@ceph.osd.3.smithi111.stdout:Sep 15 23:13:34 smithi111 podman[96628]: 2024-09-15 23:13:34.716263314 +0000 UTC m=+0.685647095 container remove c1ea6241f8c3ad29eb4817912d47259da26b172989ee6798fbdd012ea9a901d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T23:13:34.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:34 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:34.598+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:35.288 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:34 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:34.954+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:13:35.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:35 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:35.129+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:35.416 INFO:journalctl@ceph.osd.3.smithi111.stdout:Sep 15 23:13:35 smithi111 podman[96715]: 2024-09-15 23:13:35.053598247 +0000 UTC m=+0.026819794 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T23:13:35.416 INFO:journalctl@ceph.osd.3.smithi111.stdout:Sep 15 23:13:35 smithi111 podman[96715]: 2024-09-15 23:13:35.183823466 +0000 UTC m=+0.157045021 container create 40cf918a44529b371036f6cb2e94d5515732bfef27c08320be915dcde32f679a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-15T23:13:35.564 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:35 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:35.287+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:06.350849+0000 front 2024-09-15T23:13:06.351031+0000 (oldest deadline 2024-09-15T23:13:30.450780+0000) 2024-09-15T23:13:35.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:35 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:35.337+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:35.693 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:35 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:35.601+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:35.693 INFO:journalctl@ceph.osd.3.smithi111.stdout:Sep 15 23:13:35 smithi111 podman[96715]: 2024-09-15 23:13:35.435902434 +0000 UTC m=+0.409123980 container init 40cf918a44529b371036f6cb2e94d5515732bfef27c08320be915dcde32f679a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-15T23:13:35.693 INFO:journalctl@ceph.osd.3.smithi111.stdout:Sep 15 23:13:35 smithi111 podman[96715]: 2024-09-15 23:13:35.442814295 +0000 UTC m=+0.416035852 container start 40cf918a44529b371036f6cb2e94d5515732bfef27c08320be915dcde32f679a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T23:13:35.693 INFO:journalctl@ceph.osd.3.smithi111.stdout:Sep 15 23:13:35 smithi111 podman[96715]: 2024-09-15 23:13:35.527506224 +0000 UTC m=+0.500727771 container attach 40cf918a44529b371036f6cb2e94d5515732bfef27c08320be915dcde32f679a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-15T23:13:36.030 INFO:journalctl@ceph.osd.3.smithi111.stdout:Sep 15 23:13:35 smithi111 podman[96738]: 2024-09-15 23:13:35.731998502 +0000 UTC m=+0.026175416 container died 40cf918a44529b371036f6cb2e94d5515732bfef27c08320be915dcde32f679a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-15T23:13:36.256 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:35 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:35.911+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:13:36.373 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:36 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:36.156+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:36.373 INFO:journalctl@ceph.osd.3.smithi111.stdout:Sep 15 23:13:36 smithi111 podman[96738]: 2024-09-15 23:13:36.3726267 +0000 UTC m=+0.666803615 container remove 40cf918a44529b371036f6cb2e94d5515732bfef27c08320be915dcde32f679a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, io.buildah.version=1.37.2) 2024-09-15T23:13:36.404 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.3.service' 2024-09-15T23:13:36.480 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:13:36.481 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-15T23:13:36.481 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-15T23:13:36.481 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.4 2024-09-15T23:13:36.564 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:36 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:36.254+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:06.350849+0000 front 2024-09-15T23:13:06.351031+0000 (oldest deadline 2024-09-15T23:13:30.450780+0000) 2024-09-15T23:13:36.640 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 15 23:13:36 smithi121 systemd[1]: Stopping Ceph osd.4 for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:13:36.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:36 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:36.336+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:36.665 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:36 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:36.577+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:37.140 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 15 23:13:36 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-4[46255]: 2024-09-15T23:13:36.779+0000 7fc2b8550640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:13:37.140 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 15 23:13:36 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-4[46255]: 2024-09-15T23:13:36.779+0000 7fc2b8550640 -1 osd.4 92 *** Got signal Terminated *** 2024-09-15T23:13:37.140 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 15 23:13:36 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-4[46255]: 2024-09-15T23:13:36.779+0000 7fc2b8550640 -1 osd.4 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T23:13:37.231 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:36 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:36.903+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:13:37.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:37 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:37.112+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:37.564 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:37 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:37.229+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:06.350849+0000 front 2024-09-15T23:13:06.351031+0000 (oldest deadline 2024-09-15T23:13:30.450780+0000) 2024-09-15T23:13:37.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:37 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:37.289+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:37.915 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:37 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:37.589+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:38.182 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:37 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:37.897+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:13:38.417 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:38 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:38.143+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:38.564 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:38 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:38.181+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:06.350849+0000 front 2024-09-15T23:13:06.351031+0000 (oldest deadline 2024-09-15T23:13:30.450780+0000) 2024-09-15T23:13:38.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:38 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:38.334+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:38.915 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:38 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:38.591+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:39.140 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 15 23:13:38 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-4[46255]: 2024-09-15T23:13:38.761+0000 7fc2b4b55640 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.842429+0000 front 2024-09-15T23:13:12.842451+0000 (oldest deadline 2024-09-15T23:13:38.742647+0000) 2024-09-15T23:13:39.159 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:38 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:38.886+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:13:39.159 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:38 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:38.886+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:13:39.564 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:39 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:39.158+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:06.350849+0000 front 2024-09-15T23:13:06.351031+0000 (oldest deadline 2024-09-15T23:13:30.450780+0000) 2024-09-15T23:13:39.599 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:39 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:39.186+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:39.599 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:39 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:39.186+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:13:39.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:39 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:39.382+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:39.907 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:39 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:39.817+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:13:39.915 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:39 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:39.598+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:39.915 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:39 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:39.598+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:14.024590+0000 front 2024-09-15T23:13:14.024498+0000 (oldest deadline 2024-09-15T23:13:39.324528+0000) 2024-09-15T23:13:40.140 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 15 23:13:39 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-4[46255]: 2024-09-15T23:13:39.750+0000 7fc2b4b55640 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.842429+0000 front 2024-09-15T23:13:12.842451+0000 (oldest deadline 2024-09-15T23:13:38.742647+0000) 2024-09-15T23:13:40.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:39 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:39.677+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.914943+0000 front 2024-09-15T23:13:15.914917+0000 (oldest deadline 2024-09-15T23:13:39.415083+0000) 2024-09-15T23:13:40.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:39 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:39.906+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:13:40.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:39 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:39.906+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:13:40.315 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:40 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:40.139+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:06.350849+0000 front 2024-09-15T23:13:06.351031+0000 (oldest deadline 2024-09-15T23:13:30.450780+0000) 2024-09-15T23:13:40.570 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:40 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:40.176+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:40.571 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:40 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:40.176+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:13:40.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:40 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:40.371+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:40.915 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:40 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:40.569+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:40.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:40 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:40.569+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:14.024590+0000 front 2024-09-15T23:13:14.024498+0000 (oldest deadline 2024-09-15T23:13:39.324528+0000) 2024-09-15T23:13:41.140 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 15 23:13:40 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-4[46255]: 2024-09-15T23:13:40.722+0000 7fc2b4b55640 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.842429+0000 front 2024-09-15T23:13:12.842451+0000 (oldest deadline 2024-09-15T23:13:38.742647+0000) 2024-09-15T23:13:41.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:40 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:40.713+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.914943+0000 front 2024-09-15T23:13:15.914917+0000 (oldest deadline 2024-09-15T23:13:39.415083+0000) 2024-09-15T23:13:41.314 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:40 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:40.865+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:13:41.315 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:41 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:41.092+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:06.350849+0000 front 2024-09-15T23:13:06.351031+0000 (oldest deadline 2024-09-15T23:13:30.450780+0000) 2024-09-15T23:13:41.315 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:41 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:41.092+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:16.852462+0000 front 2024-09-15T23:13:16.852569+0000 (oldest deadline 2024-09-15T23:13:40.952496+0000) 2024-09-15T23:13:41.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:40 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:40.864+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:13:41.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:40 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:40.864+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:13:41.417 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:41 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:41.330+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:41.543 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:41 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:41.213+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:41.543 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:41 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:41.213+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:13:41.915 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:41 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:41.542+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:41.915 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:41 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:41.542+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:14.024590+0000 front 2024-09-15T23:13:14.024498+0000 (oldest deadline 2024-09-15T23:13:39.324528+0000) 2024-09-15T23:13:42.105 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:41 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:41.834+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:13:42.106 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:41 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:41.841+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:13:42.106 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:41 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:41.841+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:13:42.140 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 15 23:13:41 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-4[46255]: 2024-09-15T23:13:41.724+0000 7fc2b4b55640 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.842429+0000 front 2024-09-15T23:13:12.842451+0000 (oldest deadline 2024-09-15T23:13:38.742647+0000) 2024-09-15T23:13:42.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:41 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:41.749+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.914943+0000 front 2024-09-15T23:13:15.914917+0000 (oldest deadline 2024-09-15T23:13:39.415083+0000) 2024-09-15T23:13:42.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:42 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:42.209+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:42.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:42 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:42.209+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:13:42.539 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 15 23:13:42 smithi121 podman[76499]: 2024-09-15 23:13:42.267319242 +0000 UTC m=+5.591557974 container died 77125fb77304971c2b445bde59fd7b75c112ee5b4923dcab098e433e87d29b1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-4, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T23:13:42.539 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:42 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:42.377+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:42.540 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:42 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:42.377+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:13:42.564 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:42 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:42.104+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:06.350849+0000 front 2024-09-15T23:13:06.351031+0000 (oldest deadline 2024-09-15T23:13:30.450780+0000) 2024-09-15T23:13:42.564 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:42 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:42.104+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:16.852462+0000 front 2024-09-15T23:13:16.852569+0000 (oldest deadline 2024-09-15T23:13:40.952496+0000) 2024-09-15T23:13:42.890 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:42 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:42.703+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.914943+0000 front 2024-09-15T23:13:15.914917+0000 (oldest deadline 2024-09-15T23:13:39.415083+0000) 2024-09-15T23:13:42.890 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 15 23:13:42 smithi121 podman[76499]: 2024-09-15 23:13:42.646530242 +0000 UTC m=+5.970768973 container cleanup 77125fb77304971c2b445bde59fd7b75c112ee5b4923dcab098e433e87d29b1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-4, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T23:13:42.890 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 15 23:13:42 smithi121 bash[76499]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-4 2024-09-15T23:13:42.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:42 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:42.579+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:42.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:42 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:42.579+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:14.024590+0000 front 2024-09-15T23:13:14.024498+0000 (oldest deadline 2024-09-15T23:13:39.324528+0000) 2024-09-15T23:13:43.064 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:42 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:42.796+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:13:43.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:42 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:42.867+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:13:43.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:42 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:42.867+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:13:43.259 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 15 23:13:42 smithi121 podman[76526]: 2024-09-15 23:13:42.914503438 +0000 UTC m=+0.648472246 container remove 77125fb77304971c2b445bde59fd7b75c112ee5b4923dcab098e433e87d29b1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-4, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-15T23:13:43.535 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:43 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:43.253+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:43.536 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:43 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:43.253+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:13:43.551 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 15 23:13:43 smithi121 podman[76618]: 2024-09-15 23:13:43.161243052 +0000 UTC m=+0.015648556 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T23:13:43.552 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 15 23:13:43 smithi121 podman[76618]: 2024-09-15 23:13:43.307282319 +0000 UTC m=+0.161687871 container create f3f0da32da9d04acbd3e54218368185966b99ce2141c308407aead0279f96eeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T23:13:43.552 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:43 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:43.386+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:43.552 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:43 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:43.386+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:13:43.564 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:43 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:43.081+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:06.350849+0000 front 2024-09-15T23:13:06.351031+0000 (oldest deadline 2024-09-15T23:13:30.450780+0000) 2024-09-15T23:13:43.564 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:43 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:43.081+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:16.852462+0000 front 2024-09-15T23:13:16.852569+0000 (oldest deadline 2024-09-15T23:13:40.952496+0000) 2024-09-15T23:13:43.890 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:43 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:43.752+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.914943+0000 front 2024-09-15T23:13:15.914917+0000 (oldest deadline 2024-09-15T23:13:39.415083+0000) 2024-09-15T23:13:43.891 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 15 23:13:43 smithi121 podman[76618]: 2024-09-15 23:13:43.551509513 +0000 UTC m=+0.405915007 container init f3f0da32da9d04acbd3e54218368185966b99ce2141c308407aead0279f96eeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T23:13:43.891 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 15 23:13:43 smithi121 podman[76618]: 2024-09-15 23:13:43.554847276 +0000 UTC m=+0.409252769 container start f3f0da32da9d04acbd3e54218368185966b99ce2141c308407aead0279f96eeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T23:13:43.891 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 15 23:13:43 smithi121 podman[76618]: 2024-09-15 23:13:43.634748709 +0000 UTC m=+0.489154211 container attach f3f0da32da9d04acbd3e54218368185966b99ce2141c308407aead0279f96eeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-4-deactivate, RELEASE=squid-8921c8e, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T23:13:43.891 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 15 23:13:43 smithi121 podman[76618]: 2024-09-15 23:13:43.754775348 +0000 UTC m=+0.609180842 container died f3f0da32da9d04acbd3e54218368185966b99ce2141c308407aead0279f96eeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-15T23:13:43.915 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:43 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:43.534+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:43.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:43 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:43.534+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:14.024590+0000 front 2024-09-15T23:13:14.024498+0000 (oldest deadline 2024-09-15T23:13:39.324528+0000) 2024-09-15T23:13:44.064 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:43 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:43.748+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:13:44.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:43 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:43.860+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:13:44.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:43 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:43.860+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:13:44.315 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.4.service' 2024-09-15T23:13:44.355 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:13:44.355 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-15T23:13:44.355 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-15T23:13:44.355 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.5 2024-09-15T23:13:44.385 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:44 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:44.111+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:06.350849+0000 front 2024-09-15T23:13:06.351031+0000 (oldest deadline 2024-09-15T23:13:30.450780+0000) 2024-09-15T23:13:44.385 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:44 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:44.111+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:16.852462+0000 front 2024-09-15T23:13:16.852569+0000 (oldest deadline 2024-09-15T23:13:40.952496+0000) 2024-09-15T23:13:44.548 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:44 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:44.270+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:44.549 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:44 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:44.270+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:13:44.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:44 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:44.420+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:44.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:44 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:44.420+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:13:44.717 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:44 smithi192 systemd[1]: Stopping Ceph osd.5 for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:13:44.717 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:44 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:44.715+0000 7fb4ce85d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:13:44.717 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:44 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:44.715+0000 7fb4ce85d640 -1 osd.5 92 *** Got signal Terminated *** 2024-09-15T23:13:44.717 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:44 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:44.715+0000 7fb4ce85d640 -1 osd.5 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T23:13:44.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:44 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:44.547+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:44.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:44 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:44.547+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:14.024590+0000 front 2024-09-15T23:13:14.024498+0000 (oldest deadline 2024-09-15T23:13:39.324528+0000) 2024-09-15T23:13:45.064 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:44 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:44.783+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:13:45.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:44 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:44.887+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:13:45.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:44 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:44.887+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:13:45.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:44 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:44.801+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.914943+0000 front 2024-09-15T23:13:15.914917+0000 (oldest deadline 2024-09-15T23:13:39.415083+0000) 2024-09-15T23:13:45.564 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:45 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:45.068+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:06.350849+0000 front 2024-09-15T23:13:06.351031+0000 (oldest deadline 2024-09-15T23:13:30.450780+0000) 2024-09-15T23:13:45.565 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:45 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:45.068+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:16.852462+0000 front 2024-09-15T23:13:16.852569+0000 (oldest deadline 2024-09-15T23:13:40.952496+0000) 2024-09-15T23:13:45.595 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:45 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:45.290+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:45.595 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:45 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:45.290+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:13:45.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:45 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:45.381+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:45.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:45 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:45.381+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:13:45.915 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:45 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:45.593+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:45.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:45 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:45.593+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:14.024590+0000 front 2024-09-15T23:13:14.024498+0000 (oldest deadline 2024-09-15T23:13:39.324528+0000) 2024-09-15T23:13:46.064 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:45 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:45.749+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:13:46.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:45 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:45.840+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:13:46.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:45 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:45.840+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:13:46.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:45 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:45.788+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.914943+0000 front 2024-09-15T23:13:15.914917+0000 (oldest deadline 2024-09-15T23:13:39.415083+0000) 2024-09-15T23:13:46.564 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:46 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:46.096+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:06.350849+0000 front 2024-09-15T23:13:06.351031+0000 (oldest deadline 2024-09-15T23:13:30.450780+0000) 2024-09-15T23:13:46.564 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:46 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:46.096+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:16.852462+0000 front 2024-09-15T23:13:16.852569+0000 (oldest deadline 2024-09-15T23:13:40.952496+0000) 2024-09-15T23:13:46.590 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:46 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:46.286+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:46.590 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:46 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:46.286+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:13:46.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:46 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:46.367+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:46.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:46 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:46.367+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:13:46.915 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:46 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:46.589+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:46.915 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:46 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:46.589+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:14.024590+0000 front 2024-09-15T23:13:14.024498+0000 (oldest deadline 2024-09-15T23:13:39.324528+0000) 2024-09-15T23:13:47.064 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:46 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:46.774+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:13:47.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:46 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:46.873+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:13:47.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:46 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:46.873+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:13:47.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:46 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:46.873+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:13:47.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:46 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:46.750+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.914943+0000 front 2024-09-15T23:13:15.914917+0000 (oldest deadline 2024-09-15T23:13:39.415083+0000) 2024-09-15T23:13:47.564 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:47 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:47.140+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:06.350849+0000 front 2024-09-15T23:13:06.351031+0000 (oldest deadline 2024-09-15T23:13:30.450780+0000) 2024-09-15T23:13:47.565 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:47 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:47.140+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:16.852462+0000 front 2024-09-15T23:13:16.852569+0000 (oldest deadline 2024-09-15T23:13:40.952496+0000) 2024-09-15T23:13:47.578 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:47 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:47.237+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:47.578 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:47 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:47.237+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:13:47.578 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:47 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:47.237+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:13:47.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:47 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:47.322+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:47.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:47 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:47.322+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:13:47.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:47 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:47.322+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:13:47.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:47 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:47.577+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:47.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:47 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:47.577+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:14.024590+0000 front 2024-09-15T23:13:14.024498+0000 (oldest deadline 2024-09-15T23:13:39.324528+0000) 2024-09-15T23:13:48.064 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:47 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:47.742+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:13:48.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:47 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:47.846+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:13:48.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:47 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:47.846+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:13:48.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:47 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:47.846+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:13:48.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:47 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:47.719+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.914943+0000 front 2024-09-15T23:13:15.914917+0000 (oldest deadline 2024-09-15T23:13:39.415083+0000) 2024-09-15T23:13:48.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:47 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:47.719+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:24.716108+0000 front 2024-09-15T23:13:24.716135+0000 (oldest deadline 2024-09-15T23:13:47.016177+0000) 2024-09-15T23:13:48.537 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:48 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:48.248+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:48.537 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:48 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:48.248+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:13:48.537 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:48 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:48.248+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:13:48.564 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:48 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:48.140+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:06.350849+0000 front 2024-09-15T23:13:06.351031+0000 (oldest deadline 2024-09-15T23:13:30.450780+0000) 2024-09-15T23:13:48.564 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:48 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:48.140+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:16.852462+0000 front 2024-09-15T23:13:16.852569+0000 (oldest deadline 2024-09-15T23:13:40.952496+0000) 2024-09-15T23:13:48.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:48 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:48.317+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:48.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:48 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:48.317+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:13:48.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:48 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:48.317+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:13:48.915 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:48 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:48.536+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:48.915 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:48 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:48.536+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:14.024590+0000 front 2024-09-15T23:13:14.024498+0000 (oldest deadline 2024-09-15T23:13:39.324528+0000) 2024-09-15T23:13:49.064 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:48 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:48.710+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:13:49.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:48 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:48.893+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:13:49.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:48 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:48.893+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:13:49.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:48 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:48.893+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:13:49.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:48 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:48.749+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.914943+0000 front 2024-09-15T23:13:15.914917+0000 (oldest deadline 2024-09-15T23:13:39.415083+0000) 2024-09-15T23:13:49.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:48 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:48.749+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:24.716108+0000 front 2024-09-15T23:13:24.716135+0000 (oldest deadline 2024-09-15T23:13:47.016177+0000) 2024-09-15T23:13:49.564 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:49 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:49.099+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:06.350849+0000 front 2024-09-15T23:13:06.351031+0000 (oldest deadline 2024-09-15T23:13:30.450780+0000) 2024-09-15T23:13:49.564 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:49 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5[46921]: 2024-09-15T23:13:49.099+0000 7fb4cae62640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:16.852462+0000 front 2024-09-15T23:13:16.852569+0000 (oldest deadline 2024-09-15T23:13:40.952496+0000) 2024-09-15T23:13:49.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:49 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:49.271+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:49.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:49 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:49.271+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:13:49.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:49 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:49.271+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:13:49.665 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:49 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:49.545+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:49.666 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:49 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:49.545+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:14.024590+0000 front 2024-09-15T23:13:14.024498+0000 (oldest deadline 2024-09-15T23:13:39.324528+0000) 2024-09-15T23:13:49.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:49 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:49.298+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:49.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:49 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:49.298+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:13:49.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:49 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:49.298+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:13:50.064 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:49 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:49.724+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:13:50.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:49 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:49.906+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:13:50.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:49 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:49.906+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:13:50.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:49 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:49.906+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:13:50.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:49 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:49.722+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.914943+0000 front 2024-09-15T23:13:15.914917+0000 (oldest deadline 2024-09-15T23:13:39.415083+0000) 2024-09-15T23:13:50.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:49 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:49.722+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:24.716108+0000 front 2024-09-15T23:13:24.716135+0000 (oldest deadline 2024-09-15T23:13:47.016177+0000) 2024-09-15T23:13:50.550 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:50 smithi192 podman[74445]: 2024-09-15 23:13:50.274844225 +0000 UTC m=+5.722884285 container died b2b27757ef15ba84b5e68671a96b29bd2b4b262a7e1c3437996ea1bded8426ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=) 2024-09-15T23:13:50.564 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:50 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:50.270+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:50.564 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:50 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:50.270+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:13:50.564 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:50 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:50.270+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:13:50.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:50 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:50.294+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:50.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:50 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:50.294+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:13:50.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:50 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:50.294+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:13:50.814 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:50 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:50.754+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:13:50.815 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:50 smithi192 podman[74445]: 2024-09-15 23:13:50.640848699 +0000 UTC m=+6.088888758 container cleanup b2b27757ef15ba84b5e68671a96b29bd2b4b262a7e1c3437996ea1bded8426ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0) 2024-09-15T23:13:50.815 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:50 smithi192 bash[74445]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5 2024-09-15T23:13:50.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:50 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:50.563+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:50.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:50 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:50.563+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:14.024590+0000 front 2024-09-15T23:13:14.024498+0000 (oldest deadline 2024-09-15T23:13:39.324528+0000) 2024-09-15T23:13:50.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:50 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:50.563+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:24.526557+0000 front 2024-09-15T23:13:24.526379+0000 (oldest deadline 2024-09-15T23:13:50.326723+0000) 2024-09-15T23:13:51.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:50 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:50.753+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.914943+0000 front 2024-09-15T23:13:15.914917+0000 (oldest deadline 2024-09-15T23:13:39.415083+0000) 2024-09-15T23:13:51.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:50 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:50.753+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:24.716108+0000 front 2024-09-15T23:13:24.716135+0000 (oldest deadline 2024-09-15T23:13:47.016177+0000) 2024-09-15T23:13:51.257 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:50 smithi192 podman[74460]: 2024-09-15 23:13:50.925585167 +0000 UTC m=+0.650378452 container remove b2b27757ef15ba84b5e68671a96b29bd2b4b262a7e1c3437996ea1bded8426ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, io.buildah.version=1.37.2) 2024-09-15T23:13:51.257 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:50 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:50.925+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:13:51.257 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:50 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:50.925+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:13:51.257 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:50 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:50.925+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:13:51.564 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:51 smithi192 podman[74553]: 2024-09-15 23:13:51.159521761 +0000 UTC m=+0.015168328 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T23:13:51.565 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:51 smithi192 podman[74553]: 2024-09-15 23:13:51.268144546 +0000 UTC m=+0.123791138 container create c49c53b2715f6d3b175ca008161462f6074aaa5babb5731ad7d4d8735a15cccd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, io.buildah.version=1.37.2) 2024-09-15T23:13:51.565 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:51 smithi192 podman[74553]: 2024-09-15 23:13:51.495633354 +0000 UTC m=+0.351279903 container init c49c53b2715f6d3b175ca008161462f6074aaa5babb5731ad7d4d8735a15cccd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-15T23:13:51.565 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:51 smithi192 podman[74553]: 2024-09-15 23:13:51.499084486 +0000 UTC m=+0.354731041 container start c49c53b2715f6d3b175ca008161462f6074aaa5babb5731ad7d4d8735a15cccd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-8921c8e) 2024-09-15T23:13:51.597 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:51 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:51.273+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:51.598 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:51 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:51.273+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:13:51.598 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:51 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:51.273+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:13:51.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:51 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:51.259+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:51.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:51 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:51.259+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:13:51.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:51 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:51.259+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:13:51.915 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:51 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:51.596+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:51.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:51 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:51.596+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:14.024590+0000 front 2024-09-15T23:13:14.024498+0000 (oldest deadline 2024-09-15T23:13:39.324528+0000) 2024-09-15T23:13:51.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:51 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:51.596+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:24.526557+0000 front 2024-09-15T23:13:24.526379+0000 (oldest deadline 2024-09-15T23:13:50.326723+0000) 2024-09-15T23:13:51.967 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:51 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:51.760+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:13:51.967 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:51 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:51.760+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:25.298691+0000 front 2024-09-15T23:13:25.298714+0000 (oldest deadline 2024-09-15T23:13:51.198828+0000) 2024-09-15T23:13:51.968 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:51 smithi192 podman[74553]: 2024-09-15 23:13:51.587556576 +0000 UTC m=+0.443203144 container attach c49c53b2715f6d3b175ca008161462f6074aaa5babb5731ad7d4d8735a15cccd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, io.buildah.version=1.37.2) 2024-09-15T23:13:51.968 INFO:journalctl@ceph.osd.5.smithi192.stdout:Sep 15 23:13:51 smithi192 podman[74553]: 2024-09-15 23:13:51.693998851 +0000 UTC m=+0.549645402 container died c49c53b2715f6d3b175ca008161462f6074aaa5babb5731ad7d4d8735a15cccd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-5-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731) 2024-09-15T23:13:52.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:51 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:51.721+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.914943+0000 front 2024-09-15T23:13:15.914917+0000 (oldest deadline 2024-09-15T23:13:39.415083+0000) 2024-09-15T23:13:52.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:51 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:51.721+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:24.716108+0000 front 2024-09-15T23:13:24.716135+0000 (oldest deadline 2024-09-15T23:13:47.016177+0000) 2024-09-15T23:13:52.314 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:51 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:51.964+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:13:52.314 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:51 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:51.964+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:13:52.314 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:51 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:51.964+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:13:52.346 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.5.service' 2024-09-15T23:13:52.426 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:13:52.426 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-15T23:13:52.426 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-15T23:13:52.426 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.6 2024-09-15T23:13:52.599 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:52 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:52.275+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:52.599 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:52 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:52.275+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:13:52.599 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:52 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:52.275+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:13:52.600 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:52 smithi111 systemd[1]: Stopping Ceph osd.6 for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:13:52.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:52 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:52.266+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:52.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:52 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:52.266+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:13:52.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:52 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:52.266+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:13:52.915 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:52 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:52.597+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:52.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:52 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:52.597+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:14.024590+0000 front 2024-09-15T23:13:14.024498+0000 (oldest deadline 2024-09-15T23:13:39.324528+0000) 2024-09-15T23:13:52.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:52 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:52.597+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:24.526557+0000 front 2024-09-15T23:13:24.526379+0000 (oldest deadline 2024-09-15T23:13:50.326723+0000) 2024-09-15T23:13:52.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:52 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:52.827+0000 7fb7b5a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:13:52.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:52 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:52.827+0000 7fb7b5a00640 -1 osd.6 92 *** Got signal Terminated *** 2024-09-15T23:13:52.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:52 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:52.827+0000 7fb7b5a00640 -1 osd.6 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T23:13:53.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:52 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:52.797+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:13:53.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:52 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:52.797+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:25.298691+0000 front 2024-09-15T23:13:25.298714+0000 (oldest deadline 2024-09-15T23:13:51.198828+0000) 2024-09-15T23:13:53.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:52 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:52.957+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:13:53.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:52 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:52.957+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:13:53.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:52 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:52.957+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:13:53.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:52 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:52.690+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.914943+0000 front 2024-09-15T23:13:15.914917+0000 (oldest deadline 2024-09-15T23:13:39.415083+0000) 2024-09-15T23:13:53.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:52 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:52.690+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:24.716108+0000 front 2024-09-15T23:13:24.716135+0000 (oldest deadline 2024-09-15T23:13:47.016177+0000) 2024-09-15T23:13:53.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:53 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:53.317+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:53.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:53 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:53.317+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:13:53.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:53 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:53.317+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:13:53.643 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:53 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:53.316+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:53.643 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:53 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:53.316+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:13:53.643 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:53 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:53.316+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:13:53.915 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:53 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:53.642+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:53.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:53 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:53.642+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:14.024590+0000 front 2024-09-15T23:13:14.024498+0000 (oldest deadline 2024-09-15T23:13:39.324528+0000) 2024-09-15T23:13:53.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:53 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:53.642+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:24.526557+0000 front 2024-09-15T23:13:24.526379+0000 (oldest deadline 2024-09-15T23:13:50.326723+0000) 2024-09-15T23:13:54.064 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:53 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:53.766+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:13:54.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:53 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:53.766+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:25.298691+0000 front 2024-09-15T23:13:25.298714+0000 (oldest deadline 2024-09-15T23:13:51.198828+0000) 2024-09-15T23:13:54.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:53 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:53.983+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:13:54.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:53 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:53.983+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:13:54.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:53 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:53.983+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:13:54.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:53 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:53.695+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.914943+0000 front 2024-09-15T23:13:15.914917+0000 (oldest deadline 2024-09-15T23:13:39.415083+0000) 2024-09-15T23:13:54.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:53 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:53.695+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:24.716108+0000 front 2024-09-15T23:13:24.716135+0000 (oldest deadline 2024-09-15T23:13:47.016177+0000) 2024-09-15T23:13:54.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:54 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:54.300+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:54.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:54 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:54.300+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:13:54.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:54 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:54.300+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:13:54.650 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:54 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:54.328+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:54.650 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:54 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:54.328+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:13:54.650 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:54 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:54.328+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:13:54.915 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:54 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:54.649+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:54.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:54 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:54.649+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:14.024590+0000 front 2024-09-15T23:13:14.024498+0000 (oldest deadline 2024-09-15T23:13:39.324528+0000) 2024-09-15T23:13:54.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:54 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:54.649+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:24.526557+0000 front 2024-09-15T23:13:24.526379+0000 (oldest deadline 2024-09-15T23:13:50.326723+0000) 2024-09-15T23:13:54.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:54 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:54.649+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:30.827625+0000 front 2024-09-15T23:13:30.827561+0000 (oldest deadline 2024-09-15T23:13:54.327732+0000) 2024-09-15T23:13:55.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:54 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:54.806+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:13:55.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:54 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:54.806+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:25.298691+0000 front 2024-09-15T23:13:25.298714+0000 (oldest deadline 2024-09-15T23:13:51.198828+0000) 2024-09-15T23:13:55.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:55 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:55.024+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:13:55.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:55 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:55.024+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:13:55.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:55 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:55.024+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:13:55.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:54 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:54.673+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.914943+0000 front 2024-09-15T23:13:15.914917+0000 (oldest deadline 2024-09-15T23:13:39.415083+0000) 2024-09-15T23:13:55.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:54 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:54.673+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:24.716108+0000 front 2024-09-15T23:13:24.716135+0000 (oldest deadline 2024-09-15T23:13:47.016177+0000) 2024-09-15T23:13:55.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:55 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:55.258+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:55.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:55 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:55.258+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:13:55.641 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:55 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:55.258+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:13:55.641 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:55 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:55.258+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:13:55.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:55 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:55.297+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:55.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:55 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:55.297+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:13:55.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:55 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:55.297+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:13:55.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:55 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:55.297+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.081640+0000 front 2024-09-15T23:13:31.081667+0000 (oldest deadline 2024-09-15T23:13:55.082194+0000) 2024-09-15T23:13:56.064 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:55 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:55.762+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:13:56.064 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:55 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:55.762+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:25.298691+0000 front 2024-09-15T23:13:25.298714+0000 (oldest deadline 2024-09-15T23:13:51.198828+0000) 2024-09-15T23:13:56.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:55 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:55.673+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.914943+0000 front 2024-09-15T23:13:15.914917+0000 (oldest deadline 2024-09-15T23:13:39.415083+0000) 2024-09-15T23:13:56.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:55 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:55.673+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:24.716108+0000 front 2024-09-15T23:13:24.716135+0000 (oldest deadline 2024-09-15T23:13:47.016177+0000) 2024-09-15T23:13:56.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:55 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:55.673+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:29.817938+0000 front 2024-09-15T23:13:29.818017+0000 (oldest deadline 2024-09-15T23:13:55.117955+0000) 2024-09-15T23:13:56.165 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:55 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:55.677+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:56.166 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:55 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:55.677+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:14.024590+0000 front 2024-09-15T23:13:14.024498+0000 (oldest deadline 2024-09-15T23:13:39.324528+0000) 2024-09-15T23:13:56.166 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:55 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:55.677+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:24.526557+0000 front 2024-09-15T23:13:24.526379+0000 (oldest deadline 2024-09-15T23:13:50.326723+0000) 2024-09-15T23:13:56.166 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:55 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:55.677+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:30.827625+0000 front 2024-09-15T23:13:30.827561+0000 (oldest deadline 2024-09-15T23:13:54.327732+0000) 2024-09-15T23:13:56.564 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:56 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:56.064+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:13:56.564 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:56 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:56.064+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:13:56.565 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:56 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:56.064+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:13:56.625 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:56 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:56.220+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:56.625 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:56 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:56.220+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:13:56.625 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:56 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:56.220+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:13:56.625 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:56 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:56.220+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:13:56.660 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:56 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:56.252+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:56.660 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:56 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:56.252+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:13:56.660 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:56 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:56.252+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:13:56.660 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:56 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:56.252+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.081640+0000 front 2024-09-15T23:13:31.081667+0000 (oldest deadline 2024-09-15T23:13:55.082194+0000) 2024-09-15T23:13:56.890 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:56 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:56.623+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.914943+0000 front 2024-09-15T23:13:15.914917+0000 (oldest deadline 2024-09-15T23:13:39.415083+0000) 2024-09-15T23:13:56.890 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:56 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:56.623+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:24.716108+0000 front 2024-09-15T23:13:24.716135+0000 (oldest deadline 2024-09-15T23:13:47.016177+0000) 2024-09-15T23:13:56.890 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:56 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:56.623+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:29.817938+0000 front 2024-09-15T23:13:29.818017+0000 (oldest deadline 2024-09-15T23:13:55.117955+0000) 2024-09-15T23:13:56.915 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:56 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:56.659+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:56.915 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:56 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:56.659+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:14.024590+0000 front 2024-09-15T23:13:14.024498+0000 (oldest deadline 2024-09-15T23:13:39.324528+0000) 2024-09-15T23:13:56.915 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:56 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:56.659+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:24.526557+0000 front 2024-09-15T23:13:24.526379+0000 (oldest deadline 2024-09-15T23:13:50.326723+0000) 2024-09-15T23:13:56.915 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:56 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:56.659+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:30.827625+0000 front 2024-09-15T23:13:30.827561+0000 (oldest deadline 2024-09-15T23:13:54.327732+0000) 2024-09-15T23:13:57.064 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:56 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:56.720+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:13:57.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:56 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:56.720+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:25.298691+0000 front 2024-09-15T23:13:25.298714+0000 (oldest deadline 2024-09-15T23:13:51.198828+0000) 2024-09-15T23:13:57.564 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:57 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:57.064+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:13:57.564 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:57 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:57.064+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:13:57.565 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:57 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:57.064+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:13:57.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:57 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:57.267+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:57.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:57 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:57.267+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:13:57.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:57 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:57.267+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:13:57.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:57 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:57.267+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:13:57.657 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:57 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:57.297+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:57.657 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:57 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:57.297+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:13:57.657 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:57 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:57.297+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:13:57.657 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:57 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:57.297+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.081640+0000 front 2024-09-15T23:13:31.081667+0000 (oldest deadline 2024-09-15T23:13:55.082194+0000) 2024-09-15T23:13:57.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:57 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:57.656+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.622457+0000 front 2024-09-15T23:13:07.622568+0000 (oldest deadline 2024-09-15T23:13:29.922599+0000) 2024-09-15T23:13:57.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:57 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:57.656+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:14.024590+0000 front 2024-09-15T23:13:14.024498+0000 (oldest deadline 2024-09-15T23:13:39.324528+0000) 2024-09-15T23:13:57.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:57 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:57.656+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:24.526557+0000 front 2024-09-15T23:13:24.526379+0000 (oldest deadline 2024-09-15T23:13:50.326723+0000) 2024-09-15T23:13:57.916 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:57 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6[57913]: 2024-09-15T23:13:57.656+0000 7fb7ac400640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:30.827625+0000 front 2024-09-15T23:13:30.827561+0000 (oldest deadline 2024-09-15T23:13:54.327732+0000) 2024-09-15T23:13:58.063 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:57 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:57.758+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:13:58.063 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:57 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:57.758+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:25.298691+0000 front 2024-09-15T23:13:25.298714+0000 (oldest deadline 2024-09-15T23:13:51.198828+0000) 2024-09-15T23:13:58.063 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:57 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:57.758+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.199331+0000 front 2024-09-15T23:13:31.199464+0000 (oldest deadline 2024-09-15T23:13:57.099498+0000) 2024-09-15T23:13:58.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:57 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:57.653+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.914943+0000 front 2024-09-15T23:13:15.914917+0000 (oldest deadline 2024-09-15T23:13:39.415083+0000) 2024-09-15T23:13:58.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:57 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:57.653+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:24.716108+0000 front 2024-09-15T23:13:24.716135+0000 (oldest deadline 2024-09-15T23:13:47.016177+0000) 2024-09-15T23:13:58.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:57 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:57.653+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:29.817938+0000 front 2024-09-15T23:13:29.818017+0000 (oldest deadline 2024-09-15T23:13:55.117955+0000) 2024-09-15T23:13:58.314 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:58 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:58.061+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:13:58.314 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:58 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:58.061+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:13:58.314 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:58 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:58.061+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:13:58.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:58 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:58.284+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:58.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:58 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:58.284+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:13:58.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:58 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:58.284+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:13:58.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:58 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:58.284+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:13:58.665 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:58 smithi111 podman[96816]: 2024-09-15 23:13:58.490588097 +0000 UTC m=+5.774682548 container died c6b5a294c7cb1a0ca9e8ef86df03f88e10405503e95d4dff0c7aa8059933ddbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-15T23:13:58.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:58 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:58.310+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:58.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:58 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:58.310+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:13:58.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:58 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:58.310+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:13:58.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:58 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:58.310+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.081640+0000 front 2024-09-15T23:13:31.081667+0000 (oldest deadline 2024-09-15T23:13:55.082194+0000) 2024-09-15T23:13:59.064 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:58 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:58.774+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:13:59.064 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:58 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:58.774+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:25.298691+0000 front 2024-09-15T23:13:25.298714+0000 (oldest deadline 2024-09-15T23:13:51.198828+0000) 2024-09-15T23:13:59.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:58 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:58.774+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.199331+0000 front 2024-09-15T23:13:31.199464+0000 (oldest deadline 2024-09-15T23:13:57.099498+0000) 2024-09-15T23:13:59.134 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:58 smithi111 podman[96816]: 2024-09-15 23:13:58.890591677 +0000 UTC m=+6.174686118 container cleanup c6b5a294c7cb1a0ca9e8ef86df03f88e10405503e95d4dff0c7aa8059933ddbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-15T23:13:59.134 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:58 smithi111 bash[96816]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6 2024-09-15T23:13:59.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:58 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:58.663+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.914943+0000 front 2024-09-15T23:13:15.914917+0000 (oldest deadline 2024-09-15T23:13:39.415083+0000) 2024-09-15T23:13:59.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:58 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:58.663+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:24.716108+0000 front 2024-09-15T23:13:24.716135+0000 (oldest deadline 2024-09-15T23:13:47.016177+0000) 2024-09-15T23:13:59.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:58 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:58.663+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:29.817938+0000 front 2024-09-15T23:13:29.818017+0000 (oldest deadline 2024-09-15T23:13:55.117955+0000) 2024-09-15T23:13:59.415 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:59 smithi111 podman[96834]: 2024-09-15 23:13:59.133434029 +0000 UTC m=+0.635598975 container remove c6b5a294c7cb1a0ca9e8ef86df03f88e10405503e95d4dff0c7aa8059933ddbf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-15T23:13:59.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:59 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:59.330+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:13:59.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:59 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:59.330+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:13:59.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:59 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:59.330+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:13:59.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:13:59 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:13:59.330+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.081640+0000 front 2024-09-15T23:13:31.081667+0000 (oldest deadline 2024-09-15T23:13:55.082194+0000) 2024-09-15T23:13:59.564 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:59 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:59.108+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:13:59.564 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:59 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:59.108+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:13:59.565 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:13:59 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:13:59.108+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:13:59.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:59 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:59.334+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:13:59.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:59 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:59.334+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:13:59.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:59 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:59.334+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:13:59.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:13:59 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:13:59.334+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:13:59.896 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:59 smithi111 podman[96924]: 2024-09-15 23:13:59.468545476 +0000 UTC m=+0.027015662 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T23:13:59.896 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:59 smithi111 podman[96924]: 2024-09-15 23:13:59.576482522 +0000 UTC m=+0.134952741 container create 0b77124c24afb947d8f16c5beca773606bc0165987455a6d46fdae341bae4a88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-15T23:13:59.897 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:59 smithi111 podman[96924]: 2024-09-15 23:13:59.786872043 +0000 UTC m=+0.345342240 container init 0b77124c24afb947d8f16c5beca773606bc0165987455a6d46fdae341bae4a88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, ceph=True) 2024-09-15T23:13:59.897 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:59 smithi111 podman[96924]: 2024-09-15 23:13:59.793571254 +0000 UTC m=+0.352041430 container start 0b77124c24afb947d8f16c5beca773606bc0165987455a6d46fdae341bae4a88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-15T23:13:59.897 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:13:59 smithi111 podman[96924]: 2024-09-15 23:13:59.895249481 +0000 UTC m=+0.453719683 container attach 0b77124c24afb947d8f16c5beca773606bc0165987455a6d46fdae341bae4a88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-09-15T23:14:00.064 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:59 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:59.763+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:14:00.064 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:59 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:59.763+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:25.298691+0000 front 2024-09-15T23:13:25.298714+0000 (oldest deadline 2024-09-15T23:13:51.198828+0000) 2024-09-15T23:14:00.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:13:59 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:13:59.763+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.199331+0000 front 2024-09-15T23:13:31.199464+0000 (oldest deadline 2024-09-15T23:13:57.099498+0000) 2024-09-15T23:14:00.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:59 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:59.713+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.914943+0000 front 2024-09-15T23:13:15.914917+0000 (oldest deadline 2024-09-15T23:13:39.415083+0000) 2024-09-15T23:14:00.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:59 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:59.713+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:24.716108+0000 front 2024-09-15T23:13:24.716135+0000 (oldest deadline 2024-09-15T23:13:47.016177+0000) 2024-09-15T23:14:00.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:13:59 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:13:59.713+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:29.817938+0000 front 2024-09-15T23:13:29.818017+0000 (oldest deadline 2024-09-15T23:13:55.117955+0000) 2024-09-15T23:14:00.165 INFO:journalctl@ceph.osd.6.smithi111.stdout:Sep 15 23:14:00 smithi111 podman[96946]: 2024-09-15 23:14:00.095937338 +0000 UTC m=+0.029899901 container died 0b77124c24afb947d8f16c5beca773606bc0165987455a6d46fdae341bae4a88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-6-deactivate, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-15T23:14:00.422 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:00 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:00.284+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:14:00.422 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:00 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:00.284+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:14:00.422 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:00 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:00.284+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:14:00.422 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:00 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:00.284+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.081640+0000 front 2024-09-15T23:13:31.081667+0000 (oldest deadline 2024-09-15T23:13:55.082194+0000) 2024-09-15T23:14:00.564 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:00 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:00.068+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:00.564 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:00 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:00.068+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:00.564 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:00 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:00.068+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:00.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:00 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:00.343+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:00.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:00 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:00.343+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:00.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:00 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:00.343+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:00.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:00 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:00.343+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:00.757 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.6.service' 2024-09-15T23:14:00.834 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:14:00.835 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-15T23:14:00.835 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-15T23:14:00.835 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.7 2024-09-15T23:14:01.064 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:00 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:00.735+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:14:01.064 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:00 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:00.735+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:25.298691+0000 front 2024-09-15T23:13:25.298714+0000 (oldest deadline 2024-09-15T23:13:51.198828+0000) 2024-09-15T23:14:01.064 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:00 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:00.735+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.199331+0000 front 2024-09-15T23:13:31.199464+0000 (oldest deadline 2024-09-15T23:13:57.099498+0000) 2024-09-15T23:14:01.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:00 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:14:00.682+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.914943+0000 front 2024-09-15T23:13:15.914917+0000 (oldest deadline 2024-09-15T23:13:39.415083+0000) 2024-09-15T23:14:01.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:00 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:14:00.682+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:24.716108+0000 front 2024-09-15T23:13:24.716135+0000 (oldest deadline 2024-09-15T23:13:47.016177+0000) 2024-09-15T23:14:01.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:00 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:14:00.682+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:29.817938+0000 front 2024-09-15T23:13:29.818017+0000 (oldest deadline 2024-09-15T23:13:55.117955+0000) 2024-09-15T23:14:01.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:00 smithi121 systemd[1]: Stopping Ceph osd.7 for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:14:01.530 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:01 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:01.253+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:14:01.530 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:01 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:01.253+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:14:01.530 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:01 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:01.253+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:14:01.530 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:01 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:01.253+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.081640+0000 front 2024-09-15T23:13:31.081667+0000 (oldest deadline 2024-09-15T23:13:55.082194+0000) 2024-09-15T23:14:01.564 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:01 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:01.097+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:01.564 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:01 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:01.097+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:01.565 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:01 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:01.097+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:01.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:01 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:01.336+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:01.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:01 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:01.336+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:01.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:01 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:01.336+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:01.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:01 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:01.336+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:01.640 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:01 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:14:01.139+0000 7f6ab0c94640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:14:01.640 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:01 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:14:01.139+0000 7f6ab0c94640 -1 osd.7 92 *** Got signal Terminated *** 2024-09-15T23:14:01.641 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:01 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:14:01.139+0000 7f6ab0c94640 -1 osd.7 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T23:14:02.056 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:01 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:01.769+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:14:02.057 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:01 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:01.769+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:25.298691+0000 front 2024-09-15T23:13:25.298714+0000 (oldest deadline 2024-09-15T23:13:51.198828+0000) 2024-09-15T23:14:02.057 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:01 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:01.769+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.199331+0000 front 2024-09-15T23:13:31.199464+0000 (oldest deadline 2024-09-15T23:13:57.099498+0000) 2024-09-15T23:14:02.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:01 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:14:01.711+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.914943+0000 front 2024-09-15T23:13:15.914917+0000 (oldest deadline 2024-09-15T23:13:39.415083+0000) 2024-09-15T23:14:02.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:01 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:14:01.711+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:24.716108+0000 front 2024-09-15T23:13:24.716135+0000 (oldest deadline 2024-09-15T23:13:47.016177+0000) 2024-09-15T23:14:02.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:01 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:14:01.711+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:29.817938+0000 front 2024-09-15T23:13:29.818017+0000 (oldest deadline 2024-09-15T23:13:55.117955+0000) 2024-09-15T23:14:02.314 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:02 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:02.055+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:02.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:02 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:02.055+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:02.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:02 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:02.055+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:02.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:02 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:02.360+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:02.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:02 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:02.360+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:02.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:02 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:02.360+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:02.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:02 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:02.360+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:02.665 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:02 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:02.209+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:14:02.665 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:02 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:02.209+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:14:02.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:02 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:02.209+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:14:02.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:02 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:02.209+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.081640+0000 front 2024-09-15T23:13:31.081667+0000 (oldest deadline 2024-09-15T23:13:55.082194+0000) 2024-09-15T23:14:03.047 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:02 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:02.773+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:14:03.047 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:02 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:02.773+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:25.298691+0000 front 2024-09-15T23:13:25.298714+0000 (oldest deadline 2024-09-15T23:13:51.198828+0000) 2024-09-15T23:14:03.047 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:02 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:02.773+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.199331+0000 front 2024-09-15T23:13:31.199464+0000 (oldest deadline 2024-09-15T23:13:57.099498+0000) 2024-09-15T23:14:03.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:02 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:14:02.740+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.914943+0000 front 2024-09-15T23:13:15.914917+0000 (oldest deadline 2024-09-15T23:13:39.415083+0000) 2024-09-15T23:14:03.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:02 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:14:02.740+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:24.716108+0000 front 2024-09-15T23:13:24.716135+0000 (oldest deadline 2024-09-15T23:13:47.016177+0000) 2024-09-15T23:14:03.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:02 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:14:02.740+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:29.817938+0000 front 2024-09-15T23:13:29.818017+0000 (oldest deadline 2024-09-15T23:13:55.117955+0000) 2024-09-15T23:14:03.314 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:03 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:03.046+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:03.314 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:03 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:03.046+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:03.314 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:03 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:03.046+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:03.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:03 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:03.343+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:03.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:03 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:03.343+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:03.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:03 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:03.343+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:03.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:03 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:03.343+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:03.665 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:03 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:03.187+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:14:03.665 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:03 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:03.187+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:14:03.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:03 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:03.187+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:14:03.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:03 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:03.187+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.081640+0000 front 2024-09-15T23:13:31.081667+0000 (oldest deadline 2024-09-15T23:13:55.082194+0000) 2024-09-15T23:14:04.009 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:03 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:03.745+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:14:04.009 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:03 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:03.745+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:25.298691+0000 front 2024-09-15T23:13:25.298714+0000 (oldest deadline 2024-09-15T23:13:51.198828+0000) 2024-09-15T23:14:04.010 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:03 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:03.745+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.199331+0000 front 2024-09-15T23:13:31.199464+0000 (oldest deadline 2024-09-15T23:13:57.099498+0000) 2024-09-15T23:14:04.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:03 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:14:03.725+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.914943+0000 front 2024-09-15T23:13:15.914917+0000 (oldest deadline 2024-09-15T23:13:39.415083+0000) 2024-09-15T23:14:04.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:03 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:14:03.725+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:24.716108+0000 front 2024-09-15T23:13:24.716135+0000 (oldest deadline 2024-09-15T23:13:47.016177+0000) 2024-09-15T23:14:04.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:03 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:14:03.725+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:29.817938+0000 front 2024-09-15T23:13:29.818017+0000 (oldest deadline 2024-09-15T23:13:55.117955+0000) 2024-09-15T23:14:04.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:03 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:14:03.725+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:37.919155+0000 front 2024-09-15T23:13:37.919203+0000 (oldest deadline 2024-09-15T23:14:03.219237+0000) 2024-09-15T23:14:04.314 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:04 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:04.008+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:04.314 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:04 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:04.008+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:04.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:04 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:04.008+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:04.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:04 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:04.371+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:04.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:04 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:04.371+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:04.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:04 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:04.371+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:04.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:04 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:04.371+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:04.640 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:04 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:04.371+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:39.308188+0000 front 2024-09-15T23:13:39.308221+0000 (oldest deadline 2024-09-15T23:14:03.408385+0000) 2024-09-15T23:14:04.665 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:04 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:04.170+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:14:04.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:04 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:04.170+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:14:04.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:04 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:04.170+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:14:04.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:04 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:04.170+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.081640+0000 front 2024-09-15T23:13:31.081667+0000 (oldest deadline 2024-09-15T23:13:55.082194+0000) 2024-09-15T23:14:05.002 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:04 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:04.723+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:14:05.002 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:04 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:04.723+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:25.298691+0000 front 2024-09-15T23:13:25.298714+0000 (oldest deadline 2024-09-15T23:13:51.198828+0000) 2024-09-15T23:14:05.002 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:04 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:04.723+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.199331+0000 front 2024-09-15T23:13:31.199464+0000 (oldest deadline 2024-09-15T23:13:57.099498+0000) 2024-09-15T23:14:05.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:04 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:14:04.774+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.914943+0000 front 2024-09-15T23:13:15.914917+0000 (oldest deadline 2024-09-15T23:13:39.415083+0000) 2024-09-15T23:14:05.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:04 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:14:04.774+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:24.716108+0000 front 2024-09-15T23:13:24.716135+0000 (oldest deadline 2024-09-15T23:13:47.016177+0000) 2024-09-15T23:14:05.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:04 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:14:04.774+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:29.817938+0000 front 2024-09-15T23:13:29.818017+0000 (oldest deadline 2024-09-15T23:13:55.117955+0000) 2024-09-15T23:14:05.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:04 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:14:04.774+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:37.919155+0000 front 2024-09-15T23:13:37.919203+0000 (oldest deadline 2024-09-15T23:14:03.219237+0000) 2024-09-15T23:14:05.314 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:05 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:05.000+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:05.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:05 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:05.000+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:05.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:05 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:05.000+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:05.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:05 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:05.000+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:05.415 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:05 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:05.160+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:14:05.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:05 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:05.160+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:14:05.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:05 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:05.160+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:14:05.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:05 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:05.160+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.081640+0000 front 2024-09-15T23:13:31.081667+0000 (oldest deadline 2024-09-15T23:13:55.082194+0000) 2024-09-15T23:14:05.809 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:05 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:05.396+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:05.809 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:05 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:05.396+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:05.809 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:05 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:05.396+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:05.809 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:05 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:05.396+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:05.809 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:05 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:05.396+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:39.308188+0000 front 2024-09-15T23:13:39.308221+0000 (oldest deadline 2024-09-15T23:14:03.408385+0000) 2024-09-15T23:14:06.064 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:05 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:05.747+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:14:06.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:05 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:05.747+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:25.298691+0000 front 2024-09-15T23:13:25.298714+0000 (oldest deadline 2024-09-15T23:13:51.198828+0000) 2024-09-15T23:14:06.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:05 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:05.747+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.199331+0000 front 2024-09-15T23:13:31.199464+0000 (oldest deadline 2024-09-15T23:13:57.099498+0000) 2024-09-15T23:14:06.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:05 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:05.962+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:06.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:05 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:05.962+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:06.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:05 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:05.962+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:06.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:05 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:05.962+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:06.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:05 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:14:05.807+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.914943+0000 front 2024-09-15T23:13:15.914917+0000 (oldest deadline 2024-09-15T23:13:39.415083+0000) 2024-09-15T23:14:06.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:05 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:14:05.807+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:24.716108+0000 front 2024-09-15T23:13:24.716135+0000 (oldest deadline 2024-09-15T23:13:47.016177+0000) 2024-09-15T23:14:06.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:05 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:14:05.807+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:29.817938+0000 front 2024-09-15T23:13:29.818017+0000 (oldest deadline 2024-09-15T23:13:55.117955+0000) 2024-09-15T23:14:06.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:05 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7[53240]: 2024-09-15T23:14:05.807+0000 7f6aad299640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:37.919155+0000 front 2024-09-15T23:13:37.919203+0000 (oldest deadline 2024-09-15T23:14:03.219237+0000) 2024-09-15T23:14:06.665 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:06 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:06.182+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:14:06.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:06 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:06.182+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:14:06.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:06 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:06.182+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:14:06.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:06 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:06.182+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.081640+0000 front 2024-09-15T23:13:31.081667+0000 (oldest deadline 2024-09-15T23:13:55.082194+0000) 2024-09-15T23:14:06.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:06 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:06.182+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.983723+0000 front 2024-09-15T23:13:40.983756+0000 (oldest deadline 2024-09-15T23:14:05.683421+0000) 2024-09-15T23:14:06.849 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:06 smithi121 podman[76723]: 2024-09-15 23:14:06.577474397 +0000 UTC m=+5.547953752 container died 2deb1842a0f832858c6656b7f7e4522b2e41a16dcbeb6f900a2fed1670da63ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T23:14:06.849 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:06 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:06.398+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:06.849 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:06 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:06.398+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:06.849 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:06 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:06.398+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:06.850 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:06 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:06.398+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:06.850 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:06 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:06.398+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:39.308188+0000 front 2024-09-15T23:13:39.308221+0000 (oldest deadline 2024-09-15T23:14:03.408385+0000) 2024-09-15T23:14:07.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:06 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:06.765+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:14:07.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:06 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:06.765+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:25.298691+0000 front 2024-09-15T23:13:25.298714+0000 (oldest deadline 2024-09-15T23:13:51.198828+0000) 2024-09-15T23:14:07.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:06 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:06.765+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.199331+0000 front 2024-09-15T23:13:31.199464+0000 (oldest deadline 2024-09-15T23:13:57.099498+0000) 2024-09-15T23:14:07.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:06 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:06.765+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:37.099982+0000 front 2024-09-15T23:13:37.100084+0000 (oldest deadline 2024-09-15T23:14:06.500379+0000) 2024-09-15T23:14:07.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:06 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:06.947+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:07.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:06 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:06.947+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:07.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:06 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:06.947+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:07.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:06 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:06.947+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:07.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:06 smithi121 podman[76723]: 2024-09-15 23:14:06.947793861 +0000 UTC m=+5.918273207 container cleanup 2deb1842a0f832858c6656b7f7e4522b2e41a16dcbeb6f900a2fed1670da63ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, RELEASE=squid-8921c8e, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-15T23:14:07.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:06 smithi121 bash[76723]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7 2024-09-15T23:14:07.532 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:07 smithi121 podman[76738]: 2024-09-15 23:14:07.224050173 +0000 UTC m=+0.642969692 container remove 2deb1842a0f832858c6656b7f7e4522b2e41a16dcbeb6f900a2fed1670da63ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T23:14:07.533 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:07 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:07.410+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:07.533 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:07 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:07.410+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:07.533 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:07 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:07.410+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:07.533 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:07 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:07.410+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:07.533 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:07 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:07.410+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:39.308188+0000 front 2024-09-15T23:13:39.308221+0000 (oldest deadline 2024-09-15T23:14:03.408385+0000) 2024-09-15T23:14:07.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:07 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:07.185+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:14:07.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:07 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:07.185+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:14:07.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:07 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:07.185+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:14:07.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:07 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:07.185+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.081640+0000 front 2024-09-15T23:13:31.081667+0000 (oldest deadline 2024-09-15T23:13:55.082194+0000) 2024-09-15T23:14:07.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:07 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:07.185+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.983723+0000 front 2024-09-15T23:13:40.983756+0000 (oldest deadline 2024-09-15T23:14:05.683421+0000) 2024-09-15T23:14:07.786 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:07 smithi121 podman[76829]: 2024-09-15 23:14:07.434766387 +0000 UTC m=+0.016101028 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T23:14:07.786 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:07 smithi121 podman[76829]: 2024-09-15 23:14:07.532961718 +0000 UTC m=+0.114296343 container create 648cb59e12da8290fb498ad11c2a2823014a56f52347d520b7980fb76951bba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T23:14:07.786 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:07 smithi121 podman[76829]: 2024-09-15 23:14:07.785627871 +0000 UTC m=+0.366962506 container init 648cb59e12da8290fb498ad11c2a2823014a56f52347d520b7980fb76951bba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7-deactivate, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-15T23:14:08.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:07 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:07.779+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:14:08.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:07 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:07.779+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:25.298691+0000 front 2024-09-15T23:13:25.298714+0000 (oldest deadline 2024-09-15T23:13:51.198828+0000) 2024-09-15T23:14:08.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:07 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:07.779+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.199331+0000 front 2024-09-15T23:13:31.199464+0000 (oldest deadline 2024-09-15T23:13:57.099498+0000) 2024-09-15T23:14:08.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:07 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:07.779+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:37.099982+0000 front 2024-09-15T23:13:37.100084+0000 (oldest deadline 2024-09-15T23:14:06.500379+0000) 2024-09-15T23:14:08.067 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:07 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:07.928+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:08.068 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:07 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:07.928+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:08.068 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:07 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:07.928+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:08.068 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:07 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:07.928+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:08.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:07 smithi121 podman[76829]: 2024-09-15 23:14:07.788916693 +0000 UTC m=+0.370251331 container start 648cb59e12da8290fb498ad11c2a2823014a56f52347d520b7980fb76951bba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True) 2024-09-15T23:14:08.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:07 smithi121 podman[76829]: 2024-09-15 23:14:07.877157757 +0000 UTC m=+0.458492401 container attach 648cb59e12da8290fb498ad11c2a2823014a56f52347d520b7980fb76951bba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T23:14:08.140 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 15 23:14:07 smithi121 podman[76829]: 2024-09-15 23:14:07.984612558 +0000 UTC m=+0.565947191 container died 648cb59e12da8290fb498ad11c2a2823014a56f52347d520b7980fb76951bba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-15T23:14:08.415 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:08 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:08.155+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:14:08.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:08 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:08.155+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:14:08.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:08 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:08.155+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:14:08.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:08 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:08.155+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.081640+0000 front 2024-09-15T23:13:31.081667+0000 (oldest deadline 2024-09-15T23:13:55.082194+0000) 2024-09-15T23:14:08.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:08 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:08.155+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.983723+0000 front 2024-09-15T23:13:40.983756+0000 (oldest deadline 2024-09-15T23:14:05.683421+0000) 2024-09-15T23:14:08.531 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:08 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:08.394+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:08.531 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:08 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:08.394+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:08.531 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:08 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:08.394+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:08.531 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:08 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:08.394+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:08.531 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:08 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:08.394+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:39.308188+0000 front 2024-09-15T23:13:39.308221+0000 (oldest deadline 2024-09-15T23:14:03.408385+0000) 2024-09-15T23:14:08.549 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.7.service' 2024-09-15T23:14:08.587 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:14:08.587 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-15T23:14:08.587 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-15T23:14:08.587 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.8 2024-09-15T23:14:08.886 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:08 smithi192 systemd[1]: Stopping Ceph osd.8 for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:14:08.886 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:08 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:08.736+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:14:08.886 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:08 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:08.736+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:25.298691+0000 front 2024-09-15T23:13:25.298714+0000 (oldest deadline 2024-09-15T23:13:51.198828+0000) 2024-09-15T23:14:08.886 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:08 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:08.736+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.199331+0000 front 2024-09-15T23:13:31.199464+0000 (oldest deadline 2024-09-15T23:13:57.099498+0000) 2024-09-15T23:14:08.886 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:08 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:08.736+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:37.099982+0000 front 2024-09-15T23:13:37.100084+0000 (oldest deadline 2024-09-15T23:14:06.500379+0000) 2024-09-15T23:14:09.314 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:08 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:08.884+0000 7f9674e5d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:14:09.314 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:08 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:08.884+0000 7f9674e5d640 -1 osd.8 92 *** Got signal Terminated *** 2024-09-15T23:14:09.315 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:08 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:08.884+0000 7f9674e5d640 -1 osd.8 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T23:14:09.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:08 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:08.976+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:09.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:08 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:08.976+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:09.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:08 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:08.976+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:09.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:08 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:08.976+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:09.415 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:09 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:09.136+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:14:09.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:09 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:09.136+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:14:09.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:09 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:09.136+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:14:09.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:09 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:09.136+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.081640+0000 front 2024-09-15T23:13:31.081667+0000 (oldest deadline 2024-09-15T23:13:55.082194+0000) 2024-09-15T23:14:09.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:09 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:09.136+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.983723+0000 front 2024-09-15T23:13:40.983756+0000 (oldest deadline 2024-09-15T23:14:05.683421+0000) 2024-09-15T23:14:09.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:09 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:09.438+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:09.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:09 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:09.438+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:09.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:09 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:09.438+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:09.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:09 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:09.438+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:09.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:09 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:09.438+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:39.308188+0000 front 2024-09-15T23:13:39.308221+0000 (oldest deadline 2024-09-15T23:14:03.408385+0000) 2024-09-15T23:14:10.064 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:09 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:09.716+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:14:10.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:09 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:09.716+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:25.298691+0000 front 2024-09-15T23:13:25.298714+0000 (oldest deadline 2024-09-15T23:13:51.198828+0000) 2024-09-15T23:14:10.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:09 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:09.716+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.199331+0000 front 2024-09-15T23:13:31.199464+0000 (oldest deadline 2024-09-15T23:13:57.099498+0000) 2024-09-15T23:14:10.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:09 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:09.716+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:37.099982+0000 front 2024-09-15T23:13:37.100084+0000 (oldest deadline 2024-09-15T23:14:06.500379+0000) 2024-09-15T23:14:10.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:09 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:09.927+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:10.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:09 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:09.927+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:10.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:09 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:09.927+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:10.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:09 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:09.927+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:10.415 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:10 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:10.158+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:14:10.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:10 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:10.158+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:14:10.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:10 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:10.158+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:14:10.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:10 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:10.158+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.081640+0000 front 2024-09-15T23:13:31.081667+0000 (oldest deadline 2024-09-15T23:13:55.082194+0000) 2024-09-15T23:14:10.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:10 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:10.158+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.983723+0000 front 2024-09-15T23:13:40.983756+0000 (oldest deadline 2024-09-15T23:14:05.683421+0000) 2024-09-15T23:14:10.791 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:10 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:10.430+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:10.791 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:10 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:10.430+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:10.791 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:10 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:10.430+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:10.792 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:10 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:10.430+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:10.792 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:10 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:10.430+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:39.308188+0000 front 2024-09-15T23:13:39.308221+0000 (oldest deadline 2024-09-15T23:14:03.408385+0000) 2024-09-15T23:14:11.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:10 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:10.727+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:14:11.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:10 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:10.727+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:25.298691+0000 front 2024-09-15T23:13:25.298714+0000 (oldest deadline 2024-09-15T23:13:51.198828+0000) 2024-09-15T23:14:11.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:10 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:10.727+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.199331+0000 front 2024-09-15T23:13:31.199464+0000 (oldest deadline 2024-09-15T23:13:57.099498+0000) 2024-09-15T23:14:11.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:10 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:10.727+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:37.099982+0000 front 2024-09-15T23:13:37.100084+0000 (oldest deadline 2024-09-15T23:14:06.500379+0000) 2024-09-15T23:14:11.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:10 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:10.945+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:11.066 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:10 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:10.945+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:11.066 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:10 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:10.945+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:11.066 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:10 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:10.945+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:11.066 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:10 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:10.945+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:11.415 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:11 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:11.120+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:14:11.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:11 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:11.120+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:14:11.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:11 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:11.120+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:14:11.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:11 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:11.120+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.081640+0000 front 2024-09-15T23:13:31.081667+0000 (oldest deadline 2024-09-15T23:13:55.082194+0000) 2024-09-15T23:14:11.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:11 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:11.120+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.983723+0000 front 2024-09-15T23:13:40.983756+0000 (oldest deadline 2024-09-15T23:14:05.683421+0000) 2024-09-15T23:14:11.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:11 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:11.120+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:45.683935+0000 front 2024-09-15T23:13:45.683995+0000 (oldest deadline 2024-09-15T23:14:10.383882+0000) 2024-09-15T23:14:11.807 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:11 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:11.428+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:11.807 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:11 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:11.428+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:11.807 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:11 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:11.428+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:11.807 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:11 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:11.428+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:11.807 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:11 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:11.428+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:39.308188+0000 front 2024-09-15T23:13:39.308221+0000 (oldest deadline 2024-09-15T23:14:03.408385+0000) 2024-09-15T23:14:12.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:11 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:11.753+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:14:12.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:11 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:11.753+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:25.298691+0000 front 2024-09-15T23:13:25.298714+0000 (oldest deadline 2024-09-15T23:13:51.198828+0000) 2024-09-15T23:14:12.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:11 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:11.753+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.199331+0000 front 2024-09-15T23:13:31.199464+0000 (oldest deadline 2024-09-15T23:13:57.099498+0000) 2024-09-15T23:14:12.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:11 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:11.753+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:37.099982+0000 front 2024-09-15T23:13:37.100084+0000 (oldest deadline 2024-09-15T23:14:06.500379+0000) 2024-09-15T23:14:12.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:11 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:11.914+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:12.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:11 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:11.914+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:12.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:11 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:11.914+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:12.066 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:11 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:11.914+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:12.066 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:11 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:11.914+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:12.415 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:12 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:12.099+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:14:12.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:12 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:12.099+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:14:12.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:12 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:12.099+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:14:12.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:12 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:12.099+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.081640+0000 front 2024-09-15T23:13:31.081667+0000 (oldest deadline 2024-09-15T23:13:55.082194+0000) 2024-09-15T23:14:12.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:12 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:12.099+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.983723+0000 front 2024-09-15T23:13:40.983756+0000 (oldest deadline 2024-09-15T23:14:05.683421+0000) 2024-09-15T23:14:12.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:12 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:12.099+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:45.683935+0000 front 2024-09-15T23:13:45.683995+0000 (oldest deadline 2024-09-15T23:14:10.383882+0000) 2024-09-15T23:14:12.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:12 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:12.438+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:12.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:12 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:12.438+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:12.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:12 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:12.438+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:12.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:12 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:12.438+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:12.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:12 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:12.438+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:39.308188+0000 front 2024-09-15T23:13:39.308221+0000 (oldest deadline 2024-09-15T23:14:03.408385+0000) 2024-09-15T23:14:13.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:12 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:12.755+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:14:13.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:12 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:12.755+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:25.298691+0000 front 2024-09-15T23:13:25.298714+0000 (oldest deadline 2024-09-15T23:13:51.198828+0000) 2024-09-15T23:14:13.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:12 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:12.755+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.199331+0000 front 2024-09-15T23:13:31.199464+0000 (oldest deadline 2024-09-15T23:13:57.099498+0000) 2024-09-15T23:14:13.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:12 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:12.755+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:37.099982+0000 front 2024-09-15T23:13:37.100084+0000 (oldest deadline 2024-09-15T23:14:06.500379+0000) 2024-09-15T23:14:13.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:12 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:12.896+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:13.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:12 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:12.896+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:13.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:12 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:12.896+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:13.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:12 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:12.896+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:13.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:12 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:12.896+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:13.165 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:13 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:13.143+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:14:13.166 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:13 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:13.143+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:14:13.166 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:13 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:13.143+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:14:13.166 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:13 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:13.143+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.081640+0000 front 2024-09-15T23:13:31.081667+0000 (oldest deadline 2024-09-15T23:13:55.082194+0000) 2024-09-15T23:14:13.166 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:13 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:13.143+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.983723+0000 front 2024-09-15T23:13:40.983756+0000 (oldest deadline 2024-09-15T23:14:05.683421+0000) 2024-09-15T23:14:13.166 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:13 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:13.143+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:45.683935+0000 front 2024-09-15T23:13:45.683995+0000 (oldest deadline 2024-09-15T23:14:10.383882+0000) 2024-09-15T23:14:13.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:13 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:13.482+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:13.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:13 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:13.482+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:13.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:13 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:13.482+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:13.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:13 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:13.482+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:13.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:13 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:13.482+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:39.308188+0000 front 2024-09-15T23:13:39.308221+0000 (oldest deadline 2024-09-15T23:14:03.408385+0000) 2024-09-15T23:14:14.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:13 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:13.764+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:17.196849+0000 front 2024-09-15T23:13:17.196818+0000 (oldest deadline 2024-09-15T23:13:39.496756+0000) 2024-09-15T23:14:14.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:13 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:13.764+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:25.298691+0000 front 2024-09-15T23:13:25.298714+0000 (oldest deadline 2024-09-15T23:13:51.198828+0000) 2024-09-15T23:14:14.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:13 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:13.764+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.199331+0000 front 2024-09-15T23:13:31.199464+0000 (oldest deadline 2024-09-15T23:13:57.099498+0000) 2024-09-15T23:14:14.065 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:13 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8[53882]: 2024-09-15T23:14:13.764+0000 7f966d45a640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:37.099982+0000 front 2024-09-15T23:13:37.100084+0000 (oldest deadline 2024-09-15T23:14:06.500379+0000) 2024-09-15T23:14:14.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:13 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:13.868+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:14.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:13 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:13.868+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:14.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:13 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:13.868+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:14.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:13 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:13.868+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:14.066 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:13 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:13.868+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:14.415 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:14 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:14.137+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:14:14.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:14 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:14.137+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:14:14.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:14 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:14.137+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:14:14.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:14 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:14.137+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.081640+0000 front 2024-09-15T23:13:31.081667+0000 (oldest deadline 2024-09-15T23:13:55.082194+0000) 2024-09-15T23:14:14.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:14 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:14.137+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.983723+0000 front 2024-09-15T23:13:40.983756+0000 (oldest deadline 2024-09-15T23:14:05.683421+0000) 2024-09-15T23:14:14.416 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:14 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:14.137+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:45.683935+0000 front 2024-09-15T23:13:45.683995+0000 (oldest deadline 2024-09-15T23:14:10.383882+0000) 2024-09-15T23:14:14.735 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:14 smithi192 podman[74655]: 2024-09-15 23:14:14.412422334 +0000 UTC m=+5.628154240 container died f19a0a1213dbb0537d0bd6b03fa6d3bf840be9947554ec23fa53be7ca0036524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-15T23:14:14.735 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:14 smithi192 podman[74655]: 2024-09-15 23:14:14.734777997 +0000 UTC m=+5.950509894 container cleanup f19a0a1213dbb0537d0bd6b03fa6d3bf840be9947554ec23fa53be7ca0036524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2) 2024-09-15T23:14:14.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:14 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:14.524+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:14.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:14 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:14.524+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:14.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:14 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:14.524+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:14.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:14 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:14.524+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:14.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:14 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:14.524+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:39.308188+0000 front 2024-09-15T23:13:39.308221+0000 (oldest deadline 2024-09-15T23:14:03.408385+0000) 2024-09-15T23:14:15.064 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:14 smithi192 bash[74655]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8 2024-09-15T23:14:15.064 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:14 smithi192 podman[74682]: 2024-09-15 23:14:14.953116916 +0000 UTC m=+0.537416521 container remove f19a0a1213dbb0537d0bd6b03fa6d3bf840be9947554ec23fa53be7ca0036524 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-15T23:14:15.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:14 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:14.869+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:15.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:14 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:14.869+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:15.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:14 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:14.869+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:15.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:14 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:14.869+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:15.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:14 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:14.869+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:15.639 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:15 smithi192 podman[74771]: 2024-09-15 23:14:15.24602123 +0000 UTC m=+0.015354843 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T23:14:15.640 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:15 smithi192 podman[74771]: 2024-09-15 23:14:15.353755869 +0000 UTC m=+0.123089486 container create 140f7e402e71e796a6e1ae1765b6dbe18353efd3068ef4610450b852591c135f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8-deactivate, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T23:14:15.640 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:15 smithi192 podman[74771]: 2024-09-15 23:14:15.564344946 +0000 UTC m=+0.333678565 container init 140f7e402e71e796a6e1ae1765b6dbe18353efd3068ef4610450b852591c135f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-15T23:14:15.640 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:15 smithi192 podman[74771]: 2024-09-15 23:14:15.567615676 +0000 UTC m=+0.336949297 container start 140f7e402e71e796a6e1ae1765b6dbe18353efd3068ef4610450b852591c135f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8-deactivate, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-15T23:14:15.640 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:15 smithi192 podman[74771]: 2024-09-15 23:14:15.639274292 +0000 UTC m=+0.408607907 container attach 140f7e402e71e796a6e1ae1765b6dbe18353efd3068ef4610450b852591c135f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-15T23:14:15.665 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:15 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:15.173+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:14:15.665 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:15 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:15.173+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:14:15.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:15 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:15.173+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:14:15.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:15 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:15.173+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.081640+0000 front 2024-09-15T23:13:31.081667+0000 (oldest deadline 2024-09-15T23:13:55.082194+0000) 2024-09-15T23:14:15.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:15 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:15.173+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.983723+0000 front 2024-09-15T23:13:40.983756+0000 (oldest deadline 2024-09-15T23:14:05.683421+0000) 2024-09-15T23:14:15.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:15 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:15.173+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:45.683935+0000 front 2024-09-15T23:13:45.683995+0000 (oldest deadline 2024-09-15T23:14:10.383882+0000) 2024-09-15T23:14:15.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:15 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:15.553+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:15.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:15 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:15.553+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:15.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:15 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:15.553+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:15.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:15 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:15.553+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:15.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:15 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:15.553+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:39.308188+0000 front 2024-09-15T23:13:39.308221+0000 (oldest deadline 2024-09-15T23:14:03.408385+0000) 2024-09-15T23:14:15.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:15 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:15.553+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:48.709386+0000 front 2024-09-15T23:13:48.709359+0000 (oldest deadline 2024-09-15T23:14:14.609296+0000) 2024-09-15T23:14:16.043 INFO:journalctl@ceph.osd.8.smithi192.stdout:Sep 15 23:14:15 smithi192 podman[74771]: 2024-09-15 23:14:15.75744224 +0000 UTC m=+0.526775853 container died 140f7e402e71e796a6e1ae1765b6dbe18353efd3068ef4610450b852591c135f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-8-deactivate, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-15T23:14:16.043 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:15 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:15.826+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:16.043 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:15 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:15.826+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:16.043 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:15 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:15.826+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:16.043 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:15 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:15.826+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:16.044 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:15 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:15.826+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:16.318 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.8.service' 2024-09-15T23:14:16.357 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:14:16.357 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-15T23:14:16.357 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-15T23:14:16.357 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.9 2024-09-15T23:14:16.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:16 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:16.216+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:14:16.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:16 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:16.216+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:14:16.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:16 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:16.216+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:14:16.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:16 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:16.216+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.081640+0000 front 2024-09-15T23:13:31.081667+0000 (oldest deadline 2024-09-15T23:13:55.082194+0000) 2024-09-15T23:14:16.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:16 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:16.216+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.983723+0000 front 2024-09-15T23:13:40.983756+0000 (oldest deadline 2024-09-15T23:14:05.683421+0000) 2024-09-15T23:14:16.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:16 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:16.216+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:45.683935+0000 front 2024-09-15T23:13:45.683995+0000 (oldest deadline 2024-09-15T23:14:10.383882+0000) 2024-09-15T23:14:16.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:16 smithi111 systemd[1]: Stopping Ceph osd.9 for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:14:16.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:16 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:16.561+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:16.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:16 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:16.561+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:16.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:16 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:16.561+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:16.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:16 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:16.561+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:16.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:16 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:16.561+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:39.308188+0000 front 2024-09-15T23:13:39.308221+0000 (oldest deadline 2024-09-15T23:14:03.408385+0000) 2024-09-15T23:14:16.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:16 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:16.561+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:48.709386+0000 front 2024-09-15T23:13:48.709359+0000 (oldest deadline 2024-09-15T23:14:14.609296+0000) 2024-09-15T23:14:17.165 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:16 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:16.762+0000 7f3f9a000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:14:17.165 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:16 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:16.762+0000 7f3f9a000640 -1 osd.9 92 *** Got signal Terminated *** 2024-09-15T23:14:17.165 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:16 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:16.762+0000 7f3f9a000640 -1 osd.9 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T23:14:17.314 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:16 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:16.849+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:17.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:16 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:16.849+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:17.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:16 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:16.849+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:17.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:16 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:16.849+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:17.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:16 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:16.849+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:17.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:17 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:17.232+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:14:17.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:17 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:17.232+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:14:17.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:17 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:17.232+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:14:17.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:17 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:17.232+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.081640+0000 front 2024-09-15T23:13:31.081667+0000 (oldest deadline 2024-09-15T23:13:55.082194+0000) 2024-09-15T23:14:17.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:17 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:17.232+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.983723+0000 front 2024-09-15T23:13:40.983756+0000 (oldest deadline 2024-09-15T23:14:05.683421+0000) 2024-09-15T23:14:17.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:17 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:17.232+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:45.683935+0000 front 2024-09-15T23:13:45.683995+0000 (oldest deadline 2024-09-15T23:14:10.383882+0000) 2024-09-15T23:14:17.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:17 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:17.594+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:17.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:17 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:17.594+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:17.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:17 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:17.594+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:17.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:17 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:17.594+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:17.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:17 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:17.594+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:39.308188+0000 front 2024-09-15T23:13:39.308221+0000 (oldest deadline 2024-09-15T23:14:03.408385+0000) 2024-09-15T23:14:17.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:17 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:17.594+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:48.709386+0000 front 2024-09-15T23:13:48.709359+0000 (oldest deadline 2024-09-15T23:14:14.609296+0000) 2024-09-15T23:14:18.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:17 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:17.799+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:18.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:17 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:17.799+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:18.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:17 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:17.799+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:18.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:17 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:17.799+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:18.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:17 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:17.799+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:18.665 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:18 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:18.243+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:14:18.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:18 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:18.243+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:14:18.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:18 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:18.243+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:14:18.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:18 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:18.243+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.081640+0000 front 2024-09-15T23:13:31.081667+0000 (oldest deadline 2024-09-15T23:13:55.082194+0000) 2024-09-15T23:14:18.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:18 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:18.243+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.983723+0000 front 2024-09-15T23:13:40.983756+0000 (oldest deadline 2024-09-15T23:14:05.683421+0000) 2024-09-15T23:14:18.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:18 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:18.243+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:45.683935+0000 front 2024-09-15T23:13:45.683995+0000 (oldest deadline 2024-09-15T23:14:10.383882+0000) 2024-09-15T23:14:18.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:18 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:18.588+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:18.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:18 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:18.588+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:18.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:18 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:18.588+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:18.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:18 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:18.588+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:18.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:18 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:18.588+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:39.308188+0000 front 2024-09-15T23:13:39.308221+0000 (oldest deadline 2024-09-15T23:14:03.408385+0000) 2024-09-15T23:14:18.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:18 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:18.588+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:48.709386+0000 front 2024-09-15T23:13:48.709359+0000 (oldest deadline 2024-09-15T23:14:14.609296+0000) 2024-09-15T23:14:19.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:18 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:18.828+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:19.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:18 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:18.828+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:19.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:18 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:18.828+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:19.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:18 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:18.828+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:19.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:18 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:18.828+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:19.665 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:19 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:19.211+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:14:19.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:19 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:19.211+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:14:19.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:19 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:19.211+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:14:19.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:19 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:19.211+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.081640+0000 front 2024-09-15T23:13:31.081667+0000 (oldest deadline 2024-09-15T23:13:55.082194+0000) 2024-09-15T23:14:19.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:19 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:19.211+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.983723+0000 front 2024-09-15T23:13:40.983756+0000 (oldest deadline 2024-09-15T23:14:05.683421+0000) 2024-09-15T23:14:19.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:19 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:19.211+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:45.683935+0000 front 2024-09-15T23:13:45.683995+0000 (oldest deadline 2024-09-15T23:14:10.383882+0000) 2024-09-15T23:14:19.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:19 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:19.566+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:19.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:19 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:19.566+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:19.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:19 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:19.566+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:19.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:19 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:19.566+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:19.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:19 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:19.566+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:39.308188+0000 front 2024-09-15T23:13:39.308221+0000 (oldest deadline 2024-09-15T23:14:03.408385+0000) 2024-09-15T23:14:19.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:19 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:19.566+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:48.709386+0000 front 2024-09-15T23:13:48.709359+0000 (oldest deadline 2024-09-15T23:14:14.609296+0000) 2024-09-15T23:14:20.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:19 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:19.861+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:20.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:19 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:19.861+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:20.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:19 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:19.861+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:20.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:19 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:19.861+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:20.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:19 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:19.861+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:20.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:19 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:19.861+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:56.419164+0000 front 2024-09-15T23:13:56.419211+0000 (oldest deadline 2024-09-15T23:14:19.319093+0000) 2024-09-15T23:14:20.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:20 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:20.209+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:14:20.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:20 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:20.209+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:14:20.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:20 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:20.209+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:14:20.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:20 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:20.209+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.081640+0000 front 2024-09-15T23:13:31.081667+0000 (oldest deadline 2024-09-15T23:13:55.082194+0000) 2024-09-15T23:14:20.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:20 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:20.209+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.983723+0000 front 2024-09-15T23:13:40.983756+0000 (oldest deadline 2024-09-15T23:14:05.683421+0000) 2024-09-15T23:14:20.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:20 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:20.209+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:45.683935+0000 front 2024-09-15T23:13:45.683995+0000 (oldest deadline 2024-09-15T23:14:10.383882+0000) 2024-09-15T23:14:20.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:20 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:20.553+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:20.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:20 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:20.553+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:20.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:20 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:20.553+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:20.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:20 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:20.553+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:20.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:20 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:20.553+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:39.308188+0000 front 2024-09-15T23:13:39.308221+0000 (oldest deadline 2024-09-15T23:14:03.408385+0000) 2024-09-15T23:14:20.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:20 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:20.553+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:48.709386+0000 front 2024-09-15T23:13:48.709359+0000 (oldest deadline 2024-09-15T23:14:14.609296+0000) 2024-09-15T23:14:21.314 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:20 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:20.818+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:21.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:20 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:20.818+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:21.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:20 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:20.818+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:21.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:20 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:20.818+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:21.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:20 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:20.818+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:21.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:20 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:20.818+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:56.419164+0000 front 2024-09-15T23:13:56.419211+0000 (oldest deadline 2024-09-15T23:14:19.319093+0000) 2024-09-15T23:14:21.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:21 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:21.184+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.375253+0000 front 2024-09-15T23:13:07.375285+0000 (oldest deadline 2024-09-15T23:13:29.675485+0000) 2024-09-15T23:14:21.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:21 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:21.184+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:13.077473+0000 front 2024-09-15T23:13:13.077459+0000 (oldest deadline 2024-09-15T23:13:38.878226+0000) 2024-09-15T23:14:21.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:21 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:21.184+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:23.579749+0000 front 2024-09-15T23:13:23.579880+0000 (oldest deadline 2024-09-15T23:13:46.479980+0000) 2024-09-15T23:14:21.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:21 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:21.184+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:31.081640+0000 front 2024-09-15T23:13:31.081667+0000 (oldest deadline 2024-09-15T23:13:55.082194+0000) 2024-09-15T23:14:21.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:21 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:21.184+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.983723+0000 front 2024-09-15T23:13:40.983756+0000 (oldest deadline 2024-09-15T23:14:05.683421+0000) 2024-09-15T23:14:21.666 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:21 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9[64504]: 2024-09-15T23:14:21.184+0000 7f3f90a00640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:45.683935+0000 front 2024-09-15T23:13:45.683995+0000 (oldest deadline 2024-09-15T23:14:10.383882+0000) 2024-09-15T23:14:21.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:21 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:21.601+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:21.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:21 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:21.601+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:21.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:21 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:21.601+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:21.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:21 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:21.601+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:21.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:21 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:21.601+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:39.308188+0000 front 2024-09-15T23:13:39.308221+0000 (oldest deadline 2024-09-15T23:14:03.408385+0000) 2024-09-15T23:14:21.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:21 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:21.601+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:48.709386+0000 front 2024-09-15T23:13:48.709359+0000 (oldest deadline 2024-09-15T23:14:14.609296+0000) 2024-09-15T23:14:21.891 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:21 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:21.601+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:54.609880+0000 front 2024-09-15T23:13:54.609992+0000 (oldest deadline 2024-09-15T23:14:21.010331+0000) 2024-09-15T23:14:22.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:21 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:21.829+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:22.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:21 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:21.829+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:22.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:21 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:21.829+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:22.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:21 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:21.829+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:22.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:21 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:21.829+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:22.315 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:21 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:21.829+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:56.419164+0000 front 2024-09-15T23:13:56.419211+0000 (oldest deadline 2024-09-15T23:14:19.319093+0000) 2024-09-15T23:14:22.677 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:22 smithi111 podman[97038]: 2024-09-15 23:14:22.4192994 +0000 UTC m=+5.770076714 container died 0d16a3d36636e2cf137bebc70ff2d49b832cdcb353c8c713d7687d9328a466a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T23:14:22.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:22 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:22.596+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:22.891 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:22 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:22.596+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:22.891 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:22 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:22.596+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:22.891 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:22 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:22.596+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:22.891 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:22 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:22.596+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:39.308188+0000 front 2024-09-15T23:13:39.308221+0000 (oldest deadline 2024-09-15T23:14:03.408385+0000) 2024-09-15T23:14:22.891 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:22 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:22.596+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:48.709386+0000 front 2024-09-15T23:13:48.709359+0000 (oldest deadline 2024-09-15T23:14:14.609296+0000) 2024-09-15T23:14:22.891 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:22 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:22.596+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:54.609880+0000 front 2024-09-15T23:13:54.609992+0000 (oldest deadline 2024-09-15T23:14:21.010331+0000) 2024-09-15T23:14:22.960 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:22 smithi111 podman[97038]: 2024-09-15 23:14:22.775156696 +0000 UTC m=+6.125934018 container cleanup 0d16a3d36636e2cf137bebc70ff2d49b832cdcb353c8c713d7687d9328a466a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-15T23:14:22.960 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:22 smithi111 bash[97038]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9 2024-09-15T23:14:23.064 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:22 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:22.783+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:23.064 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:22 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:22.783+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:23.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:22 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:22.783+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:23.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:22 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:22.783+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:23.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:22 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:22.783+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:23.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:22 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:22.783+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:56.419164+0000 front 2024-09-15T23:13:56.419211+0000 (oldest deadline 2024-09-15T23:14:19.319093+0000) 2024-09-15T23:14:23.415 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:22 smithi111 podman[97055]: 2024-09-15 23:14:22.959697983 +0000 UTC m=+0.527203517 container remove 0d16a3d36636e2cf137bebc70ff2d49b832cdcb353c8c713d7687d9328a466a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-15T23:14:23.745 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:23 smithi111 podman[97138]: 2024-09-15 23:14:23.39611785 +0000 UTC m=+0.027001349 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T23:14:23.745 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:23 smithi111 podman[97138]: 2024-09-15 23:14:23.51873907 +0000 UTC m=+0.149622563 container create 37151a5560be0bb930374a1e15f1300657023b7791d6550bb4ddb644cbd4e6ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9-deactivate, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-15T23:14:23.745 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:23 smithi111 podman[97138]: 2024-09-15 23:14:23.73821554 +0000 UTC m=+0.369099025 container init 37151a5560be0bb930374a1e15f1300657023b7791d6550bb4ddb644cbd4e6ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T23:14:23.746 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:23 smithi111 podman[97138]: 2024-09-15 23:14:23.744774507 +0000 UTC m=+0.375657994 container start 37151a5560be0bb930374a1e15f1300657023b7791d6550bb4ddb644cbd4e6ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD) 2024-09-15T23:14:23.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:23 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:23.596+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:23.891 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:23 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:23.596+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:23.891 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:23 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:23.596+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:23.891 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:23 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:23.596+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:23.891 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:23 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:23.596+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:39.308188+0000 front 2024-09-15T23:13:39.308221+0000 (oldest deadline 2024-09-15T23:14:03.408385+0000) 2024-09-15T23:14:23.892 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:23 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:23.596+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:48.709386+0000 front 2024-09-15T23:13:48.709359+0000 (oldest deadline 2024-09-15T23:14:14.609296+0000) 2024-09-15T23:14:23.892 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:23 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:23.596+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:54.609880+0000 front 2024-09-15T23:13:54.609992+0000 (oldest deadline 2024-09-15T23:14:21.010331+0000) 2024-09-15T23:14:24.003 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:23 smithi111 podman[97138]: 2024-09-15 23:14:23.837904344 +0000 UTC m=+0.468787844 container attach 37151a5560be0bb930374a1e15f1300657023b7791d6550bb4ddb644cbd4e6ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=) 2024-09-15T23:14:24.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:23 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:23.809+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:24.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:23 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:23.809+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:24.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:23 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:23.809+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:24.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:23 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:23.809+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:24.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:23 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:23.809+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:24.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:23 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:23.809+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:56.419164+0000 front 2024-09-15T23:13:56.419211+0000 (oldest deadline 2024-09-15T23:14:19.319093+0000) 2024-09-15T23:14:24.333 INFO:journalctl@ceph.osd.9.smithi111.stdout:Sep 15 23:14:24 smithi111 podman[97162]: 2024-09-15 23:14:24.041627599 +0000 UTC m=+0.025692435 container died 37151a5560be0bb930374a1e15f1300657023b7791d6550bb4ddb644cbd4e6ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-15T23:14:24.646 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.9.service' 2024-09-15T23:14:24.721 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:14:24.721 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-15T23:14:24.721 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-15T23:14:24.721 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.10 2024-09-15T23:14:24.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:24 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:24.582+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:24.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:24 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:24.582+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:24.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:24 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:24.582+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:24.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:24 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:24.582+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:24.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:24 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:24.582+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:39.308188+0000 front 2024-09-15T23:13:39.308221+0000 (oldest deadline 2024-09-15T23:14:03.408385+0000) 2024-09-15T23:14:24.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:24 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:24.582+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:48.709386+0000 front 2024-09-15T23:13:48.709359+0000 (oldest deadline 2024-09-15T23:14:14.609296+0000) 2024-09-15T23:14:24.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:24 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:24.582+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:54.609880+0000 front 2024-09-15T23:13:54.609992+0000 (oldest deadline 2024-09-15T23:14:21.010331+0000) 2024-09-15T23:14:24.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:24 smithi121 systemd[1]: Stopping Ceph osd.10 for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:14:25.064 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:24 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:24.791+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:25.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:24 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:24.791+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:25.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:24 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:24.791+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:25.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:24 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:24.791+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:25.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:24 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:24.791+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:25.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:24 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:24.791+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:56.419164+0000 front 2024-09-15T23:13:56.419211+0000 (oldest deadline 2024-09-15T23:14:19.319093+0000) 2024-09-15T23:14:25.390 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:25 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:25.023+0000 7f59149b1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:14:25.390 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:25 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:25.023+0000 7f59149b1640 -1 osd.10 92 *** Got signal Terminated *** 2024-09-15T23:14:25.390 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:25 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:25.023+0000 7f59149b1640 -1 osd.10 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T23:14:25.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:25 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:25.552+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:25.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:25 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:25.552+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:25.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:25 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:25.552+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:25.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:25 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:25.552+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:25.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:25 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:25.552+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:39.308188+0000 front 2024-09-15T23:13:39.308221+0000 (oldest deadline 2024-09-15T23:14:03.408385+0000) 2024-09-15T23:14:25.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:25 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:25.552+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:48.709386+0000 front 2024-09-15T23:13:48.709359+0000 (oldest deadline 2024-09-15T23:14:14.609296+0000) 2024-09-15T23:14:25.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:25 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:25.552+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:54.609880+0000 front 2024-09-15T23:13:54.609992+0000 (oldest deadline 2024-09-15T23:14:21.010331+0000) 2024-09-15T23:14:26.064 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:25 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:25.744+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:26.064 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:25 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:25.744+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:26.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:25 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:25.744+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:26.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:25 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:25.744+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:26.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:25 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:25.744+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:26.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:25 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:25.744+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:56.419164+0000 front 2024-09-15T23:13:56.419211+0000 (oldest deadline 2024-09-15T23:14:19.319093+0000) 2024-09-15T23:14:26.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:26 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:26.514+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:26.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:26 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:26.514+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:26.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:26 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:26.514+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:26.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:26 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:26.514+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:26.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:26 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:26.514+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:39.308188+0000 front 2024-09-15T23:13:39.308221+0000 (oldest deadline 2024-09-15T23:14:03.408385+0000) 2024-09-15T23:14:26.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:26 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:26.514+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:48.709386+0000 front 2024-09-15T23:13:48.709359+0000 (oldest deadline 2024-09-15T23:14:14.609296+0000) 2024-09-15T23:14:26.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:26 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:26.514+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:54.609880+0000 front 2024-09-15T23:13:54.609992+0000 (oldest deadline 2024-09-15T23:14:21.010331+0000) 2024-09-15T23:14:27.064 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:26 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:26.717+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:27.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:26 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:26.717+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:27.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:26 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:26.717+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:27.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:26 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:26.717+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:27.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:26 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:26.717+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:27.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:26 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:26.717+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:56.419164+0000 front 2024-09-15T23:13:56.419211+0000 (oldest deadline 2024-09-15T23:14:19.319093+0000) 2024-09-15T23:14:27.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:27 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:27.484+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:27.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:27 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:27.484+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:27.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:27 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:27.484+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:27.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:27 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:27.484+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:27.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:27 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:27.484+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:39.308188+0000 front 2024-09-15T23:13:39.308221+0000 (oldest deadline 2024-09-15T23:14:03.408385+0000) 2024-09-15T23:14:27.891 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:27 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:27.484+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:48.709386+0000 front 2024-09-15T23:13:48.709359+0000 (oldest deadline 2024-09-15T23:14:14.609296+0000) 2024-09-15T23:14:27.891 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:27 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:27.484+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:54.609880+0000 front 2024-09-15T23:13:54.609992+0000 (oldest deadline 2024-09-15T23:14:21.010331+0000) 2024-09-15T23:14:28.064 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:27 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:27.724+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:28.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:27 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:27.724+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:28.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:27 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:27.724+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:28.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:27 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:27.724+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:28.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:27 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:27.724+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:28.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:27 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:27.724+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:56.419164+0000 front 2024-09-15T23:13:56.419211+0000 (oldest deadline 2024-09-15T23:14:19.319093+0000) 2024-09-15T23:14:28.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:27 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:27.724+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6820 osd.7 since back 2024-09-15T23:14:04.620296+0000 front 2024-09-15T23:14:04.620388+0000 (oldest deadline 2024-09-15T23:14:26.920200+0000) 2024-09-15T23:14:28.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:28 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:28.457+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:28.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:28 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:28.457+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:28.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:28 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:28.457+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:28.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:28 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:28.457+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:28.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:28 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:28.457+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:39.308188+0000 front 2024-09-15T23:13:39.308221+0000 (oldest deadline 2024-09-15T23:14:03.408385+0000) 2024-09-15T23:14:28.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:28 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:28.457+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:48.709386+0000 front 2024-09-15T23:13:48.709359+0000 (oldest deadline 2024-09-15T23:14:14.609296+0000) 2024-09-15T23:14:28.891 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:28 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:28.457+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:54.609880+0000 front 2024-09-15T23:13:54.609992+0000 (oldest deadline 2024-09-15T23:14:21.010331+0000) 2024-09-15T23:14:29.064 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:28 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:28.755+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:29.064 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:28 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:28.755+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:29.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:28 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:28.755+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:29.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:28 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:28.755+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:29.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:28 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:28.755+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:29.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:28 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:28.755+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:56.419164+0000 front 2024-09-15T23:13:56.419211+0000 (oldest deadline 2024-09-15T23:14:19.319093+0000) 2024-09-15T23:14:29.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:28 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:28.755+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6820 osd.7 since back 2024-09-15T23:14:04.620296+0000 front 2024-09-15T23:14:04.620388+0000 (oldest deadline 2024-09-15T23:14:26.920200+0000) 2024-09-15T23:14:29.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:29 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:29.434+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:07.103260+0000 front 2024-09-15T23:13:07.103470+0000 (oldest deadline 2024-09-15T23:13:30.603262+0000) 2024-09-15T23:14:29.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:29 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:29.434+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:15.904511+0000 front 2024-09-15T23:13:15.904538+0000 (oldest deadline 2024-09-15T23:13:41.804990+0000) 2024-09-15T23:14:29.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:29 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:29.434+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:21.805417+0000 front 2024-09-15T23:13:21.805442+0000 (oldest deadline 2024-09-15T23:13:46.505632+0000) 2024-09-15T23:14:29.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:29 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:29.434+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6814 osd.3 since back 2024-09-15T23:13:28.206793+0000 front 2024-09-15T23:13:28.206883+0000 (oldest deadline 2024-09-15T23:13:54.607447+0000) 2024-09-15T23:14:29.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:29 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:29.434+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:39.308188+0000 front 2024-09-15T23:13:39.308221+0000 (oldest deadline 2024-09-15T23:14:03.408385+0000) 2024-09-15T23:14:29.890 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:29 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:29.434+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:48.709386+0000 front 2024-09-15T23:13:48.709359+0000 (oldest deadline 2024-09-15T23:14:14.609296+0000) 2024-09-15T23:14:29.891 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:29 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10[60255]: 2024-09-15T23:14:29.434+0000 7f590cfae640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:54.609880+0000 front 2024-09-15T23:13:54.609992+0000 (oldest deadline 2024-09-15T23:14:21.010331+0000) 2024-09-15T23:14:30.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:29 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:29.762+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:30.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:29 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:29.762+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:30.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:29 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:29.762+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:30.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:29 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:29.762+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:30.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:29 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:29.762+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:30.066 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:29 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:29.762+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:56.419164+0000 front 2024-09-15T23:13:56.419211+0000 (oldest deadline 2024-09-15T23:14:19.319093+0000) 2024-09-15T23:14:30.066 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:29 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:29.762+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6820 osd.7 since back 2024-09-15T23:14:04.620296+0000 front 2024-09-15T23:14:04.620388+0000 (oldest deadline 2024-09-15T23:14:26.920200+0000) 2024-09-15T23:14:30.866 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:30 smithi121 podman[76943]: 2024-09-15 23:14:30.588419181 +0000 UTC m=+5.671956144 container died ea22f0a023e95b7a2e7188d84116408d52a79ffdd19f53ea1e3dc7d55f98e533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e) 2024-09-15T23:14:31.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:30 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:30.740+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:31.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:30 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:30.740+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:31.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:30 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:30.740+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:31.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:30 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:30.740+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:31.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:30 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:30.740+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:31.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:30 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:30.740+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:56.419164+0000 front 2024-09-15T23:13:56.419211+0000 (oldest deadline 2024-09-15T23:14:19.319093+0000) 2024-09-15T23:14:31.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:30 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:30.740+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6820 osd.7 since back 2024-09-15T23:14:04.620296+0000 front 2024-09-15T23:14:04.620388+0000 (oldest deadline 2024-09-15T23:14:26.920200+0000) 2024-09-15T23:14:31.140 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:30 smithi121 podman[76943]: 2024-09-15 23:14:30.947999061 +0000 UTC m=+6.031536033 container cleanup ea22f0a023e95b7a2e7188d84116408d52a79ffdd19f53ea1e3dc7d55f98e533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-15T23:14:31.140 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:30 smithi121 bash[76943]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10 2024-09-15T23:14:31.560 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:31 smithi121 podman[76958]: 2024-09-15 23:14:31.232358617 +0000 UTC m=+0.638146655 container remove ea22f0a023e95b7a2e7188d84116408d52a79ffdd19f53ea1e3dc7d55f98e533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, ceph=True, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-15T23:14:31.886 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:31 smithi121 podman[77047]: 2024-09-15 23:14:31.462165981 +0000 UTC m=+0.015659537 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T23:14:31.886 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:31 smithi121 podman[77047]: 2024-09-15 23:14:31.574928456 +0000 UTC m=+0.128422058 container create 61e9a1bbebc815e422a37aebf037245e9d28ca1a3449be308f6f95cd3483c2d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0) 2024-09-15T23:14:31.886 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:31 smithi121 podman[77047]: 2024-09-15 23:14:31.802181196 +0000 UTC m=+0.355674739 container init 61e9a1bbebc815e422a37aebf037245e9d28ca1a3449be308f6f95cd3483c2d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T23:14:31.886 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:31 smithi121 podman[77047]: 2024-09-15 23:14:31.805344069 +0000 UTC m=+0.358837613 container start 61e9a1bbebc815e422a37aebf037245e9d28ca1a3449be308f6f95cd3483c2d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-15T23:14:31.886 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:31 smithi121 podman[77047]: 2024-09-15 23:14:31.885371257 +0000 UTC m=+0.438864813 container attach 61e9a1bbebc815e422a37aebf037245e9d28ca1a3449be308f6f95cd3483c2d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-15T23:14:32.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:31 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:31.707+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:32.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:31 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:31.707+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:32.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:31 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:31.707+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:32.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:31 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:31.707+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:32.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:31 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:31.707+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:32.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:31 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:31.707+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:56.419164+0000 front 2024-09-15T23:13:56.419211+0000 (oldest deadline 2024-09-15T23:14:19.319093+0000) 2024-09-15T23:14:32.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:31 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:31.707+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6820 osd.7 since back 2024-09-15T23:14:04.620296+0000 front 2024-09-15T23:14:04.620388+0000 (oldest deadline 2024-09-15T23:14:26.920200+0000) 2024-09-15T23:14:32.140 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 15 23:14:31 smithi121 podman[77047]: 2024-09-15 23:14:31.993070958 +0000 UTC m=+0.546564512 container died 61e9a1bbebc815e422a37aebf037245e9d28ca1a3449be308f6f95cd3483c2d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-15T23:14:32.698 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.10.service' 2024-09-15T23:14:32.735 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:14:32.735 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-15T23:14:32.736 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-15T23:14:32.736 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.11 2024-09-15T23:14:33.064 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:32 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:32.732+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:33.064 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:32 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:32.732+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:33.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:32 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:32.732+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:33.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:32 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:32.732+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:33.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:32 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:32.732+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:33.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:32 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:32.732+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:56.419164+0000 front 2024-09-15T23:13:56.419211+0000 (oldest deadline 2024-09-15T23:14:19.319093+0000) 2024-09-15T23:14:33.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:32 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:32.732+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6820 osd.7 since back 2024-09-15T23:14:04.620296+0000 front 2024-09-15T23:14:04.620388+0000 (oldest deadline 2024-09-15T23:14:26.920200+0000) 2024-09-15T23:14:33.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:32 smithi192 systemd[1]: Stopping Ceph osd.11 for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:14:33.564 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:33 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:33.070+0000 7fe517e3f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:14:33.564 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:33 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:33.070+0000 7fe517e3f640 -1 osd.11 92 *** Got signal Terminated *** 2024-09-15T23:14:33.565 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:33 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:33.070+0000 7fe517e3f640 -1 osd.11 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T23:14:34.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:33 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:33.732+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:34.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:33 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:33.732+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:34.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:33 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:33.732+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:34.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:33 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:33.732+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:34.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:33 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:33.732+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:34.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:33 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:33.732+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:56.419164+0000 front 2024-09-15T23:13:56.419211+0000 (oldest deadline 2024-09-15T23:14:19.319093+0000) 2024-09-15T23:14:34.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:33 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:33.732+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6820 osd.7 since back 2024-09-15T23:14:04.620296+0000 front 2024-09-15T23:14:04.620388+0000 (oldest deadline 2024-09-15T23:14:26.920200+0000) 2024-09-15T23:14:35.064 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:34 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:34.725+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:35.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:34 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:34.725+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:35.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:34 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:34.725+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:35.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:34 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:34.725+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:35.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:34 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:34.725+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:35.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:34 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:34.725+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:56.419164+0000 front 2024-09-15T23:13:56.419211+0000 (oldest deadline 2024-09-15T23:14:19.319093+0000) 2024-09-15T23:14:35.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:34 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:34.725+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6820 osd.7 since back 2024-09-15T23:14:04.620296+0000 front 2024-09-15T23:14:04.620388+0000 (oldest deadline 2024-09-15T23:14:26.920200+0000) 2024-09-15T23:14:36.064 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:35 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:35.728+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:36.064 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:35 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:35.728+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:36.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:35 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:35.728+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:36.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:35 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:35.728+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:36.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:35 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:35.728+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:36.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:35 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:35.728+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:56.419164+0000 front 2024-09-15T23:13:56.419211+0000 (oldest deadline 2024-09-15T23:14:19.319093+0000) 2024-09-15T23:14:36.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:35 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:35.728+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6820 osd.7 since back 2024-09-15T23:14:04.620296+0000 front 2024-09-15T23:14:04.620388+0000 (oldest deadline 2024-09-15T23:14:26.920200+0000) 2024-09-15T23:14:36.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:35 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:35.728+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6820 osd.8 since back 2024-09-15T23:14:11.521354+0000 front 2024-09-15T23:14:11.521365+0000 (oldest deadline 2024-09-15T23:14:35.021355+0000) 2024-09-15T23:14:37.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:36 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:36.700+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:37.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:36 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:36.700+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:37.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:36 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:36.700+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:37.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:36 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:36.700+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:37.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:36 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:36.700+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:37.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:36 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:36.700+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:56.419164+0000 front 2024-09-15T23:13:56.419211+0000 (oldest deadline 2024-09-15T23:14:19.319093+0000) 2024-09-15T23:14:37.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:36 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:36.700+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6820 osd.7 since back 2024-09-15T23:14:04.620296+0000 front 2024-09-15T23:14:04.620388+0000 (oldest deadline 2024-09-15T23:14:26.920200+0000) 2024-09-15T23:14:37.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:36 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:36.700+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6820 osd.8 since back 2024-09-15T23:14:11.521354+0000 front 2024-09-15T23:14:11.521365+0000 (oldest deadline 2024-09-15T23:14:35.021355+0000) 2024-09-15T23:14:38.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:37 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:37.672+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6806 osd.0 since back 2024-09-15T23:13:08.612203+0000 front 2024-09-15T23:13:08.612153+0000 (oldest deadline 2024-09-15T23:13:32.712009+0000) 2024-09-15T23:14:38.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:37 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:37.672+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-15T23:13:12.712672+0000 front 2024-09-15T23:13:12.712640+0000 (oldest deadline 2024-09-15T23:13:38.012724+0000) 2024-09-15T23:14:38.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:37 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:37.672+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6804 osd.2 since back 2024-09-15T23:13:20.913676+0000 front 2024-09-15T23:13:20.913756+0000 (oldest deadline 2024-09-15T23:13:46.213769+0000) 2024-09-15T23:14:38.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:37 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:37.672+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-15T23:13:40.816477+0000 front 2024-09-15T23:13:40.816400+0000 (oldest deadline 2024-09-15T23:14:04.316398+0000) 2024-09-15T23:14:38.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:37 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:37.672+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6812 osd.5 since back 2024-09-15T23:13:44.817522+0000 front 2024-09-15T23:13:44.817455+0000 (oldest deadline 2024-09-15T23:14:10.717505+0000) 2024-09-15T23:14:38.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:37 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:37.672+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.111:6822 osd.6 since back 2024-09-15T23:13:56.419164+0000 front 2024-09-15T23:13:56.419211+0000 (oldest deadline 2024-09-15T23:14:19.319093+0000) 2024-09-15T23:14:38.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:37 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:37.672+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.121:6820 osd.7 since back 2024-09-15T23:14:04.620296+0000 front 2024-09-15T23:14:04.620388+0000 (oldest deadline 2024-09-15T23:14:26.920200+0000) 2024-09-15T23:14:38.065 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:37 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11[60617]: 2024-09-15T23:14:37.672+0000 7fe514444640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.192:6820 osd.8 since back 2024-09-15T23:14:11.521354+0000 front 2024-09-15T23:14:11.521365+0000 (oldest deadline 2024-09-15T23:14:35.021355+0000) 2024-09-15T23:14:38.922 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:38 smithi192 podman[74872]: 2024-09-15 23:14:38.641835709 +0000 UTC m=+5.701831554 container died 2804b28dbf606bd89f460d5eb86eaeddb5e9b0d16df8d44719e978344727a7de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2) 2024-09-15T23:14:39.263 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:39 smithi192 podman[74872]: 2024-09-15 23:14:39.020093934 +0000 UTC m=+6.080089779 container cleanup 2804b28dbf606bd89f460d5eb86eaeddb5e9b0d16df8d44719e978344727a7de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-15T23:14:39.263 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:39 smithi192 bash[74872]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11 2024-09-15T23:14:39.564 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:39 smithi192 podman[74891]: 2024-09-15 23:14:39.262988003 +0000 UTC m=+0.612662203 container remove 2804b28dbf606bd89f460d5eb86eaeddb5e9b0d16df8d44719e978344727a7de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-15T23:14:39.908 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:39 smithi192 podman[74981]: 2024-09-15 23:14:39.49029858 +0000 UTC m=+0.015676986 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T23:14:39.908 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:39 smithi192 podman[74981]: 2024-09-15 23:14:39.605252672 +0000 UTC m=+0.130631078 container create 80cc7c794cc1dc23d48966c81bd10534cb57805aeb25f9fbd302d5987ad2e1ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T23:14:39.908 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:39 smithi192 podman[74981]: 2024-09-15 23:14:39.832549623 +0000 UTC m=+0.357928053 container init 80cc7c794cc1dc23d48966c81bd10534cb57805aeb25f9fbd302d5987ad2e1ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T23:14:39.908 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:39 smithi192 podman[74981]: 2024-09-15 23:14:39.835700362 +0000 UTC m=+0.361078769 container start 80cc7c794cc1dc23d48966c81bd10534cb57805aeb25f9fbd302d5987ad2e1ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-15T23:14:39.908 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:39 smithi192 podman[74981]: 2024-09-15 23:14:39.907423101 +0000 UTC m=+0.432801524 container attach 80cc7c794cc1dc23d48966c81bd10534cb57805aeb25f9fbd302d5987ad2e1ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T23:14:40.314 INFO:journalctl@ceph.osd.11.smithi192.stdout:Sep 15 23:14:40 smithi192 podman[74981]: 2024-09-15 23:14:40.027130257 +0000 UTC m=+0.552508668 container died 80cc7c794cc1dc23d48966c81bd10534cb57805aeb25f9fbd302d5987ad2e1ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T23:14:40.647 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@osd.11.service' 2024-09-15T23:14:40.688 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:14:40.688 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-15T23:14:40.688 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-15T23:14:40.688 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.a 2024-09-15T23:14:41.097 INFO:journalctl@ceph.mds.a.smithi111.stdout:Sep 15 23:14:40 smithi111 systemd[1]: Stopping Ceph mds.a for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:14:41.409 INFO:journalctl@ceph.mds.a.smithi111.stdout:Sep 15 23:14:41 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-a[71271]: 2024-09-15T23:14:41.095+0000 7f92d6400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:14:41.409 INFO:journalctl@ceph.mds.a.smithi111.stdout:Sep 15 23:14:41 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-a[71271]: 2024-09-15T23:14:41.095+0000 7f92d6400640 -1 mds.a *** got signal Terminated *** 2024-09-15T23:14:42.762 INFO:journalctl@ceph.mds.a.smithi111.stdout:Sep 15 23:14:42 smithi111 podman[97242]: 2024-09-15 23:14:42.479406883 +0000 UTC m=+1.501586276 container died acfd8775c53ae73aeae3eb1abadd5e8496ba587df41921fcfc2b9195ffa9d38f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-15T23:14:43.129 INFO:journalctl@ceph.mds.a.smithi111.stdout:Sep 15 23:14:42 smithi111 podman[97242]: 2024-09-15 23:14:42.892465658 +0000 UTC m=+1.914645050 container cleanup acfd8775c53ae73aeae3eb1abadd5e8496ba587df41921fcfc2b9195ffa9d38f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-a, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-15T23:14:43.129 INFO:journalctl@ceph.mds.a.smithi111.stdout:Sep 15 23:14:42 smithi111 bash[97242]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-a 2024-09-15T23:14:43.303 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.a.service' 2024-09-15T23:14:43.376 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:14:43.376 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-15T23:14:43.376 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-15T23:14:43.376 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.d 2024-09-15T23:14:43.767 INFO:journalctl@ceph.mds.d.smithi111.stdout:Sep 15 23:14:43 smithi111 systemd[1]: Stopping Ceph mds.d for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:14:44.165 INFO:journalctl@ceph.mds.d.smithi111.stdout:Sep 15 23:14:43 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-d[70080]: 2024-09-15T23:14:43.765+0000 7f023b400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:14:44.166 INFO:journalctl@ceph.mds.d.smithi111.stdout:Sep 15 23:14:43 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-d[70080]: 2024-09-15T23:14:43.765+0000 7f023b400640 -1 mds.d *** got signal Terminated *** 2024-09-15T23:14:46.023 INFO:journalctl@ceph.mds.d.smithi111.stdout:Sep 15 23:14:45 smithi111 podman[97374]: 2024-09-15 23:14:45.74047027 +0000 UTC m=+2.091747262 container died 5dee87e37f44e9c3e4dec436d52d3a9018a563ad02f5590e5fb504d0a05cd84e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-d, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-15T23:14:46.398 INFO:journalctl@ceph.mds.d.smithi111.stdout:Sep 15 23:14:46 smithi111 podman[97374]: 2024-09-15 23:14:46.171547289 +0000 UTC m=+2.522824277 container cleanup 5dee87e37f44e9c3e4dec436d52d3a9018a563ad02f5590e5fb504d0a05cd84e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-d, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-15T23:14:46.399 INFO:journalctl@ceph.mds.d.smithi111.stdout:Sep 15 23:14:46 smithi111 bash[97374]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-d 2024-09-15T23:14:46.655 INFO:journalctl@ceph.mds.d.smithi111.stdout:Sep 15 23:14:46 smithi111 podman[97389]: 2024-09-15 23:14:46.397781079 +0000 UTC m=+0.644743314 container remove 5dee87e37f44e9c3e4dec436d52d3a9018a563ad02f5590e5fb504d0a05cd84e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-d, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T23:14:46.692 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.d.service' 2024-09-15T23:14:46.763 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:14:46.763 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-15T23:14:46.764 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-15T23:14:46.764 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.g 2024-09-15T23:14:47.154 INFO:journalctl@ceph.mds.g.smithi111.stdout:Sep 15 23:14:46 smithi111 systemd[1]: Stopping Ceph mds.g for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:14:47.415 INFO:journalctl@ceph.mds.g.smithi111.stdout:Sep 15 23:14:47 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-g[70926]: 2024-09-15T23:14:47.152+0000 7fda4a600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:14:47.416 INFO:journalctl@ceph.mds.g.smithi111.stdout:Sep 15 23:14:47 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-g[70926]: 2024-09-15T23:14:47.152+0000 7fda4a600640 -1 mds.g *** got signal Terminated *** 2024-09-15T23:14:51.569 INFO:journalctl@ceph.mds.g.smithi111.stdout:Sep 15 23:14:51 smithi111 podman[97494]: 2024-09-15 23:14:51.170102136 +0000 UTC m=+4.136435338 container died 367ebeea7f7183a2e28ae1cdd243b2074a21d5eb89d8258f09e9259a99b0778b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e) 2024-09-15T23:14:51.915 INFO:journalctl@ceph.mds.g.smithi111.stdout:Sep 15 23:14:51 smithi111 podman[97494]: 2024-09-15 23:14:51.674495433 +0000 UTC m=+4.640828632 container cleanup 367ebeea7f7183a2e28ae1cdd243b2074a21d5eb89d8258f09e9259a99b0778b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-g, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T23:14:51.916 INFO:journalctl@ceph.mds.g.smithi111.stdout:Sep 15 23:14:51 smithi111 bash[97494]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-g 2024-09-15T23:14:52.085 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.g.service' 2024-09-15T23:14:52.154 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:14:52.155 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-15T23:14:52.155 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-15T23:14:52.155 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.j 2024-09-15T23:14:52.533 INFO:journalctl@ceph.mds.j.smithi111.stdout:Sep 15 23:14:52 smithi111 systemd[1]: Stopping Ceph mds.j for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:14:52.915 INFO:journalctl@ceph.mds.j.smithi111.stdout:Sep 15 23:14:52 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-j[69665]: 2024-09-15T23:14:52.531+0000 7f66f6000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:14:52.915 INFO:journalctl@ceph.mds.j.smithi111.stdout:Sep 15 23:14:52 smithi111 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-j[69665]: 2024-09-15T23:14:52.531+0000 7f66f6000640 -1 mds.j *** got signal Terminated *** 2024-09-15T23:14:56.822 INFO:journalctl@ceph.mds.j.smithi111.stdout:Sep 15 23:14:56 smithi111 podman[97615]: 2024-09-15 23:14:56.552493689 +0000 UTC m=+4.123343558 container died 395bf4c75badffe1eb260df6b11ebe90f9a6e857c8a91c8f3c68a907c129d7e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-j, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-15T23:14:57.166 INFO:journalctl@ceph.mds.j.smithi111.stdout:Sep 15 23:14:57 smithi111 podman[97615]: 2024-09-15 23:14:57.020036636 +0000 UTC m=+4.590886517 container cleanup 395bf4c75badffe1eb260df6b11ebe90f9a6e857c8a91c8f3c68a907c129d7e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-15T23:14:57.166 INFO:journalctl@ceph.mds.j.smithi111.stdout:Sep 15 23:14:57 smithi111 bash[97615]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-j 2024-09-15T23:14:57.452 INFO:journalctl@ceph.mds.j.smithi111.stdout:Sep 15 23:14:57 smithi111 podman[97627]: 2024-09-15 23:14:57.196974321 +0000 UTC m=+0.633544340 container remove 395bf4c75badffe1eb260df6b11ebe90f9a6e857c8a91c8f3c68a907c129d7e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-j, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T23:14:57.461 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.j.service' 2024-09-15T23:14:57.529 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:14:57.529 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-15T23:14:57.529 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-15T23:14:57.529 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.b 2024-09-15T23:14:57.848 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 15 23:14:57 smithi121 systemd[1]: Stopping Ceph mds.b for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:14:58.140 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 15 23:14:57 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-b[65483]: 2024-09-15T23:14:57.846+0000 7f41608b7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:14:58.140 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 15 23:14:57 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-b[65483]: 2024-09-15T23:14:57.846+0000 7f41608b7640 -1 mds.b *** got signal Terminated *** 2024-09-15T23:15:00.589 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 15 23:15:00 smithi121 podman[77163]: 2024-09-15 23:15:00.325664852 +0000 UTC m=+2.592541022 container died 14529e1860a7900a453d514cb5d727f7e3e0ac88fe97f1e717c0eb5e98425874 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=) 2024-09-15T23:15:00.890 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 15 23:15:00 smithi121 podman[77163]: 2024-09-15 23:15:00.68785473 +0000 UTC m=+2.954730889 container cleanup 14529e1860a7900a453d514cb5d727f7e3e0ac88fe97f1e717c0eb5e98425874 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T23:15:00.890 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 15 23:15:00 smithi121 bash[77163]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-b 2024-09-15T23:15:01.137 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.b.service' 2024-09-15T23:15:01.171 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:15:01.171 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-15T23:15:01.172 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-15T23:15:01.172 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.e 2024-09-15T23:15:01.390 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 15 23:15:01 smithi121 systemd[1]: Stopping Ceph mds.e for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:15:02.140 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 15 23:15:01 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-e[64881]: 2024-09-15T23:15:01.640+0000 7fb5b5e8f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:15:02.140 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 15 23:15:01 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-e[64881]: 2024-09-15T23:15:01.640+0000 7fb5b5e8f640 -1 mds.e *** got signal Terminated *** 2024-09-15T23:15:04.619 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 15 23:15:04 smithi121 podman[77286]: 2024-09-15 23:15:04.316897009 +0000 UTC m=+2.922053452 container died 15264d5476bacf142b2bea78971b85db09613887f648840f0979ac06107ac043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-e, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-09-15T23:15:04.885 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 15 23:15:04 smithi121 podman[77286]: 2024-09-15 23:15:04.716869824 +0000 UTC m=+3.322026252 container cleanup 15264d5476bacf142b2bea78971b85db09613887f648840f0979ac06107ac043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-15T23:15:04.885 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 15 23:15:04 smithi121 bash[77286]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-e 2024-09-15T23:15:05.136 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 15 23:15:04 smithi121 podman[77298]: 2024-09-15 23:15:04.884863552 +0000 UTC m=+0.561983312 container remove 15264d5476bacf142b2bea78971b85db09613887f648840f0979ac06107ac043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-e, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0) 2024-09-15T23:15:05.140 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.e.service' 2024-09-15T23:15:05.172 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:15:05.173 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-15T23:15:05.173 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-15T23:15:05.173 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.h 2024-09-15T23:15:05.389 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 15 23:15:05 smithi121 systemd[1]: Stopping Ceph mds.h for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:15:05.890 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 15 23:15:05 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-h[65181]: 2024-09-15T23:15:05.494+0000 7f46db5d7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:15:05.890 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 15 23:15:05 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-h[65181]: 2024-09-15T23:15:05.494+0000 7f46db5d7640 -1 mds.h *** got signal Terminated *** 2024-09-15T23:15:08.588 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 15 23:15:08 smithi121 podman[77408]: 2024-09-15 23:15:08.323080146 +0000 UTC m=+2.934821933 container died c64e8fa330a3b44ad3cc7f2e63a36e1c896fafe5d33ec7afb2631c715c91b2ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-09-15T23:15:08.890 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 15 23:15:08 smithi121 podman[77408]: 2024-09-15 23:15:08.686742815 +0000 UTC m=+3.298484602 container cleanup c64e8fa330a3b44ad3cc7f2e63a36e1c896fafe5d33ec7afb2631c715c91b2ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-h, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T23:15:08.890 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 15 23:15:08 smithi121 bash[77408]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-h 2024-09-15T23:15:09.117 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.h.service' 2024-09-15T23:15:09.152 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:15:09.152 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-15T23:15:09.152 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-15T23:15:09.152 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.k 2024-09-15T23:15:09.390 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 15 23:15:09 smithi121 systemd[1]: Stopping Ceph mds.k for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:15:09.890 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 15 23:15:09 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-k[64564]: 2024-09-15T23:15:09.489+0000 7f9a782ee640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:15:09.890 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 15 23:15:09 smithi121 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-k[64564]: 2024-09-15T23:15:09.489+0000 7f9a782ee640 -1 mds.k *** got signal Terminated *** 2024-09-15T23:15:13.640 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 15 23:15:13 smithi121 podman[77530]: 2024-09-15 23:15:13.37114116 +0000 UTC m=+4.005958411 container died c856b2b02278263463ed386b9e3426a53171d52161722e3ccf5fc3c817c47a2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-k, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-15T23:15:13.989 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 15 23:15:13 smithi121 podman[77530]: 2024-09-15 23:15:13.767603517 +0000 UTC m=+4.402420765 container cleanup c856b2b02278263463ed386b9e3426a53171d52161722e3ccf5fc3c817c47a2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-09-15T23:15:13.989 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 15 23:15:13 smithi121 bash[77530]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-k 2024-09-15T23:15:14.154 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.k.service' 2024-09-15T23:15:14.188 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:15:14.188 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-15T23:15:14.189 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-15T23:15:14.189 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.c 2024-09-15T23:15:14.495 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 15 23:15:14 smithi192 systemd[1]: Stopping Ceph mds.c for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:15:14.814 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 15 23:15:14 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-c[65843]: 2024-09-15T23:15:14.494+0000 7fd44706a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:15:14.814 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 15 23:15:14 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-c[65843]: 2024-09-15T23:15:14.494+0000 7fd44706a640 -1 mds.c *** got signal Terminated *** 2024-09-15T23:15:16.545 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 15 23:15:16 smithi192 podman[75094]: 2024-09-15 23:15:16.274492816 +0000 UTC m=+1.892541751 container died 6f184d05796872f063d8b61a90784fa46aaa10ac74ed8cd3ab20b33ae56d8368 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e) 2024-09-15T23:15:16.814 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 15 23:15:16 smithi192 podman[75094]: 2024-09-15 23:15:16.635707806 +0000 UTC m=+2.253756740 container cleanup 6f184d05796872f063d8b61a90784fa46aaa10ac74ed8cd3ab20b33ae56d8368 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-c, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-15T23:15:16.814 INFO:journalctl@ceph.mds.c.smithi192.stdout:Sep 15 23:15:16 smithi192 bash[75094]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-c 2024-09-15T23:15:17.058 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.c.service' 2024-09-15T23:15:17.092 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:15:17.092 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-15T23:15:17.093 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-15T23:15:17.093 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.f 2024-09-15T23:15:17.314 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 15 23:15:17 smithi192 systemd[1]: Stopping Ceph mds.f for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:15:17.814 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 15 23:15:17 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-f[65543]: 2024-09-15T23:15:17.413+0000 7f92ee150640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:15:17.815 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 15 23:15:17 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-f[65543]: 2024-09-15T23:15:17.413+0000 7f92ee150640 -1 mds.f *** got signal Terminated *** 2024-09-15T23:15:20.565 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 15 23:15:20 smithi192 podman[75216]: 2024-09-15 23:15:20.272290067 +0000 UTC m=+2.959083379 container died 3d5ca0b95ccd4de60f7cb3b661ae6e13504bf23d3ccf0a62f6673f1fbfb2fcb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-f, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-15T23:15:21.012 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 15 23:15:20 smithi192 podman[75216]: 2024-09-15 23:15:20.606281033 +0000 UTC m=+3.293074348 container cleanup 3d5ca0b95ccd4de60f7cb3b661ae6e13504bf23d3ccf0a62f6673f1fbfb2fcb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e) 2024-09-15T23:15:21.012 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 15 23:15:20 smithi192 bash[75216]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-f 2024-09-15T23:15:21.012 INFO:journalctl@ceph.mds.f.smithi192.stdout:Sep 15 23:15:20 smithi192 podman[75228]: 2024-09-15 23:15:20.782390632 +0000 UTC m=+0.507193540 container remove 3d5ca0b95ccd4de60f7cb3b661ae6e13504bf23d3ccf0a62f6673f1fbfb2fcb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True) 2024-09-15T23:15:21.035 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.f.service' 2024-09-15T23:15:21.068 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:15:21.068 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-15T23:15:21.068 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-15T23:15:21.068 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.i 2024-09-15T23:15:21.314 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 15 23:15:21 smithi192 systemd[1]: Stopping Ceph mds.i for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:15:21.814 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 15 23:15:21 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-i[66145]: 2024-09-15T23:15:21.409+0000 7fb583c61640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:15:21.814 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 15 23:15:21 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-i[66145]: 2024-09-15T23:15:21.409+0000 7fb583c61640 -1 mds.i *** got signal Terminated *** 2024-09-15T23:15:24.529 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 15 23:15:24 smithi192 podman[75336]: 2024-09-15 23:15:24.270339981 +0000 UTC m=+2.982983344 container died 6327a8c10c2d1ff4eb78e5f5766a4bd0bc7c499b82d29d4df39bf8c4c64ff044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-i, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-15T23:15:24.786 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 15 23:15:24 smithi192 podman[75336]: 2024-09-15 23:15:24.601535588 +0000 UTC m=+3.314179017 container cleanup 6327a8c10c2d1ff4eb78e5f5766a4bd0bc7c499b82d29d4df39bf8c4c64ff044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-i, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-15T23:15:24.787 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 15 23:15:24 smithi192 bash[75336]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-i 2024-09-15T23:15:25.028 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.i.service' 2024-09-15T23:15:25.050 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 15 23:15:24 smithi192 podman[75348]: 2024-09-15 23:15:24.786228821 +0000 UTC m=+0.508490125 container remove 6327a8c10c2d1ff4eb78e5f5766a4bd0bc7c499b82d29d4df39bf8c4c64ff044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-i, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T23:15:25.050 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 15 23:15:25 smithi192 systemd[1]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.i.service: Deactivated successfully. 2024-09-15T23:15:25.050 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 15 23:15:25 smithi192 systemd[1]: Stopped Ceph mds.i for b7614d3c-73b2-11ef-bceb-c7b262605968. 2024-09-15T23:15:25.050 INFO:journalctl@ceph.mds.i.smithi192.stdout:Sep 15 23:15:25 smithi192 systemd[1]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.i.service: Consumed 3min 50.301s CPU time. 2024-09-15T23:15:25.060 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:15:25.061 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-15T23:15:25.061 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-15T23:15:25.061 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.l 2024-09-15T23:15:25.314 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 15 23:15:25 smithi192 systemd[1]: Stopping Ceph mds.l for b7614d3c-73b2-11ef-bceb-c7b262605968... 2024-09-15T23:15:25.814 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 15 23:15:25 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-l[65241]: 2024-09-15T23:15:25.404+0000 7fe1c3736640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T23:15:25.814 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 15 23:15:25 smithi192 ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-l[65241]: 2024-09-15T23:15:25.405+0000 7fe1c3736640 -1 mds.l *** got signal Terminated *** 2024-09-15T23:15:27.140 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 15 23:15:26 smithi192 podman[75455]: 2024-09-15 23:15:26.859258011 +0000 UTC m=+1.573497947 container died 9f94d35c51ebce28eaf7c475bae58406f7bad871d7b1146301f56a3d13fcc274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-15T23:15:27.482 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 15 23:15:27 smithi192 podman[75455]: 2024-09-15 23:15:27.239035782 +0000 UTC m=+1.953275717 container cleanup 9f94d35c51ebce28eaf7c475bae58406f7bad871d7b1146301f56a3d13fcc274 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-l, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-15T23:15:27.482 INFO:journalctl@ceph.mds.l.smithi192.stdout:Sep 15 23:15:27 smithi192 bash[75455]: ceph-b7614d3c-73b2-11ef-bceb-c7b262605968-mds-l 2024-09-15T23:15:27.615 DEBUG:teuthology.orchestra.run.smithi192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b7614d3c-73b2-11ef-bceb-c7b262605968@mds.l.service' 2024-09-15T23:15:27.646 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T23:15:27.647 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-15T23:15:27.647 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-15T23:15:27.918 INFO:teuthology.orchestra.run.smithi111.stdout:Deleting cluster with fsid: b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T23:15:31.916 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-15T23:15:32.118 INFO:teuthology.orchestra.run.smithi121.stdout:Deleting cluster with fsid: b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T23:15:34.609 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-15T23:15:34.810 INFO:teuthology.orchestra.run.smithi192.stdout:Deleting cluster with fsid: b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T23:15:37.232 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-15T23:15:37.297 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-15T23:15:37.330 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-15T23:15:37.357 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-15T23:15:37.359 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906262/remote/smithi111/crash 2024-09-15T23:15:37.359 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/crash -- . 2024-09-15T23:15:37.427 INFO:teuthology.orchestra.run.smithi111.stderr:tar: /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-15T23:15:37.427 INFO:teuthology.orchestra.run.smithi111.stderr:tar: Error is not recoverable: exiting now 2024-09-15T23:15:37.430 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906262/remote/smithi121/crash 2024-09-15T23:15:37.431 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/crash -- . 2024-09-15T23:15:37.469 INFO:teuthology.orchestra.run.smithi121.stderr:tar: /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-15T23:15:37.469 INFO:teuthology.orchestra.run.smithi121.stderr:tar: Error is not recoverable: exiting now 2024-09-15T23:15:37.471 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906262/remote/smithi192/crash 2024-09-15T23:15:37.472 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/crash -- . 2024-09-15T23:15:37.518 INFO:teuthology.orchestra.run.smithi192.stderr:tar: /var/lib/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-15T23:15:37.518 INFO:teuthology.orchestra.run.smithi192.stderr:tar: Error is not recoverable: exiting now 2024-09-15T23:15:37.519 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-15T23:15:37.519 DEBUG:teuthology.orchestra.run.smithi111:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-15T23:15:37.602 INFO:tasks.cephadm:Compressing logs... 2024-09-15T23:15:37.602 DEBUG:teuthology.orchestra.run.smithi111:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T23:15:37.606 DEBUG:teuthology.orchestra.run.smithi121:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T23:15:37.609 DEBUG:teuthology.orchestra.run.smithi192:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T23:15:37.634 INFO:teuthology.orchestra.run.smithi192.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-15T23:15:37.634 INFO:teuthology.orchestra.run.smithi192.stderr:: No such file or directory 2024-09-15T23:15:37.634 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-15T23:15:37.634 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-15T23:15:37.634 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-15T23:15:37.634 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-15T23:15:37.637 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-15T23:15:37.637 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mon.c.log: /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-15T23:15:37.637 INFO:teuthology.orchestra.run.smithi192.stderr: 83.1% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-15T23:15:37.637 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-15T23:15:37.637 INFO:teuthology.orchestra.run.smithi192.stderr:gzip/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.2.log: -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.log 2024-09-15T23:15:37.637 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-15T23:15:37.637 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.log: 90.7% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-15T23:15:37.640 INFO:teuthology.orchestra.run.smithi192.stderr: 87.0% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-15T23:15:37.641 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-15T23:15:37.641 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-15T23:15:37.648 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.8.log: 90.9% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-15T23:15:37.648 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-15T23:15:37.649 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mgr.z.log: 94.7% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-15T23:15:37.656 INFO:teuthology.orchestra.run.smithi111.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-15T23:15:37.656 INFO:teuthology.orchestra.run.smithi111.stderr:: No such file or directory 2024-09-15T23:15:37.656 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-15T23:15:37.657 INFO:teuthology.orchestra.run.smithi121.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-15T23:15:37.660 INFO:teuthology.orchestra.run.smithi192.stderr: 93.6% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-15T23:15:37.660 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-15T23:15:37.661 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-15T23:15:37.662 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-15T23:15:37.662 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-15T23:15:37.662 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-15T23:15:37.662 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-15T23:15:37.665 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-15T23:15:37.665 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-15T23:15:37.665 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-15T23:15:37.666 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.10.log: /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-15T23:15:37.666 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-15T23:15:37.666 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-15T23:15:37.666 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-15T23:15:37.666 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.b.log: /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-15T23:15:37.666 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-15T23:15:37.666 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-15T23:15:37.666 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.log 2024-09-15T23:15:37.666 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-15T23:15:37.666 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-15T23:15:37.666 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-15T23:15:37.667 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.h.log: /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-15T23:15:37.667 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-15T23:15:37.667 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-15T23:15:37.668 INFO:teuthology.orchestra.run.smithi111.stderr: 82.9% 90.5% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.audit.log.gz -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-15T23:15:37.668 INFO:teuthology.orchestra.run.smithi111.stderr: 2024-09-15T23:15:37.668 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-15T23:15:37.668 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.log 2024-09-15T23:15:37.670 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.6.log: /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-15T23:15:37.671 INFO:teuthology.orchestra.run.smithi111.stderr: 94.6% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-15T23:15:37.672 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-15T23:15:37.673 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-15T23:15:37.674 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.d.log: /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mgr.x.log: 94.7% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-15T23:15:37.676 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-15T23:15:37.676 INFO:teuthology.orchestra.run.smithi111.stderr: 90.9% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-15T23:15:37.677 INFO:teuthology.orchestra.run.smithi111.stderr: 87.0% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-15T23:15:37.677 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-15T23:15:37.678 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-15T23:15:37.682 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.j.log: 94.7% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-15T23:15:37.691 INFO:teuthology.orchestra.run.smithi111.stderr: 93.5% -- replaced with /var/log/ceph/cephadm.log.gz 94.6% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-15T23:15:37.691 INFO:teuthology.orchestra.run.smithi111.stderr: 2024-09-15T23:15:37.771 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-15T23:15:37.835 INFO:teuthology.orchestra.run.smithi121.stderr: 90.9% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-15T23:15:37.855 INFO:teuthology.orchestra.run.smithi121.stderr: 90.7% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-15T23:15:37.873 INFO:teuthology.orchestra.run.smithi121.stderr: 93.5% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-15T23:15:37.896 INFO:teuthology.orchestra.run.smithi121.stderr: 87.0% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-15T23:15:37.898 INFO:teuthology.orchestra.run.smithi121.stderr: 94.6% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-15T23:15:37.968 INFO:teuthology.orchestra.run.smithi111.stderr: 90.6% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-15T23:15:38.079 INFO:teuthology.orchestra.run.smithi121.stderr: 83.1% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-15T23:15:38.289 INFO:teuthology.orchestra.run.smithi192.stderr: 91.7% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-15T23:15:40.517 INFO:teuthology.orchestra.run.smithi111.stderr: 92.1% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-15T23:15:42.569 INFO:teuthology.orchestra.run.smithi192.stderr: 95.7% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-15T23:15:43.792 INFO:teuthology.orchestra.run.smithi192.stderr: 95.2% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-15T23:15:48.391 INFO:teuthology.orchestra.run.smithi111.stderr: 94.2% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-15T23:15:51.826 INFO:teuthology.orchestra.run.smithi192.stderr: 93.9% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-15T23:15:52.826 INFO:teuthology.orchestra.run.smithi111.stderr: 93.8% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-15T23:15:53.752 INFO:teuthology.orchestra.run.smithi111.stderr: 94.0% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-15T23:15:54.684 INFO:teuthology.orchestra.run.smithi192.stderr: 94.2% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-15T23:15:54.777 INFO:teuthology.orchestra.run.smithi121.stderr: 91.8% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-15T23:15:56.479 INFO:teuthology.orchestra.run.smithi111.stderr: 94.0% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-15T23:15:56.482 INFO:teuthology.orchestra.run.smithi111.stderr: 2024-09-15T23:15:56.482 INFO:teuthology.orchestra.run.smithi111.stderr:real 0m18.847s 2024-09-15T23:15:56.482 INFO:teuthology.orchestra.run.smithi111.stderr:user 0m58.721s 2024-09-15T23:15:56.482 INFO:teuthology.orchestra.run.smithi111.stderr:sys 0m5.346s 2024-09-15T23:15:56.936 INFO:teuthology.orchestra.run.smithi192.stderr: 94.1% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-15T23:15:58.273 INFO:teuthology.orchestra.run.smithi192.stderr: 94.1% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-15T23:16:49.503 INFO:teuthology.orchestra.run.smithi192.stderr: 93.7% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-15T23:16:49.505 INFO:teuthology.orchestra.run.smithi192.stderr: 2024-09-15T23:16:49.505 INFO:teuthology.orchestra.run.smithi192.stderr:real 1m11.881s 2024-09-15T23:16:49.505 INFO:teuthology.orchestra.run.smithi192.stderr:user 2m29.539s 2024-09-15T23:16:49.505 INFO:teuthology.orchestra.run.smithi192.stderr:sys 0m4.910s 2024-09-15T23:17:09.255 INFO:teuthology.orchestra.run.smithi121.stderr: 94.2% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-15T23:19:39.715 INFO:teuthology.orchestra.run.smithi121.stderr: 93.9% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-15T23:20:12.162 INFO:teuthology.orchestra.run.smithi121.stderr: 94.1% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-15T23:20:41.707 INFO:teuthology.orchestra.run.smithi121.stderr: 94.0% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-15T23:21:51.676 INFO:teuthology.orchestra.run.smithi121.stderr: 94.9% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-15T23:33:33.710 INFO:teuthology.orchestra.run.smithi121.stderr: 93.3% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-15T23:38:39.845 INFO:teuthology.orchestra.run.smithi121.stderr: 93.1% -- replaced with /var/log/ceph/b7614d3c-73b2-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-15T23:38:39.937 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-09-15T23:38:39.937 INFO:teuthology.orchestra.run.smithi121.stderr:real 23m2.312s 2024-09-15T23:38:39.938 INFO:teuthology.orchestra.run.smithi121.stderr:user 9m36.916s 2024-09-15T23:38:39.938 INFO:teuthology.orchestra.run.smithi121.stderr:sys 0m27.218s 2024-09-15T23:38:39.938 INFO:tasks.cephadm:Archiving logs... 2024-09-15T23:38:39.940 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906262/remote/smithi111/log 2024-09-15T23:38:39.942 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T23:38:48.132 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906262/remote/smithi121/log 2024-09-15T23:38:48.133 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T23:40:16.491 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906262/remote/smithi192/log 2024-09-15T23:40:16.492 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T23:40:36.843 INFO:tasks.cephadm:Removing cluster... 2024-09-15T23:40:36.844 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 --force 2024-09-15T23:40:37.122 INFO:teuthology.orchestra.run.smithi111.stdout:Deleting cluster with fsid: b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T23:40:37.749 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 --force 2024-09-15T23:40:38.649 INFO:teuthology.orchestra.run.smithi121.stdout:Deleting cluster with fsid: b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T23:40:39.406 DEBUG:teuthology.orchestra.run.smithi192:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b7614d3c-73b2-11ef-bceb-c7b262605968 --force 2024-09-15T23:40:39.610 INFO:teuthology.orchestra.run.smithi192.stdout:Deleting cluster with fsid: b7614d3c-73b2-11ef-bceb-c7b262605968 2024-09-15T23:40:40.091 INFO:tasks.cephadm:Removing cephadm ... 2024-09-15T23:40:40.091 DEBUG:teuthology.orchestra.run.smithi111:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-15T23:40:40.131 DEBUG:teuthology.orchestra.run.smithi121:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-15T23:40:40.149 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-15T23:40:40.165 INFO:tasks.cephadm:Teardown complete 2024-09-15T23:40:40.166 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-15T23:40:40.177 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-15T23:40:40.177 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T23:40:40.181 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T23:40:40.192 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T23:40:40.309 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-15T23:40:40.319 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-15T23:40:40.319 DEBUG:teuthology.orchestra.run.smithi111:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T23:40:40.323 DEBUG:teuthology.orchestra.run.smithi121:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T23:40:40.325 DEBUG:teuthology.orchestra.run.smithi192:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T23:40:40.338 INFO:teuthology.orchestra.run.smithi121.stderr:bash: line 1: ntpq: command not found 2024-09-15T23:40:40.340 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-09-15T23:40:40.345 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T23:40:40.345 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-09-15T23:40:40.345 INFO:teuthology.orchestra.run.smithi192.stdout:^* hv01.front.sepia.ceph.com 3 6 377 39 +427us[ +439us] +/- 53ms 2024-09-15T23:40:40.345 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 39 -1346us[-1346us] +/- 54ms 2024-09-15T23:40:40.345 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 +677us[ +689us] +/- 91ms 2024-09-15T23:40:40.345 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:40:40.353 INFO:teuthology.orchestra.run.smithi111.stderr:bash: line 1: ntpq: command not found 2024-09-15T23:40:40.361 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T23:40:40.361 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2024-09-15T23:40:40.361 INFO:teuthology.orchestra.run.smithi111.stdout:^* hv01.front.sepia.ceph.com 3 6 377 38 +577us[ +464us] +/- 53ms 2024-09-15T23:40:40.361 INFO:teuthology.orchestra.run.smithi111.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 42 -1165us[-1279us] +/- 54ms 2024-09-15T23:40:40.361 INFO:teuthology.orchestra.run.smithi111.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 40 +830us[ +716us] +/- 91ms 2024-09-15T23:40:40.361 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:40:40.406 INFO:teuthology.orchestra.run.smithi121.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T23:40:40.407 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================== 2024-09-15T23:40:40.407 INFO:teuthology.orchestra.run.smithi121.stdout:^* hv01.front.sepia.ceph.com 3 6 377 40 +445us[ +455us] +/- 53ms 2024-09-15T23:40:40.407 INFO:teuthology.orchestra.run.smithi121.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 37 -1311us[-1311us] +/- 54ms 2024-09-15T23:40:40.407 INFO:teuthology.orchestra.run.smithi121.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 +698us[ +707us] +/- 91ms 2024-09-15T23:40:40.407 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T23:40:40.417 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-15T23:40:40.427 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-15T23:40:40.428 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-15T23:40:40.437 DEBUG:teuthology.orchestra.run.smithi111:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T23:40:40.563 DEBUG:teuthology.orchestra.run.smithi121:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T23:40:40.692 DEBUG:teuthology.orchestra.run.smithi192:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T23:40:40.760 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T23:40:40.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:40:40.822 DEBUG:teuthology.orchestra.run.smithi121:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T23:40:40.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:40:40.855 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T23:40:40.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:40:40.884 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-15T23:40:40.896 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-15T23:40:40.905 INFO:teuthology.task.internal:Duration was 4340.700001 seconds 2024-09-15T23:40:40.906 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-15T23:40:40.916 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-15T23:40:40.916 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T23:40:40.920 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T23:40:40.923 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T23:40:40.963 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T23:40:40.969 INFO:teuthology.orchestra.run.smithi121.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T23:40:41.004 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T23:40:41.234 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-15T23:40:41.234 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi111.front.sepia.ceph.com 2024-09-15T23:40:41.235 DEBUG:teuthology.orchestra.run.smithi111:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T23:40:41.284 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi121.front.sepia.ceph.com 2024-09-15T23:40:41.284 DEBUG:teuthology.orchestra.run.smithi121:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T23:40:41.314 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi192.front.sepia.ceph.com 2024-09-15T23:40:41.315 DEBUG:teuthology.orchestra.run.smithi192:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T23:40:41.338 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-15T23:40:41.338 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T23:40:41.342 DEBUG:teuthology.orchestra.run.smithi121:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T23:40:41.357 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T23:40:41.437 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-15T23:40:41.437 DEBUG:teuthology.orchestra.run.smithi111:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T23:40:41.442 DEBUG:teuthology.orchestra.run.smithi121:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T23:40:41.444 DEBUG:teuthology.orchestra.run.smithi192:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T23:40:42.329 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-15T23:40:42.340 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-15T23:40:42.341 DEBUG:teuthology.orchestra.run.smithi111:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T23:40:42.397 DEBUG:teuthology.orchestra.run.smithi121:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T23:40:42.431 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T23:40:42.458 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-15T23:40:42.470 DEBUG:teuthology.orchestra.run.smithi111:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T23:40:42.475 DEBUG:teuthology.orchestra.run.smithi121:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T23:40:42.477 DEBUG:teuthology.orchestra.run.smithi192:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T23:40:42.514 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = core 2024-09-15T23:40:42.523 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = core 2024-09-15T23:40:42.541 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = core 2024-09-15T23:40:42.571 DEBUG:teuthology.orchestra.run.smithi111:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T23:40:42.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:40:42.605 DEBUG:teuthology.orchestra.run.smithi121:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T23:40:42.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:40:42.621 DEBUG:teuthology.orchestra.run.smithi192:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T23:40:42.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T23:40:42.637 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-15T23:40:42.648 INFO:teuthology.task.internal:Transferring archived files... 2024-09-15T23:40:42.650 DEBUG:teuthology.misc:Transferring archived files from smithi111:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906262/remote/smithi111 2024-09-15T23:40:42.650 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T23:40:42.733 DEBUG:teuthology.misc:Transferring archived files from smithi121:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906262/remote/smithi121 2024-09-15T23:40:42.733 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T23:40:42.773 DEBUG:teuthology.misc:Transferring archived files from smithi192:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906262/remote/smithi192 2024-09-15T23:40:42.774 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T23:40:42.812 INFO:teuthology.task.internal:Removing archive directory... 2024-09-15T23:40:42.812 DEBUG:teuthology.orchestra.run.smithi111:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T23:40:42.817 DEBUG:teuthology.orchestra.run.smithi121:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T23:40:42.819 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T23:40:42.861 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-15T23:40:42.873 INFO:teuthology.task.internal:Not uploading archives. 2024-09-15T23:40:42.873 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-15T23:40:42.883 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-15T23:40:42.884 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T23:40:42.892 DEBUG:teuthology.orchestra.run.smithi121:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T23:40:42.895 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T23:40:42.909 INFO:teuthology.orchestra.run.smithi121.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 23:40 /home/ubuntu/cephtest 2024-09-15T23:40:42.916 INFO:teuthology.orchestra.run.smithi192.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 23:40 /home/ubuntu/cephtest 2024-09-15T23:40:42.923 INFO:teuthology.orchestra.run.smithi111.stdout: 393754 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 23:40 /home/ubuntu/cephtest 2024-09-15T23:40:42.925 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-15T23:40:42.937 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-15T23:40:42.962 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} duration: 4340.700000762939 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-15T23:40:42.962 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T23:40:43.046 INFO:teuthology.run:pass